Today I Learned

hashrocket A Hashrocket project

194 posts by dillonhafer twitter @               

Rails' .insert_all method is too naive

Rails requires a unique index in order to use the .insert_all methods. This requirement can make this method very brittle and unusable. If your conflict target is the table’s primary key, this won’t work unless you create a redundant index on the table for this method to match against. This creates an amazing amount of waste not only of storage space, but also performance. This method would allow so many more use cases if it simply let you describe the conflict you want to match against.

More advanced method:

class ApplicationRecord
  def self.bulk_insert(array_of_hashes, conflict_targets = Array(primary_key))
    columns = array_of_hashes.first.keys
    values = array_of_hashes.flat_map(&:values)
    rows = array_of_hashes.map do |f|
      "(#{columns.size.times.map { "?" }.join(", ")})"
    end.join(", ")

    sql = sanitize_sql_array([<<~SQL, *values])
      INSERT INTO "#{table_name}"
      (#{columns.map { |c| "\"#{c}\"" }.join(",")})
      VALUES #{rows}
      ON CONFLICT (#{conflict_targets.map { |c| "\"#{c}\"" }.join(", ")}) DO NOTHING
    SQL

    connection.execute(sql.squish)
  end
end

SQL it produces:

User.bulk_insert([{email: "a@example.com"}, {email: "b@example.com"}])
INSERT INTO "users" ("email") VALUES ('a@example.com'), ('b@example.com') ON CONFLICT ("id") DO NOTHING

This would then allow you to reference any conflict you like:

alter table users add unique (email);
User.bulk_insert(
  [{email: "a@example.com"}, {email: "b@example.com"}],
  %i[email]
)

Sort array of numbers

In javascript, the Array sort function will cast everything to a string. So when you have an array of numbers, you need to repetitively cast them to numbers:

[1080, 720, 480].sort()
=> [1080, 480, 720]

So you need have to write your own sort function (don’t mess it up!)

[1080, 720, 480].sort((a, b) => Number(a) - Number(b))
=> [480, 720, 1080]




πŸ‰

ActiveStorage direct upload subfolders

When using S3, Rails does not let you configure subfolders for active storage. Every attachment lives at the root of your bucket.

This is not recommended, but cannot be helped

The only way to store attachments into subfolders is to monkey patch the direct uploads controller:

# config/initializer/direct_uploads_monkey_path.rb
Rails.application.config.to_prepare do
  class ActiveStorage::DirectUploadsController < ActiveStorage::BaseController
    def create
      key = "#{sub_dir}/#{user.id}/#{ActiveStorage::Blob.generate_unique_secure_token}"
      upload_attrs = {key:}.merge(blob_args)
      blob = ActiveStorage::Blob.create_before_direct_upload!(**upload_attrs)
      render json: direct_upload_json(blob)
    end

    private

    def user
      @user ||= User.enabled.find(session[:current_user_id])
    end

    def sub_dir
      if /cool-uploads/.match?(request.referer)
        "cool-uploads"
      elsif /company-settings/.match?(request.referer)
        "companies"
      else
        "uploads"
      end
    end
  end
end

Add custom flash keys

ActionController::Base has a default flash types of [:notice] allowing you to pass a key to #redirect_to:

class Controller < ActionController::Base
  def create
    if true
      redirect_to root_path, status: :see_other, notice: "Created"
    else
      flash[:error] = "Could not create"
      redirect_to root_path, status: :see_other
    end
  end

However, you can add :error as a custom type to get the convenience argument:

class Controller < ActionController::Base
  add_flash_types :error

  def create
    if true
      redirect_to root_path, status: :see_other, notice: "Created"
    else
      redirect_to root_path, status: :see_other, error: "Could not create"
    end
  end

Enforce TLS... except for health checks

Many infrastructure stability platforms will need to check the health of a rails application directly, not through a load balancer. Because many applications don’t terminate TLS directly (because it’s delegated to the load balancer) a health check endpoint must adhere the the force_ssl = true config option, but without TLS, causing a failure.

Rails 7 has an option to work around this (config.ssl_options):

# config/environments/production.rb
# Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
config.force_ssl = true
config.ssl_options = {
  redirect: {exclude: ->(request) { /healthz/.match?(request.path) }}
}

Clone a specific git branch

After cloning a repo, git will set your branch to whatever the value of HEAD is in the source repo:

$ ssh git@example.com
$ cat my-repo.git/HEAD
ref: refs/heads/master

If there is a different branch you want to clone, use the --branch flag:

$ git clone --branch my-feature git@example.com/my-repo.git
$ git branch
* my-feature

πŸ“§ Change the delivery method of a mailer

You can change the delivery method of a mailer with the default method:

Rails.application.config.action_mailer.delivery_method
=> :smtp

class LocalMailer < ApplicationMailer
  default delivery_method: :sendmail

  def notify_root
    mail({to: "root@localhost", subject: "Alert"})
  end
end

LocalMailer.notify_root.deliver_now

Then check the inbox:

sudo su root
mail
Mail version 8.1 6/6/93.  Type ? for help.
"/var/mail/root": 1 message 1 new
>N  1 no-reply@rails-app.example.com     Fri Aug 19 17:29 494/19390 "Alert"
?

Reverse proxy tcp/udp with nginx

Nginx has the ability to reverse proxy tcp and udp with the stream directive, similar to the http directive:

# Reverse proxy postgres server
stream {
  server {
    listen 5432;
    proxy_pass 172.18.65.97:5432;
  }
}

http {
  server { 
    ...
  }
}

This can be useful to load balance tcp streams like a database connection.

If you built nginx with --with-stream=dynamic (you can check with nginx -V) you will need to manually load a shared object:

# nginx.conf
load_module /usr/lib/nginx/modules/ngx_stream_module.so

Write a warning message to $stderr

While debugging, if you ever need to write to $stderr you might use $stderr#puts, but you can use Warning#warn which is better called from Kernel, because Kernel appends newlines and respects the -W0 flag:

$stderr.puts "you have been warned"
Warning.warn "you have been warned"
Kernel.warn "you have been warned"
warn "you have been warned"

Output:

you have been warned
you have been warnedyou have been warned
you have been warned

Listen for onfocus events on document

If you want to set an event listener on document for an input’s onFocus event, you’ll find there is no β€œonFocus" event for document. Instead you can use the focusin event listener:

document.addEventListener("focusin", function(e) {
  console.log("an input received focus");
});

So if you had dynamic events you can achieve this:

const on = (eventName, elementSelector, handler) => {
  document.addEventListener(eventName, function (e) {
    for (var target = e.target; target && target != this; target = target.parentNode) {
      if (target.matches(elementSelector)) {
        handler.call(target, e);
        break;
      }
    }
  }, false);
}

on("focusin", "#email_address", function(e) {
  const currentTarget = this;
  console.log("Email address just received focus");
});

Start multiple processes in dev env with bin/dev

Rails comes with bin/dev to start multiple processes from a Procfile.dev

# Procfile.dev
web: bin/rails server -p 3000
css: bin/rails tailwindcss:watch
jobs: bin/good_job
firebase: firebase emulators:start --import=./emulator-data

So that you can run everything at once:

$ bin/dev
08:36:10 web.1  | started with pid 40038
08:36:10 css.1  | started with pid 40039
08:36:10 firebase.1 | started with pid 40040
08:36:11 firebase.1 | i  emulators: Starting emulators: auth, firestore, storage
08:36:11 firebase.1 | i  firestore: Firestore Emulator logging to firestore-debug.log
08:36:11 web.1  | => Booting Puma
08:36:11 web.1  | => Rails 7.0.2.3 application starting in development
08:36:11 web.1  | => Run `bin/rails server --help` for more startup options
08:36:12 web.1  | Puma starting in single mode...
08:36:12 web.1  | * Puma version: 5.6.4 (ruby 3.1.2-p20) ("Birdie's Version")
08:36:12 web.1  | *  Min threads: 5
08:36:12 web.1  | *  Max threads: 5
08:36:12 web.1  | *  Environment: development
08:36:12 web.1  | *          PID: 40038
08:36:12 web.1  | * Listening on http://127.0.0.1:3000
08:36:12 web.1  | * Listening on http://[::1]:3000
08:36:12 web.1  | Use Ctrl-C to stop
08:36:12 css.1  |
08:36:12 css.1  | Rebuilding...
08:36:12 css.1  | Done in 300ms.

Avoid time discrepancies when benchmarking

You can avoid time discrepancies due to gc and allocs by using Benchmark#bmbm:

require 'benchmark'
array = (1..1000000).map { rand }
Benchmark.bmbm do |x|
  x.report("sort!") { array.dup.sort! }
  x.report("sort")  { array.dup.sort  }
end
Rehearsal -----------------------------------------
sort!   1.490000   0.010000   1.500000 (  1.490520)
sort    1.460000   0.000000   1.460000 (  1.463025)
-------------------------------- total: 2.960000sec
            user     system      total        real
sort!   1.460000   0.000000   1.460000 (  1.460465)
sort    1.450000   0.010000   1.460000 (  1.448327)

Add primary key to table

You can add a primary key to a table with alter an alter table statement:

Table with no primary key:

create table no_pks (id int generated by default as identity not null);
insert into no_pks select from generate_series(0,999);
[local] dillon@dillon=# \d no_pks
                           Table "public.no_pks"
 Column |  Type   | Collation | Nullable |             Default
--------+---------+-----------+----------+----------------------------------
 id     | integer |           | not null | generated by default as identity

You can add it later:

alter table no_pks add primary key (id);
[local] dillon@dillon=# \d no_pks
                           Table "public.no_pks"
 Column |  Type   | Collation | Nullable |             Default
--------+---------+-----------+----------+----------------------------------
 id     | integer |           | not null | generated by default as identity
Indexes:
    "no_pks_pkey" PRIMARY KEY, btree (id)

Print unknown exceptions in PL/pgSQL

When trying to figure out why a function raised an exception you can print the error code raised to lookup in the table Appendix A-1.

One method is to capture others and then raise the magic sqlstate variable (only available in exception handlers)

create or replace function do_it(name text)
  returns void
as $$
begin
  select 42 from nothing;
exception
  when others then
    raise '%: %', sqlstate, sqlerrm;
end;
$$
  security definer
  language plpgsql
;

Then you can view the error:

select do_it('hi');
ERROR:  42P01: relation "nothing" does not exist
CONTEXT:  PL/pgSQL function do_it(text) line 6 at RAISE

curl with a progress bar

You can download files with a nice progress bar using curl’s -# flag:

curl -# -O https://files.example.com/large/long_video.mp4
#################                               38.6%

This might be preferable to the verbose output:

curl --no-progress-meter -O https://files.example.com/large/long_video.mp4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 17  433M   17 75.6M    0     0  28.7M      0  0:00:15  0:00:02  0:00:13 28.8M

Use encrypted env vars with direnv

Direnv can execute shell scripts, so given that your env file is encrypted, you can automatically have it become decrypted for you:

───────┬──────────────────────
       β”‚ File: .env
───────┼──────────────────────
   1   β”‚ STRIPE_PK="123456789"
   2   β”‚ API_KEY="qwertyuiop"
───────┴──────────────────────

Say is was encrypted:

ansible-vault encrypt --vault-password-file config/master.key .env
cat .env
───────┬─────────────────────────────────────────────────────────────────────────────────
       β”‚ File: .env
───────┼─────────────────────────────────────────────────────────────────────────────────
   1   β”‚ $ANSIBLE_VAULT;1.1;AES256
   2   β”‚ 35306466356632363334643432343132356662376462333964366534393462366333623764336161
   3   β”‚ 6131336435323834623539323462626235383330346562660a323534656133653237656634346235
   4   β”‚ 30653635663438313931393966383266663535313361613339396234373164323830373262633661
   5   β”‚ 6262356131306530350a643362623636323762656132326363323736633431396463616137343139
   6   β”‚ 66666438623230333636373563393165333562633964616536663363323334343235386465346663
   7   β”‚ 3365643263643766323835356230636539353034643034346136
───────┴─────────────────────────────────────────────────────────────────────────────────

Now that we have an encrypted .env file, we just need direnv to decrypt it whenever we’re in our directory:

───────┬────────────────────────────────────────────────────────────────────────────────────────────────
       β”‚ File: .envrc
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
   1   β”‚ export $(ansible-vault decrypt --vault-password-file config/master.key --output - .env | xargs)
───────┴────────────────────────────────────────────────────────────────────────────────────────────────

Output:

$ cd rails_app
direnv: loading ~/dev/rails_app/.envrc
direnv: export +API_KEY +STRIPE_PK
echo $API_KEY
qwertyuiop

Now whenever we enter the directory, we will have the unencrypted env vars, but the file remains encrypted on disk. For whatever that’s worth.

Use + as a closure in Array reduce

In swift you can pass a method as the closure:

import Foundation

let numbers = [1, 2, 3, 4, 5]
let total = numbers.reduce(0, +)
print("Average: \(total / numbers.count)")

=> "Average: 3"

You can also use the generic closure:

import Foundation

let numbers = [1, 2, 3, 4, 5]
let total = numbers.reduce(0, { accumulator, number in 
  accumulator + number
})
print("Average: \(total / numbers.count)")

=> "Average: 3"

Prevent rails' file server from serving index.html

You can prevent rails’ file server from serving index.html, while continuing to serve other files from the public directory by changing the index_name:

class Application
  config.public_file_server.index_name = "other-index.html"
end

Rails.application.routes.draw do
  get :dashboard, to: "application#dashboard"
  root to: redirect("/dashboard")
end

Visiting the root path / will no longer serve public/index.html if you define another route for /

Implicit order column

ActiveRecord has a class method implicit_order_column= that allows you to override the behavior of the .first and .last methods.

class User < ApplicationRecord
  self.implicit_order_column = "email"
end

User.first
User Load (3.2ms)  SELECT "users".* FROM "users" ORDER BY "users"."email" ASC, "users"."id" ASC LIMIT $1  [["LIMIT", 1]]

Ignore ~/.psqlrc when using psql

You can ignore your ~/.psqlrc when running psql commands by using the -X or --no-psqlrc flags.

So when you have all this in your rc file:

/* ~/.psqlrc */
\x auto
\timing
\set PROMPT1 '%[%033[1m%]%M %n@%/%R%[%033[0m%]%# '
\set PROMPT2 '[more] %R > '
\pset null '☒'
\setenv PSQL_PAGER pspg
\setenv PAGER pspg

This command becomes quite noisy:

psql -c 'select 1'
Expanded display is used automatically.
Timing is on.
Null display is "☒".
Time: 0.210 ms
 ?column?
----------
        1
(1 row)

Time: 0.297 ms

If you run without the config file:

psql -X -c 'select 1'
 ?column?
----------
        1
(1 row)

on-line manual pages:

-X,
--no-psqlrc
    Do not read the start-up file (neither the system-wide psqlrc file nor the
    user's ~/.psqlrc file).

Get database value of model instance

AR instances have a method #attribute_in_database. Sometimes things don’t make sense, someone has created a method with the same name as the column:

create table users (id int, email citext);
class User < ApplicationRecord
  def email
    Time.current.strftime("%A, %B %C")
  end
end

User.first.email
=> "Thursday, March 20"

But you actually wan the email:

User.first.attribute_in_database("email")
=> "admin@example.com"

Array#first returns different classes based on arg

Ruby’s Array#first will have very different return types based on what arguments you provide. Without an argument, it will return nil or object. With an argument, it will always return a new array; even if your argument is 1 or 0.

a = [:foo, 'bar', 2]
a.first 
# => :foo

a.first(1) 
# => [:foo]

a.first(0)
# => []

With an empty array:

[].first
# => nil

[].first(1)
# => []

[].first(0)
# => []

Invoke procs with brackets

You can invoke a ` with bracketsProc[]`: ruby UriService = lambda do |username:, password:| def scream(n) n.upcase end "http://#{scream(username)}:#{password}@api.example.com" end UriService.call(username: "password", password: "123") => "http://PASSWORD:123@api.example.com" UriService[username: "password", password: "123"] => "http://PASSWORD:123@api.example.com"

Add TypeScript support to forms

When working with form names, it’s nice to have typescript support:

interface CustomerFormType extends HTMLFormElement {
  firstName: HTMLInputElement;
  lastName: HTMLInputElement;
}


declare global {
  interface Document {
    newCustomer: CustomerFormType;
  }
}

class CustomerForm extends Component {
  onSubmit = (e) => {
    e.preventDefault();
    const firstName = document.newCustomer.firstName.value;
    const lastName = document.newCustomer.lastName.value;
    console.log({firstName, lastName});
  };

  render() {
    return (
      
        
        
        Submit
      
    );
  }
}

Reject blank input with graphql-ruby

The graphql-ruby gem has a built-in blank validator:

class Mutations::UserUpdate < Mutations::BaseMutation
  null true

  argument :user_id,
    String,
    "Identifier of user",
    required: true,
    validates: {allow_blank: false}

  field :user_id, String, null: false

  def resolve(user_id:)
    {user_id:}
  end
end

So now a mutation with a user_id of " " will cause the graphql response to have an error:

mutation UserUpdate($userId: String!) {
  userUpdate(userId: $userId) {
    userId
  }
}