Today I Learned

A Hashrocket project

161 posts about #rails

PostgreSQL index with NO lock on Rails

Rails allow us to create a PostgreSQL index that it would not lock the table for writing meanwhile it’s being calculated.

That’s usually handful if a table has millions of rows and this operation could take hours to release the INSERT/UPDATE/DELETE lock. Maybe the lock downtime is critical to the application.

There are caveats to this approach so please read the PostgreSQL documentation.

In order to do that you’ll need to add { algorithm: :concurrently } to the add_index on a Rails migration. Additionally PostgreSQL uses DB transactions to manage this new index creation so we need to disable the Rails migration transaction. Take this example:

class AddIndexOnEventsName < ActiveRecord::Migration[5.0]
  disable_ddl_transaction!

  def change
    add_index(:events, :name, algorithm: :concurrently)
  end
end

In this case I’m using disable_ddl_transaction! to disable the DB transaction that wraps each Rails migrations and I am using algorithm: :concurrently.

This option it will generate an index like that:

CREATE INDEX CONCURRENTLY "index_events_on_name" ON "events" ("name"));

Rails will change the `not` behavior

Yay Rails will change the behavior of the ActiveRecord#not on rails 6.1. That’s great as the current behavior sometimes leads to confusion. There will be a deprecation message on the version 6.0 so let’s watch out and change our code accordingly.

As I already wrote on this TIL this function does not act as a legit boolean algebra negation. Let’s say that we have this query:

User.where.not(active: true, admin: true).to_sql

Prior to this change this will produce this query:

SELECT users.*
  FROM users
 WHERE users.active != 't'
   AND users.admin != 't'

The problem is that negating an AND clause is naturally a NAND operator, but Rails have implemented as a NOR. Some developers might wonder why regular active users are not returning.

On Rails 6.1 this will be fixed hence the new query will be:

SELECT users.*
  FROM users
 WHERE users.active != 't'
    OR users.admin != 't'

Here’s the rollout plan:

  • Rails 5.2.3 acts as NOR
  • Rails 6.0.0 acts as NOR with a deprecation message
  • Rails 6.1.0 act as NAND

This is the PR in case you want to know more.

Use `reset_column_information` to Migrate Data

If you’re generating a Rails migration, chances are you might need to facilitate migrating data to a new column. You can use reset_column_information in a migration file to pick up your changes and immediately do something with those new columns.

Assuming we have 2 models, DraftPost and Post -

  class AddColumnDraftToPosts < ActiveRecord::Migration[5.2]
    def change
      add_column :posts, :draft, :boolean, default: true
      Post.reset_column_information
      DraftPost.all.each do |draft_post|
        Post.create(content: draftPost.content)
        draft_post.delete
    end
  end

Rails 6 new ActiveRecord method pick

Rails 6 will be released with a new convenient method on ActiveRecord pick. This method picks the first result value(s) from a ActiveRecord relation.

Person.all.pick(:name)
# SELECT people.name FROM people LIMIT 1
# => 'John'

Person.all.pick(:name, :email)
# SELECT people.name, people.email FROM people LIMIT 1
# => ['John', 'john@mail.com']

So pick(:name) is basically equivalent to limit(1).pluck(:name).first.

Watch out for rails inconsistencies as pick(:name) returns a single string value and pick(:name, :email) returns an array of values.

Rails 6 Blocked Hosts

Rails 6 has a new feature where only whitelisted hosts are allowed to be accessed. By default only localhost is permitted.

When doing mobile development, you always need to test the app in a real device that connects to a backend. In order to automatically add the dev machine host to the list, just change your development.rb to:

# config/environments/development.rb 

config.hosts << "#{`hostname -s`.strip}.local"

Rails 6 upsert_all

Rails 6 and ActiveRecord introduces upsert_all 🎉

I used to have to jump into SQL and write this myself, but no more!


As a simplified use case, say you have a User and that user has many Tags. If an API wants to send you an array of strings (["Tag Name A", "Tag Name B"]), you no longer have to do any looping, find_or_create, or defining your own upsert in SQL nonsense.

@user.tags << Tag.upsert_all([
  { name: "Tag Name A"},
  { name: "Tag Name B"}
], unique_by: :name)

If the tags exist, they will be updated. If the tags are new, they will be created. And all this happens in 1 SQL Insert.

ActiveRecord not is not boolean algebra negation

Today I learned that rails ActiveRecord not query is not to be considered a boolean algebra negative. Let’s see by an example:

User.where(name: "Jon", role: "admin")

This will produce a simple sql query:

SELECT "admins".*
FROM "admins"
WHERE "admins"."name" = $1
  AND "admins"."role" = $2
[["name", "Jon"], ["role", "admin"]]

If we get the same where clause and negate it:

User.where.not(name: "Jon", role: "admin")

Then we get:

SELECT "admins".*
FROM "admins"
WHERE "admins"."name" != $1
  AND "admins"."role" != $2
[["name", "Jon"], ["role", "admin"]]

But I was expecting a query like:

SELECT "admins".*
FROM "admins"
WHERE ("admins"."name" != $1
   OR "admins"."role" != $2)
 [["name", "Jon"], ["role", "admin"]]

So if you want to produce a query like that you’ll have to build on your own:

User.where.not(name: "Jon").or(User.where.not(role: "admin"))

Just beaware of that when using not with multiple where clauses.

Delete Paranoid Records In Rails

The ActsAsParanoid gem provides soft delete functionality to ActiveRecord objects in Rails. You can enhance a model with its functionality like so:

class User < ActiveRecord::Base
  acts_as_paranoid
end

This gem hijacks ActiveRecord’s standard destroy and destroy! functionality. If you call either of these methods, instead of the record being deleted from the database, it’s deleted_at column is updated from nil to the current timestamp. Resulting in a soft deleted record.

If you call destroy or destroy! a second time (i.e. on a record that has already been soft deleted), it will be actually deleted from the database. Alternatively, you can call destroy_fully! from the beginning to skip the soft delete.

Serialize With fast_jsonapi In A Rails App

Netflix put out a Ruby gem for super fast JSON serialization — fast_jsonapi. It is great for serializing JSON responses for Rails API endpoints.

First, add gem 'fast_jsonapi' to your Gemfile and bundle install.

Then create the app/serializers directory for housing all of your JSON serializers.

Next you can create a serializer that corresponds to the model you want to serialize:

# app/serializers/recipe_serializer.rb
class RecipeSerializer
  include FastJsonapi::ObjectSerializer

  set_id :id
  attributes :name, :source_url
end

Last, use it to generate a JSON response in your controller:

# app/controllers/recipes_controller.rb
class RecipesController < ApiController
  def index
    render json: RecipeSerializer.new(@current_user.recipes)
  end
end

Requests to that endpoint will receive a response that looks something like this:

{
  data: [
    {
      id: 1,
      attributes: { name: "Old Fashioned", source_url: "http://..." },
    },
    {
      id: 2,
      attributes: { name: "Sazerac", source_url: "http://..." },
    },
  ]
}

Secure Passwords With Rails And Bcrypt

If you are using bcrypt (at least version 3.1.7), then you can easily add secure password functionality to an ActiveRecord model. First, ensure that the table backing the model has a password_digest column. Then add has_secure_password to your model.

class User < ActiveRecord::Base
  has_secure_password

  # other logic ...
end

You can now instantiate a User instance with any required fields as well as password and password_confirmation. As long as password and password_confirmation match then an encrypted password_digest will be created and stored. You can later check a given password for the user using the authenticate method.

user = User.find_by(email: user_params[:email])

if user.authenticate(user_params[:password])
  puts 'That is the correct password!'
else
  puts 'That password did not match!'
end

Rails protects production database

Rails has a mechanism to protect production databases to be dropped (and other destructive commands). In order to do that rails database tasks use a private database rake task check_protected_environments. Here’s a db task code sample from rails code databases.rake:

task drop: [:load_config, :check_protected_environments] do
  db_namespace["drop:_unsafe"].invoke
end

Under the hood it checks the database environment from a metadata table ar_internal_metadata created by rails on the first load schema attempt

SELECT * FROM ar_internal_metadata;
     key     |  value 
-------------+---------
 environment | staging 

Access Secrets In A Rails 5.2 App

For a long time the access chain for getting at secrets in your Rails app stayed the same. For instance, getting at the secret_key_base value looked something like this:

Rails.application.secrets.secret_key_base

In the world of Rails 5.2, secrets are no longer secrets. They are now credentials. This means they are under the credentials key instead of the secrets key. Here is how you can access secret_key_base now:

Rails.application.credentials.secret_key_base

source

Rails with_options

Rails has a convenient method with_options for reusing options on rails methods.

On the documentation we can see how to use on ActiveRecord classes but as this method is under Rails Object it actually can be used on several other places, like controllers.

Check this out:

before with_options

class PostsController < ApplicationController
  before_action :require_login, only: [:show, :edit]
  before_action :load_post_extra_info, only: [:show, :edit]

  ...
end

using with_options

class PostsController < ApplicationController
  with_options only: [:show, :edit] do
    before_action :require_login
    before_action :load_post_extra_info
  end

  ...
end

Thanks @mattpolito for that!

Reversible integer to string migrations in Rails

I recently needed to convert an integer column in Rails to a string, and wanted to make sure that the migration would be reversible. I specified the up and down methods, but found that I couldn’t reverse the migration because the column type couldn’t be automatically cast back into an integer.

As it turns out, Rails allows us to specify how to cast the column with the using option:

def up
  change_column :users, :zip, :string
end

def down
  change_column :users, :zip, :integer, 
    using: "zip::integer"
end

This builds the sql:

ALTER TABLE users 
ALTER COLUMN zip 
TYPE integer
USING zip::integer

Good to go!

Pass a Block on Console Load

Have some Ruby code you want to run as the Rails console loads? Here’s a technique.

In this example, I’m going to use a function from the hirb gem. Add the gem to your Gemfile and bundle.

Next, pass a block to console in your application configuration.

# config/application.rb

module MyApplication
  class Application < Rails::Application
    console do
      # Action(s) I want to run on console load...
      Hirb.enable
    end
  end
end

The next time you start the Rails console, Hirb.enable will run after load.

code / docs

Outer join with ActiveRecord `references` method

I want to join posts to comments regardless if comments exist or not.

You can use includes for that:

Post.includes(:comments).all

But that results in 2 queries, one to get all the posts, and one to get all comments that have the relevant post_id.

With references you can turn this into an outer join:

Post.includes(:comments).references(:comments).all

Now we’re getting all the information we need with just 1 query.

Check out the Active Record guides here

How to check Rails migration statuses

You can check the status of your migrations in Rails by running rails db:migrate:status

This command returns your database name, as well as a list of all your migrations, with their name and status

database: my-database-dev

Status | Migration ID | Migration Name
--------------------------------------
  up   | 201803131234 | Create users
  up   | 201803201234 | Create blogs
 down  | 201804031234 | Create posts

ActiveRecord where.not

When I need a IS NULL SQL expression with ActiveRecord it is easy to avoid a SQL string fragment.

User.where(deactivated_at: nil)

But in early versions of Rails using IS NOT NULL required a string.

User.where("deactivated_at is not null")

But modern Rails has where.not expressions which can eliminate the string.

User.where.not(deactivated_at: nil)

Add React With Webpacker To A New Rails App

Webpacker makes it easy to manage app-like JavaScript in the context of a Rails app. React is a great candidate for this kind of webpack-powered JavaScript processing pipeline.

To set up a new Rails project with Webpack and React wired up, add the --webpack=react flag:

$ rails new rails-react-app --webpack=react

As part of the generated app, you will get a app/javascript/packs directory with a hello_react.jsx file that has a really basic React component.

source

Specify different bundler groups with RAILS_GROUPS

Is there a gem that you want to include sometimes and not others? For instance, do you have multiple versions of staging with slightly different gemsets? You can manage that with custom groups.

group :apple do
  gem 'some_gem'
end

group :orange do
  gem 'some_other_gem'
end
RAILS_GROUPS=apple,orange rails console
> Rails.groups
[:default, "development", "apple", "orange"]

In the config/application.rb file, Rails.groups is passed to Bundler.require.

Bundler.require(*Rails.groups)

Change Name of :id Param in Rails Resource Routing

You can change the name of the parameter used by rails resource routing by specifying the param option to the resource route.

For example, if we have a show endpoint:

resources :pages, only: :show

Running rake routes will return the pages#show url as:

/pages/:id

We can change the id param to slug by doing:

resources :pages, only: :show, param: :slug

Then running rake routes will yield our new pages#show route as:

/pages/:slug

Delayed Job Queue Adapter in RSpec with Rails 5.1

In old versions of Rails, you were able to override the ActiveJob queue in a test like this:

describe MyJob do
  it 'works' do
    ActiveJob::Base.queue_adapter = :delayed_job
    expect {
      MyJob.perform_later(some_params)
    }.to change(Delayed::Job.count).by(1)
  end
end

With Rails 5.1, we have the ActiveJob::TestHelper class which you will need to employ in your tests. In order to override the queue a different strategy is needed.

describe MyJob do
  def queue_adapter_for_test
    ActiveJob::QueueAdapters::DelayedJobAdapter.new
  end
  
  it 'works' do
    expect {
      MyJob.perform_later(some_params)
    }.to change(Delayed::Job.count).by(1)
  end
end

You will need to add the following to your rspec config or a support file:

RSpec.configure do |config|
  config.include(ActiveJob::TestHelper)
end

# you will also need the code below for the test
# to clear out the jobs between test runs
class ActiveJob::QueueAdapters::DelayedJobAdapter
  class EnqueuedJobs
    def clear
      Delayed::Job.where(failed_at:nil).map &:destroy
    end
  end
  
  class PerformedJobs
    def clear
      Delayed::Job.where.not(failed_at:nil).map &:destroy
    end
  end
  
  def enqueued_jobs
    EnqueuedJobs.new
  end
  
  def performed_jobs
    PerformedJobs.new
  end
end

🔑 Set foreign keys to null

Sometimes, in certain circumstances, it is reasonable to have a foreign key value of null.

ActiveRecord’s .has_many method has an argument to set the foreign key column on referencing rows to null when that record is deleted.

dependent: :nullify

Example:

class Post < ApplicationRecord
  belongs_to :author
  belongs_to :category
end

class Category < ApplicationRecord
  has_many :posts, dependent: :nullify
end

In this example whenever a category is deleted, any posts referencing the categories table will have their foreign key set to null.

*References: http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html#method-i-has_many-label-Options

`disable_with` to prevent double clicks

If a link or button initiates a request that takes a long time to respond then the user (being a user) might click the link or button again. Depending on the code implemented on the server this could put the application into a bad state.

Rails has a handy feature that will disable a link or button after the user has clicked it, disable_with:

<%= link_to('something',
      something_path(something),
      data: {
        disable_with: "Please wait..."
      }
    )
%>

In Rails 5.0 and up sumbit buttons have disable_with set by default. To disable disable_with use:

data: { disable_with: false }

Storing recurring schedules in #Rails + #Postgres

If you have a scheduling component to your Rails application you may need to store the day of week and time of day in the database.

One way to store the day of week is to use an integer column with a check constraint that will check that the value is between 0 and 6.

create table schedules (
  id serial primary key,
  day_of_week integer not null check(day_of_week in (0,1,2,3,4,5,6)),
  beg_time time not null,
  end_time time not null
);

Then when you read it back from the database and need to convert it back to day name you can use Date::DAYNAMES. e.g.:

[2] pry(main)> require 'date'
=> true
[3] pry(main)> Date::DAYNAMES
=> ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]
[4] pry(main)> Date::DAYNAMES[0]
=> "Sunday"
[5] pry(main)>

If you need to store time of day as entered (in a time without timezone column - as specified above) check out the wonderful Tod gem by Jack Christensen

Convert to BigDecimal with `to_d` w/ActiveSupport

Ruby provides the BigDecimal method to convert to BigDecimal.

> require 'bigdecimal'
> BigDecimal("123.45")
#<BigDecimal:56236cc3cab8,'0.12345E3',18(18)>

But you can’t convert a float without a precision

> BigDecimal(123.12)
ArgumentError: can't omit precision for a Float.
> BigDecimal(123.12, 5).to_s
"0.12312E3"

When using Rails, and specifically with ActiveSupport required, you can use the to_d method converts to BigDecimal.

> require 'active_support'
> 123.to_d
#<BigDecimal:55ebd7800ea8,'0.123E3',9(27)>
> "123".to_d
#<BigDecimal:55ebd7800ea8,'0.123E3',9(27)>

And for floats provides a default precision of Float::DIG+1 which for me is 16. DIG is described as

The number of decimal digits in a double-precision floating point.

> 123.45.to_d
#<BigDecimal:55ebd2d6cfb8,'0.12345E3',18(36)>
> 123.45.to_d.to_s
"123.45"

Note, to_s in ActiveSupport outputs a more human readable number. Also Note, nil is not convertable with to_d

> require 'active_support'
> nil.to_d
NoMethodError: undefined method `to_d' for nil:NilClass
> BigDecimal(nil)
TypeError: no implicit conversion of nil into String

What you had before you saved w/`previous_changes`

When you set values into the ActiveRecord object the previous values are still available with changes, but when you save, however you save, those changes are wiped out. You can access what those values were before saving with previous_changes.

> thing = Thing.create({color: 'blue', status: 'active'})
> thing.color = 'red'
> puts thing.changes
{"color" => ['blue', 'red']}
> puts thing.previous_changes
{}
> thing.save
> puts thing.changes
{}
> puts thing.previous_changes
{"color" => ['blue', 'red']}

Access record from ActiveRecord::RecordInvalid

You can pass an array of hashes to Thing.create! in ActiveRecord. If one of those records is invalid, then an ActiveRecord::RecordInvalid error is thrown. You might need to know which record threw the error, in which case you can get the record from the error with record_invalid_error.record

bad_record = nil

begin
  Things.create!([{value: 'bad'}, {value: 'good'}])
rescue ActiveRecord::RecordInvalid => record_invalid_error
  bad_record = record_invalid_error.record
end

if bad_record
  puts "got a bad record with value: #{bad_record.value}"
end

Array of hashes `create` many ActiveRecord objects

Generally, you use the create method of ActiveRecord objects to create an object by passing a hash of attributes as the argument.

Thing.create(color: 'green', status: 'active')

You can also pass an array of hashes to create:

things = [
  {
    color: 'blue',
    status: 'pending'
  },
  {
    color: 'green',
    status: 'active'
]

created_things = Thing.create(things)

One disappointing thing is that this does not batch the insert statements. It is still just one insert statement per object, but it might make your code simpler in some cases.

Getting BetterErrors in Rails While Using Ngrok

By default, the BetterErrors gem only works for localhost. If you’re using ngrok to access your rails server and you want to have access to BetterErrors, you’ll need to whitelist the IP of the machine that ngrok is running on.

Add the following to development.rb:

BetterErrors::Middleware.allow_ip!(NGROK_MACHINE_PUBLIC_IP)
# - or -
# Use an IP finding service to grab your public IP each time you start the server:
# (ipecho.net, api.ipify.org, etc.)
BetterErrors::Middleware.allow_ip!(open('http://api.ipify.org').read)

Rails Ignore Pending Migrations

Ruby on Rails gives us a nice warning in development when we haven’t run a pending database migration. But what if we’re iterating on a migration, and the results can best be viewed in the browser? We might want to temporarily disable this warning.

Here’s how to do it:

# config/environments/development.rb
config.active_record.migration_error = false

Don’t forget to turn it back on when you’re finished.

Better Rails SQL Migrations

Some people on our team, including me, write raw SQL in Ruby on Rails migrations. But they can get unwieldy when iterating on a complex migration that does multiple things.

Mitigate this is to break up your statements into separate HERDOCs, like so:

def up
  execute <<-SQL
    update pixaxes set metal = 'diamond' where metal = 'iron';
    -- many things...
  SQL

  execute <<-SQL
    update swords set metal = 'diamond' where metal = 'iron';
    -- many more things...
  SQL
end

If the migration fails, we’ll get an error pointing to the specific problematic HEREDOC, instead of essentially ‘the entire statement is invalid’. You then put a debugger between any HEREDOC to iterate on the issue.

h/t Jack Christensen

Set the Domain or Host in Rails URL Helpers

You can explicitly set the domain with Rails URL Helpers:

app.root_url
=> "http://www.example.com/"

app.root_url(domain: 'www.customdomain.org')
=> "http://www.customdomain.org"

 

If we include the protocol in our domain argument, this brings up an interesting gotcha. Check it out:

app.root_url(domain: 'http://www.customdomain.org')
=> "http://www.http://www.customdomain.org/"

 

Thankfully, Rails gives us the host option. Host will use a Regex to remove the protocol from the argument:

app.root_url(host: 'https://www.customdomain.org')
=> "http://www.customdomain.org/"

 

Notice the output in the last example; our protocol was replaced removed and replaced with http. You can use the protocol argument to enforce your protocol on a given host:

app.root_url(host: 'http://www.customdomain.org', protocol: 'https')
=> "https://www.customdomain.org/"

Alias A Model Method Or Field in a GraphQL Type

If you need to override the name of a model method or field in a GraphQL type, you can use the property argument.

Let’s say we have a model called Message:

class Message < ApplicationRecord
  def bar
    "bar"
  end
end

We can then make an alias to the bar method by specifying a property in our MessageType:

Types::MessageType = GraphQL::ObjectType.define do 
  field :foo, types.String, property: :bar
end

Has ActiveRecord::Relation already been grouped?

I’ve recently run into a situation where I needed to apply a group statement to an ActiveRecord scope if a group statement had already been applied. In this case we need to examine the current scope to see if in fact a group statement has already been applied. ActiveRecord::Relation fortunately has a group_values method that returns an array of all the columns that the query has been grouped by as symbols.

if current_scope.group_values.length > 1
  current_scope.group(:another_column)
else
  current_scope
end

Authenticate With Username or Email

Today I learned a way to implement a login form that accepts email or username as the login. I’ve been on the other side of a form like this many times, but had never written one myself.

Here’s one solution, with ActiveRecord:

User.where('username = ? or email = ?', "jwworth", "jake@example.com")

title or slug are represented by the same parameter, and either can be nil.

login = params.fetch('username_or_email')
User.where('username = ? or email = ?', login, login)

Convert nested JSON object to nested OpenStructs

If you are parsing a nested JSON string such as this:

{
   "vendor": {
       "company_name": "Basket Clowns Inc",
       "website": "www.basketthecloon.com"
 }

And want to access it with dot notation, simply doing:

OpenStruct.new(JSON.parse(json_str))

will not do!

Turns out there is a cool option on JSON.parse called object_class:

JSON.parse(json_str, object_class: OpenStruct)

Now you can access the resulting object with dot notation all the way down:

obj.vendor.website #=> "www.basketthecloon.com"

Change The Nullability Of A Column

Do you have an existing table with a column that is exactly as you want it except that it needs to be changed to either null: false or null: true?

One option is to use ActiveRecord’s change_column_null method in your migration.

For example to change a nullable column to null: false, you’ll want a migration like the following:

def change
  change_column_null :posts, :title, false
end

Note, if you have existing records with null values in the title column, then you’ll need to deal with those before migrating.

If you want to make an existing column nullable, change that false to true:

def change
  change_column_null :posts, :title, true
end