Today I Learned

A Hashrocket project

101 posts by viniciusnegrisolo @vfnegrisolo

ActiveRecord not is not boolean algebra negation

Today I learned that rails ActiveRecord not query is not to be considered a boolean algebra negative. Let’s see by an example:

User.where(name: "Jon", role: "admin")

This will produce a simple sql query:

SELECT "admins".*
FROM "admins"
WHERE "admins"."name" = $1
  AND "admins"."role" = $2
[["name", "Jon"], ["role", "admin"]]

If we get the same where clause and negate it:

User.where.not(name: "Jon", role: "admin")

Then we get:

SELECT "admins".*
FROM "admins"
WHERE "admins"."name" != $1
  AND "admins"."role" != $2
[["name", "Jon"], ["role", "admin"]]

But I was expecting a query like:

SELECT "admins".*
FROM "admins"
WHERE ("admins"."name" != $1
   OR "admins"."role" != $2)
 [["name", "Jon"], ["role", "admin"]]

So if you want to produce a query like that you’ll have to build on your own:

User.where.not(name: "Jon").or(User.where.not(role: "admin"))

Just beaware of that when using not with multiple where clauses.

Elixir ExDoc has version dropdown

ExDoc released a new version that allow developers to show a version dropdown on their documentation.

Here’s how I added to my library:

Open the mix.exs file and add javascript_config_path to the docs option on your project function.

def project do
  [
    ...
    docs: [
      main: "readme",
      extras: ~w(README.md),
      javascript_config_path: "../.doc-versions.js"
    ],
    ...
end

And on my Makefile I have this:

docs: ## Generate documentation.
docs: setup
    echo "var versionNodes = [" > .doc-versions.js
    app=`mix run -e 'IO.puts(Mix.Project.config()[:app])'`; \
    for v in $$(git tag | tail -r); do echo "{version: \"$$v\", url: \"https://hexdocs.pm/$$app/$$v/\"}," >> .doc-versions.js; done
    echo "]" >> .doc-versions.js
    mix docs

So if I run make docs this will generate or update a file .doc-versions.js from what I have on my git tag

And here is how it looks like:

image

Here’s the ExDoc changelog.

How to assert Elixir doctest raises an error

Today I learned how to assert an Elixir doctest raises an error. Check this out:

defmodule MyModule do
  @doc """
  This function raises ArgumentError.

  ## Examples

      iex> MyModule.my_func()
      ** (ArgumentError) something is wrong
  """
  def my_func() do
    raise(ArgumentError, "something is really wrong")
  end
end

The previous doctest will fail with this message:

  1) doctest MyModule.my_func/0 (1) (MyModuleTest)
     test/my_module_test.exs:3
     Doctest failed: wrong message for ArgumentError
     expected:
       "something is wrong"
     actual:
       "something is really wrong"
     code: MyModule.my_func()
     stacktrace:
       lib/my_module.ex:10: MyModule (module)

Rails protects production database

Rails has a mechanism to protect production databases to be dropped (and other destructive commands). In order to do that rails database tasks use a private database rake task check_protected_environments. Here’s a db task code sample from rails code databases.rake:

task drop: [:load_config, :check_protected_environments] do
  db_namespace["drop:_unsafe"].invoke
end

Under the hood it checks the database environment from a metadata table ar_internal_metadata created by rails on the first load schema attempt

SELECT * FROM ar_internal_metadata;
     key     |  value 
-------------+---------
 environment | staging 

Parallel shell processing with xargs

Today I learned how to parallel run a slow command on my shell. We can use xargs combined with the flags -n and -P flags. Let’s see how this works:

find . -type f | xargs -n1 -P8 slow_command
  • slow_command your slow command that receives a file as the first arg
  • -n to specify how many arguments are passed to the slow_command
  • -P how many parallel workers xargs will spawn to run the slow_command

Check this out watch -d -n 0.1 "seq 10 | xargs -n2 -P8 echo":

watch-xargs

On this example xargs are spawning up to 8 workers to run the echo command and for each echo execution xargs will pass 2 arguments. The arguments are produced by a seq 10 and as multiple executions of echo runs in parallel we can highlight the output changes with watch.

Show better image diff on git

Today I learned how to improve my git diff with images. For context check this TIL first.

I gave a step further on my git config to allow a better diff of my images using imagemagick compare features.

git config --global diff.image.textconv 'imgcat'
git config --global diff.image.command 'imgdiff'

So textconv will be used by git show commands and it’s using iTerm2 imgcat.

And command will be used by git diff command and it uses my new shell script imgdiff added to my PATH:

#!/bin/bash

if [[ -f "$1" ]] && [[ -f "$2" ]]; then
  compare "$2" "$1" png:- | montage -geometry +4+4 "$2" - "$1" png:- | imgcat
else
  if [[ -f "$1" ]]; then
    echo "+ Image Added"
    imgcat "$1"
  else
    echo "- Image Removed"
    imgcat "$2"
  fi
fi

exit 0

With that I can have a diff like that:

git diff

So previous image to the left, new to the right and imagemagick comparison in the middle.

Rails with_options

Rails has a convenient method with_options for reusing options on rails methods.

On the documentation we can see how to use on ActiveRecord classes but as this method is under Rails Object it actually can be used on several other places, like controllers.

Check this out:

before with_options

class PostsController < ApplicationController
  before_action :require_login, only: [:show, :edit]
  before_action :load_post_extra_info, only: [:show, :edit]

  ...
end

using with_options

class PostsController < ApplicationController
  with_options only: [:show, :edit] do
    before_action :require_login
    before_action :load_post_extra_info
  end

  ...
end

Thanks @mattpolito for that!

Show images on git diff

Today I learned how to show binary files, more specifically images, using git diff or git show CLI. For that I am using iTerm2 imgcat.

In order to get there I had to configure git to allow a custom diff command to specific file types.

So I used git attributes to do that:

echo "*.gif diff=image" >> ~/.gitattributes
echo "*.jpg diff=image" >> ~/.gitattributes
echo "*.png diff=image" >> ~/.gitattributes
git config --global core.attributesfile '~/.gitattributes'

Then I had to change the git diff text converter to use iterm2 imgcat:

git config --global diff.image.textconv 'imgcat'

Check this out:

git-diff-images

Notes:

  • imgcat does not work with new versions of tmux
  • git pagers like less or more won’t work either, so you can run git --no-pager diff or you can pipe with cat like:
git diff | cat

Check also Open images in vim

Delimiters for sed shell command

Today I learned that the sed shell command accepts other delimiters than /. I was trying to run the following:

folder="screens/components"
echo "~/FOLDER/index.js" | sed -e "s/FOLDER/${folder}/g"
# sed: 1: "s/{{folder}}/screens/co ...": bad flag in substitute command: 'c'

But I got a bad flag error. So I changed my delimiter to | and all works fine:

folder="screens/components"
echo "~/FOLDER/index.js" | sed -e "s|FOLDER|${folder}|g"

Javascript Default Named Parameters

I wanted to create a function with some named parameters as optional values, but I wanted them to have a default value for each option. I could use function({max = 10, min = -10}){} but this only works if you pass a js object as argument. So I added a default object into that and it just works great. Check this out:

avg = function({max = 10, min = -10} = {max: 11, min: -11}) {
  let avg = (max + min) / 2;
  console.log('min', min, 'max', max, 'avg', avg);
  return avg;
}

avg();
// => min -11 max 11 avg 0

avg({});
// => min -10 max 10 avg 0

avg({max: 5});
// => min -10 max 5 avg -2.5

avg({min: 6});
// => min 6 max 10 avg 8

avg({max: 5, min: 6});
// => min 6 max 5 avg 5.5

Default values are different on this example for illustration purposes.

Git Push Force but with lease

If you need to run a git push --force to push a fixup or an amended commit you can try the --force-with-lease tag first for safety. It will protect you to overwrite a commit made by other dev.

alias gplease='git push --force-with-lease'

Check Git push documentation

--force-with-lease will protect all remote refs that are going to be updated by requiring their current value to be the same as the remote-tracking branch we have for them.

Serialize an Elixir pid

I was wondering how to serialize/deserialize a pid for some tricks with Ecto Sandbox and I found out some nice code on phoenix_ecto code. Here it goes my version:

defmodule MyApp.Serializer do
  @spec serialize(term) :: binary
  def serialize(term) do
    term
    |> :erlang.term_to_binary()
    |> Base.url_encode64()
  end

  @spec deserialize(binary) :: term
  def deserialize(str) when is_binary(str) do
    str
    |> Base.url_decode64!()
    |> :erlang.binary_to_term()
  end
end

And the test:

defmodule MyApp.SerializerTest do
  use ExUnit.Case, async: true

  alias MyApp.Serializer

  describe "serialize/1" do
    test "serializes a pid" do
      pid = self()
      assert pid |> Serializer.serialize() |> is_binary()
    end

    test "serializes a map" do
      map = %{foo: :bar}
      assert map |> Serializer.serialize() |> is_binary()
    end
  end

  describe "serialize/1, deserialize/1" do
    test "serializes and deserializes a pid" do
      pid = self()
      assert pid |> Serializer.serialize() |> Serializer.deserialize() == pid
    end

    test "serializes and deserializes a map" do
      map = %{foo: :bar}
      assert map |> Serializer.serialize() |> Serializer.deserialize() == map
    end
  end
end

Redirect outputs with subshell

Today I learned how to use a Subshell to redirect multiple outputs as a single one. This way I can run a bunch of commands and unify all the results as it was a single command.

(git diff --name-only; git diff --staged --name-only) | sort -u

The above example will show me a sorted list of changed files (including the staged ones) in a git project without duplicates.

Here’s another example:

$ echo "world"; echo "hello" | sort -u
#=> world
#=> hello

$ (echo "world"; echo "hello") | sort -u
#=> hello
#=> world

From VIM to Tmux

There’s a really good plugin tslime for sending text from VIM to another Tmux session/window/pane. In addition, sometimes it’s super useful to map short cuts like this:

:vmap <buffer> \t <Plug>SendSelectionToTmux

In this example I’m mapping \t for sending all visual selected text in VIM to Tmux.

Check this out:

tmux

h/t @joshuadavey

PostgreSQL "RETURNING"

Today I learned that PostgreSQL INSERT/UPDATE/DELETE has a RETURNING clause that returns the computed values.

We can use it to get values that are generated by database such as sequences or defaults.

Check this out:

DROP TABLE IF EXISTS users;

CREATE TABLE users (
  id    SERIAL PRIMARY KEY,
  email VARCHAR NOT NULL UNIQUE
);

INSERT INTO users (email) VALUES ('user1@example.com');
-- INSERT 0 1

INSERT INTO users (email) VALUES ('user2@example.com') RETURNING *;
--  id |       email
-- ----+-------------------
--   2 | user2@example.com
-- (1 row)

h/t @joshuadavey

Vim undo tree

Vim has an undo tree and there’s two different ways to iterate with your changes.

One way to undo/redo changes is with: u and <Ctrl>r. This moves the current state based on the current branch in the tree.

The other way is using g- and g+. In this case it moves based on the timestamp of that change.

So here’s an example to follow:

  1. add a line: added_line_1
  2. update this line: updated_line_1
  3. undo this change: u
  4. add another line: added_line_2

If you run through these steps your undo tree will be something like:

 o  [3] "added_line_2" <= "current_state"
 | *  [2] "updated_line_1"
 |/
 *  [1] "added_line_1"
 *  [0] "empty_file"

Now, if you undo with u will be in this state:

 *  [3] "added_line_2"
 | *  [2] "updated_line_1"
 |/
 o  [1] "added_line_1" <= "current_state"
 *  [0] "empty_file"

Otherwise if you undo with g- will be in this part of the tree:

 *  [3] "added_line_2"
 | o  [2] "updated_line_1" <= "current_state"
 |/
 *  [1] "added_line_1"
 *  [0] "empty_file"

You can check gundo vim plugin for tree visualization.

Aggregate Expressions with FILTER in Postgresql

Today I learned to use FILTER on aggregate expressions such as COUNT in postgresql.

See this example:

SELECT u.login,
       COUNT(1) FILTER (WHERE r.language ilike 'ruby') as ruby,
       COUNT(1) FILTER (WHERE r.language ilike 'javascript') as js,
       COUNT(1) FILTER (WHERE r.language ilike 'elixir') as elixir,
       COUNT(1) as all_langs
FROM users u
LEFT JOIN repositories r ON (u.id = r.user_id)
GROUP BY u.id;

-- login | ruby | js | elixir | all_langs
---------+------+----+--------+------------
-- bill  |    5 |  2 |      3 |         15
-- karen |    2 |  7 |      4 |         19
-- bob   |    9 |  1 |      2 |         23

h/t @joshuadavey

Using Ruby Hash in `gsub`

Ruby String#gsub method also accepts a hash as second argument. This is perfect for some situations like mapping a phone-word to phone number:

phonewords = {
  'a' => 2, 'b' => 2, 'c' => 2,
  'd' => 3, 'e' => 3, 'f' => 3,
  'g' => 4, 'h' => 4, 'i' => 4,
  'j' => 5, 'k' => 5, 'l' => 5,
  'm' => 6, 'n' => 6, 'o' => 6,
  'p' => 7, 'q' => 7, 'r' => 7, 's' => 7,
  't' => 8, 'u' => 8, 'v' => 8,
  'w' => 9, 'x' => 9, 'y' => 9, 'z' => 9,
}
phone = "1-800-map-gsub"
puts phone.gsub(/[a-z]/, phonewords)
# => 1-800-627-4782

h/t @joshuadavey

Rails Enums and PostgreSQL Enums

Today I learned that Rails Enum works pretty well with PostgreSQL Enum.

Enum in PostgreSQL work as a new type with restricted values and if you need to guarantee data integrity there’s no best way to do that than in your database. Here is how we create a new enum in PostgreSQL and use in a new column:

class AddEnumToTrafficLights < ActiveRecord::Migration
  def up
    ActiveRecord::Base.connection.execute <<~SQL
      CREATE TYPE color AS ENUM ('red', 'green', 'blue');
    SQL

    add_column :traffic_lights, :color, :color, index: true
  end

  def down
    remove_column :traffic_lights, :color

    ActiveRecord::Base.connection.execute <<~SQL
      DROP TYPE color;
    SQL
  end
end

In Rails models I prefer to use Hash syntax when mapping Ruby symbols to database values.

class TrafficLight < ApplicationRecord
  enum color: {
    red: 'red',
    green: 'green',
    blue: 'blue',
  }
end

And now Enum in action:

TrafficLight.colors
#=> {
#=>   "red"=>"red",
#=>   "green"=>"green",
#=>   "blue"=>"blue"
#=> }
TrafficLight.last.blue!
TrafficLight.last.color
#=> "blue"

h/t @ joshuadavey

Rails Enum with prefix/suffix

[enum]:http://api.rubyonrails.org/classes/ActiveRecord/Enum.html

Today I learned that Rails 5 released new options for [enum][] definition: _prefix and _suffix.

class Conversation < ActiveRecord::Base
  enum status:          [:active, :archived], _suffix: true
  enum comments_status: [:active, :inactive], _prefix: :comments
end

conversation.active_status!
conversation.archived_status? # => false

conversation.comments_inactive!
conversation.comments_active? # => false

h/t @joshuadavey

Dealing with exit signals on Ruby

When working on CLIs it’s useful to treat if the application exists. Image you have a long application and even if the user hits a <Ctrl+c> to kill the process you still want to display the results so far.

There is a Ruby Kernel method at_exit that runs the block passed to it if the current process exists. Here is an example:

# print_at_exit.rb
at_exit { puts "Come back later!" }
puts "sleeping for 5 secs"
sleep 5

If you run this file and wait it until it finishes you’ll get:

ruby print_at_exit.rb                                                                                                                {130}
#=> sleeping for 5 secs
#=> 
#=> Come back later!

But now, if you run it and hit <Ctrl+c> for finishing the process earlier you’ll get:

ruby print_at_exit.rb
#=> sleeping for 5 secs
^C
#=> Come back later!
#=> print_at_exit.rb:6:in `sleep': Interrupt
#=>         from print_at_exit.rb:6:in `<main>'

h/t @nikkypx

Ruby defined? method evaluation

Today I learned that Ruby Object method defined? evaluates if the argument is anything recognizable, not just if the variable are set. Here are some examples:

undefined_foo #=> NameError: undefined local variable or method `undefined_foo'
defined? undefined_foo #=> nil

defined? nil #=> "nil"

defined? "foo" #=> "expression"
defined? [] #=> "expression"
defined? def x; end #=> "expression"
defined? class Foo; end #=> "expression"

defined? @foo = :bar #=> "assignment"

@foo = :bar
defined? @foo #=> "instance-variable"

foo = :bar
defined? foo #=> "local-variable"

def foo; end
defined? foo() #=> "method"

class Foo; end
defined? Foo #=> "constant"

h/t @VidalEkechukwu

Elixir Binaries and Bit Strings

Today I learned that in Elixir Binary is a sub type of Bit String with all elements using a multiple of 8 bits.

"a" # => "a"
?a # => 97
<<97>> # => "a"
is_bitstring <<97>> # => true
is_binary <<97>> # => true
is_bitstring <<97::4>> # => true
is_binary <<97::4>> # => false
is_binary <<97::16>> # => true
  • So String is a UTF-8 Binary.
  • And Binary is a multiple 8 bits Bit String.

One query with Select In

Today I learned that if you pass an ActiveRecord::Relation as param to where Rails will create a single query using IN and an inner SELECT:

Repository.where(user_id: User.active.select(:id))
=>  Repository Load (1.1ms)
        SELECT "repositories".*
        FROM "repositories"
        WHERE "repositories"."user_id" IN (
          SELECT "users"."id"
          FROM "users"
          WHERE "users"."status" == 1
       )

h/t @mattpolito

Ember nested routes and it's active class

Ember adds automagically a css class active to all link-to elements under the current route.

So let’s say you have this in your router:

// app/router.js
Router.map(function() {
  this.route('blog', function() {
    this.route('post', { path: ':slug' });
  });
});

If you access a page like http://localhost:4200/blog/last-post, all the following links will have the active css class.

{{#link-to 'blog'}}Blog{{/link-to}}
{{#link-to 'blog' post}}{{post.name}}{{/link-to}}
<a id="ember661" href="/blog" class="ember-view active">Blog</a>
<a id="ember663" href="/blog/last-post" class="ember-view active">Last Post</a>

So far so good, but now, if by any reason you just want either child link or parent link with the active class you can change the parent link-to to use .index.

{{#link-to 'blog.index'}}Blog{{/link-to}}

So both blog and blog.index links to the same url, but they act differently with active class.

ActiveJobs callbacks with conditionals

Today I learned that ActiveJobs callbacks accept filters, so it’s easy to do conditionals like:

class SyncUserJob < ApplicationJob
  queue_as :default

  after_perform(if: :recursive) do
    SyncOrganizationJob.perform_later(user, recursive: true)
    SyncRepositoryJob.perform_later(user)
  end

  attr_accessor :user, :recursive

  def perform(access_token, recursive: false)
    @access_token = access_token
    @recursive    = recursive
    @user         = sync_user
  end

  def sync_user
    ...
  end

More context with git diff

So if you want to see more (or less) context when running a git diff command you can use -U<number_of_lines>.

Here it is a regular git diff (3 lines as default):

vnegrisolo@home:~/my-app(master)% git diff
index modified: app/jobs/github/sync_organization_job.rb
@ sync_organization_job.rb:24 @ module Github
      organization.login       = data['login']
      organization.description = data['description']
      organization.avatar_url  = data['avatar_url']
-      organization.user        = @user
+      organization.user        = user

      organization.save
      recursive_sync(organization) if @recursive

Now just showing the changed lines:

vnegrisolo@home:~/my-app(master)% git diff -U0
index modified: app/jobs/github/sync_organization_job.rb
@ sync_organization_job.rb:27 @ module Github
-      organization.user        = @user
+      organization.user        = user

Finally with 5 lines of context instead of 3:

vnegrisolo@home:~/my-app(master)% git diff -U5
index modified: app/jobs/github/sync_organization_job.rb
@ sync_organization_job.rb:22 @ module Github
      organization = Organization.find_or_initialize_by(github_id: data['id'])

      organization.login       = data['login']
      organization.description = data['description']
      organization.avatar_url  = data['avatar_url']
-      organization.user        = @user
+      organization.user        = user

      organization.save
      recursive_sync(organization) if @recursive
      organization
    end

Ruby Method arity

Today I learned that Method#arity does not consider blocks.

Also it returns negative with splat arguments.

Finally, all keyword arguments counts 1.

class Foo
  def one; end
  def two(a, b); end
  def three(a, *b); end
  def four(a, &b); end
  def five(a, b:, c:); end
end

puts Foo.instance_method(:one).arity
# => 0
puts Foo.instance_method(:two).arity
# => 2
puts Foo.instance_method(:three).arity
# => -2
puts Foo.instance_method(:four).arity
# => 1
puts Foo.instance_method(:five).arity
# => 2

h/t @higgaion

Rails reset counter caches

Today I learned that Rails has an easy way to reset counter caches

You just need to call reset_counters(id, *counters) method

irb> Organization.reset_counters(4, :repositories)
  Organization Load (0.3ms)  SELECT  "organizations".* FROM "organizations" WHERE "organizations"."id" = $1 LIMIT $2  [["id", 4], ["LIMIT", 1]]
   (0.5ms)  SELECT COUNT(*) FROM "repositories" WHERE "repositories"."organization_id" = $1  [["organization_id", 4]]
  SQL (2.1ms)  UPDATE "organizations" SET "repositories_count" = 6 WHERE "organizations"."id" = $1  [["id", 4]]
=> true

Rails autoload paths

Today I learned that I can see Rails autoload_paths running:

puts ActiveSupport::Dependencies.autoload_paths
/Users/vnegrisolo/projects/my-rails-app/app/business
/Users/vnegrisolo/projects/my-rails-app/app/channels
/Users/vnegrisolo/projects/my-rails-app/app/controllers
/Users/vnegrisolo/projects/my-rails-app/app/controllers/concerns
/Users/vnegrisolo/projects/my-rails-app/app/jobs
/Users/vnegrisolo/projects/my-rails-app/app/mailers
/Users/vnegrisolo/projects/my-rails-app/app/models
/Users/vnegrisolo/projects/my-rails-app/app/models/concerns
/Users/vnegrisolo/projects/my-rails-app/app/serializers
/Users/vnegrisolo/projects/my-rails-app/app/services
/Users/vnegrisolo/projects/my-rails-app/test/mailers/previews

You can always add more paths with:

# config/application.rb
config.autoload_paths << Rails.root.join('lib')

By the way, that’s why concerns (model/controller) do not have Concern as namespace.

Ember queryParams default value has to be static

Today I learned that Ember queryParams do not work with computed properties. They intend to be static values and here is an issue discussion about that.

So this code will not work properly:

export default Ember.Controller.extend({
  queryParams: ['start'],
  start: new Date().getFullYear() + '-' + (new Date().getMonth() + 1) + '-' + new Date().getDate();
});
export default Ember.Route.extend({
  queryParams: {
    start: { refreshModel: true }
  },
  model(params) {
    console.log('start', params.start);
  }
});

In this example the default value is cached and if you try to change it manually, or use a computed property with volatile, you’ll see a delegate is not a function error message.