Today I Learned

A Hashrocket project

Revealing Rails Scopes

I've been working on some Rails code that brings in ActiveRecord models from multiple gems. Often these models have default scopes, that bain of a legacy Rails codebase, and figuring that out requires source diving one or more gems. Today I hacked my way to a faster solution: just read the SQL Rails generates.

Here's a post without a default scope, and then one with a default scope:

pry(main)> Post.all.to_sql
=> "SELECT \"posts\".* FROM \"posts\""
pry(main)> Developer.all.to_sql
=> "SELECT \"developers\".* FROM \"developers\" ORDER BY \"developers\".\"username\" ASC"

I see you, ORDER BY.

Age in Postgres

On this day every year I get curious about how old I actually am. Postgres has some utilities to help me out with that, like subtraction of dates.

> select now() - '1978-3-24';
14245 days 17:39:55.623866

Ah yes, 14245 days on this earth, it really makes you think. What got returned anyways, that looks weird:

> select pg_typeof(now() - '1978-3-24');
interval

Got it, that's an interval. It doesn't really tell me much, is this interval infinite?

> select isfinite(now() - '1978-3-24');
t

I can squint and read isfinite as infinite and suddenly my outlook changes, but only momentarily.

Postgres, just tell me how many years I've been alive ok? None of this days stuff.

> select age('1978-3-24'::date);
39 years

Chaining Multiple RSpec Change Matchers

It can be handy to use RSpec's change matchers to determine if some method or process creates a new record.

expect{ Registration.create(attrs) }.to change{ User.count }.by(1)

But what if we are testing a method that creates a couple different records in the system?

RSpec allows us to chain together change matchers with and. Consider this additional contrived example.

expect {
  Project.generate(attrs)
}.to change{ Project.count }.by(1).and \
     change{ User.count }.by(1)

In addition to keeping our tests tight and concise, this approach gives some pretty nice output on failure.

If we were just beginning our implementation with a failing test, we'd see a multi-part failure like the following.

Failure/Error:
  expect {
    Project.generate(attrs)
  }.to change{ Project.count }.by(1).and \
       change{ User.count }.by(1)

     expected result to have changed by 1, but was changed by 0

  ...and:

     expected result to have changed by 1, but was changed by 0

Disk usage for just the top level dirs

Every so often your hard drive runs out of space and you need to do a survey of what could be possibly be taking up so much space. The du disk-usage command is great for that.

It can be noisy though as it traverses through all the directories and gives you a report on the size of each file. What we really want is just the size of each directory under the root dir.

du has a depth flag (-d) to help control the depth of directories that the command reports on.

du -h -d1 /

The above command gives you a report on all the directories and files at the top level.

Invisible components in React Native

Sometimes you want a component to be rendered but not visible to the user and in React Native 0.42 and below, you could accomplish that by setting the component's height and width to 0.

The good news is that React Native 0.43 will make it easier and introduce the CSS property you are already familiar with: display: 'none'.

You can already try it installing the pre-release version 0.43.0-rc.4.

Run Prettier on all #JavaScript files in a dir

If you are like me you must like formatters such as Prettier which probably prompted you to set your editor to auto format the file on save.

That's great for new projects but when working on an existing project, every file you touch will have a huge diff in git that can obscure the real changes made to the file.

To solve that you must run prettier on all your javascript files as an independent commit. You can do it with the following command:

find ./src/**/*.js | xargs prettier --write --print-width 80 --single-quote --trailing-comma es5

The flags after the prettier are all my personal preferences except for --write which tells prettier to write the file in place.

Note 1: Make sure you have all the files you are about to change committed to source control so that you can check them out if this did not go well.

Note 2: When committing this change it would be a good idea to use git add -p and go through the changes one by one (which is always a good idea...)

Note 3: To dry run and see which files will be changed run the find ./src/**/*.js by itself.

Erlang documentation with `erl -man`

Erlang can be tricky, so documentation is a must, but when you're 30,000 feet high in an airplane with no internet you might regret not having downloaded DashApp and with it all the Erlang documentation.

But it was on your machine the whole time! You have access to all the Erlang documentation with erl -man used like so:

erl -man calendar

Which pops open a man page for the calendar module.

When you are learning Erlang the docs are at your fingertips.

List Available File Types For Ack

The ack utility allows you to filter the searched files based on file type. If you'd like to know all of the file types available, you can use the --help=types flag. This will include file types you've specified in your .ackrc file.

Here is a sample of some of the output.

$ ack --help=types
    ...
    --[no]css          .css .less .scss
    --[no]dart         .dart
    --[no]delphi       .pas .int .dfm .nfm .dof .dpk .dproj .groupproj .bdsgroup .bdsproj
    --[no]elisp        .el
    --[no]elixir       .ex .exs
    --[no]erlang       .erl .hrl
    --[no]fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp
    --[no]go           .go
    --[no]groovy       .groovy .gtmpl .gpp .grunit .gradle
    --[no]haskell      .hs .lhs
    --[no]hh           .h
    --[no]html         .html .mustache .handlebars .tmpl
    --[no]jade         .jade
    --[no]java         .java .properties
    --[no]js           .js
    ...

See man ack for more details.

Extended file attributes on macOS

You can store metadata on any file in the mac filesystem(HFS+). If you want to ensure the file has a specific encoding, or if you want to place a checksum with the file you can use file attributes.

Setting and reading are easy on macOS with the xattr utility:

> touch something.txt
> xattr -w xyz 123 something.txt
> xattr -p xyz something.txt
123

In this example we wrote(xattr -w) an attribute onto the file and printed it(xattr -p) to see that it was written.

You can list the file attributes with a cryptic option for ls, ls -l@. Try doing this on your ~/Downloads dir to learn something cool about how macOS treats files coming from the internet.

Remove One List From Another

The --/2 operator allows you to subtract two lists, that is, remove all elements in the right list from the left list. Each occurrence of an element is removed if there is a corresponding element. If there is no corresponding element, it is ignored.

Here are some examples.

> [1, 2, 3] -- [2, 4]
[1, 3]
> [:a, :b, :c, :a, :d, :a] -- [:a, :a]
[:b, :c, :d, :a]

This kind of list operation is not particularly efficient, so for large lists it can be quite slow. The following example took several minutes to run.

> Enum.into(1..1000000, []) -- Enum.into(2..1000000, [])
[1]

To achieve a true set difference, you'll note that the docs for this operator recommend checking out MapSet.difference/2.

See h Kernel.-- for more details.

RSpec Covers a Range

Today I used a feature of RSpec I've never used before, cover.

cover returns true if it's argument is covered by the given range:

expect(0..1).to cover(order.risk_score)

In my use case, I had an attribute risk_score that is calculated by a third-party API. It should always be between zero and one, but it changes on every test run because the API considers the test user more and more risky each time. cover allowed me to test this attribute, and the underlying logic, in a flexible way.

Inspect with Label

IO.inspect returns the value you pass in as the first argument. This is great when you want to debug the middle state of a series of piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Here, inspect will write to stdout the values [{"A", 1}, {"B", 2"}...].

IO.inspect also takes a label option, which decorates the values written to stdout but does not disrupt the piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect(label: "Letters with index:")
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Which outputs Letters with index: [{"A", 1}, {"B", 2"}...] to stdout.

Bits of Characters

If you are curious what the binary of an ascii/utf-8 char is you can use this string of commands at the command line:

echo 'A' | xxd -b

The A character is 65 in ascii which is 64 + 1. 01000000 is 64. 00000001 is of course 1, 01000001 is 65.

echo 'a' | xxd -b

The a character is 97 in ascii which is 64 + 32 + 1. 00100000 is 32 in binary, given this and the above, 01100001 is 97.

echo '🤓' | xxd -b

This ridiculous emoji is a utf-8 char. When you look at the binary for it:

11110000 10011111 10100100 10010011

You can see that every byte begins with a 1 which means that it will combine with any subsequent byte beginning with 1 to form a unique character.

Jaro Distance determines "closeness" of 2 strings

In Elixir, the String module includes a function named jaro_distance that determines the similarity between two strings:

iex> String.jaro_distance("Chris", "Crhis")
0.9333333333333332
iex> String.jaro_distance("Chris", "Bob")
0.0
iex> String.jaro_distance("Chris", "Dennis")
0.5777777777777778

The docs say this works best with short strings. The Jaro distance article on wikipedia has lots of math if you're into that.

Referencing Values In IEx's History

Each time we execute a statement in an iex session, the counter is incremented. These numbers are references to the history of the session. We can use these references to refer to previously executed values using v/1. This is particularly handy for multi-line statements or when we forget to bind to the result of some function.

Consider the following iex session:

iex(1)> :one
:one
iex(2)> 1 + 1
2
iex(3)> "three" |> String.to_atom()
:three

If we execute v() on its own, it is the same as v(-1) in that it will give us the latest value in the history.

iex(4)> v()
:three

Providing any positive number will refer to the references we see next to each statement.

iex(5)> v(1)
:one

Negative numbers, as we saw with v(-1), will count backwards in the history from where we are.

iex(6)> v(-4)
2

See h v for more details.

Creating A PID

Often times, when invoking a function that spawns a process, the PID of the spawned process is returned and we bind to it. That PID is a reference to some BEAM process in our system.

We can create our own references using the pid/3 function.

Let's assume we have the following processes, among others, in our system at the moment.

> Process.list |> Enum.reverse |> Enum.take(3)
[#PID<0.284.0>, #PID<0.283.0>, #PID<0.282.0>]

We can create a reference to any of them using the three number parts that they are made up of.

> pid(0, 284, 0)
#PID<0.284.0>

See, it's alive.

> pid(0, 284, 0) |> Process.alive?
true

What if we make up a PID that doesn't actually reference any process?

> pid(0, 333, 0) |> Process.alive?
false

Note: there is also a pid/1 version of the function. See h pid for more details.

CSS Variables

CSS3 supports native variables that are assigned with the following syntax:

html {
  --foo-color: #FF0000;
  --bar-color: #00FF00;
}

Note the -- that precedes the variable's name. Now, the <html> element and all its descendants will be able to access the variables with the following syntax.

a {
  color: var(--foo-color);
}

button {
  background: var(--bar-color);
}

CSS uses the var function to look up variables.

You don't have to assign the variable to the <html> tag. It can be assigned to any element or selector's style block. Just remember, only that element and its children will have access to the variable ;-)

Futhermore, you can access the variables in Chrome's DevTool/Element Inspector!

NOTE: This feature is not supported at all in Internet Explorer and only has partial support in Microsoft Edge 15, the most recent Edge at this time of writing. Opera Mini users are out of luck as well :-(

Browser Support Chart

Accept Your Own Changes During Git Rebase

During a git rebase you may encounter conflicts in files between your current, HEAD, branch and the branch you're rebasing. Ordinarily, you'll want to go through each file and individually resolve each conflict to ensure that the proper changes are preserved.

Sometimes, however, you already know that you want to accept ALL the changes in a file on your local branch and discard the other branch's edits. Instead of opening the file, finding the conflict regions, then making the appropriate changes, you can more succinctly prefer your changes with the following command:

git checkout --ours /path/to/file.js

Conversely, if you want to keep the other branch's changes, run:

git checkout --theirs /path/to/file.ex

You can also do this for an entire directory:

git checkout --ours /path/to/dir/
git checkout --theirs . # Current working directory

When you [re]open the conflict files, you'll see that your preferred branch's changes have been written and the other branch's have been discarded.

After you've finished, stage the the conflict files, and continue your rebase:

git add /path/to/conflict_file.rb
git rebase --continue # or --skip

Fuzzy awesome copy to sys clipboard w/yank & fzf

Say you want to copy a pid of a process to system clipboard, you could run ps ax, maybe grep on the result, connect your trusty mouse and try to select the value and hit ⌘ + c.

Or you can use the amazing fuzzy finder FZF (brew install fzf) in combination with Yank (brew install yank).

ps ax | fzf | yank

Now simply start typing the name of the process. When you press return you will get the columns broken down into a selectable prompt - choose one and press return. It is now in your system clipboard.

Here's a demo:

demo

This will work with any column spaced or even multiline response. Try running ps ax | yank.

Determine the type of buffer

The :set <option> command will show the value of an option. In the case of :set buftype this is generally set to nothing. It's empty. This is because we're usually working with a normal buffer. The exception is the quick window. Generally, the buftype of the quick window is quickwindow. The help window also has its own buftype, help.

These values are used programmatically to help create features with vimscript.

See more with: :help buftype

Create jsonb data structure from columns

You can create a jsonb object by casting a string version of the object to jsonb:

> select '{"a": 1, "b": 2}'::jsonb;
      jsonb
------------------
 {"a": 1, "b": 2}

When using values from columns to create a jsonb object its not practical to construct a string. In this case you want to use jsonb_build_object:

> select jsonb_build_object('a', a.x) from (select x from generate_series(1, 10) as x) as a;
 jsonb_build_object
--------------------
 {"a": 1}
 {"a": 2}
 {"a": 3}
 {"a": 4}
 {"a": 5}
 {"a": 6}
 {"a": 7}
 {"a": 8}
 {"a": 9}
 {"a": 10}

jsonb_build_object is a variatic function and takes a variable number of arguments. So you can add additional values like this:

jsonb_build_object('a', 1, 'b', 2)

Making your JS files prettier on Vim

We all use VIM at Hashrocket and we can run prettier everytime we save a JS file:

yarn global add prettier
autocmd FileType javascript set formatprg=prettier\ --single-quote\ --trailing-comma\ es5\ --stdin
autocmd BufWritePre *.js :normal gggqG
autocmd BufWritePre *.js exe "normal! gggqG\<C-o>\<C-o>"

And yes. I like trailing comma,

**** Update: There is an open issue on Github when prettier fails to parse the JS file but there is a workaround for it.

Show commits in the order they were committed

When reviewing a series of commits its sometimes helpful to review them in the order they were committed.

By default, if you pass a range to git show, like git show HEAD..master (assuming you are on a different branch and master is ahead by a number of commits) the commits will be shown in the order of most recent to least recent.

To view the commits in least recent to most recent order use the --reverse flag:

git show HEAD..master --reverse

Debug the `--exclude-pattern` option in rspec.

You can exclude certain files from being run by rspec with the --exclude-pattern option like so:

rspec --exclude-pattern run_me_not_spec.rb

You can place this option into your .rspec file.

When doing this and then committing the .rspec file its helpful to make sure the exclude pattern is correct. Try this command and pipe it into grep.

rspec --dry-run -fdoc | grep 'excluded test name'

If no results are returned, then you are successfully excluding the test! The --dry-run option is important because actually running the entire test suite would be too time consuming.

Specifying The Phoenix Server Port

Running mix phx.server for a Phoenix project with the default settings will attach the server to port 4000.

If you'd like to use a different port in development, you can change it in config/dev.exs.

config :my_app, MyApp.Web.Endpoint,
  http: [port: 4444],
  ...

Alternatively, you can allow it to be configurable from the command line with an environment variable and a fallback port.

config :my_app, MyApp.Web.Endpoint,
  http: [port: System.get_env("PORT") || 4000],
  ...

Running

$ PORT=4444 mix phx.server

will launch the server on port 4444.

Logrotation for a Rails App

This week I did a some Linux server logrotation for a Ruby on Rails application. log/your_log.log can get large on a server with traffic, and sometimes we must control how long to retain data.

Here's the basic configuration file I wrote, with comments:

# Logrotater:
# - Daily
# - Timestamped
# - Doesn't error on missing log files
# - Keeps seven copies
# - Truncates log files (allows Rails to start writing 
#  to the new log file without a restart)

/var/app/current/log/your_log.log {
    daily
    dateext
    missingok
    rotate 7
    copytruncate
}

There are many other options. Check out man logrotate on a Linux machine for more info.

Prepare, Execute, and Deallocate Statements

In PostgreSQL, you can prepare a named statement to be executed later using prepare.

> prepare column_names (text) as
    select column_name from information_schema.columns where table_name = $1;
PREPARE

These statements are kept around for the duration of the session. To see the available statements, check out the pg_prepared_statements view.

> select * from pg_prepared_statements;
     name     |                                  statement                                  |         prepare_time          | parameter_types | from_sql
--------------+-----------------------------------------------------------------------------+-------------------------------+-----------------+----------
 column_names | prepare column_names (text) as                                             +| 2017-03-10 15:01:09.154528-06 | {text}          | t
              |   select column_name from information_schema.columns where table_name = $1; |                               |                 |

To run a prepared statement, use execute with the name of the statement and any arguments.

> execute column_names('users');
   column_name
-----------------
 id
 email
 password_digest
 created_at
 updated_at
 first_name
 last_name

You can also delete a statement with deallocate if you'd like.

> deallocate column_names;
DEALLOCATE

yank full line without new line character

I am often in a scenario where I want to take all the contents of the current line and paste them into the middle of another line, perhaps into some parentheses or quotation marks.

If I'm in the middle of a line I can type ^y$ which will goto the first character of the line and then yank to the end without the newline.

This is the rare palindramic vim command. It works the same backwards as it does forwards ($y^).

There is also a plugin that treats the line as a vim object called vim-textobj-line. Check it out!

Matching on directories for vim's autocmd

Generally, vim users use the autocmd command to set specific options for filetypes. Like, if you want all Ruby files to have the shiftwidth option set to 2 then you would include an autocmd line like this in your ~/.vimrc file:

autocmd BufRead *.rb set shiftwidth=2

I wanted to do something a bit different. I wanted to set the color scheme for each file in a directory rather than for a file type, and I tried this:

autocmd BufRead *writing* color shine

Nothing happened.

Via :help autocmd-patterns I was able to learn that if there isn't a slash char (/) anywhere in the pattern then it just tries to match on the filename. If a slash is included however, the whole path for that file is matched. All I needed was to add a slash:

autocmd BufRead */writing* color shine

Shine On.

Submit a pull request from the command line

Github offers a git extension for augmenting the git command line tool with github specific functionality. This extension is called hub.

hub adds the git alias pull-request to git. This alias/command creates a pull request from the current branch to master by default:

git pull-request

Additionally, you can specify which branch would be the base with the first argument and which branch would be the head with the second argument, like so:

git pull-request devolop feature/add_colors

jQuery.map returns wrapped elements

There are thousands of ways to write JavaScript and many ways to map over a collection in JavaScript. The jQuery variant of the function has a couple of peculiarities that might trip you up however.

First, the arguments to the map function are not (item, index) like they are for the Array.prototype.map function. Instead, they are (index, item).

Second, the jQuery map function does not return an array consisting of the values returned from the map function. It instead returns an array of those values wrapped by jQuery.

This can be unexpected when iterating over a set of elements to create an array of derived values.

Debug :hover selectors in CSS

When styling HTML you may need to define a hover style the css of which might look like this:

div.something:hover {
  background-color: red;
}

The div with class something will now have a red background when your mouse hovers over it.

This can be hard to debug however, if this were a regular style we could inspect it and change the style of the element in dev tools. With hovers, the style is only available in dev tools when the mouse is physically over the element.

Chrome has a facility to declare that a hover state should be on for an element. Simply open dev tools and in the Elements tab right click an element to get a context menu. The menu will have options :active, :hover, :focus, and :visited. Choose :hover and the hover state will be permenantly affixed to that element. You can now play with the hover style until it looks right.

Code splitting with Webpack 2 and Babel

Webpack 2 provides the ability to split your code into smaller files that are lazy loaded during runtime as they are needed.

When I first learned about this feature I thought it would be very intelligent in detecting which parts of the code are using a certain module and split all my modules into separate files automatically. That's not really the case.

If you want to have Webpack split your code and lazy load it you need to explicitly call import in your code. For example:

import Foo from './foo';

class Bar {
  baz() {
    Foo.someMethod(); // this will not be split and lazy loaded

    import('./lazy_module').then(function(lazyModule) {
      console.log(lazyModule.sayHello());
    }).catch(function(err) {
      console.log('Failed to load lazy module', err);
    });
  }
}

To have this work you need to install the Syntax Dynamic Import library

Then edit your .babelrc:

{
  "presets": [["es2015", { "modules": false }]],
  "plugins": ["syntax-dynamic-import"]
}

The modules: false part is really important. It basically instructs Babel to not try and parse the imports but let Webpack 2's native ability to parse imports to do the work. This was tricky.

There's more to that and it keeps changing so I recommend visiting this documentation https://webpack.js.org/guides/code-splitting-import/

Selected Text Character Count

When highlighting a word in vim (Visual Mode), type:

g<CTRL-g>

In the command section of vim you'll see something like this:

Selected 1 of 90 Lines; 2 of 715 Words; 15 of 4171 Chars; 15 of 4173 Bytes

In this case, 15 of 4171 Chars is the number of total characters and the number of characters selected.

Its not as straight forward or as easy to parse as you might expect, but when you know what to look for it becomes easy to read.

Full page screenshots in Firefox

Firefox has an interesting feature called the Developer Toolbar. This toolbar is entirely separate from the JS console and provides a cli for custom commands for specific functionality. It can opened with shift-F2 or Tools > Developers > Developer Toolbar.

In that toolbar, type:

screenshot --fullpage my-page.png

A png file with the given name will be saved to the Downloads folder. The file format will always be png.

Source

More Developer Toolbar information can be found here.

Type help in the toolbar to see all the commands.

How Rails Responds to `*/*`

Yesterday I fixed a bug in TIL. This application has a Twittercard, but it's never worked. Twitter's card validator confusingly claims the site lacks Twitter meta tags.

After experimenting, I realized that when cURL-ing our site, the response type is application/json. Why is Rails giving me JSON?

When an HTTP request's accept headers equal */*, any MIME type is accepted. And, when a respond_to block is present:

Rails determines the desired response format from the HTTP Accept header submitted by the client.

Determined how? I learned that the first MIME type declared is used for */* requests.

Notice the difference (HTML first):

# app/controllers/posts_controller.rb
 def index
    @posts = Post.all
    respond_to do |format|
      format.html
      format.json { render json: @posts }
    end
  end

Request/response:

$ curl -vI localhost:3000/
...
> Accept: */*
>
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
...

JSON first:

# app/controllers/posts_controller.rb
 def index
    @posts = Post.all
    respond_to do |format|
      format.json { render json: @posts }
      format.html
    end
  end

Request/response:

$ curl -vI localhost:3000/
...
> Accept: */*
>
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
...

Reversing the order (HTML first) solved the issue.

Grep For A Pattern On Another Branch

Git has a built-in grep command that works essentially the same as the standard grep command that unix users are used to. The benefit of git-grep is that it is tightly integrated with Git. You can search for occurrences of a pattern on another branch. For example, if you have a feature branch, my-feature, on which you'd like to search for occurrences of user.last_name, then your command would look like this:

$ git grep 'user\.last_name' my-feature

If there are matching results, they follow this format:

my-feature:app/views/users/show.html.erb:  <%= user.last_name %>
...

This formatting is handy because you can easily copy the branch and file directive for use with git-show.

See man git-grep for more details.

Viewing A File On Another Branch

Sometimes you want to view a file on another branch (without switching branches). That is, you want to view the version of that file as it exists on that branch. git show can help. If your branch is named my_feature and the file you want to see is app/models/users.rb, then your command should look like this:

$ git show my_feature:app/models/users.rb

You can even tab-complete the filename as you type it out.

See man git-show for more details.

source

New PostgreSQL 9.6 slice syntax

PostgreSQL arrays support slice operations.

jack=# select ('{a,b,c,d,e,f}'::text[])[2:4];
  text
---------
 {b,c,d}

New in PostgreSQL 9.6 is slice syntax with implicit lower and upper bounds.

jack=# select ('{a,b,c,d,e,f}'::text[])[:4];
   text
-----------
 {a,b,c,d}
jack=# select ('{a,b,c,d,e,f}'::text[])[3:];
   text
-----------
 {c,d,e,f}

Previously, array_lower and array_upper was needed to slice from a particular index to the bounds of an array.