Today I Learned

A Hashrocket project

What Is On The Runtime Path?

All of the plugins, syntax highlighting, language-specific indentation that extend the default behavior of Vim are on the runtime path. If something isn't on Vim's runtime path, then Vim won't know about and as a result will not load it at runtime.

How do we see what is on the runtime path?

The rtp option is the shorthand for runtimepath. Calling set on either of these will show us the list of runtime paths, or at least some of them.

:set rtp

This will generally be a truncated list if you have a lot of plugins. To be sure you are seeing all of them, use echo instead.

:echo &rtp

See :h rtp for more details.

h/t Chris Erin

Private vs Public Struct members

In Golang, identifiers can be either Exported or Unexported. In other languages this would be known as public or private, but in golang the question is whether the identifiers should be able to be seen outside of the module. If not? Don't export them.

The rule is simple, if an identifier is capitalized it will be exported. This manifests itself in golang when converting a struct to JSON.

type Apple struct {
    color string
    weight int
}

a := json.Marshal(Apple{"green", 10})
fmt.Println(a)

In this case this code will print {}. To print what you would expect the struct should be defined with capitalized identifiers:

type Apple struct {
    Color string
    Weight int
}

Now the marshalled struct should be {"Color": "green", "Weight": 10}

Add files with matches to the args list

If I want to change all instances of the word Apple to Orange across my entire project I can use the vim command:

:argdo %s/Apple/Orange/gc

But only if the files that have the word Apple are in the args list. To populate the args list you can use the args command in combination with shelling out to git grep with the -l flag.

:args `git grep -l Apple`

Then, when you see the list of files with :args, you'll see all the files with the word Apple.

A Mnemonic for Changing Modes

Making a script file executable involves an incantation that has something to do with chmod but what exactly? chmod has two modes, absolute and symbolic. In absolute mode, you set the entire set of permissions with a numeric value like 755.

In symbolic mode, you can add or subtract permissions to either the user (u), group (g) or owner (o). To give read permissions to the group type:

> chmod g+r myfile.txt

And to give execution priviledges to the owner of a file:

> chmod o+x myscript.rb

Netrw Special Command

Building off this previous post, today I learned a new command in Netrw: the 'special command'.

When navigating, enter x over any file, and your operating system will open it with its tool of choice (Preview for images and Finder for directories on a Mac, etc.). This is great for exploring a directory.

From the Netrw help:

Certain files, such as html, gif, jpeg, (word/office) doc, etc, files, are
best seen with a special handler (ie. a tool provided with your computer's
operating system).  Netrw allows one to invoke such special handlers by:

        * when Exploring, hit the "x" key
        * when editing, hit gx with the cursor atop the special filename
          (latter not available if the g:netrw_nogx variable exists)

Send SQL to a Tmux Window

Hashrocket.vim provides a mapping that is awesome:

autocmd FileType sql nmap <buffer> <leader>t :<C-U>w \| call Send_to_Tmux("\\i ".expand("%")."\n")<CR>

With this command, if you're editing an SQL file in a Tmux session, run <leader>t from normal mode, and your file will be read into the session and window of your choice. If there's a psql session listening there, you've got a fast way to iterate on an SQL statement from Vim.

The command even writes the file for you before sending the SQL.

Chaining expectations in Rspec

Generally, we think about expectations in RSpec one at a time. If the first expectation fails, then don't go any further. Expectations in RSpec however are chainable, meaning, I can attach one expectation to another for the same subject and then know about the failures or successes for both expections, that looks like this.

expect(1).to eq(2).and eq(3)

Which produces output like this:

 Failure/Error: expect(1).to eq(2).and eq(3)

          expected: 2
               got: 1

          (compared using ==)

       ...and:

          expected: 3
               got: 1

          (compared using ==)

The same result can be got from the below code which may appeal to you a bit more:

def chain_exp(*expects)
  expects.inject {|exps, exp| exps.and(exp)}
end

expect(1).to chain_exp(eq(2), eq(3))

Inspecting The Process Message Queue

A core tenant of Elixir is message passing between processes. So, if a process is sent a message, where does that message go? What happens if it gets sent many messages? The Process.info/2 function allows us to inspect the message queue.

First, let's send some messages (to ourself) and then keep an eye on the length of the message queue as we go.

> send self(), {:error, "this is bad"}
{:error, "this is bad"}
> Process.info(self(), :message_queue_len)
{:message_queue_len, 1}
> send self(), {:hello, "world"}
{:hello, "world"}
> Process.info(self(), :message_queue_len)
{:message_queue_len, 2}

Now, I am curious what those specific messages are. Let's ask Process.info/2 for the messages that are in the message queue.

> Process.info(self(), :messages)
{:messages, [error: "this is bad", hello: "world"]}

There are a lot of other things that Process.info/2 can tell us about a process. See the Erlang docs for process_info for more details.

Get the association with Ecto

In the schema of my post model I have this line:

belongs_to :developer

So a post is associated with a developer. When I have a post struct and try to access the developer I might see this:

> Tilex.Repo.get(Tilex.Post, 42).developer
#Ecto.Association.NotLoaded<association :developer is not loaded>

Getting the developer is now a two step process:

  1. Construct a query based on the association
  2. Use that query to acquire the struct

To construct a query based on the association we can use the Ecto.assoc/2 function:

query = Ecto.assoc(post, :developer)

Then this query is executed with the Repo.one function:

developer = Tilex.Repo.one(query)

There ya go!

Cherry Pick A Range Of Commits

Git's cherry-pick command allows you to specify a range of commits to be cherry picked onto the current branch. This can be done with the A..B style syntax -- where A is the older end of the range.

Consider a scenario with the following chain of commits: A - B - C - D.

$ git cherry-pick B..D

This will cherry pick commits C and D onto HEAD. This is because the lower-bound is exclusive. If you'd like to include B as well. Try the following:

$ git cherry-pick B^..D

See man git-cherry-pick for more details.

Rails/PG Statement Timeout 🐘⏰

By default, Rails does not set a timeout on database statements. For example, this will run for a full day, even if your ruby process goes away.

ActiveRecord::Base.connection.execute(<<~SQL)
  select pg_sleep(86400);
SQL

The good news is that Rails provides a way to prevent this from happening from the database.yml with the statement_timeout variable.

default: &default
  adapter: postgresql
  ...
  variables:
    statement_timeout: 5000

I'm starting to think that this should always be set low. And then explicitly set higher on a per-query basis, when one is expecting something to take a long time.

Login for feature test with warden test helpers

Warden provides a way to login as a user without having to go through the web interface that a user generally sees for sign in.

user = FactoryGirl.create(:user)
login_as user, scope: :user

In the config block for Rspec you would include this statement:

config.include Warden::Test::Helpers, type: :feature

If you have different models for different types of users in your system you can sign in with different scopes. Lets say you have a student user concept, you can sign in with:

student = FactoryGirl.create(:student)
login_as student, scope: :student

Customizing the output of `ps`

There's a lot to know about processes, although typically we're looking for a mapping of pids to commands.

You can customize the output of ps with -o. The -o flag expects a comma separated list of keywords which indicate the columns that you'd like to display for each process.

> ps -o pid,command

The above command shows only the pid and the command w/args for each process. You can see all the possible keywords with ps -L.

erlang memory usage

You might be be curious how much memory your erlang vm is taking. You can get that and more from :erlang.memory. Below is what I get from running this after starting iex normally.

> :erlang.memory
[total: 18133840, processes: 4975616, processes_used: 4974616, system: 13158224,
 atom: 264529, atom_used: 256719, binary: 228728, code: 6640396, ets: 414904]

This list is confusing. All numbers are bytes, so in total 18133840 bytes are allocated to the erlang process. There are two subcategories processes and system and the values of those categories sum to equal the value of total.

The remaining categories are either a component of system or a component of processes.

Listing Files In IEx

When you start an IEx session, you do so in the context of some directory -- the current working directory. This context can be important if you need to do something like import a file. In fact, you may want to know what files are available in the current working directory.

You can list them all out within IEx using ls/0.

iex(1)> ls()
           .git     .gitignore      README.md         _build         assets         config
           deps            lib        mix.exs       mix.lock           priv           test
            tmp

You can also list the contents of some other specific directory by naming it when invoking ls/1.

See h() within IEx for more details.

When Was This File Added to Version Control?

I learned a helpful Git command today. To determine when a file was added to version control, run the following:

$ git log --follow --diff-filter=A <filename>

This returns the Git history, following files through renames, filtering on when a file was added to version control. Here's the output for the README of 'Today I Learned':

$ git log --follow --diff-filter=A README.md
commit 9a7984a48db19489bb4113378298ddaa97121c7a
Author: Jake Worth <dev+jwworth@hashrocket.com>
Date:   Sat Mar 28 12:52:19 2015 -0400

    Add a basic README

Useful for explaining mystery files in the repository 🔎.

Related File in Rails.vim

Rails.vim is a staple of our Hashrocket workstations. It continues to surprise me.

Today I discovered the :R (related) command, which is a counterpart to the :A (alternate) command. :R looks for the file related to your current buffer.

This command is sophisticated. Assuming you're following Rails conventions for file organization, :R over most templates will open the associated controller, with your cursor on the line where the action that corresponds with the view name (show for a template called show.html.erb) is defined. The same thing works in reverse; :R over the show method leads you back to show.html.erb.

Use Elixir plug for only some controller actions

Phoenix allows for a flexible request processing pipeline with plugs. Your controllers are basically plugs in the plug pipeline. You can specify plugs that will execute before your controller actions with the plug macro.

defmodule MyApp.ThingController do
  use MyApp.Web, :controller

  plug MyApp.InterestingPlug
end

You can also specify a guard for this macro that will only enable this plug to be run before certain actions:

plug MyApp.InterestingPlug when action in [:new, :create]

Am I connected to a terminal via ssh?

When connecting to another computer via ssh the computer will have some environment variables that are not set otherwise. You can examine the environment variables with SSH in the name with:

> printenv | grep SSH

And in both linux and macos you'll see an environment variable:

SSH_CONNECTION=192.168.50.3 65102 192.168.50.12 22

The first ip address and corresponding port represents the ip address if one exists and the second is the machine you are currently ssh'd into.

Update The URL Of A Remote

I just changed the name of a Github repository. One of the implications of this is that the remote URL that my local git repository has on record is now out of date. I need to update it.

If I use git-remote with the -v flag. I can see what remotes I currently have.

$ git remote -v
origin  git@github.com:jbranchaud/pokemon.git (fetch)
origin  git@github.com:jbranchaud/pokemon.git (push)

Now, to update the URL for that remote, I can use git remote set-url specifying the name of the remote and the updated URL.

$ git remote set-url origin git@github.com:jbranchaud/pokemon_deluxe.git

If I check again, I can see it has been updated accordingly.

$ git remote -v
origin  git@github.com:jbranchaud/pokemon_deluxe.git (fetch)
origin  git@github.com:jbranchaud/pokemon_deluxe.git (push)

Defining Multiple Clauses In An Anonymous Function

Anonymous functions often take the approach of doing a single thing with the inputs, regardless of their shape or values. There is no need to limit ourselves though. The same pattern matching that we use all over our Elixir programs can be utilized to define multiple clauses in an anonymous function as well.

Consider the following example:

iex> my_function = fn
  {:ok, x} -> "Everything is ok: #{x}"
  {:error, x} -> "There was an error: #{x}"
end
#Function<6.52032458/1 in :erl_eval.expr/5>

We can then invoke our anonymous function using the bound variable to see what results we get with different kinds of inputs.

iex> my_function.({:ok, 123})
"Everything is ok: 123"
iex> my_function.({:error, "be warned"})
"There was an error: be warned"

source

cat from stdin and historical notes

Generally, cat takes a file as an argument and prints the file to stdout. cat can also print to stdout from stdin like this:

> echo abc | cat -
abc

Which is the equivalent to:

> echo abc | cat
abc

But when you need to concatenate a line to the beginning of a file and then process that result it comes in handy:

> echo abc | cat - 123.txt
abc
123
> echo abc | cat - 123.txt | grep abc
abc

The macosx cat man page (here) has a history section and mentions that:

Dennis Ritchie designed and wrote the first man page. It appears to have been cat(1).

So cat was the first man page, interesting! (The ubuntu man page for cat does not mention this)

Size of RSA encrypted content

RSA encryption (publickey/privatekey) is powerful encryption but its really only meant to encrypt small things like other encryption keys. The maximum size of the amount of data an rsa public key can encrypt is:

(key_size/8).floor - 11

So in the case of a key thats 1024 bits it can only encrypt data up to 117 bytes long. Not long enough for documents of any size or variable length.

Add email to github account

You can have multiple emails associated with your github account so that if your commits have user_x@example.com or on another machine your commits have user_y@example.com they'll both show up as you on the github system.

In my case, all commits on hashrocket workstations are dev+somebody@hashrocket.com and we change who that somebody is with the pear application. So when I'm working on a hashrocket workstation I commit as dev+chriserin@hashrocket.com.

When editing a profile on github there is a left hand menu labeled 'personal settings' and in that menu there is an emails link. When you click emails you have the opportunity to 'Add email address'. There ya go! Happy committing!

H/T Matt Polito

Redirect outputs with subshell

Today I learned how to use a Subshell to redirect multiple outputs as a single one. This way I can run a bunch of commands and unify all the results as it was a single command.

(git diff --name-only; git diff --staged --name-only) | sort -u

The above example will show me a sorted list of changed files (including the staged ones) in a git project without duplicates.

Here's another example:

$ echo "world"; echo "hello" | sort -u
#=> world
#=> hello

$ (echo "world"; echo "hello") | sort -u
#=> hello
#=> world

The Registry is back

The Registry is a bad Windows dream, something left behind when I switched off Windows for good circa 2011.

It's back. It's a new Elixir 1.4 module where you can store values that you'd like to access globally. Potentially, you want to store a pid of a worker you started. You can store that in the registry like so.

{:ok, pid} = MyWorker.start_link()
supervisor(Registry, [:unique, Registry.WorkerPids])
Registry.register(Registry.WorkerPids, :my_worker, worker_pid)
[{_, found_worker_pid}] = Registry.lookup(Registry.WorkerPids, :my_worker)

A lot is going on here.

  • We start a worker
  • We start the registry process for register Registry.WorkerPids
  • We register the key/value in a specific register (Registry.WorkerPids)
  • We lookup the value in that register with that key

Its important to note that the Registry.WorkerPids symbol is just an atom, not a module. There is no WorkerPids module implementing any special functionality anywhere.

Definitely a lot of ceremony for a key/value store. I expect to find practical uses for it as I experience more complex Elixir systems.

Shhh.... Curl silently.

The curl program generally displays a progress bar as its making a request and receiving a response. This can be awkward when you are trying to wrap curl programmatically.

To shut off the progress bar use:

curl -sS http://www.google.com

The first option (-s) hides the progress bar and the errors. The second option (-S) will expose an error on fail if used in conjunction with -s.

Re-mark a Word as Bad

Vim's spell checking feature is pretty great. I use it as a default for .md and .rdoc files on my machine.

Sometimes it incorrectly marks technical words like 'ExUnit' as bad (misspelled). To fix this, I type zg over the word in normal mode, which adds it to a list of white-listed words that are no longer considered bad.

The opposite of that command is zw, which comments out the line in the dictionary file. Run :runtime spell/cleanadd.vim to clean up those comments.

Revealing Rails Scopes

I've been working on some Rails code that brings in ActiveRecord models from multiple gems. Often these models have default scopes, that bane of a legacy Rails codebase, and figuring that out requires source diving one or more gems. Today I hacked my way to a faster solution: just read the SQL Rails generates.

Here's a post without a default scope, and then one with a default scope:

pry(main)> Post.all.to_sql
=> "SELECT \"posts\".* FROM \"posts\""
pry(main)> Developer.all.to_sql
=> "SELECT \"developers\".* FROM \"developers\" ORDER BY \"developers\".\"username\" ASC"

I see you, ORDER BY.

Age in Postgres

On this day every year I get curious about how old I actually am. Postgres has some utilities to help me out with that, like subtraction of dates.

> select now() - '1978-3-24';
14245 days 17:39:55.623866

Ah yes, 14245 days on this earth, it really makes you think. What got returned anyways, that looks weird:

> select pg_typeof(now() - '1978-3-24');
interval

Got it, that's an interval. It doesn't really tell me much, is this interval infinite?

> select isfinite(now() - '1978-3-24');
t

I can squint and read isfinite as infinite and suddenly my outlook changes, but only momentarily.

Postgres, just tell me how many years I've been alive ok? None of this days stuff.

> select age('1978-3-24'::date);
39 years

Chaining Multiple RSpec Change Matchers

It can be handy to use RSpec's change matchers to determine if some method or process creates a new record.

expect{ Registration.create(attrs) }.to change{ User.count }.by(1)

But what if we are testing a method that creates a couple different records in the system?

RSpec allows us to chain together change matchers with and. Consider this additional contrived example.

expect {
  Project.generate(attrs)
}.to change{ Project.count }.by(1).and \
     change{ User.count }.by(1)

In addition to keeping our tests tight and concise, this approach gives some pretty nice output on failure.

If we were just beginning our implementation with a failing test, we'd see a multi-part failure like the following.

Failure/Error:
  expect {
    Project.generate(attrs)
  }.to change{ Project.count }.by(1).and \
       change{ User.count }.by(1)

     expected result to have changed by 1, but was changed by 0

  ...and:

     expected result to have changed by 1, but was changed by 0

Disk usage for just the top level dirs

Every so often your hard drive runs out of space and you need to do a survey of what could be possibly be taking up so much space. The du disk-usage command is great for that.

It can be noisy though as it traverses through all the directories and gives you a report on the size of each file. What we really want is just the size of each directory under the root dir.

du has a depth flag (-d) to help control the depth of directories that the command reports on.

du -h -d1 /

The above command gives you a report on all the directories and files at the top level.

Invisible components in React Native

Sometimes you want a component to be rendered but not visible to the user and in React Native 0.42 and below, you could accomplish that by setting the component's height and width to 0.

The good news is that React Native 0.43 will make it easier and introduce the CSS property you are already familiar with: display: 'none'.

You can already try it installing the pre-release version 0.43.0-rc.4.

Run Prettier on all #JavaScript files in a dir

If you are like me you must like formatters such as Prettier which probably prompted you to set your editor to auto format the file on save.

That's great for new projects but when working on an existing project, every file you touch will have a huge diff in git that can obscure the real changes made to the file.

To solve that you must run prettier on all your javascript files as an independent commit. You can do it with the following command:

find ./src/**/*.js | xargs prettier --write --print-width 80 --single-quote --trailing-comma es5

The flags after the prettier are all my personal preferences except for --write which tells prettier to write the file in place.

Note 1: Make sure you have all the files you are about to change committed to source control so that you can check them out if this did not go well.

Note 2: When committing this change it would be a good idea to use git add -p and go through the changes one by one (which is always a good idea...)

Note 3: To dry run and see which files will be changed run the find ./src/**/*.js by itself.

Erlang documentation with `erl -man`

Erlang can be tricky, so documentation is a must, but when you're 30,000 feet high in an airplane with no internet you might regret not having downloaded DashApp and with it all the Erlang documentation.

But it was on your machine the whole time! You have access to all the Erlang documentation with erl -man used like so:

erl -man calendar

Which pops open a man page for the calendar module.

When you are learning Erlang the docs are at your fingertips.

List Available File Types For Ack

The ack utility allows you to filter the searched files based on file type. If you'd like to know all of the file types available, you can use the --help=types flag. This will include file types you've specified in your .ackrc file.

Here is a sample of some of the output.

$ ack --help=types
    ...
    --[no]css          .css .less .scss
    --[no]dart         .dart
    --[no]delphi       .pas .int .dfm .nfm .dof .dpk .dproj .groupproj .bdsgroup .bdsproj
    --[no]elisp        .el
    --[no]elixir       .ex .exs
    --[no]erlang       .erl .hrl
    --[no]fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp
    --[no]go           .go
    --[no]groovy       .groovy .gtmpl .gpp .grunit .gradle
    --[no]haskell      .hs .lhs
    --[no]hh           .h
    --[no]html         .html .mustache .handlebars .tmpl
    --[no]jade         .jade
    --[no]java         .java .properties
    --[no]js           .js
    ...

See man ack for more details.

Extended file attributes on macOS

You can store metadata on any file in the mac filesystem(HFS+). If you want to ensure the file has a specific encoding, or if you want to place a checksum with the file you can use file attributes.

Setting and reading are easy on macOS with the xattr utility:

> touch something.txt
> xattr -w xyz 123 something.txt
> xattr -p xyz something.txt
123

In this example we wrote(xattr -w) an attribute onto the file and printed it(xattr -p) to see that it was written.

You can list the file attributes with a cryptic option for ls, ls -l@. Try doing this on your ~/Downloads dir to learn something cool about how macOS treats files coming from the internet.

Remove One List From Another

The --/2 operator allows you to subtract two lists, that is, remove all elements in the right list from the left list. Each occurrence of an element is removed if there is a corresponding element. If there is no corresponding element, it is ignored.

Here are some examples.

> [1, 2, 3] -- [2, 4]
[1, 3]
> [:a, :b, :c, :a, :d, :a] -- [:a, :a]
[:b, :c, :d, :a]

This kind of list operation is not particularly efficient, so for large lists it can be quite slow. The following example took several minutes to run.

> Enum.into(1..1000000, []) -- Enum.into(2..1000000, [])
[1]

To achieve a true set difference, you'll note that the docs for this operator recommend checking out MapSet.difference/2.

See h Kernel.-- for more details.

RSpec Covers a Range

Today I used a feature of RSpec I've never used before, cover.

cover returns true if it's argument is covered by the given range:

expect(0..1).to cover(order.risk_score)

In my use case, I had an attribute risk_score that is calculated by a third-party API. It should always be between zero and one, but it changes on every test run because the API considers the test user more and more risky each time. cover allowed me to test this attribute, and the underlying logic, in a flexible way.

Inspect with Label

IO.inspect returns the value you pass in as the first argument. This is great when you want to debug the middle state of a series of piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Here, inspect will write to stdout the values [{"A", 1}, {"B", 2"}...].

IO.inspect also takes a label option, which decorates the values written to stdout but does not disrupt the piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect(label: "Letters with index:")
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Which outputs Letters with index: [{"A", 1}, {"B", 2"}...] to stdout.

Bits of Characters

If you are curious what the binary of an ascii/utf-8 char is you can use this string of commands at the command line:

echo 'A' | xxd -b

The A character is 65 in ascii which is 64 + 1. 01000000 is 64. 00000001 is of course 1, 01000001 is 65.

echo 'a' | xxd -b

The a character is 97 in ascii which is 64 + 32 + 1. 00100000 is 32 in binary, given this and the above, 01100001 is 97.

echo '🤓' | xxd -b

This ridiculous emoji is a utf-8 char. When you look at the binary for it:

11110000 10011111 10100100 10010011

You can see that every byte begins with a 1 which means that it will combine with any subsequent byte beginning with 1 to form a unique character.