Today I Learned

A Hashrocket project

199 posts by chriserin @mcnormalmode

Expect page to have a link... an exact link

Sometimes, while writing integration tests, we expect there to not be a link on a page, for instance if our page looked like this:

<a href="#create_jetpack">Create Jetpack</a>

And we want to make sure there is no general create link so we expect this:

expect(page).to_not have_link('Create')

But the expectation fails! By default capybara will match if the substring is anywhere in the content.

To avoid this problem use the exact flag like:

expect(page).to_not have_link('Create', exact: true)

Vim move cursor within a visual selection

When expanding a visual selection you might want to increase both sides. For instance, if you have made a selection from line 20 to line 40 you may want to expand the selection to line 18 to line 42. The visual selection can only be expanded with the cursor and the cursor is only on one side, but the cursor can be moved to the other side of the selection with o.

Read more about it with: :help visual-change.

Typing card suits in vim ♢♡♣♠

Digraphs are two letter combinations that provide a easy way to type and remember non-ascii characters in vim. In insert mode you can type a digraph with Ctrl-k and then two letters that map to the unicode character.

For card suits the pneumonic is lower case c for card and upper case C for clubs, D for diamonds, H for hearts, and S for spades.

cD ♢
cC ♣
cH ♡
cS ♠

You can see all the digraphs with :digraphs.

Now you have all the necessary unicode characters to easily create card games in the terminal!

Resetting the bundler --path

When running bundler install you can pass an option --path <dir>:

bundle install --path ./gems

Which will specify where to install the gems. This is a one time option. Everytime you invoke bundle from here on in you’ll see that its using the directory you specificed with the —path variable as the gem directory.

Bundler keeps track of this option in the .bundler/config file in the folder from where bundler was initially run. That file looks like this:

---
BUNDLE_PATH: "./gems"
BUNDLE_DISABLE_SHARED_GEMS: "true"

To reset the gems path back to the default, remove the BUNDLE_PATH line from the bundle/config file.f

Override ssh command for git

Git supports a number of environment variables one of them being GIT_SSH. You can override the default ssh command (ssh of course) with that environment variable.

GIT_SSH=./my_ssh git pull origin master

In this case my_ssh is a bash executable file that looks like this:

#!/bin/bash

ssh -i ~/.ssh/id_rsa $1 $2

An idea I got from this stack overflow answer.

Clearly the private key file I used is the default private key file so a bit redundant but its a way to add any options that might be necessary for the ssh connection, although specifying a host configuration in the ssh_config file might be simpler.

Reinstall all rubies with rvm

When compiling ruby the executable is attached to various C system libs like libreadline. If libreadline is updated then each ruby on the system attached to the old libreadline must be re-compiled. If you are using rvm then this can be done with one command:

rvm reinstall all --force

The --force flag will skip any question the process asks.

Be careful, if you are working on a machine that has installed many rubies and has had many projects for those rubies this could take an extraordinarily long time.

Specifying the private key to use in ssh

A private key is necessary in ssh to authenticate the connection securely. Generally ssh will look for a file in the ~/.ssh/ dir with a name of id_rsa (or if not using rsa a file beginning with id_ and the encryption algo name). That file should contain the private key.

You can, however, specify a different primary key at the command line with the -i flag like so:

ssh -i ~/.ssh/my_other_key.pem

Additionally, you may use the IdentityFile option the the ssh_config file to permanently configure the private key for a specific connection.

Hack to size the container of a background image

An element that contains a background image cannot have its height and width derived from that background image. An element can have a height and width that is determined by the size of a child element however. Combine the two to ensure that the container of the background image has the height and width of the image itself.

<div style="background-image: url('http://example.com/image.png')">
    <img src="http://example.com/image.png">
</div>

`border-collapse` to collapse borders

There are two settings for the css property border-collapse, separate and collapse. Table cells and the table itself can both have borders. If the value separate is set, then each cell has its own border, with the space in between the borders governed by the border-spacing property.

If collapse is set, then the table cells and the surrounding table will share the borders surrounding them.

`any` and `all` special Postgres constructs

To determine if a value exists in a subquery or scalar value list you can use the in operator like:

> select 1 in (1, 2)
true

any and all go a step further by allowing for an operator in the special construct which is used against each value in the subquery or array (no scalar value list for these constructs).

any examples:

> select 1 = any(Array[1, 2, 3])
true
> select 1 > any(Array[1, 2, 3])
false

all examples

> select 17 <> any(Array[2, 4, 8])
true
> select 17 < any(Array[2, 4, 8])
false

some is a synonym for any.

Remove, Get, And Compile a dependency

To get a better feel for how a dependency works I’ll do some IO.puts style debugging, but when I’m done I need to clean up all those IO.puts statements.

mix deps.clean ecto

The above command will remove two directories, the _build/dev/lib/ecto dir and the deps/ecto dir.

mix deps.get ecto

This command will resolve the dependency tree and get the appropriate version of ecto based on the information in your mix.exs file. It will not reach out to the internet if the package is already cached as a tar file in the .hex/packages/hexpm dir. It will unpack the tar file into the local deps/ecto dir but it will not compile it.

mix deps.compile ecto

Will compile ecto to the _build/dev/lib/ecto directory.

What file is this?

If you are working in an environment where the status line is not displayed and you don’t know what file you are currently editing, you can use the :file command to give you that information.

:file
"app/models/post.rb" line 43 of 139 --30%-- col 4

:f is the abbreviated version

Fingerpring of an ssh key in multiple formats

You might need to confirm which keys you have attached to your github account. Github provides a list of your keys with the associated fingerprints. If you have a public key on your machine you can see what fingerprint it has with:

> ssh-keygen -lf ~/.ssh/id_rsa.pub
4096 SHA256:/6Rat4zVZ0auHEwWtF4QG8NA0j4NAKWisFuXV0ZP5zk chris.erin@gmail.com (RSA)

Github displays an md5 fingerprint rather than a sha256 fingerprint. You can use the -E flag to choose a specific hash type.

> ssh-keygen -E md5 -lf ~/.ssh/id_rsa.pub
4096 MD5:e3:19:85:1c:8a:31:0f:4b:de:cb:6d:a7:64:b4:4e:e7 chris.erin@gmail.com (RSA)

Logging to a file in Elixir test environment

The default configuration for a logger in the test env is:

config :logger, level: :warn

That is, of the four logging levels (defug, info, warn and error) only log warnings and errors.

By default this logger logs to the console making the below redundant:

config :logger, level: :warn, backends: [:console]

:console is the only option available by default, but there are other backends available via hex package.

With hex package logger_file_backend you might have a different config:

config :logger, backends: [{LoggerFileBackend, :error_log}]

config :logger, :error_log, path: 'log/here.log'

Compile an Elixir dependency as you change it

In an Elixir project, the dependencies are one-time compiles. That makes sense when you know the dependency is not going to change. It stops making sense, however, when you need to explore how a dependency works by adding debug statements like IO.puts. Or when you find something in a dependency you want to change and need to test that change in an application.

In those cases Elixir provides the facility to compile a dependency each time it has a change in one of the component files just like how compilation in your mix app works. You can enable this by providing a path to the location of the dependency in the mix.exs file:

defp dep do
  [
  ...
  {:wallaby, '~> 0.16.1', path: 'deps/wallaby'}
  ...
  ]
end

In this case the path is set to the local deps dir, where the dependency already exists, but you can also set it to a working directory for that library.

Select to clipboard at the ubuntu command line

Copying and pasting without a mouse - or programmatically - can be incredibly challenging. Ubuntu provides an apt-get installable program xclip which can provide X11 clipboard access at the command line.

> echo PASTEME | xclip -sel clip

The value PASTEME is now in the clipboard buffer.

The -sel or -selection indicates the buffer that will be used, primary, secondary or clipboard. Generally, clipboard is the buffer that we want.

To output the value of the buffer use the -o or -output flag:

> xclip -sel clip -o
PASTEME

Keeping track of your CPU Heat

If you’ve ever built your own computer, you’ve had anxiety about not having put everything together right and maybe even something critical, like the mechanisms that keep the computer and the CPU cool.

On Ubuntu you can intall the lm-sensors software which provides you with the facilities to get information the various heat levels inside the computer.

> sudo apt-get install lm-sensors
> sensors
asus-isa-0000
Adapter: ISA adapter
cpu_fan:        0 RPM

acpitz-virtual-0
Adapter: Virtual device
temp1:        +27.8°C  (crit = +119.0°C)
temp2:        +29.8°C  (crit = +119.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +26.0°C  (high = +80.0°C, crit = +100.0°C)
Core 0:        +23.0°C  (high = +80.0°C, crit = +100.0°C)
Core 1:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 2:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 3:        +24.0°C  (high = +80.0°C, crit = +100.0°C)

Restart or shutdown ubuntu

Its easy to shutdown down Ubuntu from the commandline (given the correct permissions). Shutdown with shutdown which gives you 60 seconds to reverse that decision to shutdown by cancelling with shutdown -c. You can also reboot which takes effect right away.

shutdown # shutdown
shutdown -c # cancel that shutdown.
reboot # reboot the computer now

Ubuntu default desktop manager

What I like about linux is that there are configurations. You should be able to manipulate the whole system without a gui. The configuration for your default display manager, for instance, is in the default-display-manager file:

> cat /etc/X11/default-display-manager
/usr/sbin/lightdm

There’s a startup script /etc/init.d/lightdm that checks that the value stored in this file points to a lightdm program and then executes that program.

Opening Gnome terminal emulator

In an attempt to get better with keyboard shortcuts on Ubuntu I’ve - for the time being - ditched the mouse. I generally only use two applications, Firefox and Terminal.

Opening a new terminal is as easy as:

Ctrl + Shift + T

It starts rather small but I resize it by using left or right with the Super (window/apple/command) key.

Super + Left
Super + Right

Then its resized to either the left half of the screen or the right half of the screen.

Create a github repository from the cmd line

In the effort to never leave the command line anything to do with github is always a frustration. The hub command line tool - a tool that augments git with github specific commands - helps out tremendously.

In this case I want to create a github repository to push my code to without opening a browser, going to github, creating a repo, copying the remote address for that repo and then setting up the remote locally.

It can all be done with one step.

hub create "exciting_open_source_tech_repo"

git init must be run first, so that there is a local repo to link to the github repo, but then your good. Start modifying, adding and commiting as you normally would.

Webmock assertion that an http request was made

Webmock is a library that helps you manager the http requests that are made during a test. Its not advisable to hit production systems with test-generated requests and with webmock you can stub those requests. Sometimes to ensure functionality of your system you might want to assert that the request was made.

stub_request(:get, 'http://www.google.com')
assert_requested(:get, 'http://www.google.com')

The stub_request method also returns a variable that can be passed to the assert function.

get_google = stub_request(:get, 'http://www.google.com')
assert_requested get_google

Time travelling in rspec/rails

When basing logic on the current time its helpful for testing to have a stable time. A time that does not change. Rails has a module ActiveSupport::Testing::TimeHelpers that was added in Rails 4.2 to provide methods that manipulate the time during testing.

travel_to(Time.parse("2017-01-19")) do
  puts Time.now.strftime(:date)
end

puts Time.now.strftime(:date)

The above code outputs 2017-01-19 and 2017-05-02 (the current date). A fun way to time travel in modern ruby.

Private vs Public Struct members

In Golang, identifiers can be either Exported or Unexported. In other languages this would be known as public or private, but in golang the question is whether the identifiers should be able to be seen outside of the module. If not? Don’t export them.

The rule is simple, if an identifier is capitalized it will be exported. This manifests itself in golang when converting a struct to JSON.

type Apple struct {
    color string
    weight int
}

a := json.Marshal(Apple{"green", 10})
fmt.Println(a)

In this case this code will print {}. To print what you would expect the struct should be defined with capitalized identifiers:

type Apple struct {
    Color string
    Weight int
}

Now the marshalled struct should be {"Color": "green", "Weight": 10}

Add files with matches to the args list

If I want to change all instances of the word Apple to Orange across my entire project I can use the vim command:

:argdo %s/Apple/Orange/gc

But only if the files that have the word Apple are in the args list. To populate the args list you can use the args command in combination with shelling out to git grep with the -l flag.

:args `git grep -l Apple`

Then, when you see the list of files with :args, you’ll see all the files with the word Apple.

A Mnemonic for Changing Modes

Making a script file executable involves an incantation that has something to do with chmod but what exactly? chmod has two modes, absolute and symbolic. In absolute mode, you set the entire set of permissions with a numeric value like 755.

In symbolic mode, you can add or subtract permissions to either the user (u), group (g) or owner (o). To give read permissions to the group type:

> chmod g+r myfile.txt

And to give execution priviledges to the owner of a file:

> chmod o+x myscript.rb

Chaining expectations in Rspec

Generally, we think about expectations in RSpec one at a time. If the first expectation fails, then don’t go any further. Expectations in RSpec however are chainable, meaning, I can attach one expectation to another for the same subject and then know about the failures or successes for both expections, that looks like this.

expect(1).to eq(2).and eq(3)

Which produces output like this:

 Failure/Error: expect(1).to eq(2).and eq(3)

          expected: 2
               got: 1

          (compared using ==)

       ...and:

          expected: 3
               got: 1

          (compared using ==)

The same result can be got from the below code which may appeal to you a bit more:

def chain_exp(*expects)
  expects.inject {|exps, exp| exps.and(exp)}
end

expect(1).to chain_exp(eq(2), eq(3))

Get the association with Ecto

In the schema of my post model I have this line:

belongs_to :developer

So a post is associated with a developer. When I have a post struct and try to access the developer I might see this:

> Tilex.Repo.get(Tilex.Post, 42).developer
#Ecto.Association.NotLoaded<association :developer is not loaded>

Getting the developer is now a two step process:

  1. Construct a query based on the association
  2. Use that query to acquire the struct

To construct a query based on the association we can use the Ecto.assoc/2 function:

query = Ecto.assoc(post, :developer)

Then this query is executed with the Repo.one function:

developer = Tilex.Repo.one(query)

There ya go!

Login for feature test with warden test helpers

Warden provides a way to login as a user without having to go through the web interface that a user generally sees for sign in.

user = FactoryGirl.create(:user)
login_as user, scope: :user

In the config block for Rspec you would include this statement:

config.include Warden::Test::Helpers, type: :feature

If you have different models for different types of users in your system you can sign in with different scopes. Lets say you have a student user concept, you can sign in with:

student = FactoryGirl.create(:student)
login_as student, scope: :student

Customizing the output of `ps`

There’s a lot to know about processes, although typically we’re looking for a mapping of pids to commands.

You can customize the output of ps with -o. The -o flag expects a comma separated list of keywords which indicate the columns that you’d like to display for each process.

> ps -o pid,command

The above command shows only the pid and the command w/args for each process. You can see all the possible keywords with ps -L.

erlang memory usage

You might be be curious how much memory your erlang vm is taking. You can get that and more from :erlang.memory. Below is what I get from running this after starting iex normally.

> :erlang.memory
[total: 18133840, processes: 4975616, processes_used: 4974616, system: 13158224,
 atom: 264529, atom_used: 256719, binary: 228728, code: 6640396, ets: 414904]

This list is confusing. All numbers are bytes, so in total 18133840 bytes are allocated to the erlang process. There are two subcategories processes and system and the values of those categories sum to equal the value of total.

The remaining categories are either a component of system or a component of processes.

Use Elixir plug for only some controller actions

Phoenix allows for a flexible request processing pipeline with plugs. Your controllers are basically plugs in the plug pipeline. You can specify plugs that will execute before your controller actions with the plug macro.

defmodule MyApp.ThingController do
  use MyApp.Web, :controller

  plug MyApp.InterestingPlug
end

You can also specify a guard for this macro that will only enable this plug to be run before certain actions:

plug MyApp.InterestingPlug when action in [:new, :create]

Am I connected to a terminal via ssh?

When connecting to another computer via ssh the computer will have some environment variables that are not set otherwise. You can examine the environment variables with SSH in the name with:

> printenv | grep SSH

And in both linux and macos you’ll see an environment variable:

SSH_CONNECTION=192.168.50.3 65102 192.168.50.12 22

The first ip address and corresponding port represents the ip address if one exists and the second is the machine you are currently ssh’d into.

cat from stdin and historical notes

Generally, cat takes a file as an argument and prints the file to stdout. cat can also print to stdout from stdin like this:

> echo abc | cat -
abc

Which is the equivalent to:

> echo abc | cat
abc

But when you need to concatenate a line to the beginning of a file and then process that result it comes in handy:

> echo abc | cat - 123.txt
abc
123
> echo abc | cat - 123.txt | grep abc
abc

The macosx cat man page (here) has a history section and mentions that:

Dennis Ritchie designed and wrote the first man page. It appears to have been cat(1).

So cat was the first man page, interesting! (The ubuntu man page for cat does not mention this)

Size of RSA encrypted content

RSA encryption (publickey/privatekey) is powerful encryption but its really only meant to encrypt small things like other encryption keys. The maximum size of the amount of data an rsa public key can encrypt is:

(key_size/8).floor - 11

So in the case of a key thats 1024 bits it can only encrypt data up to 117 bytes long. Not long enough for documents of any size or variable length.

Add email to github account

You can have multiple emails associated with your github account so that if your commits have user_x@example.com or on another machine your commits have user_y@example.com they’ll both show up as you on the github system.

In my case, all commits on hashrocket workstations are dev+somebody@hashrocket.com and we change who that somebody is with the pear application. So when I’m working on a hashrocket workstation I commit as dev+chriserin@hashrocket.com.

When editing a profile on github there is a left hand menu labeled ‘personal settings’ and in that menu there is an emails link. When you click emails you have the opportunity to ‘Add email address’. There ya go! Happy committing!

H/T Matt Polito

The Registry is back

“The Registry” is a bad Windows dream, something left behind when I switched off Windows for good circa 2011.

It’s back. It’s a new Elixir 1.4 module where you can store values that you’d like to access globally. Potentially, you want to store a pid of a worker you started. You can store that in the registry like so.

{:ok, pid} = MyWorker.start_link()
supervisor(Registry, [:unique, Registry.WorkerPids])
Registry.register(Registry.WorkerPids, :my_worker, worker_pid)
[{_, found_worker_pid}] = Registry.lookup(Registry.WorkerPids, :my_worker)

A lot is going on here.

  • We start a worker
  • We start the registry process for register Registry.WorkerPids
  • We register the key/value in a specific register (Registry.WorkerPids)
  • We lookup the value in that register with that key

Its important to note that the Registry.WorkerPids symbol is just an atom, not a module. There is no WorkerPids module implementing any special functionality anywhere.

Definitely a lot of ceremony for a key/value store. I expect to find practical uses for it as I experience more complex Elixir systems.

Shhh.... Curl silently.

The curl program generally displays a progress bar as its making a request and receiving a response. This can be awkward when you are trying to wrap curl programmatically.

To shut off the progress bar use:

curl -sS http://www.google.com

The first option (-s) hides the progress bar and the errors. The second option (-S) will expose an error on fail if used in conjunction with -s.

Age in Postgres

On this day every year I get curious about how old I actually am. Postgres has some utilities to help me out with that, like subtraction of dates.

> select now() - '1978-3-24';
14245 days 17:39:55.623866

Ah yes, 14245 days on this earth, it really makes you think. What got returned anyways, that looks weird:

> select pg_typeof(now() - '1978-3-24');
interval

Got it, that’s an interval. It doesn’t really tell me much, is this interval infinite?

> select isfinite(now() - '1978-3-24');
t

I can squint and read isfinite as infinite and suddenly my outlook changes, but only momentarily.

Postgres, just tell me how many years I’ve been alive ok? None of this days stuff.

> select age('1978-3-24'::date);
39 years

Disk usage for just the top level dirs

Every so often your hard drive runs out of space and you need to do a survey of what could be possibly be taking up so much space. The du disk-usage command is great for that.

It can be noisy though as it traverses through all the directories and gives you a report on the size of each file. What we really want is just the size of each directory under the root dir.

du has a depth flag (-d) to help control the depth of directories that the command reports on.

du -h -d1 /

The above command gives you a report on all the directories and files at the top level.

Erlang documentation with `erl -man`

Erlang can be tricky, so documentation is a must, but when you’re 30,000 feet high in an airplane with no internet you might regret not having downloaded DashApp and with it all the Erlang documentation.

But it was on your machine the whole time! You have access to all the Erlang documentation with erl -man used like so:

erl -man calendar

Which pops open a man page for the calendar module.

When you are learning Erlang the docs are at your fingertips.

Extended file attributes on macOS

You can store metadata on any file in the mac filesystem(HFS+). If you want to ensure the file has a specific encoding, or if you want to place a checksum with the file you can use file attributes.

Setting and reading are easy on macOS with the xattr utility:

> touch something.txt
> xattr -w xyz 123 something.txt
> xattr -p xyz something.txt
123

In this example we wrote(xattr -w) an attribute onto the file and printed it(xattr -p) to see that it was written.

You can list the file attributes with a cryptic option for ls, ls -l@. Try doing this on your ~/Downloads dir to learn something cool about how macOS treats files coming from the internet.

Inspect with Label

IO.inspect returns the value you pass in as the first argument. This is great when you want to debug the middle state of a series of piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Here, inspect will write to stdout the values [{"A", 1}, {"B", 2"}...].

IO.inspect also takes a label option, which decorates the values written to stdout but does not disrupt the piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect(label: "Letters with index:")
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Which outputs Letters with index: [{"A", 1}, {"B", 2"}...] to stdout.

Bits of Characters

If you are curious what the binary of an ascii/utf-8 char is you can use this string of commands at the command line:

echo 'A' | xxd -b

The A character is 65 in ascii which is 64 + 1. 01000000 is 64. 00000001 is of course 1, 01000001 is 65.

echo 'a' | xxd -b

The a character is 97 in ascii which is 64 + 32 + 1. 00100000 is 32 in binary, given this and the above, 01100001 is 97.

echo '🤓' | xxd -b

This ridiculous emoji is a utf-8 char. When you look at the binary for it:

11110000 10011111 10100100 10010011

You can see that every byte begins with a 1 which means that it will combine with any subsequent byte beginning with 1 to form a unique character.