Today I Learned

A Hashrocket project

184 posts by chriserin @mcnormalmode

Setting the desktop picture in Gnome

Gnome's going to be the default desktop for Ubuntu as of version 17.10. To me this is exciting stuff and its never too early to start learning how to configure the gnome desktop.

Here's a command line statement that will set the desktop image.

> gsettings set org.gnome.desktop.background picture-uri file:///home/myname/Downloads/Major_Oak.JPG

Compile time purge level

Elixir can conditionally remove log statements of the form Logger.debug/2 to Logger.error/2 at compile time.

config :logger, compile_time_purge_level: :info

In this configuration, logger statements like Logger.debug("something inconsequential happened") will be compiled out altogether.

Logging to a file in Elixir test environment

The default configuration for a logger in the test env is:

config :logger, level: :warn

That is, of the four logging levels (defug, info, warn and error) only log warnings and errors.

By default this logger logs to the console making the below redundant:

config :logger, level: :warn, backends: [:console]

:console is the only option available by default, but there are other backends available via hex package.

With hex package logger_file_backend you might have a different config:

config :logger, backends: [{LoggerFileBackend, :error_log}]

config :logger, :error_log, path: 'log/here.log'

Compile an Elixir dependency as you change it

In an Elixir project, the dependencies are one-time compiles. That makes sense when you know the dependency is not going to change. It stops making sense, however, when you need to explore how a dependency works by adding debug statements like IO.puts. Or when you find something in a dependency you want to change and need to test that change in an application.

In those cases Elixir provides the facility to compile a dependency each time it has a change in one of the component files just like how compilation in your mix app works. You can enable this by providing a path to the location of the dependency in the mix.exs file:

defp dep do
  [
  ...
  {:wallaby, '~> 0.16.1', path: 'deps/wallaby'}
  ...
  ]
end

In this case the path is set to the local deps dir, where the dependency already exists, but you can also set it to a working directory for that library.

Select to clipboard at the ubuntu command line

Copying and pasting without a mouse - or programmatically - can be incredibly challenging. Ubuntu provides an apt-get installable program xclip which can provide X11 clipboard access at the command line.

> echo PASTEME | xclip -sel clip

The value PASTEME is now in the clipboard buffer.

The -sel or -selection indicates the buffer that will be used, primary, secondary or clipboard. Generally, clipboard is the buffer that we want.

To output the value of the buffer use the -o or -output flag:

> xclip -sel clip -o
PASTEME

Keeping track of your CPU Heat

If you've ever built your own computer, you've had anxiety about not having put everything together right and maybe even something critical, like the mechanisms that keep the computer and the CPU cool.

On Ubuntu you can intall the lm-sensors software which provides you with the facilities to get information the various heat levels inside the computer.

> sudo apt-get install lm-sensors
> sensors
asus-isa-0000
Adapter: ISA adapter
cpu_fan:        0 RPM

acpitz-virtual-0
Adapter: Virtual device
temp1:        +27.8°C  (crit = +119.0°C)
temp2:        +29.8°C  (crit = +119.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +26.0°C  (high = +80.0°C, crit = +100.0°C)
Core 0:        +23.0°C  (high = +80.0°C, crit = +100.0°C)
Core 1:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 2:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 3:        +24.0°C  (high = +80.0°C, crit = +100.0°C)

Restart or shutdown ubuntu

Its easy to shutdown down Ubuntu from the commandline (given the correct permissions). Shutdown with shutdown which gives you 60 seconds to reverse that decision to shutdown by cancelling with shutdown -c. You can also reboot which takes effect right away.

shutdown # shutdown
shutdown -c # cancel that shutdown.
reboot # reboot the computer now

Ubuntu default desktop manager

What I like about linux is that there are configurations. You should be able to manipulate the whole system without a gui. The configuration for your default display manager, for instance, is in the default-display-manager file:

> cat /etc/X11/default-display-manager
/usr/sbin/lightdm

There's a startup script /etc/init.d/lightdm that checks that the value stored in this file points to a lightdm program and then executes that program.

Opening Gnome terminal emulator

In an attempt to get better with keyboard shortcuts on Ubuntu I've - for the time being - ditched the mouse. I generally only use two applications, Firefox and Terminal.

Opening a new terminal is as easy as:

Ctrl + Shift + T

It starts rather small but I resize it by using left or right with the Super (window/apple/command) key.

Super + Left
Super + Right

Then its resized to either the left half of the screen or the right half of the screen.

Create a github repository from the cmd line

In the effort to never leave the command line anything to do with github is always a frustration. The hub command line tool - a tool that augments git with github specific commands - helps out tremendously.

In this case I want to create a github repository to push my code to without opening a browser, going to github, creating a repo, copying the remote address for that repo and then setting up the remote locally.

It can all be done with one step.

hub create "exciting_open_source_tech_repo"

git init must be run first, so that there is a local repo to link to the github repo, but then your good. Start modifying, adding and commiting as you normally would.

Webmock assertion that an http request was made

Webmock is a library that helps you manager the http requests that are made during a test. Its not advisable to hit production systems with test-generated requests and with webmock you can stub those requests. Sometimes to ensure functionality of your system you might want to assert that the request was made.

stub_request(:get, 'http://www.google.com')
assert_requested(:get, 'http://www.google.com')

The stub_request method also returns a variable that can be passed to the assert function.

get_google = stub_request(:get, 'http://www.google.com')
assert_requested get_google

Time travelling in rspec/rails

When basing logic on the current time its helpful for testing to have a stable time. A time that does not change. Rails has a module ActiveSupport::Testing::TimeHelpers that was added in Rails 4.2 to provide methods that manipulate the time during testing.

travel_to(Time.parse("2017-01-19")) do
  puts Time.now.strftime(:date)
end

puts Time.now.strftime(:date)

The above code outputs 2017-01-19 and 2017-05-02 (the current date). A fun way to time travel in modern ruby.

Remount a linux filesystem as writable

When booting into safe mode in Ubuntu, because of a boot issue or some other hard to correct issue, the file system will be read only. You can use the mount command to remound the filesystem as read-write.

sudo mount -o rw,remount /

Private vs Public Struct members

In Golang, identifiers can be either Exported or Unexported. In other languages this would be known as public or private, but in golang the question is whether the identifiers should be able to be seen outside of the module. If not? Don't export them.

The rule is simple, if an identifier is capitalized it will be exported. This manifests itself in golang when converting a struct to JSON.

type Apple struct {
    color string
    weight int
}

a := json.Marshal(Apple{"green", 10})
fmt.Println(a)

In this case this code will print {}. To print what you would expect the struct should be defined with capitalized identifiers:

type Apple struct {
    Color string
    Weight int
}

Now the marshalled struct should be {"Color": "green", "Weight": 10}

Add files with matches to the args list

If I want to change all instances of the word Apple to Orange across my entire project I can use the vim command:

:argdo %s/Apple/Orange/gc

But only if the files that have the word Apple are in the args list. To populate the args list you can use the args command in combination with shelling out to git grep with the -l flag.

:args `git grep -l Apple`

Then, when you see the list of files with :args, you'll see all the files with the word Apple.

A Mnemonic for Changing Modes

Making a script file executable involves an incantation that has something to do with chmod but what exactly? chmod has two modes, absolute and symbolic. In absolute mode, you set the entire set of permissions with a numeric value like 755.

In symbolic mode, you can add or subtract permissions to either the user (u), group (g) or owner (o). To give read permissions to the group type:

> chmod g+r myfile.txt

And to give execution priviledges to the owner of a file:

> chmod o+x myscript.rb

Chaining expectations in Rspec

Generally, we think about expectations in RSpec one at a time. If the first expectation fails, then don't go any further. Expectations in RSpec however are chainable, meaning, I can attach one expectation to another for the same subject and then know about the failures or successes for both expections, that looks like this.

expect(1).to eq(2).and eq(3)

Which produces output like this:

 Failure/Error: expect(1).to eq(2).and eq(3)

          expected: 2
               got: 1

          (compared using ==)

       ...and:

          expected: 3
               got: 1

          (compared using ==)

The same result can be got from the below code which may appeal to you a bit more:

def chain_exp(*expects)
  expects.inject {|exps, exp| exps.and(exp)}
end

expect(1).to chain_exp(eq(2), eq(3))

Get the association with Ecto

In the schema of my post model I have this line:

belongs_to :developer

So a post is associated with a developer. When I have a post struct and try to access the developer I might see this:

> Tilex.Repo.get(Tilex.Post, 42).developer
#Ecto.Association.NotLoaded<association :developer is not loaded>

Getting the developer is now a two step process:

  1. Construct a query based on the association
  2. Use that query to acquire the struct

To construct a query based on the association we can use the Ecto.assoc/2 function:

query = Ecto.assoc(post, :developer)

Then this query is executed with the Repo.one function:

developer = Tilex.Repo.one(query)

There ya go!

Login for feature test with warden test helpers

Warden provides a way to login as a user without having to go through the web interface that a user generally sees for sign in.

user = FactoryGirl.create(:user)
login_as user, scope: :user

In the config block for Rspec you would include this statement:

config.include Warden::Test::Helpers, type: :feature

If you have different models for different types of users in your system you can sign in with different scopes. Lets say you have a student user concept, you can sign in with:

student = FactoryGirl.create(:student)
login_as student, scope: :student

Customizing the output of `ps`

There's a lot to know about processes, although typically we're looking for a mapping of pids to commands.

You can customize the output of ps with -o. The -o flag expects a comma separated list of keywords which indicate the columns that you'd like to display for each process.

> ps -o pid,command

The above command shows only the pid and the command w/args for each process. You can see all the possible keywords with ps -L.

erlang memory usage

You might be be curious how much memory your erlang vm is taking. You can get that and more from :erlang.memory. Below is what I get from running this after starting iex normally.

> :erlang.memory
[total: 18133840, processes: 4975616, processes_used: 4974616, system: 13158224,
 atom: 264529, atom_used: 256719, binary: 228728, code: 6640396, ets: 414904]

This list is confusing. All numbers are bytes, so in total 18133840 bytes are allocated to the erlang process. There are two subcategories processes and system and the values of those categories sum to equal the value of total.

The remaining categories are either a component of system or a component of processes.

Use Elixir plug for only some controller actions

Phoenix allows for a flexible request processing pipeline with plugs. Your controllers are basically plugs in the plug pipeline. You can specify plugs that will execute before your controller actions with the plug macro.

defmodule MyApp.ThingController do
  use MyApp.Web, :controller

  plug MyApp.InterestingPlug
end

You can also specify a guard for this macro that will only enable this plug to be run before certain actions:

plug MyApp.InterestingPlug when action in [:new, :create]

Am I connected to a terminal via ssh?

When connecting to another computer via ssh the computer will have some environment variables that are not set otherwise. You can examine the environment variables with SSH in the name with:

> printenv | grep SSH

And in both linux and macos you'll see an environment variable:

SSH_CONNECTION=192.168.50.3 65102 192.168.50.12 22

The first ip address and corresponding port represents the ip address if one exists and the second is the machine you are currently ssh'd into.

cat from stdin and historical notes

Generally, cat takes a file as an argument and prints the file to stdout. cat can also print to stdout from stdin like this:

> echo abc | cat -
abc

Which is the equivalent to:

> echo abc | cat
abc

But when you need to concatenate a line to the beginning of a file and then process that result it comes in handy:

> echo abc | cat - 123.txt
abc
123
> echo abc | cat - 123.txt | grep abc
abc

The macosx cat man page (here) has a history section and mentions that:

Dennis Ritchie designed and wrote the first man page. It appears to have been cat(1).

So cat was the first man page, interesting! (The ubuntu man page for cat does not mention this)

Size of RSA encrypted content

RSA encryption (publickey/privatekey) is powerful encryption but its really only meant to encrypt small things like other encryption keys. The maximum size of the amount of data an rsa public key can encrypt is:

(key_size/8).floor - 11

So in the case of a key thats 1024 bits it can only encrypt data up to 117 bytes long. Not long enough for documents of any size or variable length.

Add email to github account

You can have multiple emails associated with your github account so that if your commits have user_x@example.com or on another machine your commits have user_y@example.com they'll both show up as you on the github system.

In my case, all commits on hashrocket workstations are dev+somebody@hashrocket.com and we change who that somebody is with the pear application. So when I'm working on a hashrocket workstation I commit as dev+chriserin@hashrocket.com.

When editing a profile on github there is a left hand menu labeled 'personal settings' and in that menu there is an emails link. When you click emails you have the opportunity to 'Add email address'. There ya go! Happy committing!

H/T Matt Polito

The Registry is back

The Registry is a bad Windows dream, something left behind when I switched off Windows for good circa 2011.

It's back. It's a new Elixir 1.4 module where you can store values that you'd like to access globally. Potentially, you want to store a pid of a worker you started. You can store that in the registry like so.

{:ok, pid} = MyWorker.start_link()
supervisor(Registry, [:unique, Registry.WorkerPids])
Registry.register(Registry.WorkerPids, :my_worker, worker_pid)
[{_, found_worker_pid}] = Registry.lookup(Registry.WorkerPids, :my_worker)

A lot is going on here.

  • We start a worker
  • We start the registry process for register Registry.WorkerPids
  • We register the key/value in a specific register (Registry.WorkerPids)
  • We lookup the value in that register with that key

Its important to note that the Registry.WorkerPids symbol is just an atom, not a module. There is no WorkerPids module implementing any special functionality anywhere.

Definitely a lot of ceremony for a key/value store. I expect to find practical uses for it as I experience more complex Elixir systems.

Shhh.... Curl silently.

The curl program generally displays a progress bar as its making a request and receiving a response. This can be awkward when you are trying to wrap curl programmatically.

To shut off the progress bar use:

curl -sS http://www.google.com

The first option (-s) hides the progress bar and the errors. The second option (-S) will expose an error on fail if used in conjunction with -s.

Reading from standard in Go

Golang takes input from stdin with the Scan set of functions in the fmt package. These functions are, Scan, Scanf and Scanln.

var input string
fmt.Scan(&input)
fmt.Println(input)

The above snippet echos the input from stdin to stdout.

Age in Postgres

On this day every year I get curious about how old I actually am. Postgres has some utilities to help me out with that, like subtraction of dates.

> select now() - '1978-3-24';
14245 days 17:39:55.623866

Ah yes, 14245 days on this earth, it really makes you think. What got returned anyways, that looks weird:

> select pg_typeof(now() - '1978-3-24');
interval

Got it, that's an interval. It doesn't really tell me much, is this interval infinite?

> select isfinite(now() - '1978-3-24');
t

I can squint and read isfinite as infinite and suddenly my outlook changes, but only momentarily.

Postgres, just tell me how many years I've been alive ok? None of this days stuff.

> select age('1978-3-24'::date);
39 years

Disk usage for just the top level dirs

Every so often your hard drive runs out of space and you need to do a survey of what could be possibly be taking up so much space. The du disk-usage command is great for that.

It can be noisy though as it traverses through all the directories and gives you a report on the size of each file. What we really want is just the size of each directory under the root dir.

du has a depth flag (-d) to help control the depth of directories that the command reports on.

du -h -d1 /

The above command gives you a report on all the directories and files at the top level.

Erlang documentation with `erl -man`

Erlang can be tricky, so documentation is a must, but when you're 30,000 feet high in an airplane with no internet you might regret not having downloaded DashApp and with it all the Erlang documentation.

But it was on your machine the whole time! You have access to all the Erlang documentation with erl -man used like so:

erl -man calendar

Which pops open a man page for the calendar module.

When you are learning Erlang the docs are at your fingertips.

Extended file attributes on macOS

You can store metadata on any file in the mac filesystem(HFS+). If you want to ensure the file has a specific encoding, or if you want to place a checksum with the file you can use file attributes.

Setting and reading are easy on macOS with the xattr utility:

> touch something.txt
> xattr -w xyz 123 something.txt
> xattr -p xyz something.txt
123

In this example we wrote(xattr -w) an attribute onto the file and printed it(xattr -p) to see that it was written.

You can list the file attributes with a cryptic option for ls, ls -l@. Try doing this on your ~/Downloads dir to learn something cool about how macOS treats files coming from the internet.

Inspect with Label

IO.inspect returns the value you pass in as the first argument. This is great when you want to debug the middle state of a series of piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Here, inspect will write to stdout the values [{"A", 1}, {"B", 2"}...].

IO.inspect also takes a label option, which decorates the values written to stdout but does not disrupt the piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect(label: "Letters with index:")
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Which outputs Letters with index: [{"A", 1}, {"B", 2"}...] to stdout.

Bits of Characters

If you are curious what the binary of an ascii/utf-8 char is you can use this string of commands at the command line:

echo 'A' | xxd -b

The A character is 65 in ascii which is 64 + 1. 01000000 is 64. 00000001 is of course 1, 01000001 is 65.

echo 'a' | xxd -b

The a character is 97 in ascii which is 64 + 32 + 1. 00100000 is 32 in binary, given this and the above, 01100001 is 97.

echo '🤓' | xxd -b

This ridiculous emoji is a utf-8 char. When you look at the binary for it:

11110000 10011111 10100100 10010011

You can see that every byte begins with a 1 which means that it will combine with any subsequent byte beginning with 1 to form a unique character.

Jaro Distance determines "closeness" of 2 strings

In Elixir, the String module includes a function named jaro_distance that determines the similarity between two strings:

iex> String.jaro_distance("Chris", "Crhis")
0.9333333333333332
iex> String.jaro_distance("Chris", "Bob")
0.0
iex> String.jaro_distance("Chris", "Dennis")
0.5777777777777778

The docs say this works best with short strings. The Jaro distance article on wikipedia has lots of math if you're into that.

Determine the type of buffer

The :set <option> command will show the value of an option. In the case of :set buftype this is generally set to nothing. It's empty. This is because we're usually working with a normal buffer. The exception is the quick window. Generally, the buftype of the quick window is quickwindow. The help window also has its own buftype, help.

These values are used programmatically to help create features with vimscript.

See more with: :help buftype

Create jsonb data structure from columns

You can create a jsonb object by casting a string version of the object to jsonb:

> select '{"a": 1, "b": 2}'::jsonb;
      jsonb
------------------
 {"a": 1, "b": 2}

When using values from columns to create a jsonb object its not practical to construct a string. In this case you want to use jsonb_build_object:

> select jsonb_build_object('a', a.x) from (select x from generate_series(1, 10) as x) as a;
 jsonb_build_object
--------------------
 {"a": 1}
 {"a": 2}
 {"a": 3}
 {"a": 4}
 {"a": 5}
 {"a": 6}
 {"a": 7}
 {"a": 8}
 {"a": 9}
 {"a": 10}

jsonb_build_object is a variatic function and takes a variable number of arguments. So you can add additional values like this:

jsonb_build_object('a', 1, 'b', 2)

Remove key/value from jsonb data structure

If you store jsonb in your postgres database, you will find yourself manipulating json data periodically in postgres.

Removing a key/value from a jsonb object is very simple, just use the subtraction operator.

> select '{"a": 1, "b": 2}'::jsonb - 'a';
 ?column?
----------
 {"b": 2}
(1 row)

Detach other users in tmux

If someone else has attached to your tmux session but by doing so shrank the tmux screen to a small size, then you are well within your rights to kick them out and regain your real estate.

<leader>D will present you with all the connections. The selection you make will be detached from the current tmux session.

Show commits in the order they were committed

When reviewing a series of commits its sometimes helpful to review them in the order they were committed.

By default, if you pass a range to git show, like git show HEAD..master (assuming you are on a different branch and master is ahead by a number of commits) the commits will be shown in the order of most recent to least recent.

To view the commits in least recent to most recent order use the --reverse flag:

git show HEAD..master --reverse

Debug the `--exclude-pattern` option in rspec.

You can exclude certain files from being run by rspec with the --exclude-pattern option like so:

rspec --exclude-pattern run_me_not_spec.rb

You can place this option into your .rspec file.

When doing this and then committing the .rspec file its helpful to make sure the exclude pattern is correct. Try this command and pipe it into grep.

rspec --dry-run -fdoc | grep 'excluded test name'

If no results are returned, then you are successfully excluding the test! The --dry-run option is important because actually running the entire test suite would be too time consuming.

Turn off folds in my vim buffer!

Sometimes, a co-worker will come over to your vim session and just start folding things.

In these cases, turn off folds with the ever valuable :set nofoldenable.

If all the buffers have been folded then consider, :bufdo nofoldenable.

yank full line without new line character

I am often in a scenario where I want to take all the contents of the current line and paste them into the middle of another line, perhaps into some parentheses or quotation marks.

If I'm in the middle of a line I can type ^y$ which will goto the first character of the line and then yank to the end without the newline.

This is the rare palindramic vim command. It works the same backwards as it does forwards ($y^).

There is also a plugin that treats the line as a vim object called vim-textobj-line. Check it out!

Get the starting hex chars of a binary file

The starting 2 to 8 bytes are used at times to identify the file type of a binary file. You can see what those bytes are in hex format with the xxd command.

> xxd -len 8 beach.png
00000000: 8950 4e47 0d0a 1a0a                      .PNG....

xxd is a command used to create hex dumps.

Convert binary to decimal at the command line

If you have a binary number that you'd like to see in decimal form you can run the command:

> printf '%d\n' 0b10001001
137

Matching on directories for vim's autocmd

Generally, vim users use the autocmd command to set specific options for filetypes. Like, if you want all Ruby files to have the shiftwidth option set to 2 then you would include an autocmd line like this in your ~/.vimrc file:

autocmd BufRead *.rb set shiftwidth=2

I wanted to do something a bit different. I wanted to set the color scheme for each file in a directory rather than for a file type, and I tried this:

autocmd BufRead *writing* color shine

Nothing happened.

Via :help autocmd-patterns I was able to learn that if there isn't a slash char (/) anywhere in the pattern then it just tries to match on the filename. If a slash is included however, the whole path for that file is matched. All I needed was to add a slash:

autocmd BufRead */writing* color shine

Shine On.

Submit a pull request from the command line

Github offers a git extension for augmenting the git command line tool with github specific functionality. This extension is called hub.

hub adds the git alias pull-request to git. This alias/command creates a pull request from the current branch to master by default:

git pull-request

Additionally, you can specify which branch would be the base with the first argument and which branch would be the head with the second argument, like so:

git pull-request devolop feature/add_colors

jQuery.map returns wrapped elements

There are thousands of ways to write JavaScript and many ways to map over a collection in JavaScript. The jQuery variant of the function has a couple of peculiarities that might trip you up however.

First, the arguments to the map function are not (item, index) like they are for the Array.prototype.map function. Instead, they are (index, item).

Second, the jQuery map function does not return an array consisting of the values returned from the map function. It instead returns an array of those values wrapped by jQuery.

This can be unexpected when iterating over a set of elements to create an array of derived values.

Debug :hover selectors in CSS

When styling HTML you may need to define a hover style the css of which might look like this:

div.something:hover {
  background-color: red;
}

The div with class something will now have a red background when your mouse hovers over it.

This can be hard to debug however, if this were a regular style we could inspect it and change the style of the element in dev tools. With hovers, the style is only available in dev tools when the mouse is physically over the element.

Chrome has a facility to declare that a hover state should be on for an element. Simply open dev tools and in the Elements tab right click an element to get a context menu. The menu will have options :active, :hover, :focus, and :visited. Choose :hover and the hover state will be permenantly affixed to that element. You can now play with the hover style until it looks right.

Selected Text Character Count

When highlighting a word in vim (Visual Mode), type:

g<CTRL-g>

In the command section of vim you'll see something like this:

Selected 1 of 90 Lines; 2 of 715 Words; 15 of 4171 Chars; 15 of 4173 Bytes

In this case, 15 of 4171 Chars is the number of total characters and the number of characters selected.

Its not as straight forward or as easy to parse as you might expect, but when you know what to look for it becomes easy to read.

Full page screenshots in Firefox

Firefox has an interesting feature called the Developer Toolbar. This toolbar is entirely separate from the JS console and provides a cli for custom commands for specific functionality. It can opened with shift-F2 or Tools > Developers > Developer Toolbar.

In that toolbar, type:

screenshot --fullpage my-page.png

A png file with the given name will be saved to the Downloads folder. The file format will always be png.

Source

More Developer Toolbar information can be found here.

Type help in the toolbar to see all the commands.

Images in markdown

Images in markdown are the same as links except for the bang (!) character preceding the syntax.

The syntax for links is squares-parens. I have a hard time remembering this, and I try parens-squares as often as not, but its squares-parens, like this:

[My text](https://mylink.com)

The syntax for images is also squares-parens but preceded by a bang (!), like this:

![My alt text](https://mylink.com/oh_my.png)

Installing the Golang tools with Vim

Go has a set of tools to help aid in go development such as a test coverage tool for go or go guru (a tool for answering questions about the go source code.

While in vim if you would like to install these tools just use the command:

:GoInstallBinaries

And likewise if you would like to update these tools use the command

:GoUpdateBinaries

These commands are provided by the vim-go vim plugin. The binaries are installed into your $GOPATH directory or if you'd like to override that dir set the g:go_bin_path vim variable.

Start rails with a different web server

The webserver for my current project is puma, which is a multi-threaded ruby server. This multithreaded nature makes it hard to but a pry statement in and break in specific places. There are multiple threads that will listen to the input the user provides at the REPL.

Using webrick would allow us to debug and step through our code but changing the applications configuration in order to enable that seems unreasonable.

Fortunately, rails provides an easy way to change servers at the command line.

> rails server webrick

Just pass the name of the server you would rather use as a command line argument.

When typing rails server --help you'll see this option available on the first line:

Usage: rails server [mongrel, thin etc] [options]

Raid 0 offers performance with worse reliability

RAID is defined:

Redundant Array of Independent Drives

Whenever I think of RAID I think of data drives that each contain the same information, for fault tolerance. Different RAID configurations are given different numeric identifiers. Recently, when shopping for motherboards I saw RAID 0 listed as a selling point for many of the boards and I assumed this was a fault tolerance feature on these motherboards meant for gamers.

RAID 0 has nothing to do with fault tolerance. Nor redundance. In a Raid 0 configuration data is spread across multiple drives with the intention of increasing the bandwidth of data from the motherboard/CPU to the drives. It is a technique for increasing performance that actually reduces fault tolerance.

Read more about RAID here -> https://en.wikipedia.org/wiki/Standard_RAID_levels

First and last arguments to a command line command

If I start with the command:

> echo a b c
a b c

Sometimes I want the last argument to a command, which in this case is c but many times can be a long file path or a hard to spell word. In that case I would use !$:

> echo !$
c

This substitutes !$ with the last argument of the previous command.

To substitute the first argument instead, use !^ like so:

> echo !^
a  

The ^ character and the $ character are ofter used as first and last. In vim those commands move the cursor to the beginning and the end of the line respectively. In regex those characters match the beginning or end of the line.

Use Mkdir! to create dir of present file in vim

In vim its easy to :edit, :write, or :saveas a file to a path that doesn't yet exist. Such as:

:edit non/existant/path/file.txt

Vim will let you have a buffer with that path, but when you try to write the file vim will throw an error:

E212: Can't open file for writing

Enter the Mkdir! command. The Mkdir command has been added by the eunuch.vim plugin to make creating directories easier from vim. The bang (!) variant takes the path of the current buffer and creates the directory from that path.

When we get the E212 error we can type:

:Mkdir!

And walla! The path will be created. No more errors when writing the file.

Alias Loaders in webpack

A loader might not be named conveniently for your project. Maybe the path to it is too long and obscures the actual configuration of the loader, in that case you can create an alias with the resolveLoader.alias webpack configuration.

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        loader: 'smrt',
      }
    ]
  },
  resolveLoader: {
    alias: {
      'smrt': require.resolve(__dirname, 'really', 'long', 'path', 'smart-loader.js')
    }
  }
}

In addition, you may only want to transform one file. Its possible to do that in a require statement and use the bang character (!) prefix syntax to declare the loader like this.

require 'smrt!dumbfile';

Clearly in this case a long loader name would obscure intent.

Export from old school libraries in Webpack

As you are refactoring your legacy Rails project towards webpack and away from the asset pipeline you discovered an old version of the facebook_sdk that is absolutely critical to the ongoing success of the legacy application. This file doesn't play nicely with CommonJS though and exports its constant with global declarations like:

var FB = {};

This isn't very global in CommonJS and your app doesn't have access to that constant anymore.

The problem can be solved with the exports-loader used like so:

module.exports = {
  module: {
    rules: [
      {
        test: /facebook_sdk/,
        loader: 'exports-loader?FB',
      }
    ]
  }
}

This is just a cute way of tacking an export line to the bottom of the file like this:

module.exports = FB;

Custom loaders for webpack

Perhaps there is a transformation you'd like to perform on your javascript files that is unique to your project. Perphaps replacing the word dumb with smart is a requirement but you don't have control over the actual files and can't make that change to the source files themseles.

In this situation, you can use a custom loader.

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        loader: './smart-loader',
      }
    ]
  },
}

The loader itself (smart-loader.js) would look something like this.

module.exports = function(source) {
  return source.replace('dumb', 'smart');
}

Exclude files from webpack transformation

Sometimes you want every file that has the .dumb extension to be transformed by the dumb-loader except for one particularly dumb file that should remain as is, as a lesson for to all the other dumb files. Just use the handy exclude configuration on modules.rules.loader to make sure that file doesn't get transformed.

module.exports = {
  module: {
    rules: [
      {
        test: /\.dumb$/,
        loader: 'dumb-loader',
        exclude: /really\.dumb/
      }
    ]
  }
}

Now that really dumb file won't get transformed by the dumb loader.

Responding with :stop allows call of `terminate`

A GenServer in Elixir has two lifecycle methods, init and terminate. init is called when the GenServer is started with GenServer.start_link.

What I learned today is that there is a terminate method in case there's any resources that need to be cleaned up when the GenServer is shut down. The terminate method is called when the :stop message is returned from either handle_cast or handle_call:

defmodule Cache do
  use GenServer

  #... a lot of other code

  def handle_cast({:something, 1}, state) do
    IO.puts "This executes first"
    {:stop, "This is my reason for stopping", state}
  end

  def terminate(reason, state)
    IO.puts "Then this executes"
  end
end

Pivoting a 2-dimensional list

In one of the exercism.io exercises, a grid is represented by a list of lists where each inner list represents a row.

The exercise then asks you to pivot the grid, that is, transform the list of lists so that each list represents a column.

From someone else's solution I learned that an easy way to conduct this transform is to use List.zip and Tuple.to_list.

grid = [[1,2,3, 4], [1,2,3, 4], [1, 2, 3, 4]]
List.zip(grid) 
|> Enum.map(&Tuple.to_list/1)
# [[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4]]

Tuple's to_list method is needed because List.zip returns an array of tuples.

Disable a pathogen plugin

Let's say I got this plugin that's just messing my editor up all day and I want to disable it. Because pathogen just looks at the plugins dir and loads whatever is there I can't just remove it from a manifest (or something similar).

There IS though a cool global vimscript var called g:pathogen_disabled that can be populated with all the vimscripts you'd like to disable.

It must be declared and populated before the call to pathogen#infect

let g:pathogen_disabled = []
call add(g:pathogen_disabled, 'vim-badplugin')
call pathogen#infect()

There ya go! Now vim-badplugin will not be included by pathogen!

Merge maps with a callback

Merging two maps together is something I'm familiar with:

iex> Map.merge(%{a: 1, b: 2}, %{c: 3})
%{a: 1, b: 2, c: 3} 

But this function suffers from a unilateral decision that's made when a key in the first argument also exists in the second argument. The value from the first map always overwrites the value from the second.

iex> Map.merge(%{a: 1, b: 2, c: 100}, %{c: 3})
%{a: 1, b: 2, c: 100}

But Elixir's Map module has a merge function that takes a function as an additional argument. It lets you decide what to do in case of a key conflict. You could write this function so that the second argument overwrites the first or better yet add the two values together.

iex> Map.merge(%{a: 1, b: 2, c: 100}, %{c: 3}, fn(k, v1, v2) -> v1 + v2 end)
%{a: 1, b: 2, c: 103}

Three data types that go `into` a Map

Enum.into can be used to put different types of things into different types of collections. For instance I can put elements from one list into another list.

iex> Enum.into([4], [1,2,3])
[1, 2, 3, 4]

It works with Maps too and there's different types of things than can go into a map. A map can certainly go into another map:

iex> Enum.into(%{"a" => 1}, %{"b" => 2})
%{"a" => 1, "b" => 2}

Keyword lists also go into a map:

iex> Enum.into([a: 1], %{"b" => 2})
%{:a => 1, "b" => 2}

And lists of tuples also go into a map:

iex> Enum.into([{"a", 1}], %{"b" => 2})
%{"a" => 1, "b" => 2}

But only when there are two elements in the tuple:

iex(32)> Enum.into([{"a", 1, 3}], %{"b" => 2})
** (ArgumentError) argument error
    (stdlib) :maps.from_list([{"a", 1, 3}])
    (elixir) lib/enum.ex:1072: Enum.into/2

Using assigned value later in same pattern match

I have an array of character lists.

['apple', 'aardvark']

And I want to make sure that each of these two items has the same beginning letter.

I can do this with pattern matching because variables can be reused in an assignment.

iex > [[c | _], [c | _]] = ['apple', 'aardvark']
['apple', 'aardvark']
iex > c
97

Here's what happens when the first letter does NOT match.

iex > [[c | _], [c | _]] = ['apple', 'hamster']
** (MatchError) no match of right hand side value: ['apple', 'hamster']

Run ExUnit tests in the order they are defined

You might have a test module like this.

ExUnit.start

defmodule TestOrderTest do
  use ExUnit.Case

  test "first" do
    assert true
  end

  test "second" do
    assert true
  end
end

And when you run it, second runs before first!

In general this is fine, because every test should pass no matter which one runs first. In certain circumstances however, lets say you're working through some Exercism.io challenges, you might want them to run in order.

To do so include the configuration seed: 0

Elixir.configure seed: 0

And tackle every test in the order they've been given to you!

Two swings at guassian distribution in Postgres

I've come across 2 ways to replicate a bell curve.

This is adding 3 random numbers together.

select 
floor(random()*3)+floor(random()*3)+floor(random()*3) as y, 
count(*) 
from generate_series(1,10000000) group by y order by y;
 y |  count
---+---------
 0 |  369905
 1 | 1111997
 2 | 2222763
 3 | 2593306
 4 | 2220452
 5 | 1111506
 6 |  370071
(7 rows)

Time: 18806.885 ms

Consistent distribution, but 18 seconds for 10 million rows!

The second method has a tail end of the curve that keeps going even as it tapers to 0. This uses the normal_rand function provided by postgres with arguments of quantity, median, and standard deviation.

select  x, count(*) 
from 
(select abs(ceil(normal_rand(10000000,2,1))) as x) as y 
group by x order by count;
 x |  count
---+---------
 0 |  214152
 1 | 1373200
 2 | 3414028
 3 | 3411424
 4 | 1359447
 5 |  214194
 6 |   13234
 7 |     319
 8 |       2
(9 rows)

Time: 3221.894 ms

I want the peak of the curve around 2 and 3 but for this method of Gaussian distribution I get results below 0 unless I use abs. And its fast, 3 seconds!

Ever so slightly faster intervals in postgres

When working with dates in postgres I often construct an interval like so:

> select (2 || ' days')::interval;
2 days

Which is fine! But there's an alternative that's ever so slightly faster.

> select 2 * '1 day'::interval;
2 days

The time difference can be seen over large quantities of data, like 10 million.

> select (x || ' days')::interval from generate_series(1, 10000000) as x;
...
Time: 10437.314 ms
 > select x * '1 days'::interval from generate_series(1, 10000000) as x;
...
Time: 7831.568 ms

Microbenchmarking in action! Remember, take all microbenchmarks with a grain of salt.

Non-Greedy Vim Regex

Regex is by default greedy, meaning, if you used this regex /.*, on the following line:

a123,b123,c123,d123

You'd match everything up to the last comma a123,b123,c123. If you just want everything to the first comma you need to use a non-greedy matcher. In posix regex you could modify the * operator (match 0 or more) with the ? operator, like this: /.*?,/.

Vim however is a bit weird and doesn't support making the * operator non-greedy in this manner. You can however use curly braces to tell vim to match 0 or more, but as few as possible like this: /.\{-}, which will match just a123,.

Remember, the leading curly needs to be escaped with a slash \.

See :help pattern-overview for more!

Slicing arguments into an array.

The arguments keyword in javascript is not of type Array, but can be turned into an Array easily by using the slice function of the Array prototype.

function varargs() {
  return Array.prototype.slice.apply(arguments);
}

> varargs(1, 2, 3);
[1, 2, 3]

zsh variable editing

Lets say I made a typo in an important environment variable and I wanted a quick fix, just to see if yes, that was indeed why my application wasn't working.

$ echo $RAILS_ENV
> porduction
# ^^^ YIKES ^^^

The Z shell has a builtin program called vared that helps you edit an existing environment/shell variable.

$ vared RAILS_ENV
porduction

Now you are in an editing context, you can edit the value with all your configured command line editing tools. Fix the error, hit RETURN, and you're good!

whois that url?

Let's say you just got a spammy text message that contains a suspicious url and you want to learn more but clicking the link just seems like a bad idea.

You can learn more from the whois program that comes installed on macosx. It sends queries to databases that store domain registration information.

When you use the command:

whois spam.com

You can see the important result

Registrant Organisation: Hormel Foods Corporation

The wikipedia article for whois contains some deep internet history.

Test out your zsh prompt

If you are trying to change or create a new prompt in Z shell you can use the print command with the -D flag to help you iterate.

print -P %B%n%b

This prints out your user name in bold to stdout.

ZSH Glob Exclusion

I want to list everything except a .jpg file from the files

foo.doc foo.gif foo.jpg

but if I use print -l * I get them all back. To exclude a pattern in zsh you first must turn on extendedglob.

$ setopt extendedglob

Then, everything after the ^ will be treated as an exclusion.

$ print -l ^*jpg
foo.doc
foo.gif

Encrypting with gpg

Encrypting and decrypting is easy with gpg.

To encrypt the text ABC and write it to a file called abc.enc:

$ echo "ABC" | gpg --symmetric --force-mdc > abc.enc

To decrypt the abc.enc file:

$ gpg -d < abc.enc
gpg: CAST5 encrypted data
gpg: gpg-agent is not available in this session
gpg: encrypted with 1 passphrase
ABC

Each step asks for a passphrase. Symmetric encryption is encryption that can be encrypted and decrypted with the same information, in this case that information is the passphrase.

--force-mdc enables the modification detection code, so that gpg will be able to know if the encrypted text has been modified or not.

Changing your default shell

If you decide that the bash shell is old news and maybe zsh is the real deal you can use the chsh command in ubuntu.

chsh dev -s /usr/bin/zsh

Logout and login for the change to take effect and then check your shell var

$ echo $SHELL
/usr/bin/zsh

Fantastic!

Checking for os in bash

If you need to conditionally run something in macosx that you wouldn't need to run in linux, you can check the $OSTYPE env var and put it in an if statement like so:

if [[ "$OSTYPE" == 'darwin'* ]]; then
  echo 'oh geez on macos';
fi 

Is that process running? pgrep!

If I want to ask the operating system if a process is running I typically use something like:

$ ps -ax | grep rails

But the output reports two processes, the rails process that is running and the actual grep. Once trick to get past this is:

$ ps -ax | grep [r]ails 

So that the grep looks for a process that starts with r followed by ails but can't find it because the r is followed by ].

But this is an old problem with a fairly old solution. Both mac and linux have a pgrep program.

$ pgrep ruby
44761

Which returns the pid of the process you're looking for. To get more information about the program and its arguments use the -lf flags.

$ pgrep -lf ruby
44761 /Users/devuser/.rvm/rubies/ruby-2.3.1/bin/ruby bin/rails s

Get current call stack with `new Error().stack`

arguments.caller use to be an acceptable way to find what function is calling the current function. Now, if 'use strict'; has been called for the current closure then an error will be thrown preventing the arguments.caller call.

Getting the current stack trace is just as easy though with new Error().stack.

function whoCalledMe() {
  console.log(new Error().stack);
  //doin' somethin'
}

An Agent reference can be pid or name.

Each function of the Agent module takes an agent as the first argument:

get(agent, fun, timeout \\ 5000)

but this can be a couple of different things. The documentation defines the agent type as:

agent :: pid | {atom, node} | name

#The agent reference

name ::
  atom |
  {:global, term} |
  {:via, module, term}

#The agent name

So really five different types of values are valid. The pid is a value returned from calling start_link.

> {:ok, agent_pid} = Agent.start_link(fn -> [:thing] end)
> Agent.get(agent_pid, fn (state) -> hd(state) end)
:thing

The agent can be referenced by a name atom also. In the documentation example that name atom is the __MODULE__ for the module that wraps access to the agent. The name for the agent is declared by passing a name option into the start_link function call.

> Agent.start_link(fn -> [:thing] end, name: __MODULE__)
> Agent.get(__MODULE__, fn (state) -> hd(state) end)
:thing

I'm note sure about the {:global, term} and {:via, module, term} values that seem to be valid. I would love to see examples for those.

Copy that ssh pub key to the remote server

When setting up a server its customary to copy the ssh key to the authorized keys file so that you don't rely on password auth to sign in to the remote server.

Its generally something like a 4 or 5 step process but linux has a utility to get it down to one step, ssh-copy-id.

$ ssh-copy-id dev@myserver.com

After entering your password you're all set! Not turn off that password authentication in your ssh config and start deploying your app!

Periodic messages in Elixir

Today I needed to implement a task that happens once every 24 hours. I immediately thought, well this should be a cron job, but my co-worker suggested using the send_after method in a GenServer instead.

    Process.send_after(self, {:"$gen_cast", :get_data}, interval)

The third argument is a milliseconds argument that determines when the call will be processed. The $gen_cast is a hackish type of thing to get the GenServer to handle the call with handle_cast.

The periodism should start in the init and whenever handle_cast is called, it should schedule a new time that the message will be sent.

  def init(state) do
    work
    Process.send_after(self, {:"$gen_cast", :get_data}, interval)
    {:ok, state}
  end

  def handle_cast(:get_data, state) do
    Process.send_after(self, {:"$gen_cast", :get_data}, interval)
    work
    {:noreply, state}
  end

H/T Micah Cooper

Read command line output into buffer

Sometimes you need some information from outside of vim in your vim buffer. A list of all the models in rails perhaps?

:read !ls app/models

Anything to the right of the bang(!) will be executed via the shell and everything given to standard out by the program will be read into the buffer.

`NULLIF` in Postgres

If there multiple values in a database column that both mean the the column for that row has no data, then the NULLIF function can be used in queries to normalize the values in that column.

Null char is 😵

chriserin=# \pset null 😵
Null display is "😵".
chriserin=# select nullif('NOTHING', 'NOTHING');
 nullif
--------
 😵
(1 row)

chriserin=# select nullif('something', 'NOTHING');
  nullif
-----------
 something
(1 row)

Does template exist?

An instance of the class ActionView::LookupContext::ViewPaths is available as lookup_context in a rails view.

If you need to dynamically construct partial paths and know that sometimes the partial may not exist, you can use the lookup_context method exists? to determine if the partial exists.

lookup_context.exists?(dynamic_path, [], true)

The third argument is partial, the second argument is prefixes.

3 parts of routing a websocket

First, there's a socket declaration in the lib/appname/endpoint.ex file.

  socket "/socket", AppName.UserSocket

Second, there's a transport declaration in the web/channels/user_socket.ex file.

  transport :ws, Phoenix.Transports.WebSocket

Now, the websocket will be available at the url ws://localhost:4000/socket/ws the final path segment of the url being added by the first argument of the transport declaration.

Finally, the channel topic is also declared in the user_socket.ex file.

channel "app_topic:*", AppName.MyChannel

Now every json body that has a topic attribute of app_topic will be routed to MyChannel.AppName where it will be handled appropriately based on the event attribute.

{"topic":"app_topic:something","event":"phx_join","payload":{},"ref":"1"}

Name that anonymous function

I use anonymous functions in javascript a whole bunch but anonymous functions don't get a name:

> a = function() {}
> a.name
undefined

Its helpful to have a name for the function when you're debugging and need to determine the caller of a function or if you want to determine which function has been passed in as an argument.

It's easy to name anonymous functions:

> a = function myName() {}
> a.name
'myName'

But weirdly, javascript doesn't know about a function named myName so you're not polluting the namespace:

> myName()
ReferenceError: myName is not defined

Getting a date

Elixir 1.3 has introduced calendars and calendar types. Lets check out how to get a Date type!

You can use the new function and pass in year, month and day separately.

iex > Date.new(2008, 6, 28)
{:ok, ~D[2008-06-28]}

If you want a nonsensical date you'll get an :error atom back.

iex > Date.new(2008, 6, 31)
{:error, :invalid_date}

Elixir knows how to handle ISO8601 date formatted strings. Don't forget to zero pad that month value!

iex > Date.from_iso8601("2008-6-28")
{:error, :invalid_format}
iex > Date.from_iso8601("2008-06-28")
{:ok, ~D[2008-06-28]}

To support Erlang the from_erl function will accept a tuple.

iex > Date.from_erl({2008, 6, 28})
{:ok, ~D[2008-06-28]}

It's interesting that the date is inspected as a sigil.

iex > {:ok, d} = Date.from_erl({2008, 6, 28})
{:ok, ~D[2008-06-28]}
iex > d
~D[2008-06-28]

Hey you can create a date with the new sigil syntax!

iex > ~D[2008-06-28]
~D[2008-06-28]

Microbenchmarking elixir with Benchee

There is a hex package to help facilitate benchmarking in elixir. Below is an example of how to compare flat_map to map().flatten(). Example stolen straight from the README.

list = Enum.to_list(1..10_000)
map_fun = fn(i) -> [i, i * i] end

Benchee.run(%{time: 3},
             [{"flat_map", fn -> Enum.flat_map(list, map_fun) end},
              {"map.flatten",
              fn -> list |> Enum.map(map_fun) |> List.flatten end}])

Loopback to different 127.*.*.* ips mac osx

You can't bind to the loopback interface automatically on mac osx.

> rails -b 127.0.0.2
ERROR: Can't assign requested address - bind(2) for 127.0.0.2:3000 (Errno::EADDRNOTAVAIL)

Mac OSX will disable all loopback addresses other than 127.0.0.1 by default, but they can be enabled with ifconfig.

> sudo ifconfig lo0 alias 127.0.0.2 up

And now your server can bind to that address.

h/t Jack Christensen

Matching *within* function parameter declaration

The below contains a matching expression within the function declaration:

iex > defmodule Hulk do
... > def print({1, a} = tt) do
... > IO.inspect(a)
... > IO.puts("**********")
... > IO.inspect(tt)
... > end
... > end

This allows the function access to the variable that was bound while matching and the full value passed in to the function. It returns:

iex > Hulk.print({1, 3})
3
**********
{1, 3}

This is particularly useful for Maps which is the only datatype to support partial matching (eg %{a: a} = %{a: 1, b: 2}) so that the unknown and unmatched portion of the map will be available.

This is a common idiom in pheonix while declaring actions in controllers:

def show(conn, %{"messenger" => messenger} = params) do

The above is from the pheonix docs

Two anonymous functions in one

Elixir has anonymous functions, and function pattern matching, and to accommodate the two concepts the language has the facility to do both at the same time.

alphaIndex = fn ("a") -> 0;  ("b") -> 1; (letter) -> :unknown end

alphaIndex.("a")
# returns 0
alphaIndex.("b")
# returns 1
alphaIndex.("c")
# returns :unknown

Using __using__ for use with the `use` macro

You might run across the use macro while doing something like.

use GenServer

This sets up the module to be able to do special things with the special module GenServer.

Behind the scenes GenServer implements

defmacro __using__(_opts) do

to ensure that the module is setup correctly for use with GenServer.

You can experiment with implementing __using__ inside of IEX like so:

iex > defmodule Fruit do
... > defmacro __using__(_opts) do
... > quote do
... > IO.puts "Good to see you've added Fruit to your meal"
... > end
... > end
... > end

iex > defmodule Meal do
... > use Fruit
... > end
"Good to see you've added Fruit to your meal"

Binary pattern matching

You might be familiar with the most popular form of Elixir pattern matching involving tuples:

iex > {:ok, x} = {:ok, 1000}
{:ok, 1000}
iex > x
1000

You can also pattern match binaries:

iex > <<1, y, 2>> = <<1, 255, 2>>
<<1, 255, 2>>
iex > y
255

This gets powerful when you're able to match binary formats.

iex > <<x :: binary-1, y :: binary-2, z :: binary>> = <<1, 255, 254, 2>>
<<1, 255, 254, 2>>
iex > x
<<1>>
iex > y
<<255, 254>>
iex > z
<<2>>

Here's an article about using binary pattern matching to parse a png file.

Observe the erlang vm with :observer

Elixir and Erlang can have fairly complex process organization structures, but they can be inspected with the observer tool.

You can start the observer tool with:

iex > :observer.start

And then you'll have access to a gui with tabs labled:

System

Load Charts

Memory Allocations

Applications

Processes

Table Viewer

Trace Overview

Determine type? no. Get info tho with `i`

There is no direct way to get a variable type in Elixir. For instance type("") is not something that exists although there are 15 Kernel methods for specific type determination such as:

iex > is_number(1)
true
iex > is_binary("") # Note that elixir checks for binary instead of string, a string is a binary.
true
iex > is_atom(:hey)
true

There is an i function available only in iex that will print out useful information about a variable.

iex > i("")
Term
  ""
Data type
  BitString
Byte size
  0
Description
  This is a string: a UTF-8 encoded binary. It's printed surrounded by
  "double quotes" because all UTF-8 encoded codepoints in it are printable.
Raw representation
  <<>>
Reference modules
  String, :binary

It's implemented as a protocol check it out.

I discovered it here.

Weird Operator Uses in Elixir

= is match, not assignment (sometimes they're equivalent)

<> is string concatenation

[a | b] is a pipe used within squares and appends the left hand side to the right hand side when the right hand side is a list.

++ is an append operator for lists

[1, 2, 3] ++ [4] == [1, 2, 3, 4]

-- is a subtraction operator for lists.

[1, 2, 3] -- [1, 2] == [3]

in checks to see if an element is included in a list.

1 in [1] == true

=== works with numbers and returns false when comparing numbers of different types that would otherwise be equivalent.

(1 === 1.0) == false; (1 == 1.0) == true

^ is the pin operator, it switches a variable from assignable to matchable.

iex > a = 1; {^a, b} = {1, 2};
{1, 2}
iex > a = 1; {^a, b} = {2, 2};
** (MatchError) no match of right hand side value: {2, 2}
iex > a = 1; {a, b} = {2, 2};
{2, 2}

Accessing a single element of a list by index

Getting a element form a list by index is a common language idiom.

In ruby you would use the squares [] method (similar to Java, C#, etc.).

> [:a, :b, :c, :d][2]
:c

But Elixir is not object oriented and instead provides the Enum module that has many functions that can operate on a list. Three functions return an element by index.

iex > Enum.at([:a, :b, :c, :d], 2)
:c
iex > Enum.fetch([:a, :b, :c, :d], 2)
{ :ok, :c }
iex > Enum.fetch!([:a, :b, :c, :d], 2)
{ :ok, :c }

They behave differently when the index doesn't correspond to the list.

iex > Enum.at([:a, :b, :c, :d], 9999)
nil
iex > Enum.fetch([:a, :b, :c, :d], 9999)
:error
iex > Enum.fetch!([:a, :b, :c, :d], 9999)
** (Enum.OutOfBoundsError) out of bounds error
    (elixir) lib/enum.ex:722: Enum.fetch!/2

I don't understand how the statuses (:ok, :error) might be used in application code yet by I'm curious and can't wait to find out!

Three ways to compile Elixir

1: Use the c function in iex

iex> c('lib/funkymath.ex')
[FunkyMath]

2: When opening iex pass the module as a command line argument

$ iex lib/funkymath.ex

3: Use the mix command line utility.

$ mix compile

This last option creates some artifacts that you can then use.

Select tag options in #angular 1

To display options for a select tag in Angular 1 use the ngOptions attribute and a comprehension expression.

 <select ng-model="myColor" ng-options="color for color in ['red']">
 </select>

The comprehension expression is like a query for your data maybe? Weird stuff.

The ng-model is necessary for the options to display.

Is the hstore extension enabled?

Have you already enabled the Hstore extension in Postgres?

Running the below command in the psql console will let you know what extensions have been installed.

\dx

Find the error description in a tmux window

At Hashrocket we have a tmux binding ctrl-z e that will find the last instance of the word Error in the current window. This is great when error output for a test might consume 9 full windows worth of spaces at times.

This comes with Hashrocket's dotmatrix command line environment and is defined as:

bind-key e copy-mode \; send-keys "?Error" C-m

Defined at this location.

H/T Micah Woods

Was that branch merged into master?

To see if a branch was merged into master, use the --merged flag.

git branch --merged master

This will show all the existing branches that have been merged into master.

Remove function from namespace in Clojure

If there is an existing unneeded function something loaded from another namespace and you want to create your own function of the same name you can use ns-unmap

(ns-unmap *ns* 'something)

Where *ns* is a reference to the current namespace.

IIFE is an immediately invoked function expression

Y'all have seen this before.

(function(myVar) {
  console.log(myVar);
})("abc");

But did you know the name?

Immediately Invoked Function Expression

IIFE

And its got more variations than the above which you can see in this 2010 blog post.

http://benalman.com/news/2010/11/immediately-invoked-function-expression/

gcc comments the code

I've been using the vim-commentary plugin for years but always by selecting a line and THEN using gc to turn the line into a language specific comment. Selecting the line isn't necessary though, you can just use gcc and the line the cursor is on will either be turned into a comment or if its a comment already it'll be uncommented.

Rails vim navigation for lib

Y'all might be familiar with :Emodel, :Econtroller, and :Eview but today I learned that there is a vim rails navigation for :Elib.

The command :Elib xyz! creates a vim buffer and file at path lib/xyz.rb.

after_commit callback in active_record

We use ActiveRecord callbacks at times to sync data between systems but if you're using transactions you don't want to have called a callback like after_create and then have the whole transaction roll back, then you'd have out of sync systems! Rails conveniently includes an after_commit hook for such a case.

class Fruit < ActiveRecord::Base
  after_commit :sync_to_fruitstand
end

ActiveRecord::Base.transaction do
  Fruit.create(name: 'Banana')
  Fruit.create(name: 'Kiwi')
  Fruit.create(name: 'Rasberry')
end

Now, the callback is called (for all 3 times) only when the final Fruit creation for a Rasberry succeeds.

Don't load gem docs by default.

You don't need to generate/install the ruby documentation of those gems if you don't want to!

Put a .gemrc file in your home dir and include the line.

gem: --no-document

H/T Vidal Ewechukwu

Ruby Hash's update method

Today I learned that update is a synonym for merge! for a hash.

a = {x: 1}
a.update({y: 2})
a == {x: 1, y: 2}

I say update is a synonym for merge! but really the C method that they share is called rb_hash_update. So who is the synonym of who? I think the etymology of all the various alias methods in ruby core would be a fun subject. Was map or collect introduced first, etc, etc.

Be aware! Postgres rounds.

Yesterday, my pair and I created a test that calculated a value and compared that to the value of a calculation in the code we were testing. This worked out great except for one hitch, we were asserting about the derived value after it had been inserted into the database. What we didn't count on is that Postgres rounds. Check this out:

create table money (amount numeric(4, 2));
insert into money (amount) values (10.342) returning amount;
 amount
--------
  10.34

insert into money (amount) values (10.347) returning amount;
 amount
--------
  10.35

Postgres rounds!

Postgres permissions to insert, but not return.

Postgres permissions can be harsh. If you don't have the select permission on a table you might not be able to do some very rails like things that you think you ought to be able to do, take my friend Jimmy.

set role 'admin';
create role 'jimmy';
create table fruits (id serial primary key, name varchar);
grant insert on fruits to jimmy;
grant usage on fruits_id_seq to jimmy;

Given these permissions it's possible for Jimmy to make inserts into the fruit table like so:

set role 'jimmy';
insert into fruits (name) values ('Apple');
INSERT 0 1

But Rails wants a little more, it wants to know the id of the thing that was just created which is a problem because Jimmy doesn't have select permissions.

set role 'jimmy';
insert into fruits (name) values ('Orange') returning id;
ERROR:  permission denied for relation fruits

Argh this complicates matters, but I'll relent and give Jimmy the appropriate permissions so that he can add records through ActiveRecord.

set role 'admin';
grant select on fruits to jimmy;
set role 'jimmy';
insert into fruits (name) values ('orange') returning id;
 id
----
  2

A high level view of Rspec tests

Test files in ruby/rspec can grow to gigantic soul crushing sizes, which makes it hard to really get a sense of what tests are in the file and where. This is troublesome when trying to determine a sensible place to add a new test to the already gigantic file.

To get a better sense of the structure of the file you can combine the dry-run and format options for readable, hierarchical documentation in a small amount of time.

rspec -fdoc --dry-run specs/my_massive_test_file_spec.rb

Ternary shortcut in PHP

Sometimes, and hopefully sometimes is rarely, you have to read some PHP in order to rewrite that code in another language. In these cases you might run across this oddity.

$a = $a ?: 2; 

This is pretty much like var a = a || 2; in javascript but it comes out of the ternary operator and definitely doesn't make sense when you first look at it. In every other language that's a syntax error, but php, ya dig?

Add value to Postgres enum type, dropping tho...

An enum type in Postgres is a type that can restrict a value to only certain values. It can be defined like this:

create type fruit as Enum ('orange', 'apple');

We can now cast a string to a fruit.

chriserin=# select 'orange'::fruit;
 fruit
--------
 orange

Well... some strings...

chriserin=# select 'rasberry'::fruit;
ERROR:  invalid input value for enum fruit: "rasberry"

It's all good! We can add rasberry to the enum type.

chriserin=# alter type fruit add value 'rasberry';
chriserin=# select 'rasberry'::fruit;
  fruit
----------
 rasberry

Postgres allows you to add values to an enum type, but you can't drop values which is a bit of an inconvenience when creating up/down db migrations like you do in Rails or with a tool like tern.

Private Class Methods in Ruby

Every object oriented language has a different take on the rules of methods and objects and classes, and I cut my teeth on Java, which means that forever and ever after I'll assume everything works like Java. That left me writing code like the Apple class below that has a private class method, but not really. You can still call it.

class Apple
  def self.latin_name
    binomial_name
  end

  private
  def self.binomial_name
    "Malus domestica"
  end
end

puts Apple.latin_name
puts Apple.binomial_name  #no error, works fine

There's a way to make class methods private in Ruby, you just gotta jump through some hoops. Err, I mean use the class << self syntax. This oddity pushes an instance singleton onto the class effectively creating class methods.

class Orange
  def self.latin_name
    binomial_name
  end

  class << self
    private def binomial_name
      "Citrus × sinensis"
    end
  end
end

puts Orange.latin_name
puts Orange.binomial_name # ERROR!!! you can't call a private method.

H/T Mike Chau

Get Chrome history from remote machine

Let's say you need to get the url you visited Friday at the office, but its Sunday and you're at home! First, ssh into the remote machine where you were browsing the internet. Second, locate the history file, on OSX its at:

~/Library/Application Support/Google/Chrome/Default/History

Third, try to open the file with a text editor. DOH! Its a sqlite file! Fortunately you have sqlite on your machine so you can use sqlite3 History at the command line. Fourth, type .table to see what tables are available. Oh that doesn't work, you get an error that says Error: database is locked. This is easy to overcome though, just make a copy of the history file.

cp History History.1
sqlite3 History.1
> .tables
downloads             meta                  urls
downloads_url_chains  segment_usage         visit_source
keyword_search_terms  segments              visits

Now you have access to the database. The url you want is definitely in the urls table, you just have to use your sql skills to find it!

Point the Gemfile.lock to a new branch version

One very flexible option for gluing your softwares together is to point a gem to a branch on Github in your Gemfile like so:

gem 'fancy_addition', github: 'chriserin/fancy_addition', branch: 'double_add'

But that reference is locked to a specific version of your branch in the Gemfile.lock

remote: git://github.com/chriserin/fancy_addition.git
revision: dca3f25fa294ee68b41b2c736064e5e200039b6d
branch: double_add

If you update that branch, the repository that depends on it will not automatically receive that update. To update the Gemfile.lock reference to the most recent version of that branch use the --source option of bundler like so:

bundle update --source double_add

Shuffle an array in postgres

Somtimes an array is just too sequential

select ARRAY[1,2,3,4,5];
chriserin=# select ARRAY[1,2,3,4,5];
    array
-------------
 {1,2,3,4,5}
(1 row)

I want to shuffle this array into an unpredictable mess of numbers, and one way to do this in PostgreSQL is to turn these array elements into rows, order them randomly and then squash the rows back into an array like so:

chriserin=# select array_agg(foo.x) from (select unnest(ARRAY[1,2,3,4,5]) x order by random()) foo;
  array_agg
-------------
 {2,5,4,3,1}
(1 row)

The key here is the order by random() clause, which is a special postgres case. order by 0.284325 which is a number that random() could return, is not valid sql. There are dragons in order by, read here for more info http://postgresql.nabble.com/select-random-order-by-random-td1891064.html

array indexes in postgres

Lets say I have an array of fruit types (Apple, Pear, Banana, Tangerine) and I want to turn each element in the array into a row I can do this.

select unnest(fruits) 
from (select Array['Apple', 'Pear', 'Tangerine', 'Banana'] fruits) as list;

which returns:

  unnest
-----------
 Apple
 Pear
 Tangerine
 Banana
(4 rows)

But it would be nice to have an array index as column in case the order of the elements in this array had some meaning. Postgres has a function generate_subscripts to help you with array index elements.

select generate_subscripts(fruits, 1), unnest(fruits) from (select Array['Apple', 'Pear', 'Tangerine', 'Banana'] fruits) as list;

which returns:

 generate_subscripts |  unnest
---------------------+-----------
                   1 | Apple
                   2 | Pear
                   3 | Tangerine
                   4 | Banana

Throw a minus 1 in there if you want to be base zero :).

Debian alternatives

Debian/Ubuntu has a number of chores for which the application to use for that chore can be configured by you. Editor and pager are 2 of the most common and generally I've set the environment variables for those in the past. Setting EDITOR and PAGER determines what happens when an application puts you into a page mode or which an application requires that you edit something.

Debian/Ubuntu though has a system for determining what application should be used in that instance, the alternatives system.

Running ls -l /usr/bin | grep 'alternative' will show you all the programs that are linked to the /etc/alternatives directory. The /etc/alternatives dir in turn has symlinks that point to the application choices a user configures.

update-alternatives is the program which manipulates the symlinks in the /etc/alternatives directory. Running sudo update-alternatives --config editor will give you a menu from which you can choose your favorite editor.

NaN being falsey leads to interesting statements

NaN is a javascript constant representing something that is 'not a number', and like almost everything else in javascript, its falsey.

Boolean(NaN)
> false

You can determine if a number that is passed to a function is valid after you give it to a mathematical operation.

function doSomething(number) {
  var importantNumber = number * 96 || 100;
  console.log(importantNumber)
}

Uncomplicated Fire Wall

Lets say you have a malicious program on your server submitting http posts over and over over. What's important is shutting down those requests before the server provider shuts them down for you. An uncomplicated way to do that is with ucf uncomplicated fire wall.

First enable ssh so that you don't lock yourself out.

ufw allow ssh

Then deny all other outgoing traffic

ufw default deny outgoing

Check out the rules with status

ufw status

And then enable the fire wall.

ufw enable 

If you've ever dealt with iptables, this might seem a little less.... complicated.

Avoid mutating array when sorting

I just ran into a hard to track down bug with code that amounted to this.

> a = [6,5,4,3,2,1]
[ 6, 5, 4, 3, 2, 1 ]
> a.map(function(x) { console.log(x); a.sort()})
6
2
3
4
5
6

Why is the second number 2 and not 5? Oh, because I was sorting the array. In my program I needed to gather some information about the collection as a whole and the sort statement was buried 2 function calls down, and didn't realize the collection was being mutated while it was being iterated.

To avoid this, a best practice might be to copy the array before sorting it, two examples of that would be:

a.concat().sort()

And

a.slice(0).sort()

Same number of characters; choose your favorite.

Examine all jQuery events for namespace

In a big system you can lose track of all the events that might be registered with jQuery. An easy way to see them all is:

$._data(window, 'events')

jQuery allows namespacing events as well. This can look like:

$(myElement).on('click.nsMyElement', function() { console.log('click'})

And check it out you can see that through data

$._data(myElement, 'events')['click'][0]['namespace']
> nsMyElement

deep_munge, I hardly knew ye.

Rails 4.1.8 has a method called deep_munge which is intended to massage params to convert empty arrays into nils in response to security issues like http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-2660.

This can be tricky when sending in a json string as post data that may have some values that should be empty arrays. Those empty arrays will get converted to nil.

To get around that rails 4.1.8 added a configuration option

config.action_dispatch.perform_deep_munge = false

Rails 5.0 intends to not have the same sql injection vulnerabilities and so have removed the deep_munge method that would change an empty array value to nil but have left in the configuration option which produces behavior best described by looking at the tests.

Rails source

All Rails http verbs are UPPERCASE

Rails supports many http verbs, and in the rails source code they're handily organized by ietf rfc.

    RFC2616 = %w(OPTIONS GET HEAD POST PUT DELETE TRACE CONNECT)
    RFC2518 = %w(PROPFIND PROPPATCH MKCOL COPY MOVE LOCK UNLOCK)
    RFC3253 = %w(VERSION-CONTROL REPORT CHECKOUT CHECKIN UNCHECKOUT MKWORKSPACE UPDATE LABEL MERGE BASELINE-CONTROL MKACTIVITY)
    RFC3648 = %w(ORDERPATCH)
    RFC3744 = %w(ACL)
    RFC5323 = %w(SEARCH)
    RFC4791 = %w(MKCALENDAR)
    RFC5789 = %w(PATCH)

source

Notice though that they're all upper case, so if you try to make an http request with lowercase patch, then you'll get a crazy UnknownHttpMethod error.

The -A -B and -Cs of Grep

Given the text file things.txt:

foo
bar
baz

When I type cat things.txt | grep 'bar' Then I see just the one line with bar and nothing else, in fact, I have no idea what comes before or after bar.

To know what comes 1 line before bar I can type cat things.txt | grep 'bar' -B 1 then I see:

foo
bar

To know what comes 1 line after bar I can type cat things.txt | grep 'bar' -A 1 then I see:

bar
baz

And to see 1 line either side of the grep result I can type cat things.txt | grep 'bar' -C 1.

Synchronous XHR is deprecated

Sometimes before I've figured out the right pattern for my app I use the synchronous xhr to get some naive green for my tests.

  var isAsync = false
  var xhr = new XMLHttpRequest()
  xhr.open('post', '/things.json', isAsync)
  xhr.send(JSON.stringify({things: things}))

But this time I got a nice deprecation notice at the console.

Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check http://xhr.spec.whatwg.org/

And low and behold whatwg.org has a deprecation notice

An ES6 javascript array of numbers

If you want an array of numbers in ruby, you might do something like (0..127).to_a. Concise! I don't really know how to do this in javascript but es6 has a couple of methods to help with a clever solution.

Array(100) gives you an array with 100 elements, all undefined.

array.keys() gives you an es6 iterator that will iterate through all the keys of the array, which happen to be 0 - 99.

Array.from() will turn an iterator into an Array!

So in just 3 weird steps you can get an array full of numbers.

Array.from(Array(127).keys())

Hopefully someone knows an easier way :).

Transaction within a transaction

When using transactional fixtures in rspec, you're likely to always be in a transaction. When that's the case, transactions in you're code don't necessarily behave as you'd expect.

def update
  ActiveRecord::Base.transaction do
    Thing.find(params[:id]).update_attributes(color: 'red')
    raise ActiveRecord::Rollback
  end
end

In this case the transaction won't rollback the changes to Thing, and if you're testing that rollback in rspec, you'll be very confused.

While not the perfect solution, it's possible to force this transaction to behave how you'd like it to with the requires_new: true named argument which makes the transaction a 'sub-transaction'.

  ActiveRecord::Base.transaction(requires_new: true) do
  end

documentation

within_window capybara

Capybara lets you test things in multiple windows. Which blows my mind for some reason.

window = page.drivers.window_handles.first
within_window(window) do
  test_the_form
end

Rebase all commits to whats been pushed

When git rebasing interatively, git rebase -i you need to specify a commit to rebase interactively back to, like:

git rebase -i HEAD^^

And generally I specify the commit to rebase to with as many carrots after HEAD as commits I needed. But generally you are rebasing commits that aren't yet pushed to master, so in that case just rebase back to 'origin/master'.

git rebase -i origin/master

Elm type annotations has return type

Here is an add function that takes 2 and 4 and turns these into 24. :)

add : Int -> Int -> String
add x y = (toString x ++ toString y)
x = add 2 4

The first line is a type annotation that lists the first argument, second argument and return type in that order.

ActiveRecord::Relation size vs count

An array in ruby has 3 methods that do the same thing. size, count, and length all return the number of items in an array.

An ActiveRecord::Relation however uses them a bit differently. count is always going to run a query in the database while size will return the number of items in the collection based on the objects currently in the object graph.

> songs  = Songs.all
> songs.size
10
> songs.count
SELECT count(*) FROM songs;
10

:vglobal helps you find all the other lines

Given a file like:

cat
dog
elephant
dog
mouse

You may want to delete all lines that are not 'dog'

:v/dog/d.

Will find all the lines that are NOT dog, and delete them.

:v is short for :vglobal, also equivalent to :g!

Creating an object from an ActiveRecord::Relation

Rails allows us devs to create a record from an ActiveRecord::Association like so:

> wheel = Car.find_by(type: 'Mazda').wheels.build
> wheel.car_id
1

And in this way we can create a wheel associated with a specific car.

But we can also create an object from an ActiveRecord::Relation that allows us to predefine attributes.

> spare_wheel = car.wheels.where(spare: true).build
> spare_wheel.spare
true

to_partial_path

Passing an active record instance to the render will look for a partial of the active record's name. This is because the active record implements to_partial_path.

This can be mimic'd by a non-active record object like so.

class Fruit
  def to_partial_path
    "fruit"
  end
end

Passing Fruit.new to render like so:

render Fruit.new

Will look for a partial name _fruit.html.erb

I need to iterate with an index! With index!

You need an index while iterating over your collection, right?

> a = ['apple', 'orange', 'banana'];
> a.map.with_index {|fruit, index| puts index}
0
1
2

with_index is a method on Enumerator, and map returns an Enumerator.

I wish I could cURL that network request

So OK, you've got this webpage that's making an xhr (XML http request) and you really want to iterate on the results you're getting back from that request. cURL seems like the answer there, but sometimes constructing a cURL request on the command line isn't worth the effort.

Enter Chrome.

In the network panel of Chrome's developer tools you can ctrl-click the request in question and choose Copy as cURL. Presto! Paste it into the cmd line and start iterating on that endpoint!

Referencing jobs (to kill them)

Generally I've killed processes by pid, like kill -9 30452, but that involves looking up the pid. If your shell started the job you can reference the process by its job id %number.

> cat &
[1] 70058
[1]  + suspended (tty input)  cat
> kill %1
[1]  + terminated  cat

Or you can reference it by name %name

> cat &
[1] 70064
[1]  + suspended (tty input)  cat
> kill %cat
[1]  + terminated  cat

Killing processes in this way surprisingly only works on the last started job.

> cat & ; cat &
[1] 70110
[2] 70111
[2]  + suspended (tty input)  cat
[1]  + suspended (tty input)  cat
> kill %cat
[2]  - terminated  cat
> jobs -l
[1]  + 70110 suspended (tty input)  cat

Moving forth and back on command line

Whenever I use bindkey to list the command line key bindings I see entries such as

"^[f" forward-word
"^[b" backward-word

Alt-f and Alt-b. But when I try to use those I get characters such as ƒ and .

To use forward and backward word in iTerm you must edit a profile in the iTerm preferences and choose the option 'Left Option acts as +Esc' and 'Right Option acts as +Esc'.

`each_cons` in ruby to get differences

I want to get the difference between consecutive elements in an array. Ruby has a method each_cons that provides a block with consecutive numbers of an enumerator.

[1, 3, 7].each_cons(2).map {|a, b| b - a }
=> [2, 4]

Be careful when editing the sudoers file

This website says: Because improper syntax in the sudoers file can leave you with a system where it is impossible to obtain elevated privileges, it is important to use the visudo command to edit the file.

RIGHT. If you screw up the syntax you may remove access to the system. visudo checks the syntax for you and won't let you save a file with incorrect syntax. It responds to the EDITOR env variable so you can edit it in an editor of your choice.

Another command that helps you edit a resource with safety restrictions is vipw.

Get first value from javascript Map

Map is a new key/value data structure in es6 (and its available in your chrome browser RIGHT NOW). But getting values out of a Map without using a key is complicated a little by having to go through a javascript iterator.

> m = new Map([[1,"a"],[2,"b"]])
Map {1 => "a", 2 => "b"}
> m.values()[0]
undefined
> m.values()
MapIterator {"a", "b"}

Its all good though, the next() function of a fresh iterator will always return the first value. Well, not really the first value but a value wrapped in an object with two values, the value you are looking for represented by value and done which tells you whether the iterator has run out of elements or not.

> m = new Map([[1,"a"],[2,"b"]])
Map {1 => "a", 2 => "b"}
> m.values().next().value
"a"

Javascript closures bind to the variable

If you're a clever kind of javascript dev you create functions with variables from outside the function definition.

var i = 5;
function printSomething() {console.log(i);}
printSomething();
> 5

But you should watch yourself when using a closure within a loop.

> fns = []
> for(var i = 0; i < 3; i++) {fns.push(function () {console.log(i); })};
> fns[0]();
3
> fns[1]();
3
> fns[2]();
3

Yikes! The closure references the variable as it is now, not the variable as it was. You should only reference the variable if you don't expect it to change (or if you want it to change).

> fns = []
> function createFn(i) { return function() { console.log(i); };}
> for (var i = 0; i < 3; i++) {fns.push(createFn(i))};
> fns[0]();
0
> fns[1]();
1
> fns[2]();
2

Javascript keys are only strings!

You can use squares to access or set a value in an object, but that key is just getting converted to a string. And you can't loop through the keys and expect the keys to be objects... they're strings!

> a = {1: "x"}
{ '1': 'x' }
> a[/x/] = "y"
'y'
> a[1]
'x'
> a[/x/]
'y'
> a['1']
'x'
> a['/x/']
'y'
> for (var i in a) { console.log(i === 1); console.log(i === /x/) }
false
false
false
false

The microseconds since navigation started

Date.now will give you milliseconds since the epoch, but maybe what you really want is the time since the page started loading.

> performance.now()
162374.73599999974

Essentially that is the microseconds since the page started loading, but really its the microseconds since performance.timing.navigationStart which itself is defined as the:

time immediately after the user agent finishes prompting to unload 

which is defined here.

Treat a subquery like a value

Postgres is cool with returning a value from a query.

select 1;

And also returning a value from a subquery.

select (select 1);

Turtles all the way down.

select (select (select 1));

But that subquery can only return one row or you'll get an error.

select (select unnest('{1, 2}'::integer[]));
ERROR:  more than one row returned by a subquery used as an expression

This is used sometimes when you need to use a count in an expression.

select round((select count(*) from pg_class where reltype = 0) / (select count(*)::float from pg_class) * 100) as percent_with_zero_reltype;
 percent_with_zero_reltype 
---------------------------
                        39

Inline your helper_method declaration

Remember, def is just a special function in ruby and as such it has a return value that is a symbol that is the name of the function being defined.

That allows us to do some cool (weird?) looking things with class methods that take a symbol that refers to a function, like the helper_method call in a controller.

  class DataController < BaseController

  #actions and whatever

  private

  helper_method def format_that_data(data)
    "#{data[0]} X #{data[1]}"
  end

  end

`gf` go-to-file mapping goes-to-gem-source

Everyone knows (or does not know) that gf when the cursor is over a word in vim will try to find a file on the system to navigate to that matches that word or path.

Rails overrides this to be used on constants and to navigate to the file where a constant is defined if the constant name matches the file name (matches in the rails sense).

When you're in the Gemfile, gf on a gem will take you to that gem, just like bundle open. Brilliant!

Log timings with javascript console

If you're curious about the time delta between two lines of code in javascript, you can use the console.time and console.timeEnd functions to get that delta.

> console.time("a")
undefined
> console.timeEnd("a")
a: 8136ms
undefined

The string passed in is just a token for the section you want to time.

h/t Chase McCarthy

Clone local git repo

I git clone github repositories all the time but I didn't realize you could clone a git repo on the file system.

git clone myproject myproject2

And what's crazier, I can clone the .git dir directly and I get the same result.

git clone myproject/.git myproject2

This is important because Capistrano creates a 'repo' dir on the target server that is a copy of the .git directory but is detached from any working tree (the set of files you'd normally see in a git repo).

To clone from a capistrano server you can copy the repo dir locally and git clone it.

Extracting values from Postgres json blob

Lets get value 'a':

select '{"a": "b"}'::json->>'a';
  ?column?  
------------
 b

Howbout a nested value 'b'?

select ('{"a": {"b": "c"}}'::json->>'a')::json->>'b';
 ?column? 
----------
 c

Howbout the first value of an array?

select ('{"a": [{"b": "c"}]}'::json->>'a')::json->>0;
  ?column?  
------------
 {"b": "c"}

There's a better way to get a nested path

select json_extract_path('{"a": {"b": "c"}}', 'a', 'b');
 json_extract_path 
-------------------
 "c"
(1 row)

But you can't grab a value from an array with that method. But at least json_extract_path returns a value of type json so you don't have to cast it before using the feather (->>) operator.

select json_extract_path('{"a": {"b": ["c"]}}', 'a', 'b')->>0;
 json_extract_path 
-------------------
 "c"
(1 row)

Time Zone offsets in the DB

When you need a timezone offset in the database use

select utc_offset from pg_timezone_names where name = 'Asia/Hong_Kong';

But in this case utc_offset is an interval and maybe you want an integer

select '1 hour'::interval::integer;   -- << THROWS ERROR

Throws an error cannot cast type interval to integer

But you can extract the hour from an interval

select extract(hour from utc_offset) from pg_timezone_names where name = 'Asia/Hong_Kong';

-[ RECORD 1 ]
date_part | 8

This operation can be unexpectedly expensive to run (100ms) so be careful where you run it or maybe put this rarely changing information into a materialized view.

Check to see if a subquery returns any records

A subquery may or may not return results. And you may want to make decisions for your data set based on that.

select 'Here exists a record' as word where not exists (select 1 where false);
      word
-----------------
 Here exists a record
(1 row)

select 'no record :(' as word where not exists (select 1 where true);
 word
------
(0 rows)

Some connections are just hangin' KILL THEM

Jake has covered in the past how to see connections and their queries in postgres with

select * from pg_stat_activity;

It returns a pid column and now you can use that pid to do these processes harm.

select pg_terminate_backend(pid) from pg_stat_activity where THERES THAT QUERY GET IT

How busy are your cores?

Linux talks about core utilization in terms of Load Average. This is, the average load over 1 minute or over 5 minutes or over 15 minutes.

For a 1 core machine a Load Average of 1.00 is 100%. For an 8 core machine 8.00 is 100%. On my 4 core mac I can check my Load Average with uptime

chriserin@:~% uptime
20:48  up 17 days,  1:42, 7 users, load averages: 1.23 1.53 1.59

Over the last 1 minute, 1.23. Over the last 5 minutes, 1.53. Over the last 15 minutes, 1.59.

My CPU is doing great! You can also see the same numbers in htop. Source

You don't need those traces!

Josh looked at disassembling ruby recently.

It outputs code like:

0000 trace            1                                               (   1)
0002 putobject_OP_INT2FIX_O_1_C_
0003 dup
0004 setlocal_OP__WC__0 2
0006 leave

The first instruction trace tells ruby to emit a trace event (trace system detailed here).

The trace instructions may not be useful when examining disassembled code and as such you can compile the code without the trace instructions.

RubyVM::InstructionSequence.compile_option = {
  trace_instruction: false
}

This site uses the Rosario font

A distinctive feature of the font this site uses is the Q with the long tail that swoops under the next letter and the letter after that.

ROSARIO is the font with the Q

So when you see Q in SQL you know its Rosario.

Flatten array in javascript

Flattening an array with nested arrays (2 dimensional array) in javascript is relatively easy if unintuitive.

> [].concat.apply([], [[1],[2,3],[4]])
[ 1, 2, 3, 4 ]

But an array with 3 or more dimensions or levels of nesting gets messy.

> [].concat.apply([], [[1], [2, [3]]])
[ 1, 2, [ 3 ] ]

This Stack Overflow thread has some solutions. This underscore implementation is a good reference as well.

How does postgres get its meta data?

When you get to a psql command line prompt and enter \dt psql will output all the tables in the current schema.

But if you open psql with

psql -E database_name

postgres will show you the query it runs to satisfy \dt.

SELECT n.nspname as "Schema",
  c.relname as "Name",
  CASE c.relkind WHEN 'r' THEN 'table' WHEN 'v' THEN 'view' WHEN 'm' THEN 'materialized view' WHEN 'i' THEN 'index' WHEN 'S' THEN 'sequence' WHEN 's' THEN 'special' WHEN 'f' THEN 'foreign table' END as "Type",
  pg_catalog.pg_get_userbyid(c.relowner) as "Owner"
FROM pg_catalog.pg_class c
     LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','')
      AND n.nspname <> 'pg_catalog'
      AND n.nspname <> 'information_schema'
      AND n.nspname !~ '^pg_toast'
  AND pg_catalog.pg_table_is_visible(c.oid)
ORDER BY 1,2;

HashWithIndifferentAccess for keyword arguments

HashWithIndifferentAccess is a cool hash-like object that allows you to access values with either a symbol OR a string. But you can't pass it to a method that expects keyword arguments.

def foo(a:, b:); return a, b; end
h = HashWithIndifferentAccess.new({a: 1, b: 2})
foo(h)
=> ArgumentError: wrong number of arguments (1 for 0)

Fear not!

foo(h.symbolize_keys)
=> 1, 2

Calling symbolize_keys on a HashWithIndifferentAccess object will create a hash that can be used for keyword arguments.

http://apidock.com/rails/v4.2.1/Hash/symbolize_keys

Using while to iterate a javascript iterator

The key to iterating over a javascript Iterator is that next returns a javascript object with two entries, value and done. If done is false, then calling next on the iterator will yield another object with a value. When done is true then you're done!

while(object = objectIterator.next(); !object.done) {
  myValue = object.value
}

Implementing an enumerator is easy

All you need to implement an enumerator is the each method.

class ChicagoDevs
  include Enumerable

  def each &block
    ['ce','jb', 'jc', 'jd', 'jw', 'bd', 'dp'].each{|member| block.call(member)}
  end
end

h/t Brian Dunn

Bird Beak Bash (or process substitution)

http://www.tldp.org/LDP/abs/html/process-sub.html

Process substitution feeds the output of a process (or processes) into the stdin of another process.

<() creates a file descriptor and is most useful when using a command that takes two files as arguments like diff or comm but you want to use output from commands rather than files.

comm <(ls dirA) <(ls dirB)

mkdir -p throws no errors

mkdir throws errors if you try to create a directory that already exists.

mkdir -p is cool with not creating and not throwing an error if you try to create a dir that already exists.

From the man page:

On the other hand, with this option specified, no error will be reported if a directory given as an operand already exists.

That feels a lot better than if [ ! -f myDir ]; then mkdir myDir end; fi

Docker volume binding

When starting a docker dir, you can map a dir on the host system to a dir in the container with the -v option.

docker start my-image -v /var/logs/app:/app/log

In this case the host dir is on the left side of the colon and the docker container dir is on the right.

*nix user ids and groups

Does a user exist on the system?

$ id -u <username>

That'll give you the id, like 1006.

$ groups <username>

That'll give you the groups a user is a member of, like ssh staff admin

Gist indexes understand ranges

Imagine you're selling time in a room, but that room has a limited capacity. Anybody can book an arbitrary amount of time in the future, but there can only be 5 people in the room at any given time. You'll want to query the db to ask it how many people are in the room for a certain amount of time before you sell that time.

For many many rooms and many many slices of time, you'll get a query like

select count(*) from rooms r join slices s on s.room_id = r.id where r.id = ? and tstzrange('2016-01-02', '2016-01-05') && s.time_range;

This is going to take in the many hundreds of ms to run, but never fear, an index is near.

alter table slices create index speed_things_up gist(room_id, time_range);

And now the above query takes 10s of ms to run!

Tmux command targets

With a new commit to our environment to enable project specific tmux configurations its been fun to dig into tmux automation.

tmux send-keys -t seq27:0.0 'vim .' C-m

Will send the keys 'vim .' to a very specific target -t seq27:0.0

seq27 = the name of the session

0 = the first window of that session

0 = the first pane of that window

Thats how you can open vim in a specific pane in your tmux session automatically!

Bash true!

We needed to inspect that a return code made it back from docker's exec command today.

  docker exec -it <docker-container> "true" && echo "TRUE"

Will print TRUE

  docker exec -it <docker-container> "! true" && echo "WONT GET PRINTED"

Won't print anything! Return codes FTW. true returns 0. ! true returns 1.

The third argument of radio_button_tag is checked!

Of course I should know this but I didn't, 3rd value of radio_button_tag is whether the input is checked or not.

radio_button_tag("name", "value", isChecked)

#stilllearningrails

Filter empty strings from array

Given an array with many empty strings

arr = ["a", "", "b", "", "c", "", "d"];

To get rid of the empty strings call filter and pass the Boolean method.

arr.filter(Boolean);
> ["a", "b", "c", "d"]

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Boolean

The most generic type of constraint

The Check constraint is the most generic type of constraint.

alter table things add CONSTRAINT corners_must_be_greater_than_0 CHECK (corners > 0);

Ignore hang ups when starting a process

Start a process, exit the current shell, process continues.

$ nohup rake qc:work &

nohup enables your program to ignore the HUP signal a controlling process sends.

Example

$ nohup cat &                                                                                                              
> [1] 2766
> appending output to nohup.out
$ kill -1 2766
$ ps -A | grep 2766
> 2766 ttys003    0:00.00 cat

RVM binaries storage

RVM binaries are stored at:

https://rvm.io/binaries

They have binaries for 9 operating systems, the versions of each operating system, and the system architecture that each version supports.

Ruby timeout

Included in the ruby std lib is a timeout lib. It provides the capacity to auto-terminate a potentially long running operation.

http://ruby-doc.org/stdlib-2.2.1/libdoc/timeout/rdoc/Timeout.html

require 'timeout'
status = Timeout::timeout(5) {
  # Something that should be interrupted if it takes more than 5 seconds...
}

Convert interger to time

Time.at(Time.now.to_i)

command line SQL help

Documentation

\h [NAME]              help on syntax of SQL commands, * for all commands

Example

\h ROLLBACK TO SAVE POINT

Output

Command:     ROLLBACK TO SAVEPOINT
Description: roll back to a savepoint
Syntax:
ROLLBACK [ WORK | TRANSACTION ] TO [ SAVEPOINT ] savepoint_name

/ht Brian

Retab

:retab resets the tabs dependent on shift width and tab stop.