Today I Learned

A Hashrocket project

Expand Emojis With The Spread Operator

There are a number of emojis that are not stand-alone unicode characters, but instead are a combination of two or more other emojis. The two main places this happens is with family emojis and emojis using non-simpsons skin tones.

You can use JavaScript's spread operator to expand these emojis to see what their base components are. Here is a screenshot of a few that I expanded from Chrome's dev tools.

source

Documentation Lookup With Vim and Alchemist

Which argument position is the accumulator for Enum.reduce/3?

How does group_by work?

I find myself fairly frequently jumping from vim to Chrome to do Google searches for Elixir standard lib documentation. It gets the job done, but it is kinda slow and I'd prefer to avoid the context switch.

With alchemist.vim, Elixir documentation lookup is at your finger tips. Just move the cursor over the module or function you are curious about and hit K (from normal mode).

Curious about Enum.reduce? Type it out in your current Vim buffer, move the cursor over it, and hit K.

Elixir with macro `<-` and `=`

So Elixir with macro accepts Matching clauses <- and Bare expressions =. Both match patterns and do not leak variables assigned inside these structures.

with {:ok, width} <- Map.fetch(%{width: 10}, :width),
  do: {:ok, 2 * width}
#=> {:ok, 20}

with {:ok, width} = Map.fetch(%{width: 10}, :width),
  do: {:ok, 2 * width}
#=> {:ok, 20}

So what is the difference between these?

When a match fails, matching clauses <- returns failed result but bare expressions = raises a MatchError:

with {:ok, width} <- Map.fetch(%{height: 10}, :width),
  do: {:ok, 2 * width}
#=> :error

with {:ok, width} = Map.fetch(%{height: 10}, :width),
  do: {:ok, 2 * width}
#=> ** (MatchError) no match of right hand side value: :error

Another difference is that when guard is only available for matching clause.

with {:ok, width} when is_number(width) <- Map.fetch(%{width: 10}, :width),
  do: {:ok, 2 * width}
#=> {:ok, 20}

Verify downloaded files from the web #security

If you download a file from the web on a public WiFi and want to run on your machine you might want to check if the file has not been tampered with by a man-in-the-middle-attack or if the file host has been breached.

The easiest way to do this is to check the publised md5 or sha-1 hash for that file (you can do that via your phone if you want to be extra secure). Not every package publishes that but if they do it will be on their website usually next to the download link.

To verify the file you will need to hash the file you downloaded using openssl. For example:

 $ openssl sha1 Kali-Linux-2016.1-vm-amd64.7z
 SHA1(Kali-Linux-2016.1-vm-amd64.7z)= 2b49bf1e77c11ecb5618249ca69a46f23a6f5d2d

Which matches the site's published sha-1 hash:

kalisha

If you want to check md5, simply replace sha1 in the command with md5.

Updating Values In A Map

When working with maps in any language, you often need a way to update key-value pairs. Furthermore, you will need a way to handle keys that are not already present in the map, generally associating some default value.

In Elixir, the Map module provides the get_and_update/3 function as a way of accomplishing such a task.

Let's use a score counting example to see it in action:

> scores = %{}
%{}
# jake scores a point
> {_, scores} = Map.get_and_update(scores, :jake, fn(x) -> {x, (x || 0) + 1} end)
{nil, %{jake: 1}}
# chris scores a point
> {_, scores} = Map.get_and_update(scores, :chris, fn(x) -> {x, (x || 0) + 1} end)
{nil, %{chris: 1, jake: 1}}
# jake scores another point
> {_, scores} = Map.get_and_update(scores, :jake, fn(x) -> {x, (x || 0) + 1} end)
{1, %{chris: 1, jake: 2}}
# final scores
> scores
%{chris: 1, jake: 2}

We use (x || 0) + 1 as a way of providing an initial score for new keys.

The update function is expected to return a tuple with the original value and the updated value.

See the docs for more details.

Return `MatchData` with `$~`

Check out this Ruby regex:

2.1.0 :001 > "totally gourmet" =~ /totally/
 => 0

There's a match at index zero. But what is the thing that actually matched? Enter $~:

 2.1.0 :002 > $~
  => #<MatchData "totally">

$~ stores the result of the last regex, and can be called later on. This is equivalent to the match method:

2.1.0 :003 > /totally/.match('totally gourmet')
 => #<MatchData "totally">

h/t Brian Dunn

Psql Command with Tuples Only

With psql, we can run any string command once and exit using the -c (command) flag. Here's an example:

$ psql hr-til_development -c 'select title, slug from posts limit 5';
                  title                   |    slug
------------------------------------------+------------
 Start rails server in production mode    | 97e26f5f68
 Enable ScreenSharing from a terminal     | 55dfe1a1a3
 Variable Hoisting in JavaScript          | eeedb8dda0
 Three ways to compile Elixir             | f2a423d1e1
 Indent and de-indent line in INSERT MODE | 8de0fb5d93
(5 rows)

If you are parsing the result, the headers can get in the way. This is where the -t (tuples only) flag is useful. Same query:

$ psql hr-til_development -tc 'select title, slug from posts limit 5';
 Start rails server in production mode    | 97e26f5f68
 Enable ScreenSharing from a terminal     | 55dfe1a1a3
 Variable Hoisting in JavaScript          | eeedb8dda0
 Three ways to compile Elixir             | f2a423d1e1
 Indent and de-indent line in INSERT MODE | 8de0fb5d93

Notice the headers and row count are gone. Toggle this in a psql session with the \t command.

Rotate Everything By 13 Letters

For some inane reason, Vim comes with a ROT-13 feature. So, if you are ever in need of rotating the letters of some portion of the file by 13, you can do that with the simple g? binding.

For example, if you hit g?? on the following line:

Six dollar eggs

you will get

Fvk qbyyne rttf

As you can see, casing is preserved.

The only practical uses of this are Vimgolf and convincing people at coffee shops that you are a hacker.

See :h g? for more details.

Show Matching Entries For Help

Looking up things with Vim's :help command can be error prone if you don't know exactly how to format what you are looking up. Bypass some of the guesswork by hitting Ctrl-d after writing part of the :help command. This will populate a wildmenu of possible matches.

For instance, if you know there is a command containing ?, but you aren't sure how to look it up, try the following:

:help ?<Ctrl-d>

You can tab through to the one you want and hit enter to read up on it. Who knew there were so many Vim bindings involving a ?.

See :h help-context for more details.

Grep your Database 🔎

Wading through a database full of text (as might support a CMS) and feeling lost? Sometimes you gotta ditch the queries and just start grepping.

This Stack Overflow post sums up the idea. Here's a version of the command we used today to pick through a text-rich database:

$ curl `heroku pg:backups public-url -rproduction` | pg_restore | egrep -o '[a-z]+\.chicagocatlounge.biz'

This found, via a regex match, all subdomains of our target URL, in any text field of the production database. Pretty awesome.

Create a Placeholder for Content Editable Divs

When you have a content editable div. Add an data attribute. We're going to use placeholder:

<div contenteditable="true" data-placeholder="(untitled)" class="title">
</div>

Then in your css, use a few pseudo classes to make it behave like a placholder:

.title:empty:not(:focus):before {
  content: attr(data-placeholder)
}

Very cool! I (for whatever reason) never knew about :empty, :not, attr() or chaining multiple pseudo classes. Learned a lot in this little snippet :)

h/t @alejandroRINC0N

Find Dead Code with Unused

Today I discovered a fascinating tool called Unused, written by Josh Clayton.

Unused leverages two of my favorite projects, Exhuberant Ctags and The Silver Searcher, to identify dead code. I installed it with homebrew.

Dead code is code that is never used by your application. It wastes electricity and mental bandwidth, and can hang around unnoticed for years. We need powerful tools to find and remove it.

Unused threw out a few false positives, as expected. Use with caution and a solid test suite.

The Unused website highlights several pull requests removing dead code from popular open source projects. I was able to use it on 'Today I Learned' itself. A small win:

https://github.com/hashrocket/hr-til/pull/101

Github Advanced Search Parameters

I really enjoy browsing Github. The search feature in particular is a fantastic way to stumble on interesting projects.

Search has an 'advanced' menu that lets you restrict your query. I've used it enough that I have some preferences that I enter directly into the search bar: is:issue is:open label:"help wanted" finds open issues actively seeking outside contributions. Not everybody uses labels, so removing that can yield even more results.

Add a language filter as a query parameter, and you're on your way to some good open source contribution opportunities. Drill down further with a search term, such as 'testing' or 'flicker'.

Treat null as if it is a known value #postgresql

When you query a table sometimes you want to check if a nullable field is not equal to a value. For example:

select * from sometable where afield != 'avalue';

However the query above will exclude rows where afield is null, so you would typically add that as an additional condition:

select * from sometable where afield is null or afield != 'avalue';

When you are doing it once it may be ok but as queries get bigger this makes the query messy and harder to read. Fortunately Postgres offers a more idiomatic way to check if a value does not equal something, including null values: is distinct from and is not distinct from.

select * from sometable where afield is distinct from 'avalue';

This query will return all the rows where afield is null or anything but avalue. Conversely:

select * from sometable where afield is NOT distinct from (select x from y limit 1);

will return all the values that are equal to the result of the subquery above and is useful when the result of the subquery could be null.

h/t Jack Christensen

Original docs: https://wiki.postgresql.org/wiki/Is_distinct_from

Ruby defined? method evaluation

Today I learned that Ruby Object method defined? evaluates if the argument is anything recognizable, not just if the variable are set. Here are some examples:

undefined_foo #=> NameError: undefined local variable or method `undefined_foo'
defined? undefined_foo #=> nil

defined? nil #=> "nil"

defined? "foo" #=> "expression"
defined? [] #=> "expression"
defined? def x; end #=> "expression"
defined? class Foo; end #=> "expression"

defined? @foo = :bar #=> "assignment"

@foo = :bar
defined? @foo #=> "instance-variable"

foo = :bar
defined? foo #=> "local-variable"

def foo; end
defined? foo() #=> "method"

class Foo; end
defined? Foo #=> "constant"

h/t @VidalEkechukwu

Checking Commit Ancestry

I have two commit shas and I want to know if the first is an ancestor of the second. Put another way, is this first commit somewhere in the history of this other commit.

Git's merge-base command combined with the --is-ancestor flag makes answering this question easy. Furthermore, because it is a plumbing command, it can be used in a script or sequence of commands as a switch based on the answer.

Here is an example of this command in action:

$ git merge-base --is-ancestor head~ head && echo 'yes, it is'
yes, it is
$ git merge-base --is-ancestor head~ head~~ && echo 'yes, it is'

In the first command, head~ is clearly an ancestor of head, so the echo command is triggered. In the second, head~ is not an ancestor of head~~ so the return status of 1 short-circuits the rest of the command. Hence, no echo.

See man git-merge-base for more details.

source

List Different Commits Between Two Branches

There are times when I want to get a sense of the difference between two branches. I don't want to look at the actual diff though, I just want to see what commits are on one and the other.

I can do just this by using the git-log command with a couple flags, most importantly the --cherry-pick flag.

To compare the feature branch against the master branch, I can run a command like the following:

$ git log --left-right --graph --cherry-pick --oneline feature...branch

This lists commits with the first line of their messages. It also includes either a < or > arrow at the front of each commit indicating whether the commit is on the left (feature) or right (master) branch, respectively.

Note: you can use more than branches in a command like this. Any two references will work. You can just use two SHAs for instance.

source

non recursive find

The find command is recursive by default. Let's say you want to find all the screen shots on your ~/Desktop, but ignore any screens in the directories in Desktop. Setting the maxdepth flag will tell how far to recurse, using the value 1 only searches the directory you specified, using 2 would go into each directory, but only not search any sub directories, and so on.

#this will recurse by default
$ find ~/Desktop -name "*Screen*"

#this will only search the Desktop and it's immediate subdirectories
$ find ~/Desktop -maxdepth 2 -name "*Screen*"

Combine Multiple Rake Commands Without &&

Ordinarily, to successively run commands whose execution depends on the previous command's success, you would separate the commands with two &'s. This includes rake commands:

rake db:migrate && rake db:redo && rake db:test:prepare

The downside to this, however, is that the Rails environment is loaded with each rake invocation. Thus, in this example the Rails environment is loaded three times.

This is slow. To speed up the process, we can load the Rails environment just once as follows:

rake db:migrate db:redo db:test:prepare

We still run all the tasks but don't have to wait an eternity for Rails to get its act together and load rake. Hooray!

Hat-tip @mrmicahcooper

Jump To The Ends Of Your Shell History

There are all sorts of ways to do things in your shell environment without reaching for the arrow keys. For instance, if you want to move up to the previous command, you can hit Ctrl-p. To move down to the next command in your shell history, you can hit Ctrl-n.

But what if you want to move to the beginning and end of your entire shell history?

Find your meta key (probably the one labeled alt) and hit META-< and META-> to move to the end and beginning of your shell history, respectively.

source

One query with Select In

Today I learned that if you pass an ActiveRecord::Relation as param to where Rails will create a single query using IN and an inner SELECT:

Repository.where(user_id: User.active.select(:id))
=>  Repository Load (1.1ms)
        SELECT "repositories".*
        FROM "repositories"
        WHERE "repositories"."user_id" IN (
          SELECT "users"."id"
          FROM "users"
          WHERE "users"."status" == 1
       )

h/t @mattpolito

zsh variable editing

Lets say I made a typo in an important environment variable and I wanted a quick fix, just to see if yes, that was indeed why my application wasn't working.

$ echo $RAILS_ENV
> porduction
# ^^^ YIKES ^^^

The Z shell has a builtin program called vared that helps you edit an existing environment/shell variable.

$ vared RAILS_ENV
porduction

Now you are in an editing context, you can edit the value with all your configured command line editing tools. Fix the error, hit RETURN, and you're good!

List Names Of Files With Matches

I often use grep and ag to search for patterns in a group or directory of files. Generally I am interested in looking at the matching lines themselves. However, sometimes I just want to know the set of files that have matches. Both grep and ag can be told to output nothing more than the names of the files with matches when given the -l flag.

This can come in particularly handy if you just want a list of files that can be piped (or copied) for use with another command. This eliminates all the extra noise.

h/t Dillon Hafer

Drop connections with Nginx

You can ignore requests sent to nginx by responding with the status 444

server {
  listen 80;
  listen [::]:80;
  listen 443;
  listen [::]:443 ssl http2;

  server_name dillonhafer.com;
  location /no-response {
    return 444;
  }
}

The primary use for this is to prevent processing requests with undefined server names:

# block for requests with undefined server names
server {
  listen 80;
  listen [::]:80;

  return 444;
}

whois that url?

Let's say you just got a spammy text message that contains a suspicious url and you want to learn more but clicking the link just seems like a bad idea.

You can learn more from the whois program that comes installed on macosx. It sends queries to databases that store domain registration information.

When you use the command:

whois spam.com

You can see the important result

Registrant Organisation: Hormel Foods Corporation

The wikipedia article for whois contains some deep internet history.

Reversing A List

To efficiently work with and transform lists in Elixir, you will likely need utilize a list reversing function from time to time. Your best bet is to reach for the Erlang implementation which is available as part of the lists module.

Here are a couple examples of how to use it:

> :lists.reverse([1,2,3])
[3, 2, 1]
> :lists.reverse([1, :a, true, "what", 5])
[5, "what", true, :a, 1]

Note: though I said _transform_ lists above, what is actually going on is that a new version of the list representing my transformation is being created, per Elixir's functional nature.

Reset Target tslime Pane

The combination of tslime and turbux makes running tests from Vim in a tmux session as easy as a single key binding. One problem that can arise from time to time is having tslime focused on an undesired tmux window/pane combination. There is no binding to tell tslime that you'd like to re-select the target window and pane.

I've often resorted to closing out of Vim in order to reset the prompt. There is a better way and it doesn't require you to wipe out your Vim session.

Just unlet the global Vim variable for the tslime plugin like so:

:unlet g:tslime

The next time you invoke turbux it will see that g:tslime isn't set and will prompt you for a new window and pane combination.

h/t Josh Davey

`Enum.chunk_by`

I discovered a pretty cool Elixir function this weekend, Enum.chunk_by.

Backstory: I wanted to group a list of strings by equality, while preserving the list order.

Enum.chunk_by takes an enumerable and a function, and breaks that enumerable into an enumerable when the function returns a new or different result.

Here's a simple example:

iex> Enum.chunk_by(['A', 'A', 'A', 'B', 'A'], fn(l) -> l end)
[['A', 'A', 'A'], ['B'], ['A']]

Anytime l changes, a new list is created.

Slightly more complex:

iex> Enum.chunk_by([1, 2, 3, 4, 5], fn(n) -> rem(n, 3) == 0 end)
[[1, 2], [3], [4, 5]]

The function only returns something different (true) on 3, so 3 is assigned to its own list.

Grep through compressed (gzipped) log files

The logrotate linux utility automatically compresses your ever-growing production log files.

If you encountered an error and wanted to search the history including all compressed logs, you may have considered unzipping all of them into a directory and run a grep in that directory.

Fortunately linux offers a more idiomatic way for grepping gzipped files: zgrep.

From the manual:

zgrep, zegrep, and zfgrep act like grep, egrep, and fgrep, respectively, but accept input files compressed with the compress(1) or gzip(1) compression utilities.

h/t Jack Christensen

Simplify System.cmd with W sigil

Say we have the following shell command to get the the unix timestamp of the last commit:

$ git log -1 --date=short --pretty=format:%ct

1470067380

In elixir. One might do it like so:

System.cmd("git", ["log", "-1", "--date=short", "--pretty=format:%ct]) 
|> elem(0)

#=> "1470067380"

And here is just a simpler syntax for the same thing:

System.cmd("git", ~w[log -1 --date=short --pretty=format:%ct])
|> elem(0)

#=> "1470067380"

This is just a little clean and feels more like the original command.

Ruby-Like `split` in Elixir

Elixir's split function is a bit different from the Ruby version you might be familiar with.

Here's Ruby's split:

2.1.0 :001 > "FOOBAR".split("")
 => ["F", "O", "O", "B", "A", "R"]

And Elixir's:

iex(1)> String.split("FOOBAR", "")
["F", "O", "O", "B", "A", "R", ""]

Whoa, what's that extra "" doing in there? Drink in the Elixir. A small amount of message board reading has led me to conclude this was very deliberate and isn't going to be changed.

Here's one way to get a Ruby-like split:

iex(1)> String.split("FOOBAR", "", trim: true)
["F", "O", "O", "B", "A", "R"]

Open Multiple Tabs with Vim

I like to use Vim to open many files at once. Today I discovered the -p flag to help with this process.

To open all the files in a directory in tab mode, use this command:

$ vim -p path/to/files/*

-o does the same thing in horizontal splits; -O in vertical splits.

When you're finished reading, :qall is a nice way to close the book.