Today I Learned

A Hashrocket project

152 posts about #command-line

More useful Homebrew searches #macOS #homebrew

Homebrew, the third-party package manager on macOS, allows searching for packages by name, but the list that comes out only contains package names. That’s not always very useful, particulary when you are not sure what you are looking for.

To get the package description along with the package name simply add --desc to your brew search command.

For example, let’s look for a library for performing file diffs with color highlighting:

$ brew search --desc diff
apgdiff: Another PostgreSQL diff tool
cdiff: View colored diff with side by side and auto pager support
cern-ndiff: Numerical diff tool
colordiff: Color-highlighted diff(1) output
cppad: Differentiation of C++ Algorithms
dhex: Ncurses based advanced hex editor featuring diff mode and more
diff-so-fancy: Good-lookin' diffs with diff-highlight and more

You can also search using regex in both the description and name of the package as long as you supply the --desc option:

$ brew search --desc /[cC]olor.*[dD]iff/
cdiff: View colored diff with side by side and auto pager support
colordiff: Color-highlighted diff(1) output
icdiff: Improved colored diff

`cd` in subshell

With many of our projects sequestering the front end javascript code into an assets directory I find myself moving between the root project directory and the assets directory to perform all the npm or yarn related tasks in that assets dir. Inevitably I’ll start doing something like this:

cd assets; npm install; cd ..

or this

pushd assets; npm install; popd

In both cases using ; instead of && puts me back in the original directory regardless of the result of the npm command.

I just learned that using cd in a subshell does not change the directory of the current shell, so I can also do this:

(cd assets; npm install)

Xargs from a file

I’ve struggled with xargs conceptually for long time, but actually its pretty easy conceptually. For commands that don’t read from stdin but do take arguments, like echo or kill, you can turn newline separated values from stdin in into arguments.

Piping to echo does not work.

> echo 123 | echo
# nothing

Using xargs it does.

> echo 123 | xargs echo

xargs can also read a file with the -a flag, turning each line of the file into an argument.

> echo "123\nabc" > test.txt
> cat test.txt
> xargs -a test.txt echo
123 abc

H/T Brian Dunn

Kill rogue shell processes

There is a particular type of attack where an inserted usb stick can act like a keyboard, open a terminal, and start something like this:

while (true); do something_malicious; sleep 3600; done & disown

This process endlessly loops and wakes every hour to do something malicious. The & puts it in the background and the disown will end its attachment to the current terminal. When the terminal is closed the process will get a parent of 1.

This process is still detectable and killable at the command line by finding all shell programs with a parent pid of 1 and killing them with -9.

ps ax -o pid,command,ppid | grep '.*zsh.*\s1$' | awk '{print $1}' | xargs kill -9

This will kill all running rogue zsh processes. There may be reasons why you’d want a process to be detached from its parent terminal, but you could easily decide that this isn’t something you want ever and place the above command into a cron job that runs every 2 seconds.

Download all of humble bundle books in parallel

Humble Bundle is a great site which offers technical book bundles. The problem is that they present the user with a huge list of links for all the different formats and it is a tedious task to right click each link and save it to your hard drive.

In order to solve this you can open the Developer Tools while on the download page and paste the following:

var pattern = /(MOBI|EPUB|PDF( ?\(H.\))?|CBZ|Download)$/i;
var nodes = document.getElementsByTagName('a');
var downloadCmd = '';
for (i in nodes) {
    var a = nodes[i];
    if (a && a.text && pattern.test(a.text.trim()) && a.attributes['data-web']) {
        downloadCmd += 'wget --content-disposition "' + a.attributes['data-web'].value + "\"\n";
var output = document.createElement("pre");
output.textContent = downloadCmd;


This will add a pre tag to the page with a bunch of wget commands. Go ahead and copy those to your clipboard.

Using GNU Parallel (brew install parallel). First save the contents of your clipboard into a file, for example download_jobs then run the following command:

parallel -j 4 < download_jobs

Replace 4 with the number of cores you have on your machine.

Then sit back and watch your directory get populated with files.

Surround every line in a file using sed

To replace every line in a file you can use linux’s built in sed utility:

For example given a file like this:


If we want to surround each line with Plug '$content_of_line' we can run the following command:

sed -e "s/\(.*\)/Plug '\1'/" .vimbundle.local


Plug 'dkarter/backpack'
Plug 'junegunn/fzf'
Plug 'junegunn/fzf.vim'
Plug 'junegunn/vim-peekaboo'
Plug 'junegunn/gv.vim'
Plug 'terryma/vim-multiple-cursors'
Plug 'scrooloose/nerdtree'
Plug 'dyng/ctrlsf.vim'
Plug 'haya14busa/incsearch.vim'
Plug 'killphi/vim-legend'
Plug 'neomake/neomake'

If the result is what we expected we can add -i flag to write the file in place, updating it with our changes:

sed -ie "s/\(.*\)/Plug '\1'/" .vimbundle.local

Keep Your Brews Bubbly

Life is too short to have Homebrew problems. Run these commands to keep your brews bubbly.

Checks your system to make sure that future installs go smoothly:

brew doctor

Upgrades Homebrew to the latest version:

brew update

Gets a list of what packages are outdated:

brew outdated

Looks through your installed packages and deletes any old versions that may still be hanging around:

brew cleanup

You could alternatively add the --dry-run flag to cleanup to see all the outdated packages that would be removed.

Deletes old symlinks:

brew prune

Updates packages to the latest version:

brew upgrade

You can add the --cleanup flag to delete older versions of the packages you are updating.

List Files Ordered By Modification Date

The ls command lists the files in a directory. Tacking on the -l flag will list it in long format. By default, everything is listed in lexicographical order. This is what ls -l looks like for this repository.

-rw-r--r--    1 jbranchaud  staff    628 Feb 14  2016
-rw-r--r--    1 jbranchaud  staff   1058 Feb 19  2015 LICENSE
-rw-r--r--    1 jbranchaud  staff  40983 Aug 18 16:59
drwxr-xr-x    5 jbranchaud  staff    170 Apr  1 14:45 ack
drwxr-xr-x    5 jbranchaud  staff    170 Feb 24 16:31 chrome

Sometimes you want a sense of what has been modified and when. Lexicographical order isn’t going to help much here. By tacking on the -t flag, the files will be listed in order of their modification dates. Here is ls -lt for the same repository.

-rw-r--r--    1 jbranchaud  staff  40983 Aug 18 16:59
drwxr-xr-x  119 jbranchaud  staff   4046 Aug 17 11:38 vim
drwxr-xr-x    5 jbranchaud  staff    170 Aug 16 10:47 internet
drwxr-xr-x   23 jbranchaud  staff    782 Aug  1 10:17 javascript
drwxr-xr-x    7 jbranchaud  staff    238 Jul 22 14:04 webpack

See man ls for more details.

Check to see if a command exists at #shell

If you want to check to see if a command exists on a user’s machine you can use command -v. command without flags will run the command passed as an argument.

> command echo "A"

command -v with the -v flag will return the path of the command, and most importantly, a non-zero status code if that command does not exist.

> command -v brew

Or on Linux

> command -v brew
# returns nothing and also returns a status code of 1

Importantly, you can check for a command before using it.

command -v brew && brew install something

String concatentation in the Bourne Again Shell

String concatenation must be different in every lanaguage, or so it seems, and Bash is no different! You can string concat just by putting too strings next to each other.

> echo "a""b"

Its ok if one of these strings is a variable.

> a = "x" 
> echo $a"b"

Its ok if you put a double quoted string next to a single quoted string.

> echo 'a'"b"

Its ok if you put spaces in between the strings, but that space now will be part of the new string.

> echo "a" "b"
a b

But multiple spaces in between strings will be squashed to one string.

> echo "a"      "b"
a b

This works when setting a variable too, but only surrounded by parens.

> c=('a'      'b')
> echo $c
a b

Don’t worry about the parens though when they are right next to each other.

> c = "a""b"
> echo $c

Edit bash command at certain point in history

Jack introduced our team to the fc command a couple weeks ago. We had a command in our history that we wanted to edit, but it was not the last command. In fact, it was several 100 commands up in our history.

It turns out that the fc command takes an id argument, and each item in history has an id. To find the id of the particular command we wanted we ran:

> history | grep curl
10256 curl
10500 curl

To edit the first command listed in curl we just use:

fc 10256

H/T Ifu Aniemeka

Weekly jobs in crontab

The syntax for crontab entries is confusing at best. Fortunately, cron provides easy to read special strings for common cases.

If you want a job to run once a week your crontab would look like this:

@weekly /usr/bin/my_job

And thats it! @weekly in this case means 0 0 * * 0. Every Sunday at midnight.

You can check other crontab special strings with man 5 crontab.

H/T Dorian Karter

Override ssh command for git

Git supports a number of environment variables one of them being GIT_SSH. You can override the default ssh command (ssh of course) with that environment variable.

GIT_SSH=./my_ssh git pull origin master

In this case my_ssh is a bash executable file that looks like this:


ssh -i ~/.ssh/id_rsa $1 $2

An idea I got from this stack overflow answer.

Clearly the private key file I used is the default private key file so a bit redundant but its a way to add any options that might be necessary for the ssh connection, although specifying a host configuration in the ssh_config file might be simpler.

Specifying the private key to use in ssh

A private key is necessary in ssh to authenticate the connection securely. Generally ssh will look for a file in the ~/.ssh/ dir with a name of id_rsa (or if not using rsa a file beginning with id_ and the encryption algo name). That file should contain the private key.

You can, however, specify a different primary key at the command line with the -i flag like so:

ssh -i ~/.ssh/my_other_key.pem

Additionally, you may use the IdentityFile option the the ssh_config file to permanently configure the private key for a specific connection.

Select to clipboard at the ubuntu command line

Copying and pasting without a mouse - or programmatically - can be incredibly challenging. Ubuntu provides an apt-get installable program xclip which can provide X11 clipboard access at the command line.

> echo PASTEME | xclip -sel clip

The value PASTEME is now in the clipboard buffer.

The -sel or -selection indicates the buffer that will be used, primary, secondary or clipboard. Generally, clipboard is the buffer that we want.

To output the value of the buffer use the -o or -output flag:

> xclip -sel clip -o

Keeping track of your CPU Heat

If you’ve ever built your own computer, you’ve had anxiety about not having put everything together right and maybe even something critical, like the mechanisms that keep the computer and the CPU cool.

On Ubuntu you can intall the lm-sensors software which provides you with the facilities to get information the various heat levels inside the computer.

> sudo apt-get install lm-sensors
> sensors
Adapter: ISA adapter
cpu_fan:        0 RPM

Adapter: Virtual device
temp1:        +27.8°C  (crit = +119.0°C)
temp2:        +29.8°C  (crit = +119.0°C)

Adapter: ISA adapter
Package id 0:  +26.0°C  (high = +80.0°C, crit = +100.0°C)
Core 0:        +23.0°C  (high = +80.0°C, crit = +100.0°C)
Core 1:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 2:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 3:        +24.0°C  (high = +80.0°C, crit = +100.0°C)

Restart or shutdown ubuntu

Its easy to shutdown down Ubuntu from the commandline (given the correct permissions). Shutdown with shutdown which gives you 60 seconds to reverse that decision to shutdown by cancelling with shutdown -c. You can also reboot which takes effect right away.

shutdown # shutdown
shutdown -c # cancel that shutdown.
reboot # reboot the computer now

Ubuntu default desktop manager

What I like about linux is that there are configurations. You should be able to manipulate the whole system without a gui. The configuration for your default display manager, for instance, is in the default-display-manager file:

> cat /etc/X11/default-display-manager

There’s a startup script /etc/init.d/lightdm that checks that the value stored in this file points to a lightdm program and then executes that program.

Change To That New Directory

The $_ variable provided by bash is always set to the last argument of the previous command. One handy use of this is for changing directories into a newly created directory.

$ mkdir new_dir && cd $_

This command will leave you in your newly created directory, new_dir.

We can imagine using this bash variable in a number of similar scenarios as well. What if we are using some language specific command that creates a directory? Will it work when creating a new Phoenix or Rails project?

It sure will.

Give it a try with Phoenix:

mix my_app && cd $_

or with Rails:

rails new app && cd $_


Undo Some Command Line Editing

When using some of the fancy command line editing shortcuts, such as ctrl-u, you may end up erroneously changing or deleting part of the current command. Retyping it may be a pain or impossible if you’ve forgotten exactly what was changed. Fortunately, bash’s command line editing has undo built in. Just hit ctrl-_ a couple times to get back to where you want to be.

h/t Chris Erin


A Mnemonic for Changing Modes

Making a script file executable involves an incantation that has something to do with chmod but what exactly? chmod has two modes, absolute and symbolic. In absolute mode, you set the entire set of permissions with a numeric value like 755.

In symbolic mode, you can add or subtract permissions to either the user (u), group (g) or owner (o). To give read permissions to the group type:

> chmod g+r myfile.txt

And to give execution priviledges to the owner of a file:

> chmod o+x myscript.rb

Customizing the output of `ps`

There’s a lot to know about processes, although typically we’re looking for a mapping of pids to commands.

You can customize the output of ps with -o. The -o flag expects a comma separated list of keywords which indicate the columns that you’d like to display for each process.

> ps -o pid,command

The above command shows only the pid and the command w/args for each process. You can see all the possible keywords with ps -L.

Am I connected to a terminal via ssh?

When connecting to another computer via ssh the computer will have some environment variables that are not set otherwise. You can examine the environment variables with SSH in the name with:

> printenv | grep SSH

And in both linux and macos you’ll see an environment variable:


The first ip address and corresponding port represents the ip address if one exists and the second is the machine you are currently ssh’d into.

Case-Insensitive Search With Ack

Use the -i flag to perform a case-insensitive search with ack.

$ ack -i easter

3:The [`ack`]( utility has a fun Easter egg that dumps

18:Eastern time.

If you are a Vim user, you may be familiar with smart-case. The --smart-case option is a related Ack feature worth checking out.

See man ack for more details.

cat from stdin and historical notes

Generally, cat takes a file as an argument and prints the file to stdout. cat can also print to stdout from stdin like this:

> echo abc | cat -

Which is the equivalent to:

> echo abc | cat

But when you need to concatenate a line to the beginning of a file and then process that result it comes in handy:

> echo abc | cat - 123.txt
> echo abc | cat - 123.txt | grep abc

The macosx cat man page (here) has a history section and mentions that:

Dennis Ritchie designed and wrote the first man page. It appears to have been cat(1).

So cat was the first man page, interesting! (The ubuntu man page for cat does not mention this)

Redirect outputs with subshell

Today I learned how to use a Subshell to redirect multiple outputs as a single one. This way I can run a bunch of commands and unify all the results as it was a single command.

(git diff --name-only; git diff --staged --name-only) | sort -u

The above example will show me a sorted list of changed files (including the staged ones) in a git project without duplicates.

Here’s another example:

$ echo "world"; echo "hello" | sort -u
#=> world
#=> hello

$ (echo "world"; echo "hello") | sort -u
#=> hello
#=> world

Shhh.... Curl silently.

The curl program generally displays a progress bar as its making a request and receiving a response. This can be awkward when you are trying to wrap curl programmatically.

To shut off the progress bar use:

curl -sS

The first option (-s) hides the progress bar and the errors. The second option (-S) will expose an error on fail if used in conjunction with -s.

List Available File Types For Ack

The ack utility allows you to filter the searched files based on file type. If you’d like to know all of the file types available, you can use the --help=types flag. This will include file types you’ve specified in your .ackrc file.

Here is a sample of some of the output.

$ ack --help=types
    --[no]css          .css .less .scss
    --[no]dart         .dart
    --[no]delphi       .pas .int .dfm .nfm .dof .dpk .dproj .groupproj .bdsgroup .bdsproj
    --[no]elisp        .el
    --[no]elixir       .ex .exs
    --[no]erlang       .erl .hrl
    --[no]fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp
    --[no]go           .go
    --[no]groovy       .groovy .gtmpl .gpp .grunit .gradle
    --[no]haskell      .hs .lhs
    --[no]hh           .h
    --[no]html         .html .mustache .handlebars .tmpl
    --[no]jade         .jade
    --[no]java         .java .properties
    --[no]js           .js

See man ack for more details.

Extended file attributes on macOS

You can store metadata on any file in the mac filesystem(HFS+). If you want to ensure the file has a specific encoding, or if you want to place a checksum with the file you can use file attributes.

Setting and reading are easy on macOS with the xattr utility:

> touch something.txt
> xattr -w xyz 123 something.txt
> xattr -p xyz something.txt

In this example we wrote(xattr -w) an attribute onto the file and printed it(xattr -p) to see that it was written.

You can list the file attributes with a cryptic option for ls, ls -l@. Try doing this on your ~/Downloads dir to learn something cool about how macOS treats files coming from the internet.

Bits of Characters

If you are curious what the binary of an ascii/utf-8 char is you can use this string of commands at the command line:

echo 'A' | xxd -b

The A character is 65 in ascii which is 64 + 1. 01000000 is 64. 00000001 is of course 1, 01000001 is 65.

echo 'a' | xxd -b

The a character is 97 in ascii which is 64 + 32 + 1. 00100000 is 32 in binary, given this and the above, 01100001 is 97.

echo '🤓' | xxd -b

This ridiculous emoji is a utf-8 char. When you look at the binary for it:

11110000 10011111 10100100 10010011

You can see that every byte begins with a 1 which means that it will combine with any subsequent byte beginning with 1 to form a unique character.

Fuzzy awesome copy to sys clipboard w/yank & fzf

Say you want to copy a pid of a process to system clipboard, you could run ps ax, maybe grep on the result, connect your trusty mouse and try to select the value and hit ⌘ + c.

Or you can use the amazing fuzzy finder FZF (brew install fzf) in combination with Yank (brew install yank).

ps ax | fzf | yank

Now simply start typing the name of the process. When you press return you will get the columns broken down into a selectable prompt - choose one and press return. It is now in your system clipboard.

Here’s a demo:


This will work with any column spaced or even multiline response. Try running ps ax | yank.

First and last arguments to a command line command

If I start with the command:

> echo a b c
a b c

Sometimes I want the last argument to a command, which in this case is c but many times can be a long file path or a hard to spell word. In that case I would use !$:

> echo !$

This substitutes !$ with the last argument of the previous command.

To substitute the first argument instead, use !^ like so:

> echo !^

The ^ character and the $ character are ofter used as first and last. In vim those commands move the cursor to the beginning and the end of the line respectively. In regex those characters match the beginning or end of the line.

Search Files Specific To A Language

The ack command makes it easy to narrow the set of searched files to those of a specific programming language. For instance, if you have a rails project and only want to search the ruby files, use the --ruby flag with your ack command.

$ ack --ruby Active

With the --ruby flag, I get a manageable number of results. Without it, not so much.

$ ack --ruby Active | wc -l
$ ack Active | wc -l

See man ack for more details.

A Better File Wiper 📂

A tried-and-true debugging technique is to tail the test log:

$ tail -f log/test.log

When the output gets noisy, we might ‘wipe’ the log by removing it, so that it only contains the output from a single action:

$ rm log/test.log && rspec spec/features/user_logs_in_spec.rb

A better way to do this is to overwrite the file, like so:

$ echo '' > log/test.log

This wipes the file without replacing it, which preserves its serial number. It’s less destructive, and allows other processes interacting with the file to finish their work.

Here’s an example of this principle in action, using ls -i. Notice how the file’s serial number changes when we replace the file, but not when we ovewrite it.

$ ls -i *.txt
27232432 foobar.txt
$ rm foobar.txt
$ touch foobar.txt
$ ls -i *.txt
27232453 foobar.txt
$ echo '' > foobar.txt
$ ls -i *.txt
27232453 foobar.txt

h/t Brian Dunn

Capitalized Letter is the Default

Today I learned something fun about command line interfaces— when presented with a list of choices such as yes, no, or quit, the capitalized choice is the default in many Unix-based CLIs.

Here’s some aptitude:

% sudo apt-get install whatever
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  python-cssselect python-glade2 python-pyquery
0 upgraded, 3 newly installed, 0 to remove and 43 not upgraded.
Need to get 497 kB of archives.
After this operation, 2,207 kB of additional disk space will be used.
Do you want to continue? [Y/n]

The interface waits for your Y (‘yes’) or n (‘no’).

Because ‘yes’ is capitalized, it’s the default, simply pressing ENTER is the same as pressing Y, then ENTER. Enjoy your extra keystrokes!

Pipe | after | grep

The grep command may not work properly if you want to pipe another command after it, specially when you are tailing a file for example. Just call grep with --line-buffered

heroku logs -t | grep --line-buffered "heroku\[router\]" | awk '{print $11" "$5}'

This command outputs an easy-to-read performance log for each HTTP request:

service=266ms path="/api/posts"
service=142ms path="/api/users"

Shell ! command

!! is well known to repeat the last shell command in a *nix terminal, it turns out there are other useful variants:

Given a shell history:

$ history
10034  echo $PATH
10035  history
10036  type ruby
10037  which ruby
10038  history
10039  ls

!n - execute a specific item:

$ !10039

!?<match>?: matches a pattern and runs the first command that matches:

$ !?uby?
type ruby
ruby is /Users/jason/.rvm/rubies/ruby-2.2.2/bin/ruby

Verify downloaded files from the web #security

If you download a file from the web on a public WiFi and want to run on your machine you might want to check if the file has not been tampered with by a man-in-the-middle-attack or if the file host has been breached.

The easiest way to do this is to check the publised md5 or sha-1 hash for that file (you can do that via your phone if you want to be extra secure). Not every package publishes that but if they do it will be on their website usually next to the download link.

To verify the file you will need to hash the file you downloaded using openssl. For example:

 $ openssl sha1 Kali-Linux-2016.1-vm-amd64.7z
 SHA1(Kali-Linux-2016.1-vm-amd64.7z)= 2b49bf1e77c11ecb5618249ca69a46f23a6f5d2d

Which matches the site’s published sha-1 hash:


If you want to check md5, simply replace sha1 in the command with md5.

non recursive find

The find command is recursive by default. Let’s say you want to find all the screen shots on your ~/Desktop, but ignore any screens in the directories in Desktop. Setting the maxdepth flag will tell how far to recurse, using the value 1 only searches the directory you specified, using 2 would go into each directory, but only not search any sub directories, and so on.

#this will recurse by default
$ find ~/Desktop -name "*Screen*"

#this will only search the Desktop and it's immediate subdirectories
$ find ~/Desktop -maxdepth 2 -name "*Screen*"