Today I Learned

A Hashrocket project

168 posts about #command-line

Use jq to filter objects list with regex

My use case is filtering an array of objects that have an attribute that matches a particular regex and extract another attribute from that object.

Here is my dataset:

[
  {name: 'Chris', id: 'aabbcc'},
  {name: 'Ryan', id: 'ddeeff'},
  {name: 'Ifu', id: 'aaddgg'}
]

First get each element of an array.

> data | jq '.[]'
{name: 'Chris', id: 'aabbcc'}
{name: 'Ryan', id: 'ddeeff'}
{name: 'Ifu', id: 'aaddgg'}

Pipe the elements to select and pass an expression that will evaluate to truthy for each object.

> data | jq '.[] | select(true) | select(.name)'
{name: 'Chris', id: 'aabbcc'}
{name: 'Ryan', id: 'ddeeff'}
{name: 'Ifu', id: 'aaddgg'}

Then use the test function to compare an attribute to a regex.

> data | jq '.[] | select(.id|test("a."))'
{name: 'Chris', id: 'aabbcc'}
{name: 'Ifu', id: 'aaddgg'}

Extract the name attribute for a more concise list.

> data | jq '.[] | select(.id|test("a.")) | .name'
Chris
Ifu

Read more about jq

Setting timezones from command line on MacOS

We wrote some tests that failed recently in a timezone other than our own (☹️). In order to reproduce this failure we had to set the timezone of our local machine to the failure timezone.

sudo systemsetup -settimezone America/New_York

Easy enough. You can get all the timezones available to you with:

sudo systemsetup -listtimezones

And also check your current timezone

sudo systemsetup -gettimezone

systemsetup has 47 different time/system oriented commands. But it needs a sudo even to run -help.

> systemsetup -help
You need administrator access to run this tool... exiting!

Don’t fret you can get this important secret information via the man page

man systemsetup

zsh change dir after hook w/`chpwd_functions`

How does rvm work? You change to a directory and rvm has determined which version of ruby you’re using.

In bash, the cd command is overriden with a cd function. This isn’t necessary in zsh because zsh has hooks that execute after you change a directory.

If you have rvm installed in zsh you’ll see this:

> echo $chpwd_functions
__rvm_cd_functions_set
> echo ${(t)chpwd_functions}
array

chpwd_functions is an array of functions that will be executed after you run cd.

Within the function you can use the $OLDPWD and $PWD env vars to programatically determine where you are like this:

> huh() { echo 'one sec...' echo $OLDPWD && echo $PWD } 
> chpwd_functions+=huh
> cd newdir
one sec...
olddir
newdir

You can read more about hook functions in the zsh docs.

Remove duplicates in zsh $PATH

How to do it?

typeset -aU path

There ya go. All entries in $PATH are now unique

How does it work? Well that’s stickier. path and PATH are not the same thing. You can determine that by examining the type of each with t variable expansion.

> echo ${(t)path}
array-unique-special
> echo ${(t)PATH}
scalar-export-special

They are linked variables however, if you change one you change the other but they have different properties. I have to admit at this point that scalar-export-special is something of which I’m ignorant.

The typeset declaration is simpler, it changes the type of path from array-special to array-unique-special. The -a flag is for array and the -U flag is for unique.

What files are being sourced by zsh at startup?

Tracking down a bug in the jagged mountains of shell files that are my zsh configuration can be tough. Enter the SOURCE_TRACE option, settable with the -o flag when opening a new shell with the zsh command.

zsh -o SOURCE_TRACE

Which outputs:

+/etc/zsh/zshenv:1> <sourcetrace>
+/etc/zsh/zshrc:1> <sourcetrace>
+/home/chris/.zshrc:1> <sourcetrace>
+/home/chris/.sharedrc:1> <sourcetrace>
+/home/chris/.zshrc.local:1> <sourcetrace>

The above output is an abbreviated representation of the actual files loaded on my system. I have language manager asdf installed which adds a couple of entries. I am an rvm user which adds 50 (!!!) entries. Lots of shell code to source for rvm. Additionally, I use Hashrocket’s dotmatrix which adds even more entries. Lots of sourcing to sort through.

This is handy in combination with print line (or echo) debugging. It gives your print lines added context with things get noisy.

Posting data from a file with curl

Iterating on an api post request with curl can be frustrating if that involves a lot of command line editing. curl however can read a file for post body contents. Generally the --data option is used like this:

curl -XPOST --data '{"data": 123}' api.example.com/data

But when using an @ symbol you can reference a file

curl -XPOST --data @data.json api.example.com/data

Now, you can run the same command for each iteration and edit the data.json file containing the data to be posted with your favorite text editor (which is vim right?).

whatis command and apropos in #linux #bash

Ever wonder what a command you are using is? Turns out linux has the answers for you!

Simply type whatis followed by the name of the command.

Examples:

$ whatis bc
bc(1)         - An arbitrary precision calculator language
$ whatis brew
brew(1)       - The missing package manager for macOS
brew-cask(1)  - a friendly binary installer for macOS

whatis uses the man pages to search your entered query.

There is also a reverse search, which searches the descriptions of commands. For example say you are looking for a calculator:

$ apropos calculator
bc(1)         - An arbitrary precision calculator language
dc(1)         - an arbitrary precision calculator

h/t this tweet by Peter Cooper

Stop #bash script on error #linux #zsh

If you are writing a procedural bash script, you may want to stop execution if one of the steps errored out.

You can write error handling for each step, but that can get quite verbose and make your script hard to read, or you might even miss something.

Fortunately bash provides another option:

set -e

Simply place the above code at the top of your script and bash should halt the script in case any of them returns a non-true exit code.

Caveats: this will not work in all cases, for example it does not work for short circuited commands using &&/||.

If you want it to work when one of your operations in a pipe fails you will need to add the pipefail flag (not supported on some systems set -o | grep pipefail to check your system):

set -e -o pipefail

If you have a script that always returns a non true return code and that’s fine you can override set -e for that command with:

set +e
your_command_goes_here
set -e

At this point I consider it a best practice to include this statement in every script I write.

List filenames of multiple filetypes in project

Ag (aka The Silver Searcher) is an amazing piece of software. It allows you to define file types (like Ack) and comes prepackeged with some file types.

Using this feature you can list all files of a specific type in your project. For example say we want to list all Ruby and JavaScript files:

ag --ruby --js -l

Ag has the added benefit over Ack, that it ignores gitignored files, so you only get the files that matter (and not stuff from node_modules etc).

To see what filetypes Ag supports:

ag --list-file-types

The list is pretty extensive! Unlike Ack however, there is currently no way to add new file types or extend the list.

Reloading shell history in zsh

When you start a shell your history list is populated from your .zsh_history file. Depending on options that you have set, when you close a shell you write your history list to that same history file. Until you close your or open new shells that history is self contained and not accessible from another shell.

There is a built-in zsh command to both write and read history from the .zsh_history file. fc -W will write to the history file. fc -R will read from the history file.

Clean Up Autojump

autojump is a command-line tool for faster directory changing. It’s a must-have on my workstation, because it lets me jump to deeply nested directories like this:

:~/$ j cli
/Users/jwworth/projects/2017/twitter-killer/client
:~/jwworth/projects/2017/twitter-killer/client$

There are two ways we can clean up autojump once it’s been running for a while. First, purge non-existant directories from the jump database:

$ j --purge
Purged 8 entries.

Second, edit the file ~/Library/autojump/autojump.txt (OSX), which stores the jump database. Here you can remove directories that should never be jumped to, or are weighted too highly because of frequent use in the past.

Happy jumping!

Call a program one time for each argument w/ xargs

Generally, I’ve used xargs in combination with programs like kill or echo both of which accept a variable number of arguments. Some programs only accept one argument.

For lack of a better example, lets try adding 1 to 10 numbers. In shell environments you can add with the expr command.

> expr 1 + 1
2

I can combine this with seq and pass the piped values from seq to expr with xargs.

> seq 10 | xargs expr 1 + 
expr: syntax error

In the above, instead of adding 1 to 1 and then 1 to 2, it trys to run:

expr 1 + 1 2 3 4 5 6 7 8 9 0

Syntax Error!

We can use the -n flag to ensure that only one argument is applied at time and the command runs 10 times.

> seq 10 | xargs -n1 expr 1 +
2
3
4
5
6
7
8
9
10
11

For more insight into what’s being called, use the -t flag to see the commands.

Use `source /dev/stdin` to execute commands

Let’s say there’s a command in a file, like a README file, and you don’t have any copy or paste tools handy. You can get the command out of the README file with:

> cat README.md | grep "^sed"
sed -ie "s/\(.*\)/Plug '\1'/" .vimbundle.local

Great! Now how do we run it? The source is generally used to read and execute commands in files, and really /dev/stdin behaves like a file.

You can use the pipe operator to place the command into stdin and then source will read from stdin.

> cat README.md | grep "^sed" | source /dev/stdin

A simpler example can be constructed with echoing

> echo "echo 'hi there'"
echo 'hi there'

And

> echo "echo 'hi there'" | source /dev/stdin
hi there

Split large file into multiple smaller files

Bash has a handy tool to split files into multiple pieces. This can be useful as a precursor to some parallelized processing of a large file. Lets say you have gigabytes of log files you need to search through, splitting the files into smaller chunks is one way to approach the problem.

> seq 10 > large_file.txt
> split -l2 large_file.txt smaller_file_
> ls -1
smaller_file_aa
smaller_file_ab
smaller_file_ac
smaller_file_ad
smaller_file_ae
large_file.txt

First, I created a “large” file with ten lines. Then, I split that file into files with the prefix smaller_file_. The -l2 option for split tells split to make every 2 lines a new file. For 10 lines, we’ll get 5 files. The suffix it adds (“aa”, “ab”, …) sorts lexigraphically so that we can reconstruct the file with cat and globs.

> cat smaller_file*
1
2
3
4
5
6
7
8
9
10

More useful Homebrew searches #macOS #homebrew

Homebrew, the third-party package manager on macOS, allows searching for packages by name, but the list that comes out only contains package names. That’s not always very useful, particulary when you are not sure what you are looking for.

To get the package description along with the package name simply add --desc to your brew search command.

For example, let’s look for a library for performing file diffs with color highlighting:

$ brew search --desc diff
apgdiff: Another PostgreSQL diff tool
cdiff: View colored diff with side by side and auto pager support
cern-ndiff: Numerical diff tool
colordiff: Color-highlighted diff(1) output
cppad: Differentiation of C++ Algorithms
dhex: Ncurses based advanced hex editor featuring diff mode and more
diff-so-fancy: Good-lookin' diffs with diff-highlight and more
...

You can also search using regex in both the description and name of the package as long as you supply the --desc option:

$ brew search --desc /[cC]olor.*[dD]iff/
cdiff: View colored diff with side by side and auto pager support
colordiff: Color-highlighted diff(1) output
icdiff: Improved colored diff

`cd` in subshell

With many of our projects sequestering the front end javascript code into an assets directory I find myself moving between the root project directory and the assets directory to perform all the npm or yarn related tasks in that assets dir. Inevitably I’ll start doing something like this:

cd assets; npm install; cd ..

or this

pushd assets; npm install; popd

In both cases using ; instead of && puts me back in the original directory regardless of the result of the npm command.

I just learned that using cd in a subshell does not change the directory of the current shell, so I can also do this:

(cd assets; npm install)

Xargs from a file

I’ve struggled with xargs conceptually for long time, but actually its pretty easy conceptually. For commands that don’t read from stdin but do take arguments, like echo or kill, you can turn newline separated values from stdin in into arguments.

Piping to echo does not work.

> echo 123 | echo
# nothing

Using xargs it does.

> echo 123 | xargs echo
123

xargs can also read a file with the -a flag, turning each line of the file into an argument.

> echo "123\nabc" > test.txt
> cat test.txt
123
abc
> xargs -a test.txt echo
123 abc

H/T Brian Dunn

Kill rogue shell processes

There is a particular type of attack where an inserted usb stick can act like a keyboard, open a terminal, and start something like this:

while (true); do something_malicious; sleep 3600; done & disown

This process endlessly loops and wakes every hour to do something malicious. The & puts it in the background and the disown will end its attachment to the current terminal. When the terminal is closed the process will get a parent of 1.

This process is still detectable and killable at the command line by finding all shell programs with a parent pid of 1 and killing them with -9.

ps ax -o pid,command,ppid | grep '.*zsh.*\s1$' | awk '{print $1}' | xargs kill -9

This will kill all running rogue zsh processes. There may be reasons why you’d want a process to be detached from its parent terminal, but you could easily decide that this isn’t something you want ever and place the above command into a cron job that runs every 2 seconds.

Download all of humble bundle books in parallel

Humble Bundle is a great site which offers technical book bundles. The problem is that they present the user with a huge list of links for all the different formats and it is a tedious task to right click each link and save it to your hard drive.

In order to solve this you can open the Developer Tools while on the download page and paste the following:

var pattern = /(MOBI|EPUB|PDF( ?\(H.\))?|CBZ|Download)$/i;
var nodes = document.getElementsByTagName('a');
var downloadCmd = '';
for (i in nodes) {
    var a = nodes[i];
    if (a && a.text && pattern.test(a.text.trim()) && a.attributes['data-web']) {
        downloadCmd += 'wget --content-disposition "' + a.attributes['data-web'].value + "\"\n";
    }
}
var output = document.createElement("pre");
output.textContent = downloadCmd;
document.getElementById("papers-content").prepend(output);

credit: https://gist.github.com/graymouser/a33fbb75f94f08af7e36

This will add a pre tag to the page with a bunch of wget commands. Go ahead and copy those to your clipboard.

Using GNU Parallel (brew install parallel). First save the contents of your clipboard into a file, for example download_jobs then run the following command:

parallel -j 4 < download_jobs

Replace 4 with the number of cores you have on your machine.

Then sit back and watch your directory get populated with files.

Surround every line in a file using sed

To replace every line in a file you can use linux’s built in sed utility:

For example given a file like this:

dkarter/backpack
junegunn/fzf
junegunn/fzf.vim
junegunn/vim-peekaboo
junegunn/gv.vim
terryma/vim-multiple-cursors
scrooloose/nerdtree
dyng/ctrlsf.vim
haya14busa/incsearch.vim
killphi/vim-legend
neomake/neomake

If we want to surround each line with Plug '$content_of_line' we can run the following command:

sed -e "s/\(.*\)/Plug '\1'/" .vimbundle.local

Output:

Plug 'dkarter/backpack'
Plug 'junegunn/fzf'
Plug 'junegunn/fzf.vim'
Plug 'junegunn/vim-peekaboo'
Plug 'junegunn/gv.vim'
Plug 'terryma/vim-multiple-cursors'
Plug 'scrooloose/nerdtree'
Plug 'dyng/ctrlsf.vim'
Plug 'haya14busa/incsearch.vim'
Plug 'killphi/vim-legend'
Plug 'neomake/neomake'

If the result is what we expected we can add -i flag to write the file in place, updating it with our changes:

sed -ie "s/\(.*\)/Plug '\1'/" .vimbundle.local

Keep Your Brews Bubbly

Life is too short to have Homebrew problems. Run these commands to keep your brews bubbly.

Checks your system to make sure that future installs go smoothly:

brew doctor

Upgrades Homebrew to the latest version:

brew update

Gets a list of what packages are outdated:

brew outdated

Looks through your installed packages and deletes any old versions that may still be hanging around:

brew cleanup

You could alternatively add the --dry-run flag to cleanup to see all the outdated packages that would be removed.

Deletes old symlinks:

brew prune

Updates packages to the latest version:

brew upgrade

You can add the --cleanup flag to delete older versions of the packages you are updating.

List Files Ordered By Modification Date

The ls command lists the files in a directory. Tacking on the -l flag will list it in long format. By default, everything is listed in lexicographical order. This is what ls -l looks like for this repository.

-rw-r--r--    1 jbranchaud  staff    628 Feb 14  2016 CONTRIBUTING.md
-rw-r--r--    1 jbranchaud  staff   1058 Feb 19  2015 LICENSE
-rw-r--r--    1 jbranchaud  staff  40983 Aug 18 16:59 README.md
drwxr-xr-x    5 jbranchaud  staff    170 Apr  1 14:45 ack
drwxr-xr-x    5 jbranchaud  staff    170 Feb 24 16:31 chrome
...

Sometimes you want a sense of what has been modified and when. Lexicographical order isn’t going to help much here. By tacking on the -t flag, the files will be listed in order of their modification dates. Here is ls -lt for the same repository.

-rw-r--r--    1 jbranchaud  staff  40983 Aug 18 16:59 README.md
drwxr-xr-x  119 jbranchaud  staff   4046 Aug 17 11:38 vim
drwxr-xr-x    5 jbranchaud  staff    170 Aug 16 10:47 internet
drwxr-xr-x   23 jbranchaud  staff    782 Aug  1 10:17 javascript
drwxr-xr-x    7 jbranchaud  staff    238 Jul 22 14:04 webpack
...

See man ls for more details.

Check to see if a command exists at #shell

If you want to check to see if a command exists on a user’s machine you can use command -v. command without flags will run the command passed as an argument.

> command echo "A"
"A"

command -v with the -v flag will return the path of the command, and most importantly, a non-zero status code if that command does not exist.

> command -v brew
/usr/local/bin/brew

Or on Linux

> command -v brew
# returns nothing and also returns a status code of 1

Importantly, you can check for a command before using it.

command -v brew && brew install something

String concatentation in the Bourne Again Shell

String concatenation must be different in every lanaguage, or so it seems, and Bash is no different! You can string concat just by putting too strings next to each other.

> echo "a""b"
ab 

Its ok if one of these strings is a variable.

> a = "x" 
> echo $a"b"
xb

Its ok if you put a double quoted string next to a single quoted string.

> echo 'a'"b"
ab

Its ok if you put spaces in between the strings, but that space now will be part of the new string.

> echo "a" "b"
a b

But multiple spaces in between strings will be squashed to one string.

> echo "a"      "b"
a b

This works when setting a variable too, but only surrounded by parens.

> c=('a'      'b')
> echo $c
a b

Don’t worry about the parens though when they are right next to each other.

> c = "a""b"
> echo $c
ab

Edit bash command at certain point in history

Jack introduced our team to the fc command a couple weeks ago. We had a command in our history that we wanted to edit, but it was not the last command. In fact, it was several 100 commands up in our history.

It turns out that the fc command takes an id argument, and each item in history has an id. To find the id of the particular command we wanted we ran:

> history | grep curl
10256 curl www.google.com
10500 curl www.amazon.com

To edit the first command listed in curl we just use:

fc 10256

H/T Ifu Aniemeka

Weekly jobs in crontab

The syntax for crontab entries is confusing at best. Fortunately, cron provides easy to read special strings for common cases.

If you want a job to run once a week your crontab would look like this:

@weekly /usr/bin/my_job

And thats it! @weekly in this case means 0 0 * * 0. Every Sunday at midnight.

You can check other crontab special strings with man 5 crontab.

H/T Dorian Karter

Override ssh command for git

Git supports a number of environment variables one of them being GIT_SSH. You can override the default ssh command (ssh of course) with that environment variable.

GIT_SSH=./my_ssh git pull origin master

In this case my_ssh is a bash executable file that looks like this:

#!/bin/bash

ssh -i ~/.ssh/id_rsa $1 $2

An idea I got from this stack overflow answer.

Clearly the private key file I used is the default private key file so a bit redundant but its a way to add any options that might be necessary for the ssh connection, although specifying a host configuration in the ssh_config file might be simpler.

Specifying the private key to use in ssh

A private key is necessary in ssh to authenticate the connection securely. Generally ssh will look for a file in the ~/.ssh/ dir with a name of id_rsa (or if not using rsa a file beginning with id_ and the encryption algo name). That file should contain the private key.

You can, however, specify a different primary key at the command line with the -i flag like so:

ssh -i ~/.ssh/my_other_key.pem

Additionally, you may use the IdentityFile option the the ssh_config file to permanently configure the private key for a specific connection.

Select to clipboard at the ubuntu command line

Copying and pasting without a mouse - or programmatically - can be incredibly challenging. Ubuntu provides an apt-get installable program xclip which can provide X11 clipboard access at the command line.

> echo PASTEME | xclip -sel clip

The value PASTEME is now in the clipboard buffer.

The -sel or -selection indicates the buffer that will be used, primary, secondary or clipboard. Generally, clipboard is the buffer that we want.

To output the value of the buffer use the -o or -output flag:

> xclip -sel clip -o
PASTEME

Keeping track of your CPU Heat

If you’ve ever built your own computer, you’ve had anxiety about not having put everything together right and maybe even something critical, like the mechanisms that keep the computer and the CPU cool.

On Ubuntu you can intall the lm-sensors software which provides you with the facilities to get information the various heat levels inside the computer.

> sudo apt-get install lm-sensors
> sensors
asus-isa-0000
Adapter: ISA adapter
cpu_fan:        0 RPM

acpitz-virtual-0
Adapter: Virtual device
temp1:        +27.8°C  (crit = +119.0°C)
temp2:        +29.8°C  (crit = +119.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +26.0°C  (high = +80.0°C, crit = +100.0°C)
Core 0:        +23.0°C  (high = +80.0°C, crit = +100.0°C)
Core 1:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 2:        +24.0°C  (high = +80.0°C, crit = +100.0°C)
Core 3:        +24.0°C  (high = +80.0°C, crit = +100.0°C)

Restart or shutdown ubuntu

Its easy to shutdown down Ubuntu from the commandline (given the correct permissions). Shutdown with shutdown which gives you 60 seconds to reverse that decision to shutdown by cancelling with shutdown -c. You can also reboot which takes effect right away.

shutdown # shutdown
shutdown -c # cancel that shutdown.
reboot # reboot the computer now

Ubuntu default desktop manager

What I like about linux is that there are configurations. You should be able to manipulate the whole system without a gui. The configuration for your default display manager, for instance, is in the default-display-manager file:

> cat /etc/X11/default-display-manager
/usr/sbin/lightdm

There’s a startup script /etc/init.d/lightdm that checks that the value stored in this file points to a lightdm program and then executes that program.

Change To That New Directory

The $_ variable provided by bash is always set to the last argument of the previous command. One handy use of this is for changing directories into a newly created directory.

$ mkdir new_dir && cd $_

This command will leave you in your newly created directory, new_dir.

We can imagine using this bash variable in a number of similar scenarios as well. What if we are using some language specific command that creates a directory? Will it work when creating a new Phoenix or Rails project?

It sure will.

Give it a try with Phoenix:

mix phx.new my_app && cd $_

or with Rails:

rails new app && cd $_

source

Undo Some Command Line Editing

When using some of the fancy command line editing shortcuts, such as ctrl-u, you may end up erroneously changing or deleting part of the current command. Retyping it may be a pain or impossible if you’ve forgotten exactly what was changed. Fortunately, bash’s command line editing has undo built in. Just hit ctrl-_ a couple times to get back to where you want to be.

h/t Chris Erin

source

A Mnemonic for Changing Modes

Making a script file executable involves an incantation that has something to do with chmod but what exactly? chmod has two modes, absolute and symbolic. In absolute mode, you set the entire set of permissions with a numeric value like 755.

In symbolic mode, you can add or subtract permissions to either the user (u), group (g) or owner (o). To give read permissions to the group type:

> chmod g+r myfile.txt

And to give execution priviledges to the owner of a file:

> chmod o+x myscript.rb

Customizing the output of `ps`

There’s a lot to know about processes, although typically we’re looking for a mapping of pids to commands.

You can customize the output of ps with -o. The -o flag expects a comma separated list of keywords which indicate the columns that you’d like to display for each process.

> ps -o pid,command

The above command shows only the pid and the command w/args for each process. You can see all the possible keywords with ps -L.

Am I connected to a terminal via ssh?

When connecting to another computer via ssh the computer will have some environment variables that are not set otherwise. You can examine the environment variables with SSH in the name with:

> printenv | grep SSH

And in both linux and macos you’ll see an environment variable:

SSH_CONNECTION=192.168.50.3 65102 192.168.50.12 22

The first ip address and corresponding port represents the ip address if one exists and the second is the machine you are currently ssh’d into.

Case-Insensitive Search With Ack

Use the -i flag to perform a case-insensitive search with ack.

$ ack -i easter

ack/ack-bar.md
3:The [`ack`](https://beyondgrep.com/) utility has a fun Easter egg that dumps

postgres/configure-the-timezone.md
18:Eastern time.

If you are a Vim user, you may be familiar with smart-case. The --smart-case option is a related Ack feature worth checking out.

See man ack for more details.

cat from stdin and historical notes

Generally, cat takes a file as an argument and prints the file to stdout. cat can also print to stdout from stdin like this:

> echo abc | cat -
abc

Which is the equivalent to:

> echo abc | cat
abc

But when you need to concatenate a line to the beginning of a file and then process that result it comes in handy:

> echo abc | cat - 123.txt
abc
123
> echo abc | cat - 123.txt | grep abc
abc

The macosx cat man page (here) has a history section and mentions that:

Dennis Ritchie designed and wrote the first man page. It appears to have been cat(1).

So cat was the first man page, interesting! (The ubuntu man page for cat does not mention this)

Redirect outputs with subshell

Today I learned how to use a Subshell to redirect multiple outputs as a single one. This way I can run a bunch of commands and unify all the results as it was a single command.

(git diff --name-only; git diff --staged --name-only) | sort -u

The above example will show me a sorted list of changed files (including the staged ones) in a git project without duplicates.

Here’s another example:

$ echo "world"; echo "hello" | sort -u
#=> world
#=> hello

$ (echo "world"; echo "hello") | sort -u
#=> hello
#=> world

Shhh.... Curl silently.

The curl program generally displays a progress bar as its making a request and receiving a response. This can be awkward when you are trying to wrap curl programmatically.

To shut off the progress bar use:

curl -sS http://www.google.com

The first option (-s) hides the progress bar and the errors. The second option (-S) will expose an error on fail if used in conjunction with -s.

List Available File Types For Ack

The ack utility allows you to filter the searched files based on file type. If you’d like to know all of the file types available, you can use the --help=types flag. This will include file types you’ve specified in your .ackrc file.

Here is a sample of some of the output.

$ ack --help=types
    ...
    --[no]css          .css .less .scss
    --[no]dart         .dart
    --[no]delphi       .pas .int .dfm .nfm .dof .dpk .dproj .groupproj .bdsgroup .bdsproj
    --[no]elisp        .el
    --[no]elixir       .ex .exs
    --[no]erlang       .erl .hrl
    --[no]fortran      .f .f77 .f90 .f95 .f03 .for .ftn .fpp
    --[no]go           .go
    --[no]groovy       .groovy .gtmpl .gpp .grunit .gradle
    --[no]haskell      .hs .lhs
    --[no]hh           .h
    --[no]html         .html .mustache .handlebars .tmpl
    --[no]jade         .jade
    --[no]java         .java .properties
    --[no]js           .js
    ...

See man ack for more details.