Today I Learned

A Hashrocket project

263 posts about #command-line

How to convert JSON to CSV with jq

I had this json file that was an array of objects and some of those objects had different keys. I wanted to visualize that data in a spreadsheet (data science, AI, machine learning stuff) so I thought about having a CSV file where each JSON key would become a CSV column.

// file.json
[
  {
    "type": "Event1",
    "time": 20
  },
  {
    "type": "Event2",
    "distance": 100
  }
]

jq to the rescue:


cat file.json | jq -r '(map(keys) | add | unique) as $cols | map(. as $row | $cols | map($row[.])) as $rows | $cols, $rows[] | @csv' > file.csv

Here is the output:

"distance","time","type"
,20,"Event1"
100,,"Event2"
Screen Shot 2020-08-07 at 4 39 16 PM

Tmux Send Keys to Pane

I wrote a script the other day designed to help me download and edit files faster. In part of the script, I wanted to open Vim in an existing Tmux pane, and in the process I learned about the tmux send-keys command. Here’s how it works:

$ tmux send-keys -t 3 "vim" Enter

send-keys, aliased send, sends your string of commands to your pane (t) of choice. Running the above opens Vim in pane #3.

The -t flag accepts negative numbers, too, like an array index. In my version of the above command, I send the keys to pane -1, the last pane on the screen, which is where I keep Vim in my Tmux session.

zsh comes with help (set the $HELPDIR)

zsh helpfully comes installed with help files for all the builtins and a run-help command to help you access those help files. There is a trick though, before setting any environment variables here’s what happens:

$ run-help
There is no list of special help topics available at this time.

This is because the HELPDIR isn’t set. You have to find the install location for zsh’s help files and set the env var to that dir. On my system that looks like this:

export HELPDIR='/usr/share/zsh/help'

Then when you run run-help you should see a list of builtins for which there is help documentation. This is the same documentation that you can get via man builtins but much more readable and discoverable. run-help will call man as well if it can’t find you’re arg in the help files.

For me run-help is cumbersome to type so I alias it. Here’s what goes into my .zshrc:

export HELPDIR='/usr/share/zsh/help'
alias help=run-help

zsh is now much more helpful!

zsh comes with Tetris

zsh comes with it’s very own tetris game. No plugins needed!

You do need to autoload the tetriscurses function:

autoload -Uz tetriscurses

And then run tetriscurses.

While in the game, you can press H to learn which keys do what, and that looks like this:

left: h, j, left
right: right, n, l
rotate: up, c, i
soft drop: down, t, k
hard drop: space
quit: q
press space to return

also, maybe you want an alias for this?

autoload -Uz tetriscurses
alias tetris=tetriscurses

I’m putting the above straight into my .zshrc! Happy Sunday!

Follow the link in linux with `readlink -e`

Sometimes linux can be a maze of symbolic links. On my system, the java command exists at /usr/bin/java which is a link that points to /etc/alternatives/java which is a link that points to /usr/lib/jvm/java-8-oracle/jre/bin/java.

Instead of looking up each of the links of these files with ls -l, readlink -e will the links all the way through to the eventual file. In my case that would look like this:

$ readlink -e `which java`
# returns /usr/lib/jvm/java-8-oracle/jre/bin/java

You can learn more with man readlink.

On Mac, there is a readlink command, but there is no -e flag and it is not recursive.

Global Alias in zsh

What makes an alias global? Well, the -g flag of course. And what does this globality give you? Well, the ability to invoke an alias anywhere in the command line.

If I like the word ‘Potateos’ but I don’t ever have the energy to type the whole thing then I can create a global alias for that word:

> alias -g PO="Potatoes"
> echo PO
Potatoes

That’s convenient and cool. What is it actually for? Maybe redirecting errors to /dev/null:

> alias -g NO='2> /dev/null'
> echo foo >> /dev/stderr
foo
> (echo foo >> /dev/stderr) NO
# no output, it got swallowed!

Looks weird and maybe not useful, but maybe you can creatively find a useful way to use it:

I learned about this zsh functionality and other functionality here.

Shortcuts with hash -d in zsh

I stumbled across this zsh tricks post yesterday and am blown away by the hash command, which allows you to see and manipulate the hash table for either commands or for directory shortcuts.

hash by itself in zsh will output the location for all the commands.

hash -d shows you all of the named directories, and check this out you can navigate to one of those directories with ~shortcutname, like this:

$ hash -d | grep bin
bin=/bin
daemon=/usr/sbin
proxy=/bin
sync=/bin
$ cd ~daemon
$ pwd
/usr/sbin

You can create your own directory shortcuts like this:

$ hash -d mydir=/home/me/very/long/path

And then cd to it:

$ cd ~mydir
$ pwd
/home/me/very/long/path

Crazy! Read more in the zsh docs.

List Files by Updated

I’m currently working on an app that forwards logging around to various locations on the Linux server. It’s a bit tricky for me to figure out where any action I take in the browser is being logged. I need those logs!

A nice way to figure out where the logging is happening is to narrow it down to one directory (say, /var/log/) then ls that directory, ordering by most recently updated. The items at the top of the list have been recently updated, and thus probably contain valuable loggings!

$ ls -lt

I’m throwing on the -l flag for more detail. If there are lot of logs, filter it down with head:

$ ls -lt | head

Thanks for the idea, Kori!

The three amigos of the current directory

I always have trouble remembering how to get the name of the current directory. So strange pneumonics is the way to go.

The first amigo is a shell variable:

echo $PWD
# returns '/home/chris/tils'

There is also a pwd command that returns the same thing.

The second amigo is basename which gives you the current directory name without its path:

basename $PWD
# returns 'tils'

The third amigo is dirname which gives you the path without the current directory name:

dirname $PWD
# returns '/home/chris'

So now I can do things like

alias tnew=tmux new -s $(basename $PWD)

because I always, always, name my tmux session after the name of the current directory.

DNS Lookup with host

Today while doing some sleuthing, I learned about the host command. host “is a simple utility for performing DNS lookups.” It helped me connect a series of domains to their respective AWS EC2 servers, without a visit to the domain registrar.

Example:

$ host jakeworth.com
jakeworth.com has address 184.168.131.241
jakeworth.com mail is handled by 10 mailstore1.secureserver.net.
jakeworth.com mail is handled by 0 smtp.secureserver.net.

More info: man host

Get actual file size with du on Linux

You can use du, to report on the size of directories or files, but when my file is smaller than the block size I don’t see the output I expect.

With a small file, this should be the size of the number of characters.

$ echo 'Every Good Boy Deserves Fudge' > staff.txt
$ cat staff.txt | wc -c
30

But when I use du to examine file, I don’t see 30.

$ du -h staff.txt
4.0K    staff.txt

du measures in block sizes because in general if any part of a block is used, then for the purposes of the operating system the entire block is used.

You can tell du to care only about the size of the file with --apparent-size which is only apparent because between the beginning and end of the file the OS can’t tell which bytes are in use or are not in use.

$ du --apparent-size staff.txt
1       staff.txt

When reporting apparent size it rounds up to kilobytes, or --block-size=1k

To get the actual size of the file, you can use -b which is the same as --apparent-size --block-size=1

$ du -b staff.txt
30      staff.txt

Object construction with jq

jq is a powerful command-line tool to help you parse, analyze and script json output.

My current problem in jq is to turn this:

{
  "modules": [
  {
    "name": "x",
    "size": 10
  },
  {
    "name": "y",
    "size": 20
  }
  ]
}

into this:

{x: 10}
{y: 20}

This is possible using object construction:

jq '.modules[] | {(.name): .size}'

You pipe the result of the initial attribute as an array syntax .modules[] to an object {}. To use an attribute as a key you put parens around the attribute (.name) and declare that the value should be a different attribute .size.

Read more in the jq docs

Tmux Shortcuts I use

Create a new window

<tmux-leader>c

Think ‘c’ as in create. Rename window

<tmux-leader>,

That’s a comma. I don’t have anything clever for this one. Create a new pane

<tmux-leader>%

Maybe you’re clever enough to come up with something for this? I just memorized it. Kill a window ( like with a hung terminal )

<tmux-leader>&

You can also create a new session without calling new-session in full, you just use

tmux new -s <session-name>

Which is a bit quicker to type. Unless you like to type, you know, you do you.

Using zsh functions with xargs

I want to call a zsh function with xargs, but the arguments passed to xargs don’t run in your environment.

$ function hi() { echo "hello world $@" }
$ hi person!
hello world person!
$ seq 3 | xargs hi
xargs: hi: No such file or directory

No such file or directory!? hi is a function, but xargs doesn’t see it. With a combination of environment variables, function output and zsh command execution, we can use that function with xargs.

First let’s read the definition of our function into an environment variable.

$ FUNCS=$(functions hi); echo $FUNCS
hi () {
  echo "hello world $@"
}

Now we can use that in combination with zsh -c to execute the function with xargs.

$ FUNCS=$(functions hi); seq 3 | xargs -I{} zsh -c "eval $FUNCS; hi {}"
hello world 1
hello world 2
hello world 3

This solution is messy but workable.

Multiline matches with ripgrep (rg)

Ripgrep has become the default file search tool in my development environment. It’s fast! It can also do multiline searches if given the correct set of flags.

First, let me introduce you to the dataset:

$ echo 'apple\norange\nbanana\nkiwi'
apple
orange
banana
kiwi

So what if I want all the lines from orange to kiwi?

$ echo 'apple\norange\nbanana\nkiwi' | rg 'orange.*kiwi'

This finds nothing! Never fear, there is a --multiline flag.

$ echo 'apple\norange\nbanana\nkiwi' | rg --multiline 'orange.*kiwi'

This also finds nothing! The problem is that . does not match \n in regex. You can change that behaviour however by using the dot all modifier which looks like (?s).

$ echo 'apple\norange\nbanana\nkiwi' | rg --multiline '(?s)orange.*kiwi'
orange
banana
kiwi

We did it! Alternately, you can use the --multiline-dotall flag to allow . to match \n.

$ echo 'apple\norange\nbanana\nkiwi' | rg --multiline --multiline-dotall 'orange.*kiwi'
orange
banana
kiwi

I prefer short incantations however, and we can shorten it by using -U instead of --multiline.

$ echo 'apple\norange\nbanana\nkiwi' | rg -U '(?s)orange.*kiwi'
orange
banana
kiwi

Search in dotfiles with ripgrep

ripgrep is a very fast searching file system searching utility written in Rust. General usage is like this:

rg something directory-name

This is great, but when searching in my dotfiles for a configuration I don’t find what I’m looking for. Any file whose filename starts with . is considered hidden and these files are not searched by default with ripgrep.

To search hidden files (or dot files) use the --hidden flag.

rg --hidden "alias git" ~

Now, if there is a configuration that is overriding git in my dot files, I’ll find it!

What's my public IP? #automation #linux #terminal

Ever wonder what your public IP looks like? When I do, I usually search duckduckgo.com for “what’s my ip” and it is nice enough to tell me that at the top above the search results.

But what if I want to do that at the terminal, and perhaps use that information in a script?

TIL about ipinfo.io! curl it and you get your ip, hostname, city, state, zipcode, country, ISP, timezone, even lat & lng coordinates of the rough location!

curl ipinfo.io
{
  "ip": "111.111.111.111",
  "hostname": "redacted.comcast.com",
  "city": "Chicago",
  "region": "Illinois",
  "country": "US",
  "loc": "41.8500,-87.6500",
  "org": "AS7922 Comcast Cable Communications, LLC",
  "postal": "60666",
  "timezone": "America/Chicago",
  "readme": "https://ipinfo.io/missingauth"
}

This information is not fully accurate, but it’s good enough for scripts, and because it comes back in json form you can pipe it to jq to extract info or read it directly into your scripts.

No ifconfig on Linux?

If you are on a Linux machine and trying to run ifconfig you may get Command not found: ifconfig. That is because some Linux distros don’t come with it bundled by default.

On Manjaro Linux which is an Arch Linux based distro you can use pacman to install the package containing ifconfig called net-tools:

sudo pacman -S net-tools

On Debian based distros you can use:

sudo apt-get install net-tools

If you don’t want to install net-tools you can try using the ip command:

ip addr

This will return the interfaces on your machine and their IP bindings.

Seeing tmux messages easier

Using tmux is pretty slick, until something doesn’t work and you want to see the error message for longer that about 750ms. That’s the default setting for the messages to be displayed.

Set a new display time

For the current session

:set-option display-time <milliseconds>

So set-option display-time 4000 would set your display time to 4 seconds.

Globally

Add the -g flag.
Or add it to your tmux.conf to hold on to the configuration:

set-option -g display-time <milliseconds>

View all messages

If you want to view all the recent messages, then you can use

:show-messages

Default hot-key

<tmux-leader>~

To close out of the messages, just press <enter>.

Use pgrep and xargs to kill (processes) #zsh #bash

Have you ever found yourself doing this:

ps aux | grep [b]eam

And then copying the pids one by one so you can pass them to kill?

There’s a better way to return just the pids of the process you care about and not having to worry about ps finding your grep call (that’s why I’m surrounding the b in beam with square brackets).

pgrep -f beam

This will return just the pids, one in each line (which is perfect for use with xargs)

Example output:

11632
11456

Use with xargs to kill (-9 for extra brutality points 😈):

pgrep -f beam | kill -9

Testing Shell Conditions

When you’re shell scripting you really want to get your head wrapped around conditions. GNU provides a command to test conditions.

test 1 -gt 0
# exits with exit code 0
echo $?
# prints 0
test 0 -gt 1
# exits with exit code 1
echo $?
# prints 1

Checking the $? env var is a bit awkward, you can chain the command with echo though.

test 1 -gt 0 && echo true
# outputs true

Just be aware that it doesn’t output false when false.

But if you’re chaining with && you might as well use the [[ compound command.

[[ 1 -gt 0]] && echo true
# outputs true

Now you’re using shell syntax directly.

Linux ZSH ls colors

ls does not colorize the output in linux.

ls --colordoes colorize the output. It’s smart to set an alias.

alias ls='ls --color=auto'

Ok, now you’ve got colors everytime, but how do you change those colors?

The color settings are defaulted, but can be overriden by the value of environment variable LS_COLORS.

The language for setting these colors is really obtuse, but you can generate the settings with the command dircolors. dircolors outputs an enivornment variable you can include into your zshrc file. This variable will give you the same colors as when LS_COLORS is not set.

You can figure out what values to set colors to with this resource.

View the `motd` after login in Ubuntu

When you ssh into an Ubuntu machine, you may see a welcome message that starts with something like this:

Welcome to Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-65-generic x86_64)

This is the motd (message of the day).

What if you clear your terminal after login but want to see that message again?

There are two ways to do this.

$ cat /run/motd.dynamic

This will show you the same message that was created for you when you logged in.

If there is dynamic information in that message and you want to see the latest version run:

$ sudo run-parts /etc/update-motd.d/

This will run all the scripts that make the motd message.

Highlight json with the `bat`

Sometimes I run a utility that outputs a whole bunch of json, like:

docker inspect hello-world 
# outputs a couple pages of json

I want to send it through bat because bat is a great output viewer, and I also want it to syntax highlight. If bat is viewing a file with the extenson of json then it will get syntax highlighting, but in this case there is no file and no extension.

You can turn on json syntax highlighting with the --language flag.

docker inspect hello-world | bat --language json
# or just use -l
docker inspect hello-world | bat -l json

Combine this with --theme and you’re looking good!

docker inspect hello-world | bat -l json --theme TwoDark

Install all versions in .tool-versions with asdf

If you get the code for a new project and it is a project where versions are managed by asdf, then you will have a .tool-versions file and it will look something like this:

elixir 1.7.4-otp-21
erlang 21.3.8

If I don’t have those versions installed, then generally I install those individually.

If your working directory is the same version as the .tool-versions file then you can install all versions specified in that file with:

asdf install

Tmux New Window and Process ⏩

This is a command I am continually a huge fan of. Here’s one way to open a new Tmux window, from Tmux command mode:

:new-window <program> <arguments>

My practical example from today:

:new-window psql my_database

This opens a new Tmux window psql with the arguments supplied, connecting me to my_database. When I terminate database connection, the window closes. For web development, this is a great way to quickly connect to a program, run some commands, then close the connection and cleanup the Tmux session.

Keeping an SSH connection alive

Do you get disconnected from your SSH session often? I do… but I’ve found a solution that helps.

An SSH configuration that can be made on the server or client side but in my instance it makes more sense for the update to be on the client side.

In your ~/.ssh/config we’ll utilize the ServerAliveInterval declaration.

ServerAliveInterval
             Sets a timeout interval in seconds after which if no data has
             been received from the server, ssh(1) will send a message through
             the encrypted channel to request a response from the server.  The
             default is 0, indicating that these messages will not be sent to
             the server.  This option applies to protocol version 2 only.

This declaration informs the client to send a keep alive packet at a certain interval to the server, ensuring the connection stays open.

Host example
  Hostname example.com
  ServerAliveInterval 30

That declaration will send a packet every 30 seconds if the connection goes idle. Now this is the desired funcationality that I’d like to see for all of my SSH connections so we can add it to the host wildcard like so:

Host *
  ServerAliveInterval 120

Tmux Kill All Other Sessions

Ok, you can kill a session in Tmux from the command line with tmux kill-session -t <session-name>. Did you know you can add a flag to kill every session other than your target? Here’s that command.

$ tmux kill-session -a -t <session-name>
$ tmux kill-session -at <session-name> # short version

All sessions but the target will die. This is my ‘clear the desk’ operation when I come back to my computer after some time away and want to shut down a bunch of random servers and processes.

Make dry run

make has an great option to validate the commands you want to run without executing them. For that just use --dry-run.

#!make
PHONY: test
test:
    echo "running something"

Then:

$ make test --dry-run
echo "running something"
$ make test
echo "running something"
running something

Thanks @DillonHafer for that tip!

Parallel xargs fails if any of its children do

We like to write about xargs. In addition to all that, turns out xargs is a great tool for easily parallelizing tests, linters or anything where some may pass, and some may fail. If any of the processes that xargs spawns fail, the xargs call will also fail.

All child processes exit zero:


% echo "0\n0\n0" | xargs -Icode -P4 sh -c 'exit code'; echo exit code: $?
exit code: 0

And so does xargs! If any exit non-zero:

echo "0\n1\n127" | xargs -Icode  -P4 sh -c 'exit code'; echo exit code: $?
exit code: 1

xargs follows suit.

Find File Case-Insensitively 🔎

When using the find command, consider the -iname flag. It works like -name:

True if the last component of the pathname being examined matches pattern.

But with case insensitivity:

-iname pattern
  Like -name, but the match is case insensitive.

Here’s a sample command, which will match utils.js and Utils.js.

$ find . -iname Utils.js

See man find for more information. h/t Raelyn.

Ack ignores node_modules by default

When searching through your JavaScript project it doesn’t make sense to search through your node_modules. But if your are on a spelunking journey into the depths of your dependencies, you may want to search through all your node_modules!

ack ignores node_modules by default, and ack being ack you can ack through ack to check it out:

> cat `which ack` | ack node_modules
--ignore-directory=is:node_modules

This is different behaviour from ag and rg which also ignore node_modules but not explicitly. They both ignore node_modules by ignoring all entries in the .gitignore file.

rg claims to implement full support for the .gitignore file while also claiming other search tools do not. The open issues list for ag bears that out.

With each of these tools, explicitly stating the directory to search through overrides the ignore.

> ack autoprefix node_modules
> rg autoprefix node_modules
> ag autoprefix node_modules

Delete a Command from ZSH history 📚

A while ago, I restored this site’s production database to a backup captured 24 hours earlier (the reason isn’t important). Now in my terminal history, heroku pg:backups:restore a719 DATABASE_URL -rproduction is just hanging out, ready for me to accidentally smash, sending our production database hurtling back into the past. How do I remove this destructive command from my history?

In my terminal, ZSH, there’s a file called ~/.zsh_history, and a similar one for Bash. To remove the command, open that file and remove the entry from the list.

cat somefile
heroku pg:backups:restore a719 DATABASE_URL -rproduction # delete me!
ls

Open a new terminal window, and the bad command is gone.

To avoid this situation, consider the following setting (thanks Dillon!):

# ~/.zshrc
setopt HIST_IGNORE_SPACE

With this, any command preceded by a space is excluded from history. One could alias a destructive command to itself, preceded by a space, and save themselves a headache. Note a weird bug here: the last command appears in history, even if it should be excluded, until another command is entered. Banish it to the ether by running just an empty space as a command.

asdf Global Versions 🌏

Spend a bit of time with asdf, and you might see an error like this:

$ npm install
asdf: No version set for command npm
you might want to add one of the following in your .tool-versions file:

nodejs 10.15.3
nodejs 8.9.1

The project I’m trying to set up via npm install doesn’t specify a Node version in .tool-versions, and since I have multiple Nodes on my machine, asdf isn’t sure which to use.

I don’t want to edit .tool-versions in this project; I’d rather asdf had a global default.

Here’s how I made that happen:

$ asdf global node 10.15.3   

How to clear a Mac terminal and its scroll-back?

Just type this: clear && printf '\e[3J'

Or even better create an alias for that, here’s mine:

alias clear='clear && printf "\e[3J"';

Here’s what I’ve learned today:

On Mac a regular clear is pretty much the same as typing Control + L on iTerm2. This clears the screen what’s good but sometimes I want to clear all the scroll-back to clean the noise and find things faster.

In order to clean the scroll-back I was performing a Command + K. This cleans the screen and the scroll-back. That’s great, except that it messes up with tmux rendering and tmux holds all the scroll-back per pane, so it won’t work at all.

So my new alias solves that as after clear the screen it also sends a terminal command to reset the scroll back through printf '\e[3J' and this keeps tmux working just fine!

Print Calendar With Week Number 📅

I use the week number (1-52) in my notetaking. How do I know what week number it is? ncal.

$ ncal -w
    May 2019
Mo     6 13 20 27
Tu     7 14 21 28
We  1  8 15 22 29
Th  2  9 16 23 30
Fr  3 10 17 24 31
Sa  4 11 18 25
Su  5 12 19 26
   18 19 20 21 22

Though not shown in the output above, today’s date (the 10th) is highlighted in my terminal output. At the bottom of the column containing today’s date is the week number (19).

Don't truncate when redirecting

A common problem in shell languages is truncating a file that you’re trying to transform by redirecting into it.

Let’s say I have a file full of “a”‘s

PROMPT> cat test.txt
aaaaaa

And I want to switch them all to “b”‘s

PROMPT> cat test.txt | tr 'a' 'b'
bbbbbb

But when I try to redirect into the same file I truncate file:

PROMPT> cat test.txt | tr 'a' 'b' > test.txt
PROMPT> cat test.txt
# nothing

This is because the shell truncates “test.txt” for redirection before cat reads the file.

An alternate approach is to use tee.

PROMPT> cat test.txt | tr 'a' 'b' | tee test.txt
bbbbbb
PROMPT> cat test.txt
bbbbbb

tee will write the output to both the file and to standard out, but will do so at the end of the pipe, after the cat command reads from the file.

Delete lines from file with sed

Imagine the following file:

sed.test

hised
hellosed
goodbyesed

If you want to delete a line matching a regular expression (e.g. hellosed), you can use d at the end of your regular expression.

sed '/hellosed/d' sed.test

Output:

hised
goodbyesed

However the file did not change:

cat sed.test

hised
hellosed
goodbyesed

To write the file in place use the -i [suffix] option. This argument allows you to specify the suffix of the backup file to be saved before committing your changes. For example:

sed -i '.bak' '/hellosed/d' sed.test

Now the file will be modified with our changes but we will also get a backup of the original file in sed.test.bak.

If you like living on the edge 🛩, and don’t want those pesky backup files littering your system, you can supply -i with an empty suffix, causing no backup file to be saved.

sed -i '' '/hellosed/d' sed.test

killall 💀

Today while pairing I learned about the killall command.

This BSD program kills processes by name, unlike kill, which uses PID.

Here’s me killing my Mac calculator:

$ ps aux | grep Calc
grep Calc
/Applications/Calculator.app/Contents/MacOS/Calculator
$ killall Calculator
$ ps aux | grep Calc
grep Calc

Thanks for the tip, Mark!

Two arguments in a command with xargs and bash -c

You can use substitution -I{} to put the argument into the middle of the command.

> echo "a\nb\nc\nd" | xargs -I{} echo {}!
a!
b!
c!
d!

I can use -L2 to provide exactly 2 arguments to the command:

> echo "a\nb\nc\nd" | xargs -I{} -L2 echo {}!
a b!
c d!

But I want to use two arguments, the first in one place, the next in another place:

> echo "a\nb\nc\nd" | xargs -I{} -L2 echo {}x{}!
a bxa b!
c dxa b!

I wanted axb! but got a bxa b!. In order to achieve this you have to pass arguments to a bash command.

> echo "a\nb\nc\nd" | xargs -L2 bash -c 'echo $0x$1!'
axb!
cxd!

Just like calling

bash -c 'echo $0x$1!' a b

Where $0 represents the first argument and $1 represents the second argument.

Parsing CSV at the command line with `csvcut`

Parsing csv at the command line is easy with the csvcut tool from csvkit.

csvkit is installable with pip

> pip install csvcut 

You can print only the columns you are interested in.

> echo "a,b,c,d" | csvcut -c 1,4
a,d
> echo "a,b,c,d" | csvcut -c 1,5
Column 5 is invalid. The last column is 'd' at index 4.

It also handles quoted csv columns:

echo 'a,"1,2,3,4",c,d' | csvcut -c 2,3

It handles new lines:

> echo 'a,b\nc,d' | csvcut -c 2
b
d

There are a virtual plethora of options check out csvcut --help.

csvkit has a number of csv processing tools, check them out here.

Get pip installed executables into the asdf path

Given I am using asdf when I install a python executable via pip, then I expect to be able to run that executable at the command line.

> asdf global python 3.7.2
> pip install some-executable-package
> some-executable
zsh: command not found: some-executable

This is because a shim has not yet been created for this executable. If you look into your shims dir for the executable:

> ls $(dirname $(which pip)) | grep some-executable

It doesn’t exist.

To place a shim into the shims dir of asdf you must reshim.

> asdf reshim python

And now you have a shim that points to the executable:

> ls $(dirname $(which pip)) | grep some-executable
some-executable
> some-executable
Hello World!

Confirming operations with `xargs -p`

xargs is a great tool to take a lot of input and execute a lot of different commands based on that input. Sometimes though, if you are performing destructive or mutative actions with xargs you want to proceed more cautiosly.

> echo "banana apple orange" | tr ' ' '\n' | xargs -n1 echo "I like"

This outputs:

I like banana
I like apple
I like orange

But maybe I don’t like some of those things, please ask! Including the p flag with xargs forces a prompt.

> echo "banana apple orange" | tr ' ' '\n' | xargs -p -n1 echo "I like"
echo I like banana ?...n
echo I like apple ?...y
I like apple
echo I like orange ?...n

Yep, I only like apples.

Get first image of animated gif

Image Magick’s convert tool has a no-option, very simple way to access the first frame of an animated gif.

convert 'animated.gif[0]' animated.first.gif

The square brackets after the file name above can contain any index for any frame of the image. 0 is the index of the first image.

To discover how many frames an animated gif has you can use:

identify animated.gif

Which will return a line for every frame in the animated gif. Those lines will look like this:

animated.gif[32] GIF 736x521 756x594+4+70 8-bit sRGB 256c 421707B 0.000u 0:00.000