Today I Learned

A Hashrocket project

11 posts by jackchristensen

Compute group proportions of a whole in SQL

It's easy with SQL to use aggregates to count members of a group. But what if you also want the proportion that group is of the whole. The answer is to use a window function combined with the aggregate. The window function is evaluated after the normal aggregation. i.e. it can reference the values of other post-aggregation rows.

select type, count(*), count(*)::numeric / sum(count(*)) over ()
from pets
group by type;
  type  | count |        ?column?        
--------+-------+------------------------
 dog    |    13 | 0.52000000000000000000
 rabbit |     3 | 0.12000000000000000000
 cat    |     9 | 0.36000000000000000000

Use git subtree to merge repositories with history

Use git subtree to merge repositories with history. For example, the following command merges the master branch of repository foo into the directory foo on the current repository.

git subtree add -P foo /path/to/foo master

See https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt for more details.

New PostgreSQL 9.6 slice syntax

PostgreSQL arrays support slice operations.

jack=# select ('{a,b,c,d,e,f}'::text[])[2:4];
  text
---------
 {b,c,d}

New in PostgreSQL 9.6 is slice syntax with implicit lower and upper bounds.

jack=# select ('{a,b,c,d,e,f}'::text[])[:4];
   text
-----------
 {a,b,c,d}
jack=# select ('{a,b,c,d,e,f}'::text[])[3:];
   text
-----------
 {c,d,e,f}

Previously, array_lower and array_upper was needed to slice from a particular index to the bounds of an array.

Create an infinite sequence in Ruby

seq = (0..Float::INFINITY).each is a one-liner that creates an infinite sequence.

irb(main):001:0> seq = (0..Float::INFINITY).each
=> #<Enumerator: 0..Infinity:each>
irb(main):002:0> seq.next
=> 0
irb(main):003:0> seq.next
=> 1
irb(main):004:0> seq.next
=> 2
irb(main):005:0> seq.next
=> 3

Edit the command in your editor

Sometimes when creating a long command at the command line it is easier to edit the command in vim. ctrl+x ctrl+e opens the current command line in your editor.

Save the "less" buffer

Sometimes I pipe a command to less and I want to save the results. To save the buffer type -o. Then type the file name and hit enter.

Quick Garbage Collector Stats from Go Programs

Set the GODEBUG=gctrace=1 in the environment to get log output from a Go program whenever the garbage collector runs.

Example:

gc 1 @10.282s 0%: 0.12+0.17+0.10 ms clock, 0.38+0/0.053/0.19+0.30 ms cpu, 4->4->0 MB, 5 MB goal, 4 P

The clock and cpu sections give us details on how long the GC cycle took. In this case we can see that it took well under 1ms.

See https://golang.org/pkg/runtime/ for details on what the output means.

Intel Speedstep and Ubuntu 14.04 Performance

Intel Speedstep works with the OS to adjust the clock speed of the CPU in real-time to save power. Older Linux kernels had a poor interaction with Speedstep that could cause the CPU to be downclocked even when running something demanding like a test suite. This can be fixed by disabling Speedstep in BIOS or upgrading the kernel. I was on kernel 3.13 and upgraded to 4.2. I saw 15-40% speed increase when running these tests.

Ubuntu 14.04.4 ships with a new kernel, but older installs of 14.04 will not be automatically upgraded. Run uname -a to see what kernel you are running. If it is not at least 4.2, then you may want to upgrade your kernel 1. Using aptitude this is simple as:

sudo aptitude install linux-generic-lts-wily

1 Upgrading the kernel in this way should be safe, but changing the kernel always has the potential to break something, so proceed at your own risk.

tar automatically detects file type

For years I have used tar xzf example.tar.gz to decompress tar.gz files and tar xjf example.tar.bz2 to decompress tar.bz2 files. It turns out specifying the compression type (the z or j) is unnecessary. tar can automatically determine the file type. Now I can use tar xf example.tar.gz.

Render 64-bit integers as strings in JSON

When rendering a server-side object containing 64-bit integers, precision can be lost if it is rendered as a number. For example, if a JSON document contains the number 9219976635399687875, JavaScript will parse it as 9219976635399688000. This is caused by JavaScript only having floating point numbers. To ensure very large numbers are not rounded, render the value out as a string, e.g. 9219976635399687875.

Cron does not like back ticks

cron may fail when trying to run a command with back ticks.

cp src backup-`date +%F`

Use dollar parenthesis instead.

cp src backup-$(date +%F)