Today I Learned

A Hashrocket project

Open iOS simulator app data directory

If you need to quickly open an app’s document directory on the iOS Simulator, you can quickly get the path from the xcrun CLI.

xcrun simctl get_app_container booted ${bundle_id} data

Here’s some ruby you can throw into a Rakefile of your iOS app 😉

# Rakefile
desc "Open the simulator document directory"
task :docs do
  bundle_id = File.read("MyAppName/MyAppName.xcodeproj/project.pbxproj")
    .scan(/PRODUCT_BUNDLE_IDENTIFIER = "(.*)"/)
    .flatten.first

  app_directory = `xcrun simctl get_app_container booted #{bundle_id} data`

  puts "Opening simulator directory: #{app_directory}"
  `open #{app_directory}`
end

Delete a Command from ZSH history 📚

A while ago, I restored this site’s production database to a backup captured 24 hours earlier (the reason isn’t important). Now in my terminal history, heroku pg:backups:restore a719 DATABASE_URL -rproduction is just hanging out, ready for me to accidentally smash, sending our production database hurtling back into the past. How do I remove this destructive command from my history?

In my terminal, ZSH, there’s a file called ~/.zsh_history, and a similar one for Bash. To remove the command, open that file and remove the entry from the list.

cat somefile
heroku pg:backups:restore a719 DATABASE_URL -rproduction # delete me!
ls

Open a new terminal window, and the bad command is gone.

To avoid this situation, consider the following setting (thanks Dillon!):

# ~/.zshrc
setopt HIST_IGNORE_SPACE

With this, any command preceded by a space is excluded from history. One could alias a destructive command to itself, preceded by a space, and save themselves a headache. Note a weird bug here: the last command appears in history, even if it should be excluded, until another command is entered. Banish it to the ether by running just an empty space as a command.

Count true values with postgres

I can try to count everything that is not null in postgres:

psql> select count(x.x) from (values (null), ('hi'), (null), ('there')) x (x);
2

But if I try to count everything that is true in postgres, I don’t get what I want:

psql> select count(x.x) from (values (false), (true), (false), (true)) x (x);
4

I can, however, take advantage of the postgres ability to cast boolean to int:

psql> select true::int;
1
psql> select false::int;
0

Using ::int I get:

psql> select count(x.x::int) from (values (false), (true), (false), (true)) x (x);
4

Postgres is still counting everything that is not null, but what if use sum instead?

psql> select sum(x.x::int) from (values (false), (true), (false), (true)) x (x);
2

Because everything is either a 0 or a 1, sum behaves like count.

Now you can do something like this:

psql> select sum((status = 'Awesome')::int) as awesomes, sum((status = 'Terrible')::int) as terribles from statuses;

Print the current stacktrace in Elixir

Stacktrace, backtrace, callstack, in Elixir its stacktrace and it’s available via Process.info/2 using the :current_stacktrace item:

Process.info(self(), :current_stacktrace)

And to print it:

IO.inspect(Process.info(self(), :current_stacktrace), label: "STACKTRACE")

I’m also learning that Process.info/2 takes a pid and an item as arguments. When you call Process.info/1 with just the pid you only get a subset of the info available, not everything.

The items available via Process.info/1 are listed in the erlang documentation here.

The additional items available via Process.info/2 are listed in the erlang documentation here.

You may note that backtrace is also an item that is available via Process.info but it contains more information than you are might need to figure out where you are in the code.

What's the Trailing Underscore in Elm?

Variables such as model_, with a trailing underscore, are allowed and conventional in Elm. If you’re familiar with a language where _model can mean an unused variable, this can cause a double-take. The trailing underscore is a nod to Haskell, telling us the variable is related, or similar, to a prior variable.

Here’s an example: model_ is the argument of the update function in the Elm architecture, and model is the updated model we’ll return:

> model_ = { start = "now" }
{ start = "now" } : { start : String }
> model = { model_ | start = "tomorrow" }
{ start = "tomorrow" } : { start : String }

This Stack Overflow answer summarizes the convention:

https://stackoverflow.com/a/5673954/2112512

A single quote (model') was used similarly in the past; that syntax was deprecated in Elm 0.18. Here’s a GitHub issue describing that decision:

https://github.com/elm-lang/elm-plans/issues/4

Give a commit a new parent

Given I have these logical commits on two branches:

normalbranch: A - B - C - D

funkybranch: Z - Y - X
git co normalbranch
git rebase --onto X C

Then the logical commits of normalbranch will now be:

normalbranch: Z - Y - X - C - D

We’ve given commit C the new parent of X. C’s hash will change and D’s hash will change.

I use this when I build on top of a branch that I’ve already submitted for a PR. That branch will get merged into the new root branch, and then I’ll need the new parent on the root branch for the commits that I’m currently working on.

PostgreSQL Triggers and Views 🐘

Today I learned a lot about PostgreSQL.

  1. Views can have triggers!
  2. Deleting such a view also deletes the trigger!

To demonstrate, let’s create a table and record, and a view that looks at our table.

create table some_table (name varchar);
insert into some_table values ('some name');
create view some_view as select name from some_table;

Now, we need a test function:

create or replace function always_null() returns trigger as $always_null$
  begin
    return null;
  end;
$always_null$ language plpgsql;

Let create a trigger for our function:

create trigger some_table_trigger
  instead of delete on some_view
  for each row execute procedure always_null();

Here’s our view… with a trigger attached!

backup=# \d some_view
                   View "public.some_view"
 Column |       Type        | Collation | Nullable | Default
--------+-------------------+-----------+----------+---------
 name   | character varying |           |          |
Triggers:
    some_table_trigger INSTEAD OF DELETE ON some_view FOR EACH ROW EXECUTE PROCEDURE always_null()

Our trigger can be inspected (lots of interesting info omitted):

backup=# select * from information_schema.triggers;
-[ RECORD 1 ]--------------+--------------------------------
trigger_name               | some_table_trigger

Drop the view, and lose the trigger:

backup=# drop view some_view;
DROP VIEW
backup=# select * from information_schema.triggers;
(0 rows)

Flunk Your ExUnit Tests ❌

Today I got to use the flunk/1 function in an ExUnit test. I like everything about this function, from its name to its signature.

Here’s how we used it:

case result do
  "bad" ->
    flunk("Your result is bad, and you should feel bad.")

  "good" ->
    assert result_passes_other_tests(result)
end

It fit well into a helper function that should fail if my result ever comes back a certain way.

flunk docs

`mix phx.digest` creates the manifest

mix phx.digest creates production ready assets: hashed, zipped and compressed.

In production, when you call:

Endpoint.static_path('asset_name.css')

This will look for a file in the priv/static directory and return the path for that file. But what you want is the hashed version of that file, because modern browsers are greedy cachers and will you use the use to bust that cache.

The static_path function can return a path that represents the cached version, but it needs to read the manifest, which maps the file to it’s hashed, zipped and compressed versions.

mix phx.digest creates the manifest along with the hashed, zipped and compressed versions of all the files in the priv/static directory.

PostgreSQL: Table or View?

I’m doing some database exploration today and needed to find out if a collection of tables are all tables, or if some of them are views. By querying the information schema via the following: select table_name, table_type from information_schema.tables;, I learned more about each table and its type:

> select table_name, table_type from information_schema.tables;
              table_name               | table_type
---------------------------------------+------------
 cats                                  | BASE TABLE
 dog                                   | BASE TABLE
 dogs_and_cats                         | VIEW

Update Map Syntax

There is a special syntax for updating a map in elixir.

thing = %{a: 1, b: 2, c: 3}
updated_thing = %{thing | b: 4}
# %{a: 1, b: 4, c: 3}

But be careful, it throws an error if you try to update a key that doesn’t exist!!

thing = %{a: 1, b: 2, c: 3}
# ** (KeyError) key :x not found in: %{a: 1, b: 2, c: 3}

Skip Pending Tests in ExUnit

In ExUnit you have the ability to tag a test with any atom:

@tag :awesome
test "my awesome test" do
end

@tag :terrible
test "my terrible test" do
end

And then you can exclude those tags at the command line so that your terrible test does not run:

mix test --exclude terrible

So you can make your own tag :pending and exclude those. If you don’t want to have to set the flag at the command line everytime, you can call ExUnit.configure. This is typically placed in the test/test_helper.exs file before ExUnit.start().

ExUnit.configure(exclude: :pending)
ExUnit.start()

Now your pending tests will not run.

That is how you can use custom tags to skip tests, but you can also just use the built in skip tag to skip tests.

@tag :skip
test "my terrible test" do
end

Timex `between?` is exclusive but can be inclusive

Timex.between? is a great function for determining if a time is in a specific time period, but it’s exclusive, so if you are testing for a time at the boundary the result will be negative.

iex> end_time = Timex.now()
iex> start_time = Timex.shift(end_time, days: -7)
iex> time = start_time
iex> Timex.between?(time, start_time, end_time)
false

Never fear, you can pass an inclusive option:

iex> Timex.between?(time, start_time, end_time, inclusive: true)
true

But you don’t have to be inclusive on both sides! Here, I pass :end so that I am only inclusive at the end of my time period.

iex> Timex.between?(time, start_time, end_time, inclusive: :end)
false

And of course I can be only inclusive at the beginning of the period if I prefer:

iex> Timex.between?(time, start_time, end_time, inclusive: :start)
true

Use `reset_column_information` to Migrate Data

If you’re generating a Rails migration, chances are you might need to facilitate migrating data to a new column. You can use reset_column_information in a migration file to pick up your changes and immediately do something with those new columns.

Assuming we have 2 models, DraftPost and Post -

  class AddColumnDraftToPosts < ActiveRecord::Migration[5.2]
    def change
      add_column :posts, :draft, :boolean, default: true
      Post.reset_column_information
      DraftPost.all.each do |draft_post|
        Post.create(content: draftPost.content)
        draft_post.delete
    end
  end

asdf Global Versions 🌏

Spend a bit of time with asdf, and you might see an error like this:

$ npm install
asdf: No version set for command npm
you might want to add one of the following in your .tool-versions file:

nodejs 10.15.3
nodejs 8.9.1

The project I’m trying to set up via npm install doesn’t specify a Node version in .tool-versions, and since I have multiple Nodes on my machine, asdf isn’t sure which to use.

I don’t want to edit .tool-versions in this project; I’d rather asdf had a global default.

Here’s how I made that happen:

$ asdf global node 10.15.3   

How to clear a Mac terminal and its scroll-back?

Just type this: clear && printf '\e[3J'

Or even better create an alias for that, here’s mine:

alias clear='clear && printf "\e[3J"';

Here’s what I’ve learned today:

On Mac a regular clear is pretty much the same as typing Control + L on iTerm2. This clears the screen what’s good but sometimes I want to clear all the scroll-back to clean the noise and find things faster.

In order to clean the scroll-back I was performing a Command + K. This cleans the screen and the scroll-back. That’s great, except that it messes up with tmux rendering and tmux holds all the scroll-back per pane, so it won’t work at all.

So my new alias solves that as after clear the screen it also sends a terminal command to reset the scroll back through printf '\e[3J' and this keeps tmux working just fine!

Print Calendar With Week Number 📅

I use the week number (1-52) in my notetaking. How do I know what week number it is? ncal.

$ ncal -w
    May 2019
Mo     6 13 20 27
Tu     7 14 21 28
We  1  8 15 22 29
Th  2  9 16 23 30
Fr  3 10 17 24 31
Sa  4 11 18 25
Su  5 12 19 26
   18 19 20 21 22

Though not shown in the output above, today’s date (the 10th) is highlighted in my terminal output. At the bottom of the column containing today’s date is the week number (19).

Friendly Zoom Personal Meeting URL ➡

If you have a Zoom account, you have a Personal Meeting URL— a permanent video conferencing room tied to your account. It looks like this:

https://zoom.us/j/5551112342

When another person with Zoom installed visits this link in a browser, Zoom starts a call in your room.

A pro move is to permenantly redirect a subdomain to this URL, like so:

chat.tomcruise.com

Now you have a memorable link you can share in online and offline conversation.

Match across lines with Elixir Regex `dotall` (s)

Elixir Regex is PCRE compliant and Ruby Regex isn’t in at least one specific way. The m (multiline) flag behaves differently in Elixir and Ruby.

Without any modifiers, Regex does not match across new lines:

iex> Regex.scan( ~r/.*/, "a\nb")
[["a"], [""], ["b"], [""]]
iex> Regex.scan( ~r/.*/, "ab")
[["ab"], [""]]

With the m (multiline) modifier, Regex will match the beginning of each line when using the ^ char.

iex> Regex.scan( ~r/.*/m, "a\nb")
[["a"], [""], ["b"], [""]]
iex> Regex.scan( ~r/^.*/, "a\nb")
[["a"]]
iex> Regex.scan( ~r/^.*/m, "a\nb")
[["a"], ["b"]]

The s (dotall) modifier is the right way in Elixir to match across newlines.

iex> Regex.scan( ~r/.*/s, "a\nb")
[["a\nb"], [""]]
iex> Regex.scan( ~r/^.*/s, "a\nb")
[["a\nb"]]

In ruby, you can match across lines with the m (modifier) and the s is ignored.

irb> "a\nb".scan(/.*/)
=> ["a", "", "b", ""]
irb> "a\nb".scan(/.*/m)
=> ["a\nb", ""]
irb> "a\nb".scan(/.*/s)
=> ["a", "", "b", ""]

Read about the history of dotall here

Rails 6 new ActiveRecord method pick

Rails 6 will be released with a new convenient method on ActiveRecord pick. This method picks the first result value(s) from a ActiveRecord relation.

Person.all.pick(:name)
# SELECT people.name FROM people LIMIT 1
# => 'John'

Person.all.pick(:name, :email)
# SELECT people.name, people.email FROM people LIMIT 1
# => ['John', 'john@mail.com']

So pick(:name) is basically equivalent to limit(1).pluck(:name).first.

Watch out for rails inconsistencies as pick(:name) returns a single string value and pick(:name, :email) returns an array of values.

Get _just one_ value from Ecto query

With the data structure you use in select’s expression argument you can specify what type of data structure the query will return for a row. I’ve used [] and %{} and %Something{} but you can also specify that each row is just one value without using a data structure at all.

Combine that with Repo.one to just return one row and you can just get the specific value you are looking for without any destructuring.

age = 
  User
  |> where(id: 42)
  |> select([u], u.age)
  |> Repo.one()

Jest toEqual on js objects

jest toEqual compares recursively all properties of object with Object.is. And in javascript an object return undefined for missing keys:

const obj = {foo: "FOO"}

obj.foo
// "FOO"

obj.bar
// undefined

So it does not make sense to compare an object against another one that has a undefined as value, so remove them all from the right side of the expectation to avoid confusion. In other words, this test will fail:

test("test fails => objects are similar", () => {
  expect({
    planet: "Mars"
  }).not.toEqual({
    planet: "Mars",
    humans: undefined
  });
});

// Expected value to not equal:
//   {"humans": undefined, "planet": "Mars"}
// Received:
//   {"planet": "Mars"}

Watch out!

Don't truncate when redirecting

A common problem in shell languages is truncating a file that you’re trying to transform by redirecting into it.

Let’s say I have a file full of “a”‘s

PROMPT> cat test.txt
aaaaaa

And I want to switch them all to “b”‘s

PROMPT> cat test.txt | tr 'a' 'b'
bbbbbb

But when I try to redirect into the same file I truncate file:

PROMPT> cat test.txt | tr 'a' 'b' > test.txt
PROMPT> cat test.txt
# nothing

This is because the shell truncates “test.txt” for redirection before cat reads the file.

An alternate approach is to use tee.

PROMPT> cat test.txt | tr 'a' 'b' | tee test.txt
bbbbbb
PROMPT> cat test.txt
bbbbbb

tee will write the output to both the file and to standard out, but will do so at the end of the pipe, after the cat command reads from the file.

Elixir sigil for DateTime with timezone

Elixir have a new sigil_U to be relased in the upcomming 1.9.0 version.

~U[2019-05-18 21:25:06.098765Z]

This new sigil creates a UTC DateTime.

Now Elixir will have all these sigils for dates & times:

Date.new(2019, 5, 18)
=> {:ok, ~D[2019-05-18]}

Time.new(23, 55, 6, 98_765)
=> {:ok, ~T[23:55:06.098765]}

NaiveDateTime.new(2019, 5, 18, 23, 55, 6, 98_765)
=> {:ok, ~N[2019-05-18 23:55:06.098765]}

DateTime.from_iso8601("2019-05-18T23:55:06.098765+02:30")
=> {:ok, ~U[2019-05-18 21:25:06.098765Z], 9000}

Install the last Elixir version with asdf

If you are using asdf for managing Elixir version through asdf-elixir then you can install the master version. This way we can use new features yet to be release as 1.9. Let’s see how:

asdf install elixir master
asdf global elixir master

Then:

elixir --version
Erlang/OTP 21 [erts-10.3.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Elixir 1.9.0-dev (6ac1b99) (compiled with Erlang/OTP 20)

Get pids for each beam application in Elixir

In Elixir, it’s hard to determine what pids belong to what applications just by using Process.info

iex> pid(0,45,0) |> Process.info
[
  current_function: {:application_master, :main_loop, 2},
  initial_call: {:proc_lib, :init_p, 5},
  status: :waiting,
  message_queue_len: 0,
  links: [#PID<0.46.0>, #PID<0.43.0>],
  dictionary: [
    "$ancestors": [#PID<0.44.0>],
    "$initial_call": {:application_master, :init, 4}
  ],
  trap_exit: true,
  error_handler: :error_handler,
  priority: :normal,
  group_leader: #PID<0.45.0>,
  total_heap_size: 376,
  heap_size: 376,
  stack_size: 7,
  reductions: 49,
  garbage_collection: [
    max_heap_size: %{error_logger: true, kill: true, size: 0},
    min_bin_vheap_size: 46422,
    min_heap_size: 233,
    fullsweep_after: 65535,
    minor_gcs: 0
  ],
  suspending: []
]

With the above output, I can’t tell what application this is!

I can use the :running key of Erlang’s :application.info to get a map of applications to pids.

iex> :application.info[:running]
[
  logger: #PID<0.93.0>,
  iex: #PID<0.85.0>,
  elixir: #PID<0.79.0>,
  compiler: :undefined,
  stdlib: :undefined,
  kernel: #PID<0.45.0>
]

Oh ok, pid(0,45,0) is the Kernel application.

Delete lines from file with sed

Imagine the following file:

sed.test

hised
hellosed
goodbyesed

If you want to delete a line matching a regular expression (e.g. hellosed), you can use d at the end of your regular expression.

sed '/hellosed/d' sed.test

Output:

hised
goodbyesed

However the file did not change:

cat sed.test

hised
hellosed
goodbyesed

To write the file in place use the -i [suffix] option. This argument allows you to specify the suffix of the backup file to be saved before committing your changes. For example:

sed -i '.bak' '/hellosed/d' sed.test

Now the file will be modified with our changes but we will also get a backup of the original file in sed.test.bak.

If you like living on the edge 🛩, and don’t want those pesky backup files littering your system, you can supply -i with an empty suffix, causing no backup file to be saved.

sed -i '' '/hellosed/d' sed.test

Test that an email was sent with correct params

assert_email_delivered_with is a function that checks if an email was delivered using Bamboo.

Sending an email is a side-effect in functional programming parlance. When testing a function where the email is sent several calls deep, you generally don’t get the email as a return value of that function.

When using Bamboo’s TestAdaptor in test mode, Bamboo captures the delivered email so that you can assert about it.

Lottery.notify_winner()
assert_email_delivered_with(subject: "You Won!!")

You can also assert on: to, cc, text_body, html_body, from, and bcc.

Happy Testing!

Hide an Elixir Module from Documentation

Recently I’ve noticed @moduledoc false sprinkled through some Elixir projects. What’s going on? To quote the docs:

Conveniently, Elixir allows developers to hide modules and functions from the documentation, by setting @doc false to hide a particular function, or @moduledoc false to hide the whole module. If a module is hidden, you may even document the functions in the module, but the module itself won’t be listed in the documentation…

Why would you see this in a project that doesn’t autogenerate docs? Possibly to satisfy a linter like Credo. In a codebase with public APIs, I think this kind of explicit statement is a good practice.

Writing Elixir Docs

Rails 6 Blocked Hosts

Rails 6 has a new feature where only whitelisted hosts are allowed to be accessed. By default only localhost is permitted.

When doing mobile development, you always need to test the app in a real device that connects to a backend. In order to automatically add the dev machine host to the list, just change your development.rb to:

# config/environments/development.rb 

config.hosts << "#{`hostname -s`.strip}.local"

Formatting the email address fields with Bamboo

When you create an email with Bamboo and deliver it:

 email = new_email(
      to: "chris@example.com",
      from: "vitamin.bot@example.com",
      subject: "Reminder",
      text_body: "Take your vitamins"
    )
 |> Mailer.deliver_now()

The to field will not be what you expect it to be:

email.to
# {nil, "chris@example.com"}

This is the normalized format for a to field. The first element in the tuple is the name of the recipient.

Bamboo allows you to format this field yourself with protocols.

defimpl Bamboo.Formatter, for: Person do
  def format_email_address(person, _opts) do
    {person.nickname, person.email}
  end
end

And now if I send an email with Person in the to field:

person = %Person{
    nickname: "shorty", 
  email: "chris@example.com"
}

email = new_email(
      to: person,
      from: "vitamin.bot@example.com",
      subject: "Reminder",
      text_body: "Take your vitamins"
    )
|> Mailer.deliver_now()

Then the to field gets formatted with our protocol.

email.to
# {"shorty", "chris@example.com"}

Read more about the Bamboo Formatter here.

Rails 6 upsert_all

Rails 6 and ActiveRecord introduces upsert_all 🎉

I used to have to jump into SQL and write this myself, but no more!


As a simplified use case, say you have a User and that user has many Tags. If an API wants to send you an array of strings (["Tag Name A", "Tag Name B"]), you no longer have to do any looping, find_or_create, or defining your own upsert in SQL nonsense.

@user.tags << Tag.upsert_all([
  { name: "Tag Name A"},
  { name: "Tag Name B"}
], unique_by: :name)

If the tags exist, they will be updated. If the tags are new, they will be created. And all this happens in 1 SQL Insert.

Improve Your Stack Overflow Feed ⬆

I like answering questions on Stack Overflow. It helps me stay aware of what people are struggling with in a particular language or framework. To do this well, I manage my tags. Here’s a screenshot of some of the tags I’m watching (subjects I care about and can contribute answers to) and ignoring (the rest).

image

With these settings, my question feed is mostly signal rather than noise.

If you visit Stack Overflow often and haven’t signed up for an account, I recommend it.

Fuzzy translation key merging with Gettext

Gettext is a great tool for i18n in elixir. It provides a mix task for extracting translation keys from your code. The translation keys (or message ids) are natural language and look like this:

gettext("Hi, Welcome to Tilex!")

After running mix gettext.extract && mix gettext.merge, an already translated Italian locale file would look like:

msgid "Hi, Welcome to Tilex!"
msgstr "Italian version of Welcome!"

There’s a chance that the natural language key (which also serves as the default string) will change.

If it changes just a little bit then the Italian locale file will look like:

#, fuzzy
msgid "Hi, Welcome to Tilex!!"
msgstr "Italian version of Welcome!"

It gets marked as #, fuzzy, and the new msgid replaced the old msgid.

Gettext determines how big of a change will constitute a fuzzy match with String.jaro_distance.

iex> String.jaro_distance("something", "nothing")
0.8412698412698413
iex> String.jaro_distance("peanuts", "bandersnatch")
0.576984126984127

The higher the number the closer the match. fuzzy_threshold is the configuration that determines whether a msgid is fuzzy or not and the default for fuzzy_threshold is 0.8, set here.

killall 💀

Today while pairing I learned about the killall command.

This BSD program kills processes by name, unlike kill, which uses PID.

Here’s me killing my Mac calculator:

$ ps aux | grep Calc
grep Calc
/Applications/Calculator.app/Contents/MacOS/Calculator
$ killall Calculator
$ ps aux | grep Calc
grep Calc

Thanks for the tip, Mark!

Two arguments in a command with xargs and bash -c

You can use substitution -I{} to put the argument into the middle of the command.

> echo "a\nb\nc\nd" | xargs -I{} echo {}!
a!
b!
c!
d!

I can use -L2 to provide exactly 2 arguments to the command:

> echo "a\nb\nc\nd" | xargs -I{} -L2 echo {}!
a b!
c d!

But I want to use two arguments, the first in one place, the next in another place:

> echo "a\nb\nc\nd" | xargs -I{} -L2 echo {}x{}!
a bxa b!
c dxa b!

I wanted axb! but got a bxa b!. In order to achieve this you have to pass arguments to a bash command.

> echo "a\nb\nc\nd" | xargs -L2 bash -c 'echo $0x$1!'
axb!
cxd!

Just like calling

bash -c 'echo $0x$1!' a b

Where $0 represents the first argument and $1 represents the second argument.

Weighted Shuffle in Elixir

Shuffling a list is easy, but what if you want to perform a weighted shuffle? That is, what if some elements in a list had a greater chance of being selected, or higher in the list, than others?

Given these colors and these weights:

weighted_colors = [{1, "orange"}, {3, "green"}, {5, "red"}]

I can create a list that is mostly represents each item the number of times of it’s weight by utilizing List.duplicate:

weighted_colors
|> Enum.map(fn ({weight, color}) -> 
  List.duplicate(color, weight)
end)
|> Enum.flatten
# ["orange", "green", "green", "green", "red", "red", "red", "red", "red"]

Then shuffle it:

|> Enum.shuffle
# ["red", "red", "orange", "red", "green", "green", "red", "green", "red"]

And then use Enum.uniq attribute of preserving the order in which it encounters each unique element from left to right.

|> Enum.uniq
# ["red", "orange", "green"]

Run this 100 times over the same weighted list and the most outcome that you would experience most often is:

# ["red", "green", "orange"]

Create a temp table from values

In postgres, if you are looking for a way to create a quick data set to experiment with you can create a temporary table using the values expression.

create temp table test as values ('a', 1), ('b', 2), ('c', 3);

In postgres, a temp table is a table that will go away at the end of the session in which it was created.

The types of the columns are inferred as you can see when examining the table in psql:

> \d test
               Table "pg_temp_3.test"
 Column  |  Type   | Collation | Nullable | Default
---------+---------+-----------+----------+---------
 column1 | text    |           |          |
 column2 | integer |           |          |

What happens if we try to mix types?

> create temp table test as values ('a', 1), (1, 'c');
ERROR:  invalid input syntax for integer: "a"

You can also be specific about types by casting the values of the first row.

> create temp table test as values ('a'::varchar(1), 1::decimal), ('b', 1);
> \d test
                    Table "pg_temp_3.test"
 Column  |       Type        | Collation | Nullable | Default
---------+-------------------+-----------+----------+---------
 column1 | character varying |           |          |
 column2 | numeric           |           |          |

Choose only one row for a given value w Distinct

If you have a table with many rows but you only want one row for a given value, you can use distinct on (<column>)

select distinct on (letter) * 
from (
values 
('y', 512), 
('y',128), 
('z', 512), 
('x',128), 
('z', 256)) 
as x (letter, number);

Which produces

letter | number
--------+--------
 x      |    128
 y      |    512
 z      |    512
(3 rows)

Distinct implicitly orders by the specified column ascending. Putting order by letter at the end of the select statement produces the exact same output (and execution plan).

Distinct chooses the first row for the given column after sorting, so changing the sort order for the second column will change the results.

Here are the results after adding a order by letter, number asc to the above select statement.

 letter | number
--------+--------
 x      |    128
 y      |    128
 z      |    256
(3 rows)

Values clause in a select statement

You’ve all seen the VALUES clause before. It is typically used to insert data.

insert into colors (name, brightness) 
values ('red', 10), ('black', 0), ('blue', 5);

You can also use the VALUES clause in a select statement:

select * from (values ('red', 10), ('black', 0), ('blue', 5)
);

This is great when experimenting with different parts of sql at the command line.

Additionally, values is a first class expression on its own:

> values (1,2), (2,3);

 column1 | column2
---------+---------
       1 |       2
       2 |       3
(2 rows)

Generating a postgres query with no rows

For testing purposes, it is nice to be able to simulate a query that does not result in any rows.

select * from generate_series(0, -1) x;
 x
---
(0 rows)

This is because (from the docs):

When step is positive, zero rows are returned if start is greater than stop.

Step (the third argument to generate series) defaults to 1, so anything where the from has to go backwards to get to the to will result in 0 rows.

select * from generate_series(4, 3) x;
 x
---
(0 rows)

And now you can see how sum() behaves without any rows:

select sum(x) from generate_series(1, 0) x;
sum
-----
   ø
(1 row)

Ohhh, it returns null. I hope you’re planning for that.

Parsing CSV at the command line with `csvcut`

Parsing csv at the command line is easy with the csvcut tool from csvkit.

csvkit is installable with pip

> pip install csvcut 

You can print only the columns you are interested in.

> echo "a,b,c,d" | csvcut -c 1,4
a,d
> echo "a,b,c,d" | csvcut -c 1,5
Column 5 is invalid. The last column is 'd' at index 4.

It also handles quoted csv columns:

echo 'a,"1,2,3,4",c,d' | csvcut -c 2,3

It handles new lines:

> echo 'a,b\nc,d' | csvcut -c 2
b
d

There are a virtual plethora of options check out csvcut --help.

csvkit has a number of csv processing tools, check them out here.

Get pip installed executables into the asdf path

Given I am using asdf when I install a python executable via pip, then I expect to be able to run that executable at the command line.

> asdf global python 3.7.2
> pip install some-executable-package
> some-executable
zsh: command not found: some-executable

This is because a shim has not yet been created for this executable. If you look into your shims dir for the executable:

> ls $(dirname $(which pip)) | grep some-executable

It doesn’t exist.

To place a shim into the shims dir of asdf you must reshim.

> asdf reshim python

And now you have a shim that points to the executable:

> ls $(dirname $(which pip)) | grep some-executable
some-executable
> some-executable
Hello World!