Today I Learned

A Hashrocket project

Ready to join Hashrocket? Find Openings here and apply today.

Reverse proxy tcp/udp with nginx

Nginx has the ability to reverse proxy tcp and udp with the stream directive, similar to the http directive:

# Reverse proxy postgres server
stream {
  server {
    listen 5432;
    proxy_pass 172.18.65.97:5432;
  }
}

http {
  server { 
    ...
  }
}

This can be useful to load balance tcp streams like a database connection.

If you built nginx with --with-stream=dynamic (you can check with nginx -V) you will need to manually load a shared object:

# nginx.conf
load_module /usr/lib/nginx/modules/ngx_stream_module.so

Override a form's action url with `formaction`

Recently, I found myself with a form that needed to be able to submit to multiple urls. You can definitely use some javascript to get around this but that seemed like overkill for my situation. That’s when I learned about the HTML formaction attribute.

You can use this attribute on a submit tag and it will override the action specified by the form. Here’s an example with a Ruby on Rails form, but you just as easily do something similar with plain HTML -

<% form_with url: false do |f| %>
  <% users.each do |user| %>
    <%= check_box_tag "user_ids[]", user.id, false %>
  <% end %>

  <%= f.submit "Assign to Users", formaction: "/users/assign" %>
  <%= f.submit "Print for Users", formaction: "/users/print" %>
<% end %>

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/button#attr-formaction

Saving screenshots to clipboard on Mac

Have you ever needed to take a screenshot just to send it to somebody, but then you are left with the cumbersome chore of having to actually delete the screenshot before they pile up on your desktop like a pile of dirty laundry?

Well, I am here to save you those 5 seconds you’ve been missing out on so you can enjoy that next sip of coffeeβ€”guilt free.

When taking a screenshot with CMD + shift + 3, hold down control as well.

When taking a screenshot with CMD + shift + 4, hold down control while dragging the crosshairs to select a section of screen to capture.

Now, instead of saving to your default screenshot location, it will just copy it to your clipboard. You can now just paste the image into your message with no cleanup necessary.

TL;DR
Hold down control when taking a screenshot to copy it to the clipboard.

Alternate ways to quit out of Vim

In an effort to increase my Vim knowledge, I stumbled across a blog post by rocketeer alumnus Josh Branchaud. His post covers a few ways to quit out of vim and it is worth the read.

In there I found these two useful ways for quitting out of Vim from Normal mode, as opposed to Command mode.

ZZ in normal mode is equivalent to :x Β  in command mode.
ZQ in normal mode is equivalent to :q! in command mode

Logging data as a table in the console

I’m sure you’re already familiar with console.log() to debug, but did you know that there is a similar console.table() that is great for displaying arrays and objects?

Here are some examples,

Assuming you had an array of names

img

Assuming you had a person object

img

Assuming you had an array of person objects

img

There is also an optional columns parameter, which takes an array containing the names of columns you want to include in the output.

console.table(data)
console.table(data, columns)

Understanding Query I/O in Postgres with BUFFERS

The EXPLAIN command in Postgres can help you understand the query plan for a given query. Furthermore, you can use EXPLAIN ANALYZE to see the estimated query plan and cost vs the actual time and rows.

To take it a step further, you can use EXPLAIN (ANALYZE, BUFFERS) to include a number that represents the I/O disk usage of certain parts of your query.

explain (analyze, buffers)
  select
    *
  from floor_plans
  order by created_at desc
;

                                                  QUERY PLAN
---------------------------------------------------------------------------------------------------------------
 Sort  (cost=2.56..2.60 rows=18 width=159) (actual time=0.062..0.065 rows=16 loops=1)
   Sort Key: created_at DESC
   Sort Method: quicksort  Memory: 28kB
   Buffers: shared hit=2
   ->  Seq Scan on floor_plans  (cost=0.00..2.18 rows=18 width=159) (actual time=0.018..0.032 rows=16 loops=1)
         Buffers: shared hit=2
 Planning Time: 0.111 ms
 Execution Time: 0.106 ms
(8 rows)

Make sure that if you run this with a query that writes, that you wrap it in a BEGIN...ROLLBACK statement.

https://www.postgresql.org/docs/current/using-explain.html

Nested table querying with ActiveRecord

Sometimes you need to query a joined table in ActiveRecord. This can be easily done via the hash syntax:

User.where(greetings: {text: "HOWDY"}).to_sql

=> SELECT "users".* FROM "users" WHERE ("greetings"."text" = 'HOWDY')

Note: the key here (greetings) is pluralized because we’re referencing a table name and not a potential relationship that ActiveRecord knows about.

Note: I’ve ommited a join to the greetings table that would have this query make sense

Cool: so we can do that but there’s even an alternative way to handle this using dot notation

User.where("greetings.text": "HOWDY"}).to_sql

=> SELECT "users".* FROM "users" WHERE ("greetings"."text" = 'HOWDY')

Under the hood, ActiveRecord will convert dots from the string key and interpret them as table and column names.

Match strings with regular expressions

In Ruby you can use String#=~ to compare a string with regexp, returning the first index where it is found. For example let’s search for the first ? in this string:

"www.example.com/search?meatloaf" =~ /\?/ 
=> 22

If there is no match, it returns nil

"www.example.com" =~ /\?/ 
=> nil

BONUS: When using Regexp#=~, which functions very similarly, you can use a regexp with named captures to store them in local variables.

/(?<search_params>\?.+)/ =~ "www.example.com/search?lasagna"
=> 22
search_params
=> "?lasagna"

Receive Javascript messages from iFrame

Due to security reasons what can be received from an iFrame can be very limited. However there is a utility built for the purpose of safely communicating with things like an iFrame, pop-up, etc.

An iframe can utilize the Window.postMessage() function to post info that may be relevant to someone embedding the iFrame.

Then in the page where the embedded iFrame is located, the message event can be watched.

window.addEventListener("message", (event) => {
  if (event.origin !== "IFRAME URL")
    return;

  // ...
}, false);

By utilizing the event’s origin you can be even safer. Above we’re declaring that if the origin of tghe event did not come from the expected iframe’s url… just stop.

If you’re good with the origin check then looks at the event’s data property. It will provide whatever was sent via postMessage().

Write a warning message to $stderr

While debugging, if you ever need to write to $stderr you might use $stderr#puts, but you can use Warning#warn which is better called from Kernel, because Kernel appends newlines and respects the -W0 flag:

$stderr.puts "you have been warned"
Warning.warn "you have been warned"
Kernel.warn "you have been warned"
warn "you have been warned"

Output:

you have been warned
you have been warnedyou have been warned
you have been warned

Listen for onfocus events on document

If you want to set an event listener on document for an input’s onFocus event, you’ll find there is no β€œonFocus” event for document. Instead you can use the focusin event listener:

document.addEventListener("focusin", function(e) {
  console.log("an input received focus");
});

So if you had dynamic events you can achieve this:

const on = (eventName, elementSelector, handler) => {
  document.addEventListener(eventName, function (e) {
    for (var target = e.target; target && target != this; target = target.parentNode) {
      if (target.matches(elementSelector)) {
        handler.call(target, e);
        break;
      }
    }
  }, false);
}

on("focusin", "#email_address", function(e) {
  const currentTarget = this;
  console.log("Email address just received focus");
});

Set the id of the root document fragment

<template id="my-template">
  <div>
    <input type="hidden" value="1" name="amount" />
  </div>
</template>

You can set the id by querying for the div:

const deep = true;
const template = document.querySelector("#my-template");
const div = document.importNode(template.content, deep);
div.querySelector("div").id = "my-id";
document.body.appendChild(div);

Output:

  <div id="my-id">
    <input type="hidden" value="1" name="amount" />
  </div>

Start multiple processes in dev env with bin/dev

Rails comes with bin/dev to start multiple processes from a Procfile.dev

# Procfile.dev
web: bin/rails server -p 3000
css: bin/rails tailwindcss:watch
jobs: bin/good_job
firebase: firebase emulators:start --import=./emulator-data

So that you can run everything at once:

$ bin/dev
08:36:10 web.1  | started with pid 40038
08:36:10 css.1  | started with pid 40039
08:36:10 firebase.1 | started with pid 40040
08:36:11 firebase.1 | i  emulators: Starting emulators: auth, firestore, storage
08:36:11 firebase.1 | i  firestore: Firestore Emulator logging to firestore-debug.log
08:36:11 web.1  | => Booting Puma
08:36:11 web.1  | => Rails 7.0.2.3 application starting in development
08:36:11 web.1  | => Run `bin/rails server --help` for more startup options
08:36:12 web.1  | Puma starting in single mode...
08:36:12 web.1  | * Puma version: 5.6.4 (ruby 3.1.2-p20) ("Birdie's Version")
08:36:12 web.1  | *  Min threads: 5
08:36:12 web.1  | *  Max threads: 5
08:36:12 web.1  | *  Environment: development
08:36:12 web.1  | *          PID: 40038
08:36:12 web.1  | * Listening on http://127.0.0.1:3000
08:36:12 web.1  | * Listening on http://[::1]:3000
08:36:12 web.1  | Use Ctrl-C to stop
08:36:12 css.1  |
08:36:12 css.1  | Rebuilding...
08:36:12 css.1  | Done in 300ms.

Avoid time discrepancies when benchmarking

You can avoid time discrepancies due to gc and allocs by using Benchmark#bmbm:

require 'benchmark'
array = (1..1000000).map { rand }
Benchmark.bmbm do |x|
  x.report("sort!") { array.dup.sort! }
  x.report("sort")  { array.dup.sort  }
end
Rehearsal -----------------------------------------
sort!   1.490000   0.010000   1.500000 (  1.490520)
sort    1.460000   0.000000   1.460000 (  1.463025)
-------------------------------- total: 2.960000sec
            user     system      total        real
sort!   1.460000   0.000000   1.460000 (  1.460465)
sort    1.450000   0.010000   1.460000 (  1.448327)

Make numbers more readable in a Rails view

Rails has a number of handy helper methods.

Specifically, in ActionView::Helpers::NumberHelper there is the method number_with_delimiter

If you want to display the number 100000 and have people know right away whether it is 1 million or 100 thousand, just pass it to number_with_delimiter and it will return "100,000".

Some useful optional keyword parameters:
delimiter:"πŸ”" replaces the default , delimiter. (100000 becomes 100πŸ”000)
separator:"πŸ•" replaces the default . separator. (100.56 becomes 100πŸ•56)

Add primary key to table

You can add a primary key to a table with alter an alter table statement:

Table with no primary key:

create table no_pks (id int generated by default as identity not null);
insert into no_pks select from generate_series(0,999);
[local] dillon@dillon=# \d no_pks
                           Table "public.no_pks"
 Column |  Type   | Collation | Nullable |             Default
--------+---------+-----------+----------+----------------------------------
 id     | integer |           | not null | generated by default as identity

You can add it later:

alter table no_pks add primary key (id);
[local] dillon@dillon=# \d no_pks
                           Table "public.no_pks"
 Column |  Type   | Collation | Nullable |             Default
--------+---------+-----------+----------+----------------------------------
 id     | integer |           | not null | generated by default as identity
Indexes:
    "no_pks_pkey" PRIMARY KEY, btree (id)

Print unknown exceptions in PL/pgSQL

When trying to figure out why a function raised an exception you can print the error code raised to lookup in the table Appendix A-1.

One method is to capture others and then raise the magic sqlstate variable (only available in exception handlers)

create or replace function do_it(name text)
  returns void
as $$
begin
  select 42 from nothing;
exception
  when others then
    raise '%: %', sqlstate, sqlerrm;
end;
$$
  security definer
  language plpgsql
;

Then you can view the error:

select do_it('hi');
ERROR:  42P01: relation "nothing" does not exist
CONTEXT:  PL/pgSQL function do_it(text) line 6 at RAISE

Proxy Development Requests with Create-React-App

Create-React-App supports an optional proxy option in development, which can be added to your package.json. In my case, I was dealing with CORS issues locally because our frontend and backend run on the same server in production, but our development setup is different.

My backend was running on http://localhost:4000, and frontend on http://localhost:3000.

In my package.json, I specified that I wanted to proxy my requests to backend development server:

{
  ...
  "proxy": "http:localhost:4000"
  ...
}

Then I changed my fetch call to be a relative URL:

// Was fetch("http://localhost:4000/api/graph", ...params)
fetch("/api/graphql")

Create-react-app will recognize that this path is not a static asset and β€œwill proxy the request (http://localhost:4000/api/graphql) as a fallback”.

Create-React-App Docs - Proxying API Requests in Development

curl with a progress bar

You can download files with a nice progress bar using curl’s -# flag:

curl -# -O https://files.example.com/large/long_video.mp4
#################                               38.6%

This might be preferable to the verbose output:

curl --no-progress-meter -O https://files.example.com/large/long_video.mp4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 17  433M   17 75.6M    0     0  28.7M      0  0:00:15  0:00:02  0:00:13 28.8M

Use encrypted env vars with direnv

Direnv can execute shell scripts, so given that your env file is encrypted, you can automatically have it become decrypted for you:

───────┬──────────────────────
       β”‚ File: .env
───────┼──────────────────────
   1   β”‚ STRIPE_PK="123456789"
   2   β”‚ API_KEY="qwertyuiop"
───────┴──────────────────────

Say is was encrypted:

ansible-vault encrypt --vault-password-file config/master.key .env
cat .env
───────┬─────────────────────────────────────────────────────────────────────────────────
       β”‚ File: .env
───────┼─────────────────────────────────────────────────────────────────────────────────
   1   β”‚ $ANSIBLE_VAULT;1.1;AES256
   2   β”‚ 35306466356632363334643432343132356662376462333964366534393462366333623764336161
   3   β”‚ 6131336435323834623539323462626235383330346562660a323534656133653237656634346235
   4   β”‚ 30653635663438313931393966383266663535313361613339396234373164323830373262633661
   5   β”‚ 6262356131306530350a643362623636323762656132326363323736633431396463616137343139
   6   β”‚ 66666438623230333636373563393165333562633964616536663363323334343235386465346663
   7   β”‚ 3365643263643766323835356230636539353034643034346136
───────┴─────────────────────────────────────────────────────────────────────────────────

Now that we have an encrypted .env file, we just need direnv to decrypt it whenever we’re in our directory:

───────┬────────────────────────────────────────────────────────────────────────────────────────────────
       β”‚ File: .envrc
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
   1   β”‚ export $(ansible-vault decrypt --vault-password-file config/master.key --output - .env | xargs)
───────┴────────────────────────────────────────────────────────────────────────────────────────────────

Output:

$ cd rails_app
direnv: loading ~/dev/rails_app/.envrc
direnv: export +API_KEY +STRIPE_PK
echo $API_KEY
qwertyuiop

Now whenever we enter the directory, we will have the unencrypted env vars, but the file remains encrypted on disk. For whatever that’s worth.

Use + as a closure in Array reduce

In swift you can pass a method as the closure:

import Foundation

let numbers = [1, 2, 3, 4, 5]
let total = numbers.reduce(0, +)
print("Average: \(total / numbers.count)")

=> "Average: 3"

You can also use the generic closure:

import Foundation

let numbers = [1, 2, 3, 4, 5]
let total = numbers.reduce(0, { accumulator, number in 
  accumulator + number
})
print("Average: \(total / numbers.count)")

=> "Average: 3"

String concatenation in Ruby

My first instinct when it comes to concatenating strings would be to use the += operator:

name = "Peter"
=> "Peter"

name.object_id
=> 15320

name += " Parker"
=> "Peter Parker"

name.object_id
=> 34480

However, as you can see, doing this creates a new object. Instead, << should be used to maintain the same string object and also improve performance when working on larger batches of strings.

name = "Peter"
=> "Peter"

name.object_id
=> 54960

name << " Parker"
=> "Peter Parker"

name.object_id
=> 54960

Prevent rails' file server from serving index.html

You can prevent rails’ file server from serving index.html, while continuing to serve other files from the public directory by changing the index_name:

class Application
  config.public_file_server.index_name = "other-index.html"
end

Rails.application.routes.draw do
  get :dashboard, to: "application#dashboard"
  root to: redirect("/dashboard")
end

Visiting the root path / will no longer serve public/index.html if you define another route for /

Implicit order column

ActiveRecord has a class method implicit_order_column= that allows you to override the behavior of the .first and .last methods.

class User < ApplicationRecord
  self.implicit_order_column = "email"
end

User.first
User Load (3.2ms)  SELECT "users".* FROM "users" ORDER BY "users"."email" ASC, "users"."id" ASC LIMIT $1  [["LIMIT", 1]]

Migrating Data in Ecto

I usually have created a mix task for data migrations to avoid putting the app down, but today I learned that ecto migrations accept an arg --migrations-path to their commands which allow us to have 2 separate folders for migrations.

With that we can easily use the default priv/repo/migrations folder for automatic migrations (for running on deployments) and a separate folder, let’s say priv/repo/data_migrations that we can run when it’s more convenient.

So in prod we run migrations on deploy and data_migrations on a quite time for the app to avoid downtime. And in dev and test env we just run them all as we usually have smaller dataset, so not a big deal.

Here’s a good post about this strategy.

Ignore ~/.psqlrc when using psql

You can ignore your ~/.psqlrc when running psql commands by using the -X or --no-psqlrc flags.

So when you have all this in your rc file:

/* ~/.psqlrc */
\x auto
\timing
\set PROMPT1 '%[%033[1m%]%M %n@%/%R%[%033[0m%]%# '
\set PROMPT2 '[more] %R > '
\pset null '☒'
\setenv PSQL_PAGER pspg
\setenv PAGER pspg

This command becomes quite noisy:

psql -c 'select 1'
Expanded display is used automatically.
Timing is on.
Null display is "☒".
Time: 0.210 ms
 ?column?
----------
        1
(1 row)

Time: 0.297 ms

If you run without the config file:

psql -X -c 'select 1'
 ?column?
----------
        1
(1 row)

on-line manual pages:

-X,
--no-psqlrc
    Do not read the start-up file (neither the system-wide psqlrc file nor the
    user's ~/.psqlrc file).

Get database value of model instance

AR instances have a method #attribute_in_database. Sometimes things don’t make sense, someone has created a method with the same name as the column:

create table users (id int, email citext);
class User < ApplicationRecord
  def email
    Time.current.strftime("%A, %B %C")
  end
end

User.first.email
=> "Thursday, March 20"

But you actually wan the email:

User.first.attribute_in_database("email")
=> "admin@example.com"

About EST versus EDT

Here in the US we have the system of daylight savings time (DST), at least for now.

But did you know that the timezone you use includes that? Cause I sure didn’t.

EST/CST/MST/etc are for the standard time version of the time zone EDT/CDT/MDT/etc are for the DST adjusted periods!

Largely, I prefer to let libraries handle this for me, so I’ve never dug into the details. But this one is also interesting from the perspective of communications with others.