Today I Learned

hashrocket A Hashrocket project

Use comm to Compare Files

Today I learned about comm, which is used to select the common lines in two files. It's pretty neat, but has a strange output format.

Say we have two text files:

# first.txt
one
two
three

# second.txt
one
three
four

We can run the below command to find the common lines. In the output, the first column is what's only in first.txt, the second column is what's in second.txt, and the third column is what's common.

$ comm first.txt second.txt                                                                                         
                one
        three
        four
two
three

Hmm, that doesn't look right - one and three are common, not just one. The caveat with comm is that the files need to be sorted lexically. You can sort easily in bash with bird beak notation for process substitution.

$ comm <(sort first.txt) <(sort second.txt)
        four
                one
                three
two

If we only want the common lines, we can apply the flags -12 to hide the first and second columns:

$ comm -12 <(sort first.txt) <(sort second.txt)
one
three

Add metadata to Stripe webhooks using TwiML <Pay>

This is kind of a specific TIL, but hopefully it saves somebody else the time it took me to find the answer.

If you are using the TwiML <Pay> verb there is a noun on <Pay> called <Parameter> which lets you pass parameters to your action.

If your payment provider is Stripe and you want to send custom metadata, you can send a <Parameter> with a name prefaced as metadata_ and it will be added to the Stripe metadata that gets sent out in the webhook.

Here's an example:

<Pay paymentConnector="stripe"> 
   <Parameter name="metadata_testKey" value="value" /> 
</Pay>

and now the object you receive in your webhook will have the following:

"metadata": {
  "testKey": "value"
}

Modify Tables with `change_table` in Rails

Ever since I used ecto in Elixir Phoenix, I've always thought about their alter table function while working on Rails projects. Turns out, I have been living under a rock and I could have been using change_table this whole time 🤦🏻‍♂️

For those new to the change_table function, it allows you to make bulk alterations to a single table with one block statement. You can adjust columns(and their types), references, indexes, indexes, etc. For a full list of transformations, see the official api documentation.

If you're using Postgres or MySQL, you can also add an optional argument bulk: true and it will tell the block to generate a bulk SQL alter statement.

class AddUserIdToPosts < ActiveRecord::Migration[7.1]
  def change
    change_table(:posts, bulk: true) do |t|
      t.remove_reference :user_post, :text, null: false
      t.references :user, null: false
    end
  end
end

API Ruby on Rails - change_table

Hooray Rails! 🥳

Convert objects into URL query strings

In Rails you can easily convert objects into URL query strings.

Here are a few examples:

string = "vanilla"
big_string = "extra chocolate"
integer = 5
array = ["vanilla", "chocolate"]
hash = {flavor: "vanilla", scoops: 2}

string.to_query("flavor")
=> "flavor=vanilla"

big_string.to_query("flavor")
=> "flavor=extra+chocolate"

integer.to_query("scoops")
=> "scoops=5"

array.to_query("flavors")
=> "flavors%5B%5D=vanilla&flavors%5B%5D=chocolate"

hash.to_query
=> "flavor=vanilla&scoops=2"

Note: Hashes don't require a key to be passed in, because they are inferred from the hash keys.

Change ActionCable's URL on the Client

ActionCable provides a handy escape hatch for changing the url path of the cable in your frontend. In my case, I wanted to specify a token on the cable url so that I could preserve some in place functionality while also being able to use the Turbo::StreamsChannel.

I did some source diving in turbo-rails and found that turbo calls out to createConsumer which comes from @rails/actioncable.

And that's where I found this (part of which is pasted below with a subtle change for readers) - 

export function createConsumer(url = getConfig("url") || "/cable") {
  return new Consumer(url)
}

export function getConfig(name) {
  const element = document.head.querySelector(`meta[name='action-cable-${name}']`)
  if (element) {
    return element.getAttribute("content")
  }
}

You can place a meta tag for the cable url in your views; this will be picked up by Action Cable and Turbo - 

<meta name="action-cable-url" content="/cable?foo=bar">

Open New Tab with HTML Form Submit

You can open a new tab on form submit by changing the target attribute of the form element: 

<form target="_blank">
 <!-- ... -->
</form>

While this is useful, it doesn't handle the scenario where you have multiple submit buttons (different actions) that need to behave differently.

For that, there's the formtarget attribute on your submit button/input. Note that by setting this attribute, you will override the form target attribute if set.

In the example below, the "Save" button will submit the form in the current window, while the Preview window will submit the form in a new tab

<form action="/liquid-templates/1">
  <!-- ... -->

  <button type="submit">Save</button>
  <button type="submit" formaction="/liquid-templates/1/preview" formtarget="_blank">Preview</button>
</form>

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/form#target https://developer.mozilla.org/en-US/docs/Web/HTML/Element/button#formtarget

Ruby Data class to replace Struct

Ruby has a new-ish class to build "immutable" structs, check this out:

Measure = Data.define(:amount, :unit)
distance = Measure.new(100, 'km')

distance.amount
#=> 100

distance.unit
#=> "km"

And if you try to use a setter, then it will fail:

 distance.amount = 101
(irb):7:in `<main>': undefined method `amount=' for an instance of Measure (NoMethodError)

Restrict Phoenix Component Attr Values

If you want to limit the permissible values of a Phoenix Component attribute, you can use the values keyword list option when you call attr/3. Here's is an example that restricts which atoms are being passed to the component:

use MyApp.Component

attr :acceptable_color, :atom, values: [:green, :blue]

if you were to pass any atom other than :green or :blue to this component, the compiler will warn you.

Specify behavior for nulls in a unique index

Postgres 15 gave us the ability to specify how we want null values to be treated when dealing with unique indexes.

By default, nulls are considered unique values in Postgres:

create table users (name text, email text unique);
-- CREATE TABLE
insert into users values ('Joe', null), ('Jane', null);
-- INSERT 0 2

This default behavior can also be explicitly set using the nulls distinct clause:

create table users (name text, email text unique nulls distinct);
-- CREATE TABLE
insert into users values ('Joe', null), ('Jane', null);
-- INSERT 0 2

To change the default behavior and prevent nulls from being considered unique values, you can use the nulls not distinct clause:

create table users (name text, email text unique nulls not distinct);
-- CREATE TABLE
insert into users values ('Joe', null), ('Jane', null);
-- ERROR:  duplicate key value violates unique constraint "users_email_key"
-- DETAIL:  Key (email)=(null) already exists.

See this change in the Postgres 15 release notes

Return Types In PSQL Pattern Match

If you use the pattern match operators in PSQL, you'll want to mind the column types passed to these statements

If you use a string, you will get a boolean return -

select 'a' like '%b%';
?column?
----------
 f
(1 row)

select 'a' like '%a%';
?column?
----------
 t
(1 row)

But if you select a null in one of these statements, the return is null as well -

select null like '%a%';
?column?
----------
 ø
(1 row)

Moral of the story - if you're expecting a boolean, you can coalesce the column before the pattern match - 

select coalesce(null, '') like '%a%';
?column?
----------
 f
(1 row)

https://www.postgresql.org/docs/current/functions-matching.html

Silence output of command in Makefile

Have you ever wanted to keep your makefile output a bit tidier? The @ symbol is your secret weapon.

Think of it like a silencer for your makefile. Every time you run a command, the makefile helpfully echos it back to you. But that echo gets silenced with the @ symbol at the beginning of a line.

This can be handy for keeping things clean, especially when you have a long list of commands in your makefile. Imagine a recipe with a million ingredients – you only care about the final dish, not every single step along the way, right?

Here's an example:

server: # Runs rails server
  @RAILS_LOG_LEVEL=debug bin/rails server

See how that works? The command executed for make server runs silently in the background.

Now, remember, this doesn't mean errors magically disappear. If something goes wrong, the error message will still show up. But for everything else, it's like a behind-the-scenes operation, keeping your makefile output focused on the important stuff.

So next time you want to streamline your makefile output, grab the @ symbol and hit the mute button on those noisy commands!

Set Env Vars with Shell Scripts

Sometimes I have limited screen real estate in my terminal, and my normal prompt of current/working/directory (git_branch) % takes up too much space. I wanted a bash script that could change my prompt to something short like & in one quick command. So I wrote a shell script:

#!/bin/bash

PS1="\[\e[32m\]& \[\e[m\]"

Nice and simple, right? 

~/src/dotfiles (main) % ./shorter.sh
~/src/dotfiles (main) %

Except, it doesn't change anything 😱. That's because changing PS1 isn't executing a command, it's setting an environment variable. So, just executing this shell script isn't enough, we need to source it to source the new PS1 in this terminal.* 

~/src/dotfiles (main) % . ./shorter.sh
&

* This is also why changing the prompt like this only affects the current terminal, and not any others that you may have open at the same time.

Regex in RSpec Argument Matchers

Today I learned you can use regular expressions in RSpec method argument expectations.

Suppose I have a method that takes an email, and a registered boolean as parameters:

def some_method(email:, registered:)
end

In a spec, it's easy enough to verify that it was called with set parameters, like test@example.com and true:

expect(subject)
  .to receive(:some_method)
  .with(email: 'test@example.com', registered: true)

But what if I want to verify that the email address just belongs to example.com? We can use a regex for that!

expect(subject)
  .to receive(:some_method)
  .with(email: /@example.com$/, registered: true)

ActiveRecord allows you to view saved changes

Say you have an Activerecord object that you are making changes to.

user = User.last
user.update(first_name: "Joe", last_name: "Hashrocket")

With ActiveModel's Dirty module, you can view the changes made to the model even after it has been saved using the saved_changes method

user.saved_changes
# {"first_name"=>[nil, "Joe"], "last_name"=>[nil, "Hashrocket"]]}

Trigger a Stimulus.js action with a window event

Stimulus.js controllers are great! However, sometimes you need to trigger them in different ways. Usually, you're looking at an event on an element.

<textarea 
  name="embed_html"
  data-action: "input->textarea-autogrow#autogrow"
  controller: "textarea-autogrow"></textarea>

Now, this is a trivial example as the Textarea-Autogrow controller is already monitoring the element without the need for the explicit data-action above.

In my case, I'm also using this text area within a tab powered by another controller from the Tailwind Stimulus Components library called Tabs. Due to the tab hiding my text area at the time of rendering, it doesn't calculate the height of the content inside. I'll need to trigger this manually. Luckily, the Tab controller emits a tsc:tab-change event on the window.

Now, I can utilize this on my textarea with the @window syntax.

<textarea 
  name="embed_html"
  data-action: "tsc:tab-change@window->textarea-autogrow#autogrow input->textarea-autogrow#autogrow"
  controller: "textarea-autogrow"></textarea>

This way, we trigger the autogrow function with each element's input and when we change tabs.

Check out the Global Events documentation for more info.

Change creation strategy in FactoryBot

I found out that's possible to change the FactoryBot strategy by invoking the to_create method inside the factory definition.

We had to do that to make factory bot linting to work on a factory that acts like an enum. So we did something like this:

FactoryBot.define do
  factory :role do
    to_create do |instance|
      instance.attributes = instance.class
        .create_with(instance.attributes)
        .find_or_create_by(slug: instance.slug)
        .attributes

      instance.instance_variable_set(:@new_record, false)
    end
  end
end

The said part here is that FactoryBot expects us to mutate that instance in order to work.

Create Accessors for JSONB column data in Rails

I was recently reading about the ActiveRecord::Store API and just today, I got the chance to use it. It's a really great API for working with serialized data in Rails.

Let's consider the following model Geolocation, with a jsonb column data. The contents of the data column look something like this -

{
  "coordinates": {
    "latitude": "5.887692972524717",
    "longitude": "-162.08234016158394"
  }
}

The store_accessor creates getters and settings for coordinates.

class Geolocation < ApplicationRecord
  store_accessor :data, :coordinates
end

Geolocation.last.coordinates
# => {"latitude" => "5.887692972524717", "longitude" => "-162.08234016158394"}

In my case, I have nested data that I'd like to create accessors for - latitude and longitude. From what I could find, this API doesn't support nested data yet, so we have to bring in a helper from ActiveModel::Attributes. We declare the coordinates portion as a jsonb attribute.

class Geolocation < ApplicationRecord
  store_accessor :data, :coordinates
  store_accessor :coordinates, :latitude, :longitude
  attribute :coordinates, :jsonb
end

geo = Geolocation.new
geo.coordinates
# => nil
geo.latitude
# => nil
geo.longitude
# => nil

geo.latitude = 5.887692972524717
# => 5.887692972524717
geo.coordinates
# => {"latitude"=>5.887692972524717}
geo.longitude = -162.08234016158394
# => -162.08234016158394
geo.coordinates
# => {"latitude"=>5.887692972524717, "longitude"=>-162.08234016158394}

I_Ran_Out_Of_Words_Hopefully_You_Get_My_Point :)

S/O Vlad & LayeredDesignForRailsApplications

https://devdocs.io/rails~7.1/activerecord/store

View Git Commits from Blame With Fugitive

I use vim-fugitive all the time to run a blame on the file I have open in (neo)vim. In the blame buffer, I typically hit enter on a commit to view the full commit in the buffer. It's nice, but then I need to jump back to view the code again.

Today I learned if you hit o instead of enter, the commit will open in a split below, which is much more convenient for me - it allows me to see the current version of the file, the blame, and the commit in one view. Game changer.

image

Handle Julian Dates in Ruby

I recently had to deal with parsing Julian Dates on a project. This was my first time seeing this in the wild and to my surprise, Ruby has some handy utilities on Date for converting to/from Julian Date Numbers.

Parse a Julian Date Number into a Date

Date.jd(2459936)
#=> #<Date: 2022-12-22 ((2459936j,0s,0n),+0s,2299161j)>

Convert a Date Object to a Julian Date Number

Date.new(2024, 02, 29).jd
#=> 2460370

https://en.wikipedia.org/wiki/Julian_day

Remove padding from values in strftime

By default, some numbers in strftime are padded, either with 0 or ' '.

For example:

best_moment_ever = DateTime.new(1996, 2, 15, 19, 21, 0, '-05:00')
=> Thu, 15 Feb 1996 19:21:00 -0500

best_moment_ever.strftime("%m/%e/%Y at %l:%M%P")
=> "02/15/1996 at  7:21pm"

As we can see, there is a big gap between at and 7:21pm. This is because the hour is being padded with empty string. Sometimes this is fine, but if you ever wanted to remove any padding, just add a - flag to the directive:

best_moment_ever.strftime("%-m/%e/%Y at %-l:%M%P")
=> "2/15/1996 at 7:21pm"

Notice how we also removed other built in padding, like the 0 in the month

There's a few other ways you can manipulate the results. Learn more here!

List Installed Fonts via the Command Line

Today I learned there's a command line utility called fc-list. It lists the fonts installed on your system.

Running fc-list will print out a lot of information - font families, the different styles available, where they're installed.

I find it's much more useful to run fc-list : family, which will print out all the font family names installed:

$ fc-list : family

Fira Code,Fira Code SemiBold
0xProto Nerd Font
Iosevka Term,Iosevka Term Extrabold
Menlo
.SF NS Mono
...

This is a lot easier to read and grep through!

My main use case for this is if I want to use a font in my alacritty config, but don't know the font's exact name. For example, if I want to use Apple's new(-ish) SF Mono font, the font family is not SF Mono as you might expect - its clearly .SF NS Mono 🤷.

Scoped Uniqueness Validation

Let's say you have a blog, with Authors and BlogPosts. A BlogPost has an Author and a title. If we have a requirement that the post titles need to be unique, our model would look something like this:

class Blog < ApplicationRecord
  has_one :author

  validates :title, uniqueness: true
end

Let's say the requirement changes, and the post title needs to be unique to the author (yeah it's convoluted, but let's roll with it). So Ann can author a post titled 'Cool Post', and Bob can also author 'Cool Post', but Ann can't publish another post titled 'Cool Post'.

Conveniently, uniqueness validations can be scoped to a particular attributes. So to satisfy the above, we can update the validation to:

class Blog < ApplicationRecord
  has_one :author

  validates :title, uniqueness: {scope: :author_id}
end

Tell Rails Your Foreign Key is a UUID

Let's say you have a blog with an Author model, and you want to create a blog_posts table. Each post has an author, and you want a foreign key on blog_posts to the Author's id.

class CreateAuthors < ActiveRecord::Migration[7.0]
  def change
    create_table :authors do |t|
      t.string :name
    end
  end
end

class CreateBlogPosts < ActiveRecord::Migration[7.0]
  def change
    create_table :blog_posts do |t|
      t.string :title
      t.text :content
      t.references :author, null: false, foreign_key: true

    end
  end
end

Pretty straightforward, right? But if Author#id is a UUID, you'll probably run into some issues with this migration. Rails by default assumes your table's IDs will be BigInt and if your IDs aren't then you need to specify the type in t.references:

class CreateAuthors < ActiveRecord::Migration[7.0]
  def change
    create_table :authors, id: :uuid do |t|
      t.string :name
    end
  end
end

class CreateBlogPosts < ActiveRecord::Migration[7.0]
  def change
    create_table :blog_posts, id: :uuid do |t|
      t.string :title
      t.text :content
      t.references :author, null: false, foreign_key: true, type: :uuid

    end
  end
end

Comparing Nullable values in PostgreSQL

PostgreSQL treats NULL values a bit different than most of the languages we work with. Said that, if you try this:

SELECT * FROM users WHERE has_confirmed_email <> TRUE

This would return users with has_confirmed_email = FALSE only, so NULL values that you have on your DB would be ignored. In this case if you want to get users with has_confirmed_email as FALSE or NULL then you can use IS DISTINCT FROM:

SELECT * FROM users WHERE has_confirmed_email IS DISTINCT FROM TRUE

Phoenix chunk huge files

So Phoenix, actually Plug, has a way to chunkify a response back to the client. This allowed us to create a stream out of an ecto query, then pipe that stream into a csv encoder and finally to Plug.Conn.chunk/2, which end up solving the huge memory spikes that we had when downloading a CSV. Here's a pseudo-code:

my_ecto_query
|> Repo.stream()
|> CSV.encode() # external lib that encodes from a stream into CSV
|> Enum.reduce_while(conn, fn chunk, conn ->
  case Conn.chunk(conn, chunk) do
    {:ok, conn} -> {:cont, conn}
    _ -> {:halt, conn}
  end
end)

Be informed of when a browser document is visible

A browser's visibilityState property is an essential aspect of web development that plays a crucial role in optimizing user experience and resource management in web applications. This property is part of the Page Visibility API, a web standard that provides developers with the ability to determine the current visibility state of a webpage.

The concept of visibility state revolves around whether a webpage is in a "visible" or "hidden" state. When a user navigates to a different tab, minimizes the browser window, or locks the screen, the webpage becomes "hidden". Conversely, when the webpage is in the foreground, and the user actively interacts with it, the state is "visible".

Add an error to an Ecto Changeset

If you want to create custom validations with ecto changeset, chances are you are going to need to add errors to the changeset to be shown to the user.

Using add_error/4 allows you to add an error to a changeset.

changeset = change(%BlogPost{}, %{content: "Hello World ...cont"})
add_error(changeset, :content, "Your blog content is too long!")

changeset.errors
[content: {"Your blog content is too long!", []}]

How to search case sensitively in Postgres

There's nothing special about this, I'm just a dummy who only ever used ilike and never thought twice about it.

Today, I learned that ilike is just a case-insensitive version of like.

This would return any rows with a name of Peter

select * from users
where name ilike '%peter%'
;

This would not return any rows with a name of Peter

select * from users
where name like '%peter%'
;

Stay tuned for tomorrow's TIL where I tell you about how I learned the sky is blue! 😂

Parseable npm audit output

In npm v6 and below, npm audit has a --parseable option that will output the audit report in plain text rows with tab delimiters. By default its a lot of information, but you can awk and grep/ripgrep to parse and filter down to what you want.

To only print the package name, vulnerability level and resolution, you can run:

% npm audit --parseable | awk -F $'\t' '{print $2, $3, $4}'

minimist critical npm update minimist --depth 15
immer high npm install react-scripts@5.0.1
loader-utils high npm install react-scripts@5.0.1
decode-uri-component high npm install react-scripts@5.0.1

And you can pipe that through to ripgrep to only show the critical vulnerabilities.

% npm audit --parseable | awk -F $'\t' '{print $2, $3, $4}' | rg critical

minimist critical npm update minimist --depth 15

In newer versions of npm, the --parseable option was dropped and instead you can use the --json option to output to json instead.

Disable database management tasks in Rails

If you wish to not have Rails manage migrations, seeds, schema changes, etc, you can add the database_tasks: false to your database.yml. This may be useful if you are "inheriting", sharing a database managed with some other system, or if you are connecting to a readonly database.

production:
  database: hashrocket
  adapter: postgresql
  encoding: unicode
  pool: 5
  database_tasks: false

https://guides.rubyonrails.org/active_record_multiple_databases.html#connecting-to-databases-without-managing-schema-and-migrations