Today I Learned

hashrocket A Hashrocket project

297 posts about #elixir surprise

Count Occurrences of Elements in List

The Enum Module in Elixir has the frequencies/1 function, which is useful for counting the occurrences of each element in a list. 

> Enum.frequencies(["dog", "cat", "dog", "bird", "dog", "cat"])
%{"bird" => 1, "cat" => 2, "dog" => 3}

There's also frequencies_by/2, where the 2nd argument is a function that normalizes the element/key before counting.

> Enum.frequencies_by(["a", "b", "a", "C", "A"], fn key ->
  String.upcase(key)
end)
%{"A" => 3, "B" => 1, "C" => 1}

https://devdocs.io/elixir~1.17/enum#frequencies/1

Sigils in Elixir

In Elixir, there's strings "hi" and charlists 'hi'.

You can also use the ~c sigil to denote a charlist - 

~c"hi there"

There's also a word list sigil ~w which behaves similarly to %w in Ruby. It constructs a list of words - 

~w(blue red green)
=> ["blue", "red", "green"]

Also worth mentioning, the ~r sigil for regexes -

"foo" =~ ~r"hashrocket"
=> false

"hashrocket" =~ ~r/hashrocket/
=> true

There's many other sigils in Phoenix and Elixir. Be sure to check out the docs!

h/t to Jack Rosa

https://hexdocs.pm/elixir/sigils.html

Enum Reject vs Filter

If you're anything like me, you regularly forget if Enum.filter/2 keeps elements that return true or filters them out.

Let existence of Enum.reject/2 be your reminder.

reject/2 does the same thing as filter/2 but discards truthy elements instead of keeping them. I wish that filter/2 was renamed to keep/2, then the 2 functions would feel more like logical opposites and perhaps more readable.

Using Enum Sort with multiple keys

You can sort an enum by multiple keys at once. For example, If we wanted to sort a list of 'Vehicle' structs by type and model at the same time, we could do this:

vehicle_list = [
  %Vehicle{type: "van", model: "Odyssey"},
  %Vehicle{type: "truck", model: "Avalanche"},
  %Vehicle{type: "van", model: "Pacifica"},
  %Vehicle{type: "truck", model: "Bronco"}
]

Enum.sort_by(vehicle_list, &{&1.type, &1.model})
#=>
[
  %Vehicle{type: "truck", model: "Avalanche"},
  %Vehicle{type: "truck", model: "Bronco"},
  %Vehicle{type: "van", model: "Odyssey"},
  %Vehicle{type: "van", model: "Pacifica"},
]

sort_by is first sorting the vehicles by the :type key and then sorting by the :model key.

Restrict Phoenix Component Attr Values

If you want to limit the permissible values of a Phoenix Component attribute, you can use the values keyword list option when you call attr/3. Here's is an example that restricts which atoms are being passed to the component:

use MyApp.Component

attr :acceptable_color, :atom, values: [:green, :blue]

if you were to pass any atom other than :green or :blue to this component, the compiler will warn you.

Phoenix chunk huge files

So Phoenix, actually Plug, has a way to chunkify a response back to the client. This allowed us to create a stream out of an ecto query, then pipe that stream into a csv encoder and finally to Plug.Conn.chunk/2, which end up solving the huge memory spikes that we had when downloading a CSV. Here's a pseudo-code:

my_ecto_query
|> Repo.stream()
|> CSV.encode() # external lib that encodes from a stream into CSV
|> Enum.reduce_while(conn, fn chunk, conn ->
  case Conn.chunk(conn, chunk) do
    {:ok, conn} -> {:cont, conn}
    _ -> {:halt, conn}
  end
end)

Add an error to an Ecto Changeset

If you want to create custom validations with ecto changeset, chances are you are going to need to add errors to the changeset to be shown to the user.

Using add_error/4 allows you to add an error to a changeset.

changeset = change(%BlogPost{}, %{content: "Hello World ...cont"})
add_error(changeset, :content, "Your blog content is too long!")

changeset.errors
[content: {"Your blog content is too long!", []}]

Compare NaiveDateTimes by a specific unit

In Elixir, you can get the difference between two NaiveDateTimes using the NaiveDateTime.diff/3 function.

The third arg allows you to pass a unit such as :microsecond, :millisecond, or any unit returned from System.time_unit. By default, the unit is set to :second. 

NaiveDateTime.diff(~N[2014-10-02 00:29:18], ~N[2014-10-02 00:29:10])
#=> 8
NaiveDateTime.diff(~N[2014-10-04 00:29:18], ~N[2014-10-02 00:29:10], :day)
#=> 2

Interestingly enough :day works as a unit, but not :month, or :year. 🤔

String Concat Pattern Matching In Elixir

This is a neat pattern matching trick in elixir, its best explained with a simple example:

invoice_message = "You owe $34"
"You owe " <> dollar_amount = invoice

IO.inspect(dollar_amount)
# => "$34"

With a slightly different situation, It may seem like you could do this:

invoice_message = "You owe 34 dollars"
"You owe " <> dollar_amount  <> " dollars"= invoice

IO.inspect(dollar_amount)
# => ** (ArgumentError) the left argument of <> operator inside
# a match should always be a literal binary because its size can't be verified. Got: dollar_amount

But sadly you'll need to use regex to do that because elixir will throw an error.

Custom Sigils In Elixir

You can create custom sigils by following the sigil_{character} definition pattern. Let's make an addition sigil sigil_a that sums up space separated numbers.

defmodule CustomSigil do
  def sigil_a(string, []) do
    string
    |> String.split(" ")
    |> Enum.map(& String.to_integer(&1))
    |> Enum.sum()
  end
end

# ~a(2 4 6)
#=> 12
# ~a(12 12)
#=> 24

Elixir's Info function

The elixir Kernel module has an interesting function i/1 that returns information about whatever you pass to it.

It will provide the argument's data type, byte size, raw representation, a description, and reference modules

i("hello world")

#=>
# Term
#  "hello world"
# Data type
#  BitString
# Byte size
#  11
# Description
#  This is a string: a UTF-8 encoded binary. It's printed surrounded by
#  "double quotes" because all UTF-8 encoded code points in it are printable.
# Raw representation
#  <<104, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100>>
# Reference modules
#  String, :binary
# Implemented protocols
#  Collectable, IEx.Info, Inspect, List.Chars, String.Chars

How to debug in Elixir

If you want to debug an elixir code execution you can use the dbg/2 macro. I was already using that nice macro to show the results on each step of a pipe execution on the logs, but I learned that if we use iex to start phoenix server then the dbg/2 calls will act like adding a breakpoints so we can pry on it. We need to start the server with:

iex -S mix phx.server

As a side note this debugging worked on the elixir 1.14.3 version but I saw that on the latest versions there's a new option to be passed to iex --dbg pry in order to swap the dbg implementation from the fancy IO output to the IEx.pry one.

Elixir Data Type Comparison

Elixir has an interesting defined sorting order for its data types. The order is as follows:

number < atom < reference < function < port < pid < tuple < map < list < bitstring

For example: 

random_data = [["Hello World"], {:hello, :world}, 16, %{key: "value"}, :atom]
Enum.sort(random_data)
#=> [16, :atom, {:hello, :world}, %{key: "value"}, ["Hello World"]]

Or even

4 < :atom
#=> true

Unix Timestamps in Elixir

Unix timestamps are a very simple way to compare times in an integer format. They are the total seconds elapsed since January 1st, 1970 UTC (The Unix Epoch).

In elixir, if you want to do some operations on a DateTime struct, and want to keep things simple, you can convert a DateTime struct to an integer with to_unix/2. By default, it will use seconds as the unit.

How to get javascript params in LiveView?

I found out that Phoenix LiveView has a function get_connect_params to be used when you want to get some value sent at the connection time from the javascript.

This way I can send params like:

// app.js
const params = {
  _csrf_token: csrfToken,
  timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
};

const liveSocket = new LiveSocket('/live', Socket, { params });

And get those like on my LiveView:

# my_live_view.ex
@impl true
def mount(_params, _session, socket) do
  tz = get_connect_params(socket)["timezone"] || "America/New_York"
  {:ok, assign(socket, :timezone, tz)}
end

Skip hidden fields on Phoenix inputs_for

I just found out that Phoenix Component inputs_for accepts a skip_hidden attr which was very useful today. We have a has_many relationship that we are using <.inputs_for> to decompose the nested fields and as it was generating some hidden fields the style was breaking.

We are actually using a Tailwind divide on the parent tag and those hidden fields were adding extra separation borders. A way to solve that was calling <.inputs_for twice as this:

<div>

  <.inputs_for field={@form[:posts]} />

  <div class="divide-y">
    <.inputs_for :let={form} field={@form[:posts]} skip_hidden>
      <.input type="text" field={form[:title]} label="Title" />
    </.inputs_for>
  </div>
</div>

Macro Guard Clauses in Elixir

Have you ever wanted to pass your own custom function into a guard clause?

Lets start by looking at a super basic guard clause here:

def greater_than_10(num) when num > 10, do: true
def greater_than_10(_), do: false

Let's say we want to get more specific with the operation inside of the when guard. This is overkill for our situation, but lets say we want the when to use a custom function of ours instead of a basic operation, but when guards don't allow you to directly pass functions to them.

Elixir has a fancy macro called defguard that allows you to define custom guards like functions. Let's change our function up and define a guard that checks if our argument is both a integer and even.

defmodule Guards do
  defguard is_even_num(num) when is_integer(value) and rem(value, 2) == 0
end

Let's use that guard in our function 

import Guards

def even_num_greater_than_10(num) when is_even_num(num) do
  case num > 10 do
    true -> true
    _ -> false
  end
end

Even though this example is a bit overkill, the option to use custom guards is a cool elixir feature.

Casting Associations in Phoenix

If you want to cast changes on a struct's associations, you can use the changeset function cast_assoc/3. This allows you to make changes to a parent struct and its associations at the same time.

Example

Lets say you have a struct called Artist and an associated schema called Album. The schema for the Album looks something like this:

def module MyApp.Music.Album do
 use Ecto.Schema
 alias MyApp.Music.Artist

  schema "albums" do
    field :title, :string
    
    belongs_to :artist, Artist
  end

  def changeset(artist, params \\ %{}) do
    artist
    |> cast(params, [:title, :artist_id])
    |> validate_required([:title])
  end
end

and the schema for artist looks like this:

def module MyApp.Music.Artist do
 use Ecto.Schema
 alias MyApp.Music.Album

  schema "albums" do
    field :name, :string
    
    has_many :albums, Album
  end
end

Now, if you wanted to build a changeset for the Artist where you can change the artist's name and the title field on an associated album:

def changeset(artist, params \\ %{}) do
    artist
    |> cast(params, [:name])
    |> cast_assoc(:albums, with: &MyApp.Album.changeset/2)
    |> validate_required([:name])
end

Convert Changeset Into Schema With `apply_changes`

Often times, it's necessary to perform some intermediate operations on a Ecto changeset before saving it to the database. It can be easier to deal with the underlying schema. And you might need the entire schema rather than only the changes in the current changeset.

You can use the apply_changes/1 function to apply that changeset and receive schema with which you can perform operations. It's worth noting that the data is not validated when changes are applied, so care needs to be taken to ensure validity before an attempt to save that record.

iex> lead_schema = Lead.changeset(%{name: "Andrew", email: "Andrew.Vogel@hashrocket.com"})
                   |> apply_changes() 
%Lead{name: "Andrew", email: "Andrew.Vogel@hashrocket.com"}

iex> lead_schema |> downcase_email() |> do_some_other_processing()

https://hexdocs.pm/ecto/Ecto.Changeset.html#apply_changes/1

Phoenix LiveView slot attributes

Today I learned how to define an attribute on a Phoenix LiveView slot using a do block:

slot :column
  attr :field, :atom
  attr :sortable, :boolean, default: false
end

def table(assigns) do
  ~H"""
  <table>
    <tr>
      <th :for={col <- @column}>
        <%= col.label %>
        <.sort_by :if={col.sortable} field={col.field}/>
      </th>
    </tr>
  </table>
  """
end

This way we can use our slots with attributes:

def render(assigns) do
  ~H"""
    <.table>
      <:colum field={:name} sortable>Name</:colum>
      <:colum field={:price}>Price</:colum>
      <:colum>Actions</:colum>
      ...
    </.table>
  """
end

Check this out

Pattern Matching Args with Exact Values

In elixir there is a handy trick for pattern matching for exact values. Let's say we have a function head_match that takes two arguments, a string and a list, that checks if the string argument is the same as the head of the list. We could use pattern matching like this:

def head_match(head, [head, _tail]) do
  IO.puts("Thats the head of the List!")
end
#this will match if we called it like head_match("first_word", ["first_word", "second_word"])

See how we named the first argument head and used that same name as the list head? This means that the function will only match if those two values are the exact same rather than just matching the structure of the arguments.

Shorthand Elixir Anonymous Functions

Elixir has a really cool syntax for writing anonymous functions (unnamed functions). It goes like this:

# The '&' operator is used to define the function and its arguments
putter = &(IO.puts &1)
putter.("Today I learned")
#...> Today I learned
#...> :ok

&1 refers to the functions first argument; in this example, we used "Today I learned". You can use more arguments with &2, &3, etc... For example:

combiner = &(&1 <> &2)
combiner.("Foo", "Bar")
#...> "FooBar"

Also notice that we are not calling either of these functions like function(arg1, arg2), instead we are calling them like function.(arg1, arg2). We have to use . because we are not actually naming these functions, we are only assigning them a reference, hence "anonymous" functions.

Pattern Match Keyword List in Function Def

TIL that you can pattern match a keyword list in a function definition.

Sometimes you'll receive the last argument as an empty keyword list; in this case, we're calling it opts. You can pattern match key-values by defining the shape in a function.

  defmodule Example do
    def hello(opts \\ [])
 
   # will match when message is the only keyword included in opts
    def hello([message: message]) do
      IO.puts("Hello #{message}")
    end

    # will match when there are multiple keywords but message is the first
    def hello([{:message, message} | _rest] = opts) do
      IO.puts("Hello #{message}")
    end
  end

> Example.hello(message: "World")
Hello World
:ok

# could also call in the more verbose way
> Example.hello([message: "World"])
Hello World
:ok

# :message but with other args after
> Example.hello(message: "World", something_else: "hi")
Hello World
:ok

Note: As a TIL reader pointed out, pattern matching Keywords will make your function args order dependent. The following would not work: 

> Example.hello(foo: "bar", message: "World")
** (FunctionClauseError) no function clause matching in Example.hello/1

If you need them to be order independent, use a map or just match on single argument, then check for each option appropriately with the Keyword module.

https://elixir-lang.org/getting-started/keywords-and-maps.html

Composite Primary Keys using Elixir Ecto

Ecto allows us to map a table with composite primary keys into our schemas. The way to do that is by using the primary_key: true option for the Ecto.Schema.field/3 instead of dealing with the special module attr @primary_key. Check this out:

defmodule MyApp.OrderItem do
  use Ecto.Schema

  @primary_key false
  schema "order_items" do
    field :product_id, :integer, primary_key: true
    field :order_id, :integer, primary_key: true
    field :quantity, :integer
  end
end

Identity Primary Keys on Ecto

Ecto, allow the type :identity to be used since 3.5 which is cool:

create table(:user, primary_key: false) do
  add :id, :identity, primary_key: true
  add :name, :string, null: false

  timestamps()
end

That generates this SQL:

CREATE TABLE public.users (
    id bigint NOT NULL,
    name character varying(255) NOT NULL,
    inserted_at timestamp(0) without time zone NOT NULL,
    updated_at timestamp(0) without time zone NOT NULL
);

ALTER TABLE public.users ALTER COLUMN id ADD GENERATED BY DEFAULT AS IDENTITY (
    SEQUENCE NAME public.users_id_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1
);

The only issue is that there's no option to change from BY DEFAULT to ALWAYS 😕

Phoenix component attr definition

The new Phoenix.Component attr/3 function is awesome. It does compile time validations to all the places that are calling the component and also it comes with nice and useful options. For instance the default option:

attr :name, :string, default: "there"

def greet(assigns) do
  ~H"""
  <h1>Hey <%= @name %>!</h1>
  """
end

That's very useful. That would totally replace most usages of assign_new like I used to do:

def greet(assigns) do
  assigns = assign_new(assigns, :name, fn -> "there" end)

  ~H"""
  <h1>Hey <%= @name %>!</h1>
  """
end

This way we can call:

<.greet />

And have this html generated:

<h1>Hey there!</h1>

Migrating Data in Ecto

I usually have created a mix task for data migrations to avoid putting the app down, but today I learned that ecto migrations accept an arg --migrations-path to their commands which allow us to have 2 separate folders for migrations.

With that we can easily use the default priv/repo/migrations folder for automatic migrations (for running on deployments) and a separate folder, let's say priv/repo/data_migrations that we can run when it's more convenient.

So in prod we run migrations on deploy and data_migrations on a quite time for the app to avoid downtime. And in dev and test env we just run them all as we usually have smaller dataset, so not a big deal.

Here's a good post about this strategy.