Today I Learned

A Hashrocket project

Ready to join Hashrocket? Find Openings here and apply today.

239 posts about #elixir

Split a string in elixir

You can split a string in elixir with String.split

defmodule Phoenix.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def up do
    execute_sql("""
      create extension citext;

      create table users (
        id bigint generated by default as identity primary key,
        email citext unique not null,
        inserted_at timestamptz not null default now(),
        updated_at timestamptz not null default now()
      );
    """)
  end

  def down do
    execute_sql("""
      drop table users;
      drop extension citext;
    """)
  end
   

  def execute_sql(sql_statements) do
     sql_statements
     |> String.split(";")
     |> Enum.filter(fn s -> String.trim(s) != "" end)
     |> Enum.each(&execute/1)
  end
end

Elixir: Sandboxing an `iex` session

We can change the default .iex.exs file for another to be pre-loaded by iex using --dot-iex:

MIX_ENV=test iex --dot-iex .iex-sandbox.exs -S mix

This way we could create a rudimentary way to “protect” our existing database of changes on the iex session. Check this out:

# .iex-sandbox.exs
:ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo, ownership_timeout: 300_000)

This approach works for existing databases, so if the app we are working has a very difficult data setup then maybe this is the way to go. Although, making the db to hold data changes for as long as iex sessions are running it can be a bit too much in terms of db memory. Row & Table locks can definately be a problem as well.

Another important note is that we may have to call Sandbox.allow/3 if we need to call a GenServer or another process that touches the db.

The way to go

If we have a simpler data, with a nice seeds file, we’d be much better this way:

(export DB_NAME=my_app_sandbox && mix ecto.reset && iex -S mix)

Don’t forget to change our ecto config to use that ENV var.

Hibernating a GenServer

I was reading a great blog post from @cloud8421 this morning and I learned that Elixir GenServer has the capability to hibernate itself, which it means that it halts the continuous looping if there’s no incoming message and it runs a garbage collection to release some memory generated by that process.

Use that with moderation, and it’s always nice to evaluate if we really need to put a GenServer to hibernate, but in some cases it’s very handy.

How to force reload associations in Ecto

Ecto by default doesn’t load associations and you have to explicitly call preload:

id = 1
user = Repo.get(User, id) |> Repo.preload(:posts)

If you call preload again on the same user, Ecto won’t make the SQL query again because the association is already loaded:

user = user |> Repo.preload(:posts)

But in some cases you do want to reload a preloaded association then you can use force: true:

user = user |> Repo.preload(:posts, force: true)

Disable capture in Elixir Regex

Today I came across a regex that had to use the ( parenthesis to match a sequence of chars but I wanted just to match them, so I’d love if I could ignore that piece from the “capture” part.

So I learned that we could use ?: in the beginning of the ( and that would turn the capture off for that piece. Check this out;

iex> Regex.scan(~r/aa(bb)(?:cc|CC)/, "aabbccdd aabbCCdd")
[["aabbcc", "bb"], ["aabbCC", "bb"]]

In this case I am matching and capturing the bb sequence and I am matching but not capturing the cc|CC sequence options.

Preloading data for Phoenix LiveView components

One of the most common problems in web development is N+1. Graphql has taught me the DataLoader pattern and since then I never had to worry about N+1 as long as I kept using that pattern.

With Phoenix LiveView you don’t need to think about APIs anymore but I’m glad that it has support for preloading data kind of the same way as if it was a DataLoader. You can define a preload function that will receive a list of all the assigns so you can preload all the data needed for multiple components in one single batch before they are mounted. Here is the sequence of a component lifecycle:

preload(list_of_assigns) -> mount(socket) -> update(assigns, socket) -> render(assigns)

On this example only one SQL query is made to load all products by id:

@impl true
def preload(list_of_assigns) do
  product_ids = Enum.map(list_of_assigns, & &1.product_id)

  products =
    from(p in Product, where: p.id in ^product_ids, select: {p.id, p})
    |> Repo.all()
    |> Map.new()

  Enum.map(list_of_assigns, fn assigns ->
    Map.put(assigns, :product, products[assigns.product_id])
  end)
end

Elixir Supervisor default child_spec

When using Supervisor elixir already declares a child_spec/1 for us with the right type: :supervisor. The generated code will be something like:

defmodule MyApp.MySupervisor do
  use Supervisor, opts
  
  ...

  def child_spec(init_arg) do
    default = %{
      id: __MODULE__,
      start: {__MODULE__, :start_link, [init_arg]},
      type: :supervisor
    }

    Supervisor.child_spec(default, unquote(Macro.escape(opts)))
  end
end

Keep in mind that’s a pseudo-code as I left the Macro.escape for better understanding of the “expanded” code that happens on the __using__/1 function.

I tried to get this info from the docs, which I could not find that easily so I dig into the code, and as a good surprise this was so easy to find. Check this out.

By the way, the generated code is very similar to the GenServer one for the same function, the only difference is that on the GenServer there’s no :type key, and it works fine because the default value for the :type key is :worker.

Using `@describetag` in Elixir Tests

Wow! ExUnit have some special tags for @moduletag and @describetag which are self explanatory if you know what tags are used for in ExUnit. They are so useful in case you want to, for example, only run that section of tests:

defmodule MyApp.AccountsTest do
  use MyApp.DataCase

  @moduletag :module_tag

  describe "search" do
    @describetag :describe_tag

    @tag :test_tag
    test "return all users" do
      user_1 = insert!(User)
      user_2 = insert!(User)

      assert Accounts.search(User) == [user_1, user_2]
    end
    ...
  end
  ...
end

In case you want to run only tests that match with a module (file) level @moduletag:

my_app git:(main) ✗ mix test --only my_module_tag
Excluding tags: [:test]
Including tags: [:my_module_tag]
............
Finished in 0.2 seconds
22 tests, 0 failures, 10 excluded

Or if we want to run all tests that match with a describe block tagged with @describetag:

my_app git:(main) ✗ mix test --only my_describe_tag
Excluding tags: [:test]
Including tags: [:my_describe_tag]
....
Finished in 0.2 seconds
22 tests, 0 failures, 18 excluded

We can see by the number of excluded tests that this works great as expected.

Phoenix LiveView with some JSX flavor

Today I learned from Vinny about this Elixir library called Surface that allows you to write LiveView components that look a lot like JSX from React.

You first create a module that uses Surface.Component and define some props:

defmodule MyApp.WelcomeMessage do
  use Surface.Component

  prop name, :string, required: true

  def render(assigns) do
    ~F"""
    <div class="title">Hello, <strong>{@name}!</strong><div>
    """
  end
end

Then your LiveView module needs to use Surface.LiveView and you are ready to render that new “JSX” component passing some props:

defmodule MyApp.ExampleLive do
  use Surface.LiveView
  alias MyApp.WelcomeMessage

  def render(assigns) do
    ~F"""
    <div class="container">
      <WelcomeMessage name="Vinny" />
    </div>
    """
  end
end

Surface also handles state and has a way to handle portals, which are called slots. I might play with this library in one of my side-projects.

Nested attributes in Phoenix (Live) Views:

To use nested attributes in a form you have to use the inputs_for helper:

<%= f = form_for @changeset, Routes.user_path(@socket, :create), phx_submit: :submit, as: :user %>
  <%= input f, :name %>
  <%= inputs_for f, :address, fn a -> %>
    <%= input a, :city %>
    <%= input a, :zipcode %>
    <% end %><
  <%= submit "Create" %>
</form>

And then you can use cast_assoc and specify the changeset for the nested schema you want to validate:

defmodule MyApp.User do
  def registration_changeset(user, attrs, opts \\ []) do
    user
      |> cast(attrs, [:name])
      |> cast_assoc(:address, with: &MyApp.Address.changeset/2)
      |> validate_required([:name, :address])
  end
end

defmodule MyApp.Address do
  def changeset(address, attrs, opts \\ []) do
    user
      |> cast(attrs, [:city, :zipcode])
      |> validate_required([:city, :zipcode])
  end
end

Session cookie with Phoenix LiveView

You can use phx-trigger-action to make sure a form submit goes through http instead through the socket.

On the example below, when @trigger_submit is true then the form will be sumitted to the path Routes.user_session_path(@socket, :create), which will hit the controller and set the cookie:

<%= f = form_for @changeset, Routes.user_session_path(@socket, :create), phx_trigger_action: @trigger_submit, phx_change: :validate, phx_submit: :submit, as: :user %>
  <%= input f, :email %>
  <%= input f, :password %>
  <%= submit "Sign In", disabled: !@changeset.valid? %>
</form>

Count Occurrences in Elixir

Elixir recently introduced an useful couple of functions to count how many times a value appears in an Enumerable. It comes in two formats: frequencies/1 and frequencies_by/2. Here’s an example:

iex> [
...>   %{name: "Falcon", power: "Flight"},
...>   %{name: "Titan Spirit", power: "Flight"},
...>   %{name: "Atom Claw", power: "Strength"},
...>   %{name: "Electro", power: "Electricity Control"},
...>   %{name: "Loki Brain", power: "Telekinesis"},
...> ] |> Enum.frequencies_by(& &1.power)
%{
  "Electricity Control" => 1,
  "Flight" => 2,
  "Strength" => 1,
  "Telekinesis" => 1
}

Elixir Date and Time conversion into US standards

Today I learned how to manually convert a Date and Time into US standards: mm/dd/YYYY and hh:mm am|pm. Here’s my code snippet:

defmodule Utils.Converter do
  def to_usa_date(%Date{day: day, month: month, year: year}) do
    "~2..0B/~2..0B/~4..0B"
    |> :io_lib.format([month, day, year])
    |> to_string()
  end

  def to_usa_time(%Time{} = time) do
    period = (time.hour < 12 && "am") || "pm"
    hour = time.hour |> rem(12)

    "~2..0B:~2..0B ~2..0s"
    |> :io_lib.format([hour, time.minute, period])
    |> to_string()
  end
end

This way I can convert dates like ~D[2020-05-29] into 05/29/2020" and times like ~T[11:00:07.001] into 11:00 am" and ~T[23:00:07.001] into 11:00 pm".

Here’s the erlang :io_lib documentation

Elixir Inspecting Big Structs and Lists

IO inspects limits your data by default in 50, so 50 Map keys and values, or 50 List items, etc. Sometimes you have a larger data, so you can play with this number by setting an option to the IO.inspect/2 function, or you can set it to :infinity. Use :infinity with moderation of course:

big_map = 1..100 |> Enum.map(&{ &1, "value #{&1}"}) |> Enum.into(%{})
IO.inspect(big_map)
IO.inspect(big_map, limit: 100)
IO.inspect(big_map, limit: :infinity)

Convert HTTPoison Request into curl command

Today I learned how to convert a HTTPoison Request into a curl command, useful for logs. This might be useful to someone else, so I am sharing:

defmodule MyHTTP do
  def curl(%HTTPoison.Request{} = req) do
    headers = req.headers |> Enum.map(fn {k, v} -> "-H \"#{k}: #{v}\"" end) |> Enum.join(" ")
    body = req.body && req.body != "" && "-d '#{req.body}'" || nil
    params = URI.encode_query(req.params || [])
    url = [req.url, params] |> Enum.filter(& &1 != "") |> Enum.join("?")

    [
      "curl -v -X",
      to_string(req.method),
      headers,
      body,
      url,
      ";"
    ]
    |> Enum.filter(& &1)
    |> Enum.map(&String.trim/1)
    |> Enum.filter(& &1 != "")
    |> Enum.join(" ")
  end
end

And here’s an example of usage:

MyHTTP.curl(%HTTPoison.Request{
  url: "https://til.hashrocket.com",
  params: [q: "elixir"]
})

"curl -v -X get https://til.hashrocket.com?q=elixir ;"

Print stacktrace in Elixir #ElixirLang

When you are trying to debug something it is usually helpful to see the stacktrace of what called a function.

If you’re inside of a try expression, and within the catch or rescue block, that’s easy - use the __STACKTRACE__ Special Form.

try do
    # NOPE
rescue
  ArgumentError ->
    IO.puts("Stacktrace #{inspect(__STACKTRACE__)}")
catch
  value ->
    IO.puts("Stacktrace #{inspect(__STACKTRACE__)}")
else
  # NOPE
after
  # NOPE
end

For all other cases the incantation is a little more complicated (and if you don’t think you can remember it, perhaps set up a snippet):

self()
|> Process.info(:current_stacktrace)
|> IO.inspect(label: "---------> stacktrace")

Ecto queries can be combined

Queries can take other queries to build on top of, which is very useful for conditional query logic:

def find_posts(group_id, title) do
  posts =
    from(posts in Phoenix.Post,
      where: posts.group_id == ^group_id,
    )
    |> title_search(title)
    |> Phoenix.Repo.all()
end

defp title_search(query, title) do
  query
    |> where([posts], posts.title == ^title)
end

defp title_search(query, nil) do
  query
end

How to remove unused deps from mix.lock

When you remove a dep from the mix.exs file it will remain in the mix.lock file. Unused deps need to be “unlocked” in order to be removed from the lock file.

For example:

mix deps.clean --unused

will only remove the unused dependencies’ files, but will keep the library in the mix.lock file. If you run the clean command the --unlock option, it will also remove it from the lock file.

For example:

mix deps.clean --unlock --unused

How to change the request body size in Phoenix

Plug.Parsers accepts a length: key with a value of approximate bytes to accept and can be used in an endpoint file.

# endpoint.ex
plug Plug.Parsers,
  pasrsers: [:urlencoded, :multipart],
  length: 100_000_000 # 100 MB body size (approximately)

You can also configure the length for a specific parser:

plug Plug.Parsers,
  parsers: [:urlencoded, {:multipart, length: 100_000_000}] # 100 MB body size (approximately)

Ecto can only execute 1 sql statement at a time

Ecto can only ever execute 1 sql statement at a time, by design. For performance concerns, every statement is wrapped in a prepared statement
*some “security” is a fringe benefit of prepare statements.

In regards to the performance concerns of the prepared statement, PostgreSQL will force re-analysis of the statement when the objects in the statement have undergone definitional changes (DDL), making its use in a migration useless.

An example of a migration if you need to perform multiple statements:

def up do
  execute("create extension if not exists \"uuid-ossp\";")
  execute("alter table schedules add column user_id uuid;")
  execute("create unique index on schedules (id, user_id);")
end

def down do
  execute("alter table schedules drop column user_id;")
  execute("drop extension if exists \"uuid-ossp\";")
end

What happens on Ecto.Repo.rollback?

The rollback function interupts the execution of the function that you pass to transaction, but what is the mechanism for interupting execution?

When you call rollback you are essentially calling this code:

throw({DBConnection, conn_ref, reason})

throw is no good without a catch and you can find the try ... catch in the DBConnection module as well. It can be boiled down to this:

try do
  # call function passed to Ecto.Repo.transaction
catch
  :throw, {__MODULE__, ^conn_ref, reason} ->
    # send rollback request to database and return with {:error, reason}
    return {:error, reason}
  kind, reason ->
    # send rollback request to database and raise with reason
    :erlang.raise(kind, reason, stack)
end

This code exploits the ability of catch to both handle a throw and handle a raise.

`ets` table gets deleted when owning process dies

You can create a new ets table with:

:ets.new(:chris_table, [:named_table])

And you can confirm it was created with:

:ets.info(:chris_table)
[
id: #Reference<0.197283434.4219076611.147360>,
...
]

Now check this:

spawn(fn -> :ets.new(:spawn_table, [:named_table]) end)

Let’s see if it was created:

:ets.info(:spawn_table)
# returns :undefined

What gives? The erlang ets docs say this:

Each table is created by a process. When the process terminates, the table is automatically destroyed.

So, spawn created the process and then terminated, so :spawn_table got deleted when the process died.

Get a writable temporary directory in Elixir

When you need to write a temporary file to disk it is common to assume a particular path exists based on the operating system the app is running on:

tmp_file = Path.join("/tmp", filename)
File.write!(tmp_file)

However depending on the configuration of the app as it is deployed in production, it may not have access to the /tmp directory. Fortunately Elixir provides a function that returns a writable path for temporary files:

dir = System.tmp_dir!()
tmp_file = Path.join(dir, filename)
File.write!(tmp_file)

From the docs (h System.tmp_dir):

Writable temporary directory.

Returns a writable temporary directory. Searches for directories in the following order:

    1. the directory named by the TMPDIR environment variable
    2. the directory named by the TEMP environment variable
    3. the directory named by the TMP environment variable
    4. C:\TMP on Windows or /tmp on Unix
    5. as a last resort, the current working directory

Returns nil if none of the above are writable.

I prefer to use the “bang” version System.tmp_dir! described as:

Same as tmp_dir/0 but raises RuntimeError instead of returning nil if no temp dir is set.

On my machine it returns something like this:

"/var/folders/66/dj7rwns53vn4db4_1npvqtrh0000gn/T/"

🔒 Serve Phoenix App Locally with HTTPS 🔒

The Phoenix Framework provides an easy mix task to automatically generate a self-signed SSL cert. This is useful if you want to test the app locally with HTTPS.

mix phx.gen.cert

The self-sigend certs will be stored in priv/cert so make sure you add that path to your .gitignore.

When finished the command will prompt you to update your endpoint configuration and a few imporant warnings:

If you have not already done so, please update your HTTPS Endpoint
configuration in config/dev.exs:

  config :tilex, TilexWeb.Endpoint,
    http: [port: 4000],
    https: [
      port: 4001,
      cipher_suite: :strong,
      certfile: "priv/cert/selfsigned.pem",
      keyfile: "priv/cert/selfsigned_key.pem"
    ],
    ...

WARNING: only use the generated certificate for testing in a closed network
environment, such as running a development server on `localhost`.
For production, staging, or testing servers on the public internet, obtain a
proper certificate, for example from [Let's Encrypt](https://letsencrypt.org).

NOTE: when using Google Chrome, open chrome://flags/#allow-insecure-localhost
to enable the use of self-signed certificates on `localhost`.

Precise timings with `monotonic_time`

Monotonic time is time from a clock that only moves forward. The system clock on your CPU can be set and reset. Even when tied to the LAN ntp protocol the system clock can be out-of-sync by a couple of milliseconds. When measuring in microseconds, that’s a lot of time, and time drift can occur at the microsecond level even when attached to NTP, requiring system clock resets.

To get monotonic time in Elixir use, System.monotonic_time:

iex> System.monotonic_time
-576460718338896000
iex> System.monotonic_time
-576460324867892860

It’s ok that this number is negative, it’s always moving positive.

The number has a time unit of :native. To get a duration in millseconds you could convert from :native to millisecond.

iex> event_time = System.monotonic_time
-576459417748861340
iex> System.convert_time_unit(System.monotonic_time - event_time, :native, :millisecond)
38803

Or you could get a millisecond duration by using the one argument of monotonic_time to specify the time unit you want.

iex> event_time = System.monotonic_time(:millisecond)
-576459079381
iex> System.monotonic_time(:millisecond) - event_time
14519

Check out the elixir docs on time for more info.

IO.inspect Label

When doing puts-driven-development in Elixir, IO.inspect/1 and IO.inspect/2 are very useful. These functions return their items unchanged, allowing us to spy on a value in our Elixir code.

I’m a fan of the :label option, which decorates our output with a label, making it easier to understand.

def channel_name_split(channel) do
  channel.name
  |> IO.inspect(label: "channel name before")
  |> String.split("")
  |> IO.inspect(label: "channel name after")
end

And here’s the output in our server log:

channel name before: "workflow"
channel name after: ["", "w", "o", "r", "k", "f", "l", "o", "w", ""]

Check out h IO.inspect in IEX for more info.

telemetry handler detaches automatically on error

Telemetry handlers should be designed to run efficiently over and over again without making a noticeable performance impact on your production system. But what happens if an occurs? Your monitoring should have no bearing on the success of your business logic.

So what happens if your monitoring has an error? With telemetry, it’s ok, the offending handler is just detached, never to run again. Business logic unaffected.

Here’s a test that demonstrates the handler being removed.

handlerFn = fn _, _, _, _ ->
  raise "something"
end

:telemetry.attach(
  :bad_handler,
  [:something_happened],
  handlerFn,
  :no_config
)

# the handler is in ets
assert [[[:something_happened]]] =
         :ets.match(:telemetry_handler_table, {:handler, :bad_handler, :"$1", :_, :_})

:telemetry.execute([:something_happened], %{}, %{})

# the handler is gone!
assert [] = :ets.match(:telemetry_handler_table, {:handler, :bad_handler, :"$1", :_, :_})

You can see the try/catch block in the telemetry source code here.

List all telemetry event handlers

Telemetry is a new library for application metrics and logging in beam applications. It was added to Phoenix in version 1.4.7 released in June 2019.

Telemetry consists of registered handler functions that are executed when specific events occur.

To see a list of what handlers are registered for which events you can call:

:telemetry.list_handlers([])

When returns a list of maps.

To see a list of handlers that have a specific event prefix, you can pass in a prefix as the only argument.

:telemetry.list_handlers([:phoenix, :endpoint])

Which returns:

[
  %{
    config: :ok,
    event_name: [:phoenix, :endpoint, :start],
    function: #Function<2.82557494/4 in Phoenix.Logger.install/0>,
    id: {Phoenix.Logger, [:phoenix, :endpoint, :start]}
  },
  %{
    config: :ok,
    event_name: [:phoenix, :endpoint, :stop],
    function: #Function<3.82557494/4 in Phoenix.Logger.install/0>,
    id: {Phoenix.Logger, [:phoenix, :endpoint, :stop]}
  }
]

`telemetry_event` Overrides Repo Query Event

Ecto gives you a single telemetry event out of the box, [:my_app, :repo, :query], where the [:my_app, :repo] is the telemetry prefix option for ecto.

This event is called whenever any request to the database is made:

  handler = fn _, measurements, _, _ ->
    send(self(), :test_message)
  end

  :telemetry.attach(
    "query",
    [:test_telemetry, :repo, :query],
    handler,
    %{}
  )

  Repo.all(TestTelemetry.Colour)

  assert_receive(:test_message)

This event is overriden when using the the :telemetry_event option, a shared option for all Repo query functions.

  handler = fn _, measurements, _, _ ->
    send(self(), :test_message)
  end

  custom_handler = fn _, measurements, _, _ ->
    send(self(), :custom_message)
  end

  :telemetry.attach(
    "query",
    [:test_telemetry, :repo, :query],
    handler,
    %{}
  )

  :telemetry.attach(
    "custom",
    [:custom],
    custom_handler,
    %{}
  )

  Repo.all(TestTelemetry.Colour, telemetry_event: [:custom])

  assert_receive(:custom_message)
  refute_receive(:test_message)

Which means for any given query you can only broadcast one event. If you have a system that keeps track of expensive queries but you also need to debug a particular query in production, you will take that query out of the system to track expensive queries.

Telemetry Attach and Execute

The telemetry api is simpler than the name tends to imply. There are only two primary functions, attach/4 and execute/4. Check out the telemetry docs to see the full api.

Attach is simple. Essentially, you want a function to be called when a certain event occurs:

handler = fn [:a_certain_event], _measuremnts, _metadata, _config ->
  IO.puts("An event occurred!!")
end

:telemetry.attach(
  :unique_handler_id,
  [:a_certain_event],
  handler,
  :no_config
)

The first argument is handler_id and can be anything be must be unique. The last argument is config and can also be anything. It will be passed along, untouched, to the handler function as the last argument.

Execute is simple. When the program calls execute, the handler that matches the event is called and the measurements and metadata are passed to the handler.

measurements = %{}
metadata = %{}
:telemetry.execute([:a_certain_event], measurements, metadata)

It’s important to note that the event name must be a list of atoms.

A simple test for attach and execute would look like this:

test "attach and execute" do
  handler = fn _, _, _, _ ->
    send(self(), :test_message)
  end

  :telemetry.attach(
    :handler_id,
    [:something_happened],
    handler,
    :no_config
  )

  :telemetry.execute([:something_happened], %{}, %{})

  assert_receive :test_message
end

Accumulating Attributes In Elixir

Typically, if you declare an attribute twice like this:

@unit_of_measure :fathom
@unit_of_measure :stone

The second declaration will override the first:

IO.inspect(@unit_of_measure)
# :stone

But by registering the attribute by calling register_attribute you get the opportunity to set the attribute to accumulate. When accumulating, each declaration will push the declared value onto the head of a list.

defmodule TriColarian do
  @moduledoc false

  Module.register_attribute(__MODULE__, :colors, accumulate: true)

  @colors :green
  @colors :red
  @colors :yellow

  def colors do
    @colors
  end
end

TriColarian.colors()
# [:yellow, :red, :green]

At compile time, perhaps when executing a macro, you have the opportunity to dynamically build a list.

I learned this when Andrew Summers gave a talk on DSLs at Chicago Elixir this past Wednesday. You can see his slides here

`sleep` is just `receive after`

While using sleep in your production application might not be for the best, it’s useful in situations where you’re simulating process behaviour in a test or when you’re trying to diagnose race conditions.

The Elixir docs say this:

Use this function with extreme care. For almost all situations where you would use sleep/1 in Elixir, there is likely a more correct, faster and precise way of achieving the same with message passing.

There’s nothing special about sleep though. The implementation for both :timer.sleep and Process.sleep is equivalent. In Elixir syntax, it’s:

receive after: (timeout -> :ok)

So, it’s waiting for a message, but doesn’t provide any patterns that would successfully be matched. It relies on the after of receive to initiate the timeout and then it returns :ok.

If you’re building out some production code that requires waiting for a certain amount of time, it might be useful to just use the receive after code so that if later you decide that the waiting can be interrupted you can quickly provide a pattern that would match the interrupt like:

timeout = 1000

receive do
  :interrupt -> 
    :ok_move_on_now
  
  after
    timeout -> :ok_we_waited
end

Assert one process gets message from another

Erlang has a very useful-for-testing function :erlang.trace/3, that can serve as a window into all sorts of behaviour.

In this case I want to test that one process sent a message to another. While it’s always best to test outputs rather than implementation, when everything is asynchronous and paralleized you might need some extra techniques to verify your code works right.

pid_a =
  spawn(fn ->
    receive do
      :red -> IO.puts("got red")
      :blue -> IO.puts("got blue")
    end
  end)

:erlang.trace(pid_a, true, [:receive])

spawn(fn ->
  send(pid_a, :blue)
end)

assert_receive({:trace, captured_pid, :receive, captured_message})

assert captured_pid == pid_a
assert :blue == captured_message

In the above example we setup a trace on receive for the first process:

:erlang.trace(pid_a, true, [:receive])

Now, the process that called trace will receive a message whenever traced process receives a message. That message will look like this:

{
  :trace,
  pid_that_received_the_message,
  :receive, # the action being traced
  message_that_was_received
}

This in combination with assert_receive allows you to test that the test process receives the trace message.

Assert Linked Process Raised Error

Linked processes bubble up errors, but not in a way that you can catch with rescue:

test "catch child process error?" do
  spawn_link(fn -> 
    raise "3RA1N1AC"
  end)
rescue 
  e in RuntimeError ->
    IO.puts e
end

This test fails because the error wasn’t caught. The error bubbles up outside of normal execution so you can’t rely on procedural methods of catching the error.

But, because the error causes on exit on the parent process (the test process) you can trap the exit with Process.flag(:trap_exit, true). This flag changes exit behavior. Instead of exiting, the parent process will now receive an :EXIT message.

test "catch child process error?" do
  Process.flag(:trap_exit, true)

  child_pid = spawn_link(fn -> 
    raise "3RA1N1AC"
  end)

  assert_receive {
    :EXIT,
    ^child_pid,
    {%RuntimeError{message: "3RA1N1AC"}, _stack}
  }
end

The error struct is returned in the message tuple so you can pattern match on it and assert about.

This method is still subject to race conditions. The child process must throw the error before the assert_receive times out.

There is a different example in the Elixir docs for catch_exit.

Assert Test Process Did or Will Receive A Message

The ExUnit.Assertions module contains a function assert_receive which the docs state:

Asserts that a message matching pattern was or is going to be received within the timeout period, specified in milliseconds.

It should possibly in addition say “received by the test process”. Let’s see if we can send a message from a different process and assert that the test process receives it:

test_process = self()

spawn(fn ->
  :timer.sleep(99)
  send(test_process, :the_message)
end)

assert_receive(:the_message, 100)
# It Passes!!!

In the above code, :the_message is sent 1 millisecond before the timeout, and the assertion passes.

Now let’s reverse the assertion to refute_receive, and change the sleep to the same time as the timeout.

test_process = self()

spawn(fn ->
  :timer.sleep(100)
  send(test_process, :the_message)
end)

refute_receive(:the_message, 100)
# It Passes!!!

Yep, it passes.

Hiding and Revealing Struct Info with `inspect`

There are two ways to hide information when printing structs in Elixir.

Hiding by implementing the inspect protol.

defmodule Thing do
  defstruct color: "blue", tentacles: 7
end

defimpl Inspect, for: Thing do
  def inspect(thing, opts) do
    "A #{thing.color} thing!"
  end
end

So now in iex I can’t tell how many tentacles the thing has:

> monster = %Thing{color: "green", tentacles: 17}
> IO.inspect(monster, label: "MONSTER")
MONSTER: A green thing!
A green thing!

Note that iex uses inspect to output data which can get confusing.

NEW IN ELIXIR 1.8: You can also hide data with @derive

defmodule Thing do
  @derive {Inspect, only: [:color]}
  defstruct color: "blue", tentacles: 7
end

And now you won’t see tentacles on inspection

> monster = %Thing{color: "green", tentacles: 17}
> IO.inspect(monster)
#Thing<color: "green", ...>

In both cases, you can reveal the hidden information with the structs: false option:

> monster = %Thing{color: "green", tentacles: 17}
> IO.inspect(monster)
#Thing<color: "green", ...>
> IO.inspect(monster, structs: false)
%{__struct__: Thing, color: "green", tentacles: 17}

Custom Validation in Ecto

Sometimes the standard validation options aren’t enough.

You can create a custom validator in Ecto using the validate_change function.

In this particular case, I want to validate that thing has an odd number of limbs. I can pass the changeset to validate_odd_number.

def changeset(thing, attrs) do
  thing
  |> cast(attrs, [
    :number_of_limbs
  ])
  |> validate_odd_number(
    :number_of_limbs
  )
end

And then define validate_odd_number like this:

def validate_odd_number(changeset, field) when is_atom(field) do
  validate_change(changeset, field, fn (current_field, value) ->
    if rem(value, 2) == 0 do
      [{f, "This field must be an odd number"}]
    else 
      []
    end
  end)
end

We pass a function to validate_change that takes the field atom and the value for that field. In that function we test for oddness and return a keyword list that contains the field name and an error message.

Remember that when f is :number_of_limbs:

    [{f, "hi"}] == [number_of_limbs: "hi"]
    # true, these data structures are equal

Ecto's `distinct` adds an order by clause

When using distinct on you may encounter this error:

select distinct on (color) id, color from fruits order by id;
-- ERROR:  SELECT DISTINCT ON expressions must match initial ORDER BY expressions

This is because distinct on needs to enounter the rows in a specific order so that it can make the determination about which row to take. The expressions must match the initial ORDER BY expressions!

So the above query works when it looks like this:

select distinct on (color) id, color from fruits order by color, id;

OK, now it works.

Ecto’s distinct function helps us avoid this common error by prepending an order by clause ahead of the order by clause you add explicitly.

This elixir statement:

Fruit |> distinct([f], f.color) |> order_by([f], f.id) |> Repo.all

Produces this sql (cleaned up for legibility):

select distinct on (color) id, color from fruits order by color, id;

Ecto added color to the order by!

Without any order by at all, distinct does not prepend the order by.

Read the docs!

Transactions can timeout in Elixir

In Ecto, transactions can timeout. So this type of code:

  Repo.transaction fn -> 
    # Many thousands of expensive queries
    # And inserts
  end

This type of code might fail after 15 seconds, which is the default timeout.

In Ecto you can specify what the timeout should be for each operation. All functions that make a request to the database have the same same shared options of which :timeout is one.

Repo.all(massive_query, timeout: 20_000)

The above query now times out after 20 seconds.

These shared options apply to a transaction as well. If you don’t care that a transaction is taking a long time you can set the timeout to :infinity.

  Repo.transaction(fn -> 
    # Many thousands of expensive queries
    # And inserts
  end, timeout: :infinity)

Now this operation will be allowed to finish, despite the time it takes.

Timing A Function In Elixir

Erlang provides the :timer module for all things timing. The oddly named function tc will let you know how long a function takes:

{uSecs, :ok} = :timer.tc(IO, :puts, ["Hello World"])

Note that uSecs is microseconds not milliseconds so divide by 1_000_000 to get seconds.

microseconds are helpful though because sometimes functions are just that quick.

:timer.tc(IO, :puts, ["Hello World"])
# {22, :ok}

You can also call :timer.tc with a function and args:

adding = fn (x, y) ->  x + y end

:timer.tc(adding, [1,3])
# {5, 4}

Or just a function:

:timer.tc(fn -> 
    # something really expensive
    :ok
    end)
# {1_302_342, :ok}