Today I Learned

A Hashrocket project

109 posts about #elixir

CamelCase to underscore and back again w/Elixir

If you have a mix project name with multiple words, then those multiple words are generally separated with underscores in the project directory name, like honey_bears. The module name for the project however is HoneyBears.

Converting a string from underscore to CamelCase is built into Elixir. Its in the Macro module:

> Macro.camelize("honey_bear")
"HoneyBear"

The reverse case also can be solved with a Macro function.

> Macro.underscore("HoneyBear")
"honey_bear"

The Macro module has a number of convience functions for working with macros.

Configure max http header size in Elixir Phoenix

If you store lots of data in cookies, you may eventually run up against the maximum header value length in Cowboy which by default is 4096.

While storing that much data in cookies is maybe not the best idea, there is a configuration setting that can help you work around this issue:

config :myapp, MyAppWeb.Endpoint,                                                                                                                 
  http: [protocol_options: [max_request_line_length: 8192, max_header_value_length: 8192]]

There ya go! Now you can accept http header values up to 8192 bytes in length!

Phoenix will watch your JS for you, just watch!

With es6 javascript ecosystems most compilation/transpilation is done with watchers, programs that watch for changes in your code and then trigger compilation based on the changes you’ve made.

Phoenix has a method of integrating those watchers into the server itself.

config :myapp, MyApp.Endpoint,
  watchers: [yarn: ["run", "watch", cd: Path.expand("../assets", __DIR__)]]

The above configuration goes in config/dev.exs and runs the watch command whenever you start the server/endpoint. The keyword option cd is as of yet undocumented, but does what you think it does, changes the directory to the path you configure, which by convention in Phoenix 1.3 would be the assets directory. cd is the only keyword option.

The watchers start when you run your server. Just make a change to your javascript and refresh the page!

Execute raw parameterized sql with Ecto in Elixir

Using sql directly is a good way to use some of the more interesting parts of sqlthat Ecto does not provide a ready abstraction for in its dsl. And parameterization is necessary to avoid sql injection and just to be able to deal with single quotes correctly. Its fairly straight forward once you find the right module (Ecto.Adapters.SQL) and function (query or query!). Parameters are indicated by $1.

sql = """
  select * from users where name = $1;
"""

results = Ecto.Adapters.SQL.query!(MyApp.Repo, sql, ["chris"])

There is also a stream function on the Ecto.Adapters.SQL module with the same signature as query but the documentation doesn’t necessarily state the advantages or situations where it may be useful. More to learn.f8

Load all records for an association in Ecto

Lets say you get a collection of Posts with:

posts = Repo.all(Post)

If you try to access the associated developer for one of these posts, you get an error:

hd(posts).developer
#Ecto.Association.NotLoaded<association :developer is not loaded>

The same is true for all posts in the collection, none of them have a loaded developer. You can load them one by one with:

developer = Tilex.Repo.one(Ecto.assoc(post, :developer))

But if you need to do this for each developer you’ll get into an N+1 situation, making an additional trip to the database for each post.

Instead you can load all the developers at one time with:

posts = Repo.preload(posts, :developer)

Clearing Phoenix npm cache on Heroku

I recently encountered an issue where the solution, as found on the internet, was to remove all the npm_modules, re-install them and build again. My issue however was on Heroku rather than local.

If you’re using Elixier and Phoenix on Heroku then you’re likely using the heroku-buildpack-phoenix-static. This buildpack compiles the static assets for Phoenix.

This buildpack caches npm_modules by default, to clear the cache you must provide a configuration file at the root level of your project named phoenix_static_buildpack.config that looks like:

clean_cache=true

The buildpack will now download the npm_modules on each deployment.

What applications are running in your Elixir app?

Ever wonder what applications are running in your Elixir or Phoenix app?

Pop open IEx and take a look with Applications.started_applications/1

iex(1)> Application.started_applications
[{:tilex, 'tilex', '0.0.1'},
 {:ueberauth_google, 'An Uberauth strategy for Google authentication.',
  '0.6.0'},
 {:ueberauth, 'An Elixir Authentication System for Plug-based Web Applications',
  '0.4.0'}, {:oauth2, 'An Elixir OAuth 2.0 Client Library', '0.9.1'},
 {:timex,
  'Timex is a rich, comprehensive Date/Time library for Elixir projects, with full timezone support via the :tzdata package.\nIf you need to manipulate dates, times, datetimes, timestamps, etc., then Timex is for you!\n',
  '3.1.24'},
 {:combine, 'A parser combinator library for Elixir projects.', '0.9.6'},
 {:tzdata, 'Tzdata is a parser and library for the tz database.\n', '0.5.12'},
 {:postgrex, 'PostgreSQL driver for Elixir.', '0.13.3'},
 {:db_connection,
  'Database connection behaviour for database transactions and connection pooling\n',
  '1.1.2'},
 {:connection, 'Connection behaviour for connection processes\n', '1.0.4'},
 {:phoenix_html,
  'Phoenix.HTML functions for working with HTML strings and templates',
  '2.10.2'}, {:phoenix_ecto, 'Integration between Phoenix & Ecto', '3.2.3'},
 {:ecto, 'A database wrapper and language integrated query for Elixir.\n',
  '2.1.4'}, {:poolboy, 'A hunky Erlang worker pool factory', '1.5.1'},
 {:decimal, 'Arbitrary precision decimal arithmetic.', '1.4.0'},
........

Enable history in IEX through #erlang (OTP 20) ⏳

If you are using the latest version of Erlang, OTP 20 now ships with shell history, so you can use Ctrl-p / Ctrl-n or the up/down arrow keys.

The shell history is turned off by default though, so you will have to turn it on by adding the following to your .zshrc/.bashrc etc.

export ERL_AFLAGS="-kernel shell_history enabled"

Once you do that make sure to source your bash config file or open a new window.

Every subsequent iex session will now have shell history. 🚀

Setting breaks in IEx for debugging

With the release of Elixir 1.5 comes some handy new IEx helpers, one of which is the ability to add break points throughout your code.

You can add a break to any function using break!/2 or break!/4:

defmodule MyModule do
  def hello(name) do
    "hello " <> name
  end
end
iex(1)> break!(MyModule.hello/ 1)
or
iex(1)> break!(MyModule, :hello, 1)

Both break/2 and break/4 accept an additional argument for how many stops you want to make. Useful for recursive functions where you may want to stop multiple times.

To see what breaks you have use breaks/0

iex(1)> breaks()
 ID   Module.function/arity   Pending stops
---- ----------------------- ---------------
 1    MyModule.hello/1        1

Now when you call the function, you’ll be placed into the a debugger and you can inspect whats being passed in:

iex(4)> MyModule.hello("world")
Break reached: MyModule.hello/1 (lib/my_module.ex:2)
    1: defmodule MyModule do
    2:   def hello(name) do
    3:     "hello " <> name
    4:   end
pry(1)> name
"world"

To exit the break and start a new shell process use respawn/0

pry(2)> respawn
Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
"hello world"
iex(1)>

GenServer child_spec/1 in Elixir 1.5

Elixir 1.5 GenServer introduces overridable child_spec/1. Now instead of, in your application supervisor, calling;

# MyExample.Application
def start(_type, _args) do
  children = [
    worker(MyExample.MyChild, [], restart: :permanent, shutdown: 5000)
  ]
end

You can now let the child decide how its supposed to be implemented by overriding child_spec/1 in the child.

#MyExample.Application
def start(_type, _args) do
  children = [ 
    MyExample.MyChild
   ]
end

# MyExample.MyChild
def child_spec(_args) do
  %{
    id: __Module__,
    start: { __Module__, :start_link, []},
    restart: :permanent,
    shutdown: 5000,
    type: :worker
   }
end

You can view the defaults that child_spec/1 implements in the source code.

Arguments can be passed to child_spec/1 which can then be used for pattern matching and custom configurations based on the supervisor accessing it:

#MyExample.Application
def start(_type, _args) do
  children = [ 
    {MyExample.MyChild, "temporary"}
   ]
end

# MyExample.MyChild
def child_spec("temporary") do
  %{
    id: __Module__,
    start: { __Module__, :start_link, []},
    restart: :temporary,
    shutdown: 5000,
    type: :worker
   }
end

def child_spec(_) do
  %{
    id: __Module__,
    start: { __Module__, :start_link, []},
    restart: :permanent,
    shutdown: 5000,
    type: :worker
   }
end

Terminate incomplete expression in IEx

Occasionally you get stuck in an incomplete expression while in IEx. Instead of figuring out which character you need to type to terminate the expression, you can terminate early with #iex:break

iex(1)> ["ab
...(1)> c"
...(1)> "
...(1)> ]
...(1)> #iex:break
** (TokenMissingError) iex:1: incomplete expression
iex(1)>

Another shortcut to achieve the same result is hit ctrl-g (User switch command), i (interrupt job), c (connect to job). This puts you right back in IEx.

iex(1)> ["ab"
iex(1)>   <<<ctrl-g typed here>>>
User switch command
 --> i
 --> c
Bug Bug ..!!** (EXIT) interrupted
iex(1)>

Require local package in mix.exs

In Elixir as you are writing your application it is recommended to split parts of it into smaller applications (also can be called micro-services if you want to be buzzword compliant).

You don’t however need to publish those dependencies to the Hex package manager in order to load them, instead you can use the path argument when defining a dependency.

In this example we have our main application Foo and in the directory above it we have an application called Bar.

To make the Bar module available in Foo we can do it like so:

mix.exs

defp deps do
  [
    {:bar, path: "../bar"},
  ]
end

Module attribute constants nil in Elixir

Module attributes in elixir (@something) can be used as constants and assigned a value, however one must make sure that the value assigned to the constant is available at compile time.

For example when using a constant that loads environment variables if the value is not available in compile time they will resolve to nil.

@salt System.get_env(:your_app_name, "AUTH_SALT")

So either export the environment variables before compiling, or don’t use module attributes, instead you can use a function:

defp salt do
  System.get_env(:your_app_name, "AUTH_SALT")
end

Consider:

  • The downside to using a function is that it will be re-evaluated each time.
  • The downside of exporting env-vars before compile is that they you might forget and your app will crash in production. You can circumvent that by writing a script for compilation.
  • You can also call System.get_env from your config.exs but make sure to run mix clean after doing so since the compiler seems to cache the config compilation.

Remove, Get, And Compile a dependency

To get a better feel for how a dependency works I’ll do some IO.puts style debugging, but when I’m done I need to clean up all those IO.puts statements.

mix deps.clean ecto

The above command will remove two directories, the _build/dev/lib/ecto dir and the deps/ecto dir.

mix deps.get ecto

This command will resolve the dependency tree and get the appropriate version of ecto based on the information in your mix.exs file. It will not reach out to the internet if the package is already cached as a tar file in the .hex/packages/hexpm dir. It will unpack the tar file into the local deps/ecto dir but it will not compile it.

mix deps.compile ecto

Will compile ecto to the _build/dev/lib/ecto directory.

Logging to a file in Elixir test environment

The default configuration for a logger in the test env is:

config :logger, level: :warn

That is, of the four logging levels (defug, info, warn and error) only log warnings and errors.

By default this logger logs to the console making the below redundant:

config :logger, level: :warn, backends: [:console]

:console is the only option available by default, but there are other backends available via hex package.

With hex package logger_file_backend you might have a different config:

config :logger, backends: [{LoggerFileBackend, :error_log}]

config :logger, :error_log, path: 'log/here.log'

Compile an Elixir dependency as you change it

In an Elixir project, the dependencies are one-time compiles. That makes sense when you know the dependency is not going to change. It stops making sense, however, when you need to explore how a dependency works by adding debug statements like IO.puts. Or when you find something in a dependency you want to change and need to test that change in an application.

In those cases Elixir provides the facility to compile a dependency each time it has a change in one of the component files just like how compilation in your mix app works. You can enable this by providing a path to the location of the dependency in the mix.exs file:

defp dep do
  [
  ...
  {:wallaby, '~> 0.16.1', path: 'deps/wallaby'}
  ...
  ]
end

In this case the path is set to the local deps dir, where the dependency already exists, but you can also set it to a working directory for that library.

Create A List Of Atoms

The ~w sigil makes it easy to create a word list — a list of strings — where each word is separated by a space.

> ~w(bulbasaur charmander squirtle)
["bulbasaur", "charmander", "squirtle"]

By appending an a onto that sigil construct, you are instructing Elixir that you would instead like a list of atoms.

> ~w(bulbasaur charmander squirtle)a
[:bulbasaur, :charmander, :squirtle]

source

Serve static files/directories in Phoenix

Phoenix will by default server some files and directories from priv/static. More specifically css/, fonts/, images/, js/, favicon.ico and robots.txt. If you need to add a new directory however simply creating it in priv/static will not make Phoenix serve it. You need to explicitly add it.

Say your project is called ChicagoElixir you will need to go to your endpoint configuration, typically in: lib/chicago_elixir/web/endpoint.ex. There you will find the following configuration:

  plug Plug.Static,
    at: "/", from: :chicago_elixir, gzip: false,
    only: ~w(css fonts images js favicon.ico robots.txt)

Simply add the new folder or file name to the list in only and restart your Phoenix server.

Inspecting The Process Message Queue

A core tenant of Elixir is message passing between processes. So, if a process is sent a message, where does that message go? What happens if it gets sent many messages? The Process.info/2 function allows us to inspect the message queue.

First, let’s send some messages (to ourself) and then keep an eye on the length of the message queue as we go.

> send self(), {:error, "this is bad"}
{:error, "this is bad"}
> Process.info(self(), :message_queue_len)
{:message_queue_len, 1}
> send self(), {:hello, "world"}
{:hello, "world"}
> Process.info(self(), :message_queue_len)
{:message_queue_len, 2}

Now, I am curious what those specific messages are. Let’s ask Process.info/2 for the messages that are in the message queue.

> Process.info(self(), :messages)
{:messages, [error: "this is bad", hello: "world"]}

There are a lot of other things that Process.info/2 can tell us about a process. See the Erlang docs for process_info for more details.

Get the association with Ecto

In the schema of my post model I have this line:

belongs_to :developer

So a post is associated with a developer. When I have a post struct and try to access the developer I might see this:

> Tilex.Repo.get(Tilex.Post, 42).developer
#Ecto.Association.NotLoaded<association :developer is not loaded>

Getting the developer is now a two step process:

  1. Construct a query based on the association
  2. Use that query to acquire the struct

To construct a query based on the association we can use the Ecto.assoc/2 function:

query = Ecto.assoc(post, :developer)

Then this query is executed with the Repo.one function:

developer = Tilex.Repo.one(query)

There ya go!

erlang memory usage

You might be be curious how much memory your erlang vm is taking. You can get that and more from :erlang.memory. Below is what I get from running this after starting iex normally.

> :erlang.memory
[total: 18133840, processes: 4975616, processes_used: 4974616, system: 13158224,
 atom: 264529, atom_used: 256719, binary: 228728, code: 6640396, ets: 414904]

This list is confusing. All numbers are bytes, so in total 18133840 bytes are allocated to the erlang process. There are two subcategories processes and system and the values of those categories sum to equal the value of total.

The remaining categories are either a component of system or a component of processes.

Listing Files In IEx

When you start an IEx session, you do so in the context of some directory — the current working directory. This context can be important if you need to do something like import a file. In fact, you may want to know what files are available in the current working directory.

You can list them all out within IEx using ls/0.

iex(1)> ls()
           .git     .gitignore      README.md         _build         assets         config
           deps            lib        mix.exs       mix.lock           priv           test
            tmp

You can also list the contents of some other specific directory by naming it when invoking ls/1.

See h() within IEx for more details.

Use Elixir plug for only some controller actions

Phoenix allows for a flexible request processing pipeline with plugs. Your controllers are basically plugs in the plug pipeline. You can specify plugs that will execute before your controller actions with the plug macro.

defmodule MyApp.ThingController do
  use MyApp.Web, :controller

  plug MyApp.InterestingPlug
end

You can also specify a guard for this macro that will only enable this plug to be run before certain actions:

plug MyApp.InterestingPlug when action in [:new, :create]

Defining Multiple Clauses In An Anonymous Function

Anonymous functions often take the approach of doing a single thing with the inputs, regardless of their shape or values. There is no need to limit ourselves though. The same pattern matching that we use all over our Elixir programs can be utilized to define multiple clauses in an anonymous function as well.

Consider the following example:

iex> my_function = fn
  {:ok, x} -> "Everything is ok: #{x}"
  {:error, x} -> "There was an error: #{x}"
end
#Function<6.52032458/1 in :erl_eval.expr/5>

We can then invoke our anonymous function using the bound variable to see what results we get with different kinds of inputs.

iex> my_function.({:ok, 123})
"Everything is ok: 123"
iex> my_function.({:error, "be warned"})
"There was an error: be warned"

source

The Registry is back

“The Registry” is a bad Windows dream, something left behind when I switched off Windows for good circa 2011.

It’s back. It’s a new Elixir 1.4 module where you can store values that you’d like to access globally. Potentially, you want to store a pid of a worker you started. You can store that in the registry like so.

{:ok, pid} = MyWorker.start_link()
supervisor(Registry, [:unique, Registry.WorkerPids])
Registry.register(Registry.WorkerPids, :my_worker, worker_pid)
[{_, found_worker_pid}] = Registry.lookup(Registry.WorkerPids, :my_worker)

A lot is going on here.

  • We start a worker
  • We start the registry process for register Registry.WorkerPids
  • We register the key/value in a specific register (Registry.WorkerPids)
  • We lookup the value in that register with that key

Its important to note that the Registry.WorkerPids symbol is just an atom, not a module. There is no WorkerPids module implementing any special functionality anywhere.

Definitely a lot of ceremony for a key/value store. I expect to find practical uses for it as I experience more complex Elixir systems.

Erlang documentation with `erl -man`

Erlang can be tricky, so documentation is a must, but when you’re 30,000 feet high in an airplane with no internet you might regret not having downloaded DashApp and with it all the Erlang documentation.

But it was on your machine the whole time! You have access to all the Erlang documentation with erl -man used like so:

erl -man calendar

Which pops open a man page for the calendar module.

When you are learning Erlang the docs are at your fingertips.

Remove One List From Another

The --/2 operator allows you to subtract two lists, that is, remove all elements in the right list from the left list. Each occurrence of an element is removed if there is a corresponding element. If there is no corresponding element, it is ignored.

Here are some examples.

> [1, 2, 3] -- [2, 4]
[1, 3]
> [:a, :b, :c, :a, :d, :a] -- [:a, :a]
[:b, :c, :d, :a]

This kind of list operation is not particularly efficient, so for large lists it can be quite slow. The following example took several minutes to run.

> Enum.into(1..1000000, []) -- Enum.into(2..1000000, [])
[1]

To achieve a true set difference, you’ll note that the docs for this operator recommend checking out MapSet.difference/2.

See h Kernel.-- for more details.

Inspect with Label

IO.inspect returns the value you pass in as the first argument. This is great when you want to debug the middle state of a series of piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Here, inspect will write to stdout the values [{"A", 1}, {"B", 2"}...].

IO.inspect also takes a label option, which decorates the values written to stdout but does not disrupt the piped calls.

"ABCDEFG"
|> String.graphemes
|> Enum.each_with_index
|> IO.inspect(label: "Letters with index:")
|> Enum.map(fn({letter, i}) -> "#{letter}#{i}" end)
|> Enum.join

Which outputs Letters with index: [{"A", 1}, {"B", 2"}...] to stdout.

Jaro Distance determines "closeness" of 2 strings

In Elixir, the String module includes a function named jaro_distance that determines the similarity between two strings:

iex> String.jaro_distance("Chris", "Crhis")
0.9333333333333332
iex> String.jaro_distance("Chris", "Bob")
0.0
iex> String.jaro_distance("Chris", "Dennis")
0.5777777777777778

The docs say this works best with short strings. The Jaro distance article on wikipedia has lots of math if you’re into that.

Referencing Values In IEx's History

Each time we execute a statement in an iex session, the counter is incremented. These numbers are references to the history of the session. We can use these references to refer to previously executed values using v/1. This is particularly handy for multi-line statements or when we forget to bind to the result of some function.

Consider the following iex session:

iex(1)> :one
:one
iex(2)> 1 + 1
2
iex(3)> "three" |> String.to_atom()
:three

If we execute v() on its own, it is the same as v(-1) in that it will give us the latest value in the history.

iex(4)> v()
:three

Providing any positive number will refer to the references we see next to each statement.

iex(5)> v(1)
:one

Negative numbers, as we saw with v(-1), will count backwards in the history from where we are.

iex(6)> v(-4)
2

See h v for more details.

Creating A PID

Often times, when invoking a function that spawns a process, the PID of the spawned process is returned and we bind to it. That PID is a reference to some BEAM process in our system.

We can create our own references using the pid/3 function.

Let’s assume we have the following processes, among others, in our system at the moment.

> Process.list |> Enum.reverse |> Enum.take(3)
[#PID<0.284.0>, #PID<0.283.0>, #PID<0.282.0>]

We can create a reference to any of them using the three number parts that they are made up of.

> pid(0, 284, 0)
#PID<0.284.0>

See, it’s alive.

> pid(0, 284, 0) |> Process.alive?
true

What if we make up a PID that doesn’t actually reference any process?

> pid(0, 333, 0) |> Process.alive?
false

Note: there is also a pid/1 version of the function. See h pid for more details.

Specifying The Phoenix Server Port

Running mix phx.server for a Phoenix project with the default settings will attach the server to port 4000.

If you’d like to use a different port in development, you can change it in config/dev.exs.

config :my_app, MyApp.Web.Endpoint,
  http: [port: 4444],
  ...

Alternatively, you can allow it to be configurable from the command line with an environment variable and a fallback port.

config :my_app, MyApp.Web.Endpoint,
  http: [port: System.get_env("PORT") || 4000],
  ...

Running

$ PORT=4444 mix phx.server

will launch the server on port 4444.

Compute md5 Hash Of A String

To compute the md5 digest of a string, we can use Erlang’s top-level md5 function.

> :erlang.md5("#myelixirstatus")
<<145, 148, 139, 99, 194, 176, 105, 18, 242, 246, 37, 69, 142, 69, 226, 199>>

This, however, gives us the result in the raw binary representation. We would like it in a base 16 encoding, as md5 digests tend to be.

We can wrap (or pipe) this with Base.encode16 to get the result we are looking for.

> Base.encode16(:erlang.md5("#myelixirstatus"), case: :lower)
"91948b63c2b06912f2f625458e45e2c7"

source

Counting Records With Ecto

Sometimes you want to know how many records there are in a table. Ecto gives us a couple ways to approach this.

We can use the count1) function that the Ecto query API provides.

> Repo.one(from p in "people", select: count(p.id))

16:09:52.759 [debug] QUERY OK source="people" db=1.6ms
SELECT count(p0."id") FROM "people" AS p0 []
168

Alternatively, we can use the fragment/1 function to use PostgreSQL’s count function.

> Repo.one(from p in "people", select: fragment("count(*)"))

16:11:19.818 [debug] QUERY OK source="people" db=1.5ms
SELECT count(*) FROM "people" AS p0 []
168

Lastly, Ecto.Repo has the aggregate/4 function which provides a :count option.

> Repo.aggregate(from(p in "people"), :count, :id)

16:11:23.786 [debug] QUERY OK source="people" db=1.7ms
SELECT count(p0."id") FROM "people" AS p0 []
168

Unique Indexes With Ecto

You can create a unique index in a migration for one or more columns using the unique_index/3 function.

For example, if you are creating a join table for followers and want to ensure that duplicate follower entries are prevented, you may want to include a unique index like so:

create table(:followers) do
  add :followed_user, references(:users), null: false
  add :following_user, references(:users), null: false
end

create unique_index(:followers, [:followed_user, :following_user])

Keep in mind that unique_index/3 is a shorthand for index/3 when you set unique: true.

Responding with :stop allows call of `terminate`

A GenServer in Elixir has two lifecycle methods, init and terminate. init is called when the GenServer is started with GenServer.start_link.

What I learned today is that there is a terminate method in case there’s any resources that need to be cleaned up when the GenServer is shut down. The terminate method is called when the :stop message is returned from either handle_cast or handle_call:

defmodule Cache do
  use GenServer
  
  #... a lot of other code

  def handle_cast({:something, 1}, state) do
    IO.puts "This executes first"
    {:stop, "This is my reason for stopping", state}
  end

  def terminate(reason, state)
    IO.puts "Then this executes"
  end
end

Creating Indexes With Ecto

Using indexes in the right places within relational databases is a great way to speed up queries.

To add a basic index in an Ecto migration, use Ecto.Migration.index\2:

create index(:users, [:email])

Creating a composite index doesn’t require jumping through any hoops; just put the relevant column names in the list:

create index(:posts, [:user_id, :title])

See h Ecto.Migration.index for more details.

Custom Phoenix Validations

Today I learned how to write a custom Phoenix model validation. It came from the Phoenix port of this very application, which requires that posts be 200 words or less.

Phoenix doesn’t have a word length validation, so I had to make my own. Here it is, minus some irrelevant lines of code:

# web/models/post.ex

def changeset(struct, params \\ %{}) do
  struct
  |> cast(params, [:title, :body, :channel_id])
  |> validate_length_of_body
end

defp validate_length_of_body(changeset) do
  body = get_field(changeset, :body)
  validate_length_of_body(changeset, body)
end

defp validate_length_of_body(changeset, body) do
  if length(String.split(body, ~r/\s+/)) > 200 do
    add_error(changeset, :body, "should be at most 200 word(s)")
  else
    changeset
  end
end

When the post body is greater than 200 words (as defined by splitting on whitespace characters), the validation adds an error to the body field that mimics the errors Phoenix provides for character-length validation failures. Including the (s), which I kind of like.

When the body is short enough, changeset is returned from the function of the same name, a form of success.

Pattern matching FTW.

Requiring Keys For Structs

When defining a struct in Elixir, we may want to ensure that certain values are provided. We can require that certain keys are used in the creation of a struct with the @enforce_keys attribute. Here is an example of a Name struct.

defmodule Name do
  @enforce_keys [:first, :last]
  defstruct [:first, :middle, :last]
end

With this defined, we can create a struct that uses all of the keys.

> jack = %Name{first: "Jack", middle: "Francis", last: "Donaghy"}
%Name{first: "Jack", last: "Donaghy", middle: "Francis"}

We can also ignore :middle and just provide the required keys.

> liz = %Name{first: "Liz", last: "Lemon"}
%Name{first: "Liz", last: "Lemon", middle: nil}

We cannot, however, omit any of the keys specified in @enforce_keys. If we do omit any of them, Elixir will raise an error.

> tracy = %Name{first: "Tracy"}
** (ArgumentError) the following keys must also be given when building struct Name: [:last]
    expanding struct: Name.__struct__/1
    iex:6: (file)

Virtual Fields With Ecto Schema

If you’d like to include a particular key-value pair in an Ecto changeset, it needs to be included as a field in the schema. In the case of something akin to a password field, you want to be able to perform validations against it, but the password itself does not have a column in the database. In other words, you want to use the password in memory as part of the validation process but not save it to the database. To accomplish this, you need to specify that it is a virtual field.

schema "users" do
  field :username, :string
  field :password_digest, :string
  field :password, :string, virtual: true
end

With that schema, you can then validate the :password and transform it into the corresponding :password_digest field.

def registration_changeset(model, params) do
  model
  |> changeset(params)                  # do other standard validations
  |> cast(params, [:password])          # include :password in the changeset
  |> validate_length(:password, min: 8) # validations
  |> put_pass_hash()                    # transform into :password_digest
end

Pivoting a 2-dimensional list

In one of the exercism.io exercises, a grid is represented by a list of lists where each inner list represents a row.

The exercise then asks you to pivot the grid, that is, transform the list of lists so that each list represents a column.

From someone else’s solution I learned that an easy way to conduct this transform is to use List.zip and Tuple.to_list.

grid = [[1,2,3, 4], [1,2,3, 4], [1, 2, 3, 4]]
List.zip(grid) 
|> Enum.map(&Tuple.to_list/1)
# [[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4]]

Tuple’s to_list method is needed because List.zip returns an array of tuples.

Merge maps with a callback

Merging two maps together is something I’m familiar with:

iex> Map.merge(%{a: 1, b: 2}, %{c: 3})
%{a: 1, b: 2, c: 3} 

But this function suffers from a unilateral decision that’s made when a key in the first argument also exists in the second argument. The value from the first map always overwrites the value from the second.

iex> Map.merge(%{a: 1, b: 2, c: 100}, %{c: 3})
%{a: 1, b: 2, c: 100}

But Elixir’s Map module has a merge function that takes a function as an additional argument. It lets you decide what to do in case of a key conflict. You could write this function so that the second argument overwrites the first or better yet add the two values together.

iex> Map.merge(%{a: 1, b: 2, c: 100}, %{c: 3}, fn(k, v1, v2) -> v1 + v2 end)
%{a: 1, b: 2, c: 103}

Three data types that go `into` a Map

Enum.into can be used to put different types of things into different types of collections. For instance I can put elements from one list into another list.

iex> Enum.into([4], [1,2,3])
[1, 2, 3, 4]

It works with Maps too and there’s different types of things than can go into a map. A map can certainly go into another map:

iex> Enum.into(%{"a" => 1}, %{"b" => 2})
%{"a" => 1, "b" => 2}

Keyword lists also go into a map:

iex> Enum.into([a: 1], %{"b" => 2})
%{:a => 1, "b" => 2}

And lists of tuples also go into a map:

iex> Enum.into([{"a", 1}], %{"b" => 2})
%{"a" => 1, "b" => 2}

But only when there are two elements in the tuple:

iex(32)> Enum.into([{"a", 1, 3}], %{"b" => 2})
** (ArgumentError) argument error
    (stdlib) :maps.from_list([{"a", 1, 3}])
    (elixir) lib/enum.ex:1072: Enum.into/2

Using assigned value later in same pattern match

I have an array of character lists.

['apple', 'aardvark']

And I want to make sure that each of these two items has the same beginning letter.

I can do this with pattern matching because variables can be reused in an assignment.

iex > [[c | _], [c | _]] = ['apple', 'aardvark']
['apple', 'aardvark']
iex > c
97

Here’s what happens when the first letter does NOT match.

iex > [[c | _], [c | _]] = ['apple', 'hamster']
** (MatchError) no match of right hand side value: ['apple', 'hamster']

Render A Template To A String

Templates in a Phoenix application ultimately get compiled to functions that can be quickly rendered with the necessary data. We can take a look at how a template will be rendered using Phoenix.View.render_to_string/3.

First, we need a template:

# user.html.eex
<h1><%= @user.first_name %></h1>
<h5><%= @user.username %> (<%= @user.email %>)</h5>

We can then render that template for the view with some user:

> user = %User{first_name: "Liz", last_name: "Lemon", username: "llemon", email: "lizlemon@nbc.com"}
%MyApp.User{...}

> Phoenix.View.Render_to_string(MyApp.UserView, "user.html", user: user)
"<h1>Liz</h1>\n<h5>llemon (lizlemon@nbc.com)</h5>\n"