Elixir: Notes about with

Posted on Thu 08 October 2020 in Elixir

This post explains how to use the keyword with for conditional code.

Introduction

Let's discuss this example from the Joy of Elixir book:

# readfile_case.ex
file_data = %{name: "haiku.txt"}
case Map.fetch(file_data, :name) do
  {:ok, name} ->
    case File.read(name) do
      {:ok, contents} ->
        line = contents
               |> String.split("\n", trim: true)
               |> Enum.map(&String.reverse/1)
               |> Enum.join("\n")

        IO.puts "#{inspect line}"

      {:error, :enoent} ->
        IO.puts "Could not find a file called #{name}"
    end
  :error -> "No key called :name in file_data map"
end

The structure of the code is as follows:

case expression_a do
  pattern ->
    case expression_b do
      pattern_ok ->
        do_something
      pattern_error_b ->
        do_something
    end
  pattern_error_a -> do_something
end

In this case, we read a text file called "haiku.txt":

  • If the map file_data contains the key :name, we pass the first case.
  • If we can read the file, then we pass the second case and return a single string.
  • Otherwise, we evaluate the clause of the pattern that matches the result of the case.

This is the output of the script:

iex(1)> c "readfile_case.exs"
"I love Elixir\nIt is so easy to learn\nGreat functional code"

We can improve the readability of that block of code by using with. The modified code is given below:

# readfile_with.exs
file_data = %{name: "haiku.txt"}
with {:ok, name} <- Map.fetch(file_data, :name),
     {:ok, contents} <- File.read(name) do
     line = contents
            |> String.split("\n", trim: true)
            |> Enum.map(&String.reverse/1)
            |> Enum.join("\n")

     IO.puts "#{inspect line}"
else
  :error -> ":name key missing in file_data"
  {:error, :enoent} -> "Couldn't read file"
end

The output is the same as above:

iex(1)> c "readfile_with.exs"
"I love Elixir\nIt is so easy to learn\nGreat functional code"

We can say that the latter code is easier to read. Its structure is:

with expression_a,
     ...,
     expression_n do

     do_something
else
  pattern_error_a -> do_something_with_a
  ...
  pattern_error_n -> do_something_with_n
end

In short, with allows us to specify all conditions that must be met before doing a given task (e.g., do_something). If any of these conditions fail, we then evaluate one of the clauses in the else block. In the previous example, it is more intuitive for us that we need to meet two conditions for reading the file:

with {:ok, name} <- Map.fetch(file_data, :name),    # if file_data contains the key :name and
     {:ok, contents} <- File.read(name) do          # if we can read the file
        # work with the contents of the file

Notice that we use the <- operator to define each expression. We could rather use =, but it will do something else. We discuss it at the end of the post. If Map.fetch fails, then we obtain an atom :error. If File.read fails, then we get a tuple {:error, msg}, like {:error, :enoent}. We can handle these errors in the else block:

with ... do
     do_something
else
  :error -> ":name key missing in file_data"
  {:error, :enoent} -> "Couldn't read file"
end

Make it fail

Let's look for :filename rather than :name in file_data:

# readfile_with.exs
file_data = %{name: "haiku.txt"}
with {:ok, name} <- Map.fetch(file_data, :filename),   # modified
     {:ok, contents} <- File.read(name) do
     line = contents
            |> String.split("\n", trim: true)
            |> Enum.map(&String.reverse/1)
            |> Enum.join("\n")

     IO.puts "#{inspect line}"
else
  :error -> ":name key missing in file_data"
  {:error, :enoent} -> "Couldn't read file"
end

We got an error message, as expected:

iex(1)> c "readfile_with.exs"
:name key missing in file_data

Now, suppose we are interested in reading other file, say "poem.txt":

# readfile_with.exs
file_data = %{name: "poem.txt"}                  # modified
with {:ok, name} <- Map.fetch(file_data, :name),
     {:ok, contents} <- File.read(name) do
     line = contents
            |> String.split("\n", trim: true)
            |> Enum.map(&String.reverse/1)
            |> Enum.join("\n")

     IO.puts "#{inspect line}"
else
  :error -> IO.puts ":name key missing in file_data"
  {:error, :enoent} -> IO.puts "Couldn't read file"
end

Again, the code works as expected and shows a proper message:

iex(2)> c "readfile_with.exs"
Couldn't read file

Some gotchas

Use <- instead of =

One thing that I noted when using with for the first time is the <- operator. Indeed, we can use either <- or = in the previous example. Although, there is a subtle difference. Let's replace <- with = in the previous example:

# readfile_with.exs
file_data = %{name: "haiku.txt"}
with {:ok, name} = Map.fetch(file_data, :name),    # modified
     {:ok, contents} = File.read(name) do          # modified
     line = contents
            |> String.split("\n", trim: true)
            |> Enum.map(&String.reverse/1)
            |> Enum.join("\n")

     IO.puts "#{inspect line}"
else
  :error -> ":name key missing in file_data"
  {:error, :enoent} -> "Couldn't read file"
end

Now, we get this warning:

iex(1)> c "readfile_with.exs"
warning: "else" clauses will never match because all patterns in "with" will always match
  readfile_with.exs:3

"I love Elixir\nIt is so easy to learn\nGreat functional code"

Actually, we can change the code to introduce an error. Let's change the name of the text file again:

# readfile_with.exs
file_data = %{name: "poem.txt"}                     # modified
with {:ok, name} = Map.fetch(file_data, :name),     # modified
     {:ok, contents} = File.read(name) do           # modified
     line = contents
            |> String.split("\n", trim: true)
            |> Enum.map(&String.reverse/1)
            |> Enum.join("\n")

     IO.puts "#{inspect line}"
else
  :error -> ":name key missing in file_data"
  {:error, :enoent} -> "Couldn't read file"
end

This time, we also get a compilation error. This error is due the result of File.read(name) does not match{:ok, contents}. In this case, the result is {:error, :enoent}:

iex(1)> c "readfile_with.exs"
warning: "else" clauses will never match because all patterns in "with" will always match
  readfile_with.exs:3


== Compilation error in file readfile_with.exs ==
** (MatchError) no match of right hand side value: {:error, :enoent}
    readfile_with.exs:4: (file)
    (elixir) lib/kernel/parallel_compiler.ex:229: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7
** (CompileError)  compile error
    (iex) lib/iex/helpers.ex:200: IEx.Helpers.c/2

Now, let's replace :name with :filename:

# readfile_with.exs
file_data = %{name: "haiku.txt"}
with {:ok, name} = Map.fetch(file_data, :filename),   # modified
     {:ok, contents} = File.read(name) do             # modified
     line = contents
            |> String.split("\n", trim: true)
            |> Enum.map(&String.reverse/1)
            |> Enum.join("\n")

     IO.puts "#{inspect line}"
else
  :error -> ":name key missing in file_data"
  {:error, :enoent} -> "Couldn't read file"
end

Again, we got an error. This time, the result of Map.fetch(file_data, :filename) does not match {:ok, name}. The result is :error:

iex(1)> c "readfile_with.exs"
warning: "else" clauses will never match because all patterns in "with" will always match
  readfile_with.exs:3


== Compilation error in file readfile_with.exs ==
** (MatchError) no match of right hand side value: :error
    readfile_with.exs:3: (file)
    (elixir) lib/kernel/parallel_compiler.ex:229: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7
** (CompileError)  compile error
    (iex) lib/iex/helpers.ex:200: IEx.Helpers.c/2

Note that if we use = instead of <-, the else block is ignored. From Programming Elixir, p. 38:

In the previous example, the head of the with expression used = for basic pattern matches. If any of these had failed, a MatchError exception would be raised. But perhaps we'd want to handle this case in a more elegant way. That's where the <- operator comes in. If you use <- instead of = in a with expression, it performs a match, but if it fails it returns the value that couldn't be matched.

Let's check our previous errors. In the first case, the resulting value is :error since :filename is not in the map:

iex(1)> file_data = %{name: "haiku.txt"}
%{name: "haiku.txt"}
iex(2)> with {:ok, name} <- Map.fetch(file_data, :name), do: name        # success
"haiku.txt"
iex(3)> with {:ok, name} <- Map.fetch(file_data, :filename), do: name    # failure
:error

In the second case, we tried to read a file that does not exist. Hence, the resulting value is {:error, :enoent}:

iex(6)> with {:ok, contents} <- File.read("haiku.txt"), do: contents    # success
"rixilE evol I\nnrael ot ysae os si tI\nedoc lanoitcnuf taerG"
iex(7)> with {:ok, contents} <- File.read("poem.txt"), do: contents     # failure
{:error, :enoent}

We can use the resulting value of with to match one of the clauses in the else block. Although, I noted that Dave did not mention that in his book. Let's assign the result of with to a variable and print it:

# readfile_with.exs
file_data = %{name: "haiku.txt"}
result = with {:ok, name} <- Map.fetch(file_data, :name),
              {:ok, contents} <- File.read(name) do
                    contents
                    |> String.split("\n", trim: true)
                    |> Enum.map(&String.reverse/1)
                    |> Enum.join("\n")
         end

IO.puts result

This is the result:

iex(3)> c "readfile_with.exs"
I love Elixir
It is so easy to learn
Great functional code

Now, introduce an error:

# readfile_with.exs
file_data = %{name: "poem.txt"}                            # invalid filename
result = with {:ok, name} <- Map.fetch(file_data, :name),
              {:ok, contents} <- File.read(name) do        # <- operator
                    contents
                    |> String.split("\n", trim: true)
                    |> Enum.map(&String.reverse/1)
                    |> Enum.join("\n")
         end

IO.puts "#{inspect result}"     # do not use IO.puts result, it will throw an exception 
                                # because result may not be a string

This is the result:

iex(6)> c "readfile_with.exs"
{:error, :enoent}

Now, use another key:

# readfile_with.exs
file_data = %{name: "haiku.txt"}
result = with {:ok, name} <- Map.fetch(file_data, :filename),   # invalid key
              {:ok, contents} <- File.read(name) do
                    contents
                    |> String.split("\n", trim: true)
                    |> Enum.map(&String.reverse/1)
                    |> Enum.join("\n")
         end

IO.puts "#{inspect result}"     # do not use IO.puts result, it will throw an exception 
                                # because result may not be a string

This is the result:

iex(9)> c "readfile_with.exs"
:error

Scope of variables

From Programming Elixir, p. 37:

[...] width allows you to define a local scope for variables. If you need a couple of temporary variables when calculating something, and you don't want those variables to leak out into the wider scope, use with.

In other words, the variables defined within a with are temporary. In the following code, we define two variables: name and contents. Note that we also use the same variable names within the with block. We then print the value of name and the value of contents. You will see that these variables are unchanged:

# readfile_with.exs
name = "unchanged"
contents = "unchanged"
file_data = %{name: "haiku.txt"}
with {:ok, name} <- Map.fetch(file_data, :name),
     {:ok, contents} <- File.read(name) do
        contents
        |> String.split("\n", trim: true)
        |> Enum.map(&String.reverse/1)
        |> Enum.join("\n")
end

IO.puts "name:     #{name}" 
IO.puts "contents: #{contents}"

This is the output:

iex(4)> c "readfile_with.exs"
name:     unchanged
contents: unchanged

Order of clauses in the else block

Finally, let's talk about the order of the clauses in the else block. At first, I believed that you must define each clause in the same order as the conditions in the header of the with block. However, that is not true. Instead, you must define the most general clause at the end. That is, something like _ -> "Generic error" must be the last clause. Just for completeness, let's see what happens when you use _to match the result of any of the conditions of the header of with:

# readfile_with.exs
file_data = %{name: "poem.txt"}
result = with {:ok, name} <- Map.fetch(file_data, :name),
              {:ok, contents} <- File.read(name) do
                line = contents
                        |> String.split("\n", trim: true)
                        |> Enum.map(&String.reverse/1)
                        |> Enum.join("\n")

                IO.puts "#{inspect line}"
        else
            _ -> "Generic error"
            {:error, :enoent} -> "Couldn't read file"
            :error -> ":name key missing in file_data"

        end

IO.puts result

This is the result:

iex(19)> c "readfile_with.exs"
Generic error

Note I believed that Elixir would give me a warning to indicate that _ -> "Generic error" will match anything. But it did not. As far as I know, this kind of warning only happens when working with modules. For example, if you try to run this code, Elixir will show a warning:

defmodule SampleModule do

    def hi(_) do
        IO.puts "Hi, stranger!"
    end

    def hi(%{name: name}) do
        IO.puts "Hi, #{name}"
    end

end

SampleModule.hi(%{name: "laura"})

This is the output:

iex(1)> c "sample_module.exs"
warning: this clause cannot match because a previous clause at line 3 always matches
  sample_module.exs:7

Hi, stranger!
[SampleModule]

Elixir