Elixir: Thoughts on the `with` Statement

Elixir has a some great syntactic sugar. A nice feature that was introduced back in Elixir 1.2 is the with statement which allows you to string together pattern matches without horrific nesting. You can now write out the whole happy path for your code in a nice, concise statement, without having to worry about all the unhappy paths. Clarity for the win! But, it has a down side. Here’s what I think that is, and an brief overview of a pattern I settled on for dealing with it.

Borrowing an example from the docs, with allows you to turn this hard-to-read code:

defp serve(socket) do
  msg =
    case read_line(socket) do
      {:ok, data} ->
        case KVServer.Command.parse(data) do
          {:ok, command} ->
            KVServer.Command.run(command)
          {:error, _} = err ->
            err
        end
      {:error, _} = err ->
        err
    end

  write_line(socket, msg)
  serve(socket)
end

into a much cleaner, and far easier to understand configuration:

defp serve(socket) do
  msg =
    with {:ok, data} <- read_line(socket),
         {:ok, command} <- KVServer.Command.parse(data),
         do: KVServer.Command.run(command)

  write_line(socket, msg)
  serve(socket)
end

This is a big win both for the syntax of writing it and for the cognitive load of reading it. But some pitfalls lurk.

The Down Side

In the hard to read first block of code, we’re at least able to see which patterns are returned out of the functions called, and can sort of, kind of reason about what the contents of msg might be once the case statements are matched, given a particular input.

The problem with the with implementation here is that the value of msg is hard to predict, and we can’t be 100% sure which of the last two functions have been called. It forces us to put a lot of the control logic into the write_line/2 function instead of in the control structure where we can see it. Even if we were to handle these before write_line, we still need to handle a bunch of possible return values.

The problem is that msg might be a string, the result of KVServer.Command.run(command) or it might be something like the tuple {:error, :uh_oh}, which presumably is not going to be useful to send to write_line/2.

This is kind of painful to debug if an unexpected value gets emitted and then passed down. And from a code standpoint it can end up trading cleanliness in this block of code for dirtiness elsehwere. You might have to, for example, change write_line/2 able to handle all the possible intermediary values. Or you introduce a follow-on case statement that checks the value of msg when returned from with. But then when you are reading that case statement, how do you know which case matches the return values from which function in the chain?

One Solution

I’ve done this a number of times now, and in the end I settled on a pattern that I like. It’s a compromise that keeps the handiness of the with block, but lets you reason about the resulting value. You might find it useful (or not).

What I’ve started doing is to make sure that each function chained into the with statement returns a tuple identifying where it went wrong. We can then still chain the whole happy path in the with statement but handle all the other cases, without accidentally pushing that down into the following functions.

And if someone changes one of the chained functions later, and it starts to return something unexpected, we’ll catch that case right here where it was returned.

Here’s what that might look like if we modified the example above:

defp serve(socket) do
  msg =
    with {:ok, data} <- read_line(socket),
         {:ok, command} <- KVServer.Command.parse(data),
         do: KVServer.Command.run(command)

  case msg do
    {:ok, value} -> # The whole chain succeeded
      write_line(socket, msg)

    {:error, :read_line, error} -> # Bailed out in `read_line/1`
      Log.error "Bad read_line/1 from socket #{inspect(socket)}"

    {:error, :command_parse, error} ->
      Log.error "Can't parse response #{inspect(data)}"
      do_something_with(error)

    unexpected ->
      Log.error "Got unexpected value #{inspect(unexpected)}"
  end

  serve(socket)
end

In this pattern, we modify the functions we call in the with block to return a tuple that contains where things went wrong. We can then easily match against that in the case statement and handle the responses appropriately. This really starts to pay off when the with statement contains three or four layers of function calls. You can be much more cerain that you’ve handled all the potential places the with might short-circuit.

When we’re debugging this, chances are we’ll get a nice log about where things went wrong, and if we got something we weren’t expecting, we’ll still log it out and move on. Only in the condition that everything went well will we call write_line/2.

The main idea here is that we’re returning enough information from each chained function to identify where things went wrong. But, in the comments on the original post, mmuskala pointed out that there is a more concise construct where with takes an else block (introduced in Elixir 1.3) which allows us to write the above more cleanly as:

defp serve(socket) do
  with {:ok, data} <- read_line(socket),
       {:ok, command} <- KVServer.Command.parse(data),
       {:ok, msg} <- KVServer.Command.run(command) do
    write_line(socket, msg)
  else
    {:error, :read_line, error} -> # Bailed out in `read_line/1`
      Log.error "Bad read_line/1 from socket #{inspect(socket)}"

    {:error, :command_parse, error} -> # Bailed in ...Command.parse/1
      Log.error "Can't parse response #{inspect(data)}"
      do_something_with(error)

    unexpected ->
      Log.error "Got unexpected value #{inspect(unexpected)}"
  end
  serve(socket)
end

This is nice because it’s even cleaner syntax. And it avoids having to handle the :ok response in the else block as we would have in a following case. Note that here we’ve also moved the write_line/2 call into the with chain.

Conclusion

Is this the right pattern for you? Maybe. I’ve found it pretty useful. It reduces the chance of passing something unexpected to a downstream function and it makes debugging a lot easier. I find the with statement useful and I like the way it makes it easy to understand the expected code flow. Using this pattern I’ve found no downsides to using it, even with a larger number of chained functions.