Desugaring for comprehensions (AKA for expressions)

Recently I stumbled upon a piece of Scala code that might leave you puzzled. Before showing you the code I must spend a few words about the compiler options.

Compiler options

Getting into the glory details of each and every compiler option is out of the scope of this post (see compiler options). Suffice to say you can use some flags to make the Scala compiler stricter and help you find code deficiencies at compile time. The flag I always use, among others, is "-Wunused". So in build.sbt I have:

scalacOptions := Seq(

Basically, if you have a piece of code that is not used you’ll be notified about it. You may also want to enable other flags and/or be more fine grained about the unused code you want to be notified about. Alex did a very good write-up about the compiler options here.

What happened?

It’s time to show you the (simplified) code:

val optA: Option[Int] = Option(42)
val optB: Option[Int] = Option(0)

val x1: Option[Int] = for {
  a <- optA
  b <- optB
  if a != b
} yield a

That code means: if both Options are of type Some and their content is different then the result is Some(a) else None.

Well, if you try to compile that code, using the flag introduced in Compiler options, you’ll get the following warning:

...parameter value b in anonymous function is never used
[warn]     b <- optB
[warn]     ^

At which point, if you know the nuts and bolts of for comprehensions you’ll go “oh right, let me fix that”, but if you don’t you could be like “huh? I’m actually using b in if a != b! Plus, what is this anonymous function the compiler is talking about?”.

In order to understand the reason behind that warning, you need to know how for expressions are translated—under the hood—by the Scala compiler. A for comprehension is just syntactic sugar for a chain of foreach, flatMap, map, withFilter or filter calls. I won’t be digging into the details of that, given you can find them here. For the scope of this post, I’m just going to show you why you get that warning by desugaring the for comprehension in the example above. That code gets translated, more or less, into this one (name of the assignment variable aside):

val x2: Option[Int] = optA.flatMap { a =>
  optB.withFilter(b => a != b).map(b => a)

See the problem? In .map(b => a) we’re just discarding b. Besides, b => a is the anonymous function the compiler is talking about. Indeed, if you try to compile the previous code you’ll get the same warning:

parameter value b in anonymous function is never used
[warn]     optB.withFilter(b => a != b).map(b => a)
[warn]                                      ^

We can solve that by simply replacing map(b => a) with map(_ => a). But what if we want to keep using the for expression? There you go:

val x3: Option[Int] = for {
  b <- optB
  a <- optA
  if a != b
} yield a

I’ve just switched the order of optB and optA. That’s desugared as (more or less):

val x4: Option[Int] = optB.flatMap { b =>
  optA.withFilter(a => a != b).map(a => a)

See? No unused values there. Here is an alternative, instead of resorting to changing the order:

val x5: Option[Int] = for {
  a <- optA
  if optB.exists(b => a != b)
} yield a

which gets translated as:

val x6: Option[Int] = optA
  .withFilter { a =>
    optB.exists(b => a != b)
  .map(a => a)

Of course, in terms of values, they’re all equivalent:

assert(x1 == x2)
assert(x2 == x3)
assert(x3 == x4)
assert(x4 == x5)
assert(x5 == x6)

At first I wasn’t sure if it can be considered or not a compiler bug, but after a small chat with Gabriele on twitter I decided to file a bug issue and see what they think about it.