Why flatMap is effectful magic

Yet again I was wondering why some Cats effect code was not running my effects and yet again I discovered I'd forgot to flatMap. Silly but it made me think why do we need to flatMap in the first place - besides being told to.

I was thinking: if map is defined as:

map[A, B]:     IO[A] => (A => B)     => IO[B]

and flatMap as

flatMap[A, B]: IO[A] => (A => IO[B]) => IO[B]

then if they both result in IO[B] then why does one necessarily run a program and one does not?

The omniscient Fabio Labella pointed me in the right direction on Gitter. He did this with a toy implementation of an IO:

class IO[A](val unsafeRun: () => A) {

  def map[B](f: A => B): IO[B] = new IO[B] { () =>

      val myResult = unsafeRun()

      val result   = f(myResult)

      result

  }

  def flatMap[B](f: A => IO[B]): IO[B] = new IO[B] { () =>

      val myResult = unsafeRun()

      val nextIO   = f(myResult)

      nextIO.unsafeRun()

   }

}

object IO {

   def apply(a: => A): IO[A] = new IO(() => a)

}

Here, we're simply suspending some functionality, a, into an IO with a very basic implementation of map and flatMap which were just ineluctably derived from the signatures I defined at the top of this post.

Fabio Labella @SystemFw 11:41

[Regarding map], say we do IO(readLine).map(_.toUpperCase)

and unsafeRun that, let's do a substitution to see what [it] translates to

IO { () =>

  val myResult = readLine

  myResult.toUpperCase

}

 

and when you unsafeRun that you get

{ 

   val myResult = readLine

   myResult.toUpperCase

}

 

which is the expected behaviour

So, map's type is fine for this use case. 

Fabio Labella @SystemFw 11:47

ok, so now let's take this other example

val read: IO[String]           = IO(readLine)

def print(s: String): IO[Unit] = IO(println(s))

val nope: IO[IO[Unit]]         = read.map(print)

nope.unsafeRun

which has not printed anything! (the function is still suspended)

Here, we want an unsafeRun that references a program not a container of a program (where "program == things of shape F[A]").

So, we have no choice but to use flatMap. It's the only signature that fits.

Fabio Labella @SystemFw 11:50    

For "map to be able to run effects", it would have to know that when B happens to be IO[C], it also needs to call unsafeRun on it, which breaks parametricity [ie, it does not conform to the signatures].

Not only that, it actually negates a useful pattern, sometimes you want an IO that produces another IO without running it.

Because you want to treat the inner one as a value (for example you want to send it to an in memory queue).

So, it's not just a theoretical problem, it has a practical impact.

Conclusion

1. Running effects via flatMap is not a convention. It's fundamental.

2. Turning on the compiler warnings would have caused the faulty code not to compile in the first place. Given:

    val printEffect:  Int => IO[Unit]     = x => IO { println(x) }

    val printEach:    Pipe[IO, Int, Unit] = { _.map(x => printEffect(x)) }

    printEach(Stream(1, 2, 3, 4, 5)).compile.toList

then the compiler will fail on printEffect(x) because although this returns a value that is not Unit, the compiler will treat it as if it does because of the type of printEach.