Monad Transformers: code smell in disguise?

What are they?

Luka Jacobowitz (Gitter @LukaJCB 01/04/20 01:48) says "the definition of monad transformer is that for any MT[F, A] if F is a monad than MT[F, *] is also a monad."

Noel  Welsh gives us an example of a 3-way monad: None, Some(x) or a (String) error:

Monad transformers allow us to stack monads. Eg, stacking an Option with \/ (that is, Scalaz's version of Either) into one handy monad. The type OptionT[M[_], A] is a monad transformer that constructs an Option[A] inside the monad M. So the first important point is the monad transformers are built from the inside out.

Here's an example:

  def transformAndAdd[F[_]: Monad](fa: OptionT[F, Int], fb: OptionT[F, Int]): OptionT[F, Int] = for {
    a <- fa
    b <- fb
  } yield a + b

Here, F can be any monad. For instance:

    val faIO = OptionT[IO, Int](IO(Some(1)))

    val fbIO = OptionT[IO, Int](IO(Some(2)))

    transformAndAdd(faIO, fbIO)

will wrap the result of the addition in an IO. You can do something similar for Future:

    val fa = OptionT[Future, Int](Future(Some(1)))

    val fb = OptionT[Future, Int](Future(Some(2)))

    println(Await.result(transformAndAdd(fa, fb).value, 1.seconds))

We care not what the outer monad is as long as the inner is an Option in this case.

Drawbacks

Not all combinations of monads can be used with transformers. This SO answer explains why:

Important thing here is that there is no FutureT defined in cats, so you can compose Future[Option[T]], but can't do that with Option[Future[T]] ... 

That means you can compose Future x Option (aka Option[Future[T]]) to one single monad (coz OptionT exists), but you can't compose Option x Future (aka Future[Option[T]]) without knowing that Future is something else besides being a monad (even though they’re inherently applicative functors - applicative is not enough to neither build a monad nor monad transformer on it)

The omniscient Rob Norris explains why:

Rob Norris @tpolecat
Like Option[IO. There's no IOT transformer. 

PhillHenry @PhillHenry
Because nobody has written one? Or it can't be done?
(Sorry for such a silly question. Friday night here.) 

Rob Norris @tpolecat
It can't be done. You would have to define Option[IO[A]] => (A => Option[IO[B]]) => Option[IO[B]] but there's no way to do that because you can't get the A out.

Should we even use them?

There appear to be two schools of thought on Monad Transformers. People aligned to Scalaz appear to hate them in Scala.

jrciii @jrciii
Are monad transformer stacks idiomatic scalaz? If not is there a better way to get either an error or a value with logging along the way? 

beezee @beezee
eh idiomatic i have to defer to others. i feel like the idioms change every time there's a conference
i used them a lot and now consider that code legacy
i dream of the day it's all gone
it seems very in fashion to mimic mtl
scalaz has MonadError and MonadWriter
the nice thing about it is you don't worry about plumbing in your business code and you can be explicit about what capabilities are required for every function
i'm sure others who have used it more than me could describe pitfalls there too 

John A. De Goes @jdegoes
Monad transformers are not idiomatic in Scala. Avoid them at all costs. [See John's article here]

Emily Pillmore
The real question you need to ask after deciding Monad Transformers are not good in Scala is why that is true. The answer is not that MT are bad, per se, but that Scala has very poor facilities for dealing with nested boxed and unboxed values alike. It simply is not a language that has reasonable data formats 

John A. De Goes @jdegoes
@PhillHenry Yeah, what Emily says. Monad transformers work well in Haskell (actually way more performant than, e.g. free monads). But Scala is not Haskell. In Scala, monad transformers destroy performance and type inference, and add tedious boilerplate unless used in combination with type classes (in which case you just have to worry about type inference and performance).

[Gitter chat, 20 & 26 Feb  2019]

But the Cats crowd don't seem to mind:

Rob Norris @tpolecat
The correct take on monad transformers is that they're fine. Write your program in terms of abstract effect F and then at the end of the world if you have to instantiate it into a transformer stack, who cares? It's awkward but just for a line or two. 

Gavin Bisesi @Daenyth
And unless your app's runtime does no database or network IO, any performance overhead from using transformers is just not worth caring about. IO drowns it out by orders of magnitude

[Gitter chat, 1 Nov 2019]

But even the Cats people say: "Strong claim: Thou shalt not use Monad Transformers in your interface" (Practical FP in Scala by Gabriel Volpe)