Unit testing in Functional Programming (part 2)

ZIO Test

This is quite beautiful. A few years ago, I blogged about a problem I had that some code was making an assertion on a bifunctor but the assertion was on the happy path. However, if the code returned an effect that represented the unhappy path, the assertion was not run and the test erroneously passed.

One could blame a remiss developer but, Dr Matthew Pocock made the point that "if key information about how to use an API is not enforced by that API, then IMHO that API is broken". I'm inclined to agree.

Fast forward five years, and now I have zio-test that will stop such code from ever being committed (see the full code here).

  override def spec: ZSpec[TestEnvironment, Any] = suite("ZIO test")(
    testM("fails on unhappy path even with no assertions"){
      val result:   Option[String]            = None
      val asserted: Option[TestResult]        = result.map(x => assert(x)(equalTo("this is naive as it assumes happy path")))
      val myZio:    zio.IO[Unit, TestResult]  = ZIO.fromOption(asserted)
      myZio
    }
  )

Cleverly, this testing framework fails with:

    Fiber failed.
    A checked error was not handled.
    ()
    
    Fiber:Id(1585237038923,15) was supposed to continue to:
      a future continuation at zio.test.package$ZTest$.apply(package.scala:104)
    
    Fiber:Id(1585237038923,15) execution trace:
      at zio.ZIO$.fromOption(ZIO.scala:2682)
      at uk.co.odinconsultants.fp.zio.ZioSpec$.spec(ZioSpec.scala:24)
      at zio.ZIO$.effectSuspendTotal(ZIO.scala:2260)

No more ScalaTest for me...

The principle is simply that if a None is passed to ZIO.fromOption, then the generated ZIO represents "the moral equivalent of `throw` for pure code" (from the ZIO ScalaDocs). This seems a little opionated but it's a stated quality of its creator ("I do think ZIO core should provide one opinionated way to do things to make scaling in larger teams easier" says John de Goes on GitHub)

There's an awful lot of fibre fun going on as well (the tests run on their own fibres). And it's useful to remember that in typical FP style, your code does not run in the testM block. It merely describes what is to be tested by setting up a testing data structure that actually runs at the end-of-the-world.

Cats Effects

Cats effects has its own testing framework. You can use it with:

    implicit val testContext: TestContext       = TestContext()
    implicit val cs:          ContextShift[IO]  = testContext.contextShift(IO.ioEffect)
    implicit val timer:       Timer[IO]         = testContext.timer(IO.ioEffect)

then fast forward time with:

testContext.tick(60 seconds)

It seems that it is usual to call IO.unsafeToFuture and then make the assertion on the subsequent Future.

However, there is a limit to TestContext if you're testing concurrency in the effect engine itself.

"You can test some things with TestContext, which is deterministic, but not everything. You can't test things like 'make interruption happen exactly at this flatMap'. So I use a mix of TestContext, which is deterministic, with tests that sleep appropriately to simulate different scenarios and sometimes tests that run on real concurrency, but run multiple times (ab)using scalacheck properties. I think the work in CE3 that provides a completely deterministic implementation of Concurrent offers some promise in that direction though." - Fabio Labella, Gitter, April 18 2010

Further examples include:

Check if the code starts on a proper thread pool
Check if elements are sent downstream in the original order
Check if effect is synchronous (for async you'd need tick() )

(Credit to Piotr Gawryś for cataloguing them).

Counting in FS2

Counting is a pretty typical thing to do in a unit test. What's the best way?

PhillHenry 
I have some code that creates a stream and now I want to test it. I'd like my 'mock' function passed to its .mapAsync to count how many times it is called (the production function will not do this). I could use a java.util.concurrent.atomic.AtomicInteger in my test class but is there are more elegant way?

Fabio Labella @SystemFw
@PhillHenry you can use Ref for that
principle is the same though: 

Stream.eval(Ref[IO].of(0)).flatMap { state =>   val f: A => IO[B] = state.update(_ + 1) >> yourThing   yourStream  } 

I'd recommend to watch my talk on shared state, it is likely the reason the Ref was not updating is that you are not yet familiar with the share-through-flatMap concept and you are creating two refs by mistake

Refs seem to be the accepted way of unit testing [SO].

A full example of can be found on my GitHub where I use a SignallingRef to count enqueues and deques.

Does Tagless Final help?

Tagless final is often touted as a good way to write testable code but:

"Ultimately, the testability of tagless-final programs requires they code to an interface, not an implementation. Yet, if applications follow this principle, they can be tested even without tagless-final!...Using tagless-final doesn’t provide any inherent benefits to testability. The testability of your application is completely orthogonal to its use of tagless-final, and comes down to whether or not you follow best practices—which you can do with or without tagless-final."

Although De Goes appreciates Tagless Final he has 5 excellent criticisms here [DZone]

Although Gavin Bisesi (Gitter, Mar 31 18:11) lists the main selling points of Tagless Final are such fine things as:

"If I have F=IO now, but later I have some ApmTracingT[F, *] effect that gives me span/duration tracking, my implementation doesn't change at all, just I just pass a different F when I construct it" and "actual interop (cats-effect vs zio vs monix? All work)"

he also says the obvious benefit to testing is:

"if a test wants a no-op version of Store, then F can be Id[_], and your test now doesn't contain IO concurrency."