Automatic Boundary Checking

Surprises in Java's Math class

ScalaTest/ScalaCheck has a rather nice little feature. It can boundary check your unit tests. Let's take this code:

import org.scalatest.WordSpec
import org.scalatest.prop.GeneratorDrivenPropertyChecks
import org.scalatest.Matchers

class MyScalaTest extends WordSpec with GeneratorDrivenPropertyChecks with Matchers {

  "java.lang.Math.abs" should {
    "always return a non-negative value" in {
      forAll { x: Int =>
        Math.abs(x) should be >= 0
      }
    }
  }
}

It nicely runs the test many times, probing it with different values. Unfortunately, it fails with:

[info] MyScalaTest:

[info] Math.abs

[info] - should always return a non-negative value *** FAILED ***

[info]   TestFailedException was thrown during property evaluation.

[info]     Message: -2147483648 was not greater than or equal to 0

[info]     Location: (MyScalaTest.scala:12)

[info]     Occurred when passed generated values (

[info]       arg0 = -2147483648

[info]     )

What? How could java.lang.Math.abs fail to return a non-negative value? Well, we hit a boundary condition. ScalaCheck is clever enough to try Integer.MIN_VALUE. However, because there are more negative Integers than positive (the range is -2³¹ to 2³¹-1), not all negative integers map to positive ones.

If we are happy with a limited range, we can write:

import org.scalacheck.Gen

.

.     

      val sensibleRange = Gen.choose(0, 100)

      forAll(sensibleRange) {

.

.

to choose numbers inclusive of 0 to 99.

All the Kings of America have beards

Consider this. I say: "everybody in the room next door is called John". You look inside the room and see nobody there and tell me so. If I then said: "Yes, everybody in the room is called John!", you might look sideways at me.

Yet this is not only mathematically correct but also what we see in functional language.

Take this Java:

        Set<String> emptySet = new HashSet<String>();
        Predicate<? super String> isHelloWorld = (String x) -> { return "hello world".equals(x); };
        System.out.println(emptySet.stream().allMatch(isHelloWorld)); // true
        System.out.println(emptySet.stream().anyMatch(isHelloWorld)); // false

Or alternatively this Scala:

    def isHelloWorld: (String) => Boolean = "hello world".equals(_)
    val emptySet: Set[String] = Set[String]()
    println(emptySet.forall(isHelloWorld)) // true
    println(emptySet.exists(isHelloWorld)) // false

Any assertion on all the elements of an empty set is always true.

Two tips for Java and Scala interoperability

I'm writing more and more Scala these days but don't want to give up on Java libraries I know and love. To this end, I've been writing ScalaTests with Mockito. It's fairly straightforward but two things initially foxed me.

Varargs

The first were varargs. "Both java and scala have varargs" but calling a Java method using varargs from Scala is not obvious.

I had code that looked like:

val x: List[X]       = ...
val ordered: InOrder = Mockito.inOrder(x)

which didn't compile because x was just being treated like an Object rather than an array of them.

The solution looks like this:

val ordered: InOrder = Mockito.inOrder(x:_*)


Conversions between Java and Scala collections

This is solved with a simple import.

import collection.JavaConversions._

Now we can do things like:

Thread.getAllStackTraces.map(x => println(x._1 + ":\n" + (x._2.mkString("\n    "))))

to print out all the stack traces of all the threads currently running. Without this import, the map function would fail as getAllStackTraces returns Map<Thread, StackTracesElement[]> - a Java Map that doesn't have the map method.

Some useful Scala I learned this week

1. Indexes of a collection

If you want to  traverse a list of elements keeping the index in mind, you could do this with a fold but a more elegant way is something like:

    val words = List("the", "quick", "brown", "fox", "jumped", "over", "the", "lazy", "dog")

    words.zipWithIndex // returns tuples like (word, index)

2. Choosing only some elements of a collection

Now, let's say we want the tuples of those words that begin with 't'. We do this with the collect function in TraversableLike.

    words.zipWithIndex collect {
        case (word, index) if word.startsWith("t") => (word, index)
    }

(You will note that we pass collect a partial function). This will return the list:

List((the,0), (the,6))

That is, the 0th and 6th words begin with 't', and they are both 'the'.

3. Counting the elements that meet a criteria

is very simple:

    words count (_ startsWith("t"))

Will, unsurprisingly, return 2 for the above collection.

What I learned this week is that you could do all these with folds etc but Scala (I am told) has methods that do pretty much anything you could want. It's just a matter of learning them.

Scala's way of overriding final Java classes

Or "pimp my library".

I was pair programming with a Scala guru when he saw me creating a function to interrogate a String. He took my code and re-wrote as  something like this:

object MyImplicits {
  
  implicit class MyString(val string: String) {
    
      def startsWithD: Boolean = string.startsWith("D")
      
  }

}

object MyImplicitsMain {
  
  import com.phenry.scala.MyImplicits._
  
  def main(args: Array[String]) = {
    val aString = "Does this start with D?"
    println(aString.startsWithD) // this returns true
  }
  
}

So, effectively he added a function to the String class for any code that imports this implicit.

Note that implicits can live in objects and packages (as they basically hold what Java programmers would recognise as statics) but not class or traits.

One is advised to be careful with implicits, however. "Implicits are very powerful but you should use them with care," said Scala's creator.

Specialized Scala

I've been playing with the maths library, Breeze (a good introduction is here). It's written in Scala and uses a cunning stunt to avoid boilerplate code.

The problem with writing an efficient library in Java is autoboxing is horribly inefficient for sufficiently complicated mathematical operations. The solution is typically to make the code less generic and use primitives (as I did in my pet project, jMathematics).

Scala has a neat trick to avoid this. It uses the @specialized annotation. So, one of Breeze's implementations of a matrix looks like this:

final class DenseMatrix[@specialized(Int, Float, Double) V](...
                                                            val data: Array[V],
                                                            ...)

This results in the compiler creating methods that use these 3 primitives as well as the more generic types. This leads to quite a lot of classes:

henryp@phillsdell:~/Code/Scala/Maths/breeze$ find ./target/scala-2.10/classes/ -name DenseMatrix* | wc -l

350

but this Scala code:

  def valueAt(row: Int, col: Int) = apply(row,col)

(where apply ultimately reads from data: Array[V] in the constructor above) is compiled into this:

henryp@phillsdell:~/Code/Scala/Maths/breeze$ for FILE in `find ./target/scala-2.10/classes/ -name DenseMatrix*` ; do { javap "$FILE" | grep "valueAt(int, int" ; } done
  public int valueAt(int, int);
  public java.lang.Object valueAt(int, int);
  public float valueAt(int, int);
  public java.lang.Object valueAt(int, int);
  public double valueAt(int, int);
  public java.lang.Object valueAt(int, int);
  public V valueAt(int, int);

Great, so a generic Object return value plus methods for our three specialized primitives. What could possibly go wrong? Well careless use can lead to literally megabytes of code being generated. That's why it's so rare in the standard Scala libraries.

However, this is a big win for mathematics library writers.

More on Java 8 Lambdas

Using the code found on this tutorial of Java 8's new Optional class, I started playing with the syntax of the new lambdas.

First let's instantiate a simple data structure using POJOs:

        Computer computer = new Computer(new Soundcard(new USB("3.0")));

and wrap it in a fairly straightforward object that's new to Java 8's functional packages:

        Optional<Computer> option = Optional.of(computer);

Now comes the more interesting code. Say Computer has a method thus:

    public Optional<Soundcard> getSoundcard() { ... }

we can flatMap so:

        Optional<Soundcard> flatMapped = option.flatMap(Computer::getSoundcard);

What interesting syntax. 

There's three curious points about this line.

Computer::getSoundcard

We call this a method reference. It's the function we call on all the elements we flatMap over. It's type can be seen by refactoring it out:

        Function<Computer, Optional<Soundcard>> asFunction = Computer::getSoundcard;

Using this newly refactored code, the flatMap line above is equivalent to:

        Optional<Soundcard> flatMapped = option.flatMap(asFunc);

Alternative syntax

We could just have easily used the Lambda Expression Syntax and the flatMap would have looked like this:

        Optional<Soundcard> flatMapped = option.flatMap( (Computer aComputer) -> aComputer.getSoundcard() );

or, equivalently, using the more verbose block form:

        Optional<Soundcard> flatMapped = option.flatMap( (Computer aComputer) -> {

            return aComputer.getSoundcard();

        } );

Again, refactoring this expression out into an isolated function shows us its type:

        Function<Computer, Optional<Soundcard>> asFunction = (Computer aComputer) -> aComputer.getSoundcard();

Map versus FlatMap

Note that the flatMap method was used rather than the map method. There is a very good reason for this.

The flatMap method signature looks like this:

    public<U> Optional<U> flatMap(Function<? super T, Optional<U>> mapper)

while the map method looks like this:

    public<U> Optional<U> map(Function<? super T, ? extends U> mapper)

(where T is the type that this Optional contains in both cases).

If we call map on this Optional, look at the return type:

        Optional<Optional<Soundcard>> mapped = option.map((Computer aComputer) -> aComputer.getSoundcard());

An Optional in an Optional is returned. Looking back at the signatures, the reason for this is clear. 

The method flatMap takes a function that returns an Optional<Soundcard> and this is exactly what the flatMap method returns.

But map's method signature says it takes a function that returns a type U and wraps that U in an Optional and returns it. Since we can pass the method reference Computer::getSoundcard to map and that obviously returns Optional<Soundcard> (because that's what getSoundcard() says it returns), the type U is Optional<Soundcard>. So, U is wrapped in an Optional and U is an Optional. Therefore, an Optional wrapped in an Optional is returned.

Monads

It may seem odd that we map and flatMap on an Optional. Perhaps you're more used to calling these methods on Collections. But Lists and Sets share properties with Optionals in that they're all monads - something I'm only just getting my head around...

Further reading

Java 8: Lambdas, Part 1, Oracle.
Processing Data with Java SE 8 Streams, Java Magazine.