class FooBuilder {
  ...
  public this withBar(Bar bar) {
    this.bar = bar;
  }
}

Now that Java permits covariant returns types, we can get nearly the same effect by having Bar extends Foo override any builder-type method that returns a Foo to return a Bar. But what a hassle!

We can also get the same effect using recursive generics, via

class FooBuilder<T extends FooBuilder<T>> {
  T someBuilderMethod(...) { ... }
}

class BarBuilder extends FooBuilder<BarBuilder> {
  ...
}

My own experience is that recursive generics used for this purpose tend to be clunky. Further, if something else extends Bar, someBuilderMethod can only return a Bar, so we can’t do something like

class BazBuilder extends BarBuilder {
  BazBuilder someOtherBuilderMethod(...) { ... }
}

class SomeTestThatNeedsABaz {
  public void testSomething() {
    Baz baz = new BazBuilder().someBuilderMethod(...).someOtherBuilderMethod(...).build(); //WRONG
  }
}

All these issues would be solved very neatly with

class FooBuilder<T extends FooBuilder<T>> {
  this someBuilderMethod(...) { ... }
}

Oh, and the syntax inside a this method is the same as a void method, permitting naked returns or dropping off the bottom with no return.

Strong visual patterns can be an indicator that you may need to remove some duplication from your tests. I say “may”, because some amount of duplication in the interest of clarity is acceptable in test code.

Assume you’re testing a method call, and that your test follows the basic, simple flow

• set-up
• call the method under test
• assertions

If the duplication is in the set-up, can you use a builder or other means to remove some of that duplication? If it’s in the assertions, are you sure that you aren’t testing the same thing in multiple test cases? Even in cases where you need to test different edge cases (“Does it work with a one-element list? With a zero-element list? With a two-element list?”) you may not need to repeat the ancillary assertions in each test. If another test is already testing that part, don’t repeat yourself. At least, don’t repeat yourself without a good reason.

Squint.

And if you see patterns, take a moment to ponder.

In that previous post, we produced the following test code.

   public void testDisplayLines() {
     Account acct = new Account();
     acct.setId("ID17");
     acct.setBalance(17);
     acct.setNewCharges(5);

     widget.addDisplayLines(acct);

     DisplayLine line = widget.getDisplayLine("ID17");
     assertEquals("17.0", line.getPreviousBalance());
     assertEquals("22.0", line.getCurrentBalance());
     assertEquals("Overdue 90 days", line.getErrorMessage());
   }

   public void testDisplayLinesWithOverdueAccount() {
     Account acct = new Account();
     acct.setOverdueMonths(3);
     acct.setId("ID17");
     acct.setBalance(17);
     acct.setNewCharges(5);

     widget.addDisplayLines(acct);

     DisplayLine line = widget.getDisplayLine("ID17");
     assertEquals("17.0", line.getPreviousBalance());
     assertEquals("22.0", line.getCurrentBalance());
     assertEquals(null, line.getErrorMessage());
   }

As well as duplication in the assertions, the test data set-up is similar. We could get rid of this duplication with a helper method

   public void testDisplayLines() {
     Account acct = createAccount("ID17", 17, 5);

     widget.addDisplayLines(acct);
     ...

   private void Account createAccount(String id, int balance, int newCharges) {
     Account acct = new Account();
     acct.setId(id);
     acct.setBalance(balance);
     acct.setNewCharges(newCharges);
     return acct;
   }

But once we have 8-10 tests with different set-up needs, we end up with telescoping helper methods, and the reader cannot tell solely by inspection of the call to the helper method what each parameter means.

While the builder pattern is usually used to create immutable data, we can use it to chain data creation.

   public void testDisplayLines() {
     Account acct = buildAccount()
         .setId("ID17").setBalance(17).setNewCharges(5).build();

     widget.addDisplayLines(acct);
     ...
   }

   public void testDisplayLinesWithOverdueAccount() {
     Account acct = buildAccount()
         .setOverdueMonths(3).setId("ID17").setBalance(17).setNewCharges(5).build();

     widget.addDisplayLines(acct);
     ...
   }

   private AccountBuilder buildAccount() {
     return new AccountBuilder();
   }

   private static class AccountBuilder {
     private Account acct = new Account();

     AccountBuilder setOverdueMonths(int overdueMonths) {
       acct.setOverdueMonths(overdueMonths);
       return this;
     }

     Account build() {
       Account result = acct;
       acct = null;  //builder is single-use
       return result;
     }

     AccountBuilder setId(String id) {
       acct.setId(id);
       return this;
     }

     AccountBuilder setBalance(int balance) {
       acct.setBalance(balance);
       return this;
     }

     AccountBuilder setNewCharges(int charges) {
       acct.setCharges(charges);
       return this;
     }
   }

Note that the data set-up in the test methods is as readable as setting the values “by hand”. But the declaration of the build class itself is fairly verbose. Some care should be taken to ensure that a builder is only used to instantiate a single object, as untangling the values persisted across multiple re-uses is confusing to read. There’s also a danger of aliasing if the same object is re-used.

For illustrative purposes, this example is only creating test data of a single type in two test methods. In the real world, this is not enough to justify its construction.

It’s often the case that we want to use multiple builders for different kinds of data.

      Contact contact = buildContact().setNames("Jack", "Endino").build();
      Account acct = buildAccount().setId("ID32").addContact(contact).build();
      ...

The builders’ methods do not need to correspond one-for-one with the objects’ data, but it must be trivially obvious to the most casual observer just what is being set.

The builder can even create collaborative objects, but only when the convenience gained greatly outweighs any loss of readaility. For example, if many accounts need to have a contact as a collaborator, a convenience method can be added to the builder.

      Account acct = buildAccount().addSimpleContact("Jack", "Endino").build();
      ...

If you took high school chemistry, you may have had to titrate a hydrochloric acid solution with a sodium hydroxide solution. The idea was to perfectly balance the hydrochloric acid with the sodium hydroxide, so that the phenolphthalein is right at the edge between pink and white.

When you use weasel testing or p3 to do TDD, and alternate rapidly between writing a little test and writing a little production code, you “titrate” your unit tests to perfectly cover your production code. You know you have enough tests, because you didn’t write any production code until you were forced to do so. And you know you didn’t write any extra tests, because you would have no motivation to write a test that won’t force a change in the production code. This is the Baby Bear approved method: not too much, not too little, but just right.

I suspect getting this just-enough-and-no-more is much harder to do when you write all your unit tests in advance, or when you write your unit tests after the fact. Maybe I should try this for comparison at some point.

assertTrue("list is exhausted", !iter.hasNext());
assertTrue("index is out of bounds", i < 0);

Part of the difficulty is that we read the same message in two different contexts. When we’re reading them in the code, it’s natural to read the messages as being the invariant which is needed to pass this test. That is, when in code, it’s easy to read the message as what is true when the test passes.

But what do we see on the console when those same two assertions fail?

java.lang.AssertionError: list is exhausted
java.lang.AssertionError: index is out of bounds

When reading the assertion error in console output, it is easy to read it as what is true when the test fails. The “list is exhausted” messages makes sense in the code (assuming exhaustion is what passes the test) while “index is out of bounds” makes more sense when read in the console output.

But if we turn them into “should statements”, they make sense in both contexts.

assertTrue("list should be exhausted", !iter.hasNext());
assertTrue("index should not be out of bounds", i < 0);

java.lang.AssertionError: list should be exhausted
java.lang.AssertionError: index should not be out of bounds

You can get nearly the same effect, even if you have to develop solo. (But it won’t be near as much fun.)

The key to doing this effectively is that you need to have two personas. I’ve tried to come up with a good name for this technique, but so far I’ve only come up with “schizophrenic programming” or “devil’s advocate programming” or “stubborn testing” or even “two hats programming”. For now, let’s call it “weasel testing”.

You wear two hats, and you alternate between them. One hat belongs to the Good Tester. He’s only allowed to edit the test code. He’s always trying to force the Bad Programmer to write the correct production code. While you’re playing the Good Tester, your job is to write the next incremental piece of an excellent test. And your job is finished — for the moment — as soon as you have a correct but failing test.

Once the tests are failing, you change hats, and Weasel takes over. The Weasel isn’t a bad programmer. “Lazy” or “Contrarian” might describe Weasel better. Wearing Weasel’s hat, your job is to make all the tests pass by writing the simplest piece of code that satisfies the test. The simplest. No matter how stupid or wrong. Be stubborn. That Good Tester is trying to make you write the correct code, and you even know what code he’s trying to force you to write. But you’ll show him! You’ll think of some cockeyed but simple way to weasel out of his trap, and make the tests go green. Thereby showing the Good Tester that his tests aren’t up to snuff yet. Once the tests go green, it’s the Good Tester’s turn again. And you cycle back and forth until the Good Tester finally paints Weasel into a corner from which he can’t escape. At that point, you have a solid chunk of production code with full test coverage.

Do not underestimate how hard this is to do well. Every time you do TDD, you already pretty much know what code you’re trying to force yourself to write. This makes it very easy to write the “final, correct” code, even without sufficient test pressure. As a result, when you transition from Good Tester to Weasel, it’s worth pausing a moment, shifting contexts, and trying to think carefully about how to cheat and still get the tests to go green.

The better you get at playing Weasel, the better your tests will become.

What are these tests supposed to do?

   public void testSomething4() {
     dataSetup(5, 17, new Account());
     verifyExpectations("ID17", "17.0", "22.0", null);
   }

   public void testSomething5() {
     Account acct = new Account();
     acct.setOverdueMonths(3);
     dataSetup(5, 17, acct);
     verifyExpectations("ID17", "17.0", "22.0", "Overdue 90 days");
   }

   private void dataSetup(int i, int j, Displayable acct) {
     acct.setId("ID" + j);
     acct.setBalance(j);
     acct.setNewCharges(i);
     widget.addDisplayLine(acct);
   }

   private void verifyExpectations(String id, String balance, String total, String errors) {
     DisplayLine line = widget.getDisplayLine(id);
     assertEquals(balance, line.getPreviousBalance());
     assertEquals(total, line.getCurrentBalance());
     assertEquals(errors, line.getErrorMessage());
   }

Each of two tests sets values on an account, and adds that account to some widget. Unfortunately, however, the helper methods dataSetup() and verifyExpectations() obscure the tests rather than clarifying. The reader will need to build an understanding of what these methods do before she can determine whether the test is needed and correct.

Ideally, a reader should be able to understand what is being tested and how by looking solely at the test method. In tests, we can tolerate a bit more redundancy if it improves clarity.

   public void testDisplayLines() {
     Account acct = new Account();
     acct.setId("ID17");
     acct.setBalance(17);
     acct.setNewCharges(5);

     widget.addDisplayLines(acct);

     DisplayLine line = widget.getDisplayLine("ID17");
     assertEquals("17.0", line.getPreviousBalance());
     assertEquals("22.0", line.getCurrentBalance());
     assertEquals("Overdue 90 days", line.getErrorMessage());
   }

   public void testDisplayLinesWithOverdueAccount() {
     Account acct = new Account();
     acct.setOverdueMonths(3);
     acct.setId("ID17");
     acct.setBalance(17);
     acct.setNewCharges(5);

     widget.addDisplayLines(acct);

     DisplayLine line = widget.getDisplayLine("ID17");
     assertEquals("17.0", line.getPreviousBalance());
     assertEquals("22.0", line.getCurrentBalance());
     assertEquals(null, line.getErrorMessage());
   }

The test methods have been given meaningful names. They have been refactored to expose the basic progression of any good test: set-up, exercise the code under test, and then assertions that test its correctness. In fact, we have formatted each test to visually separate each phase. Now we set the account id with a constant string, rather than “computing” it from the balance.

With this change, there is more redundancy, but we’ve only increased the line count by 3. By formatting the redundant parts identically, it is relatively easy to visually “diff” the two tests to understand how the two similar tests differ.

There are ways we can make these tests ever more readable, but that’s a topic for another post…

For example, let’s imagine that we have an AbstractFlyable and its subclasses, Blimp, Helicopter, and Airplane. To the extent that there is testable functionality in AbstractFlyable – maybe calculations related to speed or fuel consumption — we test those in AbstractFlyableTest. There’s really no good reason to re-test those capabilities in BlimpTest. BlimpTest should be testing the code that is blimp-specific.

This rule is often violated when frameworks are used. For example, it’s common for a web application to have a persistence framework hiding SQL usage underneath, and to be used repeatedly for storing all kinds of data for our application, whether blimps, repair parts, departure dates, or whatever. For each of these, we want to test the code that is specific to each use of the framework.

• Do test that blimps are something persisted to the database. 
• Do test specialized queries created just for blimps, such as a complex query to allow blimps to find football stadiums. 
• Don’t test behavior that is inherited from the framework, such as the ability to compute air speed from data persisted by all AbstractFlyables.

In some of the more rigorous forms of  TDD such as ping pong TDD or devil’s advocate testing (forthcoming blog post), a natural rhythm emerges:

• The developers imagine some piece of functionality they want to add to the code.
• They write a test that pushes them in the right direction.
• They implement as little production code as they can to get the test to pass. 
• And then they iterate this cycle until done. 

Done right, no production code is created without a corresponding test. But another, less obvious, advantage is that no tests are created without corresponding production code. No tests is added unless it is needed to drive development. As a result, the test code stays lean. This is harder to do if tests are written en masse before or after the production code.

Like your production code, your tests should be lean. And fast. J-E-T. Justify every test.

There may be another aspect of usefulness — I’m still waffling on this one — in that a good test is a defense against likely failures. That is, we don’t usually test simple getters and setters, because they are not expected to fail. Should you buy accidental death and dismemberment insurance? Probably not, although the insurance is relatively cheap, as actuarially accidents are far less likely than illness or disease. Likewise, providing test insurance against something that’s highly unlikely to fail is probably a waste of resources.

A good test is correct. A test that expects incorrect values, actually enforcing errors in the application, is a bad test. Almost as bad is a test that doesn’t effectively test what it purports to test, giving a false sense of security. Common examples of this are EasyMock tests which fail to verify the mocks, or tests which override methods of the class under test.

A good test is readable. Readability is what gives subsequent readers of the test confidence that the test is useful and correct. Ideally, it should not take someone familiar with your application more than 30 seconds to understand what an individual test is testing and the basic mechanism by which you are testing that feature. It might take longer to ensure the correctness of any mocks, builders, and helpers you use, but the intent of those helpers should be plain in the top-level test.

1. Create a mock with EasyMock.
2. Set the expectations on the calls to be made on that mock by making those same calls. At this point you can also set return values.
3. Put the mock into replay mode
4. Run your test.
5. Verify the mock.

As an example,

  Foo foo = EasyMock.createMock(Foo.class);
  Bar myBar = new Bar();
  EasyMock.expect(foo.someMethod(1, myBar, "baz")).andReturn(17L);
  EasyMock.replay(foo);
  ClassUnderTest underTest = new ClassUnderTest(foo);
  long result = underTest.classUnderTest("baz", myBar);
  assertEquals(17L, result);
  EasyMock.verify(foo);

Before the replay call, the call to foo.someMethod() sets an expectation that someMethod should be called on foo with exactly the parameters 1, myBar, and “baz”. EasyMock will throw an exception if any other method is called, or if that method is called with different parameters. EasyMock.verify will throw an exception if any expectations have not been met.

EasyMock can be made more general than this. Let’s say that our class under test actually instantiates a Bar, and we want to all the call to someMethod no matter which Bar is passed in. We can do this, but it requires that we use EasyMock’s matchers rather than concrete values when setting up the expectation. Specifically, EasyMock.isA(Bar.class) matches any Bar. We can’t do

  EasyMock.expect(foo.someMethod(    //WRONG!
      EasyMock.isA(Bar.class), "baz").andReturn(17L);

because it turns out that when setting expectations on an EasyMock mock, you can’t mix concrete values and matches. You need to use all one or all the other.

  EasyMock.expect(foo.someMethod(
      EasyMock.anyInt(),
      EasyMock.isA(Bar.class),
      EasyMock.eq("baz"))).andReturn(17L);

This expectation will allow any single call to someMethod which is passed any int as the first argument, any Bar as the second, and the string “baz” as the third. It further turns out that EasyMock provides a mechanism for anyone to create their own argument matchers. We could, for example, create an argument matcher that only matches instances of Bar which satisfy some more specific criterion.

Now, if you think about it, all these matchers — whether EasyMock’s or custom — need to return a value of the correct type to be passed to foo.someMethod. Who knows what constructor’s Bar might have, and what might happen if EasyMock tries to instantiate one? And how can the fake Bar that is returned from EasyMock.isA() and handed to foo.someMethod() have the information to do the correct matching operation?

The answer is, it can’t. EasyMock cheats. And that brings us to our point. When using argument matchers, EasyMock expects — nay, requires — that each matcher make exactly one call to EasyMock.reportMatcher(…). And how does it know which matcher goes with which argument? Well, it lines them up. By calling

  foo.someMethod(EasyMock.anyInt(), EasyMock.isA(Bar.class),
      EasyMock.eq("baz"));

each of the EasyMock matchers does a call to EasyMock.reportMatcher(), which squirrels away a specific matcher in a global list. When the mock foo gets a call to someMethod, it expects to find 3 matchers in that global list. (Well, either 3 or zero, zero being the case where we’re using concrete values and not matchers.) If it find 1 or 2, instead, it throws a cryptic exception, because you mixed concrete values and matchers, and it has no way to know which argument each of the matchers should go with. It simply can’t tell.

So … problem #1 rears its head when we use our IDE’s automated refactoring tools. Someone at some later time decides that they need to generalize Foo’s someMethod, so they use their IDE to add another argument, a boolean let’s say, and they tell their IDE the default value is false. Our call using matchers now looks like

  foo.someMethod(EasyMock.anyInt(),
      EasyMock.isA(Bar.class), EasyMock.eq("baz"), false)

That someone probably has no idea this test is here. When they finally run the full test suite, the cryptic error pops up, they come snooping around this code. If they’re not already familiar with EasyMock and this particular quirk, they’re in for a fun time, until they discover that they need to change this to

  foo.someMethod(EasyMock.anyInt(), EasyMock.isA(Bar.class),
      EasyMock.eq("baz"), EasyMock.anyBoolean())

The second problem is even more insidious. What happens when we refactor our test method. For whatever reason, let’s say we extract the Bar matcher into a local variable.

  Bar bar = EasyMock.isA(Bar.class);
  foo.someMethod(EasyMock.anyInt(), bar,
      EasyMock.eq("baz"), EasyMock.anyBoolean());

And now EasyMock starts failing. Why? Because each argument matcher is responsible to call the global EasyMock.reportMatcher — in the correct order. By this refactoring, the reportMatcher calls are now out of order, and EasyMock is either going to have class cast exceptions, or the expections on someMethod that previously passed will now fail.

Or worse.

One way to avoid these problems is to decide not to use EasyMock at all, instead creating your own hand-rolled doubles. And if you decide to use EasyMock in your project, …

1. Make sure at least someone on your team understands how EasyMock works under the hood.
2. Be aware of EasyMock’s all-concrete or all-matchers requirement, and that violations of this requirement can be violated by automated refactoring.
3. If you use EasyMock matchers, make sure that they are produced while marshalling the arguments to call the mock method, and not beforehand.