Monday, 18 September 2017

Fluent, descriptive testing with NExpect

About a year or so ago, I discovered AssertionHelper, a base class provided by NUnit which allowed for a more familiar style of testing when one has to bounce back and forth between (Java|Type)Script and C#. Basically, it allows one to use the Expect keyword to start an assertion, eg:

[TestFixture]
public class TestSystem: AssertionHelper
{
  [Test]
  public void TestSomething()
  {
    Expect(true, Is.True);
  }
}

And, for a while, that sufficed. But there were some aspects of this which bothered me:
  1. This was accomplished via inheritence, which just seems like "not the right way to do it".
    There have been times that I've used inheritence with test fixtures -- specifically for testing different implementations of the same interface, and also when I wrote a base class which made EF-based testing more convenient.
    Having to inherit from AssertionHelper means that I have had to push that inheritence further down, or lose the syntax.
  2. The tenses are wrong: Expect is future-tense, and Is.True is present-tense. Now, I happen to like the future-tensed Expect syntax -- it really falls in line with writing your test first:
    1. I write code to set up the test
    2. I write code to run the system-under-test
    3. I expect some results
    4. I run the test
    5. It fails!
    6. I write the code
    7. I re-run the test
    8. It passes! (and if not, I go back to #7 and practice my sad-panda face)
    9. I refactor
A few months after I started using it, a bigger bother arrived: the NUnit team was deprecating AssertionHelper because they didn't think that it was used enough in the community to warrant maintenance. A healthy discussion ensued, wherein I offered to maintain AssertionHelper and, whilst no-one objected, the discussion seemed to be moth-balled a little (things may have changed by now). Nevertheless, my code still spewed warnings and I hate warnings. I suppressed them for a while with R# comments and #pragma, but I couldn't deal -- I kept seeing them creep back in again with new test fixtures.

This led me to the first-pass: NUnit.StaticExpect where I'd realised that the existing AssertionHelper syntax could be accomplished via
  1. A very thin wrapper around Assert.That using a static class with static methods
  2. C#'s using static
This meant that the code above could become:

using static NUnit.StaticExpect.Expectations;

[TestFixture]
public class TestSystem
{
  [Test]
  public void TestSomething()
  {
    Expect(true, Is.True);
  }
}

Which was better in that:
  1. I didn't have the warning about using the Obsoleted AssertionHelper
  2. I didn't have to inherit from AssertionHelper
But there was still that odd future-present tense mix. So I started hacking about on NExpect
NExpect states as its primary goals that it wants to be:
  • Readable
    • Tests are easier to digest when they read like a story. Come to think of it, most code is. Code has two target audiences: the compiler and your co-workers (which includes you). The compiler is better at discerning meaning in a glob of logic than a human being is, which is why we try to write expressive code. It's why so many programming languages have evolved as people have sought to express their intent better.
      Your tests also form part of your documentation -- for that one target audience: developers.
  • Expressive
    • Because the intent of a test should be easy to understand. The reader can delve into the details when she cares to.
      Tests should express their intention. A block of assertions proving that a bunch of fields on one object match those on another may be technically correct and useful, but probably has meaning. Are we trying to prove that the object is a valid FrobNozzle? It would be nice if the test could say so.
  • Extensible
    • Because whilst pulling out a method like AssertIsAFrobNozzle is a good start, I was enamoured with the Jasmine way, along the lines of:

      expect(result).toBeAFrobNozzle();

      Which also negated well:

      expect(result).not.toBeAFrobNozzle();


In NExpect, you can write an extension method FrobNozzle(), dangling off of IA<T>, and write something like:
Expect(result).To.Be.A.FrobNozzle();
// or, negated
Expect(result).Not.To.Be.A.FrobNozzle();
// or, negated alternative
Expect(result).To.Not.Be.A.FrobNozzle(); The result is something which is still evolving, but is already quite powerful and useful -- and trivial to extend. I suggest checking out the demo project I made showing the evolution 
  1. from "olde" testing (Assert.AreEqual), 
  2. through the better, new Assert.That syntax of NUnit (which is quite expressive, but I really, really want to Expect and I want to be able to extend my assertions language, two features I can't get with Assert.That)
  3. through expression via AssertionHelper
  4. then NUnit.StaticExpect and finally 
  5. NExpect, including some examples of basic "matchers" (language borrowed from Jasmine): extension methods which make the tests read easier and are easy to create and re-use.
For the best effect, clone the project, reset back to the first commit and "play through" the commits.

NExpect has extensibility inspired by Jasmine and a syntax inspired by Chai (which is a little more "dotty" than Jasmine).

I've also had some great contributions from Cobus Smit , a co-worker at Chillisoft who has not only helped with extending the NExpect language, but also through trial-by-fire usage in his project.


No comments:

Post a Comment

#PeanutButter.Utils: the dictionaries Dictionaries, HashMaps, whatever you want to call them -- they can be one of the most useful constru...