Thursday, 21 September 2017

NExpect level 3: you are the key component

In previous posts, I've examined how to do simple and collection-based assertions with NExpect.

These have enabled two of the design goals of NExpect:
  • Expect(NExpect).To.Be.Readable();
    • Because code is for co-workers, not compilers. And your tests are part of your documentation.
  • Expect(NExpect).To.Be.Expressive();
    • Because the intent of a test should be easy to understand. The reader can delve into the details when she cares to.

Now, we come on to the third goal, inspired by Jasmine: easy user extension of the testing framework to facilitate expressive testing of more complex concepts.

Most of the "words" in NExpect can be "attached to" with extension methods. So the first question you have to ask is "how do I want to phrase my assertion?". You could use the already-covered .To or .To.Be:

internal static class Matchers
{
  internal static void Odd(
    this IBe be
  )
  {
    be.AddMatcher(actual =>
    {
      var passed = actual % 2 == 1;
      var message = passed
                    ? $"Expected {actual} not to be odd"
                    : $"Expected {actual} to be odd";
      return new MatcherResult(
        passed,
        message
      );
    });
  }
}

The above extension enables the following test:

  [Test]
  public void ILikeOddNumbers()
  {
    // Assert
    Expect(1).To.Be.Odd();
    Expect(2).Not.To.Be.Odd();
  }

There are a few concepts to unpack:

.AddMatcher()

This is how you add a "matcher" (term borrowed from Jasmine... Sorry, I couldn't come up with a better name, so it stuck) to a grammar continuation like .To or .Be. Note that we just create an extension method on IBe<T> where T is the type you'd like to test against, and the internals of that extension basically just add a matcher. This takes in a Func<T, IMatcherResult> so your actual matcher needs to return an IMatcher result, which really is just a flag about whether or not the test passed and the message to display if the test failed.

Pass or fail?

This is the heart of your assertion and can be as tricky as you like. Obviously, your matcher could also have multiple exit strategies with specific messages about each possible failure. But the bit that takes a little getting used to is that you're writing a matcher which could be used with .Not in the grammar chain, so you should cater for that eventuality.

Meaningful messages

There's a simple strategy here: get the passed value as if you're doing a positive assertion (ie, as if there is no .Not in the chain) and set your message as follows:
  • If you've "passed", the message will only be shown if the expectation was negated (ie, there's a .Not in the chain), so you need to negate your message (see the first message above)
  • If you've "failed", the message will only be show if the message was not negated, so you need to show the relevant message for that scenario.
It turns out that (mostly), we can write messages like so:

internal static class Matchers
{
  internal static void Odd(
    this IBe be
  )
  {
    be.AddMatcher(actual =>
    {
      var passed = actual % 2 == 1;
      var message = 
        $"Expected {actual} {passed ? "not " : ""}to be odd";
      return new MatcherResult(
        passed,
        message
      );
    });
  }
}

Doing this is tedious enough that NExpect offers a .AsNot() extension for booleans:

internal static class Matchers
{
  internal static void Odd(
    this IBe be
  )
  {
    be.AddMatcher(actual =>
    {
      var passed = actual % 2 == 1;
      var message = 
        $"Expected {actual} {passed.AsNot()}to be odd";
      return new MatcherResult(
        passed,
        message
      );
    });
  }
}

Also, NExpect surfaces a convenience extension method for printing values: .Stringify() which will:
  • print strings with quotes
  • null values as "null" 
  • and objects and collections in a "JSON-like" format. 
Use as follows:

internal static class NumberMatchers
{
  internal static void Odd(
    this IBe be
  )
  {
    be.AddMatcher(actual =>
    {
      var passed = actual % 2 == 1;
      var message = 
        $"Expected {actual.Stringify()} {passed.AsNot()}to be odd";
      return new MatcherResult(
        passed,
        message
      );
    });
  }
}

You'll have to think (a little) about your first matcher, but it starts getting easier the more you write (:
Now you can write more meaningful tests like those in the demo project.

The above is fine if, like me, you can see a pattern you'd like to test for really easily (if you have a kind of "matcher-first" mindset), but does provide a minor barrier to entry for those who like to write a few tests and refactor out common assertions.

Not to worry: NExpect has you covered with .Compose():

internal static class PersonMatchers
{
  internal static void Jane(
    this IA a
  )
  {
     a.Compose(actual =>
     {
        Expect(actual.Id).To.Equal(1);
        Expect(actual.Name).To.Equal("Jane");
     });
  }

// ....
  [Test]
  public void TestJane()
  {
    // Arrange
    var person = new Person() { Id = 1, Name = "Jane", Alive = true };

    // Assert
    Expect(person).To.Be.A.Jane();
  }
}

.Compose uses [CallerMemberName] to determine the name of your matcher and attempts to throw a useful UnmetExpectationException when one of your sub-expectations fails. You may also provide a Func to .Compose to generate a more meaningful message.

These are some rather simple examples -- I'm sure you can get much more creative! I know I have (:

Some parts of NExpect are simply there to make fluent extension easier, for example:
  • To.Be.A
  • To.Be.An
  • To.Be.For
NExpect will be extended with more "grammar parts" as the need arises. If NExpect is missing a "grammar dangler" that you'd like, open an issue on GitHub.
 

NExpect level 2: testing collections

In a prior post, I covered simple value testing with NExpect. In this post, I'd like to delve into collection assertions, since they are fairly common.

First, the simplest: asserting that a collection contains a desired value:


  [Test]
  public void SimpleContains()
  {
    // Arrange
    var collection = new[] { "a", "b", "c" };
  
    // Assert
    Expect(collection).To.Contain("a");
  }

This is what you would expect from any other assertions framework.

Something has always bothered me about this kind of testing though. In particular, the test above passes just as well as this one:


  [Test]
  public void MultiContains()
  {
    // Arrange
    var collection = new[] { "a", "b", "c", "a" };
  
    // Assert
    Expect(collection).To.Contain("a");
  }

And yet they are not functionally equivalent from where I stand. Which makes the test feel a little flaky to me. This is why NExpect actually didn't even have the above assertion first. Instead, I was interested in being more specific:

  [Test]
  public void SpecificContains()
  {
    // Arrange
    var collection = new[] { "a", "b", "c", "a" };
  
    // Assert
    Expect(collection).To.Contain.Exactly(1).Equal.To("b");
    Expect(collection).To.Contain.At.Least(1).Equal.To("c");
    Expect(collection).To.Contain.At.Most(2).Equal.To("a");
  }

Now my tests are speaking specifically about what they expect.

Sometimes you just want to test the size of a collection, but you don't really care if it's an IEnumerable<T>, a List<T> or an array. Other testing frameworks may let you down, requiring you to write a test against the Count or Length property, meaning that when your implementation changes from returning, eg, List<T> to array (which may be smart: List<T> is not only a heavier construct but implies that you can add to the collection), your tests will fail for no really good reason -- your implementation still returns 2 FrobNozzles, so who cares if the correct property to check is Length or Count? I know that I don't.

That's Ok, NExpect takes away the care of having to consider that nuance and allows you to spell out what you actually mean:

  [Test]
  public void SizeTest()
  {
    // Arrange
    var collection = new[] { "a", "b", "c" };
    var lonely = new[] { 1 };
    var none = new bool[0];

    // Assert
    Expect(collection).To.Contain.Exactly(3).Items();
    Expect(lonely).To.Contain.Exactly(1).Item();

    Expect(none).To.Contain.No().Items();
    Expect(none).Not.To.Contain.Any().Items();
    Expect(none).To.Be.Empty();
  }

Note that the last three are functionally equivalent. They are just different ways to say the same thing. NExpect is designed to help you express your intent in your tests, and, as such, there may be more than one way to achieve the same goal:

  [Test]
  public void AnyWayYouLikeIt()
  {
    // Assert
    Expect(1).Not.To.Equal(2);
    // ... is exactly equivalent to
    Expect(1).To.Not.Equal(2);

    Expect(3).To.Equal(3);
    // ... is exactly equivalent to
    Expect(3).To.Be.Equal.To(3);
  }

There are bound to be other examples. The point is that NExpect attempts to provide you with the language to write your assertions in a readable manner without enforcing a specific grammar.

Anyway, on with collection testing!

You can test for equality, meaning items match at the same point in the collection (this is not reference equality testing on the collection, but would equate to reference equality testing on items of class type or value equality testing on items of struct type:

  [Test]
  public void CollectionEquality()
  {
    // Assert
    Expect(new[] { 1, 2, 3 })
      .To.Be.Equal.To(new[] { 1, 2, 3 });
  }

You can also test out-of-order:

  [Test]
  public void CollectionEquivalence()
  {
    // Assert
    Expect(new[] { 3, 1, 2 })
      .To.Be.Equivalent.To(new[] { 1, 2, 3 });
  }

Which is all nice and dandy if you're testing value types or can do reference equality testing (or at least testing where each object has a .Equals override which does the comparison for you). It doesn't help when you have more complex objects -- but NExpect hasn't forgotten you there: you can do deep equality testing on collections too:

  [Test]
  public void CollectionDeepEquality()
  {
    var input = new[] {
      new Person() { Id = 1, Name = "Jane", Alive = true },
      new Person() { Id = 2, Name = "Bob", Alive = false }
    };
    // Assert
    Expect(input.AsObjects())
      .To.Be.Deep.Equal.To(new[] 
        {
          new { Id = 1, Name = "Jane", Alive = true },
          new { Id = 2, Name = "Bob", Alive = false }
        });
  }

Note that, much like the points on "Who's line is it, anyway?", the types don't matter. This is deep equality testing (: However, we did need to "dumb down" the input collection to a collection of objects with the provided .AsObjects() extension method so that the test would compile, otherwise there's a type mismatch at the other end. Still, this is, imo, more convenient than the alternative: item-for-item testing, property-by-property.

The above is incomplete without equivalence, of course:

  [Test]
  public void CollectionDeepEquivalence()
  {
    var input = new[] {
      new Person() { Id = 1, Name = "Jane", Alive = true },
      new Person() { Id = 2, Name = "Bob", Alive = false }
    };
    // Assert
    Expect(input.AsObjects())
      .To.Be.Deep.Equivalent.To(new[] {
        new { Id = 2, Name = "Bob", Alive = false },
        new { Id = 1, Name = "Jane", Alive = true }
      });
  }

And intersections are thrown in for good measure:

  [Test]
  public void CollectionIntersections()
  {
    var input = new[] {
      new Person() { Id = 1, Name = "Jane", Alive = true },
      new Person() { Id = 2, Name = "Bob", Alive = false }
    };
    // Assert
    Expect(input.AsObjects())
      .To.Be.Intersection.Equivalent.To(new[] {
        new { Id = 2, Name = "Bob" },
        new { Id = 1, Name = "Jane" }
      });
    Expect(input.AsObjects())
      .To.Be.Intersection.Equial.To(new[] {
        new { Id = 1, Name = "Jane" },
        new { Id = 2, Name = "Bob" }
      });
  }

You can also test with a custom IEqualityComparer<T>:

  [Test]
  public void CollectionIntersections()
  {
    var input = new[] {
      new Person() { Id = 1, Name = "Jane", Alive = true },
      new Person() { Id = 2, Name = "Bob", Alive = false }
    };
    // Assert
    Expect(input)
      .To.Contain.Exactly(1).Equal.To(
        new Person() { Id = 2, Name = "Bob" }, 
        new PersonEqualityComparer()
    );
  }

or with a quick-and-dirty Func<T>:

  [Test]
  public void CollectionIntersections()
  {
    var input = new[] {
      new Person() { Id = 1, Name = "Jane", Alive = true },
      new Person() { Id = 2, Name = "Bob", Alive = false }
    };
    // Assert
    Expect(input.AsObjects())
      .To.Contain.Exactly(1).Matched.By(
        p => p.Id == 1 && p.Name == "Jane"
      );
  }

And all of this is really just the start. The real expressive power of NExpect comes in how you extend it.

But more on that in the next episode (:

NExpect level 1: testing objects and values

I recently introduced NExpect as an alternative assertions library. I thought it might be nice to go through usage, from zero to hero.

NExpect is available for .NET Framework 4.5.2 and above as well as anything which can target .NET Standard 1.6 (tested with .NET Core 2.0)

So here goes, level 1: testing objects and values.

NExpect facilitates assertions (or, as I like to call them: expectations) against basic value types in a fairly unsurprising way:


  [Test]
  public void SimplePositiveExpectations
  {
    // Arrange
    object obj = null;
    var intValue = 1;
    var trueValue = true;
    var falseValue = false;

    // Assert
    Expect(obj).To.Be.Null();
    Expect(intValue).To.Equal(1);
    Expect(trueValue).To.Be.True();
    Expect(falseValue).To.Be.False();
  }

So far, nothing too exciting or unexpected there. NExpect also caters for negative expectations:

  [Test]
  public void SimpleNegativeExpectations
  {
    // Arrange
    object obj = new object();
    var intValue = 1;
    var trueValue = true;
    var falseValue = false;

    // Assert
    Expect(obj).Not.To.Be.Null();
    Expect(intValue).Not.To.Equal(2);
    Expect(trueValue).Not.To.Be.False();
    Expect(falseValue).Not.To.Be.True();

    Expect(intValue).To.Be.Greater.Than(0);
    Expect(intValue).To.Be.Less.Than(10);
  }

(Though, in the above, I'm sure we all agree that the boolean expectations are neater without the .Not).

Expectations carry type forward, so you won't be able to, for example:

  [Test]
  public void ExpectationsCarryType
  {
    Expect(1).To.Equal("a");  // does not compile!
  }

However, expectations around numeric values perform upcasts in much the same way that you'd expect in live code, such that you can:


  [Test]
  public void ShouldUpcastAsRequired()
  {
    // Arrange
    int a = 1;
    byte b = 2;
    uint c = 3;
    long d = 4;

    // Assert 
    Expect(b).To.Be.Greater.Than(a);
    Expect(c).To.Be.Greater.Than(b);
    Expect(d).To.Be.Greater.Than(a);
  }

All good and well, but often we need to check that a more complex object has a bunch of expected properties.

.Equal is obviously going to do reference-equality testing for class types and value equality testing for struct types. We could:

  [Test]
  public void TestingPropertiesOneByOne()
  {
    // Arrange
    var person = new {
      Id = 1,
      Name = "Jane",
      Alive = true
    };

    // Assert
    Expect(person.Id).To.Equal(1);
    Expect(person.Name).To.Equal("Jane");
    Expect(person.Alive).To.Be.True();
  }

But that kind of test, whilst perfectly accurate, comes at a cognitive overhead for the reader. Ok, perhaps not much overhead in this example, but imagine if that person had come from another method:

  [Test]
  public void TestingPropertiesOneByOne()
  {
    // Arrange
    var sut = new PersonRepository();
    
    // Act
    var person = sut.FindById(1);

    // Assert
    Expect(person).Not.To.Be.Null();
    Expect(person.Id).To.Equal(1);
    Expect(person.Name).To.Equal("Jane");
    Expect(person.Alive).To.Be.True();
  }

In this case, we'd expect the result to also have a defined type, not some anonymous type. It would be super-convenient if we could do deep equality testing. Which we can:

  [Test]
  public void DeepEqualityTesting()
  {
    // Arrange
    var sut = new PersonRepository();
    
    // Act
    var person = sut.FindById(1);

    // Assert
    Expect(person).To.Deep.Equal(new {
      Id = 1,
      Name = "Jane",
      Alive = 1
    });
  }
This exposes our test for what it's really doing: when searching for the person with the Id of 1, we should get back an object which describes Jane in our system. Our test is speaking about intent, not just confirming value equality. Notice that the type of the object used for comparison doesn't matter, and this holds for properties too.

Note that I omitted the test for null in the second variant. You don't need it because the deep equality tester will deal with that just fine. However, you are obviously still free to include it for the sake of clarity.

NExpect gets this "for free" by depending on a git submodule of PeanutButter and importing only the bits it needs. In this way, I can re-use well-tested code and consumers don't have to depend on another Nuget package. Seems like a win to me.

What if we didn't care about all of the properties? What if we only cared about, for example, Name and Id. A dead Jane is still a Jane, right?

NExpect has you covered:

  [Test]
  public void IntersectionEqualityTesting()
  {
    // Arrange
    var sut = new PersonRepository();
    
    // Act
    var person = sut.FindById(1);

    // Assert
    Expect(person).To.Intersection.Equal(new {
      Id = 1,
      Name = "Jane"
    });
  }


We can also test the type of a returned object:

  [Test]
  public void TypeTesting()
  {
    // Arrange
    var sut = new PersonRepository();

    // Act
    var person = sut.FindById(1);

    // Assert
    Expect(person).To.Be.An.Instance.Of<Person>();
    Expect(person).To.Be.An.Instance.Of<IPerson>();
    Expect(person).To.Be.An.Instance.Of<BaseEntity>();
  }

We can test for the exact type (Person), implemented interfaces (IPerson) and base types (BaseEntity).

Next, we'll explore simple collection testing. Tune in for Level 2!

Monday, 18 September 2017

Fluent, descriptive testing with NExpect

About a year or so ago, I discovered AssertionHelper, a base class provided by NUnit which allowed for a more familiar style of testing when one has to bounce back and forth between (Java|Type)Script and C#. Basically, it allows one to use the Expect keyword to start an assertion, eg:

[TestFixture]
public class TestSystem: AssertionHelper
{
  [Test]
  public void TestSomething()
  {
    Expect(true, Is.True);
  }
}

And, for a while, that sufficed. But there were some aspects of this which bothered me:
  1. This was accomplished via inheritence, which just seems like "not the right way to do it".
    There have been times that I've used inheritence with test fixtures -- specifically for testing different implementations of the same interface, and also when I wrote a base class which made EF-based testing more convenient.
    Having to inherit from AssertionHelper means that I have had to push that inheritence further down, or lose the syntax.
  2. The tenses are wrong: Expect is future-tense, and Is.True is present-tense. Now, I happen to like the future-tensed Expect syntax -- it really falls in line with writing your test first:
    1. I write code to set up the test
    2. I write code to run the system-under-test
    3. I expect some results
    4. I run the test
    5. It fails!
    6. I write the code
    7. I re-run the test
    8. It passes! (and if not, I go back to #7 and practice my sad-panda face)
    9. I refactor
A few months after I started using it, a bigger bother arrived: the NUnit team was deprecating AssertionHelper because they didn't think that it was used enough in the community to warrant maintenance. A healthy discussion ensued, wherein I offered to maintain AssertionHelper and, whilst no-one objected, the discussion seemed to be moth-balled a little (things may have changed by now). Nevertheless, my code still spewed warnings and I hate warnings. I suppressed them for a while with R# comments and #pragma, but I couldn't deal -- I kept seeing them creep back in again with new test fixtures.

This led me to the first-pass: NUnit.StaticExpect where I'd realised that the existing AssertionHelper syntax could be accomplished via
  1. A very thin wrapper around Assert.That using a static class with static methods
  2. C#'s using static
This meant that the code above could become:

using static NUnit.StaticExpect.Expectations;

[TestFixture]
public class TestSystem
{
  [Test]
  public void TestSomething()
  {
    Expect(true, Is.True);
  }
}

Which was better in that:
  1. I didn't have the warning about using the Obsoleted AssertionHelper
  2. I didn't have to inherit from AssertionHelper
But there was still that odd future-present tense mix. So I started hacking about on NExpect
NExpect states as its primary goals that it wants to be:
  • Readable
    • Tests are easier to digest when they read like a story. Come to think of it, most code is. Code has two target audiences: the compiler and your co-workers (which includes you). The compiler is better at discerning meaning in a glob of logic than a human being is, which is why we try to write expressive code. It's why so many programming languages have evolved as people have sought to express their intent better.
      Your tests also form part of your documentation -- for that one target audience: developers.
  • Expressive
    • Because the intent of a test should be easy to understand. The reader can delve into the details when she cares to.
      Tests should express their intention. A block of assertions proving that a bunch of fields on one object match those on another may be technically correct and useful, but probably has meaning. Are we trying to prove that the object is a valid FrobNozzle? It would be nice if the test could say so.
  • Extensible
    • Because whilst pulling out a method like AssertIsAFrobNozzle is a good start, I was enamoured with the Jasmine way, along the lines of:

      expect(result).toBeAFrobNozzle();

      Which also negated well:

      expect(result).not.toBeAFrobNozzle();


In NExpect, you can write an extension method FrobNozzle(), dangling off of IA<T>, and write something like:
Expect(result).To.Be.A.FrobNozzle();
// or, negated
Expect(result).Not.To.Be.A.FrobNozzle();
// or, negated alternative
Expect(result).To.Not.Be.A.FrobNozzle(); The result is something which is still evolving, but is already quite powerful and useful -- and trivial to extend. I suggest checking out the demo project I made showing the evolution 
  1. from "olde" testing (Assert.AreEqual), 
  2. through the better, new Assert.That syntax of NUnit (which is quite expressive, but I really, really want to Expect and I want to be able to extend my assertions language, two features I can't get with Assert.That)
  3. through expression via AssertionHelper
  4. then NUnit.StaticExpect and finally 
  5. NExpect, including some examples of basic "matchers" (language borrowed from Jasmine): extension methods which make the tests read easier and are easy to create and re-use.
For the best effect, clone the project, reset back to the first commit and "play through" the commits.

NExpect has extensibility inspired by Jasmine and a syntax inspired by Chai (which is a little more "dotty" than Jasmine).

I've also had some great contributions from Cobus Smit , a co-worker at Chillisoft who has not only helped with extending the NExpect language, but also through trial-by-fire usage in his project.


This week in PeanutButter

Nothing major, really -- two bugfixes, which may or may not be of interest:

  1. PropertyAssert.AreEqual allows for comparison of nullable and non-nullable values of the same underlying type -- which especially makes sense when the actual value being tested (eg nullable int) is being compared with some concrete value (eg int). 
  2. Fix for the object extension DeepClone() -- some production code showed that Enum values weren't being copied correctly. So that's fixed now.
    If you're wondering what this is, DeepClone() is an extension method on all objects to provide a copy of that object, deep-cloned (so all reference types are new types, all value types are copied), much like underscore's, _.cloneDeep() for Javascript. This can be useful for comparing a "before" and "after" from a bunch of mutations, especially using the DeepEqualityTester from PeanutButter.Utils or the object extension DeepEquals(), which does deep equality testing, much like you'd expect.

There's also been some assertion upgrading -- PeanutButter consumes, and helps to drive NExpect, an assertions library modelled after Chai for syntax and Jasmine for user-space extensibility. Head on over to Github to check it out -- though it's probably time I wrote something here about it (:

Friday, 4 August 2017

This week in PeanutButter

Ok, so I'm going to give this a go: (semi-)regularly blogging about updates to PeanutButter in the hopes that perhaps someone sees something useful that might help out in their daily code. Also so I can just say "read my blog" instead of telling everyone manually ^_^

So this week in PeanutButter, some things have happened:

  • DeepEqualityTester fuzzes a little on numerics -- so you can compare numerics of equal value and different type correctly (ie, (int)2 == (decimal)2). This affects the {object}.DeepEquals() and PropertyAssert.AreDeepEqual() methods.
  • DeepEqualityTester can compare fields now too. PropertyAssert.DeepEquals will not use this feature (hey, the name is PropertyAssert!), but {object}.DeepEquals() will, by default -- though you can disable this.
  • DuckTyper could duck Dictionaries to interfaces and objects to interfaces -- but now will also duck objects with Dictionary properties to their respective interfaces where possible.
  • String extensions for convenience:
    • ToKebabCase()
    • ToPascalCase()
    • ToSnakeCase()
    • ToCamelCase()
  • DefaultDictionary<TKey, TValue> - much like Python's defaultdict, this provides a dictionary where you give a strategy for what to return when a key is not present. So a DefaultDictionary<string, bool> could have a default value of true or false instead of throwing exceptions on unknown keys.
  • MergeDictionary<TKey, TValue> - provides a read-only "view" on a collection of similarly typed IDictionary<TKey, TValue> objects, resolving values from the first dictionary they are found in. Coupled with DefaultDictionary<TKey, TValue>, you can create layered configurations with fallback values.
  • DuckTyping can duck from string values to enums
And I'm quite sure the reverse (enum to string) will come for cheap-or-free. So there's that (: You might use this when you have, for example, a Web.Config with a config property "Priority" and you would like to end up with an interface like:

public enum Priorities
{
  Low,
  Medium,
  High
}
public interface IConfig
{
  Priorities DefaultPriority { get; }
}


And a Web.Config line like:
<appSettings>
  <add key="DefaultPriority" value="Medium" />
</appSettings>


Then you could, somewhere in your code (perhaps in your IOC bootstrapper) do:
 var config = WebConfigurationManager.AppSettings.DuckAs<IConfig>();

(This already works for string values, but enums are nearly there (:). You can also use FuzzyDuckAs<T>(), which will allow type mismatching (to a degree; eg a string-backed field can be surfaced as an int) and will also give you freedom with your key names: whitespace and casing don't matter (along with punctuation). (Fuzzy)DuckAs<T>() also has options for key prefixing (so you can have "sections" of settings, with a prefix, like "web.{setting}" and "database.{setting}". But all of that isn't really from this week -- it's just useful for lazy devs like me (:

Saturday, 25 March 2017

C# Evolution

(and why you should care)

I may be a bit of a programming language nut. I find different languages interesting not just because they are semantically different or because they offer different features or even because they just look pretty (I'm lookin' at you, Python), but because they can teach us new things about the languages we already use.

I'm of the opinion that no technology with any staying power has no value to offer. In other words, if the tech has managed to stay around for some time, there must be something there. You can hate on VB6 as much as you want, but there has to be something there to have made it dominate desktop programming for the years that it did. That's not the focus of this discussion though.

Similarly, when new languages emerge, instead of just rolling my eyes and uttering something along the lines of "Another programming language? Why? And who honestly cares?", I prefer to take a step back and have a good long look. Creating a new language and the ecosystem required to interpret or compile that language is a non-trivial task. It takes a lot of time and effort, so even when a language seems completely impractical, I like to consider why someone might spend the time creating it. Sure, there are outliers where the only reason is "because I could" (I'm sure Shakespeare has to be one of those) or "because I hate everyone else" (Brainfuck, Whitespace and my favorite to troll on, Perl -- I jest, because Perl is a powerhouse; I just haven't ever seen a program written in Perl which didn't make me cringe, though Larry promises that is supposed to change with Perl 6).

Most languages are born because someone found the language they were dealing with was getting in the way of getting things done. A great example is Go, which was dreamed up by a bunch of smart programmers who had been doing this programming thing for a while and really just wanted to make an ecosystem which would help them to get stuff done in a multi-core world without having to watch out for silly shit like deadlocks and parallel memory management. Not that you can't hurt yourself in even the most well-designed language (indeed, if you definitely can't hurt yourself in the language you're using, you're probably bound and gagged by it -- but I'm not here to judge what you're into).

Along the same lines, it's interesting to watch the evolution of languages, especially as languages evolve out of fairly mundane, suit-and-tie beginnings. I feel like C# has done that -- and will continue to do so. Yes, a lot of cool features aren't unique or even original -- deconstruction in C#7 has been around for ages in Python, for example -- but that doesn't make them any less valuable.

I'm going to skip some of the earlier iterations and start at where I think it's interesting: C#5. Please note that this post is (obviously) opinion. Whilst I'll try to cover all of the feature changes that I can dig up, I'm most likely going to focus on the ones which I think provide the programmer the most benefit.

C#5

C#5 brought async features and caller information. Let's examine the latter before I offer up what will probably be an unpopular opinion on the former.

Caller information allowed providing attributes on optional functional parameters such that if the caller didn't provide a value, the called code could glean
  • Caller file path
  • Caller line number
  • Caller member name
This is a bonus for logging and debugging of complex systems, but also allowed tricks whereby the name of a property could be automagically passed into, for example, an MVVM framework method call in your WPF app. This makes refactor-renaming easier and removes some magic strings, not to mention making debug logging way way simpler. Not ground-breaking, but certainly [CallerMemberName] became a friend of the WPF developer. A better solution  for the exact problem of property names and framework calls came with nameof in C#6, but CallerMember* attributes are still a good thing.



C# 5 also brought async/await. On the surface, this seems like a great idea. If only C#'s async/await worked anything like the async/await in Typescript, where, under the hood, there's just a regular old promise and no hidden bullshit. C#'s async/await looks like it's just going to be Tasks and some compiler magic under the hood, but there's that attached context which comes back to bite asses more often than a mosquito with a scat fetish. There are some times when things will do exactly what you expect and other times when they won't, just because you don't know enough about the underlying system.

That's the biggest problem with async/await, in my opinion: it looks like syntactic sugar to take away the pain of parallel computing from the unwashed masses but ends up just being trickier than having to learn how to do actual multi-threading. There's also the fickle task scheduler which may decide 8 cores doesn't mean 8 parallel tasks -- but that's OK: you can swap that out for your own task scheduler... as long as you understand enough of the underlying system, again (as this test code demonstrates).

Like many problems that arise out of async / parallel programming, tracking down the cause of sporadic issues in code is non-trivial. I had a bit of code which would always fail the first time, and then work like a charm -- until I figured out it was the context that was causing issues, so I forced a null context and the problem stopped. The developer has to start learning about task continuation options and start caring about how external libraries do things. And many developers aren't even aware of when it's appropriate to use async/await, opting to use it everywhere and just add overhead to something which really didn't need to be async, like most web api controllers. Async/await makes a lot of sense in GUI applications though.

Having async/await around IO-bound stuff in web api calls may be good for high-volume web requests because it allows the worker thread to be re-assigned to handle another request. I have yet to see an actual benchmark showing better web performance from simply switching to async/await for all calls.
The time you probably most want to use it is for concurrent IO to shorten the overall request time for a single request. Some thought has to go into this though -- handling too requests concurrently may just end up with many requests timing out instead of a few requests being given a quick 503, indicating that the application needs some help with scaling. In other words, simply peppering your code with async/await could result in no net gain, especially if the code being awaited is hitting the same resource (eg, your database).

Which leads to the second part of C#'s async/await that I hate: the async zombie apocalypse. Because asking the result of a method marked async to just .Wait() is suicide (I hope you know it is; please, please, please don't do that), async/await patterns tend to propagate throughout code until everything is awaiting some async function. It's the Walking Dead, traipsing through your code, leaving little async/await turds wherever they go.




You can use ConfigureAwait() to get around deadlocking on the context selected for your async code -- but you must remember to apply it to all async results if you're trying to "un-async" a block of code. You can also set the synchronization context (I suggest the useful value of null). Like the former workaround, it's hardly elegant.

As much as I hate (fear?) async/await, there are places where it makes the code clearer and easier to deal with. Mostly in desktop applications, under event-handling code. It's not all bad, but the concept has been over-hyped and over-used (and poorly at that -- like EF's SaveChangesAsync(), which you might think, being async, is thread-safe, and you'd be DEAD WRONG).

Let's leave it here: use async/await when it provides obvious value. Question it's use at every turn. For every person who loves async/await, there's a virtual alphabet soup of blogs explaining how to get around some esoteric failure brought on by the feature. As with multi-threading in C/C++: "use with caution".



C#6

Where C#5 brought low numbers of feature changes (but one of the most damaging), C#6 brought a smorgasbord of incremental changes. There was something there for everyone:

Read-only auto properties made programming read-only properties just a little quicker, especially when the property values came from the constructor. So code like:

public class VideoGamePlumber
{
  public string Name { get { return _name; } }
  private string _name;

  public VideoGamePlumber(string name)
  {
    _name = name;
  }
}
became:

public class VideoGamePlumber
{
  public string Name { get; private set; }

  public VideoGamePlumber(string name)
  {
    Name = name;
  }
}
but that still leaves the Name property open for change within the VideoGamePlumber class, so better would be the C#6 variant:

public class VideoGamePlumber
{
  public string Name { get; }

  public VideoGamePlumber(string name)
  {
    Name = name;
  }
}
The Name property can only be set from within the constructor. Unless, of course, you resort to reflection, since the mutability of Name is enforced by the compiler, not the runtime. But I didn't tell you that.



Auto-property initializers seem quite convenient, but I'll admit that I rarely use them, I think primarily because the times that I want to initialize outside of the constructor, I generally want a private (or protected) field, and when I want to set a property at construction time, it's probably getting it's value from a constructor parameter. I don't hate the feature (at all), just don't use it much. Still, if you wanted to:
public class Doggie
{
  public property Name { get; set; }

  public Doggie()
  {
    Name = "Rex"; // set default dog name
  }
}
becomes:
public class Doggie
{
  public property Name { get; set; } = "Rex";
}
You can combine this with the read-only property syntax if you like:
public class Doggie
{
  public property Name { get; } = "Rex";
}
but then all doggies are called Rex (which is quite presumptuous) and you really should have just used a constant, which you can't modify through reflection.



Expression-bodied function members can provide a succinct syntax for a read-only, calculated property. However, I use them sparingly because anything beyond a very simple "calculation", eg:

public class Person
{
  public string FirstName { get; }
  public string LastName { get; }
  public string FullName => $"{FirstName} {LastName}";

  public Person(string firstName, string lastName)
  {
    FirstName = firstName;
    LastName = lastName;
  }
}
starts to get long and more difficult to read; though that argument is simply countered by having a property like:

public class Business
{
  public string StreetAddress { get; }
  public string Suburb { get; }
  public string Town { get; }
  public string PostalCode { get; }
  public string PostalAddress => GetPostalAddress();

  public Business(string streetAddress, string suburb, string Town, string postalCode)
  {
    StreetAddress = streetAddress;
    Suburb = suburb;
    Town = town;
    PostalCode = postalCode
  }
  
  private string GetAddress()
  {
    return string.Join("\n", new[]
    {
      StreetAddress,
      Suburb,
      Town,
      PostalCode
    });
  }
}
Where the logic for generating the full address from the many bits of involved data is tucked away in a more readable method and the property itself becomes syntactic sugar, looking less clunky than just exposing the GetAddress method.

Index initializers provide a neater syntax for initializing Dictionaries, for example:
var webErrors = new Dictionary()
{
  { 404, "Page Not Found" },
  { 302, "Page moved, but left a forwarding address" },
  { 500, "The web server can't come out to play today" }
};

Can be written as:
private Dictionary webErrors = new Dictionary
{
  [404] = "Page not Found",
  [302] = "Page moved, but left a forwarding address.",
  [500] = "The web server can't come out to play today."
};

It's not ground-breaking, but I find it a little more pleasing on the eye.

Other stuff that I hardly use includes:

Extension Add methods for collection initializers  allow your custom collections to be initialized like standard ones. Not a feature I've ever used because I haven't had the need to write a custom collection.

Improved overload resolution reduced the number of times I shook my fist at the complainer compiler. Ok, so I technically use this, but this is one of those features that, when it's working, you don't even realise it.

Exception filters made exception handling more expressive and easier to read.

await in catch and finally blocks allows the async/await zombies to stumble into your exception hanlding. Yay.



On to the good bits (that I regularly use) though:

using static made using static functions so much neater -- as if they were part of the class you were currently working in. I don't push static functions in general because using them means that testing anything which uses them has to test them too, but there are places where they make sense. One is in RandomValueGen from PeanutButter.RandomGenerators, a class which provides functions to generate random data for testing purposes. A static import means you no longer have to mention the RandomValueGen class throughout your test code:

using PeanutButter.RandomGenerators;

namespace Bovine.Tests
{
  [TestFixture]
  public class TestCows
  {
    [Test]
    public void Moo_ShouldBeAnnoying()
    {
      // Arrange
      var cow = new Cow()
      {
        Name = RandomValueGen.GetRandomString(),
        Gender = RandomValueGen.GetRandom(),
        Age = RandomValueGen.GetRandomInt()
      };
      // ...
    } 
  }
}

Can become:
using static PeanutButter.RandomGenerators.RandomValueGen;

namespace Bovine.Tests
{
  [TestFixture]
  public class TestCows
  {
    [Test]
    public void Moo_ShouldBeAnnoying()
    {
      // Arrange
      var cow = new Cow()
      {
        Name = GetRandomString(),
        Gender = GetRandom(),
        Age = GetRandomInt()
      };
      // ...
    } 
  }
}


Which is way more readable simply because there's less unnecessary cruft in there. At the point of reading (and writing) the test, source library and class for random values is not only irrelevant and unnecessary -- it's just plain noisy and ugly.


Null conditional operators. Transforming fugly multi-step checks for null into neat code:

if (thing != null &&
    thing.OtherThing != null &&
    thing.OtherThing.FavoriteChild != null &&
    // ... and so on, and so forth, turtles all the way down
    //     until, eventually
    this.OtherThing.FavoriteChild.Dog.Collar.Spike.Metal.Manufacturer.Name != null)
{
  return this.OtherThing.FavoriteChild.Dog.Collar.Spike.Metal.Manufacturer.Name;
}
return "Unknown manufacturer";

becomes:
return thing
  ?.OtherThing
  ?.FavoriteChild
  ?.Dog
  ?.Collar
  ?.Spike
  ?.Metal
  ?.Manufacturer
  ?.Name
  ?? "Unknown manufacturer";

and kitties everywhere rejoiced:


String interpolation helps you to turn disaster-prone code like this:

public void PrintHello(string salutation, string firstName, string lastName)
{
  Console.WriteLine("Hello, " + salutation + " " + firstName + " of the house " + lastName);
}

or even the less disaster-prone, more efficient, but not that amazing to read:
public void PrintHello(string salutation, string firstName, string lastName)
{
  Console.WriteLine(String.Join(" ", new[]
  {
    "Hello,",
    salutation,
    firstName,
    "of the house",
    lastName
  }));
}

Into the safe, readable:
public void PrintHello(string salutation, string firstName, string lastName)
{
  Console.WriteLine($"Hello, {salutation} {firstName} of the house {lastName}");
}



nameof is also pretty cool, not just for making your constructor null-checks impervious to refactoring:

public class Person
{
  public Person(string name)
  {
    if (name == null) throw new ArgumentNullException(nameof(name));
  }
}


(if you're into constructor-time null-checks) but also for using test case sources in NUnit:
[Test, TestCaseSource(nameof(DivideCases))]
public void DivideTest(int n, int d, int q)
{
  Assert.AreEqual( q, n / d );
}

static object[] DivideCases =
{
  new object[] { 12, 3, 4 },
  new object[] { 12, 2, 6 },
  new object[] { 12, 4, 3 } 
};

C#7

This iteration of the language brings some  neat features for making code more succinct and easier to grok at a glance.
Inline declaration of out variables makes using methods with out variables a little prettier. This is not a reason to start using out parameters: I still think that there's normally a better way to do whatever it is that you're trying to achieve with them and use of out and ref parameters is, for me, a warning signal in the code of a place where something unexpected could happen. In particular, using out parameters for methods can make the methods really clunky because you have to set them before returning, making quick returns less elegant. Part of me would have liked them to be set to the default value of the type instead, but I understand the rationale behind the compiler not permitting a return before setting an out parameter: it's far too easy to forget to set it and end up with strange behavior.

I think I can count on one hand the number of times I've written a method with an out or ref parameter and I couldn't even point you at any of them. I totally see the point of ref parameters for high-performance code where it makes sense (like manipulating sound or image data). I just really think that when you use out or ref, you should always ask yourself "is there another way to do this?". Anyway, my opinions on the subject aside, there are times when you have to interact with code not under your control and that code uses out params, for example:

public void PrintNumeric(string input)
{
  int result;
  if (int.TryParse(input, out result))
  {
    Console.WriteLine($"Your number is: {input}");
  }
  else
  {
    Console.WriteLine($"'{input}' is not a number )':");
  }
}

becomes:
public void PrintNumeric(string input)
{
  if (int.TryParse(input, out int result))
  {
    Console.WriteLine($"Your number is: {input}");
  }
  else
  {
    Console.WriteLine($"'{input}' is not a number )':");
  }
}

It's a subtle change, but if you have to use out parameters enough, it becomes a lot more convenient, less noisy.

Similarly ref locals and returns, where refs are appropriate, can make code much cleaner. The general use-case for these is to return a reference to a non-reference type for performance reasons, for example when you have a large set of ints, bytes, or structs and would like to pass off to another method to find the element you're interested in before modifying it. Instead of the finder returning an index and the outer call re-referencing into the array, the finder method can simply return a reference to the found element so the outer call can do whatever manipulations it needs to. I can see the use case for performance reasons in audio and image processing as well as large sets of structs of data. The example linked above is quite comprehensive and the usage is for more advanced code, so I'm not going to rehash it here.

Tuples have been available in .NET for a long time, but they've been unfortunately cumbersome. The new syntax in C#7 is changing that. Python tuples have always been elegant and now a similar elegance comes to C#:
public static class TippleTuple
{
  public static void Main()
  {
    // good
    (int first, int second) = MakeAnIntsTuple();
    Console.WriteLine($"first: {first}, second: {second}");
    // better
    var (flag, message) = MakeAMixedTuple();
    Console.WriteLine($"first: {flag}, second: {message}");
  }

  public static (int first, int second) MakeAnIntsTuple()
  {
    return (1, 2);
  }

  public static (bool flag, string name) MakeAMixedTuple()
  {
    return (true, "Moo");
  }
}

Some notes though: the release notes say that you should need to install the System.ValueTuple nuget package to support this feature in VS2015 and lower, but I found that I needed to install the package on VS2017 too. Resharper still doesn't have a clue about the feature as of 2016.3.2, so usages within the interpolated strings above are highlighted as errors. Still, the program above compiles and is way more elegant than using Tuple<> generic types. It's very clever that language features can be delivered by nuget packages though.

Local functions provide a mechanism for keeping little bits of re-used code local within a method. In the past, I've used a Func variable where I've had a little piece of re-usable logic:
private NameValueCollection CreateRandomSettings()
{
  var result = new NameValueCollection();
  Func randInt = () => RandomValueGen.GetRandomInt().ToString();
  result["MaxSendAttempts"] = randInt();
  result["BackoffIntervalInMinutes"] = randInt();
  result["BackoffMultiplier"] = randInt();
  result["PurgeMessageWithAgeInDays"] = randInt();
  return result;
}

which can become:
public static NameValueCollection GetSomeNamedNumericStrings()
{
  var result = new NameValueCollection();
  result["MaxSendAttempts"] = RandInt();
  result["BackoffIntervalInMinutes"] = RandInt();
  result["BackoffMultiplier"] = RandInt();
  result["PurgeMessageWithAgeInDays"] = RandInt();
  return result;

  string RandInt () => GetRandomInt().ToString();
}

Which, I think, is neater. There's also possibly a performance benefit: every time the first implementation of GetSomeNamedNumericStrings is called, the randInt Func is instantiated, where the second implementation has the function compiled at compile-time, baked into the resultant assembly. Whilst I wouldn't put it past the compiler or JIT to do clever optimisations for the first implementation, I wouldn't expect it.

Throw expressions also offer a neatening to your code:

public int ThrowIfNullResult(Func source)
{
  var result = source();
  if (result == null) 
    throw new InvalidOperationException("Null result is not allowed");
  return result;
}
can become:
public int ThrowIfNullResult(Func source)
{
  return source() ?? 
    throw new InvalidOperationException("Null result is not allowd");
}

So now you can actually write a program where every method has one return statement and nothing else.



Time for the big one: Pattern Matching. This is a language feature native to F# and anyone who has done some C#-F# crossover has whined about how it's missing from C#, including me. Pattern matching not only elevates the rather bland C# switch statement from having constant-only cases to non-constant ones:

public static string InterpretInput(string input)
{
  switch (input)
  {
    case string now when now == DateTime.Now.ToString("yyyy/mm/dd"):
      return "Today's date";
    default:
      return "Unknown input";
  }
}

It allows type matching:

public static void Main()
{
  var animals = new List()
  {
    new Dog() {Name = "Rover"},
    new Cat() {Name = "Grumplestiltskin"},
    new Lizard(),
    new MoleRat()
  };
  foreach (var animal in animals)
    PrintAnimalName(animal);
}

public static void PrintAnimalName(Animal animal)
{
  switch (animal)
  {
    case Dog dog:
      Console.WriteLine($"{dog.Name} is a {dog.GetType().Name}");
      break;
    case Cat cat:
      Console.WriteLine($"{cat.Name} is a {cat.GetType().Name}");
      break;
    case Lizard _:
      Console.WriteLine("Lizards have no name");
      break;
    default:
      Console.WriteLine($"{animal.GetType().Name} is mystery meat");
      break;
  }
}

Other new features include generalized async return typesnumeric literal syntax improvements and more expression bodied members.

Wrapping it up

I guess the point of this illustrated journey is that you really should keep up to date with language features as they emerge:
  • Many features save you time, replacing tedious code with shorter, yet more expressive code
  • Some features provide a level of safety (eg string interpolations)
  • Some features give you more power
So if you're writing C# like it's still .net 2.0, please take a moment to evolve as the language already has.

NExpect level 3: you are the key component

In previous posts, I've examined how to do simple and collection-based assertions with NExpect . These have enabled two of the desig...