What's new in PeanutButter?

Retrieving the post... Please hold.

If the post doesn't load properly, you can check it out here: 
https://github.com/fluffynuts/blog/blob/master/20190412.md

PeanutButter.RandomValueGen: the builder pattern & random generation for testing purposes

Retrieving the post... Please hold.

If the post doesn't load properly, you can check it out here: 
https://github.com/fluffynuts/blog/blob/master/20180825.md

#PeanutButter.Utils: the dictionaries

Dictionaries, HashMaps, whatever you want to call them -- they can be one of the most useful constructs in any language. In Javascript, the dictionary interface to objects makes a lot of dynamic code simpler. In fact, it was the Javascript paradigm which served as the inspiration for `PeanuButter.Utils` member: `DictionaryWrappingObject`. This class was originally made to facilitate some of the functionality in `PeanutButter.Ducktyping`: a library for duck-typing arbitrary objects onto interfaces which aren't directly related. For example, if you had an anonymous object with the correct "shape", you could duck-type it onto an interface which another part of the system requires without having to manually create your own implementation of that interface and the code to copy data / forward method calls -- all of which `PeanutButter.DuckTyping` can do, with varying amounts of flexibility to matching methods and properties, according to your needs.

But I'm not here to talk about `PeanutButter.DuckTyping` today -- as interesting as it was to write and as useful as it's proven to be in a couple of projects since then.

So, back to `DictionaryWrappingObject`: this class provides the familiar `IDictionary<string, object>` interface over any other object so you can enumerate through the properties or perform functions like querying property names without directly using reflection yourself. And, of course, you can just use very Javascript-y syntax:
```csharp
var obj = new { id = 1, name = "bob" };
var wrapper = new DictionaryWrappingObject(obj);
var name = wrapper["name"];
```
You can also construct with a case-insensitive StringComparer to make those lookups a little fuzzier:
```
var obj = new { Id = 1, Name = "Mary" };
var wrapper = new DictionaryWrappingObject(obj);
var id = wrapper["ID"];
var name = wrapper["name"];
```

Another useful dictionary construct that I stumbled across in Python is the default dictionary -- implemented in `PeanutButter.Utils`, unsurprisingly, as `DefaultDictionary`. This dictionary allows you to specify a default value to return for the case where requested keys aren't found:
```
var animalsInZoo = new DefaultDictionary<string, bool>(false);
animalsInZoo["Camel"] = true;
animalsInZoo["Panda"] = true;

//... some time later:
var haveCamels = animalsInZoo["Camel"]; // true
var haveSnakes = animalsInZoo["Snake"]; // false, not KeyNotFoundException!
```

`DefaultDictionary` can be a little smarter than having a static value for the default:
```
// set up the default dictionary such that students with a name starting
// "A" exist.
var students = new DefaultDictionary<string, bool>(
  k => k.StartsWith("A")
);
students["Anna"] = false;
students["Mary"] = true;
// ... elsewhere ...

var haveAnna = students["Anna"]; // false, explicitly set
var haveAndrew = studentds["Andrew"]; // true: default value provider
var haveMary = students["Mary"]; // true: explicitly set
var haveStewart = students["Stewart"]; // false: default value provider
```

`DefaultDictionary` is expecially useful in conjunction with `MergeDictionary`, which takes one or more other dictionaries with the same key/value types and merges them, returning the value from the first in the merge list to have a value. So we could have:
```
var config = new Dictionary<string, string>()
{
  ["host"] = "database-machine",
  ["port"] = "123"
};
var defaults = new Dictionary<string, string>(k =>
{
  switch (k):
  {
    case "host":
      return "localhost";
    case "port":
      return "3306";
    case "user":
      return "mysql";
    case "password":
      return "super-secret";
    default:
      return "";
  }
});
var final = new MergeDictionary<string, string>(
  config, defaults
);

// ... elsewhere ...
var config = new 
{
  host: final["host"],        // database-machine
  port: int.Parse(final["port"]), // 123
  user: final["user"],        // mysql (default)
  password: final["password"] // super-secret (default)
};

```

Finally, there is the `CaseWarpingDictionary`, which basically acts as a wrapper for another dictionary to change the case-sensitivity of the keys, especially useful when you want a dictionary that is a little more forgiving (case-insensitive) than the one you're working with. `CaseWarpingDictionary` can be constructed with either a boolean instructing whether or not the result is case-sensitive, or with a StringComparer, so you can, for instance, switch from `Ordinal` to `CurrentCultureIgnoreCase`.

## Last words
The `IDictionary<TKey, TValue>` interface is not particularly difficult to implement, but it is quite convenient to consume. And I recommend devs wanting to learn something about the internals of .NET to implement some kind of `IDictionary<TKey, TValue>` at some point in their lives. For one thing, it will give you an appreciation for the good old `Dictionary` (:

# Unit test coverage and why it matters

Good unit test coverage can help you to improve your code in ways you might not expect. I'm not talking about just chasing some mythical value, like the agile team which ascribes to the contract of "85% coverage over the project", though chasing a number (the theoretical 100% coverage, which I've only achieved in one project ever) can lead you to some interesting discoveries.

Obviously, we can have bogus coverage:

```csharp
using NUnit.Framework;
using static NExpect.Expectations;
using NExpect;

[TestFixture]
public class TestTheThing
{
  [Test]
  public void ShouldDoTheStuff()
  {
    // Arrange
    var sut = new TheThing();
    // Act
    var result = sut.DidTheStuff();
    // Assert
    Expect(result).To.Be.True();
  }
}

public class TheThing
{
  public bool DidTheStuff()
  {
    try
    {
      WriteOutSomeFile();
      MakeSomeWebCall();
      return true;
    }
    catch (WebException ex)
    {
      return false;
    }
  }
  // ... let's imagine that WriteOutSomeFile 
  // and MakeSomeWebCall are defined below...
}
```

The above test doesn't actually check that the correct file was written out -- or even that the correct web request happened. The one test above technically provides full coverage of the class (if WriteOutSomeFile and MakeSomeWebCall have no branches), but it's a bit anemic in that the coverage doesn't tell us much.

So coverage is not a number which definitively tells you that your code (or tests) are good. However, examining coverage reports (particularly the line-by-line analysis) has helped me to discover at least three classes of error:

## I found bugs I didn't know I had

When I was still at [Chillisoft](http://www.chillisoft.co.za), I'd just finished a unit of code (TDD, of course) to my satisfaction and decided to run coverage on it for interest sake. I was convinced that I'd done a good job, writing one test before each line of production code which was required. To my dismay, I found that there was one line which _wasn't_ covered. _Shame on me_, I thought, and went back to the test fixture, _where I found a test describing the exact situation that line should be handling_. Ok, so I have this test, it names the situation, but there's no coverage on the line?

_Remember: coverage reports _can_ be faulty. It's rare, but it's worthwhile re-running your reports to just make double-sure that what you're seeing is correct._

I re-ran my coverage, but that one line remained red. And the more I looked at it, the more it looked like it should actually be causing a test to fail. So I re-examine the test which is supposed to be covering it to find... I've made a mistake in that test and it's actually not running through the branch with the uncovered line. 

The fault here most likely comes down to one faulty TDD cycle where I hadn't gotten a good "red" before my "green". Still, examining the coverage report made me find the error and fix it before the code got anywhere near production. This experience is why I advocate for running coverage after reaching a point where one expects the current unit of work to be complete -- to find any holes in testing or defects in logic hidden in those holes. It's why I (convincingly) argued for all Chillisoft programmers to be granted an Ultimate license of Resharper, which has dotCover built right in to the test runner. We have coverage reports running at the CI server, but I wanted every developer to have the ability to test coverage quickly so that they could also discover flaws in their code before that code gets to production -- or even another developer's machine!

## I found dead code

Just recently, I finally got the `gulp` build tasks for `NExpect` to include coverage by default when running `npm test`. And to my dismay, `NExpect` only had about 78% coverage. Which I thought was odd, because `NExpect` was build very-much test-first: indeed, the general method of operation was to write out a test with the desired expression and then provide the code to make that expression happen. So, for example:

```csharp
Expect(someCollection).To.Contain
  .Exactly(1).Deep.Equal.To(search);
```

would have started out with most of those words red (thanks to Resharper) and they would unhighlight as I got together the class/interface linkage to make the words flow as desired. I expected coverage to be closer to 90% (I did expect some places to have been missed in lieu of ever having scrutinised coverage reports for `NExpect` before), but 78%? I had some work to do.

In addition to finding a few minor bugs that I didn't know I had (particularly with wording of failure messages in a few cases), I found that I had bits of code which theoretically should have been under test, but which weren't covered. Especially stuff like:

```csharp
[Test]
public void ComparingFloats()
{
  Expect(1.5f).To.Be.Greater.Than(1.4f)
    .And.Less.Than(1.6f);
}
```

which works as expected, but never hit the `Than` extension methods for continuations of float.

The answer became obvious upon hovering over the usages -- each `Than` was expecting to operate on values of type `double` as the _subject (actual) value_ (ie, the value being tested, in this case, `1.5f`). This is because `Expect` upcasts certain numeric types (floats to doubles, ints to longs, etc) so that the comparison code doesn't require casting from the consumer (since `NExpect` continuations hold the type of the subject all the way through, instead of downcasting to `object` and hoping for the user to provide reasonable values. There's nothing wrong (that I can tell) with this approach -- and it works well, but it _does_ mean that the `Than` extension methods expecting to operate on `float` and `int` subjects will never be used. They were dead code! So I could safely remove them. One of the ways to make code better is to remove the unnecessary bits (:

## I found holes in my api

This is again, working in `NExpect`, where, upon providing coverage for one variant of syntax, I would find that I hadn't implemented for another. For example, `NExpect` has no opinion on which of these is better:

```csharp
Expect(1).To.Not.Equal(2);
Expect(1).Not.To.Equal(2);
```

All `NExpect` does is prevent silliness like:

```csharp
Expect(1).Not.To.Not.Equal(1);
```

`NExpect` is designed around user-extensibility as one of the primary goals. As such, there are some "dangling" words, like `A`, `An`, and `Have` so
that the user can provide her own expressive extensions:

```csharp
var dog = animalFactory.CreateDog();
Expect(dog).Not.To.Be.A.Cat();
Expect(dog).To.Be.A.Dog();
Expect(dog).To.Be.A.Mammal();
```

Where the user can use Matchers or Composition to provide the logic for the `Cat`, `Dog`, and `Mammal` extension method assertions. `NExpect` doesn't actually provide extensions on these "danglers" -- they're literally just there for custom extension.

Whilst running coverage, I found that one variant of `Contain` wasn't covered, and when I wrote a test to go through all three (positive, negative, alt. negative), I found that there were missing implementations! Which I naturally implemented (:

## Using coverage to make your code better

Coverage reports like those generated by `dotCover` and the combination of `OpenCover` and `ReportGenerator` can not only give you confidence in your code and a fuzzy feeling inside at a number which shows that you do care about automated testing for your code -- they can also help you to make your code (and tests) better. And make _you_ better, going forward, because you learn more about the mistakes you make along the way.

If you want to get started relatively easily and you're in the .net world, you can use [gulp-tasks](https://github.com/fluffynuts/gulp-tasks) as a submodule in your git repo. Follow the instructions in the `start` folder and get to a point where you can run `npm run gulp cover-dotnet` (or make this your `test` script in `package.config`). This should:
- build your project
- run your tests, using `OpenCover` and `NUnit`
- generate html reports using `ReportGenerator`, under a `buildreports` folder

You can always check out [NExpect](https://github.com/fluffynuts/NExpect) to see how I get it done there (:

# What's in PeanutButter.Utils, part 2

## Metadata extensions

I just wanted to chip away at my promise to explain more of the bits in PB, so I thought I'd pick a little one (though I've found it to be quite useful): metadata extensions.

At some point, I wanted to be able to attach some arbitrary information to an object which I didn't want to extend or wrap and which some code, far down the line, would want to read. If C# was Javascript, I would have just tacked on a property:
```js
someObject.__whatDidTheCowSay = "moo";
```

But C# is _not_ Javascript. I could have maintained some global `IDictionary` somewhere, but, even though I wanted it to support a feature in [NExpect](https://github.com/fluffynuts/NExpect), where the code wouldn't have a running lifetime of any significance, it still felt like a bad idea to keep hard references to things within NExpect. The code associating the metadata has no idea of when that metadata won't be necessary any more -- and neither does the consumer.

Then I came across [`ConditionalWeakTable`](https://msdn.microsoft.com/en-us/library/dd287757(v=vs.110).aspx) which looked very interesting: it's a way of storing data where the keys are weak references to the original objects, meaning that if the original objects are ready to GC, they can be collected and the weak reference just dies. In other words, I found a way to store arbitrary data referencing some parent object and the arbitrary data would only be held in memory until the end of the lifetime of the original object.

That's exactly what I needed.

So was born the [`MetadataExtensions`](https://github.com/fluffynuts/PeanutButter/blob/master/source/Utils/PeanutButter.Utils/MetadataExtensions.cs) class, which provides the following extension methods on all objects:

- `SetMetadata`<`T`>`(string key, object value)`
- `GetMetadata`<`T`>`(string key)`
- `HasMetadata`<`T`>`(string key)`

which we can use as follows:

```csharp
public void MethodWantingToStoreMetadata(
  ISomeType objectWeWantToStoreStateAgainst)
{
  objectWeWantToStorStateAgainst
    .SetMetadata("__whatDidTheCowSay", "moo");
}

// erstwhile, elsewhere:

public void DoSomethingInterestionIfNecessary(
  ISomeType objectWhichMightHaveMetadata)
{
  if (objectWhichMightHaveSomeMetadata
        .HasMetadata("_whatDidTheCowSay"))
  {
    var theCowSaid = objectWhichMightHaveSomeMetadata
                       .GetMetadata("_whatDidTheCowSay");
    if (theCowSaid == "moo")
    {
      Console.WriteLine("The cow is insightful.");
    } 
    else if (theCowSaid == "woof")
    {
      Console.WriteLine("That ain't no cow, son.");
    }
  }
}
```

And, of course, as soon as the associated object can be collected by the garbage collector (remembering that the reference to this object, maintained within PB, is _weak_), that object is collected and the associated metadata (if not referenced elsewhere, of course) is also freed up. This mechanism has facilitated some interesting behavior in `NExpect`, and I hope that it can be helpful to others too.

# Markdown all things

Whilst blogger.com provides a fairly good blogging platform for regular writing, I've found that it's rather painful for technical blogging. In particular, code blocks are a mission. In the past, I've wrapped code in <pre><code> ... </code></pre> and let [highlight.js](https://github.com/isagalaev/highlight.js/) do all the heavy lifting of making that actually look readable. HighlightJs has been fantastic at that, but it still hasn't been as smooth a process as I would have liked _overall_. I still tended to write the non-code parts in the WYSIWIG html editor, and had to switch the source view to work on code parts. 

When I blog, I literally want to get out the information as quickly as possible, in a readable format. I'm not here to fight with styling.

So I was quite happy to stumble across [showdown](https://github.com/showdownjs/showdown). A little Javascript in my template and suddenly I could write in possibly the simplest format ever: markdown. I had quick and easy access to simple styling elements (lists, headings, etc) as well as code blocks. All good, but not automagick out of the box. 

I thought to myself, _"I'm sure I can't be the only person who wants this"_, and _"It would be nice if that auto-bootstrapping of markdown+code could be done anywhere, not just from within my blogger template"_.

So, as is so common within the open-source world, I stand upon the very tall, very broad shoulders of [highlight.js](https://github.com/isagalaev/highlight.js/) and [showdown](https://github.com/showdownjs/showdown) to present [auto-markdown](https://github.com/fluffynuts/auto-markdown): a script you can include on any page to convert any element with the `markdown` class to be rendered as markdown. It can even be configured (script versions and code theme) via some global variables, so you don't have to fiddle with the code if you don't want to.

I trialed it with my last post and It's how I'm writing now -- I just add a shell `pre` tag with the `markdown` class and get on with the writing, without any more fighting with the html editor. As a bonus: even if the script fails for some reason (such as if the user has Javascript disabled or GitHub doesn't supply my script in time), the blog is still in a readable format: markdown.

If you're interested, follow the instructions in the [README.md](https://github.com/fluffynuts/auto-markdown). Feel free to open issues if you encounter some - for instance, I encountered some stickiness with generics in code blocks. Also, bear in mind that markdown requires html-escaping for chevrons (ie, embedding xml). 

Feel free to share it as much as you like. If you don't feel comfortable referencing my code directly, fork my repo and keep your own copy (:

Now, if only blogger's html editor had a vi mode...

# What's in `PeanutButter.Utils`, exactly?

`PeanutButter.Utils` is a package which pretty-much evolved as I had common problems that I was solving day-to-day. People joining a team that I was working on would be exposed to bits of it and, like a virus, those bits would propagate across other code-bases. Some people asked for documentation, which I answered with a middle-ground of `xmldoc`, which most agreed was good enough. People around me got to know of the more useful bits in `PeanutButter.Utils` or would ask me questions like "Does PeanutButter.Utils have something which can do [X]?". I kind of took the ubiquity amongst my team-mates for granted.

Fast-forward a little bit, and I've moved on to another company, where people don't know anything about the time-savers in `PeanutButter.Utils` -- and it occurs to me that that statement probably applies to pretty-much most people -- so I thought it might be worthwhile to have some kind of primer on what you can expect to find in there. An introduction, if you will. I think there's enough to break the content down into sections, so we can start with:

## Disposables
One of the patterns I like most in the .net world is that of `IDisposable`. It's a neat way to ensure that something happens at the end of a block of code irrespective of what goes on _inside_ that code. The code could throw or return early -- it doesn't matter: whatever happens in the `Dispose` method of the `IDisposable` declared at the top of a `using` block will be run. Usually, we use this for clearing up managed resources (eg on database connections), but it struck me that there were some other convenient places to use it. Most generically, if you wanted to run something simple at the start of a block of code and run something else at the end (think of toggling something on for the duration of a block of code), you could use the convienient `AutoResetter` class:
```csharp
using (new AutoResetter(
    () => ToggleFeatureOn(), 
    () => ToggleFeatureOff()))
{
    // code inside here has the feature toggled on
}
// code over here doesn't -- and the feature is 
//    toggled back off again even if the code 
//    above throws an exception.
```
It's very simple -- but it means that you can get the functionality of an `IDisposable` by writing two little lambda methods.

You can also have a variant where the result from the first lambda is fed into the second:
```csharp
using (new AutoResetter(
    () => GetCounterAndResetToZero(),
    originalCount => ResetCounterTo(originalCount)))
{
// counter is zero here
}
// counter is reset to original value here
```

Cool.

Other common problems that can be solved with `IDisposable` are:
### Ensuring mutexes / semaphores are reset, even if an exception is encountered
For this, we can use `AutoLocker`:
```csharp
using (new AutoLocker(someMutex))
{
}
using (new AutoLocker(someSemaphore))
{
}
using (new AutoLocker(someSemaphoreLite))
{
}
```

### Temporary files in tests
```csharp
using (var tempFile = new AutoTempFile())
{
   File.WriteAllBytes(
       Encoding.UTF8.GetBytes("moo, said the cow"),
       tempFile.Path
   );
   // we can run testing code against the file here
}
// file is gone here, like magick!
```
This uses the `Path.GetTempFileName()` system call by default -- so you don't have to care about where the file actually exists. Of course, there are constructor overloads to:
- create the file populated with data (string or bytes)
- create the file in a different location (not the system temp data location)
- create the file with a specific name

`AutoTempFile` also exposes the files contents via properties:
- `StringData` for string contents
- `BinaryData` for a byte[] array

There is also an `AutoTempFolder` if you want a scratch area to work in for a period of time. When it is disposed, it and all it's contents are deleted.

Similarly, `AutoDeleter` is an `IDisposable` which can keep track of multiple files you'd like to delete when it is disposed:
```csharp
using (var deleter = new AutoDeleter())
{
    // some files are created, then we can do:
    deleter.Add("C:\Some\File");
    deleter.Add("C:\Some\Other\File");
}
// and here, those files are deleted. If they can't be 
//    deleted, (eg they are locked by some process),
//    then the error is quietly suppressed. 
//    `AutoDeleter` works for folders too.
```

### Other disposables
As much as I love the `using` pattern, it can lead to some "arrow code", like this venerable ADO.NET code:
```csharp
using (var conn = CreateDbConnection())
{
    conn.Open();
    using (var cmd = conn.CreateCommand())
    {
        cmd.CommandText = "select * from users";
        using (var reader = cmd.ExecuteReader())
        {
            // read from the reader here
        }
    }
}
```
Sure, many people don't use ADO.NET "raw" like this any more -- it's just an easy example which comes to mind. I've seen far worse "nest-denting" of `using` blocks too.

This can be flattened out a bit with `AutoDisposer`:
```csharp
using (var disposer = new AutoDisposer())
{
    var conn = disposer.Add(CreateDbConnection());
    conn.Open();
    var cmd = disposer.Add(conn.CreateCommand());
    cmd.CommandText = "select * from users";
    var reader = disposer.Add(conn.ExecuteReader());
    // read from the db here
}
// reader is disposed
// cmd is disposed
// conn is disposed
```
`AutoDisposer` disposes of items in reverse-order in case of any disposing dependencies.


So that's part 1 of "What's in `PeanutButter.Utils`?". There are other interesting bits, like:
- extension methods to
    - make some operations more convenient
         - do conversions
         - working with `Stream` objects
         - DateTime utilities
    - facilitate more functional code (eg `.ForEach` for collections)
    - `SelectAsync` and `WhereAsync` let you use async lambdas in your LINQ
    - test and manipulate strings
- the `DeepEqualityTester`, which is at the heart of `NExpect`s `.Deep` and `.Intersection` equality testing
- `MemberExpression` helpers
- Reflection tidbits
- reading and writing arbitrary metadata for any object you encounter (think like adding property data to any object)
- Some pythonic methods (`Range` and a (imo) more useful `Zip` than the one bundled in LINQ)
- dictionaries
    - `DictionaryWrappingObject` lets you treat any object like you would in Javascript, with text property indexes
    - `DefaultDictionary` returns default values for unknown keys
    - `MergeDictionary` allows layering multiple dictionaries into one "view"
    - `CaseWarpingDictionary` provides a decorator dictionary for when the dictionary you have does indexing with inconvenient case rules

I hope to tackle these in individual posts (:

Toggle All Things!

(Note: this post was started quite a long time ago and has lived in draft mode whilst I intended to provide a concrete code example. I'm not sure when, if ever, that will come, but I think that the lessons learned are still valuable to share)

It's not an uncommon workflow: the client / company wants some features in a certain sprint and the features have to be signed off by a QA / testing team. So if we take the example of a two week sprint, what often happens is that the testers sit idle (or are tasked onto a different project) for the first week, while development happens at a furious pace, then testers are given domain to test the week's work and we enter a development freeze, where developers are unable to work ahead for fear of interrupting testing and must be on standby for any bugs picked up by QA.

Hopefully, if you're testing your own work proficiently, QA picks up nothing, or perhaps some small stuff like a misaligned label or similar. However, for that time period, you're stagnant -- which causes you to work harder when the first week of a sprint comes around again. It's not a healthy place to be in, if, for no other reason, than because it provides ammunition for the "production line" mindset and guilting you into working longer hours during the "productive" part of the sprint. And if you were on top of things, the business is also losing during the development freeze because you aren't free to forge ahead with new stories.

No-one is winning.

One possible strategy is to adopt a "feature branch per developer" strategy with a regular "merge day" where some poor soul attempts to merge the disparate work of the last two weeks. You can minimise the pain with more frequent merges during the active development phase, but whilst you're in development freeze, you're going to drift and merge hell is eventually inevitable.

One of the most useful strategies we've used to date is that of feature toggles, which, simply put, are mechanisms for turning parts of your software on or off. So here are some thoughts on that particular topic, played out over the development cycle of a SPA web application.

1. A simplistic approach: compile-time toggles (ie: releases with feature-sets)

One method is good old #if / #ifdef and compile-time constants. Bake those features in and release a version with a specified feature matrix. This works well for software where the required featureset is unlikely to change, such as for Gentoo packages, where your USE flags determine the features available to you at run-time until such time as you decide that you need more or fewer features. As the owner and maintainer of said Gentoo system, you decide what you want from your software and you compile it that way. When your requirements change, you update your flags and re-compile.

This works well when the user / tester is the developer. But not as well when the user/tester is someone else -- because then they have to contact the developer and request a rebuild and redeployment every time a feature needs to be turned on or off.

It's not an invalid strategy -- it's just not very agile. Additionally, you can't easily a automate testing (unit / integration testing) against features, because doing so requires changing compile parameters.

2. Better: static values in the code

Another version of this is compiling with static / constant values and branching in code. This allows testing against the different features (especially if you're using static values instead of constants). This alleviates the automated testing conundrum but still leaves the human testing department out in the cold. They will still contact you for a new build containing the required feature matrix.

Rebuilding and redeploying to satisfy a required feature matrix becomes impractical the more features you enable and the more agile you'd like to be. Suppose you think feature [X] is ready, but 1/2 an hour before sign-off, testing finds a defect. Now you have to rush to provide a deployment package without [X]. In addition, testing may not yet have had the time to prove that there are no unintended side-effects from not including feature [X].

3. Even better: run-time toggles determined by app / web configuration

By now you've realised that the toggles should be configurable and they have been pushed them into your web config (for example). Great! Toggles can be changed fairly quickly, without requiring a re-deployment. The problem is that you still need someone fairly technical (and who has access to the staging machines) to update the enabled feature-set.

My experience has been that if you can make the toggling of features relatively quick and painless, you can offload the decision on which features are in an accepted release to someone in testing or management. You don't want to waste your time and resources on fighting for feature inclusion / exclusion -- you want to focus on making new things, solving new problems! So whilst feature toggles in your web config are better than compile-time, you could push this just a little further for a lot of win.

4. Best: run-time toggles determined per user interaction

In the case of a web system, this would mean "per web request". Imagine if the testers had the ability to test just the stuff they were confident in, and, if they had time, could toggle on a beta feature and test if they have capacity. Imagine if, when a flaw is found in the implementation or design of a feature, it didn't have to hold up the entire release -- just toggle it off, continue testing other features and sign off what is available.

Our implementation was system-wide, re-evaluated per web request, but you could even push this further with per-user configuration included in request headers. Practically, this may be more effort than it's worth.

Features of good feature toggles

• It should be easy to toggle features and difficult to end up with a set of conflicting behaviors from the application. 
• It should be easy to determine the feature-set state of the application and difficult to be confused about interactions which cause experienced defects. I'm suggesting that if you have old feature [X] and new feature [Y], which replaces [X], there shouldn't be two toggles ("enable [X]" and "enable [Y]") -- whilst this technically solves the problem, you're expecting testers to understand the repercussions of the ability of enabling both. Instead, you should strive towards well-defined feature toggles which explain what they do and provide only two well-known states (the "off" and "on" states) which cannot be confused by interaction with other parts of the system.
• It should be as frictionless as possible to develop against (adding features and consuming them) to encourage using the framework so that the agility of feature toggles is experienced as often as possible. More toggles are better than fewer toggles, especially for non-inter-dependent features.
• Toggles should be easy to remove as well: once QA is satisfied with a feature, the toggle should be removed along with any branches of logic which disabled the feature.

This can all be achieved, and it's really not that tricky. Here are the steps we went through:

A. First pass: development freeze and the rush around sign-off time sucks. We need to be able to add new features which can be easily disabled!

So, first off, we define a class / interface which has boolean toggles on it. And we get the web app to put persist into our storage (sql, document db, whatever) on startup, adding toggles which are missing. We save three versions of this configuration, for three proposed feature-sets: "development", "staging", "production" and we let annotations on the configuration class determine default toggle states for features.

For example, we create a feature toggle with the feature defaulted on for "development" and off for "staging" and "production". In this way, developers on the team default to seeing the current work (and interacting with it) and the feature isn't inadvertently introduced to QA or the production servers. Of course, the moment QA is ready, they can manually enable the feature. Production only gets the feature enabled when a deployment to production is explicitly made with a script to enable the feature.

We allow selection of a feature set ("development", "staging", "production") in the web config, because this is unlikely to change on a deployment target, so probably won't need human interaction (let deployment scripts take care of this) and surface these toggles to the user via a simple UI. The UI itself is toggled with a feature toggle, so the production site doesn't have it at all -- no way for a user to enable a feature they shouldn't or disable a feature which should be there.

This process also makes adding / removing feature toggles trivial for other developers on the system.

Consuming toggles, server-side (back-end developer):
We could make the decision about defined feature classes to inject by manipulating the IoC container -- inject implementation "A" of interface "I" or, if the toggle is enabled, inject implementation "B". This may work with differing degrees of efficiency according to your IoC container and you can end up with two implementations which are incredibly similar, but which differ only in one minor aspect.

Alternatively, components of the system could, perhaps, request feature toggle configuration via IoC injection branch subtly as needed. We make sure that anything which needs feature toggles has a per-web-request lifetime (as does the feature toggles injectible) so that the web app doesn't have to be restarted to bring a toggle into effect

Consuming toggles, client-side (front-end developer):

The current feature toggles are exposed as a calculated stylesheet where disabled features literally have their display set to "none !important". So when the page loads, incomplete features are not available for interaction. On feature toggle, this stylesheet can be reloaded to get instant UI feedback of toggled features without even a page reload.

Toggling toggles (user):

Initially, toggling a feature simply toggles the boolean in storage and re-requests the featureset stylesheet, immediately updating the UI. Calls to the api get a new instance of the feature toggles entity and can change their behavior accordingly.

Typically, a "big hammer" approach works well: a controller action returns null or an empty collection when the toggle is off or calls into defined logic to produce a calculated result when the feature is enabled. Sometimes, a finer chisel is called into play -- some component further down the logic chain subtly changes behavior based on the toggle.

B. New features may replace older ones -- we need a toggle to turn one bit on and simultaneously turn another off

We already have this capability at the server -- we can branch code according to an injected feature toggles matrix configuration object -- but at the client, we are just hiding UI elements which expose disabled features.
Thus we added in more client-side logic: we expose a Javascript blob in a generated js file with the feature toggle matrix and emit an event when this is re-downloaded.
To ease developer consumption, a framework function is provided to tuck away the complexity of listening for the correct event.
Also, whilst we've already had calculated stylesheets to toggle incomplete features off, we can now add in calculated stylesheets to toggle fallback features (ie, legacy feature) back on again.

Winning at features

The net result of employing this tactic has meant that we've managed to pull out far in front of the testing team which used to drive deadlines. Simply put, we're able to work on sprint [N+1] whilst testing is signing off sprint [N], such that we've essentially ended up a sprint ahead and had the breathing-room to address technical debt -- a win for everyone!

Feature toggles provided:

• A way to alleviate stress for testers and developers: if a feature isn't deemed finished by sign-off time, it's simply toggled out for deployment
• A way to alleviate stress for developers: whilst features for this sprint are under human testing, developers don't have to sit idle -- they can move forward with the next set of requirements.
• A way to alleviate stress for project managers: it's far easier to say "we managed to get 4/5 of the required features deployed" than "we couldn't deploy because one feature couldn't be signed off"

Interactive, per-request feature toggles meant that:

• Testers could test at their own pace, enabling features on their environment when they were ready and disabling if they thought that those features were interacting negatively with others or if they considered them incomplete at sign-off time for the sprint.
• Testers can preview a feature during development to give early feedback to the developers if they have capacity
• "What-if" conversations could be had: "what if feature [X] was enabled for the users without [Y], which appears to be a little dodgy right now?"
• There was no rush at deployment time for getting "just the right build" to deploy: simply deploy the latest and toggle off the features which aren't considered fully-baked.

NExpect hits 100 releases!

It's probably not such a big deal, but it feels like it to me: NExpect (GitHub, Nuget) has hit its 100th release! Whilst the reason for the 100th release was trivial (adding support to assert the existence of keys by any case in a case-insensitive dictionary), it feels like some kind of milestone.

Of course, it wouldn't be here without the fantastic contributions from Cobus Smit as well as the input from everyone who uses it (even the developers on my team who had it foisted upon them!).

NExpect isn't done with its evolution though. I have a trello board that I probably should convert to GitHub issues. And I welcome bug reports and requests which align with the ethos of the project:

• Expect(NExpect).To.Be.Readable();
  • Because code is for co-workers, not compilers. And your tests are part of your documentation.
• Expect(NExpect).To.Be.Expressive();
  • Because the intent of a test should be easy to understand. The reader can delve into the details when she cares to.
• Expect(NExpect).To.Be.Extensible();
  • Because I can't predict every use-case. I believe that your assertions framework should enable expressive, readable tests through extension.

Personally, I'm finding NExpect to be a pleasure to use. But perhaps I'm a little biased (:

What's new in PeanutButter

I realise that it's been a while (again) since I've posted an update about new things in PeanutButter (GitHub, Nuget). I've been slack!

Here are the more interesting changes:

1. PeanutButter.Utils now has a netstandard2.0 target in the package. So you can use those tasty, tasty utils from netcore2.0
2. This facilitated adding a netstandard2.0 target for PeanutButter.RandomGenerators -- so now you can use the GenericBuilder and RandomValueGen, even on complex types. Yay! Spend more time writing interesting code and less time thinking of a name on your Person object (:
3. Many fixes to DeepClone(), the extension method for objects, which gives you back a deep-cloned version of the original, including support for collections and cloning the object's actual type, not the type inferred by the generic usage - so a downcast object is properly cloned now.
4. DeepEqualityTester can be used to test shapes of objects, without caring about actual values -- simply set OnlyCompareShape to true. Combined with setting FailOnMissingProperties to false, a deep equality test between an instance of your model and an instance of an api model can let you know if your models have at least the required fields to satisfy the target api.
5. Random object generation always could use NSubstitute to generate instances of interfaces, but now also supports using PeanutButter.DuckTyping's Create.InstanceOf<T>() to create your random instances. Both strategies are determined at runtime from available assemblies -- there are no hard dependencies on NSubstitute or PeanutButter.DuckTyping.
6. More async IEnumerable extensions (which allow you to await on a Linq query with async lambdas -- but bear in mind that these won't be translated to, for example, Linq-to-Sql code to run at the server-side):
1. SelectAsync
2. ToArrayAsync
3. WhereAsync
4. AggregateAsync

There was also some cleanup (I defaulted R# to show warnings, not just errors -- and squashed around 2000 of them) and made all projects depend on the newer PackageReferences mechanism for csproj files.

So, yeah, stuff happened (:

Everything sucks. And that's OK.

There is no perfect code, no perfect language, no perfect framework or methodology. Everything is, in some way, flawed.

This realization can be liberating -- when you accept that everything is flawed in some way, it puts aside petty arguments about the best language, editor, IDE, framework, database -- whatever some zealot is pedaling as being the ultimate solution to your problems. It also illuminates the need for all of the different options out there -- new programming languages are born sometimes on a lark, but often because the creator wanted to express themselves and their logical intent better than they could with the languages that they knew. Or perhaps they wanted to remove some of the complexity associated with common tasks (such as memory allocation and freeing or asynchronous logic). Whatever the reason, there is (usually) a valid one.

The same can be said for frameworks, database, libraries -- you name it. Yes, even the precious pearls that you've hand-crafted into existence. 

In fact, especially those.

We can be blind to the imperfections in our own creations, sometimes it's just ego in the way. Sometimes it's just a blind spot. Sometimes it's because we tie our own self-value to the things we create.

"You are not your job, you're not how much money you have in the bank. You are not the car you drive. You're not the contents of your wallet. You are not your fucking khakis."

For the few that didn't recognize the above, it was uttered by Tyler Durden from the cult classic Fight Club. There are many highlights one could pick from that movie, but this one has been on my mind recently. It refers to how people define themselves by their possessions, finding their identity in material items which are, ultimately, transient.

Much like how we're transient, ultimately unimportant in the grand scale of space and time that surrounds us. Like our possessions, we're just star stuff, on loan from the universe whilst we experience a tiny fraction of it.

I'd like to add another item to the list above:

You are not the code you have written.

This may be difficult to digest. It may stick in your throat, but some freedom comes in accepting this.

As a developer, you have to be continually learning, continually improving, just to remain marginally relevant in the vast expanse of languages, technologies, frameworks, companies and problems in the virtual sea in which our tiny domains float. If you're not learning, you're left behind the curve. Stagnation is a slow death which can be escaped by shaking up your entire world (eg forcing learning by hopping to a new company) or embraced by simply accepting it as your fate as you move into management, doomed to observe others actually living out their passions as they create new things and you report on it. From creator to observer. You may be able to balance this for a while, you may even have the satisfaction of moving the pieces on the board, but you've given up something of yourself, some part of your creator spirit. You can still learn here -- but you can also survive just fine as you are.

Or at least that's how it looks from the outside. And that's pretty-much how I've heard it described from the inside. I wouldn't know, personally. I'm too afraid to try. I like making things.

This journey of constant learning and improvement probably applies to other professions, especially those requiring some level of creativity and craftsmanship from the member. You either evolve to continue creating or your fade away into obscurity.

And if you are passionate about what you're doing, if you are continually learning, continually hungry to be better at what you do, continually looking for ways to evolve your thought processes and code, then inevitably, you have to look back on your creations of the past and feel...

Displeased.

Often I can add other emotions to the mix: embarrassed, appalled, even loathing. But at the very least, looking back on something you've created, you should be able to see how much better you'd be able to do it now. This doesn't mean that your past creations have no value -- especially if they are actually useful and in use. It just means that a natural part of continual evolution is the realization that everything you've ever done, everything you ever will do, given enough distance of time, upon reflection, sucks.

It starts when you recognize that code you wrote a decade ago sucks. It grows as you realize that code you wrote 5 years, even 2 years ago sucks. It crescendos as you realize that the code you wrote 6 months ago sucks -- indeed, even the code you wrote a fortnight ago could be done better with what you've learned in the last sprint. 

It's not a bad thing. The realization allows you to divorce your self-worth from your creations. If anything, you could glean some of your identity from the improvements you've been able to make throughout your career. Because, if you realize that your past creations, in some way or another, all suck, if you realize that this truth will come to pass for your current and future creations, then you have to also come to conclusion that realizing deficiencies in your past accomplishments highlights your own personal evolution.

If you look back over your past code and you don't feel some kind of disappointment, if you can't point out the flaws in your prior works, then I'd have to conclude that you're either stagnating or you're deluding yourself -- perhaps out of pride, perhaps because you've attached your self-worth to the creations you've made. Neither is a good place to be -- you're either becoming irrelevant or you're unteachable and you will become irrelevant.

If you can come to accept this as truth, it also means that you can accept criticism with valid reasoning as an opportunity to learn instead of an attack on your character. 

I watched a video where the speaker posits that code has two audiences: the compiler and your co-workers. The compiler doesn't care about style or readability. The compiler simply cares about syntactical correctness. I've tended to take this a little further with the mantra:

Code is for co-workers, not compilers.

I can't claim to be the original author -- but I also can't remember where I read it. In this case, it does boil down to a simple truth: if your code is difficult for someone else to extend and maintain, it may not be the sparkling gem you think it is. If a junior tasked with updating the code gets lost, calls on a senior for help and that senior can't grok your code -- it's not great. If that senior points that out, then they are doing their job as the primary audience of that code. This is not a place for conflict -- it is a place for learning. Yes, sometimes code is just complex. Sometimes language or framework features are not obvious to an outside programmer looking in. But it's unusual to find complex code which can't be made understandable. Einstein said:

“If you can't explain it to a six year old, you don't understand it yourself.” 

And I'd say this extends to your code -- if your co-workers can't extend it, learn of the (potentially complex) domain, work with the code you've made, then you're missing a fundamental reason for the code to exist: explaining the domain to others through modelling it and solving problems within it.

Working with people who haven't accepted this is difficult -- you can't point out flaws that need to be fixed or, in extreme cases, even fix them without inciting their wrath. You end up having to guerilla-code to resolve code issues or just bite your tongue as you quietly sweep up after them. Or worse -- working around the deficiencies in their code because they insist you depend on it whilst simultaneously denying you the facility to better it.

At some point, we probably all felt the pang when someone pointed out a flaw in our code. Hopefully, as we get older and wiser, this falls away. Personally, I think that divorcing your self-image from your creations, allowing yourself to be critical of the things you've made -- this is one of the marks of maturity that defines the difference between senior developer and junior developer. Not to say that a junior can't master this already -- more to say that I question the "seniority" of a senior who can't do this. It's just one of the skills you need to progress. Like typing or learning new languages.

All of this isn't to say that you can't take pride in your work or that there's no point trying to do your best. We're on this roundabout trying to learn more, to evolve, to be better. You can only get better from a place where you were worse. Once of the best sources of learning is contemplated failure. You can also feel pride in the good parts of what you've created, as long as that is tempered by a realistic, open view on the not-so-good parts. 

You may even like something you've made, for a while, at least. I'm currently in a bit of a honeymoon phase with NExpect (GitHub, Nuget): it's letting me express myself better in tests, it's providing value to handful of other people -- but this too shall pass. At some point, I'm going to look at the code and wonder what I was thinking. I'm going to see a more elegant solution, and I'm going to see the inadequacies of the code. Indeed, I've already experienced this in part -- but it's been overshadowed by the positive stuff that I've experienced, so I'm not quite at the loathing state yet.

You are not your fucking code. When it's faults become obvious, have the grace to learn instead of being offended. 

NExpect level 3: you are the key component

Retrieving the post... Please hold. If the post doesn't load properly, you can check it out here: https://raw.githubusercontent.com/fluffynuts/blog/master/20170917_NExpectLevel3.md

NExpect level 2: testing collections

Retrieving the post... Please hold. If the post doesn't load properly, you can check it out here: https://raw.githubusercontent.com/fluffynuts/blog/master/20170917_NExpectLevel2.md

NExpect level 1: testing objects and values

Retrieving the post... Please hold. If the post doesn't load properly, you can check it out here: https://raw.githubusercontent.com/fluffynuts/blog/master/20170917_NExpectLevel1.md

Fluent, descriptive testing with NExpect

Retrieving the post... Please hold. If the post doesn't load properly, you can check it out here: https://github.com/fluffynuts/blog/blob/master/20180918_IntroducingNExpect.md

This week in PeanutButter

Nothing major, really -- two bugfixes, which may or may not be of interest:

1. PropertyAssert.AreEqual allows for comparison of nullable and non-nullable values of the same underlying type -- which especially makes sense when the actual value being tested (eg nullable int) is being compared with some concrete value (eg int). 
2. Fix for the object extension DeepClone() -- some production code showed that Enum values weren't being copied correctly. So that's fixed now.
   If you're wondering what this is, DeepClone() is an extension method on all objects to provide a copy of that object, deep-cloned (so all reference types are new types, all value types are copied), much like underscore's, _.cloneDeep() for Javascript. This can be useful for comparing a "before" and "after" from a bunch of mutations, especially using the DeepEqualityTester from PeanutButter.Utils or the object extension DeepEquals(), which does deep equality testing, much like you'd expect.

There's also been some assertion upgrading -- PeanutButter consumes, and helps to drive NExpect, an assertions library modelled after Chai for syntax and Jasmine for user-space extensibility. Head on over to Github to check it out -- though it's probably time I wrote something here about it (:

This week in PeanutButter

Ok, so I'm going to give this a go: (semi-)regularly blogging about updates to PeanutButter in the hopes that perhaps someone sees something useful that might help out in their daily code. Also so I can just say "read my blog" instead of telling everyone manually ^_^

So this week in PeanutButter, some things have happened:

• DeepEqualityTester fuzzes a little on numerics -- so you can compare numerics of equal value and different type correctly (ie, (int)2 == (decimal)2). This affects the {object}.DeepEquals() and PropertyAssert.AreDeepEqual() methods.
• DeepEqualityTester can compare fields now too. PropertyAssert.DeepEquals will not use this feature (hey, the name is PropertyAssert!), but {object}.DeepEquals() will, by default -- though you can disable this.
• DuckTyper could duck Dictionaries to interfaces and objects to interfaces -- but now will also duck objects with Dictionary properties to their respective interfaces where possible.
• String extensions for convenience:
• ToKebabCase()
• ToPascalCase()
• ToSnakeCase()
• ToCamelCase()
• DefaultDictionary<TKey, TValue> - much like Python's defaultdict, this provides a dictionary where you give a strategy for what to return when a key is not present. So a DefaultDictionary<string, bool> could have a default value of true or false instead of throwing exceptions on unknown keys.
• MergeDictionary<TKey, TValue> - provides a read-only "view" on a collection of similarly typed IDictionary<TKey, TValue> objects, resolving values from the first dictionary they are found in. Coupled with DefaultDictionary<TKey, TValue>, you can create layered configurations with fallback values.
• DuckTyping can duck from string values to enums

And I'm quite sure the reverse (enum to string) will come for cheap-or-free. So there's that (: You might use this when you have, for example, a Web.Config with a config property "Priority" and you would like to end up with an interface like:

public enum Priorities
{
  Low,
  Medium,
  High
}
public interface IConfig
{
  Priorities DefaultPriority { get; }
}

And a Web.Config line like:

<appSettings>
  <add key="DefaultPriority" value="Medium" />
</appSettings>

Then you could, somewhere in your code (perhaps in your IOC bootstrapper) do:

 var config = WebConfigurationManager.AppSettings.DuckAs<IConfig>();

(This already works for string values, but enums are nearly there (:). You can also use FuzzyDuckAs<T>(), which will allow type mismatching (to a degree; eg a string-backed field can be surfaced as an int) and will also give you freedom with your key names: whitespace and casing don't matter (along with punctuation). (Fuzzy)DuckAs<T>() also has options for key prefixing (so you can have "sections" of settings, with a prefix, like "web.{setting}" and "database.{setting}". But all of that isn't really from this week -- it's just useful for lazy devs like me (:

C# Evolution

(and why you should care)

I may be a bit of a programming language nut. I find different languages interesting not just because they are semantically different or because they offer different features or even because they just look pretty (I'm lookin' at you, Python), but because they can teach us new things about the languages we already use.

I'm of the opinion that no technology with any staying power has no value to offer. In other words, if the tech has managed to stay around for some time, there must be something there. You can hate on VB6 as much as you want, but there has to be something there to have made it dominate desktop programming for the years that it did. That's not the focus of this discussion though.

Similarly, when new languages emerge, instead of just rolling my eyes and uttering something along the lines of "Another programming language? Why? And who honestly cares?", I prefer to take a step back and have a good long look. Creating a new language and the ecosystem required to interpret or compile that language is a non-trivial task. It takes a lot of time and effort, so even when a language seems completely impractical, I like to consider why someone might spend the time creating it. Sure, there are outliers where the only reason is "because I could" (I'm sure Shakespeare has to be one of those) or "because I hate everyone else" (Brainfuck, Whitespace and my favorite to troll on, Perl -- I jest, because Perl is a powerhouse; I just haven't ever seen a program written in Perl which didn't make me cringe, though Larry promises that is supposed to change with Perl 6).

Most languages are born because someone found the language they were dealing with was getting in the way of getting things done. A great example is Go, which was dreamed up by a bunch of smart programmers who had been doing this programming thing for a while and really just wanted to make an ecosystem which would help them to get stuff done in a multi-core world without having to watch out for silly shit like deadlocks and parallel memory management. Not that you can't hurt yourself in even the most well-designed language (indeed, if you definitely can't hurt yourself in the language you're using, you're probably bound and gagged by it -- but I'm not here to judge what you're into).

Along the same lines, it's interesting to watch the evolution of languages, especially as languages evolve out of fairly mundane, suit-and-tie beginnings. I feel like C# has done that -- and will continue to do so. Yes, a lot of cool features aren't unique or even original -- deconstruction in C#7 has been around for ages in Python, for example -- but that doesn't make them any less valuable.

I'm going to skip some of the earlier iterations and start at where I think it's interesting: C#5. Please note that this post is (obviously) opinion. Whilst I'll try to cover all of the feature changes that I can dig up, I'm most likely going to focus on the ones which I think provide the programmer the most benefit.

C#5

C#5 brought async features and caller information. Let's examine the latter before I offer up what will probably be an unpopular opinion on the former.

Caller information allowed providing attributes on optional functional parameters such that if the caller didn't provide a value, the called code could glean

• Caller file path
• Caller line number
• Caller member name

This is a bonus for logging and debugging of complex systems, but also allowed tricks whereby the name of a property could be automagically passed into, for example, an MVVM framework method call in your WPF app. This makes refactor-renaming easier and removes some magic strings, not to mention making debug logging way way simpler. Not ground-breaking, but certainly [CallerMemberName] became a friend of the WPF developer. A better solution  for the exact problem of property names and framework calls came with nameof in C#6, but CallerMember* attributes are still a good thing.

C# 5 also brought async/await. On the surface, this seems like a great idea. If only C#'s async/await worked anything like the async/await in Typescript, where, under the hood, there's just a regular old promise and no hidden bullshit. C#'s async/await looks like it's just going to be Tasks and some compiler magic under the hood, but there's that attached context which comes back to bite asses more often than a mosquito with a scat fetish. There are some times when things will do exactly what you expect and other times when they won't, just because you don't know enough about the underlying system.

That's the biggest problem with async/await, in my opinion: it looks like syntactic sugar to take away the pain of parallel computing from the unwashed masses but ends up just being trickier than having to learn how to do actual multi-threading. There's also the fickle task scheduler which may decide 8 cores doesn't mean 8 parallel tasks -- but that's OK: you can swap that out for your own task scheduler... as long as you understand enough of the underlying system, again (as this test code demonstrates).

Like many problems that arise out of async / parallel programming, tracking down the cause of sporadic issues in code is non-trivial. I had a bit of code which would always fail the first time, and then work like a charm -- until I figured out it was the context that was causing issues, so I forced a null context and the problem stopped. The developer has to start learning about task continuation options and start caring about how external libraries do things. And many developers aren't even aware of when it's appropriate to use async/await, opting to use it everywhere and just add overhead to something which really didn't need to be async, like most web api controllers. Async/await makes a lot of sense in GUI applications though.

Having async/await around IO-bound stuff in web api calls may be good for high-volume web requests because it allows the worker thread to be re-assigned to handle another request. I have yet to see an actual benchmark showing better web performance from simply switching to async/await for all calls.
The time you probably most want to use it is for concurrent IO to shorten the overall request time for a single request. Some thought has to go into this though -- handling too requests concurrently may just end up with many requests timing out instead of a few requests being given a quick 503, indicating that the application needs some help with scaling. In other words, simply peppering your code with async/await could result in no net gain, especially if the code being awaited is hitting the same resource (eg, your database).

Which leads to the second part of C#'s async/await that I hate: the async zombie apocalypse. Because asking the result of a method marked async to just .Wait() is suicide (I hope you know it is; please, please, please don't do that), async/await patterns tend to propagate throughout code until everything is awaiting some async function. It's the Walking Dead, traipsing through your code, leaving little async/await turds wherever they go.

You can use ConfigureAwait() to get around deadlocking on the context selected for your async code -- but you must remember to apply it to all async results if you're trying to "un-async" a block of code. You can also set the synchronization context (I suggest the useful value of null). Like the former workaround, it's hardly elegant.

As much as I hate (fear?) async/await, there are places where it makes the code clearer and easier to deal with. Mostly in desktop applications, under event-handling code. It's not all bad, but the concept has been over-hyped and over-used (and poorly at that -- like EF's SaveChangesAsync(), which you might think, being async, is thread-safe, and you'd be DEAD WRONG).

Let's leave it here: use async/await when it provides obvious value. Question it's use at every turn. For every person who loves async/await, there's a virtual alphabet soup of blogs explaining how to get around some esoteric failure brought on by the feature. As with multi-threading in C/C++: "use with caution".

Read more about C#5 here

C#6

Where C#5 brought low numbers of feature changes (but one of the most damaging), C#6 brought a smorgasbord of incremental changes. There was something there for everyone:

Read-only auto properties made programming read-only properties just a little quicker, especially when the property values came from the constructor. So code like:

public class VideoGamePlumber
{
  public string Name { get { return _name; } }
  private string _name;

  public VideoGamePlumber(string name)
  {
    _name = name;
  }
}

became:

public class VideoGamePlumber
{
  public string Name { get; private set; }

  public VideoGamePlumber(string name)
  {
    Name = name;
  }
}

but that still leaves the Name property open for change within the VideoGamePlumber class, so better would be the C#6 variant:

public class VideoGamePlumber
{
  public string Name { get; }

  public VideoGamePlumber(string name)
  {
    Name = name;
  }
}

The Name property can only be set from within the constructor. Unless, of course, you resort to reflection, since the mutability of Name is enforced by the compiler, not the runtime. But I didn't tell you that.

Auto-property initializers seem quite convenient, but I'll admit that I rarely use them, I think primarily because the times that I want to initialize outside of the constructor, I generally want a private (or protected) field, and when I want to set a property at construction time, it's probably getting it's value from a constructor parameter. I don't hate the feature (at all), just don't use it much. Still, if you wanted to:

public class Doggie
{
  public property Name { get; set; }

  public Doggie()
  {
    Name = "Rex"; // set default dog name
  }
}

becomes:

public class Doggie
{
  public property Name { get; set; } = "Rex";
}

You can combine this with the read-only property syntax if you like:

public class Doggie
{
  public property Name { get; } = "Rex";
}

but then all doggies are called Rex (which is quite presumptuous) and you really should have just used a constant, which you can't modify through reflection.

Expression-bodied function members can provide a succinct syntax for a read-only, calculated property. However, I use them sparingly because anything beyond a very simple "calculation", eg:

public class Person
{
  public string FirstName { get; }
  public string LastName { get; }
  public string FullName => $"{FirstName} {LastName}";

  public Person(string firstName, string lastName)
  {
    FirstName = firstName;
    LastName = lastName;
  }
}

starts to get long and more difficult to read; though that argument is simply countered by having a property like:

public class Business
{
  public string StreetAddress { get; }
  public string Suburb { get; }
  public string Town { get; }
  public string PostalCode { get; }
  public string PostalAddress => GetPostalAddress();

  public Business(string streetAddress, string suburb, string Town, string postalCode)
  {
    StreetAddress = streetAddress;
    Suburb = suburb;
    Town = town;
    PostalCode = postalCode
  }
  
  private string GetAddress()
  {
    return string.Join("\n", new[]
    {
      StreetAddress,
      Suburb,
      Town,
      PostalCode
    });
  }
}

Where the logic for generating the full address from the many bits of involved data is tucked away in a more readable method and the property itself becomes syntactic sugar, looking less clunky than just exposing the GetAddress method.

Index initializers provide a neater syntax for initializing Dictionaries, for example:

var webErrors = new Dictionary()
{
  { 404, "Page Not Found" },
  { 302, "Page moved, but left a forwarding address" },
  { 500, "The web server can't come out to play today" }
};

Can be written as:

private Dictionary webErrors = new Dictionary
{
  [404] = "Page not Found",
  [302] = "Page moved, but left a forwarding address.",
  [500] = "The web server can't come out to play today."
};

It's not ground-breaking, but I find it a little more pleasing on the eye.

Other stuff that I hardly use includes:

Extension Add methods for collection initializers  allow your custom collections to be initialized like standard ones. Not a feature I've ever used because I haven't had the need to write a custom collection.

Improved overload resolution reduced the number of times I shook my fist at the complainer compiler. Ok, so I technically use this, but this is one of those features that, when it's working, you don't even realise it.

Exception filters made exception handling more expressive and easier to read.

await in catch and finally blocks allows the async/await zombies to stumble into your exception hanlding. Yay.

On to the good bits (that I regularly use) though:

using static made using static functions so much neater -- as if they were part of the class you were currently working in. I don't push static functions in general because using them means that testing anything which uses them has to test them too, but there are places where they make sense. One is in RandomValueGen from PeanutButter.RandomGenerators, a class which provides functions to generate random data for testing purposes. A static import means you no longer have to mention the RandomValueGen class throughout your test code:

using PeanutButter.RandomGenerators;

namespace Bovine.Tests
{
  [TestFixture]
  public class TestCows
  {
    [Test]
    public void Moo_ShouldBeAnnoying()
    {
      // Arrange
      var cow = new Cow()
      {
        Name = RandomValueGen.GetRandomString(),
        Gender = RandomValueGen.GetRandom(),
        Age = RandomValueGen.GetRandomInt()
      };
      // ...
    } 
  }
}

Can become:

using static PeanutButter.RandomGenerators.RandomValueGen;

namespace Bovine.Tests
{
  [TestFixture]
  public class TestCows
  {
    [Test]
    public void Moo_ShouldBeAnnoying()
    {
      // Arrange
      var cow = new Cow()
      {
        Name = GetRandomString(),
        Gender = GetRandom(),
        Age = GetRandomInt()
      };
      // ...
    } 
  }
}

Which is way more readable simply because there's less unnecessary cruft in there. At the point of reading (and writing) the test, source library and class for random values is not only irrelevant and unnecessary -- it's just plain noisy and ugly.

Null conditional operators. Transforming fugly multi-step checks for null into neat code:

if (thing != null &&
    thing.OtherThing != null &&
    thing.OtherThing.FavoriteChild != null &&
    // ... and so on, and so forth, turtles all the way down
    //     until, eventually
    this.OtherThing.FavoriteChild.Dog.Collar.Spike.Metal.Manufacturer.Name != null)
{
  return this.OtherThing.FavoriteChild.Dog.Collar.Spike.Metal.Manufacturer.Name;
}
return "Unknown manufacturer";

becomes:

return thing
  ?.OtherThing
  ?.FavoriteChild
  ?.Dog
  ?.Collar
  ?.Spike
  ?.Metal
  ?.Manufacturer
  ?.Name
  ?? "Unknown manufacturer";

and kitties everywhere rejoiced:

String interpolation helps you to turn disaster-prone code like this:

public void PrintHello(string salutation, string firstName, string lastName)
{
  Console.WriteLine("Hello, " + salutation + " " + firstName + " of the house " + lastName);
}

or even the less disaster-prone, more efficient, but not that amazing to read:

public void PrintHello(string salutation, string firstName, string lastName)
{
  Console.WriteLine(String.Join(" ", new[]
  {
    "Hello,",
    salutation,
    firstName,
    "of the house",
    lastName
  }));
}

Into the safe, readable:

public void PrintHello(string salutation, string firstName, string lastName)
{
  Console.WriteLine($"Hello, {salutation} {firstName} of the house {lastName}");
}

nameof is also pretty cool, not just for making your constructor null-checks impervious to refactoring:

public class Person
{
  public Person(string name)
  {
    if (name == null) throw new ArgumentNullException(nameof(name));
  }
}

(if you're into constructor-time null-checks) but also for using test case sources in NUnit:

[Test, TestCaseSource(nameof(DivideCases))]
public void DivideTest(int n, int d, int q)
{
  Assert.AreEqual( q, n / d );
}

static object[] DivideCases =
{
  new object[] { 12, 3, 4 },
  new object[] { 12, 2, 6 },
  new object[] { 12, 4, 3 } 
};

C#7

This iteration of the language brings some  neat features for making code more succinct and easier to grok at a glance.

Inline declaration of out variables makes using methods with out variables a little prettier. This is not a reason to start using out parameters: I still think that there's normally a better way to do whatever it is that you're trying to achieve with them and use of out and ref parameters is, for me, a warning signal in the code of a place where something unexpected could happen. In particular, using out parameters for methods can make the methods really clunky because you have to set them before returning, making quick returns less elegant. Part of me would have liked them to be set to the default value of the type instead, but I understand the rationale behind the compiler not permitting a return before setting an out parameter: it's far too easy to forget to set it and end up with strange behavior.

I think I can count on one hand the number of times I've written a method with an out or ref parameter and I couldn't even point you at any of them. I totally see the point of ref parameters for high-performance code where it makes sense (like manipulating sound or image data). I just really think that when you use out or ref, you should always ask yourself "is there another way to do this?". Anyway, my opinions on the subject aside, there are times when you have to interact with code not under your control and that code uses out params, for example:

public void PrintNumeric(string input)
{
  int result;
  if (int.TryParse(input, out result))
  {
    Console.WriteLine($"Your number is: {input}");
  }
  else
  {
    Console.WriteLine($"'{input}' is not a number )':");
  }
}

becomes:

public void PrintNumeric(string input)
{
  if (int.TryParse(input, out int result))
  {
    Console.WriteLine($"Your number is: {input}");
  }
  else
  {
    Console.WriteLine($"'{input}' is not a number )':");
  }
}

It's a subtle change, but if you have to use out parameters enough, it becomes a lot more convenient, less noisy.

Similarly ref locals and returns, where refs are appropriate, can make code much cleaner. The general use-case for these is to return a reference to a non-reference type for performance reasons, for example when you have a large set of ints, bytes, or structs and would like to pass off to another method to find the element you're interested in before modifying it. Instead of the finder returning an index and the outer call re-referencing into the array, the finder method can simply return a reference to the found element so the outer call can do whatever manipulations it needs to. I can see the use case for performance reasons in audio and image processing as well as large sets of structs of data. The example linked above is quite comprehensive and the usage is for more advanced code, so I'm not going to rehash it here.

Tuples have been available in .NET for a long time, but they've been unfortunately cumbersome. The new syntax in C#7 is changing that. Python tuples have always been elegant and now a similar elegance comes to C#:

public static class TippleTuple
{
  public static void Main()
  {
    // good
    (int first, int second) = MakeAnIntsTuple();
    Console.WriteLine($"first: {first}, second: {second}");
    // better
    var (flag, message) = MakeAMixedTuple();
    Console.WriteLine($"first: {flag}, second: {message}");
  }

  public static (int first, int second) MakeAnIntsTuple()
  {
    return (1, 2);
  }

  public static (bool flag, string name) MakeAMixedTuple()
  {
    return (true, "Moo");
  }
}

Some notes though: the release notes say that you should need to install the System.ValueTuple nuget package to support this feature in VS2015 and lower, but I found that I needed to install the package on VS2017 too. Resharper still doesn't have a clue about the feature as of 2016.3.2, so usages within the interpolated strings above are highlighted as errors. Still, the program above compiles and is way more elegant than using Tuple<> generic types. It's very clever that language features can be delivered by nuget packages though.

Local functions provide a mechanism for keeping little bits of re-used code local within a method. In the past, I've used a Func variable where I've had a little piece of re-usable logic:

private NameValueCollection CreateRandomSettings()
{
  var result = new NameValueCollection();
  Func randInt = () => RandomValueGen.GetRandomInt().ToString();
  result["MaxSendAttempts"] = randInt();
  result["BackoffIntervalInMinutes"] = randInt();
  result["BackoffMultiplier"] = randInt();
  result["PurgeMessageWithAgeInDays"] = randInt();
  return result;
}

which can become:

public static NameValueCollection GetSomeNamedNumericStrings()
{
  var result = new NameValueCollection();
  result["MaxSendAttempts"] = RandInt();
  result["BackoffIntervalInMinutes"] = RandInt();
  result["BackoffMultiplier"] = RandInt();
  result["PurgeMessageWithAgeInDays"] = RandInt();
  return result;

  string RandInt () => GetRandomInt().ToString();
}

Which, I think, is neater. There's also possibly a performance benefit: every time the first implementation of GetSomeNamedNumericStrings is called, the randInt Func is instantiated, where the second implementation has the function compiled at compile-time, baked into the resultant assembly. Whilst I wouldn't put it past the compiler or JIT to do clever optimisations for the first implementation, I wouldn't expect it.

Throw expressions also offer a neatening to your code:

public int ThrowIfNullResult(Func source)
{
  var result = source();
  if (result == null) 
    throw new InvalidOperationException("Null result is not allowed");
  return result;
}

can become:

public int ThrowIfNullResult(Func source)
{
  return source() ?? 
    throw new InvalidOperationException("Null result is not allowd");
}

So now you can actually write a program where every method has one return statement and nothing else.

Time for the big one: Pattern Matching. This is a language feature native to F# and anyone who has done some C#-F# crossover has whined about how it's missing from C#, including me. Pattern matching not only elevates the rather bland C# switch statement from having constant-only cases to non-constant ones:

public static string InterpretInput(string input)
{
  switch (input)
  {
    case string now when now == DateTime.Now.ToString("yyyy/mm/dd"):
      return "Today's date";
    default:
      return "Unknown input";
  }
}

It allows type matching:

public static void Main()
{
  var animals = new List()
  {
    new Dog() {Name = "Rover"},
    new Cat() {Name = "Grumplestiltskin"},
    new Lizard(),
    new MoleRat()
  };
  foreach (var animal in animals)
    PrintAnimalName(animal);
}

public static void PrintAnimalName(Animal animal)
{
  switch (animal)
  {
    case Dog dog:
      Console.WriteLine($"{dog.Name} is a {dog.GetType().Name}");
      break;
    case Cat cat:
      Console.WriteLine($"{cat.Name} is a {cat.GetType().Name}");
      break;
    case Lizard _:
      Console.WriteLine("Lizards have no name");
      break;
    default:
      Console.WriteLine($"{animal.GetType().Name} is mystery meat");
      break;
  }
}

Other new features include generalized async return types, numeric literal syntax improvements and more expression bodied members.

Updates in C# 7.1 and 7.2

The full changes are here (7.1) and here (7.2). Whilst the changes are perhaps not mind-blowing, there are some nice ones:

• Async Main() method now supported
• Default literals, eg: Func foo = default;
• Inferred tuple element names so you can construct tuples like you would with anonymous objects
• Reference semantics with value types allow specifying modifiers:
  • in which specifies the argument is passed in by reference but may not be modified by the called code
  • ref readonly which specifies the reverse: that the caller may not modify the result set by the callee
  • readonly struct makes the struct readonly and passed in by ref
  • ref struct which specifies that a struct type access managed memory directly and must always be allocated on the stack
• More allowance for _ in numeric literals
• The private protected modifier, which is like protected internal except that it restricts usage to derived types only

Wrapping it up

I guess the point of this illustrated journey is that you really should keep up to date with language features as they emerge:

• Many features save you time, replacing tedious code with shorter, yet more expressive code
• Some features provide a level of safety (eg string interpolations)
• Some features give you more power

So if you're writing C# like it's still .net 2.0, please take a moment to evolve as the language already has.