PeanutButter.RandomValueGen: the builder pattern & random generation for testing purposes

Retrieving the post... Please hold.

If the post doesn't load properly, you can check it out here: 
https://github.com/fluffynuts/blog/blob/master/20180825.md

# What's in `PeanutButter.Utils`, exactly?

`PeanutButter.Utils` is a package which pretty-much evolved as I had common problems that I was solving day-to-day. People joining a team that I was working on would be exposed to bits of it and, like a virus, those bits would propagate across other code-bases. Some people asked for documentation, which I answered with a middle-ground of `xmldoc`, which most agreed was good enough. People around me got to know of the more useful bits in `PeanutButter.Utils` or would ask me questions like "Does PeanutButter.Utils have something which can do [X]?". I kind of took the ubiquity amongst my team-mates for granted.

Fast-forward a little bit, and I've moved on to another company, where people don't know anything about the time-savers in `PeanutButter.Utils` -- and it occurs to me that that statement probably applies to pretty-much most people -- so I thought it might be worthwhile to have some kind of primer on what you can expect to find in there. An introduction, if you will. I think there's enough to break the content down into sections, so we can start with:

## Disposables
One of the patterns I like most in the .net world is that of `IDisposable`. It's a neat way to ensure that something happens at the end of a block of code irrespective of what goes on _inside_ that code. The code could throw or return early -- it doesn't matter: whatever happens in the `Dispose` method of the `IDisposable` declared at the top of a `using` block will be run. Usually, we use this for clearing up managed resources (eg on database connections), but it struck me that there were some other convenient places to use it. Most generically, if you wanted to run something simple at the start of a block of code and run something else at the end (think of toggling something on for the duration of a block of code), you could use the convienient `AutoResetter` class:
```csharp
using (new AutoResetter(
    () => ToggleFeatureOn(), 
    () => ToggleFeatureOff()))
{
    // code inside here has the feature toggled on
}
// code over here doesn't -- and the feature is 
//    toggled back off again even if the code 
//    above throws an exception.
```
It's very simple -- but it means that you can get the functionality of an `IDisposable` by writing two little lambda methods.

You can also have a variant where the result from the first lambda is fed into the second:
```csharp
using (new AutoResetter(
    () => GetCounterAndResetToZero(),
    originalCount => ResetCounterTo(originalCount)))
{
// counter is zero here
}
// counter is reset to original value here
```

Cool.

Other common problems that can be solved with `IDisposable` are:
### Ensuring mutexes / semaphores are reset, even if an exception is encountered
For this, we can use `AutoLocker`:
```csharp
using (new AutoLocker(someMutex))
{
}
using (new AutoLocker(someSemaphore))
{
}
using (new AutoLocker(someSemaphoreLite))
{
}
```

### Temporary files in tests
```csharp
using (var tempFile = new AutoTempFile())
{
   File.WriteAllBytes(
       Encoding.UTF8.GetBytes("moo, said the cow"),
       tempFile.Path
   );
   // we can run testing code against the file here
}
// file is gone here, like magick!
```
This uses the `Path.GetTempFileName()` system call by default -- so you don't have to care about where the file actually exists. Of course, there are constructor overloads to:
- create the file populated with data (string or bytes)
- create the file in a different location (not the system temp data location)
- create the file with a specific name

`AutoTempFile` also exposes the files contents via properties:
- `StringData` for string contents
- `BinaryData` for a byte[] array

There is also an `AutoTempFolder` if you want a scratch area to work in for a period of time. When it is disposed, it and all it's contents are deleted.

Similarly, `AutoDeleter` is an `IDisposable` which can keep track of multiple files you'd like to delete when it is disposed:
```csharp
using (var deleter = new AutoDeleter())
{
    // some files are created, then we can do:
    deleter.Add("C:\Some\File");
    deleter.Add("C:\Some\Other\File");
}
// and here, those files are deleted. If they can't be 
//    deleted, (eg they are locked by some process),
//    then the error is quietly suppressed. 
//    `AutoDeleter` works for folders too.
```

### Other disposables
As much as I love the `using` pattern, it can lead to some "arrow code", like this venerable ADO.NET code:
```csharp
using (var conn = CreateDbConnection())
{
    conn.Open();
    using (var cmd = conn.CreateCommand())
    {
        cmd.CommandText = "select * from users";
        using (var reader = cmd.ExecuteReader())
        {
            // read from the reader here
        }
    }
}
```
Sure, many people don't use ADO.NET "raw" like this any more -- it's just an easy example which comes to mind. I've seen far worse "nest-denting" of `using` blocks too.

This can be flattened out a bit with `AutoDisposer`:
```csharp
using (var disposer = new AutoDisposer())
{
    var conn = disposer.Add(CreateDbConnection());
    conn.Open();
    var cmd = disposer.Add(conn.CreateCommand());
    cmd.CommandText = "select * from users";
    var reader = disposer.Add(conn.ExecuteReader());
    // read from the db here
}
// reader is disposed
// cmd is disposed
// conn is disposed
```
`AutoDisposer` disposes of items in reverse-order in case of any disposing dependencies.


So that's part 1 of "What's in `PeanutButter.Utils`?". There are other interesting bits, like:
- extension methods to
    - make some operations more convenient
         - do conversions
         - working with `Stream` objects
         - DateTime utilities
    - facilitate more functional code (eg `.ForEach` for collections)
    - `SelectAsync` and `WhereAsync` let you use async lambdas in your LINQ
    - test and manipulate strings
- the `DeepEqualityTester`, which is at the heart of `NExpect`s `.Deep` and `.Intersection` equality testing
- `MemberExpression` helpers
- Reflection tidbits
- reading and writing arbitrary metadata for any object you encounter (think like adding property data to any object)
- Some pythonic methods (`Range` and a (imo) more useful `Zip` than the one bundled in LINQ)
- dictionaries
    - `DictionaryWrappingObject` lets you treat any object like you would in Javascript, with text property indexes
    - `DefaultDictionary` returns default values for unknown keys
    - `MergeDictionary` allows layering multiple dictionaries into one "view"
    - `CaseWarpingDictionary` provides a decorator dictionary for when the dictionary you have does indexing with inconvenient case rules

I hope to tackle these in individual posts (:

What's new in PeanutButter

I realise that it's been a while (again) since I've posted an update about new things in PeanutButter (GitHub, Nuget). I've been slack!

Here are the more interesting changes:

1. PeanutButter.Utils now has a netstandard2.0 target in the package. So you can use those tasty, tasty utils from netcore2.0
2. This facilitated adding a netstandard2.0 target for PeanutButter.RandomGenerators -- so now you can use the GenericBuilder and RandomValueGen, even on complex types. Yay! Spend more time writing interesting code and less time thinking of a name on your Person object (:
3. Many fixes to DeepClone(), the extension method for objects, which gives you back a deep-cloned version of the original, including support for collections and cloning the object's actual type, not the type inferred by the generic usage - so a downcast object is properly cloned now.
4. DeepEqualityTester can be used to test shapes of objects, without caring about actual values -- simply set OnlyCompareShape to true. Combined with setting FailOnMissingProperties to false, a deep equality test between an instance of your model and an instance of an api model can let you know if your models have at least the required fields to satisfy the target api.
5. Random object generation always could use NSubstitute to generate instances of interfaces, but now also supports using PeanutButter.DuckTyping's Create.InstanceOf<T>() to create your random instances. Both strategies are determined at runtime from available assemblies -- there are no hard dependencies on NSubstitute or PeanutButter.DuckTyping.
6. More async IEnumerable extensions (which allow you to await on a Linq query with async lambdas -- but bear in mind that these won't be translated to, for example, Linq-to-Sql code to run at the server-side):
1. SelectAsync
2. ToArrayAsync
3. WhereAsync
4. AggregateAsync

There was also some cleanup (I defaulted R# to show warnings, not just errors -- and squashed around 2000 of them) and made all projects depend on the newer PackageReferences mechanism for csproj files.

So, yeah, stuff happened (:

This week in PeanutButter

Nothing major, really -- two bugfixes, which may or may not be of interest:

1. PropertyAssert.AreEqual allows for comparison of nullable and non-nullable values of the same underlying type -- which especially makes sense when the actual value being tested (eg nullable int) is being compared with some concrete value (eg int). 
2. Fix for the object extension DeepClone() -- some production code showed that Enum values weren't being copied correctly. So that's fixed now.
   If you're wondering what this is, DeepClone() is an extension method on all objects to provide a copy of that object, deep-cloned (so all reference types are new types, all value types are copied), much like underscore's, _.cloneDeep() for Javascript. This can be useful for comparing a "before" and "after" from a bunch of mutations, especially using the DeepEqualityTester from PeanutButter.Utils or the object extension DeepEquals(), which does deep equality testing, much like you'd expect.

There's also been some assertion upgrading -- PeanutButter consumes, and helps to drive NExpect, an assertions library modelled after Chai for syntax and Jasmine for user-space extensibility. Head on over to Github to check it out -- though it's probably time I wrote something here about it (:

This week in PeanutButter

Ok, so I'm going to give this a go: (semi-)regularly blogging about updates to PeanutButter in the hopes that perhaps someone sees something useful that might help out in their daily code. Also so I can just say "read my blog" instead of telling everyone manually ^_^

So this week in PeanutButter, some things have happened:

• DeepEqualityTester fuzzes a little on numerics -- so you can compare numerics of equal value and different type correctly (ie, (int)2 == (decimal)2). This affects the {object}.DeepEquals() and PropertyAssert.AreDeepEqual() methods.
• DeepEqualityTester can compare fields now too. PropertyAssert.DeepEquals will not use this feature (hey, the name is PropertyAssert!), but {object}.DeepEquals() will, by default -- though you can disable this.
• DuckTyper could duck Dictionaries to interfaces and objects to interfaces -- but now will also duck objects with Dictionary properties to their respective interfaces where possible.
• String extensions for convenience:
• ToKebabCase()
• ToPascalCase()
• ToSnakeCase()
• ToCamelCase()
• DefaultDictionary<TKey, TValue> - much like Python's defaultdict, this provides a dictionary where you give a strategy for what to return when a key is not present. So a DefaultDictionary<string, bool> could have a default value of true or false instead of throwing exceptions on unknown keys.
• MergeDictionary<TKey, TValue> - provides a read-only "view" on a collection of similarly typed IDictionary<TKey, TValue> objects, resolving values from the first dictionary they are found in. Coupled with DefaultDictionary<TKey, TValue>, you can create layered configurations with fallback values.
• DuckTyping can duck from string values to enums

And I'm quite sure the reverse (enum to string) will come for cheap-or-free. So there's that (: You might use this when you have, for example, a Web.Config with a config property "Priority" and you would like to end up with an interface like:

public enum Priorities
{
  Low,
  Medium,
  High
}
public interface IConfig
{
  Priorities DefaultPriority { get; }
}

And a Web.Config line like:

<appSettings>
  <add key="DefaultPriority" value="Medium" />
</appSettings>

Then you could, somewhere in your code (perhaps in your IOC bootstrapper) do:

 var config = WebConfigurationManager.AppSettings.DuckAs<IConfig>();

(This already works for string values, but enums are nearly there (:). You can also use FuzzyDuckAs<T>(), which will allow type mismatching (to a degree; eg a string-backed field can be surfaced as an int) and will also give you freedom with your key names: whitespace and casing don't matter (along with punctuation). (Fuzzy)DuckAs<T>() also has options for key prefixing (so you can have "sections" of settings, with a prefix, like "web.{setting}" and "database.{setting}". But all of that isn't really from this week -- it's just useful for lazy devs like me (:

EF-based testing, with PeanutButter: Shared databases

The PeanutButter.TestUtils.Entity Nuget package provides a few utilities for testing EntityFramework-based code, backed by TempDb instances so you can test that your EF code works as in production instead of relying on (in my experience) flaky substitutions.
One is the EntityPersistenceTester, which provides a fluent syntax around proving that your data can flow into and out of a database. I'm not about to discuss that in-depth here, but it does allow (neat, imo) code like the following to test POCO persistence:

// snarfed from EmailSpooler tests
[Test]
public void EmailAttachment_ShouldBeAbleToPersistAndRecall()
{
    EntityPersistenceTester.CreateFor<EmailAttachment>()
        .WithContext<EmailContext>()
        .WithDbMigrator(MigratorFactory)
        .WithSharedDatabase(_sharedTempDb)
        .WithAllowedDateTimePropertyDelta(_oneSecond)
        .ShouldPersistAndRecall();
}

which prove that an EmailAttachment POCO can be put into, and successfully retrieved from a database, allowing DateTime properties to drift by a second. All very interesting, and Soon To Be Documented™, but not the focus of this entry.

I'd like to introduce a new feature, but to do so, I have to introduce where it can be used. A base class TestFixtureWithTempDb<T> exists within PeanutButter.TestUtils.Entity. It provides some of the scaffolding required to do more complex testing than just "Can I put a POCO in there?". The generic argument is some implementation of DbContext and it's most useful when testing a repository as it provides a protected GetContext() method which provides a spun-up context of type T, with an underlying temporary database. By default, this is a new, clean database every test, but you can invoke the protected DisableDatabaseRegeneration() method in your test fixture's [OneTimeSetup]-decorated method (or constructor, if you prefer) to make this database live for the lifetime of your test fixture. The base class takes care of disposing of the temporary database when appropriate so you can focus on the interesting stuff: getting your tests (and then code) to work. A full (but simple) example of usage can be found here: https://github.com/fluffynuts/PeanutButter/blob/master/source/Utils/PeanutButter.FluentMigrator.Tests/TestDbMigrationsRunner.cs

My focus today is on a feature which can help to eliminate a pain-point I (and others) have experienced with EF testing backed onto a TempDb instance: time to run tests. EF takes a second or two to generate internal information about a database the first time some kind of activity (read/write) to that database is done via an EF DbContext. Not too bad on application startup, but quite annoying if it happens at every test. DisableDatabaseRegeneration() helps, but still means that each test fixture has a spin-up delay, meaning that when there are a few test fixtures, other developers on your team become less likely to run the entire test suite -- which is bad for everyone.

However, after some nudging in the right direction by co-worker Mark Whitfeld, I'd like to announce the availability of the UseSharedTempDb attribute in PeanutButter.TestUtils.Entity as of version 1.2.120, released today.
To use, decorate your test fixture:

[TestFixture]
[UseSharedTempDb("SomeSharedTempDbIdentifier")]
public class TestCrossFixtureTempDbLifetimeWithInheritence_Part1
    : TestFixtureWithTempDb
{
    // .. actual tests go here ..
}

And run your tests. And see an exception:

PeanutButter.TestUtils.Entity.SharedTempDbFeatureRequiresAssemblyAttributeException : 
The UseSharedTempDb class attribute on TestSomeStuff 
requires that assembly SomeProject.Tests have the attribute AllowSharedTempDbInstances.

Try adding the following to the top of a class file:
[assembly: PeanutButter.TestUtils.Entity.Attributes.AllowSharedTempDbInstances]

So, follow the instructions and add the assembly attribute to the top of your test fixture source file and re-run your tests. Congratulations, you're using a shared instance of a TempDb which will be cleaned up when NUnit is finished, providing, of course, that you don't interrupt the test run yourself (:

Legacy database testing with PeanutButter

Preamble

Recently, I've been asked about a strategy for testing code which works against an existing database. It's not the first time I've been asked about this -- and it probably won't be the last. I should have listened to Scott Hanselman's advice long ago and blogged it.

Better late than never, I suppose. Let's get to it!

The problem

You have some code which hits a MSSQL database to do something. It could be hitting a stored procedure for a report. It could be inserting or updating rows. Whatever the use-case, you'd like to have that code under test, if not just for your own sanity, then because you're about to extend that code and are just plain scared that you're about to break something which already exists. This is a valid reason -- and it drove me to the strategy I'll outline below.

You may also simply wish to write your code test-first, but have this great existing legacy mass which you have to work with (and around) and you're just struggling to get the first test out.

Please note: this strategy outlines more integration-style testing than true unit testing. However, I'd rather have an integration test than no test any day. This kind of testing also leads to tests which take a few seconds to run (instead of the preferred milliseconds) -- but I'd rather have slow tests than no tests.

Note that I'm tackling MSSQL because:

1. It's a common production database
2. If you were dealing with simpler databases like SQLite or SQLCE, you may already have a strategy to deal with this (though PB can still make it easier, so read on)
3. I haven't found (yet) a nice way to do temporary in-process MySQL or PostgreSQL. You could use this strategy with Firebird since server and embedded can even use the same file (but not concurrently, of course) -- though currently PeanutButter.TempDb has no baked-in Firebird provider. I guess I should fix that!

So, let's really get to it!

The general idea

Ideally, I'd like to have some kind of test framework which would spin up a temporary database, create all the structures (tables, views) that I need, perhaps the programmability (procedures, functions) I'd like to test (if applicable) and also provides a mechanism for loading in data to test against so that I can write "when {condition} then {expectation}"-style tests.

I'd also like that temporary database to die by fire when my tests are done. I don't want to have to clean anything up manually.

Traditionally, running instances of database services have been used for this style of testing -- but that leaves you with a few sticky bits:

1. Your development and CI environments have to be set up the same, with credentials baked into the test suite. Or perhaps you can use environment overrides -- still, authorization to the test database has to be a concern
2. Test isolation is less than trivial and as the test suite grows, the tests start interacting with each other, even if there is cleanup at some point along the way.
3. Getting a new developer on the team is more effort than it really should be, mainly because of (1) above. For the same reasons, developing on a "loaner" or laptop is also more effort than it should be. You can't just "check out and go".

Some prior strategies exist to cope with this, but they are unsatisfactory:

1. A shared development/test database which accumulates cruft and potentially causes strange test behaviour when unexpected data is matched by systems under test
2. Swapping out the production database layer for something like SQLite. Whilst I really like SQLite, the problem boils down to differences in functionality between SQLite and whatever production database you're using. I've come across far to many recently, in a project where tests are run against SQLite and production code runs against PostgreSQL. I've seen similar issues with testing code targeting SQL Server on SQLCE. Even if you have a fairly beefy ORM layer (EF, NHibernate, etc) to abstract a lot of the database layer away from you, you're going to hit issues. I can think of too many to put them out here -- if you really want a list of the issues I've hit in this kind of scenario, feel free to ask. I've learned enough to feel fear when someone suggests testing on a database other than the engine you're going to deploy on.
   Sometimes you have tests which work when production code fails. Sometimes your tests simply can't test what you want to do because the test database engine is "differently capable".
3. For similar reasons to (2) above, even if you're testing down to the ORM layer, mocked / substituted database contexts (EF) can provide you with tests which work when your production code is going to fail.

So we'd like to test against "real iron", but we'd like that iron to be transient.

PeanutButter to the rescue (:

The strategy that emerged

1. Create a temporary database (PeanutButter.TempDb.(some flavor))
1. Fortunately, when production code is going to be talking to SQL Server, we can use a LocalDb database for testing -- all the functionality of SQL Server (well, pretty-much all of it, enough for application code -- you'll be missing full text search for example, but the engine is basically the same).
2. Create the database structures required (via migrations or scripts)
3. Run the tests on the parts of the system to be tested
4. Dispose of the temporary database when done, leaving no artifacts and no cruft for another test or test fixture.

public void TestSomeLegacySystem
{
  [Test]
  public void TestSomeLegacyMethod()
  {
    // Arrange
    using (var db = new TempDbLocalDb())
    { 
      using (var conn = db.CreateConnection())
      {
        // run in database schema script(s)
        // insert any necessary data for the test 
      }
      // Act
      // write actual test action here
      // Assert 
      using (var conn = db.CreateConnection())
      {
        // perform assertions on the database with the new connection 
      } 
    } // our TempDb is destroyed here -- nothing to clean up!
  } 
}

This is very raw, ADO-style code. Of course, it's not too difficult to extrapolate to using an EF context since TempDb exposes both a connection string or you could pass a new connection from CreateConnection to your context's constructor, which would call into DbContext's constructor which can take a DbConnection -- and you would set the second (boolean) parameter based on whether or not you'd like to dispose of the connection yourself. 

I did this often enough that it became boring and my laziness kicked in. Ok, so that happened at about the third test...

And so TestFixtureWithTempDb was born: this forms a base class for a test fixture requiring a temporary database of the provided type. It has a protected Configure() method which must be used to instruct the base class how to create the database (with an initial migrator to get the database up to speed), as well as providing hints; for example, by default, a new TempDb is spun up for every test, but if you're willing to take care of cleaning out crufty data after each test (perhaps with a [Teardown]-decorated method), then you can share the database between tests in the fixture for a little performance boost. The boost is more noticable when you also have EF talking to the temporary database as EF will cache model information per database on first access -- so even if you have the same structures in two different iterations, EF will go through the same mapping steps for each test, adding a few seconds (I find typically about 2-5) per test.

Indeed, if you have an EF context, you perhaps want to step one up the chain to EntityPersistenceTestFixtureBase, which is inherited with a generic type that is your DbContext. Your implementation must have a constructor which takes just a DbConnection for this base class to function. If you've created your context from EDMX, you'll have to create another partial class with the same name and the expected constructor signature, passing off to an intermediatary static method which transforms a DbConnection into an EntityConnection; otherwise, just add a constructor.

And this is where my laziness kicks in again: the test fixture for EntityPersistenceTestFixtureBase provides a reasonable example for usage. Note the call to Configure() in the constructor -- you could also have this call in a [OneTimeSetup] method. If you forget it, PeanutButter will bleat at you -- but helpfully, instructing you to Configure() the test fixture before running tests (:

Some interesting points:

1. Configure's first parameter is a boolean: when true, the configuration will create ASP.NET tables in the temporary database for you (since it's highly unlikely you'll have them in your own migrations). This is useful only if you're intending to test, for example, an MVC controller which will change behaviour based on the default ASP.NET authentication mechanisms. Mostly, you'll want this to be false.
2. The second parameter is a factory function: it takes in a connection string and should emit something which implements IDBMigrationsRunner -- this is an instance of a class with a MigratoToLatest() method which performs whatever is necessary to build the required database structures. You could wrap a FluentMigrator instance, or you can use the handy DbSchemaImporter, given a dump of your database as scripts (without the use statement!) to run in your existing schema. When doing the latter, I simply import said script in a regular old .net resource -- when doing so, you'll get a property on that resource which is a string: the script to run (:
3. You can configure a method to run before providing an new EF context -- when configured, this method will be given the EF context which is first created in a test so that it can, for example, clear out old data. Obviously, this only makes sense if you're going full-EF.
4. If you have a hefty legacy database, expect some minor issues that you'll have to work through. I've found, for instance, procedures which compiled in SSMS, but not when running in the script for said procedure because it was missing a semi-colon. Don't despair: the effort will be worth it. You can also try only scripting out the bare minimum of the target database that is required for your tests.

Enough blathering!

Ok, this has been a post well-worth a TL;DR. It's the kind of thing would would probably work better as a 15-minute presentation, but I suppose some blog post is better than no post (:

Questions? Comments? They're all welcome. If there's something you'd like me to go more in-depth with, shout out -- I can always re-visit this topic (:

Legacy database testing with PeanutButter

Preamble

Recently, I've been asked about a strategy for testing code which works against an existing database. It's not the first time I've been asked about this -- and it probably won't be the last. I should have listened to Scott Hanselman's advice long ago and blogged it.

Better late than never, I suppose. Let's get to it!

The problem

You have some code which hits a MSSQL database to do something. It could be hitting a stored procedure for a report. It could be inserting or updating rows. Whatever the use-case, you'd like to have that code under test, if not just for your own sanity, then because you're about to extend that code and are just plain scared that you're about to break something which already exists. This is a valid reason -- and it drove me to the strategy I'll outline below.

You may also simply wish to write your code test-first, but have this great existing legacy mass which you have to work with (and around) and you're just struggling to get the first test out.

Please note: this strategy outlines more integration-style testing than true unit testing. However, I'd rather have an integration test than no test any day. This kind of testing also leads to tests which take a few seconds to run (instead of the preferred milliseconds) -- but I'd rather have slow tests than no tests.

Note that I'm tackling MSSQL because:

1. It's a common production database
2. If you were dealing with simpler databases like SQLite or SQLCE, you may already have a strategy to deal with this (though PB can still make it easier, so read on)
3. I haven't found (yet) a nice way to do temporary in-process MySQL or PostgreSQL. You could use this strategy with Firebird since server and embedded can even use the same file (but not concurrently, of course) -- though currently PeanutButter.TempDb has no baked-in Firebird provider. I guess I should fix that!

So, let's really get to it!

The general idea

Ideally, I'd like to have some kind of test framework which would spin up a temporary database, create all the structures (tables, views) that I need, perhaps the programmability (procedures, functions) I'd like to test (if applicable) and also provides a mechanism for loading in data to test against so that I can write "when {condition} then {expectation}"-style tests.

I'd also like that temporary database to die by fire when my tests are done. I don't want to have to clean anything up manually.

Traditionally, running instances of database services have been used for this style of testing -- but that leaves you with a few sticky bits:

1. Your development and CI environments have to be set up the same, with credentials baked into the test suite. Or perhaps you can use environment overrides -- still, authorization to the test database has to be a concern
2. Test isolation is less than trivial and as the test suite grows, the tests start interacting with each other, even if there is cleanup at some point along the way.
3. Getting a new developer on the team is more effort than it really should be, mainly because of (1) above. For the same reasons, developing on a "loaner" or laptop is also more effort than it should be. You can't just "check out and go".

Some prior strategies exist to cope with this, but they are unsatisfactory:

1. A shared development/test database which accumulates cruft and potentially causes strange test behaviour when unexpected data is matched by systems under test
2. Swapping out the production database layer for something like SQLite. Whilst I really like SQLite, the problem boils down to differences in functionality between SQLite and whatever production database you're using. I've come across far to many recently, in a project where tests are run against SQLite and production code runs against PostgreSQL. I've seen similar issues with testing code targeting SQL Server on SQLCE. Even if you have a fairly beefy ORM layer (EF, NHibernate, etc) to abstract a lot of the database layer away from you, you're going to hit issues. I can think of too many to put them out here -- if you really want a list of the issues I've hit in this kind of scenario, feel free to ask. I've learned enough to feel fear when someone suggests testing on a database other than the engine you're going to deploy on.
   Sometimes you have tests which work when production code fails. Sometimes your tests simply can't test what you want to do because the test database engine is "differently capable".
3. For similar reasons to (2) above, even if you're testing down to the ORM layer, mocked / substituted database contexts (EF) can provide you with tests which work when your production code is going to fail.

So we'd like to test against "real iron", but we'd like that iron to be transient.

PeanutButter to the rescue (:

The strategy that emerged

1. Create a temporary database (PeanutButter.TempDb.(some flavor))
1. Fortunately, when production code is going to be talking to SQL Server, we can use a LocalDb database for testing -- all the functionality of SQL Server (well, pretty-much all of it, enough for application code -- you'll be missing full text search for example, but the engine is basically the same).
2. Create the database structures required (via migrations or scripts)
3. Run the tests on the parts of the system to be tested
4. Dispose of the temporary database when done, leaving no artifacts and no cruft for another test or test fixture.

public void TestSomeLegacySystem
{
  [Test]
  public void TestSomeLegacyMethod()
  {
    // Arrange
    using (var db = new TempDbLocalDb())
    { 
      using (var conn = db.CreateConnection())
      {
        // run in database schema script(s)
        // insert any necessary data for the test 
      }
      // Act
      // write actual test action here
      // Assert 
      using (var conn = db.CreateConnection())
      {
        // perform assertions on the database with the new connection 
      } 
    } // our TempDb is destroyed here -- nothing to clean up!
  } 
}

This is very raw, ADO-style code. Of course, it's not too difficult to extrapolate to using an EF context since TempDb exposes both a connection string or you could pass a new connection from CreateConnection to your context's constructor, which would call into DbContext's constructor which can take a DbConnection -- and you would set the second (boolean) parameter based on whether or not you'd like to dispose of the connection yourself. 

I did this often enough that it became boring and my laziness kicked in. Ok, so that happened at about the third test...

And so TestFixtureWithTempDb was born: this forms a base class for a test fixture requiring a temporary database of the provided type. It has a protected Configure() method which must be used to instruct the base class how to create the database (with an initial migrator to get the database up to speed), as well as providing hints; for example, by default, a new TempDb is spun up for every test, but if you're willing to take care of cleaning out crufty data after each test (perhaps with a [Teardown]-decorated method), then you can share the database between tests in the fixture for a little performance boost. The boost is more noticable when you also have EF talking to the temporary database as EF will cache model information per database on first access -- so even if you have the same structures in two different iterations, EF will go through the same mapping steps for each test, adding a few seconds (I find typically about 2-5) per test.

Indeed, if you have an EF context, you perhaps want to step one up the chain to EntityPersistenceTestFixtureBase, which is inherited with a generic type that is your DbContext. Your implementation must have a constructor which takes just a DbConnection for this base class to function. If you've created your context from EDMX, you'll have to create another partial class with the same name and the expected constructor signature; otherwise, just add a constructor.

And this is where my laziness kicks in again: the test fixture for EntityPersistenceTestFixtureBase provides a reasonable example for usage. Note the call to Configure() in the constructor -- you could also have this call in a [OneTimeSetup] method. If you forget it, PeanutButter will bleat at you -- but helpfully, instructing you to Configure() the test fixture before running tests (:

Some interesting points:

1. Configure's first parameter is a boolean: when true, the configuration will create ASP.NET tables in the temporary database for you (since it's highly unlikely you'll have them in your own migrations). This is useful only if you're intending to test, for example, an MVC controller which will change behaviour based on the default ASP.NET authentication mechanisms. Mostly, you'll want this to be false.
2. The second parameter is a factory function: it takes in a connection string and should emit something which implements IDBMigrationsRunner -- this is an instance of a class with a MigratoToLatest() method which performs whatever is necessary to build the required database structures. You could wrap a FluentMigrator instance, or you can use the handy DbSchemaImporter, given a dump of your database as scripts (without the use statement!) to run in your existing schema. When doing the latter, I simply import said script in a regular old .net resource -- when doing so, you'll get a property on that resource which is a string: the script to run (:
3. You can configure a method to run before providing an new EF context -- when configured, this method will be given the EF context which is first created in a test so that it can, for example, clear out old data. Obviously, this only makes sense if you're going full-EF.
4. If you have a hefty legacy database, expect some minor issues that you'll have to work through. I've found, for instance, procedures which compiled in SSMS, but not when running in the script for said procedure because it was missing a semi-colon. Don't despair: the effort will be worth it. You can also try only scripting out the bare minimum of the target database that is required for your tests.

Enough blathering!

Ok, this has been a post well-worth a TL;DR. It's the kind of thing would would probably work better as a 15-minute presentation, but I suppose some blog post is better than no post (:

Questions? Comments? They're all welcome. If there's something you'd like me to go more in-depth with, shout out -- I can always re-visit this topic (:

Update PeanutButter - now with less pain!

 (and possibly more fibre)

Necessity Laziness is the mother of all invention. It's the reason PeanutButter (https://github.com/fluffynuts/PeanutButter) even exists and is on Nuget. But as PeanutButter has expanded in its modular fashion, one thing has bugged me: updating.

When I change one Nuget library, I update all Nuget packages to avoid any confusion about which version of what plays nicely with the other - to the point where I've even made each package which depends on another PeanutButter package depend on the same version as itself. I wanted an easy way to update all PeanutButter packages since I release quite often - indeed, pretty-much whenever I add any functionality.

An approach to this problem might be running update-package from the package manager console. Whilst I'm a fan of keeping libraries up to date, sometimes you can get unexpected consequences from this action such as breaking a site depending on an older version of ASP.NET MVC. I have no control over those other packages -- but I do control PeanutButter, so what I need is something more like update-peanutbutter.

And now I (and anyone else using PeanutButter) have it.

Thanks to an excellent tutorial by Phil Haack on his blog You've Been Haacked, I've added the command via a module loaded from the PeanutButter.Utils Nuget package init script. PeanutButter.Utils is one of the "lowest-down" packages, so chances are very good that if you're using any of the others, you're using PeanutButter.Utils. The change is available from version 1.2.15 and the easiest way to take advantage of it would be to update one of your projects to use the latest PeanutButter.Utils and then use update-peanutbutter from the package manager console to update all the other projects in your solution.

Happy hacking (:

PeanutButter is updated to NUnit 3

NUnit (https://github.com/nunit/nunit) has been updated to version 3 for a little while now, so I thought it best to move PeanutButter forward as the test utilities which use NUnit for assertions and such have required the user to explicitly install NUnit 2.6.4.

In particular, this will affect any new installations or updates to projects using the following:

•  PeanutButter.TestUtils.Generic 
•    Which includes PropertyAssert
• PeanutButter.TestUtils.Entity
• Which includes the helpers around testing database persistence of your entity models

What this means is:

• If you’re not depending on any of these PeanutButter modules, of course nothing changes for you. Though I’d still recommend moving forward to NUnit 3 at some point when you have some time. There are some changes and some things to deal with, so don't rush it.
•  If you have a project which is currently stable and not in active development, don’t bother updating anything unless you really want to stay up-to-date.
•  If you’re going to install-package PeanutButter.<something>, you may find that you’re automatically updated to NUnit 3. Mostly, it’s not a problem and the differences are easy to work around. If you get a PeanutButter package with version 1.1.x instead of 1.0.x, you’re going with NUnit 3. If you don’t want to go this route, install the last 1.0.x version: 1.0.155. You may have to specify an NUnit version to install, ie:
  install-package NUnit –project <your project> -version 2.6.4.
  I used to make my packages not depend on specific versions of anything, but I’ve recently started adding minimum versioning to packages to try to alleviate some of this headache.

Please welcome PeanutButter 1.0.118!

 (if you don't know what PeanutButter is -- apart from a yummy spread and a rather silly name for an open-source project -- have a gander here: http://davydm.blogspot.co.za/2014/06/introducing.html)

What's new:

• Updates to DatabaseHelpers:
• Fluent syntax for conditions (.And() and .Or() extension methods)
• EqualityOperators changes
• Like_ is deprecated (but still functions). It is replaced by Contains, with the same functionality
• Like is new: does not insert any wildcards -- that's up to you! (also, finally, has a reasonable name, now that I know how to use keywords as identifiers in VB.NET. Yes, the DBHelpers are written in VB.NET. Get over it.)
• StartsWith: does what it says on the tin
• EndsWith: also does what it says on the tin
• Added ConcatenatedField for when you want some composite result from a few fields
• Updated CalculatedField: All *Fields now implement IField, so all of the existing funcationality which SelectFields could give you can also be found for the others (conditions, etc)
• Enumerable convenience functions
• And() -- like Union() but produces an Array
• eg new[] { 1, 2 }.And(3) is equivalent to new[] { 1,2,3}
• ButNot() -- like Except(), but produces an Array
• RandomGenerators:
• RandomValueGen.GetRandomCollection
• Takes a factory function and optionally min/max number of items
• eg to get a random-sized IEnumerable<int> of random ints, do:
  RandomValueGen.GetRandomCollection(() => RandomValueGen.GetRandomInt())
• RandomValueGen.GetRandomSelectionFrom
• picks some random items from another collection
• ExpressionHelper: 
• GetMemberPathFor:
  use to get full property paths from lambdas, eg
  ExpressionHelper.GetMemberPathFor<Foo>(fooItem => fooItem.SomeProperty.SomeValue)
  gets you
  "SomeProperty.SomeValue"
  Useful for testing or somewhere where you need to get to properties dynamically
• GetPropertyTypeFor:
  Similar usage to GetMemberPathFor, but returns the property type
• Of course, there are some bugfixes
• And probably other stuff I forgot... It's been a while since I tooted on the PeanutButter horn.

 Like other PeanutButter bits and pieces, these have been born out of a desire for convenience, cleaner code and/or getting the compiler to check stuff before hitting the runtime. I hope they benefit others -- I know that I use a little PB every day (:

As always, usage, source availability and distribution remain totally free.

A little PeanutButter housekeeping

PeanutButter, that collection of libraries that does a whole bunch of stuff, has always been extended with the philosophy that it would be better to have a bunch of small libraries each with only their own dependencies rather than a monolithic installation of a bunch of crap you'd never want.

Well, with the exception of PeanutButter.TestUtils.Generic, which is where interesting test helper code went to lurk and generally be useful. This was where the first incarnation of TempDb was born. It was either Sqlite or SqlCe, I don't remember, but whichever it was, the other was soon to follow, in the same package. So if you wanted a temporary SqlCe database, you ended up getting all of the Sqlite dependencies smooshed into your project. Boo!

I recognised this as sub-optimal, but left it just so, partly because I'm a lazy bugger and partly because I was afraid of fallout for anyone who might upgrade packages and find stuff to be no longer working as it had been moved out into another package.

Not too long ago, I got the itch to add LocalDb support and, of course, it ended up in the same assembly. I was getting less stoked, but remained at the same level of lazy.

Finally, tonight, I got my ass in gear and split these out. So there are 4 new packages:

• PeanutButter.TempDb
• PeanutButter.TempDb.Sqlite
• PeanutButter.TempDb.SqlCe
• PeanutButter.TempDb.LocalDb

The first contains the generic TempDb<> logic and the remainder are concrete implementations. PeanutButter.TestUtils.Generic no longer provides these classes (and no longer spams your projects with the unused dependencies). Apologies if this causes some "WAT?!" for anyone, but the split was inevitable. As a bonus side-effect, PeanutButter.TestUtils now depends on NUnit, so installing the former will bring in the latter -- which is actually less WAT because of funky things like PropertyAssert.

Anyway, as always, they're totally free. So if they break, you can keep all the pieces.

PeanutButter TempDb gets LocalDb support

PeanutButter.TestUtils.Generic contains a lot of stuff, including TempDb<>, a generic abstract class which can be used to create temporary database instances and two derivatives, TempDbSqlite and TempDbSqlCe which, as the names might suggest, provide temporary databases of the Sqlite and SqlCe flavours.

Tonight, I added TempDbLocalDb which uses SQL Server's LocalDB mode (as is the default mechanism for MVC web applications. Whilst this does seem to take a little longer to spin up, it means that your tests can now have full SQLServer support with a temporary database which is automatically cleaned up as your tests end.

TempDb<> implements IDisposable, so the natural flow would be something like:

using (var db = new TempDbLocalDb())
{
    var connection = db.CreateConnection();
    connection.Open(); // if you need it open first
    // do stuff with the connection
    // for example, create an Entity context using the DbConnection
}

Thanks to Andrew Russell for the idea and pointing me at some resources (http://blog.developers.ba/localdb-for-database-integration-testing-in-asp-net-5-project-and-xunit-net/) which got me going.

You can install via nuget with:

install-package peanutbutter.testutils.generic

Hope this is of use to someone. Whilst it's true that providing a repository layer, for example, and mocking out data get/set operations is a lot faster (and probably better design), there are times when you just have to test against a database engine (for example, when testing that your database migrations work and that your entity models align with your migrated database structures... you are testing that, aren't you?)

Update (2015/06/03):  the TempDb packages have (finally) been split out and swept up so you don't have to install all of the providers you're not using. Check out the post here: http://davydm.blogspot.com/2015/06/a-little-peanutbutter-housekeeping.html

Shout-out to the kind folks at JetBrains

The other day I received a reminder mail that my R# license is due to expire in a few weeks. Now, I bought that when I was doing some work for a personal client and I really needed both VB.NET and C# functionality from my R#. So unfortunately, despite the kind offer from my company to use one of the roaming licenses they have for C#, that wasn't going to cut the mustard. I bit the bullet and got my own R# and reaped the productivity benefits, even though I'm quite sure I use only around 40% of the functionality offered by R#.
Anyways, I responded to the mail with a query regarding PeanutButter and the JetBrains R# OpenSource licensing program and... have been awarded an OpenSource (ie, no-charge) license for this year for R#, Full Edition. Thanks JetBrains! If you use C# and/or VB.NET (and, now, I hear C++ is in EAP) for development, in Visual Studio, Resharper can probably boost your productivity with navigation, code cleanup and refactoring functionality. Head on over to http://www.jetbrains.com/resharper/ to check it out if you don't use it already. The pricing may seem like quite a bit (and I'll freely admit that it's not cheap), but you can make that back quite quickly with the time it saves you, especially if you program TDD-style and take pride in your code, refactoring it to make it expressive and therefore more easily maintained.

A little PeanutButter for your MVC

ASP.NET MVC is one of the best things that I could possibly think of to have happened to the web from a Windows-centric point of view. At last, there's a way to sweep that abomination that is WebForms under the proverbial carpet.

Like anything, though, it does have its caveats. You have useful constructs like Script and Style Bundles -- but no easy way to test them from a CI environment. Also script inclusion becomes a bit more manual than it needs to be when viewed though the lens of AMDs like require.js. But you do get the advantage of bundling in that a single request can satisfy multiple code/style requirements. (Let me be clear here: AMDs are good. I like require.js. But it does take a little more effort to set up and get working correctly and you don't (without even more configuration) get the hit-reduction that bundles provide. Both methods have their advantages. Select your tools for your tasks as they fit best for you, on the day.)

PeanutButter.MVC was built out of a need to make those processes slightly more testable and elegant.

First of all, there are two facade classes:

• ScriptBundleFacade
• StyleBundleFacade

They wrap ScriptBundle and StyleBundle accordingly and implement interfaces of the expected names (IScriptBundle and IStyleBundle). You would use them like you'd use ScriptBundle and StyleBundle instances from the MVC framework. However, since they implement an interface, you can also create substitutes for them so that you can test-constrain your bundle registration process. This is important because I found that it was not uncommon to add a new javascript or css file to the solution, build-and-run, and be surprised that my changes weren't in play -- until I realised that I hadn't bundled them.

For example, I have the following method on my BundleConfig:

public static void RegisterBundles(BundleCollection bundles, 
        Func<string, IScriptBundle> withScriptBundleCreator = null,
        Func<string, IStyleBundle> withStyleBundleCreator = null)
{
    withScriptBundleCreator = withScriptBundleCreator ?? ((bundleName) => new ScriptBundleFacade(bundleName));
    withStyleBundleCreator = withStyleBundleCreator ?? ((bundleName) => new StyleBundleFacade(bundleName));

    AddJQueryBundlesTo(bundles, withScriptBundleCreator);
    AddStyleBundlesTo(bundles, withStyleBundleCreator);
    AddHandlebarsBundleTo(bundles, withScriptBundleCreator);

    AddApplicationScriptBundlesTo(bundles, withScriptBundleCreator);
}

We can see that the function would ordinarily be invoked without lambda factories to produce Script- and StyleBundleFacades, so it produces its own, very straight-forward ones. However, the tests that constrain this method can inject lambda factories so that the bundling methods can be tested to ensure that they include the required bundles from the relevant sources. Indeed, the tests are quite straight-forward:

    [TestFixture]
    public class TestBundleConfig
    {
        private Func<string, IScriptBundle< CreateSubstituteScriptBundleCreator(List>IScriptBundle< withTrackingList)
        {
            return (name) =>
                {
                    var scriptBundle = Substitute.For<IScriptBundle>();
                    scriptBundle.Name.Returns(name);
                    var includedPaths = new List<string>();
                    var includedDirs = new List<IncludeDirectory>();
                    scriptBundle.IncludedPaths.ReturnsForAnyArgs(args =>
                        {
                            return includedPaths.ToArray();
                        });
                    scriptBundle.IncludedDirectories.ReturnsForAnyArgs(args =>
                        {
                            return includedDirs.ToArray();
                        });
                    scriptBundle.Include(Arg.Any<string>()).ReturnsForAnyArgs(args =>
                        {
                            includedPaths.AddRange(args[0] as string[]);
                            return new Bundle("~/");
                        });
                    scriptBundle.IncludeDirectory(Arg.Any<string>(), Arg.Any<string>())
                        .ReturnsForAnyArgs(args =>
                        {
                            includedDirs.Add(new IncludeDirectory(args[0] as string, args[1] as string));
                            return new Bundle("~/");
                        });
                    scriptBundle.IncludeDirectory(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<bool>())
                        .ReturnsForAnyArgs(args =>
                        {
                            includedDirs.Add(new IncludeDirectory(args[0] as string, args[1] as string, (bool)args[2]));
                            return new Bundle("~/");
                        });
                    withTrackingList.Add(scriptBundle);
                    return scriptBundle;
                };
        }

        private Func<string, IStyleBundle> CreateSubstituteStyleBundleCreator(List<IStyleBundle> withTrackingList)
        {
            return (name) =>
            {
                var styleBundle = Substitute.For<IStyleBundle>();
                var includedPaths = new List<string>();
                styleBundle.IncludedPaths.ReturnsForAnyArgs(args =>
                    {
                        return includedPaths.ToArray();
                    });
                styleBundle.Include(Arg.Any<string>()).ReturnsForAnyArgs(args =>
                    {
                        var paths = args[0] as string[];
                        includedPaths.AddRange(paths);
                        return new Bundle("~/");
                    });
                withTrackingList.Add(styleBundle);
                return styleBundle;
            };
        }

        [Test]
        public void RegisterBundles_GivenBundleCollection_RegistersSharedScripts()
        {
            //---------------Set up test pack-------------------
            var collection = new BundleCollection();
            var scriptBundles = new List<IScriptBundle>();
            var styleBundles = new List<IStyleBundle>();
            //---------------Assert Precondition----------------

            //---------------Execute Test ----------------------
            BundleConfig.RegisterBundles(collection, 
                CreateSubstituteScriptBundleCreator(scriptBundles), 
                CreateSubstituteStyleBundleCreator(styleBundles));

            //---------------Test Result -----------------------
            Assert.AreNotEqual(0, scriptBundles.Count);
            Assert.AreNotEqual(0, styleBundles.Count);

            Assert.IsTrue(scriptBundles.Any(sb => sb.Name == "~/bundles/js/shared" &&
                                                    sb.IncludedDirectories.Any(d => d.Path == "~/Scripts/js/shared" && 
                                                                                    d.SearchPattern == "*.js" && 
                                                                                    d.SearchSubdirectories == true)));
        }
    }

All good and well. We can ensure that our MVC application is creating all of the required bundles. It would also be super-neat if we could streamline the inclusion process. Of course, we can.

PeanutButter.MVC also includes a utility called AutoInclude. If we decide to set up our bundles under /bundles/js/{controller} (for scripts for any action on the controller) and /bundles/js/{action}, then a lot of inclusion work can be done for us in our base _Layout view with a single line (assuming you've included the relevant @using clause at the top):

@AutoInclude.AutoIncludeScriptsFor(ViewContext)

AutoInclude uses the convention of scripts sitting under folders with names corresponding to the controller, with the casing of the scripts folders lowered to be more consistent with how script folders are named. This one line has, in conjunction with judicial bundling (and testing of that bundling!) allowed all views to just "magically" get their relevant scripts. In my project, I can create script bundles which include similarly-named folders and not have to worry about how my views get relevant logic scripts from there on out.

So, for example, I might perform registrations like the following (where scriptBundleCreator is a passed in Func<iscriptbundle>):

bundles.Add(scriptBundleCreator("~/bundles/js/policy")
    .IncludeDirectory("~/Scripts/js/policy", "*.js", false));
bundles.Add(scriptBundleCreator("~/bundles/js/policy/accept")
    .IncludeDirectory("~/Scripts/js/policy/accept", "*.js", false));
bundles.Add(scriptBundleCreator("~/bundles/js/policy/edit")
    .IncludeDirectory("~/Scripts/js/policy/edit", "*.js", false));

Now, from the Policy controller, I have two actions, Accept and Edit. Both have their relevant views, of course, and the AutoInclude is done automatically for them by virtue of the fact that they use the default _Layout.cshtml. Under my Scripts folder in my project, I have a file structure layout like:

policy
policy/common.js
policy/accept
policy/accept/accept.js
policy/accept/lead-autocompletion.js
policy/accept/proposalEmailer.js
policy/edit
policy/edit/clientDetailsDisplayUpdater.js
policy/edit/edit.js
policy/livePolicyUpdater.js

And the result is that the Policy/Accept view gets common.js, accept.js, lead-autocompletion.js and proposalEmailer.js. The Policy/Edit view gets common.js, clientDetailsDisplayUpdater.js, edit.js and livePolicyUpdater.js.

So now I'm free to create small, easily-testable javascript files (which I'll test with Jasmine and whatever works best for my purposes (eg karma or the Resharper unit test runner -- which works, mostly, with Jasmine, but has a few rough edges)). And when I want them in a page, I just drop them in the appropriate folder to get them on the next compile/debug run. And because of bundling, the end-user doesn't have to get many little hits for javascript files, instead, just getting two per view.

Apart from the testability of it and the simplicity of adding another piece of javascript functionality to the site, there's a huge bonus in grokkability. Let's face it: one of the reasosn why tests are good on your code is for when a new developer comes onto the project (or some unlucky person is tasked with maintaining some code they had nothing to do with). Tests provide feedback for when something breaks but also provide a communication mechanism for the new developer to figure out how discreet parts of the overall machine work. To the same end, understandable symbol and file naming and unsurprising project layout can really help with a new developer (or when you just have to get back on to the project for maintenance or extension and it's a couple of months down the line...)

Anyway, so there it is: PeanutButter.MVC. Free, small, doesn't depend on much, and hopefully useful. I'm certainly reaching for it the next time I'm in MVC land.