Monday, 9 January 2017

EF-based testing, with PeanutButter: Shared databases

The PeanutButter.TestUtils.Entity Nuget package provides a few utilities for testing EntityFramework-based code, backed by TempDb instances so you can test that your EF code works as in production instead of relying on (in my experience) flaky substitutions.
One is the EntityPersistenceTester, which provides a fluent syntax around proving that your data can flow into and out of a database. I'm not about to discuss that in-depth here, but it does allow (neat, imo) code like the following to test POCO persistence:

// snarfed from EmailSpooler tests
[Test]
public void EmailAttachment_ShouldBeAbleToPersistAndRecall()
{
    EntityPersistenceTester.CreateFor<EmailAttachment>()
        .WithContext<EmailContext>()
        .WithDbMigrator(MigratorFactory)
        .WithSharedDatabase(_sharedTempDb)
        .WithAllowedDateTimePropertyDelta(_oneSecond)
        .ShouldPersistAndRecall();
}

which prove that an EmailAttachment POCO can be put into, and successfully retrieved from a database, allowing DateTime properties to drift by a second. All very interesting, and Soon To Be Documented™, but not the focus of this entry.

I'd like to introduce a new feature, but to do so, I have to introduce where it can be used. A base class TestFixtureWithTempDb<T> exists within PeanutButter.TestUtils.Entity. It provides some of the scaffolding required to do more complex testing than just "Can I put a POCO in there?". The generic argument is some implementation of DbContext and it's most useful when testing a repository as it provides a protected GetContext() method which provides a spun-up context of type T, with an underlying temporary database. By default, this is a new, clean database every test, but you can invoke the protected DisableDatabaseRegeneration() method in your test fixture's [OneTimeSetup]-decorated method (or constructor, if you prefer) to make this database live for the lifetime of your test fixture. The base class takes care of disposing of the temporary database when appropriate so you can focus on the interesting stuff: getting your tests (and then code) to work. A full (but simple) example of usage can be found here: https://github.com/fluffynuts/PeanutButter/blob/master/source/Utils/PeanutButter.FluentMigrator.Tests/TestDbMigrationsRunner.cs

My focus today is on a feature which can help to eliminate a pain-point I (and others) have experienced with EF testing backed onto a TempDb instance: time to run tests. EF takes a second or two to generate internal information about a database the first time some kind of activity (read/write) to that database is done via an EF DbContext. Not too bad on application startup, but quite annoying if it happens at every test. DisableDatabaseRegeneration() helps, but still means that each test fixture has a spin-up delay, meaning that when there are a few test fixtures, other developers on your team become less likely to run the entire test suite -- which is bad for everyone.

However, after some nudging in the right direction by co-worker Mark Whitfeld, I'd like to announce the availability of the UseSharedTempDb attribute in PeanutButter.TestUtils.Entity as of version 1.2.120, released today.
To use, decorate your test fixture:

[TestFixture]
[UseSharedTempDb("SomeSharedTempDbIdentifier")]
public class TestCrossFixtureTempDbLifetimeWithInheritence_Part1
    : TestFixtureWithTempDb
{
    // .. actual tests go here ..
}


And run your tests. And see an exception:

PeanutButter.TestUtils.Entity.SharedTempDbFeatureRequiresAssemblyAttributeException : 
The UseSharedTempDb class attribute on TestSomeStuff 
requires that assembly SomeProject.Tests have the attribute AllowSharedTempDbInstances.

Try adding the following to the top of a class file:
[assembly: PeanutButter.TestUtils.Entity.Attributes.AllowSharedTempDbInstances]


So, follow the instructions and add the assembly attribute to the top of your test fixture source file and re-run your tests. Congratulations, you're using a shared instance of a TempDb which will be cleaned up when NUnit is finished, providing, of course, that you don't interrupt the test run yourself (:

Wednesday, 4 January 2017

Gentoo adventures: Diablo 1 HD!

I recently wrote about switching to Gentoo. It's been an overall quite positive experience. The following is a story about getting Diablo 1 HD running through WINE, because:
  • Diablo 1 is one of the most iconic games ever
  • Diablo 1 (vanilla) has always had some issues running through WINE (most notably, the main menu -- though I discovered today that if you alt-tab (in Plasma shell, at least), then the window preview actually does show the menu)
  • I stumbled across someone's post on Reddit stating that it could be done (though it turned out that his advice didn't work for me, but I did eventually get there)

Just in case you didn't follow the link, this is what I'm talking about:

Before we begin: WINE doesn't deal well with an app which changes resolution and then dies unexpectedly. I found this out the hard way. I'd highly recommend installing (if you don't have it already) xrandr and binding a hotkey to set your native resolution. If you're using xbindkeys (yes, I like old-school... it works, reliably and I have an existing config with tweaks to allow volume changing via the buttons on the side of my mouse), you can add this to your .xbindkeysrc:


"xrandr -s 1920x1080"
    Control + Alt + Shift + F12



to be able to swap back to 1920x1080 by pressting ctrl-alt-shift-f12, in case, somewhere along the way, you end up with a strange-looking desktop.

On with it then!

First of all, you're going to need an (ahem) original Diablo 1 CD. Just so happens that I do have one of those, living on a shelf of other games, many of which I own on Steam because optical media is so last century. I'm not advocating other methods for doing this. When I got this Diablo 1 CD, it cost around R50 (~$13). Not really gonna break the bank. If you really can't get one, you could try alternative methods, if you know what I mean.

Naturally, my instructions are going to be in Gentoo dialect, but I expect that there won't be a lot different for others, except the USE flag fiddling I did -- which you won't have to do in a binary distribution and also which may not be entirely necessary, though not hurtful. Some were attempts to get around sound issues and I'll point those out.

Ok, so you have your original Diablo 1 CD, right? First we need to install that, which means we need wine. I installed with the following package.use:

app-emulation/wine dos openal s3tc staging

Note that I added dos simply in case I wanted dosbox later, s3tc because equery u wine claims it may be needed for some games, staging because I was curious what I would be missing out on from pre-release features and openal because of that aforementioned Redditer. So I don't think you actually need any of this.

Once you have WINE installed, run the autorun.exe from your original Diablo 1 CD. Setup should be quick and launch into the game -- cinematics first and then a black screen as the menu isn't properly displayed. You can alt-tab out of this and just kill it.

Now download the latest Diablo 1 HD release and unpack it where you installed Diablo 1. If you stuck with all defaults, you'll find it under ~/.wine/drive_c/Diablo. This adds a Belzebub.exe executable, which you can attempt to WINE -- but you'll get an error about OpenAL. Belzebub is shipped with an OpenAL32 dll -- the problem is that it depends on DirectSound. The false trail to follow is to use the inbuilt openal in WINE -- for me, that stopped the error, but also left me with no in-game sound (though the cinematics did have sound).

What worked better for me was to run winetricks, which I installed with the following package.use:

app-emulation/winetricks rar
 
simply because I didn't want some arbitrary download to fail simply because it came down as a rar archive. I'm pretty sure you don't need it for this process though -- winetricks was downloading the directx9 installer and pulling out dsound.dll (source at time of writing). Once again, I don't think you need my USE flags at all -- but they don't hurt. If your distro doesn't bindle Winetricks, you can always get it here: https://github.com/Winetricks/winetricks.
Run winetricks and go through:
  1. Select default wineprefix
  2. Install a Windows DLL or component
  3. Select dsound and hit the Ok button
Downloading may take a little while, depending on your pipe. Installation was quite quick. Running winecfg afterwards, looking at the Libraries tab, you should see *dsound set native. And now you should be able to run

wine Belzebub.exe

in the ~/.wine/drive_c/Diablo folder, with the original CD in your optical drive and the game should start up and work properly. You can bask in the glory of a 20-year-old game, running at a modern resolution on an OS it wasn't designed for (:

For bonus points, you can pack away that CD if you're prepared to keep an ISO on disk:
  1. mkdir ~/DIABLO1_CD
  2. dd if=/dev/sr0 of=~/diablo1.iso (substitute your optical drive for /dev/sr0 if it differs)
  3. add the following line to your /etc/fstab
    /home/{USER}/diablo1.iso /home/{USER}/DIABLO1_CD/ auto loop,users 0 0
  4. mount ~/DIABLO1_CD

You could even put that into a script in your Diablo folder:  

#!/bin/sh 
cd $(dirname $0) 
mount ~/DIABLO1_ISO 
wine Belzebub.exe
cd - 

You will probably see some odd flickering as the app starts up. Just hang in there! Mount won't barf on multiple runs -- it will just refuse to remount. Happy hunting!

Monday, 2 January 2017

Like changing religion, only harder, part 2

Gentoo is not the distribution for you if:

  • You hate reading
  • You don't want to learn new stuff
  • You're not willing to do a lot of it manually
The Handbook is an excellent resource, rich in detail. I'd actually recommend that anyone wanting to learn more about how Linux-based stuff works give an installation a go, even if just in a safe VirtualBox VM.

That's where I starrted -- a VM. I need my desktop to stay in a fully working state, so I can't afford to have it partially working or offline for days on end. And even a preliminary look at the handbook was enough to inform me that this was no trivial task.

Now, I've been a little coddled, possibly, by how good even the early Debian (curses-based) installers were (and still are). Debian subscribes to the "chicken-peck philosophy": basically, once the installer is going, you're going to be asked a lot of questions, but the default answer, which a chicken would get by just pecking on the enter key, is sane and will work for most people, Debian also attempts to install with a sane set of default packages which would be useful (or at least not get in the way) for most people.

Gentoo, on the other hand follows the "your machine, your rules" philosophy -- you're basically not guided with much at the terminal. You are going to need that handbook, unless you've done this a few times before.

After a few hours each night for a few nights, I felt like I had a handle on this and discovered that I had a hard drive I wasn't using. My only niggle was that I hadn't managed to get sound to work in the VM -- whenever I tried to play any sound, the player would just lock up. Still, I thought it was worth giving a go, whilst leaving my default boot sequence intact (and manually selecting the spare drive to boot from, when I needed to). I also found that I could do a lot just in a chroot whilst being booted into my existing Debian installation, so that helped a lot.

I'm still learning, but so far, I've gotten to the point where my default boot is the Gentoo installation, launching into a Plasma5 desktop. It's appreciably more responsive than I ever had Plasma5 before (which is why I had been quite happy to downgrade to KDE4 when I last installed Debian over Ubuntu) -- indeed it's more responsive than my Debian install in pretty-much every way. Some points that were interesting in the process:

  • The dmix ALSA plugin no longer requires magick to make it work (as it did when I first encountered it, ages ago). DMIX was, for me, this unattainable holy grail -- I never managed to muster the .asoundrc-foo to get it working. Which is why I had resigned myself to having to be bound to PulseAudio for all these years. But the good news is that I didn't have to configure a thing -- ALSA worked with software mixing out of the box, with no coaxing. WIN!
  • Installing a service doesn't mean it will be automatically started at system boot. This was a little surprising to me, but it fits in with the Gentoo philosophy -- you need to consciously decide that you want a service running at startup. It doesn't just happen. Fortunately, OpenRC is very easy to deal with, not like SystemD (at least, that's my experience)
  • Debian users will know that you pretty-much just need apt-get (install stuff and update lists of stuff you can install) and apt-cache (search for stuff to install). Gentoo has emerge for the searching and installing, but you're really going to want equery (from gentoolkit) and probably genlop.
  • Coming from a binary-based distro, USE flags are both interesting and scary. So now I have software compiled on my system with exactly the feature-set I asked for -- unlike binary packages, where I get whatever an upstream developer thought was a good feature-set. Not that that upstream developer was wrong or needed second-guessing: I never had a complaint with that. It's just an interesting paradigm shift and that Gentoo philosophy of consciously selecting stuff comes through again: I've often found that packages have practically the bare minimum of features aligned (Handbrake / ffpmeg is a good example) and I really wish I'd found equery uses a lot earlier. That handy little command not only tells you what use flags are available (which you can figure out with emerge -pv), but (mostly) what they actually mean.
  • Compiling. Eish. Be prepared to let your pc grind for an hour or so. Sometimes you can avoid this: there are binary packages for some of the more arduous bits (like LibreOffice and PaleMoon, though I chose to install those from source for funsies). Sometimes you can't (when I found that something I'd installed required that 32-bit versions of 177 other libraries be built). It's an elegant system though.
So I guess it's "So long, Debian, and thanks for all the fish". Perhaps Devuan will get a little more impetus and woo me back again. Who knows? But for now, I'm an habitually-tweaking Gentoo user.

Like changing religion, only harder, part 1.

I've used Debian or some derivative (UbuntuLinux Mint and even a brief stint with Corel Linux) as my primary desktop OS at home for around 16 years. Ok, I'll admit there was a brief period when I used Mandrake, before it was Mandriva and long before OpenMandriva. I also used OpenSUSE for a good few months, until RPM reminded me just how much I hate it.

Sure, I reboot to "that other OS" to game and sometimes dev (though that's mainly on a notebook now) and to observe windows updates. Gotta have a hobby. But my home pc spends upwards of 90% of its life in Linux land -- even more recently because I've been too lazy to reboot to game :/

I liked most of Debian, only choosing a derivative when I wanted more current software. I used Ubuntu long before it was the OS of the kool kids, all the way from Warty. It was all I liked about Debian with more current software but not the mad downloads required to keep up with Debian testing. I gave up on Ubuntu partly because of Unity / GNOME3 -- I had swiched to KDE, but it was never particularly fast. But something had already begun to bug me: PulseAudio.

Sound in Linux has traditionally been hard. I started before ALSA, when OSS was the way and there were a few sound daemons you could choose from, but honestly the best approach was to just get a sound card with hardware mixing -- I found a Creative AWE32 and things were good. But this PulseAudio beast was foisted on me and I never quite figured out how to shake it. I dislike it because it introduces noticable latency (even to my non-studio ears) and was always a little flaky. Last time I reloaded (from Ubuntu back to Debian plain, about a year ago), it was after finally simply having had enough of PA dying randomly. Nothing I tried would fix it for long and my desktop was laggy too -- it was just time for a clean house.

Going back to Debian was like a breath of fresh air -- things were faster, but there's still no (easy) hiding from PulseAudio. And Debian has also chosen to go the SystemD route -- another piece of software which I gave a chance, but just grew to hate more and more. Where are my logs? I have to learn journalctl commands, because they're not just in nice little text files under /var/log any more. How do I start/stop/restart a service? I have to learn systemctl commands, because /etc/init.d/(some service) (start|stop|restart) is only available for "antiquated" packages which haven't "upgraded" to systemd yet.

And don't get me started on boot (and even worse: shutdown / reboot) times. I'm running an i7 3770k with 16 gig RAM and an NVidia 660ti. Yeah, none of that is cutting edge -- but none of it is slothful either. Windows 10 on the same machine was making my precious Linux desktop feel positively sluggish.

We can't have that!

So I did what any other pedantic geek would do: downloaded ISOs and started checking out other distros in VirtualBox VMs. Primarily, I was interested in finding something that didn't use systemd, but it also had to be actively maintained, have a decently-sized community, have a lot of packages, have a good history (or be built on the shoulders of a giant with good history, like Debian). I found a nice comparison chart here: https://en.wikipedia.org/wiki/Comparison_of_Linux_distributions#Technical and set to work on evaluating in virtual environments. Ideally, I'd like to have found something Debian-based as I know the layout and tooling fairly well, but I was open to new experiences.

Most of the options I tried out didn't wow me, but a quick glance at the list linked above shows that one of the systemd-independent distributions is Gentoo.

Rewind a decade or so, and I remember having a good friend who loved Gentoo. Personally, it always felt like the OS of choice for an habitual tweaker. It just didn't seem to be worth the effort. For a while, the argument that compiling all packages locally would give a faster desktop was the primary reason I heard that I should consider it. It was also the primary reason for angst back then -- when getting a Gentoo box up and running with a desktop environment, browser, email client, office suite, etc, would take around a week. Those were the days when an Athlon 1800 was the business -- and there's no way an Athlon 1800 can compete with even a low-power current chip for compiling. Gentoo was the distro for people with nothing better to do than watch compilation output scrolling across their screens.

But I noticed that

  • OpenRC is the default init system for Gentoo (not some unsupported "you can do it, but good luck" option, like with Arch Linux, another distro I did consider because I've read a lot of good things about it),
  • Gentoo had a clear howto on setting up ALSA for sound cards which don't support hardware mixing and 
  • Gentoo is documented to the max -- there are wiki page upon wiki page of information
The last point is good -- but it's also a little scary for the same reason that over-documented code is scary: at some point, the documenting comments fall out of sync with the actual software logic and become misleading lies (and I've found some rabbit-holes in Gentoo wiki pages -- but mostly, they've been really helpful)

So Gentoo looked like a good choice.... Onwards and upwards!

Tuesday, 6 September 2016

Legacy database testing with PeanutButter

Preamble

Recently, I've been asked about a strategy for testing code which works against an existing database. It's not the first time I've been asked about this -- and it probably won't be the last. I should have listened to Scott Hanselman's advice long ago and blogged it.

Better late than never, I suppose. Let's get to it!

The problem

You have some code which hits a MSSQL database to do something. It could be hitting a stored procedure for a report. It could be inserting or updating rows. Whatever the use-case, you'd like to have that code under test, if not just for your own sanity, then because you're about to extend that code and are just plain scared that you're about to break something which already exists. This is a valid reason -- and it drove me to the strategy I'll outline below.

You may also simply wish to write your code test-first, but have this great existing legacy mass which you have to work with (and around) and you're just struggling to get the first test out.

Please note: this strategy outlines more integration-style testing than true unit testing. However, I'd rather have an integration test than no test any day. This kind of testing also leads to tests which take a few seconds to run (instead of the preferred milliseconds) -- but I'd rather have slow tests than no tests.

Note that I'm tackling MSSQL because:
  1. It's a common production database
  2. If you were dealing with simpler databases like SQLite or SQLCE, you may already have a strategy to deal with this (though PB can still make it easier, so read on)
  3. I haven't found (yet) a nice way to do temporary in-process MySQL or PostgreSQL. You could use this strategy with Firebird since server and embedded can even use the same file (but not concurrently, of course) -- though currently PeanutButter.TempDb has no baked-in Firebird provider. I guess I should fix that!

So, let's really get to it!

The general idea

Ideally, I'd like to have some kind of test framework which would spin up a temporary database, create all the structures (tables, views) that I need, perhaps the programmability (procedures, functions) I'd like to test (if applicable) and also provides a mechanism for loading in data to test against so that I can write "when {condition} then {expectation}"-style tests.

I'd also like that temporary database to die by fire when my tests are done. I don't want to have to clean anything up manually.

Traditionally, running instances of database services have been used for this style of testing -- but that leaves you with a few sticky bits:
  1. Your development and CI environments have to be set up the same, with credentials baked into the test suite. Or perhaps you can use environment overrides -- still, authorization to the test database has to be a concern
  2. Test isolation is less than trivial and as the test suite grows, the tests start interacting with each other, even if there is cleanup at some point along the way.
  3. Getting a new developer on the team is more effort than it really should be, mainly because of (1) above. For the same reasons, developing on a "loaner" or laptop is also more effort than it should be. You can't just "check out and go".
Some prior strategies exist to cope with this, but they are unsatisfactory:
  1. A shared development/test database which accumulates cruft and potentially causes strange test behaviour when unexpected data is matched by systems under test
  2. Swapping out the production database layer for something like SQLite. Whilst I really like SQLite, the problem boils down to differences in functionality between SQLite and whatever production database you're using. I've come across far to many recently, in a project where tests are run against SQLite and production code runs against PostgreSQL. I've seen similar issues with testing code targeting SQL Server on SQLCE. Even if you have a fairly beefy ORM layer (EF, NHibernate, etc) to abstract a lot of the database layer away from you, you're going to hit issues. I can think of too many to put them out here -- if you really want a list of the issues I've hit in this kind of scenario, feel free to ask. I've learned enough to feel fear when someone suggests testing on a database other than the engine you're going to deploy on.
    Sometimes you have tests which work when production code fails. Sometimes your tests simply can't test what you want to do because the test database engine is "differently capable".
  3. For similar reasons to (2) above, even if you're testing down to the ORM layer, mocked / substituted database contexts (EF) can provide you with tests which work when your production code is going to fail.
So we'd like to test against "real iron", but we'd like that iron to be transient.

PeanutButter to the rescue (:

The strategy that emerged

  1. Create a temporary database (PeanutButter.TempDb.(some flavor))
    1. Fortunately, when production code is going to be talking to SQL Server, we can use a LocalDb database for testing -- all the functionality of SQL Server (well, pretty-much all of it, enough for application code -- you'll be missing full text search for example, but the engine is basically the same).
  2. Create the database structures required (via migrations or scripts)
  3. Run the tests on the parts of the system to be tested
  4. Dispose of the temporary database when done, leaving no artifacts and no cruft for another test or test fixture.


public void TestSomeLegacySystem
{
  [Test]
  public void TestSomeLegacyMethod()
  {
    // Arrange
    using (var db = new TempDbLocalDb())
    { 
      using (var conn = db.CreateConnection())
      {
        // run in database schema script(s)
        // insert any necessary data for the test 
      }
      // Act
      // write actual test action here
      // Assert 
      using (var conn = db.CreateConnection())
      {
        // perform assertions on the database with the new connection 
      } 
    } // our TempDb is destroyed here -- nothing to clean up!
  } 
}

This is very raw, ADO-style code. Of course, it's not too difficult to extrapolate to using an EF context since TempDb exposes both a connection string or you could pass a new connection from CreateConnection to your context's constructor, which would call into DbContext's constructor which can take a DbConnection -- and you would set the second (boolean) parameter based on whether or not you'd like to dispose of the connection yourself. 

I did this often enough that it became boring and my laziness kicked in. Ok, so that happened at about the third test...

And so TestFixtureWithTempDb was born: this forms a base class for a test fixture requiring a temporary database of the provided type. It has a protected Configure() method which must be used to instruct the base class how to create the database (with an initial migrator to get the database up to speed), as well as providing hints; for example, by default, a new TempDb is spun up for every test, but if you're willing to take care of cleaning out crufty data after each test (perhaps with a [Teardown]-decorated method), then you can share the database between tests in the fixture for a little performance boost. The boost is more noticable when you also have EF talking to the temporary database as EF will cache model information per database on first access -- so even if you have the same structures in two different iterations, EF will go through the same mapping steps for each test, adding a few seconds (I find typically about 2-5) per test.

Indeed, if you have an EF context, you perhaps want to step one up the chain to EntityPersistenceTestFixtureBase, which is inherited with a generic type that is your DbContext. Your implementation must have a constructor which takes just a DbConnection for this base class to function. If you've created your context from EDMX, you'll have to create another partial class with the same name and the expected constructor signature, passing off to an intermediatary static method which transforms a DbConnection into an EntityConnection; otherwise, just add a constructor.

And this is where my laziness kicks in again: the test fixture for EntityPersistenceTestFixtureBase provides a reasonable example for usage. Note the call to Configure() in the constructor -- you could also have this call in a [OneTimeSetup] method. If you forget it, PeanutButter will bleat at you -- but helpfully, instructing you to Configure() the test fixture before running tests (:

Some interesting points:
  1. Configure's first parameter is a boolean: when true, the configuration will create ASP.NET tables in the temporary database for you (since it's highly unlikely you'll have them in your own migrations). This is useful only if you're intending to test, for example, an MVC controller which will change behaviour based on the default ASP.NET authentication mechanisms. Mostly, you'll want this to be false.
  2. The second parameter is a factory function: it takes in a connection string and should emit something which implements IDBMigrationsRunner -- this is an instance of a class with a MigratoToLatest() method which performs whatever is necessary to build the required database structures. You could wrap a FluentMigrator instance, or you can use the handy DbSchemaImporter, given a dump of your database as scripts (without the use statement!) to run in your existing schema. When doing the latter, I simply import said script in a regular old .net resource -- when doing so, you'll get a property on that resource which is a string: the script to run (:
  3. You can configure a method to run before providing an new EF context -- when configured, this method will be given the EF context which is first created in a test so that it can, for example, clear out old data. Obviously, this only makes sense if you're going full-EF.
  4. If you have a hefty legacy database, expect some minor issues that you'll have to work through. I've found, for instance, procedures which compiled in SSMS, but not when running in the script for said procedure because it was missing a semi-colon. Don't despair: the effort will be worth it. You can also try only scripting out the bare minimum of the target database that is required for your tests.

Enough blathering!

Ok, this has been a post well-worth a TL;DR. It's the kind of thing would would probably work better as a 15-minute presentation, but I suppose some blog post is better than no post (:

Questions? Comments? They're all welcome. If there's something you'd like me to go more in-depth with, shout out -- I can always re-visit this topic (:

Legacy database testing with PeanutButter

Preamble

Recently, I've been asked about a strategy for testing code which works against an existing database. It's not the first time I've been asked about this -- and it probably won't be the last. I should have listened to Scott Hanselman's advice long ago and blogged it.

Better late than never, I suppose. Let's get to it!

The problem

You have some code which hits a MSSQL database to do something. It could be hitting a stored procedure for a report. It could be inserting or updating rows. Whatever the use-case, you'd like to have that code under test, if not just for your own sanity, then because you're about to extend that code and are just plain scared that you're about to break something which already exists. This is a valid reason -- and it drove me to the strategy I'll outline below.

You may also simply wish to write your code test-first, but have this great existing legacy mass which you have to work with (and around) and you're just struggling to get the first test out.

Please note: this strategy outlines more integration-style testing than true unit testing. However, I'd rather have an integration test than no test any day. This kind of testing also leads to tests which take a few seconds to run (instead of the preferred milliseconds) -- but I'd rather have slow tests than no tests.

Note that I'm tackling MSSQL because:
  1. It's a common production database
  2. If you were dealing with simpler databases like SQLite or SQLCE, you may already have a strategy to deal with this (though PB can still make it easier, so read on)
  3. I haven't found (yet) a nice way to do temporary in-process MySQL or PostgreSQL. You could use this strategy with Firebird since server and embedded can even use the same file (but not concurrently, of course) -- though currently PeanutButter.TempDb has no baked-in Firebird provider. I guess I should fix that!

So, let's really get to it!

The general idea

Ideally, I'd like to have some kind of test framework which would spin up a temporary database, create all the structures (tables, views) that I need, perhaps the programmability (procedures, functions) I'd like to test (if applicable) and also provides a mechanism for loading in data to test against so that I can write "when {condition} then {expectation}"-style tests.

I'd also like that temporary database to die by fire when my tests are done. I don't want to have to clean anything up manually.

Traditionally, running instances of database services have been used for this style of testing -- but that leaves you with a few sticky bits:
  1. Your development and CI environments have to be set up the same, with credentials baked into the test suite. Or perhaps you can use environment overrides -- still, authorization to the test database has to be a concern
  2. Test isolation is less than trivial and as the test suite grows, the tests start interacting with each other, even if there is cleanup at some point along the way.
  3. Getting a new developer on the team is more effort than it really should be, mainly because of (1) above. For the same reasons, developing on a "loaner" or laptop is also more effort than it should be. You can't just "check out and go".
Some prior strategies exist to cope with this, but they are unsatisfactory:
  1. A shared development/test database which accumulates cruft and potentially causes strange test behaviour when unexpected data is matched by systems under test
  2. Swapping out the production database layer for something like SQLite. Whilst I really like SQLite, the problem boils down to differences in functionality between SQLite and whatever production database you're using. I've come across far to many recently, in a project where tests are run against SQLite and production code runs against PostgreSQL. I've seen similar issues with testing code targeting SQL Server on SQLCE. Even if you have a fairly beefy ORM layer (EF, NHibernate, etc) to abstract a lot of the database layer away from you, you're going to hit issues. I can think of too many to put them out here -- if you really want a list of the issues I've hit in this kind of scenario, feel free to ask. I've learned enough to feel fear when someone suggests testing on a database other than the engine you're going to deploy on.
    Sometimes you have tests which work when production code fails. Sometimes your tests simply can't test what you want to do because the test database engine is "differently capable".
  3. For similar reasons to (2) above, even if you're testing down to the ORM layer, mocked / substituted database contexts (EF) can provide you with tests which work when your production code is going to fail.
So we'd like to test against "real iron", but we'd like that iron to be transient.

PeanutButter to the rescue (:

The strategy that emerged

  1. Create a temporary database (PeanutButter.TempDb.(some flavor))
    1. Fortunately, when production code is going to be talking to SQL Server, we can use a LocalDb database for testing -- all the functionality of SQL Server (well, pretty-much all of it, enough for application code -- you'll be missing full text search for example, but the engine is basically the same).
  2. Create the database structures required (via migrations or scripts)
  3. Run the tests on the parts of the system to be tested
  4. Dispose of the temporary database when done, leaving no artifacts and no cruft for another test or test fixture.


public void TestSomeLegacySystem
{
  [Test]
  public void TestSomeLegacyMethod()
  {
    // Arrange
    using (var db = new TempDbLocalDb())
    { 
      using (var conn = db.CreateConnection())
      {
        // run in database schema script(s)
        // insert any necessary data for the test 
      }
      // Act
      // write actual test action here
      // Assert 
      using (var conn = db.CreateConnection())
      {
        // perform assertions on the database with the new connection 
      } 
    } // our TempDb is destroyed here -- nothing to clean up!
  } 
}

This is very raw, ADO-style code. Of course, it's not too difficult to extrapolate to using an EF context since TempDb exposes both a connection string or you could pass a new connection from CreateConnection to your context's constructor, which would call into DbContext's constructor which can take a DbConnection -- and you would set the second (boolean) parameter based on whether or not you'd like to dispose of the connection yourself. 

I did this often enough that it became boring and my laziness kicked in. Ok, so that happened at about the third test...

And so TestFixtureWithTempDb was born: this forms a base class for a test fixture requiring a temporary database of the provided type. It has a protected Configure() method which must be used to instruct the base class how to create the database (with an initial migrator to get the database up to speed), as well as providing hints; for example, by default, a new TempDb is spun up for every test, but if you're willing to take care of cleaning out crufty data after each test (perhaps with a [Teardown]-decorated method), then you can share the database between tests in the fixture for a little performance boost. The boost is more noticable when you also have EF talking to the temporary database as EF will cache model information per database on first access -- so even if you have the same structures in two different iterations, EF will go through the same mapping steps for each test, adding a few seconds (I find typically about 2-5) per test.

Indeed, if you have an EF context, you perhaps want to step one up the chain to EntityPersistenceTestFixtureBase, which is inherited with a generic type that is your DbContext. Your implementation must have a constructor which takes just a DbConnection for this base class to function. If you've created your context from EDMX, you'll have to create another partial class with the same name and the expected constructor signature; otherwise, just add a constructor.

And this is where my laziness kicks in again: the test fixture for EntityPersistenceTestFixtureBase provides a reasonable example for usage. Note the call to Configure() in the constructor -- you could also have this call in a [OneTimeSetup] method. If you forget it, PeanutButter will bleat at you -- but helpfully, instructing you to Configure() the test fixture before running tests (:

Some interesting points:
  1. Configure's first parameter is a boolean: when true, the configuration will create ASP.NET tables in the temporary database for you (since it's highly unlikely you'll have them in your own migrations). This is useful only if you're intending to test, for example, an MVC controller which will change behaviour based on the default ASP.NET authentication mechanisms. Mostly, you'll want this to be false.
  2. The second parameter is a factory function: it takes in a connection string and should emit something which implements IDBMigrationsRunner -- this is an instance of a class with a MigratoToLatest() method which performs whatever is necessary to build the required database structures. You could wrap a FluentMigrator instance, or you can use the handy DbSchemaImporter, given a dump of your database as scripts (without the use statement!) to run in your existing schema. When doing the latter, I simply import said script in a regular old .net resource -- when doing so, you'll get a property on that resource which is a string: the script to run (:
  3. You can configure a method to run before providing an new EF context -- when configured, this method will be given the EF context which is first created in a test so that it can, for example, clear out old data. Obviously, this only makes sense if you're going full-EF.
  4. If you have a hefty legacy database, expect some minor issues that you'll have to work through. I've found, for instance, procedures which compiled in SSMS, but not when running in the script for said procedure because it was missing a semi-colon. Don't despair: the effort will be worth it. You can also try only scripting out the bare minimum of the target database that is required for your tests.

Enough blathering!

Ok, this has been a post well-worth a TL;DR. It's the kind of thing would would probably work better as a 15-minute presentation, but I suppose some blog post is better than no post (:

Questions? Comments? They're all welcome. If there's something you'd like me to go more in-depth with, shout out -- I can always re-visit this topic (:

Friday, 26 August 2016

The story of M&m

(or, as I like to think of it, "how I wasted over an hour trying to figure out what was breaking the code I was learning, only to discover a convoluted chain of fail which ends (imo) at some code doing something other than what its name suggests because someone had an opinionated idea that it should)


Brace yourself. The rant is strong with this post. There may be a useful nugget in there, but I can't guarantee anything.

One of the greatest cores of our culture at Chillisoft, is a commitment to learning. Learning new technologies, new techniques, new life skills. Part of that commitment is our "Deliberate Practice" time, a time set aside on Fridays from 11h00 to COB in which we put aside production work and focus on learning new things.

The topics vary -- and sometimes the aim is just to experiment with technologies and figure out what may be useful. The current topic is that of TypeScript and Angular 2 and the current focus point with those two topics is for everyone to persevere in producing their own "TODO" app with TypeScript and Angular 2, hopefully from scratch, but using any available online resources.

One of those resources is the Tour of Heroes tutorial on angular.io. That tutorial goes through a lot of the basics of Angular, but suggests starting with the 5 Minute Quickstart tutorial which uses SystemJS for building. Personally, I have a few gripes about the output from that method, mostly that you're expected to reference scripts within node_modules (so ship your entire node_modules folder with your app? Hand-pick the bits you need? Post-process with another tool?) but also because I didn't see a way to process in other resources, like templates -- which I most definitely want to keep out of the code, for the day when I'm fortunate enough to once again have an html-capable designer on my team.

I could well be missing something fundamental with SystemJS -- feel free to drop comments if you can set me straight!

Having been recently exposed to Webpack and both wanting to learn more about that technology whilst avoiding the things I didn't like about what I'd seen of SystemJS, I was excited to see that the kind Angular tutorial fairies had provided a Webpack variant of the 5 Minute Quickstart tutorial.

Naturally, I dove right in.

I'd like to stop here for a moment and talk about tutorials.

So often, I've come across tutorials and other learning material which follows this kind of instruction pattern:
  1. Type something into an IDE or terminal. No explanation, just type it!
  2. Copy this file somewhere
  3. Run this command
  4. Copy these seven other files somewhere else
  5. Run this other command
  6. Congratulations! You are a 1337 developer!
Sorry to burst your bubble, but you're not. You can follow instructions. Bully for you. This is why I stopped at the end of chapter 3 of my MCSD materials, well over a decade ago: I got to a point where I'd followed all of the instructions and had been declared a proficient C++ developer by the materials -- but I had to disagree. I couldn't even repeat what I had done, let alone solve a brand new problem.

Without understanding the simple principles, a lot seems like magick and we end up copying about revered configuration files from project to project, oblivious of the cruft they bring with them.

When you are going through a tutorial (like the ones linked above) and you're instructed to copy some files into your source folder the very least you can do is type them out yourself. At least then you're forced to properly read them and start asking questions. Better, you might start wondering where these files come from and how you're going to create them next time.
In all of these tutorials, I see no mention of the following commands -- but I see their artifacts as "copy-and-paste-into-your-project-folder" instructions:
  • npm init
  • tsc --init
  • typings init
  • karma init
(When tooling doesn't have an init function, tutorials should point to documentation references like the Webpack configuration documentation. And perhaps walk you through some of the basics.)

The result is that juniors learning these techs have no idea how their fundamentals work -- and are unable to stand on their own two feet. I see a similar philosophy coming out of colleges, where graduates can program in C# but have no idea how a program is started by the OS, what JIT is, how to use a REPL -- simple things that really should be the bedrock of understanding.

As an analogy, I'm not saying that a professional car drivers should understand every part of the combustion engine -- but at least knowing about the roles played by pistons, clutch, gears, fuel, braking systems, etc.

But I digress -- back to the story of m&M... or was that M&m...

So between the Tour of Heroes and the 5 Minute Quick Start (and the Webpack variant). I finally get something up and building. I'm feeling all proud of myself until I try to do my first two-way binding, using the Angular2 FormsModule. I read that I can use a template string or a templateUrl to a file of my choosing -- and I consider the latter a better idea because of separation of concerns (and the whole designer theme above). So I point my templateUrl at a file in the same folder as the component, fire up the webpack dev server, bust out a browser and get an error like so:

Can't bind to 'ngmodel' since it isn't a known property of 'input'

Note that my template has the following code only:


<main>
  <h1>TODO</h1>
<input type="text" [(ngModel)]="newTodoText" (keyup.enter)="addTodo()" />
</main>

Nothing particularly exciting, elegant or informative in there and it takes me a while to realise that I've specified ngModel but the error is complaining about ngmodel.

I suspect that Angular is being case-sensitive about these "attributes" and my camelCasing is being clubbed by something. The Angular devs have stirred the pot a little (apparently) with this decision, but I'm not to burned up about it -- it's a template, to be pre-processed: it doesn't have to be conformant html5

Now, since I'm loading a template by url, I'm using the Webpack html loader -- that's what the Angular tutorial suggests, and it makes sense to me. I get the brainwave to use a string instead of the templateUrl -- just stuff the above code into a template attribute and try again... Success! So my suspicion are confirmed:
  1. Angular does care about case
  2. Somehow my ngModel is getting through to the code as ngmodel when the template is loaded from a file and that's causing the issue.
Ok, try another loader: raw. Success! It works! So my second suspicion is confirmed: it's the html-loader.  But this is supposed to be the "correct" way to load html -- the internet says so, and the internet is never wrong!

I suspect that html-loader is lower-casing attributes for some arbitrary reason -- but no documentation at https://www.npmjs.com/package/html-loader gives me any clues -- except that I see that html-loader is minifying by default, using https://www.npmjs.com/package/html-minifier. Ok, so perhaps minification could be considered to be part of loading, especially for packing. Fair enough.

I discover that html-minifier has lots of options with one of them being caseSensitive. Better than that, this option is disabled by default.

And this is where I start losing the plot.

Remember, folks, this package is a minifier. It has one job: make what comes in smaller and push it back out again. Unless the developers are privy to some fantastic new knowledge about bytes, I'm pretty sure that lower-casing text does not save space. Technically, proper html should have lower-cased attributes -- but that's really an opinion which belongs in another plugin (perhaps a tidy plugin?), not something which has a job of minifying. Just my opionion, mind you.

Just as confusing is that html-loader is using this extra plugin, so the only point of configuration for the minifier is html-loader, which can be configured with a query-string like syntax (blech -- someone else has to grok out all the options that I put in there? horrid!), or via a property on the exported configuration from your webpack.conf.js. Except that that mechanism only works for a select few configuration options. And html-loader doesn't have comprehensive documentation, so I guess I'll have to clone their code and go trawling through it -- a practice which is becoming annoyingly common for me.

(Side-note: I know that I'm not the best documenter either -- PeanutButter could do with online documentation -- I tend to answer questions rather, and that's a bad habit )':  The only defense I have is that at least I don't mislead anyone with partial documentation. You knew you would have to check out the code the moment you stepped through the door.)

Anyway, after trudging through code a bit and finding this github issue of interest, I eventually figure out that the magick to fix my problems can appear in my webpack.config.js, like so:


module.exports = {
...
      {
        test: /\.html$/,
        loader: 'html?caseSensitive'
      },
...
}

That little ?caseSensitive and the resultant m/M debacle took me an hour to figure out >_<

And at the end of the day, I'm still of the opinion that having a minifier change the case of code it outputs is a silly idea. Haters gonna hate, but I really don't see the point. At the very least, it produces "WAT?!" moments like the one above. Worse than that, it promotes the idea of writing sloppy code and getting a tool to fix it (for real html attributes, which should jolly-well be lower-case when the developer writes them!)

What's new in PeanutButter?

Retrieving the post... Please hold. If the post doesn't load properly, you can check it out here: https://github.com/fluffynuts/blog/...