03 Feb 2015

Clipboard01I am having a lot of fun with a new project, one where the customer said we had to do unit tests (how rare is that Smile). It is still VERY early days in the project so by the end maybe I will realise this is wrong but for now, but right I am pretty proud of one specific test. It is 50 lines of long (C# + XUnit) and 18 of those lines are asserts. If the normal practise of unit tests produces asteroids, then this is a planet sized unit test. When I wrote it, I immediately felt the weight of hundreds of purists telling me it is wrong or that it has too much complexity & will hurt me.

Is this the norm for the project?

No – it totally will not be. Most unit tests will be sub 10 lines and have one or two asserts. I believe planet sized unit tests will account for 5% of the tests at the end of the day.

You’re doing unit tests wrong!

No – there is some advice I am choosing to ignore, but really it is just advice. Why is that advice there in the first place? It is there because people with far more experience and knowledge have learnt that the larger a unit test becomes (for this just assume size = complexity), the more brittle it becomes and the more time will be spent trying to understand and maintain it. In short the value vs. risk equation breaks down (i.e. high value & low risk is awesome but low value & high risk is a bad idea). So here is the rub, for 5% of the tests we have a very high risk but they exercise the code in ways that provide a massive amount of value.

So what does it do?

I think calling this a unit test is wrong, I feel that even though it uses unit test tools & is part of the unit test project it really isn’t a unit test – it is more of a process test. We are testing how the user would run through a specific process from end to end. These process tests exercise the interactions between various pieces which a single unit test might not. I had thought to call these integration tests, but we aren’t integrating to any external systems – everything is still mocked out and injected in. Another way to look at this, and in fact this is how we structured it, it is multiple unit tests (each with a arrange, act, assert pattern in them) chained together.

I have used these in previous projects and the value of them is stupidly massive. The often find issues long before a unit test will because so much is coupled together, you could say that the increase in complexity actually increases their value.

Why so many asserts?

How I figure out who to assert is simple, I follow the advice: Assert all your assumptions. For unit tests since they are simple, the assumptions can be simple. For a process test, since it is complex you should have more assumptions during each part of it. 

This is what I am doing, have you ever tried a planet size test or a process test? What did you find in it?

Tags: 

Add new comment