Testing without ExpectedExceptionAttribute

The Windows Store version of the Microsoft Test Framework does not include an ExpectedExceptionAttribute class. I’ve already received questions on how people should go about creating tests where the code under test should generate exceptions.

First, a note on the seriousness (or lack of) for this change:  The existing .NET libraries for testing do not display this problem.  Your existing unit test libraries are fine.

But moving forward, how are you supposed to test failures?

Instead of adding the ExpectedExceptionAttribute, you should use the new method:

public static void ThrowsException<T>(Action code) where T : Exception

There are reasons for this change.

ExpectedException is a course-grained check that can cause tests to pass that really should fail. Consider this test:

[TestMethod,
ExpectedException(ArgumentException)]
public void ExceptionPath()
{
setupTestEnvironment();
var underTest = new Worker();
setupTestObject(underTest);
var systemParms = GenerateCollaborator();
underTest.DoSomeWork(systemParms);
}

The ExcpectedException attribute means that this test passes if *any* code in this test method throws the expected exception. It may not where you expect. It may be that some of your setup fails, and the particulars of the failure mean that the expected exception is what’s thrown. That could mask problems in your production code.  If the setup method setupTestObject() throws an ArgumentException, this test passes, even though it shouldn’t.

Because of the nature of this issue, it can crop up anytime. You could be updating test code and you could accidentally introduce this error. It would masked because the test passed when you started, and the test passed when you finished.

Assert.ThrowsException fixes this issue.  Here’s the new version of the same test:

[TestMethod]
public void ExceptionPath()
{
setupTestEnvironment();
var underTest = new Worker();
setupTestObject(underTest);
var systemParms = GenerateCollaborator();
underTest.DoSomeWork(systemParms);
Assert.ThrowsException<ArgumentException>(() => underTest.DoSomeWork(default(string)));
}

This different version of the test passes if and only if the DoSomeWork() method throws the ArgumentException. If the ExceptionPath() test method throws an exception anywhere else, this test fails. As it should.

If you update tests as you add features, or change defined behavior, I recommend changing your failure path tests to make use of this technique. I wouldn’t retrofit every test you have, though. That feels like a whole lot of busy work, with only small benefit.

Next, we’ll look at how to write tests for async methods that throw exceptions. There are many nuances there that can mask issues in your tests, and in your production code.