Blog

In the previous blog post, we’ve talked about indirect inputs and outputs when a unit test exercises a Subject Under Test. The examples shown previously demonstrate the use of test doubles.

A test double is a type of object that replaces a real instance of a collaborator, for the specific purpose of executing tests. Test doubles make it more easy for us to manipulate indirect inputs and verify indirect outputs. But this ease of use also comes with a certain cost. Depending on the type of test double being used, it also couples unit tests more tightly to their Subject Under Test, unless specific measures are applied to somewhat mitigate this coupling.

Test doubles are put in place using a technique called Dependency Injection. Instead of letting the Subject Under Test directly create an instance for each of its collaborators, these instances are provided from the outside. They are usually specified as arguments to the constructor of the Subject Under Test.

Let’s have a look at the different kinds of test doubles and their use.

Dummy

The first type of test double is called a Dummy. A Dummy object supports some kind of interface that is expected by the Subject Under Test, but it doesn’t provide any behaviour or return any values. Basically, it doesn’t do anything. It just exists. A Dummy is most often passed as a placeholder argument, but is never actually used by the Subject Under Test.

Let’s have a look at an example of a Dummy.

public class CreateExpenseSheetHandler : ICommandHandler<CreateExpenseSheet>
{
    private readonly IEmployeeRepository _employeeRepository;
    private readonly IExpenseSheetRepository _expenseSheetRepository;

    public CreateExpenseSheetHandler(
        IEmployeeRepository employeeRepository,
        IExpenseSheetRepository expenseSheetRepository)
    {
        _employeeRepository = employeeRepository;
        _expenseSheetRepository = expenseSheetRepository;
    }
    
    public Result Handle(CreateExpenseSheet command)
    {
        if(null == command)
            throw new ArgumentNullException(nameof(command));

        var employee = _employeeRepository.Get(command.EmployeeId);
        if(null == employee)
        {
            var message = $"The employee with identifier '{command.EmployeeId}' could not be found.";
            return Result.Failure(new DomainViolation(message));    
        }
        
        var expenseSheet = new ExpenseSheet(command.Id, employee, command.SubmissionDate);
        _expenseSheetRepository.Save(expenseSheet);
        
        return Result.Success();
    }
}

public class CreateExpenseSheet
{
    public Guid Id { get; }
    public Guid EmployeeId { get; }
    public DateTime SubmissionDate { get; }
    
    public CreateExpenseSheet(Guid id, Guid employeeId, DateTime submissionDate)
    {
        Id = id;
        EmployeeId = employeeId;
        SubmissionDate = submissionDate;
    }
}

Here we have the implementation of a command handler that creates a new expense sheet. The command object specifies the identifier for the new expense sheet, the identifier of the employee and a submission date. We use the Employee repository to retrieve an Employee object based on its unique identifier. We let the operation fail in case the required Employee object could not be found. Then the new ExpenseSheet object is created and saved into the repository. In that case, the operation is completed successfully. Note that a guard condition has also been added in order to enforce the caller of the Handle method to provide an instantiated CreateExpenseSheet command object.

public class DummyEmployeeRepository : IEmployeeRepository
{
    public Employee Get(Guid id)
    {
        throw new NotImplementedException(
            "The Get method of the EmployeeRepository shouldn't get called.");
    }
}

public class DummyExpenseSheetRepository : IExpenseSheetRepository
{
    public ExpenseSheet Get(Guid id)
    {
        throw new NotImplementedException(
            "The Get method of the ExpenseSheetRepository shouldn't get called.");
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        throw new NotImplementedException(
            "The Save method of the ExpenseSheetRepository shouldn't get called.");
    }
}

The DummyEmployeeRepository class provides a dummy implementation by implementing the IEmployeeRepository interface. The Get method throws a NotImplementedException in case it gets called. The same is true for the Get/Save methods of the DummyExpenseSheetRepository class.

[Specification]
public class When_creating_a_new_expense_sheet_using_a_non_existent_command
{
    [Establish]
    public void Context()
    {
        var dummyEmployeeRepository = new DummyEmployeeRepository();
        var dummyExpenseSheetRepository = new DummyExpenseSheetRepository();
        
        _sut = new CreateExpenseSheetHandler(dummyEmployeeRepository, dummyExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        _createExpenseSheet = () => _sut.Handle(null);
    }
    
    [Observation]
    public void Then_an_exception_should_be_thrown()
    {
        Assert.That(_createExpenseSheet, Throws.ArgumentNullException);
    }
    
    private CreateExpenseSheetHandler _sut;
    private TestDelegate _createExpenseSheet;
}

This particular test scenario verifies whether an ArgumentNullException is thrown by the Subject Under Test when a null reference is passed to the Handle method. We pass an instance of both the DummyEmployeeRepository and the DummyExpenseSheetRepository classes to the constructor of the CreateExpenseSheetHandler.

We could also get away with just passing two null references instead of these dummy objects. But then we just get a NullReferenceException when the test fails. Using an actual dummy implementation results in slightly more meaningful diagnostic information due to the custom exception messages in each method. Implementing and using a dummy object this way, we can ensure that a collaborator is never used in a particular scenario.

Stub

The second type of test double is called a Stub. A Stub object is the same as a Dummy. It doesn’t do anything, except providing the Subject Under Test an indirect input that can be controlled by a test. So, the only thing a Stub does more than a Dummy is returning values for query methods. A test can instruct a Stub to return either a valid or an invalid indirect input depending on the test scenario. But it can also instruct a Stub to raise a particular exception as well.

Let’s have a look at an example of a Stub.

public class StubEmployeeRepository : IEmployeeRepository
{
    private readonly Employee _result;

    public StubEmployeeRepository(Employee result)
    {
        _result = result;
    }
    
    public Employee Get(Guid id)
    {
        return _result;
    }
}

The StubEmployeeRepository class implements the IEmployeeRepository interface. Its constructor expects an instance of an Employee that can be returned by the Get method when and if called.

[Specification]
public class When_creating_a_new_expense_sheet_for_an_unknown_employee 
{
    [Establish]
    public void Context()
    {
        Employee unknownEmployee = null;
        
        var stubEmployeeRepository = new StubEmployeeRepository(unknownEmployee);
        var dummyExpenseSheetRepository = new DummyExpenseSheetRepository();
        
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, dummyExpenseSheetRepository);
    }
    
    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(Guid.NewGuid(), Guid.NewGuid(), new DateTime(2018, 11, 11));
        _result = _sut.Handle(command);
    }
    
    [Observation]
    public void Then_the_operation_should_fail()
    {
        Assert.That(_result.IsSuccessful, Is.False);
    }

    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario verifies whether the operation fails when the specified employee could not be found by the repository. This means that the stubbed EmployeeRepository returns a null reference when the Get method is called by the CreateExpenseSheetHandler. Further execution of the Handle method should be aborted. This is enforced by using a dummy implementation of the ExpenseSheetRepository.

A Stub is usually the type of test double that is most often used throughout an entire suite of unit tests.

Spy

The third type of test double is called a Spy. A Spy does the same things as a Stub. It provides returns values for the query methods of its public interface. But a Spy also captures the indirect outputs of the Subject Under Test and afterwards provides the ability for the test to verify these outputs. This is for the command methods of the public interface that it supports.

Let’s have a look at an example of a Spy.

public class SpyExpenseSheetRepository : IExpenseSheetRepository
{
    private readonly ExpenseSheet _result;
    
    public ExpenseSheet SavedExpenseSheet { get; private set; }
    
    public SpyExpenseSheetRepository(ExpenseSheet result)
    {
        _result = result;
    }
    
    public ExpenseSheet Get(Guid id)
    {
        return _result;
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        SavedExpenseSheet = expenseSheet;
    }
}

The SpyExpenseSheetRepository class implements the IExpenseSheetRepository interface. The constructor of the spy expects an instance of an ExpenseSheet that can be returned by the Get method when and if called. But it also assigns the ExpenseSheet object that is specified to the Save method in the SavedExpenseSheet public property.

[Specification]
public class When_creating_a_new_expense_sheet__spy_example
{
    [Establish]
    public void Context()
    {
        var employee = Example.Employee().WithId(new Guid("680D0C0A-E445-4344-B67A-363589E2746A"));
        
        var stubEmployeeRepository = new StubEmployeeRepository(employee);
        _spyExpenseSheetRepository = new SpyExpenseSheetRepository(null);
        
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, _spyExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(
            new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"), 
            new Guid("680D0C0A-E445-4344-B67A-363589E2746A"), 
            new DateTime(2018, 11, 11));

        _result = _sut.Handle(command);
    }
    
    [Observation]
    public void Then_the_approved_expense_sheet_should_be_saved()
    {
        var savedExpenseSheet = _spyExpenseSheetRepository.SavedExpenseSheet;
        
        Assert.That(savedExpenseSheet, Is.Not.Null);
        Assert.That(savedExpenseSheet.Id, 
            Is.EqualTo(new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3")));
        Assert.That(savedExpenseSheet.EmployeeId, 
            Is.EqualTo(new Guid("680D0C0A-E445-4344-B67A-363589E2746A")));
        Assert.That(savedExpenseSheet.SubmissionDate, 
            Is.EqualTo(new DateTime(2018, 11, 11)));
    }

    [Observation]
    public void Then_the_operation_should_succeed()
    {
        Assert.That(_result.IsSuccessful);
    }

    private SpyExpenseSheetRepository _spyExpenseSheetRepository;
    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario verifies the happy path of the Subject Under Test. When creating an instance of the CreateExpenseSheetHandler class, we pass an instance of the StubEmployeeRepository for providing an Employee as an indirect input and an instance of the SpyExpenseSheetRepository that we just saw. The first observation we make uses the SavedExpenseSheet property of the SpyExpenseSheetRepository in order to verify whether the saved ExpenseSheet object contains the information that we specified through the command. The second observation merely verifies whether the return value of the Handle method indicates a successful outcome.

A Spy acts as an observation point for the indirect outputs of the Subject Under Test. It functions as a hatch for verifying these outputs by unit tests. It doesn’t do any verifications of its own in contrast to the next type of test double.

Mock

The fourth type of test double is called a Mock. A Mock does the exact same things as a Spy, only it’s more intelligent than a Spy and it knows what should happen regarding the indirect outputs of the Subject Under Test. A Mock returns values for the query methods of its public interface. It also captures the interactions with the command methods of its public interface. But it also provides the ability for a unit test to state its expectations, and after exercising the Subject Under Test, provides the ability to instruct the Mock to verify the outcome according to these expectations.

Let’s have a look at an example of a Mock.

public class MockExpenseSheetRepository : IExpenseSheetRepository
{
    private readonly ExpenseSheet _result;
    private ExpenseSheet _expectedExpenseSheetToBeSaved;
    private ExpenseSheet _savedExpenseSheet;
    
    public MockExpenseSheetRepository(ExpenseSheet result)
    {
        _result = result;
    }
    
    public ExpenseSheet Get(Guid id)
    {
        return _result;
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        _savedExpenseSheet = expenseSheet;
    }

    public void ExpectSaveToBeCalled(ExpenseSheet expenseSheetToBeSaved)
    {
        _expectedExpenseSheetToBeSaved = expenseSheetToBeSaved;
    }
    
    public void Verify()
    {
        if(null == _expectedExpenseSheetToBeSaved)
            return;
        
        Assert.That(_savedExpenseSheet, Is.Not.Null);
        Assert.That(_savedExpenseSheet.Id, 
            Is.EqualTo(_expectedExpenseSheetToBeSaved.Id));
        Assert.That(_savedExpenseSheet.EmployeeId, 
            Is.EqualTo(_expectedExpenseSheetToBeSaved.EmployeeId));
        Assert.That(_savedExpenseSheet.SubmissionDate, 
            Is.EqualTo(_expectedExpenseSheetToBeSaved.SubmissionDate));
    }
}

The MockExpenseSheetRepository class implements the IExpenseSheetRepository interface. The constructor of the mock expects an instance of an ExpenseSheet that can be returned by the Get method when and if called. It also assigns the ExpenseSheet object that is specified to the Save method in the _savedExpenseSheet member variable. Notice that there are two additional methods compared to the implementation of the spy that we’ve seen earlier. One is the ExpectSaveToBeCalled method which simply stores a specified ExpenseSheet object in the _expectedExpenseSheetToBeSaved member variable. The other one is the Verify method. It first checks whether the _expectedExpenseSheetToBeSaved member variable has been set. If not, then there’s nothing to verify. Otherwise, the ExpenseSheet object that is stored through the Save method is verified and compared against the expected ExpenseSheet object.

[Specification]
public class When_creating_a_new_expense_sheet__mock_example
{
    [Establish]
    public void Context()
    {
        var employee = Example.Employee().WithId(new Guid("680D0C0A-E445-4344-B67A-363589E2746A"));
        var stubEmployeeRepository = new StubEmployeeRepository(employee);

        var expenseSheetToBeSaved = Example.ExpenseSheet()
            .WithId(new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"))
            .WithEmployee(employee)
            .WithSubmissionDate(new DateTime(2018, 11, 11));
            
        _mockExpenseSheetRepository = new MockExpenseSheetRepository(null);
        _mockExpenseSheetRepository.ExpectSaveToBeCalled(expenseSheetToBeSaved);
        
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, _mockExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(
            new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"), 
            new Guid("680D0C0A-E445-4344-B67A-363589E2746A"), 
            new DateTime(2018, 11, 11));

        _result = _sut.Handle(command);
    }
    
    [Observation]
    public void Then_the_approved_expense_sheet_should_be_saved()
    {
        _mockExpenseSheetRepository.Verify();
    }

    [Observation]
    public void Then_the_operation_should_succeed()
    {
        Assert.That(_result.IsSuccessful);
    }

    private MockExpenseSheetRepository _mockExpenseSheetRepository;
    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario also verifies the happy path of the Subject Under Test. Instead of a Spy we now create an instance of the CreateExpenseSheetHandler class using the MockExpenseSheetRepository that we just saw. The mock is instructed to expect a call to the Save method with a specific ExpenseSheet object containing the data that we expect. The observation that verifies whether a new ExpenseSheet object is saved by the CreateExpenseSheetHandler simply calls the Verify method on the mock. If things don’t work out as expected, then the MockExpenseSheetRepository will fail the test on its behalf.

There are generally two different types of mocks: strict and non-strict mock objects. Strict mocks expect that recorded calls are performed in a particular order. On the contrary, non-strict mocks don’t impose this restriction. The general guideline and best practice is to avoid strict mocks as much as possible. Unit tests that make use of strict mocks impose even more constraints on the implementation and are therefore more strongly coupled to the concrete implementation of the Subject Under Test. Therefore mock objects should almost always be non-strict.

So now we’ve seen the four different flavours of test doubles that are important for solitary tests. There’s one more type of test double that we’re going to briefly discuss now for the sake of completeness.

Fake

A different kind of test double compared to the other types of test doubles that we’ve seen so far is called a Fake. A Fake is an object that, from an outside view, simulates the exact same functionality and behaviour as the real implementation that it replaces. The difference with the other types of test doubles exists in the fact that a Fake is not controlled or observed by a test.

Let’s have a look at an example of a Fake.

public class FakeExpenseSheetRepository : IExpenseSheetRepository
{
    private readonly Dictionary<Guid, ExpenseSheet> _expenseSheets;
    
    public FakeExpenseSheetRepository()
    {
        _expenseSheets = new Dictionary<Guid, ExpenseSheet>();
    }
    
    public ExpenseSheet Get(Guid id)
    {
        var isFound = _expenseSheets.TryGetValue(id, out var expenseSheet);
        return isFound ? expenseSheet : null;
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        var current = Get(expenseSheet.Id);
        var currentVersion = current?.Version ?? 0;
        
        if(currentVersion != expenseSheet.Version)
            throw new OptimisticConcurrencyException<ExpenseSheet>(expenseSheet.Id, currentVersion, 
                expenseSheet.Version);

        expenseSheet.Version += 1;
        _expenseSheets[expenseSheet.Id] = expenseSheet;
    }
}

The FakeExpenseSheetRepository class provides a fully functional implementation of an ExpenseSheetRepository, including code for enforcing optimistic concurrency. The only difference with the real implementation is that the FakeExpenseSheetRepository doesn’t access a data store. Instead it uses a Dictionary to store ExpenseSheet objects in memory.

[Specification]
public class When_creating_a_new_expense_sheet__fake_example
{
    [Establish]
    public void Context()
    {
        var employee = Example.Employee().WithId(new Guid("680D0C0A-E445-4344-B67A-363589E2746A"));
        var stubEmployeeRepository = new StubEmployeeRepository(employee);
            
        _fakeExpenseSheetRepository = new FakeExpenseSheetRepository();            
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, _fakeExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(
            new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"), 
            new Guid("680D0C0A-E445-4344-B67A-363589E2746A"), 
            new DateTime(2018, 11, 11));

        _result = _sut.Handle(command);
    }

    [Observation]
    public void Then_the_operation_should_succeed()
    {
        Assert.That(_result.IsSuccessful);
    }

    private FakeExpenseSheetRepository _fakeExpenseSheetRepository;
    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario now uses the FakeExpenseSheetRepository instead of a spy or a mock. Because the test fixture uses a fake, we are no longer able to verify whether a new ExpenseSheet object is saved in the repository. The only thing that we can do is verify whether the return value of the Handle method indicates a successful outcome.

One might argue that using fakes no longer have anything to do with behaviour verification. That’s indeed a statement that has a certain truth to it. They’re used by tests for reasons other than verification of indirect outputs. Generally speaking, Fake objects are not well suited for solitary tests. So I usually try to avoid them for solitary tests when possible.

On the other hand, Fake objects are more than useful for sociable tests. But caution is still advisable when using them on a large scale though. A fake implementation needs to be kept in sync with the real implementation at all times.

One way to accomplish this is to use Contract Tests. Contract tests not only verify the outside observable behaviour of out-of-process services like databases, web services, etc. … . They also run against the fake implementation to ensure 100% behavioural compatibility. This can involve some significant development effort. So carefully consider when and where to use this powerful technique.

Summary

This diagram provides an overview of the different types of test doubles and for which kinds of tests they are most often applicable. Learning about the behaviour of each type of test double is essential to understanding the different levels of coupling that they impose. More on that in a next blog post.