Blog

In a previous blog post, we discussed the test pyramid and why it is a useful model to reason about a healthy mix of solitary and sociable tests.

I mentioned that at the bottom of the test pyramid, we have the most isolation. The more we move up the test pyramid, the more integration is employed which results in less design feedback. As soon as we move away from testing a single concrete class or Subject Under Test and start considering collaborations between several concrete implementations, we’re bound to encounter cascading failures. This means that a slightest change of the production code or a bug can result in a high number of failing tests that, from a conceptual point of view, don’t have a direct relation to the changed code.

Let’s have a look at an example.

public class Customer
{
    public int Id { get; }
    public string Email { get; private set; }
    public CustomerType Type { get; private set; }

    public Customer(int id, string email)
    {
        Id = id;
    }
    
    public void MakePreferred()
    {
        Type = CustomerType.Preferred;
    }

    public void ChangeEmail(string newEmail)
    {
        Email = newEmail;
    }
}

public enum CustomerType
{
    Regular = 0,
    Preferred = 1
}

Here we have a part of an application that manages customers. Users of the system can make a customer a preferred customer or change its email. A preferred customer receives some additional discounts and faster shipping. This is the implementation of the customer entity.

We also have two handler classes that receive commands for either making a customer preferred or changing its email.

//
// Make a customer preferred
//
public class MakeCustomerPreferredHandler
{
    private readonly AuthorizationService _authorizationService;
    private readonly ICustomerRepository _customerRepository;

    public MakeCustomerPreferredHandler(
        AuthorizationService authorizationService,
        ICustomerRepository customerRepository)
    {
        _authorizationService = authorizationService;
        _customerRepository = customerRepository;
    }
    
    public void Handle(MakeCustomerPreferred command)
    {
        if(! _authorizationService.IsAllowed(command.Action))
            ThrowUnauthorizedException(command.CustomerId);

        var customer = _customerRepository.Get(command.CustomerId);
        if(null == customer)
            ThrowUnknownCustomerException(command.CustomerId);
        
        customer.MakePreferred();
        _customerRepository.Save(customer);
    }

    private static void ThrowUnauthorizedException(int customerId)
    {
        var errorMessage = $"Not authorized to make customer (ID: {customerId}) a preferred 
                             customer.";
        throw new UnauthorizedException(errorMessage);
    }

    private static void ThrowUnknownCustomerException(int customerId)
    {
        var errorMessage = $"The customer with ID ${customerId} is not known by the system and 
                            therefore could not be made a preferred customer.";
        throw new UnknownCustomerException(errorMessage);
    }
}

public class MakeCustomerPreferred
{
    public CustomerAction Action { get; }
    public int CustomerId { get; }

    public MakeCustomerPreferred(int customerId)
    {
        Action = CustomerAction.MakePreferred;
        CustomerId = customerId;
    }
}

//
// Change the email of a customer
//
public class ChangeCustomerEmailHandler
{
    private readonly AuthorizationService _authorizationService;
    private readonly ICustomerRepository _customerRepository;

    public ChangeCustomerEmailHandler(
        AuthorizationService authorizationService,
        ICustomerRepository customerRepository)
    {
        _authorizationService = authorizationService;
        _customerRepository = customerRepository;
    }

    public void Handle(ChangeCustomerEmail command)
    {
        if(! _authorizationService.IsAllowed(command.Action))
            ThrowUnauthorizedException(command.CustomerId);

        var customer = _customerRepository.Get(command.CustomerId);
        if(null == customer)
            ThrowUnknownCustomerException(command.CustomerId);
        
        customer.ChangeEmail(command.NewEmail);
        _customerRepository.Save(customer);
    }
    
    private static void ThrowUnknownCustomerException(int customerId)
    {
        var errorMessage = $"The customer with ID ${customerId} is not known by the system and 
                             therefore it's email could not be changed.";
        throw new UnknownCustomerException(errorMessage);
    }

    private static void ThrowUnauthorizedException(int customerId)
    {
        var errorMessage = $"Not authorized to make customer (ID: {customerId}) a preferred 
                             customer.";
        throw new UnauthorizedException(errorMessage);
    }
}

public class ChangeCustomerEmail
{
    public CustomerAction Action { get; }
    public int CustomerId { get; }
    public string NewEmail { get; }

    public ChangeCustomerEmail(int customerId, string newEmail)
    {
        Action = CustomerAction.ChangeEmail;
        CustomerId = customerId;
        NewEmail = newEmail;
    }
}

Notice that these handler classes make use of an authorisation service that verifies whether the user is allowed to perform the operation.

We have two types of users in the system; help desk staff and back-office managers.

public class UserContext
{
    public UserRole Role { get; }

    public UserContext(UserRole role)
    {
        Role = role;
    }
}

public enum UserRole
{
    Unknown = 0,
    HelpDeskStaff = 1,
    BackOfficeManager = 2
}

Currently both types of users are allowed to make customers preferred or change their email. Only users that are not known by the system are disallowed all actions.

public class AuthorizationService
{
    private readonly UserContext _userContext;

    public AuthorizationService(UserContext userContext)
    {
        _userContext = userContext;
    }
    
    public bool IsAllowed(CustomerAction customerAction)
    {
        if(_userContext.Role == UserRole.Unknown)
            return false;
        
        // ...
        
        return true;
    }
}

public enum CustomerAction
{
    ChangeEmail = 0,
    MakePreferred = 1,
    // ...
}

Now the business has come up with a new requirement. From now on, only back-office managers are allowed to make customers preferred. Help desk staff are still allowed to change the email of our customers.

At this point, we make a change to the authorisation service to reflect this new requirement.

public bool IsAllowed(CustomerAction customerAction)
{
    if(_userContext.Role == UserRole.Unknown)
        return false;

    if(_userContext.Role == UserRole.HelpDeskStaff && customerAction == CustomerAction.MakePreferred)
        return false;

    // ...
    
    return true;
}

What we see now is that suddenly some tests for the MakeCustomerPreferredHandler start failing due to this change in the AuthorizationService. When we take a closer look, we see that the failing tests are using a concrete instance of the AuthorizationService.

[TestFixture]
public class When_an_authorized_user_makes_a_customer_preferred
{        
    [Test]
    public void Then_the_specified_customer_should_be_made_a_preferred_customer()
    {
        var customer = new Customer(354, "john@doe.com");
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.HelpDeskStaff);
        var authorizationService = new AuthorizationService(userContext);

        var customerRepository = Substitute.For<ICustomerRepository>();
        customerRepository.Get(Arg.Any<int>()).Returns(customer);
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        sut.Handle(command); 
        
        Assert.That(customer.Type, Is.EqualTo(CustomerType.Preferred));
    }
    
    [Test]
    public void Then_the_specified_customer_should_henceforth_be_treated_as_a_preferred_customer_by_the_system()
    {
        var customer = new Customer(354, "john@doe.com");
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.HelpDeskStaff);
        var authorizationService = new AuthorizationService(userContext);

        var customerRepository = Substitute.For<ICustomerRepository>();
        customerRepository.Get(Arg.Any<int>()).Returns(customer);
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        sut.Handle(command); 
        
        customerRepository.Received().Save(customer);
    }
}

[TestFixture]
public class When_an_unauthorized_user_attempts_to_make_a_customer_preferred
{        
    [Test]
    public void Then_an_exception_should_be_thrown()
    {
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.Unknown);
        var authorizationService = new AuthorizationService(userContext);
        
        var customerRepository = Substitute.For<ICustomerRepository>();
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        TestDelegate makeCustomerPreferred = () => sut.Handle(command);
        
        Assert.That(makeCustomerPreferred, Throws.InstanceOf<UnauthorizedException>());    
    }
}

[TestFixture]
public class When_an_authorized_user_attempts_to_make_a_customer_preferred_that_is_not_known_by_the_system
{               
    [Test]
    public void Then_an_exception_should_be_thrown()
    {
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.BackOfficeManager);
        var authorizationService = new AuthorizationService(userContext);
        
        var customerRepository = Substitute.For<ICustomerRepository>();
        customerRepository.Get(Arg.Any<int>()).ReturnsNull();
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        TestDelegate makeCustomerPreferred = () => sut.Handle(command);
        
        Assert.That(makeCustomerPreferred, Throws.InstanceOf<UnknownCustomerException>());
    }
}

This means that these tests are not testing the handler in isolation. They include, and are therefore dependent on, the implementation of the AuthorizationService. Therefore these are not 100% solitary tests.

Suppose that we have a few dozen handler classes like these with corresponding tests that each use a concrete instance of the AuthorizationService. Now this small change to the implementation of the AuthorizationService results in lots of failing tests. A simple change requested by the business can make these tests into a maintenance nightmare.

In this particular case we can easily fix these tests by changing the role that we pass to the UserContext. However, we can also take the path of further decoupling the tests.

We can introduce an interface for the AuthorizationService.

public interface IAuthorizationService
{
    bool IsAllowed(CustomerAction customerAction);
}

public class AuthorizationService : IAuthorizationService
{
    private readonly UserContext _userContext;

    public AuthorizationServiceV2(UserContext userContext)
    {
        _userContext = userContext;
    }
    
    public bool IsAllowed(CustomerAction customerAction)
    {
        if(_userContext.Role == UserRole.Unknown)
            return false;

        if(_userContext.Role == UserRole.HelpDeskStaff && customerAction == CustomerAction.MakePreferred)
            return false;

        // ...
        
        return true;
    }
}

Now we are able to use a test double, either written manually or generated dynamically using a mocking framework like we already did for the CustomerRepository. In our example we’ve used the NSubstitute library.

[TestFixture]
public class When_an_unauthorized_user_attempts_to_make_a_customer_preferred
{        
    [Test]
    public void Then_an_exception_should_be_thrown()
    {
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a test double for the AuthorizationService and disallow the action
        var authorizationService = Substitute.For<IAuthorizationService>();
        authorizationService.IsAllowed(CustomerAction.MakePreferred).Returns(false);
        
        var customerRepository = Substitute.For<ICustomerRepository>();
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        TestDelegate makeCustomerPreferred = () => sut.Handle(command);
        
        Assert.That(makeCustomerPreferred, Throws.InstanceOf<UnauthorizedException>());    
    }
}

This way we have completely isolated the Subject Under Test and converted the tests for the handler classes to 100% solitary tests. This way we can make more changes to the AuthorizationService without breaking other tests (e.g. help desk staff is no longer allowed to change the email of a customer either).

Whenever a Subject Under Test requires collaborators for achieving its promised functionality, we basically have three options:

• Use an instance of a concrete class.
• Use an instance of a handwritten test double.
• Use an instance provided by a test double library or framework.

If we choose the first option, we’re all set up for cascading failures. Cascading failures, as shown in this example, is one of the most important reasons why people lose faith in their tests. There are few things that can kill productivity and motivation faster than cascading failures. The goal should always be to have a loosely coupled design for both the production code as well as the test code that accompanies it. Therefore we should favour more fine-grained, solitary tests over course-grained, sociable tests. This way, changes to the Subject Under Test will only create failures in tests that are directly associated with it. Failures can be resolved more easily and quickly with fine-grained tests, sometimes even by just looking at the code. Failures of course-grained tests more frequently involves debugging the code under test. Debugging code requires a lot more mental cycles from software developers, which results in more time spent trying to find the offending code. And time takes money.

We can reduce the effects of cascading failures by replacing cross-boundary collaborators with test doubles, converting all tests into solitary tests. But do make sure to take a pragmatic approach. I like to emphasise that it’s still fine to involve the collaborators of a Subject Under Test when they live within the same boundary. An example of this is when writing solitary tests for an aggregate root inside the domain of an application.

The term “Subject Under Test” is a reminder to think carefully about the granularity of the unit of code that we want to design and therefore consume.

Jan Van Ryswyck

Thank you for visiting my blog. I’m a professional software developer since Y2K. A blogger since Y2K+5. Provider of training and coaching in XP practices. Curator of the Awesome Talks list. Past organizer of the European Virtual ALT.NET meetings. Thinking and learning about all kinds of technologies since forever.