Blog

When stating the definition of a solitary test, we mentioned that they have two constraints:

1. The code being exercised by these tests never cross the process boundary in which they are executed.
2. A single class or module is being tested.

The first constraint means that a solitary test never executes code that talks to a database, communicates across the network, touches the file system, etc. … This basically implies that a solitary test never requires any configuration whatsoever in order for it to execute correctly.

The second constraint is definitely the most controversial one. The most fundamental solitary test exercises a single class or module. Does this imply that a test which exercises code of multiple concrete classes is by definition a sociable test? Well, that kind of depends on a number of factors.

Do these cluster of classes all live within the same dependency inversion boundary? Is there a single class within this cluster that takes up the role as the main entry point? Are the other classes in the cluster only used by the main entry class? Are all these classes part of a single conceptual whole? If the answer to these questions are all positive, then I would argue that a test which exercises code of such a cluster of classes is still a solitary test and not a sociable test.

I do recognise that this hugely depends on the person you ask the question. There just isn’t a wide consensus on this particular topic. In this blog post I’m going to make a (probably futile) attempt to clarify the approach that I usually employ.

I personally consider the test pyramid to be more of a spectrum and less as a pile of discrete buckets. The moment a test exercises code of more than a single concrete class, it moves up the pyramid towards the area where the sociable tests live. How much the test rises depends on the constraints and criteria mentioned earlier.

Let’s have a look at an example.

public class ExpenseSheetController : Controller
{
    private readonly ICommandHandler<CreateExpenseSheet> _commandHandler;
    private readonly CreateExpenseSheetCommandMapper _commandMapper;
    private readonly CreateExpenseSheetFormModelValidator _validator;

    public ExpenseSheetController(ICommandHandler<CreateExpenseSheet> commandHandler)
    {
        _commandHandler = commandHandler;
        _commandMapper = new CreateExpenseSheetCommandMapper();
        _validator = new CreateExpenseSheetFormModelValidator();
    }
    
    [HttpPost]
    public IActionResult Create(CreateExpenseSheetFormModel formModel)
    {
        var isValid = _validator.Validate(formModel);
        if(! isValid)
            return BadRequest();
        
        var command = _commandMapper.MapFrom(formModel);
        var result = _commandHandler.Handle(command);

        if(! result.IsSuccessful)
            return BadRequest();
            
        return Ok();
    }
}

The subject under test is a web controller that exposes a single endpoint. This endpoint accepts a form model which contains the data necessary for creating a new expense sheet. First the specified form model is validated. In case the form model contains incorrect data, HTTP status code 400 (bad request) is returned to the client of our endpoint. If valid, then form model is mapped to a command object after which it is sent to the corresponding command handler. When the command handler is able to successfully process the command, then HTTP status 200 (OK) is returned to the client. Otherwise, we return HTTP status code 400 (bad request).

public class CreateExpenseSheetFormModel
{
    public Guid EmployeeId { get; set; }
    public IEnumerable<ExpenseModel> Expenses { get; set; }
    public DateTime SubmissionDate { get; set; }

    public CreateExpenseSheetFormModel()
    {
        Expenses = Enumerable.Empty<ExpenseModel>();    
    }
}

public class ExpenseModel
{
    public decimal Amount { get; set; }
    public DateTime Date { get; set; }
    public string Description { get; set; }
}

This is how the code of the form model looks like …

public class CreateExpenseSheetFormModelValidator
{
    public bool Validate(CreateExpenseSheetFormModel formModel)
    {
        return Exists(formModel.Expenses) &&
               formModel.Expenses.All(IsValid);
    }

    private static bool Exists(IEnumerable<ExpenseModel> expenses)
    {
        return expenses != null && expenses.Any();
    }
    
    private static bool IsValid(ExpenseModel expenseModel)
    {
        return expenseModel.Amount > 0 &&
            ! string.IsNullOrWhiteSpace(expenseModel.Description);
    }
}

public class CreateExpenseSheetCommandMapper
{
    public CreateExpenseSheet MapFrom(CreateExpenseSheetFormModel formModel)
    {
        var expenses = formModel.Expenses
            .Select(expenseModel => new ExpenseData(
                expenseModel.Amount, expenseModel.Date, expenseModel.Description));
        
        return new CreateExpenseSheet(Guid.NewGuid(), formModel.EmployeeId, 
            formModel.SubmissionDate, expenses);
    }
}

… and these are the implementations for the validator of the form model, and the mapper that maps the form model to the corresponding command.

Let’s have a look at the implementation of the tests.

[Specification]
public class When_handling_a_request_for_creating_a_new_expense_sheet
{
    [Establish]
    public void Context()
    {
        _commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        _commandHandler.Handle(Arg.Any<CreateExpenseSheet>()).Returns(Result.Success());
        
        _sut = new ExpenseSheetController(_commandHandler);    
    }

    [Because]
    public void Of()
    {
        var formModel = new CreateExpenseSheetFormModel
        {
            EmployeeId = new Guid("A881CF9F-888E-482B-A5EE-ACDD10D01CB2"),
            Expenses = new[]
            {
                new ExpenseModel 
                    { Amount = 12.8m, Date = new DateTime(2020, 04, 14), Description = "Lunch" },
                new ExpenseModel 
                    { Amount = 46.2m, Date = new DateTime(2020, 04, 15), Description = "Fuel" }
            },
            SubmissionDate = new DateTime(2019, 03, 16)
        };
        
        _result = _sut.Create(formModel);    
    }

    [Observation]
    public void Then_a_command_should_be_dispatched_to_the_domain()
    {
        var expectedCommand = new CreateExpenseSheet(
            Guid.Empty, 
            new Guid("A881CF9F-888E-482B-A5EE-ACDD10D01CB2"), 
            new DateTime(2019, 03, 16), 
            new[]
            {
                new ExpenseData(12.8m, new DateTime(2020, 04, 14), "Lunch"), 
                new ExpenseData(46.2m, new DateTime(2020, 04, 15), "Fuel")
            });
        
        _commandHandler.Should_have_received(expectedCommand, command => command.Id);
    }
    
    [Observation]
    public void Then_HTTP_status_code_200_should_be_sent_back_to_the_client()
    {
        _result.Should_be_an_instance_of<OkResult>();        
    }

    private ICommandHandler<CreateExpenseSheet> _commandHandler;
    private IActionResult _result;
    private ExpenseSheetController _sut;
}

[Specification]
public class When_handling_an_invalid_request_for_creating_a_new_expense_sheet
{
    [Establish]
    public void Context()
    {
        var commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        commandHandler.Handle(Arg.Any<CreateExpenseSheet>())
            .Throws(
                new InvalidOperationException("Handle method of command handler shouldn't get called"));
        
        _sut = new ExpenseSheetController(commandHandler);
    }

    [Because]
    public void Of()
    {
        var invalidFormModel = new CreateExpenseSheetFormModel();
        _result = _sut.Create(invalidFormModel);
    }

    [Observation]
    public void Then_HTTP_status_code_400_should_be_sent_back_to_the_client()
    {
        _result.Should_be_an_instance_of<BadRequestResult>();
    }

    private ExpenseSheetController _sut;
    private IActionResult _result;
}

[Specification]
public class When_handling_a_request_for_creating_a_new_expense_sheet_that_fails_the_domain_criteria
{
    [Establish]
    public void Context()
    {
        var commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        commandHandler.Handle(Arg.Any<CreateExpenseSheet>())
            .Returns(Result.Failure(new DomainViolation("This is highly irregular")));
        
        _sut = new ExpenseSheetController(commandHandler);
    }

    [Because]
    public void Of()
    {
        var formModel = new CreateExpenseSheetFormModel();
        _result = _sut.Create(formModel);
    }

    [Observation]
    public void Then_HTTP_status_code_400_should_be_sent_back_to_the_client()
    {
        _result.Should_be_an_instance_of<BadRequestResult>();
    }

    private IActionResult _result;
    private ExpenseSheetController _sut;
}

Notice that these tests provide a test double for the command handler when creating an instance of the controller. We use a test double here because the command handler resides in a different dependency inversion boundary, namely the core domain of the application. But what about the validator and the mapper? When looking at the constructor of the controller, notice that an instance of the validator and the mapper are created instead of just being injected.

public ExpenseSheetController(ICommandHandler<CreateExpenseSheet> commandHandler)
{
    _commandHandler = commandHandler;
    _commandMapper = new CreateExpenseSheetCommandMapper();
    _validator = new CreateExpenseSheetFormModelValidator();
}

The reasoning behind this is that the validator and the mapper are only used by this controller. In this specific case, these classes are an implementation detail of the controller. As a matter of fact, I could have chosen not to create separate classes, neglecting the single-responsibility principle, and just go for private methods in the controller itself. In that case it would also be much harder and less efficient to write tests for driving the implementation of the validator.

If for some reason the validator or mapper would be used by another controller, then I would have chosen to use dependency injection instead of creating the instance in the constructor. I would also have used a test double in the tests as well. But I don’t consider it very likely that the validator and mapper classes would be used in other parts of the code.

By letting the controller create the instance of the validator and mapper classes, the controller acts as the main entry point of the cluster. This entire approach leans more towards the Detroit School of TDD. When following The London School of TDD, we would use dependency injection for both the validator and mapper classes, and use test doubles for them as well in out tests. Which, for the record, is just a valid approach as well. We would also have separate test suites for both the validator and mapper classes.

Going back to our example, we created separate tests for the validator but not for the mapper.

[TestFixture]
public class When_validating_a_form_model_for_creating_an_expense_sheet  
{
    private CreateExpenseSheetFormModelValidator _sut;

    [Establish]
    public void Context()
    {
        _sut = new CreateExpenseSheetFormModelValidator();
    }
    
    [Observation]
    public void Then_it_should_indicate_truthy_for_a_valid_form_model()
    {
        var validFormModel = new CreateExpenseSheetFormModel
        {
            Expenses = new[]
            {
                new ExpenseModel 
                    { Amount = 23.8m, Date = new DateTime(2020, 04, 15), Description = "Lunch" }
            }
        };

        var isValid = _sut.Validate(validFormModel);
        isValid.Should_be_truthy();
    }
    
    [Observation]
    public void Then_it_should_indicate_falsy_for_a_form_model_with_an_invalid_list_of_expenses()
    {
        var invalidFormModel = new CreateExpenseSheetFormModel { Expenses = null };
        
        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
    
    [Observation]
    public void Then_it_should_indicate_falsy_for_a_form_model_without_expenses()
    {
        var invalidFormModel = new CreateExpenseSheetFormModel();
        
        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
    
    [ObservationFor(-1)]
    [ObservationFor(0)]
    public void Then_it_should_indicate_falsy_for_a_form_model_specifying_an_expense_with_an_insufficient_amount
        (decimal invalidAmount)
    {
        var invalidFormModel = new CreateExpenseSheetFormModel
        {
            Expenses = new[] { 
                new ExpenseModel
                {
                    Amount = invalidAmount, 
                    Date = new DateTime(2020, 04, 15), 
                    Description = "Dinner"
                } 
            }
        };

        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
    
    [ObservationFor(null)]
    [ObservationFor("")]
    [ObservationFor(" ")]
    public void Then_it_should_indicate_falsy_for_a_form_model_specifying_an_expense_without_description
        (string invalidDescription)
    {
        var invalidFormModel = new CreateExpenseSheetFormModel
        {
            Expenses = new[] { 
                new ExpenseModel
                {
                    Amount = 15.65m, 
                    Date = new DateTime(2020, 04, 15), 
                    Description = invalidDescription
                } 
            }
        };

        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
}

The reasoning behind this is that the implementation of the validator has a cyclomatic complexity that is greater than one, while for the mapper the cyclomatic complexity is exactly one. It’s much easier to use a separate test suite to exercise the implementation of the validator instead of through the test suite of the controller. While for the mapper class, we’ve chosen to exercise its code through the test suite of the controller itself.

The following diagram provides an overview of the approach we’ve taken in our example.

There’s one thing that we need to watch out for though, and that is cascading failures. When an issue arises in the implementation of the validator or the mapper classes, then multiple controller tests would probably fail. But the cluster of classes is still small enough to easily pinpoint the cause of the issue. When following The London School of TDD, this is less of an issue.

The tests for the controller a placed higher on the spectrum of the test pyramid, while the tests for the validator appear more near the bottom. But I do still consider these tests as solitary tests. If, for example, I would have used a concrete instance for the command handler in the test suite of the controller, then I would say that these tests are sociable tests instead.

Determining the unit of solitary tests is not really an exact science. It usually comes down to just following your intuition and relying on past experiences to determine a certain boundary that is useful. Personally I find the test pyramid a very useful model to think and reason about these boundaries.

Jan Van Ryswyck

Thank you for visiting my blog. I’m a professional software developer since Y2K. A blogger since Y2K+5. Provider of training and coaching in XP practices. Curator of the Awesome Talks list. Past organizer of the European Virtual ALT.NET meetings. Thinking and learning about all kinds of technologies since forever.