Share the intellectual wealth

Fixing vs Preventing Defects

2026-03-12T09:00:00Z

The way a team handles defects reveals much about how their organisation approaches quality. When every defect must be added to a sprawling backlog with excruciatingly detailed information, it often signals a codebase burdened by technical debt and a product suffering from the consequences of that. Some of these defects may (or may never) be fixed in so-called “stabilisation sprints”.

In contrast, consider a team that is focused on preventing defects in the first place. They rigorously apply techniques like TDD, ATDD, and BDD to accomplish this. Whenever a defect does slip through, the team pauses all other work and addresses it immediately. This reflects a fundamentally different mindset: no reliance on bug-tracking tools, and no time wasted on defect reporting. But more importantly, a focus on building in quality.

Tales Of TDD: One Test Double To Rule Them All

2026-03-05T09:00:00Z

So I’ve encountered this specific test case, which is quite elaborate.

What do you mean by that?

Well, it contains a lot of code, and also looks more difficult to understand compared to the other test cases.

Yeah, that’s because of the Fake we’re using in our test code.

So, the test code is complex because we’re using a Fake?

Yes, I’m afraid so. The test scenario embodies a specific edge case. The Subject Under Test should initiate a recovery only when the collaborator raises an InvalidValueException.

That does align with the functionality we discussed earlier.

I had to do all this setup in the test method just to force the Fake to throw an InvalidValueException.

I see. I agree that it’s sometimes not easy to make a Fake behave exactly as you want.

Tell me about it. It took me almost an hour to figure out how to make it throw an exception. I really like using Fakes in our tests because they behave just as the real implementation. But sometimes, they can be a real pain to work with as well.

Have you considered using a Stub instead of a Fake?

A Stub? No, not really. We decided in a team meeting to use Fakes instead of Stubs, Spies, or Mocks

I understand. I’m not suggesting you stop using Fakes altogether. But have you considered not using a Fake just for this specific test case? Would it make the test easier to understand if you used a Stub that raises an InvalidValueException instead?

Yeah, but that’s not what we agreed on in the team meeting.

I’m sure what was agreed on was more of a guideline than a strict rule. How about we just try it out? Let’s change the test code to use a Stub instead of a Fake.

OK. We can do that.

… After short while …

Now the test is just four lines of code. What do you think?

Yeah, it’s a lot more concise and readable.

Do you want to keep it like this?

Sure! But shouldn’t we change all the other tests to use a Stub for that collaborator as well?

Why do you want to do that?

Because that way, it’s more consistent.

If consistency is what we’re after, then I agree with you. But what do we ultimately want from our tests?

That they fail when we have an issue in the production code.

Yes, and what else?

That they are readable and easy to understand.

Exactly! There’s no reason to be dogmatic about test doubles. We can just use the one that best fits our purpose. So it’s perfectly fine to use Fakes for most of the tests, while using a Stub for this specific test case as a one-off.

I see that now. So I don’t need to stick with one specific type of test double? That’s very cool. I can’t wait to show this off at our next team meeting.

Perspectives On Software Quality

2026-02-03T18:30:00Z

The quality of software, or any product, can be viewed from different perspectives. This is because there are usually multiple stakeholders, each bringing their own unique views and interests to the table.

Some of these perspectives include:

The quality of a product as experienced by customers.
The quality of a product as perceived by the user, which can be highly subjective and personal.
The quality of the process used to build the product, which is particularly important to members of a software team.

A common issue in software organisations is the tendency to prioritize one specific perspective over other quality perspectives. For example, when customer or user quality is considered more important than process quality. However, this is a fallacy. Although these quality perspectives are often represented by different people, they are inextricably intertwined and each cannot stand on its own.

Focusing solely on customer or user quality while neglecting process quality can lead to a deteriorating codebase, making it difficult or impossible to add new features. Conversely, prioritizing only process quality, while neglecting customer or user quality, may result in a technically impressive product that fails to attract users and ultimately serves no purpose.

Organisations that empower their software teams to invest equally in all quality perspectives will outperform all others who choose not to apply this balance.

The Five Underplayed Premises Of TDD

2025-08-08T17:30:00Z

Today I got reminded of a great video of GeePaw Hill about the “Five Underplayed Premises Of TDD” .

1. The money premise:

When writing tests, we don’t do it for the art of it or other moral reasons. We do TDD because it lets us build more features faster.

2. The judgement premise:

There’s no one-size-fits all approach to building quality software. Developers applying TDD use human judgement while building software systems.

3. The chain premise:

We prefer testing very tiny subsystems that make up our larger system. Why? Because small tests are not only easier to read, comprehend and write, but also execute a lot faster compared to large-scale end-to-end tests. This is a preference, not a hard rule.

4. The correlation premise:

The internal quality of the code for a system is directly correlated to the productivity of the developers working in this code base. They go up together, and they go down together.

5. The driving premise:

Tests and testability help drive the design of a system. A code base that is highly testable, exposes certain characteristics that are generally considered as good design. People new to TDD often resist the idea of changing their design to facilitate tests. However, this is inevitable when doing TDD. It drives the design.

These 5 premises are at the foundation of every step a developer makes when using TDD. Check out GeePaw’s video if you want to learn more. The best 10 minutes you’ll probably spend this week. I promise 😄

Contract Tests - Parameterised Test Cases

2023-06-28T13:00:00Z

This is the final installment of a three-part series about contract tests. In the first blog post we’ve discussed the rationale behind contract tests. Next we’ve looked at how to implement contract tests using Abstract Test Cases. In this blog post, we’re going to look into an alternative approach to Abstract Test Cases by using parameterised tests instead.

With this alternative approach, we no longer rely on inheritance, where concrete classes are derived from an abstract base class that contains the test cases. Instead, Sociable tests are added to a regular test class, just as we would normally do. We apply the “Abstract Factory” design pattern to create the respective Subject Under Test, either the real implementation or the fake implementation. Then we use this factory to execute every parameterised test case for each subject that the factory instantiates.

Let’s have a look at an example to demonstrate this approach. Although we’ve used Java in our example, the same pattern can be implemented in a similar way by using other object-oriented languages like C#, Python, Ruby, etc. …

We’re going to implement the same example as we’ve used in the previous blog post. The Subject Under Test is still a repository for storing and retrieving employee data to and from a database.

We start out by defining an interface.

interface EmployeeRepositoryStrategy extends AutoCloseable {
    EmployeeRepository getSubjectUnderTest();
}

This interface defines a method for creating the Subject Under Test, which in our example is an instance of an EmployeeRepository. This interface also extends the AutoCloseable interface which provides a close method. This method can be used to perform some cleanup.

The following piece of code shows the implementation of the SQLiteEmployeeRepositoryStrategy, which implements the EmployeeRepositoryStrategy interface.

static class SQLiteEmployeeRepositoryStrategy implements EmployeeRepositoryStrategy {

    private final NamedParameterJdbcTemplate jdbcTemplate;
    private final SQLiteEmployeeRepository sqliteEmployeeRepository;

    public SQLiteEmployeeRepositoryStrategy() {
        var database = getClass().getClassLoader().getResource("database.db");
        var connectionUrl = String.format("jdbc:sqlite:%s", database);

        var sqliteDataSource = new SQLiteDataSource();
        sqliteDataSource.setUrl(connectionUrl);

        this.jdbcTemplate = new NamedParameterJdbcTemplate(sqliteDataSource);
        this.sqliteEmployeeRepository = new SQLiteEmployeeRepository(jdbcTemplate);
    }

    @Override
    public EmployeeRepository getSubjectUnderTest() {
        return sqliteEmployeeRepository;
    }

    @Override
    public void close() {
        jdbcTemplate.getJdbcOperations().execute("DELETE FROM Employee");
    }
}

The constructor initialises an instance of the SQLiteEmployeeRepository. This instance is returned by the implementation of the getSubjectUnderTest method. The close method simply removes all records from the Employee table in the database.

The following piece of code shows the implementation of the FakeEmployeeRepositoryStrategy, which also implements the EmployeeRepositoryStrategy interface.

static class FakeEmployeeRepositoryStrategy implements EmployeeRepositoryStrategy {

    private final FakeEmployeeRepository fakeEmployeeRepository;

    public FakeEmployeeRepositoryStrategy() {
        fakeEmployeeRepository = new FakeEmployeeRepository();
    }

    @Override
    public EmployeeRepository getSubjectUnderTest() {
        return fakeEmployeeRepository;
    }

    @Override
    public void close() {
        fakeEmployeeRepository.clear();
    }
}

The constructor initialises an instance of the FakeEmployeeRepository. This instance is also returned by the implementation of the getSubjectUnderTest method. The close method removes all the data from the repository by calling the clear method.

The contract tests for the EmployeeRepository now look like this:

public class EmployeeRepositoryTests {

    @ParameterizedTest
    @ArgumentsSource(EmployeeRepositoryStrategyProvider.class)
    public void Should_return_nothing_for_a_non_existing_employee(EmployeeRepositoryStrategy strategy) {

        var unknownId = UUID.fromString("753350fb-d9a2-4e4b-8ca4-c969ca54ef5f");

        var SUT = strategy.getSubjectUnderTest();
        var retrievedEmployee = SUT.get(unknownId);

        assertThat(retrievedEmployee).isNull();
    }

    @ParameterizedTest
    @ArgumentsSource(EmployeeRepositoryStrategyProvider.class)
    public void Should_return_employee_for_identifier(EmployeeRepositoryStrategy strategy) {

        var SUT = strategy.getSubjectUnderTest();

        var employee = new Employee(
                UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"),
                "Dwight", "Schrute",
                LocalDate.of(1966, 1, 20));
        SUT.save(employee);

        var retrievedEmployee = SUT.get(UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"));
        assertThat(retrievedEmployee).usingRecursiveComparison().isEqualTo(employee);
    }

    @ParameterizedTest
    @ArgumentsSource(EmployeeRepositoryStrategyProvider.class)
    public void Should_save_employee(EmployeeRepositoryStrategy strategy) {

        var SUT = strategy.getSubjectUnderTest();

        var newEmployee = new Employee(
                UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"),
                "Dwight", "Schrute",
                LocalDate.of(1966, 1, 20));
        SUT.save(newEmployee);

        var persistedEmployee = SUT.get(UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"));
        assertThat(persistedEmployee).usingRecursiveComparison().isEqualTo(newEmployee);
    }
}

Here we have three parameterised tests. Each test receives a particular EmployeeRepositoryStrategy instance when it gets executed. The tests themselves interact with the SUT through this specified interface. After the test has been executed, the test runner will automatically call the close method as it recognises that the parameter implements the AutoCloseable interface.

Notice that we provide an instance of the EmployeeRepositoryStrategyProvider as an argument source. The implementation of this factory class looks like this:

static class EmployeeRepositoryStrategyProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments> provideArguments(ExtensionContext extensionContext) {
        return Stream.of(
            Arguments.of(new SQLiteEmployeeRepositoryStrategy()),
            Arguments.of(new FakeEmployeeRepositoryStrategy())
        );
    }
}

The provideArguments method simply creates and returns an instance of the SQLiteEmployeeRepositoryStrategy class and the FakeEmployeeRepositoryStrategy respectively. Applying this provider as the parameter source ensures that each test is executed twice; once for the SQLiteEmployeeRepository and once for the FakeEmployeeRepository. The test runner will therefore execute six tests in total.

The advantage of using this approach is that we use composition over class inheritance as we no longer rely on subclasses. A disadvantage to this approach is that it’s slightly more complicated compared to Abstract Test Cases, which can be a debatable subject.

To conclude, I would like to thank Mario Pio Gioiosa for teaching me about the Parameterised Test Cases approach.

Contract Tests - Abstract Test Cases

2023-04-12T13:00:00Z

In the previous blog post, we’ve discussed the rationale behind contract tests. They are used for exercising those parts of an application that communicate with other parts of the system by crossing the process boundary. In this blog post, we’re going to have a look at how to implement contract tests in practice. The most common approach you’ll likely encounter is Abstract Test Cases .

With this approach, we write Sociable tests as one normally would write them. However, instead of adding them to a regular test class we’ll add them to an abstract base class instead. Then we derive a subclass for each implementation. A subclass thereby inherits all the test cases from the base class. In essence, this is the “Template Method” design pattern in action, where each test becomes a template method. Concrete subclasses implement primitive operation methods for creating and interacting with their respective Subject Under Test, either the real implementation or the fake implementation.

Let’s have a look at an example to demonstrate this. Although we’ve used Java in our example, the same pattern can be implemented in a similar way by using other object-oriented languages like C#, Python, Ruby, etc. …

The following example shows a Subject Under Test that is a repository for storing and retrieving employee data to and from a database. The contract tests for the EmployeeRepository look like this:

abstract class EmployeeRepositoryTests {

    private EmployeeRepository SUT;

    @BeforeEach
    public void setUp() {
        SUT = getSubjectUnderTest();
    }

    @AfterEach
    public void tearDown() {
        cleanup();
    }

    @Test
    public void Should_return_nothing_for_non_existing_employee() {

        var unknownId = UUID.fromString("753350fb-d9a2-4e4b-8ca4-c969ca54ef5f");
        var retrievedEmployee = SUT.get(unknownId);
        assertThat(retrievedEmployee).isNull();
    }

    @Test
    public void Should_return_employee_for_identifier() {

        var employee = new Employee(
            UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"),
            "Dwight", "Schrute",
            LocalDate.of(1966, 1, 20));
        SUT.save(employee);

        var retrievedEmployee = SUT.get(UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"));
        assertThat(retrievedEmployee).usingRecursiveComparison().isEqualTo(employee);
    }

    @Test
    public void Should_save_employee() {

        var newEmployee = new Employee(
            UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"),
            "Dwight", "Schrute",
            LocalDate.of(1966, 1, 20));
        SUT.save(newEmployee);

        var persistedEmployee = SUT.get(UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"));
        assertThat(persistedEmployee).usingRecursiveComparison().isEqualTo(newEmployee);
    }

    abstract EmployeeRepository getSubjectUnderTest();
    abstract void cleanup();
}

Notice that the EmployeeRepositoryTests class is abstract. It also defines two abstract methods: getSubjectUnderTest and cleanup. These abstract methods are being called by the setUp and tearDown method respectively, which in turn are executed before and after each test. The tests themselves interact with the SUT through the EmployeeRepository interface.

public interface EmployeeRepository {
    Employee get(UUID id);
    void save(Employee employee);
}

We’ve derived two concrete classes from EmployeeRepositoryTests, one for the real repository implementation (SQLiteEmployeeRepository) and one for the fake repository implementation (FakeEmployeeRepository). Both of these implement the EmployeeRepository interface.

The following piece of code shows the implementation of the SQLiteEmployeeRepositoryTests, which exercises the code of the SQLiteEmployeeRepository.

class SQLiteEmployeeRepositoryTests extends EmployeeRepositoryTests {

    private final NamedParameterJdbcTemplate jdbcTemplate;
    private final SQLiteEmployeeRepository sqliteEmployeeRepository;

    public SQLiteEmployeeRepositoryTests() {
        var database = getClass().getClassLoader().getResource("database.db");
        var connectionUrl = String.format("jdbc:sqlite:%s", database);

        var sqliteDataSource = new SQLiteDataSource();
        sqliteDataSource.setUrl(connectionUrl);

        this.jdbcTemplate = new NamedParameterJdbcTemplate(sqliteDataSource);
        this.sqliteEmployeeRepository = new SQLiteEmployeeRepository(jdbcTemplate);
    }

    @Override
    EmployeeRepository getSubjectUnderTest() {
        return sqliteEmployeeRepository;
    }

    @Override
    void cleanup() {
        jdbcTemplate.getJdbcOperations().execute("DELETE FROM Employee");
    }
}

The constructor initialises an instance of the SQLiteEmployeeRepository. This instance is returned by the implementation of the getSubjectUnderTest method. The cleanup method simply removes all records from the Employeetable in the database.

The implementation of the SQLiteEmployeeRepository itself looks like this:

public class SQLiteEmployeeRepository implements EmployeeRepository {

    private final NamedParameterJdbcTemplate jdbcTemplate;

    public SQLiteEmployeeRepository(NamedParameterJdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    @Override
    public Employee get(UUID id) {
        var sql = "SELECT Id, FirstName, LastName, BirthDate " +
            "FROM Employee " +
            "WHERE Id = :id";

        var parameters = Map.of("id", id);

        try {
            return jdbcTemplate
                .queryForObject(sql, parameters, SQLiteEmployeeRepository::mapEmployeeFromResultSet);
        } catch (EmptyResultDataAccessException e) {
            return null;
        }
    }

    private static Employee mapEmployeeFromResultSet(ResultSet resultSet, int rowNumber) 
        throws SQLException {

        return new Employee(
            UUID.fromString(resultSet.getString("Id")),
            resultSet.getString("FirstName"),
            resultSet.getString("LastName"),
            LocalDate.parse(resultSet.getString("BirthDate"))
        );
    }

    @Override
    public void save(Employee employee) {
        var sql = "INSERT INTO Employee (Id, FirstName, LastName, BirthDate) " +
            "VALUES(:id, :firstName, :lastName, :birthDate)";

        var parameters = Map.of(
            "id", employee.getId(),
            "firstName", employee.getFirstName(),
            "lastName", employee.getLastName(),
            "birthDate", employee.getBirthDate());

        jdbcTemplate.update(sql, parameters);
    }
}

The following piece of code shows the implementation of the FakeEmployeeRepositoryTests, which exercises the code of the FakeEmployeeRepository.

public class FakeEmployeeRepositoryTests extends EmployeeRepositoryTests {

    private final FakeEmployeeRepository fakeEmployeeRepository;

    public FakeEmployeeRepositoryTests() {
        fakeEmployeeRepository = new FakeEmployeeRepository();
    }

    @Override
    EmployeeRepository getSubjectUnderTest() {
        return fakeEmployeeRepository;
    }

    @Override
    void cleanup() {
        fakeEmployeeRepository.clear();
    }
}

Again, the constructor initialises an instance of the FakeEmployeeRepository. This instance is also returned by the implementation of the getSubjectUnderTest method. The cleanup method removes all data from the repository by calling the clear method. This method, in turn, simply clears the internal employees map as shown by the code of the FakeEmployeeRepository.

public class FakeEmployeeRepository implements EmployeeRepository {

    private final Map<UUID, Employee> employees;

    public FakeEmployeeRepository() {
        employees = new HashMap<>();
    }

    @Override
    public Employee get(UUID id) {
        return employees.get(id);
    }

    @Override
    public void save(Employee employee) {
        employees.put(employee.getId(), employee);
    }

    public void clear() {
        employees.clear();
    }
}

That’s it. When running the EmployeeRepositoryTests, the test runner will execute six tests; three tests for the SQLiteEmployeeRepository and the same three tests for the FakeEmployeeRepository.

To conclude, Abstract Test Cases are a quick and easy way to get started with contract tests. However, as the Gang of Four already expressed in their well-known book Design Patterns , we should favour composition over class inheritance. Following this principle brings us to another approach for implementing contract tests, which we’re going to discuss in the next blog post.

Contract Tests

2023-02-01T09:00:00Z

One of my favorite awesome talks of all time is Integrated Tests Are a Scam by J.B. Rainsberger. I must admit that the title has a clickbait vibe to it. However, I very much recommend watching the video.

So what’s the deal with integrated tests? Why should they be considered as a scam? First of all, let’s clarify what an integrated test actually is. An integrated test is a high-level test that exercises multiple different behaviours of the system all together in one go. At first glance this doesn’t sound like a huge problem. We need tests like that, don’t we? Well, it kind of depends on the audience of the integrated test.

Remember from my previous post about the Testing Quadrant, that there are two quadrants of tests that support the team:

Developer tests: this is the quadrant where the Test Pyramid resides.
Functional tests: these are the kind of tests, as described by Behaviour-Driven Development (BDD) or Acceptance Test-Driven Development (ATDD), that prove to business stakeholders that certain features are available in the system.

When it comes to using integrated “functional” tests for providing the necessary confidence to business stakeholders, there’s not much of a problem there. Such tests often do require different parts of the system working together to provide the requested capabilities.

However, an issue arises when software developers start using integrated “developer” tests for proving the correctness of the system. So why do such integrated tests become an issue in this situation? For all the same reasons as when having too much Sociable tests:

They can be grudgingly slow, which results in a poor feedback cycle and a long-running CI pipeline.
They are quite often very non-deterministic. This undermines trust and confidence.
They are overly verbose as they require a lot of code to set up the system under test. This requires a serious investment when it comes to time and effort.
They don’t provide useful feedback about the design of the system.
They are highly susceptible to cascading failures.

Therefore, my advice, ad nauseam, is to adhere to the Test Pyramid as it’s still a highly valuable model to think about when it comes to the developer tests quadrant.

What about integration tests? Are those a scam as well? It kind of depends on how you use them. An integration test, as opposed to an integrated test, only verifies whether the different components for a single path running through the system correctly work together. If a sociable test is used for that purpose alone, then things are just fine. However, troubles usually start appearing the moment software developers decide to start using Sociable tests, or even worse integrated tests, as the sole means to verify correctness of the system at the expense of Solitary tests.

“ Strong integration tests, consisting of collaboration tests (clients using test doubles in place of collaborating services) and contract tests (showing that service implementations correctly behave the way clients expect) can provide the same level of confidence as integrated tests at a lower total cost of maintenance.

— J.B. Rainsberger - Clearing Up the Integrated Tests Scam

What J.B. suggests here is the use of contract tests for exercising those parts of the application that cross the process boundary to communicate with other parts of the system like a database, a queue, the file system, etc. … The purpose of a contract test is to verify whether an adapter for a collaborating service correctly implements a particular interface. This implies that the same suite of tests verifies the outside-observable behaviour of both the real implementation and a fake implementation.

In the context of an hexagonal architecture, this suite of contract tests ensures that the fake adapter behaves exactly the same as the real adapter. We can then use this fake implementation for Sociable “integration” tests that verify whether the different components of the application correctly work together.

In the next blog post we’ll discuss how to approach contract tests from a practical point of view. In the meantime, you can check out the Integrated Tests Are a Scam blog series to learn more first-hand.

The Testing Quadrant

2022-06-15T08:00:00Z

Back in 2003, Brian Marick wrote an excellent article series about agile testing. There he described the concept of the Testing Quadrant. To my recollection this used to be a well-know model that most software developers were familiar with. Although still very much relevant today, I’ve noticed over the years that the knowledge about the different types of testing as described by the Testing Quadrant somewhat got lost.

The four quadrants as shown in the diagram above each match a specific perspective of testing. These are:

At the bottom left: Developer tests. This is the quadrant where the test pyramid lives. These tests are typically automated.
At the top left: Functional tests. These are customer story tests as described by Behaviour-Driven Development (BDD) or Acceptance Test-Driven Development (ATDD). These tests are usually automated.
At the top right: Exploratory tests and usability tests. By definition these are manual tests.
At the bottom right: Performance tests, load tests, security tests, … etc. These tests are mostly semi-automated, semi-manual.

Now that we’ve named the four quadrants, let’s have a look at their vertical and horizontal position in the quadrant.

The top two quadrants are concerned with tests that are obvious to business stakeholders. These tests should indicate whether the system actually provides a solution to a business problem. The bottom two quadrants on the other hand are meaningful for technical people.

“ We will write tests before we code, minute by minute. We will preserve these tests forever, and run them all together frequently. We will also derive tests from the customer’s perspective.

— Kent Beck - Extreme Programming Explained (1st Edition - Chapter 18)

The left two quadrants are focused on preventing defects before and during the coding process. This matches the quote from the opening paragraph of chapter 18 in Kent Beck’s excellent book Extreme Programming Explained (1st Edition) . In essence, by writing tests before writing the code, we build in a certain quality into the system right from the start. These kinds of tests therefore prevent defects instead of merely finding them.

The right two quadrants are focused on evaluating for defects or finding missing features. Note that this kind of inspection is still necessary. These two quadrants merely reflect whether we have a low or high quality system.

Most software systems will involve tests from across all four quadrants. Also, testing shouldn’t be a single person’s
responsibility. The quality of a software system is in fact a team effort. Every team member should be involved in these activities related to the four quadrants. Please note that the Testing Quadrant doesn’t prescribe any order of tests, nor the importance of certain quadrants and definitely not the amount of tests for each quadrant.

“ All models are wrong, but some are useful.

— George Box

The Testing Quadrant is a useful model to help software teams plan their testing efforts, while also creating awareness that it’s generally a good idea to cover all four bases.

Tales Of TDD: The Big Refactoring

2022-02-02T09:00:00Z

I’ve heard that you’ve been working on this new feature for the payroll system based on the newly voted government legislation?

Yes, I am.

How is it going? Seems like a major change.

Well, I’m glad you asked. I’ve refactored a large part of the domain based on some new insights I’ve had. This morning I’ve pushed all the code to the git repository. It took me almost two weeks to make the necessary changes.

Great!

However, I could use some help though.

Sure thing. How can I help?

You see, in order to refactor the code base I commented out most of the tests. I just hate it when they get in the way. Anyway, can you fix these tests? There shouldn’t be more than 350 of them. That way I’m able to start working on the next user story.

Uhm, … that’s … uhm … interesting.

What’s interesting?

A couple of things come to mind.

OK. Tell me.

For starters, you mentioned the term refactoring a couple of times. I’m interested in the process that you call refactoring.

A process?

Yes. How did you take on this endeavour?

Well, I did what I always do. I started out by changing a bunch of existing methods on several classes, adding and removing parameters, moving code around, etc. … I’ve also added a couple of new classes. Just the usual stuff.

How soon did you execute the tests?

After a couple of hours I wanted to compile the source code. Then I noticed that a bunch of the tests didn’t compile anymore. There were just too many, so I’ve put them in comments. You know, this bothers me the most about tests. They prevent me from refactoring the code. Maybe we should just get rid of these unit tests. We still have the acceptance tests, right?

So you didn’t run the tests?

No. They didn’t even compile.

Did you execute the acceptance tests?

Are you kidding me? They take more than two hours to run. I don’t have time for that. Besides, that’s why we pay the QA folks. It’s their job to make sure that the acceptance tests run smoothly. They’re also a mess. I won’t go anywhere near those.

In summary, you’ve been “restructuring” code for two weeks without executing any kind of automated tests?

I’m afraid so. I could have done a better job on that department, which reminds me to try running the application later on to see if it still works. But I’m curious, you just said “restructuring the code”. Isn’t that the same as refactoring?

Refactoring is a form of restructuring, but they aren’t necessarily the same thing. Refactoring is making small adjustments to the code without altering its observable behaviour. By adding up all these small changes you basically end up with a bigger change.

I don’t get it. That’s exactly what I’ve been doing. It’s not that I changed thousands of lines of code at once.

There are some very important aspects that I didn’t mention yet.

Ok. What are those aspects then?

The most important aspect of all when it comes to refactoring is to execute the tests after each and every change, no matter how small. That way you know that the system still works as expected. Whenever a test fails, you also know exactly where to find and fix the issue.

I’ve tried that once while doing a code kata with a bunch of people. It made me feel ridiculous. I mean, the code changes became so small that it was almost impossible to mess them up.

That’s what you want, right?

I don’t know. Maybe. But who has time for such things?

You just mentioned that you want to try running the application to see if most of the features still work, right?

Yes

What are you going to do when you encounter something that doesn’t work?

In that case, I’ll try to find the code for that broken feature, set out some breakpoints and start debugging to see what’s going on.

Do you have time for that?

Uhm, … I guess so.

More often than not, a debugging session can easily waste a lot of time. Alternatively, by enabling a tight feedback loop we keep ourselves out of such a mess.

Suppose that I only make small changes as you suggest, and I’m unable to find the issue whenever a test fails. Then I still have to debug the code right?

Another important aspect is to also commit the code changes after each cycle. That way you’ll be able to revert to the last good commit and start over, only now using even smaller steps.

So you’re saying that besides making only tiny changes, I have to commit those changes as well? Isn’t that a lot of overhead?

Not if you want to have a fast feedback loop in place.

Who ever said that I want to have a feedback loop?

Well, didn’t you just mention that you wanted to move on to the next user story?

Yes

Why’s that? You didn’t even finish your current user story.

I just want to work on something else. I’m kind of fed up with the current user story.

You could have moved on if you’d only made small refactorings, ensured that all the tests still passed and committed the code after each short cycle.

I don’t understand.

Well, if you’d rigorously followed this short feedback cycle that I’ve been describing, then the code would always be in a releasable state, wouldn’t it?

Yeah, maybe that’s true.

Be quick but don’t hurry.

You always say things like that. I still have one question though.

Shoot!

What if I don’t want to refactor a particular part of the code? What if I just want to replace it with some entirely new implementation? Surely I can’t fall back to having a short feedback cycle in that case, right?

When you want to make changes on a larger scale, you can use the “Branch by Abstraction” technique.

What’s that all about?

Going back to your specific case, you develop the new part of the domain just as you would implement it from scratch. This way you can flesh out the design of the new code using Test-Driven Development alongside the existing implementation. When this is finished, you can gradually replace the old implementation with the new implementation at the “seams”.

What’s a seam? I never heard about this in the context of software development before.

This is a term that Michael Feathers uses in his excellent book Working Effectively with Legacy Code . According to his definition, a seam is a place where you can alter behavior in your program without editing in that place. This involves that some kind of abstraction layer is in place that is used by the client code. This abstraction layer enables you to swap out the old implementation with the new implementation.

That way I can work on the new implementation using short feedback cycles. I’ll be able to commit the code without affecting the existing implementation, while also keeping the application in a deployable state.

Exactly.

I’ll have to try that some time for one of the next user stories.

Let me know how that works out.

I will. Now, about those 350 failing tests …

Implementing Approval Tests For PDF Document Generation

2021-12-15T09:00:00Z

In the previous blog post, we discussed how to use Approval Tests for verifying generated PDF documents. In this blog post I’m going to show how to extend the Approval Test library for Java in order to support PDF documents. Let’s just dive right into the code.

The first thing that needs to happen is making a new implementation of the ApprovalApprover interface, which is provided by the Approval Test library. The following code demonstrates how this can be implemented.

public class PdfFileApprover implements ApprovalApprover {

    public static final Field<String> PDF_DIFF_OUTPUT_DIRECTORY =
        new Field<>("PdfDiffOutputDirectory", String.class);

    private final ApprovalNamer namer;
    private final ApprovalWriter writer;

    private final double allowedDiffInPercent;
    private final List<PageArea> excludedAreas;

    private File received;
    private final File approved;

    public PdfFileApprover(ApprovalWriter writer, PdfFileOptions options) {
        this.writer = writer;
        this.allowedDiffInPercent = options.getAllowedDiffInPercent();
        this.excludedAreas = options.getExcludedAreas();

        namer = options.getParent().forFile().getNamer();

        received = namer.getReceivedFile(writer.getFileExtensionWithDot());
        approved = namer.getApprovedFile(writer.getFileExtensionWithDot());
    }

    public VerifyResult approve()
    {
        received = writer.writeReceivedFile(received);
        return approvePdfFile(received, approved);
    }

    public void cleanUpAfterSuccess(ApprovalFailureReporter reporter)
    {
        received.delete();
        if(reporter instanceof ApprovalReporterWithCleanUp) {
            ((ApprovalReporterWithCleanUp) reporter)
                .cleanUp(received.getAbsolutePath(), approved.getAbsolutePath());
        }
    }

    public VerifyResult reportFailure(ApprovalFailureReporter reporter)
    {
        reporter.report(received.getAbsolutePath(), approved.getAbsolutePath());
        if (reporter instanceof ReporterWithApprovalPower)
        {
            ReporterWithApprovalPower reporterWithApprovalPower = (ReporterWithApprovalPower) reporter;
            return reporterWithApprovalPower.approveWhenReported();
        }

        return VerifyResult.FAILURE;
    }

    public void fail()
    {
        throw new Error(String.format("Failed Approval\n  Approved:%s\n  Received:%s", 
            approved.getAbsolutePath(), received.getAbsolutePath()));
    }

    private VerifyResult approvePdfFile(File received, File approved) {
        try {
            SimpleEnvironment environment = new SimpleEnvironment();
            environment.setAllowedDiffInPercent(this.allowedDiffInPercent);

            PdfComparator<CompareResultImpl> pdfComparator = new PdfComparator<>(approved, received)
                .withEnvironment(environment);

            excludedAreas.forEach(pdfComparator::withIgnore);

            CompareResultImpl comparisonResult = pdfComparator.compare();
            if(comparisonResult.isNotEqual()) {
                String outputFileName = determineDiffOutputFileName();
                comparisonResult.writeTo(outputFileName);
            }

            return VerifyResult.from(comparisonResult.isEqual());
        } catch (IOException e) {
            return VerifyResult.FAILURE;
        }
    }

    private String determineDiffOutputFileName() {
        var outputDirectory = PackageLevelSettings.getValueFor(PDF_DIFF_OUTPUT_DIRECTORY);
        if(null == outputDirectory) {
            outputDirectory = namer.getSourceFilePath();
        }

        return Path.of(outputDirectory, "diffOutput").toString();
    }
}

The PdfFileApprover class provides the core implementation for supporting PDF documents. Most of the code is quite similar to the FileApprover class of the Approval Test library. However, the approvePdfFile method is the most important part. This method expects two arguments; the received PDF file and the approved PDF file. The purpose of the approvePdfFile method is to compare both incoming PDF files. For doing the actual comparison we make use of the PDFCompare library .

When there’s a difference between these PDF files, we save the result of the comparison to a PDF file so that we can visually inspect the differences as well as instruct the framework to fail the test. Also notice that the PDFCompare library has the ability to exclude certain areas within a PDF file from the comparison as well as allowing a certain percentage of differences.

public class PdfApprovals {

    public static void verify(ByteArrayOutputStream outputStream) {
        verify(outputStream, PdfFileOptions.DEFAULT_ALLOWED_DIFF_IN_PERCENT, Collections.emptyList());
    }

    public static void verify(ByteArrayOutputStream outputStream, List<PageArea> excludedAreas) {
        verify(outputStream, PdfFileOptions.DEFAULT_ALLOWED_DIFF_IN_PERCENT, excludedAreas);
    }

    public static void verify(ByteArrayOutputStream outputStream, double allowedDiffInPercentage) {
        verify(outputStream, allowedDiffInPercentage, Collections.emptyList());
    }

    public static void verify(ByteArrayOutputStream outputStream, double allowedDiffInPercentage, 
                              List<PageArea> excludedAreas) {
        ByteArrayInputStream inputStream = new ByteArrayInputStream(outputStream.toByteArray());
        ApprovalBinaryFileWriter binaryFileWriter = new ApprovalBinaryFileWriter(inputStream, "pdf");

        PdfFileOptions options = new PdfFileOptions()
            .withAllowedDiffInPercent(allowedDiffInPercentage)
            .withExcludedAreas(excludedAreas);

        PdfApprovals.verify(binaryFileWriter, options);
    }

    private static void verify(ApprovalWriter writer, PdfFileOptions options) {
        PdfFileApprover pdfFileApprover = new PdfFileApprover(writer, options);
        Approvals.verify(pdfFileApprover, options.getParent());
    }
}

The PdfApprovals class provides a number of static helper methods that can be used by the tests themselves. These helper methods ultimately use the PdfFileApprover class to perform the actual comparison. A number of overloaded methods are available to provide the ability of excluding certain areas and/or tweaking the allowed percentage of differences when performing the comparison.

These can be used as follows:

PdfApprovals.verify(result);

...

PdfApprovals.verify(result, 0.18);

...

var excludedAreas = Arrays.asList(
    new PageArea(4, 4, 12, 18),
    new PageArea(35, 38, 42, 45)
);
PdfApprovals.verify(result, excludedAreas);

...

PdfApprovals.verify(result, 0.18, excludedAreas);

For completeness, the following code shows the implementation of the PdfFileOptions and PackageSettings classes. These are necessary to provide the PdfFileApprover class with the necessary configuration settings for performing the comparison and saving the output files.

public class PdfFileOptions {

    public static final double DEFAULT_ALLOWED_DIFF_IN_PERCENT = 0.001;

    private enum CustomFields {
        ALLOWED_DIFF_IN_PERCENT,
        EXCLUDED_AREAS
    }

    private final Map<CustomFields, Object> customFields;
    private final Options options;

    public PdfFileOptions() {

        customFields = new HashMap<>();
        customFields.put(CustomFields.ALLOWED_DIFF_IN_PERCENT, DEFAULT_ALLOWED_DIFF_IN_PERCENT);
        customFields.put(CustomFields.EXCLUDED_AREAS, Collections.emptyList());

        options = new Options();
        options.forFile().withExtension(".pdf");
    }

    public double getAllowedDiffInPercent() {
        return (double) customFields.get(CustomFields.ALLOWED_DIFF_IN_PERCENT);
    }

    public List<PageArea> getExcludedAreas() {

        return (List<PageArea>) customFields.get(CustomFields.EXCLUDED_AREAS);
    }

    public Options getParent() {
        return options;
    }

    public PdfFileOptions withAllowedDiffInPercent(double allowedDiffInPercent) {
        customFields.put(CustomFields.ALLOWED_DIFF_IN_PERCENT, allowedDiffInPercent);
        return this;
    }

    public PdfFileOptions withExcludedAreas(List<PageArea> excludedAreas) {
        customFields.put(CustomFields.EXCLUDED_AREAS, excludedAreas);
        return this;
    }
}

public class PackageSettings {
    public static String ApprovalBaseDirectory = "../resources";
    public static String PdfDiffOutputDirectory = 
        String.format("%s/build/tmp", System.getProperty("user.dir"));
}

That’s all there is to it. With just this tiny bit of code we’re able to use Approval Tests for PDF documents.

Approval Tests For PDF Document Generation

2021-10-07T08:00:00Z

A while back I was confronted with a part of a legacy system that generates PDF documents. This legacy system used a well known library for generating the requested PDF files. The good news was that there were a decent amount of tests available. The not so good news was that these tests made heavy use of test doubles for swapping out most of the types provided by the third-party API. I strongly believe that using test doubles in such cases is not a good choice. In the past I already wrote about why to avoid using test doubles for types that you don’t own.

Tests like these might become a large impediment whenever we upgrade to a newer version of the third-party library. Major versions quite often introduce breaking changes to existing API’s. Whenever tests are strongly coupled to such an API, they basically need to be rewritten during the upgrade.

In order to reduce the coupling of these tests, I decided to replace them with Approval Tests instead. This technique is also known as “Golden Master” or “Characterization Test”. The idea behind an Approval Test is that after the first test run, some output needs to be visually verified and approved. During subsequent test runs, the approved output will be compared to current output. When there’s a difference in the output, the test will fail.

This is especially useful whenever we have to deal with code of a legacy system. You can check out this video from Emily Bache where she demonstrates the Gilded Rose refactoring kata using Approval Tests . Highly recommended!

Usually the output of Approval Tests is captured in plain text files, which has nothing to do with PDF files. So I decided to extend the Java Version of an Approval Test library to support PDF documents as well. Let’s have a look at some example code to demonstrate this extension.

Suppose that we have a small application that generates a PDF document. A generated document contains the refrain of a well-known song lyric. The following code shows a possible implementation.

public class SingAlongPdfGenerator {

    public ByteArrayOutputStream generate(SingAlongData pdfData) {

        var outputStream = new ByteArrayOutputStream();
        PdfDocument pdf = new PdfDocument(new PdfWriter(outputStream));

        Document document = new Document(pdf);
        document.add(new Paragraph("Let's sing-a-long:"));

        List list = new List()
            .setSymbolIndent(12)
            .setListSymbol("\u2022");

        for(var refrainLine : pdfData.getRefrainLines()) {
           list.add(new ListItem(refrainLine));
        }

        document.add(list);
        document.close();

        return outputStream;
    }
}

public class SingAlongData {

    private final List<String> refrainLines;

    public SingAlongData(List<String> refrainLines) {

        this.refrainLines = refrainLines;
    }

    public List<String> getRefrainLines() {
        return refrainLines;
    }
}

The SingAlongPdfGenerator class provides a method named generate that accepts a SingAlongData instance as its only parameter. The SingAlongData class is merely a DTO that provides a list of refrain lines. The generate method creates a new document containing a paragraph of text and a list of the refrain lines.

Let’s have a look at the code of the corresponding Approval Test.

public class SingAlongPdfGeneratorTests {
    
    @Test
    public void generateRickRollPdf() {

        var refrainLines = Arrays.asList(
            "Never gonna give you up",
            "Never gonna let you down",
            "Never gonna run around and desert you",
            "Never gonna make you cry",
            "Never gonna say goodbye",
            "Never gonna tell a lie and hurt you"
        );
        var data = new SingAlongData(refrainLines);

        var pdfGenerator = new SingAlongPdfGenerator();
        var result = pdfGenerator.generate(data);

        PdfApprovals.verify(result);
    } 
}

First we create an instance of the SingAlongData DTO, specifying some test data. Next we create an instance of the Subject Under Test, which in this case is the SingAlongPdfGenerator class and call the generate method. This returns a ByteArrayOutputStream containing the data of the PDF document. Then we verify the result by calling the PdfApprovals.verify method. Notice that the anatomy of an Approval Test is identical to any other type of test as it also adheres to the Arrange, Act, Assert pattern.

A new Approval Test always fails the very first time that it gets executed. After the initial test run, a PDF file is generated that needs to be visually verified and approved. So when we first run the generateRickRollPdf test, a file with the name SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf appears which has the following content:

After we verified the PDF document, we approve it using the following command:

mv ~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf 
~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf

At this point we have a PDF file named SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf. When we now execute our test again, it passes as the newly generated PDF document matches the approved PDF document.

Note that we also have to make sure to commit the approved PDF file alongside our test code. Otherwise, we have to approve the output of the test again when executed on another machine.

Let’s say that we want to make a change to the implementation of our SingAlongPdfGenerator class. For example, we’re going to make a small change to the text inside the paragraph.

document.add(new Paragraph("Let's sing-a-long shall we?"));

When we execute our test again, it fails due to the change that we’ve made.

Failed Approval
  Approved:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf
  Received:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf

The test created a “received” PDF file again alongside the existing PDF file that we’ve approved earlier. We now have to visually compare the received and the approved file side-by-side. Needless to mention that this is going to be quite cumbersome. For this reason I’ve added the capability that in case of failing test, a third PDF file is generated that contains the annotated differences between the two PDF files.

The purple markers indicate where the changes are located in the document. The green colon indicates what is expected (approved). The red text that we’ve added to the paragraph shows the actual text found in the document (received). We now have to decide whether we want to approve the changes that we’ve made or not. Let’s say that we are happy with the change that we’ve made. To do that we simply run the approval command again as shown earlier. And that’s it.

With just a handful of these Approval Tests I was able to eliminate all the tightly coupled tests. Generating PDF files is more often than not an infrastructure concern. Sociable tests tests are much more appropriate in this case compared to solitary tests. Using Approval Tests this way turned out to be a very valuable approach.

Ignoring Tests

2021-06-22T08:00:00Z

Most test frameworks out there have the capability to disable tests. This is usually done by adding some kind of annotation that instructs the test runner to ignore an individual test method.

[TestFixture]
public class SomeTests {

    [Test]
    public void RegularTest() 
    {
        ...
    }
    
    [Test]
    [Ignore]    // Ignore a single test
    public void IgnoredTest() 
    {
        ...
    }
}

It’s also possible to disable all tests of an entire test fixture by adding an annotation at the class level.

[TestFixture]
[Ignore]    // Ignore all tests
public class IgnoredTests {

    [Test]
    public void IgnoredTest() 
    {
        ...
    }
    
    [Test]
    public void AnotherIgnoredTest() 
    {
        ...
    }
}

A while back, I came to realise that I never use this kind of functionality to prevent the execution of tests. I became aware of this when I started observing other people’s development workflow. For some developers, being able to ignore tests seems to be a very important feature of a test framework. I’ve come to believe that the need for ignoring tests is more prevalent in a Test-After approach than with a Test-First approach. Let’s have a look at a very simple code example.

public class Task
{
    public Person Assignee { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }
}

Suppose that we’ve built an application that provides an online Kanban board service to our customers. In the domain of this application there’s a class named Task. Currently, a task can be assigned to a particular person. This is how the test for the Assign method looks like.

[TestFixture]
public class TaskTests
{
    [Test]
    public void It_should_be_able_to_assign_a_person_to_a_task()
    {
        var person = new Person("Joe");
        
        var SUT = new Task();
        SUT.Assign(person);
        
        Assert.That(SUT.Assignee, Is.SameAs(person));
    }
}

These days, being able to assign only a single person to a task is a relic from the past. Therefore, in light of everything “ensemble”, the business requested whether it would be possible to assign multiple people to a single task.

Let’s see how a Test-First approach would look like. Obviously, we would start by writing a failing test. So we’ll add the following test method to the TaskTests test fixture.

[Test]
public void It_should_be_able_to_assign_multiple_people_to_a_task()
{
    var people = new[]
    {
        new Person("Joe"),
        new Person("Annie")
    };
    
    var SUT = new Task();
    SUT.Assign(people);
}

This new code doesn’t compile at this point as the Assign method currently accepts only a single Person object instead of a collection. This is a good thing because a test method that doesn’t compile qualifies as a failing test. Let’s fix this by adding a new overload of the Assign method.

public class Task
{
    public Person Assignee { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }
    
    public void Assign(IEnumerable<Person> peopleToAssign)
    {
        
    }
}

We’re going to leave the implementation of this new method empty for the time being. Now the code of the application compiles again. Also, all the tests pass when we run them. We have working software again. However, it’s still not very useful as we didn’t completely finish the implementation of the new test method. An assert statement is still missing, so we’re going to add that as our next step.

[Test]
public void It_should_be_able_to_assign_multiple_people_to_a_task()
{
    var people = new[]
    {
        new Person("Joe"),
        new Person("Annie")
    };
    
    var SUT = new Task();
    SUT.Assign(people);

    Assert.That(SUT.Assignees, Is.SameAs(people));
}

By adding the assert statement we find out that the code doesn’t compile anymore. This is a good thing because a test method that doesn’t compile qualifies as a failing test. In order to fix this, we’re need to add an Assignees property to the Task class.

public class Task
{
    public Person Assignee { get; private set; }
    public IEnumerable<Person> Assignees { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }
    
    public void Assign(IEnumerable<Person> peopleToAssign)
    {
        
    }
}

Now we’re able to compile the code again. However, when we run all the tests again we find out that the assert statement that we’ve just added fails the test. In order to make this test pass, we need to add the necessary implementation to the newly overloaded Assign method.

public class Task
{
    public Person Assignee { get; private set; }
    public IEnumerable<Person> Assignees { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }
    
    public void Assign(IEnumerable<Person> peopleToAssign)
    {
        Assignees = peopleToAssign;
    }
}

When we run the test suite again, we notice that all the tests are green. We have working software again.

The rest of the code base still uses the Assign method that accepts a single Person object. So our next step is to migrate all the client code of the Task class to use the new version of the Assign method. We go about this migration by writing failing tests and making them pass by using the Assign method that accepts a collection of Person objects. When the original Assign method has been fully replaced, we can remove the corresponding test as well as the method itself.

Notice that during this workflow, we were always able to run all the tests without ignoring any existing ones. Let’s compare the Test-First approach to a Test-Last approach using the same example. We’ll immediately start off by making the necessary changes in the Task class.

public class Task
{
    public IEnumerable<Person> Assignees { get; private set; }

    public void Assign(IEnumerable<Person> peopleToAssign)
    {
        Assignees = peopleToAssign;
    }
}

We’ve renamed the Assignee property to Assignees while we also changed its type to IEnumerable<Person>. Likewise, we’ve changed the parameter of the Assign method to IEnumerable<Person> peopleToAssign. By making this change the way we did, we broke the contract of the Task class. The result is that the code doesn’t compile anymore.

First, the code of the test that verifies the Assign method needs to be changed. We complain a bit to our colleagues about how tests are holding us back while we’re developing code. After we’re done complaining, we comment out the line SUT.Assign(person); and slap the Ignore attribute on the test method. We tell ourselves that we’re going to fix this test later.

[TestFixture]
public class TaskTests
{
    [Test]
    [Ignore("Meh")]
    public void It_should_be_able_to_assign_a_person_to_a_task()
    {
        var person = new Person("Joe");
        
        var SUT = new Task();
        //SUT.Assign(person);
        
        Assert.That(SUT.Assignees, Is.SameAs(person));
    }
}

Next, we find out that there are other places in the production code that need attention as well. There could be a single caller of the Assign method, or there could be several places throughout the code base that call this method. Suppose that there are three places in the code that need to be altered to reflect the changes that we’ve made. After adding some additional Ignore attributes for disabling some other tests, everything compiles now.

Suppose that we’re now dragged into a meeting. When the meeting has ended, we (conveniently) forgot about the ignored tests. After all, we want to continue working on our feature. “Stressed and always in a hurry”.

Notice that after this endeavour we don’t even have a clue whether we have working software or not. Please note that the simple example that has been presented here in and of itself is not important. We could have made the necessary changes using the refactoring capabilities of any modern IDE. That’s not the point. The point is the difference between both workflows.

Whenever we feel the need to disable the execution of a test, we might want to consider finding a better approach. Tests should not be there to stand in our way. Rather they can help guide us throughout the development process. Test-Driven Development is not only about writing tests first. It’s about being able to work in very short iterations. The important part is that by the end of every iteration, we have code that works. Writing tests first is just the means that guides us through that workflow.

Writing Maintainable Unit Tests In Print

2021-04-23T08:00:00Z

I’m very happy to announce that Writing Maintainable Unit Tests is also available as a paper book. Both the ebook and the paper book have been completely self-published. I must say that this whole endeavour was quite an educational experience.

The ebook is still available on LeanPub. The paper book can be purchased at most leading book stores:

For more information, you can have a look at the full table of contents.

Tales Of TDD: The Case Of Overused Test Doubles

2021-03-31T12:00:00Z

Do you have 15 minutes to spare?

Sure, what can I do for you?

Well, you know I’ve been working on that new feature we were discussing at the whiteboard last week. I’ve finished the implementation of that user story. All the automated tests are green, the CI build has completed successfully, and I’ve deployed the latest version on the test environment where Luke and I already did some exploratory testing.

Sounds good.

So I was wondering whether we could do a code review together? I’ll walk you through the code, if you have some time to spare off course.

Sure thing. Show me the code!

During our design session we talked about integrating a third-party REST service for retrieving quote prices.

Yes, I remember that discussion.

So, as we agreed back then, I’ve put all the code for calling this REST service in an adapter class.

Perfect! I also see that you’ve defined and exposed your own custom data objects.

Yes indeed. I didn’t want to expose the data objects of the REST service itself. When the API of the REST service gets changed by the host, we only have to modify the implementation of the adapter class. By using separate data objects for the interface of the adapter, we prevent that those kinds of third-party API changes ripple through the rest of the system.

Well done.

Let me quickly show you the unit tests for the adapter.

Interesting …

These are the test scenarios that cover every path through the code of the adapter. I did spend quite some time figuring out how the REST client library that we’re using is actually implemented.

How so? I heard from Sarah that she used this same library a couple of months ago. She mentioned how easy it was to use this REST client library.

She’s right! It’s definitely quite easy to use. This part of the code in the adapter class demonstrates this. You can see that it only requires just a few lines of code. However, I did have a lot of difficulties to get the proper stubs in place in order to get the unit tests running green.

Can you show me the code for this?

Sure. I’ve refactored all this nasty setup code into a separate method inside the test fixture class. This method is then called by all the test methods for setting up the correct stub values.

The implementation of that method sure looks very complicated. I also notice that you’ve used a lot of test doubles.

Yes, *sigh*. I did have to create a lot of test doubles. You see, the API of the REST client library defines a domain-specific language. That meant that I had to set up a test double that returns other test doubles for every method being used. This was needed in order to avoid null reference exceptions, and also being able to return a list of quote prices.

That looks quite problematic, don’t you think?

I agree that it did require quite some effort. However, why do you consider this to be problematic? Everything went smooth once I figured out how to set up all these test doubles to work with the DSL. Also, the tests execute without too much hassle.

I’m quite sure these unit tests are running just fine. However, can I ask you this? After you’ve cobbled together the implementation of the adapter, did you have to make any changes when you first tested this implementation against the real REST service?

Well yes, quite a few actually. I did have to specify some additional headers and a couple of other things related to authentication.

You also needed to make the necessary changes to the unit tests then?

Yes, I did. At first there weren’t that many tests doubles compared to what I’ve ended up with.

I’m willing to believe that’s true. Here’s the thing. Suppose I’d tell you that this REST client library is no longer being maintained anymore and that it needs to be replaced with another library. What would be the implications for the code of the adapter class?

Well, then we obviously need to replace these couple lines of DSL code with something else. That shouldn’t be that difficult.

What would happen to the unit tests of the adapter?

Well I guess that these tests need to be completely rewritten from scratch.

Bingo!

Now I’m a bit confused. Why should that be a problem? In your hypothetical case we’ve switched to another REST client library after all, right? Doesn’t that automatically imply that these unit tests should be changed as well?

These unit tests in their current form need to be changed indeed. However, the actual reason for this change is that they’re too tightly coupled to the code structure of the adapter class. You see, using a REST client library is an implementation detail of the adapter class. It shouldn’t matter to the tests which library is being used and how it’s API is shaped. When we decide to replace this REST client library with another one, then the tests shouldn’t be modified at all. You know why?

Why?

Because they should tell us whether the behaviour of the adapter is still exactly the same as before we swapped out the REST client library. We need these tests as a fall back in order to verify whether the adapter still works as it’s supposed to.

But then they’re no longer unit tests. Do you want to replace them with integration tests?

Yes indeed, and that’s OK. Unit tests don’t make much sense for production code that lives at the edge of a software system. The adapter class you’ve created is located at the boundary of our application. Integration tests are very much warranted here. In fact, how would we otherwise know for certain whether things work correctly without actually carrying out HTTP calls?

I see your point. How can we move forward then?

Well, for starters, let’s remove all of these test doubles. Then we’ll refactor the tests by decoupling them from the implementation of the adapter.

— After a short while —

I must say that the tests now look a lot better than before.

Indeed. The most important thing is that they now fail for the right reason. At least one of these tests will turn red as soon as there’s a change in the behaviour of the adapter, not when its implementation gets changed.

I wish I had found out about this sooner.

Your original unit tests actually warned you about this.

How so?

You mentioned earlier that it was quite difficult to write unit tests for this adapter, right? That was in fact a first warning sign. You also talked about test doubles returning other test doubles because the API of the REST client library is a DSL. That was a second warning sign. Also, the REST client library itself is not being maintained by our team. Using test doubles for types that we don’t own is your third warning sign. And last but not least, just don’t use test doubles for I/O operations.

That were a lot of warning signs indeed. Maybe I just needed more coffee at the time I wrote the tests?

Good idea! Let’s grab ourselves a drink.

Book Launch: Writing Maintainable Unit Tests

2021-02-17T10:00:00Z

I’m very happy to announce that I’ve finished writing my first book Writing Maintainable Unit Tests. What an amazing journey it has been. I would never have thought that I would be able to write a book. Well now, here we are.

The ebook has been published on LeanPub, which is available in PDF, EPUB and MOBI formats. There are six chapters:

Chapter 1 provides an overview of the different kinds of automated tests and how to apply a healthy mix in a code base. It also touches on the different flavours of Test-Driven Development.
Chapter 2 describes the characteristics and principles that make tests maintainable. It touches on a number of design principles like DRY, DAMP and the Single-Responsibility Principle.
Chapter 3 discusses the anatomy of automated tests and how a good structure is essential to keep them readable for our fellow software developers.
Chapter 4 demonstrates a number of patterns and techniques to keep tests decoupled from the production code. This is the longest chapter of the book.
Chapter 5 shows a number of patterns and techniques for writing clear assertions and observations.
Chapter 6 touches on some miscellaneous principles that are useful for writing maintainable and readable tests.

For more information, you can have a look at the full table of contents.

I’ve learned a lot over these past couple of years. It’s my hope that others can learn something as well from reading this book. As always, any and all feedback is very much appreciated.

Avoid Inheritance For Test Classes

2021-01-06T09:00:00Z

Using inheritance for test classes is not a desirable thing as it introduces a number of issues. An abstract base class quite often originates from the desire to share and reuse some code with a number of derived test classes. Maintainable and readable test code should exhibit a nice balance between the DRY principle and the DAMP principle. This balance gets disturbed whenever we introduce a base class for other test classes to derive from, just in the name of DRY. Let’s have a look at an example to demonstrate this.

public class BankCard
{
    public bool Blocked { get; private set; }
    
    internal BankCard(bool blocked)
    {
        Blocked = blocked;
    }

    public static BankCard IssueNewBankCard()
    {
        return new BankCard(false);
    }
    
    public void ReportStolen()
    {
        Blocked = true;
    }

    public void Expire()
    {
        Blocked = true;
    }

    public void MakePayment(ActiveAccount fromAccount, ActiveAccount toAccount, double amount)
    {
        if(Blocked)
            throw new InvalidOperationException("Making payment is not allowed.");

        fromAccount.Withdraw(amount);
        toAccount.Deposit(amount);
    }
}

The Subject Under Test of this example is the part of a domain model that revolves around banking and payments. Here we have a simple BankCard class which can be used to make payments. Obviously a bank card can also expire or reported stolen. In that case the bank card becomes blocked. This implies that no more payments can be made.

[Specification]
public class When_issuing_a_new_bank_card
{
    [Because]
    public void Of()
    {
        _result = BankCard.IssueNewBankCard();
    }

    [Observation]
    public void Then_the_bank_card_should_be_active()
    {
        _result.Blocked.Should_be_false();
    }

    private BankCard _result;
}

[Specification]
public class When_a_bank_card_is_reported_stolen
    : Bank_card_specification
{
    [Because]
    public void Of()
    {
        SUT.ReportStolen();
    }
    
    [Observation]
    public void Then_the_bank_card_should_be_blocked()
    {
        SUT.Blocked.Should_be_true();
    }
}

[Specification]
public class When_a_bank_card_is_expired
    : Bank_card_specification
{
    [Because]
    public void Of()
    {
        SUT.Expire();
    }
    
    [Observation]
    public void Then_the_bank_card_should_be_blocked()
    {
        SUT.Blocked.Should_be_true();
    }
}

[Specification]
public class When_making_a_payment
    : Bank_card_payment_specification
{
    [Because]
    public void Of()
    {
        SUT.MakePayment(FromAccount, ToAccount, 354.76);        
    }

    [Observation]
    public void Then_the_specified_amount_should_be_withdrawn_from_the_source_account()
    {
        FromAccount.Balance.Should_be_equal_to(1645.24);
    }
    
    [Observation]
    public void Then_the_specified_amount_should_be_deposited_to_the_target_account()
    {
        ToAccount.Balance.Should_be_equal_to(1354.76);
    }
}

[Specification]
public class When_making_a_payment_using_a_blocked_bank_card
    : Bank_card_payment_specification
{
    [Because]
    public void Of()
    {
        _makePayment = () => SUTBlocked.MakePayment(FromAccount, ToAccount, 162.88);
    }

    [Observation]
    public void Then_the_payment_should_not_be_allowed()
    {
        _makePayment.Should_throw_an<InvalidOperationException>();
    }
    
    private Action _makePayment;
}

Here we have the implementation of the tests that exercise the functionality provided by the BankCard class. Notice that some of these tests derive from the Bank_card_specification base class, while others derive from the Bank_card_payment_specification base class. Let’s consider the fourth test scenario, which performs a payment for an amount of 354.76 from one account to another account. The new balances of these two accounts, 1645.24 and 1354.76 respectively, are verified after the payment has been made. However, it’s quite difficult to determine whether these values are correct just by reading the code of the test. This is because we’re missing an important part of the context that is relevant for this test scenario, which lives in the base class of the test. So in order to visually verify the correctness of the test, we also have to make a switch to a different location in the code base.

public abstract class Bank_card_specification
{
    [Establish]
    public void BaseContext()
    {
        SUT = Example.BankCard();
        SUTBlocked = Example.BankCard().AsBlocked();
    }

    protected BankCard SUT { get; private set; }
    protected BankCard SUTBlocked { get; private set; }
}

public abstract class Bank_card_payment_specification : Bank_card_specification
{
    [Establish]
    public void PaymentContext()
    {
        FromAccount = Example.ActiveAccount()
            .WithAccountName("From account")
            .WithBalance(2000);
        
        ToAccount = Example.ActiveAccount()
            .WithAccountName("To account")
            .WithBalance(1000);
    }
    
    protected ActiveAccount FromAccount { get; private set; }
    protected ActiveAccount ToAccount { get; private set; }
}

This is how the base classes for the tests have been implemented. Here we find that the balance of the FromAccount is 2000, while the balance of the ToAccount is 1000. When switching back to the implementation of the test that verifies the payment, the new balances of these two accounts start to make more sense.

By moving test code to a base class, we’ve actually made it more difficult to comprehend what a particular test scenario is all about. It negatively impacts the readability and maintainability of these tests. A developer reading this implementation has to make a number of context switches between the test class and the abstract base class. These context switches result in a mental overhead.

The tests are coupled to the properties of the base class, which is also referred to as Subclass coupling. This makes it more difficult to move the tests around in the code base when needed.

Let’s get rid of these base classes.

[Specification]
public class When_issuing_a_new_bank_card
{
    [Because]
    public void Of()
    {
        _result = BankCard.IssueNewBankCard();
    }

    [Observation]
    public void Then_the_bank_card_should_be_active()
    {
        _result.Blocked.Should_be_false();
    }

    private BankCard _result;
}

[Specification]
public class When_a_bank_card_is_reported_stolen
{
    [Establish]
    public void Context()
    {
        _sut = Example.BankCard();
    }

    [Because]
    public void Of()
    {
        _sut.ReportStolen();
    }
    
    [Observation]
    public void Then_the_bank_card_should_be_blocked()
    {
        _sut.Blocked.Should_be_true();
    }

    private BankCard _sut;
}

[Specification]
public class When_a_bank_card_is_expired
{
    [Establish]
    public void Context()
    {
        _sut = Example.BankCard();
    }

    [Because]
    public void Of()
    {
        _sut.Expire();
    }
    
    [Observation]
    public void Then_the_bank_card_should_be_blocked()
    {
        _sut.Blocked.Should_be_true();
    }

    private BankCard _sut;
}

[Specification]
public class When_making_a_payment
{
    [Establish]
    public void Context()
    {
        _fromAccount = Example.ActiveAccount()
            .WithAccountName("From account")
            .WithBalance(2000);
        
        _toAccount = Example.ActiveAccount()
            .WithAccountName("To account")
            .WithBalance(1000);

        _sut = Example.BankCard();
    }

    [Because]
    public void Of()
    {
        _sut.MakePayment(_fromAccount, _toAccount, 354.76);        
    }

    [Observation]
    public void Then_the_specified_amount_should_be_withdrawn_from_one_account()
    {
        _fromAccount.Balance.Should_be_equal_to(1645.24);
    }
    
    [Observation]
    public void Then_the_specified_amount_should_be_deposited_to_another_account()
    {
        _toAccount.Balance.Should_be_equal_to(1354.76);
    }

    private ActiveAccount _fromAccount;
    private ActiveAccount _toAccount;
    private BankCard _sut;
}

[Specification]
public class When_making_a_payment_using_a_blocked_bank_card
{
    [Establish]
    public void Context()
    {
        _fromAccount = Example.ActiveAccount()
            .WithAccountName("From account")
            .WithBalance(2000);
        
        _toAccount = Example.ActiveAccount()
            .WithAccountName("To account")
            .WithBalance(1000);

        _sut = Example.BankCard().AsBlocked();
    }

    [Because]
    public void Of()
    {
        _makePayment = () => _sut.MakePayment(_fromAccount, _toAccount, 162.88);
    }

    [Observation]
    public void Then_the_payment_should_not_be_allowed()
    {
        _makePayment.Should_throw_an<InvalidOperationException>();
    }

    private ActiveAccount _fromAccount;
    private ActiveAccount _toAccount;
    private BankCard _sut;
    private Action _makePayment;
}

All the test scenarios are now self-containing. By simultaneously applying the DRY principle as well as the DAMP principle we achieve more readability and better maintainability for our solitary tests.

There’s a widespread misconception amongst developers that inheritance is a cheap way to add some behaviour to a base class, so that one or more derived classes can benefit from that behaviour. However, that’s not truly the point. The point of inheritance is to provide polymorphic behaviour. It is definitely not the right tool for reusing code. We should use composition for that instead of inheritance.

According to Wikipedia:

“ Polymorphism is the provision of a single interface to entities of different types or the use of a single symbol to represent multiple different types.

— Wikipedia

This is something entirely different from just slapping some duplicate code on a base class and calling it a day. As the Gang of Four already expressed in their book Design Patterns: favour composition over class inheritance. This applies to both production code and test code.

How To Write Unit Tests For Logging

2020-11-11T10:00:00Z

Once in a while I get asked the question whether one should write solitary tests for logging functionality. My answer to this question is the typical consultant answer: “It depends”. In essence, logging is an infrastructure concern. The end result is log data, which is written to a resource that is external to an application. Usually the generated data ends up in a file, a database or it might even end up in a cloud service.

Because logging crosses the process boundary of an application, it is more useful to write sociable tests to verify this particular functionality. It doesn’t make sense to use solitary tests in this particular case.

That being said, there are situations where business requirements explicitly state that logging should be a part of the interface of an application. In this situation, the intent of logging should be expressed explicitly by the code which in turn should also be exercised by solitary tests. The excellent book Growing Object Oriented Software Guided By Tests, written by Steve Freeman and Nat Pryce, mentions that there are generally two separate types of logging:

Support logging
Diagnostic logging

A support log contains messages that are intended for those that perform operational activities. These messages are used to determine whether the system behaves correctly or not. The log level for these messages is usually of type error or info.

A diagnostic log on the other hand holds messages that are targeted towards software developers. These messages provide valuable insights into the details of a running system. The log level for these messages is usually of type debug or trace.

Given these two types of logging, the basic idea is that code which expresses the intent of support logging should be exercised by solitary tests. Code statements that initiate diagnostic logging are usually not covered by tests.

Let’s have a look at an example that demonstrates both support and diagnostic logging in action.

public class ExpenseSheetController : Controller
{
    private readonly ICommandHandler<CreateExpenseSheet> _commandHandler;
    private readonly ISupportNotifier _supportNotifier;

    public ExpenseSheetController(ICommandHandler<CreateExpenseSheet> commandHandler,
                                  ISupportNotifier supportNotifier)
    {
        _commandHandler = commandHandler;
        _supportNotifier = supportNotifier;
    }
    
    [HttpPost]
    [ServiceFilter(typeof(PerformanceTracing))]
    public IActionResult Create(CreateExpenseSheetFormModel formModel)
    {
        try
        {
            var command = new CreateExpenseSheet(Guid.NewGuid(), formModel.EmployeeId);
            _commandHandler.Handle(command);
        }
        catch(Exception ex)
        {
            _supportNotifier.ErrorDuringExpenseSheetCreation(ex, formModel.EmployeeId);
            return BadRequest();
        }
        
        _supportNotifier.ExpenseSheetCreated(formModel.EmployeeId);
        return Ok();
    }
}

Here we have the implementation of a controller that can receive a request for creating a new expense sheet. Notice that the constructor of this controller class expects an instance of the ISupportNotifier interface. This dependency is being used by the implementation of the Create method for logging an error when an exception occurs. It is also used for logging when an expense sheet has been successfully created.

This is how the implementation of the SupportNotifier looks like.

public class SupportNotifier : ISupportNotifier
{
    private readonly ILogger<SupportNotifier> _logger;

    public SupportNotifier(ILogger<SupportNotifier> logger)
    {
        _logger = logger;
    }
    
    public void ExpenseSheetCreated(Guid employeeId)
    {
        _logger.LogInformation("Expense sheet created for employee with ID '{employeeId}'.");
    }

    public void ErrorDuringExpenseSheetCreation(Exception ex, Guid employeeId)
    {
        _logger.LogError(ex, $"Unable to create a new expense sheet for employee with ID '{employeeId}'");
    }
}

This code demonstrates that support logging uses log levels error or info depending on the context. Verifying the code of the SupportNotifier class itself can be done by using sociable tests. It’s not a good idea to write solitary tests for the SupportNotifier class. This would imply that a test double should be used as an instance of ILogger. As we already touched on in a previous blog post, it’s much better to avoid using test doubles for types that you don’t own. In this particular case it would even be quite hard to do as the Logxx methods of ILogger are actually extension methods and not regular methods.

Let’s have a look at the tests for the ExpenseSheetController.

[Specification]
public class When_handling_a_request_for_creating_a_new_expense_sheet
{
    [Establish]
    public void Context()
    {
        var commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        _supportNotifier = Substitute.For<ISupportNotifier>();

        _sut = new ExpenseSheetController(commandHandler, _supportNotifier);
    }

    [Because]
    public void Of()
    {
        var formModel = new CreateExpenseSheetFormModel 
        { 
            EmployeeId = new Guid("94EDE8F3-9675-4DD7-A18F-E37B1F323699") 
        };

        _sut.Create(formModel);
    }
    
    [Observation]
    public void Then_it_should_notify_support()
    {
        _supportNotifier.Received()
            .ExpenseSheetCreated(new Guid("94EDE8F3-9675-4DD7-A18F-E37B1F323699"));
    }

    private ExpenseSheetController _sut;
    private ISupportNotifier _supportNotifier;
}

[Specification]
public class When_an_error_occurs_while_handling_a_request_for_creating_a_new_expense_sheet
{
    [Establish]
    public void Context()
    {
        _supportNotifier = Substitute.For<ISupportNotifier>();
        _exception = new InvalidOperationException("Meltdown");
        
        var commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        commandHandler.WhenForAnyArgs(ch => ch.Handle(null))
            .Throw(_exception);
        
        _sut = new ExpenseSheetController(commandHandler, _supportNotifier);
    }
    
    [Because]
    public void Of()
    {
        var formModel = new CreateExpenseSheetFormModel 
        { 
            EmployeeId = new Guid("D1067157-5C73-4140-9D29-0FE5C1C4C2FB") 
        };

        _sut.Create(formModel);
    }
    
    [Observation]
    public void Then_it_should_notify_support_that_a_new_expense_sheet_has_been_created()
    {
        _supportNotifier.Received()
            .ErrorDuringExpenseSheetCreation(_exception, 
                new Guid("D1067157-5C73-4140-9D29-0FE5C1C4C2FB"));
    }
    
    private ExpenseSheetController _sut;
    private ISupportNotifier _supportNotifier;
    private Exception _exception;
}

These tests verify whether support logging occurs when an expense sheet is being created or when an exception gets raised. This way we express the intent of the operational requirements.

Notice also that the controller method has been decorated with a ServiceFilter attribute.

[HttpPost]
[ServiceFilter(typeof(PerformanceTracing))]
public IActionResult Create(CreateExpenseSheetFormModel formModel)
{
    ...
}

By applying this attribute, the PerformanceTracing action filter is being registered to surround the execution of the controller method. Let’s have a look at the implementation of this action filter.

public class PerformanceTracing : ActionFilterAttribute
{
    private readonly ILogger<PerformanceTracing> _logger;
    private readonly Stopwatch _stopWatch;

    public PerformanceTracing(ILogger<PerformanceTracing> logger)
    {
        _logger = logger;
        _stopWatch = new Stopwatch();
    }

    public override void OnActionExecuting(ActionExecutingContext context)
    {
        _stopWatch.Start();
    }

    public override void OnActionExecuted(ActionExecutedContext context)
    {
        _stopWatch.Stop();

        var controllerName = context.Controller.GetType().Name;
        var controllerActionName = context.ActionDescriptor.DisplayName;
        
        _logger.LogTrace($"Action '{controllerActionName}' of controller {controllerName} executed in " + 
            $"{_stopWatch.ElapsedMilliseconds} ms.");
    }
}

This implementation is a nice example of diagnostic logging. The action filter measures the execution time of a controller method and logs the result. Notice that we’re injecting the ILogger interface directly into the constructor. By registering the PerformanceTracing action filter using the ServiceFilter attribute, we ensure that an instance of ILogger gets resolved and properly injected. We didn’t provide any tests for this implementation.

I think it’s useful to consider support logging and diagnostic logging as two separate concepts, even though they quite often use the same mechanisms under the hood.

Prevent Domain Knowledge From Sneaking Into Solitary Tests

2020-09-30T09:00:00Z

Previously we discussed why solitary tests should be easy to read. Sometimes, the readability of solitary tests is affected by those developers who overcomplicate or overengineer things. Well intentioned no doubt, but in the end quite harmful nonetheless. Complex solitary tests can cause some serious headaches for other members of the team.

One example of this is an issue that I see popping up from time to time. It’s the case where domain logic sneaks into the implementation of solitary tests. This seems to occur most often in solitary tests that exercise algorithms or business logic in domain objects.

Let’s have a look at an example to see this in action.

public class SolarPanelInstallation
{
    public IEnumerable<SolarPanel> SolarPanels { get; }

    public SolarPanelInstallation(IEnumerable<SolarPanel> solarPanels)
    {
        SolarPanels = solarPanels;
    }

    public Watts CalculateTheoreticalCapacity()
    {
        return SolarPanels.Aggregate(Watts.Of(0), (accumulator, solarPanel) 
            => accumulator + solarPanel.Capacity);
    }
}

public class SolarPanel
{
    public Watts Capacity { get; }

    public SolarPanel(Watts capacity)
    {
        Capacity = capacity;
    }
}

public readonly struct Watts
{
    public int Value { get; }

    private Watts(int value)
    {
        Value = value;
    }

    public static Watts Of(int value)
    {
        return new Watts(value);
    }

    public static Watts operator +(Watts a, Watts b)
    {
        return Of(a.Value + b.Value);
    }
    
    public override string ToString()
    {
        return $"{Value} Watts";
    }
}

Here we’ve entered the realm of generating green energy using solar panels. A solar panel installation can be comprised of one or multiple solar panels. Each solar panel has its own capacity which is expressed in watts. The SolarPanelInstallation class provides a method that calculates the theoretical capacity of the entire installation.

Let’s have a look at the test code.

[Specification]
public class When_calculating_the_theoretical_capacity_of_a_solar_panels_installation
{
    [Establish]
    public void Context()
    {
        var solarPanels = new[]
        {
            new SolarPanel(Watts.Of(368)), 
            new SolarPanel(Watts.Of(368)), 
            new SolarPanel(Watts.Of(278)) 
        };
        
        _sut = new SolarPanelInstallation(solarPanels);
    }
    
    [Because]
    public void Of()
    {
        _theoreticalCapacity = _sut.CalculateTheoreticalCapacity();
    }    
    
    [Observation]
    public void Then_it_should_yield_the_total_capacity_of_all_solar_panels_of_the_installation()
    {
        var expectedCapacity = _sut.SolarPanels
            .Select(solarPanel => solarPanel.Capacity.Value)
            .Sum();
        
        _theoreticalCapacity.Should_be_equal_to(Watts.Of(expectedCapacity));
    }

    private SolarPanelInstallation _sut;
    private Watts _theoreticalCapacity;
}

Notice how the value of the expectedCapacity variable is being calculated. This is very similar to the calculation in the CalculateTheoreticalCapacity method of the SolarPanelInstallation class. Although the way their respective implementation calculates the capacity is slightly different, we can conclude that in this example the test code contains the same knowledge as the production code. Sometimes, I even encounter tests where the developer just “borrowed” from the production code directly.

This is also a nice example where state verification tests are too tightly coupled to the production code. So when the implementation of the CalculateTheoreticalCapacity method is refactored, chances are quite high that the test code needs to be modified as well.

It’s also more difficult to read this test and figure out what the expected value should be. For an easy example like this it doesn’t require that much additional brain cycles. However, with more complex algorithms or business logic, developers often execute the test in debug mode just to figure out what the expected value should be. How’s that for readability?

Let’s have a look at an improved version of this test.

[Specification]
public class When_calculating_the_theoretical_capacity_of_a_solar_panels_installation
{
    [Establish]
    public void Context()
    {
        var solarPanels = new[]
        {
            new SolarPanel(Watts.Of(368)), 
            new SolarPanel(Watts.Of(368)), 
            new SolarPanel(Watts.Of(278)) 
        };
        
        _sut = new SolarPanelInstallation(solarPanels);
    }
    
    [Because]
    public void Of()
    {
        _theoreticalCapacity = _sut.CalculateTheoreticalCapacity();
    }    
    
    [Observation]
    public void Then_it_should_yield_the_total_capacity_of_all_solar_panels_of_the_installation()
    {
        _theoreticalCapacity.Should_be_equal_to(Watts.Of(1014));
    }

    private SolarPanelInstallation _sut;
    private Watts _theoreticalCapacity;
}

Here we just provide the value that we expect to be the result of the calculation. That’s it! No more duplicate domain knowledge, no more tight coupling of the test and no more debugging. The expected value is just right there. Simplicity can be a beautiful thing.

Domain knowledge that sneaks into your tests is something to be avoided. Be very mindful about this.

Why Solitary Tests Should Be Easy To Read

2020-09-09T12:00:00Z

Occasionally someone asks me the question whether it’s necessary to test the code of solitary tests? Usually this question is ushered by folks who are opposed to the idea of automated tests. Somehow they have the notion that one can keep writing tests for tests for tests for tests, etc. Obviously this is a complex and absurd thing to do. This attitude often stems from test code being overly complex to the point that an average developer is no longer able to understand the scenario at hand.

Solitary tests should be simple and easy to read because we need to be able to easily verify their correctness. This is also where the age-old principle of double-entry bookkeeping comes in.

According to Wikipedia, double-entry bookkeeping:

“ Is a system so named because every entry to an account requires a corresponding and opposite entry to a different account. The double entry has two equal and corresponding sides known as debit and credit. The left-hand side is debit and right-hand side is credit.

— Wikipedia

Applying this principle to automated tests means that the test code verifies the production code, and that the production code verifies the test code. This means that solitary tests can be tested by introducing the very bugs in the production code that they are intended to verify and detect. The easiest and fastest way to accomplish this is by first writing a failing test before writing the implementation that fixes the test failure.

When we add solitary tests after writing the production code, then it’s more difficult to determine whether these tests are making the proper observations. The only way to determine this is to temporarily remove or comment the production code again to see if the test fails or not. This approach adds more overhead to the development process with an extra back-and-forth between the production code and the test code compared to a test-first approach. Regardless of whether we use Test-Driven Development or not, we should always see our solitary tests fail. This makes them more trustworthy. Never ever trust a test unless you have seen it fail!

Even when rigorously following the principles of Test-Driven Development the test code is only verified once, right after it has actually been written. What will happen when we change any of the code later on? In order to avoid going down the rabbit hole of writing tests for tests for tests, we need to be able to visually verify the correctness of our solitary tests, by using our eyes and our brains. And in order to easily do this, test code should be simple, small, discoverable and easy to read.

How can we accomplish this? For starters, test code should always have a cyclomatic complexity of 1. This is a software metric used as a quantity measure for the number of linearly independent paths through a program’s source code. It’s a very useful indicator as it indicates the number of pathways through a (test) method. This directly translates in never, ever using any branch constructs in the code of a solitary test like “if” statements or “for” loops.

It’s also quite important to avoid testing too much functionality in a single test. We should therefore try to focus on a single concern at a time. That is why designing production code for testability is such an important consideration.

Tales Of TDD: Stressed And Always In A Hurry

2020-08-19T10:00:00Z

So I noticed that you completed that feature that we’ve been pair programming yesterday?

Yes, I was a little bored yesterday evening. So I decided to continue working on it

Great job!

Thanks. I’m glad the feature is ready to go.

Definitely! I still have one question though. I’ve noticed that you added fifty lines of code to that controller method without adding any tests. What I also noticed is that you’ve added that new mapper class, but there aren’t any tests for that either.

Uhm, yes, uhm … about that …

Did you forget to check in the source files of the tests?

Well, in a sense that’s true. Because I didn’t write any tests.

Oh.

I know that we’ve been using a test-first approach to build up this feature. I just wanted to finish up the implementation for this user story. I knew what the remaining code had to do, so I just wrote it. Sure, it required some more lines than I initially anticipated, but hey, it’s finished now. We can move on to the next user story. So I already …

Can I ask you another question?

Sure.

Why are you in such a hurry?

I’m not in a hurry!

Well, to me it seems that you’re a bit pressed for time.

I just wanted to move on to the next user story.

Can we talk some more about the user story that we’ve been working on yesterday?

Sure.

You’ve added some code yesterday evening. How do you know that this code actually works?

It works properly because I tested it.

Great! How did you exercise the code that you’ve added?

By running the application and clicking around in the UI.

You’ve added some code that builds a URL, right?

Indeed

Did you execute the code inside this if-statement?

I don’t know. How can I be sure of that?

Well, you could have used some code coverage tool while running the application.

That sounds complicated.

Or you could have added a breakpoint for every execution path. There are four if-statements, so you need five breakpoints. One for each if-statement and an additional one in case all four conditions evaluate to false. Then you can start the application in debug-mode and go through the UI until you’ve hit all five breakpoints.

Well, that sounds like a lot of work. Debugging the application usually takes a lot of time.

You’re right. It does require a lot of work and focus. Did you verify all those cases yesterday?

I don’t know for sure. I guess so.

What about the design of the code? How did you manage to get some feedback there?

Uhm, I didn’t. I just added the code, and it looked just fine to me.

Do you mind we pick up where we left off yesterday? How about we try to add one test for verifying whether the URL is build correctly.

Shouldn’t we get going on the next user story?

We do have some time to add a single unit test, don’t we?

I guess so.

Can you try to add the test?

Sure.

— After a short while —

So, now we have a test that verifies whether the URL is build correctly. It seems to pass as well. However, … it does seem to involve a lot of code compared to the other test methods that we already have. There’s a lot of setup code and then there’s also this “hack” that we needed in order to get the test to pass.

How could we improve this?

I’m not really sure …

How about we just add another test for when the condition of that if-statement evaluates to true?

OK. Let’s do that!

— After another short while —

The amount of setup code for these tests really starts to bother me. How about we just move the implementation for composing that URL to a separate class?

What’s your reasoning?

Then we don’t need the overhead of setting up the entire controller just for testing the implementation that builds the URL. Composing the URL is just some data in, and the URL comes out. We could more easily test this in isolation.

Excellent idea! Let’s try that. Can I make one suggestion?

Sure.

How about we revert all the changes that we’ve made thus far?

You want to throw away the unit tests that we just added? Why?

For starters, I don’t really trust them.

Why don’t you trust the tests that we’ve added? They do seem to work and pass correctly.

They sure do. However, do they fail correctly as well? What I mean by that is whether they fail for the right reason?

Why shouldn’t they?

We didn’t see them fail. Right?

That’s true.

Let’s just revert our changes.

Are you sure about that?

Definitely. We’ve learned enough by experimenting. Let’s get rid of them.

Done. What’s next?

Let’s write a failing test for that class you’ve mentioned earlier. You know, the class which composes URL’s.

So we’re back at writing our tests first?

Yes, we are.

Shouldn’t we just copy and paste the code from the controller method into a new class?

In that case, we’re back to adding tests after the fact. Then we don’t know whether those tests would fail for the right reason or not.

I guess so. So we’re going to rewrite the code again?

Yes. You can consider it as a refactoring exercise. First we’re going to use TDD to drive the implementation of the URLComposer class. After that, we’re going to remove these fifty lines of code in the controller method. Then finally, we’re going to add another failing test to the test suite of the controller. You know the drill.

Let’s get started then.

— After a while —

This code definitely looks a lot cleaner now. Good thing that we performed this refactoring. Otherwise, we wouldn’t have discovered those two bugs that existed in the untested implementation.

Yes indeed.

I always have this feeling TDD slows me down too much. But not this time.

How come?

I can’t really explain it. Usually, everything seems so obvious and simple to the point that it feels that we’re just wasting time. Then again, we’re not. We’ve now spent less time reimplementing the URL composition compared to the time I’ve spent yesterday evening. How come you never seem to be stressed out while working on a user story?

Because I work as fast as I can, but as slow as I need to.

So, let’s move to the next user story then!

Almost. Now, regarding that mapper class you’ve added yesterday evening …

Dealing With Date/Time In Solitary Tests

2020-08-04T17:00:00Z

Solitary tests never cross the process boundary in which they are executed. This means that a solitary test never executes code that talks to a database, communicates across the network, touches the file system, etc. This also implies that a solitary test never deals with the system clock either, whether it’s directly or indirectly.

Let’s have a look at some example code in order to grasp the implications of this.

public class CreateCouponHandler
{
    private readonly ICouponRepository _couponRepository;

    public CreateCouponHandler(ICouponRepository couponRepository)
    {
        _couponRepository = couponRepository;
    }

    public void Handle_Coupled(CreateCoupon command)
    {
        var coupon = new Coupon(command.CouponCode, DateTime.Today);
        _couponRepository.Save(coupon);
    }
}

public class CreateCoupon
{
    public string CouponCode { get; }

    public CreateCoupon(string couponCode)
    {
        CouponCode = couponCode;
    }
}

Here we have a handler that creates a new discount coupon and saves it in the database. A coupon requires a code, and a creation date. The coupon code is specified through an incoming CreateCoupon command object. The creation date is set to the current date by calling the DateTime.Today static property.

Let’s have a look at the test code.

[Specification]
public class When_creating_a_new_coupon
{
    [Establish]
    public void Context()
    {
        _couponRepository = Substitute.For<ICouponRepository>();
        _sut = new CreateCouponHandler(_couponRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateCoupon("COUPON_CODE");
        _sut.Handle(command);
    }
    
    [Observation]
    public void Then_a_newly_created_coupon_should_be_saved()
    {
        var expectedCouponToBeSaved = new Coupon("COUPON_CODE", DateTime.Today);
        _couponRepository.Should_have_received(expectedCouponToBeSaved);        
    }

    private ICouponRepository _couponRepository;
    private CreateCouponHandler _sut;
}

According to its definition, this test doesn’t entirely qualify as a solitary test. Sure, it doesn’t require any external configuration file in order for it to execute properly. However, both the implementation and the test code have a direct dependency on the system clock.

This test will pass most of the time. However, on rare occasions this test might also fail. Why? Suppose that this test is being executed on a build server a couple of milliseconds before midnight. The call to DateTime.Today in the Handle method of the CreateCouponHandler class returns the current date. When the test method itself is executed, midnight has already passed. So the call to DateTime.Today returns the date of the next day which causes the test to fail.

I do recognise that the chances are pretty slim for this to actually happen. Nonetheless, this definitely affects the reliability of the test. Suppose that besides the date, we also need to store the time when a coupon gets created. In that case, we would replace the calls to DateTime.Today with DateTime.UtcNow. This would also make the test even less deterministic as it would sometimes fail due to a potential difference of just a couple of milliseconds.

In order to make both the implementation and the test code more loosely coupled, we could remove the direct dependency on the system clock by encapsulating the calls to DateTime.Today and DateTime.UtcNow.

public interface IClock
{
    DateTime GetCurrentDate();
    DateTime GetCurrentDateTime();
}

Here we’ve defined an interface named IClock. This defines a contract for retrieving the current date, or the current date/time.

public class SystemClock : IClock
{
    public DateTime GetCurrentDate()
    {
        return DateTime.Today;
    }

    public DateTime GetCurrentDateTime()
    {
        return DateTime.UtcNow;
    }
}

The SystemClock class provides an implementation of the IClock interface by encapsulating the calls to DateTime.Today and DateTime.UtcNow.

public class CreateCouponHandler
{
    private readonly ICouponRepository _couponRepository;
    private readonly IClock _clock;

    public CreateCouponHandler(ICouponRepository couponRepository, IClock clock)
    {
        _couponRepository = couponRepository;
        _clock = clock;
    }

    public void Handle(CreateCoupon command)
    {
        var creationDate = _clock.GetCurrentDate();
        
        var coupon = new Coupon(command.CouponCode, creationDate);
        _couponRepository.Save(coupon); 
    }
}

An instance of the IClock interface is injected into the constructor of the CreateCouponHandler class. The IoC container of the application will provide the necessary instance of the SystemClock class. In order to get the current date, we just call the GetCurrentDate method.

Let’s see what this means for our test code.

[Specification]
public class When_creating_a_new_coupon
{
    [Establish]
    public void Context()
    {
        _couponRepository = Substitute.For<ICouponRepository>();

        // The clock stub returns a concrete date
        var clock = Substitute.For<IClock>();
        clock.GetCurrentDate().Returns(new DateTime(2020, 08, 01));
        
        _sut = new CreateCouponHandler(_couponRepository, clock);
    }

    [Because]
    public void Of()
    {
        var command = new CreateCoupon("COUPON_CODE");
        _sut.Handle(command);
    }
    
    [Observation]
    public void Then_a_newly_created_coupon_should_be_saved()
    {
        // Here we specify a concrete date for the creation date
        var expectedCouponToBeSaved = new Coupon("COUPON_CODE", new DateTime(2020, 08, 01));
        _couponRepository.Should_have_received(expectedCouponToBeSaved);        
    }

    private ICouponRepository _couponRepository;
    private CreateCouponHandler _sut;
}

Notice that we no longer have a dependency on DateTime.Today inside the test method. Instead, we just specify a concrete date.

var expectedCouponToBeSaved = new Coupon("COUPON_CODE", new DateTime(2020, 08, 01));
_couponRepository.Should_have_received(expectedCouponToBeSaved);

In the setup of the test context, we also specify that a call to the GetCurrentDate method would return the same concrete date.

var clock = Substitute.For<IClock>();
clock.GetCurrentDate().Returns(new DateTime(2020, 08, 01));

By removing the direct dependency on the system clock, we turned our test into a full-fledged solitary test. Not only that. We’ve also made the test itself more deterministic as well. The test no longer suffers from potential failures depending on the time of day that it’s being executed. And then there’s also the aspect of readability.

Making use of DateTime.Today or DateTime.UtcNow lacks readability as a developer has to mentally translate this to a concrete date. Removing this mental translation step just makes things more convenient for other developers on the team. Especially when there’s some kind of date arithmetic involved, making use of a concrete date and/or time communicates more clearly what the inputs are and what the expected outputs should be.

Announcing Book: Writing Maintainable Unit Tests

2020-07-15T13:00:00Z

I’m very happy to announce that the first draft of my book Writing Maintainable Unit Tests has been published on LeanPub. It’s the written counterpart of my video course, with some significant revisions as well as additional content and examples.

The book currently contains the first three chapters. I’m still working on the final two chapters which I expect to be finished somewhere in autumn.

The book, as well as the video course, is my attempt to teach software developers how to write maintainable and readable unit tests. Have a look at the table of contents and let me know what you think. All feedback is very much appreciated.

Only Test Through Public Interfaces

2020-07-01T10:00:00Z

One of the most commonly asked questions in developer communities regarding Test-Driven Development and unit testing is whether one should write unit tests for private methods, and if so, how to accomplish this. The answer to this question is quite simply:

“Of course. As many lines of code as possible should be covered by unit tests”.

However, the way we accomplish this is very important.

The behaviour of a class or module should only be tested through its public members. So if we’ve followed the steps of “Red, Green, Refactor”, every line of code that is used by the public methods of a class or module has been exercised. Whenever a private method is not directly or indirectly used by these public methods, then it’s probably no longer needed. A private method that is not being used can therefore be removed. There’s no need to keep dead code around. It only confuses other developers of the team later on. Besides, that’s why we make use of version control systems.

When we only test through public methods, we also guarantee to exercise the code of the private methods that are being used. Private methods should be considered as an implementation detail. So there is no reason to directly try to put them under test.

However, there might be some cases where we just want to write unit tests for a private method. The first and best approach is to simply turn this private method into a public method instead. If this bothers us, we have to question ourselves why that is. This usually means that our class or module has too many responsibilities and that we should refactor this particular part of the code anyway. One way might be to move the private method in question as a public method of a new class.

A good design implies that the code is easy to test. When it’s not easy to test, then the design of the system is not that very good to begin with. This is why some developers refer to the expression “listening to the tests”. Unit tests very quickly amplify when something is not right with the design of the system.

Some programming languages or platforms enable developers to use reflection, or some kind of meta-programming, in order to reach and execute private methods at runtime. I would strongly advice against using such techniques for unit tests.

Nonetheless, this might come in handy when refactoring a very nasty, legacy code base. In that case, we can sparingly use such an approach to enable us to make some progress in our refactoring endeavours. However, we don’t want to keep these kinds of tests around for very long. Unit tests that employ reflection or meta-programming are too tightly coupled to the implementation of the Subject Under Test. At some point, we want to end up with a nice and clean public interface that is easy to test and for which implementation can be easily changed as well. So make sure to remove unit tests that make use of reflection or meta-programming as soon as possible.

Inside-Out and Outside-In TDD

2020-06-16T10:00:00Z

Test-Driven Development is a discipline that exists for about two decades now. Unfortunately, to this very day, it is still not without controversy. Most professional developers know that writing some form of automated tests can be quite beneficial for any type of codebase. What still seems to be quite controversial is whether to write a test before or after the production code has been laid out. This discussion seems to stir its head every other day on many discussion forums and on social media.

When I use the term TDD, I mean the repeating process of first writing a failing test, then writing as little production code as possible to make the test pass, after which the important step of refactoring the code ensues. This process is also commonly referred to as “Red, Green, Refactor”.

Some people firmly follow this mantra of “Red, Green, Refactor”. Others don’t like to follow this strict process for whatever reason and prefer to write tests after they’ve completed the implementation. Personally I like to write tests before the production code, and I highly encourage anyone to adopt this practice if they haven’t already.

Nevertheless, I would like to touch on two different approaches to Test-Driven Development that are practiced throughout the world of software development, namely:

Inside-Out TDD
Outside-In TDD

Inside-Out TDD

As its names implies, this approach allows us to start with the smallest unit of code - exercising either an individual class or module - by following the “Red, Green, Refactor” cycle. During this process, the design of the implementation happens during the “Refactor” step. This step is quite important and becomes more involved compared to the “Red” and “Green” steps. Each entity of the system is created in a TDD fashion using solitary tests until the whole feature is built up. Then a few sociable tests are added to verify that all the parts are working together as expected.

Inside-Out TDD is basically the approach where you start at the bottom of the the test pyramid and work your way up.

The most notable advantage by first focusing on the individual parts of the system is the ability to work in very small increments. This also enables that the development work can be parallelized within a software team.

The downside of Inside-Out TDD is that by initially focusing on the individual parts, there’s a higher risk of these entities not working together correctly with the possibility of rework.

Outside-In TDD

This approach focuses on creating a complete flow between the larger parts of the system right from the start. All the entities that make up a feature of the system are being created from the get-go, immediately verifying the wiring and interactions between them. The design of the system happens upfront during the “Red” step of “Red, Green, Refactor”. This results in the “Refactor” step becoming much more shallow. The different parts that make up a feature of the system are put in place while writing a failing sociable test. Such a thin slice of real functionality is also referred to as a walking skeleton (see Growing Object Oriented Software Guided By Tests). This failing sociable test serves as a beacon, making sure that no implementation is being forgotten.

The first test being written usually exercises a controller or a service at the system boundary as the starting point. A sociable test is usually employed here instead of a solitary test. Some parts of the code need to be swapped out by fake implementations that mimic the behaviour of the real-world implementations.

When every piece of the puzzle is in place, solitary tests are then used to further flesh out the concrete implementations of these individual classes or modules.

Outside-In TDD is basically the approach where you start at the top of the the test pyramid and work your way down.

The advantage of this approach is that it feels more exploratory, and is ideal for those kinds of situations where the high-level parts of the system are known without committing to the more fine-grained implementation details. This results in a “think like the client” mindset well before we start thinking as a software developer.

The downside of Outside-In TDD is that the design of the larger feature and its different parts should be known right from the start as opposed to driving the design of the smaller parts of the system. This usually takes significantly more time to write a first failing sociable test.

Conclusion

One might come to the conclusion that it’s somehow important to choose one approach over the other. However, this is definitely not the case. Please note that Inside-Out TDD and Outside-In TDD are not mutually exclusive. There’s no point in choosing one approach over the other and rigorously sticking to a particular choice. We should practice and master both ways of writing automated tests in order to develop a “gut instinct” for applying a certain approach.

Test Double Heuristics

2020-06-03T10:00:00Z

In the previous blog post, we’ve talked about avoiding excessive specification of test doubles. This is just one in a series of “good practices” for using test doubles in solitary tests. Let’s have a look at a few more guidelines that increase the maintainability of solitary tests when using test doubles.

Avoid using test doubles for types that you don’t own

Suppose that we’re using a third-party library in our application. We have some solitary tests that uses test doubles for some interface types that are exposed through the public API.

When upgrading this third-party library to a newer version, we might run into some subtle and possibly less subtle issues whenever breaking changes have been introduced by its maintainers. This implies that we potentially have to fix several of our solitary tests every time we perform such an upgrade.

Using test doubles for types that are not under our control might also be a good indication that the current design is too strongly coupled to a third-party library. Another issue might be that due to the potential complexity of said library, a large amount of test doubles need to be configured which brings us back to the realm of complex test fixtures and excessive specification. We want to steer clear from these kinds of situations whenever possible.

A solution is to introduce an adapter layer that isolates the use of the third-party library from the rest of the application. Breaking changes in the third-party library are now limited to the sociable tests for this adapter layer. These adapters provide us with our own interface that we can control and which we are able to substitute when needed.

Avoid test doubles for concrete classes

Creating test doubles for concrete classes is a technique that should be avoided unless it’s absolutely necessary. When designing and writing new code, make sure to create test doubles for interfaces instead of concrete classes. Creating a test double of a concrete class might only be feasible when you’re working in a legacy application where you need to swap out a concrete collaborator.

One way to approach this is to manually derive a class and override the necessary methods. This newly derived class can then be used as a test double by the solitary tests. Note that this is usually only a temporary solution. After further refactoring the code, at some point, an interface can be introduced which removes the need for the concrete test double.

So the general guideline is to stick with creating test doubles for roles, not concrete classes.

Don’t let test doubles return other test doubles

We can be very brief when it comes to this guideline. Whenever you encounter the need to let a test double return an instance of another test double, then this is usually a clear sign to reconsider the design of the system instead. Test doubles that return other test doubles is considered an anti-pattern.

Don’t implement behaviour in test doubles

Implementing behaviour in test doubles becomes quite problematic really fast. Let’s have a look at an example.

HeadOfDepartment headOfDepartment = Example.HeadOfDepartment()
    .WithId(new Guid("224FE5B8-EDBB-4F8B-8654-715C1C294CFD"));

var approverRepository = Substitute.For<IApproverRepository>();
approverRepository.Get(Arg.Any<Guid>()).Returns( callInfo =>
{
    var id = callInfo.Arg<Guid>();
    return id != Guid.Empty ? headOfDepartment : null;
});

Here we’re using the NSubstitute library to create a stub for the IApproverRepository interface. Instead of just returning a value when the Get method of the stub is called, we specify a callback lambda function. This function first retrieves the value of the specified parameter. In this case it’s the identifier (GUID) for an approver. If the specified identifier is any GUID, then an instance of an HeadOfDepartment is returned. Otherwise, a null reference is returned for an empty GUID.

At first sight there doesn’t seem to be anything wrong. However, a big issue arises when the real implementation of the IApproverRepository implements a different behaviour. For example, it might not return a null reference for an empty GUID identifier. This implies that the Subject Under Test is tested against incorrect behaviour.

Therefore, it’s better to avoid implementing behaviour in test doubles like dummies, stubs, spies and mocks. However, there’s one type of test double that is an exception to this guideline. A Fake is a test double for which it is perfectly fine to implement behaviour. In fact, that’s the whole point of having a fake object. As we already mentioned in a previous blog post, fake objects are not well suited for solitary tests anyway.

Reduce the number of collaborators

Every time we need to instantiate a vast number of test doubles in order to create an instance of the Subject Under Test, then we need to reflect once more on the design of the system. When applying the dependency injection pattern, a bloated constructor is usually a clear sign the Subject Under Test has too many responsibilities. Such a violation of the Single Responsibility Principle can be mitigated by extracting functionality into separate classes.

We should then look for collaborators that are always used together to get a sense of the parts of the implementation that can be grouped and extracted behind a more course-grained interface. Without imposing a hard number, because there can be exceptions, whenever a Subject Under Test has more than five different collaborators then this might be an indication of a potential design issue. Be very mindful about this, especially when the need arises to add even more collaborators.

These are a couple of guidelines that I find useful when working with test doubles.

Excessive Specification of Test Doubles

2020-05-20T10:00:00Z

Previously we’ve discussed the different kinds of test doubles. By using test doubles in our solitary tests, we also introduce more coupling between the test code and the implementation. However, the good news is that there are also some things we can do in order to somewhat reduce this coupling.

A very important item of caution is to avoid excessive specification of method behaviour. Let’s have a look at an example.

var approverId = new Guid("224FE5B8-EDBB-4F8B-8654-715C1C294CFD");
HeadOfDepartment headOfDepartment = Example.HeadOfDepartment().WithId(approverId);

var approverRepositoryMock = Substitute.For<IApproverRepository>();
approverRepositoryMock.Received().Get(approverId).Returns(headOfDepartment);

Here we create a mock for the IApproverRepository interface, created by the NSubstitute framework. When the subject under test makes a call to the Get method using a specific identifier, then an instance of the HeadOfDepartment is returned. However, that’s not the only thing that happens. The mock also verifies whether the subject under test actually makes a call to the Get method. If not, then the test will fail. The approach taken in this example illustrates the concept of excessive specification.

With behaviour verification, the degree of coupling between the test code and the implementation depends on the type of test double being used. Using a Dummy or a Stub results in the least amount of coupling, while using a Spy or a Mock induces the highest amount of coupling.

For indirect inputs, as is the case in our example, we should avoid using mocks at all times. We should use stubs instead. Let’s have a look at an improved version of the previous example.

HeadOfDepartment headOfDepartment = Example.HeadOfDepartment()
    .WithId(new Guid("224FE5B8-EDBB-4F8B-8654-715C1C294CFD"));

var approverRepositoryStub = Substitute.For<IApproverRepository>();
approverRepositoryStub.Get(Guid.Empty).ReturnsForAnyArgs(headOfDepartment);

Here we no longer verify whether the Get method is being called. We only state that whenever the Get method of the ApproverRepository is called, no matter what the identifier might be, just return the specified HeadOfDepartment object. So we basically switched from using a mock to using a stub.

For solitary tests that make use of test doubles, prefer the use of stubs over spies and mocks. Generalisation of stub method arguments make solitary tests more resilient to changes. I always try to use the most generic argument matcher that is feasible for a particular scenario.

Obviously, we sometimes need to use spies or mocks. For those cases, it’s very important to restrict their use to only verifying indirect outputs. We should only specify what should happen and nothing more.

The following piece of test code is an example where the use of a mock is warranted.

[Observation]
public void Then_the_approved_expense_sheet_should_be_saved()
{
    _expenseSheetRepositoryMock.Received().Save(_expenseSheet);
}

When observing solitary tests that make use of test doubles in a decent code base, you might see that the usage ratio of dummies and/or stubs is significantly higher than that of spies and/or mocks. This should give us a decent idea about the usage of the different types of test doubles.

The Boundaries of Solitary Tests

2020-05-06T08:00:00Z

When stating the definition of a solitary test, we mentioned that they have two constraints:

The code being exercised by these tests never cross the process boundary in which they are executed.
A single class or module is being tested.

The first constraint means that a solitary test never executes code that talks to a database, communicates across the network, touches the file system, etc. … This basically implies that a solitary test never requires any configuration whatsoever in order for it to execute correctly.

The second constraint is definitely the most controversial one. The most fundamental solitary test exercises a single class or module. Does this imply that a test which exercises code of multiple concrete classes is by definition a sociable test? Well, that kind of depends on a number of factors.

Do these cluster of classes all live within the same dependency inversion boundary? Is there a single class within this cluster that takes up the role as the main entry point? Are the other classes in the cluster only used by the main entry class? Are all these classes part of a single conceptual whole? If the answer to these questions are all positive, then I would argue that a test which exercises code of such a cluster of classes is still a solitary test and not a sociable test.

I do recognise that this hugely depends on the person you ask the question. There just isn’t a wide consensus on this particular topic. In this blog post I’m going to make a (probably futile) attempt to clarify the approach that I usually employ.

I personally consider the test pyramid to be more of a spectrum and less as a pile of discrete buckets. The moment a test exercises code of more than a single concrete class, it moves up the pyramid towards the area where the sociable tests live. How much the test rises depends on the constraints and criteria mentioned earlier.

Let’s have a look at an example.

public class ExpenseSheetController : Controller
{
    private readonly ICommandHandler<CreateExpenseSheet> _commandHandler;
    private readonly CreateExpenseSheetCommandMapper _commandMapper;
    private readonly CreateExpenseSheetFormModelValidator _validator;

    public ExpenseSheetController(ICommandHandler<CreateExpenseSheet> commandHandler)
    {
        _commandHandler = commandHandler;
        _commandMapper = new CreateExpenseSheetCommandMapper();
        _validator = new CreateExpenseSheetFormModelValidator();
    }
    
    [HttpPost]
    public IActionResult Create(CreateExpenseSheetFormModel formModel)
    {
        var isValid = _validator.Validate(formModel);
        if(! isValid)
            return BadRequest();
        
        var command = _commandMapper.MapFrom(formModel);
        var result = _commandHandler.Handle(command);

        if(! result.IsSuccessful)
            return BadRequest();
            
        return Ok();
    }
}

The subject under test is a web controller that exposes a single endpoint. This endpoint accepts a form model which contains the data necessary for creating a new expense sheet. First the specified form model is validated. In case the form model contains incorrect data, HTTP status code 400 (bad request) is returned to the client of our endpoint. If valid, then form model is mapped to a command object after which it is sent to the corresponding command handler. When the command handler is able to successfully process the command, then HTTP status 200 (OK) is returned to the client. Otherwise, we return HTTP status code 400 (bad request).

public class CreateExpenseSheetFormModel
{
    public Guid EmployeeId { get; set; }
    public IEnumerable<ExpenseModel> Expenses { get; set; }
    public DateTime SubmissionDate { get; set; }

    public CreateExpenseSheetFormModel()
    {
        Expenses = Enumerable.Empty<ExpenseModel>();    
    }
}

public class ExpenseModel
{
    public decimal Amount { get; set; }
    public DateTime Date { get; set; }
    public string Description { get; set; }
}

This is how the code of the form model looks like …

public class CreateExpenseSheetFormModelValidator
{
    public bool Validate(CreateExpenseSheetFormModel formModel)
    {
        return Exists(formModel.Expenses) &&
               formModel.Expenses.All(IsValid);
    }

    private static bool Exists(IEnumerable<ExpenseModel> expenses)
    {
        return expenses != null && expenses.Any();
    }
    
    private static bool IsValid(ExpenseModel expenseModel)
    {
        return expenseModel.Amount > 0 &&
            ! string.IsNullOrWhiteSpace(expenseModel.Description);
    }
}

public class CreateExpenseSheetCommandMapper
{
    public CreateExpenseSheet MapFrom(CreateExpenseSheetFormModel formModel)
    {
        var expenses = formModel.Expenses
            .Select(expenseModel => new ExpenseData(
                expenseModel.Amount, expenseModel.Date, expenseModel.Description));
        
        return new CreateExpenseSheet(Guid.NewGuid(), formModel.EmployeeId, 
            formModel.SubmissionDate, expenses);
    }
}

… and these are the implementations for the validator of the form model, and the mapper that maps the form model to the corresponding command.

Let’s have a look at the implementation of the tests.

[Specification]
public class When_handling_a_request_for_creating_a_new_expense_sheet
{
    [Establish]
    public void Context()
    {
        _commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        _commandHandler.Handle(Arg.Any<CreateExpenseSheet>()).Returns(Result.Success());
        
        _sut = new ExpenseSheetController(_commandHandler);    
    }

    [Because]
    public void Of()
    {
        var formModel = new CreateExpenseSheetFormModel
        {
            EmployeeId = new Guid("A881CF9F-888E-482B-A5EE-ACDD10D01CB2"),
            Expenses = new[]
            {
                new ExpenseModel 
                    { Amount = 12.8m, Date = new DateTime(2020, 04, 14), Description = "Lunch" },
                new ExpenseModel 
                    { Amount = 46.2m, Date = new DateTime(2020, 04, 15), Description = "Fuel" }
            },
            SubmissionDate = new DateTime(2019, 03, 16)
        };
        
        _result = _sut.Create(formModel);    
    }

    [Observation]
    public void Then_a_command_should_be_dispatched_to_the_domain()
    {
        var expectedCommand = new CreateExpenseSheet(
            Guid.Empty, 
            new Guid("A881CF9F-888E-482B-A5EE-ACDD10D01CB2"), 
            new DateTime(2019, 03, 16), 
            new[]
            {
                new ExpenseData(12.8m, new DateTime(2020, 04, 14), "Lunch"), 
                new ExpenseData(46.2m, new DateTime(2020, 04, 15), "Fuel")
            });
        
        _commandHandler.Should_have_received(expectedCommand, command => command.Id);
    }
    
    [Observation]
    public void Then_HTTP_status_code_200_should_be_sent_back_to_the_client()
    {
        _result.Should_be_an_instance_of<OkResult>();        
    }

    private ICommandHandler<CreateExpenseSheet> _commandHandler;
    private IActionResult _result;
    private ExpenseSheetController _sut;
}

[Specification]
public class When_handling_an_invalid_request_for_creating_a_new_expense_sheet
{
    [Establish]
    public void Context()
    {
        var commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        commandHandler.Handle(Arg.Any<CreateExpenseSheet>())
            .Throws(
                new InvalidOperationException("Handle method of command handler shouldn't get called"));
        
        _sut = new ExpenseSheetController(commandHandler);
    }

    [Because]
    public void Of()
    {
        var invalidFormModel = new CreateExpenseSheetFormModel();
        _result = _sut.Create(invalidFormModel);
    }

    [Observation]
    public void Then_HTTP_status_code_400_should_be_sent_back_to_the_client()
    {
        _result.Should_be_an_instance_of<BadRequestResult>();
    }

    private ExpenseSheetController _sut;
    private IActionResult _result;
}

[Specification]
public class When_handling_a_request_for_creating_a_new_expense_sheet_that_fails_the_domain_criteria
{
    [Establish]
    public void Context()
    {
        var commandHandler = Substitute.For<ICommandHandler<CreateExpenseSheet>>();
        commandHandler.Handle(Arg.Any<CreateExpenseSheet>())
            .Returns(Result.Failure(new DomainViolation("This is highly irregular")));
        
        _sut = new ExpenseSheetController(commandHandler);
    }

    [Because]
    public void Of()
    {
        var formModel = new CreateExpenseSheetFormModel();
        _result = _sut.Create(formModel);
    }

    [Observation]
    public void Then_HTTP_status_code_400_should_be_sent_back_to_the_client()
    {
        _result.Should_be_an_instance_of<BadRequestResult>();
    }

    private IActionResult _result;
    private ExpenseSheetController _sut;
}

Notice that these tests provide a test double for the command handler when creating an instance of the controller. We use a test double here because the command handler resides in a different dependency inversion boundary, namely the core domain of the application. But what about the validator and the mapper? When looking at the constructor of the controller, notice that an instance of the validator and the mapper are created instead of just being injected.

public ExpenseSheetController(ICommandHandler<CreateExpenseSheet> commandHandler)
{
    _commandHandler = commandHandler;
    _commandMapper = new CreateExpenseSheetCommandMapper();
    _validator = new CreateExpenseSheetFormModelValidator();
}

The reasoning behind this is that the validator and the mapper are only used by this controller. In this specific case, these classes are an implementation detail of the controller. As a matter of fact, I could have chosen not to create separate classes, neglecting the single-responsibility principle, and just go for private methods in the controller itself. In that case it would also be much harder and less efficient to write tests for driving the implementation of the validator.

If for some reason the validator or mapper would be used by another controller, then I would have chosen to use dependency injection instead of creating the instance in the constructor. I would also have used a test double in the tests as well. But I don’t consider it very likely that the validator and mapper classes would be used in other parts of the code.

By letting the controller create the instance of the validator and mapper classes, the controller acts as the main entry point of the cluster. This entire approach leans more towards the Detroit School of TDD. When following The London School of TDD, we would use dependency injection for both the validator and mapper classes, and use test doubles for them as well in out tests. Which, for the record, is just a valid approach as well. We would also have separate test suites for both the validator and mapper classes.

Going back to our example, we created separate tests for the validator but not for the mapper.

[TestFixture]
public class When_validating_a_form_model_for_creating_an_expense_sheet  
{
    private CreateExpenseSheetFormModelValidator _sut;

    [Establish]
    public void Context()
    {
        _sut = new CreateExpenseSheetFormModelValidator();
    }
    
    [Observation]
    public void Then_it_should_indicate_truthy_for_a_valid_form_model()
    {
        var validFormModel = new CreateExpenseSheetFormModel
        {
            Expenses = new[]
            {
                new ExpenseModel 
                    { Amount = 23.8m, Date = new DateTime(2020, 04, 15), Description = "Lunch" }
            }
        };

        var isValid = _sut.Validate(validFormModel);
        isValid.Should_be_truthy();
    }
    
    [Observation]
    public void Then_it_should_indicate_falsy_for_a_form_model_with_an_invalid_list_of_expenses()
    {
        var invalidFormModel = new CreateExpenseSheetFormModel { Expenses = null };
        
        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
    
    [Observation]
    public void Then_it_should_indicate_falsy_for_a_form_model_without_expenses()
    {
        var invalidFormModel = new CreateExpenseSheetFormModel();
        
        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
    
    [ObservationFor(-1)]
    [ObservationFor(0)]
    public void Then_it_should_indicate_falsy_for_a_form_model_specifying_an_expense_with_an_insufficient_amount
        (decimal invalidAmount)
    {
        var invalidFormModel = new CreateExpenseSheetFormModel
        {
            Expenses = new[] { 
                new ExpenseModel
                {
                    Amount = invalidAmount, 
                    Date = new DateTime(2020, 04, 15), 
                    Description = "Dinner"
                } 
            }
        };

        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
    
    [ObservationFor(null)]
    [ObservationFor("")]
    [ObservationFor(" ")]
    public void Then_it_should_indicate_falsy_for_a_form_model_specifying_an_expense_without_description
        (string invalidDescription)
    {
        var invalidFormModel = new CreateExpenseSheetFormModel
        {
            Expenses = new[] { 
                new ExpenseModel
                {
                    Amount = 15.65m, 
                    Date = new DateTime(2020, 04, 15), 
                    Description = invalidDescription
                } 
            }
        };

        var isValid = _sut.Validate(invalidFormModel);
        isValid.Should_be_falsy();
    }
}

The reasoning behind this is that the implementation of the validator has a cyclomatic complexity that is greater than one, while for the mapper the cyclomatic complexity is exactly one. It’s much easier to use a separate test suite to exercise the implementation of the validator instead of through the test suite of the controller. While for the mapper class, we’ve chosen to exercise its code through the test suite of the controller itself.

The following diagram provides an overview of the approach we’ve taken in our example.

There’s one thing that we need to watch out for though, and that is cascading failures. When an issue arises in the implementation of the validator or the mapper classes, then multiple controller tests would probably fail. But the cluster of classes is still small enough to easily pinpoint the cause of the issue. When following The London School of TDD, this is less of an issue.

The tests for the controller a placed higher on the spectrum of the test pyramid, while the tests for the validator appear more near the bottom. But I do still consider these tests as solitary tests. If, for example, I would have used a concrete instance for the command handler in the test suite of the controller, then I would say that these tests are sociable tests instead.

Determining the unit of solitary tests is not really an exact science. It usually comes down to just following your intuition and relying on past experiences to determine a certain boundary that is useful. Personally I find the test pyramid a very useful model to think and reason about these boundaries.

Test Doubles

2020-04-08T10:30:00Z

In the previous blog post, we’ve talked about indirect inputs and outputs when a unit test exercises a Subject Under Test. The examples shown previously demonstrate the use of test doubles.

A test double is a type of object that replaces a real instance of a collaborator, for the specific purpose of executing tests. Test doubles make it more easy for us to manipulate indirect inputs and verify indirect outputs. But this ease of use also comes with a certain cost. Depending on the type of test double being used, it also couples unit tests more tightly to their Subject Under Test, unless specific measures are applied to somewhat mitigate this coupling.

Test doubles are put in place using a technique called Dependency Injection. Instead of letting the Subject Under Test directly create an instance for each of its collaborators, these instances are provided from the outside. They are usually specified as arguments to the constructor of the Subject Under Test.

Let’s have a look at the different kinds of test doubles and their use.

Dummy

The first type of test double is called a Dummy. A Dummy object supports some kind of interface that is expected by the Subject Under Test, but it doesn’t provide any behaviour or return any values. Basically, it doesn’t do anything. It just exists. A Dummy is most often passed as a placeholder argument, but is never actually used by the Subject Under Test.

Let’s have a look at an example of a Dummy.

public class CreateExpenseSheetHandler : ICommandHandler<CreateExpenseSheet>
{
    private readonly IEmployeeRepository _employeeRepository;
    private readonly IExpenseSheetRepository _expenseSheetRepository;

    public CreateExpenseSheetHandler(
        IEmployeeRepository employeeRepository,
        IExpenseSheetRepository expenseSheetRepository)
    {
        _employeeRepository = employeeRepository;
        _expenseSheetRepository = expenseSheetRepository;
    }
    
    public Result Handle(CreateExpenseSheet command)
    {
        if(null == command)
            throw new ArgumentNullException(nameof(command));

        var employee = _employeeRepository.Get(command.EmployeeId);
        if(null == employee)
        {
            var message = $"The employee with identifier '{command.EmployeeId}' could not be found.";
            return Result.Failure(new DomainViolation(message));    
        }
        
        var expenseSheet = new ExpenseSheet(command.Id, employee, command.SubmissionDate);
        _expenseSheetRepository.Save(expenseSheet);
        
        return Result.Success();
    }
}

public class CreateExpenseSheet
{
    public Guid Id { get; }
    public Guid EmployeeId { get; }
    public DateTime SubmissionDate { get; }
    
    public CreateExpenseSheet(Guid id, Guid employeeId, DateTime submissionDate)
    {
        Id = id;
        EmployeeId = employeeId;
        SubmissionDate = submissionDate;
    }
}

Here we have the implementation of a command handler that creates a new expense sheet. The command object specifies the identifier for the new expense sheet, the identifier of the employee and a submission date. We use the Employee repository to retrieve an Employee object based on its unique identifier. We let the operation fail in case the required Employee object could not be found. Then the new ExpenseSheet object is created and saved into the repository. In that case, the operation is completed successfully. Note that a guard condition has also been added in order to enforce the caller of the Handle method to provide an instantiated CreateExpenseSheet command object.

public class DummyEmployeeRepository : IEmployeeRepository
{
    public Employee Get(Guid id)
    {
        throw new NotImplementedException(
            "The Get method of the EmployeeRepository shouldn't get called.");
    }
}

public class DummyExpenseSheetRepository : IExpenseSheetRepository
{
    public ExpenseSheet Get(Guid id)
    {
        throw new NotImplementedException(
            "The Get method of the ExpenseSheetRepository shouldn't get called.");
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        throw new NotImplementedException(
            "The Save method of the ExpenseSheetRepository shouldn't get called.");
    }
}

The DummyEmployeeRepository class provides a dummy implementation by implementing the IEmployeeRepository interface. The Get method throws a NotImplementedException in case it gets called. The same is true for the Get/Save methods of the DummyExpenseSheetRepository class.

[Specification]
public class When_creating_a_new_expense_sheet_using_a_non_existent_command
{
    [Establish]
    public void Context()
    {
        var dummyEmployeeRepository = new DummyEmployeeRepository();
        var dummyExpenseSheetRepository = new DummyExpenseSheetRepository();
        
        _sut = new CreateExpenseSheetHandler(dummyEmployeeRepository, dummyExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        _createExpenseSheet = () => _sut.Handle(null);
    }
    
    [Observation]
    public void Then_an_exception_should_be_thrown()
    {
        Assert.That(_createExpenseSheet, Throws.ArgumentNullException);
    }
    
    private CreateExpenseSheetHandler _sut;
    private TestDelegate _createExpenseSheet;
}

This particular test scenario verifies whether an ArgumentNullException is thrown by the Subject Under Test when a null reference is passed to the Handle method. We pass an instance of both the DummyEmployeeRepository and the DummyExpenseSheetRepository classes to the constructor of the CreateExpenseSheetHandler.

We could also get away with just passing two null references instead of these dummy objects. But then we just get a NullReferenceException when the test fails. Using an actual dummy implementation results in slightly more meaningful diagnostic information due to the custom exception messages in each method. Implementing and using a dummy object this way, we can ensure that a collaborator is never used in a particular scenario.

Stub

The second type of test double is called a Stub. A Stub object is the same as a Dummy. It doesn’t do anything, except providing the Subject Under Test an indirect input that can be controlled by a test. So, the only thing a Stub does more than a Dummy is returning values for query methods. A test can instruct a Stub to return either a valid or an invalid indirect input depending on the test scenario. But it can also instruct a Stub to raise a particular exception as well.

Let’s have a look at an example of a Stub.

public class StubEmployeeRepository : IEmployeeRepository
{
    private readonly Employee _result;

    public StubEmployeeRepository(Employee result)
    {
        _result = result;
    }
    
    public Employee Get(Guid id)
    {
        return _result;
    }
}

The StubEmployeeRepository class implements the IEmployeeRepository interface. Its constructor expects an instance of an Employee that can be returned by the Get method when and if called.

[Specification]
public class When_creating_a_new_expense_sheet_for_an_unknown_employee 
{
    [Establish]
    public void Context()
    {
        Employee unknownEmployee = null;
        
        var stubEmployeeRepository = new StubEmployeeRepository(unknownEmployee);
        var dummyExpenseSheetRepository = new DummyExpenseSheetRepository();
        
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, dummyExpenseSheetRepository);
    }
    
    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(Guid.NewGuid(), Guid.NewGuid(), new DateTime(2018, 11, 11));
        _result = _sut.Handle(command);
    }
    
    [Observation]
    public void Then_the_operation_should_fail()
    {
        Assert.That(_result.IsSuccessful, Is.False);
    }

    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario verifies whether the operation fails when the specified employee could not be found by the repository. This means that the stubbed EmployeeRepository returns a null reference when the Get method is called by the CreateExpenseSheetHandler. Further execution of the Handle method should be aborted. This is enforced by using a dummy implementation of the ExpenseSheetRepository.

A Stub is usually the type of test double that is most often used throughout an entire suite of unit tests.

Spy

The third type of test double is called a Spy. A Spy does the same things as a Stub. It provides returns values for the query methods of its public interface. But a Spy also captures the indirect outputs of the Subject Under Test and afterwards provides the ability for the test to verify these outputs. This is for the command methods of the public interface that it supports.

Let’s have a look at an example of a Spy.

public class SpyExpenseSheetRepository : IExpenseSheetRepository
{
    private readonly ExpenseSheet _result;
    
    public ExpenseSheet SavedExpenseSheet { get; private set; }
    
    public SpyExpenseSheetRepository(ExpenseSheet result)
    {
        _result = result;
    }
    
    public ExpenseSheet Get(Guid id)
    {
        return _result;
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        SavedExpenseSheet = expenseSheet;
    }
}

The SpyExpenseSheetRepository class implements the IExpenseSheetRepository interface. The constructor of the spy expects an instance of an ExpenseSheet that can be returned by the Get method when and if called. But it also assigns the ExpenseSheet object that is specified to the Save method in the SavedExpenseSheet public property.

[Specification]
public class When_creating_a_new_expense_sheet__spy_example
{
    [Establish]
    public void Context()
    {
        var employee = Example.Employee().WithId(new Guid("680D0C0A-E445-4344-B67A-363589E2746A"));
        
        var stubEmployeeRepository = new StubEmployeeRepository(employee);
        _spyExpenseSheetRepository = new SpyExpenseSheetRepository(null);
        
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, _spyExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(
            new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"), 
            new Guid("680D0C0A-E445-4344-B67A-363589E2746A"), 
            new DateTime(2018, 11, 11));

        _result = _sut.Handle(command);
    }
    
    [Observation]
    public void Then_the_approved_expense_sheet_should_be_saved()
    {
        var savedExpenseSheet = _spyExpenseSheetRepository.SavedExpenseSheet;
        
        Assert.That(savedExpenseSheet, Is.Not.Null);
        Assert.That(savedExpenseSheet.Id, 
            Is.EqualTo(new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3")));
        Assert.That(savedExpenseSheet.EmployeeId, 
            Is.EqualTo(new Guid("680D0C0A-E445-4344-B67A-363589E2746A")));
        Assert.That(savedExpenseSheet.SubmissionDate, 
            Is.EqualTo(new DateTime(2018, 11, 11)));
    }

    [Observation]
    public void Then_the_operation_should_succeed()
    {
        Assert.That(_result.IsSuccessful);
    }

    private SpyExpenseSheetRepository _spyExpenseSheetRepository;
    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario verifies the happy path of the Subject Under Test. When creating an instance of the CreateExpenseSheetHandler class, we pass an instance of the StubEmployeeRepository for providing an Employee as an indirect input and an instance of the SpyExpenseSheetRepository that we just saw. The first observation we make uses the SavedExpenseSheet property of the SpyExpenseSheetRepository in order to verify whether the saved ExpenseSheet object contains the information that we specified through the command. The second observation merely verifies whether the return value of the Handle method indicates a successful outcome.

A Spy acts as an observation point for the indirect outputs of the Subject Under Test. It functions as a hatch for verifying these outputs by unit tests. It doesn’t do any verifications of its own in contrast to the next type of test double.

Mock

The fourth type of test double is called a Mock. A Mock does the exact same things as a Spy, only it’s more intelligent than a Spy and it knows what should happen regarding the indirect outputs of the Subject Under Test. A Mock returns values for the query methods of its public interface. It also captures the interactions with the command methods of its public interface. But it also provides the ability for a unit test to state its expectations, and after exercising the Subject Under Test, provides the ability to instruct the Mock to verify the outcome according to these expectations.

Let’s have a look at an example of a Mock.

public class MockExpenseSheetRepository : IExpenseSheetRepository
{
    private readonly ExpenseSheet _result;
    private ExpenseSheet _expectedExpenseSheetToBeSaved;
    private ExpenseSheet _savedExpenseSheet;
    
    public MockExpenseSheetRepository(ExpenseSheet result)
    {
        _result = result;
    }
    
    public ExpenseSheet Get(Guid id)
    {
        return _result;
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        _savedExpenseSheet = expenseSheet;
    }

    public void ExpectSaveToBeCalled(ExpenseSheet expenseSheetToBeSaved)
    {
        _expectedExpenseSheetToBeSaved = expenseSheetToBeSaved;
    }
    
    public void Verify()
    {
        if(null == _expectedExpenseSheetToBeSaved)
            return;
        
        Assert.That(_savedExpenseSheet, Is.Not.Null);
        Assert.That(_savedExpenseSheet.Id, 
            Is.EqualTo(_expectedExpenseSheetToBeSaved.Id));
        Assert.That(_savedExpenseSheet.EmployeeId, 
            Is.EqualTo(_expectedExpenseSheetToBeSaved.EmployeeId));
        Assert.That(_savedExpenseSheet.SubmissionDate, 
            Is.EqualTo(_expectedExpenseSheetToBeSaved.SubmissionDate));
    }
}

The MockExpenseSheetRepository class implements the IExpenseSheetRepository interface. The constructor of the mock expects an instance of an ExpenseSheet that can be returned by the Get method when and if called. It also assigns the ExpenseSheet object that is specified to the Save method in the _savedExpenseSheet member variable. Notice that there are two additional methods compared to the implementation of the spy that we’ve seen earlier. One is the ExpectSaveToBeCalled method which simply stores a specified ExpenseSheet object in the _expectedExpenseSheetToBeSaved member variable. The other one is the Verify method. It first checks whether the _expectedExpenseSheetToBeSaved member variable has been set. If not, then there’s nothing to verify. Otherwise, the ExpenseSheet object that is stored through the Save method is verified and compared against the expected ExpenseSheet object.

[Specification]
public class When_creating_a_new_expense_sheet__mock_example
{
    [Establish]
    public void Context()
    {
        var employee = Example.Employee().WithId(new Guid("680D0C0A-E445-4344-B67A-363589E2746A"));
        var stubEmployeeRepository = new StubEmployeeRepository(employee);

        var expenseSheetToBeSaved = Example.ExpenseSheet()
            .WithId(new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"))
            .WithEmployee(employee)
            .WithSubmissionDate(new DateTime(2018, 11, 11));
            
        _mockExpenseSheetRepository = new MockExpenseSheetRepository(null);
        _mockExpenseSheetRepository.ExpectSaveToBeCalled(expenseSheetToBeSaved);
        
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, _mockExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(
            new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"), 
            new Guid("680D0C0A-E445-4344-B67A-363589E2746A"), 
            new DateTime(2018, 11, 11));

        _result = _sut.Handle(command);
    }
    
    [Observation]
    public void Then_the_approved_expense_sheet_should_be_saved()
    {
        _mockExpenseSheetRepository.Verify();
    }

    [Observation]
    public void Then_the_operation_should_succeed()
    {
        Assert.That(_result.IsSuccessful);
    }

    private MockExpenseSheetRepository _mockExpenseSheetRepository;
    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario also verifies the happy path of the Subject Under Test. Instead of a Spy we now create an instance of the CreateExpenseSheetHandler class using the MockExpenseSheetRepository that we just saw. The mock is instructed to expect a call to the Save method with a specific ExpenseSheet object containing the data that we expect. The observation that verifies whether a new ExpenseSheet object is saved by the CreateExpenseSheetHandler simply calls the Verify method on the mock. If things don’t work out as expected, then the MockExpenseSheetRepository will fail the test on its behalf.

There are generally two different types of mocks: strict and non-strict mock objects. Strict mocks expect that recorded calls are performed in a particular order. On the contrary, non-strict mocks don’t impose this restriction. The general guideline and best practice is to avoid strict mocks as much as possible. Unit tests that make use of strict mocks impose even more constraints on the implementation and are therefore more strongly coupled to the concrete implementation of the Subject Under Test. Therefore mock objects should almost always be non-strict.

So now we’ve seen the four different flavours of test doubles that are important for solitary tests. There’s one more type of test double that we’re going to briefly discuss now for the sake of completeness.

Fake

A different kind of test double compared to the other types of test doubles that we’ve seen so far is called a Fake. A Fake is an object that, from an outside view, simulates the exact same functionality and behaviour as the real implementation that it replaces. The difference with the other types of test doubles exists in the fact that a Fake is not controlled or observed by a test.

Let’s have a look at an example of a Fake.

public class FakeExpenseSheetRepository : IExpenseSheetRepository
{
    private readonly Dictionary<Guid, ExpenseSheet> _expenseSheets;
    
    public FakeExpenseSheetRepository()
    {
        _expenseSheets = new Dictionary<Guid, ExpenseSheet>();
    }
    
    public ExpenseSheet Get(Guid id)
    {
        var isFound = _expenseSheets.TryGetValue(id, out var expenseSheet);
        return isFound ? expenseSheet : null;
    }

    public void Save(ExpenseSheet expenseSheet)
    {
        var current = Get(expenseSheet.Id);
        var currentVersion = current?.Version ?? 0;
        
        if(currentVersion != expenseSheet.Version)
            throw new OptimisticConcurrencyException<ExpenseSheet>(expenseSheet.Id, currentVersion, 
                expenseSheet.Version);

        expenseSheet.Version += 1;
        _expenseSheets[expenseSheet.Id] = expenseSheet;
    }
}

The FakeExpenseSheetRepository class provides a fully functional implementation of an ExpenseSheetRepository, including code for enforcing optimistic concurrency. The only difference with the real implementation is that the FakeExpenseSheetRepository doesn’t access a data store. Instead it uses a Dictionary to store ExpenseSheet objects in memory.

[Specification]
public class When_creating_a_new_expense_sheet__fake_example
{
    [Establish]
    public void Context()
    {
        var employee = Example.Employee().WithId(new Guid("680D0C0A-E445-4344-B67A-363589E2746A"));
        var stubEmployeeRepository = new StubEmployeeRepository(employee);
            
        _fakeExpenseSheetRepository = new FakeExpenseSheetRepository();            
        _sut = new CreateExpenseSheetHandler(stubEmployeeRepository, _fakeExpenseSheetRepository);
    }

    [Because]
    public void Of()
    {
        var command = new CreateExpenseSheet(
            new Guid("4048A482-1CBA-45AA-8709-6409B9FD32E3"), 
            new Guid("680D0C0A-E445-4344-B67A-363589E2746A"), 
            new DateTime(2018, 11, 11));

        _result = _sut.Handle(command);
    }

    [Observation]
    public void Then_the_operation_should_succeed()
    {
        Assert.That(_result.IsSuccessful);
    }

    private FakeExpenseSheetRepository _fakeExpenseSheetRepository;
    private CreateExpenseSheetHandler _sut;
    private Result _result;
}

The test scenario now uses the FakeExpenseSheetRepository instead of a spy or a mock. Because the test fixture uses a fake, we are no longer able to verify whether a new ExpenseSheet object is saved in the repository. The only thing that we can do is verify whether the return value of the Handle method indicates a successful outcome.

One might argue that using fakes no longer have anything to do with behaviour verification. That’s indeed a statement that has a certain truth to it. They’re used by tests for reasons other than verification of indirect outputs. Generally speaking, Fake objects are not well suited for solitary tests. So I usually try to avoid them for solitary tests when possible.

On the other hand, Fake objects are more than useful for sociable tests. But caution is still advisable when using them on a large scale though. A fake implementation needs to be kept in sync with the real implementation at all times.

One way to accomplish this is to use Contract Tests. Contract tests not only verify the outside observable behaviour of out-of-process services like databases, web services, etc. … . They also run against the fake implementation to ensure 100% behavioural compatibility. This can involve some significant development effort. So carefully consider when and where to use this powerful technique.

Summary

This diagram provides an overview of the different types of test doubles and for which kinds of tests they are most often applicable. Learning about the behaviour of each type of test double is essential to understanding the different levels of coupling that they impose. More on that in a next blog post.

Indirect Inputs and Outputs

2020-04-01T10:30:00Z

Previously we discussed what state and behaviour verification is about. Depending on the nature of the production code that we’re designing, we might decide to apply one approach over the other. Sometimes this can be quite the dilemma that we face as developers. Which approach should we choose and in which circumstance? Robert C. Martin, also known as Uncle Bob, has named this tension field the “Uncertainty Principle of TDD”.

In this blog post, we’re going to dive deeper into the realm of behaviour verification. More specifically, we’re going to discuss indirect inputs and outputs.

When calling a method or a function that expects one or more arguments, we need to specify a value for these arguments. These are the (direct) inputs for that method or function. Likewise, a method or a function can also return a particular value. This is the (direct) output for that method or function.

Only a very few classes in a system can stand on their own while fulfilling their promised functionality. Most classes interact with one or more collaborators, also called dependencies, in order to provide the necessary capabilities.

When a class calls a method provided by a collaborator, and this method returns a particular value, then this value serves as an indirect input for the calling method. Such a method being called on a collaborator is called a query. Using this kind of method, we can query the dependency for a particular value. Note that in a well modelled system, queries have no side effects.

Likewise, when a class calls a method provided by a collaborator that doesn’t return any value, but only receives one or more arguments, then this invocation along with its arguments serves as an indirect output of the calling method. A method being called on a collaborator that provides no return value is called a command. Using such a method, we can instruct or “command” the dependency to do something for the caller. A command is most likely to have side effects.

A method or a function that is either a query or a command is described by the CQS principle. This acronym stands for Command Query Separation. and was devised by Bertrand Meyer, the creator of the Eiffel programming language. This principle simply states that every method should either be a query that returns data to the caller or a command that performs an action, but not both. In other words, “Asking a question should not change the answer”.

This is an important concept for unit tests that are using the behaviour verification approach. The general guideline and best practice is that a unit test should never verify whether a particular query method of a collaborator is being called. Indirect inputs should be treated the same way as a direct input. For a direct input, a unit test just needs to specify the necessary arguments for invoking a method of the Subject Under Test. Similarly, all required indirect input values should also be specified through the test doubles that are put in place. This is usually set up during the Arrange stage of a unit test. Direct inputs should never be verified in the Assert stage of a unit test. Avoid setting expectations for query methods at all costs.

[Establish]
public void Context()
{
    HeadOfDepartment headOfDepartment = Example.HeadOfDepartment().WithId(ApproverId);
    
    _expenseSheet = Example.ExpenseSheet()
        .WithId(ExpenseSheetId)
        .WithExpense(36.50m, new DateTime(2018, 11, 01), "Sushi dinner");

    var approverRepository = Substitute.For<IApproverRepository>();
    approverRepository.Get(ApproverId).Returns(headOfDepartment);

    _expenseSheetRepository = Substitute.For<IExpenseSheetRepository>();
    _expenseSheetRepository.Get(ExpenseSheetId).Returns(_expenseSheet);

    _sut = new ApproveExpenseSheetHandler(_expenseSheetRepository, approverRepository);    
}

Here we’ve specified two indirect inputs. One being an instance of the HeadOfDepartment class. This object is returned by the Get method of the ApproverRepository test double. The other being an instance of the ExpenseSheet class. This object is returned by the Get method of the ExpenseSheetRepository test double.

The test doubles that we’ve just highlighted in our example are called “Stubs”. Stubs just return fake results to the Subject Under Test. We do not perform any expectation checks on them whatsoever.

When the Subject Under Test provides an indirect output to a collaborator by means of calling a command method, we need to verify that this action has occurred. Indirect outputs should be treated the same way as a direct output. A direct output is verified by a unit test using an assert statement. Therefore an indirect output should also be verified. This verification is usually done during the Assert phase of a unit test by verifying an expectation on a test double. This is a slightly different way of asserting but should be treated the same way as a classic assert statement.

[Observation]
public void Then_the_approved_expense_sheet_should_be_saved()
{
    var lastReceivedCall = _expenseSheetRepository.ReceivedCalls().Last();
    Assert.That(lastReceivedCall, Is.Not.Null);
    Assert.That(lastReceivedCall.GetMethodInfo().Name, Is.EqualTo("Save"));
    
    var savedExpenseSheet = lastReceivedCall.GetArguments().First() as ExpenseSheet;
    Assert.That(savedExpenseSheet, Is.Not.Null);
    Assert.That(savedExpenseSheet, Is.SameAs(_expenseSheet));
}

Here we’ve made an observation whether the last called method on the ExpenseSheetRepository test double has been the Save method. We also ensured that the saved ExpenseSheet object is the one that we expect. If this is the case, then the test will pass. If not, then the test will fail.

The test double that we’ve just seen in our example is called a “Spy”. Spies are used to verify whether the collaboration between the Subject Under Test and the dependency has been carried out. A “Spy” is considered to spy on the relationship between the two parties.

Now that we’ve touched on the concepts of “Stubs” and “Spies”, in the next blog post we’ll discuss the other different types of test doubles that we have at our disposal.

State versus Behaviour Verification

2020-03-25T11:30:00Z

When reasoning about types of automated tests, I find it quite useful to reason about two different categories, namely solitary and sociable tests. Also see the test pyramid for a more detailed explanation.

Focusing on solitary tests, there are generally two different styles that are being used:

Solitary tests that perform state verification.
Solitary tests that perform behaviour verification.

State verification

Applying state verification means that we first exercise the Subject Under Test by calling one or more methods on an object. Then we use assertions to verify the state of the object. We may also verify any results returned by the method.

This is also known as the Detroit School of TDD. This nickname comes from its origins out of Extreme Programming, a well known development methodology used by Chrysler’s C3 project at the end of the 1990s. Kent Beck’s book Test-Driven Development By Example best describes this approach.

Let’s have a look at an example of state verification.

public class Resume
{
    private readonly IList<Experience> _experiences;
    public IEnumerable<Experience> Experiences => _experiences;

    public Resume()
    {
        _experiences = new List<Experience>();
    }

    public void AddExperience(string employer, string role, DateTime from, DateTime until)
    {            
        var newExperience = new Experience(employer, role, from, until);
        _experiences.Add(newExperience);
    }
}

public class Experience
{
    public string Employer { get; }
    public string Role { get; }
    public DateTime From { get; }
    public DateTime Until { get; }

    public Experience(string employer, string role, DateTime from, DateTime until)
    {
        Employer = employer;
        Role = role;
        From = from;
        Until = until;
    }
}

Here we have a system for managing online résumés. A user can add one or more experiences to a résumé. Therefore we have a Resume class with a method AddExperience. This method adds an Experience object to a collection.

[TestFixture]
public class When_adding_experience_to_a_resume
{
    [Test]
    public void Then_the_specified_experience_should_now_appear_on_the_resume()
    {
        var experienceFrom = new DateTime(2014, 09, 12);
        var experienceUntil = new DateTime(2017, 12, 31);
        
        var resume = new Resume();
        resume.AddExperience("Google", "Data analyst", experienceFrom, experienceUntil);

        var addedExperience = resume.Experiences.SingleOrDefault();
        Assert.That(addedExperience, Is.Not.Null);
        Assert.That(addedExperience.Employer, Is.EqualTo("Google"));
        Assert.That(addedExperience.Role, Is.EqualTo("Data analyst"));
        Assert.That(addedExperience.From, Is.EqualTo(experienceFrom));
        Assert.That(addedExperience.Until, Is.EqualTo(experienceUntil));
    }
}

Looking at the implementation of the corresponding unit test, we see that a Resume object is created and that the AddExperience method is being called. Then we verify whether a correct Experience object has been added to the collection of experiences. With this test we verify the new state of the Resume object.

State verification tests do not instrument the Subject Under Test to verify its interactions with other parts of the system. We only inspect the observable state of an object and the direct outputs of its methods. This approach tests the least possible implementation detail. It has the most notable advantage that these tests will continue to pass even if the internals of the SUT’s methods are changed without altering their observable behaviour.

There are also two slightly different styles of state verification, namely Procedural State Verification and Object State Verification. We’ll cover these two styles more in-depth in a later blog post.

Behaviour verification

Verifying the behaviour of the Subject Under Test implies the ability for instrumenting and verifying its interactions with other objects or other parts of the system. This is mostly done by using test doubles, either written manually or by using a framework.

This is also known as the London School of TDD. This nickname comes from the practices applied by the Extreme Programming community in London. The concepts of this process is most clearly described by the book Growing Object Oriented Software Guided By Tests, written by Steve Freeman and Nat Pryce, also known as the GOOS book.

Let’s have a look at an example of behaviour verification.

public class RegisterUserHandler
{
    private readonly IUserRepository _userRepository;
    private readonly IEmailSender _emailSender;

    public RegisterUserHandler(
        IUserRepository userRepository,
        IEmailSender emailSender)
    {
        _userRepository = userRepository;
        _emailSender = emailSender;
    }
    
    public void Handle(RegisterUser command)
    {
        var user = new User(command.Email);
        _userRepository.Save(user);
        
        var emailMessage = new EmailMessage(user.Email, "Confirm email", "...");
        _emailSender.Send(emailMessage);
    }
}

public class RegisterUser
{
    public string Email { get; }
    
    public RegisterUser(string email)
    {
        Email = email;
    }
}

Here we have a system that manages user registration. We have a class named RegisterUserHandler that creates and saves a new user. Also a confirmation email is sent to verify the existence of the specified email address.

[TestFixture]
public class When_registering_a_new_user
{
    [Test]
    public void Then_the_new_user_should_be_registered_in_the_system()
    {
        var userRepository = Substitute.For<IUserRepository>();
        var emailSender = Substitute.For<IEmailSender>();
        
        var sut = new RegisterUserHandler(userRepository, emailSender);
        
        var command = new RegisterUser("john@doe.com");
        sut.Handle(command);
        
        userRepository.Received().Save(Arg.Any<User>());
    }

    [Test]
    public void Then_a_confirmation_email_should_be_sent()
    {
        var userRepository = Substitute.For<IUserRepository>();
        var emailSender = Substitute.For<IEmailSender>();
        
        var sut = new RegisterUserHandler(userRepository, emailSender);
        
        var command = new RegisterUser("john@doe.com");
        sut.Handle(command);
        
        emailSender.Received().Send(Arg.Any<EmailMessage>()); 
    }
}

Looking at the implementation of the unit tests, notice that we’re using the NSubstitute mocking library to verify that the collaborators of the RegisterUserHandler class are being called.

We pass those test doubles to the RegisterUserHandler class using constructor injection. For more information on Dependency Injection, you can check out the excellent book Dependency Injection by Mark Seemann. With these tests we verify the behaviour of the RegisterUserHandler class.

Applying this approach is a bit more complicated and harder to reason about than using state verification. These tests are also more brittle as they imply that the test code has intimate knowledge about the implementation details of the Subject Under Test. Changing the implementation details without changing the overall functionality will more likely result in failing tests.

There are a few techniques and design principles that can somewhat reduce this brittleness. The most important one is to strive for the least amount of collaborators. This way interactions can be verified effectively without sacrificing maintainability.

From this explanation, we might come to the conclusion that only applying state verification is the best approach for writing tests. And to a certain degree this is true. We should favour state verification over behaviour verification most of the time. But in practice, we actually need both kinds of verifications. Not every object in our system has the same role or responsibility. Some are more algorithmic and involves logic based code, while others involve more interactions between objects.

As you might have guessed, state verification is most useful for verifying algorithms and business logic, like domain objects, validators, static functions like helper or extension methods, etc. … Behaviour verification is most useful for verifying interactions, like services, controllers, gateways, etc. …

Being aware about state and behaviour verification is the first step to learn about how to write maintainable unit tests.

Cascading Failures

2020-03-18T11:30:00Z

In a previous blog post, we discussed the test pyramid and why it is a useful model to reason about a healthy mix of solitary and sociable tests.

I mentioned that at the bottom of the test pyramid, we have the most isolation. The more we move up the test pyramid, the more integration is employed which results in less design feedback. As soon as we move away from testing a single concrete class or Subject Under Test and start considering collaborations between several concrete implementations, we’re bound to encounter cascading failures. This means that a slightest change of the production code or a bug can result in a high number of failing tests that, from a conceptual point of view, don’t have a direct relation to the changed code.

Let’s have a look at an example.

public class Customer
{
    public int Id { get; }
    public string Email { get; private set; }
    public CustomerType Type { get; private set; }

    public Customer(int id, string email)
    {
        Id = id;
    }
    
    public void MakePreferred()
    {
        Type = CustomerType.Preferred;
    }

    public void ChangeEmail(string newEmail)
    {
        Email = newEmail;
    }
}

public enum CustomerType
{
    Regular = 0,
    Preferred = 1
}

Here we have a part of an application that manages customers. Users of the system can make a customer a preferred customer or change its email. A preferred customer receives some additional discounts and faster shipping. This is the implementation of the customer entity.

We also have two handler classes that receive commands for either making a customer preferred or changing its email.

//
// Make a customer preferred
//
public class MakeCustomerPreferredHandler
{
    private readonly AuthorizationService _authorizationService;
    private readonly ICustomerRepository _customerRepository;

    public MakeCustomerPreferredHandler(
        AuthorizationService authorizationService,
        ICustomerRepository customerRepository)
    {
        _authorizationService = authorizationService;
        _customerRepository = customerRepository;
    }
    
    public void Handle(MakeCustomerPreferred command)
    {
        if(! _authorizationService.IsAllowed(command.Action))
            ThrowUnauthorizedException(command.CustomerId);

        var customer = _customerRepository.Get(command.CustomerId);
        if(null == customer)
            ThrowUnknownCustomerException(command.CustomerId);
        
        customer.MakePreferred();
        _customerRepository.Save(customer);
    }

    private static void ThrowUnauthorizedException(int customerId)
    {
        var errorMessage = $"Not authorized to make customer (ID: {customerId}) a preferred 
                             customer.";
        throw new UnauthorizedException(errorMessage);
    }

    private static void ThrowUnknownCustomerException(int customerId)
    {
        var errorMessage = $"The customer with ID ${customerId} is not known by the system and 
                            therefore could not be made a preferred customer.";
        throw new UnknownCustomerException(errorMessage);
    }
}

public class MakeCustomerPreferred
{
    public CustomerAction Action { get; }
    public int CustomerId { get; }

    public MakeCustomerPreferred(int customerId)
    {
        Action = CustomerAction.MakePreferred;
        CustomerId = customerId;
    }
}

//
// Change the email of a customer
//
public class ChangeCustomerEmailHandler
{
    private readonly AuthorizationService _authorizationService;
    private readonly ICustomerRepository _customerRepository;

    public ChangeCustomerEmailHandler(
        AuthorizationService authorizationService,
        ICustomerRepository customerRepository)
    {
        _authorizationService = authorizationService;
        _customerRepository = customerRepository;
    }

    public void Handle(ChangeCustomerEmail command)
    {
        if(! _authorizationService.IsAllowed(command.Action))
            ThrowUnauthorizedException(command.CustomerId);

        var customer = _customerRepository.Get(command.CustomerId);
        if(null == customer)
            ThrowUnknownCustomerException(command.CustomerId);
        
        customer.ChangeEmail(command.NewEmail);
        _customerRepository.Save(customer);
    }
    
    private static void ThrowUnknownCustomerException(int customerId)
    {
        var errorMessage = $"The customer with ID ${customerId} is not known by the system and 
                             therefore it's email could not be changed.";
        throw new UnknownCustomerException(errorMessage);
    }

    private static void ThrowUnauthorizedException(int customerId)
    {
        var errorMessage = $"Not authorized to make customer (ID: {customerId}) a preferred 
                             customer.";
        throw new UnauthorizedException(errorMessage);
    }
}

public class ChangeCustomerEmail
{
    public CustomerAction Action { get; }
    public int CustomerId { get; }
    public string NewEmail { get; }

    public ChangeCustomerEmail(int customerId, string newEmail)
    {
        Action = CustomerAction.ChangeEmail;
        CustomerId = customerId;
        NewEmail = newEmail;
    }
}

Notice that these handler classes make use of an authorisation service that verifies whether the user is allowed to perform the operation.

We have two types of users in the system; help desk staff and back-office managers.

public class UserContext
{
    public UserRole Role { get; }

    public UserContext(UserRole role)
    {
        Role = role;
    }
}

public enum UserRole
{
    Unknown = 0,
    HelpDeskStaff = 1,
    BackOfficeManager = 2
}

Currently both types of users are allowed to make customers preferred or change their email. Only users that are not known by the system are disallowed all actions.

public class AuthorizationService
{
    private readonly UserContext _userContext;

    public AuthorizationService(UserContext userContext)
    {
        _userContext = userContext;
    }
    
    public bool IsAllowed(CustomerAction customerAction)
    {
        if(_userContext.Role == UserRole.Unknown)
            return false;
        
        // ...
        
        return true;
    }
}

public enum CustomerAction
{
    ChangeEmail = 0,
    MakePreferred = 1,
    // ...
}

Now the business has come up with a new requirement. From now on, only back-office managers are allowed to make customers preferred. Help desk staff are still allowed to change the email of our customers.

At this point, we make a change to the authorisation service to reflect this new requirement.

public bool IsAllowed(CustomerAction customerAction)
{
    if(_userContext.Role == UserRole.Unknown)
        return false;

    if(_userContext.Role == UserRole.HelpDeskStaff && customerAction == CustomerAction.MakePreferred)
        return false;

    // ...
    
    return true;
}

What we see now is that suddenly some tests for the MakeCustomerPreferredHandler start failing due to this change in the AuthorizationService. When we take a closer look, we see that the failing tests are using a concrete instance of the AuthorizationService.

[TestFixture]
public class When_an_authorized_user_makes_a_customer_preferred
{        
    [Test]
    public void Then_the_specified_customer_should_be_made_a_preferred_customer()
    {
        var customer = new Customer(354, "john@doe.com");
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.HelpDeskStaff);
        var authorizationService = new AuthorizationService(userContext);

        var customerRepository = Substitute.For<ICustomerRepository>();
        customerRepository.Get(Arg.Any<int>()).Returns(customer);
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        sut.Handle(command); 
        
        Assert.That(customer.Type, Is.EqualTo(CustomerType.Preferred));
    }
    
    [Test]
    public void Then_the_specified_customer_should_henceforth_be_treated_as_a_preferred_customer_by_the_system()
    {
        var customer = new Customer(354, "john@doe.com");
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.HelpDeskStaff);
        var authorizationService = new AuthorizationService(userContext);

        var customerRepository = Substitute.For<ICustomerRepository>();
        customerRepository.Get(Arg.Any<int>()).Returns(customer);
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        sut.Handle(command); 
        
        customerRepository.Received().Save(customer);
    }
}

[TestFixture]
public class When_an_unauthorized_user_attempts_to_make_a_customer_preferred
{        
    [Test]
    public void Then_an_exception_should_be_thrown()
    {
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.Unknown);
        var authorizationService = new AuthorizationService(userContext);
        
        var customerRepository = Substitute.For<ICustomerRepository>();
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        TestDelegate makeCustomerPreferred = () => sut.Handle(command);
        
        Assert.That(makeCustomerPreferred, Throws.InstanceOf<UnauthorizedException>());    
    }
}

[TestFixture]
public class When_an_authorized_user_attempts_to_make_a_customer_preferred_that_is_not_known_by_the_system
{               
    [Test]
    public void Then_an_exception_should_be_thrown()
    {
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a concrete instance of the AuthorizationService
        var userContext = new UserContext(UserRole.BackOfficeManager);
        var authorizationService = new AuthorizationService(userContext);
        
        var customerRepository = Substitute.For<ICustomerRepository>();
        customerRepository.Get(Arg.Any<int>()).ReturnsNull();
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        TestDelegate makeCustomerPreferred = () => sut.Handle(command);
        
        Assert.That(makeCustomerPreferred, Throws.InstanceOf<UnknownCustomerException>());
    }
}

This means that these tests are not testing the handler in isolation. They include, and are therefore dependent on, the implementation of the AuthorizationService. Therefore these are not 100% solitary tests.

Suppose that we have a few dozen handler classes like these with corresponding tests that each use a concrete instance of the AuthorizationService. Now this small change to the implementation of the AuthorizationService results in lots of failing tests. A simple change requested by the business can make these tests into a maintenance nightmare.

In this particular case we can easily fix these tests by changing the role that we pass to the UserContext. However, we can also take the path of further decoupling the tests.

We can introduce an interface for the AuthorizationService.

public interface IAuthorizationService
{
    bool IsAllowed(CustomerAction customerAction);
}

public class AuthorizationService : IAuthorizationService
{
    private readonly UserContext _userContext;

    public AuthorizationServiceV2(UserContext userContext)
    {
        _userContext = userContext;
    }
    
    public bool IsAllowed(CustomerAction customerAction)
    {
        if(_userContext.Role == UserRole.Unknown)
            return false;

        if(_userContext.Role == UserRole.HelpDeskStaff && customerAction == CustomerAction.MakePreferred)
            return false;

        // ...
        
        return true;
    }
}

Now we are able to use a test double, either written manually or generated dynamically using a mocking framework like we already did for the CustomerRepository. In our example we’ve used the NSubstitute library.

[TestFixture]
public class When_an_unauthorized_user_attempts_to_make_a_customer_preferred
{        
    [Test]
    public void Then_an_exception_should_be_thrown()
    {
        var command = new MakeCustomerPreferred(354);
        
        // Instantiate a test double for the AuthorizationService and disallow the action
        var authorizationService = Substitute.For<IAuthorizationService>();
        authorizationService.IsAllowed(CustomerAction.MakePreferred).Returns(false);
        
        var customerRepository = Substitute.For<ICustomerRepository>();
        
        var sut = new MakeCustomerPreferredHandler(authorizationService, customerRepository);
        TestDelegate makeCustomerPreferred = () => sut.Handle(command);
        
        Assert.That(makeCustomerPreferred, Throws.InstanceOf<UnauthorizedException>());    
    }
}

This way we have completely isolated the Subject Under Test and converted the tests for the handler classes to 100% solitary tests. This way we can make more changes to the AuthorizationService without breaking other tests (e.g. help desk staff is no longer allowed to change the email of a customer either).

Whenever a Subject Under Test requires collaborators for achieving its promised functionality, we basically have three options:

Use an instance of a concrete class.
Use an instance of a handwritten test double.
Use an instance provided by a test double library or framework.

If we choose the first option, we’re all set up for cascading failures. Cascading failures, as shown in this example, is one of the most important reasons why people lose faith in their tests. There are few things that can kill productivity and motivation faster than cascading failures. The goal should always be to have a loosely coupled design for both the production code as well as the test code that accompanies it. Therefore we should favour more fine-grained, solitary tests over course-grained, sociable tests. This way, changes to the Subject Under Test will only create failures in tests that are directly associated with it. Failures can be resolved more easily and quickly with fine-grained tests, sometimes even by just looking at the code. Failures of course-grained tests more frequently involves debugging the code under test. Debugging code requires a lot more mental cycles from software developers, which results in more time spent trying to find the offending code. And time takes money.

We can reduce the effects of cascading failures by replacing cross-boundary collaborators with test doubles, converting all tests into solitary tests. But do make sure to take a pragmatic approach. I like to emphasise that it’s still fine to involve the collaborators of a Subject Under Test when they live within the same boundary. An example of this is when writing solitary tests for an aggregate root inside the domain of an application.

The term “Subject Under Test” is a reminder to think carefully about the granularity of the unit of code that we want to design and therefore consume.

The Designing versus Testing Mindset

2020-03-12T11:30:00Z

Previously, we touched on two different categories of automated tests, namely solitary and sociable tests. Also we discussed how and why the test pyramid is a useful model to have a healthy mix of both kinds of tests. The bottom line of the previous blog post is that only having high-level sociable tests without any solitary tests, basically an “ice cream cone” testing strategy, can be quite detrimental.

Why? Because solitary and sociable tests each serve different purposes. They serve different purposes because they stem from a different mindset.

Solitary tests originate from a “design” mindset. Solitary tests apply a certain pressure on the design of the system. They clearly indicate design flaws when they pop up. In order to accomplish this, we isolate small parts of the system and drive their design using Test-Driven Development. The process of TDD is about learning and designing. The solitary tests that come out of this process are merely a useful side-effect.

Sociable tests are only about verifying an expected outcome, which is just fine and very much needed as well. Sociable tests originate from a “testing” mindset. They don’t drive or put any pressure on the design of the system. The reason for this is that they are inherently farther removed from the detailed implementation. Sociable tests are usually more verbose and therefore take longer to write. Because their inherit nature of taking more time to write and to run, it’s not really feasible to let them participate in a fast feedback process.

There is a process named Acceptance Test-Driven Development (ATDD), which is related to TDD, but is more focused on being a collaborative process between users, testers and developers. This can be a useful process for driving out sociable tests.

The focus of solitary and sociable tests are completely different. And again, this is all fine because designing and testing software are two entirely different things. We need a lot of “designing” as well as “verification” activities in order to build useful software systems. Make sure to have both instead of choosing only one over the other.

Why Write Maintainable Unit Tests?

2020-03-04T17:00:00Z

Last week I published my very first video course titled Writing Maintainable Unit Tests. It surely has been a very learnful experience. Going through the process of creating the content, recording and editing the videos was definitely quite challenging. But why a video course specifically about writing unit tests?

Well, for starters, everything related to Test-Driven Development is something that is near and dear to my heart. I think that shouldn’t come as a big surprise to anyone who knows me personally.

Secondly, I hear more and more developers complain about unit tests these days. I find this trend to be both quite interesting as well as disturbing. I find it interesting in the sense that I want to learn more about what specifically bothers these developers about their unit tests. At the same time it disturbs me as our industry definitely needs more software developers writing unit tests instead of less.

About two years ago, I attended a well known developer conference. One of the talks was about refactoring legacy code. Not an unusual topic you might say. Indeed, tech people have been debating this topic at developer conferences for almost two decades now. Being the most interesting topic for that time slot, I decided to take a seat amidst the listening audience. About fifteen minutes into the talk, the speaker starts ranting against writing unit tests.

I still think it’s kind of strange to start a plea against unit tests during a talk about refactoring code. But anyway, in this person’s point of view, unit tests are a waste of time and valuable resources. The argument was that unit tests break all the time if and when someone decides to change or refactor some code. Accordingly, the solution was to just stop writing unit tests. They were causing a lot of pain. At this point, a human tendency tends to kick in. Instead of improving on the cause of the issue, the most common solution is to just get rid of the messenger. If that would be any kind of useful solution, we might as well stop writing code altogether. I guess that would save the end-users of the systems that we build a lot of grief. Obviously, we’re not having any of that.

Anyway, now comes the most interesting part. Instead of writing unit tests, the speaker highly recommended to only write sociable tests (aka functional tests) instead. The most important reason cited was that they don’t break as often as unit tests. Bam!

After I picked up my jaw from the floor, I decided then and there to create a course for other software developers on how to write maintainable unit tests. Nobody would argue that a well-designed software system is a pleasure to work with. The same goes for the unit tests that we write. Unit test code should meet the same quality standards as the production code that it consumes. When unit tests start causing grief, then either the design of the production code or their own design isn’t that optimal.

My course is all about facing such design issues head-on when they pop up instead of shying away from them. In a previous blog post, I already argued that we should listen to the vital signs of TDD. It is at these moments that unit tests try to teach us something. The only thing that we need to do is be willing to learn.

Announcing Video Course: Writing Maintainable Unit Tests

2020-02-26T11:00:00Z

I’m very proud to announce that my first video course has been published on Udemy. This course teaches software developers how to write maintainable and readable unit tests. It’s for people who are serious about investing in their own skills.

The launch sale opens today with an additional discount. You can use coupon code TDD-IS-AWESOME to get a huge saving until Monday, March 2, 2020. Have a look at the curriculum and let me know what you think. Any feedback is very much appreciated.

The Test Pyramid

2019-11-07T09:00:00Z

In the previous blog post, we touched on two different categories of automated tests, namely solitary and sociable tests. We also mentioned that in order to build maintainable and high-quality software systems, we need both solitary as well as sociable tests, but not in an equal amount. Although tests from both of these categories have their strong points and weaknesses, it’s more desirable to have several solitary tests combined with only a few accompanying sociable tests.

This is where the test pyramid comes in. Initially described by Mike Cohn in the book Succeeding with Agile, the test pyramid provides a visual representation of a beneficial balance of automated tests.

The concept of the test pyramid is made up from the following parts:

At the base, we have solitary tests. These should make up the largest amount of automated tests in the software system.
In the middle, we have sociable tests. These should definitely be part of the suite of automated tests, but in a significantly lesser amount than the number of sociable tests.
At the top, we have a handful to a dozen broad system tests. This is usually in the form of UI or API tests that exercise the most important parts of the system. Try to resist the urge to create lots of these kind of tests as they can quickly turn against you and soon become a maintenance nightmare.

At the bottom of the test pyramid, we have the most isolation and also the fastest performance. This is where we get the most feedback about the design of the system. The more we move up the test pyramid, the more integration is employed and verified which results in slower tests and less feedback.

There are four primary reasons as to why we prefer that the majority of tests are solitary tests:

Sociable tests can be very slow and non-deterministic. This is due to the fact that they often cross their process boundary. They make failure diagnostics more difficult because they are further removed from the cause of the failure.
Sociable tests can therefore be overly verbose as they require a lot more code in order to setup parts of the system under test. It requires more effort and therefore takes more time to write.
Solitary tests apply a certain pressure on the design of the system. They clearly indicate the design flaws that might exist. Sociable tests on the other hand don’t provide such useful feedback about the design of the system because they are inherently farther removed from the details.
Sociable tests are highly susceptible to cascading failures. We’re going to elaborate further on this issue in a next post.

Do not bet on only a single category of these tests! Applying all of these categories to the system is the best way to achieve a well-tested code-base. However, a surprisingly large number of development teams have made the mistake of applying a testing pyramid that is upside down. Such an “ice cream cone” shaped automated testing strategy implies that there are more sociable tests than solitary tests. This anti-pattern usually stems from an overreliance on manual testing and/or a lack of applying the Test-Driven Development process.

Do bear in mind that the exact number of tests in each layer of the test pyramid is not really that relevant. However, the test pyramid is a very useful model to think and reason about. Let it be your guide on all your test automation efforts.

A Taxonomy Of Tests

2019-10-08T08:00:00Z

It’s more than fair to say that the terminology used in the world of automated tests can be a bit overwhelming. Software people have uncovered all sorts of tests in a wide variety of flavours.

For example, there are:

Unit tests
Integration tests
API tests
Database tests
Acceptance tests
UI tests
Performance tests
Regression tests
And much more …

You might have heard about some of these kinds of tests. All of these do have their usefulness for the specific purpose that they are serving. Nonetheless, for modern day software developers it’s sometimes quite hard to understand when a particular test falls into one or even multiple of these categories.

What’s even more intimidating is how you determine which of these kinds of tests are applicable to your work and the specific use cases that apply for you. Personally, I don’t feel that having a gazillion types of tests is very useful.

So having a more useful system for categorising tests seems to be in order. There has been a time where I’ve found the distinction between “fast” and “slow” tests to be useful. But coming up with a decent definition for both of these categories still remained to be somewhat non-deterministic.

At some point I’ve come to adopt the terminology used in the excellent book Working Effectively with Unit Tests, written by Jay Fields. Here the author proposes two broad categories, solitary and sociable tests. These can be seen as the equivalent of tests that either run very fast, and tests that have a wider variety of slowness.

Solitary tests only have two constraints:

The code being exercised by these tests never cross the process boundary in which they are executed.
A single class or module is being tested. This is also called the Subject Under Test or SUT for short.

Solitary tests focus on the individual parts, assuming that their collaborators work as expected. These are the cheapest tests that will cover the most ground. Solitary tests run very fast due to their nature of focusing on a single programmable unit. They are very fine-grained.

Sociable tests on the other hand are tests that cannot be classified as a solitary test. Tests that fall into this category are more course-grained as they usually exercise multiple classes or modules at the same time. They are more focused on the collaboration and integration of the different parts that make up a software system.

Sociable tests execute more slowly than solitary unit tests. This is due to the fact that they typically exercise more code and often cross their process boundary to communicate with other parts of the system like a database, a queue, the file system, the system clock, etc. …

Sociable tests can be easily identified by their need for configuration. This is usually by means of configuration files where all sorts of connection strings or file locations are being stored that can be used by both the part of the application under test as well as the sociable tests themselves for verification of the outcome.

In order to build maintainable and high-quality software systems, we need both solitary as well as sociable tests, but not in an equal amount. Although tests from both of these categories have their strong points and weaknesses, it’s more desirable to have several solitary tests combined with only a few accompanying sociable tests.

Fast Feedback

2019-09-03T08:00:00Z

Test-Driven Development is a discipline that exists for almost two decades now. Unfortunately, to this very day, it is still not without controversy. Most professional developers know that writing some form of automated tests can be quite beneficial for any type of codebase. But what still seems to be somewhat controversial is whether to write a test before or after the production code has been layed out. This discussion seems to stir its head every other day on many discussion forums and on social media.

Some people firmly follow the mantra of “Red, Green, Refactor”, writing a failing test before making it pass by adding a line or two of production code. Others don’t like to follow this strict process for whatever reason and prefer to write tests after they’ve written a part or the complete implementation. Personally I like to write my tests first. But I also would like to emphasize that I’m not writing this article to pass any judgement over anyone. For this article, I would just like to approach this topic from a slightly different angle.

Whether someone writes their tests before or after the implementation isn’t really a useful discussion. What I do believe to be valuable is the concept of a short feedback loop. This is what moves us forward. Before we dive into the topic of feedback loops, I would like to mention a small anecdote.

I once interviewed a software developer who claimed that he was a firm believer of TDD and the test-first approach. During the interview we sat down together and wrote some code. After I explained the requirements, he opened up a new source file, and started writing a simple unit test for a new method on a domain class. This new method would ultimately verify some invariants and change some of its internal state. The first unit test he wrote verified the happy path scenario. As this new method wasn’t implemented yet, the unit test obviously failed. Writing this failing test only took a couple of minutes. Then he switched over to the domain class. He did not only write just enough production code to make the unit test pass. He also wrote the full implementation for the method according to the assignment! This took him about 20 minutes in total, writing lots of code. He then ran the single unit test from before, and it passed. Obviously there where several test cases that simply weren’t there. Not only wasn’t he really following the process of “Red, Green, Refactor”. He just completely missed the point of having a short feedback process.

The term “feedback loop” is defined as follows:

“ The section of a control system that allows for feedback and self-correction and that adjusts its operation according to differences between the actual and the desired or optimal output.

— The Free Dictionary

For me, the most important aspect of a software system is it’s ability to provide feedback as fast as possible. Do we have a working system? Does all of it’s parts integrate correctly? Can we deploy the system right now? Does it run correctly in any of the environments in wich it is deployed? Does the system serve its users well? These are all important questions. But the really nice part is whether we can answer these questions with the blink of an eye. The same holds true while we’re actually working on the software itself.

Not only do we want to know whether the production code conforms to my current understanding of the business requirements. We also want fast feedback about the design of the system as well as the correctness of the unit test itself. We should never trust a test until we have seen it fail, regardless of whether you write a unit test first or after the fact. We should always make sure to see the test fail for the right reason. Writing tests first is more efficient than writing tests after the implementation because we start out with the premise of seeing a test fail. But I’ve also seen some developers, although not many, who have become quite proficient in writing tests after the production code while ensuring another form of short feedback process.

They first write a small amount of production code, then they write a test. They see the test pass, comment out the code they just added and see the test fail. Then they uncomment the code again and see the test pass (again). Again, this is somewhat less efficient and it might even take more discipline than simply following the traditional “Red, Green, Refactor”. But this still qualifies as a valuable feedback loop. So no judgement there.

What’s most important is being able to quickly verify that we’re making progress, that we’re heading into the right direction. That is what I find so appealing in Test-Driven Development.

But why stop there? Test-Driven Development is the best approach we have on most well-known platforms that we use. But there are other forms of feedback loops that can be highly beneficial as well. Take the tooling around the Clojure programming language for example. They have this thing called REPL-Driven Development where a developer constantly executes small pieces of code in a REPL. Sometimes, a few automated tests are added afterwards for regression or integration purposes. But most of the time, the calling code from the REPL is lost when its session has been closed. I truly find this process of REPL-Driven Development fascinating as code is being exercised in mere seconds instead of minutes. This approach can be applied for most LISP-family programming languages.

If you’re interested in finding out more, this excellent article manages to provide a good first impression about this process. I must admit that being able to witness a developer who is proficient at using Clojure is even more impressive.

All things summed up, for me the most valuable indicator for evaluating new technologies, methodologies and approaches is their ability to provide me with fast feedback. Learning, discussing and improving these feedback loops is what matters most.

Debugability Considered Useful

2019-01-03T09:00:00Z

Developers are lazy. There’s nothing new about that. We even pride ourselves on it. Sometimes being lazy is a good thing, but most of the time it’s not something we should brag too much about. Let’s have a look at a few examples of developer laziness in action.

public SomeViewModel Map(Something something)
{
    return new SomeViewModel
    {
        Name = something.Name,
        ListA = something.itemsA.Select(itemA => MapItemA(itemA, additionalParameter1, additionalParameter2)).ToList(),
        ListB = something.itemsB.Select(MapItemB).ToList(),
        ListC = something.itemsC.Select(itemC => MapItemC(itemC, additionalParameter3)).ToList(),
        ListD = something.itemsD.Select(itemD => MapItemD(itemD, additionalParameter4)).ToList()
    };
}

This is an example of some code that we’ve all seen before. Heck, I encounter this on a regular basis. You know, inlining code as much as possible in order to avoid declaring variables. The developer in question probably had a hard time coming up with variable names. Completely understandable. Coming up with names that express intent is one of the most difficult things that we face when writing code. But this is also where laziness becomes quite hurtful. Let’s have a look at another example.

foo.SomeMethod(new Bar(parameter1, parameter2), items.Where(item => item.Property1 == someFilterValue).Select(item => new Buzz { PropertyX = item.Property2, PropertyY = x.Property3 }).ToList());

This is a nasty one, isn’t it? While the previous example is still somewhat nicely formatted, this one is pretty bad. I’ve seen method calls like this where I had to horizontally scroll several pages in order to get to the closing semicolon. I sometimes wonder whether this is still laziness or just mere hatred towards the next person reading this code. Every time I’m horizontally scrolling in my code editor, I wonder what I did wrong to deserve such apathy. These examples communicate sloppiness and a disregard when it comes to readable code. In such cases I argue to improve the debuggability of the code.

“How difficult is it going to be in order to debug this code?” That is the question that we should ask ourselves more often. How difficult is it going to be to debug the code snippets from these two examples? It can be done, for sure. But is it going to be easy? Or is it going to be a bit more difficult than we would like?

For the first example, suppose that we want to inspect the items of the ListC property of the returned view model object. Where should we add a breakpoint? For the second example, suppose that we would like to inspect the items passed as the second argument to SomeMethod. How would we do that without any additional effort like stepping into the method to inspect the parameter or evaluating the LINQ expression itself during a debug session?

I’m definitely not advocating that we should spend more time using the debugger. In fact, debugging code is wasteful and we should avoid it as much as possible. But I do advocate for the debugability of code.

“ Debugging is worthless, but debuggability is everything.

— Some smart person

What do I mean by that? Let’s have a look at the improved versions of the two code example shown earlier.

public SomeViewModel Map(Something something)
{
    var listA = something.itemsA
        .Select(itemA => MapItemA(itemA, additionalParameter1, additionalParameter2))
        .ToList();
    
    var listB = something.itemsB
        .Select(MapItemB)
        .ToList();
    
    var listC = something.itemsC
        .Select(itemC => MapItemC(itemC, additionalParameter3))
        .ToList();
    
    var listD = something.itemsD
        .Select(itemD => MapItemD(itemD, additionalParameter4))
        .ToList();

    var result = new SomeViewModel
    {
        Name = something.Name,
        ListA = listA,
        ListB = listB,
        ListC = listC,
        ListD = listD
    };

    return result;
}

var bar = new Bar(parameter1, parameter2);
var filteredItems = items
    .Where(item => item.Property1 == someFilterValue)
    .Select(item => new Buzz { PropertyX = item.Property2, PropertyY = item.Property3 })
    .ToList()

foo.SomeMethod(bar, filteredItems);

This sure looks like more code compared to the initial version. That might be true. But we also immensely improved the debugabbility of the code. For the first example, suppose that we would like to inspect the items of the ListC property of the returned view model. At this point we even have two options. We can set a breakpoint on the line where we return the result variable. But we can also set a breakpoint earlier on in order to inspect the listC variable. The same goes for the second example where we have better options to debug the code.

But most importantly, we also improved the readability of the code. We also no longer have to horizontally scroll in order to grasp what’s going on. These days, decent code editors like Rider display a vertical line at the 120 characters mark. Whenever crossing this line while typing, a mental alarm bell should start to go off and we should consider the readability of the code.

Code that is easy to debug, is also very readable and easy to reason about. Therefore I suggest we should consider this attribute of code quality more often.

Blaming Mocks For Bad Design

2018-12-18T09:00:00Z

A while back I stumbled upon this talk titled Built-in Fake Objects. After I’ve watched the first 20 minutes or so, I was so worked up that I almost threw away my iPad just out of complete annoyance. I did eventually came back to watch the rest of the talk. But what bothered me so much?

Well, the general theme of this talk is that using mocking frameworks is bad. In itself, there’s really nothing extraordinary about that. There are so many talks out there that all compete to bring you this Luddite message. But what particularly bothers me in this case are the arguments made against using mocking frameworks.

This is the example used to demonstrate that mocks are the reincarnation of the devil itself.

String report(User user) {
    return String.format(
        "Balance of %s is %d USD",
        user.profile().name(),
        user.account().balance().usd()
    );
}

This is the simple implementation of a method named report. It returns a string containing some information about a specified user. The following code snippet shows the implementation of the accompanying unit test.

@Test
void printsReport() {
    Profile profile = mock(Profile.class);
    Account account = mock(Account.class);
    doReturn("Jeffrey").when(account).name();
    Balance balance = mock(Balance.class);
    doReturn(123).when(balance).usd();
    doReturn(balance).when(account).balance();
    User user = mock(User.class);
    doReturn(profile).when(user).profile();
    doReturn(account).when(user).account();
    assertThat(
    	Foo.report(user),
        containsString("Balance of Jeffrey is 123 USD")
    );
}

First of all, I’m not going to disagree that the test code looks pretty bad. I just wonder how intentional the omission of blank lines actually was. But using this unit test as the poster child example of why one should never use a mocking library ever again is purposely jumping to conclusions. To be fair, I would never use mocks for this implementation either. But that doesn’t automatically mean that mocking libraries should be avoided at all cost, just because they shouldn’t be applied in a large number of cases. Its basically the same as saying that all web frameworks are evil because using them to create desktop applications causes a lot of pain. In a sense, one cannot argue with that. But I wouldn’t be so dogmatic about it either.

In my previous post, I argued that unit tests amplify the signals that are being sent by the design of the system. If this is the case, then using mocks in the test code amplify those same signals with a factor ten or even more. And instead of shooting the messenger, we should learn how to listen instead.

Again, I wouldn’t use mocks for testing the report method. I would just use plain old state verification instead, creating an instance of a User using the Test Data Builder pattern and passing this instance as an argument to the report method. No need for any more complexity as this would reduce the implementation of the unit test to just a couple of lines of code.

But this is not the path taken in this particular example. So let’s run with it to see how mocks are telling us that the design of the system is flawed. First of all, a mock is used for a Profile, and an Account and a Balance. Even the User is a mock object. Interfaces have been put in place in order to create these mock objects.

Notice how these interfaces only have properties. They consist only of data, not behaviour. This is alarm bell number 1. Now let’s have a closer look at the test code itself. Notice how mocks are returning instances of other mocks. This is alarm bell number 2. Then having a closer look at the implementation of the report method. Notice the chaining of the properties in order to get the necessary data. Bingo! Alarm bell number 3. We found a train wreck. A clear violation of the Law of Demeter.

Fixing this design issue would drastically improve the test code if we still wanted to hang on to the decision of using mocks. For me this drives home the point that developers tend to blame their unit tests, the use of mocks, etc. instead of blaming their own design choices.

Presenting such a dogmatic view that mocking frameworks are evil is quite harmful in itself. It doesn’t help anyone, especially those who are new to Test-Driven Development. I strongly believe that developers should educate themselves about the particular scenarios in which mocking frameworks are an excellent choice and when they are not a good option. This tension field, this choice between applying state verification or behaviour verification is also called the “Uncertainty Principle of TDD”. Personally, I tend to use test doubles when the class under test communicates with collaborators that cross a dependency inversion boundary which is clearly not the case in this particular example.

During the second part of the talk, the speaker presents his solution to the problem by using fakes instead of mocks. A fake is an object that, from an outside view, simulates the exact same functionality and behaviour as the real implementation that it replaces. Using fakes in a unit test is another anti-pattern, which is a whole new blog post in and of itself.

My conclusion it this: mocking libraries are a valuable tool when applied correctly. We should learn by making mistakes and by listening to the signals that the design of both the system as the tests themselves are sending us. For me this is the path to enlightenment, and not by throwing the baby out with the bath water.

Listening to the Vital Signs of TDD

2018-11-08T09:00:00Z

People who know me personally know that I like to go for a long run on a regular basis. I sit and work at a desk all day. So as part of the work I do, labouring the codes, I go for a one or two hour run every other day. I already wrote about this in the past.

Besides the obvious benefits one gets from physical exercise, I also learned something quite valuable. I learned how to listen to my body. Every runner knows that running long distances involves some kind of pain. Over time, one develops a certain threshold for enduring this suffering. I know this doesn’t sound like much fun, but it really is. You can take my word for it 😀.

But over time I learned how to read the signals that my body sends me. These signals can be very subtle. How easy is it to breathe and get air? How are my legs feeling? Do my muscles feel strained? If so, to what degree? I’m constantly evaluating. For example, after a minute or two I know exactly whether I’m able to go for a fast run or that I should take it rather slowly. Even on a number of occasions, I was able to predict upcoming health issues or injuries. In such case, I have the option to reduce the intensity of a workout or even stop altogether. In any case, when something’s up, I have to act accordingly and prevent worse.

I have a similar experience when I’m writing code using Test-Driven Development. I write a small, failing test. I make the test pass as quickly as possible. And then the most important step: I refactor. Very short, successive cycles where each cycle shouldn’t take up more than just a couple of minutes. But why do I point out the “Refactor phase” as the most important one? Because this is the moment where I listen to what the unit tests are trying to tell me. Just as listening to the signals of one’s body is the most important part of running long distances, the same holds true for the whole Test-Driven Development process. In order to write sustainable software, we as developers have to learn about how to be receptive to these signals. But what exactly should we listen for?

Experienced developers often tell you that unit tests drive the design, and in a sense that’s true. But there’s an important step that comes before that. First and foremost, unit tests provide very valuable feedback about the design of the system. From this feedback, design decisions start to emerge. Test-Driven Development can be very unforgiving when the code quality of the system under test is quite poor. When it takes a long time to write a single, failing unit test, then it’s already telling us that we need to take some things into consideration during the refactor phase. From the “Red, Green, Refactor” cycle, the “Red” and “Green” stages should move as quickly as possible. The “Refactor” stage can take bit longer.

This is usually the point where newcomers to Test-Driven Development are put off by this discipled practice. The “Refactor” stage is oftentimes reduced or skipped altogether. And as soon as it becomes difficult to write and maintain unit tests, they shoot the messenger. They blame the test themselves instead of listening and blaming the design of the system being tested. Just as newcomers to running often blame the excessive pain they endure instead of just acting according to the signals their body is sending them along the way. Bad design of the system leads to brittle unit tests of poor quality.

Here’s a small example to illustrate this.

public abstract class CustomerHandlers
{
    public void Handle(RemoveCustomer command)
    {
        // Remove a customer ...
    }
}

public class RegularCustomerHandlers : CustomerHandlers
{
    public void Handle(CreateRegularCustomer command)
    {
        // Create a new regular customer ...
    }
}

public class RegularVipCustomerHandlers : CustomerHandlers
{
    public void Handle(CreateVipCustomer command)
    {
        // Create a new VIP customer ...
    }
}

We have a system that models two different types of customers: a regular customer and a VIP customer. Creating one of these involves different kind of business logic, but removing a customer is the same for both types. The developer that implemented this functionality decided to create a different handler class for each type of customer. These specific handler classes in turn derive from an abstract base class that provides the implementation for removing a customer. Let’s have a look at the unit tests.

[TestFixture]
public class RegularCustomerHandlersTests
{
    [Test]
    public void TestScenario01ForRemoveCustomer()
    {
        // Test scenario 1 for removing a customer
    }

    [Test]
    public void TestScenario02ForRemoveCustomer()
    {
        // Test scenario 2 for removing a customer
    }

    [Test]
    public void TestScenarioForCreateRegularCustomer()
    {
        // Test scenario for creating a regular customer
    }
}

[TestFixture]
public class VipCustomerHandlersTests
{
    [Test]
    public void TestScenario01ForRemoveCustomer()
    {
        // Test scenario 1 (duplicate) for removing a customer
    }

    [Test]
    public void TestScenario02ForRemoveCustomer()
    {
        // Test scenario 2 (duplicate) for removing a customer
    }

    [Test]
    public void TestScenarioForCreateVipCustomer()
    {
        // Test scenario for creating a VIP customer
    }
}

A test fixture has been used for both concrete handler classes. But notice that both test fixtures contain identical unit tests for removing a customer. This is one example where the design of the production code somewhat looks reasonable for a developer, but where the tests are claiming otherwise.

Suppose that we need to make a change to the functionality of removing a customer. If we want use Test-Driven Development, which one of these unit tests should we change first. Those in the RegularCustomerHandlersTests, or in the VipCustomerHandlersTests or both? This smells rather fishy.

Also the developer must have noticed that it somehow wasn’t that easy to write unit tests for the removal functionality. The abstract base class cannot be instantiated, so a concrete class must be used in order to invoke the Handle method. Which one should be chosen? The RegularCustomerHandlers class or the VipCustomerHandlers, or maybe a creating a third one specifically for testing? In the end, probably one of the two has been chosen. In order to make up for the bad feeling, the unit tests for removing a customer have been copied over to the test fixture of the other handler once its functionality has been finished.

And this is what we end up with when we do not properly pick up the signals that unit tests are broadcasting. A slightly better design could be the following:

public class RemoveCustomerHandler
{
    public void Handle(RemoveCustomer command)
    {
        // Remove a customer ...
    }
}

public class CreateRegularCustomerHandler
{
    public void Handle(CreateRegularCustomer command)
    {
        // Create a new regular customer ...
    }
}

public class CreateRegularVipCustomerHandler
{
    public void Handle(CreateVipCustomer command)
    {
        // Create a new VIP customer ...
    }
}

Here we have a specific handler class for each command. Likewise the unit tests now look like this:

[TestFixture]
public class RemoveCustomerHandlerTests
{
    [Test]
    public void TestScenario01ForRemoveCustomer()
    {
        // Test scenario 1 for removing a customer
    }

    [Test]
    public void TestScenario02ForRemoveCustomer()
    {
        // Test scenario 2 for removing a customer
    }
}

[TestFixture]
public class CreateRegularCustomerHandlerTests
{
    [Test]
    public void TestScenarioForCreateRegularCustomer()
    {
        // Test scenario for creating a regular customer
    }
}

[TestFixture]
public class VipCustomerHandlerTests
{
    [Test]
    public void TestScenarioForCreateVipCustomer()
    {
        // Test scenario for creating a VIP customer
    }
}

Here we have a dedicated test fixture for each handler class.

When it’s difficult to write a unit test, it hints us that the production code needs to be changed. We need to refactor the code so that it becomes easy to test. Production code that is very easy to test, that allows us to write a unit test in just minutes or even seconds, is code that is responsive to change. This is what we should strive for.

But how can we learn to listen? Unfortunately, we can only learn this by doing. Practice, practice and then practice some more. There are plenty of code katas out there. It can not be overstated how important it is to constantly practice outside of the typical work scenarios. But also rigorously apply Test-Driven Development in your daily work as well. The same goes for running. Going out for a workout on a regular basis is how you can learn about yourself, what you’re physical capabilities are, and most importantly what you’re (currently) not capable of. This is how you can improve. Going for longer distances or running faster. This is how you can move forward.

Just as important as learning about Test-Driven Development, is to learn about software design as well. Learning about both at the same time should go hand in hand. When you learn how to pick up the signals from your unit tests, you’re bound to learn something about the design of the code as well. Try out different design approaches. Keep it going. All the time.

All in the Name of DRY

2018-09-25T08:00:00Z

A while back, I was doing a code review. At some point I encountered some constructs that I see quite often. While I was describing and building my case, I was thinking that I might as well write a blog post about it. The code that I encountered looked similar to the following:

public interface IUser
{
    Int32 Id { get; }
    String FirstName { get; }
    String LastName { get; }
    Boolean IsValid { get; }
}

public class ValidUser : IUser
{
    public Int32 Id { get; }
    public String FirstName { get; }
    public String LastName { get; }
    public virtual Boolean IsValid => true;

    public ValidUser(Int32 id, String firstName, String lastName)
    {
        Id = id;
        FirstName = firstName;
        LastName = lastName;
    }
}

public class InvalidUser : ValidUser
{
    public override Boolean IsValid => false;
    
	public InvalidUser(int id)
        : base(id, string.Empty, string.Empty)
    {}
}

There are several issues with this code. The first thing that I would like to mention is that I don’t like declaring properties on interfaces. But that’s not what I want to elaborate on in this blog post. The thing that really interested me was the starting premise that led the developer in question to make these design choices. And this starting premise is called the DRY principle.

DRY

The DRY principle is an acronym that stands for “Don’t Repeat Yourself”. This principle originates from the excellent book The Pragmatic Programmer written by Andrew Hunt and Dave Thomas. Around the same time, this principle has also been stated in the book Extreme Programming Explained by Kent Beck where it is described as the “Once and Only Once Rule”.

It states that:

“Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.”

Code that isn’t DRY has the same algorithms (=knowledge) expressed in two or more places. If you change one, you have to remember to change the code at the other places as well. A developer new to the team that performs this change probably doesn’t know about these other places. Therefore we say that such code is hard to change and maintain.

The DRY principle is a very important design rule that luckily has become quite popular over the years. It is said that it promotes the orthogonality of the code by isolating changes to only those parts of the system that must change.

Too DRY

Now that we know about the DRY principle and it’s benefits, let’s have another look at our code example. It seems that there’s a concept of a “valid user” and an “invalid user”. When looking at their implementation, the InvalidUser is derived from the ValidUser class. In object-oriented terms, when a class inherits from another class, this basically means that an “IS A” relationship is being established. In our example, this results in an “invalid user” also being a “valid user”, which is definitely a bit fishy to say the least.

I’m pretty sure that this code was written with the best intentions in mind. The developer in question wanted to adhere to the (too) DRY principle in order to avoid the same property definitions on both the ValidUser and the InvalidUser class. But here’s the thing.

Properties that happen to have the same name, that are defined on more than one class, do NOT qualify as code duplication!

So I wonder why some developers go to great lengths and introduce unnecessary complexity just to avoid a few property definitions. Going back to our code example, we notice that this design choice also has some indirect consequences as well.

By deriving the InvalidUser class from the ValidUser class, we basically give up type checking. There’s no way that I can write code that verifies whether a user is valid or not based on it’s type.

IUser user = ... // Retrieve an instance of a user
if(user is ValidUser)
    Console.WriteLine("Woohoo, a valid user");

The condition of the if statement will always be true, regardless of the type of user. This is probably the reason why the IsValid property has been added.

IUser user = ... // Retrieve an instance of a user
if(user.IsValid)
    Console.WriteLine("Woohoo, a valid user");

Less DRY

After performing some refactoring, the code looks a tiny bit better.

public interface IAmUser
{
    Int32 Id { get; }
    String FirstName { get; }
    String LastName { get; }
    Boolean IsValid { get; }
}

public class ValidUser : IAmUser
{
    public Int32 Id { get; }
    public String FirstName { get; }
    public String LastName { get; }
    public virtual Boolean IsValid => true;

    public ValidUser(Int32 id, String firstName, String lastName)
    {
        Id = id;
        FirstName = firstName;
        LastName = lastName;
    }
	
    public static IAmUser Invalidate(Int32 id)
    {
        return new InvalidUser(id);
    }
}

public class InvalidUser : IAmUser
{
    public Int32 Id { get; }
    public String FirstName { get; }
    public String LastName { get; }
    public virtual Boolean IsValid => false;
    
    public InvalidUser(Int32 id)
    {
        Id = id;
        FirstName = string.Empty;
        LastName = string.Empty;
    }
}

We got rid of the inheritance structure and just implemented the properties that are needed. We also added a static Invalidate method on the ValidUser class that creates an instance of the InvalidUser class.

There are several more things that can and should be improved, like the violation of the Interface Seggregation Principle (ISP) (again, I don’t like to define properties on interfaces), checking for equality by implementing the Equals/GetHashCode methods, etc. … But I hope I made my point.

Memo on O-Ring and Software Erosion

2016-03-03T09:00:00Z

One of the most fascinating documents I’ve read to date is the memo from Roger Boisjoly on O-Ring Erosion. The original target audience for this memo he’d written were the management folks of Morton Thiokol back in 1985, about six months before the Challenger disaster.

What I find so striking about this whole story is its resemblance to the field of software engineering. We software developers can relate to this story all too well. I’ve personally been in this situation more than once. Heck, some of us are in similar kinds of situations all the time.

Roger Boisjoly attempted to halt the launch of the Space Shuttle, unfortunately to no avail. In his particular case, a failure of the Space Shuttle would cost the lives of the astronauts on board. But what about the software that we are crafting? In most cases, a failure of the software would not result in the loss of human life. It would cost the company a particular amount of money. Worst case scenario, its reputation would be flushed down the drain alongside the money.

But what should we do when we encounter reliability issues in the software that we’re working on? Well, first of all, we make an assessment about the severity and the impact of these problems and fix them accordingly. But what if it takes us a couple of days, a week, two weeks or even a few months to fix things? Well, then we usually turn to the management staff, explaining in our best non-technical terms what’s going on and ask them for the time (aka budget) we need. But here comes the tricky part. What if they don’t want to listen? What if they don’t take us serious? What if they don’t trust us that our findings are correct? What if they simply ignore us, wondering why we still have the energy to even come talk to them after or during this dead march that we’re in? What if they do listen to our plea, wholeheartedly agree with everything we say, walk us back to the hallway and then simply don’t give us anything except more business requirements and feature requests?

But now on to the million-dollar question. What do we software developers do when we don’t get the time to do the right thing? Well, we could try to fix things in our spare time, sacrificing the precious time we have with our family and friends or perhaps even our sleep for the next couple of days, weeks, months, perhaps even years? I don’t think that this is very sustainable. Or we could simply ignore management, stop adding new features and business capabilities and start fixing things on our own behalf, like some kind of software development team mutiny? I guess this will end up with one or more developers getting fired. Or we could start a campaign on social media about how software developers are being wronged at this or that particular company. Would anyone even care? And this will probably end up in some layoffs as well.

So what should we as professional software developers do to make things right in these kinds of scenarios? Maybe the people of the software craftsmanship movement could draft a script with concrete actions that guide software developers when they find themselves in the same situation as Roger Boisjoly?

I can only wonder what would happen to our industry if only 10% of all software developers would follow the actions described in such a manifesto. Wouldn’t that be useful advice?

Learning F# - The Thunderdome Principle for Functions

2015-12-26T09:00:00Z

Back in 2008, Jeremy Miller introduced the Thunderdome Principle, a technique he used for building maintainable ASP.NET MVC applications which later led to the FubuMVC open-source project. The basic premise of the Thunderdome Principle is to have all controller methods take in one ViewModel object (or none in some cases), and also return a single ViewModel object. One object enters, one object leaves!

What I like about F# is that the language designers took a similar approach when they implemented functions. In F#, every function accepts exactly one input and returns exactly one output. This is a different approach compared to C# where there is a distinction between functions that return values and those that don’t return values. This choice made by the designers of the C# programming language even bleeds into the .NET framework itself by incorporating a Func delegate and an Action delegate.

But F# is having none of that! Let’s have a look at the following function and it’s corresponding type annotation:

let multiply x y = x * y
> val multiply : x:int -> y:int -> int

At first glance this looks like a function that takes two arguments and returns a result. But the type annotation tells us a different story. Here multiply is bound to a function that takes an integer argument “x” and returns a function. This second function takes another integer argument “y” and returns an integer.

This approach of “one input value, one output value” has several benefits. One example is that the F# compiler can assume that the last expression in a function is also the return value. Therefore we didn’t need to use the return keyword in our example.

But another major benefit is partial application. Let’s have a look at the following code:

let multiplyByTen = multiply 10
> val multiplyByTen : (int -> int)

let result = multiplyByTen 5
> val result : int = 50

Here the F# compiler evaluated the call to the multiply function as far as possible, and simply bound multiplyByTen to the function for which it didn’t receive the parameter value. So partial application only works for function arguments from left to right.

That’s all fine, but what about functions that don’t take any arguments or don’t have a value to return? This is where the unit type comes in.

let logSomethingUseful = printfn "Hi there"
> val logSomethingUseful : unit = ()

Here we have a function with no arguments and no return value. However, behind the scenes, unit is passed to the function whereas unit is also returned by the function.

In my humble opinion, this whole “one input value, one output value” approach is by far a more cleaner model that is easier to understand. On several occasions while developing C# code, I wished that the .NET framework provided me with only Func delegates and a first-class void type. F# grants me this wish.

Until next time.

Learning F# - Passing Parameters to Functions

2015-12-17T09:00:00Z

One of the first issues I faced when learning F# was finding out how to specify multiple parameters to a function. While this might sound obvious when learning a functional programming language, I had a few confronting moments that forced me to unlearn things before I could make any progress.

I wanted to create a function that wrapped the Contains method of the String class. In C#, it would be implemented like this:

public class StringHelper
{
    public static Boolean Contains(String substr, String str)
    {
        return str.Contains(substr);
    } 
}

Calling this simple function is quite obvious:

StringHelper.Contains("pdf", "document.pdf")

My first attempt at writing the equivalent function in F# looked like this:

let contains (substr, str: string) = 
    str.Contains substr

I had to make a type annotation for the second argument because otherwise the F# compiler was unable to infer its type as a string. Calling this function looks like this:

contains ("pdf", "document.pdf")

All very nice. The code works and life is good. But as it turns out, there’s something going on with this implementation that I didn’t realise at the time. Turns out that the contains function isn’t a function that accepts two string arguments, but a single tuple argument of two strings!

Executing this code in the F# REPL shows the following type annotation for the contains function:

val contains : substr:string * str:string -> bool

I noticed a lot of examples where F# functions were being called without the braces and without commas separating the parameters. Calling the current implementation of the contains function this way gave me the following error:

contains "pdf" "document.pdf"

error FS0003: This value is not a function and cannot be applied

So instead, I removed the braces and comma from the function definition like so:

let contains substr str: string = 
    str.Contains substr

The compiler then gave me the following error message:

error FS0072: Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved.

I had quite some head scratching going on before I was finally able to figure this out. Apparently the second argument needs to be enclosed with braces because of the explicit type annotation.

let contains substr (str: string) = 
    str.Contains substr

contains "pdf" "document.pdf"

This time the F# REPL shows the following type annotation for the contains function:

val contains : substr:string -> t:string -> bool

This wasn’t quite what I expected at the time. As a developer who mostly writes C#, JavaScript code, I noticed how using comma-separated parameter/argument lists within braces was so engrained in my ability to read and write code. Even when dabbling with Clojure in my spare time I got no shortage of braces either. Even when writing Ruby code, I held on to the habit of using comma-separated parameter/argument lists enclosed in braces. I told myself that this would improve the readability of my code. But in fact it was my brain trying to keep me in the comfort zone.

At this point I’m quite comfortable with this syntax in F#. But it definitely took some time getting used to not adding braces/commas all over the place. I must say that the Troubleshooting F# page on Scott Wlaschin’s website was a great help!

Until next time.

The Burden of Features in Software

2015-11-21T11:00:00Z

I’ve been removing a couple of dead features this week. You know, those features that senior people in organisations like to tell epic war stories about. Those mighty conversations at dinner parties, where a person involved talks about all the pain and sorrow, about how a particular capability ended up in the software, how (crappy) it got implemented, etc … . Everyone at the table is laughing. Some of them who were also there, adding more personal anecdotes to make the story more tragic.

But the good news is that these features are no longer relevant to the end-users. In some cases, these features don’t even work correctly anymore. But they are still there to haunt you as a software developer. They wander throughout the application as ghosts. Usually they are totally at odds with the current architecture and/or design. Whenever I come across something like that in the code, I turn it into a personal crusade to get the code out as soon as possible. Removing dead weight is equally important in our profession as adding new code. But how on earth did this feature get into the code base in the first place?

In a previous post I wrote about product and project focused software. In all cases, adding features to software adds a burden. There’s not only the cost of implementing and deploying it, but there’s also the (hidden) cost of maintaining it and further evolving it as the capabilities of the software grow or when the architecture evolves through time. Then there’s also the added complexity for the end-users, need for documentation, etc. …

In my experience, there tends to be more awereness in organisations where there is some form of a product focused mindset compared to “project” organizations. In “project” organizations, a particular feature might be necessary in order to succeed with a project or two. Afterwards, there might no longer be a need for this particular capability, but a (new) need for something else might arise. They pile up feature after feature after feature until they end up with a Big Ball of Mud. In these kind of organizations there is no threshold for adding new features.

I consider being cautious before adding new features a good thing. Adding capabilities over time that are harvested from actual needs tend to be more useful anyway compared to “we need this thing, and we need it by yesterweek”. As described in my previous post, things are usually no that absolute. Most organizations sit somewhere between completely “product” focused and completely “project” focused. But the really excellent ones refuse to add new capabilities to their software until not adding it becomes almost unresponsible.

The thing is that we should create awereness that crafting software is quite a fragile proces, both with ourselves as with our stakeholders.

Product or Project Focused

2015-10-24T10:00:00Z

A software development team in an organization should be able to focus on the core domain that reflects the business it’s serving. Developers on the team should be able to iterate and further refine the domain model based on the evolving input and feedback of the domain experts. The business people, domain experts and developers treat the software as a product, further evolving throughout a long period of time so far as it provides real value.

Sounds like an ideal world. As we all experienced during our careers, reality is almost never that shiny. Lots of businesses don’t see their software as a product. Instead they pride themselves as “project” organizations, always ready to sacrifice long-term thinking and quality for impossible deadlines. The result is usually a code-base with a lot of technical debt spread with project specific features. This is what we developers call the Big Ball of Mud.

What I just described here are two complete opposites. There are lots of businesses that have a product mindset, and there are lots that have a project mindset. But most organizations sit somewhere in between. In these organizations there is some kind of balance.

A while ago, fellow Elegant Coder David Starr wrote one of the best articles I’ve read in quite some time which is related to this topic. I urge you to read this excellent writing at least a couple of times.

As per the agile manifesto, the only real measure of success is working software. Success on a development team should never be measured in volume of tickets serviced. That sort of measure is appropriate for an operational support team.

In a project organization, development teams are commonly treated as an operational support team, usually not given the right amount of time to focus on the core domain of the business.

If your team is spending less than 75% of its time doing this, you may be on the verge of “going operational” which ultimately means even less time available for new capability development. “Going operational” often represents defeat for a development team.

As with all things in life, there should be a nice balance between product and project thinking with a majority of the focus on long-term development of the core domain model. But then again, there are things that science knows, and then there are things that companies do.

Premature Abstraction

2015-10-17T18:00:00Z

The first time I read the GoF book, I didn’t understand it. This was because I didn’t had a decent understanding of the principles of object-oriented programming at the time. A while after, I read the book Design Patterns Explained. In this excellent book the author formulated the core thought behind the design patterns in the GoF book. He states that when a particular concept varies, it should be encapsulated (by means of an abstract class or and interface). This was a real eye-opener for me and guided me to understanding the design patterns described in the GoF book.

But what I also learned after several years is that introducing abstractions in code is not a free ride either. There is a time and place where you want abstractions. I used to add abstractions in domain models all over the place, as soon as I possible could. Now I try to avoid them until the last responsible moment. Only when I gain a deeper understanding, when I start to see variations of the same concept popping up, when concrete implementations start to cause pain, only then am I prepared to consider adding a new abstraction.

Suppose that we have the concept of a communication channel in a particular domain model. If we have more than one communication channel, say SMS, telephone and email, then an abstraction is warranted. Then we might want to introduce an ICommunicationChannel interface. But if we only have SMS as communication channel, then no abstraction is needed. If new communication channels are being added over time, only then do we refactor the code.

Sometimes my brain tricks me into the premature abstraction path. It also happens that I’ve postponed introducing an abstraction a bit too long. I do make mistakes. Who knew? But I do gained some awareness over the years.

The Quest for the One True Static Site Generator

2015-10-10T08:00:00Z

When I started blogging back in 2005 I created my personal blog on Blogger. I’ve been using this excellent blogging service over the years, enjoying the luxuries of not having to deal with the intricacies and complexities of hosting providers and blogging engines. Somewhere last year I started to get interested by the rise of static site generators. These days every self-respecting programming language has one or more associated static site generator tools.

What particularly piques my interest in static site generators is their promise of simplicity. Just spitting out some static HTML, CSS and JavaScript files. Fast to serve by any web server. Back to basics.

So at some point I started my quest to move away from Blogger. The tool I used for the job was Octopress, built on top of the ever popular Jekyll. This meant that I could host my personal blog on GitHub. What’s not to like?

Everything migrated smoothly and I got things set up in no time. But somehow I lost my excitement when everything went online. During the migration I constantly had to Google questions and answers, never having the feeling that I knew what I was doing. As soon as I started working on a new blog post, I got frustrated by the impediments it posed on my writing flow. Having a backlog of 10 years of blog posts ultimately seemed to have a negative impact on the performance of the output generation process. Who knew?

So after almost a year of not blogging, I decided to migrate again to another static site generator. This time I went for raw performance by adopting DocPad. At least, that’s what I thought at the time.

I spent a lot of time in order to get things working. I chose Jade as my templating engine, but got really frustrated by the terseness of its syntax, constantly struggling with the significant whitespace issues. I’ve used Jade in a couple of other projects in the past but I did not remember that it caused me so much pain.

Furthermore, some DocPad plugins were generating errors and/or warnings because they were out-of-date or using some part of the internal DocPad API that had been rendered obsolete in a more recent version, etc. … I constantly had to make compromises regarding the blog features that I wanted to implement because I didn’t get things to work just the way I wanted. After a frustrating couple of weeks I got something going. I pushed it to a brand new server at DigitalOcean and lost interest again.

When I decided to write another blog post, I noticed how very slow the generation process went. Again the impediments and negative impact on my writing flow. Not cool.

A couple of months ago I ran into yet another static site generator named Nanoc via One Thing Well. This one has been around for several years now. I started reading the documentation and instantly got a prototype working. Before migrating my personal blog again, I decided to use it for another static website that I created for my son’s soccer team in order to discover its features and applicability.

I totally loved how simple and easy it was to get things set up. For this website I had a couple of specific issues that I wanted to solve, like quickly adding photo albums and news items. I was able to implement a new version of the website in no time, growing ever more enthusiastic about Nanoc. After this successful venture, I decided to migrate my blog yet again, this time to Nanoc.

Having created two static websites with Nanoc, I couldn’t be happier about how everything turned out. One of the things I like the most is that I can just use plain old HTML interspersed with some erb. Layouts, partials and captures are all at your disposal.

Compilation and routing rules contained by the rules file are pretty amazing and easy to set up. Suppose I want to set up custom routing for all articles on my blog, using the year, month an the slug. Here’s the code to accomplish this:

route '/articles/*' do
  	year, month, day, slug = /([0-9]+)\-([0-9]+)\-([0-9]+)\-([^\/.]+)/
  		.match(item.identifier).captures
  	"/#{year}/#{month}/#{slug}/index.html"
end

Processing markdown files and/or HTML files is as easy as adding the following compilation rule:

compile '/articles/*.{html,md}' do
	filter :erb
  	filter :kramdown, :enable_coderay => false, :input => 'GFM', :hard_wrap => false
  	layout '/article.*'
end

Here we first process the erb code, then process the markdown syntax using the GitHub dialect and render the output using the article layout. This is all I had to do in order to correctly process all legacy blog posts (HTML) as well as the newer ones (markdown).

Oh, and rendering the entire blog including 10 years of blog posts takes about 30 to 40 seconds on my machine, which is pretty awesome compared to Jekyll and especially Docpad.

Generating output pages from virtual items added from code is a pretty advanced scenario, but almost as easy to set up as the basic stuff. An example of this is generating a page for every unique tag used by a blog article.

def create_tag_pages
	get_list_of_tags.each do |tag|
		sanitizedTagName = sanitize_tag_name(tag)

		@items.create(
			"= render('_tag_page', :tag => '#{tag}')",           	
		  	{ :title => "#{tag}", :tag => "#{tag}", :is_hidden => true }, 
		  	"/tags/#{sanitizedTagName}/",
		  	:binary => false
		)
  	end
end

If you’re in need for a decent static site generator, then look no further. Using Nanoc is an easy and above all, a very fun way to generate static HTML content.

Until next time.

Using F# in Sublime Text on Linux Mint

2015-06-05T08:00:00Z

I’ve been learning about functional programming for quite some time now, trying to wrap my head around the various concepts that this paradigm has to offer. One of the languages that spiked my interest besides Clojure is F#.

The reason for this is quite obvious. As a software developer who uses the .NET framework on a daily basis, I regularly run into the limitations of C#. Embracing a powerful functional language that targets the CLR and mitigates most of these limitations definitely looks quite appealing.

My goal has been to limit my use of Visual Studio to the absolute bare minimum. Especially during my spare time where I develop all my hobby projects on Linux Mint. I’m currently dabbling with F# using my preferred editor, Sublime Text.

Setting up F# on Linux Mint is quite easy. You just have to run the following commands:

$ sudo apt-get update
$ sudo apt-get install mono-complete
$ sudo apt-get install fsharp

This couldn’t be more easy. However, using F# in Sublime Text takes a bit more configuration.

First of all, you have to install the F# package which provides F# syntax highlighting for Sublime Text.

Secondly, in order to run F# code from Sublime Text it is recommended that you install the SublimeREPL package. Unfortunately, there is a minor issue with SublimeREPL what makes that the F# REPL doesn’t work in Sublime Text on Linux. In order to fix this issue you can just install SublimeREPL using Package Control and manually apply the changes from this pull request.

After installing these packages, you can add the following custom key bindings to the Key Bindings - User file in Sublime Text.

{
	"keys": ["ctrl+alt+f"], 
	"command": "run_existing_window_command",
	"args": {
	    "id": "repl_f#", 
	    "file": "config/F/Main.sublime-menu"
	}
},
{
    "keys": ["ctrl+alt+r"], 
    "command": "repl_transfer_current",
    "args": {
        "scope": "selection"
    }
}

Now when you open a new group in Sublime Text and press the CTRL-ALT-F key combination, you get the following layout:

Selecting the block of F# code in the left pane and pressing the CTRL-ALT-R key combination executes the code in the REPL in the right pane. I personally find this very useful during my explorations of the F# language.

Until next time.

Update 01:

In response to this blog post, Guillermo López mentioned that he has been working on the FSharp package for SublimeText.

@JanVanRyswyck @dsyme This one adds autocompletion to F# in ST: https://t.co/FPICSZxo8W
— Guillermo López (@guillermoooo) June 8, 2015

This package brings autocompletion, live error checking and several other cool things to the table and can easily be installed using package control. Simply a must have when using F# in SublimeText.

Hacking Endeavours - A Tale About Having Fun

2015-01-29T09:00:00Z

Some time ago, I was talking to a colleague of mine at a company event. While we were there talking, one of us popped the following question:

What's the latest totally crazy thing that you hacked together in your spare time, which was a totally whacked thing to do as a professional developer, where normally all your alarm bells and whistles would be going of the chart, but totally carried you away at that exact moment and had boatloads of fun doing?

We exchanged several (war) stories. In the end, the conclusion of this conversation was how much fun we had doing these crazy things and how much we learned.

I want to share one of my stories. It’s not as crazy as creating your own database on top of Git, or creating your own operating system using just regular expressions. But still, it’s one of my numerous conquests.

A couple of years ago I created a website for my son’s soccer team. One of my weekly routines is publishing all the games that need to be played by all the teams of the soccer club. These games are all published at the beginning of the season on the website of the Belgian soccer league. During the season, the soccer league regularly posts new updates on their website without mentioning exactly what they changed. In order to save some work, I wanted to create a small tool just for me that I could run on weekly basis to retrieve all the data that I wanted, format it into HTML and automatically post on the website. I noticed that the website of the soccer league was using a Java applet that captured all the input and showed the games accordingly for a single team. So on one evening, I hacked together a small node.js script that would drive this Java applet (using request) and scrape the data that I needed (using cheerio), then output the data in HTML and automatically upload the HTML file to the new website using the Git command-line.

I had so much fun doing this. I was taking every shortcut I could make in order to get what I wanted. No unit tests, no code analysis, no design, no nothing. Just code. I started using this tool week after week, expecting it to stop working any time soon because some developer would make changes to the Java applet or something. The funny thing is that I’ve been using this tool for almost three years now without ever breaking down on me. I still didn’t polish it, or made it more “professional”. Heck, I still need to change it every week just to fill in the dates.

While I would highly protest doing something like this on the job, I recommend doing this kind of stuff all the time during your spare time. We should consider learning a new programming language, some new technology or a new tool as a free pass for doing crazy things with it. We should allow ourselves some time besides work to hack on as many things as possible, without commitment, deadlines or other hassles. Just releasing our brains into the wild and who knows with what we might end up with.

What’s the latest and greatest that you hacked together?