Blog

Contract Tests

February 1, 2023

One of my favorite awesome talks of all time is Integrated Tests Are a Scam by J.B. Rainsberger. I must admit that the title has a clickbait vibe to it. However, I very much recommend watching the video.

So what’s the deal with integrated tests? Why should they be considered as a scam? First of all, let’s clarify what an integrated test actually is. An integrated test is a high-level test that exercises multiple different behaviours of the system all together in one go. At first glance this doesn’t sound like a huge problem. We need tests like that, don’t we? Well, it kind of depends on the audience of the integrated test.

Remember from my previous post about the Testing Quadrant, that there are two quadrants of tests that support the team:

  • Developer tests: this is the quadrant where the Test Pyramid resides.
  • Functional tests: these are the kind of tests, as described by Behaviour-Driven Development (BDD) or Acceptance Test-Driven Development (ATDD), that prove to business stakeholders that certain features are available in the system.

The testing quadrant and the testing pyramid

When it comes to using integrated “functional” tests for providing the necessary confidence to business stakeholders, there’s not much of a problem there. Such tests often do require different parts of the system working together to provide the requested capabilities.

However, an issue arises when software developers start using integrated “developer” tests for proving the correctness of the system. So why do such integrated tests become an issue in this situation? For all the same reasons as when having too much Sociable tests:

  • They can be grudgingly slow, which results in a poor feedback cycle and a long-running CI pipeline.
  • They are quite often very non-deterministic. This undermines trust and confidence.
  • They are overly verbose as they require a lot of code to set up the system under test. This requires a serious investment when it comes to time and effort.
  • They don’t provide useful feedback about the design of the system.
  • They are highly susceptible to cascading failures.

Therefore, my advice, ad nauseam, is to adhere to the Test Pyramid as it’s still a highly valuable model to think about when it comes to the developer tests quadrant.

What about integration tests? Are those a scam as well? It kind of depends on how you use them. An integration test, as opposed to an integrated test, only verifies whether the different components for a single path running through the system correctly work together. If a sociable test is used for that purpose alone, then things are just fine. However, troubles usually start appearing the moment software developers decide to start using Sociable tests, or even worse integrated tests, as the sole means to verify correctness of the system at the expense of Solitary tests.

“ Strong integration tests, consisting of collaboration tests (clients using test doubles in place of collaborating services) and contract tests (showing that service implementations correctly behave the way clients expect) can provide the same level of confidence as integrated tests at a lower total cost of maintenance.
— J.B. Rainsberger - Clearing Up the Integrated Tests Scam

What J.B. suggests here is the use of contract tests for exercising those parts of the application that cross the process boundary to communicate with other parts of the system like a database, a queue, the file system, etc. … The purpose of a contract test is to verify whether an adapter for a collaborating service correctly implements a particular interface. This implies that the same suite of tests verifies the outside-observable behaviour of both the real implementation and a fake implementation.

A general overview of the concept of contract tests

In the context of an hexagonal architecture, this suite of contract tests ensures that the fake adapter behaves exactly the same as the real adapter. We can then use this fake implementation for Sociable “integration” tests that verify whether the different components of the application correctly work together.

In the next blog post we’ll discuss how to approach contract tests from a practical point of view. In the meantime, you can check out the Integrated Tests Are a Scam blog series to learn more first-hand.

If you and your team want to learn more about how to write maintainable unit tests and get the most out of TDD practices, make sure to have look at our trainings and workshops or check out the books section. Feel free to reach out at infonull@nullprincipal-itnull.be.

Profile picture of Jan Van Ryswyck

Jan Van Ryswyck

Thank you for visiting my blog. I’m a professional software developer since Y2K. A blogger since Y2K+5. Provider of training and coaching in XP practices. Curator of the Awesome Talks list. Past organizer of the European Virtual ALT.NET meetings. Thinking and learning about all kinds of technologies since forever.

Comments

Latest articles

Tags

Disclaimer

The opinions expressed on this blog are my own personal opinions. These do NOT represent anyone else’s view on the world in any way whatsoever.

About

Thank you for visiting my website. I’m a professional software developer since Y2K. A blogger since Y2K+5. Author of Writing Maintainable Unit Tests. Provider of training and coaching in XP practices. Curator of the Awesome Talks list. Thinking and learning about all kinds of technologies since forever.

Latest articles

Contact information

(+32) 496 38 00 82

infonull@nullprincipal-itnull.be

principal-it.eu
principal-it.be
janvanryswyck.com