Blog

A Taxonomy Of Tests

October 8, 2019

It’s more than fair to say that the terminology used in the world of automated tests can be a bit overwhelming. Software people have uncovered all sorts of tests in a wide variety of flavours.

For example, there are:

  • Unit tests
  • Integration tests
  • API tests
  • Database tests
  • Acceptance tests
  • UI tests
  • Performance tests
  • Regression tests
  • And much more …

You might have heard about some of these kinds of tests. All of these do have their usefulness for the specific purpose that they are serving. Nonetheless, for modern day software developers it’s sometimes quite hard to understand when a particular test falls into one or even multiple of these categories.

What’s even more intimidating is how you determine which of these kinds of tests are applicable to your work and the specific use cases that apply for you. Personally, I don’t feel that having a gazillion types of tests is very useful.

So having a more useful system for categorising tests seems to be in order. There has been a time where I’ve found the distinction between “fast” and “slow” tests to be useful. But coming up with a decent definition for both of these categories still remained to be somewhat non-deterministic.

At some point I’ve come to adopt the terminology used in the excellent book Working Effectively with Unit Tests, written by Jay Fields. Here the author proposes two broad categories, solitary and sociable tests. These can be seen as the equivalent of tests that either run very fast, and tests that have a wider variety of slowness.

Solitary tests only have two constraints:

  1. The code being exercised by these tests never cross the process boundary in which they are executed.
  2. A single class or module is being tested. This is also called the Subject Under Test or SUT for short.

Solitary tests focus on the individual parts, assuming that their collaborators work as expected. These are the cheapest tests that will cover the most ground. Solitary tests run very fast due to their nature of focusing on a single programmable unit. They are very fine-grained.

Sociable tests on the other hand are tests that cannot be classified as a solitary test. Tests that fall into this category are more course-grained as they usually exercise multiple classes or modules at the same time. They are more focused on the collaboration and integration of the different parts that make up a software system.

Sociable tests execute more slowly than solitary unit tests. This is due to the fact that they typically exercise more code and often cross their process boundary to communicate with other parts of the system like a database, a queue, the file system, the system clock, etc. …

Sociable tests can be easily identified by their need for configuration. This is usually by means of configuration files where all sorts of connection strings or file locations are being stored that can be used by both the part of the application under test as well as the sociable tests themselves for verification of the outcome.

In order to build maintainable and high-quality software systems, we need both solitary as well as sociable tests, but not in an equal amount. Although tests from both of these categories have their strong points and weaknesses, it’s more desirable to have several solitary tests combined with only a few accompanying sociable tests.

More on that later in a next post.

If you and your team want to learn more about how to write maintainable unit tests and get the most out of TDD practices, make sure to have look at our trainings and workshops or check out the books section. Feel free to reach out at infonull@nullprincipal-itnull.be.

Profile picture of Jan Van Ryswyck

Jan Van Ryswyck

Thank you for visiting my blog. I’m a professional software developer since Y2K. A blogger since Y2K+5. Provider of training and coaching in XP practices. Curator of the Awesome Talks list. Past organizer of the European Virtual ALT.NET meetings. Thinking and learning about all kinds of technologies since forever.

Comments

Latest articles

Tags

Disclaimer

The opinions expressed on this blog are my own personal opinions. These do NOT represent anyone else’s view on the world in any way whatsoever.

About

Thank you for visiting my website. I’m a professional software developer since Y2K. A blogger since Y2K+5. Author of Writing Maintainable Unit Tests. Provider of training and coaching in XP practices. Curator of the Awesome Talks list. Thinking and learning about all kinds of technologies since forever.

Latest articles

Contact information

(+32) 496 38 00 82

infonull@nullprincipal-itnull.be

principal-it.eu
principal-it.be
janvanryswyck.com