Blog

Plausibility Is Not Truth

November 10, 2026

A couple of years ago, I received a phone call while I was in a meeting. It was my mother. I left the meeting to take the call. After asking if I had a few moments to spare, she calmly told me that she thought she had found a hand grenade.

Once I had recovered from my initial shock, she asked me what to do next. I suggested she call the police and ask them to send a patrol to investigate. I also told her that it would be best to stay away from the place where she found the thing.

A couple of hours later, a police patrol dropped by and investigated the supposed hand grenade. They didn’t think it was an explosive of any kind, but they weren’t entirely sure. So they decided that the best way forward was to call in the bomb squad.

The next day, the bomb squad arrived at my parents’ house fully geared up. After a thorough investigation, they concluded that it was fortunately just a part of an old chandelier that bore a strong resemblance to a hand grenade and not an explosive. Case closed without too much harm.

What we didn’t know was that a local journalist had picked up the story and written a short article about it. Much to my parents’ surprise, the short article appeared in the newspaper the next day, together with a picture of their house. After reading it a couple of times, I concluded that none of what the journalist had written about the incident had actually happened. Not a single word was true!

Then I realised that everything we see in a newspaper looks plausible. It could all have happened. But when you’re actually part of the story and have some expertise on the matter, you’ll notice that it’s highly inaccurate, if not a complete lie.

I noticed the same thing when using LLMs. I don’t know much about Infrastructure as Code tools such as Terraform, OpenTofu and Pulumi. When I ask an LLM to generate a Terraform configuration, for example, it provides me with a set of configuration files. When I inspect the output, it all looks pretty decent. It looks very plausible. When I apply the configuration, it usually works on the first try. This is the point at which we humans are impressed by the capabilities of an LLM. However, when I proudly show the result to a DevOps expert, they are usually much less enthusiastic and have lots of suggestions for improvements.

Conversely, when I ask an LLM to implement some code using the TDD workflow, the quality of the generated tests is often quite poor. This is not surprising, as the data used to train the model might not contain many examples of good, well-factored test code. I often have to request the LLM for improvements. I recognise that I might be biased, given that I wrote a book on how to write maintainable unit tests. So I have a few opinions on how to write decent tests.

As with the newspaper article, the output generated by LLMs appears highly plausible unless we have a strong grasp of the subject. However, as soon as we are familiar with the topic, when we have some level of expertise, we start to notice the flaws in what the LLM actually produces.

Yes, some people try to be helpful by suggesting that I should provide better context or skills or switch to the latest and greatest LLM model altogether. Essentially, they’re telling me that I need to get better at the ‘black magic’ skills to make the Magic 8-Ball more accurate. But what we actually need more of in this case is more human understanding.

The point is not to avoid all LLM usage. Rather, we and the organisations we work for need to recognise that learning is the primary objective. It’s not about churning out pull requests with 5000 lines of code day in and day out. As Toyota has taught us for so many years, learning organisations perform much better than non-learning organisations. Understanding comes from learning, so anything that prevents humans from building mental models is simply waste.

LLMs are very useful. However, that doesn’t mean we should lose our judgement. We must avoid being overly impressed by their capabilities. As Richard Feynman once said:

“ The first principle is that you must not fool yourself. And you are the easiest person to fool

If you and your team want to learn more about how to write maintainable unit tests and get the most out of TDD practices, make sure to have look at my trainings and workshops or check out the books section. Feel free to reach out at infonull@nullprincipal-itnull.be.

Profile picture of Jan Van Ryswyck

Jan Van Ryswyck

Thank you for visiting my blog. I’m a professional software developer since Y2K. A blogger since Y2K+5. Author of Writing Maintainable Unit Tests. Teaching and coaching XP / Lean practices. Speaker at tech conferences. Curator of the Awesome Talks list. Thinking and learning about all kinds of technologies since forever.

Comments

Latest articles

Tags

Disclaimer

The opinions expressed on this blog are my own personal opinions. These do NOT represent anyone else’s view on the world in any way whatsoever.

About

Thank you for visiting my website. I’m a professional software developer since Y2K. A blogger since Y2K+5. Author of Writing Maintainable Unit Tests. Teaching and coaching XP / Lean practices. Speaker at tech conferences. Curator of the Awesome Talks list. Thinking and learning about all kinds of technologies since forever.

Latest articles

Contact information

(+32) 496 38 00 82

infonull@nullprincipal-itnull.be

principal-it.eu
principal-it.be
janvanryswyck.com