Quality Architecture

The other day I was chatting to a good friend of mine about a webinar that she had watched. It was about testing strategies, such as testing pyramids and inverted testing pyramids. She wasn’t convinced that simply shifting the bulk of testing from unit tests to end-to-end tests can dramatically improve the quality of software.

Her point was that after adopting such strategies the tests will become more complex and brittle, and that’s expensive to support. Would such an increase in complexity lead to the superior quality of the final product? It’s something that’s difficult to measure at start of a project.

The answer is “it depends”, of course. Developers will happily push the responsibility of writing tests to the QA engineers. Dev managers will be happy as well - more lines of “useful” code is being written. The losing sides will be the QA teams and the end users.

The main problem with the modern quality assurance is that it starts too late in the overall software development lifecycle (SDLC). Quality should be provided at every stage of the process.

The quality of software doesn’t depend solely on the developers’ experience or powerful tooling. It doesn’t even directly depend on code coverage: I have seen too many examples of 100% covered code that was horribly broken. It depends on whether software was designed with quality in mind.

Software architects need to involve their QA engineers into discussions at the ‘boxes-and-lines’ stage. A good QA person will immediately ask questions, which help them identify their testing goals. For example, they can ask about the entry and exit points into a specific module. If the QA team cannot get access to the module (whether it’s a microservice, a stored procedure in the database, or a lambda in AWS), they need to understand how to test it without that access. UI and API tests won’t cover your periodically running tasks. Your database upgrade scripts may mangle your data and all tests still will be green.

My favourite QA people have been extremely curious about every aspect of the systems I have been building as an architect. Often, they identify potentials issues very early during the design stage, when the price of an error is nil.

Another good outcome of such an approach is that the QA team can estimate their real workload on the project. Often, developers say something like “it will take me 2 weeks to develop, and then it’s up to QA and however long it takes them to test”. In my experience, such projects take from 3 to 5 times longer than their original estimates. With the Quality Architecture approach, the guesstimates of delivery times get much more precise and closer to reality.

Also, the QA folks can start working towards their deliverables earlier and in parallel, without being considered as stumbling blocks for the whole team. How often have you seen project managers pestering a QA engineer asking when they’ll “close that Jira task”? “Not too often” is my answer as we have always given our QA people enough time by running alongside them to the finish line.

Does it sound too rosy? Yes. There are some issues with Quality Architecture. The first one: communication.

Software architects are often treated as demigods that speak 90% jargon and the rest in bad English (or their own language). Their time is precious, and they need to talk to too many people. My suggestion is to include your team’s QA person into some of these discussions; this will allow cutting out additional meetings.

The second problem is expertise. Not every QA engineer (or developer) is interested in the final result: they may not understand the purpose of the project, or the project may require some additional knowledge like networking or finances. Well, the choices are quite limited here: find another person, or better yet, educate your people.

Of course, this is a very quick intro into Quality Architecture. The actual approach spans into multiple dimensions, with some involving maths. I will try to get those put into writing at regular intervals, at least for my friend’s sake. It’s always good to have an interested reader!