Quality Gate

Acceptance testing in business critical development projects is to make sure that the implementation is successful. All phases before acceptance can be agile. But if the acceptance phase is started before all the criteria is met, there will be problems.

Problems with acceptance testing, could this be the reason?

Too often the problems are discovered during acceptance testing.

• Testing is not progressing – too many errors
• Training is not completed
• Planning implementation is behind schedule
• Implementation is delayed etc.

Usually the problem is passing the quality gate before the criteria for acceptance testing is met.

• Execution is not complete
• Functional testing is not complete
• Implementation has not been planned from the business view
• Open issues, serious and critical errors
• Acceptance testing is poorly planned

The real number of open errors

If there are 100 open errors to be fixed, it’s wrong to calculate that just by fixing them the system is ready. Even with normal quality, re-testing 100 errors produces around 40 new errors. Testing these errors produces again 10 new errors. –> If there are now a 100 errors that need fixing, the final number will be about 155 errors to be fixed for the system to be ready for the implementation. –> Processing and re-testing 155 errors takes about 240hrs client’s worktime. (There’s a post coming soon about the method to calculate this!)

Why the quality gate needs to be strict

The idea of acceptance testing is to make sure the implementation is done safely, without compromising the business. One could say it is a way to marvel how the solution supports the business. Testers are usually people from the business side, not testing professionals.

The picture below shows the situation where acceptance testing has started too early. If the project processes are ok, testers have been informed beforehand about what is not yet ready. Quite often testers still need to find that issue only to get feedback like “Oh yeah, that doesn’t work, there’s a change request open about it”.

Note

• There’s no reason to accept processes that are not ready
• Most of the processes need to be re-tested, when execution is complete
• Testing unfinished processes is slow
• Testers take negative feedback to colleagues: “Nothing works, I wonder how we’re ever going to make sales after implementation”
• Every error logged from an unfinished function, takes about 2.5hrs project worktime, for nothing.
• Acceptance testing does not give the right impression of the quality of the software being implemented.

“We have to start acceptance testing, because we need to stay on schedule!”

Or do we, or is it even smart?

Project is late, execution is not complete, errors have not been fixed. But if we change the schedule, there’s that threat that people stop working. Resources start working on something else with a tighter deadline. People believe in the miracle that when you start acceptance testing on schedule, the implementation can be done on schedule as well.

Here’s the problem:

If you scheduled the acceptance testing to last three weeks, do you want that time to be used to work on unfinished tasks from previous phases as well?
Have you taken into account all those unfinished issues resource-wise? Keeping in mind that the unfinished issues slow down all progress.

Or would be better to first finish everything and then move on to acceptance testing? Which has less impact on implementation schedule?

Well there is always that possibility that everything works out perfectly fine ;-)