When Quality is taken for granted – the results can be Costly or Lethal

When you think of Quality, especially software quality, what comes to mind?

Most people tend to consider data breaches as a really costly example. The recent MyHeritage website data security breach exposed details of 92 million users, and the MyFitnessPal hack exposed 150 million user’s private information.

However, even (easy to discover) software defects can be quite costly.

In 2017, Lottery wins on Christmas Day were a glitch so South Carolina won’t pay.

According to CNBC & Fox News, South Carolina residents who appeared to have won some money on their state’s Christmas-themed lottery game discovered they wouldn’t be getting their winnings, after all.

Lottery officials determined that there had been a glitch in the system and the state wouldn’t be paying for numerous tickets they believe suffered a printing error.

Imagine that.

This is something that ‘should’ have been easily caught. But if you don’t have a Quality mindset, then you won’t put together a testing process to catch these defects before they cause problems.

The tickets gave players the chance to win up to $500 if they matched Christmas-themed symbols in a tic-tac-toe game. The defect was obvious: the same symbol was repeated in nine different areas on the ticket, giving the player a $500 win.

State Lottery Officials said more than 70,000 erroneous tickets were printed, and ticket holders who cashed in their games before the error was discovered collected about $1.7 million, that they get to keep.

Hmmmmm….

Rather than take the time to create a simple scanning application, designed to look for duplications, mathematical inconsistencies and other known errors – which they could update over time – they cranked out these games in time for the Holidays.

How many times have we seen easily discovered, abnormal behavior defects escape to production?

For FDA regulated products they become CAPA (corrective action, preventative action) issues, which are properly reported and followed up on. Yet, they continue to appear.

Think of the Samsung G7 phone batteries catching fire or exploding, and the costs associated with that disaster – a $1 Billion write off. That’s a Billion with a B…

Whether it’s a data breach, erroneous lottery ticket, faulty cell phone battery or a deadly pacemaker embedded in someone’s chest – all of these issues are preventable if we’re willing to think ‘out of the box’ to find them.

What’s the problem?

All too often, teams trying to meet critical deadlines will rush products out the door before they’re ready.

Think about it. All the requirements are fully covered by endless happy path test cases, with complete, automated regression tests running around the clock. The hazards are listed and accounted for as well, probably included in the risk management plan.

Yet – these critical issues keep appearing.

Why is that?

I believe, if you want different results, then you have to take a different approach.

For example; how much time and effort are put into scenario-based, risk identification sessions to increase testing effectiveness, and reveal those hidden, critical defects?

How often do teams get together to create scenarios based upon actual user input – in order to discover the unexpected user interactions (or abnormal system behaviors) that generate the critical defects?

If you look at the FDA Recalls List you’ll observe the majority of the ‘reasons for recall’ are linked to either unexpected user interaction or abnormal system behavior.

Almost none of the root causes listed would be identified during typical FDA mandated (requirements-based) testing.

Yet, teams still define test coverage by the requirements traceability matrix – and they miss critical issues lurking in the code. Issues that are waiting to be exposed in the clinical environment where the abnormal system behavior and unexpected user interactions occur.

Which takes us back to the Lottery Ticket issue.

Two lawsuits, in Richland and Sumter Counties, have already been filed against the state lottery and Introlat (former lottery vendor) on behalf of people who bought tickets. Their legal counsel considers this a breach of contract if the lottery commission decided to deny them their rightful winnings.

Whether you agree with this lawsuit or not, it was all caused by a complete lack of basic Quality steps; a simple verification and validation program would’ve caught this issue before the tickets were ever issued.

Was it worth skipping a thorough Quality Process to beat a deadline – what do you think?