If there’s one thing that a piece of software or an app has to do, it’s work properly. Few things are more irritating than a program that suddenly crashes or spits out the wrong answer or loses your data. On a large scale, a bug-plagued program or app can be very costly to both the developer and the client. And if a bad piece of software is installed in a motor vehicle or a machine, it can even be dangerous.
According to a 2002 report by the US Department of Commerce’s National Institute of Standards and Technology (NIST), software is particularly prone to errors because of its mind-boggling complexity. Software products are routinely measured in millions of lines of code, and software developers spend up to eighty percent of development costs on identifying and correcting defects. According to the study, factors contributing to quality problems include limited liability by software vendors, marketing strategies, and decreasing returns on testing and debugging.
Software testing is a critical part of the development process. It provides stakeholders, developers and users with information about the functionality of the product or service under test. Depending on the testing method employed, software testing can be implemented at any time in the software development process. Traditionally, testing has most often occurred after the requirements have been defined and the coding process has been completed, while in the Agile approach the testing effort is ongoing.
The Economic Savings of Testing:
The 2002 NIST study reported that software bugs are so prevalent and so detrimental that they cost the US economy an estimated $59.5 billion annually. At the national level, over half of the costs are borne by software users and the remainder by software developers/vendors. The report states that more than a third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. Currently, over half of all errors are not found until late in the development process or after the formal launch.
The catastrophic impacts of some failures are well known. In 2013, we saw a massive software failure that could have been avoided by rigorous testing before release. The rollout of Healthcare.gov, the healthcare exchange website operated by the United States federal government in support of the Affordable Care Act, was pronounced by all observers regardless of political affiliation to be a disaster. Analysis by the Reuters news agency in mid-October stated that the total contract-based cost of building Healthcare.gov swelled by a full three times from its initial estimate of $93.7 million to about $292 million. It’s getting fixed, but at a steep price.
What were the problems? Changing and complex requirements are by no means unusual in a large, expensive custom software project, but methodologies such as agile software development have been developed to cope with them. As reported in Bloomberg BusinessWeek, the solution is to create more transparency, not less: code reviews, repetitive “unit tests” to automatically find flaws, and the constant release of new iterations into the real world, where it’s used by consumers. Unfortunately, regulations pertaining to large government contracts in many countries, including the United States, stifle effective agile software development.
It’s an axiom that the earlier a defect is found, the cheaper it is to fix it. For example, if a problem in the requirements is found only post-release, then it’s estimated to cost up to one hundred times more to fix than if it had already been found by the requirements review or in an agile iteration.
Agile Testing:
The more testing the better, and agile development recognizes that testing is not a separate phase, but an integral part of software development, along with coding. Agile testing empowers all members of a cross-functional agile team to ensure delivering the product value expected by the customer at frequent iterations. Agile teams use a “whole-team” approach to elicit feedback from customers, collaborating with the development team to turn these responses into executable specifications that guide coding. Testing and coding are done incrementally and iteratively, building up each feature until it provides enough value to release to production.
According to a 2002 report by the US Department of Commerce’s National Institute of Standards and Technology (NIST), software is particularly prone to errors because of its mind-boggling complexity. Software products are routinely measured in millions of lines of code, and software developers spend up to eighty percent of development costs on identifying and correcting defects. According to the study, factors contributing to quality problems include limited liability by software vendors, marketing strategies, and decreasing returns on testing and debugging.
Software testing is a critical part of the development process. It provides stakeholders, developers and users with information about the functionality of the product or service under test. Depending on the testing method employed, software testing can be implemented at any time in the software development process. Traditionally, testing has most often occurred after the requirements have been defined and the coding process has been completed, while in the Agile approach the testing effort is ongoing.
The Economic Savings of Testing:
The 2002 NIST study reported that software bugs are so prevalent and so detrimental that they cost the US economy an estimated $59.5 billion annually. At the national level, over half of the costs are borne by software users and the remainder by software developers/vendors. The report states that more than a third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. Currently, over half of all errors are not found until late in the development process or after the formal launch.
The catastrophic impacts of some failures are well known. In 2013, we saw a massive software failure that could have been avoided by rigorous testing before release. The rollout of Healthcare.gov, the healthcare exchange website operated by the United States federal government in support of the Affordable Care Act, was pronounced by all observers regardless of political affiliation to be a disaster. Analysis by the Reuters news agency in mid-October stated that the total contract-based cost of building Healthcare.gov swelled by a full three times from its initial estimate of $93.7 million to about $292 million. It’s getting fixed, but at a steep price.
What were the problems? Changing and complex requirements are by no means unusual in a large, expensive custom software project, but methodologies such as agile software development have been developed to cope with them. As reported in Bloomberg BusinessWeek, the solution is to create more transparency, not less: code reviews, repetitive “unit tests” to automatically find flaws, and the constant release of new iterations into the real world, where it’s used by consumers. Unfortunately, regulations pertaining to large government contracts in many countries, including the United States, stifle effective agile software development.
It’s an axiom that the earlier a defect is found, the cheaper it is to fix it. For example, if a problem in the requirements is found only post-release, then it’s estimated to cost up to one hundred times more to fix than if it had already been found by the requirements review or in an agile iteration.
Agile Testing:
The more testing the better, and agile development recognizes that testing is not a separate phase, but an integral part of software development, along with coding. Agile testing empowers all members of a cross-functional agile team to ensure delivering the product value expected by the customer at frequent iterations. Agile teams use a “whole-team” approach to elicit feedback from customers, collaborating with the development team to turn these responses into executable specifications that guide coding. Testing and coding are done incrementally and iteratively, building up each feature until it provides enough value to release to production.
No comments:
Post a Comment