Quest For Quality

May 25, 2003 (08:05 PM EDT)

Read the Original Article at

At West Bend Mutual Insurance Co., the list of job responsibilities for software developers has grown in recent months. In addition to creating applications that perform well and meet users' needs, developers must now use standard terms and templates, follow a new project-management methodology, learn to use tools that track defects, and study a slew of new guidelines for various stages of the development process.

It has required a huge commitment in time and a pretty penny in consulting fees. But the IT department's overhaul of its software quality-assurance program is necessary if it's to effectively serve the business as it moves more processes to the Web, says Amy Buechel, VP of IT business solutions. "Many IT departments are under budget constraints, and a short-sighted department probably wouldn't think this approach is realistic," Buechel says. "It's a commitment, but when you invest for the long term there can be short-term pain."

Quality Payoffs

Buechel is far from alone on her stand against poor-quality software. In a survey of 250 business-technology professionals conducted by InformationWeek Research in April, 39% say that improving the quality of software is a high priority at their companies; half name it somewhat of a priority. Only 10% call it a low priority. That's because poor software, whether it's developed internally or purchased from a vendor, directly hurts the business. Nearly half of survey respondents say software bugs or errors have led to higher costs, and more than one-third say these bugs have led to both higher costs and lost revenue.

Companies are taking a variety of approaches to address software quality, but nearly all require focusing on quality issues far earlier in software- or product-development cycles than what's been typical. Among InformationWeek survey respondents, more than half say their organizations have changed the development processes for in-house-developed software within the past 12 months to focus more on early detection of glitches, coding errors, and other flaws. And for packaged applications, some companies are spending more time to assess and test how those systems will perform once they're integrated into a company's existing IT infrastructure.

At West Bend, the changes to quality management have even extended to a pretest process. The company uses a new methodology from Compuware Corp., called Compuware application reliability solution, that lets developers assess the risk to the business of a particular application's failure, how much risk can be tolerated, and how much testing is needed to achieve that tolerance level. Test strategies and plans are then developed based on the risk assessment.

Widespread Problems

West Bend is using its more formalized development and testing procedures for a new Web application, due in July, that will automate tasks previously performed by clerical workers for insurance claims adjusters. To get an idea of how much time West Bend has spent on the process, Buechel makes a "conservative" estimate of 18 months of training for developers, and she says the training isn't over yet. It's those types of realities that may keep some companies from deploying a more formal testing process: One-third of respondents to InformationWeek's survey say cost is a significant barrier to improving the quality of their organization's software, but twice as many say lack of time is a deterrent.

Still, those who've invested in more-thorough methods that spotlight quality earlier in the development process say they're developing software faster and more efficiently and, according to 62% of survey respondents, they're increasing staff productivity. That's one reason L-3 Communication Systems, a developer of secure telecommunication systems for the military and commercial businesses, uses tools from MKS Inc. for testing and managing source code for devices such as spy-resistant military cell phones.

Because of the types of products it develops, L-3's need for software quality is an easy decision. But the increase in business that started after the Sept. 11, 2001, terrorist attacks and escalated with U.S. military action in the Middle East has required more software work, prompting the company to increase its development staff by 20% last year, says Robert Greene, software-process-improvement manager. And that's where most of the quality issues arise. "We're hiring younger engineers and bringing in engineers from other companies," he says.

L-3 requires its senior engineers to review the code of new hires so that errors are caught before the integration phase. That review process used to require senior engineers to spend a half-day poring over lines of code, but a new automated process using the MKS software lets junior engineers check for and correct errors before submitting the code to senior peers for review. The MKS software provides senior engineers with a status report on the code, which speeds up the review process to about 30 minutes, Greene says. "We're being asked to produce products faster than we've done in the past, but we can't reduce the quality of our products." The solution is to address and fix problems earlier in the development cycle. Greene acknowledges that it takes engineers hours of training to learn how to use the MKS software, but that's far better than tying up the time of several developers to find a problem that crops up later in the development cycle.

At Home Buyers Warranty Corp., improvements to quality-assurance efforts have reduced the project cycle by about 75% in the past few years, says Jim Tallant, director of MIS. The company used tools and methodologies from IBM Rational to look at internally developed software. But Tallant, who has been with the company seven years, remembers when things were different. "We had a lot of problems with lack of structure and lack of method and process," he says. "When you don't have a good process, the scope gets out of hand, you blow the budget, you deliver everything late, and you deliver applications that don't perform well."

The biggest change has been the implementation of IBM Rational's unified process methodology, which, in its most basic form, requires developers to produce chunks of proof-of-concept, executable code every eight weeks, rather than spend months producing code and then seeking feedback. By getting early user feedback on these pieces of code, Home Buyers' IT department can put together building blocks of code that have already satisfied users' needs in terms of functionality and performance, rather than having to go back many steps in the process to find poorly written software.

Few Raves

"Each eight weeks becomes a project cycle in itself, and you're going forward with everyone having complete knowledge of what's going on," Tallant says. But it's a process that was counterintuitive to Tallant at first: "I thought, if you do things faster, you're gong to make more errors." But the Rational methodology has let the company get the most difficult development work done--using software that lets developers model applications for risk assessment--even before the first piece of code is written. "Once you get that difficult stuff done, you're off to the horse races," Tallant says.

All of this comes at a cost, of course. Tallant says the IBM Rational software costs thousands of dollars per developer PC--one PC has $16,000 in licensed Rational software on it. VF Corp. uses software from Mercury Interactive Corp. to test packaged applications before integration from vendors such as i2 Technologies, PeopleSoft, and SAP--a process that costs thousands of dollars in license fees per app tested per developer seat. Still, the clothing maker can't risk disruptions to the business if software doesn't function the way it's expected to, says Leigh Koetter, VF's quality-assurance and application-testing manager. InformationWeek survey respondents agree: 88% say that improved software quality results in improved reliability, making it the leading benefit of software-quality processes among those surveyed.

VF continues to invest in automated testing tools for packaged applications as Mercury makes them available, because it's the only way the company can know how software will work in its environment without doing an actual rollout. "The best thing about packaged software is flexibility, and the worst thing about packaged software is flexibility," Koetter says. "There are thousands of ways to set up packaged software, and you have to make sure you've set it up correctly for your company."

Meanwhile, software companies have gotten serious about improving packaged apps. One of the most promising developments in the past year is the growing willingness of vendors to acknowledge that software quality is a problem. It was common to hear vendors blame quality problems on customers' inability to properly implement and use software, but vendors now realize that their responsibility for quality doesn't end once products ship. And well they should: 82% of InformationWeek survey respondents say their companies decided not to buy software from a vendor because it had a reputation for poor-quality code. And more than half say the industry is doing an unsatisfactory job ensuring that commercial software is bug-free.

Highest Hurdles

"In the past, the average developer didn't care what the customer experience was, he just cared about defects," says Alan Fletcher, VP of operations for the E-business suite-development group at Oracle. "We shifted our focus so that development is measured on customer experience." So if 20% of customers complain about performance problems with a particular module--even if there's nothing technically wrong with the code--Oracle will investigate other quality problems besides bugs. In such a case, Fletcher says, maybe the software is too difficult to implement, resulting in poor implementations and, ultimately, substandard performance.

Oracle also has increased its focus on finding software defects earlier in the development cycle, Fletcher says. For one, it has developed a formal methodology that its 5,000 engineers must follow, including standard terms and phrases and a set of metrics used to measure the effectiveness of the processes in place. A new site on the company's intranet serves as the main collaboration center for engineers, letting them track the business requirements for the software they're developing and manage development cycles, among other things. Previously, development status was sent via E-mail.

Collaboration among software vendors and users also is on the upswing. The Sustainable Computing Consortium, launched last year by Carnegie Mellon University to develop standards for software quality, is up to 45 member companies, including FedEx, General Motors, Hewlett-Packard, Mellon Financial, Microsoft, and Oracle. Member companies recently met in Phoenix to develop working committees.

As these companies recognize, software quality can't be ignored. The industry is young compared with most others, and many believe its growth will be stunted without better standards and processes around quality.

Illustration by Felix Sockwell