News

GAO and VA Squabble Over Agile Development Project, Procedures

• By David Ramel
• December 13, 2010

Exactly what Agile software development means and how it should be implemented came under unusual scrutiny in government circles recently as two federal agencies squabbled about delays in a project.

A Government Accountability Office (GAO) report released Dec. 1 claims the Department of Veterans Affairs (VA) has not implemented Agile software development practices correctly, resulting in delays to an automated system to process education benefits for veterans.

The report stated: "The department [VA] had not ensured that several key Agile practices were performed."

But the VA fired back, stating that the GAO may not know what it's talking about when it comes to Agile development, because of its possible "limited exposure to Agile methodology."

That prompted the GAO to retort: "Our audit team, which included a trained ScrumMaster, examined the department's use of Agile Scrum practices against documented and accepted methodologies and consulted with an expert in the field that is not only a ScrumMaster, but also an Agile Scrum trainer that has extensive experience in evaluating Agile system development projects."

All this back-and-forth occurred in one document (PDF here), because the VA's comments were based on a draft GAO report and were included in the final report to the House of Representatives Committee on Veterans' Affairs, along with the GAO's response to those comments.

Overall, the GAO reported that the VA "made important progress" in the project. "In addition," the report said, "the Agile process allowed the department the flexibility to accommodate legislative changes and provide functionality according to business priorities."

In another area of the report, the GAO said "VA has demonstrated key Agile practices that are essential to effectively managing its system development, but certain practices can be improved."

And the VA agreed with most of the GAO's recommendations regarding the project. Out of five recommendations, the VA concurred on three but did not concur on two.

It concurred with recommendations regarding the establishment of performance measures for goals; establishment of "bidirectional traceability between requirements and legislation, policies and business rules"; and the definition of conditions to consider work "done."

The two recommendations that met with non-concurrence by the VA, and its responses, were:

• Recommendation 4: Implement an oversight tool to clearly communicate velocity and the changes to project scope over time.

  VA Response: Non-Concur. Development metrics and models were established and implemented to forecast and measure development velocity. Based on lessons learned, these models were expanded to forecast and measure velocity at each scrum team.

• Recommendation 5: Improve the adequacy of the unit and functional testing processes to reduce the system rework.

  VA Response: Non-Concur. VA's testing approach is compatible with Agile development, where unit, functional and end-user testing are collaboratively accomplished and all significant errors are identified and resolved prior to deployment.

With the last word in its own report, the GAO stated: "We stand by our position that there is still an opportunity for the department to improve its new development process in accordance with our recommendations."