NEWS AT SEI
This article was originally published in News at SEI on: April 1, 2005
Most organizations that employ the Capability Maturity Model approach (Software CMM and CMMI methodologies) use it to assess and improve the maturity of their software and systems engineering processes. Software maturity was, indeed, on the minds of officials at the Social Security Administration (SSA) Office of Systems as it began its software process improvement (SPI) program in the late 1990s. But it was another type of maturity that spurred them to take action immediately: nearly one-third of their workforce was due to retire within five years.
Couple with this turnover the impending explosion in the SSA workload as millions of baby boomers retire, and the SSA was facing a potentially devastating “perfect storm” of conditions—managing the largest workload in the organization’s history with a staff that did not have the experience of the departing veterans.
Moving from Heroes to Processes
The Office of Systems manages the development, acquisition, and use of SSA’s information-technology resources to support the agency’s business functions. The key to setting the office on a steady course to deal with the staff changes and evolving IT initiatives was to have well-documented systems and well-defined processes in place, says Rod Waltersdorff, director of the SPI program. Waltersdorff also directs the Division of Process Engineering, Project, and Customer Service in the Office of Systems.
Waltersdorff describes the pre-SPI climate at the Office of Systems as a “culture of heroes” that always depended on people—not processes—to get things done. Consequently, on every important project, the Office of Systems involved its most knowledgeable, experienced people. In many cases, however, these were the same staff members who were counting the days until retirement. A culture change was needed.
SPI: What, Why, and How
Led by Waltersdorff, the SPI group set out some defining principles—what they hoped to achieve, reasons for change, and the method for improvement—in three areas: process improvement, knowledge management, and customer relationship management.
The SPI group defined process improvement as the ability to develop and implement processes, standards, and architectures that help improve the likelihood of success and reduce the risk to a systems project. Process improvement was necessary because the Systems office could no longer depend on individual heroes for success and because interdependencies in development activity and sponsor involvement require discipline. Waltersdorff summarized the goals of SPI goals this way: “Systems wants to deliver what it promises, when it is promised, with the highest quality at the lowest cost.”
To make this happen, Systems began to work with project managers and teams to define best practices, standards, and architectures; provide process and technical guidance to Systems and non-Systems team members; and advise project leaders on project management techniques.
The SSA approach was to use two Software Engineering Institute (SEI)-related models, the Capability Maturity Model for Software (SW-CMM) and the IDEAL model. IDEAL is an organizational improvement model that serves as a roadmap for initiating, planning, and implementing improvement actions. Named for the five phases it describes—initiating, diagnosing, establishing, acting, and learning—IDEAL supports an infrastructure to guide organizations in planning and implementing an effective SPI program and is the founding strategy employed in delivering many SEI services.
A parallel effort in knowledge management was rolled into the SPI program to document corporate knowledge, transmit legacy knowledge, and exchange knowledge both within the division and with its customers, while a related skills inventory initiative helped the Office of Systems identify upcoming skill gaps. These efforts would ease the anticipated increase in workload of the baby-boom retirement wave, and it would allow Systems to respond better to changing customer-service expectations, which were driven by rapid advances in technology.
Likewise, improving responsiveness to customer needs and accountability to customers became an SPI priority. Project teams in Systems accomplished these goals by implementing a defined IT planning and tracking process, identifying key areas of customer satisfaction, and implementing service and customer-communication improvements.
The Right Tools
Throughout the SPI initiative, SSA developed several tools to improve communications and to document risks and processes.
To track project status and to keep project managers and customers informed, Systems developed an intranet-based management tool called the Vital Signs and Observation Report (VISOR) that is accessible to all Systems project managers, executives, sponsors, and customers.
VISOR uses a dashboard with color-coded blocks to provide at-a-glance indicators of project status. Each project, for example, is measured in categories including scope, plans, resources, and risks. Blue blocks indicate satisfactory status while yellow or red blocks represent potential or definite flags. The status blocks link to additional pages with more detailed information. Using VISOR, both project managers and customers can access status information in real time.
Figure 1: Vital Signs and Observation Report (VISOR)
The SSA Office of Systems developed an intranet-based management tool called the Vital Signs and Observation Report (VISOR), which is accessible to all Systems project managers, executives, sponsors, and customers. VISOR uses a dashboard with color-coded blocks to provide at-a-glance indicators of project status.
For projects that pose a particular challenge or risk, Systems employs another intranet-based tool called Risk Identification and Mitigation System (RIMS) to document risks and guide potential solutions.
The Project Resources Guide (PRIDE), a Web-based tool that defines project lifecycles, provides detailed process documentation for the different types of projects SSA Systems staff may need to plan and execute. Lifecycle categories include Internet, collaboration, and standard projects. The PRIDE documentation guides SSA staff through work requirements, processes, and products. Using the tool during project planning and analysis has helped to control scope creep and requirements volatility and by doing so has produced greater predictability in budget, timeliness, and quality.
The Journey to Maturity
By 2000, the Office of Systems decided to establish a baseline set of core practices that would be followed across the entire development organization and in November 2001, met those goals and achieved Maturity Level 2 of the SW-CMM.
As SSA embraced the Internet as a new and important service delivery channel for the public, Systems staff members realized they needed to develop and employ their most disciplined methods in this area of new technology. Consequently, the Office of Systems Electronic Services (OSES) became the focus of the push to Maturity Level 3.
Steve Kautsch, associate commissioner, OSES, led the effort, which culminated in achieving a Maturity Level 3 rating in February 2005. “We’ve faced enormous challenges over the past five years—building a new organization from the ground up, learning new development tools and technologies, shortening our development timeframes from 12 to 18 months down to 6 months, and growing from 0 to 200 employees,” said Kautsch. “The Office of Systems is often audited to ensure that we follow applicable laws, standards, and best practices, so we cannot cut corners to save time or money. We needed streamlined, efficient processes that manage risk and build in quality. The Office of Systems had selected CMM as the framework, and it helped us define, document, and manage the new processes needed to achieve these goals. OSES began working with the CMM framework in late 2001 and by 2003 had decided to target Maturity Level 3.
“First I made sure every member of the organization understood that process improvement was a serious commitment from the top,” said Kautsch. “I assigned dedicated resources to the effort. We worked closely with the SPI team and the SEI to understand CMM, then established an office-level working group to oversee the development and implementation of the new processes. We worked with all our stakeholders here at SSA to define and document the procedures in a centralized repository for easy access. We trained the staff extensively and enforced compliance using steps that were built into the process. We continually used measurement data such as test results and lessons-learned exercises to refine the procedures. It took three years of focused effort, but it was worth it.”
Kautsch cited some of the benefits OSES achieved through process improvement.
- It created standard, well-documented, reusable processes and procedures that reduce the learning curve for new staff and enable staff mobility among projects.
- It streamlined and overlapped tasks that reduced development time by 6 to12 months without sacrificing quality or incurring additional risks.
- It achieved a solid track record based on implementation of more than 50 interactive/transactional public Internet services on time, with the expected scope, and within budget.
- It developed reliable estimates for the cost and schedule of new projects that benefit SSA planners.
The SPI initiative measured key findings from its efforts in five categories: project team characteristics (including evaluations of application experience, language experience, and requirements volatility), productivity, predictability, project estimation, and effort trends. Most measures showed some improvement when comparing 2003 and 2004 results.
Productivity, for example, measured in function points per work month, improved by more than 30%, and the ability to accurately predict the amount of effort to complete a project increased by 12%. Although predictions of project duration based on work months have remained statistically flat, that deviation from the actual duration reported is extremely low at 3.5%. Even so, Waltersdorff interprets this as a success because predictability remained consistent although increasingly volatile projects were being managed by less-experienced staff. In fact, more than 25% of the current Systems staff fall into this category, and he anticipates that the number will rise to more than 30% by the end of the year. Waltersdorff attributes much of the success in these areas to the foundation of defined processes, knowledge management, and attrition planning through Systems’ skills-inventory initiative.
Another indication of the program’s success is public feedback. Early scores for three of SSA’s Internet services from the American Customer Satisfaction Index surveys range from 86-92 and are among the highest scores to date for government or private-sector applications.
SSA continues to refine its data collection and analysis to help gain greater insights into the effects of the SPI initiative.
As the SSA Office of Systems moves forward from its latest appraisal, the SPI initiative will continue the transition from SW-CMM to CMMI. The Office of Systems will also continue to explore its potential role as a contractor for other government organizations; outside customers now include the Centers for Medicare and Medicaid Services, for which SSA is developing an application to manage the recently approved Medicare prescription drug program.
In addition to the measurable data, Waltersdorff sees many intangible results from the intensive SPI effort. “There has been a conscious, noticeable change in the fabric of our organization. You can feel and see the difference in the way people act and talk. They have confidence in their ability,” he said. “Changes in the organization go well beyond our process improvement. We feel more mature, more disciplined about what we do.”
Kautsch agrees. “Staff members—even our new members—are highly trained and have the tools they need to succeed. They take satisfaction in producing quality services that are easy and convenient for our clients. Every success builds morale within the organization and raises our credibility both inside and outside the SSA. Project sponsors are satisfied because we do not surprise them with last-minute, unexpected delays or scope changes. It’s a win-win for everyone involved.”