document: OPM’s mission is to ensure that the federal government has an effective civilian workforce. In this regard, one of the agency’s major human resources tasks is to manage and administer the retirement program for federal employees. According to the agency, the program serves federal employees by providing (1) retirement compensation and (2) tools and options for retirement planning. OPM’s Center for Retirement and Insurance Services administers the two defined benefit retirement plans that provide retirement, disability, and survivor benefits to federal employees. The first plan, the Civil Service Retirement System (CSRS), provides retirement benefits for most federal employees hired before 1984. The second plan, the Federal Employees Retirement System (FERS), covers most employees hired in or after 1984 and provides benefits that include Social Security and a defined contribution system. According to OPM, there are approximately 2.9 million active federal employees and nearly 2.5 million retired federal employees. The agency’s March 2008 analysis of federal employment retirement data estimates that nearly 1 million active federal employees will be eligible to retire and almost 600,000 will most likely retire by 2016. Figure 1 summarizes the estimated number of employees eligible and likely to retire. OPM and employing agencies’ human resources and payroll offices are responsible for processing federal employees’ retirement applications. The process begins when an employee submits a paper retirement application to his or her employer’s human resources office and is completed when the individual begins receiving regular monthly benefit payments (as illustrated in fig. 2). Once an employee submits an application, the employing agency’s human resources office provides retirement counseling services to the employee and augments the retirement application with additional paperwork, such as a separation form that finalizes the date the employee will retire. Then the agency provides the retirement package to the employee’s payroll office. After the employee separates for retirement, the payroll office is responsible for reviewing the documents for correct signatures and information, making sure that all required forms have been submitted, and adding any additional paperwork that will be necessary for processing the retirement package. Once the payroll office has finalized the paperwork, the retirement package is mailed to OPM to continue the retirement process. Payroll offices are expected to submit the package to OPM within 30 days of the retiree’s separation date. Upon receipt of the retirement package, OPM calculates an interim payment based on information provided by the employing agency. The interim payments are partial payments that typically provide retirees with 80 percent of the total monthly benefit they will eventually receive. OPM then starts the process of analyzing the retirement application and associated paperwork to determine the total monthly benefit amount to which the retiree is entitled. This process includes collecting additional information from the employing agency’s human resources and payroll offices or from the retiree to ensure that all necessary data are available before calculating benefits. After OPM completes its review and authorizes payment, the retiree begins receiving 100 percent of the monthly retirement benefit payments. OPM then stores the paper retirement folder at the Retirement Operations Center in Boyers, Pennsylvania. According to the agency’s 2008 performance report, the average processing time from the date OPM receives the initial application to the time the retiree receives a full payment is 42 days. According to the Deputy Associate Director for the Center of Retirement and Insurance Services, about 200 employees are directly involved in processing the approximately 100,000 retirement applications OPM receives annually. This processing includes functions such as determining retirement eligibility, inputting data into benefit calculators, and providing customer service. The agency uses over 500 different procedures, laws, and regulations, which are documented on the agency’s internal Web site, to process retirement applications. For example, the site contains memorandums that outline new procedures for handling special retirement applications, such as those for disability or court orders. Further, OPM’s retirement processing involves the use of over 80 information systems that have approximately 400 interfaces with other internal and external systems. For instance, 26 internal systems interface with the Department of the Treasury to provide, among other things, information regarding the total amount of benefit payments to which an employee is entitled. OPM has stated that the federal employee retirement process currently does not provide prompt and complete benefit payments upon retirement, and that customer service expectations for more timely payments are increasing. The agency also reports that a greater workload is expected due to an anticipated increase in the number of retirement applications over the next decade, yet current retirement processing operations are at full capacity. Further, the agency has identified several factors that limit its ability to process retirement benefits in an efficient and timely manner. Specifically, it noted that current processes are paper-based and manually intensive, resulting in a higher number of errors and delays in providing benefit payments; the high costs, limited capabilities, and other problems with the existing information systems and processes pose increasing risks to the accuracy of benefit payments; current manual capabilities restrict customer service; federal employees have limited access to their retirement records, making planning for retirement difficult; and attracting qualified personnel to operate and maintain the antiquated retirement systems, which have about 3 million lines of custom programming, is challenging. In the late 1980s, OPM recognized the need to automate and modernize its retirement processing and began retirement modernization initiatives that have continuously called for automating its antiquated paper-based processes. The agency’s previously established program management plans included the objectives of having timely and accurate retirement benefit payments and more efficient and flexible processes. For example, the agency’s plans call for processing retirement applications and providing retirees 100 percent of their monthly benefit payments the day it is due versus providing interim monthly payments. Its initial modernization vision called for providing prompt and complete benefit payments by developing an integrated system and automated processes. However, the agency has faced significant and long-standing challenges in doing so. In early 1987, OPM began a program called the FERS Automated Processing System (FAPS). However, after 8 years of planning, the agency decided it needed to reevaluate the program, and the Office of Management and Budget (OMB) requested that an independent board conduct a review to identify critical issues impeding progress and recommend ways to address the issues. The review identified various management weaknesses, including the lack of an established strategic plan, cost estimation methodologies, and baseline; improperly defined and ineffectively managed requirements; and no clear accountability for decision making and oversight. Accordingly, the board suggested areas for improvement and recommended terminating the program if immediate action was not taken. In mid-1996, OPM terminated the program. In 1997, OPM began planning a second modernization initiative, called the Retirement Systems Modernization (RSM) program. The agency originally intended to structure the program as an acquisition of commercially available hardware and software that would be modified in-house to meet its needs. From 1997 to 2001, OPM developed plans and analyses and began developing business and security requirements for the program. However, in June 2001, it decided to change the direction of the retirement modernization initiative. In late 2001, retaining the name RSM, the agency embarked upon its third initiative to modernize the retirement process and examined the possibility of privately sourced technologies and tools. To this end, OPM issued a request for information to obtain private sourcing options and determined that contracting was a viable alternative that would be cost efficient, less risky, and more likely to be completed on time and on budget. In 2006, the agency awarded three contracts for: (1) a commercially available, defined benefits technology solution (DBTS) to automate retirement processing; (2) services to convert paper records to electronic files; and (3) consulting services to support the redesign of its retirement operations. The contract for DBTS was awarded to Hewitt Associates, and the additional contracts to support the technology were awarded to Accenture Ltd. and Northrop Grumman Corporation, as reflected in table 1. OPM produced a December 2007 program management plan that, among other things, described capabilities the agency expected to implement as outcomes of retirement modernization. Among these capabilities, the agency expected to implement retirement benefit modeling and planning tools for active federal employees, a standardized retirement benefit calculation system, and a consolidated system to support all aspects of retirement processing. In February 2008, OPM renamed the program RetireEZ and deployed a limited initial version of DBTS. As the foundation of the modernization initiative, DBTS was to be a comprehensive technology solution that would provide capabilities to substantially automate retirement processing. This technology was to be provided by the contractor for a period of 10 years and was intended to provide, among other things, an integrated database with calculation functionality for retirement processing. In addition to calculating retirement benefit amounts, DBTS was intended to provide active and retired federal employees with self- service, Internet-based tools for accessing accounts, updating retirement records, submitting transactions, monitoring the status of claims, and forecasting retirement income. The technology was also expected to enhance customer service by providing OPM and agency personnel with the capability to access retirement information online. Further, the technology was expected to be integrated with OPM and federal agency electronic retirement records and processes. When fully implemented, the modernized program was expected to serve OPM retirement processing personnel, federal agency human resources and payroll offices, active federal employees, retirees, and the beneficiaries of retirees. According to the agency, in late February 2008, the DBTS was deployed with limited functionality to 26,000 federal employees serviced by the General Services Administration’s (GSA) payroll offices. In April 2008, OPM reported that 13 of the 37 retirement applications received from GSA’s payroll office had been processed through DBTS with manual intervention and provided the retirees 100 percent of their monthly benefits within 30 days from their retirement date. However, a month later, the agency determined that DBTS had not worked as expected and suspended system operation. In October 2008, after 5 months of attempting to address system quality issues, the agency terminated the contract. In November 2008, OPM began restructuring the program and reported that its efforts to modernize retirement processing would continue. Figure 3 illustrates the timeline of retirement modernization initiatives from 1987 to the present. Various entities within OPM are responsible for managing RetireEZ. Specifically, the management is composed of committees, a program office, and operational support, as reflected in table 2. Since 2005, we have conducted several studies of OPM’s retirement modernization noting weaknesses in its management of the initiative. In February of that year, we reported that the agency lacked processes for retirement modernization acquisition activities, such as determining requirements, developing acquisition strategies, and implementing a risk program. Further, the agency had not established effective security management, change management, and program executive oversight. We recommended that the Director of OPM ensure that the retirement modernization program office expeditiously establish processes for effective oversight of the retirement modernization in the areas of system acquisition management, information security, organizational change management, and information technology (IT) investment management. In response, between 2005 and 2007, the agency initiated steps toward establishing management processes for retirement modernization and demonstrated the completion of activities with respect to each of our nine recommendations. However, in January 2008, we reported that the agency still needed to improve its management of the program to ensure a successful outcome for its modernization efforts. Specifically, we reported that initial test results had not provided assurance that DBTS would perform as intended, the testing schedule increased the risk that the agency would not have sufficient resources or time to ensure that all system components were tested before deployment, and trends in identifying and resolving system defects had indicated a growing backlog of problems to be resolved prior to deployment. Further, we reported that although the agency had established a risk management process, it had not reliably estimated the program costs, and its progress reporting was questionable because it did not reflect the actual state of the program. We recommended that the Director of OPM address these deficiencies by conducting effective system tests and resolving urgent and high priority system defects prior to system deployment, in addition to improving program cost estimation and progress reporting. In response to our report, OPM stated that it concurred with our recommendations and was taking steps to address them. However, in March 2008, we determined that the agency was moving forward with system deployment and had not yet implemented its planned actions. OPM subsequently affirmed its agreement with our recommendations in April 2008 and reported that it had implemented or was in the process of implementing each recommendation. As of March 2009, however, these recommendations still had not been fully addressed. OPM remains far from fully implementing the retirement modernization capabilities described when it documented its plans for RetireEZ in 2007. The agency only partially implemented two of eight capabilities that it identified to modernize retirement processing. The remaining six capabilities, which were to be delivered through the DBTS contract, have not been implemented, and OPM’s plans to continue implementing them are uncertain. While the agency has taken steps to restructure the RetireEZ program without the DBTS contract, it has not developed a plan to guide its future modernization efforts. OPM’s retirement modernization plans from 2007 described eight capabilities that were to be implemented to achieve modernized processes and systems. As of late March 2009, the agency had partially implemented two of these capabilities while the remaining six had not been implemented (see table 3). Specifically, it had achieved partial implementation of an integrated database of retirement information that was intended to be accessible to OPM and agency retirement processing personnel. In this regard, the agency implemented a new database, populated with images of retirement information, which is accessible to OPM retirement processing personnel online. This database contains over 8 million files which, according to agency officials, represent approximately 80 to 90 percent of the available retirement information for all active federal employees. However, the capability for the information in the database to be integrated with OPM’s legacy retirement processing systems and to be accessible to other agency retirement processing personnel has not yet been implemented. OPM has also partially implemented enhanced customer service capabilities. Specifically, the agency acquired a new telephone infrastructure (i.e., additional lines) and hired additional customer service representatives to reduce wait times and abandonment rates. However, the agency has not yet developed the capabilities for OPM retirement processing personnel to provide enhanced customer support to active and retired federal employees through online account access and management. Moreover, six other capabilities have not been implemented—and plans to implement them are uncertain—because they were to be delivered through the now-terminated DBTS contract, which had been expected to provide a single system that would automate the processing of retirement applications, calculations, and benefit payments. Among the capabilities not implemented was one for other agencies’ automated submissions of retirement information to OPM that could be used to process retirement applications. While OPM began developing this capability by establishing interfaces with other agencies as part of its effort to implement DBTS, it discontinued the use of the interfaces for processing retirement applications when the DBTS contract was terminated. Thus, federal agencies that submit retirement information to OPM continue to provide paper packages and information when employees are ready to retire. Further, OPM has not implemented a planned capability for active and retired federal employees to access online retirement information through self-service tools. While the agency provided demonstrations of DBTS in April 2008 that showed the ability for employees to access information online, including applying for retirement and modeling future retirement benefits, this capability was to be provided by DBTS, and thus, no longer exists. The contractor had also been expected to deliver a consolidated system to support all aspects of retirement processing and an electronic case management system to support retirement processing. In the absence of these capabilities, the agency continues to manage cases through paper tracking and stand-alone systems. Additionally, OPM and federal agencies continue to rely on nonstandardized systems to determine and calculate retirement benefits, and federal retirees currently have only limited online, self-service tools. Program management principles and best practices emphasize the importance of using a program management plan that, among other things, establishes a complete description that ties together all program activities. An effective plan includes a description of the program’s scope, implementation strategy, lines of responsibility and authority, management processes, and a schedule. Such a plan incorporates all the critical areas of system development and is to be used as a means of determining what needs to be done, by whom, and when. Furthermore, establishing results-oriented (i.e., objective, quantifiable, and measurable) goals and measures, that can be included in a plan, provides stakeholders with the information they need to effectively oversee and manage programs. A plan for the future of the RetireEZ program has not been completed. In November 2008, OPM began restructuring the program and reported it was continuing toward retirement modernization without the DBTS contract. The restructuring efforts have resulted in a wide variety of documentation, including multiple descriptions of the program in formal agency reports, budget documentation, agency briefing slides, and related documents. For example, OPM’s November Fiscal Year 2008 Agency Financial Report described what the RetireEZ program is expected to achieve (e.g., provide retirement modeling tools for federal employees) once implemented. The agency’s Annual Performance Report, dated January 2009, outlined that the new vision for the restructured program is “to support benefit planning and management throughout a participant’s lifecycle through an enhanced federal retirement program.” The agency also presented information to OMB that identified eight fiscal year 2009 program initiatives, as listed in table 4. The agency has developed a variety of informal program documents and briefing slides that describe retirement modernization activities. For instance, one document prepared by the program office describes a five- phased approach that is intended to replace its previous DBTS-reliant strategy. The approach includes the following activities: (1) collecting electronic retirement information, (2) automating the retirement application process, (3) integrating retirement information, (4) developing retirement calculation technologies and tools, and (5) improving post- retirement processes through a technology solution. In addition, briefing slides also prepared by the program office outline a schedule for efforts to identify new technologies to support retirement modernization by drafting a request for information, which OPM expects to issue in late April 2009. Regardless, OPM’s various reports and documents describing its planned retirement modernization activities do not provide a complete plan for its restructured program. Specifically, although agency documents describe program implementation activities, they do not include a definition of the program, its scope, lines of responsibility and authority, management processes, and schedule. Also, the modernization program documentation does not describe results-oriented (i.e., objective, quantifiable, and measurable) performance goals and measures. According to the RetireEZ program manager, the agency is developing plans, but they will not be ready for release until the new OPM director has approved them, which is expected to occur in April 2009. Until the agency completes and uses a plan that includes all of the above elements to guide its efforts, it will not be properly positioned to obtain agreement with relevant stakeholders (e.g., Congress, OMB, federal agencies, and OPM senior executives) for its restructured retirement modernization initiative. Further, the agency will also not have a key mechanism that it needs to help ensure successful implementation of future modernization efforts. OPM has significant management weaknesses in five areas that are important to the success of its retirement modernization program: cost estimating, EVM, requirements management, testing, and program oversight. For example, the agency has not performed key steps, including the development of a cost estimating plan or completion of a work breakdown structure, both of which are necessary to develop a reliable program cost estimate. Also, OPM has not established and validated a performance measurement baseline, which is essential for reliable EVM. Further, although OPM is revising its previously developed system requirements, it has not established processes and plans to guide this work. Nor has the agency addressed test activities, even though developing processes and planning test activities early in the life cycle are recognized best practices for effective testing. Furthermore, although OPM’s Executive Steering Committee and Investment Review Board have recently become more active regarding RetireEZ, these bodies did not exercise effective oversight in the past, which has allowed the aforementioned management weaknesses to persist. Notably, OPM has not established guidance regarding how these entities are to engage with the program when corrective actions are needed. Until OPM addresses these weaknesses, many of which we and others made recommendations to correct, the agency’s retirement modernization initiative remains at risk of failure. The establishment of a reliable cost estimate is a necessary element for informed investment decision making, realistic budget formulation, and meaningful progress measurement. A cost estimate is the summation of individual program cost elements that have been developed by using established methods and validated data to estimate future costs. According to federal policy, programs must maintain current and well- documented estimates of program costs, and these estimates must span the full expected life of the program. Our Cost Estimating and Assessment Guide includes best practices that agencies can use for developing and managing program cost estimates that are comprehensive, well-documented, accurate, and credible, and provide management with a sound basis for establishing a baseline to measure program performance and formulate budgets. This guide identifies a cost estimating process that includes initial steps such as defining the estimate’s purpose (i.e., its intended use, scope, and level of detail); developing the estimating plan (i.e., the estimating approach, team, and timeline); defining the program (e.g., technical baseline description); and determining the estimating structure (e.g., work breakdown structure). According to best practices, these initial steps in the cost estimating process are of the utmost importance, and should be fully completed in order for the estimate to be considered valid and reliable. OPM officials stated that they intend to complete a modernization program cost estimate by July 2009. However, the agency has not yet fully completed initial steps for developing the new estimate. Specifically, the agency has not yet fully defined the estimate’s purpose, developed the estimating plan, defined program characteristics in a technical baseline description, or determined the estimating structure. With respect to the estimate’s purpose, agency officials stated that the estimate will inform the budget justification of RetireEZ for fiscal year 2011 and beyond. However, the agency has not clearly defined the scope or level of detail of the estimate. Regarding the estimating plan, agency officials stated that they have created a timeline to complete the estimate by July 2009. However, the agency has not documented an estimating plan that includes the approach and resources required to complete the estimate in the time period identified. With respect to the technical baseline description, agency officials stated that they are in the advanced stages of developing a request for information and a concept of operations that will serve as the basis for a technical baseline description. These documents are expected to be reviewed for approval in April 2009. Regarding the estimating structure, the agency has developed a work breakdown structure that identifies elements of the program to be estimated. However, the agency has not yet developed a work breakdown structure dictionary that clearly defines each element. Weaknesses in the reliability of OPM’s retirement modernization cost estimate have been long-standing. We first reported on the agency’s lack of a reliable cost estimate in January 2008 when we noted that critical activities, including documentation of a technical baseline description, had not been performed, and we recommended that the agency revise the estimate. Although OPM agreed to produce a reliable program cost estimate, the agency has not yet done so. Until OPM fully completes each of the steps, the agency increases the risk that it will produce an unreliable estimate and will not have a sound basis for measuring program performance and formulating retirement modernization program budgets. OMB and OPM policies require major IT programs to use EVM to measure and report program progress. EVM is a tool for measuring program progress by comparing the value of work accomplished with the amount of work expected to be accomplished. Such a comparison permits actual performance to be evaluated, based on variances from the planned cost and schedule, and future performance to be forecasted. Identification of significant variances and analysis of their causes helps program managers determine the need for corrective actions. Before EVM analysis can be reliably performed, developing a credible cost estimate is necessary. In addition to developing a cost estimate, an integrated baseline review must be conducted to validate a performance measurement baseline and attain agreement of program stakeholders (e.g., agency and contractor officials) before reliable EVM reporting can begin. The establishment of a baseline depends on the completion of a work breakdown structure, an integrated master schedule, and budgets for planned work. Although the agency plans to begin reporting on the restructured program’s progress using EVM in April 2009, the agency is not yet prepared to do so because initial steps have not been completed and are dependent on decisions about the program that have not been made. Specifically, the agency has not yet developed a reliable cost estimate for the program; such an estimate, which is critical for establishing reliable EVM, is not expected to be complete until July 2009; the agency does not plan to conduct an integrated baseline review to establish a reliable performance measurement baseline before beginning EVM reporting; and the work breakdown structure and integrated master schedule that agency officials report they have developed may not accurately reflect the full scope and schedule because key program documentation, such as the concept of operations, has not been completed. This situation resembles the state of affairs that existed in January 2008, when we reported that OPM’s EVM was unreliable because an integrated baseline review had not been conducted to validate the program baseline. At that time we recommended, among other things, that the agency establish a basis for effective use of EVM by validating a program performance measurement baseline through a program-level integrated baseline review. Although the agency stated that it agreed, it did not address this recommendation. Until the agency has developed a reliable cost estimate, performed an integrated baseline review, and validated a performance measurement baseline that reflect its program restructuring, the agency is not prepared to perform reliable EVM. Engaging in EVM reporting without first performing these fundamental steps could again render the agency’s assessment unreliable. Well-defined and managed requirements are a cornerstone of effective system development and acquisition. According to recognized guidance, disciplined processes for developing and managing requirements can help reduce the risks of developing a system that does not meet user and operational needs. Such processes include (1) developing detailed requirements that have been derived from the organization’s concept of operations and are complete and sufficiently detailed to guide system development and (2) establishing policies and plans, including defining roles and responsibilities, for managing changes to requirements and maintaining bidirectional requirements traceability. OPM’s retirement modernization requirements processes include some, but not all, of the elements needed to effectively develop and manage requirements. The agency began an effort to better develop its retirement modernization requirements in November 2008. This effort was in response to the agency’s recognition that its over 1,400 requirements lacked sufficient detail, were incomplete, and required further development. The agency intends to complete this requirements development effort in April 2009. However, the requirements will not be derived from OPM’s concept of operations because the agency is revising the concept of operations expected to be completed by April 2009, to reflect the program restructuring. Further, OPM documentation indicates that the agency has not yet determined the level of detail to which requirements should be developed. Additionally, agency officials stated that OPM is developing a requirements development process for retirement modernization. With respect to requirements management, OPM developed an organizational charter that outlined roles and responsibilities for supporting efforts to manage requirements. However, the agency does not yet have a requirements management plan. OPM’s prior experience with DBTS illustrates the importance of effective requirements development and management. According to RetireEZ program officials, insufficiently detailed requirements, poorly controlled requirements changes, and inadequate requirements traceability were factors that contributed to DBTS not performing as expected. Moreover, these requirements development and management weaknesses were identified, and recommendations for improvement were made by OPM’s independent verification and validation contractor before DBTS deployment. However, the agency has not yet corrected these weaknesses. Until OPM fully establishes requirements development and management processes, the agency increases the risk that it will (1) identify requirements that are neither complete nor sufficiently detailed and (2) not effectively manage requirements changes or maintain bidirectional traceability, thus further increasing agency risk that it will produce a system that does not meet user and operational needs. Effective testing is an essential component of any program that includes developing systems. Generally, the purpose of testing is to identify defects or problems in meeting defined system requirements and satisfying user needs. To be effectively managed, testing should be planned and conducted in a structured and disciplined fashion that adheres to recognized guidance and is coordinated with the requirements development process. Beginning the test planning process in the early stages of a program life cycle can reduce rework later in the program. Early test planning in coordination with requirements development can provide major benefits. For example, planning for test activities during the development of requirements may reduce the number of defects identified later and the costs related to requirements rework or change requests. Further, planning test activities early in a program’s life cycle can inform requests for proposals and help communicate testing expectations to potential vendors. OPM has not begun to plan test activities in coordination with developing its requirements for the RetireEZ program. According to OPM officials, the agency intends to begin its test planning by revising the previously developed DBTS test plans after requirements have been developed. However, the agency has not yet added test planning to its project schedule. Early test planning is especially important to avoid repeating the agency’s experience during DBTS testing when it identified more defects than it could resolve before system deployment. In January 2008, we reported that an unexpectedly high number of defects were identified during testing; yet, the deployment schedule had increased the risk of not resolving all defects that needed to be corrected before deploying DBTS. According to the RetireEZ program officials, the failure to fully address these defects contributed to the limited number of federal employees who were successfully processed by the system when it was deployed in February 2008. If it does not plan test activities early in the life cycle of RetireEZ, OPM increases the risk that it will again deploy a system that does not satisfy user expectations and meet requirements (i.e., accurately calculate retirement benefits) because of its potential inability to address a higher number of defects than expected. Moreover, criteria used to develop requests for proposals and communicate testing expectations to potential vendors could be better informed if the agency plans RetireEZ test activities early in the life cycle. GAO and OMB guidance calls for agencies to ensure effective oversight of IT projects throughout all life-cycle phases. Critical to effective oversight are investment management boards made up of key executives who regularly track the progress of IT projects such as system acquisitions or modernizations. These boards should maintain adequate oversight and track project performance and progress toward predefined cost and schedule goals, as well as monitor project benefits and exposure to risk. Another element of effective IT oversight is employing early warning systems that enable management boards to take corrective actions at the first sign of cost, schedule, and performance slippages. OPM’s Investment Review Board was established to ensure that major investments are on track by reviewing their progress and determining appropriate actions when investments encounter challenges. Despite meeting regularly and being provided with information that indicated problems with the retirement modernization, the board did not ensure that the investment was on track, nor did it determine appropriate actions for course correction when needed. For example, from January 2007 to August 2008 the board met and was presented with reports that described problems the retirement modernization program was facing, such as the lack of an integrated master schedule and earned value data that did not reflect the “reality or current status” of the program. However, meeting minutes indicate that no discussion or action was taken to address these problems. According to a member of the board, OPM guidance regarding how the board is to communicate recommendations and corrective actions when needed for the investments it is responsible for overseeing has not been established. In addition, OPM established an Executive Steering Committee to oversee retirement modernization. According to its charter, the committee is to provide strategic direction, oversight, and issue resolution to ensure that the program maintains alignment with the mission, goals, and objectives of the agency and is supported with required resources and expertise. However, the committee was inactive for most of 2008 and, consequently, did not exercise oversight of the program during a crucial period in its development. For example, from January 2008 until October 2008, the committee discontinued its formal meetings, and as a result, it was not involved in key program decisions, including the deployment of DBTS. Further, a member of the committee noted that OPM guidance for making recommendations and taking corrective actions also has not been provided. The ineffectiveness of the board and the inactivity of the committee allowed program management weaknesses in the areas of cost estimation, EVM, requirements management, and testing to persist and raise concerns about OPM’s ability to provide meaningful oversight as the agency proceeds with its retirement modernization. Without fully functioning oversight bodies, OPM cannot monitor modernization activities and make the course corrections that effective boards and committees are intended to provide. OPM’s retirement modernization initiative is in transition from a program that was highly dependent on the success of a major contract that no longer exists, to a restructured program that has yet to be fully defined. Although the agency has been able to partially implement a database of retirement information and improvements to customer service, it remains far from implementing six other key capabilities. Recognizing that much work remains, OPM has undertaken steps to restructure the retirement modernization program, but it has not yet produced a complete description of its planned program, including fundamental information about the program’s scope, implementation strategy, lines of responsibility and authority, management processes, and schedule. Further, OPM’s retirement modernization program restructuring does not yet include definitions of results-oriented goals and measures against which program performance can be objectively and quantitatively assessed. In addition, OPM has not overcome managerial shortcomings in key areas of program management, including areas that we have previously reported. Specifically, the agency is not yet positioned to develop a reliable program cost estimate or perform reliable EVM, both of which are critical to effective program planning and oversight. Nor has OPM overcome weaknesses in its management of system testing and defects, two activities that proved problematic as the agency was preparing to deploy the RetireEZ system that subsequently was terminated. Adding to these long-standing concerns are weaknesses in OPM’s process to effectively develop and manage requirements for whatever system or service it intends to acquire or develop. Finally, these weaknesses have been allowed to persist by entities within the agency that were ineffective in overseeing the retirement modernization program. As a consequence, the agency is faced with significant challenges on two fronts: defining and transitioning to its restructured program, and addressing new and previously identified managerial weaknesses. Until OPM addresses these weaknesses, many of which were previously identified by GAO and others, the agency’s retirement modernization initiative remains at risk of failure. Institutionalizing effective planning and management is critical not only for the success of this initiative, but also for that of other modernization efforts within the agency. To improve OPM’s effort toward planning and implementing its retirement modernization program by addressing management weaknesses, we recommend that the Director of the Office of Personnel Management provide immediate attention to ensure the following six actions are taken: Develop a complete plan for the restructured program that defines the scope, implementation strategy, lines of responsibility and authority, management processes, and schedule. Further, the plan should establish results-oriented (i.e., objective, quantifiable, and measurable) goals and associated performance measures for the program. Develop a reliable cost estimate by following the best practice steps outlined in our Cost Estimating and Assessment Guide, including definition of the estimate’s purpose, development of an estimating plan, definition of the program’s characteristics, and determination of the estimating structure. Establish a basis for reliable EVM, when appropriate, by developing a reliable program cost estimate, performing an integrated baseline review, and validating a performance measurement baseline that reflects the program restructuring. Develop a requirements management plan and execute processes described in the plan to develop retirement modernization requirements in accordance with recognized guidance. Begin RetireEZ test planning activities early in the life cycle. Develop policies and procedures that would establish meaningful program oversight and require appropriate action to address management deficiencies. The Director of the Office of Personnel Management provided written comments on a draft of this report. (The comments are reproduced in app. II.) In the comments, OPM agreed with our recommendations and stated that it had begun to address them. To this end, the Director stated that the agency had, among other actions, begun revising its retirement modernization plans, developing a new program cost estimate, planning for accurate EVM reporting, incorporating recognized guidance in requirements management planning, and planning test activities during requirements development. If the recommendations are properly implemented, they should better position OPM to effectively manage its retirement modernization initiative. The agency also provided comments on the draft report regarding our description of the federal retirement application process, as well as our characterizations of OPM’s EVM and requirements management capabilities vis-à-vis the retirement modernization program. In each of these instances, we made revisions as appropriate. We are sending copies of this report to the Director of the Office of Personnel Management, appropriate congressional committees, and other interested parties. In addition, the report is available at no charge on the GAO Web site at http://www.gao.gov. If you or your staffs have questions about this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III. As requested, the objectives of our study were to (1) assess the status of the Office of Personnel Management’s (OPM) efforts toward planning and implementing the RetireEZ program and (2) evaluate the effectiveness of the agency’s management of the modernization initiative. To assess the status of OPM’s efforts toward planning and implementing the RetireEZ program, we reviewed and analyzed program documentation, including program management plans, briefing slides, and project status documentation, to identify planned retirement modernization capabilities and determine to what extent these capabilities have been implemented; evaluated the agency’s documentation about restructuring the program and analyzed the extent to which the documentation describes current and planned RetireEZ program activities; identified and evaluated the agency’s program goals and measures and compared them to relevant guidance to determine the extent to which the goals and measures are described in results-oriented terms; supplemented agency program documentation and our analyses by interviewing agency and contractor officials, including the OPM Director, Chief Information Officer, Chief Financial Officer, Director of Modernization, Associate Director for Human Resources Products and Services Division, and executives from Hewitt Associates and Northrop Grumman Corporation; and observed retirement operations and ongoing modernization activities at OPM and contractor facilities in Washington, D.C.; Boyers, Pennsylvania; and Herndon, Virginia. To determine the effectiveness of OPM’s management of the retirement modernization initiative, we evaluated the agency’s management of program cost estimating, earned value management (EVM), requirements, test planning, and oversight and compared the agency’s work in each area with recognized best practices and guidance. Specifically, to evaluate whether OPM effectively developed a reliable program cost estimate, we analyzed the agency’s program documentation and determined to what extent the agency had completed key activities described in our Cost Estimating and Assessment Guide; to assess OPM’s implementation of EVM, we reviewed program progress reporting documentation and compared the agency’s plans for restarting its EVM-based progress reporting against relevant guidance, including our Cost Estimating and Assessment Guide; regarding requirements management, we evaluated OPM’s processes for developing and managing retirement systems modernization requirements and compared the effectiveness of those processes against recognized guidance; to determine the effectiveness of the agency’s test planning for the retirement modernization, we reviewed program activities and test plans against best practices and evaluated the extent to which the agency has begun planning for these activities; and we reviewed and analyzed documentation from program oversight entities and evaluated the extent to which these entities took actions toward ensuring the RetireEZ program was being effectively overseen. We also evaluated OPM’s progress toward implementing our open recommendations and interviewed OPM and contractor officials as noted. We conducted this performance audit at OPM headquarters in Washington, D.C., the Retirement Operations Center for OPM in Boyers, Pennsylvania, and contractor facilities in Herndon, Virginia, from May 2008 through April 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, key contributions to this report were made by Mark T. Bird, Assistant Director; Barbara S. Collier; Neil J. Doherty; David A. Hong; Thomas J. Johnson; Rebecca E. LaPaze; Lee A. McCracken; Teresa M. Neven; Melissa K. Schermerhorn; Donald A. Sebers; and John P. Smith.
summary: For the past two decades, the Office of Personnel Management (OPM) has been working to modernize the paper-intensive processes and antiquated systems used to support the retirement of federal employees. By moving to an automated system, OPM intends to improve the program's efficiency and effectiveness. In January 2008, GAO recommended that the agency address risks to successful system deployment. Nevertheless, OPM deployed a limited initial version of the modernized system in February 2008. After unsuccessful efforts to address system quality issues, OPM suspended system operation, terminated a major contract, and began restructuring the modernization effort, also referred to as RetireEZ. For this study, GAO was asked to (1) assess the status of OPM's efforts to plan and implement the RetireEZ program and (2) evaluate the effectiveness of the agency's management of the modernization initiative. To do this, GAO reviewed OPM program documentation and interviewed agency and contractor officials. OPM remains far from achieving the modernized capabilities it had planned. Specifically, the agency has partially implemented two of eight planned capabilities: (1) an integrated database of retirement information accessible to OPM and agency retirement processing personnel and (2) enhanced customer service capabilities that support customer needs and provide self-service tools. However, the remaining six capabilities have yet to be implemented because they depended on deliverables that were to be provided by a contract that is now terminated. Examples of these missing capabilities include: (1) automated submission of retirement information through interfaces with federal agencies and (2) Web-accessible self-service retirement information for active and retired federal employees. Further, OPM has not yet developed a complete plan that describes how the program is to proceed without the system that was to be provided under the terminated contract. Although agency documents describe program implementation activities, they do not include a definition of the program, its scope, lines of responsibility and authority, management processes, and a schedule. Also, modernization program documentation does not describe results-oriented performance goals and measures. Until the agency completes and uses a plan that includes all of the above elements to guide its efforts, it will not be properly positioned to move forward with its restructured retirement modernization initiative. Further, OPM has significant weaknesses in five key management areas that are vital for effective development and implementation of its modernization program: cost estimating, earned value management (a recognized means for measuring program progress), requirements management, testing, and oversight. For example, the agency has not developed a cost estimating plan or established a performance measurement baseline--prerequisites for effective cost estimating and earned value management. Further, although OPM is revising its previously developed system requirements, it has not established processes and plans to guide this work or addressed test activities even though developing processes and plans, as well as planning test activities early in the life cycle, are recognized best practices for effective requirements development and testing. Finally, although OPM's Executive Steering Committee and Investment Review Board have recently become more active regarding RetireEZ, these bodies did not exercise effective oversight in the past, which has allowed the aforementioned management weaknesses to persist and OPM has not established guidance regarding how these entities are to intervene when corrective actions are needed. Until OPM addresses these weaknesses, many of which GAO and others made recommendations to correct, the agency's retirement modernization initiative remains at risk of failure. Institutionalizing effective management is critical not only for the success of this initiative, but also for that of other modernization efforts within the agency.