21st Century Project Update

Background

Project Suspended Due to Problems in Pilot Stage

In 2004, SCO proposed the TFC Project, the IT effort to replace the existing statewide human resources management and payroll systems used to pay roughly 260,000 state employees. The new system, also called MyCalPAYS, was intended to allow the state to improve management processes such as payroll, benefits administration, and timekeeping, and include self–service access for employees and managers, among other capabilities. In February 2013, after the project experienced various problems during the pilot stage, which included payroll and related functions for about 1,500 SCO employees, the SCO terminated the contract with the project’s primary vendor, SAP Public Services, Inc. (SAP) and the project was suspended. For a full history of the TFC Project through contract termination in February 2013, please see the Appendix.

Close–Out Activities Funded by 2013–14 Budget

The 2013–14 budget package provided SCO with various resources for close–out activities associated with the suspension of the TFC Project. Specifically, the Governor proposed and the Legislature approved 40 one–year, limited–term positions and $14.6 million ($11.9 million General Fund) for the activities described below.

Reconciling SCO Payroll. In light of the many errors produced by the MyCalPAYS system during the eight months it was operational, the budget provided 24.2 positions to ensure that employees and other payroll system payees—primarily healthcare providers, retirement savings systems, and tax agencies—were paid accurately during the pilot stage. Reconciliation includes a systematic comparison between actual payroll outcomes from the MyCalPAYS system and what would have occurred had the existing functional legacy system been operative during those months. This process includes recreating past payroll cycles, a time–and resource–intensive task that relies on original source documents—including timesheets, employee histories, and other human resource documentation—to reenact past payrolls.

Defending the State’s Legal Position. The 2013–14 budget provided 9.2 positions—mostly data and systems specialists—to support SCO’s legal team in preparing for contractually mandated mediation and potential legal proceedings with SAP. In addition, SCO received $1 million for outside legal counsel.

Project Suspension Activities. Because the MyCalPAYS system was the “system of record” for SCO employees during the months it was operative, the budget provided resources for preserving data and system architecture. In addition, SCO began decommissioning system components that were no longer in use to reduce costs.

Recent Developments

Below, we describe the major developments since enactment of the 2013–14 budget.

Reconciliation for Employees Virtually Complete; Reconciliation for Other Payroll System Payees Behind Schedule. According to SCO, reconciliation efforts for employees are virtually complete, while efforts for other payroll system payees are behind schedule and will not be completed by the end of 2013–14, as originally estimated. (The SCO explained that many tasks took longer than originally estimated and additional testing was required.) Reconciliation includes: (1) analyzing payroll to identify inaccuracies, (2) notifying the employee or other payroll payee of inaccuracies, and (3) compensating employees and payees or reimbursement of the state, as appropriate. Analysis regarding employee over– and under–compensation is complete. During reconciliation, SCO found that—for 1,542 employees paid during the period for which MyCalPAYS was in place—267 employees were underpaid and 541 employees were overpaid. The SCO has notified all employees of the inaccuracies in their payroll, compensated most under–paid employees (with the exception of some whose cases are particularly complex), and has secured repayment plans from those employees that were overpaid. (The SCO prioritized efforts towards employees who were underpaid first, and then towards reconciling payroll for SCO employees who were overpaid or whose cases were particularly complex.) Analysis for other payroll system payees is currently underway—SCO has completed reconciliation analysis for one–third of approximately 100 payroll system payees that required adjustments—and the entire reconciliation process is expected to be complete by December 2014.

Mediation Fails, SCO Files Lawsuit. Pursuant to the state’s contractual obligation with SAP, SCO entered into mediation with SAP to attempt to resolve disputes between the two parties. That mediation was unsuccessful and resulted in SCO filing a lawsuit against SAP last November for breach of contract.

2014–15 Governor’s Budget Proposal

Proposal Supports Legal Activities. The Governor’s budget proposes five one–year, limited–term positions and $6.5 million ($3.6 million General Fund) to support ongoing legal activities. The positions would be used primarily to support the legal effort, including responding to public records and discovery requests and providing technical assistance to the outside legal counsel. In addition, the request includes $2.5 million for outside legal counsel to assist with legal proceedings. The proposal states that this amount is one–half of the total that SCO estimates it will be required to spend in 2014–15 to support the legal effort.

The administration also proposes provisional language that would authorize DOF to augment SCO’s budget “to fund additional litigation and related support efforts associated with the 21st Century Project payroll system” following written notification to the Legislature. The provisional language includes no cap on the amount that DOF could authorize for the project through this budgeting mechanism.

LAO Findings

Administration’s Proposal to Fund Legal Activities Underbudgeted

The 2013–14 Spending on Legal Effort Far Exceeds Budgeted Amount. As described earlier, the 2013–14 budget provided SCO with $1 million for outside legal counsel. According to SCO, spending on outside counsel is estimated to total $5.7 million for 2013–14, exceeding the budget appropriation by $4.7 million. The SCO is currently assessing whether the additional legal costs can be absorbed or if a request for current–year deficiency funding will be necessary.

Projected Costs for Legal Effort in 2014–15 Exceed Requested Amount. As noted above, the administration’s budget plan proposes $2.5 million for SCO’s outside legal counsel in 2014–15 and includes provisional language authorizing DOF to augment SCO’s budget to fund additional litigation beyond that amount. The SCO estimates that costs associated with outside counsel will total between $4.5 million and $5 million in 2014–15. That estimate is based on monthly actuals through September 2013, combined with projected costs for the remainder of 2013–14. The SCO’s estimate of costs for outside counsel appears reasonable.

Administration Does Not Propose Needed Project Assessment

Legacy Systems Only Alternative in Near Term. The state’s legacy payroll systems are comprised of numerous individual components that were created to address discrete payroll needs—such as leave balance management, basic payroll, and warrants to state vendors—as those needs arose. Although the systems lack a central architecture, including consistent data methodology and programming languages, they have been running the state’s payroll for decades. In general, the legacy systems—though outdated, inflexible, costly to maintain, and at some risk of eventual failure—seem a credible (and, in practice, the only) alternative for the state to use over the next few years.

Update to State’s Payroll Process and Systems Still Necessary. The current suspension of the TFC Project means that the state’s need for an updated human resources management and payroll system remains unmet. There are numerous functionality and stability issues that justify the continued pursuit of an updated payroll configuration, including the capabilities to respond quickly to payroll changes, issue reports to other state agencies and stakeholders, and allow employees online access to payroll and tax information. Updating the state’s payroll systems also likely could reduce ongoing maintenance costs—as they could be less technically challenging and labor–intensive to update—and the likelihood of significant payroll disruptions in future years.

Continued Need for Comprehensive Independent Assessment. In our view, an independent assessment is a necessary precursor to addressing the state’s unmet need for an updated human resources management and payroll system. A timely project assessment could identify the issues that contributed to the suspension of the project, recommend opportunities for improvement, and inform the state’s decision making on a path forward. Furthermore, a timely assessment would move the state towards a modernized payroll system more quickly, reduce the length of time that the state must depend on the aged legacy system, and thereby reduce the risk of a significant legacy system disruption in future years.

If the state is to proceed with an independent assessment, it will maximize the benefits of such an assessment by doing certain components sooner rather than later. Specifically, components that draw on the expertise of current or former TFC Project staff—such as the documentation of lessons learned—are best done as soon as possible as staff recollection of events fades with the passage of time.

Legal Considerations Appear to Take Precedence. The state faces considerable legal uncertainty about its fiscal position as its contractual disagreements with SAP are litigated. The SCO indicates that the fiscal outcomes of the legal effort range from up to $50 million in losses—which could result from a finding that the state breached the contract with SAP by terminating the contract “for convenience”—to up to $150 million in recovery that could result from a finding that SAP breached the contract. Conversations with project staff indicate that the SCO believes that litigation does not preclude that state from moving forward with an independent project assessment as described above. In contrast, the administration seems concerned that any inward–looking assessment of the TFC Project’s failure could endanger the state’s position in the legal proceedings, apparently leading it to prioritize the state’s legal efforts over such an assessment.

In our view, however, an immediate and thorough assessment (which we detail below) is arguably at least an equal if not a higher long–term priority for the state, given the importance of capturing lessons learned that might impact other current and future state IT projects. The state currently has over 35 approved significant IT projects. The total cost of completing all of these IT projects over a number of fiscal years is estimated to be nearly $4 billion. While we recognize that pursuing some aspects of an assessment could result in findings that serve to impair the state’s position in the SAP litigation, we also think that lessons captured from an assessment could increase the probability of current and future projects’ successes, a value to the state that, while uncertain, is potentially large and greater than a recovery from the state’s legal effort.

Analyst’s Recommendations

Recommend Budgeting Estimated Costs for Legal Effort

Budgeting is by its very nature uncertain. Yet a fundamental part of the budget process involves budgeting the estimate of spending for the coming fiscal year when a reasonable estimate is available. While actual spending requirements sometime exceed budgeted amounts, the budget process allows the administration to submit proposals after the budget is enacted to address these shortfalls.

Instead of budgeting estimated costs for the legal activities in 2014–15, the administration proposes to budget roughly one–half of SCO’s estimate, and includes provisional language allowing DOF to augment SCO’s budget without dollar limitation. While the language requires notification to the Legislature, we are concerned that spending authorized pursuant to this alternate budget process is subject to less public and legislative scrutiny than spending appropriated through the regularbudget process. The SCO’s estimate of costs for outside counsel is based in part on actual spending levels and appears reasonable. We therefore recommend that the Legislature budget $4.75 million for outside counsel—the midpoint between SCO’s two estimates. In addition, because the new appropriation would fund a reasonable estimate of anticipated costs and the budget process allows the administration to later submit a request for contingency funding under item 9840 of the annual budget act or a supplemental appropriation, we recommend deleting the provisional language, as it would be unnecessary.

Recommend Independent Assessment in Addition to Legal Efforts

We recommend the Legislature appropriate additional resources to SCO for an assessment of the TFC Project. While we think that the amount to be allocated should be determined in consultation with project staff and the Department of Technology, we believe the cost of an assessment could easily be over $1 million. An assessment is necessary because (1) development of a reenvisioned statewide payroll system should not commence until a thorough review has concluded and (2) the state’s other ongoing IT projects may prove more successful if they benefit from an analysis of what led to the TFC Project’s failure. In addition, we recommend initiating the assessment now rather than later to take the best possible advantage of tapping into the institutional memory of current and former staff familiar with the project.

In keeping with our recommendation in prior analyses of the TFC Project, we recommend the Legislature direct the SCO to contract with an independent entity to perform the assessment. In addition, we recommend that the Health and Human Services Agency’s Office of Systems Integration (OSI) be charged with managing the assessment contract. Although OSI currently provides IT project management and oversight for health and human services projects, its success and expertise with large–scale IT systems development would likely prove beneficial as SCO undergoes assessment of the TFC Project. We recommend that the assessment include at minimum the following components to help inform the path forward for the state’s legacy payroll systems.

Lessons Learned. We recommend the evaluator assess the management of the project and document lessons learned and strategies to prevent a repeat of previous problems. This analysis should include, but not be limited to, assessing whether (1) system requirements were identified, documented, and incorporated into the system design appropriately; (2) the primary vendor contract with SAP was managed appropriately; (3) quality assurance and control measures were effective; (4) risk and issues were identified and managed appropriately; and (5) the testing plan was appropriate.

Viability of SAP Software and Work Completed by SAP. We recommend the evaluator determine if the SAP software solution and the work completed by SAP (or any portion thereof) are viable moving forward should the state continue the TFC Project in something like its prior form. Given the rapid change in technology, other software platforms may now be on the market that more closely meet the state’s payroll needs. If the SAP software is determined to be the best way forward for the state, the evaluator should also estimate the cost and timeline for completing the project as most recently envisioned.

Potential Simplification of State Payroll Practices. We recommend that the evaluator complete an assessment, in collaboration with the California Department of Human Resources, evaluating the state’s current payroll practices to determine if these practices can by simplified. The potential applicability of commercial payroll practices to the state payroll procedures should also be included in this assessment. Payroll software packages, like the SAP software, are largely based on commercial payroll practices, which track time and issue payroll differently than the state. Incorporating commercial payroll practices into the state’s payroll procedures may reduce needed software customizations, thereby facilitating integration and enhancing the prospects of successfully implementing a statewide payroll system. Simplifying the state’s payroll practices may reduce needed software customizations, thereby facilitating integration and enhancing the prospects of successfully implementing a statewide payroll system. Simplifications would likely require statutory changes and modifications through the state’s collective bargaining process. We believe this component of an assessment has merit whether the state decides to move forward with the current centralized payroll system using the SAP software or moves forward with a different approach, as discussed further below.

Evaluation of Options to Update Payroll System. We recommend that the evaluator, in collaboration with SCO, conduct an analysis that compares the costs and feasibility of several options for moving forward to address the state’s unmet need for an updated human resources management and payroll system, including:

• Working Towards Integrated Payroll System Using Incremental Approach. The TFC Project was a huge undertaking. The state’s human resource management and payroll systems are very complex and integrating those systems within the scope of one project produced a significant level of risk that factored into the project’s failure. For this reason, the evaluator should assess the merits of integrating the state’s payroll systems using an incremental approach. In this option, SCO would implement the technical functions of the TFC Project progressively through a series of smaller projects with narrowly tailored scopes. For example, one project could focus on implementing the personnel administration functions proposed in the TFC Project, while a subsequent project could focus on benefits administration. At the conclusion of this iterative process, the state would accomplish the objective of the TFC Project—an integrated payroll system. Compared with integrating all systems at once, this would be a less risky approach, as the limited scope of each project would diffuse risk across many projects. Should any single project face challenges, the negative consequences would be contained.
• Updating Existing Legacy Systems Using Decentralized Model. Due to recent events, it is unclear whether integrating all of the state’s payroll systems, as envisioned in the TFC Project, is feasible. Payroll practices can vary significantly across state departments, and some departments, such as the Department of Forestry and Fire Protection and the Department of Corrections and Rehabilitation, have particularly complex payroll needs that complicate the state’s ability to integrate these systems. This option would assess updating payroll systems in a decentralized fashion that integrates less complex payroll departments together and considers alternative approaches for modernizing the payroll systems of complex departments.
• Completing Project Using Existing SAP Software Solution. In this option, a vendor would leverage work completed to date by SAP and continue to use the underlying SAP software—which the state owns—to modernize the state’s payroll system. The evaluator would need to consider the technical merits of updating a payroll and human resources management system with a software solution that was procured nearly ten years ago.
• Restart Project, but Using New Software Solution. In this option, a vendor would integrate the state’s payroll and human resource management systems, as envisioned in the TFC Project, but would leverage a different—presumably more modern—underlying software solution.

The findings and recommendations from the independent assessment should be made available to the Legislature upon completion.

Recommend Self–Assessment by Department of Technology

The Legislature vested the Department of Technology with responsibilities to provide greater assurances that large IT projects of this type are implemented successfully. Managing major projects of this type is inherently difficult. Both public and private entities often experience major delays, cost overruns, and operational problems when implementing these efforts. Nevertheless, this is now the second time that the TFC Project has terminated its primary vendor contract, and preventing subsequent delays to this specific project was one of the key objectives behind recent efforts to focus and consolidate the state’s technology oversight in the Department of Technology. Understanding the shortcomings of the department regarding its technology oversight functions and identifying opportunities for improvement are important activities if costly failures of this kind are to be avoided in the future. Accordingly, we recommend that the Legislature initiate a review of the performance of the Department of Technology concerning its oversight role for this major IT project. Specifically, we recommend that the Legislature require the department to undertake a self–assessment of (1) its current oversight policies and practices, (2) how these policies were applied to the TFC Project, and (3) impediments—statutory or otherwise—that limit its ability to exercise effective oversight in this and other IT projects. The lessons derived from the self–assessment have broad implications for the success of the state’s other large and complex IT projects—including projects currently under development and future projects.

The findings and recommendations from the self–assessment should be made available to the Legislature by March 1, 2015.