Legislative Analyst's Office Analysis of the 2003-04 Budget Bill

Department of Transportation (2660)

The Department of Transportation (Caltrans) is responsible for planning, coordinating, and implementing the development and operation of the state's transportation systems. These responsibilities are carried out in five programs. Three programs—Highway Transportation, Mass Transportation, and Aeronautics—concentrate on specific transportation modes. Transportation Planning seeks to improve the planning for all travel modes and Administration encompasses management of the department.

The budget proposes total expenditures of $6.4 billion by Caltrans in 2003-04. This is about $670 million, or 9.5 percent, less than estimated current-year expenditures. This decrease is largely due to a significant projected drop in funding for capital improvements on state highways.

Highway Transportation

Budget Proposes Decrease in Highway Program Expenditures

The budget proposes expenditures of $5.6 billion for the highway transportation program, about $350 million, or 6 percent, less than estimated current-year expenditures. This includes a 26 percent decrease in proposed capital outlay expenditures and a 20 percent increase in local assistance expenditures. To aid the General Fund and address projected State Highway Account shortfalls, the budget proposes several specific expenditure cuts.

The major responsibilities of the highway program are to design, construct, maintain, and operate state highways. In addition, the highway program provides local assistance funds and technical support for local roads. For 2003-04, the budget proposes to spend $5.6 billion on the high way transportation program, approximately 87 percent of the department's proposed budget. This is a decrease of $350 million, or 6 percent, from estimated current-year expenditures. This is primarily due to a sizable decrease in projected expenditures for capital outlay, as discussed below.

Of the $5.6 billion, the budget proposes $1.5 billion in capital outlay expenditures, a decrease of 26 percent below estimated 2002-03 levels. This decrease is primarily due to lower estimated expenditures for new projects to be delivered in the five-year State Transportation Improvement Program (STIP). Conversely, the budget proposes an increase of 20 percent in the local assistance program, to $1.9 billion. This increase, however, mistakenly includes certain programs administered by the federal government. After adjusting for this factor, the local assistance program declines as well.

As shown in Figure 1, Caltrans expects that state funds would support about $2.8 billion (50 percent) of highway program expenditures in the budget year. Federal funds would fund about $2.6 billion (46 percent) of the program, while the remaining $236 million (4 percent) would be paid through reimbursements, primarily from local governments.

Specific Expenditure Reductions Proposed. As discussed in our analysis of transportation funding (in the "Crosscutting Issues" section of this chapter), the Governor's budget proposes to use Traffic Congestion Relief Program (TCRP) funding to aid the General Fund in the budget year, and the State Highway Account (SHA) faces pressures that have begun to significantly lower its balance. To deal with these situations, the California Transportation Commission (CTC) stopped allocating money for new TCRP and STIP projects in December 2002. The Governor's budget has also proposed several actions to enhance SHA revenue, including increasing weight fees and redirecting federal funds from local agencies to Caltrans. In addition to these actions, the budget proposes the following specific actions to reduce Caltrans' expenditures:

• Reduce state staff for TCRP projects by 1,214 personnel in the budget year.
• Reduce state operations expenditures by a total of $177 million in the current and budget years, with related staff reductions of about 500 personnel.
• Reduce expenditures on local assistance programs totaling $60 million in the current and budget years.

We discuss some of these proposals in more detail below.

Capital Outlay Support Request Will Be Revised

The budget proposes $1.1 billion to fund capital outlay support (COS), a 12 percent decrease from current-year estimated expenditures. Most of the proposed decrease is due to a reduction in Traffic Congestion Relief Program (TCRP)-related staff, but the administration will revise its proposal in the spring based on new information. We withhold recommendation on the COS budget pending the administration's revised proposal. However, we also note that the Legislature could take separate action to provide budget-year funding for TCRP projects. Any such action would directly affect the administration's proposed staff reduction.

Budget Proposes COS Reduction. The budget proposes $1.1 billion to fund COS in the budget year. This represents a 12 percent decrease from estimated current-year funding. The budget's proposed COS level includes funding for 9,160 personnel-years, a net decrease of 906 personnel-years from the estimated 2002-03 level. The bulk of this drop is due to a proposed reduction of 1,110 TCRP-related COS personnel-years. This reduction conforms to the administration's proposal to suspend all TCRP funding in the budget year.

Proposal Is Not Finalized. The administration typically revises its proposed COS budget in the spring based on its most recent available workload projections, as we describe in our analysis of COS budgeting later in this write up. However, for the budget year, the administration must take into account not only its estimated workload, but its available funding as well. This is particularly true for the level of TCRP staff, since the budget proposes no funding for TCRP projects in 2003-04. The administration states that it intends to reassess in the spring the mix of STIP and TCRP projects that Caltrans will work on in the budget year and revise the COS budget accordingly. Because of these factors, Caltrans' COS request is likely to change substantially. Therefore, we withhold recommendation on Caltrans' COS budget pending finalization of the proposal.

Other Legislative Actions May Affect COS Staffing. Even though we do not recommend that the Legislature take action on this proposal at this time, other actions the Legislature may take would directly affect Caltrans' COS staffing level. In our analysis of transportation funding, we list several options for the Legislature to consider regarding the level of funding to provide TCRP projects. We also recommend that the Legislature act quickly to reduce the uncertainty created by the Governor's proposals. Any action the Legislature takes regarding its commitment to TCRP projects will affect the level of TCRP-related staff Caltrans needs. For example, if the Legislature commits to funding TCRP projects with a new, predictable funding source, Caltrans may need many of the TCRP-related positions in the budget year to continue developing these projects.

State Operations Reduction Proposals Need Development

The Governor's budget proposes several actions that would reduce Caltrans' state operations expenditures by $177 million in the current and budget years. However, the administration indicates that it has not yet determined exactly where many of these cuts would be taken. Therefore, we withhold recommendation on these proposals pending the administration's revisions in the spring.

Budget Proposes State Operations Reductions. As indicated in our analysis of the condition of the state's transportation funds, Caltrans projects that the SHA will experience a shortfall of about $630 million by the end of the budget year, absent corrective actions. One of the corrective actions proposed in the Governor's budget is to reduce Caltrans' state operations expenditures by $177 million. The specific reductions proposed include:

• Reduce operating expenses by $40 million in the current year and $88 million in the budget year.
• Reduce staff department-wide by 471 personnel-years through attrition in the budget year, for a savings of $36 million.
• Reduce overtime expenditures in the budget year by $10 million.
• Remove funding for 31 state personnel-years, equating to $2.8 million, from the Local Assistance Enhanced Level of Service Program in the budget year.

The budget also proposes to redirect $72 million in operating expense funding to pay staff salaries in the budget year.

Proposal Details Remain Unknown. While the administration has proposed specific dollar amounts to be saved by each of the reductions listed above, it has not determined exactly where many of the cuts would be made. This is particularly true for $18 million of the $88 million budget-year operating expense reductions and the entire $72 million operating expense redirections to personnel costs. The Governor's budget indicates that the administration will submit a finance letter in the spring to allocate these cuts.

The Governor's budget did allocate the rest of the proposed cuts listed above to general program areas. However, discussions with administration staff indicate that they are in the process of reviewing these proposals and will revise them in the spring as well. These revisions will likely change the allocation of some of the cuts from the current proposals. Therefore, we withhold recommendation on these proposed cuts pending new administration proposals in the spring. 

Caltrans Ignores State Policy; Has Not Addressed Problems On Information Technology Project

The budget includes $1 million to develop and implement the Transportation Permits Management System (TPMS) in order to automate Caltrans' manual permit writing process. Caltrans was directed by the Department of Finance (DOF) to amend its contract for TPMS in order to improve its efficiency, but to date Caltrans has not complied with this directive. In addition, the TPMS project has experienced several schedule delays and is expected to have additional costs. We recommend that Caltrans and DOF report at budget hearings on steps they will to take to address problems with the contractor, increased costs, and schedule delays.

Caltrans issues permits to oversize vehicles—those that exceed statutory limits on vehicle size, weight, or loading—to allow them to travel on the state's highways. These permits specify the routes oversize vehicles are allowed to take in order to ensure the safety of the highway system. Currently, Caltrans issues these permits manually.

The budget includes $1 million to develop and implement the TPMS. The purpose of this system is to replace Caltrans' manual permit writing process with an automated system that is used in other states. The new system will (1) verify eligibility of permit applicants, (2) determine safe truck routes, (3) track applications through the permitting process, (4) issue and automatically deliver permits, (5) assess fees, and (6) maintain financial accounting records. The cost to develop the system is estimated at $7 million, with an annual ongoing cost of $1 million. Upon full implementation, TPMS is estimated to save $2.7 million and 54 positions annually.

TPMS Is Vital for Public Safety. Caltrans received funding for TPMS to address a poor safety record related to its oversize vehicle permits. From 1996 through early 2000, there were 31 accidents in California involving oversize vehicles that struck and damaged bridges, one of which resulted in a fatality. As an interim measure to reduce errors in its permitting process, Caltrans requested and received funding for personnel to manually double-check each permit before it was issued. Caltrans still uses this process. While this is a good interim step, it is not an acceptable permanent solution. The Bureau of State Audits found in May 2000 that Caltrans' permitting system, including the manual double-checkers, was an inefficient use of resources and did not adequately protect public safety. The audit cited the proposed TPMS system as a way to reduce the potential for human error and improve the efficiency of the process.

TPMS Contract Has Not Been Amended. During our review, we found several problems with the TPMS project. First, in December 2001, DOF notified the Legislature that Caltrans would amend its primary TPMS contract in order to comply with the administration's policy of locating servers at state data centers. This policy is appropriate since state data centers are able to support servers more efficiently and provide technical services that may not be provided by departments. Our review shows that to date, Caltrans has not yet amended the contract. Moreover, it is unclear to us why Caltrans has not complied with DOF's direction.

Caltrans Expects Amendment to Increase Contract Costs. Caltrans has received preliminary information from the contractor on the proposed amendment suggesting that contract costs will increase to meet the state's policy on the location of servers. Based on our review, we conclude that TPMS hardware's location should not affect the contractor's workload and therefore should not require increased contract costs.

Caltrans Reports Hardware Will Cost More at Data Center. According to Caltrans, the Stephen P. Teale Data Center (TDC) estimates that it will cost $1 million to locate hardware at TDC's site, whereas there would be no cost to house the hardware at Caltrans. It is typically more cost effective to locate hardware at state data centers since they are able to support more equipment with less staff at a lower cost. It is unclear to us then, why the costs to locate the hardware at TDC will increase and not decrease the project's costs.

Caltrans Continues to Spend, But Contractor Has Not Delivered Products. The Feasibility Study Report that supports this project estimated that Caltrans would spend $6.6 million and use nine positions in developing the system from December 2001 through June 2003. According to Caltrans, the primary contractor has been unable to deliver a satisfactory product since the contract was signed in 2001 and failed to meet the project's first due date in October 2002. Caltrans, however, has continued to use staff resources in developing the project. Based on this information and Caltrans' current projection of additional contract and TDC costs, the current project budget ($7 million) will be insufficient to develop and implement the system authorized by the Legislature.

Unclear If Administration Is Addressing Problems. According to the Governor's executive order on oversight for state information technology (IT) projects, department directors and agency secretaries are responsible for overseeing IT projects. In addition, the Legislature provided current-year funding to DOF to oversee and direct corrective actions for state IT projects. The DOF is aware of the problems on the TPMS project. However, it has not taken formal action to correct the problems. In addition, it is unclear what actions, if any, DOF or Caltrans has taken regarding the contractor, who has neither delivered the first product nor met the first due date.

Administration Should Report on Actions Taken to Address Problems. The administration has not adequately addressed the problems on the TPMS project. Caltrans has ignored state policy and has not addressed problems with its contractor. In addition, DOF has not directed corrective actions to a project that is delayed by at least one year and will be unable to meet budget estimates. For these reasons, we recommend Caltrans and DOF report at budget hearings on (1) steps the administration has taken or will take to address problems with the contractor and Caltrans' compliance with state policy, (2) an explanation of TDC costs, (3) estimated costs and timeframes to complete the project, and (4) revised program savings based on the new project costs. This would provide information the Legislature needs for oversight of the project.

COS: Budgeting and Performance Measurement

Simply put, COS is the work required to produce capital outlay projects. Before a capital outlay project can be constructed, Caltrans must assess environmental impacts, acquire rights-of-way, and design and engineer the project. Caltrans is also responsible for overseeing the progress of project construction. Caltrans' COS budget consists primarily of the salaries, wages, benefits, and operating expenses of the more than 10,000 state staff who perform these functions. It also includes the costs of consultants who perform a portion of this work. The COS budget does not, however, include the salaries and benefits of the contractors who construct the actual projects; these costs are part of the capital outlay budget.

In 2001-02, COS expenditures totaled almost $1.2 billion. This was about 19 percent of Caltrans' expenditures on the Highway Transportation program. Relative to expenditures on highway capital outlay, Caltrans spent 44 cents on COS for every dollar it spent on capital outlay in 2001-02. For 2003-04, the budget includes $1.1 billion for COS. This level will likely be amended in the spring after Caltrans has a better estimate of the total project workload.

Realistic COS Budget Requires an Accurate Budget Model. In any one year, Caltrans works on various phases of hundreds of STIP and State Highway Operation and Protection Program (SHOPP) projects. As Caltrans is responsible for the delivery of these projects, it is essential that the department has the necessary support resources to produce them in a timely and cost-efficient manner. Accordingly, a budget model that provides a reasonable and accurate estimate of staffing and resource needs is critical. If the budget model underestimated COS needs, Caltrans would not have enough resources to complete the work it committed to do. If the model overestimated COS needs, on the other hand, then resources would be spent on staff support inefficiently, leaving less funds for capital outlay expenditures.

In this section, we review Caltrans' COS budgeting process and the department's ability to measure the performance of the COS program. We also make recommendations to improve the budget process and provide for more thorough reporting to the Legislature.

Budgeting for COS: Inconsistent Process

Each of Caltrans' district offices estimates the resource needs to deliver and construct capital outlay projects in a different way. This can lead to inconsistencies among their estimates, which could in turn make Caltrans' capital outlay support (COS) budget request inaccurate. We recommend budget bill language directing Caltrans to develop a standard workload estimation methodology for districts to use in estimating COS needs.

Since 1998, Caltrans has developed its annual COS budget using primarily a bottom-up process, by aggregating the estimated staffing needs for each of the capital outlay projects that it plans to work on in a given year. This methodology appears to improve on the previous model by tying support resources directly to capital project workload. By contrast, the previous budget model was based on a statistical formula that tended to overestimate staffing needs.

Districts Estimate COS Resource Needs. The COS budget process begins at the district level, where the project team for each capital outlay project estimates its project workload requirements. The workload requirements are the expected number of person-hours needed to perform each phase of the work. Major phases of highway capital outlay projects include environmental studies and mitigation, project design, right-of-way acquisition, and construction oversight. These estimates form the core of the COS budget. They are also used by project teams to manage project progress. Each of Caltrans' 12 district offices submits its project workload to Caltrans headquarters, where the estimates are aggregated.

Headquarters Reviews District Estimates for Reasonableness. Headquarters staff perform several checks of the aggregated data to ensure that the district estimates of workload and resource requests are reasonable in terms of both cost and timing. For example, staff compare the ratio of projected COS costs to projected capital outlay costs for each project to statewide average ratios from 1992 through 1997. If the ratio of COS to capital outlay for a given project falls outside the ranges of the statewide average ratios, headquarters staff will review the project with the district to determine whether the district's estimate is justified or should be modified. Once headquarters is satisfied with the districts' project workload estimates, they are accepted as the basis for the COS budget request.

Headquarters Adjusts COS Estimates to Include Other Workload. Headquarters staff make additional adjustments to the COS request. First, the districts' staffing requests are adjusted to reflect the assumption that 5 percent of the work done by state staff will be overtime work. Second, headquarters determines for each district the amount of work to be accomplished through contracts. These two adjustments determine how the COS workload will be accomplished, but do not alter the total amount requested by the districts. Third, the resources Caltrans estimates that it needs to use to oversee work done by local governments on the state highway system, overhead costs such as headquarters COS staff, and other activities that do not directly relate to capital outlay on new Caltrans projects, such as storm water cleanup, are added to the final COS budget request. According to Caltrans, these costs make up about 15 percent of its total COS costs on an annual basis.

Districts Have Inconsistent COS Estimation Processes. Our primary concern with the current COS budgeting process is that each district uses a different method for estimating workload. This is problematic because the difference in approaches can result in very different estimates of time and resources needed to produce the same type of projects. For example, in one district, each member of a project team develops three estimates of the workload for each phase of a project—high, low, and most likely. The project manager then compares these estimates to the statewide average project workload requirements in order to determine which estimates to submit to headquarters. In a second district, the project team first makes a rough guess of the order of magnitude of the total resource costs for a project, based on the team members' experiences with similar projects. It then uses a computer program maintained only in that district to estimate the amount of time and staff resources each activity should take on average. Project team members then further refine the estimates based on unique features of the project. Other districts use similar but varying methods to estimate their COS workload requirements.

The districts use different estimation methods because Caltrans headquarters does not require a consistent method to be used, although it does provide the districts with an example of a process to estimate workload. Although headquarters reviews all district project workload estimates, the potential inconsistencies in project cost and schedule estimates resulting from the use of different estimation approaches may not be totally addressed. In part, this is because for certain types of capital outlay projects, the database used by headquarters contains only a small sample of these projects, with wide ranges of costs and schedule estimates. For these types of projects, headquarters' review would likely not be able to identify any inconsistencies in estimation among districts. Indeed, Caltrans concurs that a consistent department wide estimation process would provide a more sound basis for its COS budget.

Department Should Adopt Consistent Workload Estimation Methods. Caltrans indicated that it is in the process of updating its average COS to capital outlay ratios with information from projects completed within the past five years (1998 through 2002). Caltrans also indicated that it intends to impose a consistent COS estimation process when it implements a new central workload database and new project management software.

However, our review shows that the software project is several years behind schedule, and its implementation is in doubt. While it may be preferable to implement the two new systems at the same time, the implementation of one is not dependent on the other. We believe that given the status of the new database implementation, Caltrans should proceed to create a workload estimation process that is consistent across all districts. In fact, absent a new database that provides better information for estimation, it is even more important that districts use a consistent methodology to determine their project workload.

Using a consistent methodology does not mean that support resources for each project should align exactly with the average resources budgeted for that type of project. In fact, the districts should be able to take into account factors that are unique to each project and each region while using the same process statewide. A standard process would ensure that unique features of transportation projects are considered and weighted in a consistent manner, no matter their location in the state. Therefore, we recommend the following budget bill language in Item 2660-001-0042:

The Department of Transportation shall create a standard process for the estimation of capital outlay support workload, to be implemented by all of the department's district offices. The department shall create this process in conjunction with the districts and begin implementing it in all district offices no later than July 1, 2004.

Actual Project Support Expenditures Higher Than Budgeted

Even though the capital outlay support (COS) budget is directly based on the workload needs of individual projects in the budget year, our review finds that actual COS costs per project exceeded budgeted costs for the period reviewed. There are plausible explanations for this cost increase, but little data are available to validate these explanations. We recommend the Legislature request an audit by the Bureau of State Audits to determine the primary causes for the growth in support costs for capital outlay projects.

Actual COS Expenditures Per Project Are Higher Than Originally Estimated. Our review of the COS budget process also shows that the estimated support workload per project, on average, is understated when compared to actual expenditures. An examination of the budgeted and actual expenditures for COS over the past decade provides one measure of this cost increase. Figure 2 (see next page) shows budgeted COS costs as a percentage of budgeted capital outlay costs in each year, compared to actual COS costs as a percentage of actual capital outlay costs. The figure shows that the actual ratio of COS to capital outlay is consistently higher than the budgeted ratio.

The reason for the difference between the budgeted and actual COS-to-capital outlay ratios is not that Caltrans spent more on COS than it was budgeted in any given year. In fact, actual COS expenditures have consistently been very close to the budgeted amounts. The difference lies in the fact that capital outlay expenditures are consistently lower than budgeted. In part, this is because large capital projects inevitably encounter delays. The fact that Caltrans spends its entire COS budget every year while capital outlay expenditures are consistently lower than budgeted implies that the actual support costs to produce a project are higher than budgeted.

A more direct measure of how COS costs increase over time is to examine the changes in support cost estimates for a sample of capital outlay projects as they are developed. The DOF did this in a December 2000 report entitled, How the California Department of Transportation Budgets Capital Outlay Projects. It examined the support costs of 76 capital outlay projects in four Caltrans districts and compared the latest cost estimates (at the time of the study) to the original estimates made one or two years earlier. The study found that, on average, estimates of project support costs grew. For instance, in one district, the support hours for the 15 projects sampled increased over initial estimates by an average of 71 percent in two years. In another two districts, support cost estimates were revised upward by 10 percent and 22 percent, respectively.

Reasons for Cost Increases Should Be Identified to Improve Cost and Project Management. Caltrans argues that an optimistic—that is, low—projection of workload needs provides a target against which to measure the department's performance in producing projects. However, the approach could result in the setting of unrealistic targets and the adoption of a budget that overstates the department's cost-efficiency in project delivery. Because there are many possible reasons why COS costs could grow as projects progress, it is important that the causes of the cost increases be identified. Absent this information, it is difficult to determine what are the necessary corrective actions to better manage projects and their costs and to improve the estimation of workload needs. For example, if the bulk of the COS cost growth is due to prolonged environmental review processes, then efforts should focus on environmental review streamlining. If the cost increase is mainly due to public opposition to projects, then more effort should be directed toward public outreach. If, however, the bulk of the growth is due to inefficiencies in Caltrans' engineering processes, then reforms of departmental policies and processes may be warranted.

To determine the primary causes of COS cost growth would require an in-depth, statistically valid study to examine COS expenditures associated with a sample number of projects. To do so, we recommend that the Legislature request the Bureau of State Audits to perform an audit of Caltrans' COS expenditures to determine both the extent of COS cost growth and its primary causes.

Evaluating COS Performance: Imperfect Information

While Caltrans has established a number of performance measures for its capital outlay support (COS) program, it has not set targets for those that measure COS expenditures. Additionally, the current performance measures are not sufficient to allow the Legislature to evaluate whether Caltrans' COS budget requests are warranted because they do not tie COS costs to specific capital outlay projects. We recommend the adoption of budget bill language to require that Caltrans set targets for each of its COS program performance measures. We further recommend the enactment of legislation to require Caltrans to establish new performance measures beginning with the 2004-05 Governor's Budget.

Current COS Performance Measures Have No Targets. In response to a 1994 consultant report on Caltrans' organizational structure and management practices and a requirement in the Supplemental Report of the 1995-96 Budget Act, Caltrans developed performance measures for its COS program. These measures were intended to (1) provide an accurate measure of annual efficiency, (2) provide a consistent basis for year-to-year comparisons, and (3) evaluate both the department's cost and its timeliness in the delivery of capital projects. Caltrans subsequently adopted a number of performance measures for the COS program. Of these, we have identified three that most closely relate to COS expenditures. These measures are detailed in Figure 3.

However, as Figure 3 shows, Caltrans has not set target values for any of the three performance measures even though data are available for the department to do so. Specifically, data are available for the project development and construction support cost change measures for three years, and data exist for the COS versus capital outlay expenditure measure back to at least 1988-89. In fact, Caltrans at one time had set a target of 33 percent for the COS versus capital outlay measure. That is, COS costs in a given year should not be more than 33 percent of capital outlay expenditures. However, Caltrans has only met this target once since 1988-89, and it abandoned the target in 2000-01.

Performance Should Be Measured Against Targets. We believe that to fully evaluate the performance of a program, the program's accomplishments must be assessed against defined targets for performance measures. This is consistent with the recommendations of the Government Accounting Standards Board (GASB), a standard-setting body for state and local government organizations. A GASB principle is that, to facilitate the budgetary decision-making process, each government entity should "establish and communicate clear, relevant goals and objectives; set measurable targets for accomplishment; and develop and report indicators that measure its progress in achieving those goals and objectives." Caltrans' COS performance measures do not currently meet these standards. Without defined targets, it is difficult for the Legislature to evaluate the department's efficiency and effectiveness in completing projects in terms of cost and timeliness.

We recommend the adoption of budget bill language requiring Caltrans to set targets for each of its performance measures. However, as we discuss in greater detail below, there are shortcomings with some of the measures that Caltrans currently uses. Caltrans has indicated that several of its COS performance measures should be revised. Therefore, we recommend that the budget bill language exclude existing performance measures from the target-setting requirement if they will be replaced by the beginning of 2004-05. 

Item 2660-001-0042: The Department of Transportation shall develop measurable targets for each of the performance measures reported in its Capital Support Performance Measures Report for 2001-02, excluding those performance measures that will be replaced by July 1, 2004. These targets shall be included in the department's Capital Support Performance Measures Report for 2002-03.

Existing Performance Measures Inadequate. More important than the lack of targets, our review shows that existing COS performance measures are inadequate and do not allow for a full evaluation of the COS program's performance. One measure—the COS versus capital outlay expenditure measure—is of particular concern.

Specifically, this measure compares COS and capital outlay expenditures in the same year, but most of the COS expenditures in a given year do not relate to the capital outlay expenditures in that year. This is because capital outlay projects often take several years to develop and construct. For instance, expenditures for environmental review and design of a project—which occur before any capital outlay dollars are spent—may take multiple years. Thus, part of the COS costs in any year would not have any capital outlay expenditures associated with them. Likewise, projects that are at the height of construction in any year would incur large capital outlay expenditures but relatively low COS expenditures. Therefore, comparing the capital outlay expenditures in any one year with the COS costs incurred in the same year does not provide an accurate measure of the cost efficiency in producing capital outlay projects. If, for example, Caltrans is in the process of expanding its workload in a given year, the COS expenditures in that year may be a much larger percentage of capital outlay than is desirable for a single project, because COS expenditures are weighted toward the beginning of a project. Conversely, as workload declines, COS costs should shrink before capital outlay does. A comparison of COS to capital outlay expenditures in one year cannot adequately recognize these situations.

Better Measures Needed to Properly Evaluate COS Budget Request and Performance. In order to allow the Legislature to more effectively evaluate Caltrans' performance, better performance measures are needed. Our review shows that the following three measures would, in combination, provide a more accurate picture of the COS program's performance. In particular, the third measure also provides better information for the Legislature to assess the COS budget request. These measures are summarized in Figure 4.

One measure should show aggregate COS and capital outlay expenditures over time for all the projects that complete construction in a given year. This measure—past efficiency—would allow a comparison of the ratio of COS to capital outlay expenditures over the life of a single set of projects. This would improve the existing COS versus capital outlay measure by avoiding a comparison of two numbers that do not directly relate to each other. In fact, we believe this should be the primary measure of the efficiency of the COS program. It would provide a consistent basis for evaluating Caltrans' COS expenditures over time, regardless of fluctuations in the amount of COS workload.

While a measure of COS efficiency for completed projects would be valuable, it would primarily measure Caltrans' past efficiency, rather than its current performance. This is because most of the work Caltrans performs on a project occurs in years prior to the date the measure would be produced. To provide some indication of Caltrans' expenditures on ongoing projects, we believe a second measure—estimated current efficiency—is needed. This measure should aggregate for all projects that begin construction in a given year their project development COS costs over time as well as their projected total capital outlay costs.

In fact, current law requires Caltrans to calculate a version of this measure for STIP projects. Current law also requires that the ratio of project development COS costs to capital outlay costs calculated by this measure average 20 percent or lower over any given three-year period. Caltrans indicates that this ratio averaged just over 16 percent for the three-year period from 1999-00 through 2001-02. 

However, the above two measures would not account for all of Caltrans' annual COS expenditures. This is because Caltrans works on many projects in any given year that neither begin nor end construction in that year. To capture ongoing expenditures on these remaining projects, a third measure would be needed to track projected COS costs in the budget year for groups of projects grouped by the year the construction contract is to be awarded. Current- and prior-year figures should also be provided for comparison purposes. This measure—detailed budget breakdown—would allow the Legislature to better evaluate Caltrans' annual COS budget request by breaking the request down into its component parts. Thus, if Caltrans asked for a large COS budget increase without a similar capital outlay budget increase, for example, this measure would allow the Legislature to determine whether this increase was related to new projects to be delivered in the future or more support work needed for the set of current projects.

Measures Should Cover STIP and SHOPP Projects Separately. Caltrans works on many different types of projects at any given time. Some projects are generally large efforts to add capacity to the highway system—STIP projects—while others are usually smaller efforts to improve the condition of existing parts of the system—SHOPP projects. Caltrans works on other categories of projects as well, but STIP and SHOPP projects are the two primary types. These different types of projects have different timelines as well as different patterns of support and capital outlay expenditures. If the performance measures we propose above grouped all capital outlay projects together, they would intermingle projects with different expected ratios of COS to capital outlay expenditure, rendering any targets for the measures meaningless. Therefore, we believe Caltrans should calculate separate measures for STIP and SHOPP projects.

Recommend Legislation Requiring New Measures. We believe that the above measures would provide the information needed for the Legislature to evaluate Caltrans' annual COS budget request accurately. The measures would also allow a more complete assessment of Caltrans' efficiency and effectiveness in developing and overseeing the construction of capital outlay projects. Accordingly, we recommend the enactment of legislation to require Caltrans to begin tracking and reporting data for these performance measures in 2004-05.

Project Delivery

Project delivery is arguably the most critical element in Caltrans' mission to improve mobility. Because of concerns over project delays, the Legislature requires our office to report on the department's progress in delivering projects that are scheduled for construction in the STIP and the SHOPP.

Our findings related to Caltrans' project delivery are summarized in Figure 5. First, we found that Caltrans delivered a lower percentage of its planned STIP and SHOPP projects in 2001-02 than in 2000-01, but the total number and value of projects delivered increased substantially. We also found that the reason for the large fluctuation in STIP projects delivered from year to year was due to a record amount of project rescheduling in the past three years, by which Caltrans and local agencies moved their scheduled project delivery dates to later years. Because of this, neither the adopted STIP nor the annual budget reflect Caltrans' actual project delivery schedule, and there is no baseline against which to measure delays in Caltrans' project delivery. Also, the number of STIP and SHOPP environmental documents Caltrans completed in 2001-02 and plans to complete in 2002-03 are significantly lower than in the previous two years. This suggests that fewer projects will be delivered in the future, contrary to the administration's stated intent to accelerate the delivery of transportation projects.

In the following section, we discuss these project delivery issues, as well as project delivery for local agencies and the seismic retrofit program.

Caltrans Project Delivery Mixed

In 2001-02, Caltrans delivered 86 percent of programmed State Transportation Improvement Program (STIP) projects and 89 percent of programmed expenditures. These percentages represent decreases from 2000-01, but the total number and value of projects delivered increased substantially. Delivery of State Highway Operation and Protection Program projects remained high.

In the Analysis of the 2001-02 Budget Bill, we adopted the CTC's definition of project delivery. This definition compares the number of projects that were allocated funding by CTC to the number of projects programmed in the STIP or SHOPP for delivery in that year. (Please see page A-37 of the 2001-02 Analysis.)

Figure 6 summarizes the number of projects Caltrans delivered ("allocated funding") compared to the number programmed in the STIP and SHOPP. Figure 7 (see next page) shows delivery in terms of dollar volume.

STIP Delivery Percentage Decreased From Previous Year, But Amount of Delivery Was Significantly Larger. According to information provided by CTC, in 2001-02 Caltrans delivered 86 percent of STIP projects that were programmed for delivery in that year, as shown in Figure 6. These are projects that primarily expand highway capacity. In terms of expenditures (Figure 7), the department delivered $675 million, 89 percent, of the programmed level for 2001-02.

The percentage of programmed STIP projects delivered in 2001-02 was lower than in the previous year. However, Caltrans delivered more projects for a higher total dollar value of work in 2001-02. Specifically, Caltrans delivered $675 million worth of projects—214 percent more than the value of projects delivered in 2000-01.

Delivery of SHOPP Projects Remains Strong. With respect to SHOPP projects, the department delivered 175 projects, or 97 percent of the projects that were programmed for delivery. The SHOPP projects provide safety, operation, or rehabilitation improvements to the state highway system. In terms of funding allocations, the department delivered $825 million, or 98 percent, of the amount in programmed funds. In general, SHOPP projects are less complicated from a design standpoint and require less extensive environmental review. This makes them, in general, easier to deliver on schedule than STIP projects.

Department Delivered Some Projects Programmed for Different Years. Figures 6 and 7 only show delivery of projects programmed for 2001-02. They do not include the delivery of projects scheduled for delivery in other years. In 2001-02, the department delivered ten STIP projects ahead of schedule and seven projects that had originally been programmed for delivery in 2000-01. With these projects, STIP delivery in 2001-02 totaled $813 million. With respect to SHOPP projects, the department delivered 16 projects that were advanced from future years and no projects from prior years. 

We support the department's practice of advancing projects ahead of schedule when possible. However, we do not include these projects in our main calculations because the Legislature's primary concern has been how well Caltrans meets its intended delivery schedule, which more closely reflects its original priority of projects. Likewise, including delivery of delayed projects would not provide a true representation of Caltrans' project delivery.

Extensive Project Rescheduling Decreases Value Of Adopted STIP and Annual Budget

Caltrans has experienced large fluctuations in State Transportation Improvement Program (STIP) delivery over the past three years due to record levels of project rescheduling. This level of rescheduling of STIP projects calls into question the value of the STIP as a scheduling tool because it allows Caltrans to annually move the baseline against which its performance is measured. Furthermore, rescheduling project delivery after Caltrans' annual budget renders its capital outlay and capital outlay support budget meaningless. We recommend that Caltrans and the California Transportation Commission (CTC) report at budget hearings on the reasons for the high levels of project rescheduling in recent years. We further recommend adoption of budget bill language requiring CTC to include strategies for reducing this level of rescheduling in its next annual report.

Large STIP Fluctuation Due to Project Rescheduling. As we indicated in the prior section, Caltrans delivered $675 million worth of projects in 2001-02—214 percent more than the dollar value of projects delivered in 2000-01. Our review shows that the reason for the large increase in STIP delivery in 2001-02 was not that Caltrans' STIP project delivery was abnormally high in that year. Rather, its delivery was abnormally low in the previous year. Figure 8 (see next page) shows the value of Caltrans' STIP project delivery over the past three years. The reason for the dip in 2000-01 was the rescheduling of an unprecedented level of projects to later years. Specifically, $788 million in projects were rescheduled, including $646 million that was originally programmed to be delivered in 2000-01. As a result, the delivery goal for 2000-01 shrank significantly. Many of these projects were rescheduled to be delivered in 2001-02, which would have caused a spike in project delivery in that year given the number of projects already scheduled for delivery. However, Caltrans again rescheduled a large number of projects from 2001-02 to later years, so that the level of project delivery was not abnormally high. This practice has continued in the current year, with STIP projects worth about $870 million rescheduled from delivery in 2002-03 and 2003-04 to later years. 

Extensive Rescheduling Decreases Value of STIP and Budget. This magnitude of rescheduling calls into question the value of the biennial STIP as a project scheduling tool. In effect, rescheduling allows Caltrans to annually move the baseline against which its performance is measured, thereby making it difficult to get a picture of Caltrans' "true" project delivery performance.

Furthermore, rescheduling moves the project delivery baseline after Caltrans develops its annual budget request. As we describe in our analysis of COS budgeting earlier, Caltrans bases its annual COS budget request (submitted in the spring) on its workload projections as of the January prior to the budget year. Its capital outlay budget request is set even earlier. However, Caltrans adjusts its budget-year delivery schedule with CTC throughout the spring preceding the budget year. Therefore, the capital outlay and COS funding that Caltrans requests in the budget do not match the projects Caltrans actually plans to deliver that year. Because this funding is not tied to specific deliverables, the Legislature is unable to hold Caltrans accountable for its annual expenditure of billions of dollars in capital outlay and COS money.

While some project rescheduling will always occur, the amount of STIP rescheduling witnessed in the past few years should be reduced. To begin to address the discrepancies between Caltrans' project delivery schedule, the annual budget, and the adopted STIP, we recommend that Caltrans and CTC report at budget hearings on the reasons for the high level of STIP project rescheduling exhibited in the past three years. Furthermore, we recommend the adoption of the following budget bill language in Item 2600-001-0046:

The California Transportation Commission shall include in its 2003 annual report to the Legislature pursuant to Government Code 14535 and 14536 an analysis of strategies to reduce the high level of rescheduling of State Transportation Improvement Program projects exhibited in 2000, 2001, and 2002.

Local Project Delivery Good, Largely Unchanged

In 2001-02, local agencies delivered 81 percent of programmed STIP projects and expenditures. They also obligated more than 100 percent of their annual allocation of federal funds, continuing to reduce their previously high balance of unobligated federal funds.

Local Agencies Delivered 81 Percent of Programmed Expenditures. Under Chapter 622, Statutes of 1997 (SB 45, Kopp), local agencies are responsible for determining how to spend 75 percent of STIP funds. To the extent that local agencies decide to spend their share of STIP funds on highway capacity improvements, they have traditionally depended on Caltrans to deliver the projects. However, to the extent that they choose to spend their share of funds on transit projects or local road improvements, they are responsible for that delivery.

In 2001-02, local agencies delivered 453, or 81 percent, of the local street and road or mass transit projects programmed in the STIP for delivery in that year. These projects totaled $400 million. Like Caltrans, however, local agencies also delivered a significant amount of projects that were scheduled for different years. Specifically, local agencies delivered 84 projects from future and prior years. These additional projects bring total delivery by local agencies to $491 million.

Local Agencies Continue Strong Use of Federal Funds. Over the past few years, local agencies' use of federal funds that they receive directly has improved significantly. In 1998 and 1999, the first two years of the current federal transportation act (TEA-21), local agencies underspent their allotment of federal funds by 41 percent and 57 percent, respectively. As a result, by October 1999, local agencies had accumulated $1.2 billion in unobligated federal allocations.

To remedy this situation, Chapter 783, Statutes of 1999 (AB 1012, Torlakson) was enacted to allow CTC to redirect most of a local agency's unused federal funds to another agency before the federal obligation authority expired. As a result, in 2000 and 2001, local agencies increased markedly their use of the major categories of federal funds, obligating 154 percent and 124 percent of their share of federal funds, respectively. Local agencies continued their strong delivery in 2002, obligating 101 percent of their federal funds. As a result, the amount of unobligated federal funds has been reduced to about $600 million. This is a big step toward lowering the backlog to a reasonable level of between $100 million and $200 million.

Number of Completed STIP and SHOPP Environmental Documents Falls

Caltrans completed 73 percent of the State Transportation Improvement Program (STIP) environmental documents it planned to complete in 2001-02, its highest percentage in five years. However, the total number of documents completed declined 41 percent from the previous year. Completion of State Highway Operation and Protection Program (SHOPP) environmental documents exhibited the same trend, with a higher percentage completed than in the previous year, but a drop of 42 percent in total number of documents delivered. These low levels of environmental document completion suggest that future STIP and SHOPP project delivery levels will be low.

Number of Completed STIP Environmental Documents Drops 41 Percent. Our review of planned environmental documents completed for STIP projects last year reveals that of the 44 environmental documents the department planned to complete during 2001-02, 32 were completed. The remaining 12 rolled forward to 2002-03 and beyond. This completion rate (73 percent) represents the highest percentage of STIP environmental document completion in the last five years. However, as Figure 9 shows, the total number of documents completed decreased by 41 percent from the number delivered in 2000-01. Unfortunately, this trend of decreased delivery appears to extend into the current year, when Caltrans plans to complete only 41 final environmental documents for STIP projects. Caltrans completed only three of these documents in the first quarter of the current year.

Environmental Document Completion for SHOPP Declines 42 Percent. Completion of environmental documents for SHOPP projects in 2001-02 exhibited the same trend as noted above for STIP environmental documents. Caltrans completed 59 of 78 planned SHOPP environmental documents, for a completion rate of 76 percent. This surpasses the previous year's completion rate, but the total number of documents completed dropped 42 percent from the 101 documents delivered in 2000-01. Similar to the trend for STIP projects, Caltrans plans to complete even fewer SHOPP environmental documents in the current year. Only 62 final SHOPP environmental documents are planned for 2002-03, of which Caltrans completed eight in the first quarter.

Decreased Environmental Document Completion Signals Lower Project Delivery in the Future. Since all transportation construction projects must receive environmental clearance, Caltrans' level of environmental document completion is a good indicator of future project delivery. The low levels of environmental documents planned and completed in 2001-02 and 2002-03 suggest that future levels of STIP and SHOPP project delivery will be low as well. For STIP projects, this is a reversal of a trend toward increased environmental document completion exhibited in the past few years, which was attributed to the department's efforts at streamlining its environmental process. This reversal is due in part to a decrease in the number of documents being rolled over from the previous year. However, our review shows that there is also a decrease in the number of new documents initiated. A decrease in environmental document completion suggests that fewer projects will be delivered in the future, contrary to the administration's declared intent to accelerate the delivery of transportation projects. 

Delays Continue in Bridge Seismic Retrofit Work

Phase 1 of the highway bridge seismic retrofit program is complete. Phase 2 is 98 percent complete, but work will not be completed on some bridges until 2010. Seismic retrofit of the state-owned toll bridges continues to be delayed.

Caltrans inspects all state and local bridges at least once every two years. Since 1971, when the Sylmar earthquake struck the Los Angeles area, Caltrans has had an ongoing bridge retrofit program. The retrofit program involves a variety of different improvements, depending on the needs of the particular structure. The improvements include strengthening the columns of existing bridges by encircling certain columns with a steel casing, adding pilings to better anchor the footings to the ground, and enlarging the size of the hinges that connect sections of bridge decks to prevent them from separating during an earthquake.

Following the 1994 Northridge earthquake, Caltrans expanded its seismic retrofit program for state highway bridges, creating a Phase 1 and a Phase 2 program. Phase 1 included 1,039 bridges identified for strengthening after the 1989 Loma Prieta earthquake at a total construction cost of $800 million, as shown in Figure 10. These projects were completed by May 2000. Phase 2 consists of an additional 1,155 bridges that were identified for strengthening following the Northridge earthquake. To date, Caltrans has completed the work on 1,135 (98 percent) of the Phase 2 bridges and estimates Phase 2 construction costs to be $1 billion. However, Caltrans estimates some Phase 2 projects will not be completed until 2010 due to more complex retrofit and replacement work on a number of these bridges. 

Phase 2 seismic retrofit work is funded primarily by bonds authorized by Proposition 192 in 1996. As of September 2002, only about $43 million of Proposition 192's authorized bonding authority remained unallocated. However, Caltrans has not yet allocated any funds for four of the remaining Phase 2 bridges, and their costs are unknown. If costs for the seismic retrofit of Phase 2 bridges exceed the amount of available funding, Caltrans plans to use federal highway bridge funds to complete the work. This, in turn, will place pressure on the SHA to fund projects that otherwise would have been funded by the federal funds.

Schedule Continues to Slip for Toll Bridge Retrofit. Caltrans is also retrofitting seven of the state's toll bridges for seismic safety. As Figure 11 indicates, Caltrans has completed work on five of the seven bridges, but the scheduled completion dates for the retrofit of the remaining bridges are now much later than Caltrans' original projections. 

Caltrans indicates that the delays in the retrofit work are due to numerous factors, and each bridge's delays are unique. For example, the east span of the Bay Bridge has been delayed more than three years from its original projected completion date. Caltrans indicates that this delay is due partly to the United States Navy's initial refusal to grant an encroachment permit to allow Caltrans to drill on Yerba Buena Island and partly to Caltrans' inability to release the bid for the first contract on the east span until a federal loan was approved.

The west span of the Bay Bridge, on the other hand, has been delayed almost six years from its original completion date, which Caltrans indicates is due in part to safety issues with the ongoing work on the bridge. As a further example, the Richmond-San Rafael Bridge was delayed four and a half years according to Caltrans because the department redesigned the retrofit schedule to reduce the scope of work in environmentally sensitive locations during certain seasons. Because many of the factors causing delays are unanticipated, it is likely that Caltrans could encounter more delays as projects progress.

Mass Transportation

Heavy Maintenance of Rail Equipment Not Yet Justified

We withhold recommendation on $10.3 million requested for heavy maintenance of intercity rail equipment pending further information from the department. 

Caltrans owns a fleet of passenger rail equipment, including passenger cars and locomotives for its Intercity Rail program. This program supports and funds intercity passenger rail service on three corridors—the Pacific Surfliner in Southern California, the San Joaquin in the Central Valley, and the Capitol in Northern California.

In 2000-01, the program's support level was increased by $5.8 million to provide ongoing funding for heavy maintenance and parts replacement of the equipment. For 2003-04, the budget requests $10.3 million for these activities. However, the department did not submit a budget change proposal for the increase, and at the time this analysis was prepared, had not provided sufficient information to justify the increase. Accordingly, pending further information from the department, we withhold recommendation on $10.3 million from the Public Transportation Account for heavy maintenance of rail equipment