
Performance Measure Completeness and Reliability Details
Each table includes a description of a performance measure and associated data provided by the agencies in charge of the measure. The Scope statement gives an overview of the data collection strategy for the underlying data behind the performance measure. The Source statement identifies the data system(s) from which the data for each measure was taken. The Statistical Issues statement has comments, provided by the Bureau of Transportation Statistics (BTS) and the agency in charge of the measure, which discuss variability of the measure and other points. The Completeness statement indicates limitations due to missing data or availability of current measures, methods used to develop projections are also provided, as appropriate. The Reliability statement gives the reader a feel for how the performance data are used in program management decision making inside DOT.
For further information about the source and accuracy (S&A) of these data, and DOT’s data quality guidelines in accordance with Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (P.L. 106-554), please refer to the BTS S&A compendium available at http://www.bts.gov/programs/statistical_policy_and_research/source_and_accuracy_compen dium/index.html.
Details on DOT Safety Measures
Highway Fatality Rate
(NHTSA – Fatality Analysis Reporting System (FARS), FHWA—Traffic Volume Trends (TVT))
|
Measure: |
Highway fatalities per 100 million vehicle-miles traveled (VMT) Calendar Year (CY) |
|---|---|
|
Scope: |
The number of fatalities is a count of occupant and non-motorist deaths which occur within 30 days of a crash involving motor vehicle traffic traveling on a trafficway customarily open to the public within the 50 States and Washington, D.C. VMT represent the total number of vehicle miles traveled by motor vehicles on public roadways within the 50 States and Washington, D.C. |
|
Sources: |
Motor vehicle traffic fatality data are obtained from the National Highway Traffic Safety Administration’s Fatality Analysis Reporting System (FARS). The FARS database is based on police crash reports and other State data. VMT data for 2005 are estimated based on preliminary 2004 VMT data from FHWA’s Traffic Volume Trends (TVT); a monthly report based on hourly traffic count data in the Highway Performance Monitoring System (HPMS). VMT data for 2004 and prior years are from the HPMS system based on State samples of road segments. |
|
Statistical Issues: |
The primary source of uncertainty in the fatality rate measure is the denominator, VMT. While the number of total fatalities used in the numerator is derived from census data and is relatively accurate, the VMT estimate in the denominator has far more variability. The TVT data used for the 2005 VMT are an early estimate from the 2004 VMT. These data, collected at approximately 4,000 continuous traffic counting locations nationwide, are used to determine the percentage change in traffic for the current month from the same month of the previous year. The percentage change is applied to the nationwide travel for the same month of the previous year to obtain an estimate of nationwide travel for the current month. The 2004 and earlier VMT are compiled from data provided to FHWA from each State. They are estimates based on a sample of road segments, so the numbers have associated sampling errors. The methodology used by each of the States to estimate VMT varies and may introduce additional non-sampling errors. Although States provide VMT estimates on an annual basis, they are only required to update their traffic counts at all sampling sites once every 3 years. Thus, an annual VMT estimate from a particular State may be based, in part, on data collected during a previous year. |
|
Completeness: |
FARS has been in use for many years and is generally accepted as a complete measure for describing safety on the Nation’s highways. Total annual fatalities are available through CY 2004. The fatality estimates used to calculate the 2005 rates shown in this report were forecasted using the most recent data from 1997 to 2003. Inputs were monthly fatality counts from FARS from 1997 to 2003. NHTSA’s first official estimates for 2005, the Early Projections, will be completed in spring 2006. Differences between the official Early Projection estimates and those in this report are to be expected. VMT data for 2004 are preliminary estimates provided by the Federal Highway Administration (FHWA). VMT data used to calculate the 2005 rates shown in this report are projected assuming an increase rate of 1.5 percent (based on previous increases in VMT) from the 2004 VMT estimate. The final measure of VMT for CY 2005 from the HPMS system will not be available until October 2006. |
|
Reliability: |
The measure informs and guides NHTSA highway safety policy, safety program planning, regulatory development, resource allocation, and operational mission performance, and tracks progress toward the goal of saving lives by preventing highway crashes. |
Large Truck-Related Fatalities
(FMCSA, NHTSA—Fatality Analysis Reporting System (FARS), FHWA—Traffic Volume Trends (TVT))
| Measure: |
Fatalities involving large trucks per 100 million truck VMT. (CY) |
|---|---|
| Scope: |
The measure includes all fatalities (e.g., drivers and occupants of passenger cars, motorcycles, large trucks, or pedestrians) associated with crashes involving trucks with a gross vehicle weight rating of 10,000 pounds or more. Truck Vehicle Miles of Travel (TVMT) represents the total number of vehicle miles traveled by large trucks on public roadways within the 50 States and Washington, D.C. |
| Sources: |
The number of fatalities comes from NHTSA’s Fatality Analysis Reporting System (FARS) data, a census of fatal traffic crashes within the 50 States and Washington, D.C. The TVMT data are derived from the FHWA’s Highway Performance Monitoring System (HPMS) |
| Statistical Issues: |
The fatality counts in FARS are generally quite accurate. The major sources of error are underreporting by some precincts and inconsistent use of the definition of a truck. Because the TVMT data provided to FHWA from each State are estimates based on a sample of road segments, the numbers have associated sampling errors. The methodology used by each of the States to estimate TVMT varies and may introduce additional non-sampling error. Although States provide TVMT estimates on an annual basis, they are only required to update their traffic counts at all sampling sites once every three years. Thus, a portion of each States’ sample sites will report estimated traffic rather then actual traffic counts. |
| Completeness: |
The FARS has been in use for many years and is generally accepted as a complete measure for describing safety on the Nation’s highways. Truck-related fatality data is complete through 2004. For 2005, the FARS data for crashes involving large trucks are not available. The value used for the 2004 rate is projected from 1997–2004 trend data. The actual fatality count for 2005 will be available in October 2006. The TVMT is complete through 2003. For 2004 and 2005, it is projected using the historical trend with adjustments for observed change in the total VMT in 2003. The final TVMT estimate for 2004 will be available in December 2005, and the final TVMT estimate for 2005 will be available in December 2006. |
| Reliability: |
The measure informs and guides FMCSA highway safety policy, safety program planning, regulatory development, resource allocation, and operational mission performance, and tracks progress toward the goal of saving lives by preventing truck and bus crashes. |
Air Carrier Fatal Accident Rate
| Measure: |
U.S. commercial fatal aviation accidents per 100,000 (Last 3 years’ average) (FY) |
|---|---|
| Scope: |
This measure includes both scheduled and nonscheduled flights of large U.S. air carriers (14 CFR Part 121) and scheduled flights of regional operators (14 CFR Part 135). It excludes on –demand (i.e., air taxi) service and general aviation. |
| Sources: |
Fatal aviation accidents: The data on commercial and general aviation fatalities come from the National Transportation Safety Board's Aviation Accident Database. Aviation accident investigators under the auspices of the National Transportation Safety Board develop the data. Departures Performed: The Office of Airline Information (OAI) within the Bureau of Transportation Statistics (BTS) collects the data on Form 41, Schedule T-100—U.S. Air Carrier Traffic and Capacity Data By Nonstop Segment and On-flight Market and Form 41, Schedule T-100 (f)—Foreign Air Carrier Traffic and Capacity Data by Nonstop Segment and On-flight Market. |
| Statistical Issues: |
The joint government/industry group working on improving the level of safety for U.S. commercial aviation has determined that the number of departures is a better denominator measure to use for determining accident rates and the Government Accountability Office recommended that FAA use departures. Both accidents and departures are censuses, having no sampling error. However, missing data, particularly in the departure counts, will result in bias to some degree. |
| Completeness: |
The FAA does comparison checking of the departure data collected by BTS. However, FAA has no independent data sources against which to validate the numbers submitted to BTS. FAA compares its list of carriers to the DOT list to validate completeness and places the carriers in the appropriate category (i.e., Part 121 or Part 135). NTSB and FAA's Office of Accident Investigation meet regularly to validate the accident count. To overcome reporting delays of 60 to 90 days, FAA must rely on historical data, partial internal data sources, and Official Airline Guide (OAG) scheduling information to project at least part of the fiscal year activity data. Due to reporting procedures in place, it is unlikely that calculation of future fiscal year departure data will be markedly improved. Lacking complete historical data on a monthly basis and independent sources of verification increases the risk of error in the activity data. |
| Reliability: |
Results are considered preliminary based on projected activity data. FAA uses performance data extensively for program management, personnel evaluation, and accountability. Most accident investigations are a joint undertaking. National Transportation Safety Board (NTSB) has the statutory responsibility, but, in fact, most of the accident investigations related to general aviation are conducted by FAA Aviation Safety Inspectors without NTSB direct involvement. FAA’s own accident investigators and other FAA employees participate in all accident investigations led by NTSB investigators. |
General Aviation Fatal Accidents
| Measure: |
Number of fatal general aviation accidents. (FY) |
|---|---|
| Scope: |
The measure includes on-demand (non-scheduled FAR Part 135) and general aviation. General aviation includes a diverse range of aviation activities. The range of general aviation aircraft includes single-seat homebuilt aircraft, helicopters, balloons, single and multiple engine land and seaplanes including highly sophisticated extended range turbojets. |
| Sources: |
The data on commercial and general aviation fatalities come from the National Transportation Safety Board's Aviation Accident Database. Aviation accident investigators under the auspices of the National Transportation Safety Board develop the data. |
| Statistical Issues: |
There is no major error in the accident counts. Random variation in air crashes results in a significant variation in the number of fatal accidents over time. |
| Completeness: |
NTSB and FAA’s Office of Accident Investigation meet regularly to validate information on the number of accidents. It would be preferable to use fatal accident rates rather than fatal accidents as the performance measure. However, general aviation flight hours are based on an annual survey conducted by the FAA. Response to the survey is voluntary. The accuracy of the flight hours collected is suspect and there is no readily available way to verify or validate the data. For this reason, the General Aviation community is unwilling to use a rate measure until the validity and reliability of the survey data can be assured. Results are considered preliminary. NTSB continues to review accident results from FY 2004. |
| Reliability: |
FAA uses performance data extensively for program management and personnel evaluation and accountability. Most accident investigations are a joint undertaking. NTSB has the statutory responsibility, but, in fact, most of the accident investigations related to general aviation are conducted by FAA Aviation Safety Inspectors without NTSB direct involvement. FAA’s own accident investigators and other FAA employees participate in all accident investigations led by NTSB investigators. |
Train Accidents Rate
| Measure: |
Rail-related accidents and incidents per million train-miles. (FY) |
|---|---|
| Scope: |
The RSIS is the principal monitoring strategy used by the FRA for the management, processing and reporting on railroad reported accidents/incidents; railroad inspections; highway-rail grade crossing data; and related railroad safety activities. The Railroad Accident/Incident Reporting Subsystem (RAIRS) is the repository of all FRA-mandated reports of railroad accidents, incidents, casualties, highway-rail grade crossing collisions, and operating information. A train accident is any collision, derailment, fire, explosion, act of God, or other event involving the operation of railroad on-track equipment (standing and moving), which results in damages greater than the current reporting threshold to railroad on-track equipment, signals, track, track structures, and roadbed. Train accidents are reported on form FRA F6180.54, Rail Equipment Accident/Incident Report. The reporting threshold for 2005 is $6,700. A train incident is any event involving the movement of on-track equipment that results in a reportable casualty but does not cause reportable damage above the current threshold established for train accidents. Operational data, including Train Miles, is reported on the form FRA F6180.55, Railroad Injury Illness Summary. |
| Sources: |
FRA’s Railroad Accident/Incident Reporting Subsystem. |
| Statistical Issues: |
None. |
| Completeness: |
Railroads are required by regulation (49 CFR 225) to file monthly reports to the FRA of all train accidents that meet a dollar threshold (currently $6,700). They are also required to file monthly operations reports of train-miles, employee hours, and passenger train-miles. Reports must be filed within 30 days after the close of the month. Data must be updated when the costs associated with an accident vary by more than 10% (higher or lower) from that initially reported. Railroad systems that do not connect with the general rail system are excluded from reporting to FRA. Examples include subway systems (e.g., Washington, D.C. Metro, New York City subway, San Francisco Bay Area Rapid Transit District), track existing inside an industrial compound, and insular rail (e.g., rail that is not connected to the general system and does not have a public highway rail crossing or go over a navigable waterway). |
| Reliability: |
FRA uses the data in prioritizing its inspections and safety reviews, and for more long-term strategic management of its rail safety program. FRA has inspectors who review the railroads’ reporting records, and who have the authority to write violations if railroads are not reporting accurately. Violations may result in monetary fines. |
Transit Fatality Rate
| Measure: |
Transit fatalities per 100 million passenger-miles traveled (CY) |
|---|---|
| Scope: |
Transit fatality data includes passengers, revenue facility occupants, trespassers, employees, other transit workers (contractors), and others. A transit fatality is a death within 30 days after the incident, which occurs under the collision, derailment, personal casualty (not otherwise classified), fire, or bus going off the road categories of National Transit Database (NTD) reporting. Previous to 2002, transit involved parties were defined as patrons, employees, and others (the safety data was collected on a fiscal as opposed calendar year basis). Fatalities for the performance measurement use only transit agency Directly Operated (DO) mode data. Purchased Transportation (PT) data is not part of this measure. Certain fatalities are excluded as they are not considered to be directly related to the operation of transit vehicles. Those include suicides and fatalities occurring in parking facilities and stations, as well as fires in right-of-ways and stations. Also, the measure includes only the major transit modes (motor/trolleybus, light rail, heavy rail, commuter rail with vanpool, automated guideway, and demand response) and excludes ferryboat, monorail, inclined plane, cable car, and jitney. The passenger-miles traveled on public transit vehicles (buses, heavy and light railcars, commuter railcars, ferries, paratransit vans, vanpools, etc.) only refer to miles while in actual revenue service to the general public. These data are reported annually by operators to the FTA National Transit Database (NTD) and to the Federal Railroad Administration’s (FRA) Rail Accident and Incident Reporting System (RAIRS). FRA RAIRS data is used exclusively for commuter rail (CR) safety data. NTD and RAIRS data are an input to FTA’s Transit Safety & Security Statistics & Analysis program (formerly known as Safety Management Information Statistics [SAMIS]). |
| Sources: |
The Transit Safety & Security Statistics & Analysis Annual Report, formerly SAMIS, is a compilation and analysis of transit accident, casualty, and crime statistics reported under the Federal Transit Administration’s (FTA’s) NTD Reporting System by transit systems that are beneficiaries of FTA Urbanized Area Formula funds. Starting in 2002, Commuter Rail safety data are being collected from the FRA Rail Accident Reporting System (RAIRS) in order to avoid redundant reporting to NTD. Transit fatalities: Transit Safety & Security Statistics & Analysis Annual Report. Transit passenger miles: Transit Safety & Security Statistics & Analysis Annual Report. |
| Statistical Issues: |
The fatality counts in FTA’s Transit Safety & Security Statistics & Analysis are a census. The major source of uncertainty in the measure relates to passenger-miles traveled. Passenger-miles are an estimate derived from reported passenger trips and average trip length. Passenger-miles are the cumulative sum of the distances ridden on passenger trips. Transit authorities have accurate counts of unlinked passenger trips and fares. An unlinked trip is recorded each time a passenger boards a transit vehicle, even though the rider may be on the same journey. Transit authorities do not routinely record trip length. To calculate passenger-miles, total unlinked trips are multiplied by average trip length. To obtain an average trip length for their bus routes and rail routes, transit authorities use a FTA-approved sampling technique. Passenger-miles are the only data element that is sampled in the NTD. Validation based on annual trend analysis is performed on the passenger mile inputs from the transit industry. The validation is performed by statistical analysts at the NTD contractor (Veridian/General Dynamics Corp.). |
| Completeness: |
The information for this measure comes from the FTA’s Transit Safety & Security Statistics & Analysis program, formerly FTA’s Safety Management Information System (SAMIS), which uses data reported by transit operators to the NTD. Many categories and definitions were added or changed in the new NTD in 2002, and have allowed for improvements and more timely analysis of trends and contributing factors. The 2005 measure is an extrapolation of partial-year data, particularly of passenger-miles traveled. |
| Reliability: |
An independent auditor and the transit agency’s CEO certify that data reported to the NTD are accurate. Using data from the NTD to compile the Transit Safety & Security Statistics & Analysis program (formerly SAMIS) data, the USDOT Volpe National Transportation Systems Center compares current safety statistics with previous years, identifies questionable trends, and seeks explanation from operators. |
Natural Gas and Hazardous Liquid Pipeline Incidents
| Measure: |
Number of incidents for natural gas and hazardous liquid pipelines. (CY) |
|---|---|
| Scope: |
This measure is based on reported hazardous liquid and natural gas accidents that meet Federal reporting criteria as defined in 49 CFR 191.1 and 191.15 for natural gas pipeline incidents and in 49 CFR 195.50 for hazardous liquid pipelines. |
| Source: |
Office of Pipeline Safety (OPS) Incident and Accident Reports |
| Statistical Issues: |
Reports are required to be submitted by the responsible operators within 30 days of an incident or face penalties for non-compliance. OPS routinely cross-checks incident/accident reports against other sources of data, such as the telephonic reporting system for incidents requiring immediate notification provided to the National Response Center (NRC). Compliance is very high and most incidents that meet reporting requirements are submitted. OPS is developing a Best Management Practice to ensure quality of the incident data. A response percentage cannot be calculated as the actual population of reportable incidents cannot be precisely determined. |
| Completeness: |
The reported estimates are based upon incident data reported in January through June 2005. In reporting pipeline safety, there is both a safety and environmental measure. There may be a 60-day lag in reporting and compiling information in the database for analysis. Operators have 30 days to report incidents. Traditionally, there are more incidents in the summer than the winter. Preliminary estimates are based on data available as of middle of August, with 6 months of data through the end of June. The CY 2005 estimate is based on a straight line extrapolation of that data (i.e., multiplying the cumulative incidents reported through June by 12/6 or doubling the number available from January through June). Projection of the environmental measure is less precise due to the nature of pipeline spills. A single large spill (10,000 barrels or more) can easily dwarf the total for all other CY spills combined. These large spills cannot be factored into a projection model due to their magnitude and infrequent and unpredictable occurrences. Thus, projections for the remaining six months of this CY assume that the average spill volume in the past six months will remain the same in the next six months. However, any large spill of non-highly volatile hazardous liquid in the next six months can move the projection upwards. |
| Reliability: |
PHMSA uses these data in prioritizing its inspections and safety reviews, and for more long-term strategic management of its pipeline safety program. |
Serious Hazardous Materials Incidents
| Measure: |
Number of serious hazardous materials transportation incidents. (CY) |
|---|---|
| Scope: |
Serious reported hazardous materials incidents were initially defined by PHMSA to be those that result in a fatality or major injury (for most purposes, an injury resulting in hospitalization) due to a hazardous material, closure of a major transportation artery or facility, or evacuation of six or more persons due to the presence of a hazardous material, or a vehicle accident or derailment resulting in the release of a hazardous material. The definition includes those incidents resulting in a fatality or major injury, the evacuation of 25 or more employees or responders or any number of the general public, the closure of a major transportation artery, the alteration of an aircraft flight plan or operation caused by the release of a hazardous material, or the exposure of hazardous material to fire; plus any release of radioactive materials from Type B packaging, Risk Group 3 or 4 infectious substance, over 11.9 gallons or 88.2 pounds of a severe marine pollutant, or a bulk quantity (over 119 gallons or 882 pounds) of a hazardous material. This measure tracks only transportation-related releases of hazardous materials that are in commerce. Volume of spills is not tracked, as this does not necessarily indicate risk. |
| Sources: |
Hazardous Material Information System (HMIS)—Office of Hazardous Materials Safety (OHM) on Form DOT F 5800.1. |
| Statistical Issues: |
Data are collected by the carrier involved in each reportable incident and submitted to PHMSA’s Office of Hazardous Materials Safety (OHM) on Form DOT F 5800.1. Carriers are required by regulation to report incidents and face significant penalties for failing to do so. Incident reports are received continuously by OHM. Carriers are required to submit incident reports to PHMSA within 30 days of an incident. Once received by OHM, it takes approximately one month for incident reports to be processed and verified. The data are then made available in the HMIS database during the next monthly update. Although the number of incidents may be underreported, such recording error is probably small in comparison to the annual variation due to chance. |
| Completeness: |
PHMSA continues to receive reports from calendar year 2005. By the end of September 2005 actual incident data was received through August 31, 2005. PHMSA is projecting the remainder of the calendar year using the actual number of incidents that occurred during September, October, November, and December of 2004—the previous calendar year. This methodology for projecting the CY 2005 estimate is expected to be within 2-4% of the final estimate which becomes available during the second quarter of CY 2006. |
| Reliability: |
Annual hazmat incident data are used to track program performance, plan regulatory and outreach initiatives, and provide a statistical basis for research and analysis. The data is also used on a daily basis to target entities for enforcement efforts, and review of applications for exemption renewals. |
Details on DOT Mobility Measures
Highway Infrastructure Condition
| Measure: |
Percentage of travel on the National Highway System (NHS) meeting pavement performance standards for “good” rated ride. (CY) |
|---|---|
| Scope: |
Data include vehicle-miles traveled on the Highway Performance Monitoring System (HPMS) reported NHS sections and pavement ride quality data reported using the International Roughness Index (IRI). IRI is a quantitative measure of the accumulated response of a quarter-car vehicle suspension experienced while traveling over a pavement. Vehicle-Miles of Travel (VMT) represents the total number of vehicle-miles traveled by motor vehicles on public roadways within the 50 States, Washington, D.C., and Puerto Rico. |
| Source: |
Data for this measure are collected by the State Highway Agencies using calibrated measurement devices that meet industry set standards and reported to FHWA. Measurement procedures are included in the FHWA HPMS Field Manual. The VMT data are derived from the HPMS. |
| Statistical Issues: |
The major source of error in the percentages is the differences in data collection methodologies between the States and the differences in data collection intervals. FHWA is working on revisions to the HPMS data collection guidelines to minimize these potential errors. VMT data are also subject to sampling errors. The magnitude of error depends on how well the sites of the continuous counting stations represent nationwide traffic rates. HPMS is also subject to estimation differences between the States, even though FHWA works to minimize such differences and differing projections on growth, population, and economic conditions that impact driving behavior. |
| Completeness: |
Data up to 2003 are final estimates. The 2004 measure is not available, as States report highway performance data to FHWA as late as October in the following calendar year. FY 2004 data are not complete. Even with complete FY 2004 data, projections must be made for FY 2005. |
| Reliability: |
The HPMS data are collected by the 50 States, the District of Columbia, and Puerto Rico in cooperation with local governments. While many of the geometric data items, such as type of median, rarely change; other items, such as traffic volume, change yearly. Typically, the States maintain data inventories that are the repositories of a wide variety of data. The HPMS data items are simply extracted from these inventories, although some data are collected just to meet Agency requirements. The FHWA provides guidelines for data collection in the HPMS Field Manual. Adherence to these guidelines varies by State, depending on issues such as staff, resources, internal policies, and uses of the data at the data provider level. An annual review of reported data is conducted by the FHWA, both at the headquarters level and in the Division Offices in each State. The reported data are subjected to intense editing and comparison with previously reported data and reasonability checks. A written annual evaluation is provided to each State to document potential problems and to encourage corrective actions. Data re-submittal is requested in cases where major problems are identified. |
Highway Congestion
| Measure: |
Percent of total annual urban-area travel occurring in congested conditions. (CY) |
|---|---|
| Scope: |
Data are derived from approximately 400 urban areas. The data reflects travel conditions on freeway and principal arterial street networks. Definitions:
|
| Source: |
Data collected and provided by the State Departments of Transportation from existing State or local government databases, including those of Metropolitan Planning Organizations. FHWA’s Highway Performance Monitoring System (HPMS) serves as the repository of the data. The Texas Transportation Institute utilizes HPMS data to derive the above measures. |
| Statistical Issues: |
The methodology used to calculate performance measures has been developed by the Texas Transportation Institute (TTI) and reported in their annual Mobility Study. A detailed description the of TTI’s methodology is available at http://mobility.tamu.edu/ums/report/methodology_appB.pdf. |
| Completeness: |
The 2003 and prior measures are final. The 2004 measure is preliminary, as partial 2004 HPMS data were used to construct the estimates. HPMS data is compiled from the States and verified approximately 10 months from the base year, e.g., 2005 actual numbers will not be available from HPMS until October 2006. The 2005 measure is a projection based on recent year trends. |
| Reliability: |
The HPMS data are collected by the 50 States, the District of Columbia, and Puerto Rico in cooperation with local governments. While many of the geometric data items, such as type of median, rarely change; other items, such as traffic volume, change yearly. Typically, the States maintain data inventories that are the repositories of a wide variety of data. The HPMS data items are simply extracted from these inventories, although some data are collected just to meet Agency requirements. The FHWA provides guidelines for data collection in the HPMS Field Manual. Adherence to these guidelines varies by State, depending on issues such as staff, resources, internal policies, and uses of the data at the data provider level. An annual review of reported data is conducted by the FHWA, both at the headquarters level and in the Division Offices in each State. The reported data are subjected to intense editing and comparison with previously-reported data and reasonability checks. A written annual evaluation is provided to each State to document potential problems and to encourage corrective actions. Data re-submittal is requested in cases where major problems are identified. |
Transit Ridership
| Measure: |
Average percent change in transit boardings per transit market (150 largest transit agencies), adjusted for changes in employment levels. (CY) |
|---|---|
| Scope: |
The metric is average percent change in transit boardings adjusted for employment levels. The components are transit passenger boardings and employment levels within a transit market. The modes covered are: Motor Bus (MB), Heavy Rail (HR), Light Rail (LR), Commuter Rail (CR), Demand Response (DR), Vanpool (VP), and Automated Guideway (AG). Employment data are collected and reported by the Bureau of Labor Statistics. |
| Sources: |
Transit Passengers: Data derived from counts made on bus and rail routes by transit agencies that are beneficiaries of FTA Urbanized Area Formula funds as part of their monthly National Transit Database (NTD) Reporting System submissions. Data is collected from the 150 largest transit systems. Employment: Bureau of Labor Statistics Current Employment Statistics (CES) Survey. |
| Statistical Issues: |
The sources of uncertainty include coverage errors and auditing issues. These data are validated by the FTA Office of Oversight’s NTD contractor staff. By statute, every FTA formula grant recipient in an urbanized area (defined by the Census as having a population of 50,000 or more) must report to the National Transit Database (NTD). In cities of this size, virtually every transit authority receives FTA funding, and there are only a few cities with over 50,000 persons that do not provide public transit service. Publicly-funded transit service can be directly-operated or purchased transportation. Transit authorities have accurate counts of unlinked passenger trips and fares. An unlinked trip is recorded each time a passenger boards a transit vehicle, even though the rider may be on the same journey. The sources of uncertainty include coverage errors and auditing issues. Until 2002, reports were required only on an annual basis. Beginning in 2002, monthly NTD reports were required of the largest 150 transit operators on certain safety, service level, and service utilization statistics. In 2003 and part of 2004, due to lack of NTD funding, there were many months without contract support to perform monthly data collection. With contract support, by the end of 2005, almost all transit agencies are reporting on a monthly basis. However, the 150 is dynamic, not static. Because much of transit is contracted out, or purchased transportation, there are often reporting gaps in the top 150 when contracts are lost or contractors go out of business. For example, in New York City, the top six private bus contractors went out of business in 2005. Another common example is Dallas Area Rapid Transit (DART). DART dropped their large bus contract and provided the service in-house. The monthly data is being reported, but it looks like one reporter didn’t report. Further, in the past, for the sake of an accurate comparison between years, the FTA procedure has been to eliminate the entire agency when only one mode has a reporting gap for a single month. That means, if one of Boston’s vanpool contractors did not report one month, FTA would eliminate all of Boston’s data entirely. Boston has heavy rail, light rail, bus, ferry, demand response, etc. In 2006, we will only drop the mode not reported. All these factors overstated the number of agencies that did not report. Through June 2005 data had been reported on a monthly basis for all modes for the full year of 2004 and 2005 by 123 of the largest 150 transit operators. Operators reporting data represent 88 % of nationwide transit utilization; all 150 operators represent 94 % of nationwide transit utilization. Employment data are reported by Bureau of Labor Statistics. The Current Employment Statistics (CES) Survey is a monthly survey of business establishments that provides estimates of employment, hours, and earnings data by industry for the Nation as a whole, all States, and most major metropolitan areas. The CES survey is a Federal-State cooperative endeavor in which State employment security agencies prepare the data using concepts, definitions, and technical procedures prescribed by the Bureau of Labor Statistics. All estimates from a sample survey are subject to sampling and other types of errors. Survey data are also subject to nonsampling errors, such as those that can be introduced into the data collection and processing operations. Estimates not directly derived from sample surveys are subject to additional errors resulting from the special estimation processes used. |
| Completeness: |
DOT has revised this measure to better account for the impact of economic conditions on transit use by adjusting for changes in the level of employment in each urbanized area and to improve timeliness. An increase in average transit ridership per market, adjusted for changes in employment, represents an increase in transit’s share of the personal travel market. In order to improve the timeliness of the data reported, and to make the period being reported more comparable across areas, in the future, the measure will utilize data on transit unlinked passenger trips (used as a surrogate for passenger-miles) from the new monthly National Transit Database that was initiated in 2002. This data is to be available for the largest 150 transit operators, which account for about 94 % of all transit ridership. Thus, for 2005, the indicator will compare transit ridership for the urbanized areas containing the 150 largest transit agencies (normalized for employment levels) for the year ending in September 2005 with the year ending in September 2004. Data on employment is based on monthly employment levels for metropolitan statistical areas reported by the Bureau of Labor Statistics. |
| Reliability: |
An independent auditor and the transit agency’s CEO certify that data reported to the NTD are accurate. FTA also compares data to key indicators such as vehicle revenue-miles, number of buses in service during peak periods, etc. FTA has undertaken a major initiative to increase ridership nationwide. This measure has been built into all FTA senior executive performance accountabilities. |
Transportation Accessibility
| Measure: |
1. Percentage of bus fleets that are compliant with the Americans with Disabilities Act (ADA). (CY) 2. Percent of key rail stations that are compliant with the Americans with Disabilities Act (ADA). (CY) |
|---|---|
| Scope: |
Accessibility for bus fleet means that vehicles are equipped with wheelchair lifts or ramps. Transit buses are buses used in urbanized areas to provide public transit service to the general public. Transit buses do not include private intercity buses (e.g., Greyhound), private shuttle buses, charter buses, or school buses. The percentage of bus fleets that are equipped with lifts or ramps is only a partial measure of overall accessibility under the ADA as it measures only the availability of transit buses in our National fleet that can accommodate wheelchairs through the use of mechanical lifts or ramps. Accessibility for transit vehicles under the ADA includes other equipment and operational practices that are not reflected in this indicator. Accessibility for key rail facilities is determined by standards for ADA compliance. Transit systems were required to identify key stations. A key station is one designated as such by public entities that operate existing commuter, light, or rapid rail systems. Each public entity has determined which stations on its system have been designated as key stations through its planning and public participation process using criteria established by DOT regulations. All new rail stations are required to be ADA compliant upon completion and must meet standards for new rail stations, not key stations. All altered stations are required to be ADA compliant upon completion and must meet standards for alterations of transportation facilities by public entities |
| Sources: |
Compliant bus fleets: National Transportation Database (NTD). Compliant rail stations: Rail Station status reports to the FTA. |
| Statistical Issues: |
Data is obtained from a census of publicly-funded transit buses in urbanized areas. Information on the ADA key rail stations is reported to FTA by transit authorities. These data are not based on a sample. |
| Completeness: |
At a transit authority, vehicle purchases are significant capital expenditures. Vehicles purchased with FTA funds must have a useful life of 12 years. Whether a bus is purchased or leased, the equipment on the bus is recorded, including lifts and ramps. For the last 20 years, transit agencies have reported on the equipment in their bus fleets to the FTA in their annual NTD submissions. There is a census of publicly-funded transit buses in urbanized areas. It is not a sample. Urbanized areas have more than 50,000 persons, and are defined by the Census Department. By statute, every FTA formula grant recipient in an urbanized area must report to the NTD. In cities of this size, virtually every transit authority receives FTA funding. There are only a few cities of over 50,000 persons that do not provide public transit service. Publicly funded transit service can be directly operated or purchased transportation. |
| Reliability: |
All data in the NTD is self-reported by the transit industry. The transit agency’s Chief Executive Officer and an independent auditor for the transit agency certify the accuracy of this self-reported data. The data is also compared with fleet data reported in previous years and cross-checked with other related operating and financial data in the report. Fleet inventory is also reviewed as part of FTA’s Triennial Review, and a visual inspection is made at that time. Information on ADA key rail stations is reported to FTA by transit authorities. The FTA’s Office of Civil Rights conducts oversight assessments to verify the information on key rail station accessibility. Quarterly rail station status reports, and key rail station assessments have significantly increased the number of key rail stations that have come into compliance over the last several years. FTA will primarily influence the goal through Federal transit infrastructure investment, which speeds the rate at which transit operators can transition to ADA-compliant facilities and equipment, oversight, and technical assistance. |
Access to Jobs
| Measure: |
Number of employment sites (in thousands) that are made accessible by Job Access and Reverse Commute (JARC) transportation services. (FY) |
|---|---|
| Scope: |
This measure assesses one part of the JARC program—the numbers of employment sites made accessible that were not previously accessible. The new employment sites represented new sites connected geographically by the new service or new employment sites reached during time periods not previously covered (late night and weekend service). An employment site is a new stop reaching employers not previously reached either directly by demandresponsive services or that are within ¼ mile of the new service stop for fixedroute service. Services that make an employment site accessible may include, but are not limited to, carpools, vanpools, and other demand-responsive services as well as traditional bus and rail public transit. This measure does not account for those JA RC activities that encourage riders to use already existing sources of public transit. |
| Source: |
FTA Grantees |
| Statistical Issues: |
In previous years, FTA has had difficulty in getting complete information from its grantees. Changes resulting from a FTA analysis of this issue have improved grantee reporting compliance to 80 % of those JARC grantees expected to report. |
| Completeness: |
JARC grantees are requested to report the new employment sites reached by the transportation services initiated under their grant. Approximately 80 % of the JARC grantees have reported this data for FY 2004 and similar or better results are expected for FY 2005. FTA projects these results to estimate the total new employment sites reached by all grantees. The calculation methodology is based on the expenditures of selected grantees when compared to the total expenditures of all grantees during the same two-fiscal-year period. In subsequent years, FTA further proposes to supplement this approach by simplifying the data-reporting process, developing profiles of all grantees, and conducting on-site surveys to collect qualitative information about program performance from selected grantees. The preliminary methodology for projecting the number of employment sites reached in FY 2005 has two elements. Phase I will use existing data collected for FY 2004 to project employment sites reached, based on expenditure level for FY 2005. Phase 2 will involve projections based on actual FY 2004 and FY 2005 cumulative data that will be available in early 2006. Phase 2 involves the collection of 2005 data collected from grantees. If data collected is incomplete, then projections will be made for grantees not reporting, based on data collected in FY 2004 / FY 2005. |
| Reliability: |
Oversight contractors review the data and contact grantees to ascertain methodologies on a sample basis, or when the information warrants review. |
Aviation Delay
| Measure: |
Percent of all flights arriving within 15 minutes of schedule at the 35 Operational Evolution Plan airports due to NAS-related delays (FY) |
|---|---|
| Scope: |
NAS On-Time Arrival is the percentage of all flights arriving at the 35 OEP (Operational Evolution Plan) airports equal to or less than 15 minutes late, based on the carrier flight plan filed with the FAA, and excluding minutes of delay attributed by air carriers to weather, carrier action, security delay, and prorated minutes for late arriving flights at the departure airport. The adjusted sum of flights arriving on or before 15 minutes of flight plan arrival time is divided by the total number of completed flights. A flight is considered on-time if it arrives no later than 15 minutes after it’s published, scheduled arrival time. This definition is used in both the DOT Airline Service Quality Performance (ASQP), and Aviation System Performance Metrics (ASPM) reporting systems. Air carriers, however, also file up-to-date flight plans for their services with the FAA that may differ from their published flight schedules. This metric measures on-time performance against the carriers filed flight plan, rather than what may be a dated published schedule. The time of arrival of completed passenger flights to and from the OEP 35 airports is compared to their flight plan scheduled time of arrival. For delayed flights, delay minutes attributable to extreme weather, carrier caused delay, security delay, and a prorated share of delay minutes due to a late arriving flight at the departure airport are subtracted from the total minutes of delay. If the flight is still delayed, that delay is attributed to the National Aviation System (NAS) and the FAA, and counted as a delayed flight. |
| Sources: |
The ASPM database, maintained by the FAA’s Office of Aviation Policy and Plans, and supplemented by DOT’s ASQP causation data, contains the data for on-time arrivals. By agreement with the FAA, ASPM flight data is filed by certain major air carriers for all flights to and from most large and medium hubs, and is supplemented by flight records contained in the Enhanced Traffic Management System (ETMS) and flight movement times provided by Aeronautical Radio, Inc. (AIRINC). Data are sufficient to complete ASPM data files for 55 airports. The 35 OEP airports are a sub-set of these 55 airports. |
| Statistical Issues: |
None, all flight data to/from the 35 OEP airports are reported. |
| Completeness: |
The FY 2005 data will not be finalized until about 90 days after the close of the fiscal year; essentially the start of the next calendar year. |
| Reliability: |
Flight plan data is extracted from carrier records supplied under ASPM, which contains flight data. Summary data are compared and supplemented with data filed monthly with DOT under 14 CFR Part 234, Airline Service Quality Performance Reports, which separately requires reporting by major air carriers on flights to and from all large hubs. |
Details on DOT Global Connectivity Measures
Disadvantaged and Women-Owned Small Businesses
| Measure: |
1. Percent share of the total dollar value of DOT direct contracts that are awarded to women-owned businesses. (FY) 2. Percent share of the total dollar value of DOT direct contracts that are awarded to small disadvantaged businesses. (FY) |
|---|---|
| Scope: |
Includes contracts awarded by DOT Operating Administrations through direct procurement. It does not include FAA contracts exempt from the Small Business Act. |
| Sources: |
Prior to October 1, 2003, these data are derived from the USDOT Contract Information System (CIS, which fed the old Federal Procurement Data System (FPDS). The CIS included all USDOT contracting activities that reported to the Federal Procurement Data Center (FPDC). Migration to the new Federal Procurement Data System on October 1, 2003 enabled the removal of agency FPDS feeder systems government-wide (including CIS). New data reports will come directly from FPDS. Data are compiled by USDOT Contracting staff from Department contract documents. Selected information is either transmitted from the operating administration contract writing systems, or manually data-keyed via the FPDS web site, into the FPDS database, which can be queried to compute needed statistics. All USDOT contracts are enumerated. |
| Statistical Issues: |
Until recently the reliability of the Federal Procurement Data System/Next Generation (FPDS/NG) was an issue with DOT and other federal agencies including the General Accountability Office (GAO). The FPDS is designed to be an accurate and reliable system, as required by the Small Business Act, Section 644(g). However, it is recognized that at least through the transitional periods of FY 2003 through FY 2006, there may be issues of synchronization and data reliability between federal agencies and the FPDS/NG. DOT currently is required to scrub FPDS/NG data and resubmit it for validation. After reverifying these data against internal sources, there are no known major errors present in the data. Business types are as identified in the Central Contractor Registration (CCR) database. However, random variation in the number of DOT contracts as well as the number of women-owned and smalldisadvantaged businesses each year results in some random variation in these measures from year to year. |
| Completeness: |
The Federal Procurement Data System (FPDS) is prescribed by regulations as the official data collection mechanism for DOT acquisitions. |
| Reliability: |
There is extensive regulatory coverage to ensure data reliability. The system is used to prepare many reports to Congress, the Small Business Administration, and others. Performance goals actual data, as finalized by the Small Business Administration is the only reliable basis for program evaluations as mandated by the Small Business Act, Section 644(g). |
St. Lawrence Seaway System Availability
| Measure: |
Percent of days in the shipping season that the U.S. portion of the St. Lawrence Seaway is available. (FY) |
|---|---|
| Scope: |
The availability and reliability of the U.S. sectors of the St. Lawrence Seaway, including the two U.S. Seaway locks in Massena, N.Y., are critical to continuous commercial shipping during the navigation season (late March to late December). System downtime due to any condition (weather, vessel incidents, malfunctioning equipment) causes delays to shipping, affecting international trade to and from the Great Lakes region of North America. Downtime is measured in hours/minutes of delay for weather (visibility, fog, snow, ice); vessel incidents (human error, electrical and/or mechanical failure); water level and rate of flow regulation; and lock equipment malfunction. |
| Sources: |
Saint Lawrence Seaway Development Corporation (SLSDC) Office of Lock Operations and Marine Services |
| Statistical Issues: |
None. |
| Completeness: |
As the agency responsible for the operation and maintenance of the U.S. portion of the St. Lawrence Seaway, SLSDC’s lock operations unit gathers primary data for all vessel transits through the U.S. Seaway sectors and locks, including any downtime in operations. Data is collected on site, at the U.S. locks, as vessels are transiting or as operations are suspended. This information measuring the System’s reliability is compiled and delivered to SLSDC senior staff and stakeholders each month. In addition, SLSDC compiles annual System availability data for comparison purposes. Since SLSDC gathers data directly from observation, there are no limitations. Historically, the SLSDC has reported this performance metric for its entire navigation season (late March to late December). Unfortunately due to reporting timelines, system availability data is only reported through September in this report. |
| Reliability: |
SLSDC verifies and validates the accuracy of the data through review of 24-hour vessel traffic control computer records, radio communication between the two Seaway entities and vessel operators, and video and audiotapes of vessel incidents. |
Bilateral Agreements
| Measure: |
Number of Bilateral and Multilateral agreements completed. The Bilateral Aviation Safety Agreement (BASA) is made up of two parts: (1) an executive agreement signed by the Department of State and Ministry of Foreign Affairs, and (2) one or more implementation procedures signed by the FAA and the other civil aviation authority. The measure is the number of agreements signed with foreign governments. |
|---|---|
| Scope: |
Global. The purpose of a BASA is to promote aviation safety and environmental quality; and to enhance cooperation and increase efficiency in matters related to civil aviation. By building a network of competent civil aviation authorities and concluding agreements with additional countries and/or regional authorities, the FAA can increase safety globally. Improved global understanding of U.S. safety regulations, processes, and procedures leads to better international regulatory oversight. Since the BASA is based on recognition of comparability of the U.S. and foreign systems for approval and surveillance of aviation industry, the BASA allows the FAA to rely upon capabilities and technical expertise of other civil aviation authorities in particular areas of aviation safety. This minimizes duplication of efforts and opens new lines of communication between authorities. FAA can then better focus on U.S. safety priorities and rely on competent civil aviation authorities for those activities taking place overseas. |
| Sources: |
FAA. |
| Statistical Issues: |
None. |
| Completeness: |
Signed BASA. This is achieved when an executive agreement and at least one implementation procedures is concluded with a given country or regional authority. |
| Reliability: |
None. |
Reduced Barriers to Trade in Transportation
| Measure: |
The number of potential air transportation consumers (in billions) in international markets traveling between the U.S. and countries with open skies and open transborder aviation agreements. |
|---|---|
| Scope: |
The number of potential air transportation consumers is the total population of the U.S. and countries with open skies aviation agreements with the U.S. By the end of FY 2005, there were 69 open skies agreements. This measurement includes the annual increase in population for the countries where open skies have been achieved, as well as the additional populations for newly negotiated open skies agreements. The estimate for the additional population is based on the median population size of the countries without open skies agreements. The measurement thus reflects the extent to which the liberalization resulting from open skies agreements, negotiated by DOT, increases travel opportunities between the U.S. and countries with previously restricted aviation agreements. |
| Source: |
Estimate of the population of the U.S. and countries with open skies agreements with the U.S., Midyear Population, international Data Base, U.S. Bureau of the Census (per website). |
| Statistical Issues: |
The International Data Base of the U.S. Bureau of the Census is a reliable source of population estimates. The Bureau’s website and publications provide qualifying data notes that more fully describe technical and other issues. These qualifying notes do not significantly affect our analyses. |
| Completeness: |
The International Data Base of the U.S. Bureau of the Census is a reliable source of population estimates. The Bureau’s website and publications provide qualifying data notes that more fully describe technical and other issues. These qualifying notes do not significantly affect our analyses. |
| Reliability: |
The International Data Base of the U.S. Bureau of the Census is a reliable source of population estimates. The Bureau’s website and publications provide qualifying data notes that more fully describe technical and other issues. These qualifying notes do not significantly affect our analyses. |
Enhanced International Competitiveness of U.S. Transport Providers
| Measure: |
The number of international negotiations conducted annually to remove market-distorting barriers to trade in transportation. |
|---|---|
| Scope: |
The number of international negotiations conducted annually to remove market-distorting barriers to trade in transportation is the number (or rounds) of meetings and negotiations that are conducted in an effort to reach open skies agreements, other liberalized aviation agreements, or to resolve problems. By the end of FY 2005, there were 69 open skies agreements, an open transborder agreement with Canada and 19 liberalized (but not open skies) agreements. These numbers, however, do not represent, but understate, the number of negotiating sessions that have historically been held to complete these agreements. The measurement thus reflects an estimate of the extent of and manner by which the DOT might best apply the necessary resources to open the competitive environment and provide increased travel opportunities and economic benefits. |
| Source: |
Estimate of the number of annual negotiating sessions that are required to achieve further international aviation liberalization. It is an internal estimate generated by the Office of the Assistant Secretary for Aviation and international Affairs based on a number of analytical, economic and geopolitical factors. |
| Statistical Issues: |
Due to geopolitical factors, the nature of international aviation negotiations can follow an unpredictable course. It is impossible to gauge or comment upon the data limitations, statistical issues, data completeness and data reliability. |
| Completeness: |
Due to geopolitical factors, the nature of international aviation negotiations can follow an unpredictable course. It is impossible to gauge or comment upon the data limitations, statistical issues, data completeness and data reliability. |
| Reliability: |
Due to geopolitical factors, the nature of international aviation negotiations can follow an unpredictable course. It is impossible to gauge or comment upon the data limitations, statistical issues, data completeness and data reliability. |
Details on DOT Security Measures
Strategic Mobility
| Measure: |
Percentage of DOD-required shipping capacity complete with crews available within mobilization timelines. (FY) |
|---|---|
| Scope: |
This measure is based on the material availability of over 50 ships in the Maritime Administration’s Ready Reserve Force (RRF) and over 120 ships enrolled in the Voluntary Intermodal Sealift Agreement (VISA) program, which includes 47 ships enrolled in the Maritime Security Program (MSP). A second factor pertinent to this measure is the availability of sufficient licensed and unlicensed mariners to operate the available ships. The performance measure represents the number of available ships (compared to the total number of ships in the RRF and VISA) that can be fully crewed within the established readiness timelines. While other Government (primarily Military Sealift Command) owned or controlled sealift type vessels are not included in this measure, they draw their crews from the same pool of mariners. Accordingly, the availability measure is adjusted to reflect expected requirements during the early stages of a military crisis. |
| Sources: |
Ship availability data is from MARAD’s Office of Sealift Support (MSP/VISA ships) and the Office of Ship Operations (RRF ships). Mariner availability data is compiled and measured based on data obtained from the U.S. Coast Guard Mariner Licensing and Documentation data, MARAD's Office of Sealift Support estimates of the size of the sailing workforce and their availability for duty during a mobilization, and Department of Defense requirements. |
| Statistical Issues: |
None. |
| Completeness: |
Data are complete. |
| Reliability: |
MARAD’s data is reasonably reliable and useful in managing its reserve fleet readiness program. |
DOD-Designated Port Facilities
| Measure: |
Percentage of DOD-designated commercial strategic ports for military use that are available for military use within DOD established readiness timelines. |
|---|---|
| Scope: |
The measure consists of the total number of DOD-designated commercial strategic ports for military use that are forecast as able to meet DOD-readiness requirements on 48-hour notice, expressed as a percentage of the total number of DOD-designated commercial strategic ports. Presently there are 15 DOD-designated commercial strategic ports. Port readiness is based on monthly forecasts submitted by the ports and semi-annual port readiness assessments by MARAD in cooperation with other NPRN partners. The MARAD/DOD semiannual port assessments provide data or other information on a variety of factors, including the following: the capabilities of channels, anchorages, berths, and pilots/tugboats to handle larger ships; rail access, rail restrictions, rail ramp offloading areas, and rail storage capacities; the availability of trained labor gangs and bosses; number and capabilities of available cranes; long-term leases and contracts for the port facility; distances from ports to key military installations; intermodal capabilities for handling containers; highway and rail access; number of port entry gates; available lighting for night operations; and number and capacity of covered storage areas and marshalling areas off the port. |
| Sources: |
Data consists of the responses received from representatives of the port authorities for those commercial ports designated by the Department of Defense as strategic ports. Forecasts are requested on a monthly basis. One hundred percent of the strategic ports provide availability forecasts. The MARAD Office of National Security Plans maintains continuing dialog with respondents. |
| Statistical Issues: |
None. |
| Completeness: |
Data are complete. |
| Reliability: |
MARAD’s data is reasonably reliable according to the Bureau of Transportation Statistics and useful in managing its port readiness program. |
Transportation Capability Assessment for Readiness
| Measure: |
Transportation Capability Assessment for Readiness Index Score |
|---|---|
| Scope: |
The Office of Intelligence, Security, and Emergency Response (OET) measures its performance in meeting the Homeland and National Security Performance goal to “prepare the nation’s transportation system for a rapid recovery from intentional harm and natural disasters” by assessing progress in six functional areas: (1) Crisis Management Center, (2) U. S. Disaster Response, (3) Training and Exercises, (4) Continuity of Operations (COOP), (5) Continuity of Government (COG), and (6) International Response. |
| Sources: |
This measure is based on a self-assessment score determined by OET. Each functional area is rated based on between 1 and 5 specific criteria. The criteria are: Function 1—Crisis Management Center (20 points) 1. Does the Secretary’s Crisis Management Center (CMC) have adequate resources, such as communications, technology, and fully ready technical staff? (10 points) 2. Have the CMC workers been trained and participated in at least two exercises per year? (10 points) Function 2—U. S. Disaster Response (20 points) 1. Do the Regional Emergency Transportation Coordinators (RETCO) and Regional Emergency Response Teams have the necessary time, skills and equipment to successfully carry out their natural disaster and WMD functions? (6 points) 2. Is there adequate secure communications with state and local government and the transportation community when dealing with WMD or national security crises? (5 points) 3. Has the National Response Plan (NRP) Transportation Annex been updated in the past 2 years? (3 points) 4. Within the past 2 years, have all ten regions updated their NRP Transportation Annexes? (3 points) 5. Have DOT and DOD sufficiently coordinated their transportation functions? (3 points) Function 3—Training and Exercises (20 points) 1. Have Regional Response Teams and key personnel from state and local government and industry participated in DOT sponsored training and exercises, and did the training and exercises include both natural disasters and national security crises? (20 points) Function 4—Continuity of Operations (COOP) (20 points) 1. Is DOT’s primary COOP site fully functional? (10 points) 2. Is the OST COOP plan updated at least once every two years? (3 points) 3. Have the Operating Administrations COOP Plans been updated in the last 2 years? (4 points) 4. Has there been at least one COOP exercise or activation for both OST as well as all DOT modes in the last 12 months? (3 points) Function 5—Continuity of Government (COG) (10 points) 1. Does DOT have a complete National Emergency Management Team (NEMT)? (5 points) 2. Have the NEMT team members received at least 1 training/exercise session during the year? (5 points) Function 6—International Response (10 points) 1. Has DOT, as a U.S. representative to NATO, participated in at least 4 key NATO meetings and 2 exercises annually? (8 points) 2. Has DOT sufficiently coordinated its international disaster role with the U.S. State Department and its Civil Reserve Air Fleet activities with the DOD? (2 points) |
| Statistical Issues: |
None. |
| Completeness: |
The measure is complete and reflects the combined score of all evaluation criteria. |
| Reliability: |
Scores are reliable to the extent that specific quantitative evaluation criteria are available for each of the questions used to rate the functions. |
Details on DOT Environmental Stewardship Measures
Wetland Protection and Recovery
| Measure: |
Ratio of wetlands replaced for every acre affected by Federal-aid Highway projects. (FY) |
|---|---|
| Scope: |
Measure includes acreage of wetlands associated with all Federal-aid highway projects funded during the fiscal year. To be included, wetland replacement (or investment in a wetland bank) must have begun. |
| Source: |
State DOTs input Federal-aid related wetland degradation and replacement data into either locally-developed wetland mitigation databases or the FHWA Wetlands Management Database. FHWA compiles and reports the final data. |
| Statistical Issues: |
The uniformity of the data is not guaranteed, since it is subject to interpretation by the State DOT. In particular, there is no uniform definition of what should be reported as acres mitigated. The FHWA has provided guidance to the States as to which mitigation activities are to be reported. |
| Completeness: |
Data are compiled by State DOTs using local sources. |
| Reliability: |
All Federal agencies including FHWA and other DOT modes must comply with National Environmental Policy Act (NEPA) and the Clean Water Act (specifically section 404(b)(1) of the CWA) regarding disruption of wetlands. These laws require agencies to identify project alternatives that would avoid or minimize impacts to wetlands as a first consideration. These alternatives are subjected to analysis under both NEPA and the Clean Water Act. Under the law, these alternatives must be chosen unless the project sponsors clearly demonstrate that they are not viable because they do not meet the project purpose and need, or will lead to other more significant environmental impacts. If, in compliance with the law, wetland disruption is unavoidable, FHWA then works to achieve this goal of wetland replacement. |
DOT Facility Cleanup
| Measure: |
Percentage of DOT facilities categorized as No Further Remedial Action under the Superfund Amendments and Reauthorization Act (SARA). (FY) |
|---|---|
| Scope: |
EPA maintains a Federal Facility Hazardous Waste docket which contains information regarding Federal facilities that manage hazardous wastes or from which hazardous substances have been or may be released. DOT facilities listed on the docket are discussed in the Annual SARA report sent to Congress each year. EPA regional offices make the determination to change facility status to NFRAPs on the docket. |
| Sources: |
EPA Federal Facility Hazardous Waste docket which is issued twice a year. |
| Statistical Issues: |
None. |
| Completeness: |
The primary criterion for NFRAP is a determination that the facility does not pose a significant threat to the public health or environment. Responsibility for these facilities may be with FAA, FHWA, or FRA. NFRAP decisions may be reversed if future information reveals that additional remedial actions are warranted. The OAs’ activities are controlled, to a degree, by interaction and decisions made by EPA Regional personnel. This measure is current and has no missing data. |
| Reliability: |
DOT uses this data to prioritize cleanup activities and attendant resource levels. However, there is insufficient time to complete remediation prior to the close of the FY for any sites added in the July report. |
Mobile Source Emissions
| Measure: |
12-month moving average number of area transportation emissions conformity lapses. (FY) |
|---|---|
| Scope: |
The transportation conformity process is intended to ensure that transportation plans, programs, and projects will not create new violations of the National Ambient Air Quality Standards (NAAQS), increase the frequency or severity of existing NAAQS violations, or delay the attainment of the NAAQS in designated non-attainment (or maintenance) areas. |
| Sources: |
The FHWA and FTA jointly make conformity determinations within air quality non-attainment and maintenance areas to ensure that Federal actions conform to the purpose of State Implementation Plans (SIP). With DOT concurrence, the EPA has issued regulations pertaining to the criteria and procedures for transportation conformity, which were revised based on stakeholder comment. |
| Statistical Issues: |
None. |
| Completeness: |
If conformity cannot be determined within certain time frames after amending the SIP, or if three years have passed since the last conformity determination, a conformity lapse is deemed to exist and no new non-exempt projects may advance until a new determination for the plan and Transportation Improvement Program (TIP) can be made. This affects transit as well as highway projects. During a conformity lapse, FHWA and FTA can only make approvals or grants for projects that are exempt from the conformity process (pursuant to Sections 93.126 and 93.127 of the conformity rule) such as a safety project, and transportation control measures (TCM) that are included in an approved SIP. Only those project phases that have received approval of the project agreement, and transit projects that have received a full funding grant agreement, or equivalent approvals, prior to the conformity lapse may proceed. This measure is current and has no missing data. |
| Reliability: |
There are no reliability issues. FHWA and FTA jointly make conformity determinations within air quality non-attainment and maintenance areas to ensure that Federal actions conform to the purpose of State Implementation Plans (SIPs). |
Hazardous Liquid Materials Spilled
| Measure: |
Tons of hazardous liquid materials spilled per million ton-miles shipped by pipelines. (CY) |
|---|---|
| Scope: |
This measure is based on reports each operator files using PHMSA F 7000-1 for an accident that meets the criteria in 49 CFR §195.50. An accident report is required for each failure in a pipeline system subject to this part in which there is a release of the hazardous liquid or carbon dioxide transported resulting in any of the following: (a) Explosion or fire not intentionally set by the operator. (b) Release of 5 gallons (19 liters) or more of hazardous liquid or carbon dioxide, except that no report is required for a release of less than 5 barrels (0.8 cubic meters) resulting from a pipeline maintenance activity if the release is: (1) Not otherwise reportable under this section; (2) Not one described in Sec. 195.52(a)(4); (3) Confined to company property or pipeline right-of-way; and (4) Cleaned up promptly; (c) Death of any person; (d) Personal injury necessitating hospitalization; (e) Estimated property damage, including cost of clean-up and recovery, value of lost product, and damage to the property of the operator or others, or both, exceeding $50,000. |
| Sources: |
Office of Pipeline Safety (OPS) Accident Report for Hazardous Liquid Pipeline Systems F 7000-1 |
| Statistical Issues: |
Spill data are collected by the pipeline operator involved in each reportable incident and submitted to PHMSA's Office of Pipeline Safety (OPS) on Form PHMSA F 7000-1. Pipeline operators are required by regulation to report incidents and face significant penalties for failing to do so. Pipeline operators are required to submit incident reports to DOT within 30 days of the incident. Any accident discovered by OPS to be reportable and for which an accident report was not submitted is referred to the Office of Pipeline Safety’s Enforcement, which ensures compliance with the reporting requirement. Post-1985 ton-mile data are calculated using a base figure reported in a 1982 USDOT study entitled Liquid Pipeline Director and then combined with data from the Association of Oil Pipe Lines and the Oil Pipeline Research Institute. Lack of additional information raises definitional and methodological uncertainties about the data’s reliability. Moreover, the three different information sources introduce data discontinuities, making time comparisons unreliable. (National Transportation System (NTS) 2002). The performance measure is a ratio of “Tons Net Loss” and “Ton-Miles Shipped.” Uncertainty in either the numerator or the denominator can have a large effect on the overall uncertainty. Some factors of possible variance in the numerator include: 1) a few large spills can make OPS miss this goal and 2) even when the total number of spills fluctuates, the net volume lost may increase. The denominator may fluctuate with the overall economy, i.e., the volume shipped increases with economic boom and decreases when the economy slows down. The environmental metric tracks a highly variable trend and OPS has noted in the past that the variability of this metric warrants close study. OPS has moved this year to "non-HVL (highly volatile liquids)" only for the environmental metric as HVLs evaporate on release and don't impact the environment in the usual way that other liquid petroleum products do. In previous years OPS had included HVLs in the metric and the variability was even more pronounced. The past long term pattern for the trend was to generally meet or miss the goal every other year as the actual performance bounced above and below the trend line regularly. OPS continues to lessen the overall standard deviation of the metric over time (the performance of the trend is getting statistically more sound over time). This measure also has continued a general downward trend even though it bounces above and below the trend line over time. |
| Completeness: |
The data for this measure fluctuate year to year. PHMSA is studying the spill data to determine the nature of this fluctuation and improve this measure. The 2005 measure is projected by extrapolating partial year (January-June) reported data. |
| Reliability: |
PHMSA uses this data in conjunction with pipeline safety data in prioritizing compliance and enforcement plans and in strategic management of the pipeline safety program. |
Aircraft Noise Exposure
| Measure: |
Percent reduction in number of people within the U.S. who are exposed to significant aircraft noise levels (Day/Night Average Sound Level (DNL) 65 decibels or more) from the three-year average for 2000 to 2002. |
|---|---|
| Scope: |
Residential population exposed to aircraft noise above Day-Night Sound Level of 65 decibels around U.S. airports. |
| Sources: |
A statistical modeling technique (Model for Assessing the Global Exposure of Noise because of Transport Airplanes (the MAGENTA model)) is applied using U.S. population data from the Department of Commerce, locally-developed traffic distribution (route and runway utilization), and aircraft distributions developed using the Enhanced Traffic Management System (ETMS) and current aircraft registration databases. The local traffic utilization data is available for the busiest U.S. airports in the form of studies developed for the FAA’s Integrated Noise Model (INM). For smaller airports, a generic statistical procedure was employed. |
| Statistical Issues: |
This measure is derived from model estimates that are subject to errors in model specification. |
| Completeness: |
No actual count is made of the number of people exposed to aircraft noise. Aircraft type and event level are current. However, some of the databases used to establish route and runway utilization were developed from 1990 to 1997, with many of them now over seven years old. Changes in airport layout including expansions may not be reflected. The benefits of Federally-funded mitigation, such as buyout, are accounted. FAA has replaced the actual number of people exposed to significant noise with the percent decrease in the number of people exposed, measured from the threeyear average for calendar year 2000-2002. Moving to the 3-year average stabilizes noise trends, which can fluctuate from year to year and are affected by unusual events such as the 9/11 attacks and the subsequent economic downturn. The FY 2000– FY 2002 base time periods includes these events and is the same 3year period used for the emissions goal. The move from actual numbers to percent helps avoid confusion over U.S. noise exposure trends caused by annual improvements to the noise exposure model. A major change to MAGENTA resulted in a significant improvement in the estimate of the number of people exposed to significant noise levels around US airports. Until now, the scope of the measure included scheduled commercial jet transport airplane traffic at major U.S. airports. With access to better operational data sources, the scope of the MAGENTA calculation has expanded to include unscheduled freight, general aviation, and military traffic. The expanded scope of operations results in an increase in the estimate of the number of people exposed to significant noise. The growth in the number of people exposed results from improvements in measurement, not a worsening in aviation noise trends. Planned improvements to MAGENTA will continue to increase the estimate of the number of people exposed to aircraft noise, giving the false impression that aircraft noise exposure is increasing. Changing the noise performance goal to an annual percent change in aircraft noise exposure will better show the trend in aircraft noise exposure. The change will also make the Government Performance Review Act (GPRA) goal consistent with the FAA Flight Plan goal. |
| Reliability: |
The Integrated Noise Model (the core of the MAGENTA model) has been validated with actual acoustic measurements at both airports and other environments such as areas under aircraft at altitude. External forecast data are from primary sources. The MAGENTA population exposure methodology has been thoroughly reviewed by an ICAO task group and was most recently validated for a sample of airport-specific cases. |
Details on DOT Organizational Excellence Measures
DOT Major System Acquisition Performance
| Measure: |
For major DOT systems, the percentage of cost, schedule, and performance goals established in acquisition project baselines that are met. (FY) |
|---|---|
| Scope: |
This performance measure encompasses acquisition management data for all of DOT’s major systems acquisition contracts, primarily in the FAA, but also from any office procuring a major system as defined in OMB Circular A-11, and DOT’s Capital Programming and Investment Control order. |
| Source: |
The data for acquisition programs comes from each DOT organization procuring major systems. |
| Statistical Issues: |
FAA: Performance is measured separately for schedule and cost goals. Schedule performance is measured by calculating the number of schedule milestones met divided by the total schedule milestones planned. Cost performance is measured by comparing the total F&E budget-at-completion amount established in the January FAA Capital Investment Plan (CIP) against the projected budget-at-completion amount published in the August CIP. Any program with a total variance of more than a 10% threshold would be considered not meeting the established fiscal year performance goal. |
| Completeness: |
This measure is current with no missing data. Each DOT organization maintains its own quality control checks for cost, schedule, and technical performance data of each major systems acquisition in accordance with OMB Circulars A-11, A109, and A-130, Federal Acquisition Regulations, and Departmental orders implementing those directives and regulations. |
| Reliability: |
Each DOT organization having major system acquisitions uses the data during periodic acquisition program reviews, for determining resource requests. It is also used during the annual budget preparation process, for reporting progress made in the President’s budget and for making key program management decisions. |
Major DOT Infrastructure Project Cost and Schedule Performance
| Measure: |
1. Achieve 95% of schedule milestones for major Federally-funded transportation infrastructure projects, or miss those milestones by less than 10%. (FY) 2. Achieve 95% of cost estimates for major Federally-funded infrastructure projects, or miss them by less than 10%. (FY) |
|---|---|
| Scope: |
Active FTA New Starts projects with Full Funding Grant Agreements larger than $1 billion; FHWA projects with a total cost of $1 billion or more, or projects approaching $1 billion with a high level of interest by the public, Congress, or the Administration; and FAA runway projects with a total cost of $1 billion or more. |
| Sources: |
FTA: FTA uses independent reviews and third-party assessments such as the Corps of Engineers and other oversight contractors to validate the accuracy of project budgets and schedules before grantees are awarded Full Funding Grant Agreements. Project/Financial Management Oversight contractors review project budgets on a monthly basis and FTA assesses projected total project costs against baseline cost estimates and schedules. FHWA: The percent cost estimates and scheduled milestones for a FHWA Major Project are measured from when the Initial Financial Plan (IFP) is prepared and approved to the required Annual Project Update. The update contains the latest information about the cost and schedule for each of the Major Projects. Division Office Project Oversight Managers provide monthly status reports as a supplement to the Annual Update. FAA: Project cost performance for each major project is measured from cost estimates submitted by the airport sponsor to support its letter of intent (LOI) and actual expenditure data from FAA data sources (for grants) and airport sponsor submissions (for overall project cost). Project schedule performance is measured from the Runway Template Action Plan (RTAP), as specified in the National Airspace System Operational Evolution Plan. |
| Statistical Issues: |
FTA: Scheduled milestone achievement is measured by the difference between the actual Revenue Operations Date and the date of the execution of the Full Funding Grant Agreement divided by the difference between the Revenue Operations Date in the Full Funding Grant Agreement and the date of execution of the Full Funding Grant Agreement. Cost estimate achievement is measured by the actual Total Project Cost divided by the Total Project Cost in the Full Funding Grant Agreement. FHWA: A scheduled milestone is defined as being achieved upon completion of the project. Major Projects generally require 6-10 years from an IFP to completion. Cost estimates are prepared by comparing the costs in the most recent Annual Update to the IFP estimate. Because of the small number of Major Projects, FHWA may not meet its target if only a few projects show cost increases. FAA: Schedule completion performance is measured for two milestones—the project design and the project construction. A project milestone is considered to meet the performance target if actual cumulative rate of completion is not more than 10 % behind scheduled cumulative rate of completion, using the RTAP schedule as a base. For example, a 36-month schedule would allow a 3.6 month delay at any point in the schedule. In FY 2005, all of the three major runway projects met the performance target for completion. Cost performance now will be measured by comparing cumulative actual costs incurred at the end of each fiscal year with cumulative costs shown in the scheduled of costs submitted with the LOI application. A project will be considered to meet the cost performance target if cumulative costs are no more than 10 % higher than projected costs in the cost schedule. For the three current major projects, the baseline of scheduled costs is $3.4 billion. The source of this baseline cost is the most recent LOI amendment for each project, which reflects unanticipated cost overruns and project scope changes that are beyond the control of the airport sponsor. This includes costs directly or indirectly related to litigation, additional mitigation costs and material and supply cost increases due to contracting delays. Based on this measure, two of three major runway projects met the cost performance target in FY 2005. |
| Completeness: |
FTA: This measure is current with no missing data. The information is currently tracked with an in-house MS Excel database. A Web-based database, FASTTrak, is being developed to track this type of project information in the future. The measures are calculated monthly by an FTA Headquarters Engineer, checked by the Team Leader and reviewed by the Office Director. FHWA: The FHWA Major Projects Team maintains the project schedules and cost estimate information in a spreadsheet, which is updated when a Project IFP is approved and/or the Annual Update is received and accepted. The data is available and reported on a semi-annual basis. FAA: Federal financial commitments to airport sponsors are tracked by two automated systems, the System of Airports Reporting (SOAR) and the Delphi financial system. These systems are updated immediately when a grant payment is made or a grant is amended or closed-out. The FAA relies on the airport sponsor to report actual project costs on a quarterly basis. Project design and construction milestones (scheduled and actual) are contained in the RTAP and developed by all involved FAA lines of business, the airport sponsor and airlines. The RTAP is comprised of tasks that must be considered when commissioning the runway and assigns accountability to the airport, airline, and FAA allowing early identification and resolution of issues that might impact the runway schedule. |
| Reliability: |
FTA: Calculations of schedule achievement are based on month of this report, and not on projected Revenue Operations Date. Re-calculations of schedule and cost baselines are made to reflect amendments to the Full Funding Grant Agreements. FTA uses independent reviews and third-party assessments such as the Corps of Engineers and other oversight contractors to validate the accuracy of project budgets and schedules before grantees’ are awarded Full Funding Grant Agreements. FTA continues to work to improve its rigorous oversight program and has made project cost and budget performance a core accountability of every senior manager in the agency. FHWA: Both the IFP and the Annual Update undergo a rigorous review by the Division Office and the Major Projects Team prior to approval and acceptance. FAA: Reporting of Federal financial commitments to airport sponsors is done in accordance with FAA policy and guidance related to administering the Airport Improvement Program (AIP) and the authorizing statute. The FAA’s AIP Branch monitors FAA regional offices for compliance with policy and guidance, including input into SOAR and Delphi, and conducts periodic regional evaluations. Actual project costs reported by the airport sponsor are verified by an annual single audit required b OMB. Such audits, cover the entire financial and compliance operation of the airport sponsor’s governing body. Status of the project design and construction schedule contained in the RTAP is updated quarterly, based on meetings held with the airport sponsor and airlines. |
Transit Grant Process Efficiency
| Measure: |
Percentage of transit grants obligated within 60 days after submission of a completed application. (FY) |
|---|---|
| Scope: |
FTA grants obligated during a fiscal year period for major programs: Urbanized area, non-Urbanized area, and Elderly and Persons with Disabilities formula grants; Capital grants; Job Access and Reverse Commute grants; Over-The-Road Bus grants; and Planning grants. |
| Sources: |
FTA internal databases including the Transportation Electronic Award Management (TEAM) system. |
| Statistical Issues: |
Processing time is calculated from submission date to obligation date. $0 dollar, non-funding grant amendments are excluded from analysis. |
| Completeness: |
Data are current with no missing data, since FTA uses internal databases, including the Transportation Electronic Award Management (TEAM) system. All grants obligated during the fiscal year for the selected programs (see scope) are included in the original data set. In rare cases where the submission date is omitted (which prevents processing time calculation), missing dates are researched and added to the database prior to reporting. The “$0” amendments are excluded because they are not representative of the grant processing action being tested. |
| Reliability: |
The files that contain raw data from TEAM have been tested to ensure that all fiscal-year-to-date obligated grants are included and that data is current. Report programs screen various date fields to identify any missing or out-of-sequence dates that would skew averages; dates are corrected prior to reporting. Reconciliation reports of TEAM data are produced monthly and anomalies are explored and resolved. Detailed monthly grant processing progress reports provide management tools to the Regional Administrators, who continue to make this goal a top priority. |
Environmental Justice
| Measure: |
Number of environmental justice (EJ) cases that remain unresolved after one year. (FY) |
|---|---|
| Scope: |
Data will cover complaints filed with DOT under Title VI of the Civil Rights Act of 1964 and that have had environmental justice elements, such as allegations of substantially adverse environmental or health impact on a minority or low-income community by a transportation project. Case resolutions are actions that end or administratively close out complaints. These include such actions as determinations of no jurisdiction, withdrawals by complainants, resolutions achieved through alternative dispute resolution, findings of no violation, and negotiated settlements after discrimination findings under Title VI. |
| Sources: |
Data are collected from the entire population of interest. Data for External Complaint Tracking System (XTRAK) will cover all complaints filed with DOT that involve allegations of discrimination by an entity that received DOT funding, or in situations where DOT has statutory enforcement authority. Valid bases for allegations of discrimination include: age, color, disability, ethnicity, National origin, race, religion, and sex. Upon receipt of information alleging discrimination, data will be entered by the Departmental Office of Civil Rights (DOCR) staff and DOT Civil Rights office personnel. Data will be entered continuously by DOCR as cases are filed and as the responsible DOT Civil Rights office processes the case. XTRAK includes information on all external administrative civil rights complaints filed with DOT. |
| Statistical Issues: |
None. |
| Completeness: |
This indicator does not measure the impact of DOT’s efforts to prevent the conditions that give rise to complaints. It does provide an initial measure of response timeliness, which is important to the public. The measure was expanded in 2000 to include the percentage of cases that remain unresolved after one year as a further indicator of the timeliness of resolution. All environmental justice cases by definition relate to the concerns of a community of low-income and/or minority people. In addition, the number of cases indicates the pervasiveness of community perception of significantly adverse environmental and health concerns. It should be noted that environmental justice complaints can include allegations of discrimination on the basis of low income, which is not covered by Federal civil rights statutes. Thus, although most EJ complaints are also under Title VI of the Civil Rights Act of 1964, not all are. Finally, there is no firm definition of what constitutes an EJ complaint, and thus, views can differ on what should be entered into XTRAK as an EJ complaint. The measure is current with no missing data. |
| Reliability: |
Performance data are used by the DOCR and other DOT OAs in strategic management of this program. |