Performance Data and
Performance Measurement
Performance measurement is dependent on the availability of useful data. Useful data will indicate level of performance and progress toward organizational goals. All data are imperfect in some fashion. Pursuing “perfect” data, however, may consume public resources without creating appreciable value. For this reason, there must be an approach that provides sufficient accuracy and timeliness but at a reasonable cost. This section of the Performance Plan/Performance Report provides information on how DOT reports on performance, verifies and validates data, assesses limitations of the data, and plans for improving DOT’s data.
In an attempt to bring consistency and quality to
its performance reporting, DOT has implemented some general rules regarding the data it uses and how
it is evaluated.
Annual
data – Whenever
available, the data in this document are reported on a Federal Government
fiscal year basis. However, there are instances where this is not possible so
calendar year data are used instead. This
often occurs when data are collected and reported to DOT by external sources
and a calendar year reporting requirement is specified in the implementing
regulation. The reporting timeframe (FY or CY) for each measure is included in
the Data Details in Appendix I.
Annual
results – If
available, the results for the most recent year in the Report are listed as
“Actual” in the Performance Goals & Results box for each performance
measure. However, given the March deadline for submission of the Performance Report,
quite often data have not been compiled and finalized for the entire year. When
this occurs and an actual value is not available for the current year, either
an estimate or projection is provided instead. In general, estimates are based
on partial year data that are extrapolated to cover a full 12-month period. For
example, if six months of data are available, they will be compared to prior
years for the same six-month period to determine any variation from past
levels. Historical trend information, supplemented by program expertise, will
then be applied to estimate the remaining six months of performance. The result
will be identified as a “preliminary estimate” in the Report. If partial year
data are not available, then past trend information will be analyzed and
supplemented by program knowledge to develop a projected value for the annual
performance measure. The result will be identified as a “projection” in the
Report. As data are finalized, the projections and preliminary estimates will
be replaced by actual results. Results
may be amended as errors and omissions are identified in the data verification
process, because updated information is provided by the reporting sources, or
because of legal or other action that changes a previously reported value. For
example, updated pipeline spill reports may change the status of a previously
reported value used in performance measurement.
In
measuring progress toward the majority of performance goals, DOT is moving to a
system of monthly performance measurements.
This will make it much easier to internally gauge periodic progress
toward goals as the year progresses, and will enable more timely performance
reporting after the years’ end.
Completeness
of Data – As
described above, actual data and “preliminary estimates” incorporate complete
or partial data from 2001. Results
listed as “projections” are not based on data from 2001, but on trend data from
prior years.
Reliability
of Measurement Data
– Because performance results in a given year are influenced by multiple
factors, some of which are beyond DOT’s control, and some of which are due to
random chance, there may be considerable variation from year to year. (See
discussion in Appendix I.) A better “picture” of performance may be gained by
looking at results over time to determine if there is a trend. Therefore,
graphs are provided for each measure showing trend lines back to 1990, or as
many years as possible if data are not available back to 1990. Additionally, a
table is included at the beginning of each strategic goal section giving the
available data from 1995 through 2001 for measures with performance goals
specified for 2001.
Integral
to performance measurement is understanding data limitations, addressing these
limitations where necessary and cost-effective, and acknowledging those that
remain when interpreting results. This section on verification and validation
provides a DOT-wide overview of our plan for assessing the quality of the data
DOT uses to measure its performance, and follows the GAO definitions for
verification and validation:
“Verification
is the assessment of data completeness, accuracy, consistency, timeliness, and
related quality control practices.”
“Validation
is the assessment of whether data are appropriate for the performance measure.”
Virtually
all data have errors. In Appendix I we have provided the following information
about the data used for each performance measure: source of the data,
limitations of the data, observations about the quality of the data, work
planned or ongoing to improve data quality, and any known biases.
Additionally,
we have compiled Source and Accuracy Statements for each of the DOT data
programs used in this report, which can be found at www.bts.gov/statpol/SAcompendium.html.
The Source and Accuracy Statements give more detail on the methods used to
collect the data, sources of variation and bias in the data, and methods used
to verify and validate the data.
By
validating data used in the DOT performance plan, we are ensuring that those
data are reflective of the phenomena they purport to measure. The Office of the
DOT Inspector General (OIG) plans to selectively verify and validate
performance measurement data each year. When pertinent to the conduct of
ongoing projects, OIG will also assess performance measures to determine their
appropriateness for measuring progress toward stated goals. These assessments
may lead to changes in the goals, improvements to or additions of data
collection systems, or both.
Assessing
and, where possible, eliminating sources of error in DOT data collection
programs has always been an important task for data program managers. As a part
of their ongoing work, managers of Departmental data programs use quality
control techniques, such as flowcharting the data collection process, to
identify where errors can be introduced into the data collection system.
Program managers also use computerized edit checks and range checks to minimize
errors that may be introduced into the data of their respective programs. In
addition, quality measurement techniques are employed to measure the effects of
unanticipated errors. These include verification of data collection and coding,
as well as coverage, response and non-response error studies to measure the
extent of human error affecting the data. As sources of error are identified,
steps are initiated to improve the data collection process.
The
data used in measuring performance come from a wide variety of sources. Much of
the data originates from sources outside the Department and, therefore, outside
the direct control of the Department. The data often come from administrative
records or from sample surveys. While DOT may not have a strong voice in
improving the quality of outside data, the Department takes all available
information about the limitations and known biases in outside data into account
when using the data.
The
myriad data sources make the task of assessing and, where possible, eliminating
error a challenging one for DOT. Different data systems contain different types
of errors. For example, data from administrative records systems may have
missing or incorrect records, and data from sample surveys will contain
sampling error.
Several
measures (particularly in safety) require aggregation across transportation
modes. This can be particularly problematic because of the use of different
definitions in different transportation modes. Also, data from outside the
Department may have unknown error properties.
To
help the operating administrations address these issues, the Bureau of
Transportation Statistics (BTS) is developing a statistical policy framework
where the operating administrations will work together to identify and
implement the current statistical “best practices” in all aspects of their data
collection programs. This project is consistent with the data capacity
discussions found in the DOT Strategic Plan.
In
2001, a DOT intermodal working group addressing DOT data quality issues
continued to:
▪
develop
Departmental statistical standards;
▪
update Source and Accuracy Statements for all DOT data programs to
document limitations and known errors and biases;
▪
improve
quality assurance procedures;
▪
evaluate
sampling and non-sampling error; and;
▪
develop common definitions for data across modes.
BTS's statistical staff is
consulting with the DOT operating administrations’ data program managers to
assist in data evaluation and validation, documenting data sources, and
determining the reliability of performance measurement estimates.
Departmental data systems
managers use these data verification methods:
▪
Comparisons with previous data from the same source.
▪
Comparisons with another reliable source of the same type of data
within DOT for the same time period.
▪
Comparisons with another reliable source of the same type of data
within DOT for a previous time period.
▪
Comparisons with another reliable source of the same type of data
outside DOT for the same time period.
▪
Comparisons with another reliable source of the same type of data
outside DOT for a previous time period.
In addition to
computerized edit checks and clerical review procedures to look for outliers,
duplicate records, and data inconsistencies, data managers also verify data
quality at each step of the data collection process using these procedures:
▪
Re-collecting/re-interviewing all (or a sample of) records and
reconciling with the original collection. (This applies to census or sample
survey data collections from administrative records, organizations, or
individuals.)
▪
Conducting 100 percent (or a sample of) data re-coding and
reconciliation to assess and correct coding errors.
▪
Conducting 100 percent (or a sample of) data re-entry and
reconciliation to assess and correct data entry errors.
The American Travel Survey’s re-interview program,
in which a sample of households were re-contacted and differences reconciled,
is an example of a verification system within a data collection program.
DOT
Data Source Limitations – Timeliness is the most significant limitation for DOT performance
measurement data. Some DOT data are not collected annually. For example, the
National Household Travel Survey and the Commodity Flow Survey each collect
data every five years. Data that are
collected each year (or more frequently) require time to analyze, confirm and
report results. For example, Highway Performance Monitoring System
vehicle-miles traveled (VMT) data require several months of post-collection
processing, making final results unavailable for this performance report.
Other
performance measurement data limitations can be found in the previously
mentioned Source and Accuracy Statements for DOT data programs. These
statements contain descriptions of data collection program design, estimates of
sampling error (if applicable), and discussions of non-sampling errors.
Non-sampling errors include under-coverage, item and unit non-response,
interviewer and respondent response error, processing error, and errors made in
data analysis.
As
part of its mandate in the Intermodal Surface Transportation Efficiency Act of
1991 (ISTEA), the Transportation Equity Act for the 21st Century (TEA-21), and
its plans for a statistical policy framework in the Department, BTS is working
on a program of research, technical assistance, and data quality enhancement to
support the continued improvement of data programs in DOT. This will help data
program managers throughout DOT improve data quality and better document known
data limitations. BTS also assists operating administrations with data
collection and documentation.
Many of DOT’s internal data programs
rely on State DOTs to collect reliable statistics within cost constraints.
While we work closely with our State DOT partners, we do not have direct
control over these data.
External
Data Source Limitations – Timeliness is also a significant limitation for external or
third-party data. Other limitations of external data are noted in the comments
for each performance measure in Appendix I. In some cases, DOT has replaced
external data, where little is known about the quality of the data, with
internal data. For example, DOT has used estimates of person-miles traveled
(PMT) from private organizations, absent any better estimate. The 1995 Nationwide
Personal Transportation Survey and American Travel Survey give DOT data with
known error properties that allow a better estimate of PMT.
The
DOT Strategic Plan 2000 – 2005 identifies data needs for each of the Department’s
strategic goals. They include:
Safety – DOT is undertaking major efforts
over the next several years to improve safety data. Safety has always been our
primary strategic goal, and in 1999 DOT created a Safety Data Action Plan to
better organize data improvement efforts. BTS will lead efforts to: 1) develop
common criteria for reporting injuries and deaths; 2) develop common data on
accident circumstances; 3) improve data quality; 4) develop better data on
accident precursors; 5) expand the collection of near-miss data to all
transportation modes; 6) develop a variety of common denominators for safety
measures; 7) advance the timeliness of safety data; 8) link safety data with
other data; 9) explore options for using technology in data collection; and 10)
expand, improve and coordinate safety data analysis.
Homeland
Security –
Existing performance data sources are generally good, but DOT will collect data
to better understand the transportation system’s vulnerability to intentional
acts of disruption or destruction.
Mobility – All mobility outcomes present
complex measurement issues. Accordingly, DOT will: 1) develop ways of measuring
user transportation cost, time, and reliability with time-series data; 2)
develop better approaches for measuring access; 3) develop straightforward
measures of congestion and its costs; 4) produce more timely and comprehensive
data on the condition and use of the transportation system; and 5) develop a
more complete understanding of variables influencing travel behavior.
Economic
Growth – DOT needs
aggregate data for measuring the productivity, effectiveness and efficiency of
the U.S. transportation system. We plan to collect, analyze and disseminate
data and information that identify critical trends and issues relating to
transportation’s nexus to the U.S. economy. DOT will: 1) develop a means of
measuring transportation cost, time, and reliability – at an aggregate level –
with time-series data; 2) develop a comprehensive measure of the transportation
capital stock; 3) improve our view of changes in the transportation workforce;
4) develop better measures of productivity in the transportation sector, and
other issues concerning use of Producer Price Indices; and 5) develop a better
picture of transportation-related variables influencing U.S. competitiveness in
the global economy.
Human
and Natural Environment – DOT will: 1) develop comparable and complete data on transportation
emissions, noise, hazardous materials releases, and wetlands impacts; 2)
improve our understanding of collateral damage to the human natural
environment; 3) create better leading indicators for potential environmental
issues; and 4) develop a reliable method of measuring the environmental
benefits of bicycling and walking.
Each
table includes a description of a performance measure and associated data
provided by the agencies in charge of the measure. The Scope statement gives an overview of the data collection strategy
for the underlying data behind the performance measure. The Source statement identifies the databases
used for the measure and their proprietary agencies. The Limitations statement describes some of
the shortcomings of the data in quantifying the particular performance characteristics
of interest. The Statistical Issues
statement has
comments, provided by the Bureau of Transportation Statistics (BTS)
and the agency in charge of the measure, that discuss variability of the measure
and other points. The Verification and Validation statement indicates steps taken
by the proprietary agencies to address data quality issues.
DOT
feels strongly that full compliance with the Government Performance and Results Act
requires impartial reporting of the statistical uncertainty associated with
numerical performance measures. A
portion of this uncertainty is related to the methodology used to calculate the
performance measure and the accuracy of the underlying data. For example, the use of samples introduces
uncertainty because estimates are used in lieu of actual counts. Also, there may be errors in the data
collected. However, there are many
other sources of variation (e.g., nonsampling errors, climate effects, new
technology) and they are often difficult to quantify. Nonetheless, a combination of past data and expert judgment can
enable uncertainty statements that are order-of-magnitude correct for even the
most difficult problems.
The
standard error of a performance measure indicates the likely size of the chance
variation in the reported number. It
incorporates both the effects of measurement error, survey error, and so forth,
as well as the variation that occurs naturally from year to year (i.e., even if
there were no change in laws, infrastructure conditions, or human behavior,
there would still be chance variation in an annual count of fatalities). DOT success in meeting GPRA goals must be
viewed in the context of this background noise.
In
many of the following Statistical Issues statements, BTS refers to regression
standard error. This is a modification
of the standard error to take into account of linear trends in the
recent past. Such adjustment is
generally needed to incorporate consistent trends due to cumulative effects of
such things as education programs, changing demographics, the gradual adoption
of new technologies, and so forth. The
underlying assumptions are that: over a short time period the trend of the
measurement data is linear; for any given year the performance measure values
are normally distributed; and the standard deviation is the same for all
years. We believe that these
assumptions lead to a conservative estimate of variability.
The
regression standard error is an estimate, calculated from the annual
performance results, of this common standard deviation. It may be used in the same way as a regular
standard error to set confidence intervals or describe uncertainty. For the purposes of performance measurement,
it may be considered a rough approximation of the annual variability in a
measure, and it will include the affects of program initiatives, influences
beyond the control of DOT (e.g., weather, petroleum prices, etc.), random
chance, and errors inherent in the data.
For
further information about the source and accuracy (S&A) of these data,
please refer to the BTS S&A compendium available at
www.bts.gov/statpol/SAcompendium.html
|
Measures:
|
1. Transportation
fatalities. (CY) 2. Fatalities
per 100 million passenger-miles. (CY) 3. Fatalities
per 100 million ton-miles of freight. (CY) 4. Transportation
injuries. (CY) 5. Injuries
per 100 million passenger-miles. (CY) 6. Injuries
per 100 million ton-miles of freight. (CY) 7.
Transportation incidents. (CY) |
|
Scope: |
This family of measures
aggregates fatalities, injuries and incidents across all modes of transportation
(air, highway, railroad, transit, waterborne and pipeline). The fatality and injury rates per 100 million
passenger-miles exclude pipeline fatalities and injuries due to minimal
interaction with passenger miles.
Highway-rail grade crossing fatalities and injuries are not counted
since they are included in data for highways.
The
fatality and injury rates per 100 million ton-miles of freight include
fatalities and injuries from large truck, rail, waterborne and pipeline
transportation. Highway-rail grade
crossing fatalities and injuries are also included since these involve
freight transportation-related fatalities and injuries that would not
otherwise be counted. Ton-miles of
freight covers intercity truck, rail, water and oil pipeline transportation. Aviation fatalities, injuries and
ton-miles are excluded because the fatality and injury data are not separated
from passenger air carriers. Transportation incidents include crashes, system
failures, spills, releases, and other accidents of a similar nature. |
|
Source: |
The data for these measures are
obtained from National Transportation Statistics published annually by the
Bureau of Transportation Statistics.
Information is taken from the following tables: Transportation Fatalities
by Mode; Injured Persons by Transportation Mode; U.S. Passenger-Miles
(Millions); U.S. Ton-Miles of Freight (Millions); and Transportation
Accidents by Mode. The one exception
is the data on large truck fatalities and injuries used for calculating
fatality and injury rates per 100 million ton-miles of freight are obtained
from the Federal Motor Carrier Safety Administration.
|
|
Limitations: |
Double counting of fatalities and
injuries may occur when an accident involves more than one mode of
transportation. Differing definitions
of injuries or transportation-related fatalities makes comparison across
modes of transportation problematic.
Highway injuries and incidents are obtained from a nationally
representative probability sample and are estimates, while the totals for
other modes of transportation are actual counts. The highway estimates are based on crashes where a police
accident report was completed and the crash resulted in property damage,
injury or death. Accidents that were
not reported to the police or did not result in property damage are not
included. Highway passenger miles are
calculated by multiplying vehicle-miles of travel (VMT) by the average number
of occupants for each vehicle type.
VMT is based on a nation-wide sample of vehicle travel. The average number of vehicle occupants
comes from survey information.
Therefore, vehicle passenger miles is an estimate, whereas
passenger-miles for other modes of transportation are calculated based on
actual passenger counts and recorded trip lengths. |
|
Statistical Issues: |
All fatality totals, and the injury and incident
numbers where actual counts are recorded, are relatively accurate. Any double counting or omissions are
expected to be fairly small. The
primary source of uncertainty in these measures comes from sampling and
survey errors related to estimation of highway injuries, incidents, VMT and
vehicle occupancy. Based on data from
1994-2000, the annual variations in the transportation safety measures are as
follows: the regression standard
error for the number of
transportation fatalities is 0.5 thousand.
For fatality rates by passenger-miles and ton-miles, it is 0.010 and
0.007, respectively. For number of
injuries, it is 0.10 million. For
injury rates by passenger-miles and ton-miles, it is 2.50 and 0.24,
respectively. For incidents, it is
0.16 million. |
|
Verification &
Validation: |
BTS compiles the data for the National
Transportation Statistics from information it gathers directly in its own
data systems (e.g. airlines information), information published by other
sources (e.g. FHWA highway statistics), or by personal communication with the
agency/organization responsible for collecting the data. Each data source conducts error checks and
monitors the accuracy of its data.
Most of these sources and their verification and validation procedures
are described in subsequent data details in this report for performance
measures of individual modes of transportation. |
|
Comment: |
While caution should be exercised in comparing fatalities,
injuries and incidents between modes of transportation due to differences in
definitions and calculations, the aggregation of these values still provides
useful information. Because the
methodology for calculating these measures has remained consistent over the
years, the trend information should provide a reasonably accurate picture of
results. |
|
Measure: |
Fatalities
per 100 million vehicle-miles-traveled (VMT) (CY) |
|
Scope: |
The number of fatalities is the total
number of motor vehicle traffic fatalities which occur on public roadways
within the 50 states and Washington, D.C. Vehicle
Miles of Travel (VMT) represent the total number of vehicle miles traveled by
motor vehicles on public roadways within the 50 states and Washington, D.C.
|
Source:
|
Motor vehicle traffic
fatality data are obtained from NHTSA’s Fatality Analysis Reporting System
(FARS). To be included in FARS, a motor
vehicle traffic crash must result in the death of a vehicle occupant or a
non-motorist within 30 days of the crash.
The FARS database is based on police crash reports and other state
data. FARS includes fatalities on all roadways open to the public, using the
National Highways System classification of roads. Pedestrian and bicycle fatalities that occur on public
highways, but do not involve a motor vehicle, are not recorded in FARS. However, they constitute only a small
number of fatalities. VMT data are derived from FHWA’s Traffic Volume Trends
(TVT), a monthly report based on hourly traffic count data in the Highway
Performance Monitoring System (HPMS).
Information is transmitted to NHTSA where it is reviewed for
consistency and accuracy before being entered into the system. These data,
collected at approximately 4,000 continuous traffic counting locations
nationwide, are used to determine the percentage change in traffic for the
current month from the same month of the previous year. The percentage change is applied to the
nationwide travel for the same month of the previous year to obtain an
estimate of nationwide travel for the current month. The data are recorded as monthly totals
and cumulative yearly totals.
|
|
Limitations: |
VMT data are subject
to sampling errors, whose magnitude depends on how well the locations of the
continuous counting locations represent nationwide traffic rates. HPMS is also subject to estimating
differences in the states, even though FHWA works to minimize such differences
and differing projections on growth, population, and economic conditions that
impact driving behavior. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating fatality rates is the denominator. While the estimate of total fatalities
used in the numerator is relatively accurate, the estimate of total vehicle
miles in the denominator has far more variability. Based on data from 1994-2000, the annual
variation in the fatality rate has a regression standard error of 0.029. The estimates of the
number and percentages of persons killed in motor vehicle traffic crashes
during 2001 are preliminary and are based on incomplete data and statistical
models. NHTSA’s first official
estimates for 2001, the Early Assessment, are being developed and will be
completed in early April 2002.
Differences between the Official Early Assessment estimates and those
in this report are to be expected. |
|
Verification & Validation: |
Fatality data from FARS
are reviewed and analyzed by NHTSA’s National Center for Statistics and
Analysis. Quality control procedures
are built into annual data collection at 6 and 9 months, and at year’s end. A study was completed in 1993, looking at
samples of FARS cases in 1989 through 1990 to assess the accuracy of data
being reported. VMT data are reviewed
by FHWA for consistency and reasonableness. |
|
Comment: |
This data program has
been in use for many years and is generally accepted for describing safety on
the Nation’s highways. Adjusting raw
highway fatalities and injuries by VMT provides a means of portraying the
changes in highway fatalities on a constant exposure basis and facilitates
year-to-year comparisons. |
|
Measure: |
Number
and rate (per million commercial VMT) of fatalities in crashes involving
large trucks. (CY) |
|
Scope: |
The measure includes all fatalities (e.g., drivers and occupants
of passenger cars, motorcycles, large trucks, or pedestrians) associated with
crashes involving trucks with a gross vehicle weight rating of 10,000 pounds
or more. The number of fatalities
comes from NHTSA’s Fatality Analysis Reporting System (FARS) data, a census
of fatal traffic crashes within the 50 states, Puerto Rico, and Washington,
D.C. The fatal crash rate is the
number of fatalities per 100 million vehicle miles of large truck travel
(VMT). |
Source:
|
NHTSA’s Fatality
Analysis Reporting System (FARS) provides fatality data. The VMT data are derived from the Federal
Highway Administration’s (FHWA) Highway Performance Monitoring System (HPMS). |
|
Limitations: |
FARS data elements are modified
from year to year to respond to emphasis areas, vehicle fleet changes, and
other needs for improvement. Large truck VMT reported to FHWA by each state
is based on a sample of road segments and is not a census. In addition, the methods used to calculate
total VMT may vary from state to state. The methods used by the states to
estimate the VMT contribution from rural and urban minor collectors are
unknown. |
|
Statistical Issues: |
The fatality counts in FARS are generally quite accurate. The major sources of error are
underreporting by some precincts and inconsistent use of the definition of a
truck. Based on 1994-2000 data, the
chance variation in a given year has a regression standard error of
approximately 157 fatalities. Because
the VMT data provided to FHWA from each state are estimates based on a sample
of road segments, the numbers have associated sampling errors. The methodology used by each of the states
to estimate VMT is not known and may introduce additional non-sampling
error. Although states provide VMT
estimates on an annual basis, they are only required to update their traffic
counts at all sampling sites once every three years. Thus an annual VMT estimate from a
particular state may be based, in part, on data collected during a previous
year. Based on 1994-2000 data, the
chance variation in a given year in the number of fatalities per 100 million
vehicle miles of large truck travel has a regression standard error of 0.053. |
|
Verification & Validation: |
Fatality data are reviewed and analyzed by NHTSA’s National
Center for Statistics and Analysis.
Quality control procedures are built into data collection and data
processing. A study using samples of
1989-1990 FARS cases was completed in 1993 to assess the accuracy of data being
reported. FHWA routinely works with
state data providers to modify reported VMT values that do not appear
reasonable before incorporating them into its final master file. |
|
Comment: |
The FARS data have been around for many years and are generally accepted
as a good source for describing fatal crashes on the Nation’s highways. The large truck VMT data used to calculate
fatal crash rates have both sampling and non-sampling (i.e., bias) error
associated with it. The impact of
these errors on FMCSA’s estimates of large truck crash rates is considered to
be minimal. |
|
Measure: |
1. Alcohol-related fatalities per 100 million
vehicle-miles traveled. 2. Percentage of highway fatalities that are alcohol
related. (CY) (2001) |
|
Scope: |
The number of
fatalities resulting from motor vehicle traffic crashes that are alcohol
related and occur on public roadways within the 50 states and Washington,
D.C. |
Source:
|
Motor vehicle traffic fatality data are obtained from NHTSA’s
Fatality Analysis Reporting System (FARS). FARS is a census of fatal motor
vehicle traffic crashes within the 50 states, Puerto Rico, and Washington,
D.C. To be included in FARS, a crash
must result in the death of a vehicle occupant or a non-motorist within 30
days of the crash. The FARS data are
based on police crash reports and other state data. FARS includes fatalities
on all roadways open to the public, using the National Highways System classification
of roads. Pedestrian and bicycle
fatalities that occur on public highways, but do not involve a motor vehicle,
are not recorded in FARS. However,
they constitute only a small number of fatalities. A fatal motor vehicle traffic crash is alcohol-related if
either a driver or a non-motorist (such as a pedestrian) involved in the
crash had a measured or estimated blood alcohol concentration (BAC) of 0.01
grams per deciliter or above.
|
|
Limitations: |
Blood Alcohol Concentration test
results are not available for all drivers and non-occupants involved in fatal
crashes. Missing data can result for
a number of reasons -- the most frequent of which is that persons are not
always tested for alcohol. To address
the missing data issue, NHTSA has developed a statistical model (Multiple
Imputation) to estimate specific
values of BAC across the full range of possible values.
Estimating missing BAC in
this manner will permit the estimation of valid statistics such as variances,
measures of central tendency, confidence intervals and standard deviations. The statistical model is based on
important characteristics of the crash including crash factors, vehicle
factors, and person factors. While
this measure does not link alcohol with fault in fatal crashes, the more
comprehensive scope of the measure compensates for a possible undercount of
the extent of the alcohol impaired driving problem. Multiple Imputation differs from the statistical model used in
previous years. However, all historical series of alcohol
involvement will be revised back to the 1982 data year to reflect the
estimates from the new methodology. |
|
Statistical Issues: |
The primary sources of uncertainty
in this performance measure arise from information gaps in the number of
intoxicated non-motorists, and from using the statistical model to estimate
the number of intoxicated drivers. The estimates of the number and
percentages of persons killed in motor vehicle traffic crashes during 2001 included in this section are preliminary and are based on incomplete data and
statistical models. They were provided to meet the time restraints required
for this report. NHTSA’s first
official estimates for 2001, the Early Assessment, are being developed and
will be completed in early April.
Differences between the Official Early Assessment estimates and those
in this report are to be expected. |
|
Verification &
Validation: |
Data are reviewed and analyzed by
NHTSA’s National Center for Statistics and Analysis. Quality control
procedures are built into annual data collection at 6 and 9 months, and at
year’s end. In 1987 and 1988, an
independent panel of academics reviewed and commented on the statistical
methods used in measuring alcohol-related highway fatalities. This report recommended that research and
development utilize a model that would permit the imputation of missing BACs
as a semi-continuous variable. |
|
Comment: |
This data program has been used
for many years and is generally accepted for describing safety on the
Nation’s highways. |
|
Measure: |
Injured
persons per 100 million vehicle-miles-traveled (VMT) (CY) (2001) |
|
Scope: |
The number of injured persons
is an estimate of the total number of persons injured in motor vehicle
traffic crashes that occur on public roadways in the 50 states and
Washington, D.C. Vehicle Miles of
Travel (VMT) represent the total number of vehicle miles traveled by motor vehicles
on public roadways within the 50 states and Washington, D.C. |
Source:
|
The number of injured persons data are derived from the
NHTSA’s National Automotive Sampling System (NASS) General Estimates System
(GES). The NASS GES is a nationally representative
probability sample that yields national estimates of total nonfatal injury
crashes, injured persons, and property-damage-only crashes. NASS GES data
cover all roadways open to the public, using the National Highways System
classification of roads.
VMT data are derived
from FHWA’s monthly report, Traffic Volume Trends (TVT), a monthly report
based on hourly traffic count data in the Highway Performance Monitoring
System (HPMS). Information is
transmitted to NHTSA where it is reviewed for consistency and accuracy before
being entered into the system. These data, collected at approximately 4,000
continuous traffic counting locations nationwide, are used to determine the
percentage change in traffic for the current month from the same month of the
previous year. The percentage change
is applied to the nationwide travel for the same month of the previous year
to obtain an estimate of nationwide travel for the current month. The data are recorded as monthly totals
and cumulative yearly totals. |
|
Limitations: |
GES data are obtained
from a nationally representative sample of 60 sites. The results provide only national data,
not state level data, and are subject to sampling error. The magnitude of the sampling error
depends on the number of Primary Sampling Units (PSUs) in the sample and the
number of crash reports sampled within each PSU. VMT data are subject
to sampling errors, whose magnitude depends upon how well the continuous
counting locations represent nationwide traffic rates. HPMS is subject to estimating differences
in the states, although FHWA works to minimize such differences and differing
projections on growth, population, and economic conditions which impact
driving behavior. |
|
Statistical
Issues: |
The estimate of the injury
rate includes three main sources of uncertainty. The numerator count of injuries has a standard error of 5.1%
(cf. Appendix C of Traffic Safety Facts). The denominator estimate of VMT contains both complex sampling
and non-sampling errors. Based on data from 1994-2000, the annual
variation in the injury rate has a regression standard error of 4.04. The estimates of the
number and percentages of persons injured in motor vehicle traffic crashes
during 2001 are preliminary and are based on incomplete data and statistical
models. NHTSA’s first official
estimates for 2001, the Early Assessment, are being developed and will be
completed in early April. Differences
between the Official Early Assessment estimates and those in this report are
to be expected. |
|
Verification &
Validation: |
Data are reviewed and
analyzed by NHTSA’s National Center for Statistics and Analysis. Quality
control procedures are built into annual data collection at 6 and 9 months,
and at year’s end. A study was completed
in 1993, looking at samples of FARS cases in 1989 through 1990 to assess the
accuracy of data being reported. VMT
data is reviewed by FHWA for consistency and reasonableness. |
|
Comment: |
This data program has
been in use for many years and is generally accepted for describing safety on
the Nation’s highways. GES records
injury severity in four classes: incapacitating injury, evident but not
incapacitating injury, possible but not visible injury, and injury of unknown
severity. Adjusting raw highway
fatalities and injuries by VMT provides a means of portraying the changes in
highway fatalities on a constant exposure basis – to facilitate year-to-year
comparisons. |
|
Measure: |
Number
and rate of injured persons involving large trucks. (CY) (2001) |
|
Scope: |
The measure includes all injured persons (e.g., drivers and
occupants of passenger cars, motorcycles, large trucks, or pedestrians)
associated with crashes involving trucks with a gross vehicle weight rating
of 10,000 pounds or more. The
number of injured persons is derived from NHTSA’s General Estimates System
(GES). The injury rate is the number of injured persons per 100 million
vehicle miles of large truck travel (VMT). |
Source:
|
NHTSA’s General Estimates System (GES)
provides injury data. VMT data are derived from the Federal Highway
Administration’s (FHWA) Highway Performance Monitoring System (HPMS).
|
|
Limitations: |
GES data are obtained from a nationally representative sample of 60
sites. The results provide only
national data, not state-by-state data.
Large truck VMT reported to FHWA by each state is based on a sample of
road segments and is not a census. In
addition, the methods used to calculate total VMT may vary from state to
state. The methods used by the states to estimate the VMT contribution from
rural and urban minor collectors are unknown. |
|
Statistical Issues: |
The GES data have a
standard error of 6.9% for injuries from truck and automobile crashes (cf.
Appendix C of Traffic Accident Reports). They are less accurate than the
corresponding fatality counts. Based
on 1994-2000 data, the variation due to random chance in the number of injuries,
which includes sampling variability, has a regression standard error of
approximately 7,091. Because the VMT
data provided to FHWA from each state are estimates based on a sample of road
segments, the numbers have associated sampling errors. The methodology used by each of the states
to estimate VMT is not known and may introduce additional non-sampling error
into the estimates. Although states
provide VMT estimates on an annual basis, they are only required to update
their traffic counts at all sampling sites once every three years. Thus an annual VMT estimate from a
particular state may be based, in part, on data collected during a previous
year. Based on 1994-2000 data, the
chance variation in a given year in the number of injured persons per 100
million vehicle miles of large truck travel has a regression standard error
of 4.39. |
|
Verification &
Validation: |
Injury data are reviewed
and analyzed by NHTSA’s National Center for Statistics and Analysis. Quality
control procedures are built into data collection and data processing. FHWA routinely works with state data
providers to modify reported VMT values that do not appear reasonable before
incorporating them into its final master file. |
|
Comment: |
The data program has been around for many years and is generally
accepted for describing safety on the Nation’s highways. GES records injury severity in four
classes: incapacitating injury, evident injury but not incapacitating, possible
but not visible injury, and injury of unknown severity. The large truck VMT data used to calculate
injured persons rates have both sampling and non-sampling (i.e., bias) error
associated with it. The impact of
these errors on FMCSA’s estimates of large truck crash rates is considered to
be minimal. |
|
Measure: |
Percentage
of front occupants using seat belts. (CY) (2001) |
|
Scope: |
The proportion of
front seat outboard passenger vehicle occupants using shoulder belts during
daylight hours. |
Source:
|
Data for 1998, 1999, and 2000 are from
the National Occupant Protection Use Survey (NOPUS). NOPUS is a National, multi-stage
probability sample. In the first
stage, counties or groups of counties (Primary Sampling Units or PSUs) were
grouped by region (Northeast, Midwest, South, and West), level of
urbanization (metropolitan or not), and level of belt use (high, medium, or
low). Fifty PSUs were selected based
on the vehicle miles of travel in those locations. In the next stage, a random sample of eight (8) Census Tracts
was selected within each of the PSUs.
In the final stage a sample of ten (10) roadway segments for all types
of roads was selected within each Census Tract. In the even numbered years, shoulder belt use of front seat
outboard (driver and right front seat) passenger vehicle (passenger cars,
vans, sport utility vehicles, and pickup trucks) occupants was observed
during daylight hours at each of the 4,000 sampled roadway segments. In 1999, a Mini-NOPUS consisting of
observation at a subsample of 2,000 of the 4,000 roadway segments was
conducted.
Estimates of national
shoulder belt use for other years shown in the graph are based on state belt
use surveys. These surveys are
conducted by most of the 50 States and the District of Columbia. For the years shown, these surveys varied
in coverage, design, and observation methods. National averages were obtained by weighting the most recently
provided state belt use estimate by the population of the state. |
|
Limitations: |
NOPUS data are based on
a random sample of sites and, therefore, are subject to sampling error. For the estimate of overall National
shoulder belt use from the 2000 NOPUS Survey, sampling error was estimated to
be 1.4 percentage points.
Additionally, observation of shoulder belt use is restricted to
daylight hours. State belt use surveys
have been conducted in many different ways.
Less than half of the states conducted probability based surveys and
the rest were based on other methods.
Additionally, most states conducted surveys that observed use only for
those occupants and vehicles covered by their state belt use law. After enactment of a grant program in the
ISTEA of 1991, some 24 states had surveys that met design criteria specified
by NHTSA. |
|
Statistical Issues: |
The primary source of
uncertainty in NOPUS is sampling errors.
The most recent estimate shown in this report is based on a
probability sample, and the survey bias and reweighting are complex. For State surveys, uncertainty derives
from disparities among the different surveys conducted by the states, the use
of non-probability samples by many of the states, the differences in persons
and vehicles observed, the differing methodologies and processes followed to
collect data on the persons and vehicles observed, and the procedures used to
estimate overall belt use. To compute
the National average from state rates for a specific year, when a state did
not conduct a survey or provide NHTSA with an estimate, the most recent rate
provided by that state was substituted.
Also, weighting state averages by population may have overstated the
contributions of some states. Based on data from 1994-2000, the annual
variation in the seat belt use rate has a regression standard error of 1.31
percent. |
|
Verification & Validation: |
NOPUS data collection
is managed by a survey research contractor who has responsibility to hire and
train the data collectors/observers.
Before data collection begins, NHTSA reviews and approves all the
training materials and Data collectors/observers must pass a 2-day training
course. The data contractor also
conducts on-scene “surprise” quality control visits to ensure that
observations are made correctly and data are coded properly. Numerous edits are also employed in the
data processing. NHTSA reviews the
data provided by the contractor for consistency. NHTSA reviewed and approved
the survey designs and data collection procedures for 24 states as a result
of a grant program authorized by the ISTEA of 1991. NHTSA, however, did not conduct any quality review or
validation of the data collection and estimation processes employed by the
states during or after data collection for the years shown. |
|
Comment: |
None. |
|
Measure: |
Fatal
aviation accidents (U.S. commercial air carriers) per 100,000 departures.
(FY) |
|
Scope: |
This measure includes both scheduled and nonscheduled flights of
large U.S. air carriers (14 CFR Part 121) and scheduled flights of commuter
airlines (14 CFR Part 135). It
excludes on-demand (i.e., air taxi) service and general aviation. |
Source:
|
Part 121 and Part 135 departure data
is submitted to BTS under 14 CFR Parts 241 and 298, respectively. NTSB provides accident data. |
|
Limitations: |
The fatal accident rate
in these categories is small and could significantly fluctuate from year to
year due to the occurrence or non-occurrence of a single accident. |
|
Statistical Issues: |
The switch from calendar to fiscal year in 2001, combined with the
use of departures rather than flight hours as the activity measure for the
denominator, present new problems.
The FAA has no independent data sources to validate BTS-collected
departure data as it did with flight hour data. To overcome reporting delays of 60 to 90 days, FAA must rely on
historical data, partial internal data sources, and Official Airline Guide
(OAG) scheduling information to project at least part of the fiscal year
activity data. Due to the reporting
procedures in place, it is unlikely that calculation of future fiscal year
departure data will be markedly improved.
Lacking complete historical data on a monthly basis and independent
sources of verification increases the risk of error in the activity
data. The regression standard error
for the annual variation in the fatality rate, based on data from 1994 –
2000, is 0.023. |
|
Verification &
Validation |
The FAA does comparison checking of the departure data collected by
BTS; however, FAA has no independent data sources against which to validate
the numbers submitted to BTS. FAA
compares its list of carriers to the DOT list to validate completeness of the
reporting list and places the carriers in the appropriate category (i.e.,
Part 121 or Part 135). NTSB and FAA’s
Office of Accident Investigation meet regularly to validate the accident
count. |
|
Comment: |
The joint
government/industry group working on improving the level of safety for U.S.
commercial aviation has determined that the number of departures is a better
denominator measure to use for determining accident rates. In a recent report on the Safer Skies
effort the Government Accounting Office agreed and recommended that the FAA
use departures. |
|
Measure: |
Number
of fatal general aviation accidents. (FY) |
|
Scope: |
The measure includes on-demand (non-scheduled FAR Part 135) and
general aviation. General aviation
comprises a diverse range of aviation activities. The range of general aviation aircraft includes single-seat
homebuilt aircraft, helicopters, balloons, single and multiple engine land
and seaplanes including highly sophisticated extended range turbojets. |
Source:
|
National Transportation Safety Board (NTSB). |
|
Limitations: |
The
use of the 1996-1998 timeframe for the baseline represents one of the safest
periods in general aviation history in terms of a decline in fatal
accidents. The number of general aviation
accidents reported in any given year might change in subsequent years. There are many reasons for these changes
to the historical data. Primary among
them is that the accident had not been reported to the NTSB, or that it was
misreported and the information corrected at a later date. |
|
Statistical Issues: |
There is no major error in the accident counts. Random variation in air crashes results in
a significant variation in the number of fatal accidents over time. The regression standard error in this
variation for 1996 through 2000 is 16.5.
|
|
Verification &
Validation: |
NTSB and FAA’s Office of
Accident Investigation meet regularly to validate the information on the
number of accidents. |
|
Comment: |
It would be preferable to use fatal accident rates rather than
fatal accidents as the performance measure. However, general aviation flight
hours are based on an annual survey conducted by the FAA. Response to the survey is voluntary. The accuracy of the flight hours collected
is suspect and there is no readily available way to verify or validate the
data. For this reason, the General
Aviation community is unwilling to use a rate measure until the validity and
reliability of the survey data can be assured. |
|
Measures: |
1. Operational errors
per 100,000 activities, or per 1 million activities. (2001) 2. Number of operational errors where less than 80 percent of
required separation is maintained. |
|
Scope: |
An error occurs
when separation between aircraft is less than the separation determined
necessary for the specific phase of flight.
“Activities” are total facility activities, as defined in Aviation System Indicators 1997 Annual
Report. Total facility activities
are the sum of en route and terminal facility activities. |
Source:
|
FAA air traffic facilities have a software program called
Operational Error Detection Patch (OEDP) that detects possible operational
errors and sends alert messages to supervisory personnel. Facility management reviews OEDP alerts
and data provided from the National Track Analysis Program (NTAP) to
determine if an operational error has occurred. Controllers are required to report operational errors. The information is summarized in the FAA
Air Traffic Operational Error and Deviation Database.
|
|
Limitations: |
There is a few months’
lag in reporting data because of the need to investigate major
incidents. The severity of errors is
not measured. Minor errors such as a
4.5-mile rather than a 5-mile separation are counted in the same way as more
serious errors. Data are available
for 1994 and following years. The DOT IG conducted an
audit of reporting on operational errors.
The IG believes that there is a potential for underreporting of
operational errors, as some errors are self-reported. The FAA disagrees with this assessment
because there are substantial penalties for not reporting an operational error. |
|
Statistical Issues: |
There are no major sources of systematic error in the operational
errors data that have been quantified.
Again, random variation in operational errors results in a significant
variation in the measured rates over time. The regression standard error in
the operational error rate using 100,000 activities denominator and the 1
million activities denominator, based on 1994-2000 data, are .048 and .48,
respectively. |
|
Verification &
Validation: |
FAA performs system
checks and counts daily against reported data to ensure the accuracy of
information reported. |
|
Comment: |
In August 1998,
the FAA discovered and corrected a misunderstanding of the procedures used in
interpreting separation reported by the National Track Analysis Program and
the data provided by the Operational Error Detection Patch. The corrected application of these
procedures, while not affecting safety, has resulted in an overall increase
in the number of errors reported between 4.6 and 4.9 miles separation
(Standard separation in these cases is 5 miles). |
|
Measures: |
1. Number of runway incursions. (FY) (2001) 2. Number and rate (per 100,000 operations)
of highest risk runway incursions. |
|
Scope: |
Runway incursions are the result of ground collision hazards or
loss of separation for aircraft in the process of taking off or landing. They
are grouped in three general categories:
operational errors, surface pilot deviations, and vehicle/pedestrian
deviations. Incursions are reported and tracked at airports that have an
operational air traffic control tower.
|
Source:
|
Air traffic controllers and pilots are
the primary source of runway incursion reports. The data is recorded in the
FAA National Incident Monitoring System (NAIMS).
|
|
Limitations: |
Preliminary incident
reports are evaluated when received. Evaluation can take up to 90 days. |
|
Statistical Issues: |
There are no major sources of systematic error in quantified runway
incursion data. The regression standard error in the reported number of
incursions, based on 1994-2000 data, is approximately 15.4. Based on 1998 – 2001 data, the regression
standard error for the number and rate of highest risk runway incursions are
8.8 and 0.01, respectively. |
|
Verification &
Validation: |
Surface incidents are
reported in the Administrator’s Daily Bulletin at the beginning of each
weekday. Surface incidents are evaluated to determine if they should be classified
as incursions. Incidents are evaluated against the official runway incursion
definition. The Air Traffic Runway Safety Program Manager, ATP-20, makes the
final decision regarding runway incursions. |
|
Comment: |
None. |
|
Measure: |
Percent
of all mariners in imminent danger who are rescued. (FY) |
|
Scope: |
Includes people in water; on shore; and aboard a vessel,
offshore structure, pier, or vehicle that is in distress or in urgent need of
assistance. The Coast Guard makes a
final determination on scene whether there is imminent danger, based on
criteria that include the nature of distress, the condition of the vessel,
the people onboard, and the environmental conditions. Criteria for this decision are discussed
in search and rescue doctrine publications. |
Source:
|
CG Search and Rescue Management
Information System (SARMIS). Data is collected from Coast Guard field units
that conduct search and rescue responses.
|
|
Limitations: |
It is probable that some number of imminent danger cases, and the
associated lives, are not reported in SARMIS. This includes situations where no distress call was received by
the Coast Guard and the persons in distress were rescued by private citizens
or local government personnel, or where the persons in distress perished
without trace. The extent of this under-reporting is not known. There is some
judgment involved in assessing whether mariners are in danger. However, there
is likely to be consistency in these assessments across years. 1994 data is skewed upward by a large
surge of migrants interdicted at sea, most of whom were counted as “rescued,”
thus increasing the percentage of lives reported as saved. Reporting no longer includes migrants
interdicted; they are counted directly as migrants interdicted under law
enforcement activity. Prior to the
introduction of the next generation data system in October 2000, data entry
was limited to closed cases, after a rescue has been successfully completed
or after the recovery of a body. The
new data system now allows missing bodies to be tracked. In this first year of data, more cases
than expected were found where bodies were not recovered. Before adding this number into our data
analysis, we will track this number to assure that this represents a data
trend and not an unusual aberration.
Errors may be introduced in SARMIS through data entry, but are likely
rare for lives saved data elements. |
|
Statistical Issues: |
The primary source of uncertainty consists of non-sampling errors.
The second generation data system, brought on-line on October 1, 2000,
reduces error due to miscoding through the use of more extensive drop down
menus, machine generated case numbers, structured data boxes, and more extensive
business rules eliminating the selection of data not consistent with other
entered data. The regression standard error for year-to-year chance variation
is 2.6 percent mariners rescued, based on data from 1994 through 2000. |
|
Verification &
Validation: |
SARMIS data entry system
uses structured entry values, check boxes, and pull down selection lists to
limit entry errors. The use of plain
language descriptions eliminates a majority of erroneous data code selection.
Additional system business rules also eliminate the selection of data not
appropriate with other entered data. The SAR Mission Coordinator (SMC) is
responsible for accurate entry of particular case data by all units involved
in the case. CG Program Managers annually validate the data in SARMIS.
Entries are reviewed at Coast Guard District offices as first step in
validation – errors and inconsistencies are identified and corrected. Finally, Coast Guard Headquarters program
managers review compiled data annually to assess consistency with historic
variance and trends. This review
includes curvilinear regression analysis to compare current data to historic
data and a program review analysis to identify and resolve aberrations. |
|
Comment: |
Beginning in FY01, this measure will cover
all mariners in distress reported in SARMIS.
The previous measure covered only mariners reported in distress that
were rescued. The significance of the
87.5% result for FY99 is uncertain at this point; FY95-98 data show a flat
trend at 84%. It is not known if the
FY99 result was produced by anomalous factors, or if it is the product of
program strategies and a changing external environment. Therefore, the goal target remains at 85%
until more analysis is completed. For
FY 2001, the preliminary estimate of the measure was 84.2 percent of all
lives, bringing the percentage about equal to the average since 1995 and
slightly below the goal, but certainly within normal variation about the
average. |
|
Measure: |
Number
of recreational boating fatalities. (CY) (2001) |
|
Scope: |
Measure includes fatalities occurring aboard vessels that are
being operated for recreational purposes.
Surfboards, iceboats, and vessels engaged in sanctioned racing events
are not considered recreational vessels.
Fatalities are included if caused by a fire, explosion, sinking or
other occurrence involving a recreational vessel, and the vessel or
associated equipment caused or contributed to the fatality. Fatalities are not included if they
occurred aboard a recreational vessel, but were caused by self-inflicted
wounds or natural causes. Fatalities
are also excluded if they occurred while the victim was engaged in other
activity such as swimming or diving, where the vessel was used as a platform
only and was not a contributing factor to the fatality. Beginning two years
ago, the measure for Recreational Boating was revised by adding an additional
6% to the aggregate number of reported fatalities, to correct for an
estimated 6% underreporting of recreational boating fatalities. |
Source:
|
Coast Guard Boating Accident Report
Database (BARD). Data is entered into
BARD by state administrators who collect data from boat owners and operators
through formal Boating Accident Reports, as instructed in 33 CFR 173c.
|
|
Limitations: |
Fatality data is derived from reports submitted by
the public along with accompanying state investigation reports. There is
consensus among the Coast Guard, the states, safety professionals, and other
researchers that most fatalities that occur on inland and most coastal waters
are under-reported. To better
quantify the extent of possible under-reporting the Coast Guard initiated and
funded an analysis of BARD data conducted by the Boat Owners Association of
the United States (BOAT/U.S.) Foundation for Boating Safety. The study found some fatalities involving
recreational boating in the Coast Guard’s Search and Rescue Management Information
System (SARMIS) that were not in BARD. However, although the study reported a
9% discrepancy, further analysis revealed that some of these findings would
not be reportable as recreational boating fatalities. There is also consensus
that under-reporting exists for fatalities occurring offshore, and aboard
U.S. recreational boats operating overseas.
Also, although there are guidelines as to what constitutes a
recreational boating fatality, there is still an element of interpretation at
the state level in reporting fatalities.
It is probable that the states do not always interpret the guidelines
in the same manner. Overall, the best estimate indicates that total
fatalities are currently under-reported by at least 6%. |
|
Statistical Issues: |
The discrepancy between
BARD and the Search & Rescue Management Information System (SARMIS) amounts to 6% of the total reports for those states covered by SARMIS. The numbers given in this report have been
adjusted to correct the deficiency.
Also, note that since the boating fatality counts are influenced by
weather, gasoline prices and other external factors, annual chance variation
should be large. Using data from 1994
to 2000, the annual variation in the number of fatalities attributable to
random chance has a regression standard error of 50.7. |
|
Verification &
Validation: |
Fatality data in BARD is
verified and validated by state boating administrators and Coast Guard
program managers. At the end of the calendar year, the Coast Guard compiles
state fatality data and sends a report to each state for confirmation. Both State and Coast Guard officials
review the statistics, including sampling of cases to ensure guidelines for
classifying fatalities were followed.
Any discrepancy is reconciled jointly by the State and Coast Guard
program manager. |
|
Comment: |
Data are not normalized for increases or decreases in the number
or usage of boats, which tends to limit data use in making comparisons over
time. The number and usage of
recreational boats has increased over the past 2 decades, while the raw number
of fatalities has generally decreased. The BOAT/US review of BARD data for
1993 through 1997 identified underreporting in BARD of 8% in 1993 and 1994,
12% in 1995, 13% in 1996 and 8% in 1997.
The Coast Guard reviewed BOAT/US’s findings for 1995, 1996, and
1997. Each record for these years was
checked and fatalities that were incorrectly labeled as recreational boating
fatalities by BOAT/US were removed from the count. Based on this revised count of recreational boating fatalities
with mislabeled fatalities removed, the Coast Guard estimates that 7%, 8% and 4% of all recreational boating
fatalities were not captured in its Boating Accident Report Database (BARD)
in 1995, 1996 and 1997 respectively for purposes of this report. The median
of these numbers – 6% - has been used to adjust recreational boating safety
data for 1993, 1994, 1998 and 1999, and to reset the goals for 1999 through
2001. The original goal of 720 has been
increased by 6% to 763 for 2000. The Coast Guard is in the
process of commissioning a comprehensive
National Boating Survey to obtain valid and reliable information on boating
practices, safety, and exposure. This
information will enable safety officials to assess boating risk, implement appropriate
safety intervention strategies, and measure the effectiveness of program
activities in reducing the risk and negative outcomes associated with the use
of recreational boats. Data from
this study will be used to further address underreporting issues and estimate
reporting discrepancies in BARD. The
study was originally set to begin in Fall 2001, however data collection is
now scheduled to begin in April 2002. |
|
Measure: |
Fatalities and rate (per million passenger capacity) aboard passenger
vessels. (2001) |
|
Scope: |
This measure is an indicator of passenger safety. It includes reportable marine casualties
resulting in the death or disappearance of a passenger aboard any U.S. vessel
(regardless of type or location) or aboard foreign flag vessels in U.S.
waters. Exceptions include death/disappearance of “non-passengers”, whenever
the cause of death/disappearance is classified as being from diving, natural
causes, (e.g. heart attack) or whenever the death/disappearance is the result
of an intentional act (e.g. suicide, altercation). Fatalities on recreational vessels are not included for two
principal reasons: Recreational vessels are prohibited from carrying
“passengers” and recreational vessel fatalities are measured and reported
separately. |
Source:
|
Passenger fatality source data is
obtained from the Coast Guard Marine Safety Information System (MSIS). Passenger fatalities are reported to the
Coast Guard as required by federal regulations. Sources of reports are most often vessel masters, operators,
owners, insurance companies, legal representatives, and other mariners.
|
|
Limitations: |
The investigation,
retrieval, analysis and reporting processes result in under-reporting for the
most recent year, with the most significant effects over the most recent 5
months. Estimates are often used to
compensate for this known data-lag. The Coast Guard initiates about 40-50
civil penalty cases for failure to report marine casualties, although many of
these are for minor casualties. In addition, some passenger fatalities may
not be reported to the Coast Guard. This number is unknown. Some passenger injuries may ultimately
prove fatal and lead to death; some missing passengers may be found. These numbers may not be updated to
reflect the changes in status. The
number is believed to be small.
Duplicate casualty entries are sometimes entered into MSIS, and some
casualties are mistakenly omitted or coded incorrectly. Verification
procedures strive to correct these errors, but it is probable that a small
number are not corrected. The data retrieval & reporting processes do not
allow automated distinction between all death types (e.g. natural vs.
accidental). As a result, some
natural deaths or suicides may be inadvertently included. |
|
Statistical Issues: |
The major sources of
uncertainty in this measure are the estimation error (as a result of the
data-lag) and the reporting error (as a result of the inability to
distinguish between which deaths should be included and which should be
excluded). |
|
Verification &
Validation: |
Verification and
validation occurs at several levels.
Edit checks within MSIS software can detect some incorrect or missing
data and force review and correction before data entry is completed. Selection lists for certain data fields
also reduce the opportunity for data entry error. All investigations go through review at the field unit for
accuracy. Investigations of serious
marine casualties are also usually reviewed at district and headquarters
offices. The headquarters Data
Administration staff conducts periodic quality control checks to identify
entry errors such as missing data or miscoding, and corrects any errors
identified. Errors identified are
referred to either the Data Administration staff or the Investigations and
Analysis staff for correction. |
|
Comment: |
During FY 2002, the Marine Safety Information System (MSIS) will
be replaced by the Marine Information System for Safety and Law Enforcement
(MISLE). While the new system will be
a major improvement, it is expected to cause serious difficulties in making
performance comparisons. One factor
is that many business processes were re-designed in conjunction with system
development. Another factor is that
data quality under MISLE is expected to be superior to that of MSIS. While this represents improvement, it may
cause near-term problems in making meaningful comparisons of data between the
two systems. |
|
Measure: |
1.
Train accidents per million train-miles. 2.
Rail-related fatalities per million train-miles. (CY) (2001) |
|
Scope: |
The fatality measure includes anyone on rail property, any on-duty
railroad employee, and anyone killed by a train or its contents. It does not include fatalities on trains
or rail lines that do not connect to the national rail network, such as mass
transit operations, certain excursion and tourist railroads, and some
industrial railroads not connected to the general system. The only railroad fatalities that are not
counted are suicides (as determined by a public official) and death by
natural cause not associated with railroad operations. Train accidents do not include
those at grade crossings. They are
reported under the performance goal for highway-rail grade crossing
accidents. |
Source:
|
Railroad
Safety Statistics – Annual Report. Statistical data, tables,
and charts depict the causes and nature of rail-related fatalities. Data on fatalities and train miles are
reported to FRA by railroad companies. |
|
Limitations: |
Because of the scope of
the reporting criteria, some fatalities that are counted are not associated
directly with operation of the trains, and some railroad fatalities are not
counted. This scope is consistent
with the regulatory authority of the agency, but not consistent with other
modes of transportation for comparative purposes. |
|
Statistical Issues: |
The reported estimates
are based upon partially reported data from 2001. Based on data from 1994-2000, chance variation from year to
year, as reflected in the regression standard error, is 0.055 for rail
fatalities. |
|
Verification &
Validation: |
Railroads are required by
law to submit monthly accident/incident reports to FRA. They are also required to update any
inaccurate or incomplete information.
FRA conducts routine data audits (records inspections) to verify the
adequacy of railroad reporting and record keeping requirements. |
|
Comment: |
None. |
|
Measure: |
Grade-crossing accidents divided by the product of: 1) million train
miles and 2) trillion vehicle-miles-traveled. (CY) |
|
Scope: |
The measure includes all collisions with on-track equipment and
highway users at public and private grade crossings. |
Source:
|
Collisions and train-miles are reported in FRA’s Railroad Safety Statistics – Annual
Report. Vehicle-miles-traveled (VMT) are obtained from the FHWA
Office of Highway Information Management.
|
|
Limitations: |
Because the denominator includes all highway vehicle-miles-traveled
(VMT), and not just VMT that are exposed to grade crossings, the rate
portrayed may be lower than the actual risk. |
|
Statistical Issues: |
Trains and automobiles
have different exposures at rail crossings---the denominator used here
attempts to combine these. The
numerator is based on partially reported 2001 data. The annual variation by
chance from year to year as measured by the regression standard error is
0.109, based on data from 1994-2000. |
|
Verification &
Validation: |
FRA’s Office of Safety
has a review process to ensure that railroads and the States comply with
Federal reporting requirements in the preparation of the FRA Railroad Safety Statistics - Annual Report. |
|
Comment: |
None |
|
Measure: |
1. Transit
fatalities per 100 million passenger miles traveled. (CY) 2. Transit
injured persons per 100 million passenger miles traveled. (CY) |
|
Scope: |
The data include both riders and employees. A fatality is defined as a transit-caused
death from collision, personal casualty, fire, derailment, or bus going off
the road. An injury is defined as any
physical damage or harm to a person requiring medical treatment caused by a
transit collision, personal casualty, fire, derailment, or bus going off the
road. |
Source:
|
FTA’s Safety Management Information
System (SAMIS), with data reported by transit operators to the National
Transit Database (NTB).
|
|
Limitations: |
Because of the scope of
the reporting criteria, some fatalities that are counted are not associated
directly with transit operation. This
scope is consistent with the regulatory authority of the agency, but not
consistent with other modes of transportation for comparative purposes. |
|
Statistical Issues: |
The fatality and injury counts in SAMIS are generally quite
accurate---the major source of error in the measure comes from uncertainty in
the passenger miles traveled. Based on
1994-2000 data, the chance variation in a given year has a regression
standard error of 0.039 for the transit fatality rates and 2.210 for the
transit injury rates. |
|
Verification &
Validation: |
An independent auditor and the transit agency’s CEO certify that data
reported to the NTD are accurate.
Using data from the NTD to compile the SAMIS data, the Transportation
Systems Center compares current safety statistics with previous years,
identifies questionable trends, and seeks explanation from operators. |
|
Comment: |
None. |
|
Measure: |
Excavation
damages to natural gas and hazardous liquid pipelines. (FY) |
|
Scope: |
This measure is based on reported hazardous liquid and natural gas
accidents that meet federal reporting criteria as defined in 49 CFR 191.1 and
191.15 for natural gas transmission pipeline incidents and in 49 CFR 195.50
for hazardous liquid pipelines. |
Source:
|
RSPA’s Natural Gas Distribution and Transmission
Incident Reports and Hazardous Liquid Pipeline Accident Reports. Failure
reports are filed within 30 days of the occurrence of reportable
incidents. Complete calendar year
data are available by March 1 of the following year. Data may change as operators file
supplemental reports.
|
|
Limitations: |
RSPA lacks adequate infrastructure information on pipeline operations
and maintenance needed to fully characterize problems when they occur and
lacks information on precursor conditions that contribute to incidents. RSPA seeks further improvements in data
collection in 2002 to address these concerns. |
|
Statistical Issues: |
Reduction in excavation
damages is tied to economic growth and expansion as populations increasingly
are encroaching on once rural areas where major interstate pipelines are
located. Because of delays in mail
delivery associated with 9/11/2001 terrorist activities, statistical
close-out of the 2001 tally requires an extrapolation of number of reports
anticipated for the last quarter of 2001. |
|
Verification &
Validation: |
RSPA reviews/verifies
data provided for accuracy and requests supplemental reports where
shortcomings are indicated. |
|
Comment: |
RSPA discontinues this measure after 2002, replacing this safety
measure with pipeline excavation damages measure. |
|
Measure: |
Failures
of natural gas transmission pipelines. (CY) (2001) |
|
Scope: |
This measure is based on reported hazardous natural gas leaks that
meet federal reporting criteria as defined in 49 CFR 191.1 and 191.15 for
natural gas transmission pipeline incidents.
|
Source:
|
RSPA’s Natural Gas Transmission
Incident Report. Failure reports are filed within 30 days of the occurrence
of reportable incidents. Complete
calendar year data are available by March 1 of the following year. Data may change as operators file
supplemental reports.
|
|
Limitations: |
RSPA lacks adequate infrastructure information on pipeline operations
and maintenance needed to fully characterize problems when they occur and
lacks information on precursor conditions that contribute to incidents. Joint Federal, state and industry teams
have been formed to devise a new course to improve information availability. |
|
Statistical Issues: |
The number of failures of
natural gas transmission pipelines is likely to be underreported. The annual variation in the number of
failures from year to year due to chance has a regression standard error of
528 for natural gas pipeline failures based on data from 1994 to 2000. |
|
Verification &
Validation: |
RSPA reviews/verifies
data provided for accuracy and requests supplemental reports where
shortcomings are indicated. |
|
Comment: |
None. |
|
Measure: |
Number of serious
hazardous materials incidents in transportation. (CY) |
|
Scope: |
Serious reported hazardous
materials incidents were initially defined by RSPA to be those that result in a fatality
or major injury (for most purposes, an injury resulting in hospitalization)
due to a hazardous material, closure of a major transportation artery or
facility, or evacuation of six or more persons due to the presence of a
hazardous material, or a vehicle accident or derailment resulting in the
release of a hazardous material. For
the 2003 Plan, the definition is revised to include those incidents resulting
in a fatality or major injury, the evacuation of 25 or more employees or
responders or any number of the general public, the closure of a major
transportation artery, the alteration of an aircraft flight plan or operation
caused by the release of a hazardous material or the exposure of hazardous
material to fire; plus any release of radioactive materials from Type B
packaging, Risk Group 3 or 4 infectious substance, over 11.9 gallons or 88.2
pounds of a severe marine pollutant, or a bulk quantity (over 119 gallons or
882 pounds) of a hazardous material. This measure tracks only
transportation related releases of hazardous materials that are in
commerce. Volume of spills is not
tracked, as this does not necessarily indicate risk. |
Source:
|
Hazardous Materials
carriers report data to RSPA for entry into the Hazardous Materials
Information System (HMIS).
|
|
Limitations: |
Data for all hazardous
materials incidents is suspected of being incomplete due to under-reporting
for minor incidents. Most reportable
serious incidents are in the system, making this a more consistent measure
for program management. However, it
does not reflect all incidents.
RSPA has issued an NPRM to revise the reporting system. |
|
Statistical Issues: |
Although the
number of incidents is likely to be underreported, such recording error is
probably small in comparison to the annual variation due to chance. The annual variation in the number of
failures (original definition) from year to year due to chance has a regression
standard error of 37.2 based on data from 1994 to 2000. The new incident definition has a
regression standard error of 30.6 based on data from 1997 to 2000. |
|
Verification &
Validation: |
RSPA verifies the data by periodic
follow-up reviews of data entry by the manager of the Hazardous Materials
Information System, and verification audits of the data entry process. RSPA crosswalks HMIS reports against the
National Response Center log of accidents.
RSPA is improving compliance with reporting requirements by
correlating HMIS reports with FRA’s Accident Report data and the HMIS
telephonic data. RSPA is piloting and
plans to incorporate procedures to correlate HMIS reports with FHWA’s
Safetynet Accident File data. |
|
Comment: |
None. |
|
Measure: |
1. Average waiting time
in minutes for passengers in line for screening. (FY) 2. [Measure on passenger
and baggage screening effectiveness.] (FY) |
|
Scope: |
TBD |
Source:
|
TBD
|
|
Limitations: |
|
|
Statistical Issues: |
|
|
Verification &
Validation: |
|
|
Comment: |
|
|
Measure: |
Detection
rate for explosive devices and weapons that may be brought aboard aircraft.
(FY) (2001) |
|
Scope: |
Machine performance test results, automated threat-image
projection (TIP) and FAA field agent testing of aviation security screener
proficiency to detect and resolve images or FAA test objects that simulate
weapons and explosive devices in checked and carry-on baggage, or carried on
the person through an airport security checkpoint. |
Source:
|
FAA Office of Civil Aviation Security Airport
and Air Carriers Information Reporting System (AAIRS). Laboratory test results from the William
J. Hughes Technical Center.
|
|
Limitations: |
No comment. |
|
Statistical Issues: |
There is no major error
present in the subject data. |
|
Verification &
Validation: |
Special “red team”
testing led by agents based at FAA headquarters is used to validate field
test results. AAIRS data is subject
to multiple layers of review.
|
|
Comment: |
The White House Commission recommended more aggressive,
realistic testing. Funding that began
in 1997 enabled an increase in testing as more field agents were hired and
trained. Prior to 1998, data from
realistic testing were too sparse to be conclusive. |
|
Measure: |
Percent
of high interest vessels screened. |
|
Scope: |
High Interest Vessel (HIV) inspection or escort is measured by
the a ratio of the number of HIV vessel inspected or escorted to the number
of HIV vessels arriving at US ports. HIV
designation is determined using specific criteria. Coast Guard inspection or escort standards are to inspect 100%
of HIV. |
Source:
|
The data for this measure is collected
using a manual count from situation reports sent after a vessel inspection or
escort.
|
|
Limitations: |
This is an interim
activity-based measure. Appropriate
outcome-based measures are under development that will improve our ability to
measure and reduce security risks in US ports. |
|
Statistical Issues: |
This is a new measure and
data systems have not yet been developed or modified to capture this
information. It is possible that
errors in the data could result due to manual data collection. |
|
Verification &
Validation: |
Verification and
validation is conducted through cross checks with situation reports. |
|
Comment: |
None. |
|
Measure: |
Percentage
of days that the designated number of critical defense assets (high endurance
cutters, patrol boats, and port security units needed to support Defense
Department operational plans) maintain a combat readiness rating of 2 or
better. (FY) (2001) |
|
Scope: |
Only high endurance cutters, patrol boats, and port security
units that are designated as necessary for defense plans are included. The
specific units required are classified. |
Source:
|
DOD Status of Readiness and Training
System (SORTS) – Database used by the Coast Guard in applying DOD standards
to its assets to determine a readiness score.
|
|
Limitations: |
SORTS uses a multi-factor matrix to calculate the readiness
status. Although specific criteria
are outlined for each factor, some judgment is required in applying
criteria. Different units and personnel
may apply standard criteria in slightly different ways depending on the
nature of the unit’s mission. |
|
Statistical Issues: |
This particular
performance measure in FY 2001 is based on readiness levels of two types of
vessels, patrol boats and high endurance cutters, which have extremely
different levels of readiness. In
addition, a third resource, Port Security Units (PSUs), is measured for its
readiness. PSUs are comprised of
Coast Guard Reservists and Active Duty personnel, trained to protect foreign
ports for expeditionary forces. The
drastic change between FY 1999 and FY 2000 performance was caused in large
part due to the fact that the requirement to report the Contingency Personnel
Requirements List (CPRL) (the full wartime personnel strength requirement) in
the unit SORTS report was waived for FY 2000 and subsequent years pending
validation of personnel requirements that have changed due to new equipment
and operational procedures. The Navy
has been informed of this waiver and has not objected to reporting personnel
strength using the less demanding Coast Guard standards for peacetime
operations in view of the fact that Reserve Unit personnel are available to
quickly bring Coast Guard units up to the full wartime personnel strength requirements
in the event of a war. |
|
Verification &
Validation: |
Units self assess and
report readiness using objective standards.
Unit readiness is periodically validated through inspections,
assistance visits, and in some cases training and assessment at Navy
facilities. These assessments are
conducted by external, field level commands (such as Coast Guard areas,
districts, and groups). |
|
Comment: |
Coast Guard will continue to reassess the overall adequacy of
this measure. |
|
Measure: |
Percentage
of DOD-required shipping capacity complete with crews available within
mobilization timelines (FY) |
|
Scope: |
As of
March 2002, this measure is based on the material availability of 76 ships in
the Maritime Administration’s Ready Reserve Force (RRF) and 115 ships
enrolled in the Voluntary Intermodal Sealift Agreement (VISA) program, which
includes 47 ships enrolled in the Maritime Security Program (MSP). A second factor pertinent to this measure
is the availability of sufficient licensed and unlicensed mariners to operate
the available ships. The performance
measure represents the number of available ships (compared to the total
number of ships in the RRF and VISA) that can be fully crewed within the
established readiness timelines.
While other Government (primarily Military Sealift Command) owned or
controlled sealift type vessels are not included in this measure, they draw
their crews from the same pool of mariners.
Accordingly, the availability measure is adjusted to reflect expected
requirements during the early stages of a military crisis. |
Source:
|
Material availability of ships: MARAD records (and
reports to DOD) on the readiness/availability status of each RRF ship each
month. Typical reasons why a ship is
not materially available include: the ship is in drydock, the ship is
undergoing a scheduled major overhaul, or the ship is undergoing an
unscheduled repair. MARAD and DOD
also maintain records of the sealift ships enrolled in the MSP and VISA and
their crew requirements. Availability of mariners: Information on the
available supply of licensed and unlicensed mariners is extrapolated from
data received from the U.S. Coast Guard’s Merchant Mariner Licensing and
Documentation (MMLD) system.
|
|
Limitations: |
The information on the available
supply of licensed and unlicensed mariners is an estimate. Because the MMLD
also does not contain all of the information on individual mariners contained
in their paper records, and provides no information on the availability and
willingness of individuals to accept a sealift position in an emergency, it
does not provide sufficient assurance of mariner availability. |
|
Statistical Issues: |
None |
|
Verification &
Validation: |
The MARAD Regional Offices (and
contracted ship managers) monitor the condition and overall readiness of each
assigned RRF ship to meet its DOD mission.
When a ship is determined not capable of meeting its activation
timeframe (mission), it is given one of several vessel condition ratings that
are reported to DOD. The monthly
report contains an explanation of the deficiency and an estimated date when
the ship will become fully capable of meeting its mission. MSP contract performance is monitored
throughout the year in order to assure proper payment of the MSP payment to
the ship operators. Recently, MARAD
attempted to validate mariner availability estimates by conducting a survey
of the mariner population. A second
survey is expected to commence in April 2002 to refine and improve the
information needed to determine availability. Because the decision to serve is a matter of individual choice
and is subject to change, MARAD intends to develop a plan for maintaining
current information on mariner availability based on the results of the 2002
mariner survey. |
|
Comment: |
None. |
|
Measure: |
Percentage
of DOD-designated commercial strategic ports for
military use that are available for military use within DOD established
readiness timelines. |
|
Scope: |
The measure consists of the total number of DOD-designated
commercial strategic ports for military use that are assessed as
able to meet DOD-readiness requirements on 48-hour notice, expressed as a
percentage of the total number of DOD-designated commercial strategic
ports. Presently there are 14
DOD-designated commercial strategic ports. Port readiness is based on monthly reports submitted by the ports
and semi-annual port readiness assessments by MARAD in cooperation with other
NPRN partners. The MARAD/DOD
semi-annual port assessments provide data or other information on a variety
of factors, including the following: the capabilities of channels,
anchorages, berths, and pilots/tugboats to handle larger ships; rail access,
rail restrictions, rail ramp offloading areas, and rail storage capacities;
the availability of trained labor gangs and bosses; number and capabilities
of available cranes; long-term leases and contracts for the port facility;
distances from ports to key military installations; intermodal capabilities
for handling containers; highway and rail access; number of port entry gates;
available lighting for night operations; and number and capacity of covered
storage areas and marshalling areas off the port. |
Source:
|
MARAD data are derived from monthly
reports submitted by the commercial strategic
ports and from MARAD/DOD semi-annual port assessments.
|
|
Limitations: |
Port readiness assessments were not made prior to 1995; therefore,
data are available only for 1995 and later years. MARAD conducts a monthly survey of all strategic facilities to
determine whether they meet the DOD availability requirement. This information is provided to MARAD as a
self-assessment by the port agency that owns the facility. There is some degree of subjectivity in
determining the availability of the port facilities. As part of the overall planning process,
MARAD and DOD conduct semiannual visits to independently verify and reassess
port capability and availability. The
indicator is by definition a point-in-time judgment. The results of the monthly and semi-annual
reports used to measure port readiness can vary in accordance with the intensity
of commercial activity at a given port at the time of the assessment. Also, the monthly reports do not include
the same level of detail as the semi-annual assessments, although MARAD is in
continuous contact with port officials to minimize response error. |
|
Statistical Issues: |
The measurement of port
readiness is an overall measure derived from MTMC comments, monthly readiness
reports, and semi-annual assessments.
As such, it is a subjective measure. |
|
Verification &
Validation: |
The MARAD/DOD semi-annual
port visits independently verify and reassess not only the DOD-designated
facilities, but also the total capability of the commercial strategic port. |
|
Comment: |
None. |
|
Measure: |
Ship capacity (in twenty-foot container equivalent
units, or TEUs) available to meet DOD’s requirements for intermodal sealift
capacity. (FY) (2001) |
|
Scope: |
Includes the aggregate TEUs (or estimated square footage) of
cargo capacity for ships enrolled in the Maritime Security Program (MSP) and
Voluntary Intermodal Sealift Agreement (VISA). |
Source:
|
MARAD/USTRANSCOM database of the
militarily useful sealift capacity for ships enrolled in the MSP and VISA programs,
based on vessel capacity data obtained from the vessel operators.
|
|
Limitations: |
MARAD, DOD and operator data on vessel characteristics (e.g., deck
strength in pounds per square feet, deck height, container stowage factors),
which are used to determine the portion of a vessel suitable for carrying
military cargo, are not always consistent.
For example, the majority of ships in MSP/VISA are containerships,
which normally are measured in TEUs; however, DOD generally measures surge
sealift ships, most of which are Roll-on/Roll-off vessels, in square
feet. Historical data prior to FY
1997 are unavailable since the MSP and VISA programs were not enacted until
that year. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
MARAD works with DOD and
the maritime industry to use the most accurate information. MARAD validates the vessel capacity data,
which are obtained from the vessel operators, through comparisons with
internationally recognized databases of vessel characteristics (such as
Lloyd’s Register data), vessel trim and stability information, stowage plans
and other cargo loading documents. |
|
Comment: |
None. |
|
Measure: |
1. Percent
of RRF no-notice activations that meet assigned readiness timelines. (FY)
(2001) 2. Percent
of days that RRF ships are mission-capable while under DOD control. (FY 2001) |
|
Scope: |
DOD conducts no-notice exercises, called “Turbo-Activations,”
annually to assess RRF activation readiness.
The USTRANSCOM, via MSC, randomly selects and orders the activation of
a number of RRF ships on an annual basis to test their capability to be
ready-for-sea (i.e., mission-capable) within their assigned readiness
timeframes of 4, 5, 10, or 20 days. |
Source:
|
MARAD maintains a database on the
number of days it takes to activate each RRF ship and its operational
reliability. The MSC activation order
is received either by phone call or message.
Documents produced during the no-notice activation period comprise the
data source for determining the amount of time taken to activate each
ship. Non-performance time is based
on the MSC Casualty Reporting (CASREP) system, which identifies casualties
that are of a severity to prevent the ship from performing the mission. These messages are passed from the ship's
Captain to MSC, the Ship Manager, and MARAD.
The reliability of activated RRF ships is measured as the
percent of days that RRF ships are mission-capable while under DOD
control. Mission-capability is
determined, in part, by the number of days it takes to repair a ship that has
become inoperative. For example, the low percent of
mission capability in 1997 (95.2) was the result of one ship being out of
service for 156 days while undergoing repairs.
|
|
Limitations: |
None. |
|
Statistical Issues: |
Since the population of
vessels covered by these measures often consists of a very small number of
vessels (as low as 13 vessels in FY 2000), a large swing in results can occur
from just one ship not being available on time or one ship having operational
problems. |
|
Verification &
Validation: |
The source of the
activation data is the actual activation order from DOD to MARAD and the
documents produced during a no-notice activation. These fix the actual time of call-up and the time when the
vessel is "Ready for Sea" (or tendered to MSC). The Ready for Sea time is agreed to by
MARAD and the on-board MSC representative and reported to DOD by official
message. The time taken to activate
each ship is maintained in the ship’s logbook and in official DOD, MSC, and
MARAD records. The collection of data
regarding mission capability under MSC operational control starts when MSC
officially accepts delivery of RRF ships with date and time
documentation. The Captain of the
ship reports all problems that are of a severity to prevent the ship from
performing its mission to MSC, the Ship Manager, and MARAD. The Captain also reports when the problem
has been corrected. This information
is entered by MSC into its CASREP system. |
|
Measure: |
Of
the mariners needed to crew combined sealift and commercial fleets during national
emergencies, the percent of the total that are available. (FY) (2001) |
|
Scope: |
The pool
of licensed and unlicensed mariners consists of mariners that have had sea
service on U.S.-flag oceangoing merchant vessels over 1,000 gross tons within
five years. The mariner pool includes
licensed and unlicensed actively sailing mariners and inactive mariners,
employed shoreside, who have the necessary skills and retain the appropriate
license/rating to operate sealift ships, defined by shipboard position and
U.S. Coast Guard certification. This
pool is then compared to the DOD and commercial manpower requirements to
determine sufficiency of the labor force.
Only oceangoing merchant vessels over 1,000 gross tons are considered
because mariners on these vessels have skills required for emergency sealift
operations. The targets are based on
a sealift operation that extends beyond 6 months, necessitating relief for
the mariners who were sailing at the start-up of the operation. |
Source:
|
U.S. Coast Guard Merchant
Mariner Licensing and Documentation (MMLD) system. The Coast Guard is the lead Federal agency
for regulating, licensing, and documenting professional merchant
mariners. MMLD provides information
on both actively sailing mariners and inactive mariners, including their
skill level and seafaring employment. Lloyd’s Maritime
Information Systems. MARAD obtains information to
track the use of U.S.-flag commercial ships active in international trade and
projects the size of the active, ocean-going, commercial fleet. The size of this fleet has a direct
correlation to the size of the commercial pool of mariners, based upon
commercial crewing rules. MARAD/DOT Mariner
Survey. New for
FY 2001, a random sample of mariners with current qualifications is now being
surveyed, in conjunction with the Bureau of Transportation Statistics. The Survey will provide a more accurate
determination of the number of currently qualified mariners as well as
information on mariner availability for sealift employment during national
defense emergencies.
|
|
Limitations: |
The size
of the active and inactive mariner pool can be estimated from the MMLD. MARAD integrates these data into its own
system for analysis and reporting.
Because the MMLD does not contain all of the information on individual
mariners contained in their paper records, and provides no information on the
availability and willingness of individuals to accept a sealift position in
an emergency, it does not provide sufficient assurance of mariner
availability. |
|
Statistical Issues: |
The
primary area of uncertainty lies within the MMLD system, which MARAD uses to
define the population of available mariners.
While the accuracy of the data continues to improve as all licenses
and documents are now on a five-year renewal cycle, gaps still exist in the
database. Because the MMLD system was not designed to contact mariners,
address and telephone information in the system is incomplete and
out-of-date. Also, operators of some
large oceangoing vessels are not required to report mariner employment to the
USCG, and evidence of sea service provided by individual mariners to fulfill
requirements for upgrading their rating is not entered in the MMLD. |
|
Verification &
Validation: |
The MMLD
system is currently the only source of electronic data on mariner
qualifications and employment. MARAD
continues to work with the USCG to improve the MMLD system. The new MARAD/DOT Mariner Survey data will
be used to estimate the number of qualified mariners available and willing to
support sealift crewing positions.
Because this determination is a matter of individual choice and is
subject to change, MARAD intends to develop a plan for maintaining current
information on mariner availability based on the results of the Survey. |
|
Comment: |
None. |
|
Measure: |
Amount
of drugs seized or destroyed at sea (metric tons). (FY) |
|
Scope: |
Total amount of drugs (cocaine, marijuana,
hashish, heroin, etc.) seized, jettisoned, or destroyed at sea by the United
States Coast Guard. Cocaine currently
constitutes the largest drug threat to the U.S., but the Coast Guard seeks to
interdict all illegal narcotics moving by non-commercial maritime
conveyances. |
Source:
|
The amount of drugs seized is measured
by Coast Guard crews and reported through the Coast Guard Law Enforcement Drug Interdiction Data
Base. Seizures are officially credited
to the Coast Guard via Federal Drug Identification Numbers (FDINs) and are
recorded in the federal Consolidated Counter-Drug Data Base, which is
administered by the U.S. Interdiction Coordinator (USIC). |
|
Limitations: |
It is possible that non-entry,
duplication, and coding errors are present in seizure amount data; however,
the chance of this error is small. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
Verification and validation
occurs in several places in the data reporting and collection process. Data entry software helps ensure data
quality and consistency by employing selection lists and logic checks. Internal analysis and review of published
data by external parties help identify errors. CG data is further reviewed at a quarterly
Consolidated Counter-Drug Data Base Conference, where all agencies that input
data into the database review all agency data for consistency and accuracy.
|
|
Comment: |
This measure aligns with the goals contained in the President’s
National Drug Control Strategy. |
|
Measure: |
Seizure
rate for cocaine that is shipped through the transit zone. (FY) (2001) |
|
Scope: |
Seizure rate is a measure consisting of the
amount of cocaine seized by the Coast Guard divided by the noncommercial
maritime cocaine flow, expressed as a percentage. Noncommercial is defined as any vessel or aircraft not engaged
in port-to-port transfer of cargo/passengers for the primary purpose of
business profit. Examples are pleasure craft, fishing vessels, offshore
work-boats, or freighters carrying cargo as a means of disguising illegal
drugs. |
Source:
|
The amount of
cocaine flow shipped by non-commercial means through the transit zone is
estimated in the Interagency Assessment of Cocaine Movement (IACM) published
by the Office of National Drug Control Policy (ONDCP). The amount of cocaine seized is measured
by Coast Guard crews and reported through the Coast Guard Law Enforcement Information
System.
|
|
Limitations: |
It is probable that
non-entry, duplication, and coding errors are present in seizure amount data
(numerator); however, this error is likely to be small. The cocaine flow amount (denominator) is
estimated through a complex process using many different sources of
information. Due to the secretive nature of the illegal drug trade, cocaine
flow estimates may contain significant errors. The size of this error may vary from year to year; the extent
of this is not known. The estimation process changes slightly each year as
improvements are made, so year-to-year comparisons of the flow are not
completely consistent. The accuracy of the official cocaine flow estimate has
been questioned by some individuals and organizations outside of government
that have an interest in U.S. drug policy.
ONDCP continuously attempts to refine this estimate to improve the
measurement of interdiction activities.
This measure only addresses cocaine; formal flow assessments do not
exist for other major drugs. This
measure is not designed to include cocaine shipped by commercial means such
as large container vessels; however, it is probable that a small amount of
cocaine included in the numerator is actually related to commercial
shipping. This distinction between
commercial and noncommercial is better for program management; at-sea
interdiction of commercially conveyed cocaine, particularly when shipped in
containers, is extremely difficult, and not the focus of the Coast Guard
program. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating seizure rate for cocaine is the denominator. Although the numerator estimate of cocaine
seized is relatively accurate, the estimate of the amount shipped in the
denominator is far more variable. The regression standard error for
year-to-year chance variation in the cocaine seizure rate is 4.0 percent,
based on data from 1995 through 2000.
|
|
Verification &
Validation: |
Verification and
validation occurs in several places in the data reporting and collection
process. Data entry software helps
ensure data quality and consistency by employing selection lists and logic
checks. Internal analysis and review
of published data by external parties help identify errors. |
|
Comment: |
This measure is consistent with the goals contained in the
President’s National Drug Control Strategy. |
|
Measure: |
Interdict
and/or deter at least 87 percent of undocumented migrants who consider attempting
to enter the U. S. via maritime routes. (FY) |
|
Scope: |
Measure includes Cuban, Dominican, Haitian, and Chinese
migrants, as these are the primary groups using maritime channels and the
groups for which formal flow estimates exist. Success rate is the estimated number arriving by maritime
channels divided by those that pose a threat of migration (estimated
intent). The interdiction rate is
just 1 minus the success rate. |
Source:
|
Data is obtained from Coast Guard and from
the Immigration and Naturalization Service (INS). Estimates of migrants who successfully arrive and estimates of
those with a high potential for undertaking the voyage are derived (with a
consistent methodology) from investigations of incidents, interviews of
detainees, and intelligence gathering.
Sources for this information are the Coast Guard, INS, and other
authorities.
|
|
Limitations: |
The numbers of illegal
migrants entering the U.S., and the numbers of potential migrants, are
derived numbers subject to estimating error.
Because of the speculative nature of the information used, and the
secretive nature of illegal migration, particularly where professional
smuggling organizations are involved, the estimated potential flow of
migrants may contain significant error.
The measure only tracks four migrant groups at this time. A small
number of migrants (approximately 10%) from various source countries are not
included because formal flow estimates of migrants leaving these countries
are not available. Using the number
of potential migrants in the denominator helps address the deterrence value
of Coast Guard operations, but could lead to confusion of this measure with a
simple interdiction rate. Trend information prior to 1995 is not available. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating the success rate for undocumented migrants is the
denominator, which is an estimate of the flow of migrants, both documented
and undocumented. |
|
Verification &
Validation: |
The numbers of migrants
reaching the U.S. via maritime routes and the number of “potential migrants”
are estimated. Methodologies and data
are continuously reviewed. The Coast
Guard has developed the estimation techniques that support this indicator
over the last six years in order to more consistently use intelligence
information. They are seeking
independent assessment of the methods, and look to improve the process in the
future. |
|
Comment: |
Partly because maritime threats of illegal migration have come
from a limited number of sources, the Coast Guard and others have developed
quantified threat estimates to better manage interdiction. Over the past six years, estimation
techniques have been improved to remove as much subjectivity and inconsistency
as possible. It should be noted that past information reflects the success of
intentional illegal activity. While
some DOT measures allow accurate projection of likely future outcomes, the
highly variable nature of illegal migrant activity limits the ability to
project future outcomes based on performance in the immediate past. |
|
Measure: |
Of
those who need to act, percent who receive threat information within 24
hours. (FY) (2001) |
|
Scope: |
Threat information, in this context, is defined as credible
information (both time-sensitive/action-oriented and informational) received
by the Intelligence Community, analyzed by OIS and distributed in the form of
a Transportation Security Information Report, generated by OIS for
distribution by the Operating Administrations (OAs). Figure is derived from the percentage of
transportation security officials and industry representatives that receive
threat information from OIS through the OAs within the 24-hour period. Security representatives and officials
will be randomly sampled by OIS within 48 hours of information dissemination
and asked if and how soon they received the subject material. |
Source:
|
Internally prepared. Survey conducted by OIS of both DOT
personnel and industry security contacts.
|
|
Limitations: |
Data: Relies on the reporting of the customers
and consumers of this information.
Reporting could be skewed to reflect positively on the dissemination
process within the Operating Administrations. Indicator: This measure only identifies whether there
are possible breakdowns and bottlenecks in the dissemination process. It does not identify where those
breakdowns specifically may be in the dissemination chain. |
|
Statistical Issues: |
Since these data are
collected through a sample survey, they are subject to sampling and
non-sampling errors. |
|
Verification &
Validation: |
Customers will be randomly
surveyed at all levels within the dissemination process, not solely the end
users. Consequently, the reporting of
dissemination times and officials who are in receipt of the information can
be crosschecked for verification and validity of data. |
|
Comment: |
None. |
|
Measure: |
Transportation-related
petroleum consumption (in quadrillion BTUs) per trillion dollars of Real
Gross Domestic Product (GDP). (CY) (2001) |
|
Scope: |
Measure includes primary consumption of petroleum for
transportation, in quadrillion BTUs.
This does not account for petroleum-produced electricity that is used
in transportation; however, this is less than 1% of petroleum use. Petroleum use is normalized to real GDP,
in constant 1996 dollars. |
Source:
|
U.S. Department of Energy Annual Energy Review 1999 and Annual Energy Review 2000.
|
|
Limitations: |
Energy consumption does not include petroleum-produced transportation electricity. Measure does not capture the fraction of
this petroleum use that is imported, nor does it capture actual energy
efficiency (BTUs per
passenger-mile-traveled). |
|
Statistical Issues: |
These data are external
to DOT. They are subject to both
sampling and nonsampling errors.
Based on 1994-2000 data, chance variation from year to year in the
transportation energy efficiency measure has a regression standard error of
0.016. |
|
Verification &
Validation: |
Data is taken from external
sources, which conduct their own verification and validation. |
|
Comment: |
Petroleum use is normalized to the nation’s real GDP in order to
capture the nation’s economic exposure to petroleum use in
transportation. Beginning in 1999,
the GDP baseline was changed from constant 1992 dollars to 1996 dollars. |
Highway
infrastructure condition Page
60
|
Measure: |
Percentage of travel on
the National Highway System (NHS) meeting pavement performance standards for
acceptable ride. (CY) |
|
|
Scope: |
Data include vehicle miles traveled on the HPMS reported NHS sections
and pavement ride quality data reported using the International Roughness Index
(IRI). IRI is a quantitative measure of the accumulated response of a
"quarter-car" vehicle suspension experienced while traveling over a
pavement. Vehicle Miles of Travel (VMT) represent the total number of vehicle
miles traveled by motor vehicles on public roadways within the 50 states and
Washington, D.C. |
|
Source:
|
Data collected by the State Highway Agencies and reported to FHWA for
the Highway Performance Monitoring System (HPMS). They are obtained from calibrated measurement devices that meet
industry set standards. Measurement
procedures are included in the HPMS Field Manual. VMT is a calculated product of the annual average daily traffic
(AADT) and the centerline length of the section for which the AADT is reported.
In the HPMS, travel is accumulated for each universe section to develop
appropriate totals for the higher functional systems. AADT is required for
each section of Interstate, NHS, and other principal arterial; as a result,
travel is computed for these functional systems on a 100-percent basis. For
minor arterial, rural major collector and urban collector systems, travel is
calculated from samples using the AADT, centerline length reported for each
sample section and the HPMS sample expansion factor for each section. Travel
for the NHS on all functional systems is computed from the universe AADT
data. For the most part, travel for the rural minor collector and
rural/urban local functional systems is calculated by the States using their
own procedures and is provided in HPMS on a summary basis. Some States use
supplemental traffic counts outside of the HPMS procedures; others employ
estimating techniques, such as fuel use, to determine travel on these
systems. In general, these methods are used in both rural and urban areas,
including the donut areas of nonattainment areas to meet Clean Air Act
requirements. |
|
|
Limitations: |
IRI
data for the approved NHS exist from 1995 onward. Past data (1993 and 1994)
contain some variation as this data was on the proposed, rather than the
existing NHS. No NHS IRI data are available prior to 1993. The HPMS requires States to report IRI
data every two years; however, following the requirements is not mandated,
but voluntary. VMT
estimates reported via the HPMS should be of reasonable quality particularly
for the higher order functional systems. AADT and travel data are edited by
the HPMS software for unusual values and for unusual changes to previously
reported values. FHWA routinely works with State data providers to modify
reported AADT values that do not appear to be reasonable before final use.
Although AADT is required to be updated annually in HPMS, counts are only
required to be updated on a 3-year cycle. For any reporting year, AADT for
uncounted sections is usually derived by factoring the latest year's count
for those sections. |
|
|
Statistical Issues: |
The
major source of error in the percentages is probably the sampling error from
selecting the segments of highway tested for smoothness. VMT
data are subject to sampling errors, whose magnitude depends on how well the
locations of the continuous counting locations represent nationwide traffic
rates. HPMS is also subject to
estimating differences in the states, even though FHWA works to minimize such
differences and differing projections on growth, population, and economic
conditions which impact driving behavior. |
|
|
Verification &
Validation: |
FHWA
validates the data based on consistency reviews. States that follow the HPMS
sampling instructions in developing traffic counting programs (Appendix F in
the HPMS Field Manual) and the practices advocated in the Traffic Monitoring
Guide have adequate counting and classification tools to prepare quality AADT
and travel estimates for HPMS. The consistency of the sampling and counting
procedures should also provide comparable State-to-State traffic data. |
|
|
Comment: |
None. |
|
|
Measure: |
Percentage of miles on
the National Highway System (NHS) that meet pavement performance standards for acceptable ride. (CY) (2001) |
|
|
Scope: |
International Roughness Index (IRI) is compiled annually for every
section of the NHS, using data reported from the States. |
|
Source:
|
Data collected by the State Highway Agencies and reported to FHWA for
the Highway Performance Monitoring System (HPMS). They are obtained from calibrated measurement devices that meet
industry set standards. Measurement
procedures are included in the HPMS Field Manual. |
|
|
Limitations: |
IRI
data for the approved NHS exist from 1995 onward. Past data (1993 and 1994)
contain some variation as this data was on the proposed, rather than the
existing NHS. No NHS IRI data are available prior to 1993. The HPMS requires States to report IRI
data every two years; however, following the requirements is not mandated,
but voluntary. |
|
|
Statistical Issues: |
The
major source of error in the percentages is probably the sampling error from
selecting the segments of highway tested for smoothness. The annual variation in the percentage due
to chance has a regression standard error of approximately 0.44 percent based
on data from 1995-2000. |
|
|
Verification &
Validation: |
FHWA
validates the data based on consistency reviews |
|
|
Comment: |
None. |
|
|
Measure: |
Percentage
of deficient bridges on the NHS. (CY) (2001) |
|
Scope: |
Measure includes the number of deficient (structurally deficient
and functionally obsolete) bridges on the NHS functional system divided by
the total number of NHS bridges in the inventory, expressed as a
percent. |
Source:
|
Bridge information is collected by
State DOTs and other bridge owners and provided to FHWA annually for
inclusion in the FHWA maintained National Bridge Inventory (NBI).
|
|
Limitations: |
NBI includes information
on all 114,567 NHS bridges. States are
required to update the system annually, but many States update
quarterly. The system contains 95
data items for each of the bridges, and 20 of these items relate to bridge
condition and appraisal. There are
specific instructions as to how to assess bridges based on these items,
including a grading scale from 0 to 9 with specific definitions and specific
criteria to follow |
|
Statistical Issues: |
Even with the item
specific grading system, differences in the grading between individual
inspectors and between inspection days are probably the largest component of
variation in the percentages. Based
on 1994-2000 data, the estimated regression standard error for year-to-year
variation in the percentages due to chance is approximately 0.65 percent. |
|
Verification &
Validation: |
DOT evaluates accuracy
and reliability of the submitted NBI information through data checks and
field reviews by both Headquarter and field office personnel. This is done as a part of FHWA’s NBI, the
National Bridge Inventory System (NBIS), and Highway Bridge Replacement and
Rehabilitation Program. Evaluation of
the State’s compliance with the NBIS most often includes a sample of bridge
inspection reports and a comparison of condition data with field visits to
the bridge site. In addition, there
is an edit update program that identifies potential data errors in the NBIS. |
|
Comment: |
None. |
|
Measure: |
Miles
of the Appalachian Development Highway System (ADHS) completed. (FY) (2001) |
|
Scope: |
Measure includes actual miles completed on the 3,025 mile ADHS,
within 13 member States. |
Source:
|
States submit annual status updates on
ADHS miles completed within their State each fiscal year to the Appalachian
Regional Commission (ARC). The ARC
compiles the data.
|
|
Limitations: |
This is an output
measure. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
Completed by ARC. |
|
Comment: |
ARC estimates that the TEA-21 funding level will result in
completion of approximately 37 additional miles each FY 1999 through 2003. |
|
Measure: |
Number
of metropolitan areas where integrated ITS infrastructure is deployed. (FY)
(2001) |
|
Scope: |
The level of integrated deployment in 75 of the nation’s largest
metropolitan areas has been established using a set of indicators that
consider two factors: (1) How much ITS infrastructure is in place at each
metropolitan area; and, (2) How much integration is going on at each
area. The process for determining the
level of “component” deployment in a metropolitan area employs a set of
indicators that measure the magnitude of deployment for selected ITS
components. These are typically
expressed as a ratio of actual deployment divided by the total possible, for
example the number of freeway miles under electronic surveillance divided by
the total freeway mileage. Components
are considered deployed once the level of deployment attains a specified
threshold level based on the indicators.
Integration is defined as the sharing of data between agencies
associated with the different jurisdictions responsible for ITS
infrastructure. Typically there are
three: State DOTs responsible for management of freeways and incident
management programs; city governments, which manage most of the traffic
signal systems; and public transit authorities, which manage most bus and
rail services. The level of
integration is determined by the extent that these three major transportation
organizations employ technology to share and use transportation data to
increase system capacity. Two
examples of integration are: 1) a city traffic signal system receiving data
from the state freeway management center about the queues at freeway ramp
meters and then adjusting the signal timings on the arterial streets, or 2) a
transit agency providing the state freeway management center with the
real-time location of the buses so that freeway speeds can be determined. Metropolitan areas are rated as low,
medium, or high separately for deployment and integration and then assigned
an overall combined rating. An
overall score of medium or high meets the goal for a metropolitan area. |
|
Source: |
Metropolitan ITS
Deployment Tracking Database developed by the Oak Ridge National Laboratory
for the ITS Joint Program Office.
Data are collected by means of surveys from designated metropolitan
areas. |
|
Limitations: |
This indicator is designed
to track and encourage basic steps toward component deployment and systems
integration. However, it does not
reflect the full breadth of deployment or integration activities. For example, while it establishes the
existence of basic integration of essential components, it does not confirm
that all possible or desirable integration links exist in a metropolitan
area. Similarly, the attainment of a
deployment threshold only confirms a substantial commitment to the use of ITS
technology but does not indicate that all needed deployment is complete. |
|
Statistical Issues: |
These data come from
sample surveys that, like all sample surveys, contain sampling and
non-sampling errors. |
|
Verification & Validation: |
The DOT Joint Program
Office reviews deployment tracking indicators and methodology. Results are distributed to DOT
headquarters and field staff as well as to state and local survey responders
for confirmation of accuracy and completeness before the final reports are
issued. Independent experts in
statistics and transportation review procedures for survey construction and
data collection prior to each survey iteration. A steering committee of Federal, state, and local
transportation officials review and approve changes to methodology and indicators
prior to implementation. |
|
Comment: |
The FY 1997 baseline is
36 areas. |
|
Measure: |
1. Billion transit passenger miles traveled.
(CY) (2001) 2. Average percent change in transit passenger-miles
traveled per transit market. (FY) |
|
Scope: |
Includes revenue-passenger miles on publicly sponsored bus,
transit rail, commuter rail, ferry, and vanpools in urbanized areas. |
Source:
|
National Transit Database
(NTD), with information gathered from transit operators. |
|
Limitations: |
Data is self-reported by
transit agencies using an FTA-approved sampling methodology. Although most data is reported in the
National Transit Database each year, sample cycles may be annual, every three
years, or every five years depending on the size of the urban area and the
number of vehicles operated.
Ridership is an outcome indicator that reflects a variety of factors,
including the capital investment of the Federal Government. Ridership is also influenced by
operational decisions of transit authorities, and the availability and cost
of alternative modes of transportation. |
|
Statistical Issues: |
The sources of uncertainty include sampling error, annual chance variation,
and auditing issues. The regression
standard error from 1994-2000 indicates that the magnitude of the combination
of the first two sources of error is approximately 0.67. |
|
Verification &
Validation: |
An independent auditor and the transit agency’s CEO certify that data
reported to the NTD are accurate. FTA
also compares data to key indicators such as vehicle revenue miles, number of
buses in service during peak periods, etc. |
|
Comment: |
None. |
|
Measure: |
Percentage
of transit grants obligated within 60 days after submission of a completed
application. |
|
Scope: |
FTA grants
obligated during a fiscal year period for major programs: Urbanized area,
non-Urbanized area, and Elderly and Persons with Disabilities formula grants;
Capital grants; Job Access and Reverse Commute grants; Over-The-Road Bus
grants; and Planning grants. |
Source:
|
FTA TEAM database. |
|
Limitations: |
Several factors that contribute
to grant delays are beyond FTA’s ability to control. These factors include
the processing of flexible funds from FHWA through the Treasury, and the
Congressional grant release process. |
|
Statistical Issues: |
Processing time is calculated from submission date to obligation
date. $0 dollar non-funding grant amendments are excluded from analysis. |
|
Verification &
Validation: |
TEAM output file is crosschecked against other system generated files
for consistency; inconsistencies are investigated and corrected prior to
reporting. Grants with missing or out-of-sequence dates are excluded for
calculating averages. |
|
Comment: |
An FTA task force meeting was held in February 2002 to identify causes
for grant processing delays. The resulting action plan is now being
circulated for final review and approval. Implementation of the plan will
follow. |
|
Measure: |
Percentage
of on-time flights. (FY) |
|
Scope: |
The time of arrival of
completed, scheduled passenger flights to and from the 32 DOT large-hub
airports is compared to their scheduled time of arrival. The sum of flights arriving on or before
15 minutes of scheduled arrival time is divided by the total number of
completed flights. |
Source:
|
The Aviation System Performance Metrics (ASPM) database,
maintained by the FAA’s Office of Aviation Policy and Plans. By agreement with the FAA, ASPM flight data
is filed by certain major air carriers for all flights to and from 21 large
and medium hubs, and is supplemented by flight records contained in the
Enhanced Traffic Management System (ETMS) and flight movement times provided
by Aeronautical Radio, Inc. (AIRINC).
Data are sufficient to complete ASPM data files for 49 airports. |
|
Statistical Issues: |
There is little major error in the count of completed
flights or the count of flights that arrive on-time. |
|
Limitations: |
Some
ASPM data is constructed from ETMS records, a small portion of which may not
be maintained in FAA traffic control computers when they are under heavy use.
|
|
Verification &
Validation: |
Flight
data is extracted from the Official Airline Guide (OAG) and compared
to& data from carrier records, which contains carrier computer
reservation flight schedule data.
Summary data is compared to data filed monthly with DOT under 14 CFR
Part 234, Airline Service Quality Performance Reports, which
separately requires reporting by major air carriers on flights to and from
the 32 large hubs. |
|
Comment: |
FAA’s percentage of flights
arriving on-time derived from ASPM data differs only by fractions of a
percent from the on-time percentage derived from DOT’s slightly different
database. |
|
Measure: |
Aviation
delays per 100,000 activities. (FY) (2001) |
|
Scope: |
An FAA reported
delay occurs when an aircraft is delayed fifteen minutes or more because of
constraints that prevent the aircraft from making a scheduled landing. Delays are counted in five categories: FAA
equipment, volume, weather, runway related, and other. Delays due to airline equipment are not
considered. “Activities” are total facility activities, as defined in
Aviation System Indicators 1998 Annual Report. Total facility activities are the sum of en route and terminal
facility activities. |
Source:
|
FAA air traffic facilities report the data to
headquarters, which incorporates the data into the Air Traffic Operations
Management System.
|
|
Statistical Issues: |
There is no major error in either the delay counts (numerator) or the
flight operations data (denominator) for this rate. However, random variation in aviation delays results in a
significant variation in the delay rate from year to year. The regression standard
error in the delay rate, based on 1994-2000 data, is approximately 20.0. |
|
Limitations: |
By collecting information
on delays of fifteen minutes or more, FAA does not capture the aggregate amount
of system delay, but only the most significant delays. |
|
Verification &
Validation: |
Data is analyzed and
checked by an Air Traffic Service headquarters office on a daily basis to
ensure accuracy of the information reported. |
|
Comment: |
Total delays in
all five categories are what the traveling public experience. |
|
Measure: |
Cumulative
increase in throughput during peak periods at certain major airports. (FY)
(2001) |
|
Scope: |
This measure
focuses on the arrival rates during peak traffic periods comparing pre-CTAS
rates to post CTAS rates. |
Source:
|
Radar system (HOST and ARTS) data is collected and aircraft
flight tracks are obtained from those systems and analyzed to determine
arrival and departure times.
|
|
Limitations: |
The radar systems produce
very large data files requiring a substantial effort to extract relevant data
for analysis. The extracted data sets need to be of sufficient size to
produce statistically significant results. |
|
Statistical Issues: |
Conditions (weather,
runways in use, aircraft mix) vary, affecting rates. Data must be normalized
and data sets must be of sufficient size to produce valid results. |
|
Verification &
Validation: |
Methodologies and
detailed results are available for review in semi-annual FFP1 Metrics Updates
(December and June). Results are
coordinated with FAA and User stakeholders. |
|
Comment: |
None. |
|
Measure: |
Cumulative
increase in direct routings for en route flight phase. (FY) (2001) |
|
Scope: |
This measure
focuses on the number of direct routings provided by en route controllers
comparing pre and post-URET installation. |
Source:
|
URET provides data on routing amendments, which is then
analyzed to determine the number of direct amendments.
|
|
Limitations: |
The radar systems produce
very large data files requiring a substantial effort to extract relevant data
for analysis. The extracted data sets need to be of sufficient size to
produce statistically significant results. |
|
Statistical Issues: |
Extreme weather
conditions, particularly thunderstorms, will significantly affect routing
amendments; therefore, data is sampled for days when weather is not a factor. |
|
Verification &
Validation: |
Methodologies and
detailed results are available for review in semi-annual FFP1 Metrics Updates
(December and June). Results are
coordinated with FAA and User stakeholders. |
|
Comment: |
None. |
|
Measure: |
Percent
of runways in good or fair condition (commercial service, reliever, and
selected general aviation airports). (CY)
(2001)
|
|
Scope: |
Paved runways at the 3,300+ airports in FAA’s National Plan of
Integrated Airport Systems (NPIAS) are assessed for pavement condition. The NPIAS airports include all commercial
service and reliever airports and those general aviation airports that are
significant to national air transportation. |
Source:
|
The FAA’s Airport Safety Data Program
(ASDP) provides extensive data about the facilities that are available at
public-use airports. Data are provided
approximately annually by FAA inspectors for airports certified under FAR
139. Data for other airports,
including most public use general aviation airports, are provided under an
FAA contract.
|
|
Limitations: |
FAA contracts for a visual survey of the runways to categorize their condition based on criteria developed by the FAA Office of Airports. “Good” condition means all cracks and joints are sealed; “fair” condition means there is mild surface cracking, unsealed joints, and slab edge spalling; and “poor” condition means there are large open cracks, surface and edge spalling, and vegetation growing through cracks and joints. Since the reports are based on a visual inspection, underlying drainage or strength problems are not reported. However, these problems normally create surface defects that are visible. The more detailed pavement condition index (PCI) inspections require a section-by-section examination of the runway rather than an overall assessment used for this performance measure. FAA has been aggregating the ADSP data from all NPIAS airports only every several years for inc |