Performance Data and
Performance Measurement
Performance measurement is dependent on the availability of useful data. Useful data will indicate level of performance and progress toward organizational goals. All data are imperfect in some fashion. Pursuing “perfect” data, however, may consume public resources without creating appreciable value. For this reason, there must be an approach that provides sufficient accuracy and timeliness but at a reasonable cost. This section of the Performance Plan/Performance Report provides information on how DOT reports on performance, verifies and validates data, assesses limitations of the data, and plans for improving DOT’s data.
In an attempt to bring consistency and quality to
its performance reporting, DOT has implemented some general rules regarding the data it uses and how
it is evaluated.
Annual
data – Whenever
available, the data in this document are reported on a Federal Government
fiscal year basis. However, there are instances where this is not possible so
calendar year data are used instead. This
often occurs when data are collected and reported to DOT by external sources
and a calendar year reporting requirement is specified in the implementing
regulation. The reporting timeframe (FY or CY) for each measure is included in
the Data Details in Appendix I.
Annual
results – If
available, the results for the most recent year in the Report are listed as
“Actual” in the Performance Goals & Results box for each performance
measure. However, given the March deadline for submission of the Performance Report,
quite often data have not been compiled and finalized for the entire year. When
this occurs and an actual value is not available for the current year, either
an estimate or projection is provided instead. In general, estimates are based
on partial year data that are extrapolated to cover a full 12-month period. For
example, if six months of data are available, they will be compared to prior
years for the same six-month period to determine any variation from past
levels. Historical trend information, supplemented by program expertise, will
then be applied to estimate the remaining six months of performance. The result
will be identified as a “preliminary estimate” in the Report. If partial year
data are not available, then past trend information will be analyzed and
supplemented by program knowledge to develop a projected value for the annual
performance measure. The result will be identified as a “projection” in the
Report. As data are finalized, the projections and preliminary estimates will
be replaced by actual results. Results
may be amended as errors and omissions are identified in the data verification
process, because updated information is provided by the reporting sources, or
because of legal or other action that changes a previously reported value. For
example, updated pipeline spill reports may change the status of a previously
reported value used in performance measurement.
In
measuring progress toward the majority of performance goals, DOT is moving to a
system of monthly performance measurements.
This will make it much easier to internally gauge periodic progress
toward goals as the year progresses, and will enable more timely performance
reporting after the years’ end.
Completeness
of Data – As
described above, actual data and “preliminary estimates” incorporate complete
or partial data from 2001. Results
listed as “projections” are not based on data from 2001, but on trend data from
prior years.
Reliability
of Measurement Data
– Because performance results in a given year are influenced by multiple
factors, some of which are beyond DOT’s control, and some of which are due to
random chance, there may be considerable variation from year to year. (See
discussion in Appendix I.) A better “picture” of performance may be gained by
looking at results over time to determine if there is a trend. Therefore,
graphs are provided for each measure showing trend lines back to 1990, or as
many years as possible if data are not available back to 1990. Additionally, a
table is included at the beginning of each strategic goal section giving the
available data from 1995 through 2001 for measures with performance goals
specified for 2001.
Integral
to performance measurement is understanding data limitations, addressing these
limitations where necessary and cost-effective, and acknowledging those that
remain when interpreting results. This section on verification and validation
provides a DOT-wide overview of our plan for assessing the quality of the data
DOT uses to measure its performance, and follows the GAO definitions for
verification and validation:
“Verification
is the assessment of data completeness, accuracy, consistency, timeliness, and
related quality control practices.”
“Validation
is the assessment of whether data are appropriate for the performance measure.”
Virtually
all data have errors. In Appendix I we have provided the following information
about the data used for each performance measure: source of the data,
limitations of the data, observations about the quality of the data, work
planned or ongoing to improve data quality, and any known biases.
Additionally,
we have compiled Source and Accuracy Statements for each of the DOT data
programs used in this report, which can be found at www.bts.gov/statpol/SAcompendium.html.
The Source and Accuracy Statements give more detail on the methods used to
collect the data, sources of variation and bias in the data, and methods used
to verify and validate the data.
By
validating data used in the DOT performance plan, we are ensuring that those
data are reflective of the phenomena they purport to measure. The Office of the
DOT Inspector General (OIG) plans to selectively verify and validate
performance measurement data each year. When pertinent to the conduct of
ongoing projects, OIG will also assess performance measures to determine their
appropriateness for measuring progress toward stated goals. These assessments
may lead to changes in the goals, improvements to or additions of data
collection systems, or both.
Assessing
and, where possible, eliminating sources of error in DOT data collection
programs has always been an important task for data program managers. As a part
of their ongoing work, managers of Departmental data programs use quality
control techniques, such as flowcharting the data collection process, to
identify where errors can be introduced into the data collection system.
Program managers also use computerized edit checks and range checks to minimize
errors that may be introduced into the data of their respective programs. In
addition, quality measurement techniques are employed to measure the effects of
unanticipated errors. These include verification of data collection and coding,
as well as coverage, response and non-response error studies to measure the
extent of human error affecting the data. As sources of error are identified,
steps are initiated to improve the data collection process.
The
data used in measuring performance come from a wide variety of sources. Much of
the data originates from sources outside the Department and, therefore, outside
the direct control of the Department. The data often come from administrative
records or from sample surveys. While DOT may not have a strong voice in
improving the quality of outside data, the Department takes all available
information about the limitations and known biases in outside data into account
when using the data.
The
myriad data sources make the task of assessing and, where possible, eliminating
error a challenging one for DOT. Different data systems contain different types
of errors. For example, data from administrative records systems may have
missing or incorrect records, and data from sample surveys will contain
sampling error.
Several
measures (particularly in safety) require aggregation across transportation
modes. This can be particularly problematic because of the use of different
definitions in different transportation modes. Also, data from outside the
Department may have unknown error properties.
To
help the operating administrations address these issues, the Bureau of
Transportation Statistics (BTS) is developing a statistical policy framework
where the operating administrations will work together to identify and
implement the current statistical “best practices” in all aspects of their data
collection programs. This project is consistent with the data capacity
discussions found in the DOT Strategic Plan.
In
2001, a DOT intermodal working group addressing DOT data quality issues
continued to:
▪
develop
Departmental statistical standards;
▪
update Source and Accuracy Statements for all DOT data programs to
document limitations and known errors and biases;
▪
improve
quality assurance procedures;
▪
evaluate
sampling and non-sampling error; and;
▪
develop common definitions for data across modes.
BTS's statistical staff is
consulting with the DOT operating administrations’ data program managers to
assist in data evaluation and validation, documenting data sources, and
determining the reliability of performance measurement estimates.
Departmental data systems
managers use these data verification methods:
▪
Comparisons with previous data from the same source.
▪
Comparisons with another reliable source of the same type of data
within DOT for the same time period.
▪
Comparisons with another reliable source of the same type of data
within DOT for a previous time period.
▪
Comparisons with another reliable source of the same type of data
outside DOT for the same time period.
▪
Comparisons with another reliable source of the same type of data
outside DOT for a previous time period.
In addition to
computerized edit checks and clerical review procedures to look for outliers,
duplicate records, and data inconsistencies, data managers also verify data
quality at each step of the data collection process using these procedures:
▪
Re-collecting/re-interviewing all (or a sample of) records and
reconciling with the original collection. (This applies to census or sample
survey data collections from administrative records, organizations, or
individuals.)
▪
Conducting 100 percent (or a sample of) data re-coding and
reconciliation to assess and correct coding errors.
▪
Conducting 100 percent (or a sample of) data re-entry and
reconciliation to assess and correct data entry errors.
The American Travel Survey’s re-interview program,
in which a sample of households were re-contacted and differences reconciled,
is an example of a verification system within a data collection program.
DOT
Data Source Limitations – Timeliness is the most significant limitation for DOT performance
measurement data. Some DOT data are not collected annually. For example, the
National Household Travel Survey and the Commodity Flow Survey each collect
data every five years. Data that are
collected each year (or more frequently) require time to analyze, confirm and
report results. For example, Highway Performance Monitoring System
vehicle-miles traveled (VMT) data require several months of post-collection
processing, making final results unavailable for this performance report.
Other
performance measurement data limitations can be found in the previously
mentioned Source and Accuracy Statements for DOT data programs. These
statements contain descriptions of data collection program design, estimates of
sampling error (if applicable), and discussions of non-sampling errors.
Non-sampling errors include under-coverage, item and unit non-response,
interviewer and respondent response error, processing error, and errors made in
data analysis.
As
part of its mandate in the Intermodal Surface Transportation Efficiency Act of
1991 (ISTEA), the Transportation Equity Act for the 21st Century (TEA-21), and
its plans for a statistical policy framework in the Department, BTS is working
on a program of research, technical assistance, and data quality enhancement to
support the continued improvement of data programs in DOT. This will help data
program managers throughout DOT improve data quality and better document known
data limitations. BTS also assists operating administrations with data
collection and documentation.
Many of DOT’s internal data programs
rely on State DOTs to collect reliable statistics within cost constraints.
While we work closely with our State DOT partners, we do not have direct
control over these data.
External
Data Source Limitations – Timeliness is also a significant limitation for external or
third-party data. Other limitations of external data are noted in the comments
for each performance measure in Appendix I. In some cases, DOT has replaced
external data, where little is known about the quality of the data, with
internal data. For example, DOT has used estimates of person-miles traveled
(PMT) from private organizations, absent any better estimate. The 1995 Nationwide
Personal Transportation Survey and American Travel Survey give DOT data with
known error properties that allow a better estimate of PMT.
The
DOT Strategic Plan 2000 – 2005 identifies data needs for each of the Department’s
strategic goals. They include:
Safety – DOT is undertaking major efforts
over the next several years to improve safety data. Safety has always been our
primary strategic goal, and in 1999 DOT created a Safety Data Action Plan to
better organize data improvement efforts. BTS will lead efforts to: 1) develop
common criteria for reporting injuries and deaths; 2) develop common data on
accident circumstances; 3) improve data quality; 4) develop better data on
accident precursors; 5) expand the collection of near-miss data to all
transportation modes; 6) develop a variety of common denominators for safety
measures; 7) advance the timeliness of safety data; 8) link safety data with
other data; 9) explore options for using technology in data collection; and 10)
expand, improve and coordinate safety data analysis.
Homeland
Security –
Existing performance data sources are generally good, but DOT will collect data
to better understand the transportation system’s vulnerability to intentional
acts of disruption or destruction.
Mobility – All mobility outcomes present
complex measurement issues. Accordingly, DOT will: 1) develop ways of measuring
user transportation cost, time, and reliability with time-series data; 2)
develop better approaches for measuring access; 3) develop straightforward
measures of congestion and its costs; 4) produce more timely and comprehensive
data on the condition and use of the transportation system; and 5) develop a
more complete understanding of variables influencing travel behavior.
Economic
Growth – DOT needs
aggregate data for measuring the productivity, effectiveness and efficiency of
the U.S. transportation system. We plan to collect, analyze and disseminate
data and information that identify critical trends and issues relating to
transportation’s nexus to the U.S. economy. DOT will: 1) develop a means of
measuring transportation cost, time, and reliability – at an aggregate level –
with time-series data; 2) develop a comprehensive measure of the transportation
capital stock; 3) improve our view of changes in the transportation workforce;
4) develop better measures of productivity in the transportation sector, and
other issues concerning use of Producer Price Indices; and 5) develop a better
picture of transportation-related variables influencing U.S. competitiveness in
the global economy.
Human
and Natural Environment – DOT will: 1) develop comparable and complete data on transportation
emissions, noise, hazardous materials releases, and wetlands impacts; 2)
improve our understanding of collateral damage to the human natural
environment; 3) create better leading indicators for potential environmental
issues; and 4) develop a reliable method of measuring the environmental
benefits of bicycling and walking.
Each
table includes a description of a performance measure and associated data
provided by the agencies in charge of the measure. The Scope statement gives an overview of the data collection strategy
for the underlying data behind the performance measure. The Source statement identifies the databases
used for the measure and their proprietary agencies. The Limitations statement describes some of
the shortcomings of the data in quantifying the particular performance characteristics
of interest. The Statistical Issues
statement has
comments, provided by the Bureau of Transportation Statistics (BTS)
and the agency in charge of the measure, that discuss variability of the measure
and other points. The Verification and Validation statement indicates steps taken
by the proprietary agencies to address data quality issues.
DOT
feels strongly that full compliance with the Government Performance and Results Act
requires impartial reporting of the statistical uncertainty associated with
numerical performance measures. A
portion of this uncertainty is related to the methodology used to calculate the
performance measure and the accuracy of the underlying data. For example, the use of samples introduces
uncertainty because estimates are used in lieu of actual counts. Also, there may be errors in the data
collected. However, there are many
other sources of variation (e.g., nonsampling errors, climate effects, new
technology) and they are often difficult to quantify. Nonetheless, a combination of past data and expert judgment can
enable uncertainty statements that are order-of-magnitude correct for even the
most difficult problems.
The
standard error of a performance measure indicates the likely size of the chance
variation in the reported number. It
incorporates both the effects of measurement error, survey error, and so forth,
as well as the variation that occurs naturally from year to year (i.e., even if
there were no change in laws, infrastructure conditions, or human behavior,
there would still be chance variation in an annual count of fatalities). DOT success in meeting GPRA goals must be
viewed in the context of this background noise.
In
many of the following Statistical Issues statements, BTS refers to regression
standard error. This is a modification
of the standard error to take into account of linear trends in the
recent past. Such adjustment is
generally needed to incorporate consistent trends due to cumulative effects of
such things as education programs, changing demographics, the gradual adoption
of new technologies, and so forth. The
underlying assumptions are that: over a short time period the trend of the
measurement data is linear; for any given year the performance measure values
are normally distributed; and the standard deviation is the same for all
years. We believe that these
assumptions lead to a conservative estimate of variability.
The
regression standard error is an estimate, calculated from the annual
performance results, of this common standard deviation. It may be used in the same way as a regular
standard error to set confidence intervals or describe uncertainty. For the purposes of performance measurement,
it may be considered a rough approximation of the annual variability in a
measure, and it will include the affects of program initiatives, influences
beyond the control of DOT (e.g., weather, petroleum prices, etc.), random
chance, and errors inherent in the data.
For
further information about the source and accuracy (S&A) of these data,
please refer to the BTS S&A compendium available at
www.bts.gov/statpol/SAcompendium.html
|
Measures:
|
1. Transportation
fatalities. (CY) 2. Fatalities
per 100 million passenger-miles. (CY) 3. Fatalities
per 100 million ton-miles of freight. (CY) 4. Transportation
injuries. (CY) 5. Injuries
per 100 million passenger-miles. (CY) 6. Injuries
per 100 million ton-miles of freight. (CY) 7.
Transportation incidents. (CY) |
|
Scope: |
This family of measures
aggregates fatalities, injuries and incidents across all modes of transportation
(air, highway, railroad, transit, waterborne and pipeline). The fatality and injury rates per 100 million
passenger-miles exclude pipeline fatalities and injuries due to minimal
interaction with passenger miles.
Highway-rail grade crossing fatalities and injuries are not counted
since they are included in data for highways.
The
fatality and injury rates per 100 million ton-miles of freight include
fatalities and injuries from large truck, rail, waterborne and pipeline
transportation. Highway-rail grade
crossing fatalities and injuries are also included since these involve
freight transportation-related fatalities and injuries that would not
otherwise be counted. Ton-miles of
freight covers intercity truck, rail, water and oil pipeline transportation. Aviation fatalities, injuries and
ton-miles are excluded because the fatality and injury data are not separated
from passenger air carriers. Transportation incidents include crashes, system
failures, spills, releases, and other accidents of a similar nature. |
|
Source: |
The data for these measures are
obtained from National Transportation Statistics published annually by the
Bureau of Transportation Statistics.
Information is taken from the following tables: Transportation Fatalities
by Mode; Injured Persons by Transportation Mode; U.S. Passenger-Miles
(Millions); U.S. Ton-Miles of Freight (Millions); and Transportation
Accidents by Mode. The one exception
is the data on large truck fatalities and injuries used for calculating
fatality and injury rates per 100 million ton-miles of freight are obtained
from the Federal Motor Carrier Safety Administration.
|
|
Limitations: |
Double counting of fatalities and
injuries may occur when an accident involves more than one mode of
transportation. Differing definitions
of injuries or transportation-related fatalities makes comparison across
modes of transportation problematic.
Highway injuries and incidents are obtained from a nationally
representative probability sample and are estimates, while the totals for
other modes of transportation are actual counts. The highway estimates are based on crashes where a police
accident report was completed and the crash resulted in property damage,
injury or death. Accidents that were
not reported to the police or did not result in property damage are not
included. Highway passenger miles are
calculated by multiplying vehicle-miles of travel (VMT) by the average number
of occupants for each vehicle type.
VMT is based on a nation-wide sample of vehicle travel. The average number of vehicle occupants
comes from survey information.
Therefore, vehicle passenger miles is an estimate, whereas
passenger-miles for other modes of transportation are calculated based on
actual passenger counts and recorded trip lengths. |
|
Statistical Issues: |
All fatality totals, and the injury and incident
numbers where actual counts are recorded, are relatively accurate. Any double counting or omissions are
expected to be fairly small. The
primary source of uncertainty in these measures comes from sampling and
survey errors related to estimation of highway injuries, incidents, VMT and
vehicle occupancy. Based on data from
1994-2000, the annual variations in the transportation safety measures are as
follows: the regression standard
error for the number of
transportation fatalities is 0.5 thousand.
For fatality rates by passenger-miles and ton-miles, it is 0.010 and
0.007, respectively. For number of
injuries, it is 0.10 million. For
injury rates by passenger-miles and ton-miles, it is 2.50 and 0.24,
respectively. For incidents, it is
0.16 million. |
|
Verification &
Validation: |
BTS compiles the data for the National
Transportation Statistics from information it gathers directly in its own
data systems (e.g. airlines information), information published by other
sources (e.g. FHWA highway statistics), or by personal communication with the
agency/organization responsible for collecting the data. Each data source conducts error checks and
monitors the accuracy of its data.
Most of these sources and their verification and validation procedures
are described in subsequent data details in this report for performance
measures of individual modes of transportation. |
|
Comment: |
While caution should be exercised in comparing fatalities,
injuries and incidents between modes of transportation due to differences in
definitions and calculations, the aggregation of these values still provides
useful information. Because the
methodology for calculating these measures has remained consistent over the
years, the trend information should provide a reasonably accurate picture of
results. |
|
Measure: |
Fatalities
per 100 million vehicle-miles-traveled (VMT) (CY) |
|
Scope: |
The number of fatalities is the total
number of motor vehicle traffic fatalities which occur on public roadways
within the 50 states and Washington, D.C. Vehicle
Miles of Travel (VMT) represent the total number of vehicle miles traveled by
motor vehicles on public roadways within the 50 states and Washington, D.C.
|
Source:
|
Motor vehicle traffic
fatality data are obtained from NHTSA’s Fatality Analysis Reporting System
(FARS). To be included in FARS, a motor
vehicle traffic crash must result in the death of a vehicle occupant or a
non-motorist within 30 days of the crash.
The FARS database is based on police crash reports and other state
data. FARS includes fatalities on all roadways open to the public, using the
National Highways System classification of roads. Pedestrian and bicycle fatalities that occur on public
highways, but do not involve a motor vehicle, are not recorded in FARS. However, they constitute only a small
number of fatalities. VMT data are derived from FHWA’s Traffic Volume Trends
(TVT), a monthly report based on hourly traffic count data in the Highway
Performance Monitoring System (HPMS).
Information is transmitted to NHTSA where it is reviewed for
consistency and accuracy before being entered into the system. These data,
collected at approximately 4,000 continuous traffic counting locations
nationwide, are used to determine the percentage change in traffic for the
current month from the same month of the previous year. The percentage change is applied to the
nationwide travel for the same month of the previous year to obtain an
estimate of nationwide travel for the current month. The data are recorded as monthly totals
and cumulative yearly totals.
|
|
Limitations: |
VMT data are subject
to sampling errors, whose magnitude depends on how well the locations of the
continuous counting locations represent nationwide traffic rates. HPMS is also subject to estimating
differences in the states, even though FHWA works to minimize such differences
and differing projections on growth, population, and economic conditions that
impact driving behavior. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating fatality rates is the denominator. While the estimate of total fatalities
used in the numerator is relatively accurate, the estimate of total vehicle
miles in the denominator has far more variability. Based on data from 1994-2000, the annual
variation in the fatality rate has a regression standard error of 0.029. The estimates of the
number and percentages of persons killed in motor vehicle traffic crashes
during 2001 are preliminary and are based on incomplete data and statistical
models. NHTSA’s first official
estimates for 2001, the Early Assessment, are being developed and will be
completed in early April 2002.
Differences between the Official Early Assessment estimates and those
in this report are to be expected. |
|
Verification & Validation: |
Fatality data from FARS
are reviewed and analyzed by NHTSA’s National Center for Statistics and
Analysis. Quality control procedures
are built into annual data collection at 6 and 9 months, and at year’s end. A study was completed in 1993, looking at
samples of FARS cases in 1989 through 1990 to assess the accuracy of data
being reported. VMT data are reviewed
by FHWA for consistency and reasonableness. |
|
Comment: |
This data program has
been in use for many years and is generally accepted for describing safety on
the Nation’s highways. Adjusting raw
highway fatalities and injuries by VMT provides a means of portraying the
changes in highway fatalities on a constant exposure basis and facilitates
year-to-year comparisons. |
|
Measure: |
Number
and rate (per million commercial VMT) of fatalities in crashes involving
large trucks. (CY) |
|
Scope: |
The measure includes all fatalities (e.g., drivers and occupants
of passenger cars, motorcycles, large trucks, or pedestrians) associated with
crashes involving trucks with a gross vehicle weight rating of 10,000 pounds
or more. The number of fatalities
comes from NHTSA’s Fatality Analysis Reporting System (FARS) data, a census
of fatal traffic crashes within the 50 states, Puerto Rico, and Washington,
D.C. The fatal crash rate is the
number of fatalities per 100 million vehicle miles of large truck travel
(VMT). |
Source:
|
NHTSA’s Fatality
Analysis Reporting System (FARS) provides fatality data. The VMT data are derived from the Federal
Highway Administration’s (FHWA) Highway Performance Monitoring System (HPMS). |
|
Limitations: |
FARS data elements are modified
from year to year to respond to emphasis areas, vehicle fleet changes, and
other needs for improvement. Large truck VMT reported to FHWA by each state
is based on a sample of road segments and is not a census. In addition, the methods used to calculate
total VMT may vary from state to state. The methods used by the states to
estimate the VMT contribution from rural and urban minor collectors are
unknown. |
|
Statistical Issues: |
The fatality counts in FARS are generally quite accurate. The major sources of error are
underreporting by some precincts and inconsistent use of the definition of a
truck. Based on 1994-2000 data, the
chance variation in a given year has a regression standard error of
approximately 157 fatalities. Because
the VMT data provided to FHWA from each state are estimates based on a sample
of road segments, the numbers have associated sampling errors. The methodology used by each of the states
to estimate VMT is not known and may introduce additional non-sampling
error. Although states provide VMT
estimates on an annual basis, they are only required to update their traffic
counts at all sampling sites once every three years. Thus an annual VMT estimate from a
particular state may be based, in part, on data collected during a previous
year. Based on 1994-2000 data, the
chance variation in a given year in the number of fatalities per 100 million
vehicle miles of large truck travel has a regression standard error of 0.053. |
|
Verification & Validation: |
Fatality data are reviewed and analyzed by NHTSA’s National
Center for Statistics and Analysis.
Quality control procedures are built into data collection and data
processing. A study using samples of
1989-1990 FARS cases was completed in 1993 to assess the accuracy of data being
reported. FHWA routinely works with
state data providers to modify reported VMT values that do not appear
reasonable before incorporating them into its final master file. |
|
Comment: |
The FARS data have been around for many years and are generally accepted
as a good source for describing fatal crashes on the Nation’s highways. The large truck VMT data used to calculate
fatal crash rates have both sampling and non-sampling (i.e., bias) error
associated with it. The impact of
these errors on FMCSA’s estimates of large truck crash rates is considered to
be minimal. |
|
Measure: |
1. Alcohol-related fatalities per 100 million
vehicle-miles traveled. 2. Percentage of highway fatalities that are alcohol
related. (CY) (2001) |
|
Scope: |
The number of
fatalities resulting from motor vehicle traffic crashes that are alcohol
related and occur on public roadways within the 50 states and Washington,
D.C. |
Source:
|
Motor vehicle traffic fatality data are obtained from NHTSA’s
Fatality Analysis Reporting System (FARS). FARS is a census of fatal motor
vehicle traffic crashes within the 50 states, Puerto Rico, and Washington,
D.C. To be included in FARS, a crash
must result in the death of a vehicle occupant or a non-motorist within 30
days of the crash. The FARS data are
based on police crash reports and other state data. FARS includes fatalities
on all roadways open to the public, using the National Highways System classification
of roads. Pedestrian and bicycle
fatalities that occur on public highways, but do not involve a motor vehicle,
are not recorded in FARS. However,
they constitute only a small number of fatalities. A fatal motor vehicle traffic crash is alcohol-related if
either a driver or a non-motorist (such as a pedestrian) involved in the
crash had a measured or estimated blood alcohol concentration (BAC) of 0.01
grams per deciliter or above.
|
|
Limitations: |
Blood Alcohol Concentration test
results are not available for all drivers and non-occupants involved in fatal
crashes. Missing data can result for
a number of reasons -- the most frequent of which is that persons are not
always tested for alcohol. To address
the missing data issue, NHTSA has developed a statistical model (Multiple
Imputation) to estimate specific
values of BAC across the full range of possible values.
Estimating missing BAC in
this manner will permit the estimation of valid statistics such as variances,
measures of central tendency, confidence intervals and standard deviations. The statistical model is based on
important characteristics of the crash including crash factors, vehicle
factors, and person factors. While
this measure does not link alcohol with fault in fatal crashes, the more
comprehensive scope of the measure compensates for a possible undercount of
the extent of the alcohol impaired driving problem. Multiple Imputation differs from the statistical model used in
previous years. However, all historical series of alcohol
involvement will be revised back to the 1982 data year to reflect the
estimates from the new methodology. |
|
Statistical Issues: |
The primary sources of uncertainty
in this performance measure arise from information gaps in the number of
intoxicated non-motorists, and from using the statistical model to estimate
the number of intoxicated drivers. The estimates of the number and
percentages of persons killed in motor vehicle traffic crashes during 2001 included in this section are preliminary and are based on incomplete data and
statistical models. They were provided to meet the time restraints required
for this report. NHTSA’s first
official estimates for 2001, the Early Assessment, are being developed and
will be completed in early April.
Differences between the Official Early Assessment estimates and those
in this report are to be expected. |
|
Verification &
Validation: |
Data are reviewed and analyzed by
NHTSA’s National Center for Statistics and Analysis. Quality control
procedures are built into annual data collection at 6 and 9 months, and at
year’s end. In 1987 and 1988, an
independent panel of academics reviewed and commented on the statistical
methods used in measuring alcohol-related highway fatalities. This report recommended that research and
development utilize a model that would permit the imputation of missing BACs
as a semi-continuous variable. |
|
Comment: |
This data program has been used
for many years and is generally accepted for describing safety on the
Nation’s highways. |
|
Measure: |
Injured
persons per 100 million vehicle-miles-traveled (VMT) (CY) (2001) |
|
Scope: |
The number of injured persons
is an estimate of the total number of persons injured in motor vehicle
traffic crashes that occur on public roadways in the 50 states and
Washington, D.C. Vehicle Miles of
Travel (VMT) represent the total number of vehicle miles traveled by motor vehicles
on public roadways within the 50 states and Washington, D.C. |
Source:
|
The number of injured persons data are derived from the
NHTSA’s National Automotive Sampling System (NASS) General Estimates System
(GES). The NASS GES is a nationally representative
probability sample that yields national estimates of total nonfatal injury
crashes, injured persons, and property-damage-only crashes. NASS GES data
cover all roadways open to the public, using the National Highways System
classification of roads.
VMT data are derived
from FHWA’s monthly report, Traffic Volume Trends (TVT), a monthly report
based on hourly traffic count data in the Highway Performance Monitoring
System (HPMS). Information is
transmitted to NHTSA where it is reviewed for consistency and accuracy before
being entered into the system. These data, collected at approximately 4,000
continuous traffic counting locations nationwide, are used to determine the
percentage change in traffic for the current month from the same month of the
previous year. The percentage change
is applied to the nationwide travel for the same month of the previous year
to obtain an estimate of nationwide travel for the current month. The data are recorded as monthly totals
and cumulative yearly totals. |
|
Limitations: |
GES data are obtained
from a nationally representative sample of 60 sites. The results provide only national data,
not state level data, and are subject to sampling error. The magnitude of the sampling error
depends on the number of Primary Sampling Units (PSUs) in the sample and the
number of crash reports sampled within each PSU. VMT data are subject
to sampling errors, whose magnitude depends upon how well the continuous
counting locations represent nationwide traffic rates. HPMS is subject to estimating differences
in the states, although FHWA works to minimize such differences and differing
projections on growth, population, and economic conditions which impact
driving behavior. |
|
Statistical
Issues: |
The estimate of the injury
rate includes three main sources of uncertainty. The numerator count of injuries has a standard error of 5.1%
(cf. Appendix C of Traffic Safety Facts). The denominator estimate of VMT contains both complex sampling
and non-sampling errors. Based on data from 1994-2000, the annual
variation in the injury rate has a regression standard error of 4.04. The estimates of the
number and percentages of persons injured in motor vehicle traffic crashes
during 2001 are preliminary and are based on incomplete data and statistical
models. NHTSA’s first official
estimates for 2001, the Early Assessment, are being developed and will be
completed in early April. Differences
between the Official Early Assessment estimates and those in this report are
to be expected. |
|
Verification &
Validation: |
Data are reviewed and
analyzed by NHTSA’s National Center for Statistics and Analysis. Quality
control procedures are built into annual data collection at 6 and 9 months,
and at year’s end. A study was completed
in 1993, looking at samples of FARS cases in 1989 through 1990 to assess the
accuracy of data being reported. VMT
data is reviewed by FHWA for consistency and reasonableness. |
|
Comment: |
This data program has
been in use for many years and is generally accepted for describing safety on
the Nation’s highways. GES records
injury severity in four classes: incapacitating injury, evident but not
incapacitating injury, possible but not visible injury, and injury of unknown
severity. Adjusting raw highway
fatalities and injuries by VMT provides a means of portraying the changes in
highway fatalities on a constant exposure basis – to facilitate year-to-year
comparisons. |
|
Measure: |
Number
and rate of injured persons involving large trucks. (CY) (2001) |
|
Scope: |
The measure includes all injured persons (e.g., drivers and
occupants of passenger cars, motorcycles, large trucks, or pedestrians)
associated with crashes involving trucks with a gross vehicle weight rating
of 10,000 pounds or more. The
number of injured persons is derived from NHTSA’s General Estimates System
(GES). The injury rate is the number of injured persons per 100 million
vehicle miles of large truck travel (VMT). |
Source:
|
NHTSA’s General Estimates System (GES)
provides injury data. VMT data are derived from the Federal Highway
Administration’s (FHWA) Highway Performance Monitoring System (HPMS).
|
|
Limitations: |
GES data are obtained from a nationally representative sample of 60
sites. The results provide only
national data, not state-by-state data.
Large truck VMT reported to FHWA by each state is based on a sample of
road segments and is not a census. In
addition, the methods used to calculate total VMT may vary from state to
state. The methods used by the states to estimate the VMT contribution from
rural and urban minor collectors are unknown. |
|
Statistical Issues: |
The GES data have a
standard error of 6.9% for injuries from truck and automobile crashes (cf.
Appendix C of Traffic Accident Reports). They are less accurate than the
corresponding fatality counts. Based
on 1994-2000 data, the variation due to random chance in the number of injuries,
which includes sampling variability, has a regression standard error of
approximately 7,091. Because the VMT
data provided to FHWA from each state are estimates based on a sample of road
segments, the numbers have associated sampling errors. The methodology used by each of the states
to estimate VMT is not known and may introduce additional non-sampling error
into the estimates. Although states
provide VMT estimates on an annual basis, they are only required to update
their traffic counts at all sampling sites once every three years. Thus an annual VMT estimate from a
particular state may be based, in part, on data collected during a previous
year. Based on 1994-2000 data, the
chance variation in a given year in the number of injured persons per 100
million vehicle miles of large truck travel has a regression standard error
of 4.39. |
|
Verification &
Validation: |
Injury data are reviewed
and analyzed by NHTSA’s National Center for Statistics and Analysis. Quality
control procedures are built into data collection and data processing. FHWA routinely works with state data
providers to modify reported VMT values that do not appear reasonable before
incorporating them into its final master file. |
|
Comment: |
The data program has been around for many years and is generally
accepted for describing safety on the Nation’s highways. GES records injury severity in four
classes: incapacitating injury, evident injury but not incapacitating, possible
but not visible injury, and injury of unknown severity. The large truck VMT data used to calculate
injured persons rates have both sampling and non-sampling (i.e., bias) error
associated with it. The impact of
these errors on FMCSA’s estimates of large truck crash rates is considered to
be minimal. |
|
Measure: |
Percentage
of front occupants using seat belts. (CY) (2001) |
|
Scope: |
The proportion of
front seat outboard passenger vehicle occupants using shoulder belts during
daylight hours. |
Source:
|
Data for 1998, 1999, and 2000 are from
the National Occupant Protection Use Survey (NOPUS). NOPUS is a National, multi-stage
probability sample. In the first
stage, counties or groups of counties (Primary Sampling Units or PSUs) were
grouped by region (Northeast, Midwest, South, and West), level of
urbanization (metropolitan or not), and level of belt use (high, medium, or
low). Fifty PSUs were selected based
on the vehicle miles of travel in those locations. In the next stage, a random sample of eight (8) Census Tracts
was selected within each of the PSUs.
In the final stage a sample of ten (10) roadway segments for all types
of roads was selected within each Census Tract. In the even numbered years, shoulder belt use of front seat
outboard (driver and right front seat) passenger vehicle (passenger cars,
vans, sport utility vehicles, and pickup trucks) occupants was observed
during daylight hours at each of the 4,000 sampled roadway segments. In 1999, a Mini-NOPUS consisting of
observation at a subsample of 2,000 of the 4,000 roadway segments was
conducted.
Estimates of national
shoulder belt use for other years shown in the graph are based on state belt
use surveys. These surveys are
conducted by most of the 50 States and the District of Columbia. For the years shown, these surveys varied
in coverage, design, and observation methods. National averages were obtained by weighting the most recently
provided state belt use estimate by the population of the state. |
|
Limitations: |
NOPUS data are based on
a random sample of sites and, therefore, are subject to sampling error. For the estimate of overall National
shoulder belt use from the 2000 NOPUS Survey, sampling error was estimated to
be 1.4 percentage points.
Additionally, observation of shoulder belt use is restricted to
daylight hours. State belt use surveys
have been conducted in many different ways.
Less than half of the states conducted probability based surveys and
the rest were based on other methods.
Additionally, most states conducted surveys that observed use only for
those occupants and vehicles covered by their state belt use law. After enactment of a grant program in the
ISTEA of 1991, some 24 states had surveys that met design criteria specified
by NHTSA. |
|
Statistical Issues: |
The primary source of
uncertainty in NOPUS is sampling errors.
The most recent estimate shown in this report is based on a
probability sample, and the survey bias and reweighting are complex. For State surveys, uncertainty derives
from disparities among the different surveys conducted by the states, the use
of non-probability samples by many of the states, the differences in persons
and vehicles observed, the differing methodologies and processes followed to
collect data on the persons and vehicles observed, and the procedures used to
estimate overall belt use. To compute
the National average from state rates for a specific year, when a state did
not conduct a survey or provide NHTSA with an estimate, the most recent rate
provided by that state was substituted.
Also, weighting state averages by population may have overstated the
contributions of some states. Based on data from 1994-2000, the annual
variation in the seat belt use rate has a regression standard error of 1.31
percent. |
|
Verification & Validation: |
NOPUS data collection
is managed by a survey research contractor who has responsibility to hire and
train the data collectors/observers.
Before data collection begins, NHTSA reviews and approves all the
training materials and Data collectors/observers must pass a 2-day training
course. The data contractor also
conducts on-scene “surprise” quality control visits to ensure that
observations are made correctly and data are coded properly. Numerous edits are also employed in the
data processing. NHTSA reviews the
data provided by the contractor for consistency. NHTSA reviewed and approved
the survey designs and data collection procedures for 24 states as a result
of a grant program authorized by the ISTEA of 1991. NHTSA, however, did not conduct any quality review or
validation of the data collection and estimation processes employed by the
states during or after data collection for the years shown. |
|
Comment: |
None. |
|
Measure: |
Fatal
aviation accidents (U.S. commercial air carriers) per 100,000 departures.
(FY) |
|
Scope: |
This measure includes both scheduled and nonscheduled flights of
large U.S. air carriers (14 CFR Part 121) and scheduled flights of commuter
airlines (14 CFR Part 135). It
excludes on-demand (i.e., air taxi) service and general aviation. |
Source:
|
Part 121 and Part 135 departure data
is submitted to BTS under 14 CFR Parts 241 and 298, respectively. NTSB provides accident data. |
|
Limitations: |
The fatal accident rate
in these categories is small and could significantly fluctuate from year to
year due to the occurrence or non-occurrence of a single accident. |
|
Statistical Issues: |
The switch from calendar to fiscal year in 2001, combined with the
use of departures rather than flight hours as the activity measure for the
denominator, present new problems.
The FAA has no independent data sources to validate BTS-collected
departure data as it did with flight hour data. To overcome reporting delays of 60 to 90 days, FAA must rely on
historical data, partial internal data sources, and Official Airline Guide
(OAG) scheduling information to project at least part of the fiscal year
activity data. Due to the reporting
procedures in place, it is unlikely that calculation of future fiscal year
departure data will be markedly improved.
Lacking complete historical data on a monthly basis and independent
sources of verification increases the risk of error in the activity
data. The regression standard error
for the annual variation in the fatality rate, based on data from 1994 –
2000, is 0.023. |
|
Verification &
Validation |
The FAA does comparison checking of the departure data collected by
BTS; however, FAA has no independent data sources against which to validate
the numbers submitted to BTS. FAA
compares its list of carriers to the DOT list to validate completeness of the
reporting list and places the carriers in the appropriate category (i.e.,
Part 121 or Part 135). NTSB and FAA’s
Office of Accident Investigation meet regularly to validate the accident
count. |
|
Comment: |
The joint
government/industry group working on improving the level of safety for U.S.
commercial aviation has determined that the number of departures is a better
denominator measure to use for determining accident rates. In a recent report on the Safer Skies
effort the Government Accounting Office agreed and recommended that the FAA
use departures. |
|
Measure: |
Number
of fatal general aviation accidents. (FY) |
|
Scope: |
The measure includes on-demand (non-scheduled FAR Part 135) and
general aviation. General aviation
comprises a diverse range of aviation activities. The range of general aviation aircraft includes single-seat
homebuilt aircraft, helicopters, balloons, single and multiple engine land
and seaplanes including highly sophisticated extended range turbojets. |
Source:
|
National Transportation Safety Board (NTSB). |
|
Limitations: |
The
use of the 1996-1998 timeframe for the baseline represents one of the safest
periods in general aviation history in terms of a decline in fatal
accidents. The number of general aviation
accidents reported in any given year might change in subsequent years. There are many reasons for these changes
to the historical data. Primary among
them is that the accident had not been reported to the NTSB, or that it was
misreported and the information corrected at a later date. |
|
Statistical Issues: |
There is no major error in the accident counts. Random variation in air crashes results in
a significant variation in the number of fatal accidents over time. The regression standard error in this
variation for 1996 through 2000 is 16.5.
|
|
Verification &
Validation: |
NTSB and FAA’s Office of
Accident Investigation meet regularly to validate the information on the
number of accidents. |
|
Comment: |
It would be preferable to use fatal accident rates rather than
fatal accidents as the performance measure. However, general aviation flight
hours are based on an annual survey conducted by the FAA. Response to the survey is voluntary. The accuracy of the flight hours collected
is suspect and there is no readily available way to verify or validate the
data. For this reason, the General
Aviation community is unwilling to use a rate measure until the validity and
reliability of the survey data can be assured. |
|
Measures: |
1. Operational errors
per 100,000 activities, or per 1 million activities. (2001) 2. Number of operational errors where less than 80 percent of
required separation is maintained. |
|
Scope: |
An error occurs
when separation between aircraft is less than the separation determined
necessary for the specific phase of flight.
“Activities” are total facility activities, as defined in Aviation System Indicators 1997 Annual
Report. Total facility activities
are the sum of en route and terminal facility activities. |
Source:
|
FAA air traffic facilities have a software program called
Operational Error Detection Patch (OEDP) that detects possible operational
errors and sends alert messages to supervisory personnel. Facility management reviews OEDP alerts
and data provided from the National Track Analysis Program (NTAP) to
determine if an operational error has occurred. Controllers are required to report operational errors. The information is summarized in the FAA
Air Traffic Operational Error and Deviation Database.
|
|
Limitations: |
There is a few months’
lag in reporting data because of the need to investigate major
incidents. The severity of errors is
not measured. Minor errors such as a
4.5-mile rather than a 5-mile separation are counted in the same way as more
serious errors. Data are available
for 1994 and following years. The DOT IG conducted an
audit of reporting on operational errors.
The IG believes that there is a potential for underreporting of
operational errors, as some errors are self-reported. The FAA disagrees with this assessment
because there are substantial penalties for not reporting an operational error. |
|
Statistical Issues: |
There are no major sources of systematic error in the operational
errors data that have been quantified.
Again, random variation in operational errors results in a significant
variation in the measured rates over time. The regression standard error in
the operational error rate using 100,000 activities denominator and the 1
million activities denominator, based on 1994-2000 data, are .048 and .48,
respectively. |
|
Verification &
Validation: |
FAA performs system
checks and counts daily against reported data to ensure the accuracy of
information reported. |
|
Comment: |
In August 1998,
the FAA discovered and corrected a misunderstanding of the procedures used in
interpreting separation reported by the National Track Analysis Program and
the data provided by the Operational Error Detection Patch. The corrected application of these
procedures, while not affecting safety, has resulted in an overall increase
in the number of errors reported between 4.6 and 4.9 miles separation
(Standard separation in these cases is 5 miles). |
|
Measures: |
1. Number of runway incursions. (FY) (2001) 2. Number and rate (per 100,000 operations)
of highest risk runway incursions. |
|
Scope: |
Runway incursions are the result of ground collision hazards or
loss of separation for aircraft in the process of taking off or landing. They
are grouped in three general categories:
operational errors, surface pilot deviations, and vehicle/pedestrian
deviations. Incursions are reported and tracked at airports that have an
operational air traffic control tower.
|
Source:
|
Air traffic controllers and pilots are
the primary source of runway incursion reports. The data is recorded in the
FAA National Incident Monitoring System (NAIMS).
|
|
Limitations: |
Preliminary incident
reports are evaluated when received. Evaluation can take up to 90 days. |
|
Statistical Issues: |
There are no major sources of systematic error in quantified runway
incursion data. The regression standard error in the reported number of
incursions, based on 1994-2000 data, is approximately 15.4. Based on 1998 – 2001 data, the regression
standard error for the number and rate of highest risk runway incursions are
8.8 and 0.01, respectively. |
|
Verification &
Validation: |
Surface incidents are
reported in the Administrator’s Daily Bulletin at the beginning of each
weekday. Surface incidents are evaluated to determine if they should be classified
as incursions. Incidents are evaluated against the official runway incursion
definition. The Air Traffic Runway Safety Program Manager, ATP-20, makes the
final decision regarding runway incursions. |
|
Comment: |
None. |
|
Measure: |
Percent
of all mariners in imminent danger who are rescued. (FY) |
|
Scope: |
Includes people in water; on shore; and aboard a vessel,
offshore structure, pier, or vehicle that is in distress or in urgent need of
assistance. The Coast Guard makes a
final determination on scene whether there is imminent danger, based on
criteria that include the nature of distress, the condition of the vessel,
the people onboard, and the environmental conditions. Criteria for this decision are discussed
in search and rescue doctrine publications. |
Source:
|
CG Search and Rescue Management
Information System (SARMIS). Data is collected from Coast Guard field units
that conduct search and rescue responses.
|
|
Limitations: |
It is probable that some number of imminent danger cases, and the
associated lives, are not reported in SARMIS. This includes situations where no distress call was received by
the Coast Guard and the persons in distress were rescued by private citizens
or local government personnel, or where the persons in distress perished
without trace. The extent of this under-reporting is not known. There is some
judgment involved in assessing whether mariners are in danger. However, there
is likely to be consistency in these assessments across years. 1994 data is skewed upward by a large
surge of migrants interdicted at sea, most of whom were counted as “rescued,”
thus increasing the percentage of lives reported as saved. Reporting no longer includes migrants
interdicted; they are counted directly as migrants interdicted under law
enforcement activity. Prior to the
introduction of the next generation data system in October 2000, data entry
was limited to closed cases, after a rescue has been successfully completed
or after the recovery of a body. The
new data system now allows missing bodies to be tracked. In this first year of data, more cases
than expected were found where bodies were not recovered. Before adding this number into our data
analysis, we will track this number to assure that this represents a data
trend and not an unusual aberration.
Errors may be introduced in SARMIS through data entry, but are likely
rare for lives saved data elements. |
|
Statistical Issues: |
The primary source of uncertainty consists of non-sampling errors.
The second generation data system, brought on-line on October 1, 2000,
reduces error due to miscoding through the use of more extensive drop down
menus, machine generated case numbers, structured data boxes, and more extensive
business rules eliminating the selection of data not consistent with other
entered data. The regression standard error for year-to-year chance variation
is 2.6 percent mariners rescued, based on data from 1994 through 2000. |
|
Verification &
Validation: |
SARMIS data entry system
uses structured entry values, check boxes, and pull down selection lists to
limit entry errors. The use of plain
language descriptions eliminates a majority of erroneous data code selection.
Additional system business rules also eliminate the selection of data not
appropriate with other entered data. The SAR Mission Coordinator (SMC) is
responsible for accurate entry of particular case data by all units involved
in the case. CG Program Managers annually validate the data in SARMIS.
Entries are reviewed at Coast Guard District offices as first step in
validation – errors and inconsistencies are identified and corrected. Finally, Coast Guard Headquarters program
managers review compiled data annually to assess consistency with historic
variance and trends. This review
includes curvilinear regression analysis to compare current data to historic
data and a program review analysis to identify and resolve aberrations. |
|
Comment: |
Beginning in FY01, this measure will cover
all mariners in distress reported in SARMIS.
The previous measure covered only mariners reported in distress that
were rescued. The significance of the
87.5% result for FY99 is uncertain at this point; FY95-98 data show a flat
trend at 84%. It is not known if the
FY99 result was produced by anomalous factors, or if it is the product of
program strategies and a changing external environment. Therefore, the goal target remains at 85%
until more analysis is completed. For
FY 2001, the preliminary estimate of the measure was 84.2 percent of all
lives, bringing the percentage about equal to the average since 1995 and
slightly below the goal, but certainly within normal variation about the
average. |
|
Measure: |
Number
of recreational boating fatalities. (CY) (2001) |
|
Scope: |
Measure includes fatalities occurring aboard vessels that are
being operated for recreational purposes.
Surfboards, iceboats, and vessels engaged in sanctioned racing events
are not considered recreational vessels.
Fatalities are included if caused by a fire, explosion, sinking or
other occurrence involving a recreational vessel, and the vessel or
associated equipment caused or contributed to the fatality. Fatalities are not included if they
occurred aboard a recreational vessel, but were caused by self-inflicted
wounds or natural causes. Fatalities
are also excluded if they occurred while the victim was engaged in other
activity such as swimming or diving, where the vessel was used as a platform
only and was not a contributing factor to the fatality. Beginning two years
ago, the measure for Recreational Boating was revised by adding an additional
6% to the aggregate number of reported fatalities, to correct for an
estimated 6% underreporting of recreational boating fatalities. |
Source:
|
Coast Guard Boating Accident Report
Database (BARD). Data is entered into
BARD by state administrators who collect data from boat owners and operators
through formal Boating Accident Reports, as instructed in 33 CFR 173c.
|
|
Limitations: |
Fatality data is derived from reports submitted by
the public along with accompanying state investigation reports. There is
consensus among the Coast Guard, the states, safety professionals, and other
researchers that most fatalities that occur on inland and most coastal waters
are under-reported. To better
quantify the extent of possible under-reporting the Coast Guard initiated and
funded an analysis of BARD data conducted by the Boat Owners Association of
the United States (BOAT/U.S.) Foundation for Boating Safety. The study found some fatalities involving
recreational boating in the Coast Guard’s Search and Rescue Management Information
System (SARMIS) that were not in BARD. However, although the study reported a
9% discrepancy, further analysis revealed that some of these findings would
not be reportable as recreational boating fatalities. There is also consensus
that under-reporting exists for fatalities occurring offshore, and aboard
U.S. recreational boats operating overseas.
Also, although there are guidelines as to what constitutes a
recreational boating fatality, there is still an element of interpretation at
the state level in reporting fatalities.
It is probable that the states do not always interpret the guidelines
in the same manner. Overall, the best estimate indicates that total
fatalities are currently under-reported by at least 6%. |
|
Statistical Issues: |
The discrepancy between
BARD and the Search & Rescue Management Information System (SARMIS) amounts to 6% of the total reports for those states covered by SARMIS. The numbers given in this report have been
adjusted to correct the deficiency.
Also, note that since the boating fatality counts are influenced by
weather, gasoline prices and other external factors, annual chance variation
should be large. Using data from 1994
to 2000, the annual variation in the number of fatalities attributable to
random chance has a regression standard error of 50.7. |
|
Verification &
Validation: |
Fatality data in BARD is
verified and validated by state boating administrators and Coast Guard
program managers. At the end of the calendar year, the Coast Guard compiles
state fatality data and sends a report to each state for confirmation. Both State and Coast Guard officials
review the statistics, including sampling of cases to ensure guidelines for
classifying fatalities were followed.
Any discrepancy is reconciled jointly by the State and Coast Guard
program manager. |
|
Comment: |
Data are not normalized for increases or decreases in the number
or usage of boats, which tends to limit data use in making comparisons over
time. The number and usage of
recreational boats has increased over the past 2 decades, while the raw number
of fatalities has generally decreased. The BOAT/US review of BARD data for
1993 through 1997 identified underreporting in BARD of 8% in 1993 and 1994,
12% in 1995, 13% in 1996 and 8% in 1997.
The Coast Guard reviewed BOAT/US’s findings for 1995, 1996, and
1997. Each record for these years was
checked and fatalities that were incorrectly labeled as recreational boating
fatalities by BOAT/US were removed from the count. Based on this revised count of recreational boating fatalities
with mislabeled fatalities removed, the Coast Guard estimates that 7%, 8% and 4% of all recreational boating
fatalities were not captured in its Boating Accident Report Database (BARD)
in 1995, 1996 and 1997 respectively for purposes of this report. The median
of these numbers – 6% - has been used to adjust recreational boating safety
data for 1993, 1994, 1998 and 1999, and to reset the goals for 1999 through
2001. The original goal of 720 has been
increased by 6% to 763 for 2000. The Coast Guard is in the
process of commissioning a comprehensive
National Boating Survey to obtain valid and reliable information on boating
practices, safety, and exposure. This
information will enable safety officials to assess boating risk, implement appropriate
safety intervention strategies, and measure the effectiveness of program
activities in reducing the risk and negative outcomes associated with the use
of recreational boats. Data from
this study will be used to further address underreporting issues and estimate
reporting discrepancies in BARD. The
study was originally set to begin in Fall 2001, however data collection is
now scheduled to begin in April 2002. |
|
Measure: |
Fatalities and rate (per million passenger capacity) aboard passenger
vessels. (2001) |
|
Scope: |
This measure is an indicator of passenger safety. It includes reportable marine casualties
resulting in the death or disappearance of a passenger aboard any U.S. vessel
(regardless of type or location) or aboard foreign flag vessels in U.S.
waters. Exceptions include death/disappearance of “non-passengers”, whenever
the cause of death/disappearance is classified as being from diving, natural
causes, (e.g. heart attack) or whenever the death/disappearance is the result
of an intentional act (e.g. suicide, altercation). Fatalities on recreational vessels are not included for two
principal reasons: Recreational vessels are prohibited from carrying
“passengers” and recreational vessel fatalities are measured and reported
separately. |
Source:
|
Passenger fatality source data is
obtained from the Coast Guard Marine Safety Information System (MSIS). Passenger fatalities are reported to the
Coast Guard as required by federal regulations. Sources of reports are most often vessel masters, operators,
owners, insurance companies, legal representatives, and other mariners.
|
|
Limitations: |
The investigation,
retrieval, analysis and reporting processes result in under-reporting for the
most recent year, with the most significant effects over the most recent 5
months. Estimates are often used to
compensate for this known data-lag. The Coast Guard initiates about 40-50
civil penalty cases for failure to report marine casualties, although many of
these are for minor casualties. In addition, some passenger fatalities may
not be reported to the Coast Guard. This number is unknown. Some passenger injuries may ultimately
prove fatal and lead to death; some missing passengers may be found. These numbers may not be updated to
reflect the changes in status. The
number is believed to be small.
Duplicate casualty entries are sometimes entered into MSIS, and some
casualties are mistakenly omitted or coded incorrectly. Verification
procedures strive to correct these errors, but it is probable that a small
number are not corrected. The data retrieval & reporting processes do not
allow automated distinction between all death types (e.g. natural vs.
accidental). As a result, some
natural deaths or suicides may be inadvertently included. |
|
Statistical Issues: |
The major sources of
uncertainty in this measure are the estimation error (as a result of the
data-lag) and the reporting error (as a result of the inability to
distinguish between which deaths should be included and which should be
excluded). |
|
Verification &
Validation: |
Verification and
validation occurs at several levels.
Edit checks within MSIS software can detect some incorrect or missing
data and force review and correction before data entry is completed. Selection lists for certain data fields
also reduce the opportunity for data entry error. All investigations go through review at the field unit for
accuracy. Investigations of serious
marine casualties are also usually reviewed at district and headquarters
offices. The headquarters Data
Administration staff conducts periodic quality control checks to identify
entry errors such as missing data or miscoding, and corrects any errors
identified. Errors identified are
referred to either the Data Administration staff or the Investigations and
Analysis staff for correction. |
|
Comment: |
During FY 2002, the Marine Safety Information System (MSIS) will
be replaced by the Marine Information System for Safety and Law Enforcement
(MISLE). While the new system will be
a major improvement, it is expected to cause serious difficulties in making
performance comparisons. One factor
is that many business processes were re-designed in conjunction with system
development. Another factor is that
data quality under MISLE is expected to be superior to that of MSIS. While this represents improvement, it may
cause near-term problems in making meaningful comparisons of data between the
two systems. |
|
Measure: |
1.
Train accidents per million train-miles. 2.
Rail-related fatalities per million train-miles. (CY) (2001) |
|
Scope: |
The fatality measure includes anyone on rail property, any on-duty
railroad employee, and anyone killed by a train or its contents. It does not include fatalities on trains
or rail lines that do not connect to the national rail network, such as mass
transit operations, certain excursion and tourist railroads, and some
industrial railroads not connected to the general system. The only railroad fatalities that are not
counted are suicides (as determined by a public official) and death by
natural cause not associated with railroad operations. Train accidents do not include
those at grade crossings. They are
reported under the performance goal for highway-rail grade crossing
accidents. |
Source:
|
Railroad
Safety Statistics – Annual Report. Statistical data, tables,
and charts depict the causes and nature of rail-related fatalities. Data on fatalities and train miles are
reported to FRA by railroad companies. |
|
Limitations: |
Because of the scope of
the reporting criteria, some fatalities that are counted are not associated
directly with operation of the trains, and some railroad fatalities are not
counted. This scope is consistent
with the regulatory authority of the agency, but not consistent with other
modes of transportation for comparative purposes. |
|
Statistical Issues: |
The reported estimates
are based upon partially reported data from 2001. Based on data from 1994-2000, chance variation from year to
year, as reflected in the regression standard error, is 0.055 for rail
fatalities. |
|
Verification &
Validation: |
Railroads are required by
law to submit monthly accident/incident reports to FRA. They are also required to update any
inaccurate or incomplete information.
FRA conducts routine data audits (records inspections) to verify the
adequacy of railroad reporting and record keeping requirements. |
|
Comment: |
None. |
|
Measure: |
Grade-crossing accidents divided by the product of: 1) million train
miles and 2) trillion vehicle-miles-traveled. (CY) |
|
Scope: |
The measure includes all collisions with on-track equipment and
highway users at public and private grade crossings. |
Source:
|
Collisions and train-miles are reported in FRA’s Railroad Safety Statistics – Annual
Report. Vehicle-miles-traveled (VMT) are obtained from the FHWA
Office of Highway Information Management.
|
|
Limitations: |
Because the denominator includes all highway vehicle-miles-traveled
(VMT), and not just VMT that are exposed to grade crossings, the rate
portrayed may be lower than the actual risk. |
|
Statistical Issues: |
Trains and automobiles
have different exposures at rail crossings---the denominator used here
attempts to combine these. The
numerator is based on partially reported 2001 data. The annual variation by
chance from year to year as measured by the regression standard error is
0.109, based on data from 1994-2000. |
|
Verification &
Validation: |
FRA’s Office of Safety
has a review process to ensure that railroads and the States comply with
Federal reporting requirements in the preparation of the FRA Railroad Safety Statistics - Annual Report. |
|
Comment: |
None |
|
Measure: |
1. Transit
fatalities per 100 million passenger miles traveled. (CY) 2. Transit
injured persons per 100 million passenger miles traveled. (CY) |
|
Scope: |
The data include both riders and employees. A fatality is defined as a transit-caused
death from collision, personal casualty, fire, derailment, or bus going off
the road. An injury is defined as any
physical damage or harm to a person requiring medical treatment caused by a
transit collision, personal casualty, fire, derailment, or bus going off the
road. |
Source:
|
FTA’s Safety Management Information
System (SAMIS), with data reported by transit operators to the National
Transit Database (NTB).
|
|
Limitations: |
Because of the scope of
the reporting criteria, some fatalities that are counted are not associated
directly with transit operation. This
scope is consistent with the regulatory authority of the agency, but not
consistent with other modes of transportation for comparative purposes. |
|
Statistical Issues: |
The fatality and injury counts in SAMIS are generally quite
accurate---the major source of error in the measure comes from uncertainty in
the passenger miles traveled. Based on
1994-2000 data, the chance variation in a given year has a regression
standard error of 0.039 for the transit fatality rates and 2.210 for the
transit injury rates. |
|
Verification &
Validation: |
An independent auditor and the transit agency’s CEO certify that data
reported to the NTD are accurate.
Using data from the NTD to compile the SAMIS data, the Transportation
Systems Center compares current safety statistics with previous years,
identifies questionable trends, and seeks explanation from operators. |
|
Comment: |
None. |
|
Measure: |
Excavation
damages to natural gas and hazardous liquid pipelines. (FY) |
|
Scope: |
This measure is based on reported hazardous liquid and natural gas
accidents that meet federal reporting criteria as defined in 49 CFR 191.1 and
191.15 for natural gas transmission pipeline incidents and in 49 CFR 195.50
for hazardous liquid pipelines. |
Source:
|
RSPA’s Natural Gas Distribution and Transmission
Incident Reports and Hazardous Liquid Pipeline Accident Reports. Failure
reports are filed within 30 days of the occurrence of reportable
incidents. Complete calendar year
data are available by March 1 of the following year. Data may change as operators file
supplemental reports.
|
|
Limitations: |
RSPA lacks adequate infrastructure information on pipeline operations
and maintenance needed to fully characterize problems when they occur and
lacks information on precursor conditions that contribute to incidents. RSPA seeks further improvements in data
collection in 2002 to address these concerns. |
|
Statistical Issues: |
Reduction in excavation
damages is tied to economic growth and expansion as populations increasingly
are encroaching on once rural areas where major interstate pipelines are
located. Because of delays in mail
delivery associated with 9/11/2001 terrorist activities, statistical
close-out of the 2001 tally requires an extrapolation of number of reports
anticipated for the last quarter of 2001. |
|
Verification &
Validation: |
RSPA reviews/verifies
data provided for accuracy and requests supplemental reports where
shortcomings are indicated. |
|
Comment: |
RSPA discontinues this measure after 2002, replacing this safety
measure with pipeline excavation damages measure. |
|
Measure: |
Failures
of natural gas transmission pipelines. (CY) (2001) |
|
Scope: |
This measure is based on reported hazardous natural gas leaks that
meet federal reporting criteria as defined in 49 CFR 191.1 and 191.15 for
natural gas transmission pipeline incidents.
|
Source:
|
RSPA’s Natural Gas Transmission
Incident Report. Failure reports are filed within 30 days of the occurrence
of reportable incidents. Complete
calendar year data are available by March 1 of the following year. Data may change as operators file
supplemental reports.
|
|
Limitations: |
RSPA lacks adequate infrastructure information on pipeline operations
and maintenance needed to fully characterize problems when they occur and
lacks information on precursor conditions that contribute to incidents. Joint Federal, state and industry teams
have been formed to devise a new course to improve information availability. |
|
Statistical Issues: |
The number of failures of
natural gas transmission pipelines is likely to be underreported. The annual variation in the number of
failures from year to year due to chance has a regression standard error of
528 for natural gas pipeline failures based on data from 1994 to 2000. |
|
Verification &
Validation: |
RSPA reviews/verifies
data provided for accuracy and requests supplemental reports where
shortcomings are indicated. |
|
Comment: |
None. |
|
Measure: |
Number of serious
hazardous materials incidents in transportation. (CY) |
|
Scope: |
Serious reported hazardous
materials incidents were initially defined by RSPA to be those that result in a fatality
or major injury (for most purposes, an injury resulting in hospitalization)
due to a hazardous material, closure of a major transportation artery or
facility, or evacuation of six or more persons due to the presence of a
hazardous material, or a vehicle accident or derailment resulting in the
release of a hazardous material. For
the 2003 Plan, the definition is revised to include those incidents resulting
in a fatality or major injury, the evacuation of 25 or more employees or
responders or any number of the general public, the closure of a major
transportation artery, the alteration of an aircraft flight plan or operation
caused by the release of a hazardous material or the exposure of hazardous
material to fire; plus any release of radioactive materials from Type B
packaging, Risk Group 3 or 4 infectious substance, over 11.9 gallons or 88.2
pounds of a severe marine pollutant, or a bulk quantity (over 119 gallons or
882 pounds) of a hazardous material. This measure tracks only
transportation related releases of hazardous materials that are in
commerce. Volume of spills is not
tracked, as this does not necessarily indicate risk. |
Source:
|
Hazardous Materials
carriers report data to RSPA for entry into the Hazardous Materials
Information System (HMIS).
|
|
Limitations: |
Data for all hazardous
materials incidents is suspected of being incomplete due to under-reporting
for minor incidents. Most reportable
serious incidents are in the system, making this a more consistent measure
for program management. However, it
does not reflect all incidents.
RSPA has issued an NPRM to revise the reporting system. |
|
Statistical Issues: |
Although the
number of incidents is likely to be underreported, such recording error is
probably small in comparison to the annual variation due to chance. The annual variation in the number of
failures (original definition) from year to year due to chance has a regression
standard error of 37.2 based on data from 1994 to 2000. The new incident definition has a
regression standard error of 30.6 based on data from 1997 to 2000. |
|
Verification &
Validation: |
RSPA verifies the data by periodic
follow-up reviews of data entry by the manager of the Hazardous Materials
Information System, and verification audits of the data entry process. RSPA crosswalks HMIS reports against the
National Response Center log of accidents.
RSPA is improving compliance with reporting requirements by
correlating HMIS reports with FRA’s Accident Report data and the HMIS
telephonic data. RSPA is piloting and
plans to incorporate procedures to correlate HMIS reports with FHWA’s
Safetynet Accident File data. |
|
Comment: |
None. |
|
Measure: |
1. Average waiting time
in minutes for passengers in line for screening. (FY) 2. [Measure on passenger
and baggage screening effectiveness.] (FY) |
|
Scope: |
TBD |
Source:
|
TBD
|
|
Limitations: |
|
|
Statistical Issues: |
|
|
Verification &
Validation: |
|
|
Comment: |
|
|
Measure: |
Detection
rate for explosive devices and weapons that may be brought aboard aircraft.
(FY) (2001) |
|
Scope: |
Machine performance test results, automated threat-image
projection (TIP) and FAA field agent testing of aviation security screener
proficiency to detect and resolve images or FAA test objects that simulate
weapons and explosive devices in checked and carry-on baggage, or carried on
the person through an airport security checkpoint. |
Source:
|
FAA Office of Civil Aviation Security Airport
and Air Carriers Information Reporting System (AAIRS). Laboratory test results from the William
J. Hughes Technical Center.
|
|
Limitations: |
No comment. |
|
Statistical Issues: |
There is no major error
present in the subject data. |
|
Verification &
Validation: |
Special “red team”
testing led by agents based at FAA headquarters is used to validate field
test results. AAIRS data is subject
to multiple layers of review.
|
|
Comment: |
The White House Commission recommended more aggressive,
realistic testing. Funding that began
in 1997 enabled an increase in testing as more field agents were hired and
trained. Prior to 1998, data from
realistic testing were too sparse to be conclusive. |
|
Measure: |
Percent
of high interest vessels screened. |
|
Scope: |
High Interest Vessel (HIV) inspection or escort is measured by
the a ratio of the number of HIV vessel inspected or escorted to the number
of HIV vessels arriving at US ports. HIV
designation is determined using specific criteria. Coast Guard inspection or escort standards are to inspect 100%
of HIV. |
Source:
|
The data for this measure is collected
using a manual count from situation reports sent after a vessel inspection or
escort.
|
|
Limitations: |
This is an interim
activity-based measure. Appropriate
outcome-based measures are under development that will improve our ability to
measure and reduce security risks in US ports. |
|
Statistical Issues: |
This is a new measure and
data systems have not yet been developed or modified to capture this
information. It is possible that
errors in the data could result due to manual data collection. |
|
Verification &
Validation: |
Verification and
validation is conducted through cross checks with situation reports. |
|
Comment: |
None. |
|
Measure: |
Percentage
of days that the designated number of critical defense assets (high endurance
cutters, patrol boats, and port security units needed to support Defense
Department operational plans) maintain a combat readiness rating of 2 or
better. (FY) (2001) |
|
Scope: |
Only high endurance cutters, patrol boats, and port security
units that are designated as necessary for defense plans are included. The
specific units required are classified. |
Source:
|
DOD Status of Readiness and Training
System (SORTS) – Database used by the Coast Guard in applying DOD standards
to its assets to determine a readiness score.
|
|
Limitations: |
SORTS uses a multi-factor matrix to calculate the readiness
status. Although specific criteria
are outlined for each factor, some judgment is required in applying
criteria. Different units and personnel
may apply standard criteria in slightly different ways depending on the
nature of the unit’s mission. |
|
Statistical Issues: |
This particular
performance measure in FY 2001 is based on readiness levels of two types of
vessels, patrol boats and high endurance cutters, which have extremely
different levels of readiness. In
addition, a third resource, Port Security Units (PSUs), is measured for its
readiness. PSUs are comprised of
Coast Guard Reservists and Active Duty personnel, trained to protect foreign
ports for expeditionary forces. The
drastic change between FY 1999 and FY 2000 performance was caused in large
part due to the fact that the requirement to report the Contingency Personnel
Requirements List (CPRL) (the full wartime personnel strength requirement) in
the unit SORTS report was waived for FY 2000 and subsequent years pending
validation of personnel requirements that have changed due to new equipment
and operational procedures. The Navy
has been informed of this waiver and has not objected to reporting personnel
strength using the less demanding Coast Guard standards for peacetime
operations in view of the fact that Reserve Unit personnel are available to
quickly bring Coast Guard units up to the full wartime personnel strength requirements
in the event of a war. |
|
Verification &
Validation: |
Units self assess and
report readiness using objective standards.
Unit readiness is periodically validated through inspections,
assistance visits, and in some cases training and assessment at Navy
facilities. These assessments are
conducted by external, field level commands (such as Coast Guard areas,
districts, and groups). |
|
Comment: |
Coast Guard will continue to reassess the overall adequacy of
this measure. |
|
Measure: |
Percentage
of DOD-required shipping capacity complete with crews available within
mobilization timelines (FY) |
|
Scope: |
As of
March 2002, this measure is based on the material availability of 76 ships in
the Maritime Administration’s Ready Reserve Force (RRF) and 115 ships
enrolled in the Voluntary Intermodal Sealift Agreement (VISA) program, which
includes 47 ships enrolled in the Maritime Security Program (MSP). A second factor pertinent to this measure
is the availability of sufficient licensed and unlicensed mariners to operate
the available ships. The performance
measure represents the number of available ships (compared to the total
number of ships in the RRF and VISA) that can be fully crewed within the
established readiness timelines.
While other Government (primarily Military Sealift Command) owned or
controlled sealift type vessels are not included in this measure, they draw
their crews from the same pool of mariners.
Accordingly, the availability measure is adjusted to reflect expected
requirements during the early stages of a military crisis. |
Source:
|
Material availability of ships: MARAD records (and
reports to DOD) on the readiness/availability status of each RRF ship each
month. Typical reasons why a ship is
not materially available include: the ship is in drydock, the ship is
undergoing a scheduled major overhaul, or the ship is undergoing an
unscheduled repair. MARAD and DOD
also maintain records of the sealift ships enrolled in the MSP and VISA and
their crew requirements. Availability of mariners: Information on the
available supply of licensed and unlicensed mariners is extrapolated from
data received from the U.S. Coast Guard’s Merchant Mariner Licensing and
Documentation (MMLD) system.
|
|
Limitations: |
The information on the available
supply of licensed and unlicensed mariners is an estimate. Because the MMLD
also does not contain all of the information on individual mariners contained
in their paper records, and provides no information on the availability and
willingness of individuals to accept a sealift position in an emergency, it
does not provide sufficient assurance of mariner availability. |
|
Statistical Issues: |
None |
|
Verification &
Validation: |
The MARAD Regional Offices (and
contracted ship managers) monitor the condition and overall readiness of each
assigned RRF ship to meet its DOD mission.
When a ship is determined not capable of meeting its activation
timeframe (mission), it is given one of several vessel condition ratings that
are reported to DOD. The monthly
report contains an explanation of the deficiency and an estimated date when
the ship will become fully capable of meeting its mission. MSP contract performance is monitored
throughout the year in order to assure proper payment of the MSP payment to
the ship operators. Recently, MARAD
attempted to validate mariner availability estimates by conducting a survey
of the mariner population. A second
survey is expected to commence in April 2002 to refine and improve the
information needed to determine availability. Because the decision to serve is a matter of individual choice
and is subject to change, MARAD intends to develop a plan for maintaining
current information on mariner availability based on the results of the 2002
mariner survey. |
|
Comment: |
None. |
|
Measure: |
Percentage
of DOD-designated commercial strategic ports for
military use that are available for military use within DOD established
readiness timelines. |
|
Scope: |
The measure consists of the total number of DOD-designated
commercial strategic ports for military use that are assessed as
able to meet DOD-readiness requirements on 48-hour notice, expressed as a
percentage of the total number of DOD-designated commercial strategic
ports. Presently there are 14
DOD-designated commercial strategic ports. Port readiness is based on monthly reports submitted by the ports
and semi-annual port readiness assessments by MARAD in cooperation with other
NPRN partners. The MARAD/DOD
semi-annual port assessments provide data or other information on a variety
of factors, including the following: the capabilities of channels,
anchorages, berths, and pilots/tugboats to handle larger ships; rail access,
rail restrictions, rail ramp offloading areas, and rail storage capacities;
the availability of trained labor gangs and bosses; number and capabilities
of available cranes; long-term leases and contracts for the port facility;
distances from ports to key military installations; intermodal capabilities
for handling containers; highway and rail access; number of port entry gates;
available lighting for night operations; and number and capacity of covered
storage areas and marshalling areas off the port. |
Source:
|
MARAD data are derived from monthly
reports submitted by the commercial strategic
ports and from MARAD/DOD semi-annual port assessments.
|
|
Limitations: |
Port readiness assessments were not made prior to 1995; therefore,
data are available only for 1995 and later years. MARAD conducts a monthly survey of all strategic facilities to
determine whether they meet the DOD availability requirement. This information is provided to MARAD as a
self-assessment by the port agency that owns the facility. There is some degree of subjectivity in
determining the availability of the port facilities. As part of the overall planning process,
MARAD and DOD conduct semiannual visits to independently verify and reassess
port capability and availability. The
indicator is by definition a point-in-time judgment. The results of the monthly and semi-annual
reports used to measure port readiness can vary in accordance with the intensity
of commercial activity at a given port at the time of the assessment. Also, the monthly reports do not include
the same level of detail as the semi-annual assessments, although MARAD is in
continuous contact with port officials to minimize response error. |
|
Statistical Issues: |
The measurement of port
readiness is an overall measure derived from MTMC comments, monthly readiness
reports, and semi-annual assessments.
As such, it is a subjective measure. |
|
Verification &
Validation: |
The MARAD/DOD semi-annual
port visits independently verify and reassess not only the DOD-designated
facilities, but also the total capability of the commercial strategic port. |
|
Comment: |
None. |
|
Measure: |
Ship capacity (in twenty-foot container equivalent
units, or TEUs) available to meet DOD’s requirements for intermodal sealift
capacity. (FY) (2001) |
|
Scope: |
Includes the aggregate TEUs (or estimated square footage) of
cargo capacity for ships enrolled in the Maritime Security Program (MSP) and
Voluntary Intermodal Sealift Agreement (VISA). |
Source:
|
MARAD/USTRANSCOM database of the
militarily useful sealift capacity for ships enrolled in the MSP and VISA programs,
based on vessel capacity data obtained from the vessel operators.
|
|
Limitations: |
MARAD, DOD and operator data on vessel characteristics (e.g., deck
strength in pounds per square feet, deck height, container stowage factors),
which are used to determine the portion of a vessel suitable for carrying
military cargo, are not always consistent.
For example, the majority of ships in MSP/VISA are containerships,
which normally are measured in TEUs; however, DOD generally measures surge
sealift ships, most of which are Roll-on/Roll-off vessels, in square
feet. Historical data prior to FY
1997 are unavailable since the MSP and VISA programs were not enacted until
that year. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
MARAD works with DOD and
the maritime industry to use the most accurate information. MARAD validates the vessel capacity data,
which are obtained from the vessel operators, through comparisons with
internationally recognized databases of vessel characteristics (such as
Lloyd’s Register data), vessel trim and stability information, stowage plans
and other cargo loading documents. |
|
Comment: |
None. |
|
Measure: |
1. Percent
of RRF no-notice activations that meet assigned readiness timelines. (FY)
(2001) 2. Percent
of days that RRF ships are mission-capable while under DOD control. (FY 2001) |
|
Scope: |
DOD conducts no-notice exercises, called “Turbo-Activations,”
annually to assess RRF activation readiness.
The USTRANSCOM, via MSC, randomly selects and orders the activation of
a number of RRF ships on an annual basis to test their capability to be
ready-for-sea (i.e., mission-capable) within their assigned readiness
timeframes of 4, 5, 10, or 20 days. |
Source:
|
MARAD maintains a database on the
number of days it takes to activate each RRF ship and its operational
reliability. The MSC activation order
is received either by phone call or message.
Documents produced during the no-notice activation period comprise the
data source for determining the amount of time taken to activate each
ship. Non-performance time is based
on the MSC Casualty Reporting (CASREP) system, which identifies casualties
that are of a severity to prevent the ship from performing the mission. These messages are passed from the ship's
Captain to MSC, the Ship Manager, and MARAD.
The reliability of activated RRF ships is measured as the
percent of days that RRF ships are mission-capable while under DOD
control. Mission-capability is
determined, in part, by the number of days it takes to repair a ship that has
become inoperative. For example, the low percent of
mission capability in 1997 (95.2) was the result of one ship being out of
service for 156 days while undergoing repairs.
|
|
Limitations: |
None. |
|
Statistical Issues: |
Since the population of
vessels covered by these measures often consists of a very small number of
vessels (as low as 13 vessels in FY 2000), a large swing in results can occur
from just one ship not being available on time or one ship having operational
problems. |
|
Verification &
Validation: |
The source of the
activation data is the actual activation order from DOD to MARAD and the
documents produced during a no-notice activation. These fix the actual time of call-up and the time when the
vessel is "Ready for Sea" (or tendered to MSC). The Ready for Sea time is agreed to by
MARAD and the on-board MSC representative and reported to DOD by official
message. The time taken to activate
each ship is maintained in the ship’s logbook and in official DOD, MSC, and
MARAD records. The collection of data
regarding mission capability under MSC operational control starts when MSC
officially accepts delivery of RRF ships with date and time
documentation. The Captain of the
ship reports all problems that are of a severity to prevent the ship from
performing its mission to MSC, the Ship Manager, and MARAD. The Captain also reports when the problem
has been corrected. This information
is entered by MSC into its CASREP system. |
|
Measure: |
Of
the mariners needed to crew combined sealift and commercial fleets during national
emergencies, the percent of the total that are available. (FY) (2001) |
|
Scope: |
The pool
of licensed and unlicensed mariners consists of mariners that have had sea
service on U.S.-flag oceangoing merchant vessels over 1,000 gross tons within
five years. The mariner pool includes
licensed and unlicensed actively sailing mariners and inactive mariners,
employed shoreside, who have the necessary skills and retain the appropriate
license/rating to operate sealift ships, defined by shipboard position and
U.S. Coast Guard certification. This
pool is then compared to the DOD and commercial manpower requirements to
determine sufficiency of the labor force.
Only oceangoing merchant vessels over 1,000 gross tons are considered
because mariners on these vessels have skills required for emergency sealift
operations. The targets are based on
a sealift operation that extends beyond 6 months, necessitating relief for
the mariners who were sailing at the start-up of the operation. |
Source:
|
U.S. Coast Guard Merchant
Mariner Licensing and Documentation (MMLD) system. The Coast Guard is the lead Federal agency
for regulating, licensing, and documenting professional merchant
mariners. MMLD provides information
on both actively sailing mariners and inactive mariners, including their
skill level and seafaring employment. Lloyd’s Maritime
Information Systems. MARAD obtains information to
track the use of U.S.-flag commercial ships active in international trade and
projects the size of the active, ocean-going, commercial fleet. The size of this fleet has a direct
correlation to the size of the commercial pool of mariners, based upon
commercial crewing rules. MARAD/DOT Mariner
Survey. New for
FY 2001, a random sample of mariners with current qualifications is now being
surveyed, in conjunction with the Bureau of Transportation Statistics. The Survey will provide a more accurate
determination of the number of currently qualified mariners as well as
information on mariner availability for sealift employment during national
defense emergencies.
|
|
Limitations: |
The size
of the active and inactive mariner pool can be estimated from the MMLD. MARAD integrates these data into its own
system for analysis and reporting.
Because the MMLD does not contain all of the information on individual
mariners contained in their paper records, and provides no information on the
availability and willingness of individuals to accept a sealift position in
an emergency, it does not provide sufficient assurance of mariner
availability. |
|
Statistical Issues: |
The
primary area of uncertainty lies within the MMLD system, which MARAD uses to
define the population of available mariners.
While the accuracy of the data continues to improve as all licenses
and documents are now on a five-year renewal cycle, gaps still exist in the
database. Because the MMLD system was not designed to contact mariners,
address and telephone information in the system is incomplete and
out-of-date. Also, operators of some
large oceangoing vessels are not required to report mariner employment to the
USCG, and evidence of sea service provided by individual mariners to fulfill
requirements for upgrading their rating is not entered in the MMLD. |
|
Verification &
Validation: |
The MMLD
system is currently the only source of electronic data on mariner
qualifications and employment. MARAD
continues to work with the USCG to improve the MMLD system. The new MARAD/DOT Mariner Survey data will
be used to estimate the number of qualified mariners available and willing to
support sealift crewing positions.
Because this determination is a matter of individual choice and is
subject to change, MARAD intends to develop a plan for maintaining current
information on mariner availability based on the results of the Survey. |
|
Comment: |
None. |
|
Measure: |
Amount
of drugs seized or destroyed at sea (metric tons). (FY) |
|
Scope: |
Total amount of drugs (cocaine, marijuana,
hashish, heroin, etc.) seized, jettisoned, or destroyed at sea by the United
States Coast Guard. Cocaine currently
constitutes the largest drug threat to the U.S., but the Coast Guard seeks to
interdict all illegal narcotics moving by non-commercial maritime
conveyances. |
Source:
|
The amount of drugs seized is measured
by Coast Guard crews and reported through the Coast Guard Law Enforcement Drug Interdiction Data
Base. Seizures are officially credited
to the Coast Guard via Federal Drug Identification Numbers (FDINs) and are
recorded in the federal Consolidated Counter-Drug Data Base, which is
administered by the U.S. Interdiction Coordinator (USIC). |
|
Limitations: |
It is possible that non-entry,
duplication, and coding errors are present in seizure amount data; however,
the chance of this error is small. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
Verification and validation
occurs in several places in the data reporting and collection process. Data entry software helps ensure data
quality and consistency by employing selection lists and logic checks. Internal analysis and review of published
data by external parties help identify errors. CG data is further reviewed at a quarterly
Consolidated Counter-Drug Data Base Conference, where all agencies that input
data into the database review all agency data for consistency and accuracy.
|
|
Comment: |
This measure aligns with the goals contained in the President’s
National Drug Control Strategy. |
|
Measure: |
Seizure
rate for cocaine that is shipped through the transit zone. (FY) (2001) |
|
Scope: |
Seizure rate is a measure consisting of the
amount of cocaine seized by the Coast Guard divided by the noncommercial
maritime cocaine flow, expressed as a percentage. Noncommercial is defined as any vessel or aircraft not engaged
in port-to-port transfer of cargo/passengers for the primary purpose of
business profit. Examples are pleasure craft, fishing vessels, offshore
work-boats, or freighters carrying cargo as a means of disguising illegal
drugs. |
Source:
|
The amount of
cocaine flow shipped by non-commercial means through the transit zone is
estimated in the Interagency Assessment of Cocaine Movement (IACM) published
by the Office of National Drug Control Policy (ONDCP). The amount of cocaine seized is measured
by Coast Guard crews and reported through the Coast Guard Law Enforcement Information
System.
|
|
Limitations: |
It is probable that
non-entry, duplication, and coding errors are present in seizure amount data
(numerator); however, this error is likely to be small. The cocaine flow amount (denominator) is
estimated through a complex process using many different sources of
information. Due to the secretive nature of the illegal drug trade, cocaine
flow estimates may contain significant errors. The size of this error may vary from year to year; the extent
of this is not known. The estimation process changes slightly each year as
improvements are made, so year-to-year comparisons of the flow are not
completely consistent. The accuracy of the official cocaine flow estimate has
been questioned by some individuals and organizations outside of government
that have an interest in U.S. drug policy.
ONDCP continuously attempts to refine this estimate to improve the
measurement of interdiction activities.
This measure only addresses cocaine; formal flow assessments do not
exist for other major drugs. This
measure is not designed to include cocaine shipped by commercial means such
as large container vessels; however, it is probable that a small amount of
cocaine included in the numerator is actually related to commercial
shipping. This distinction between
commercial and noncommercial is better for program management; at-sea
interdiction of commercially conveyed cocaine, particularly when shipped in
containers, is extremely difficult, and not the focus of the Coast Guard
program. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating seizure rate for cocaine is the denominator. Although the numerator estimate of cocaine
seized is relatively accurate, the estimate of the amount shipped in the
denominator is far more variable. The regression standard error for
year-to-year chance variation in the cocaine seizure rate is 4.0 percent,
based on data from 1995 through 2000.
|
|
Verification &
Validation: |
Verification and
validation occurs in several places in the data reporting and collection
process. Data entry software helps
ensure data quality and consistency by employing selection lists and logic
checks. Internal analysis and review
of published data by external parties help identify errors. |
|
Comment: |
This measure is consistent with the goals contained in the
President’s National Drug Control Strategy. |
|
Measure: |
Interdict
and/or deter at least 87 percent of undocumented migrants who consider attempting
to enter the U. S. via maritime routes. (FY) |
|
Scope: |
Measure includes Cuban, Dominican, Haitian, and Chinese
migrants, as these are the primary groups using maritime channels and the
groups for which formal flow estimates exist. Success rate is the estimated number arriving by maritime
channels divided by those that pose a threat of migration (estimated
intent). The interdiction rate is
just 1 minus the success rate. |
Source:
|
Data is obtained from Coast Guard and from
the Immigration and Naturalization Service (INS). Estimates of migrants who successfully arrive and estimates of
those with a high potential for undertaking the voyage are derived (with a
consistent methodology) from investigations of incidents, interviews of
detainees, and intelligence gathering.
Sources for this information are the Coast Guard, INS, and other
authorities.
|
|
Limitations: |
The numbers of illegal
migrants entering the U.S., and the numbers of potential migrants, are
derived numbers subject to estimating error.
Because of the speculative nature of the information used, and the
secretive nature of illegal migration, particularly where professional
smuggling organizations are involved, the estimated potential flow of
migrants may contain significant error.
The measure only tracks four migrant groups at this time. A small
number of migrants (approximately 10%) from various source countries are not
included because formal flow estimates of migrants leaving these countries
are not available. Using the number
of potential migrants in the denominator helps address the deterrence value
of Coast Guard operations, but could lead to confusion of this measure with a
simple interdiction rate. Trend information prior to 1995 is not available. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating the success rate for undocumented migrants is the
denominator, which is an estimate of the flow of migrants, both documented
and undocumented. |
|
Verification &
Validation: |
The numbers of migrants
reaching the U.S. via maritime routes and the number of “potential migrants”
are estimated. Methodologies and data
are continuously reviewed. The Coast
Guard has developed the estimation techniques that support this indicator
over the last six years in order to more consistently use intelligence
information. They are seeking
independent assessment of the methods, and look to improve the process in the
future. |
|
Comment: |
Partly because maritime threats of illegal migration have come
from a limited number of sources, the Coast Guard and others have developed
quantified threat estimates to better manage interdiction. Over the past six years, estimation
techniques have been improved to remove as much subjectivity and inconsistency
as possible. It should be noted that past information reflects the success of
intentional illegal activity. While
some DOT measures allow accurate projection of likely future outcomes, the
highly variable nature of illegal migrant activity limits the ability to
project future outcomes based on performance in the immediate past. |
|
Measure: |
Of
those who need to act, percent who receive threat information within 24
hours. (FY) (2001) |
|
Scope: |
Threat information, in this context, is defined as credible
information (both time-sensitive/action-oriented and informational) received
by the Intelligence Community, analyzed by OIS and distributed in the form of
a Transportation Security Information Report, generated by OIS for
distribution by the Operating Administrations (OAs). Figure is derived from the percentage of
transportation security officials and industry representatives that receive
threat information from OIS through the OAs within the 24-hour period. Security representatives and officials
will be randomly sampled by OIS within 48 hours of information dissemination
and asked if and how soon they received the subject material. |
Source:
|
Internally prepared. Survey conducted by OIS of both DOT
personnel and industry security contacts.
|
|
Limitations: |
Data: Relies on the reporting of the customers
and consumers of this information.
Reporting could be skewed to reflect positively on the dissemination
process within the Operating Administrations. Indicator: This measure only identifies whether there
are possible breakdowns and bottlenecks in the dissemination process. It does not identify where those
breakdowns specifically may be in the dissemination chain. |
|
Statistical Issues: |
Since these data are
collected through a sample survey, they are subject to sampling and
non-sampling errors. |
|
Verification &
Validation: |
Customers will be randomly
surveyed at all levels within the dissemination process, not solely the end
users. Consequently, the reporting of
dissemination times and officials who are in receipt of the information can
be crosschecked for verification and validity of data. |
|
Comment: |
None. |
|
Measure: |
Transportation-related
petroleum consumption (in quadrillion BTUs) per trillion dollars of Real
Gross Domestic Product (GDP). (CY) (2001) |
|
Scope: |
Measure includes primary consumption of petroleum for
transportation, in quadrillion BTUs.
This does not account for petroleum-produced electricity that is used
in transportation; however, this is less than 1% of petroleum use. Petroleum use is normalized to real GDP,
in constant 1996 dollars. |
Source:
|
U.S. Department of Energy Annual Energy Review 1999 and Annual Energy Review 2000.
|
|
Limitations: |
Energy consumption does not include petroleum-produced transportation electricity. Measure does not capture the fraction of
this petroleum use that is imported, nor does it capture actual energy
efficiency (BTUs per
passenger-mile-traveled). |
|
Statistical Issues: |
These data are external
to DOT. They are subject to both
sampling and nonsampling errors.
Based on 1994-2000 data, chance variation from year to year in the
transportation energy efficiency measure has a regression standard error of
0.016. |
|
Verification &
Validation: |
Data is taken from external
sources, which conduct their own verification and validation. |
|
Comment: |
Petroleum use is normalized to the nation’s real GDP in order to
capture the nation’s economic exposure to petroleum use in
transportation. Beginning in 1999,
the GDP baseline was changed from constant 1992 dollars to 1996 dollars. |
Highway
infrastructure condition Page
60
|
Measure: |
Percentage of travel on
the National Highway System (NHS) meeting pavement performance standards for
acceptable ride. (CY) |
|
|
Scope: |
Data include vehicle miles traveled on the HPMS reported NHS sections
and pavement ride quality data reported using the International Roughness Index
(IRI). IRI is a quantitative measure of the accumulated response of a
"quarter-car" vehicle suspension experienced while traveling over a
pavement. Vehicle Miles of Travel (VMT) represent the total number of vehicle
miles traveled by motor vehicles on public roadways within the 50 states and
Washington, D.C. |
|
Source:
|
Data collected by the State Highway Agencies and reported to FHWA for
the Highway Performance Monitoring System (HPMS). They are obtained from calibrated measurement devices that meet
industry set standards. Measurement
procedures are included in the HPMS Field Manual. VMT is a calculated product of the annual average daily traffic
(AADT) and the centerline length of the section for which the AADT is reported.
In the HPMS, travel is accumulated for each universe section to develop
appropriate totals for the higher functional systems. AADT is required for
each section of Interstate, NHS, and other principal arterial; as a result,
travel is computed for these functional systems on a 100-percent basis. For
minor arterial, rural major collector and urban collector systems, travel is
calculated from samples using the AADT, centerline length reported for each
sample section and the HPMS sample expansion factor for each section. Travel
for the NHS on all functional systems is computed from the universe AADT
data. For the most part, travel for the rural minor collector and
rural/urban local functional systems is calculated by the States using their
own procedures and is provided in HPMS on a summary basis. Some States use
supplemental traffic counts outside of the HPMS procedures; others employ
estimating techniques, such as fuel use, to determine travel on these
systems. In general, these methods are used in both rural and urban areas,
including the donut areas of nonattainment areas to meet Clean Air Act
requirements. |
|
|
Limitations: |
IRI
data for the approved NHS exist from 1995 onward. Past data (1993 and 1994)
contain some variation as this data was on the proposed, rather than the
existing NHS. No NHS IRI data are available prior to 1993. The HPMS requires States to report IRI
data every two years; however, following the requirements is not mandated,
but voluntary. VMT
estimates reported via the HPMS should be of reasonable quality particularly
for the higher order functional systems. AADT and travel data are edited by
the HPMS software for unusual values and for unusual changes to previously
reported values. FHWA routinely works with State data providers to modify
reported AADT values that do not appear to be reasonable before final use.
Although AADT is required to be updated annually in HPMS, counts are only
required to be updated on a 3-year cycle. For any reporting year, AADT for
uncounted sections is usually derived by factoring the latest year's count
for those sections. |
|
|
Statistical Issues: |
The
major source of error in the percentages is probably the sampling error from
selecting the segments of highway tested for smoothness. VMT
data are subject to sampling errors, whose magnitude depends on how well the
locations of the continuous counting locations represent nationwide traffic
rates. HPMS is also subject to
estimating differences in the states, even though FHWA works to minimize such
differences and differing projections on growth, population, and economic
conditions which impact driving behavior. |
|
|
Verification &
Validation: |
FHWA
validates the data based on consistency reviews. States that follow the HPMS
sampling instructions in developing traffic counting programs (Appendix F in
the HPMS Field Manual) and the practices advocated in the Traffic Monitoring
Guide have adequate counting and classification tools to prepare quality AADT
and travel estimates for HPMS. The consistency of the sampling and counting
procedures should also provide comparable State-to-State traffic data. |
|
|
Comment: |
None. |
|
|
Measure: |
Percentage of miles on
the National Highway System (NHS) that meet pavement performance standards for acceptable ride. (CY) (2001) |
|
|
Scope: |
International Roughness Index (IRI) is compiled annually for every
section of the NHS, using data reported from the States. |
|
Source:
|
Data collected by the State Highway Agencies and reported to FHWA for
the Highway Performance Monitoring System (HPMS). They are obtained from calibrated measurement devices that meet
industry set standards. Measurement
procedures are included in the HPMS Field Manual. |
|
|
Limitations: |
IRI
data for the approved NHS exist from 1995 onward. Past data (1993 and 1994)
contain some variation as this data was on the proposed, rather than the
existing NHS. No NHS IRI data are available prior to 1993. The HPMS requires States to report IRI
data every two years; however, following the requirements is not mandated,
but voluntary. |
|
|
Statistical Issues: |
The
major source of error in the percentages is probably the sampling error from
selecting the segments of highway tested for smoothness. The annual variation in the percentage due
to chance has a regression standard error of approximately 0.44 percent based
on data from 1995-2000. |
|
|
Verification &
Validation: |
FHWA
validates the data based on consistency reviews |
|
|
Comment: |
None. |
|
|
Measure: |
Percentage
of deficient bridges on the NHS. (CY) (2001) |
|
Scope: |
Measure includes the number of deficient (structurally deficient
and functionally obsolete) bridges on the NHS functional system divided by
the total number of NHS bridges in the inventory, expressed as a
percent. |
Source:
|
Bridge information is collected by
State DOTs and other bridge owners and provided to FHWA annually for
inclusion in the FHWA maintained National Bridge Inventory (NBI).
|
|
Limitations: |
NBI includes information
on all 114,567 NHS bridges. States are
required to update the system annually, but many States update
quarterly. The system contains 95
data items for each of the bridges, and 20 of these items relate to bridge
condition and appraisal. There are
specific instructions as to how to assess bridges based on these items,
including a grading scale from 0 to 9 with specific definitions and specific
criteria to follow |
|
Statistical Issues: |
Even with the item
specific grading system, differences in the grading between individual
inspectors and between inspection days are probably the largest component of
variation in the percentages. Based
on 1994-2000 data, the estimated regression standard error for year-to-year
variation in the percentages due to chance is approximately 0.65 percent. |
|
Verification &
Validation: |
DOT evaluates accuracy
and reliability of the submitted NBI information through data checks and
field reviews by both Headquarter and field office personnel. This is done as a part of FHWA’s NBI, the
National Bridge Inventory System (NBIS), and Highway Bridge Replacement and
Rehabilitation Program. Evaluation of
the State’s compliance with the NBIS most often includes a sample of bridge
inspection reports and a comparison of condition data with field visits to
the bridge site. In addition, there
is an edit update program that identifies potential data errors in the NBIS. |
|
Comment: |
None. |
|
Measure: |
Miles
of the Appalachian Development Highway System (ADHS) completed. (FY) (2001) |
|
Scope: |
Measure includes actual miles completed on the 3,025 mile ADHS,
within 13 member States. |
Source:
|
States submit annual status updates on
ADHS miles completed within their State each fiscal year to the Appalachian
Regional Commission (ARC). The ARC
compiles the data.
|
|
Limitations: |
This is an output
measure. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
Completed by ARC. |
|
Comment: |
ARC estimates that the TEA-21 funding level will result in
completion of approximately 37 additional miles each FY 1999 through 2003. |
|
Measure: |
Number
of metropolitan areas where integrated ITS infrastructure is deployed. (FY)
(2001) |
|
Scope: |
The level of integrated deployment in 75 of the nation’s largest
metropolitan areas has been established using a set of indicators that
consider two factors: (1) How much ITS infrastructure is in place at each
metropolitan area; and, (2) How much integration is going on at each
area. The process for determining the
level of “component” deployment in a metropolitan area employs a set of
indicators that measure the magnitude of deployment for selected ITS
components. These are typically
expressed as a ratio of actual deployment divided by the total possible, for
example the number of freeway miles under electronic surveillance divided by
the total freeway mileage. Components
are considered deployed once the level of deployment attains a specified
threshold level based on the indicators.
Integration is defined as the sharing of data between agencies
associated with the different jurisdictions responsible for ITS
infrastructure. Typically there are
three: State DOTs responsible for management of freeways and incident
management programs; city governments, which manage most of the traffic
signal systems; and public transit authorities, which manage most bus and
rail services. The level of
integration is determined by the extent that these three major transportation
organizations employ technology to share and use transportation data to
increase system capacity. Two
examples of integration are: 1) a city traffic signal system receiving data
from the state freeway management center about the queues at freeway ramp
meters and then adjusting the signal timings on the arterial streets, or 2) a
transit agency providing the state freeway management center with the
real-time location of the buses so that freeway speeds can be determined. Metropolitan areas are rated as low,
medium, or high separately for deployment and integration and then assigned
an overall combined rating. An
overall score of medium or high meets the goal for a metropolitan area. |
|
Source: |
Metropolitan ITS
Deployment Tracking Database developed by the Oak Ridge National Laboratory
for the ITS Joint Program Office.
Data are collected by means of surveys from designated metropolitan
areas. |
|
Limitations: |
This indicator is designed
to track and encourage basic steps toward component deployment and systems
integration. However, it does not
reflect the full breadth of deployment or integration activities. For example, while it establishes the
existence of basic integration of essential components, it does not confirm
that all possible or desirable integration links exist in a metropolitan
area. Similarly, the attainment of a
deployment threshold only confirms a substantial commitment to the use of ITS
technology but does not indicate that all needed deployment is complete. |
|
Statistical Issues: |
These data come from
sample surveys that, like all sample surveys, contain sampling and
non-sampling errors. |
|
Verification & Validation: |
The DOT Joint Program
Office reviews deployment tracking indicators and methodology. Results are distributed to DOT
headquarters and field staff as well as to state and local survey responders
for confirmation of accuracy and completeness before the final reports are
issued. Independent experts in
statistics and transportation review procedures for survey construction and
data collection prior to each survey iteration. A steering committee of Federal, state, and local
transportation officials review and approve changes to methodology and indicators
prior to implementation. |
|
Comment: |
The FY 1997 baseline is
36 areas. |
|
Measure: |
1. Billion transit passenger miles traveled.
(CY) (2001) 2. Average percent change in transit passenger-miles
traveled per transit market. (FY) |
|
Scope: |
Includes revenue-passenger miles on publicly sponsored bus,
transit rail, commuter rail, ferry, and vanpools in urbanized areas. |
Source:
|
National Transit Database
(NTD), with information gathered from transit operators. |
|
Limitations: |
Data is self-reported by
transit agencies using an FTA-approved sampling methodology. Although most data is reported in the
National Transit Database each year, sample cycles may be annual, every three
years, or every five years depending on the size of the urban area and the
number of vehicles operated.
Ridership is an outcome indicator that reflects a variety of factors,
including the capital investment of the Federal Government. Ridership is also influenced by
operational decisions of transit authorities, and the availability and cost
of alternative modes of transportation. |
|
Statistical Issues: |
The sources of uncertainty include sampling error, annual chance variation,
and auditing issues. The regression
standard error from 1994-2000 indicates that the magnitude of the combination
of the first two sources of error is approximately 0.67. |
|
Verification &
Validation: |
An independent auditor and the transit agency’s CEO certify that data
reported to the NTD are accurate. FTA
also compares data to key indicators such as vehicle revenue miles, number of
buses in service during peak periods, etc. |
|
Comment: |
None. |
|
Measure: |
Percentage
of transit grants obligated within 60 days after submission of a completed
application. |
|
Scope: |
FTA grants
obligated during a fiscal year period for major programs: Urbanized area,
non-Urbanized area, and Elderly and Persons with Disabilities formula grants;
Capital grants; Job Access and Reverse Commute grants; Over-The-Road Bus
grants; and Planning grants. |
Source:
|
FTA TEAM database. |
|
Limitations: |
Several factors that contribute
to grant delays are beyond FTA’s ability to control. These factors include
the processing of flexible funds from FHWA through the Treasury, and the
Congressional grant release process. |
|
Statistical Issues: |
Processing time is calculated from submission date to obligation
date. $0 dollar non-funding grant amendments are excluded from analysis. |
|
Verification &
Validation: |
TEAM output file is crosschecked against other system generated files
for consistency; inconsistencies are investigated and corrected prior to
reporting. Grants with missing or out-of-sequence dates are excluded for
calculating averages. |
|
Comment: |
An FTA task force meeting was held in February 2002 to identify causes
for grant processing delays. The resulting action plan is now being
circulated for final review and approval. Implementation of the plan will
follow. |
|
Measure: |
Percentage
of on-time flights. (FY) |
|
Scope: |
The time of arrival of
completed, scheduled passenger flights to and from the 32 DOT large-hub
airports is compared to their scheduled time of arrival. The sum of flights arriving on or before
15 minutes of scheduled arrival time is divided by the total number of
completed flights. |
Source:
|
The Aviation System Performance Metrics (ASPM) database,
maintained by the FAA’s Office of Aviation Policy and Plans. By agreement with the FAA, ASPM flight data
is filed by certain major air carriers for all flights to and from 21 large
and medium hubs, and is supplemented by flight records contained in the
Enhanced Traffic Management System (ETMS) and flight movement times provided
by Aeronautical Radio, Inc. (AIRINC).
Data are sufficient to complete ASPM data files for 49 airports. |
|
Statistical Issues: |
There is little major error in the count of completed
flights or the count of flights that arrive on-time. |
|
Limitations: |
Some
ASPM data is constructed from ETMS records, a small portion of which may not
be maintained in FAA traffic control computers when they are under heavy use.
|
|
Verification &
Validation: |
Flight
data is extracted from the Official Airline Guide (OAG) and compared
to& data from carrier records, which contains carrier computer
reservation flight schedule data.
Summary data is compared to data filed monthly with DOT under 14 CFR
Part 234, Airline Service Quality Performance Reports, which
separately requires reporting by major air carriers on flights to and from
the 32 large hubs. |
|
Comment: |
FAA’s percentage of flights
arriving on-time derived from ASPM data differs only by fractions of a
percent from the on-time percentage derived from DOT’s slightly different
database. |
|
Measure: |
Aviation
delays per 100,000 activities. (FY) (2001) |
|
Scope: |
An FAA reported
delay occurs when an aircraft is delayed fifteen minutes or more because of
constraints that prevent the aircraft from making a scheduled landing. Delays are counted in five categories: FAA
equipment, volume, weather, runway related, and other. Delays due to airline equipment are not
considered. “Activities” are total facility activities, as defined in
Aviation System Indicators 1998 Annual Report. Total facility activities are the sum of en route and terminal
facility activities. |
Source:
|
FAA air traffic facilities report the data to
headquarters, which incorporates the data into the Air Traffic Operations
Management System.
|
|
Statistical Issues: |
There is no major error in either the delay counts (numerator) or the
flight operations data (denominator) for this rate. However, random variation in aviation delays results in a
significant variation in the delay rate from year to year. The regression standard
error in the delay rate, based on 1994-2000 data, is approximately 20.0. |
|
Limitations: |
By collecting information
on delays of fifteen minutes or more, FAA does not capture the aggregate amount
of system delay, but only the most significant delays. |
|
Verification &
Validation: |
Data is analyzed and
checked by an Air Traffic Service headquarters office on a daily basis to
ensure accuracy of the information reported. |
|
Comment: |
Total delays in
all five categories are what the traveling public experience. |
|
Measure: |
Cumulative
increase in throughput during peak periods at certain major airports. (FY)
(2001) |
|
Scope: |
This measure
focuses on the arrival rates during peak traffic periods comparing pre-CTAS
rates to post CTAS rates. |
Source:
|
Radar system (HOST and ARTS) data is collected and aircraft
flight tracks are obtained from those systems and analyzed to determine
arrival and departure times.
|
|
Limitations: |
The radar systems produce
very large data files requiring a substantial effort to extract relevant data
for analysis. The extracted data sets need to be of sufficient size to
produce statistically significant results. |
|
Statistical Issues: |
Conditions (weather,
runways in use, aircraft mix) vary, affecting rates. Data must be normalized
and data sets must be of sufficient size to produce valid results. |
|
Verification &
Validation: |
Methodologies and
detailed results are available for review in semi-annual FFP1 Metrics Updates
(December and June). Results are
coordinated with FAA and User stakeholders. |
|
Comment: |
None. |
|
Measure: |
Cumulative
increase in direct routings for en route flight phase. (FY) (2001) |
|
Scope: |
This measure
focuses on the number of direct routings provided by en route controllers
comparing pre and post-URET installation. |
Source:
|
URET provides data on routing amendments, which is then
analyzed to determine the number of direct amendments.
|
|
Limitations: |
The radar systems produce
very large data files requiring a substantial effort to extract relevant data
for analysis. The extracted data sets need to be of sufficient size to
produce statistically significant results. |
|
Statistical Issues: |
Extreme weather
conditions, particularly thunderstorms, will significantly affect routing
amendments; therefore, data is sampled for days when weather is not a factor. |
|
Verification &
Validation: |
Methodologies and
detailed results are available for review in semi-annual FFP1 Metrics Updates
(December and June). Results are
coordinated with FAA and User stakeholders. |
|
Comment: |
None. |
|
Measure: |
Percent
of runways in good or fair condition (commercial service, reliever, and
selected general aviation airports). (CY)
(2001)
|
|
Scope: |
Paved runways at the 3,300+ airports in FAA’s National Plan of
Integrated Airport Systems (NPIAS) are assessed for pavement condition. The NPIAS airports include all commercial
service and reliever airports and those general aviation airports that are
significant to national air transportation. |
Source:
|
The FAA’s Airport Safety Data Program
(ASDP) provides extensive data about the facilities that are available at
public-use airports. Data are provided
approximately annually by FAA inspectors for airports certified under FAR
139. Data for other airports,
including most public use general aviation airports, are provided under an
FAA contract.
|
|
Limitations: |
FAA contracts for a
visual survey of the runways to categorize their condition based on criteria
developed by the FAA Office of Airports.
“Good” condition means all cracks and joints are sealed; “fair”
condition means there is mild surface cracking, unsealed joints, and slab
edge spalling; and “poor” condition means there are large open cracks,
surface and edge spalling, and vegetation growing through cracks and
joints. Since the reports are based
on a visual inspection, underlying drainage or strength problems are not
reported. However, these problems
normally create surface defects that are visible. The more detailed pavement condition index (PCI) inspections
require a section-by-section examination of the runway rather than an overall
assessment used for this performance measure. FAA has been aggregating the
ADSP data from all NPIAS airports only every several years for inclusion in
the NPIAS report to Congress. This
information exists for 1993, 1997, and 1998. |
|
Statistical Issues: |
Less than half of the
ADSP records were updated during CY 2000.
The relatively subjective nature of judging pavement quality means
this measure is also subject to random variation due to measurement error. |
|
Verification &
Validation: |
Efforts continue to
correlate PCI and ADSP data. |
|
Comment: |
A contract was initiated in FY 2001 to coordinate efforts by
state agencies to conduct safety inspections at selected general aviation
airports. |
|
Measure: |
Number
of runways that are accessible in low visibility conditions. (FY) (2001) |
|
Scope: |
This performance measure counts the total number of airport
runways with published ground-based and/or satellite-based landing
systems. The intent of this measure
is to reflect increased accessibility using satellite-based technology for
vertically guided approaches. |
Source:
|
Internal FAA Aviation Systems
Standards tracking system.
|
|
Limitations: |
Increasing the number of runways
with satellite- based landing systems as well as augmenting existing
satellite-based landing systems with vertical altitude guidance will improve
access to airports and increase schedule reliability. Both improved access and increased
reliability are considered benefits to the aviation industry and the
individual air traveler. However,
individual use of landing systems is not tracked by current FAA information
systems. In addition, aircraft must
be appropriately equipped to use the new technology. The FAA does not track these equipment
additions. |
|
Statistical Issues: |
There is no major error in the counts of published landing
systems. However, like the above measure, random changes in the number of published approaches result in
random variation in the count from year to year. |
|
Verification &
Validation: |
The number of airport
runways with a satellite-based landing system is computed monthly by Aviation
Systems Standards. |
|
Comment: |
None. |
|
Measure: |
Total
number of commercial vessel collisions, allisions, and groundings. (FY) |
|
Scope: |
The measure includes all commercial ships regardless of tonnage.
Intentional groundings are excluded. “Allisions”
refers to incidents wherein ships collide with a fixed object such as a
bridge or aid to navigation. |
Source:
|
Coast Guard Marine Safety Information
System (MSIS). Sources of reports are most often vessel masters, operators, owners,
insurance companies, legal representatives, and other mariners. Collisions,
Allisions, and Groundings are reported to the Coast Guard as required by
federal regulations.
|
|
Limitations: |
The investigation, retrieval, analysis and reporting processes
result in under-reporting for the most recent year, with the most significant
effects over the most recent 5 months.
Estimates are often used to compensate for this known data-lag. It is probable that some collisions,
allisions and groundings are not reported to the Coast Guard. This number is
unknown. Serious events such as major
collisions and hard groundings are more likely to be reported than minor
events such as a temporary grounding where a vessel could remove itself
without assistance. Duplicate event
entries are sometimes entered into MSIS, and some events are mistakenly
omitted or coded incorrectly.
Verification procedures strive to correct these errors, but it is
probable that a small number are not corrected. Because this count of
incidents is not normalized to exposure, it does not provide a sensitive
indicator of changes in risk. |
|
Statistical Issues: |
The major sources of
uncertainty in these measures are the estimation error (as a result of the data-lag) the response error (as a result of parties failing to report
casualties to the Coast Guard), and recording error (based on differences in
the training and judgment of Coast Guard investigators in recording the
accident). The regression standard error for year-to-year chance variation in
the number of collisions, allisions and groundings under the new measure is
approximately 70, based on data from 1996 through 2000. |
|
Verification &
Validation: |
Verification and
validation occur at several levels. Edit
checks within MSIS software can detect some incorrect or missing data and
force review and correction before data entry is completed. Selection lists for certain data fields
also reduce the opportunity for data entry error. All investigations go through one level of review at the field
unit for accuracy. Investigations of
serious marine casualties are also usually reviewed at district and
headquarters offices. The headquarters
Data Administration staff conducts periodic quality control checks to identify
entry errors such as missing data or miscoding, and corrects any errors
identified. Each investigation
involving a vessel accident is reviewed before it is included in the measure. Errors identified are referred to either
the Data Administration staff or the Investigations and Analysis staff for
correction. |
|
Comment: |
During FY 2002, the
Marine Safety Information System (MSIS) will be replaced by the Marine
Information System for Safety and Law Enforcement (MISLE). While the new system will be a substantial
improvement, it is expected to cause serious difficulties in making
performance comparisons. One factor
is that many business processes were re-designed in conjunction with system
development. Another factor is that
data quality under MISLE is expected to be superior to that of MSIS. While this represents improvement, it may
cause near-term problems in making meaningful comparisons of data between the
two systems. |
|
Measure: |
Percentage of days in the shipping season that the U.S. sectors of
the St. Lawrence Seaway locks are available, including the two U.S. Seaway
locks in Massena, N.Y. (CY) |
|
Scope: |
The
availability and reliability of the U.S. sectors of the St. Lawrence Seaway,
including the two U.S. Seaway locks in Massena, N.Y., are critical to
continuous commercial shipping during the navigation season (late March to
late December). System downtime due
to any condition (weather, vessel incidents, malfunctioning equipment) causes
delays to shipping, affecting international trade to and from the Great Lakes
region of North America. Downtime is
measured in minutes/hours of delay for weather (visibility, fog, snow, ice);
vessel incidents (human error, electrical and/or mechanical failure); water
level and rate of flow regulation; and lock equipment malfunction. |
Source:
|
SLSDC gathers the data from internal
Lock Operations records.
|
|
Limitations: |
As the agency responsible
for the operation and maintenance of the U.S. portion of the St. Lawrence
Seaway, SLSDC’s lock operations unit gathers primary data for all vessel
transits through the U.S. Seaway sectors and locks, including any downtime in
operations. Data is collected on
site, at the U.S. locks, as vessels are transiting or as operations are
suspended. This information measuring
the System’s reliability is compiled and delivered to SLSDC senior staff each
month. In addition, SLSDC compiles
annual System availability data for comparison purposes. Since SLSDC gathers data directly from
observation, there are no limitations. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
SLSDC verifies and validates the accuracy of the data through review of
24-hour vessel traffic control computer records, radio communication between
the two Seaway entities and vessel operators; and video and audiotapes of
vessel incidents. |
|
Comment: |
SLSDC influences the measure primarily through capital planning,
and consistent facilities maintenance and investment. |
|
Measure: |
Days
critical waterways are closed due to ice. (FY) (2001) |
|
Scope: |
Seven waterways are designated critical to icebreaking on the Great
Lakes based on historical ice conditions, volume of traffic, and potential
for flooding due to ice dams on rivers.
The Coast Guard measure is the number of days critical waterways are
closed for more than 24 hours due to ice. |
Source:
|
Data comes from U.S. Coast Guard and U.S. Army Corps of Engineers
observations. Waterways closure data
is reported to the Ninth Coast Guard District by operating units via
operational situation reports. |
|
Limitations: |
The data set associated
with this measure is relatively small and simple; hence it tends to be fairly
accurate. However, it is possible
that small errors exist. This measure captures only Great Lakes winter
navigation, and not all domestic icebreaking. The observation of closures in
critical waterways is a surrogate for mobility over the whole Great Lakes
waterway system. |
|
Statistical Issues: |
This particular
performance measure is highly sensitive to the severity of winter weather,
which will dramatically affect the ability to predict the number of days the
waterways are closed due to ice. The
Coast Guard expects a lower rate of waterways closures due to ice during mild
winters and a corresponding higher rate of waterways closures during severe
winters. The Coast Guard uses a
standard severity index (based on average temperatures) to measure the
severity of winter weather (–6.2 or
milder defines average severity; less than –6.2 defines severe, e.g. –6.5).
The term “waterway closure” is also subject to differences in definition by
districts or sub-units reporting the data. |
|
Verification &
Validation: |
Coast Guard district
program managers review and validate data from situation reports and provide
Coast Guard headquarters with an End of Season report. |
|
Comment: |
Great Lakes data reflect initial measurement methodology. Further refinements are being explored
that will make this measure a more comprehensive gauge of winter navigation. |
|
Measure: |
1. Percentage
of bus fleets that are Americans with Disabilities Act (ADA) compliant. (CY) 2. Percentage
of key rail stations that are ADA compliant. (CY) |
|
Scope: |
Accessibility
for bus fleet means that vehicles are lift or wheel chair ramp equipped. Accessibility for key rail facilities is
determined by standards for ADA compliance. |
Source:
|
Data on bus accessibility is collected in the National Transit
Database (NTD), with information gathered from transit operators. Data on rail accessibility is reported to FTA
by the transit authorities. |
|
Limitations: |
Measure does not capture
ADA compliance (or transportation accessibility) for modes other than
transit. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
For bus accessibility, an independent auditor and the transit
agency’s CEO certify that data reported to the NTD are accurate. Data are also compared with fleet data
reported in previous years, and crosschecked with other related
operating/financial data in the report.
Fleet inventory is reviewed as a part of FTA’s Triennial Review, and a
visual inspection is made at that time.
FTA’s Office of Civil Rights conducts oversight reviews in order to
verify the information on key rail station accessibility which has been
self-reported by the transit authorities. |
|
Comment: |
FTA will
primarily influence the goal through Federal transit infrastructure
investment, which speeds the rate at which transit operators can transition
to ADA-compliant facilities and equipment. |
|
Measure: |
Number
of employment sites that are made accessible by Job Access and Reverse
Commute transportation services. (FY) (2001) |
|
Scope: |
This measure assesses one part of the Job Access and Reverse Commute
program – the number of employment sites made accessible that were not
previously accessible. An employment
site is considered accessible if located within 1/4 mile of services provided
by the grantee. Employment sites must
offer jobs that require a high school diploma or less. Services that make an employment site
accessible may include, but are not limited to, carpools, vanpools, and
demand-responsive services as well as traditional bus and rail public transit. The measure cannot account for those Job
Access and Reverse Commute activities that encourage riders to use already
existing sources of public transit. |
Source:
|
Data are provided to FTA by grantees
of the Job Access and Reverse Commute program in their quarterly progress
reports.
|
|
Limitations: |
The goal and measurement
is the primary evaluation measure aimed at capturing results of the Job
Access and Reverse Commute program.
Three elements are key to job access – the residence of the employee,
the commute, and the job location. This measure includes the “goal” of the commute and the job, but
it does not include the “starting line” of the commute, the rider’s
home. Although jobs may be made more
accessible to transportation services, these services may not provide access
to potential workers’ communities.
This measure also cannot account for improved accessibility due to
lower fares or shorter commute times – it only addresses the gap in service
delivery. FTA requires a greater
level of precision from larger, urban grantees than rural grantees that may
have fewer resources at their disposal. |
|
Statistical Issues: |
There are major problems
in obtaining accurate estimates of the number of entry-level jobs within a
quarter-mile of grantee services.
Surveys are costly and prone to systematic biases. The uncertainty in this estimate is both
large and difficult to quantify. |
|
Verification &
Validation: |
FTA will use an oversight
contractor to verify reported information on a sample basis. |
|
Comment: |
None. |
|
Measure: |
Number
of passengers (in millions) in international markets with open skies aviation
agreements. (FY) |
|
Scope: |
These data are collected
by DOT for all flight segments to/from a U.S. point. The data for this
measure include all passengers on U.S. and foreign carrier flights to and
from 47 “open-skies” countries and Canada.
This indicator reflects (barring significant, unrelated macroeconomic
and political influences) the extent to which the competitive environment
promoted by DOT increases travel opportunities. |
Source:
|
U.S. air carriers file
domestic and foreign data in the T-100 system. Foreign carrier data are from the T-100F database. Foreign air carriers file data for all
nonstop flight segments involving a U.S. point. |
|
Limitations: |
These data are considered
a reliable measure of airline passenger traffic between the U.S. and foreign
nations. The annual increase in air
traffic, however, is affected by economic strength as well as market
liberalization in bilateral aviation trade agreements. Furthermore, only part of the growth rate
in open skies markets can be attributed to new traffic – some of the increase
may reflect diversion of traffic from less competitive routes with higher
taxes and/or inferior service options.
The goal of 3% annual growth reflects aviation analysts’ judgment of
the net impact of these agreements above the estimated growth expected in the
industry. For these reasons, this goal must be considered more of a
forecast than a "target." |
|
Statistical Issues: |
Like other counts of aviation-related activities, there are no major
sources of systematic error in these data that have been quantified. However, random variation in the number
and distribution of airline passengers, as well as the changes in the number
of "open-skies" agreements, results in variation in the measure
over time. The regression standard
error in this variation for 1994 through 2000 is 2.20. |
|
Verification &
Validation: |
Airlines are required to
certify that these data are accurate.
Also, these data are a 100% enumeration of traffic and capacity and
can be verified for reasonableness against other databases, such as flight
schedules. |
|
Comment: |
U.S. policy has favored
the linking of networks. Networks
allow improved service and marketing in many thousands of small city-pair
markets. All of this traffic flows
over flights captured by the T-100 and T-100F reports for international flights. |
|
Measures: |
1. Percent
of subsidized communities with at least 2 round trips/day, 6 days/week (12
round trips/week). (This measure will
be discontinued after FY 2001.) (FY) (2001) 2. Percent
of subsidized communities with at least 3 round trips/day, 6 days/week (18
round trips/week). (FY) (2001) |
|
Scope: |
The measure
shows the number of weekly round trips at subsidized EAS communities in the
continental U.S. EAS communities are
those that were on the certificated airline map in 1978. |
Source:
|
Air carrier filings, airport managers and community
officials.
|
|
Limitations: |
Service frequency is
closely associated with program funding levels and the number of EAS communities
that require subsidy; the number of communities may change. Service frequency may also be affected by
conditions such as an air carrier going out of business, airline strikes, or
carrier shutdowns. DOT’s goal assumes
a fairly constant level of communities in the base (76 in 1998). This measure will not show instances in
which the Department is successfully able to effect a carrier transition to
commercially viable service without a subsidy. Data has only been gathered for 1996 and later years. |
|
Statistical Issues: |
There is no major error
present in the subject data. |
|
Verification &
Validation: |
Continued contact with
airport and civic parties, carrier officials, and Congressional staffs. |
|
Comment: |
Consideration of
alternate strategies or performance measures may be prompted by developments
such as budget constraints and the makeup of commuter’s aircraft fleet. |
|
Measure: |
Gross
tonnage (in thousands) of commercial vessels on order or under construction
in U.S. shipyards. (CY) (2001) |
|
Scope: |
Includes all commercial self-propelled vessels 100 GT or larger
that are on order or under construction (i.e., the orderbook) in the United
States, as of December 31. Vessels
such as drill rigs and inland barges are not included in these figures. |
Source:
|
In addition to MARAD’s compilation of
data, information is drawn from commercial suppliers of worldwide vessel
characteristics data. These include
Lloyd’s Register of Shipping (marketed through Lloyd’s Maritime Information
Services), Clarkson’s Research Service, and Fairplay.
|
|
Limitations: |
No single commercial
supplier of vessel data has complete information on shipyard orders and
construction activity in the U.S.
None of the major data suppliers collect information on
non-self-propelled vessels. In 1998,
MARAD began direct semi-annual shipyard surveys. However, as the overall response rate was about 40 percent and
did not produce any significant increase in either the quantity or quality of
the data, MARAD is seeking alternative methods to obtain this data. The commercial sources used are the best
available, and consequently the data reported represents an amalgam of their
collection efforts. |
|
Statistical Issues: |
One anomaly with the data
is a gap in the statistics for vessels between 100 and 1,000 GT. Only Lloyd’s data provides data in this
category, but their data does not cover the full spectrum of vessels. Orderbook data on December 31 of each year
represents information available at that time and may not reflect complete
information. |
|
Verification &
Validation: |
MARAD compares
information obtained from the different data sources to verify its accuracy. |
|
Comment: |
It has become evident that the available data does not
adequately measure the value or complexity of the commercial shipbuilding
program; therefore, MARAD plans to develop a new goal and measure. |
|
Measure: |
Number
of students graduating with transportation-related advanced degrees from
universities receiving DOT funding. (SY) (2001) |
|
Scope: |
University Transportation Center (UTC) data includes recipients
of Masters and Ph.D. degrees in programs considered to be transportation
related. |
Source:
|
UTC data to be derived from university
records provided to RSPA as part of the UTCs’ grant application.
|
|
Limitations: |
While baseline data has been
obtained for the UTC program, no data currently exists for other education
programs that can result in graduate degrees. |
|
Statistical Issues: |
There is a possibility of
undercounting, due to difficulty in specifying degree programs that are
transportation-related. Additionally,
some universities may not fully comply. |
|
Verification &
Validation: |
Comparison with data
reported for all degree programs by host universities and specific reports on
each recipient of an advanced degree. |
|
Comment: |
None. |
|
Measure: |
Cumulative
number of students (in thousands) reached through the Garrett A. Morgan
Technology and Transportation Futures Program. (SY) (2001) |
|
Scope: |
Includes students of all ages reached through specific
activities such as internships, job shadowing, career days, video
conferences, classroom visits, and teacher externship visits that inform them
of the opportunities available in the transportation field and ensure that
they have the skills and knowledge required for transportation jobs. |
Source:
|
RSPA maintained database to aggregate
responses from program organizers.
|
|
Limitations: |
The inherent nature of
this measure does not allow us to gauge the quality of contacts made with
students “reached” or provide a means to track outcomes in terms of students
entering the transportation field as a direct result of the activities
sponsored through the Garrett A. Morgan Technology and Transportation Futures
Program. |
|
Statistical Issues: |
Some variability is inevitable
in classroom attendance counts, videoconferences, and other measures of
exposure. But this uncertainty should
be small. |
|
Verification &
Validation: |
RSPA works to ensure that
the quantitative data being reported is complete and accurately reflects the
associated student activity before it is entered into RSPA’s database. |
|
Comment: |
None. |
|
Measure: |
Millions
of passengers on Amtrak’s intercity routes. (FY) (2001) |
|
Scope: |
The measure includes all revenue paying passengers on intercity
routes. |
Source:
|
Amtrak Annual Report and Amtrak’s Monthly Train Earnings Report.
|
|
Limitations: |
Data collection relies on
accuracy of Amtrak report. Ridership
is an outcome indicator that reflects a variety of factors, not
insignificantly the capital investment of the Federal Government. Operational decisions of Amtrak and the
availability and cost of alternative modes of transportation also influence
ridership. |
|
Statistical Issues: |
Chance variation from
year to year, as estimated by the regression standard error from 1994-2000,
is 0.81. This calculation assumes
stable operations over the seven-year period; since new runs and lines are
added and removed fairly often, the standard error is only a rough
approximation. |
|
Verification &
Validation: |
Amtrak conducts monthly
verification and validation of data. |
|
Comment: |
A 3.6 million increase in ridership was projected from 1998-2001
as a result of the initiation of the Northeast Corridor high-speed rail
service. |
|
Measure: |
Percentage of ports
reporting landside and waterside impediments to the flow of commerce. (FY) (2001) |
|
Scope: |
81 U.S. deep
and shallow draft ports. |
Source:
|
Informal
telephone surveys of some port officials. |
|
Limitations: |
The informal
surveys did not encompass all of the intended ports within the scope of this
measure. These surveys were not
scientifically rigorous and the questions asked varied from one region of the
country to another. |
|
Statistical Issues: |
(See Verification and
Validation section.) |
|
Verification &
Validation: |
Impediments data was incomplete
and inconsistent. After reexamining
the available data and the methods for obtaining it, MARAD has concluded that
these data do not provide any valid indication as to whether the goal was met
or not. MARAD was not successful in
clearing up inconsistencies or filling in data gaps. |
|
Comment: |
MARAD has also reached the conclusion that MARAD programs do not
have a measurable impact in reducing impediments at U.S. ports. MARAD efforts in this area are limited in scope
to facilitating dialogue between stakeholders in the Marine Transportation
System or technology demonstrations at one or two ports. Therefore, this measure will no longer be
used. |
|
Measure: |
1. Average
condition of motor bus fleet (on a scale of 1 (poor) to 5 (excellent)). (CY)
(2001) 2. Average
condition of rail vehicle fleet (on a scale of 1 (poor) to 5 (excellent)).
(CY) (2001) |
|
Scope: |
The measure
includes bus, demand response, and rail fleets. |
Source:
|
National
Transit Database (NTD), with information gathered from transit operators;
Transit Economic Requirements Model (TERM), which estimates average vehicle
condition using NTD data. |
|
Limitations: |
Average vehicle
condition may not fully reflect the average condition that transit passengers
face, since vehicles in worse condition tend to be utilized less. There are
also lags in reporting of data to the NTD (thereby requiring preliminary
estimates for recent years) and in the effects of federal government capital
assistance (since it may take five years from the time that funds are
appropriated to the time that new or rehabilitated vehicles are placed in
service) |
|
Statistical Issues: |
Condition is
generated from NTD data using an econometric model, which in turn is based on
a random national sample of vehicles. Average condition changes very slowly
due to the steady replacement of vehicles and the relationships in the
estimated model. |
|
Verification &
Validation: |
An independent
auditor and transit agency’s CEO certify that data reported to the NTD are
accurate. Data are also compared with fleet data reported in previous years,
and crosschecked with other related operating/financial data in the report.
The econometric model used to translate NTD data into average condition
ratings is based on visual inspections of a national sample of bus and rail
vehicles. The sample will need to be repeated periodically in the future in
order to keep the econometric model current with developments in vehicle
conditions. |
|
Comment: |
None. |
Details on DOT Measures of Human & Natural Environment
|
Measure: |
Number
of significant domestic and foreign fishery violations found. (FY) |
|
Scope: |
Fishery protection is measured by the number of significant
fishery violations recorded by the United States Coast Guard. Significant violations are defined as those Living Marine
Resource violations which result in one or more of the following conditions: 1) Significant damage/impact to the
resource/fisheries management plan; or 2) Significant monetary advantage to
the violator over their competitors. |
Source:
|
Significant fishery violations are
detected by Coast Guard law enforcement units in the course of living marine
resource law enforcement boardings.
The information from the boarding is reported through the Coast Guard
Marine Information for Safety and Law Enforcement (MISLE) System.
|
|
Limitations: |
It is possible that non-entry, duplication, and coding errors are
present in MISLE data; however, the likelihood of this error is small. |
|
Statistical Issues: |
None. |
|
Verification &
Validation: |
Verification and
validation of data occurs in several places in the data reporting and
collection process. Data entry
software helps ensure data quality and consistency by employing selection
lists and logic checks. Internal
analysis and review of published data by external parties help identify
errors. |
| Comment: |
None. |
|
Measure: |
Percent
change in number of species that are designated as overfished (includes only
the areas where Coast Guard has enforcement responsibility in fisheries
management plans). (FY) (2001) |
|
Scope: |
This measure includes species covered under formal fisheries
management plans that contain Coast Guard enforcement responsibilities, and
that are formally assessed by the National Marine Fisheries Service and
designated as either over-fished, approaching over-fished, or not
over-fished. |
Source:
|
National Marine Fisheries Service. Data is provided through the annual NMFS report to Congress
"Status of Fisheries of the United States." This report is mandated by the Sustainable
Fisheries Act of 1996. |
|
Limitations: |
Historical data are limited – 1997 - 2000 only. Not all species required to be assessed
were formally assessed as over-fished or not over-fished until 2000. Hence,
the number of reported over-fished species rose in NMFS’ 2001 assessment. Assessments of over-fishing are based on
biological sampling methods and estimations that are subject to error. |
|
Statistical Issues: |
As noted in the Limitations section, this measure is likely to rise
as NMFS continues its search for currently unknown fish stocks. In addition, NMFS revisions to data
definitions of over-fished stocks, including their reclassification of
over-fished into categories of over-fished and over-fishing has affected the
calculation of this measure. |
|
Verification &
Validation: |
Data are provided by
NMFS. DOT does not independently
verify or validate the data. |
|
Comment: |
This measure is aligned with the Sustainable Fisheries Act and
the National Marine Fisheries Service (NMFS) related goal. The Coast Guard also measures the rate of compliance with
federal regulations as a critical component of enforcing fisheries management
plans designed to improve species health, and prevent over-fishing. |
|
Measure: |
On a program-wide basis, acres of wetlands replaced for every
acre affected by Federal-aid Highway projects (where impacts are
unavoidable). (FY) |
|
Scope: |
Measure includes wetlands associated with all Federal-aid
highway projects each fiscal year. To
be included, wetland replacement (or investment in a wetland bank) must have
begun. |
Source:
|
State DOTs input Federal-aid related
wetland degradation and replacement data into either locally developed
wetland mitigation databases or the FHWA Wetlands Management Database. FHWA compiles the final data.
|
|
Limitations: |
Data only exists on
Federal-aid related wetland replacement.
Also, uniformity of the data is not guaranteed, as it is subject to
interpretation by the reporting State DOTs.
In particular, there is no uniform understanding of what should be
reported as mitigation acreage. The
FHWA has provided guidance on mitigation activities to report and will soon
issue the Wetlands Management Database that should reduce the current
variations in data received from the States.
Data on wetland replacement is available for the past five fiscal
years (FY 1996 - FY 2000). |
|
Statistical Issues: |
The non-uniformity of the
data is problematic. Definitional
ambiguity also makes formal statements of statistical uncertainty
problematic. |
|
Verification &
Validation: |
Data are gathered from
established mitigation amounts required by section 404 (Clean Water Act)
permits that states must acquire for their projects. In addition, FHWA provides guidance to
help states consistently report mitigation data. This process will be further improved through a standard
mitigation database under development for the states. At present, there is no external audit of
state data. |
|
Comment: |
All Federal
agencies (including FHWA and other modes) must comply with National
Environmental Policy Act (NEPA) and the Clean Water Act (specifically section
404(b)(1) of the CWA) regarding disruption of wetlands. These laws require
agencies to identify project alternatives that would avoid or minimize
impacts to wetlands as a first consideration. These alternatives are subjected to analysis under both NEPA
and the Clean Water Act. Under the
law, these alternatives must be chosen unless the project sponsors clearly
demonstrate that they are not viable because they do not meet the project purpose
and need, or will lead to other more significant environmental impacts. If, in compliance with the law, wetland
disruption is unavoidable, FHWA then works to achieve this goal of wetland
replacement. |
|
Measure: |
Percentage
of DOT facilities categorized as No Further Remedial Action Planned (NFRAP)
under the Superfund Amendments and Reauthorization Act (SARA). (FY) |
|
|
Scope: |
EPA maintains a Federal Facility Hazardous Waste docket (docket),
which contains information regarding Federal facilities that manage hazardous
wastes or from which hazardous substances have been or may be released. DOT facilities listed on the docket are
discussed in the Annual SARA report sent to Congress each year. EPA regional offices make the determination
to change facility status to NFRAPs on the docket. |
|
Source:
|
Annual SARA Report to Congress.
|
|
|
Limitations: |
The number of DOT
facilities listed on the docket can and has fluctuated over the years. Several of the DOT facilities listed have
more than one site requiring cleanup and a facility is not removed from the
list until all of the sites have no further remedial action planned. Some facilities are listed erroneously and
it may take several years to remove them from the docket. NFRAP decisions may be reversed by EPA if
future information reveals that additional remedial actions are warranted. |
|
|
Statistical Issues: |
There is no major error
present in the subject data. |
|
|
Verification & Validation: |
The data used in
measuring this performance is based on restoration activities at field
locations for USCG, FAA, FHWA, and FRA.
These field sites report their activities to their respective
headquarters management who verifies the data by periodic follow-up
reviews. The data is then reported
yearly to the Office of the Secretary, who crosschecks it against data
received from EPA and the states. |
|
|
Comment: |
The primary criterion for NFRAP is a determination that the
facility does not pose a significant threat to the public health or
environment. NFRAP decisions may be
reversed if future information reveals that additional remedial actions are
warranted. The Operating Administrations’ activities are controlled, to a
degree, by interaction and decisions made by EPA Regional personnel. |
|
|
Management Discussion |
The
number of obsolete vessels removed from the National Defense Reserve Fleet
(NDRF) sites for subsequent disposal. (FY) |
|
|
Scope: |
As of
January 2002, there were 136 vessels in the NDRF designated for
disposal. MARAD estimates this number
will increase, as more Ready Reserve Force (RRF) merchant-type vessels become
obsolete. This increase is primarily
due to obsolescence of additional non-combatant, merchant-type vessels from
MARAD’s RRF, but also from other Federal agencies (e.g. Coast Guard, NOAA,
etc.) for disposal. MARAD notified
the Navy in October 2001 that it would not accept titles to obsolete Navy
merchant-type ships until significant progress is made in disposing of
MARAD’s current backlog of obsolete ships.
A vessel is not removed from the list of vessels awaiting disposal
until it is physically removed from the NDRF sites. |
|
Source:
|
MARAD maintains records on each of the vessels located at
its three Reserve Fleet sites and the entity responsible for disposal of each
of the vessels.
|
|
|
Limitations: |
None |
|
|
Statistical Issues: |
None |
|
|
Verification &
Validation: |
Vessels
removed from the NDRF sites are tracked by MARAD. MARAD has oversight authority for the vessels that it has
contracted to be scrapped and continually monitors the operation of the
contract holders to make sure that the ships are scrapped in a safe and
environmentally sound manner.
Additionally, the Environmental Protection Agency and State and local
environmental agencies are made aware of ships being scrapped or recycled,
and they also monitor progress. MARAD
requires written certification from respective entities that all recycled
activities are completed in accordance with Federal, State and local laws. |
|
|
Comment: |
None |
|
|
Measure: |
Monthly
average number of area transportation emissions conformity lapses. (FY) |
|
|
Scope: |
The transportation conformity process is intended to
ensure that transportation plans, programs, and projects will not create new
violations of the National Ambient Air Quality Standards (NAAQS), increase
the frequency or severity of existing NAAQS violations, or delay the
attainment of the NAAQS in designated non-attainment (or maintenance)
areas. The publication, Transportation
Conformity: A Basic Guide for State and Local Officials |
|
Source:
|
FHWA and FTA jointly make conformity determinations
within air quality non-attainment and maintenance areas to ensure that
Federal actions conform to the purpose of State Implementation Plans
(SIPs). With DOT concurrence, the EPA
has issued regulations pertaining to the criteria and procedures for
transportation conformity, which were revised based on stakeholder comment. |
|
|
Limitations: |
Conformity determinations are required by law to be
updated once every three years. One
reason for an area to be in a conformity lapse is due to the fact that it missed
the deadlines for making a conformity determination on the transportation
plan and program. Under this
scenario, the conformity lapse is not a result of the emissions problem in
that area. In
addition, certain State Implementation Plan (SIP)-related deficiency findings
by EPA (such as a disapproval of a submitted SIP without a protective
finding) may also put an area in a conformity lapse. This may take a long time before the
SIP-related issue(s) are addressed through the complex and time-consuming SIP
revision process. In this situation,
FHWA/FTA will have little control over the duration of the conformity lapse. |
|
|
Statistical Issues: |
None. |
|
|
Verification &
Validation: |
The MPO and U.S. DOT (FHWA/FTA) have a responsibility
to ensure that transportation plans and programs within metropolitan
boundaries conform to the SIP. In metropolitan areas, the governing board of
each MPO must formally make a conformity determination on its transportation
plan/TIP prior to submitting them to the U.S. DOT (FHWA/FTA) for review and
approval. Conformity determinations for projects outside of these boundaries
are the responsibility of the U.S. DOT (FHWA/FTA) and the project sponsor,
which usually is the State DOT. In
addition, the National Memorandum of Understanding issued on April 19, 2001,
provides the EPA and DOT with a framework for coordinating and working
through issues in the conformity and SIP processes. Specifically, the MOU's
provisions ensure that: 1. EPA and DOT consult on conformity
determinations before DOT's approval process; 2. the conformity rule's provisions are
appropriately applied with regard to conformity determinations; and 3. adequate interagency consultation persists
through the planning and conformity processes to identify and resolve issues
prior to a conformity lapse or freeze. |
|
|
Comment: |
If
conformity cannot be determined within certain time frames after amending the
SIP, or if three years has passed since the last conformity determination, a
conformity lapse is deemed to exist and no new non-exempt projects may
advance until a new determination for the plan and TIP can be made. This
affects transit as well as highway projects.
During a conformity lapse, FHWA and FTA can only make approvals or
grants for: projects that are exempt from the conformity process (pursuant to
'93.126 and '93.127 of the conformity rule) such as safety projects, and
transportation control measures (TCMs) that are included in approved SIP.
Only those project phases that have received approval of the project
agreement, and transit projects that have received a full funding grant
agreement (FFGA), or equivalent approvals, prior to the conformity lapse may
proceed during a conformity lapse. |
|
|
Measure: |
Tons
(in millions) of mobile source emissions from on-road motor vehicles. (FY)
(2001) | |
|
Scope: |
Figure is the
sum of on-road mobile source emissions of carbon monoxide, hydrocarbons,
nitrogen oxides, and particulate matter less than 10 microns in diameter
(PM-10). |
|
Source:
|
National Air
Quality and Emissions Trends Report published annually by EPA. (EPA uses data from FHWA’s Highway
Performance Monitoring System – HPMS.) |
|
|
Limitations: |
On-road mobile source
emissions estimates are modeled using vehicle data. Past data contain some variations due to changes in methodology
used to obtain on-road mobile source emissions estimates. EPA revises emission estimates
periodically based on revised methodology.
In 1999, EPA increased the annual emission burden trend based on the
knowledge that heavy-duty diesel trucks manufactured since the early 1990’s
produce higher emissions during high-speed operations. Emissions data are
reported in a 2-year time lag. Indicator
captures all major mobile source emissions from on-road vehicles. It does not capture off-road mobile
sources, such as agriculture and construction machinery, lawn mowers,
aircraft, trains, and boats. |
|
|
Statistical Issues: |
The EPA’s use
of a mathematical model poses issues of model validation. The annual variation in the model’s
estimates, as measured by the regression standard error for data from years
1994 to 1999, is 2.53. The HPMS data
used as input to the model are subject to sampling and non-sampling errors. |
|
|
Verification &
Validation: |
EPA conducts
verification and validation of data.
FHWA field offices perform annual reviews of HPMS data that EPA uses
as a part of its model. |
|
|
Comment: |
The National
Ambient Air Quality Standards (NAAQS), as revised in July 1997, may create
new challenges for DOT in meeting the air quality goal. Targets may need to
be modified to reflect these changes. |
|
|
Measure:
|
Metric tons
(in millions) of carbon equivalent emissions from transportation sources.
(CY) (2001) |
|
Scope: |
Measure includes GHGs such as those subject to the
Kyoto Protocol (e.g., CO2, CH4), but not other GHGs
(e.g., water vapor). Emissions from
fossil fuels combusted in civilian and military ships and aircraft engaged in
international transport of passengers and cargo (i.e., those that are
recorded separately as international bunkers) are not included. Does not
include emissions from non-transportation mobile sources such as farm and
construction equipment. |
Source:
|
Inventory of U.S. Greenhouse Gas Emissions and
Sinks: 1990-1999, published by EPA, supplemented with EPA’s Draft Inventory of U.S. Greenhouse Gas Emissions
and Sinks: 1990-2000. Estimates
are based on data from EPA and other agencies.
|
|
Limitations: |
GHG emissions are estimated based on
DOE estimates of aggregate supply of energy products such as motor gasoline
and distillate fuel oil. Further
disaggregation (e.g., of transportation modes and other uses such as
agriculture) is not always available.
Related “upstream” emissions and sequestration (e.g., from petroleum
refining) are in separate categories.
Procedures for calculating and applying GHG credits and permits have
not yet been established. |
|
Statistical Issues: |
These data are external to DOT. They are subject to both sampling and
non-sampling errors. |
|
Verification & Validation: |
EPA conducts verification and
validation of data. DOT will participate
as appropriate in reviewing data, methodology, and results. |
|
Comment: |
None. |
|
Measure: |
Gallons
spilled per million gallons shipped by maritime sources. (FY) |
|
Scope: |
Spills from vessels and waterfront facilities that are the
target of Coast Guard regulatory prevention efforts are counted; other spills
are not. Oil spills of 1 million gallons or more are excluded (or shown
separately) from data since they are rare (they do not occur every year) and
would have an inordinate influence on statistical trends. |
Source:
|
Spill amounts (numerator) are obtained
from the Coast Guard Marine Safety Information System (MSIS). By regulation, spills
are reported to the National Response Center or to the Coast Guard Federal
On-scene Coordinator. Spill reports
are normally made by the representatives of the party spilling the oil.
Sometimes spill reports are received from third parties, or spills are
discovered by Coast Guard personnel.
Data on waterborne oil shipments (denominator) is from US Army Corps
of Engineers “Waterborne Commerce Statistics”.
|
|
Limitations: |
The investigation,
retrieval, analysis and reporting processes result in under-reporting for the
most recent year, with the most significant effects over the most recent 5
months. Estimates are often used to
compensate for this known data-lag.
It is probable that some spills are not reported. Large spills that impact a large area, or
are located in heavily transited areas are more likely to be reported than
small spills or spills in remote locations.
The actual amount of oil spilled may vary significantly from the
amount estimated. The significance of
this error depends on the unique circumstances of each case. However, the error rate for volume of oil
spilled is estimated to be less than 5% because large spills receive a high
level of review and account for most of the volume spilled. Duplicate spill
entries are sometimes entered into MSIS, and some spills are mistakenly
omitted or entered incorrectly.
Verification procedures strive to correct these errors, but it is
probable that some are not corrected. By excluding non-regulated sources and
major oil spills, the measure does not capture the amount spilled annually
from all sources. However, the exclusions are helpful in assessing the impact
of existing Coast Guard regulations and policies (program management). |
|
Statistical Issues: |
The major sources of
uncertainty in this measure are the reporting error (as a result of the
data-lag), estimation error (actual amount of oil spilled may vary from the
amount estimated), and response error (as a result of spills not being
reported to or discovered by the Coast Guard). The regression standard error
for year-to-year chance variation is 1.8 for the number of gallons spilled
per million gallons shipped, based on data from 1995 through 2000. |
|
Verification &
Validation: |
Verification and
validation occurs at several levels. Edit
checks within MSIS can detect some incorrect or missing data and force review
and correction before data entry is completed. Selection lists for certain data fields also reduce the
opportunity for data entry error. All
investigations go through one level of review at the field unit for
accuracy. Investigations of spills
are also usually reviewed at district and headquarters offices. The headquarters Data Administration staff
conducts periodic quality control checks to identify entry errors such as missing
data or miscoding, and corrects any errors identified. Each spill involving 1,000 gallons or more
is reviewed before it is included in the measure. Errors identified are referred to either the Data
Administration staff or the Investigations and Analysis staff for correction.
|
|
Comment: |
During FY 2002, the
Marine Safety Information System (MSIS) will be replaced by the Marine
Information System for Safety and Law Enforcement (MISLE). While the new system will be a significant
improvement, it is expected to cause serious difficulties in making
performance comparisons. One factor
is that many business processes were re-designed in conjunction with system
development. Another factor is that
data quality under MISLE is expected to be superior to that of MSIS. While this represents improvement, it may
cause near-term problems in making meaningful comparisons of data between the
two systems. |
|
Measure: |
Tons
of hazardous liquid materials spilled per million ton-miles shipped by
pipelines. (CY) |
|
Scope: |
Hazardous
liquid pipeline incidents are those that result in a fatality or injury
resulting in hospital treatment or hospitalization, property damage equal to
or greater than $50,000, or more than 50 barrels spilled. (A rulemaking proposes to lower the
reporting threshold for spill amount from 50 barrels to five gallons.) This measure tracks only releases from
hazardous liquid pipelines to the environment. Natural gas pipeline releases vaporize into the atmosphere and
do not have long-term significant impact on the environment, and thus are not
included in this measure. |
Source:
|
Pipeline operators report to RSPA on form
7000-1, Hazardous Liquid Accident Report.
RSPA records the data in RSPA’s Hazardous Materials Information
System.
|
|
Limitations: |
Because of the magnitude
and frequency of fluctuations in the historical data for this measure, a
short-term goal will be of limited use in tracking program performance. RSPA does not collect volume shipped data
but uses the Association of Oil Pipelines annual Fact Sheet as source for
this part of the measure. |
|
Statistical Issues: |
These spill incidents are rare and probably
not independent events. The
performance measure is a ratio, so uncertainty in the denominator can have a
large effect on the overall uncertainty.
|
|
Verification &
Validation: |
RSPA reviews the data for accuracy. Supplemental reports are requested where
obvious reporting shortcomings are indicated. Additionally, the ASME B31.4 liquid pipeline data review
subcommittee performs an annual examination of the hazardous liquid incident
reports. Known problems with
under-reporting property damages and spill quantities are being addressed by
a rulemaking to revise accident reporting requirements to implement a new
“open and closed” status to insure that operators continue to file supplemental
reports until the spill consequence is fully reported. A new industry data improvement effort
piloted in 1999 will provide better precursor data and more extensive data
about impacts to the environment of hazardous liquid pipeline spills. The American Petroleum Institute is
housing the voluntary data repository, which will collect information on
spills down to five gallons (down to one gallon in water) for all pipeline
spills, including those currently not jurisdictional to RSPA. |
|
Comment: |
The data for this measure fluctuate year to year. RSPA is studying the spill data to
determine the nature of this fluctuation and improve this measure. |
|
Measure: |
Number
of people in the U.S. (in thousands) who are exposed to significant noise
levels (65 decibels or more). (FY) |
|
Scope: |
Residential
population exposed to aircraft noise above Day-Night Sound Level of 65
decibels around U.S. airports with the greatest number of commercial jet
take-offs and landings. |
Source:
|
A statistical modeling technique (the MAGENTA model) is
applied using U.S. population data from the Department of Commerce, locally developed
traffic distribution (route and runway utilization), and aircraft
distributions developed using the Official Airline Guide and current aircraft
registration databases. The local traffic utilization data is available for
the busiest U.S. airports in the form of studies developed for the FAA’s
Integrated Noise Model (INM). For smaller airports, a generic statistical
procedure was employed.
|
|
Limitations: |
No actual count (i.e.,
using a local survey) is made of the number of people exposed to aircraft
noise. No military or general
aviation aircraft are included in the FAA’s model. Aircraft type and event
level can be considered current. However, the majority of the databases used
to establish route and runway utilization were developed from 1990 to 1997,
with many of them now over seven years old. Changes in airport layout
including expansions may not be reflected. The benefits of federally funded
mitigation, such as sound insulation or buyout, are not accounted at present.
Future development of the methodology will attempt to quantify the gains
(reduction in people exposed) due to these actions. |
|
Statistical Issues: |
This measure is derived
from model estimates that are subject to errors in model specification. The estimates of population data will be
revised once the new U.S. Census data for 2000 is released and the model
software is updated accordingly. |
|
Verification &
Validation: |
The Integrated Noise
Model has been validated with actual acoustic measurements at both airports
and other environments such as areas under aircraft at altitude. External forecasts data are from primary
sources. The MAGENTA population exposure methodology has been thoroughly
reviewed by an ICAO task group and was validated for several airport specific
cases. |
|
Comment: |
FY 2000 was the
first year measuring using the MAGENTA model. |
|
Measure: |
Percent
of urban population living within a quarter mile of a transit stop with service
frequency of 15 minutes or less (non-rush-hour). (CY) (2001) |
|
Scope: |
A transit stop is defined as a bus stop, but does not include
rail stations unless associated with a bus stop. |
Source:
|
FTA compiled information from bus schedules across the country. Population statistics come from the Census
Bureau. Information from both of
these sources was formatted using the Geographic Information System. |
|
Limitations: |
Transit stops do not include
rail stations (such as light rail or subway). However, rail stations are almost always served by bus lines,
so most persons who live near a rail station also live near a bus line. |
|
Statistical Issues: |
The extrapolation of
population statistics from the Census Bureau at a level fine enough to
support inferences within a geographic radius of a quarter mile is
difficult. The measurement aspects of
this estimate require careful examination. |
|
Verification &
Validation: |
Under development. |
|
Comment: |
The Federal Transit Administration is working to develop the
Transit Performance Monitoring System.
Fully instituted, the TPMS will allow the Department to measure not
only how many people live close to public transit, but also how many people use
public transit for basic mobility. |
|
Measure: |
1. Percent share of the total
dollar value of DOT direct contracts that are awarded to women-owned
businesses. (FY) 2. Percent share
of the total dollar value of DOT direct contracts that are awarded to small
disadvantaged businesses. (FY) |
|
Scope: |
Includes contracts awarded by DOT contracting activities (except FAA)
through direct procurement (i.e., not including contracts issued by
grantees). |
Source:
|
All DOT contracting activities except
the FAA report data to the Contract Information System (CIS). This data is reported to the Federal
Procurement Data Center (FPDC) by the CIS.
|
|
Limitations: |
Contracting data is reported by procurement offices directly into the
CIS. Data can still be entered into CIS and reported to FPDC after
performance measurement results are submitted so small variations in prior
year performance data may result. |
|
Statistical Issues: |
There is no major error
present in the subject data. However,
random variation in the number of DOT contracts as well as the number of
women-owned and small-disadvantaged businesses each year results in some
random variation in these measures from year to year. The regression standard error for
1994-2000 is 0.64 percent for women-owned small businesses and 1.23 percent
for small-disadvantaged businesses. |
|
Verification &
Validation: |
DOT conducts comparison
checks of CIS data with FPDC data to reconcile discrepancies. On occasion, GSA audits the accuracy of
DOT contracting data. |
|
Comment: |
None. |
|
Measure: |
Number
of environmental justice cases that remain unresolved after one year. (FY) |
|
Scope: |
Data will cover complaints filed with DOT under Title VI of the
Civil Rights Act of 1964 and which have had environmental justice elements,
such as allegations of substantially adverse environmental or health impact
on a minority or low-income community by a transportation project. Case
resolutions are actions that end or administratively close out
complaints. These include such
actions as determinations of no jurisdiction, withdrawals by complainants,
resolutions achieved through alternative dispute resolution, findings of no
violation, and negotiated settlements after discrimination findings under
Title VI. |
Source:
|
DOT will collect this data through the External Complaint Tracking
System (XTRAK). |
|
Limitations: |
This measure is an
initial indicator of how well DOT processes EJ complaints. Variables that will not necessarily be
assessed include such factors as magnitude of injury, number of beneficiaries
adversely affected, pervasiveness, and time constraints before irreparable
damage occurs. Other statutory requirements exist for NEPA concerns. |
|
Statistical Issues: |
There is no major error
present in the subject data. |
|
Verification &
Validation: |
Data will cover the
entire universe of external complaints received by DOT, and will be entered
into the system by operating administrations and DOT Office of Civil Rights
staff. |
|
Comment: |
This indicator does not measure the impact of DOT’s efforts to
prevent the conditions that give rise to complaints. It does provide an initial measure of
response timeliness, which is important to the public. The measure was expanded in 2000 to
include the percent of cases that remain unresolved after one year as a
further indicator of the timeliness of resolution. All environmental justice cases by definition relate to the
concerns of a community of low income and/or minority people. In addition, the number of cases indicates
the pervasiveness of community perception of significantly adverse
environmental and health concerns. |