Performance Data and
Performance Measurement
Performance measurement is dependent on the availability of useful data. Useful data will indicate level of performance and progress toward organizational goals. All data are imperfect in some fashion. Pursuing “perfect” data, however, may consume public resources without creating appreciable value. For this reason, there must be an approach that provides sufficient accuracy and timeliness but at a reasonable cost. This section of the Performance Plan/Performance Report provides information on how DOT reports on performance, verifies and validates data, assesses limitations of the data, and plans for improving DOT’s data.
In an attempt to bring consistency and quality to
its performance reporting, DOT has implemented some general rules regarding the data it uses and how
it is evaluated.
Annual
data – Whenever
available, the data in this document are reported on a Federal Government
fiscal year basis. However, there are instances where this is not possible so
calendar year data are used instead. This
often occurs when data are collected and reported to DOT by external sources
and a calendar year reporting requirement is specified in the implementing
regulation. The reporting timeframe (FY or CY) for each measure is included in
the Data Details in Appendix I.
Annual
results – If
available, the results for the most recent year in the Report are listed as
“Actual” in the Performance Goals & Results box for each performance
measure. However, given the March deadline for submission of the Performance Report,
quite often data have not been compiled and finalized for the entire year. When
this occurs and an actual value is not available for the current year, either
an estimate or projection is provided instead. In general, estimates are based
on partial year data that are extrapolated to cover a full 12-month period. For
example, if six months of data are available, they will be compared to prior
years for the same six-month period to determine any variation from past
levels. Historical trend information, supplemented by program expertise, will
then be applied to estimate the remaining six months of performance. The result
will be identified as a “preliminary estimate” in the Report. If partial year
data are not available, then past trend information will be analyzed and
supplemented by program knowledge to develop a projected value for the annual
performance measure. The result will be identified as a “projection” in the
Report. As data are finalized, the projections and preliminary estimates will
be replaced by actual results. Results
may be amended as errors and omissions are identified in the data verification
process, because updated information is provided by the reporting sources, or
because of legal or other action that changes a previously reported value. For
example, updated pipeline spill reports may change the status of a previously
reported value used in performance measurement.
In
measuring progress toward the majority of performance goals, DOT is moving to a
system of monthly performance measurements.
This will make it much easier to internally gauge periodic progress
toward goals as the year progresses, and will enable more timely performance
reporting after the years’ end.
Completeness
of Data – As
described above, actual data and “preliminary estimates” incorporate complete
or partial data from 2001. Results
listed as “projections” are not based on data from 2001, but on trend data from
prior years.
Reliability
of Measurement Data
– Because performance results in a given year are influenced by multiple
factors, some of which are beyond DOT’s control, and some of which are due to
random chance, there may be considerable variation from year to year. (See
discussion in Appendix I.) A better “picture” of performance may be gained by
looking at results over time to determine if there is a trend. Therefore,
graphs are provided for each measure showing trend lines back to 1990, or as
many years as possible if data are not available back to 1990. Additionally, a
table is included at the beginning of each strategic goal section giving the
available data from 1995 through 2001 for measures with performance goals
specified for 2001.
Integral
to performance measurement is understanding data limitations, addressing these
limitations where necessary and cost-effective, and acknowledging those that
remain when interpreting results. This section on verification and validation
provides a DOT-wide overview of our plan for assessing the quality of the data
DOT uses to measure its performance, and follows the GAO definitions for
verification and validation:
“Verification
is the assessment of data completeness, accuracy, consistency, timeliness, and
related quality control practices.”
“Validation
is the assessment of whether data are appropriate for the performance measure.”
Virtually
all data have errors. In Appendix I we have provided the following information
about the data used for each performance measure: source of the data,
limitations of the data, observations about the quality of the data, work
planned or ongoing to improve data quality, and any known biases.
Additionally,
we have compiled Source and Accuracy Statements for each of the DOT data
programs used in this report, which can be found at www.bts.gov/statpol/SAcompendium.html.
The Source and Accuracy Statements give more detail on the methods used to
collect the data, sources of variation and bias in the data, and methods used
to verify and validate the data.
By
validating data used in the DOT performance plan, we are ensuring that those
data are reflective of the phenomena they purport to measure. The Office of the
DOT Inspector General (OIG) plans to selectively verify and validate
performance measurement data each year. When pertinent to the conduct of
ongoing projects, OIG will also assess performance measures to determine their
appropriateness for measuring progress toward stated goals. These assessments
may lead to changes in the goals, improvements to or additions of data
collection systems, or both.
Assessing
and, where possible, eliminating sources of error in DOT data collection
programs has always been an important task for data program managers. As a part
of their ongoing work, managers of Departmental data programs use quality
control techniques, such as flowcharting the data collection process, to
identify where errors can be introduced into the data collection system.
Program managers also use computerized edit checks and range checks to minimize
errors that may be introduced into the data of their respective programs. In
addition, quality measurement techniques are employed to measure the effects of
unanticipated errors. These include verification of data collection and coding,
as well as coverage, response and non-response error studies to measure the
extent of human error affecting the data. As sources of error are identified,
steps are initiated to improve the data collection process.
The
data used in measuring performance come from a wide variety of sources. Much of
the data originates from sources outside the Department and, therefore, outside
the direct control of the Department. The data often come from administrative
records or from sample surveys. While DOT may not have a strong voice in
improving the quality of outside data, the Department takes all available
information about the limitations and known biases in outside data into account
when using the data.
The
myriad data sources make the task of assessing and, where possible, eliminating
error a challenging one for DOT. Different data systems contain different types
of errors. For example, data from administrative records systems may have
missing or incorrect records, and data from sample surveys will contain
sampling error.
Several
measures (particularly in safety) require aggregation across transportation
modes. This can be particularly problematic because of the use of different
definitions in different transportation modes. Also, data from outside the
Department may have unknown error properties.
To
help the operating administrations address these issues, the Bureau of
Transportation Statistics (BTS) is developing a statistical policy framework
where the operating administrations will work together to identify and
implement the current statistical “best practices” in all aspects of their data
collection programs. This project is consistent with the data capacity
discussions found in the DOT Strategic Plan.
In
2001, a DOT intermodal working group addressing DOT data quality issues
continued to:
▪
develop
Departmental statistical standards;
▪
update Source and Accuracy Statements for all DOT data programs to
document limitations and known errors and biases;
▪
improve
quality assurance procedures;
▪
evaluate
sampling and non-sampling error; and;
▪
develop common definitions for data across modes.
BTS's statistical staff is
consulting with the DOT operating administrations’ data program managers to
assist in data evaluation and validation, documenting data sources, and
determining the reliability of performance measurement estimates.
Departmental data systems
managers use these data verification methods:
▪
Comparisons with previous data from the same source.
▪
Comparisons with another reliable source of the same type of data
within DOT for the same time period.
▪
Comparisons with another reliable source of the same type of data
within DOT for a previous time period.
▪
Comparisons with another reliable source of the same type of data
outside DOT for the same time period.
▪
Comparisons with another reliable source of the same type of data
outside DOT for a previous time period.
In addition to
computerized edit checks and clerical review procedures to look for outliers,
duplicate records, and data inconsistencies, data managers also verify data
quality at each step of the data collection process using these procedures:
▪
Re-collecting/re-interviewing all (or a sample of) records and
reconciling with the original collection. (This applies to census or sample
survey data collections from administrative records, organizations, or
individuals.)
▪
Conducting 100 percent (or a sample of) data re-coding and
reconciliation to assess and correct coding errors.
▪
Conducting 100 percent (or a sample of) data re-entry and
reconciliation to assess and correct data entry errors.
The American Travel Survey’s re-interview program,
in which a sample of households were re-contacted and differences reconciled,
is an example of a verification system within a data collection program.
DOT
Data Source Limitations – Timeliness is the most significant limitation for DOT performance
measurement data. Some DOT data are not collected annually. For example, the
National Household Travel Survey and the Commodity Flow Survey each collect
data every five years. Data that are
collected each year (or more frequently) require time to analyze, confirm and
report results. For example, Highway Performance Monitoring System
vehicle-miles traveled (VMT) data require several months of post-collection
processing, making final results unavailable for this performance report.
Other
performance measurement data limitations can be found in the previously
mentioned Source and Accuracy Statements for DOT data programs. These
statements contain descriptions of data collection program design, estimates of
sampling error (if applicable), and discussions of non-sampling errors.
Non-sampling errors include under-coverage, item and unit non-response,
interviewer and respondent response error, processing error, and errors made in
data analysis.
As
part of its mandate in the Intermodal Surface Transportation Efficiency Act of
1991 (ISTEA), the Transportation Equity Act for the 21st Century (TEA-21), and
its plans for a statistical policy framework in the Department, BTS is working
on a program of research, technical assistance, and data quality enhancement to
support the continued improvement of data programs in DOT. This will help data
program managers throughout DOT improve data quality and better document known
data limitations. BTS also assists operating administrations with data
collection and documentation.
Many of DOT’s internal data programs
rely on State DOTs to collect reliable statistics within cost constraints.
While we work closely with our State DOT partners, we do not have direct
control over these data.
External
Data Source Limitations – Timeliness is also a significant limitation for external or
third-party data. Other limitations of external data are noted in the comments
for each performance measure in Appendix I. In some cases, DOT has replaced
external data, where little is known about the quality of the data, with
internal data. For example, DOT has used estimates of person-miles traveled
(PMT) from private organizations, absent any better estimate. The 1995 Nationwide
Personal Transportation Survey and American Travel Survey give DOT data with
known error properties that allow a better estimate of PMT.
The
DOT Strategic Plan 2000 – 2005 identifies data needs for each of the Department’s
strategic goals. They include:
Safety – DOT is undertaking major efforts
over the next several years to improve safety data. Safety has always been our
primary strategic goal, and in 1999 DOT created a Safety Data Action Plan to
better organize data improvement efforts. BTS will lead efforts to: 1) develop
common criteria for reporting injuries and deaths; 2) develop common data on
accident circumstances; 3) improve data quality; 4) develop better data on
accident precursors; 5) expand the collection of near-miss data to all
transportation modes; 6) develop a variety of common denominators for safety
measures; 7) advance the timeliness of safety data; 8) link safety data with
other data; 9) explore options for using technology in data collection; and 10)
expand, improve and coordinate safety data analysis.
Homeland
Security –
Existing performance data sources are generally good, but DOT will collect data
to better understand the transportation system’s vulnerability to intentional
acts of disruption or destruction.
Mobility – All mobility outcomes present
complex measurement issues. Accordingly, DOT will: 1) develop ways of measuring
user transportation cost, time, and reliability with time-series data; 2)
develop better approaches for measuring access; 3) develop straightforward
measures of congestion and its costs; 4) produce more timely and comprehensive
data on the condition and use of the transportation system; and 5) develop a
more complete understanding of variables influencing travel behavior.
Economic
Growth – DOT needs
aggregate data for measuring the productivity, effectiveness and efficiency of
the U.S. transportation system. We plan to collect, analyze and disseminate
data and information that identify critical trends and issues relating to
transportation’s nexus to the U.S. economy. DOT will: 1) develop a means of
measuring transportation cost, time, and reliability – at an aggregate level –
with time-series data; 2) develop a comprehensive measure of the transportation
capital stock; 3) improve our view of changes in the transportation workforce;
4) develop better measures of productivity in the transportation sector, and
other issues concerning use of Producer Price Indices; and 5) develop a better
picture of transportation-related variables influencing U.S. competitiveness in
the global economy.
Human
and Natural Environment – DOT will: 1) develop comparable and complete data on transportation
emissions, noise, hazardous materials releases, and wetlands impacts; 2)
improve our understanding of collateral damage to the human natural
environment; 3) create better leading indicators for potential environmental
issues; and 4) develop a reliable method of measuring the environmental
benefits of bicycling and walking.
Each
table includes a description of a performance measure and associated data
provided by the agencies in charge of the measure. The Scope statement gives an overview of the data collection strategy
for the underlying data behind the performance measure. The Source statement identifies the databases
used for the measure and their proprietary agencies. The Limitations statement describes some of
the shortcomings of the data in quantifying the particular performance characteristics
of interest. The Statistical Issues
statement has
comments, provided by the Bureau of Transportation Statistics (BTS)
and the agency in charge of the measure, that discuss variability of the measure
and other points. The Verification and Validation statement indicates steps taken
by the proprietary agencies to address data quality issues.
DOT
feels strongly that full compliance with the Government Performance and Results Act
requires impartial reporting of the statistical uncertainty associated with
numerical performance measures. A
portion of this uncertainty is related to the methodology used to calculate the
performance measure and the accuracy of the underlying data. For example, the use of samples introduces
uncertainty because estimates are used in lieu of actual counts. Also, there may be errors in the data
collected. However, there are many
other sources of variation (e.g., nonsampling errors, climate effects, new
technology) and they are often difficult to quantify. Nonetheless, a combination of past data and expert judgment can
enable uncertainty statements that are order-of-magnitude correct for even the
most difficult problems.
The
standard error of a performance measure indicates the likely size of the chance
variation in the reported number. It
incorporates both the effects of measurement error, survey error, and so forth,
as well as the variation that occurs naturally from year to year (i.e., even if
there were no change in laws, infrastructure conditions, or human behavior,
there would still be chance variation in an annual count of fatalities). DOT success in meeting GPRA goals must be
viewed in the context of this background noise.
In
many of the following Statistical Issues statements, BTS refers to regression
standard error. This is a modification
of the standard error to take into account of linear trends in the
recent past. Such adjustment is
generally needed to incorporate consistent trends due to cumulative effects of
such things as education programs, changing demographics, the gradual adoption
of new technologies, and so forth. The
underlying assumptions are that: over a short time period the trend of the
measurement data is linear; for any given year the performance measure values
are normally distributed; and the standard deviation is the same for all
years. We believe that these
assumptions lead to a conservative estimate of variability.
The
regression standard error is an estimate, calculated from the annual
performance results, of this common standard deviation. It may be used in the same way as a regular
standard error to set confidence intervals or describe uncertainty. For the purposes of performance measurement,
it may be considered a rough approximation of the annual variability in a
measure, and it will include the affects of program initiatives, influences
beyond the control of DOT (e.g., weather, petroleum prices, etc.), random
chance, and errors inherent in the data.
For
further information about the source and accuracy (S&A) of these data,
please refer to the BTS S&A compendium available at
www.bts.gov/statpol/SAcompendium.html
|
Measures:
|
1. Transportation
fatalities. (CY) 2. Fatalities
per 100 million passenger-miles. (CY) 3. Fatalities
per 100 million ton-miles of freight. (CY) 4. Transportation
injuries. (CY) 5. Injuries
per 100 million passenger-miles. (CY) 6. Injuries
per 100 million ton-miles of freight. (CY) 7.
Transportation incidents. (CY) |
|
Scope: |
This family of measures
aggregates fatalities, injuries and incidents across all modes of transportation
(air, highway, railroad, transit, waterborne and pipeline). The fatality and injury rates per 100 million
passenger-miles exclude pipeline fatalities and injuries due to minimal
interaction with passenger miles.
Highway-rail grade crossing fatalities and injuries are not counted
since they are included in data for highways.
The
fatality and injury rates per 100 million ton-miles of freight include
fatalities and injuries from large truck, rail, waterborne and pipeline
transportation. Highway-rail grade
crossing fatalities and injuries are also included since these involve
freight transportation-related fatalities and injuries that would not
otherwise be counted. Ton-miles of
freight covers intercity truck, rail, water and oil pipeline transportation. Aviation fatalities, injuries and
ton-miles are excluded because the fatality and injury data are not separated
from passenger air carriers. Transportation incidents include crashes, system
failures, spills, releases, and other accidents of a similar nature. |
|
Source: |
The data for these measures are
obtained from National Transportation Statistics published annually by the
Bureau of Transportation Statistics.
Information is taken from the following tables: Transportation Fatalities
by Mode; Injured Persons by Transportation Mode; U.S. Passenger-Miles
(Millions); U.S. Ton-Miles of Freight (Millions); and Transportation
Accidents by Mode. The one exception
is the data on large truck fatalities and injuries used for calculating
fatality and injury rates per 100 million ton-miles of freight are obtained
from the Federal Motor Carrier Safety Administration.
|
|
Limitations: |
Double counting of fatalities and
injuries may occur when an accident involves more than one mode of
transportation. Differing definitions
of injuries or transportation-related fatalities makes comparison across
modes of transportation problematic.
Highway injuries and incidents are obtained from a nationally
representative probability sample and are estimates, while the totals for
other modes of transportation are actual counts. The highway estimates are based on crashes where a police
accident report was completed and the crash resulted in property damage,
injury or death. Accidents that were
not reported to the police or did not result in property damage are not
included. Highway passenger miles are
calculated by multiplying vehicle-miles of travel (VMT) by the average number
of occupants for each vehicle type.
VMT is based on a nation-wide sample of vehicle travel. The average number of vehicle occupants
comes from survey information.
Therefore, vehicle passenger miles is an estimate, whereas
passenger-miles for other modes of transportation are calculated based on
actual passenger counts and recorded trip lengths. |
|
Statistical Issues: |
All fatality totals, and the injury and incident
numbers where actual counts are recorded, are relatively accurate. Any double counting or omissions are
expected to be fairly small. The
primary source of uncertainty in these measures comes from sampling and
survey errors related to estimation of highway injuries, incidents, VMT and
vehicle occupancy. Based on data from
1994-2000, the annual variations in the transportation safety measures are as
follows: the regression standard
error for the number of
transportation fatalities is 0.5 thousand.
For fatality rates by passenger-miles and ton-miles, it is 0.010 and
0.007, respectively. For number of
injuries, it is 0.10 million. For
injury rates by passenger-miles and ton-miles, it is 2.50 and 0.24,
respectively. For incidents, it is
0.16 million. |
|
Verification &
Validation: |
BTS compiles the data for the National
Transportation Statistics from information it gathers directly in its own
data systems (e.g. airlines information), information published by other
sources (e.g. FHWA highway statistics), or by personal communication with the
agency/organization responsible for collecting the data. Each data source conducts error checks and
monitors the accuracy of its data.
Most of these sources and their verification and validation procedures
are described in subsequent data details in this report for performance
measures of individual modes of transportation. |
|
Comment: |
While caution should be exercised in comparing fatalities,
injuries and incidents between modes of transportation due to differences in
definitions and calculations, the aggregation of these values still provides
useful information. Because the
methodology for calculating these measures has remained consistent over the
years, the trend information should provide a reasonably accurate picture of
results. |
|
Measure: |
Fatalities
per 100 million vehicle-miles-traveled (VMT) (CY) |
|
Scope: |
The number of fatalities is the total
number of motor vehicle traffic fatalities which occur on public roadways
within the 50 states and Washington, D.C. Vehicle
Miles of Travel (VMT) represent the total number of vehicle miles traveled by
motor vehicles on public roadways within the 50 states and Washington, D.C.
|
Source:
|
Motor vehicle traffic
fatality data are obtained from NHTSA’s Fatality Analysis Reporting System
(FARS). To be included in FARS, a motor
vehicle traffic crash must result in the death of a vehicle occupant or a
non-motorist within 30 days of the crash.
The FARS database is based on police crash reports and other state
data. FARS includes fatalities on all roadways open to the public, using the
National Highways System classification of roads. Pedestrian and bicycle fatalities that occur on public
highways, but do not involve a motor vehicle, are not recorded in FARS. However, they constitute only a small
number of fatalities. VMT data are derived from FHWA’s Traffic Volume Trends
(TVT), a monthly report based on hourly traffic count data in the Highway
Performance Monitoring System (HPMS).
Information is transmitted to NHTSA where it is reviewed for
consistency and accuracy before being entered into the system. These data,
collected at approximately 4,000 continuous traffic counting locations
nationwide, are used to determine the percentage change in traffic for the
current month from the same month of the previous year. The percentage change is applied to the
nationwide travel for the same month of the previous year to obtain an
estimate of nationwide travel for the current month. The data are recorded as monthly totals
and cumulative yearly totals.
|
|
Limitations: |
VMT data are subject
to sampling errors, whose magnitude depends on how well the locations of the
continuous counting locations represent nationwide traffic rates. HPMS is also subject to estimating
differences in the states, even though FHWA works to minimize such differences
and differing projections on growth, population, and economic conditions that
impact driving behavior. |
|
Statistical Issues: |
The primary source of
uncertainty in estimating fatality rates is the denominator. While the estimate of total fatalities
used in the numerator is relatively accurate, the estimate of total vehicle
miles in the denominator has far more variability. Based on data from 1994-2000, the annual
variation in the fatality rate has a regression standard error of 0.029. The estimates of the
number and percentages of persons killed in motor vehicle traffic crashes
during 2001 are preliminary and are based on incomplete data and statistical
models. NHTSA’s first official
estimates for 2001, the Early Assessment, are being developed and will be
completed in early April 2002.
Differences between the Official Early Assessment estimates and those
in this report are to be expected. |
|
Verification & Validation: |
Fatality data from FARS
are reviewed and analyzed by NHTSA’s National Center for Statistics and
Analysis. Quality control procedures
are built into annual data collection at 6 and 9 months, and at year’s end. A study was completed in 1993, looking at
samples of FARS cases in 1989 through 1990 to assess the accuracy of data
being reported. VMT data are reviewed
by FHWA for consistency and reasonableness. |
|
Comment: |
This data program has
been in use for many years and is generally accepted for describing safety on
the Nation’s highways. Adjusting raw
highway fatalities and injuries by VMT provides a means of portraying the
changes in highway fatalities on a constant exposure basis and facilitates
year-to-year comparisons. |
|
Measure: |
Number
and rate (per million commercial VMT) of fatalities in crashes involving
large trucks. (CY) |
|
Scope: |
The measure includes all fatalities (e.g., drivers and occupants
of passenger cars, motorcycles, large trucks, or pedestrians) associated with
crashes involving trucks with a gross vehicle weight rating of 10,000 pounds
or more. The number of fatalities
comes from NHTSA’s Fatality Analysis Reporting System (FARS) data, a census
of fatal traffic crashes within the 50 states, Puerto Rico, and Washington,
D.C. The fatal crash rate is the
number of fatalities per 100 million vehicle miles of large truck travel
(VMT). |
Source:
|
NHTSA’s Fatality
Analysis Reporting System (FARS) provides fatality data. The VMT data are derived from the Federal
Highway Administration’s (FHWA) Highway Performance Monitoring System (HPMS). |
|
Limitations: |
FARS data elements are modified
from year to year to respond to emphasis areas, vehicle fleet changes, and
other needs for improvement. Large truck VMT reported to FHWA by each state
is based on a sample of road segments and is not a census. In addition, the methods used to calculate
total VMT may vary from state to state. The methods used by the states to
estimate the VMT contribution from rural and urban minor collectors are
unknown. |
|
Statistical Issues: |
The fatality counts in FARS are generally quite accurate. The major sources of error are
underreporting by some precincts and inconsistent use of the definition of a
truck. Based on 1994-2000 data, the
chance variation in a given year has a regression standard error of
approximately 157 fatalities. Because
the VMT data provided to FHWA from each state are estimates based on a sample
of road segments, the numbers have associated sampling errors. The methodology used by each of the states
to estimate VMT is not known and may introduce additional non-sampling
error. Although states provide VMT
estimates on an annual basis, they are only required to update their traffic
counts at all sampling sites once every three years. Thus an annual VMT estimate from a
particular state may be based, in part, on data collected during a previous
year. Based on 1994-2000 data, the
chance variation in a given year in the number of fatalities per 100 million
vehicle miles of large truck travel has a regression standard error of 0.053. |
|
Verification & Validation: |
Fatality data are reviewed and analyzed by NHTSA’s National
Center for Statistics and Analysis.
Quality control procedures are built into data collection and data
processing. A study using samples of
1989-1990 FARS cases was completed in 1993 to assess the accuracy of data being
reported. FHWA routinely works with
state data providers to modify reported VMT values that do not appear
reasonable before incorporating them into its final master file. |
|
Comment: |
The FARS data have been around for many years and are generally accepted
as a good source for describing fatal crashes on the Nation’s highways. The large truck VMT data used to calculate
fatal crash rates have both sampling and non-sampling (i.e., bias) error
associated with it. The impact of
these errors on FMCSA’s estimates of large truck crash rates is considered to
be minimal. |
|
Measure: |
1. Alcohol-related fatalities per 100 million
vehicle-miles traveled. 2. Percentage of highway fatalities that are alcohol
related. (CY) (2001) |
|
Scope: |
The number of
fatalities resulting from motor vehicle traffic crashes that are alcohol
related and occur on public roadways within the 50 states and Washington,
D.C. |
Source:
|
Motor vehicle traffic fatality data are obtained from NHTSA’s
Fatality Analysis Reporting System (FARS). FARS is a census of fatal motor
vehicle traffic crashes within the 50 states, Puerto Rico, and Washington,
D.C. To be included in FARS, a crash
must result in the death of a vehicle occupant or a non-motorist within 30
days of the crash. The FARS data are
based on police crash reports and other state data. FARS includes fatalities
on all roadways open to the public, using the National Highways System classification
of roads. Pedestrian and bicycle
fatalities that occur on public highways, but do not involve a motor vehicle,
are not recorded in FARS. However,
they constitute only a small number of fatalities. A fatal motor vehicle traffic crash is alcohol-related if
either a driver or a non-motorist (such as a pedestrian) involved in the
crash had a measured or estimated blood alcohol concentration (BAC) of 0.01
grams per deciliter or above.
|
|
Limitations: |
Blood Alcohol Concentration test
results are not available for all drivers and non-occupants involved in fatal
crashes. Missing data can result for
a number of reasons -- the most frequent of which is that persons are not
always tested for alcohol. To address
the missing data issue, NHTSA has developed a statistical model (Multiple
Imputation) to estimate specific
values of BAC across the full range of possible values.
Estimating missing BAC in
this manner will permit the estimation of valid statistics such as variances,
measures of central tendency, confidence intervals and standard deviations. The statistical model is based on
important characteristics of the crash including crash factors, vehicle
factors, and person factors. While
this measure does not link alcohol with fault in fatal crashes, the more
comprehensive scope of the measure compensates for a possible undercount of
the extent of the alcohol impaired driving problem. Multiple Imputation differs from the statistical model used in
previous years. However, all historical series of alcohol
involvement will be revised back to the 1982 data year to reflect the
estimates from the new methodology. |
|
Statistical Issues: |
The primary sources of uncertainty
in this performance measure arise from information gaps in the number of
intoxicated non-motorists, and from using the statistical model to estimate
the number of intoxicated drivers. The estimates of the number and
percentages of persons killed in motor vehicle traffic crashes during 2001 included in this section are preliminary and are based on incomplete data and
statistical models. They were provided to meet the time restraints required
for this report. NHTSA’s first
official estimates for 2001, the Early Assessment, are being developed and
will be completed in early April.
Differences between the Official Early Assessment estimates and those
in this report are to be expected. |
|
Verification &
Validation: |
Data are reviewed and analyzed by
NHTSA’s National Center for Statistics and Analysis. Quality control
procedures are built into annual data collection at 6 and 9 months, and at
year’s end. In 1987 and 1988, an
independent panel of academics reviewed and commented on the statistical
methods used in measuring alcohol-related highway fatalities. This report recommended that research and
development utilize a model that would permit the imputation of missing BACs
as a semi-continuous variable. |
|
Comment: |
This data program has been used
for many years and is generally accepted for describing safety on the
Nation’s highways. |
|
Measure: |
Injured
persons per 100 million vehicle-miles-traveled (VMT) (CY) (2001) |
|
Scope: |
The number of injured persons
is an estimate of the total number of persons injured in motor vehicle
traffic crashes that occur on public roadways in the 50 states and
Washington, D.C. Vehicle Miles of
Travel (VMT) represent the total number of vehicle miles traveled by motor vehicles
on public roadways within the 50 states and Washington, D.C. |
Source:
|
The number of injured persons data are derived from the
NHTSA’s National Automotive Sampling System (NASS) General Estimates System
(GES). The NASS GES is a nationally representative
probability sample that yields national estimates of total nonfatal injury
crashes, injured persons, and property-damage-only crashes. NASS GES data
cover all roadways open to the public, using the National Highways System
classification of roads.
VMT data are derived
from FHWA’s monthly report, Traffic Volume Trends (TVT), a monthly report
based on hourly traffic count data in the Highway Performance Monitoring
System (HPMS). Information is
transmitted to NHTSA where it is reviewed for consistency and accuracy before
being entered into the system. These data, collected at approximately 4,000
continuous traffic counting locations nationwide, are used to determine the
percentage change in traffic for the current month from the same month of the
previous year. The percentage change
is applied to the nationwide travel for the same month of the previous year
to obtain an estimate of nationwide travel for the current month. The data are recorded as monthly totals
and cumulative yearly totals. |
|
Limitations: |
GES data are obtained
from a nationally representative sample of 60 sites. The results provide only national data,
not state level data, and are subject to sampling error. The magnitude of the sampling error
depends on the number of Primary Sampling Units (PSUs) in the sample and the
number of crash reports sampled within each PSU. VMT data are subject
to sampling errors, whose magnitude depends upon how well the continuous
counting locations represent nationwide traffic rates. HPMS is subject to estimating differences
in the states, although FHWA works to minimize such differences and differing
projections on growth, population, and economic conditions which impact
driving behavior. |
|
Statistical
Issues: |
The estimate of the injury
rate includes three main sources of uncertainty. The numerator count of injuries has a standard error of 5.1%
(cf. Appendix C of Traffic Safety Facts). The denominator estimate of VMT contains both complex sampling
and non-sampling errors. Based on data from 1994-2000, the annual
variation in the injury rate has a regression standard error of 4.04. The estimates of the
number and percentages of persons injured in motor vehicle traffic crashes
during 2001 are preliminary and are based on incomplete data and statistical
models. NHTSA’s first official
estimates for 2001, the Early Assessment, are being developed and will be
completed in early April. Differences
between the Official Early Assessment estimates and those in this report are
to be expected. |
|
Verification &
Validation: |
Data are reviewed and
analyzed by NHTSA’s National Center for Statistics and Analysis. Quality
control procedures are built into annual data collection at 6 and 9 months,
and at year’s end. A study was completed
in 1993, looking at samples of FARS cases in 1989 through 1990 to assess the
accuracy of data being reported. VMT
data is reviewed by FHWA for consistency and reasonableness. |
|
Comment: |
This data program has
been in use for many years and is generally accepted for describing safety on
the Nation’s highways. GES records
injury severity in four classes: incapacitating injury, evident but not
incapacitating injury, possible but not visible injury, and injury of unknown
severity. Adjusting raw highway
fatalities and injuries by VMT provides a means of portraying the changes in
highway fatalities on a constant exposure basis – to facilitate year-to-year
comparisons. |
|
Measure: |
Number
and rate of injured persons involving large trucks. (CY) (2001) |
|
Scope: |
The measure includes all injured persons (e.g., drivers and
occupants of passenger cars, motorcycles, large trucks, or pedestrians)
associated with crashes involving trucks with a gross vehicle weight rating
of 10,000 pounds or more. The
number of injured persons is derived from NHTSA’s General Estimates System
(GES). The injury rate is the number of injured persons per 100 million
vehicle miles of large truck travel (VMT). |
Source:
|
NHTSA’s General Estimates System (GES)
provides injury data. VMT data are derived from the Federal Highway
Administration’s (FHWA) Highway Performance Monitoring System (HPMS).
|
|
Limitations: |
GES data are obtained from a nationally representative sample of 60
sites. The results provide only
national data, not state-by-state data.
Large truck VMT reported to FHWA by each state is based on a sample of
road segments and is not a census. In
addition, the methods used to calculate total VMT may vary from state to
state. The methods used by the states to estimate the VMT contribution from
rural and urban minor collectors are unknown. |
|
Statistical Issues: |
The GES data have a
standard error of 6.9% for injuries from truck and automobile crashes (cf.
Appendix C of Traffic Accident Reports). They are less accurate than the
corresponding fatality counts. Based
on 1994-2000 data, the variation due to random chance in the number of injuries,
which includes sampling variability, has a regression standard error of
approximately 7,091. Because the VMT
data provided to FHWA from each state are estimates based on a sample of road
segments, the numbers have associated sampling errors. The methodology used by each of the states
to estimate VMT is not known and may introduce additional non-sampling error
into the estimates. Although states
provide VMT estimates on an annual basis, they are only required to update
their traffic counts at all sampling sites once every three years. Thus an annual VMT estimate from a
particular state may be based, in part, on data collected during a previous
year. Based on 1994-2000 data, the
chance variation in a given year in the number of injured persons per 100
million vehicle miles of large truck travel has a regression standard error
of 4.39. |
|
Verification &
Validation: |
Injury data are reviewed
and analyzed by NHTSA’s National Center for Statistics and Analysis. Quality
control procedures are built into data collection and data processing. FHWA routinely works with state data
providers to modify reported VMT values that do not appear reasonable before
incorporating them into its final master file. |
|
Comment: |
The data program has been around for many years and is generally
accepted for describing safety on the Nation’s highways. GES records injury severity in four
classes: incapacitating injury, evident injury but not incapacitating, possible
but not visible injury, and injury of unknown severity. The large truck VMT data used to calculate
injured persons rates have both sampling and non-sampling (i.e., bias) error
associated with it. The impact of
these errors on FMCSA’s estimates of large truck crash rates is considered to
be minimal. |
|
Measure: |
Percentage
of front occupants using seat belts. (CY) (2001) |
|
Scope: |
The proportion of
front seat outboard passenger vehicle occupants using shoulder belts during
daylight hours. |
Source:
|
Data for 1998, 1999, and 2000 are from the National Occupant Protection Use Survey (NOPUS). NOPUS is a National, multi-stage probability sample. In the first stage, counties or groups of counties (Primary Sampling Units or PSUs) were grouped by region (Northeast, Midwest, South, and West), |