Performance Data and Performance Measurement

Performance measurement is dependent on the availability of useful data. Useful data will indicate level of performance and progress toward organizational goals. All data are imperfect in some fashion. Pursuing “perfect” data, however, may consume public resources without creating appreciable value. For this reason, there must be an approach that provides sufficient accuracy and timeliness but at a reasonable cost. This section of the Performance Plan/Performance Report provides information on how DOT reports on performance, verifies and validates data, assesses limitations of the data, and plans for improving DOT’s data.


PerformanceData Completeness and Reliability

In an attempt to bring consistency and quality to its performance reporting, DOT has implemented some general rules regarding the data it uses and how it is evaluated.

Annual data – Whenever available, the data in this document are reported on a Federal Government fiscal year basis. However, there are instances where this is not possible so calendar year data are used instead.  This often occurs when data are collected and reported to DOT by external sources and a calendar year reporting requirement is specified in the implementing regulation. The reporting timeframe (FY or CY) for each measure is included in the Data Details in Appendix I.

Annual results – If available, the results for the most recent year in the Report are listed as “Actual” in the Performance Goals & Results box for each performance measure. However, given the March deadline for submission of the Performance Report, quite often data have not been compiled and finalized for the entire year. When this occurs and an actual value is not available for the current year, either an estimate or projection is provided instead. In general, estimates are based on partial year data that are extrapolated to cover a full 12-month period. For example, if six months of data are available, they will be compared to prior years for the same six-month period to determine any variation from past levels. Historical trend information, supplemented by program expertise, will then be applied to estimate the remaining six months of performance. The result will be identified as a “preliminary estimate” in the Report. If partial year data are not available, then past trend information will be analyzed and supplemented by program knowledge to develop a projected value for the annual performance measure. The result will be identified as a “projection” in the Report. As data are finalized, the projections and preliminary estimates will be replaced by actual results.  Results may be amended as errors and omissions are identified in the data verification process, because updated information is provided by the reporting sources, or because of legal or other action that changes a previously reported value. For example, updated pipeline spill reports may change the status of a previously reported value used in performance measurement.

In measuring progress toward the majority of performance goals, DOT is moving to a system of monthly performance measurements.  This will make it much easier to internally gauge periodic progress toward goals as the year progresses, and will enable more timely performance reporting after the years’ end. 

Completeness of Data – As described above, actual data and “preliminary estimates” incorporate complete or partial data from 2001.  Results listed as “projections” are not based on data from 2001, but on trend data from prior years. 

Reliability of Measurement Data – Because performance results in a given year are influenced by multiple factors, some of which are beyond DOT’s control, and some of which are due to random chance, there may be considerable variation from year to year. (See discussion in Appendix I.) A better “picture” of performance may be gained by looking at results over time to determine if there is a trend. Therefore, graphs are provided for each measure showing trend lines back to 1990, or as many years as possible if data are not available back to 1990. Additionally, a table is included at the beginning of each strategic goal section giving the available data from 1995 through 2001 for measures with performance goals specified for 2001.

Verifying & Validating Performance Measures

Integral to performance measurement is understanding data limitations, addressing these limitations where necessary and cost-effective, and acknowledging those that remain when interpreting results. This section on verification and validation provides a DOT-wide overview of our plan for assessing the quality of the data DOT uses to measure its performance, and follows the GAO definitions for verification and validation:

“Verification is the assessment of data completeness, accuracy, consistency, timeliness, and related quality control practices.”

“Validation is the assessment of whether data are appropriate for the performance measure.”

Virtually all data have errors. In Appendix I we have provided the following information about the data used for each performance measure: source of the data, limitations of the data, observations about the quality of the data, work planned or ongoing to improve data quality, and any known biases.

Additionally, we have compiled Source and Accuracy Statements for each of the DOT data programs used in this report, which can be found at www.bts.gov/statpol/SAcompendium.html. The Source and Accuracy Statements give more detail on the methods used to collect the data, sources of variation and bias in the data, and methods used to verify and validate the data.

By validating data used in the DOT performance plan, we are ensuring that those data are reflective of the phenomena they purport to measure. The Office of the DOT Inspector General (OIG) plans to selectively verify and validate performance measurement data each year. When pertinent to the conduct of ongoing projects, OIG will also assess performance measures to determine their appropriateness for measuring progress toward stated goals. These assessments may lead to changes in the goals, improvements to or additions of data collection systems, or both.

Assessing and, where possible, eliminating sources of error in DOT data collection programs has always been an important task for data program managers. As a part of their ongoing work, managers of Departmental data programs use quality control techniques, such as flowcharting the data collection process, to identify where errors can be introduced into the data collection system. Program managers also use computerized edit checks and range checks to minimize errors that may be introduced into the data of their respective programs. In addition, quality measurement techniques are employed to measure the effects of unanticipated errors. These include verification of data collection and coding, as well as coverage, response and non-response error studies to measure the extent of human error affecting the data. As sources of error are identified, steps are initiated to improve the data collection process.

The data used in measuring performance come from a wide variety of sources. Much of the data originates from sources outside the Department and, therefore, outside the direct control of the Department. The data often come from administrative records or from sample surveys. While DOT may not have a strong voice in improving the quality of outside data, the Department takes all available information about the limitations and known biases in outside data into account when using the data.

The myriad data sources make the task of assessing and, where possible, eliminating error a challenging one for DOT. Different data systems contain different types of errors. For example, data from administrative records systems may have missing or incorrect records, and data from sample surveys will contain sampling error.

Several measures (particularly in safety) require aggregation across transportation modes. This can be particularly problematic because of the use of different definitions in different transportation modes. Also, data from outside the Department may have unknown error properties.

To help the operating administrations address these issues, the Bureau of Transportation Statistics (BTS) is developing a statistical policy framework where the operating administrations will work together to identify and implement the current statistical “best practices” in all aspects of their data collection programs. This project is consistent with the data capacity discussions found in the DOT Strategic Plan.

In 2001, a DOT intermodal working group addressing DOT data quality issues continued to:

          develop Departmental statistical standards;

      update Source and Accuracy Statements for all DOT data programs to document limitations and known errors and biases;

          improve quality assurance procedures;

          evaluate sampling and non-sampling error; and;

      develop common definitions for data across modes.

BTS's statistical staff is consulting with the DOT operating administrations’ data program managers to assist in data evaluation and validation, documenting data sources, and determining the reliability of performance measurement estimates.

Departmental data systems managers use these data verification methods:

      Comparisons with previous data from the same source.

      Comparisons with another reliable source of the same type of data within DOT for the same time period.

      Comparisons with another reliable source of the same type of data within DOT for a previous time period.

      Comparisons with another reliable source of the same type of data outside DOT for the same time period.

      Comparisons with another reliable source of the same type of data outside DOT for a previous time period.

In addition to computerized edit checks and clerical review procedures to look for outliers, duplicate records, and data inconsistencies, data managers also verify data quality at each step of the data collection process using these procedures:

      Re-collecting/re-interviewing all (or a sample of) records and reconciling with the original collection. (This applies to census or sample survey data collections from administrative records, organizations, or individuals.)

      Conducting 100 percent (or a sample of) data re-coding and reconciliation to assess and correct coding errors.

      Conducting 100 percent (or a sample of) data re-entry and reconciliation to assess and correct data entry errors.

The American Travel Survey’s re-interview program, in which a sample of households were re-contacted and differences reconciled, is an example of a verification system within a data collection program.

Data Limitations in Performance Measures

DOT Data Source Limitations – Timeliness is the most significant limitation for DOT performance measurement data. Some DOT data are not collected annually. For example, the National Household Travel Survey and the Commodity Flow Survey each collect data every five years.  Data that are collected each year (or more frequently) require time to analyze, confirm and report results. For example, Highway Performance Monitoring System vehicle-miles traveled (VMT) data require several months of post-collection processing, making final results unavailable for this performance report.

Other performance measurement data limitations can be found in the previously mentioned Source and Accuracy Statements for DOT data programs. These statements contain descriptions of data collection program design, estimates of sampling error (if applicable), and discussions of non-sampling errors. Non-sampling errors include under-coverage, item and unit non-response, interviewer and respondent response error, processing error, and errors made in data analysis.

As part of its mandate in the Intermodal Surface Transportation Efficiency Act of 1991 (ISTEA), the Transportation Equity Act for the 21st Century (TEA-21), and its plans for a statistical policy framework in the Department, BTS is working on a program of research, technical assistance, and data quality enhancement to support the continued improvement of data programs in DOT. This will help data program managers throughout DOT improve data quality and better document known data limitations. BTS also assists operating administrations with data collection and documentation.

Many of DOT’s internal data programs rely on State DOTs to collect reliable statistics within cost constraints. While we work closely with our State DOT partners, we do not have direct control over these data.

External Data Source Limitations – Timeliness is also a significant limitation for external or third-party data. Other limitations of external data are noted in the comments for each performance measure in Appendix I. In some cases, DOT has replaced external data, where little is known about the quality of the data, with internal data. For example, DOT has used estimates of person-miles traveled (PMT) from private organizations, absent any better estimate. The 1995 Nationwide Personal Transportation Survey and American Travel Survey give DOT data with known error properties that allow a better estimate of PMT.

Our Data Needs

The DOT Strategic Plan 2000 – 2005 identifies data needs for each of the Department’s strategic goals. They include:

Safety – DOT is undertaking major efforts over the next several years to improve safety data. Safety has always been our primary strategic goal, and in 1999 DOT created a Safety Data Action Plan to better organize data improvement efforts. BTS will lead efforts to: 1) develop common criteria for reporting injuries and deaths; 2) develop common data on accident circumstances; 3) improve data quality; 4) develop better data on accident precursors; 5) expand the collection of near-miss data to all transportation modes; 6) develop a variety of common denominators for safety measures; 7) advance the timeliness of safety data; 8) link safety data with other data; 9) explore options for using technology in data collection; and 10) expand, improve and coordinate safety data analysis.

Homeland Security – Existing performance data sources are generally good, but DOT will collect data to better understand the transportation system’s vulnerability to intentional acts of disruption or destruction.

Mobility – All mobility outcomes present complex measurement issues. Accordingly, DOT will: 1) develop ways of measuring user transportation cost, time, and reliability with time-series data; 2) develop better approaches for measuring access; 3) develop straightforward measures of congestion and its costs; 4) produce more timely and comprehensive data on the condition and use of the transportation system; and 5) develop a more complete understanding of variables influencing travel behavior.

Economic Growth – DOT needs aggregate data for measuring the productivity, effectiveness and efficiency of the U.S. transportation system. We plan to collect, analyze and disseminate data and information that identify critical trends and issues relating to transportation’s nexus to the U.S. economy. DOT will: 1) develop a means of measuring transportation cost, time, and reliability – at an aggregate level – with time-series data; 2) develop a comprehensive measure of the transportation capital stock; 3) improve our view of changes in the transportation workforce; 4) develop better measures of productivity in the transportation sector, and other issues concerning use of Producer Price Indices; and 5) develop a better picture of transportation-related variables influencing U.S. competitiveness in the global economy.

Human and Natural Environment – DOT will: 1) develop comparable and complete data on transportation emissions, noise, hazardous materials releases, and wetlands impacts; 2) improve our understanding of collateral damage to the human natural environment; 3) create better leading indicators for potential environmental issues; and 4) develop a reliable method of measuring the environmental benefits of bicycling and walking.

 


Appendix I – Performance Measures (Detail)

 

Each table includes a description of a performance measure and associated data provided by the agencies in charge of the measure.  The Scope statement gives an overview of the data collection strategy for the underlying data behind the performance measure.  The Source statement identifies the databases used for the measure and their proprietary agencies.  The Limitations statement describes some of the shortcomings of the data in quantifying the particular performance characteristics of interest.  The Statistical Issues statement has comments, provided by the Bureau of Transportation Statistics (BTS) and the agency in charge of the measure, that discuss variability of the measure and other points.  The Verification and Validation statement indicates steps taken by the proprietary agencies to address data quality issues.

DOT feels strongly that full compliance with the Government Performance and Results Act requires impartial reporting of the statistical uncertainty associated with numerical performance measures.  A portion of this uncertainty is related to the methodology used to calculate the performance measure and the accuracy of the underlying data.  For example, the use of samples introduces uncertainty because estimates are used in lieu of actual counts.  Also, there may be errors in the data collected.  However, there are many other sources of variation (e.g., nonsampling errors, climate effects, new technology) and they are often difficult to quantify.  Nonetheless, a combination of past data and expert judgment can enable uncertainty statements that are order-of-magnitude correct for even the most difficult problems.

The standard error of a performance measure indicates the likely size of the chance variation in the reported number.  It incorporates both the effects of measurement error, survey error, and so forth, as well as the variation that occurs naturally from year to year (i.e., even if there were no change in laws, infrastructure conditions, or human behavior, there would still be chance variation in an annual count of fatalities).  DOT success in meeting GPRA goals must be viewed in the context of this background noise.

In many of the following Statistical Issues statements, BTS refers to regression standard error.  This is a modification of the standard error to take into account of linear trends in the recent past.  Such adjustment is generally needed to incorporate consistent trends due to cumulative effects of such things as education programs, changing demographics, the gradual adoption of new technologies, and so forth.  The underlying assumptions are that: over a short time period the trend of the measurement data is linear; for any given year the performance measure values are normally distributed; and the standard deviation is the same for all years.  We believe that these assumptions lead to a conservative estimate of variability.

The regression standard error is an estimate, calculated from the annual performance results, of this common standard deviation.  It may be used in the same way as a regular standard error to set confidence intervals or describe uncertainty.  For the purposes of performance measurement, it may be considered a rough approximation of the annual variability in a measure, and it will include the affects of program initiatives, influences beyond the control of DOT (e.g., weather, petroleum prices, etc.), random chance, and errors inherent in the data.

For further information about the source and accuracy (S&A) of these data, please refer to the BTS S&A compendium available at

www.bts.gov/statpol/SAcompendium.html

 


Details on DOT Measures of Overall Safety

Transportation Safety                                                                       Page 12

Measures:

1.      Transportation fatalities. (CY)

2.      Fatalities per 100 million passenger-miles. (CY)

3.      Fatalities per 100 million ton-miles of freight. (CY)

4.      Transportation injuries. (CY)

5.      Injuries per 100 million passenger-miles. (CY)

6.      Injuries per 100 million ton-miles of freight. (CY)

7.      Transportation incidents. (CY)

Scope:

This family of measures aggregates fatalities, injuries and incidents across all modes of transportation (air, highway, railroad, transit, waterborne and pipeline).

The fatality and injury rates per 100 million passenger-miles exclude pipeline fatalities and injuries due to minimal interaction with passenger miles.  Highway-rail grade crossing fatalities and injuries are not counted since they are included in data for highways.

The fatality and injury rates per 100 million ton-miles of freight include fatalities and injuries from large truck, rail, waterborne and pipeline transportation.  Highway-rail grade crossing fatalities and injuries are also included since these involve freight transportation-related fatalities and injuries that would not otherwise be counted.  Ton-miles of freight covers intercity truck, rail, water and oil pipeline transportation.  Aviation fatalities, injuries and ton-miles are excluded because the fatality and injury data are not separated from passenger air carriers. Transportation incidents include crashes, system failures, spills, releases, and other accidents of a similar nature.

Source:

The data for these measures are obtained from National Transportation Statistics published annually by the Bureau of Transportation Statistics.  Information is taken from the following tables: Transportation Fatalities by Mode; Injured Persons by Transportation Mode; U.S. Passenger-Miles (Millions); U.S. Ton-Miles of Freight (Millions); and Transportation Accidents by Mode.  The one exception is the data on large truck fatalities and injuries used for calculating fatality and injury rates per 100 million ton-miles of freight are obtained from the Federal Motor Carrier Safety Administration.

Limitations:

Double counting of fatalities and injuries may occur when an accident involves more than one mode of transportation.  Differing definitions of injuries or transportation-related fatalities makes comparison across modes of transportation problematic.  Highway injuries and incidents are obtained from a nationally representative probability sample and are estimates, while the totals for other modes of transportation are actual counts.  The highway estimates are based on crashes where a police accident report was completed and the crash resulted in property damage, injury or death.  Accidents that were not reported to the police or did not result in property damage are not included.  Highway passenger miles are calculated by multiplying vehicle-miles of travel (VMT) by the average number of occupants for each vehicle type.  VMT is based on a nation-wide sample of vehicle travel.  The average number of vehicle occupants comes from survey information.  Therefore, vehicle passenger miles is an estimate, whereas passenger-miles for other modes of transportation are calculated based on actual passenger counts and recorded trip lengths. 

Statistical Issues:

All fatality totals, and the injury and incident numbers where actual counts are recorded, are relatively accurate.  Any double counting or omissions are expected to be fairly small.  The primary source of uncertainty in these measures comes from sampling and survey errors related to estimation of highway injuries, incidents, VMT and vehicle occupancy.  Based on data from 1994-2000, the annual variations in the transportation safety measures are as follows:  the regression standard error for the  number of transportation fatalities is 0.5 thousand.  For fatality rates by passenger-miles and ton-miles, it is 0.010 and 0.007, respectively.  For number of injuries, it is 0.10 million.  For injury rates by passenger-miles and ton-miles, it is 2.50 and 0.24, respectively.  For incidents, it is 0.16 million.

Verification & Validation:

BTS compiles the data for the National Transportation Statistics from information it gathers directly in its own data systems (e.g. airlines information), information published by other sources (e.g. FHWA highway statistics), or by personal communication with the agency/organization responsible for collecting the data.  Each data source conducts error checks and monitors the accuracy of its data.  Most of these sources and their verification and validation procedures are described in subsequent data details in this report for performance measures of individual modes of transportation.

Comment:

While caution should be exercised in comparing fatalities, injuries and incidents between modes of transportation due to differences in definitions and calculations, the aggregation of these values still provides useful information.  Because the methodology for calculating these measures has remained consistent over the years, the trend information should provide a reasonably accurate picture of results. 

 

Highway fatality rate                                                                   Page 15

Measure:

Fatalities per 100 million vehicle-miles-traveled (VMT) (CY)

Scope:

The number of fatalities is the total number of motor vehicle traffic fatalities which occur on public roadways within the 50 states and Washington, D.C.

Vehicle Miles of Travel (VMT) represent the total number of vehicle miles traveled by motor vehicles on public roadways within the 50 states and Washington, D.C.

Source:

Motor vehicle traffic fatality data are obtained from NHTSA’s Fatality Analysis Reporting System (FARS).  To be included in FARS, a motor vehicle traffic crash must result in the death of a vehicle occupant or a non-motorist within 30 days of the crash.  The FARS database is based on police crash reports and other state data. FARS includes fatalities on all roadways open to the public, using the National Highways System classification of roads.  Pedestrian and bicycle fatalities that occur on public highways, but do not involve a motor vehicle, are not recorded in FARS.  However, they constitute only a small number of fatalities.

VMT data are derived from FHWA’s Traffic Volume Trends (TVT), a monthly report based on hourly traffic count data in the Highway Performance Monitoring System (HPMS).  Information is transmitted to NHTSA where it is reviewed for consistency and accuracy before being entered into the system. These data, collected at approximately 4,000 continuous traffic counting locations nationwide, are used to determine the percentage change in traffic for the current month from the same month of the previous year.  The percentage change is applied to the nationwide travel for the same month of the previous year to obtain an estimate of nationwide travel for the current month.  The data are recorded as monthly totals and cumulative yearly totals.

Limitations:

VMT data are subject to sampling errors, whose magnitude depends on how well the locations of the continuous counting locations represent nationwide traffic rates.  HPMS is also subject to estimating differences in the states, even though FHWA works to minimize such differences and differing projections on growth, population, and economic conditions that impact driving behavior.

Statistical Issues:

The primary source of uncertainty in estimating fatality rates is the denominator.  While the estimate of total fatalities used in the numerator is relatively accurate, the estimate of total vehicle miles in the denominator has far more variability.  Based on data from 1994-2000, the annual variation in the fatality rate has a regression standard error of 0.029.

The estimates of the number and percentages of persons killed in motor vehicle traffic crashes during 2001 are preliminary and are based on incomplete data and statistical models.  NHTSA’s first official estimates for 2001, the Early Assessment, are being developed and will be completed in early April 2002.  Differences between the Official Early Assessment estimates and those in this report are to be expected.

Verification & Validation:

Fatality data from FARS are reviewed and analyzed by NHTSA’s National Center for Statistics and Analysis.  Quality control procedures are built into annual data collection at 6 and 9 months, and at year’s end.  A study was completed in 1993, looking at samples of FARS cases in 1989 through 1990 to assess the accuracy of data being reported.  VMT data are reviewed by FHWA for consistency and reasonableness.

Comment:

This data program has been in use for many years and is generally accepted for describing safety on the Nation’s highways.  Adjusting raw highway fatalities and injuries by VMT provides a means of portraying the changes in highway fatalities on a constant exposure basis and facilitates year-to-year comparisons.

 

Large truck-related fatalities                                                       Page 15

Measure:

Number and rate (per million commercial VMT) of fatalities in crashes involving large trucks. (CY)

Scope:

The measure includes all fatalities (e.g., drivers and occupants of passenger cars, motorcycles, large trucks, or pedestrians) associated with crashes involving trucks with a gross vehicle weight rating of 10,000 pounds or more. The number of fatalities comes from NHTSA’s Fatality Analysis Reporting System (FARS) data, a census of fatal traffic crashes within the 50 states, Puerto Rico, and Washington, D.C.  The fatal crash rate is the number of fatalities per 100 million vehicle miles of large truck travel (VMT).

Source:

NHTSA’s Fatality Analysis Reporting System (FARS) provides fatality data.  The VMT data are derived from the Federal Highway Administration’s (FHWA) Highway Performance Monitoring System (HPMS).

Limitations:

FARS data elements are modified from year to year to respond to emphasis areas, vehicle fleet changes, and other needs for improvement. Large truck VMT reported to FHWA by each state is based on a sample of road segments and is not a census.  In addition, the methods used to calculate total VMT may vary from state to state. The methods used by the states to estimate the VMT contribution from rural and urban minor collectors are unknown. 

Statistical Issues:

The fatality counts in FARS are generally quite accurate.  The major sources of error are underreporting by some precincts and inconsistent use of the definition of a truck.  Based on 1994-2000 data, the chance variation in a given year has a regression standard error of approximately 157 fatalities.  Because the VMT data provided to FHWA from each state are estimates based on a sample of road segments, the numbers have associated sampling errors.  The methodology used by each of the states to estimate VMT is not known and may introduce additional non-sampling error.  Although states provide VMT estimates on an annual basis, they are only required to update their traffic counts at all sampling sites once every three years.  Thus an annual VMT estimate from a particular state may be based, in part, on data collected during a previous year.  Based on 1994-2000 data, the chance variation in a given year in the number of fatalities per 100 million vehicle miles of large truck travel has a regression standard error of 0.053.

Verification & Validation:

Fatality data are reviewed and analyzed by NHTSA’s National Center for Statistics and Analysis.  Quality control procedures are built into data collection and data processing.  A study using samples of 1989-1990 FARS cases was completed in 1993 to assess the accuracy of data being reported.  FHWA routinely works with state data providers to modify reported VMT values that do not appear reasonable before incorporating them into its final master file.

Comment:

The FARS data have been around for many years and are generally accepted as a good source for describing fatal crashes on the Nation’s highways.  The large truck VMT data used to calculate fatal crash rates have both sampling and non-sampling (i.e., bias) error associated with it.  The impact of these errors on FMCSA’s estimates of large truck crash rates is considered to be minimal.

 

 

Alcohol related highway fatalities                                               Page 18

Measure:

1.  Alcohol-related fatalities per 100 million vehicle-miles traveled.

2.  Percentage of highway fatalities that are alcohol related. (CY) (2001)

Scope:

The number of fatalities resulting from motor vehicle traffic crashes that are alcohol related and occur on public roadways within the 50 states and Washington, D.C. 

Source:

Motor vehicle traffic fatality data are obtained from NHTSA’s Fatality Analysis Reporting System (FARS). FARS is a census of fatal motor vehicle traffic crashes within the 50 states, Puerto Rico, and Washington, D.C.  To be included in FARS, a crash must result in the death of a vehicle occupant or a non-motorist within 30 days of the crash.  The FARS data are based on police crash reports and other state data. FARS includes fatalities on all roadways open to the public, using the National Highways System classification of roads.  Pedestrian and bicycle fatalities that occur on public highways, but do not involve a motor vehicle, are not recorded in FARS.  However, they constitute only a small number of fatalities.  A fatal motor vehicle traffic crash is alcohol-related if either a driver or a non-motorist (such as a pedestrian) involved in the crash had a measured or estimated blood alcohol concentration (BAC) of 0.01 grams per deciliter or above.

Limitations:

Blood Alcohol Concentration test results are not available for all drivers and non-occupants involved in fatal crashes.  Missing data can result for a number of reasons -- the most frequent of which is that persons are not always tested for alcohol.  To address the missing data issue, NHTSA has developed a statistical model (Multiple Imputation) to estimate specific values of BAC across the full range of possible values.  Estimating missing BAC in this manner will permit the estimation of valid statistics such as variances, measures of central tendency, confidence intervals and standard deviations. The statistical model is based on important characteristics of the crash including crash factors, vehicle factors, and person factors.  While this measure does not link alcohol with fault in fatal crashes, the more comprehensive scope of the measure compensates for a possible undercount of the extent of the alcohol impaired driving problem.  Multiple Imputation differs from the statistical model used in previous years.  However, all historical series of alcohol involvement will be revised back to the 1982 data year to reflect the estimates from the new methodology.

Statistical Issues:

The primary sources of uncertainty in this performance measure arise from information gaps in the number of intoxicated non-motorists, and from using the statistical model to estimate the number of intoxicated drivers. 

The estimates of the number and percentages of persons killed in motor vehicle traffic crashes during 2001 included in this section are preliminary and are based on incomplete data and statistical models. They were provided to meet the time restraints required for this report.  NHTSA’s first official estimates for 2001, the Early Assessment, are being developed and will be completed in early April.  Differences between the Official Early Assessment estimates and those in this report are to be expected.

Verification & Validation:

Data are reviewed and analyzed by NHTSA’s National Center for Statistics and Analysis. Quality control procedures are built into annual data collection at 6 and 9 months, and at year’s end.  In 1987 and 1988, an independent panel of academics reviewed and commented on the statistical methods used in measuring alcohol-related highway fatalities.  This report recommended that research and development utilize a model that would permit the imputation of missing BACs as a semi-continuous variable.

Comment:

This data program has been used for many years and is generally accepted for describing safety on the Nation’s highways. 

 

 

 

 

Highway injured persons rate                                                      Page 18

Measure:

Injured persons per 100 million vehicle-miles-traveled (VMT) (CY) (2001)

Scope:

The number of injured persons is an estimate of the total number of persons injured in motor vehicle traffic crashes that occur on public roadways in the 50 states and Washington, D.C.

Vehicle Miles of Travel (VMT) represent the total number of vehicle miles traveled by motor vehicles on public roadways within the 50 states and Washington, D.C.

Source:

The number of injured persons data are derived from the NHTSA’s National Automotive Sampling System (NASS) General Estimates System (GES).  The NASS GES is a nationally representative probability sample that yields national estimates of total nonfatal injury crashes, injured persons, and property-damage-only crashes. NASS GES data cover all roadways open to the public, using the National Highways System classification of roads.

VMT data are derived from FHWA’s monthly report, Traffic Volume Trends (TVT), a monthly report based on hourly traffic count data in the Highway Performance Monitoring System (HPMS).  Information is transmitted to NHTSA where it is reviewed for consistency and accuracy before being entered into the system. These data, collected at approximately 4,000 continuous traffic counting locations nationwide, are used to determine the percentage change in traffic for the current month from the same month of the previous year.  The percentage change is applied to the nationwide travel for the same month of the previous year to obtain an estimate of nationwide travel for the current month.  The data are recorded as monthly totals and cumulative yearly totals.

Limitations:

GES data are obtained from a nationally representative sample of 60 sites.  The results provide only national data, not state level data, and are subject to sampling error.  The magnitude of the sampling error depends on the number of Primary Sampling Units (PSUs) in the sample and the number of crash reports sampled within each PSU. 

VMT data are subject to sampling errors, whose magnitude depends upon how well the continuous counting locations represent nationwide traffic rates.  HPMS is subject to estimating differences in the states, although FHWA works to minimize such differences and differing projections on growth, population, and economic conditions which impact driving behavior.

Statistical Issues:

The estimate of the injury rate includes three main sources of uncertainty.  The numerator count of injuries has a standard error of 5.1% (cf. Appendix C of Traffic Safety Facts).  The denominator estimate of VMT contains both complex sampling and non-sampling errors.  Based on data from 1994-2000, the annual variation in the injury rate has a regression standard error of 4.04.

The estimates of the number and percentages of persons injured in motor vehicle traffic crashes during 2001 are preliminary and are based on incomplete data and statistical models.  NHTSA’s first official estimates for 2001, the Early Assessment, are being developed and will be completed in early April.  Differences between the Official Early Assessment estimates and those in this report are to be expected.

Verification & Validation:

Data are reviewed and analyzed by NHTSA’s National Center for Statistics and Analysis. Quality control procedures are built into annual data collection at 6 and 9 months, and at year’s end.   A study was completed in 1993, looking at samples of FARS cases in 1989 through 1990 to assess the accuracy of data being reported.  VMT data is reviewed by FHWA for consistency and reasonableness.

Comment:

This data program has been in use for many years and is generally accepted for describing safety on the Nation’s highways.  GES records injury severity in four classes: incapacitating injury, evident but not incapacitating injury, possible but not visible injury, and injury of unknown severity.  Adjusting raw highway fatalities and injuries by VMT provides a means of portraying the changes in highway fatalities on a constant exposure basis – to facilitate year-to-year comparisons.

 

 

Large truck-related injured persons                                            Page 18

Measure:

Number and rate of injured persons involving large trucks. (CY) (2001)

Scope:

The measure includes all injured persons (e.g., drivers and occupants of passenger cars, motorcycles, large trucks, or pedestrians) associated with crashes involving trucks with a gross vehicle weight rating of 10,000 pounds or more. The number of injured persons is derived from NHTSA’s General Estimates System (GES). The injury rate is the number of injured persons per 100 million vehicle miles of large truck travel (VMT).

Source:

NHTSA’s General Estimates System (GES) provides injury data. VMT data are derived from the Federal Highway Administration’s (FHWA) Highway Performance Monitoring System (HPMS).

Limitations:

GES data are obtained from a nationally representative sample of 60 sites.  The results provide only national data, not state-by-state data.  Large truck VMT reported to FHWA by each state is based on a sample of road segments and is not a census.  In addition, the methods used to calculate total VMT may vary from state to state. The methods used by the states to estimate the VMT contribution from rural and urban minor collectors are unknown.

Statistical Issues:

The GES data have a standard error of 6.9% for injuries from truck and automobile crashes (cf. Appendix C of Traffic Accident Reports).  They are less accurate than the corresponding fatality counts.  Based on 1994-2000 data, the variation due to random chance in the number of injuries, which includes sampling variability, has a regression standard error of approximately 7,091.  Because the VMT data provided to FHWA from each state are estimates based on a sample of road segments, the numbers have associated sampling errors.  The methodology used by each of the states to estimate VMT is not known and may introduce additional non-sampling error into the estimates.  Although states provide VMT estimates on an annual basis, they are only required to update their traffic counts at all sampling sites once every three years.  Thus an annual VMT estimate from a particular state may be based, in part, on data collected during a previous year.  Based on 1994-2000 data, the chance variation in a given year in the number of injured persons per 100 million vehicle miles of large truck travel has a regression standard error of 4.39.

Verification & Validation:

Injury data are reviewed and analyzed by NHTSA’s National Center for Statistics and Analysis. Quality control procedures are built into data collection and data processing.  FHWA routinely works with state data providers to modify reported VMT values that do not appear reasonable before incorporating them into its final master file.

Comment:

The data program has been around for many years and is generally accepted for describing safety on the Nation’s highways.  GES records injury severity in four classes: incapacitating injury, evident injury but not incapacitating, possible but not visible injury, and injury of unknown severity.  The large truck VMT data used to calculate injured persons rates have both sampling and non-sampling (i.e., bias) error associated with it.  The impact of these errors on FMCSA’s estimates of large truck crash rates is considered to be minimal.

 

Seat belt use                                                                                 Page 18

Measure:

Percentage of front occupants using seat belts. (CY) (2001)

Scope:

The proportion of front seat outboard passenger vehicle occupants using shoulder belts during daylight hours.

Source:

Data for 1998, 1999, and 2000 are from the National Occupant Protection Use Survey (NOPUS).  NOPUS is a National, multi-stage probability sample.  In the first stage, counties or groups of counties (Primary Sampling Units or PSUs) were grouped by region (Northeast, Midwest, South, and West),