Section I
Describe your agency's risk assessment premise(s) and process(es) that you performed subsequent to compiling your full program inventory. List the risk-susceptible programs identified through your risk assessments. Be sure to include the programs previously identified in the former Section 57 of OMB Circular A-11.
In the first year of implementation of the Improper Payments Information Act (IPIA), the Department of Transportation (DOT) reviewed the majority of its programs and activities to identify those that may be susceptible to significant improper payments. This improper payments risk assessment was conducted in two phases. For the first phase, the DOT engaged KPMG, LLP to research and develop an improper payment risk assessment process and methodology. DOT identified ten programs with the highest potential for improper payments based on the highest 2003 Fiscal Year expenditures, which comprised the majority of FY 2003 DOT expenditures.
The following programs were identified as most susceptible to improper payments based on DOT's assessment of their full program inventory:
The ten identified programs were subject to an initial risk assessment to determine the sampling size to be used. Each program then underwent an in-depth review based on OMB guidelines.
The second risk assessment phase for the balance of the DOT programs focused on lower dollar value programs, which used a self-assessment risk based methodology. The risk self-assessment methodology was based on the KPMG-developed gross risk assessment tool with detailed criteria guidelines, which would determine which programs may have a higher improper payment risk.
Section II
Describe the statistical sampling process conducted to estimate the improper payment rate for each program identified.
DOT engaged KPMG, LLP to conduct its improper payment review of Fiscal Year 2003 payments for the ten identified programs. KPMG acquired knowledge of the programs through research, questionnaires and interviews and based on the obtained knowledge, identified risk criteria that were used as the basis for further assessing program risk. Risk criteria included gross expended amount; complexity of payment calculations; established internal controls and oversight; type of program recipients; number of program recipients; volume of payments; probability of program growth; and changes in the program. Results of the risk assessment were utilized to create a sampling plan.
A stratified sampling design was used for testing payments based on the FY 2003 disbursement amounts and the assessed risk of the program. This sampling plan provided statistical confidence of 95 percent by measuring the actual variability of the dollar data and, through a weighted set of formulas, providing a natural measure of the relative sampling error. Use of appropriate population weights with the stratification methodology produced an unbiased estimate for the whole file. This statistical sampling design allowed for calculation of statistical projections for the amount of improper payments for each tested program and for projection of attributes across the selected population.
The sample selection of payments was random within each stratum. Allocations to each stratum were based on the dollar value of the payment using the Neyman Optimization formula with a design precision ranging from one percent to 10 percent of the estimated dollar amount, depending on the assessed risk level. For high risk, the design precision was between one percent and three percent; for moderate risk, between five percent and seven percent; and for low risk, 10 percent. A two-sided 95 percent confidence limit was achieved. The stratification design relies on dollar amount ranges to generate better precision than simple random sampling for a given sample size. The stratification ensured that all strata were mutually exclusive and collectively exhaustive, thus covering the entire population of payments.
DOT provided all payment transactions for the fiscal year 2003, for nine of the ten programs. For the tenth program, payroll, DOT provided all payment activities for three pay periods in FY 2003, randomly selected by KPMG. A total sample size of 1,030 payments was randomly selected among a minimum of three quantitative strata based on the payment amount. The table below shows the overall sample sizes by program.
* Based on FY 2003 Estimated Amount
** Section 57 Program
For the balance of DOT programs, each DOT Operating Administration reviewed its remaining programs following the gross risk assessment methodology developed by KPMG. Those DOT programs comprise the minority of DOT expenditures and were in many cases not subject to the reporting requirements of the IPIA. The final self-assessment reviews were certified by each Operating Administration CFO and reviewed by the Office of Financial Management in the Office of the Secretary. No other DOT programs with lower dollar values were deemed a high risk for improper payments. In many cases, the very low dollar value of the program alone would have disqualified that program as an improper payment risk.
Section III
Explain the corrective actions your agency plans to implement to reduce the estimated rate of improper payments. Include in this discussion what is seen as the cause(s) of errors and the corresponding steps necessary to prevent future occurrences. If efforts are already underway, and/or have been ongoing for some length of time, it is appropriate to include that information in this section.
KPMG did not identify any improper payments exceeding both 2.5% of program payments and $10 million. The results to date have shown that no corrective actions have been warranted. However, KPMG's scope was limited in three ways. First, there was an inadvertent sample population reduction in the FHWA Federal Aid program based on the extract requirements provided by FHWA. DOT and KPMG will work to identify the missing population amounts and review the additional program. Second, FAA was not able to provide data or answers to outstanding questions for the FAA Operations and FAA Facilities and Equipment Programs on time. Therefore, the items with outstanding data were considered and projected as questionable transactions.
Third, for electronically-processed grants there was limited data available for KPMG to review based the Federal Financial Assistance Management Improvement Act. To address this shortcoming, DOT has devised an innovative research and development (R&D) strategy for effectively addressing the grants program review limitations. This strategy involves using a proof of concept project to test the feasibility of using the Single Audit process to meet the improper payment estimation and remediation requirements of the Improper Payments Information Act. This proposal has been approved by OMB, and DOT has executed a contract with a consultant to begin the process of this proof of concept effort. DOT will present a R&D project agreement to OMB after the state selection and agreement has been formalized.
In this first year of execution of the IPIA, developing process, methodology and risk assessment procedures, numerous lessons refinements and enhancements were needed. For example, standardized financial measurement points were needed to ensure accurate data were available. Extensive staff learning curves were required to educate operating administration financial personnel on the requirements of this newly-developed methodology. Staff misunderstandings between what was requested by the contractor and what was provided need further clarification and process improvements to resolve those discrepancies.
Section IV
The table of Improper Payment Reduction Outlook FY 2003 - FY 2007.
Program
Over Payments Projection
Under Payments Projection
FY03 Outlays
FY03 IP%
FY04 IP%
FY05 IP%
FY06 IP%
FY07 IP%
Airport Improvement Program (FAA)*
$0
($36,568)
$2,577,000,000
0%**
***
***
***
***
Formula Grants (FTA)*
$0
$0
$4,979,000,000
0%
***
***
***
***
Capital Investment Grants (FTA)*
$0
$0
$2,812,000,000
0%
***
***
***
***
Federal Aid (FHWA)*
$0
$0
$17,767,863,023
0%
***
***
***
***
* Section 57 Program
** The FAA Airport Improvement program improper underpayment projection error was statistically insignificant.
*** It is likely that information developed by DOT's proof of concept project will result in improved identification and measurement methods for electronically-processed grant payments. As a result, any increases in future improper payments made in connection with DOT's grant programs should be analyzed to determine if the increases were caused by improvements in identification or by internal control or process weaknesses.
KPMG was able to test electronically-processed grants for eligibility, award and payment approval, incurrence of cost during the funding period, payment within the award or other funding limitations and that payment was sent to the proper recipient. However, KPMG encountered data limitations that will be encountered by all Federal agencies that electronically processed grant payments in compliance with PL 106-107.
As a result of PL 106-107 streamlining the payment process, documentation needed to determine if the payment was calculated correctly, discounts and credits were properly taken and all costs were allowable, is not maintained. Accordingly, KPMG was not able to assess compliance with these requirements.
To resolve the issue of limited data in support of grant payments made in compliance with PL 106-107, DOT has devised an innovative research and development (R&D) strategy. This strategy involves using a proof of concept project to test the feasibility of using the Single Audit process to provide the information needed to determine if grant payments made in compliance with PL 106-107 meet the improper payment estimation and remediation requirements of the IPIA.
Section V
Discuss your agency's recovery auditing effort, if applicable, including the amount of recoveries expected, the actions taken to recover them, and the business process changes and internal controls instituted and/or strengthened to prevent further occurrences.
For the past two years the Department of Transportation has engaged PRG-Schultz to provide recovery audit services. During that time PRG-Schultz has reviewed payments made by the DOT agencies to their commercial vendors for the fiscal years 2000, 2001, and 2002. The recovery audit produced just $216,382 (0.014 percent) in recoveries as of September 2004 out of a reviewed base of $1,543,058,000.
The monies recovered resulted from a combination of PRG-Schultz review of vendor statements of accounts, results from proprietary duplicate payment queries, and a manual review of invoices and contracts. A review of vendor statements was facilitated by a mass mailing requesting statements of accounts from the department's commercial vendors. The open credits identified on the statements were verified through correspondence with the vendor and research of all electronic and hard copy data available. Secondly, PRG-Schultz executed proprietary queries on each Operating Administration's financial data set to extract potential duplicate or erroneous payments. These potential duplicate or erroneous payments were researched and proven out through a review of invoices, contracts, and financial data. A manual review of contracts and invoices was conducted to round out the review.
Overall, the recoveries have been minimal relative to the Department's total commercial spend. The DOT is continuing to pursue the use of recovery audits and has expanded the scope of recovery audits to include all transactions older than one year in the recovery audit scope. To date, no internal controls or business process change recommendations have resulted from recovery audits.
The following departmental recovery audit management report measures the recovery audit progress by each DOT Operating Administration and recovery audit errors rates as a function of overpayments recovered.
** Error rate for DOT is based on total overpayments divided by total adjusted amount complete. It is not the sum of the error rates for each agency. Individual agencies' error rates are based on their specific overpayments in relationship to their specific audit dollar volume.
Section VI
Describe the steps the agency has taken and plans to take (including time line) to ensure that agency managers (including the agency head) are held accountable for reducing and recovering improper payments.
DOT has taken a very strong role in ensuring that agency managers are held accountable for reducing and recovering improper payments. The DOT CFO has taken the lead in announcing the improper payment program and follows the program closely. Each Operating Administration CFO was briefed at the agency CFO meeting and follow up meetings have included improper payments as an agenda item. Operating Administration financial managers are frequently briefed at the DOT Financial Management Council on program updates, progress and problem resolutions. Further, each Operating Administration CFO was required to certify the improper payments review results for his or her Operating Administration. The DOT CFO Office of Financial Management (OFM) provides a strong oversight and program management role in reviewing Operating Administrations' progress and resolving programmatic obstacles.
Section VII
Describe whether the agency has the information systems and other infrastructure it needs to reduce improper payments to the levels the agency has targeted.
Currently the DOT possesses the information systems and other infrastructure it needs to measure improper payments. DOT is striving to improve the quality of its information by refining its internal process and measurement procedures. DOT has also devised an innovative research and development strategy for effectively addressing the grants program review limitations. This strategy involves using a proof of concept R&D project to test the feasibility of using the Single Audit process to meet the improper payment estimation and remediation requirements of the Improper Payments Information Act. The results of this R&D project may require additional infrastructures.
Section VIII
A description of any statutory or regulatory barriers that may limit the agencies' corrective actions in reducing improper payments.
KPMG encountered difficulty in timely identifying disallowed costs charged on sampled grant payments. Federal agencies rely on two primary sources for identification of disallowed costs for grants - the Federal Audit Clearinghouse and grant closeout process. The Federal Audit Clearinghouse serves as the central source for OMB Circular A-133 audit reports for entities receiving federal funds in excess of $500,000 and is responsible for initiating the distribution of entity audit reports to the grant-making Federal agencies when the audit reports reveal findings relevant to grants issued by the agency. At the time of award closeout, Federal agencies are required to obtain and review all final program and financial reports and may have additional procedures for reviewing all costs charged to the award.
The majority of sampled payments were from grants that had not been closed at the time of testing. For this reason, KPMG relied heavily on review of the audit results posted to the Clearinghouse Web site (which appear on the entity SF-SAC forms) to identify disallowed costs. KPMG found that the Clearinghouse did not allow for timely identification of disallowed costs. A majority of the entity audit results for the period covering the sampled payment were not available. KPMG contacted the Clearinghouse regarding the unavailability of entity FY 2002 and FY 2003 reports and was told that missing reports could be due to the Clearinghouse reporting requirements, which allow entities nine months after the end of the entity fiscal year to submit reporting packages. In some cases the sampled payment occurred in the Federal fiscal year ending 9/30/03 but the payment was to an entity with a fiscal year ending December 31, 2003, thus the reporting package was not due to the Clearinghouse until September 30, 2004. Another reason for unavailability is the Clearinghouse reviews all reporting packages prior to posting SF-SAC forms with audit results to the Web site. If a reporting package is incomplete or has formatting or substantive inadequacies, the package is returned to the recipient. A Clearinghouse representative stated that in some cases it may take several years for the entity to provide a reporting package that meets standards and can be uploaded to the Web site. Finally, the Clearinghouse has a backlog of reporting packages and in some cases it may take up to nine months after receipt of reporting packages to upload the audit results to the Web site.
In addition to unavailability of audit results on the Clearinghouse Web site, KPMG had difficulty obtaining copies of reports with findings in a timely manner. Review of the entity SF-SAC form allows for identification of allowable costs/cost principles findings for a particular grant program for the fiscal year. In order to determine if the audit report included more specific information that would allow for identification of the award or payment for which disallowed costs were charged requires reviewing the audit report. KPMG noted that it takes approximately 2.5 months from the date of request to receive report copies. This is because the request is submitted to the Clearinghouse who then coordinates with the cognizant agency to mail report copies.
Grants Streamlining
The Federal Financial Assistance Management Improvement Act of 1999 (P.L 106-107) was designed to simplify and streamline the grants management process including the grants payment process. As part of the grants simplification and streamlining effort, P.L. 106-107 required Federal agencies to adopt payment systems that allow for recipients to request and receive grant drawdowns electronically. Both FTA and FAA grants are processed through the ECHO electronic payment system and FHWA Federal-Aid grants are processed through the payment system RASPS. Outside the extensive data in the respective grants management system, in most cases there is little supporting documentation for electronic payment making it difficult to identify improper payments. The electronic payment systems typically require the recipient to indicate the project for which the drawdown request is being made, but do not require additional information be provided to support the payment beyond the information in the grants management system. KPMG was able to test the electronically processed grants for eligibility, award and payment approval, incurrence of cost during the funding period, payment within the award or other funding limitations and that the payment was sent to the proper recipient. However, without supporting detail KPMG could not determine whether the payment was calculated correctly, discounts and credits were properly taken and all costs were allowable. It should be noted that all Federal agencies with electronically processed grants in compliance with PL 106-107 would encounter this same limitation. Furthermore, to resolve this shortcoming, DOT has devised an innovative research and development (R&D) strategy for effectively addressing the grants program review limitations. This strategy involves using a proof of concept project to test the feasibility of using the Single Audit process to meet the improper payment estimation and remediation requirements of the IPIA. This proposal has been presented to OMB, and DOT has executed a contract with a consultant to begin the process of this proof of concept effort.
Section IX
Additional comments, if any, on overall agency efforts, specific programs, best practices, or common challenges identified as a result of IPIA implementation.
KPMG had some difficulty obtaining accurate program payment populations. KPMG relied on the OAs to provide accounting specifications for extracts containing all fiscal year 2003 program payments. In several cases the obtained populations included payments not related to the program under review and therefore had to be further refined. Based on lessons learned implementing this new program, DOT will be refining its process and procedures, along with generating standardized measurement points.
In addition to problems encountered in obtaining accurate program populations, KPMG requested that the program populations only include payment transactions. However, KPMG found the extracts that DOT provided included reclassifications and adjustments that were not easily identifiable. Based on KPMG's review of documentation, it was determined that items were not payments and the population had to be scrubbed and sample items replaced by several OAs.
Despite the above-described difficulties, DOT completed several noteworthy events. DOT has put into place a Departmental-wide improper payments review process and methodology to review all programs. Second, DOT has devised an innovative research and development strategy for effectively addressing the grants program review limitations. This strategy involves using a proof of concept project to test the feasibility of using the Single Audit process to meet the improper payment estimation and remediation requirements of the Improper Payments Information Act. This proposal has been presented to OMB, and DOT has executed a contract with a consultant to begin the process of this proof of concept effort.
Lastly, DOT has successfully implemented a Department-wide recovery audit program which has been a model as noted by OMB. The recovery auditor has access to our financial system to review payment records and has seamlessly been integrated into our business process with minimal cost to the government.