document: As of September 30, 2010—the end of the 2 fiscal years during which Recovery Act awards were made—NIH made more than 21,500 grant awards using Recovery Act funds. In August 2010, we reported that NIH used standard review processes—peer review or administrative review— and standard criteria to award extramural scientific research grants with Recovery Act funding. These NIH Recovery Act grant awards were made to three grant categories. The grants varied in award size, geographic distribution, award duration, and research methods, consistent with scientific research grants funded with annual appropriations. The act required that these funds be obligated by NIH within a 2-year window—specifically, in fiscal years 2009 and 2010, though the activities funded by the grant may occur after fiscal year 2010. OMB guidance requires recipients of Recovery Act funding—including NIH Recovery Act grantees—to report on the number of jobs supported by the Recovery Act on a quarterly basis to the nationwide data collection system. OMB developed recipient reporting guidance and deployed a nationwide data collection system at www.federalreporting.gov. According to OMB guidance, a grantee’s estimate of the number of jobs supported by the Recovery Act each quarter must be expressed in terms of FTEs, which are calculated as the total number of hours worked and funded by the Recovery Act within a reporting quarter divided by the quarterly hours in a full-time schedule, as defined by the recipient. According to the OMB guidance, federal agencies that award Recovery Act funds should establish internal controls to ensure data quality, completeness, accuracy, and timely reports to the www.federalreporting.gov Web site. In reviewing a selection of the reports submitted to www.federalreporting.gov by grantees of agencies across the Department of Health and Human Services (HHS), the HHS Office of Inspector General found that HHS had processes in place for reporting the use of Recovery Act funds. NIH officials also reported that HHS assesses the quality of reports filed by NIH Recovery Act grantees. For example, using data assessments performed by NIH, HHS assesses the quality of the data reported by Recovery Act grantees. NIH and NIH Recovery Act grantees collect information about the FTEs supported by NIH Recovery Act funding as well as information on the other impacts of this funding from a variety of sources. Specifically, NIH collects information about FTEs supported by the Recovery Act from the www.federalreporting.gov Web site. NIH grantees, including NIH Recovery Act grantees, also submit annual progress reports to NIH that include information such as the goals and progress of their research. NIH is also participating in the development of a multiagency collaboration (called Star Metrics) to track the employment, scientific, and economic impacts of its funded research projects—including Recovery Act grants. In addition, NIH gathers information from principal investigators working on priority research areas and prepares publicly available reports (known as Investment Reports) about the potential scientific impacts of NIH- funded research. NIH Institutes and Centers (IC) select the topics featured in these reports based on (1) the importance of the topic area within the body of research funded by the IC, (2) the level of funding provided by the IC to the topic area, and (3) the level of public interest in the topic area. NIH grantees also collect information about the jobs as well as other impacts of NIH grants, including those funded by the Recovery Act, using payroll records, and effort reporting systems—such as time cards, other internal accounting records, and publications. Data reported by all NIH Recovery Act grantee institutions to the nationwide data collection system and available to NIH indicate that the number of FTEs supported by NIH Recovery Act funds generally increased from December 2009 through September 2010, then generally remained steady from December 2010 through June 2011—the most recent quarters for which data are available. As shown in figure 1, the number of FTEs supported by NIH Recovery Act funding ranged from about 12,000 in the reporting quarter ending December 2009 to about 21,000 in the quarter ending in June 2011. According to NIH officials, Recovery Act funds could eventually support a total of approximately 54,000 FTEs. This figure represents NIH’s estimated total of FTEs that could be supported throughout the Recovery Act. According to NIH officials, this estimate is projected based on the quarterly expenditure of funds reported by grantee institutions and the projected number of FTEs that NIH expects that these funds could support over the life of the Recovery Act. NIH expects that the Star Metrics program will provide additional information about the number and types of jobs funded by the Recovery Act. NIH officials reported that the Star Metrics program is an ongoing initiative and that the program is expected to release preliminary results regarding jobs in 2012. Like other NIH Recovery Act grantee institutions, data reported by our five grantee institutions also showed a general increase in FTEs. Specifically, the five institutions combined reported almost 1,000 FTEs in the quarter ending in December 2009, increasing to almost 2,000 supported FTEs in the most recent quarter for which data are available that ended in June 2011. (See fig. 2). Through responses to our data collection instrument 50 selected principal investigators at five grantee institutions provided additional information explaining how the Recovery Act funding supported FTEs. Nearly 30 percent of the 50 selected principal investigators reported that the NIH funding they received supported new positions, and about half of the principal investigators reported that the funding they received allowed them to avoid reductions in the number of employees at their institution or avoid a reduction in the number of hours worked by current employees. For example, according to the selected principal investigators, 29 percent of the jobs supported by NIH Recovery Act funding at the five grantee institutions were new employees hired by the institution using Recovery Act funding, and 54 percent were current employees. One principal investigator reported using NIH Recovery Act funding to hire more than 10 employees, many of whom had recently been laid off or had been out of work for several months. According to selected principal investigators, a majority (54 percent) of the job positions supported by NIH Recovery Act funds were parttime and the mean number of hours worked per week for all supported positions was about 20, including for example, a mean of 9 hours per week for professors and 35 hours per week for students pursuing postgraduate degrees. (See app. II for more descriptive information about the FTEs supported by NIH Recovery Act funding.) NIH officials currently receive some information reported by NIH grantees about other impacts of NIH’s Recovery Act funding, and NIH is participating in a program that NIH officials expect could help track these other impacts. In response to our data collection instrument, two-thirds of our 50 selected principal investigators—who direct research at the grantee institutions—reported that the Recovery Act funding received in fiscal years 2009 and 2010 was used to purchase research supplies and equipment and lab testing services. In addition, the majority of our 50 selected principal investigators and NIH also reported preliminary results from research projects funded by the Recovery Act. NIH officials we interviewed said that principal investigators—who direct research at the grantee institutions—including those which received Recovery Act funding—currently report some information to NIH about the other impacts of NIH-funded research. This information generally includes purchases made by the principal investigators, as well as preliminary research results submitted to NIH in their annual progress reports. NIH is participating in the Star Metrics program—a multiagency collaboration currently involving about 77 grantee institutions—to track, among other things, the scientific and nonscientific impacts of its funded research grants, including social and workforce outcomes and economic growth. NIH officials expect that the Star Metrics program could provide more information about these other impacts. Officials told us that Star Metrics is currently developing an approach to capture this information, and that they expect to pilot the approach in 2012. However, at this time there is no expected completion date for reporting this information. In their responses to our data collection instrument, many of our 50 selected principal investigators reported that they used the Recovery Act funding they received from fiscal years 2009 through 2010 to purchase supplies, equipment, and testing services used in research. Some of the principal investigators also reported that in the course of conducting some of their Recovery Act-funded research, they were able to provide scientific training to health care professionals. The selected principal investigators provided anecdotal information about the other impacts of the selected grants. Recipients of Recovery Act funding, such as grantee institutions, do not systematically track these other impacts; however, they are not required by the Recovery Act to do so. In previous work on the Recovery Act, GAO identified difficulties in assessing other impacts, particularly in instances when data on the other impacts are not readily available. (See app. III for more details of the other impacts of NIH Recovery Act funding as reported by selected principal investigators.) Purchasing Supplies and Equipment. In their responses to our data collection instrument, two-thirds of our 50 selected principal investigators reported that they used the Recovery Act funding they received from NIH to purchase or lease laboratory equipment and supplies needed to conduct research. These transactions, which we corroborated by conducting a selected review of NIH Annual Progress Reports and Recovery Act recipient reports, could translate into additional sales and revenues for the vendors. According to the principal investigators, their transactions included biomedical equipment and supplies, office supplies, computer equipment, and software licenses. For example, one principal investigator reported purchasing highly specialized imaging equipment for $27,000, as well as other medical, laboratory, and office supplies. Purchasing Specialized Services. Over a quarter of our 50 selected principal investigators reported that they used NIH’s Recovery Act funding to purchase certain laboratory testing services—such as genetic sequencing—from other research facilities that were better equipped to perform the testing and analyses. For example, one principal investigator reported contracting with a small local research company to perform specialized DNA analysis needed to determine the causes of immune deficiency disorders. In addition, a couple of principal investigators reported that they used NIH’s Recovery Act funding to contract for consultations services, such as statistical analyses and the design of models needed for their research. Some principal investigators also purchased ancillary services that they said were needed to support clinical trials, such as services providing patient transportation, recruitment, and care. Scientific Training for Health Care Professionals. Nine of our 50 selected principal investigators also reported in our data collection instrument that in the course of conducting their Recovery Act-funded research they were able to provide scientific training to health care professionals. Some of these principal investigators cited the importance of exposing current and future physicians to research-based approaches for diagnosing and treating patients. For example, one principal investigator reported that while researching how to select treatments for cancer patients, new oncology researchers—fellows and junior faculty—were trained about the effects of human genetics on care delivery for cancer patients. According to this principal investigator, understanding the effects of genetics on cancer allows physicians to personalize the treatment options they offer to patients. The principal investigator also noted that the next generation of physicians needs to be knowledgeable about genomic approaches to cancer care, while developing the foundation for their research careers. According to another principal investigator, as part of research to determine why certain genes contribute to Alzheimer’s disease, health care professionals were trained to analyze complex genetic datasets and to develop software packages needed to efficiently perform the analysis. In responses to our data collection instrument, a majority of our 50 selected principal investigators who direct research at the grantee institutions reported on the preliminary results from their research projects supported with Recovery Act funds. According to the majority of our selected principal investigators these preliminary results could contribute to future scientific developments in preventive medicine, the early detection of diseases, and medical therapies. Additionally, one principal investigator reported that some of the results of their research could lead to the development of research capabilities to be used by other researchers. A few principal investigators, however, stated that it was premature to report any preliminary results from their NIH Recovery Act- funded research, because they were still conducting clinical trials and analyzing data. In general, scientific research—including NIH-funded projects—can be lengthy and complex, and take years to obtain results. Grantee institutions and principal investigators in our review and NIH officials we interviewed reported that they track the scientific impact of NIH research—including preliminary results from research funded through the Recovery Act—primarily through peer-reviewed publications. NIH officials also reported that they track certain priority research areas and communicate potential scientific impacts through its Investment Reports. According to NIH, when a sufficiently large body of research results have accumulated the agency plans to prepare reports (similar to its Investment Reports) that highlight the impact of its Recovery Act- funded research. Other metrics used to track scientific impacts—including for Recovery Act-funded research—as reported by principal investigators in our review include the filing and approval of patent applications, the ability to secure future grant funding, presentations at professional meetings, utilization of products produced from their research, and changes to health care policies and clinical practices implemented as a result of their research. As noted earlier, the majority of our selected principal investigators provided preliminary results from their research projects supported with Recovery Act funds. The following are examples of these preliminary results:  Prevention of Diseases. One principal investigator reported that their Recovery Act-supported research on coronary heart disease indicated that high levels of calcified and noncalcified plaque, which can clog arteries and contribute to heart disease, is present in young healthy people who have a family history of premature coronary disease. According to this principal investigator, the results of this research could be used to identify persons who would benefit from heart imaging tests and preventative therapy for coronary heart disease.  Early Detection of Diseases. One principal investigator reported that their Recovery Act-supported research resulted in the identification of several hundred proteins that are associated with chronic pancreatitis. According to this principal investigator, this research could contribute towards creating new blood tests for detecting chronic pancreatitis. Another principal investigator reported identifying the symptoms that are the most important and efficient for making a diagnosis of autism in young children. Improvements in Medical Therapies. One principal investigator reported that data collected for their Recovery Act grant has yielded results in developing personalized therapeutic approaches for patients with idiopathic pulmonary fibrosis, a fatal disorder. This principal investigator noted that these results could help to simplify decision making regarding therapeutic interventions, such as for patients undergoing an organ transplant. Another principal investigator cited progress toward overcoming the resistance of colon cancer to existing treatment therapies, and another assessed two alternative therapies for coronary heart disease. A principal investigator also reported that their Recovery Act-supported research contributed to the development of a kidney dialysis monitoring device that could be less invasive and more cost-effective than the current surgically implanted monitoring systems. Improved Research Capabilities. One principal investigator reported that their Recovery Act-supported research contributed to the development of a new approach that is being utilized by other researchers studying the connections between different genes and traits, such as those that may lead to heart disease. A draft of this report was provided to HHS for review and comment. HHS provided technical comments that were incorporated as appropriate. As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its issue date. At that time, we will send copies of this report to other interested congressional committees, the Secretary of Health and Human Services, and the Director of the National Institutes of Health. This report will also be available on the GAO Web site at http://www.gao.gov. If you or your staff have any questions regarding this report, please contact Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV. To obtain the information National Institutes of Health (NIH) and selected NIH Recovery Act grantees have on the jobs supported with NIH Recovery Act funding, we interviewed NIH officials about the information they have on the full-time-equivalents (FTE) supported by the Recovery Act, and reviewed (1) NIH data containing information reported by grantee institutions to a nationwide data collection system at www.federalreporting.gov on the FTEs supported by NIH Recovery Act funding, (2) annual progress reports for fiscal year 2010 that NIH Recovery Act grantees are required to submit to NIH, and (3) other jobs information that NIH gathers from other sources. To assess the reliability of the data provided by NIH, we obtained information from agency officials knowledgeable about (1) NIH grant award data, (2) NIH Recovery Act grantee recipient reports, and (3) the jobs information that NIH gathers from other sources. We also performed data quality checks to assess the reliability of the Recovery Act grants data file received from NIH. These data quality checks involved an assessment to identify incorrect and erroneous entries or outliers. Based on the information we obtained and analyses we conducted, we determined that the data were sufficiently reliable for the purposes of this report. In addition, we selected five grantee institutions, which were universities that employ principal investigators who received NIH Recovery Act funding. The five selected grantee institutions met the following criteria: (1) received the largest portion of Recovery Act funds from NIH, (2) received the largest number of grants, and (3) reported the highest number of FTEs supported by NIH Recovery Act funds. The selected institutions were Johns Hopkins University, University of Michigan, University of Washington, University of Pennsylvania, and Duke University. The selected grantee institutions are not representative of all institutions that received Recovery Act funding. (See table 1 for more information about the five selected grantee institutions.) To gather more specific information about individual grants, we created a Web-based data collection instrument (DCI) and disseminated it to 50 selected principal investigators—10 principal investigators at each of the same five grantee institutions. The Web-based DCI contained questions about the types and number of jobs supported by the Recovery Act funding received from NIH. The selected principal investigators received grant awards that met the following criteria: (1) the grant was a new grant award and not a supplement to an existing grant, (2) the grant award was for $500,000 or greater (see table 2 for more details), and (3) the grant award was made on or before December 1, 2009. We reviewed the abstracts for all the grants that met the above criteria and made a judgmental selection of the final 50 grants—making sure to include a variety of grant types such as Challenge grants and Grand Opportunity (GO) grants that were developed for the Recovery Act. The 50 selected grant awards ranged in size from $500,000 to about $11,000,000. The principal investigators for these selected grants are not representative of all principal investigators who received NIH Recovery Act funding. To gather information about the grants from an institutional perspective, we also created a second Web-based DCI and disseminated it to an official involved in coordinating Recovery Act reporting at each of the five selected grantee institutions. We performed follow-up information gathering from selected principal investigators and administrators at grantee institutions that completed the DCI to supplement the information provided in the DCI. We also obtained and reviewed information reported by grantee institutions to the nationwide data collection system at www.federalreporting.gov about the number of jobs supported by the Recovery Act. The information on the number of FTEs supported by NIH Recovery Act funding reported to the nationwide data collection system by recipients of Recovery Act funding has certain limitations. First, the Office of Management and Budget (OMB) guidance requires FTE numbers to be reported quarterly and FTEs should not be added across quarters to obtain a cumulative number of FTEs. In addition, the calculation of FTEs may reflect full-time and/or multiple part-time jobs, therefore FTEs cannot be used to determine the total number of individual jobs. Moreover, because of a change in OMB’s reporting guidelines, FTE data for the first reporting quarter may not be comparable to the data reported for subsequent reporting quarters. The number of FTEs represents only the jobs directly supported by the Recovery Act but does not capture the jobs indirectly supported by the act or other impacts of the spending. To identify the information NIH and selected grantee institutions and principal investigators have on the other impacts of the NIH Recovery Act funding they received, we utilized the Web-based DCI disseminated to the same 50 principal investigators—10 principal investigators at each of the five selected grantee institutions—noted earlier, and interviewed NIH officials. We asked the grantee institution and principal investigators to identify other impacts such as scientific impacts, impacts in the local community, and impacts on the grantee institution and principal investigators. We also asked NIH and principal investigators to identify the metrics they use to measure and track these other impacts. We contacted the State Recovery Act representative in two of the states in which our selected universities are located (North Carolina and Pennsylvania) to identify information on the other impacts of NIH Recovery Act funding in their jurisdictions. Finally, we reviewed relevant NIH Recovery Act grant guidance as well as OMB’s Recovery Act guidance to identify Recovery Act grantee requirements for reporting information on FTEs and on the impacts of the Recovery Act grants to NIH and the nationwide data collection system at www.federalreporting.gov. We disseminated a Web-based data collection instrument (DCI) to a total of 50 selected principal investigators (10 principal investigators at each of five selected grantee institutions). The data collection instrument included questions about the jobs supported by NIH Recovery Act funding. Detailed results from selected questions in our data collection instrument related to the jobs supported by Recovery Act funding cited in this report are listed below in tables 3-6. For example, information about (1) the number of supported positions that existed before the Recovery Act and (2) the average number of hours worked by each supported job category. Not all totals add to 100 percent because respondents were given multiple answers and asked to check all that apply. We disseminated a Web-based data collection instrument to a total of 50 selected principal investigators (10 principal investigators at each of five selected grantee institutions). The data collection instrument included questions about the other impacts of NIH Recovery Act funding. Detailed results from selected questions in our data collection instrument related to the other impacts of Recovery Act funding cited in this report are listed in tables 7-10. For example, information about (1) the types of nonscientific impacts reported by selected principal investigators, and (2) the metrics used to track and measure scientific impacts. In addition to the contact named above, Will Simerl, Assistant Director; N. Rotimi Adebonojo; Leonard Brown; Carolyn Garvey; Krister Friday; Daniel S. Ries; and Monica Perez-Nelson made key contributions to this report.
summary: The American Recovery and Reinvestment Act of 2009 (Recovery Act) included $8.2 billion in funding for the National Institutes of Health (NIH) to be used to support additional scientific research-including extramural grants at universities and other research institutions. In 2009, the Acting Director of NIH testified that each extramural grant awarded with Recovery Act funding had the potential of supporting employment--full- or part-time scientific jobs--in addition to other impacts, such as contributing to advances in improving public health. GAO was asked to examine the use of Recovery Act funds by NIH grantees. Specifically, GAO addresses the information available from NIH and its grantees about the extent to which NIH Recovery Act funding (1) supported jobs, and (2) had other impacts. To obtain information on job impacts, GAO reviewed a database containing information NIH Recovery Act grantees reported to the national data collection system and interviewed NIH officials. To obtain more specific jobs information about individual grants, GAO administered a Web-based data collection instrument to 50 selected principal investigators who direct research at grantee institutions--10 principal investigators at each of five selected grantee institutions. The selected principal investigators had generally received awards of $500,000 or more. To obtain information on other Recovery Act impacts, GAO used information from the data collection instrument and interviewed NIH officials. Data reported by all of NIH's Recovery Act grantee institutions to the national data collection system at www.federalreporting.gov and available to NIH indicate that the number of full-time equivalent (FTEs) jobs supported by NIH Recovery Act funds increased from December 2009 through September 2010, and then remained steady from December 2010 through June 2011--the most recent quarter for which data are available. The number of FTEs supported by NIH Recovery Act funds increased from about 12,000 in the reporting quarter ending December 2009 to about 21,000 in the quarter ending in June 2011. The 50 selected principal investigators who direct research at the grantee institutions in GAO's review provided additional information explaining how the Recovery Act funding supported FTEs. Nearly one-third of the selected principal investigators reported that the NIH Recovery Act funding they received supported new positions, and about half of the principal investigators reported that the funding they received allowed them to avoid reductions in jobs or avoid a reduction in the number of hours worked by current employees. The selected principal investigators also reported that the Recovery Act funding they received primarily supported scientists and other faculty. NIH officials we interviewed reported that they receive some information from principal investigators about the other impacts of NIH-funded research, such as preliminary research results included in annual progress reports. NIH is also participating in the Star Metrics program--a multiagency venture to monitor the scientific, social, and economic impacts of federally funded science--which NIH officials expect could provide more information about these impacts. While Star Metrics is currently developing an approach to capture information about the other impacts of NIH grant funding, there is no expected completion date for reporting this information. In response to GAO's data collection instrument, selected principal investigators who direct research at the grantee institutions in GAO's review reported that the use of Recovery Act funds resulted in purchases of research supplies, equipment, laboratory testing services, and scientific training of health care professionals. The majority of the 50 selected principal investigators in GAO's review also reported preliminary results from their Recovery Act-funded research that could contribute to future scientific developments in prevention and early detection of disease, improvements in medical therapies, and improved research capabilities. The principal investigators in GAO's review and NIH officials GAO interviewed reported that they track the scientific impact of NIH research--including the impact of research funded through the Recovery Act--primarily through peer-reviewed publications, but also through other metrics such as the filing and approval of patent applications. According to NIH officials, when a sufficiently large body of research results has accumulated, NIH plans to prepare reports--similar to its existing publicly available Investment Reports--that will highlight the impact of its Recovery Act-funded research. The Department of Health and Human Services provided technical comments on a draft of this report, which GAO incorporated as appropriate.