The validation of peer review through research impact measures and the implications for funding strategies.

There is a paucity of data in the literature concerning the validation of the grant application peer review process, which is used to help direct billions of dollars in research funds. Ultimately, this validation will hinge upon empirical data relating the output of funded projects to the predictions implicit in the overall scientific merit scores from the peer review of submitted applications. In an effort to address this need, the American Institute of Biological Sciences (AIBS) conducted a retrospective analysis of peer review data of 2,063 applications submitted to a particular research program and the bibliometric output of the resultant 227 funded projects over an 8-year period. Peer review scores associated with applications were found to be moderately correlated with the total time-adjusted citation output of funded projects, although a high degree of variability existed in the data. Analysis over time revealed that as average annual scores of all applications (both funded and unfunded) submitted to this program improved with time, the average annual citation output per application increased. Citation impact did not correlate with the amount of funds awarded per application or with the total annual programmatic budget. However, the number of funded applications per year was found to correlate well with total annual citation impact, suggesting that improving funding success rates by reducing the size of awards may be an efficient strategy to optimize the scientific impact of research program portfolios. This strategy must be weighed against the need for a balanced research portfolio and the inherent high costs of some areas of research. The relationship observed between peer review scores and bibliometric output lays the groundwork for establishing a model system for future prospective testing of the validity of peer review formats and procedures.

Tags
Data and Resources
To access the resources you must log in

This item has no data

Identity

Description: The Identity category includes attributes that support the identification of the resource.

Field Value
PID pmid:25184367
PID https://www.doi.org/10.1371/journal.pone.0106474
PID pmc:PMC4153641
URL https://dx.doi.org/10.1371/journal.pone.0106474
URL https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0106474&type=printable
URL https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0106474
URL https://dx.plos.org/10.1371/journal.pone.0106474
URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4153641/
URL https://ui.adsabs.harvard.edu/abs/2014PLoSO...9j6474G/abstract
URL https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/25184367/?tool=EBI
URL https://paperity.org/p/60446840/the-validation-of-peer-review-through-research-impact-measures-and-the-implications-for
URL http://dx.doi.org/10.1371/journal.pone.0106474
URL http://dx.plos.org/10.1371/journal.pone.0106474
URL https://doaj.org/toc/1932-6203
URL https://europepmc.org/articles/PMC4153641/
URL http://europepmc.org/articles/PMC4153641
URL https://core.ac.uk/display/150516982
URL https://academic.microsoft.com/#/detail/2092309740
Access Modality

Description: The Access Modality category includes attributes that report the modality of exploitation of the resource.

Field Value
Access Right Open Access
Attribution

Description: Authorships and contributors

Field Value
Author Stephen Gallo, 0000-0001-6043-2153
Publishing

Description: Attributes about the publishing venue (e.g. journal) and deposit location (e.g. repository)

Field Value
Collected From Europe PubMed Central; PubMed Central; ORCID; UnpayWall; DOAJ-Articles; Crossref; Microsoft Academic Graph
Hosted By Europe PubMed Central; PLoS ONE
Publication Date 2014-09-03
Additional Info
Field Value
Language Undetermined
Resource Type Article; UNKNOWN
keyword Q
keyword R
keyword keywords.General Biochemistry, Genetics and Molecular Biology
system:type publication
Management Info
Field Value
Source https://science-innovation-policy.openaire.eu/search/publication?articleId=dedup_wf_001::3804a4f577e917c32866de4719894e49
Author jsonws_user
Last Updated 23 December 2020, 09:22 (CET)
Created 23 December 2020, 09:22 (CET)