Studying grant decision-making: a linguistic analysis of review reports

Peer and panel review are the dominant forms of grant decision-making, despite its serious weaknesses as shown by many studies. This paper contributes to the understanding of the grant selection process through a linguistic analysis of the review reports. We reconstruct in that way several aspects of the evaluation and selection process: what dimensions of the proposal are discussed during the process and how, and what distinguishes between the successful and non-successful applications? We combine the linguistic findings with interviews with panel members and with bibliometric performance scores of applicants. The former gives the context, and the latter helps to interpret the linguistic findings. The analysis shows that the performance of the applicant and the content of the proposed study are assessed with the same categories, suggesting that the panelists actually do not make a difference between past performance and promising new research ideas. The analysis also suggests that the panels focus on rejecting the applications by searching for weak points, and not on finding the high-risk/high-gain groundbreaking ideas that may be in the proposal. This may easily result in sub-optimal selections, in low predictive validity, and in bias. QC 20180918

Tags
Data and Resources
To access the resources you must log in

This item has no data

Identity

Description: The Identity category includes attributes that support the identification of the resource.

Field Value
PID https://www.doi.org/10.1007/s11192-018-2848-x
PID pmc:PMC6132964
URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6132964/
URL https://www.narcis.nl/publication/RecordID/oai%3Aresearch.vu.nl%3Apublications%2Fb8d34c7e-9a4c-4ff4-89c1-be947008d517
URL https://dblp.uni-trier.de/db/journals/scientometrics/scientometrics117.html#BesselaarSS18
URL https://research.vu.nl/en/publications/b8d34c7e-9a4c-4ff4-89c1-be947008d517
URL http://dx.doi.org/10.1007/s11192-018-2848-x
URL https://link.springer.com/article/10.1007/s11192-018-2848-x
URL http://europepmc.org/articles/PMC6132964
URL https://ideas.repec.org/a/spr/scient/v117y2018i1d10.1007_s11192-018-2848-x.html
URL http://link.springer.com/content/pdf/10.1007/s11192-018-2848-x.pdf
URL http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-235172
URL https://academic.microsoft.com/#/detail/2883965652
URL https://doi.org/10.1007/s11192-018-2848-x
URL http://kth.diva-portal.org/smash/record.jsf?pid=diva2:1248653
URL http://link.springer.com/article/10.1007/s11192-018-2848-x/fulltext.html
URL https://link.springer.com/content/pdf/10.1007%2Fs11192-018-2848-x.pdf
URL https://research.vu.nl/en/publications/studying-grant-decision-making-a-linguistic-analysis-of-review-re
Access Modality

Description: The Access Modality category includes attributes that report the modality of exploitation of the resource.

Field Value
Access Right Open Access
Attribution

Description: Authorships and contributors

Field Value
Author Peter van den Besselaar, 0000-0002-8304-8565
Author Ulf Sandström
Author Hélène Schiffbaenker
Contributor Organization Sciences
Contributor Network Institute
Contributor Organization & Processes of Organizing in Society (OPOS)
Publishing

Description: Attributes about the publishing venue (e.g. journal) and deposit location (e.g. repository)

Field Value
Collected From PubMed Central; Datacite; UnpayWall; Publikationer från KTH; NARCIS; Crossref; Microsoft Academic Graph
Hosted By Europe PubMed Central; Scientometrics; DSpace at VU; Publikationer från KTH
Publication Date 2018-07-13
Publisher KTH, Hållbarhet och industriell dynamik
Additional Info
Field Value
Country Sweden
Format application/pdf
Language English
Resource Type Other literature type; Article
keyword Samhällsvetenskap
keyword Peer review; Panel review; Research grants; Decision-making Linguistics LIWC; European Research Council _ERC_
system:type publication
Management Info
Field Value
Source https://science-innovation-policy.openaire.eu/search/publication?articleId=dedup_wf_001::636365c9f9c427bef77fce0642256fa4
Author jsonws_user
Last Updated 26 December 2020, 02:59 (CET)
Created 26 December 2020, 02:59 (CET)