Methods to account for citation inflation in research evaluation

Quantitative research evaluation requires measures that are transparent, relatively simple, and free of disciplinary and temporal bias. We document and provide a solution to a hitherto unaddressed temporal bias – citation inflation – which arises from the basic fact that scientific publication is steadily growing at roughly 4% per year. Moreover, because the total production of citations grows by a factor of 2 every 12 years, this means that the real value of a citation depends on when it was produced. Consequently, failing to convert nominal citation values into real citation values produces significant mis-measurement of scientific impact. To address this problem, we develop a citation deflator method, outline the steps to generalize and implement it using the Web of Science portal, and analyze a large set of researchers from biology and physics to demonstrate how two common evaluation metrics – total citations and h-index – can differ by a remarkable amount depending on whether the underlying citation counts are deflated or not. In particular, our results show that the scientific impact of prior generations is likely to be significantly underestimated when citations are not deflated, often by 100% or more of the nominal value. Thus, our study points to the need for a systemic overhaul of the counting methods used evaluating citation impact – especially in the case of researchers, journals, and institutions – which can span several decades and thus several doubling periods. Peer reviewed

Tags
Data and Resources
To access the resources you must log in

This item has no data

Identity

Description: The Identity category includes attributes that support the identification of the resource.

Field Value
PID https://www.doi.org/10.1016/j.respol.2019.04.009
PID https://www.doi.org/10.2139/ssrn.3193712
URL https://api.elsevier.com/content/article/PII:S0048733319301003?httpAccept=text/plain
URL http://hdl.handle.net/11311/1088228
URL https://www.sciencedirect.com/science/article/pii/S0048733319301003
URL https://escholarship.org/uc/item/31r806zg
URL https://aaltodoc.aalto.fi/handle/123456789/38415
URL https://doi.org/10.1016/j.respol.2019.04.009
URL https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3193712
URL https://research.aalto.fi/fi/publications/methods-to-account-for-citation-inflation-in-research-evaluation(ff6168f0-73a0-440f-a258-f2d76ffaf827).html
URL https://escholarship.org/content/qt31r806zg/qt31r806zg.pdf?t=q4giwa
URL http://juuli.fi/Record/0339518819
URL http://dx.doi.org/10.1016/j.respol.2019.04.009
URL https://api.elsevier.com/content/article/PII:S0048733319301003?httpAccept=text/xml
URL https://research.aalto.fi/files/33938546/1_s2.0_S0048733319301003_main_1.pdf
URL https://academic.microsoft.com/#/detail/2935482946
URL http://dx.doi.org/10.2139/ssrn.3193712
URL https://aaltodoc.aalto.fi:443/handle/123456789/38415
URL https://re.public.polimi.it/handle/11311/1088228
Access Modality

Description: The Access Modality category includes attributes that report the modality of exploitation of the resource.

Field Value
Access Right Open Access
Attribution

Description: Authorships and contributors

Field Value
Author Fabio Pammolli, 0000-0001-7056-1303
Contributor University of California Merced
Contributor Department of Computer Science
Contributor Polytechnic University of Milan
Contributor Indiana University Bloomington
Contributor Aalto-yliopisto
Contributor Aalto University
Publishing

Description: Attributes about the publishing venue (e.g. journal) and deposit location (e.g. repository)

Field Value
Collected From Aaltodoc Publication Archive; VIRTA; ORCID; UnpayWall; RE.PUBLIC@POLIMI Research Publications at Politecnico di Milano; Crossref; Microsoft Academic Graph
Hosted By Aaltodoc Publication Archive; VIRTA; RE.PUBLIC@POLIMI Research Publications at Politecnico di Milano; Research Policy; SSRN Electronic Journal
Publication Date 2019-09-01
Publisher ELSEVIER SCIENCE BV
Additional Info
Field Value
Country Finland; Italy
Format application/pdf
Language English
Resource Type Article
system:type publication
Management Info
Field Value
Source https://science-innovation-policy.openaire.eu/search/publication?articleId=dedup_wf_001::a1091e56ab40fd3056252b2551912304
Author jsonws_user
Last Updated 22 December 2020, 18:06 (CET)
Created 22 December 2020, 18:06 (CET)