A specific quality of the discussion about innovation indices (scoreboards) is that more often than not the subject is dealt with from a purely technical point of view. Such a narrow approach silently assumes that indices used as a policy tool are an accurate reflection of the phenomenon and should not be questioned, and also that the whole discussion concerning them should refer to methodological aspects and is best left to the statisticians. This author is of the opinion that for an accurate evaluation of the value of indices as a policy tool, it is necessary to consider the matter from the broader point of view and from the context in which such indices are generated and used. This article puts forward the thesis that progress in science and innovation policy studies depends on a diversity of issues, approaches and perspectives. If that is the case, maintaining thematic and methodological variety may be more important than creating coherent and closed analytical tools, i.e. indices. The advantage of indices is that they focus attention on those variables which are deemed to be key. Among their disadvantages, however, are their highly abstract nature (in order to understand innovation-related phenomena, it is necessary to study them in tangible, composite forms); their tendency to skip unmeasurable determinants; their prior acceptance of definitions and concepts of innovation (instead of searching for them); the way they apply a single yardstick to diverse countries and regions, assumed linearity and causality in a complex and non-linear world, the way they direct policy towards implementing indicators (rather than identifying and solving problems). It is suggested that big data revolution will allow the emergence of a new measurement tools that will replace innovation indices.