The "Black-Box" of institutional scores : analyzing the distribution of the values of the H and G Indexes in medicine schools in Romania

Abstract

Measuring the university research performance has been an important focus of the higher education policies in past decade in Romania. In the present study we considered alternative methodologies for evaluating quality of research in the faculties of medicine. We set to compare the perspectives of past official evaluations with alternatives based on h and g indexes of the academics within these faculties and subsequent successive indexes and averages. We analyzed the distribution of the values of the individual h and g indexes and we rejected the universality claim hypothesis, according to which all university h- and g-index distributions follow a single functional form, proportional with the size of the universities. However, using the Characteristic Scores and Scales approach, we show that the shape of distributions is quite similar across universities revealing the skewness of scientific productivity. Given the high skewness of all distributions, we conclude that all three collective aggregation rules considered, averages, h- and g-successive indexes fail to provide an accurate measure of the differences between the individual academics within the six medical schools, and fail to provide scientific achievement incentives for the wide majority of the academic staff within the analysed faculties.