Skip to Main Content

Research Impacts Using Citation Metrics

Email this link:

Liaison Librarian

Profile Photo
Julia Gelfand
Office: Science Library 228

Phone: 949-824-4971


Other Resources & LibGuides

Many people have compiled interesting guides about how to introduce and present the growing resources on research and scholarly impacts. They continue to inspire me to revise this guide. Some of them are here:

Library Perspective

Libraries monitor their collection strength using a variety of indicators.

There are numerous collection assessment tools that document for libraries how their collections and services compare with similar institutions. Disciplinary information such as Digital Humanities: Questions and Answers addresses many issues related to trends and research directions.

Examples may include OCLC's WorldCat Collection Assessment Tool, and Comparative Collection Assessment for Books.

Membership in national consortia such as the American Association of Universities (AAU) and the Association of Research Libraries (ARL) provide annual demographics about a library collection, size, staff  and the operating costs. 

The American Library Association and the The Bowker Annual of Library and Book Trade Information give statistical analysis of library resources. 

For International impacts, there are a number of resources that describe library holdings, WorldCat, an international consortia of libraries; governmental funding structures, rankings and assessment exercises such as in the UK with SCONUL ; in Australia, Libraries Australia

Bibliometrics PowerPoint

Contains information on:

  • Google Scholar Profiles
  • Google Scholar Universal Gadget for Scientific Publication Citation Counting
  • Microsoft Academic Search
  • Web of Science Citation Reports

Role of Negative Impact

Recent trends in bogus metrics and predatory publishers encourage scholars to be very cautious about publishing claims regarding impact.  There has been a call for what can be considered negative metrics.  The Center for the Study of Interdisciplinarity has developed some lists and they appear in the following publications:

Sources about Predatory Publishing Practices

There is an increase in predatory sources that do not honor peer review and other standard methods of professional collaboration in scholarly publishing activities. This has extended to conferences as well as invitations to serve on editorial boards, etc.  You can verify the sources in these sources:

Bibliometric Limitations

This series of videos from Irish academics discuss some of the limitations of bibliometrics, and the importance of using a variety of methods to evaluate impact.

Social Media


Metrics Toolkit launches to help academics better understand research impact - January 30, 2018

 Metrics Toolkit, a resource built to help researchers and evaluators navigate the ever-changing research metrics landscape, launched on January 30, 2018..
The Metrics Toolkit includes 27 expert-written, time-saving summaries for the most popular research metrics including the Journal Impact Factor and Altmetric Attention Score.
Even better, the Metrics Toolkit includes an app that can recommend discipline-specific metrics to satisfy your specific use cases.
Best of all, the Metrics Toolkit carries a CC-BY license so you can reuse its content as you wish!

Selected References

New Journals devoted to altmetrics and scholarly publishing include:

Scholarly Assessment Reports, began in 2020 and is an Open Access, international, peer reviewed journal in the area of assessment of scholarly activities, including research, teaching and social service. The mission of this journal is to enhance among a wide scholarly and policy audience the knowledge on the potential and limits of scholarly assessment methodologies, in order to establish optimal conditions for an informed, responsible, effective and fair use of such methodologies and their metrics in actual scholarly assessment practices

The Journal of Altmetrics  launched in 2019 as an Open Access, international, peer reviewed journal in the areas of alternative metrics as they pertain to scholarly, social media and science communications as well as research policy and higher education. 

Frontiers in Research Metrics, launched in 2016 and publishes rigorously peer-reviewed research on the development, applications, and evaluation of scholarly metrics, including bibliometric, scientometric, informetric, and altmetric studies.

Several recent journal issues and articles have explored different metrics, and cover altmetrics in particular. 

 New Study Identifies Half-Life of Journal Articles ( LJ, January 2014)

Abbott, A., et al. (2010).  Do metrics matter? Nature, 465(7300), 860-862.   

Altmetrics Bibliography, version 1:10/14/2013 (includes content from January 2001-September 2013)

Armbruster, C. (2010). Whose metrics? citation, usage and access metrics as scholarly information service. Learned Publishing, 23(1), 33-38. doi:10.1087/20100107

Chen, C., Ibekwe-SanJuan, F., Hou, J. (2010). The structure and dynamics of cocitation clusters: A multiple perspective cocitation analysis. Journal of the American Society for Information Science and Technology 61(7): 1386-1409. doi:10.1002/asi.21309
Communication in Science: Pressure and Predators (2013), Science, Special Issue, October 4.
Egghe, L. (2006) Theory and practice of the g-index, Scientometrics, 69 (1): 131–152. doi:10.1007/s11192-006-0144-7
Emmanuel, Jenny (2013) Users and Citation Management Tools: Use and Support, Reference Services Review, 44(4):

Garfield, E. (2007).  The evolution of the Science Citation Index.  International Microbiology, 10(1), 65-69.

Gordon, G. J. (2010). What we don't know we don't know. Against the Grain, 22(4), 18-20.

Hirsch, J. E. (2005). An index to quantify an individual's scientific research output, PNAS 102 (46): 16569–16572. doi:10.1073/pnas.0507655102
Howard, J. (2013). Rise of altmetrics revives questions about how to measure impact of research.  Chronicle of Higher Education, June 3.

Jacs, P. (2010). Eigenfactor and article influence scores in the journal citation reports. Online Information Review, 34(2), 339-348.

Kraker, P., Körner, C., Jack, K., & Granitzer, M. (2012). Harnessing user library statistics for research evaluation and knowledge domain visualization. Proceedings of the 21st international conference companion on World Wide Web WWW 12 Companion (p. 1017). ACM. doi: 10.1145/2187980.2188236

Larsen, P. O., & Von Ins, M. (2010). The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics, 84(3), 575-603. doi: 10.1007/s11192-010-0202-z
Lopez-Cozar, E.D., Robinson-Garcia, N., Torres-Salinas, D. (2013). Manipulating Google Scholar Citations and Google Scholar Metrics: Simple, easy and tempting. (this version, v2)
Lapinski, S., Piwowar, H., and Priem, J. (2013). Riding the crest of the altmetrics wave: How librarians can help prepare faculty for the next generation of research impact metrics.  College & Research Libraries News 74(6), 292-300. 
Piwowar, H., ed.  (2013) Special issue devoted to Altmetrics. Bulletin of the American Society for Information Science and Technology, 39 (4), April.
Piwowar, H. (2013). Altmetrics: Value all research products.  Nature 493 (7431) 159.  doi:10.1038/493159a
Priem, P., Taraborelli, D., Groth, P. and Neylon, C. (2010) Alt-metrics Manifesto.
Priem, J. (2013). Scholarship: Beyond the paper.  Nature, 495 (7442). doi:10.1038/495437a

Priem, J., & Hemminger B.M. (2010).  Scientometrics 2.0: Toward new metrics of scholarly impact on the social web. First Monday, 15(7), article 2.
Roemer, R. C. & Borchardt, R. (2012). From bibliometrics to altmetrics: A changing scholarly landscape. College and Research Libraries News, 73(10), 596-600.
Rovner, S. L. (2008). The import of impact: new types of journal metrics grow more influential in the scientific community.  Chemical & Engineering News, 86(20), 39-42.

Schreiber, M. (2008).  An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 53(9), 1513-1522.
Shaikh-Lesko, R. (2014). Self Citation Gender Gap.  The Scientist, March 18. 

Taraborelli, D. (2008). Soft peer review. Social software and distributed scientific evaluation, Proceedings of the 8th International Conference on the Design of Cooperative Systems (COOP '08)

Tous, R., Guerrero, M., & Delgado, J. (2011). Semantic web for reliable citation analysis in scholarly publishing. Information Technology and Libraries, 30(1), 24-33.

Van Leeuwen, T. (2008).  Testing the validity of the Hersch-index for research assessment purposes.  Research Evaluation, 17(2), 157-160.

Van Noorden, R. (2010).  Metrics: a profusion of measures.  Nature, 465(7300), 864-866.