What are research metrics?
Research metrics, sometimes known as indicators, are defined by Research England as “quantitative measurements designed to evaluate research outputs and their impacts.” Many of these metrics are based on statistical analysis of citation data of traditional publications such as books and journal articles. However, more recently “alternative metrics” have been developed to capture data such as page views, downloads and social media mentions.
For any metric you use it is essential to understand the source of the data and any limitations, for example citation data is generated from academic databases and will not include references to research from sources which are not indexed. There are differences in the applicability of specific metrics between scholarly disciplines, for example journal ranking, impact factor and citation indicators are not well-developed in arts and humanities disciplines.
When choosing to use research metrics, please remember to consider the limitations of bibliometrics and use them responsibly.
An online resource called the Metrics Toolkit explains how different metrics are calculated, what their limitations are and provides appropriate and inappropriate use cases.
Publication metrics
Use to find out how often a publication, or a group of publications, has been cited by others. The number of times a particular article, book or other research output has been cited can be an indication of the impact of that output on others' work. Remember that citation practices vary across the different disciplines.
The key tools to obtain citation information are listed below. In each source, search for your target output to view a list of the papers that cited it.
Publication metrics - tools
| Scopus |
You can access Scopus through NUsearch databases. Look for the 'Cited by' number on the right-hand side of each record: click to see the citing papers. |
|---|
| Web of Science |
You can access Web of Science through NUsearch databases. Look for 'Times cited' alongside each record and click the number to see the citing papers. |
|---|
| Google Scholar |
Look for the 'Cited by' link underneath the record (first link on the left). |
|---|
You will find the figure is different depending on the source you use. Each source contains a different set of journal citation data and therefore draws upon a different list of citations. The figure is also likely to increase over time as more citations are received.
If you are looking for metrics related to books or monographs, see the section below.
Author metrics
Use to explore the impact of an author, or a group of authors, based on the citation rates of their publications. How the author’s name is denoted in a database can vary so it’s important to search for all the name variants.
A number of metrics exist to assess the number of citations for a given author. Often the Hirsch Index (h-index) is shown: for more information on the h-index, and its limitations, see the Metrics Toolkit.
Databases used to obtain any metrics have their own boundaries or limitations, which is important to remember when working with data about an individual or group of individuals. For example, Scopus and Web of Science each have their own set of selection criteria applied by in-house academic editors, however these criteria are sensitive to biases and exclude some good quality scholarly documents and/or activity. The citations counted within these databases, and their associated analytical tools, enable deep and powerful analyses of an academic environment that is closely curated and managed under the ownership of commercial entities, Elsevier and Clarivate.
Google Scholar follows a more inclusive but less supervised approach by crawling the open web for scholarly documents, and aggregating the citations into Google Scholar Profiles. The results are a more diverse coverage in all aspects, but can lead to more errors, duplicates and fewer tools to normalize highly cited papers or discount self-citations.
Further reading on this topic: Google Scholar, Web of Science, and Scopus: Which is best for me? (LSE blog)
If you are the author, make sure your author profile on Scopus and/or Web of Science is correct so that your papers are identified as yours. Sign up to and use your ORCID iD across different platforms to allow publishers and aggregators of scholarly literature to distinguish you from researchers with similar names.
Journal metrics
Use to assess the impact of a journal and to compare journals. Remember to consider the audience you are trying to reach and use journal metrics to aid the decision rather than be the driver. Note: Journal-level metrics should not be used as a proxy for the quality of individual papers in the journal. Journal metrics are reviewed every year and be aware that journal metrics values can go up and down.
Journal metrics
| Source Normalised Impact per Paper (SNIP) |
The SNIP value measures the average citation impact of the publications of a journal and corrects for the differences in citation patterns between subject areas. This makes comparisons between disciplines easier. The SNIP value is calculated by the Leiden University's Centre for Science and Technology Studies (CWTS) and is based on Scopus data. Available on the CWTS website or via Scopus. |
|---|
| Scimago Journal Rank (SJR) |
Measures the average number of weighted citations received in a year, by articles published in a journal in the previous 3 years. Normalisation occurs so can journals can be compared across disciplines and a prestige factor is applied. The data is provided by Scopus. Available via a public portal or via Scopus. |
|---|
| Journal Impact Factor (JIF) |
Measures the frequency with which published papers are cited up to two years after publication. It uses citation data from the Web of Science database, with full JIF information available in Journal Citation Reports. You will often see this displayed on journal webpages. Limitations: No normalisation occurs so take care when comparing journals across different disciplines. |
|---|
Altmetrics
Altmetrics also known as alternative metrics, are measures that capture the attention an output generates on social media, websites, news resources and preprint servers. Attention includes tweets, mentions, shares, recommendations, webpage views, PDF downloads, mass media mentions and data and code usage.
Article-level metrics applied to research papers can help researchers understand how their research is being shared or discussed online. The attention is immediate and goes beyond academia. Altmetrics can be used to complement traditional metrics.
Many publishers now present altmetric data within their own websites, where you will see the Altmetric badges or 'donuts' alongside the article. Key tools showing altmetric information are listed below (note this list is not exhaustive).
The University of Nottingham has access to the Altmetric Explorer for Institutions web platform on a 1-year trial, ending July 2024. To get started visit the Altmetric Explorer website, and register using your institutional email address. An introductory video is available to learn how to navigate and search the database.
To change the homepage view to show UoN data, select the Edit Search button, then Account Settings, change the Default Search Scope to “My institution only”, followed by the Change Search Scope button.
To find out more about which sources are tracked and how the score is calculated, see the Altmetric Explorer solutions page.
If you have any questions about altmetrics, please contact the Library's Research Support team.
Remember: attention does not necessarily indicate the article is important or give any indication of quality.
Book and monograph metrics
Locating evidence showing the impact for books, monographs and book chapters is more difficult than traditional journal articles. Some recommended resources are listed below.
Book and monograph metrics - resources
| Google Scholar |
Google Scholar gives citation counts of book titles and book chapters with link to citing works. Google Scholar is a particulary helpful tool for book-based disciplines (social sciences, arts and humanities). |
|---|
| Google Books |
Google Books provides book reviews with a breakdown by user rating, star review ranking, and site review name. |
|---|
| Scopus |
Accessible through NUsearch databases, Scopus is one of the major multi-disciplinary citation databases. It contains 230,000 books, including monographs and edited volumes, major reference works and graduate-level textbooks. Authors of books and book chapters can obtain citation counts for their works. |
|---|
| Book Citation Index (Web of Science) |
The Book Citation Index (Web of Science) comprises scholarly books in series and non-series format, including textbooks, published dissertations, translations of non-English titles, and book chapters. The Book Citation Index gives the number of citation counts when a book or book chapter has been cited in other works. Currently there are 116,000 books indexed, with 10,000 new books added each year. |
|---|
Other sources to check are Amazon and WorldCat.
In addition to citations, other indicators of the reception of a book or book chapter may include:
- use as a university textbook;
- listed on 'bestseller' lists;
- awards and honours received; or
- number of copies sold, editions and translations into other languages.
Institution and group metrics
World University Rankings
A number of international rankings are freely available, including:
World university rankings
| QS World University Rankings |
The QS World University Rankings rank more than 900 universities from over 80 countries by assessing university performance across four areas: research; teaching; employability; and internationalisation. Based on 'hard data', data from Scopus and from surveys. |
|---|
| THE World University Rankings |
Produced by Times Higher Education and based on Scopus and survey data, the THE World University Rankings rank research-intensive universities across 'their core missions: teaching, research, knowledge transfer and international outlook.' |
|---|
| Academic Ranking of World Universities (ARWU) |
The ARWU is published by ShanghaiRanking Consultancy, using Web of Science data plus other criteria, for example winners of Nobel Prizes and Fields Medals. More than 1,200 universities are ranked each year and the best 500 are published. |
|---|
SciVal
SciVal is a tool which allows users to view and analyse the publication and citation data for individual articles and authors. It uses data from the Scopus database. Scopus citation and publication data underpins the Times Higher Education and QS World University Rankings.
SciVal can be used to benchmark research groups, schools and institutions using a variety of different indicators, including outputs in the top percentiles. All members of the University have access to SciVal via NUsearch databases.
It is essential that all users of SciVal use the data and tools it provides appropriately and in line with University policy.
For further information and support on Scival, please refer to the following resources: