Responsible use of research metrics

Assessment of the quality and impact of research is part of the research activity lifecycle. The University recognises the importance of assessing research responsibly and the value of peer review and expert judgment as a central component in the assessment of research outputs and wider research performance. However, we recognise that quantitative metrics can have a supporting role in research assessment.

Aligned with our university values and research culture, which aims to nurture the highest quality research, the responsible use of metrics principles recognise and value the intrinsic merit of research over and above the publication channel. All contributions should be recognised, and new knowledge disseminated as openly as possible, maximising the potential for impact.

In January 2019 the University of Nottingham UK became a proud signatory of the San Francisco Declaration on Research Assessment (DORA). A set of principles for the responsible use of metrics were approved by Research Committee in September 2019. In line with our commitment to review the principles, a revised version covering the broader remit of research assessment, and incorporating recommendations from HEFCE commissioned report, Harnessing the Metric Tide, has been approved by the University Open Research Group [10 Oct 2024]. The revised principles, listed below, are designed to encapsulate current good practice and to act as a guide for future activities.

Research metrics guide DORA indicators guidance Email the Research Support team Reporting issues and concerns

UoN UK Guiding Principles for Responsible use of Metrics in Research Assessment

1. Use Expert Judgement

Research Assessment is based upon expert judgement and peer review. Quantitative measures can be used to support or inform judgements but should not be used to replace evaluation and critique provided through expert peer opinion.

2. Be Transparent

Be clear what the rationale for using particular metrics is and how they will be used.
Make data collection and analytical processes transparent as to whether, and how metrics will be used, including their calculation.
Give researchers the opportunity to verify their data. This may involve releasing the data underpinning decisions, identifying data sources and metrics which will be used or asking researchers to source and submit data for review themselves.

3. Be Contextual

The use of metrics should be appropriate to the research assessment being conducted and will vary according to the context e.g. institution-level metrics may be inappropriate to use at an individual-level. It should be clear from the outset of any research evaluation which metrics are being tracked and why.
Value the intrinsic merit of the research output over the reputation of the publisher, the publication channel or associated metrics. This is a particularly important consideration if quantitative metrics are used as part of recruitment and promotion processes.

4. Be Fair

Respect diversity of disciplines, career stages, employment pattern, gender and minoritised groups.
Recognise that within scholarly communication (including peer review), systemic publication and citation biases exist which can disadvantage researchers from specific groups and consider ways to mitigate this.
Variations between disciplines should be acknowledged. Use normalised or field-weighted citation data where possible and consider whether comparisons are meaningful.
Recognise there are a wide range of research outputs beyond journal articles, including monographs, conference papers, research data, preprints, working papers, protocols, and software.
Recognise the range of contributions to 'team research'
Recognise that some limitations in coverage of sources of publication data exists, e.g. unindexed publication venues and outputs written in languages other than English.

5. Be Robust

All data should be accurate, and of an appropriate scale and scope, ensuring what is valued is measured rather than what is easy to measure.
Choose a combination of metrics, this will ensure any weaknesses in one particular measure does not skew the data.
Ensure the metrics and any associated evaluation methods are reviewed and updated regularly, recognising any unintended systemic effects of assessment and update to encourage appropriate behaviours.

University of Nottingham, Authored by UoN Libraries, Approved by the University Open Research Group (10 Oct 2024)

Reporting issues and concerns 

If you have any concerns regarding research metrics at the University of Nottingham UK not being used in accordance with the guiding principles there are a number of routes you can follow, please choose the most appropriate for the situation.

For general guidance on responsible metrics email the Libraries Research Support team. For specific issues, raise the concern with members of staff involved or contact the relevant School’s Director of Research and Knowledge Exchange.

For guidance on reporting serious breaches of the policy please consult the University of Nottingham Whistleblowing Code and Complaints Procedures.