Interest Group Supporting Output July 3, 2024

RDA Value for the Evaluation of Research

Output Type: Interest Group Supporting Output
Output Status: Endorsed
Review Period End: 2024-08-03
DOI: 10.15497/RDA00118

Primary Domain: Domain Agnostic
RDA Pathways: Other
Group Technology Focus: Policy-Related
Stakeholders: Funders & Policy makers, Infrastructure Providers, Libraries, Research Performing Organisations, Researchers & Scientists
Sustainable Development Goals: Gender Equality, Industry, Innovation and Infrastructure
Language: English

Abstract

The evaluation of research is evolving from being mostly bibliographic index-based to a broader context, which is now recognised as indispensable for enabling Open Research. In the same way that Open Research promotes the open sharing of FAIR data and other research outputs, evaluators must also value and consider these outputs as part of the wider research evaluation framework. The RDA is an appropriate forum for addressing the evolving requirements of research evaluation as they relate to Open Research practices. The RDA Interest Group “Evaluation of Research” organises the discussion on the subject in RDA, particularly as it relates to data and RDA activities on research software.

The document describes the RDA value for the evaluation of research. The topic is perfectly aligned with its mission. RDA’s reach across the international community, including research organisations and funders, and the diverse profiles of its individual members, can help to facilitate and align a global discussion of research evaluation, drawing on data experts from a range of domains. Liaison are built with international initiatives tackling the topic, in particular the Coalition for Advancement of Research Assessment. A number of RDA Groups have activities relevant to the many changes surrounding the evaluation of research. Building on the diversity of the RDA
community and activities, possible RDA contributions to the debate fall in several categories: possible metrics and criteria, stakeholder representation, research outputs and their relevant properties, disciplinary aspects. The groups dealing with support activities contribute to the ecosystem’s progression by providing discussions and outputs that help researchers to engage in and with the transforming context.

Impact Statement

RDA’s contribution to the international effort aimed at the evolution of research evaluation is potentially among the important RDA’s contribution to Open Research policies. The document is proposed to be included among in the collection of documents describing the value of RDA to different stakeholders and endeavours.

Citations

Genova, F., Crott, E., Madalli, D., Nurnberger, A., & RDA Evaluation of Research IG. (2024). RDA Value for the Evaluation of Research (Version 1.0). Research Data Alliance. https://doi.org/10.15497/RDA/00118

Files

Name Date Added

RDA-Value-for-the-Evaluation-of-Research

July 3, 2024 By Bridget Walker

Download
RDA Value for the Evaluation of Research V1.1 COMPLETE

October 31, 2024 By Bridget Walker

Download

Comments

August 5, 2024 at 4:11 pm

Julien Colomb says:

Hello,

I have difficulties understanding what this output is about, it looks like a charter for a new group to me. Is it one ? Is it a call for more interaction in the RDA on this topic? Or is it a claim that RDA is the good place to have these conversation (in this case, one could consider alternatives ?)
I may just be lost in the "new" RDA website ?

Could you try to clarify what your objectives are.

PS: Does it relate to "Sharing Rewards and Credit (SHARC) IG" and their output in review ?

Reply
- August 6, 2024 at 6:36 am
  
  Francoise Genova says:
  
  The RDA has activities in a number of domain. Evaluation of Research is a policy topic, and many initiatives are tackling it at the international, continental, national and local level. The output aim is to have an available statement about the possible RDA role, which can be shared with the policy level (e.g. Ministries interested in the topic), possible international partners, and the RDA community, the members of which are often deeply affected by the way evaluation of research is currently performed. It is also useful to inform the community members that their own activities in the RDA can be of interest in the evaluation of research context. When the output is endorsed, it could be posted in the collection of 'RDA Value for...' web pages, which are linked from the RDA web page.
  
  Reply
August 5, 2024 at 1:11 pm

Veronika Stoka says:

In this context, it is worth mentioning and maybe timely, to include the recent RDA and GA4GH agreement to strategic relationship to advance
responsible data sharing as stated in the "Memorandum of Understanding between the Research Data Alliance (RDA) and the Global Alliance for Genomics and Health (GA4GH)".
I would be glad to contribute to these initiatives of common interest to both organizations (currently ongoing), thus optimizing our efforts by preventing work duplication.

Reply
- August 6, 2024 at 6:26 am
  
  Francoise Genova says:
  
  Thanks, Veronika. It is good to build a relationship with GA4GH on the topic. We will get in touch with you to discuss how this can be built, and we will cite the possible liaison in the output. Is there a link to the GA4GH activities in the domain which could be cited?
  
  Reply
August 4, 2024 at 1:11 pm

Francoise Genova says:

Thanks for your comments. We will check how to mentions AIDV WG. Re. your second comment, it is good indeed to be inclusive but the idea of this text is not to make a catalogue of the relevant international initiatives. At present liaison is established with CoARA. The output can be clarified on that point and updated to reflect the evolutions to come in this domain. Thanks for your proposal to contribute, we can discuss the initiatives you would suggest to liaise with.

Reply
July 23, 2024 at 4:13 pm

Francis P. Crawley says:

This is an interesting statement. It makes the important link between the development of open science and the need to reconsider how we do research evaluation. Perhaps it is appropriate to include a reference to the 'EOSC-Future/RDA Working Group on Artificial Intelligence & Data Visitation (AIDV)' in the paragraph statement's paragraph: 'The most straightforward connections are related to possible metrics and criteria. Several
Groups have been active in the domain, in particular the Data Usage Metrics WG and the
Sharing Rewards and Credit (SHARC) IG. The IG Understanding and Capturing the Usage of
Digital Research Infrastructure, currently under discussion, will also have the capacity to assess
possible new criteria if it gathers enough support to be endorsed'. This group has developed important guidance in this area and partially gave rise to the CoARA-ERIP Working Group.
It may also be good to indicate other groups in the world that are working on this beyond CoARA. We should strive for a more inclusive approach. I would be happy to contribute to this.

Reply
July 22, 2024 at 5:23 am

Anca Monica Marin says:

Thank you very much for the output and for the possibility of giving feedback. To me, one of the key aspects for advancing research is to promote more prominently its cross-disciplinary aspects and for doing this, standardized connectors are needed - like NUTS codes, or other international classifications, etc. Not sure whether this is the closest WG for this idea, but I thought one reference to Standardisation of Data Categories and Codes WG, might also be useful. thank you

Reply
- September 10, 2024 at 8:20 am
  
  Francoise Genova says:
  
  @Anca Monica Marin - Thank you again for your comment on the "RDA Value for the Evaluation of Research" document. Standardised connectors are indeed important facilitators, and we had identified Vocabularies with that in mind. As you suggest, it would make sense in that respect to add a reference to the Standardisation of Data Categories and Codes WG.
  
  While working on finalising the output taking the comments into account, we checked the WG page. It is now in the "Historical" category, with the indication that it was "Withdrawn", which happened in 2015. It seems that they did not produce any output which could be relevant.
  
  In the current situation as displayed in the public WG history, we have the feeling that it would be better not to cite it explicitly in our output. The topic remains really interesting, do you have any indication that it might come back in the RDA?
  
  Reply
- August 4, 2024 at 1:04 pm
  
  Francoise Genova says:
  
  Thanks, we will take your comment into account and check the RDA Groups for relevance.
  
  Reply

You must be logged in or join the group to leave a comment.