Interest Group Output July 3, 2024

RDA Value for the Evaluation of Research

  • Primary Domain: Domain Agnostic
  • RDA Pathways: Other
  • Group Technology Focus: Policy-Related
  • Stakeholders: Funders & Policy makers, Infrastructures, Libraries, Research Performing Organisations, Researchers & Scientists
  • Sustainable Development Goals: Gender Equality, Industry, Innovation and Infrastructure
  • Language: English

Abstract

The evaluation of research is evolving from being mostly bibliographic index-based to a broader context, which is now recognised as indispensable for enabling Open Research. In the same way that Open Research promotes the open sharing of FAIR data and other research outputs, evaluators must also value and consider these outputs as part of the wider research evaluation framework. The RDA is an appropriate forum for addressing the evolving requirements of research evaluation as they relate to Open Research practices. The RDA Interest Group “Evaluation of Research” organises the discussion on the subject in RDA, particularly as it relates to data and RDA activities on research software.

The document describes the RDA value for the evaluation of research. The topic is perfectly aligned with its mission. RDA’s reach across the international community, including research organisations and funders, and the diverse profiles of its individual members, can help to facilitate and align a global discussion of research evaluation, drawing on data experts from a range of domains. Liaison are built with international initiatives tackling the topic, in particular the Coalition for Advancement of Research Assessment. A number of RDA Groups have activities relevant to the many changes surrounding the evaluation of research. Building on the diversity of the RDA
community and activities, possible RDA contributions to the debate fall in several categories: possible metrics and criteria, stakeholder representation, research outputs and their relevant properties, disciplinary aspects. The groups dealing with support activities contribute to the ecosystem’s progression by providing discussions and outputs that help researchers to engage in and with the transforming context.

 

 

Impact Statement

RDA’s contribution to the international effort aimed at the evolution of research evaluation is potentially among the important RDA’s contribution to Open Research policies. The document is proposed to be included among in the collection of documents describing the value of RDA to different stakeholders and endeavours.

Citations

Genova, F., Crott, E., Madalli, D., Nurnberger, A., & RDA Evaluation of Research IG. (2024). RDA Value for the Evaluation of Research (Version 1.0). Research Data Alliance. https://doi.org/10.15497/RDA/00118

Comments

  • Profile Picture

    August 5, 2024 at 1:26 pm

    Julien Colomb says:

    Hello, I have difficulties understanding what this output is about, it looks like a charter for a new group to me. Is it one ? Is it a call for more interaction in the RDA on this topic? Or is it a claim that RDA is the good place to have these conversation (in this case, one could consider alternatives ?) I may just be lost in the "new" RDA website ? Could you try to clarify what your objectives are. PS: Does it relate to "Sharing Rewards and Credit (SHARC) IG" and their output in review ?

    • Profile Picture

      August 6, 2024 at 1:26 pm

      Francoise Genova says:

      The RDA has activities in a number of domain. Evaluation of Research is a policy topic, and many initiatives are tackling it at the international, continental, national and local level. The output aim is to have an available statement about the possible RDA role, which can be shared with the policy level (e.g. Ministries interested in the topic), possible international partners, and the RDA community, the members of which are often deeply affected by the way evaluation of research is currently performed. It is also useful to inform the community members that their own activities in the RDA can be of interest in the evaluation of research context. When the output is endorsed, it could be posted in the collection of 'RDA Value for...' web pages, which are linked from the RDA web page.

  • Profile Picture

    August 5, 2024 at 1:26 pm

    Veronika Stoka says:

    In this context, it is worth mentioning and maybe timely, to include the recent RDA and GA4GH agreement to strategic relationship to advance responsible data sharing as stated in the "Memorandum of Understanding between the Research Data Alliance (RDA) and the Global Alliance for Genomics and Health (GA4GH)". I would be glad to contribute to these initiatives of common interest to both organizations (currently ongoing), thus optimizing our efforts by preventing work duplication.

    • Profile Picture

      August 6, 2024 at 1:26 pm

      Francoise Genova says:

      Thanks, Veronika. It is good to build a relationship with GA4GH on the topic. We will get in touch with you to discuss how this can be built, and we will cite the possible liaison in the output. Is there a link to the GA4GH activities in the domain which could be cited?

  • Profile Picture

    August 4, 2024 at 1:26 pm

    Francoise Genova says:

    Thanks for your comments. We will check how to mentions AIDV WG. Re. your second comment, it is good indeed to be inclusive but the idea of this text is not to make a catalogue of the relevant international initiatives. At present liaison is established with CoARA. The output can be clarified on that point and updated to reflect the evolutions to come in this domain. Thanks for your proposal to contribute, we can discuss the initiatives you would suggest to liaise with.

  • Profile Picture

    July 23, 2024 at 1:26 pm

    Francis P. Crawley says:

    This is an interesting statement. It makes the important link between the development of open science and the need to reconsider how we do research evaluation. Perhaps it is appropriate to include a reference to the 'EOSC-Future/RDA Working Group on Artificial Intelligence & Data Visitation (AIDV)' in the paragraph statement's paragraph: 'The most straightforward connections are related to possible metrics and criteria. Several Groups have been active in the domain, in particular the Data Usage Metrics WG and the Sharing Rewards and Credit (SHARC) IG. The IG Understanding and Capturing the Usage of Digital Research Infrastructure, currently under discussion, will also have the capacity to assess possible new criteria if it gathers enough support to be endorsed'. This group has developed important guidance in this area and partially gave rise to the CoARA-ERIP Working Group. It may also be good to indicate other groups in the world that are working on this beyond CoARA. We should strive for a more inclusive approach. I would be happy to contribute to this.

  • Profile Picture

    July 22, 2024 at 1:26 pm

    Anca Monica Marin says:

    Thank you very much for the output and for the possibility of giving feedback. To me, one of the key aspects for advancing research is to promote more prominently its cross-disciplinary aspects and for doing this, standardized connectors are needed - like NUTS codes, or other international classifications, etc. Not sure whether this is the closest WG for this idea, but I thought one reference to Standardisation of Data Categories and Codes WG, might also be useful. thank you

    • Profile Picture

      September 10, 2024 at 1:26 pm

      Francoise Genova says:

      @Anca Monica Marin - Thank you again for your comment on the "RDA Value for the Evaluation of Research" document. Standardised connectors are indeed important facilitators, and we had identified Vocabularies with that in mind. As you suggest, it would make sense in that respect to add a reference to the Standardisation of Data Categories and Codes WG. While working on finalising the output taking the comments into account, we checked the WG page. It is now in the "Historical" category, with the indication that it was "Withdrawn", which happened in 2015. It seems that they did not produce any output which could be relevant. In the current situation as displayed in the public WG history, we have the feeling that it would be better not to cite it explicitly in our output. The topic remains really interesting, do you have any indication that it might come back in the RDA?

    • Profile Picture

      August 4, 2024 at 1:26 pm

      Francoise Genova says:

      Thanks, we will take your comment into account and check the RDA Groups for relevance.

You must be logged in or join the group to leave a comment.