The evolution of the evaluation of research from being mostly bibliographic index-based is now recognized to be indispensable to enable Open Science. In the same way that Open Science promotes the open sharing of FAIR data and other research outputs, evaluators must also value and consider these outputs as part of the wider research evaluation framework.  The RDA is an appropriate forum for addressing the evolving requirements of research evaluation as they relate to Open Science practices, and in particular to addressing challenges around the evaluation of datasets as valuable assets in their own right and the recognition of participation in the definition and maintenance of disciplinary or generic data sharing frameworks and of data services, as well as more generally of team work. RDA’s reach across the international community, including research organisations and funders, can help to facilitate and align a global discussion of research evaluation, drawing on data experts from a range of domains.

The BoF objectives are

• To hold a first discussion about the subject in the RDA;

• To assess whether there is an interest in the RDA community to develop activities in the domain;

• To identify useful activities which could be tackled by the RDA in the domain, such as a concrete list of existing initiatives on the subject, and create a realistic working plan on how RDA could contribute on the evaluation aspects, particularly regarding data.

The meeting is co-organised by Francoise Genova (RAB co-chair), Ari Asmi (RAB), Natalie Harrower, Rosie Hicks (RAB), Rebecca Koskela (RAB co-chair), Shelley Stall (OAB co-chair) 

Collaborative session notes: https://docs.google.com/document/d/1VQvDSyrnPnXJg9e5Iowtak_O9tXQIwy1AB93…

The meeting agenda will include:

• Short presentations by different stakeholders. Confirmed speakers

  • Javier Lopez Albacete (European Commission)

  • Serge Bauin, Centre National de la Recherche Scientifique (France)

  • Carlos Montero Corrales (National University of Costa Rica) – Evaluation of results of Veterinary Medicine School of the National University of Costa Rica

  • Claudio Monge Hernandez (Costa Rica) – The challenges of the new scientific evaluation in an open science world: a latin american perspective.

     

• A short presentation of the outcome of a global questionnaire survey of the scholarly communication attitudes and behaviours of early career researchers (Harbingers), presented by Chérifa Boukacem-Zeghmouri

• Time reserved for discussion of the possible follow-up RDA activities.

Participants will be invited to gather information about relevant documents and web sites and to post their input in a shared document Evaluation of Research – Stories and Links. The document is available and input can also be provided before and after the BoF session.

The necessary evolution of the evaluation of research has been discussed for years now in the international context – one can of course cite DORA), with also activities on how to enable it in regional, national and organisational contexts. It was recently a major topic of the Paris Open Science European Conference (OSEC 2022) organised in the context of the French Presidency of the European Union with a strong European and international participation, including speakers from the European Commission, South America, UNESCO and the United States. The meeting will be an occasion to gather more information from the participants about the activities they are aware of as well as to discuss possible follow-up activities in the RDA.

The RDA Data Usage Metrics Working Group produced a recommendation which can be useful in this context.

One aim of the meeting is to gather more relevant links. The initial list shown below is now included in the shared document Evaluation of Research – Stories and Links with additional material, which can be completed by the participants or any person willing to share information on the subject. The document is organised into the following sections: International and regional endeavours; National endeavours; Organisational endeavours; Relevant meetings; Studies and documents; Alternative incentives; Relevant RDA Groups; Any other relevant material.

DORA – Declaration on Research Assessment

UNESCO Recommendation on Open Science

Paris Open Science Conference

Paris Call on Research Assessment

Towards a reform of the research assessment system (Scoping paper, European Commission)

Process towards an agreement on reforming research assessment (European Commission)

Science Europe Research Assessment Priority Science Europe Position Statement and Recommendations on Research Assessment Processes

Open Science in university approaches to academic assessment – Follow-up to the 2020-21 EUA (European University Association) Open Science survey

Turning FAIR into Reality, Final report and action plan from the European Expert Group on FAIR data

How Do We Make Research Assessment More Responsible? – A Multi-stakeholder Discussion, Phil Jones, The Scholarly Kitchen

RDA Data Usage Metrics WG Recommendation

Harbingers survey