Evaluation of Research
Submitted by Francoise Genova
The evolution of the evaluation of research from being mostly bibliographic index-based is now recognized to be indispensable to enable Open Science. In the same way that Open Science promotes the open sharing of FAIR data and other research outputs, evaluators must also value and consider these outputs as part of the wider research evaluation framework. The RDA is an appropriate forum for addressing the evolving requirements of research evaluation as they relate to Open Science practices. RDA’s reach across the international community, including research organisations and funders, can help to facilitate and align a global discussion of research evaluation, drawing on data experts from a range of domains.
A successful BoF on Evaluation of Research was held during P19 (web page, meeting session notes). It demonstrated that there is significant interest in the community to organise an activity on the topic in the RDA, under the form of an Interest Group at this stage. A number of people registered during the BoF or later to help in organising the IG or to indicate that they are interested to participate in its activities. The IG charter is being prepared. It should be submitted before P20. If the IG is accepted before the Plenary, this session will be its first Plenary session.
The BoF objectives are:
-
To discuss activities to be performed by the IG, or their status if the IG has been created;
-
To liaise with relevant RDA Groups;
-
To share information about relevant initiatives.
Collaborative session notes: https://docs.google.com/document/d/11hu6NPiaGF01b-7axO1WWr4Vwse6C8s-um2E...
The meeting agenda will include:
-
Presentation and discussion of the activities proposed for the IG (30 min);
-
Presentation of the Coalition for Advancing Research Assessment (CoARA) - Eva Mendéz (20 min);
-
Panel of representatives of relevant RDA Groups to discuss IG activities and possible liaison (20 min). The Groups could include in particular Early Career and Engagement IG, Research Funders and Stakeholders on Open Research and Data Management Policies and Practices IG, Sharing Rewards and Credit (SHARC) IG, Software Source Code IG;
-
Conclusions on future IG activities (20 min).
The meeting will be organised as a working meeting and participants will be invited to provide their questions and comments during each sequence. Written feedback and input will be gathered in the collaborative session notes on the following topics: IG efforts; relevant initiatives; relevant RDA Bodies and Groups; IG outcomes; proposed activities; usefulness of a document gathering relevant links and eventually stories. The collaborative note document is organised accordingly.
The draft IG charter is also available for comment.
The necessary evolution of the evaluation of research has been discussed for years now in the international context - one can of course cite DORA), with also activities on how to enable it in regional, national and organisational contexts. It was a major topic of the Paris Open Science European Conference (OSEC 2022) organised in February 2022 in the context of the French Presidency of the European Union with a strong European and international participation, including speakers from the European Commission, South America, UNESCO and the United States. The Paris Call on Research Assessment was presented during the OSEC 2022 Conference. It “calls for the creation of a coalition of research funding organisations, research performing organisations, and assessment authorities, willing and committed to reform the current research assessment system along commonly agreed objectives, principles and actions (such as mutual learning, shared documentation and commonly agreed monitoring effort). The success of such a coalition will be deeply connected to its capacity to propose concrete implementation processes and to its capacity to associate and involve researchers at all levels.” More recently the Coalition for Advancing Research Assessment (CoARA) was created with the vision that “the assessment of research, researchers and research organisations recognises the diverse outputs, practices and activities that maximise the quality and impact of research. This requires basing assessment primarily on qualitative judgement, for which peer review is central, supported by responsible use of quantitative indicators.”
Organisations are invited to sign an Agreement on Reforming Research Assessment which sets a shared direction for changes in assessment practices for research, researchers and research performing organisations, with the overarching goal to maximise the quality and impact of research.
- 368 reads