Interest Group Output June 25, 2024

Recommendations on Open Science Rewards and Incentives: Guidance for multiple stakeholders in Research.

  • Primary Domain: Domain Agnostic
  • RDA Pathways: Data Lifecycles - Versioning, Provenance, Citation, and Reward
  • Group Technology Focus: Policy-Related
  • Stakeholders: Funders & Policy makers, Research Performing Organisations, Researchers & Scientists
  • Sustainable Development Goals: Partnerships to achieve the Goal, Reduced Inequality
  • Language: English

Abstract

Open Science contributes to the collective building of scientific knowledge and societal progress. However, academic research currently fails to recognise and reward efforts to share research outputs. Yet it is crucial that such activities be valued, as they require considerable time, energy, and expertise to make scientific outputs usable by others, as stated by the FAIR principles. To address this challenge, several bottom-up and top-down initiatives have emerged to explore ways to assess and credit Open Science activities (e.g., Research Data Alliance, RDA) and to promote the assessment of a broad spectrum of research outputs, including datasets and software (e.g., Coalition for Advancement of Research Assessment, CoARA). As part of the RDA-SHARC (SHAring Rewards and Credit) interest group, we have developed a set of recommendations to help implement various rewarding schemes at different levels. The recommendations target a broad range of stakeholders. For instance, institutions are encouraged to provide digital services and infrastructure, organise training and cover expenses associated with making data available for the community. The funders should establish policies requiring open access to data produced by funded research and provide corresponding support. The publishers should favour open peer-review models and open access to articles, data and software. Government policymakers should set up a comprehensive Open Science strategy, as recommended by UNESCO and followed by a growing number of countries. The present work details different measures that are proposed to the stakeholders. The need to include sharing activities in research evaluation schemes as an overarching mechanism to promote Open Science practices is specifically emphasised.

RDA-SHARC Recommendations website: https://bienflorencia.github.io/rda-sharc-reco/

Impact Statement

RDA-SHARC’s recommendations are meant to be adopted by researchers and the various stakeholders of the research environment and evaluation system in order to enable rewarding schemes for open science activities. The objective is to fill the gaps between the current OS activities and the recognition system. The ultimate goal is to foster and facilitate open science practice in all scientific communities.

Explanation of Sustainable Development Goals

We believe that our research output emphasising the need to seamlessly integrate Open Science practices in research evaluation and rewarding schemes is particularly associated with two Sustainable Development Goals: Reduced Inequality and Partnerships for the Goals.

Reduced Inequality: Our recommendations aim to address the inequalities that are linked to current evaluation and reward systems, driven by narrow and inappropriate definitions of research excellence.

Partnerships for the Goals: Our recommendations encourage cooperation between multiple stakeholders to share knowledge, expertise and resources for implementing rewards and recognition for Open Science practices. These practices, when properly incentivised and widely adopted, have the potential to accelerate progress across a broader scope of Sustainable Development Goals.

Citations

Mabile, L., Shmagun, H., Erdmann, C., Cambon-Thomsen, A., Thomsen, M., Grattarola, F., & RDA Sharing Rewards and Credit (SHARC) IG. (2024). Recommendations on Open Science Rewards and Incentives: Guidance for multiple stakeholders in Research. Research Data Alliance. https://doi.org/10.15497/RDA/000117

 

 

Comments

  • Profile Picture

    September 16, 2024 at 1:26 pm

    Laurence Mabile says:

    The APA style was meant as an example, we’ll make this clearer in the table. The important thing is to use a style that is structured and includes information about what type of object is published, e.g. data, software, etc and not just defaulting to paper. This will enable proper credit and also help indexing services identify the object type.

  • Profile Picture

    July 26, 2024 at 1:26 pm

    Katie Eve says:

    We are unclear on the reason for the recommendation to Researchers to “Use american psychological association (apa) style”. Different disciplines, journals and authors use a range of reference styles. Flexibility should be afforded to accommodate diverse needs and preferences.

  • Profile Picture

    July 26, 2024 at 1:26 pm

    Katie Eve says:

    We suggest that recommendations to RPOs regarding characteristics of trusted repositories be expanded to reference the US National Institutes of Health (NIH) Generalist Repository Ecosystem Initiative (GREI), and the and the Desirable Characteristics for All Data Repositories, available here: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-016.html.

    • Profile Picture

      September 16, 2024 at 1:26 pm

      Laurence Mabile says:

      Thank you for your suggestion. We fully agree that the GREI and NIH references are helpful resources and will include them in the relevant table.

  • Profile Picture

    July 26, 2024 at 1:26 pm

    Katie Eve says:

    We support the recommendation to Funders to “For all research projects, systematically allocate a portion of the proposal budget to OS activities, such as data management and sharing.” This should be further expanded to also include OA as an example of an OS activity for which a proportion of proposal budget should be allocated.

    • Profile Picture

      September 16, 2024 at 1:26 pm

      Laurence Mabile says:

      We have preferentially recommended to allocate some funds to support OA while transitioning to diamond models. However, we do not recommend to allocate the proposal budget for APCs.

  • Profile Picture

    July 26, 2024 at 1:26 pm

    Katie Eve says:

    We are unclear on the basis for the recommendation to RPOs to transition to the diamond OA model. From our experience over the last few years, Read and Publish/ Transformative agreements are currently the most effective way of enabling immediate access to research cost effectively and at scale, and gold open access is the most sustainable. Transformative agreements supported the open access status of more than 300,000 publications globally in 2023 (see: https://www.stm-assoc.org/oa-dashboard/transformative-agreements/). At Elsevier we already have such agreements in place for over 2,000 institutions, and we are committed to working with institutions who wish to establish such agreements. As part of our commitment to meet the evolving needs of the research community, we will continue to analyse, monitor and experiment with different publication models. For example, we already publish over 130 diamond open access journals on behalf of institutions. However, before widescale adoption of such alternative models, questions need to be addressed, such as commitments to long-term funding, preservation policies, metadata collection/ tagging policies to support discoverability, and more. Importantly, diamond open access journals are typically regionally- or institutionally-focused with lower readership, usage and impact than established international journals. These factors make them a less realistic and attractive option as a publication venue for many researchers, who are typically most motivated by publishing in a journal which will afford their work the greatest reach and impact, irrespective of the publishing model.

    • Profile Picture

      September 16, 2024 at 1:26 pm

      Laurence Mabile says:

      Transformative agreements were necessary/intended as part of a transition process and not as permanent ones. Some recent studies have explicitly challenged the effectiveness and sustainability of transformative agreements [cOAlition S 'Transformative Journals: analysis from the 2023 reports'; 'How open are hybrid journals included in transformative agreements?']. Moreover, TAs are not included in the cOAlition S priorities anymore [https://zenodo.org/records/11243942; this is based on the results of a global multi-stakeholder consultation across various research communities]. Overall, our recommendations are guided by principles to benefit society in the long term. Thus, as we mentioned in other responses, we recommend the transition to the Diamond OA path.

  • Profile Picture

    July 26, 2024 at 1:26 pm

    Katie Eve says:

    We are concerned about the recommendation to RPOs to “Promote that non-oa (closed) outputs should not be reported for performance evaluation procedures”. Different subject disciplines have different priorities with respect to OA. Researchers from the Social Sciences and Humanities, Early Career Researchers, and women, are typically in receipt of smaller sized (or even no) funding grants. This can affect their ability to publish OA in their journal of choice. As such, this recommendation could have unintended consequences by excluding large groups from research evaluation exercises, which would perpetuate and exacerbate existing funding disparities.

    • Profile Picture

      September 16, 2024 at 1:26 pm

      Laurence Mabile says:

      We agree that we need to be more explicit about non-OA. By closed outputs we mean outputs only accessible over paywall. Those should not be considered in the evaluation process if the at least the accepted versions are not openly (i.e., freely) accessible elsewhere, such as in institutional archives. As for the disparities you are mentioning, they are first fundamentally the result of either dysfunctional research evaluation rules abusively focused on JIFs, or entrenched in social organisation & cultural habits. Accelerating knowledge dissemination about those topics is even more important. OS hold such boosting potential and promoting that non-oa (closed) outputs should not be reported for performance evaluation procedures is an efficient way to push for open science.The issues at stake should not be used as an excuse not to promote OS. Meanwhile, considering the so far unequal reality of research, implementation should indeed be done with caution and framed by principles of human equity, research integrity and responsible conduct.

  • Profile Picture

    July 24, 2024 at 1:26 pm

    Katie Eve says:

    Given publishers’ journals and submission platforms typically serve researchers globally, we need to provide flexibility in the identifiers we request to ensure we are supporting all authors to meet their local requirements/ recommendations, and avoid creating barriers for some researcher groups which would create inequities. Where authors so wish, we enable them to provide ORCiDs during the submission workflow and where they provide this we make it available alongside their published article/ surface with CrossRef.

    • Profile Picture

      September 16, 2024 at 1:26 pm

      Laurence Mabile says:

      Thanks Katie for all your comments. We believe that asking for ORCID doesn’t create a barrier, on the contrary a barrier will be generated if we promote diversifying the individual IDs in many different identifiers (per country, institution, etc.). Having a PID for researchers that is free, universal, interoperable, and easy to get is highly beneficial for the community. And if publishers start requiring them, then more researchers will create ORCIDs, which will facilitate interconnectivity of research outputs. (see https://projects.tib.eu/pid-service/en/persistent-identifiers/persistent-identifiers-pids/) Nevertheless, your comment helped us see that a link to create an ORCID was needed in the recommendation and we will now add it.

  • Profile Picture

    July 17, 2024 at 1:26 pm

    Francoise Genova says:

    The Evaluation of Research (EoR) IG posted information about this RfC (https://www.rd-alliance.org/groups/evaluation-research-ig/forum/topic/output-from-the-sharing-rewards-and-credit-ig-in-rfc-until-july-26th/), which triggered the following discussion in comment to this post (the links to the persons' profile is available in the original discussed linked above): - from Ingvill Constanze Ødegaard: This is a very important document with many excellent recommendations and the structured overviews are very helpful. Only one aspect that always seems to be missing when we talk about research, is that research takes place in many different fields, amongst others also in civil society organisations, NGOs and IGOs. A lot of private and public funding goes into research/data collection in these organisations and in my experience OS, data management, FAIR,CARE etc are concepts not even known to many. Furthermore, when I address the issue OS is for a variety of reasons even opposed. In particular with respect to the SDGs it would thus be important to widen the actors we address with regard to OS strategies to include all spheres where reseach takes place and funding agencies should be recommended to require OS also in these areas. - from Francoise Genova: Dear Ingvill, Thank you very much for your comment. To my knowledge the SHARC IG chairs will see this discussion, so I think that we can continue it here. I will make sure to report it in the RfC page comments when we have discussed a bit further. For me your comment introduces a new dimension in the RDA ‘evaluation space’, and likely more generally a new dimension in the global, very active discussion about the necessary evolution of research evaluation. The RDA may be able to contribute, in particular because the RDA for the Sustainable Development Goals IG, of which you are a co-chair, knows about NGOs and IGOs. At this stage I think that introducing this new dimension in the SHARC output would significantly delay it, whereas it is a very useful contribution to the on-going work on research evaluation. Would you, and/or the SDG IG, be interested in working on a document on research evaluation taking the NGOs and IGOs specificities into consideration? This could be a very useful RDA Output! Best regards Francoise - from Ingvill Constanze Ødegaard: Thanks for your feedback Francoise, just back from meetings. Will look into your suggestion and discuss it with the co-chairs. Also in the IG Data for Development NGOs and IGOs play a big role. Best Ingvill

    • Profile Picture

      July 22, 2024 at 1:26 pm

      Laurence Mabile says:

      Many thanks Ingvill and Françoise for this discussion. Although our first target has been the academic sphere, the recommendations are inclusive and can be applied to any organisation doing research, including NGOs, IGOs or private institutes, as long as they perform research. We will make it more explicit in the Objectives section of our recommmendations. To make this inclusivity effective, it is though key to disseminate the recommendations to the relevant stakeholders. The Evaluation IG could also help here…

  • Profile Picture

    July 9, 2024 at 1:26 pm

    Laurence Mabile says:

    Thank you Daniel for your time and your comments. A quick response to the first one: Indeed the output is the text diplayed on the GitHub website. We could not find other ways to point to it while complying to the output submission template. A DOI will be created for the final version once the community review is completed. We'll be back soon to respond to your other comments...

  • Profile Picture

    July 2, 2024 at 1:26 pm

    Daniel S. Katz says:

    In In https://bienflorencia.github.io/rda-sharc-reco/introduction.html#the-current-sharing-practice-of-academics 1. software isn't mentioned 2. software leads to an additional challenge, as it needs to be actively maintained to avoid software collapse (ref Hinsen). This maintenance requires human work, and one method to support that work is to commercialize the software or some uses of it.

    • Profile Picture

      July 22, 2024 at 1:26 pm

      Laurence Mabile says:

      1- In the introduction, which you referred to, we mentioned that activities and efforts to make research outputs such as data and software findable, accessible, interoperable and reusable, in accordance with the FAIR principles, should be properly recognised. 2- The issues you raised regarding software collapse should definitely be considered in the Software Management Plan, which we recommended for research performing organisations to require. We have though considered that commercialisation is not a reward per se. The focus of this work was to make recommendations which could stimulate open source research software by recognising and rewarding relevant efforts. It was out of the scope of this work to discuss commercialisation or non-commercialisation of software.

  • Profile Picture

    July 2, 2024 at 1:26 pm

    Daniel S. Katz says:

    In https://bienflorencia.github.io/rda-sharc-reco/introduction.html#the-current-sharing-practice-of-academics, the comma in "Now a clear consensus on how outputs should be shared and rewarded, needs to be established" should be removed.

    • Profile Picture

      July 22, 2024 at 1:26 pm

      Laurence Mabile says:

      Good point, thanks, the comma will be removed in the new version.

  • Profile Picture

    July 2, 2024 at 1:26 pm

    Daniel S. Katz says:

    In https://bienflorencia.github.io/rda-sharc-reco/introduction.html#how-modern-science-is-recognised, I don't understand how "credit is allocated by the community through ... peer review ...". If you are going to say this, why not also say that credit is allocated through prizes?

    • Profile Picture

      July 22, 2024 at 1:26 pm

      Laurence Mabile says:

      We believe we provided a more specific description of the processes at stake in the text: credit is a milestone in the rewarding process, it is a non tangible recognition element as opposed to prizes that are tangible rewards. We have though clarified the sentence as follows : « In the case of published discoveries, credit is allocated by the community through attribution, peer review approval, and citation ».

  • Profile Picture

    July 2, 2024 at 1:26 pm

    Daniel S. Katz says:

    In https://bienflorencia.github.io/rda-sharc-reco/introduction.html, I don't agree with "Crediting is the explicit recognition for one’s contribution to a work, the process whereby ownership of a scientific work is attributed to an individual, a group of individuals or an institution". I think this is mixing credit/contribution and ownership, which are separate concepts.

    • Profile Picture

      July 22, 2024 at 1:26 pm

      Laurence Mabile says:

      The word ‘ownership’ is here used in the sense of ‘paternity’ of the work (as referred to in Shibayama et al.). We guess it can be confusing to some extent so we did change the sentence as follows : « Crediting is the explicit recognition for one’s contribution to a work, the process whereby the origin of a scientific work is attributed to an individual, a group of individuals or an institution ». We hope it does clarifiy the meaning of the sentence…

  • Profile Picture

    July 2, 2024 at 1:26 pm

    Daniel S. Katz says:

    Some of the grammar in https://bienflorencia.github.io/rda-sharc-reco/ is awkward. For example, "The funders" should probably be "Funders", and "The publishers" should probably be "Publishers".

    • Profile Picture

      July 22, 2024 at 1:26 pm

      Laurence Mabile says:

      Indeed, thanks! We will correct the relevant parts of the text you referred to and check the entire text again for grammar.

  • Profile Picture

    July 2, 2024 at 1:26 pm

    Daniel S. Katz says:

    The download object here that is called the output isn't actually the output but instead a word document (why word rather than a more portable text or even PDF?) with a paragraph of text and a link to a website. I think the website itself should be the output, and perhaps should be archived and pointed to in the DOI record directly.

    • Profile Picture

      August 5, 2024 at 1:26 pm

      Julien Colomb says:

      I do agree, RDA outputs must be persistent. You may consider making a pdf via quarto as the RDA output (starting with a link to the current deployed website if you want).

You must be logged in or join the group to leave a comment.