Reproducibility IG Charter

22 May 2023

Reproducibility IG Charter

 

This interest group seeks to advance and enable reproducibility in research based on or producing datasets that require code. Our goals are to build bridges with the many RDA external efforts addressing reproducibility, doing so in coordination with the other RDA Working and Interest Groups where appropriate. Our efforts may also produce Working Groups that can develop recommendations and solutions.

At the 20 th Plenary in Gothenburg, Limor Peer and Tom Honeyman co-organized a well-attended session on Computational Reproducibility: What’s Next for RDA? in which the question of reinstating the Reproducibility IG was posed to the community. A primary goal of the session was to gauge the interest in computational reproducibility and in an Interest Group on the topic among RDA members. We also invited a distinguished panel of speakers to share their work on reproducibility outside RDA as a way of mapping the landscape and illustrating the potential role the IG can play as a mechanism for bridging between these and related RDA efforts. As we summarized in the notes, the conversation indicated that there appears to be value in establishing a coordinating function within the RDA, likely through reestablishing the historical Reproducibility IG; that there are many external efforts, and RDA internal efforts that need to be taken into consideration; and that if there was a clear point of entry into RDA activities around reproducibility, it would be much easier to align RDA efforts with external efforts.

 

Download the full Charter to read the IG's focus and objectives 

 

Co-Chairs

Limor PeerLauren CadwalladerTom HoneymanMandy Gooch

 

 

Review period start: 
Wednesday, 24 May, 2023 to Saturday, 24 June, 2023
  • Nicolas Rougier's picture

    Author: Nicolas Rougier

    Date: 20 Jun, 2023

    Among the different objectives, it might be good to have a dedicated effort for defining terms properly (reproducibility, replicability, reusability, etc) and to related them to other group efforts (see for example https://arxiv.org/abs/1802.03311, "Terminologies for reproducible reseach"). Said differently, what does the term "reproduciblility" means for code? Maybe it has been addressed by the IG in the past IG but I did not find the referrence

  • Lauren Cadwallader's picture

    Author: Lauren Cadwallader

    Date: 10 Jul, 2023

    Thanks for your comment on the Charter. Our intention is for this Group is to facilitate discussions around reproducibility and we think that defining reproducibility and related terms would be a natural and important discussion for our Group to cover. Given that several definitions already exist, we wouldn't look to create an Output around the definitions of key terms but instead would record and share the facilitated conversations about the strengths and limitations of existing definitions. We've clarified the definition of reproducibility that helps define our intentions for the Group in the revised Charter and included the facilitation role around community definitions in our Objectives.

    Lauren, on behalf of the co-chairs

  • Francis P. Crawley's picture

    Author: Francis P. Crawley

    Date: 20 Jun, 2023

    The principal objective of 'acting as a bridge between internal and external [RDA] efforts around reproducibility' is to be welcome. Some small thoughts:

    1. As Nicolas Rougier suggests above, you will likely want to define 'reproducibility data, data sets, digital tools, and digital ecosystems.

    2. It would be of value to see some reflection on why reproducitility is valuable, how is it limited, and where it may not be applicable.

    3. A discussion on data quality, reliability, and integrity in connection with reproducibility would appear worthwhile.

    Much appreciation, Francis

  • Lauren Cadwallader's picture

    Author: Lauren Cadwallader

    Date: 10 Jul, 2023

    Thanks for your comments on the Charter. Points 1 & 2 are covered in our response to Nicolas. On your third point, we agree that would be a worthwhile discussion and we’ve included data integrity as an area of interest in the revised Charter. There are other groups in the RDA who are working on these issues and in this instance (and many others) we would look to generate joint sessions to address issues that overlap our Group and existing work in RDA.

    Lauren, on behalf of the co-chairs

submit a comment