With over 10000 members from 145 countries, RDA provides a neutral space where its members can come together to develop and adopt infrastructure that promotes data-sharing and data-driven research
This whiteboard is open to all RDA discipline specialists willing to give a personal account of what data-related challenges they are facing and how RDA is helping them
The Global Open Research Commons (GORC) is an ambitious vision of a global set of interoperable resources necessary to enable researchers to address societal grand challenges including climate change, pandemics, and poverty. The realized vision of GORC will provide frictionless access to all research artifacts including, but not limited to: data, publications, software and compute resources; and metadata, vocabulary, and identification services to everyone everywhere, at all times.
The GORC is being built by a set of national, pan-national and domain specific organizations such as the European Open Science Cloud, the African Open Science Platform, and the International Virtual Observatory Alliance (see Appendix A for a fuller list). The GORC IG is working on a set of deliverables to support coordination amongst these organizations, including a roadmap for global alignment to help set priorities for Commons development and integration. In support of this roadmap, this WG will develop and collect a set of benchmarks for GORC organizations to measure their user engagement and development internally within the organization, gauge their maturity and compare features across commons.
In the first case, the WG will collect information about how existing commons are measuring success, adoption or use of their services within their organization, such as data downloads, contributed software, and similar KPI and access statistics.
Secondly, we will also develop, validate, collect and curate a set of benchmarks that will allow Commons developers to compare features across science clouds. In the latter case for example, we would consider benchmarks such as evidence or the existence of :
A well defined decision making process
A consistent and openly available data privacy policy
Federated Authentication and Authorization infrastructure
Community supported and well documented metadata standard(s)
A workflow for adding and maintaining PIDs for managed assets
A mechanism for utilizing vocabulary services or publishing to the semantic web
A process to inventory research artefacts and services
An Open Catalogue of these artefacts and services
A proven workflow to connect multiple different research artefact types (e.g. data and publications; data and electronic laboratory notebooks; data and related datasets)
A mechanism to capture provenance for research artefacts
Mechanisms for community engagement and input; an element or scale for inclusion
We anticipate that the first set of metrics will be quantitative measures used within an organization, while the second set of benchmarks will be comparable across organizations.
Value Proposition
This WG is motivated by the broader goal of openly sharing data and related services across technologies, disciplines, and countries to address the grand challenges of society. The deliverables of the WG itself will inform roadmaps for development of the infrastructure necessary to meet that goal, while engagements and relationships formed during the work period will help forge strong partnerships across national, regional and domain focused members which are crucial to its success. Identifying observable and measurable benchmarks in pursuit of the global open science commons will help create a tangible path for development and support strategic planning within and across science commons infrastructures. In the future, best practices for commons development will emerge based on the experience of what actions led to successful outcomes. This work will provide a forum for discussion that will allow members to identify the most important features and the minimal elements required to guide their own development and build a commons that is globally interoperable. Finally, it will support developers as they seek resources to build the global commons by helping them respond to funding agencies requirements for measurable deliverables.
The proposed WG was discussed at the RDA 16 virtual plenary.[1] Participants discussed the initial work packages and agreed during the meeting this was a worthy goal and an appropriate approach.
Engagement with Existing Work
The GORC IG builds on, and incorporates the previous National Data Services IG. The Commons that will be investigated in this WG are likely either to have considered or implemented outputs from other RDA groups, such as the Domain Repositories IG, the Data Fabric IG, and the Virtual Research Environment IG, just to name a few. These groups and many others outside of RDA will have recommendations that speak to functionality and features of various components of Commons; for example the re3data.org schema for collecting information on research data repositories for registration, the EOSC FAIR WG and Sustainability WG that seek to define the EOSC as a Minimum Viable Product (MVP). We will review these and other related outputs to see if they have identified benchmarks that we feel will support our goals. This review period will ensure that we do not duplicate existing efforts. Appendix B of this case statement identifies a few of these existing efforts, both within and without RDA; this list will be expanded and reviewed by the WG members.
Work Plan
To create these deliverables, members of the group will:
Review public facing documentation of each Commons to extract benchmarking information (both KPIs and feature lists).
Review public facing documentation of recommendations and roadmaps from related communities to extract benchmarking information (Appendix B). This evaluation phase will include an examination of the outputs from other RDA WGs and position papers available in the wider science infrastructure community, along with experiences gathered by the WG’s members.
Because benchmarking information may not be easily found in public documents we will conduct outreach to Commons representatives and related organizations to ask for additional feedback and information about benchmarks used by their community. This may include benchmarks already in use, as well as benchmarks that organizations feel would be useful but which are not yet implemented.
Synthesize and document the benchmarks into 2 deliverables, described below..
We anticipate that the WG will create sub-working groups or task groups. The WG will decide if they would rather define the task group according to the deliverables, creating a Commons Internal Benchmarking TG and a Commons External Benchmarking TG, or if they would rather subdivide according to a typology of the commons, for example with some members looking at pan-national, national, or domain specific commons, or by some other subdivision of labor.
The WG will proceed according to the following schedule:
Month
Activity
Jan-Mar
2020
Group formation
Agreement on the scope of work and deliverables (broad scope)
Case statement community review
Creation of sub-working groups
Apr-Sept
2021
Begin literature review of public facing documents from Science Commons and related organizations
Refine scope: Meeting point to consolidate list of topics to be addressed in the deliverables and assess level of resource available to achieve them
Oct-Dec
2021
Begin outreach to Science Commons and related organizations
First draft: Internal Benchmarks distributed for community review
Mar-Jun
2022
First draft: External Benchmarks distributed for community review
July
2022
Final deliverables
Deliverables
This group will create Supporting Outputs in furtherance of the goals of the GORC IG. Specifically, 2 documents:
D1: a non-redundant set of KPIs and success metrics currently utilized, planned or desired for existing science commons, and
D2: a list of observable international benchmarks of features, structures and functionality that can help define a Commons and that will feed into a roadmap of Commons interoperability.
D3: Adoption Plan: described in section 9 below.
Mode and Frequency of Operation
The WG will meet monthly over Zoom, at a time to be determined by the membership. The WG will also communicate asynchronously online using the mailing list functionality provided by RDA and via shared online documents. If and when post-Covid international travel is restored during the 18 month work period of this WG then we will propose and schedule meetings during RDA plenaries and at other conferences where a sufficient number of group members are in attendance.
Addressing Consensus and Conflicts
The WG will adhere to the stated RDA Code of Conduct and will work towards consensus, which will be achieved primarily through mailing list discussions and online meetings, where opposing views will be openly discussed and debated amongst members of the group. If consensus cannot be achieved in this manner, the group co-chairs will make the final decision on how to proceed.
The co-chairs will keep the working group on track by reviewing progress relative to the deliverables. Any new ideas about deliverables or work that the co-chairs deem to be outside the scope of the WG defined here will be referred back to the GORC IG to determine if a new WG should be formed.
Community Engagement
The working group case statement will be disseminated to RDA mailing lists and communities of practice related to Commons development that are identified by the GORC IG in an effort to cast a wide net and attract a diverse, multi-disciplinary membership. Similarly, when appropriate, draft outputs will also be published to relevant stakeholders and mailing lists to encourage broad community feedback.
Adoption Plan
The WG will create an adoption plan for distributing and maintaining the deliverables. A specific plan will be developed to facilitate adoption or implementation of the WG Recommendation and other outcomes within the organizations and institutions represented by WG members. This will include possible strategies for adoption more broadly within the global community, and in such a way as to facilitate interoperability of global infrastructures. Pilot adoptions or implementations would ideally start within the 18 month timeframe before the WG is complete. We envision implementation occurring when developers of commons compare themselves with similar organizations. We also envision the adoption plan will speak to howwe include the benchmarks in the larger GORC roadmap being created by the parent IG.
European Roadmaps -The European Commission and European Strategy Forum on Research Infrastructures (ESFRI) encourage Member States and Associated Countries to develop national roadmaps for research infrastructures.
The European Strategy Forum on Research Infrastructures (ESFRI) Landscape Analysis “provides the current context of the most relevant Research Infrastructures that are available to European scientists and to technology developers”
GORC International Benchmarking WG Case Statement
RDA Case Statement
GORC International Benchmarking WG
The Global Open Research Commons (GORC) is an ambitious vision of a global set of interoperable resources necessary to enable researchers to address societal grand challenges including climate change, pandemics, and poverty. The realized vision of GORC will provide frictionless access to all research artifacts including, but not limited to: data, publications, software and compute resources; and metadata, vocabulary, and identification services to everyone everywhere, at all times.
The GORC is being built by a set of national, pan-national and domain specific organizations such as the European Open Science Cloud, the African Open Science Platform, and the International Virtual Observatory Alliance (see Appendix A for a fuller list). The GORC IG is working on a set of deliverables to support coordination amongst these organizations, including a roadmap for global alignment to help set priorities for Commons development and integration. In support of this roadmap, this WG will develop and collect a set of benchmarks for GORC organizations to measure their user engagement and development internally within the organization, gauge their maturity and compare features across commons.
In the first case, the WG will collect information about how existing commons are measuring success, adoption or use of their services within their organization, such as data downloads, contributed software, and similar KPI and access statistics.
Secondly, we will also develop, validate, collect and curate a set of benchmarks that will allow Commons developers to compare features across science clouds. In the latter case for example, we would consider benchmarks such as evidence or the existence of :
We anticipate that the first set of metrics will be quantitative measures used within an organization, while the second set of benchmarks will be comparable across organizations.
This WG is motivated by the broader goal of openly sharing data and related services across technologies, disciplines, and countries to address the grand challenges of society. The deliverables of the WG itself will inform roadmaps for development of the infrastructure necessary to meet that goal, while engagements and relationships formed during the work period will help forge strong partnerships across national, regional and domain focused members which are crucial to its success. Identifying observable and measurable benchmarks in pursuit of the global open science commons will help create a tangible path for development and support strategic planning within and across science commons infrastructures. In the future, best practices for commons development will emerge based on the experience of what actions led to successful outcomes. This work will provide a forum for discussion that will allow members to identify the most important features and the minimal elements required to guide their own development and build a commons that is globally interoperable. Finally, it will support developers as they seek resources to build the global commons by helping them respond to funding agencies requirements for measurable deliverables.
The proposed WG was discussed at the RDA 16 virtual plenary.[1] Participants discussed the initial work packages and agreed during the meeting this was a worthy goal and an appropriate approach.
The GORC IG builds on, and incorporates the previous National Data Services IG. The Commons that will be investigated in this WG are likely either to have considered or implemented outputs from other RDA groups, such as the Domain Repositories IG, the Data Fabric IG, and the Virtual Research Environment IG, just to name a few. These groups and many others outside of RDA will have recommendations that speak to functionality and features of various components of Commons; for example the re3data.org schema for collecting information on research data repositories for registration, the EOSC FAIR WG and Sustainability WG that seek to define the EOSC as a Minimum Viable Product (MVP). We will review these and other related outputs to see if they have identified benchmarks that we feel will support our goals. This review period will ensure that we do not duplicate existing efforts. Appendix B of this case statement identifies a few of these existing efforts, both within and without RDA; this list will be expanded and reviewed by the WG members.
To create these deliverables, members of the group will:
We anticipate that the WG will create sub-working groups or task groups. The WG will decide if they would rather define the task group according to the deliverables, creating a Commons Internal Benchmarking TG and a Commons External Benchmarking TG, or if they would rather subdivide according to a typology of the commons, for example with some members looking at pan-national, national, or domain specific commons, or by some other subdivision of labor.
The WG will proceed according to the following schedule:
Month
Activity
Jan-Mar
2020
Group formation
Apr-Sept
2021
Begin literature review of public facing documents from Science Commons and related organizations
Refine scope: Meeting point to consolidate list of topics to be addressed in the deliverables and assess level of resource available to achieve them
Oct-Dec
2021
Begin outreach to Science Commons and related organizations
Update at RDA17
Jan-Mar
2021
First draft: Internal Benchmarks distributed for community review
Mar-Jun
2022
First draft: External Benchmarks distributed for community review
July
2022
Final deliverables
This group will create Supporting Outputs in furtherance of the goals of the GORC IG. Specifically, 2 documents:
D1: a non-redundant set of KPIs and success metrics currently utilized, planned or desired for existing science commons, and
D2: a list of observable international benchmarks of features, structures and functionality that can help define a Commons and that will feed into a roadmap of Commons interoperability.
D3: Adoption Plan: described in section 9 below.
The WG will meet monthly over Zoom, at a time to be determined by the membership. The WG will also communicate asynchronously online using the mailing list functionality provided by RDA and via shared online documents. If and when post-Covid international travel is restored during the 18 month work period of this WG then we will propose and schedule meetings during RDA plenaries and at other conferences where a sufficient number of group members are in attendance.
The WG will adhere to the stated RDA Code of Conduct and will work towards consensus, which will be achieved primarily through mailing list discussions and online meetings, where opposing views will be openly discussed and debated amongst members of the group. If consensus cannot be achieved in this manner, the group co-chairs will make the final decision on how to proceed.
The co-chairs will keep the working group on track by reviewing progress relative to the deliverables. Any new ideas about deliverables or work that the co-chairs deem to be outside the scope of the WG defined here will be referred back to the GORC IG to determine if a new WG should be formed.
The working group case statement will be disseminated to RDA mailing lists and communities of practice related to Commons development that are identified by the GORC IG in an effort to cast a wide net and attract a diverse, multi-disciplinary membership. Similarly, when appropriate, draft outputs will also be published to relevant stakeholders and mailing lists to encourage broad community feedback.
The WG will create an adoption plan for distributing and maintaining the deliverables. A specific plan will be developed to facilitate adoption or implementation of the WG Recommendation and other outcomes within the organizations and institutions represented by WG members. This will include possible strategies for adoption more broadly within the global community, and in such a way as to facilitate interoperability of global infrastructures. Pilot adoptions or implementations would ideally start within the 18 month timeframe before the WG is complete. We envision implementation occurring when developers of commons compare themselves with similar organizations. We also envision the adoption plan will speak to howwe include the benchmarks in the larger GORC roadmap being created by the parent IG.
Co-chairs:
Appendix A: List of Commons
Pan National Commons
National Commons
European Roadmaps - The European Commission and European Strategy Forum on Research Infrastructures (ESFRI) encourage Member States and Associated Countries to develop national roadmaps for research infrastructures.
Non-European
Domain Commons
Gateway/Virtual Research Environment/Virtual Laboratory communities and other Services
Appendix B: Draft List of WG/IG, documents, recommendations, frameworks and roadmaps from related and relevant communities
[1] P16 session notes and presentation