This meeting is a follow up to the inaugural P16 (November 2020) Benchmarking BoF, held under the umbrella of the Global Open Research Commons (GORC) IG, where we gauged the interest in creating a working group to evaluate and recommend a minimal set of international benchmarks for global commons that allow researchers and developers to coordinate services and create roadmaps for international interoperability. In short, this WG will collect information about 2 different types of benchmarks used by GORC organizations:
-
Benchmarks that measure success, adoption, engagement or use of their services within their organization, such as data downloads, number of contributed software packages, HPC cycles and similar KPI and access statistics and
-
Benchmarks that allow Commons developers to compare features across science clouds, as evidence of their organizations maturity and adherence to best practices. In the latter case for example, we would consider benchmarks such as evidence or the existence of a consistent and openly available data privacy policy or a workflow for adding and maintaining PIDs for managed assets.
Participants discussed the proposed work packages and agreed during the meeting this was a worthy goal and an appropriate approach. The WG case statement was submitted to RDA in January of 2022 and at the time of this submission is under community review.
The objective of this meeting will be:
-
High level review of discussions from P16 and GORC IG leadership calls
-
Review the timeline of proposed work packages
-
Determine next steps for outreach and recruitment of members, and collecting, organizing and reviewing benchmarks
-
This will include a discussion about how the typology developed by the GORC IG can be used as an organizing principle for some deliverables proposed within the WG
-
-
Create a standing meeting for the WG
Collaborative session notes (main session): https://docs.google.com/document/d/1RSlRDheHARG1l-MSmwzK3Ae7tO9t1gxc2WdFXMVPUEs/edit?usp=sharing
Collaborative session notes (repeat session): https://docs.google.com/document/d/1LA2Aj1Yd9bEfmvrzppVpgWxoUA8qlD4xLyusNky7lg4/edit?usp=sharing
-
Introduction to meeting and previous work (10 mins) - Mark Leggott
-
Take away messages from P16
-
-
The proposed timeline and outline scope of work (15 mins) - Karen Payne
-
Coordination with GORC IG
-
-
Create an outreach list to recruit for the WG (10 min) - All
-
Group activity to refine deliverables and approach (40 mins) - All
Set up standing meeting and assign action items for next steps (15 mins) - Karen Payne and Mark Leggott
Research funders, infrastructure providers and data services active in delivering Research / Data Commons initiatives
The Global Open Research Commons International Benchmarking WG will examine benchmarks that are useful to national, pan-national and domain specific developers of Commons globally. The benchmarks will fall into one of two classes: internal benchmarks to measure user engagement or content generation, and external benchmarks as indicators of Commons maturity and adherence to best practices. We anticipate looking to the typology of commons under development by the GORC IG as an organizing principle for the benchmarks. The WG will also create a dissemination plan aimed at promoting the benchmarks across the Commons community.
Following an initial BoF session at P16 (ostensibly in Costa Rica but held virtually), the chairs of the WG submitted a case statement to RDA which at the time of this submission is under community review. The Chairs of the WG have continued to coordinate with the GORC IG as they refine both the definition and typology of the Commons. In addition, the Data Together leadership (representing RDA, CODATA, GOFAIR and WDS) have convened a series of meetings to discuss how to effectively coordinate their GORC initiatives. This session will build on the discussions at P16, the work of the IG leadership and DataTogether meetings, and will articulate the next steps and larger path forward for the WG.
The most recent IG session was held at P16 in Costa Rica. Slides and notes are available online.
The most recent Benchmarking BoF session was held at P16 in Costa Rica. Slides and notes are available online.
Originally, three BoF sessions for the IG were run at previous events:
-
At the 11th RDA plenary in Berlin in March 2018 there was a BoF entitled Towards a Global Open Science Commons. This included presentations on the African Open Science Platform, the Australian Research Data Commons, the European Open Science Cloud, the NIH data commons and Canadian activities.
-
At International Data Week in Gaborone in November 2018, a SciDataCon session was held on Delivering a Global Open Science Commons. Again a number of presentations were given to profile existing work, after which the group discussion pointed to a number of next steps.
-
At the 13th Plenary in Philadelphia in April 2019 there was a BoF entitled Coordinating Global Open Science Commons initiatives. This session focused on group work to advance a typology and identification of areas in which it makes sense to collaborate and coordinate work.
In addition, at the CODATA 2019 'Towards next-generation data-driven science: policies, practices and platforms' conference in Beijing in September 2019, a session was held on Coordinating Global Open Science Commons Initiatives. A number of presentations were given and further discussion was focused on understanding possible areas for consensus building and collaboration, surveying initiatives that aim to coordinate Open Science activities and reduce ’siloisation’, refining the initial typology and framework and examining the potential for further coordination and interoperability.
- 679 reads