International Benchmarking the Global Open Research Commons (GORC)

You are here

21 Jan 2021
Meeting objectives: 

This meeting is a follow up to the inaugural P16 (November 2020) Benchmarking BoF, held under the umbrella of the Global Open Research Commons (GORC) IG, where we gauged the interest in creating a working group to evaluate and recommend a minimal set of international benchmarks for global commons that allow researchers and developers to coordinate services and create roadmaps for international interoperability.  In short, this WG will collect information about 2 different types of benchmarks used by GORC organizations:

  • Benchmarks that measure success, adoption, engagement or use of their services within their organization, such as data downloads, number of contributed software packages, HPC cycles and similar KPI and access statistics and

  • Benchmarks that allow Commons developers to compare features across science clouds, as evidence of their organizations maturity and adherence to best practices. In the latter case for example, we would consider benchmarks such as evidence or the existence of a consistent and openly available data privacy policy or a workflow for adding and maintaining PIDs for managed assets.

Participants discussed the proposed work packages and agreed during the meeting this was a worthy goal and an appropriate approach.  The WG case statement was submitted to RDA in January of 2022 and at the time of this submission is under community review.

 

The objective of this meeting will be:

  1. High level review of discussions from P16 and GORC IG leadership calls

  2. Review the timeline of proposed work packages

  3. Determine next steps for outreach and recruitment of members, and collecting, organizing and reviewing benchmarks

    1. This will include a discussion about how the typology developed by the GORC IG can be used as an organizing principle for some deliverables proposed within the WG

  4. Create a standing meeting for the WG

Meeting agenda: 

Collaborative session notes (main session): https://docs.google.com/document/d/1RSlRDheHARG1l-MSmwzK3Ae7tO9t1gxc2WdFXMVPUEs/edit?usp=sharing

Collaborative session notes (repeat session)https://docs.google.com/document/d/1LA2Aj1Yd9bEfmvrzppVpgWxoUA8qlD4xLyusNky7lg4/edit?usp=sharing

  • Introduction to meeting and previous work (10 mins) - Mark Leggott

    • Take away messages from P16

  • The proposed timeline and outline scope of work (15 mins) - Karen Payne

    • Coordination with GORC IG

  • Create an outreach list to recruit for the WG (10 min) - All

  • Group activity to refine deliverables and approach (40 mins) - All

Set up standing meeting and assign action items for next steps (15 mins) - Karen Payne and Mark Leggott

Target Audience: 

Research funders, infrastructure providers and data services active in delivering Research / Data Commons initiatives

Group chair serving as contact person: 
Brief introduction describing the activities and scope of the group: 

The Global Open Research Commons International Benchmarking WG will examine benchmarks that are useful to national, pan-national and domain specific developers of Commons globally. The benchmarks will fall into one of two classes: internal benchmarks to measure user engagement or content generation, and external benchmarks as indicators of Commons maturity and adherence to best practices. We anticipate looking to the typology of commons under development by the GORC IG as an organizing principle for the benchmarks. The WG will also create a dissemination plan aimed at promoting the benchmarks across the Commons community.

Short Group Status: 

Following an initial BoF session at P16 (ostensibly in Costa Rica but held virtually), the chairs of the WG submitted a case statement to RDA which at the time of this submission is under community review.  The Chairs of the WG have continued to coordinate with the GORC IG as they refine both the definition and typology of the Commons. In addition, the Data Together leadership (representing RDA, CODATA, GOFAIR and WDS) have convened a series of meetings to discuss how to effectively coordinate their GORC initiatives.  This session will build on the discussions at P16, the work of the IG leadership and DataTogether meetings, and will articulate the next steps and larger path forward for the WG.

Estimate of the required room capacity (Hybrid plenary): 
40-60
Type of Meeting: 
Working meeting
Remote participation availability (only for physical Plenaries): 
Yes
Avoid conflict with the following group (1): 
Avoid conflict with the following group (2): 
Avoid conflict with the following group (3): 
Meeting presenters: 
WG/IG chairs plus invited speakers from Commons initiatives