This meeting is a continuation of the regular meetings of the Global Open Research Commons International Benchmarking WG. The GORC Benchmarking WG is conducted under the umbrella, and coordinates with the Global Open Research Commons (GORC) IG as they refine the definition and typology of the Commons and work towards a roadmap for global development. The WG’s goal is to evaluate and recommend a minimal set of international benchmarks for global commons that allow researchers and developers to coordinate services and create roadmaps for international interoperability.
The attendees of the Benchmarking WG at P17 agreed to the general purpose and approach to the work proposed by the WG leads. Subsequent to P17, approximately 29 individuals joined the WG. We held our first regular meeting in June of 2021 to respond to Council’s feedback to the WG Case statement. It was informative feedback and helped us better define the meaning of benchmarks. Moreover, it provided the opportunity to clarify our role in the community, not as evaluators or an organization that seeks to certify commons, but rather as supporting commons developers who are creating interoperable services.
Between the end of July 2021 and P18 we anticipate making decisions about the form of the deliverable for the WG and will be in the process of testing and utilizing the workflow developed for researching, proposing, discussing and capturing benchmarks. In light of this, we propose the Benchmarking WG meeting at P18 essentially be a standard working meeting where members can meet and discuss potential benchmarks. This would allow anyone who is interested but not yet involved in the WG to see the process in action and provide an opportunity for the WG to be transparent about the state of our deliverables.
The objective of this meeting will be:
Recap and discuss any proposed refinements to the benchmark submission and approval process
Review the status of deliverables
Determine next steps
Introduction to meeting and previous work (10 mins) - Mark Leggott
Take away messages from P17
Review and update: The process for proposing benchmarks, KPIs and implementations; current status of the deliverable
Group activity discussion of submitted benchmarks (40 mins) - All
Next steps (15 mins) - Karen Payne and Mark Leggott
Research funders, infrastructure providers and data services active in delivering Research / Data Commons initiatives
The Global Open Research Commons International Benchmarking WG will examine benchmarks that are useful to national, pan-national and domain specific developers of Commons globally. The benchmarks will fall into one of two classes: internal benchmarks to measure user engagement or content generation, and external benchmarks as indicators of Commons maturity and adherence to best practices. We anticipate using the typology of commons under development by the GORC IG as an organizing principle for the benchmarks. The WG will also capture exemplars of well executed benchmarks or Commons features when available, and use the entire collection as input to the GORC IG Roadmap.
At the time of this writing the WG isn't officially endorsed (mid-July 2021). As a BoF this group submitted an original case statement to the TAB in early January 2021. We then created a second version based on their review and re-submitted April 6, 2021. That version was elevated to Council and based on their feedback we created a third version and are waiting to hear the response. The group’s first regular session took place June 17, 2021. The next scheduled meeting is for July 22, 2021.
The most recent WG session was held at P17 in Edinburgh/Virtual. Slides and notes (main, repeat) are available online.
The most recent IG session was held at P17 in Edinburgh/Virtual. Slides and notes (main, repeat) are available online.
An IG session was held at P16 in Costa Rica. Slides and notes are available online.
A Benchmarking BoF session was held at P16 in Costa Rica. Slides and notes are available online.
Originally, three BoF sessions for the IG were run at previous events:
At the 11th RDA plenary in Berlin in March 2018 there was a BoF entitled Towards a Global Open Science Commons. This included presentations on the African Open Science Platform, the Australian Research Data Commons, the European Open Science Cloud, the NIH data commons and Canadian activities.
At International Data Week in Gaborone in November 2018, a SciDataCon session was held on Delivering a Global Open Science Commons. Again a number of presentations were given to profile existing work, after which the group discussion pointed to a number of next steps.
At the 13th Plenary in Philadelphia in April 2019 there was a BoF entitled Coordinating Global Open Science Commons initiatives. This session focused on group work to advance a typology and identification of areas in which it makes sense to collaborate and coordinate work.
The Data Together leadership (representing RDA, CODATA, GOFAIR and WDS) have convened a series of meetings to discuss how to effectively coordinate their GORC initiatives. At the CODATA 2019 'Towards next-generation data-driven science: policies, practices and platforms' conference in Beijing in September 2019, a session was held on Coordinating Global Open Science Commons Initiatives. A follow up introduction event was held in June 2021 . In both cases a number of presentations were given and further discussion was focused on understanding possible areas for consensus building and collaboration, surveying initiatives that aim to coordinate Open Science activities and reduce ’siloing’, refining the initial typology and framework and examining the potential for further coordination and interoperability. In addition, WG chairs are also participating in OSCER, the Open Science Commons Executive Roundtable.
- 188 reads