GORC International Benchmarking WG Case Statement

08 Jan 2021

GORC International Benchmarking WG Case Statement

GORC International Benchmarking WG Case Statement

The original Case Statement which underwent community review is now superseded by the revised Case Statement following TAB review.

The full and revised Case Statement of the WG is attached to this page together with the original Case Statement.

 

Introduction

 

The Global Open Research Commons (GORC) is an ambitious vision of a global set of interoperable resources necessary to enable researchers to address societal grand challenges including climate change, pandemics, and poverty. The realized vision of GORC will provide frictionless access to all research artifacts including, but not limited to: data, publications, software and compute resources; and metadata, vocabulary, and identification services to everyone everywhere, at all times.

 

The GORC is being built by a set of national, pan-national and domain specific organizations such as the European Open Science Cloud, the African Open Science Platform, and the International Virtual Observatory Alliance (see Appendix A for a fuller list). The GORC IG is working on a set of deliverables to support coordination amongst these organizations, including a roadmap for global alignment to help set priorities for Commons development and integration. In support of this roadmap, this WG will establish benchmarks to compare features across commons.  We will not coordinate the use of specific benchmarks by research commons. Rather, we will review and identify features currently implemented by a target set of GORC organizations and determine how they measure their user engagement with these features.

Review period start: 
Friday, 8 January, 2021 to Monday, 8 February, 2021
  • Francoise Genova's picture

    Author: Francoise Genova

    Date: 03 Feb, 2021

    I used to be the vice-chair of the FAIR Working Group of the EOSC Executive Board, which completed its task at the end of 2020. I would like to strongly support the proposal to create this GORC International Benchmarking WG. The EOSC FAIR WG recommended in particular that its proposal for FAIR Metrics in the EOSC, inspired from the FAIR Data Maturity Model WG, be reviewed in an international context, and we suggested the GORC IG. The point was discussed during the P16 pre-WG BoF and it seems to fit well as one of the points which could be addressed in the International Benchmarking WG. I am also pleased to see the domain specific organisations are considered, and that the subdivision of tasks will be defined in a flexible way.

    A few minor comments:

    - the long list of possible topics in Section 1 can be frightening when organisations will be contacted to participate. It would be useful to say explicitly that not all commons-developing organisations are expected to develop all these features, depending on their mission and community requirements.

    - I suggest to add the FAIR Data Maturity Model WG in the list of relevant RDA Groups in Section 3. It is cited elsewhere, but FAIR plays an essential role for enabling seamless access to data and other digital objects.

    Very minor comment: in Section 9, last sentence, howwe > how we

    I will disseminate information about this WG proposal in the IVOA, which is cited as one of the relevant organisations among the domain commons.

    Best wishes

    Francoise Genova

  • Ville Tenhunen's picture

    Author: Ville Tenhunen

    Date: 08 Feb, 2021

    Hi all,

    We (Yin Chen and Ville Tenhunen) discussed in the EGI office (nowadays virtual one) about the RDA GORC Benchmarking IG Charter proposal. 

    Generally, the idea of the IG is good and easy to support. This is an activity which supports GORC WG work and global research data collaboration.

    We hope you could consider a few points of views to clarify or sharpen the charter text:

    • The value proposition has described values of having a set of benchmarks. It might be better if it defines who are the targeting readers and what benefits they can gain from this work?

    • Some phase of the WG lifecycle it is necessary to provide more information how such a set of benchmarks will be developed, so that whether the approach is appropriate, whether the result is valid can be evaluated.

    • How will the benchmark be used? It should provide some use scenarios to justify the usefulness of these benchmarks.

    Couple detailed comments about the Charter and Appendixes:

    1. Charter

    There is also the list of some benchmarks to consider. It is understandable that here is some examples about benchmarks, but couple of proposals for further considerations:

    • Is it possible to benchmark budgeting of commons per BKT or research funding in general (OECD has pretty much data about this)? Budgeting structure is also an important aspect. Some commons has funded internationally, some nationally and some has institution based funding etc.
    • Training and support infrastructures and methods are an important part of success in many places and for example EOSC had Skills and Training WG (https://www.eoscsecretariat.eu/working-groups/skills-training-working-group). Perhaps this should be also considered as a part of the benchmarks list.
    • Point "9. A proven workflow to connect multiple different research artefact types (e.g. data and publications; data and electronic laboratory notebooks; data and related datasets)". Currently also softwares has their increasing role as a research artefact. Please see: Report from the EOSC Executive Board Working Group (WG) Architecture Task Force (TF) SIRS (https://op.europa.eu/s/oK7d)

    2. Appendix A

    • There is mentioned “ATT (Finland)”. This initiative has ended. Currently these activities has organised by national open science coordination: https://www.avointiede.fi/en
    • Appendix contain mix of projects/initiatives/recommendations etc. It is not clear what ‘common’ refers to -- standards, recommendations, or practice? 

    3. Appendix B; Draft List of WG/IG, documents, recommendations, frameworks and roadmaps from related and relevant communities

     

    With the best regards,

    Yin & Ville

submit a comment