Evaluating FAIRness Practise - Need for Integrated Ecosystem of Assessment Linked to Crediting and Rewarding Mechanisms (Remote Access Instructions)
Collaborative session notes: https://docs.google.com/document/d/1n2jimW-RB22o9C8XWEAKvx0ZIQI1grHQh6w7hsNj-m0/edit
Meeting Location: Congress A
Evaluating FAIRness practise: need for an integrated ecosystem of assessment linked to crediting and rewarding mechanisms.
Collaborative session notes:
Short introduction describing the scope of the group and if any previous activities
Data sharing statements and promotion face a challenging reality, as many obstacles remain on several fronts. Among them is the lack of relevant and recognized rewarding mechanisms for the very specific efforts required to share organized datasets and physical resources (Cambon-Thomsen et al., 2011; Mabile et al., 2016). The RDA-SHARC interest group is an interdisciplinary group set up to unpack and improve crediting and rewarding mechanisms in the data/resource sharing process. The main objective is to encourage the adoption of data sharing activities-related criteria in the research evaluation process at the institutional, national and European/international levels. As a step forward, FAIR sharing assessment grids are being built by RDA-SHARC IG members. To be generic and trans-disciplinary, assessment grids should be understandable by all scientist including those who are not experts in data science. The two grids displayed as a tree-graph structure are based on previous works on FAIR data management (Reymonet et al., 2018; Wilkinson et al., 2016; Wilkinson et al., 2018; and E.U.Guidelines about FAIRness DMPs):
1/ the self-assessment grid is conceived as a checklist for scientists to identify whether their own activities are compliant to FAIR principles and to pinpoint the hurdles that hinder efficient sharing and reuse of data.
2/ the two-level grid (simplified / extensive) is conceived as a chart for the evaluator to assess the quality of the researcher/scientist sharing practices, over a given period of time, taking into account the means & supports available over that period. Assessment criteria are classified according to their level of stringency regarding FAIRness (essential / recommended / desirable).
The scope of this session is to build the stepwise processes required to enhance adoption of the proposed FAIR assessment grids-prototypes by the various stakeholders. The questions to be discussed are: What are the pros and cons of the proposed assessment prototypes/grids? What are the steps required by the various actors in the evaluation ecosystem for grids adoptions? How to use this ecosystem for credit and rewarding? Which levels of implementation should be considered? The next step would be to design a pilot use of the modified grids in order to gather empirical data.
Additional links to informative material related to the group
● Sharc group page: www.rd-alliance.org/groups/sharing-rewards-and-credit-sharc-ig
● SHARC Posters: https://zenodo.org/record/1745374#.XA-v6HRKg2w
● Simplified poster grid : https://zenodo.org/record/1995646#.XA-wpXRKg2w
● Working document including draft recommendations and their background rationale: https://docs.google.com/document/d/14_HxIrrkB0128EQpmTqrwXtuJTy3zQ_TLFOD...
To identify drivers, locks and means to elaborate stepwise processes for adoption and implementation measures of grids prototypes for Fairness assessment by the research community
● To identify the various levels of possible implementation (European, national and institutional)
● To identify potential users and stakeholders (policy makers, researchers, funders, institutional evaluators)
● To identify and rank coercitives vs incentives tools for adoption and future evolutions of the grids
Targeted session outcomes:
● Guideline on “How to use results from FAIR grids assessment for rewarding and credit”
● Landscape of users and drivers for grids adoption by each user
● List and classification of tools for adoption as a pilot study
● Agenda for grid dissemination and quality improvement
● Goals of the group’s project; current standing; objectives of the meeting: Introduction to the group?
A. Cambon-Thomsen (FR) 10 min
● Grids in SHARC recommendations, L. Mabile (FR) 5 min
● Grids construction and structure, R.David (FR) 10 min
● Eliane Frankhauser, DANS (NL) 10 min:
DANS FAIR assessment tools and implementation (provisional title).
● Luiz Olavo Bonino, GO FAIR (NL) 10 min:
Essential FAIR criteria, prioritisation and rules for implementation (provisional title)
● Questions-mediated discussion with panelists and audience (approx. 45 min)
Chairs: Alison Specht, Romain David, Laurence Mabile, Anne Cambon-Thomsen
Group chair serving as contact person
Type of meeting
● Everyone interested by the project is welcome to come and make input, especially policy makers and scientists from biological, medical, environmental, astronomical sciences, but not exclusively.
Remote Access Instructions