Test comments: proposals for changes in the indicators

04 Mar 2020
Groups audience: 

Dear members of the RDA FAIR Data working group
As part of the work on defining a set of indicators for the evaluation of
FAIRness, it was planned to test the indicators that were developed in 2019
in a test period to gather feedback that could lead to revision of the draft
indicators.
After the round of testing of the indicators in January and early February,
as reported in the meeting of 13 February 2020 [1
], the editorial team analysed the
comments from the testers and from discussions on the Guidelines. From those
comments, we have identified several issues about the indicators which we
would like to bring to the attention of the working group.
Some of the comments were requests for clarifications - to be addressed in
the Guidelines [2

Wmeg/edit> ] -, others were about the formulation of indicators, but we also
received comments that suggested addition, removal or combination of
indicators.
We have created issues on GitHub to facilitate discussion and to attempt to
reach consensus by early April 2020, before the start of a public review
period of the indicators and guidelines in mid-April. We would appreciate it
if you could contribute your comments by the 3rd of April 2020.
This message to the group is intended to make all working group members
aware of the proposed changes and to encourage anyone who wants to
participate in the discussion to access GitHub and contribute there. Please
let us know if you have difficulties to connect to GitHub and the editorial
team will help you to get connected.
The issues are the following (please go to
https://github.com/RDA-FAIR/FAIR-data-maturity-model-WG/issues/39 for the
links to the individual issues):
Drop indicator because out of scope
1. A1.1-02M+D: Out of scope - indicators test for open-source protocol,
which is not mentioned in FAIR principle A1.1.
2. I2-01M+D: Out of scope - indicators test for use of standard
vocabularies, but principle I2 only refers to FAIR vocabularies and says
nothing about 'standard'.
3. R1.1-03M: Out of scope - indicator tests that licence information is
in the 'right' element, but this is basically a quality aspect; we don't
test this for other metadata requirements either.
4. R1.1-05M: Out of scope - indicator tests information on consent for
personal data; this is not mentioned in principle R1.1 which is about
licensing.
5. I1-03M+D: Out of scope - principle I1 does not mention
'self-describing'. Furthermore, it has been noted that the term itself is
not entirely clear and could be seen to be very close to the representation
being machine-understandable.
New indicator
1. A1-0xM: New indicator alongside A1-01D for manual access to
metadata; it was pointed out that there is no indicator for manual access to
metadata while there is an indicator for manual access to data (A1-01D Data
can be accessed manually (i.e. with human intervention) ). An argument was
made that there are cases that metadata can only be accessed with human
intervention.
Combining indicators
1. A1-01M: Merge closely related indicators A1-01M, A1.1-03D and
A1.2-01M into A1-01M with definition "Metadata contains information to
enable the user to get access to the data"; in guidelines include that this
can be information about access control, e.g. need to register or provide
username/password.
2. A1.2-01D, A1.2-02D: Merge these two: having separate indicators for
authentication and authorisation puts undue emphasis on these aspects; they
will always be evaluated together.
Rephrasing
1. R1-01M: Rephrase: align with wording of principle R1; not
"sufficient metadata" but "plurality of accurate and relevant attributes".
2. F2-01M: Rephrase: delete the mention of standard in indicator for F2
- the standard aspect is covered in R1.3 - the indicators in F2 and R1 will
just focus on the amount of metadata, and R1.3 will test that metadata is
standard.
3. F1-02M+D: Rephrase: use 'globally unique' instead of 'universally
unique' - to align with the principle and because it might be confusing as
UUID has a very specific meaning.
4. I3-02D, I3-03M, I3-04M: Rephrase: drop 'sufficiently' qualified -
that's not in the principle, only 'qualified'
Please note that these are proposed changes that are open for discussion. If
there are no objections, the editorial team will proceed with the proposed
changes in the first week of April.
We are looking forward to your comments!
Kind regards,
Makx Dekkers and the editorial team
[1] https://www.rd-alliance.org/workshop-7
[2]
https://docs.google.com/document/d/1pDGGL3-BbBJu18KlfZUI3AizKLHXGXdIi_mP...
meg/edit