Proposal to develop a mapping (set) FAIRness score

[cthoyt]

Currently sitting in the RDA FAIR Mappings WG and had the idea that we should be able to automatically calculate a FAIRness score both for individual mappings and a mapping set.

For an individual mapping, this could work by assigning a weight/importance to each field in the Mapping, then taking a weighted sum over all of the fields that are present (then normalizing against the highest possible score).

For a mapping set, it could be a combination of a weighted sum over the mapping set-level metadata fields and the aggregate FAIRness of individual mappings.

[gouttegd]

This could take the form of custom annotations in the LinkML model, similar to what we already use to mark “propagatable” slots and to record the version in which a slot was introduced.

For example:

slots:
  license:
    description: A URL to the license of the mapping.
    slot_uri: dcterms:license
    annotations:
      fairness_weight: 1.0
  mapping_provider:
    description: A URL pointing to the source that provided the mapping.
    annotations:
      fairness_weight: 0.9

This should then make it fairly easy for implementations to automatically calculate the overall score.

[matentzn]

Hahha this is a bit geeky but I definitely like it!

@gouttegd what would be the SOP though for defining the weights? I am sure there will be plenty of widely diverging views on "how important" adding the subject_type is..

My concrete concern is right now to implement a "FAIR Mappings WG" score, and I wonder how this would relate to what is proposed here, or if indeed they are very different and we will have two systems for "scoring fairness" in the end?

[gouttegd]

what would be the SOP though for defining the weights?

Since this “FAIRness score” is @cthoyt ’s idea, I’d suggest you ask him.

I am sure there will be plenty of widely diverging views on "how important" adding the subject_type is..

Quite likely, yes. Which is why, in my infinite wisdom, I was planning to stay outside of the discussion entirely. I merely outlined how those factors can be represented in the model, once they will be agreed upon.