[Feature Request]: A new keyword arg for MPRester, `allowed_licenses`

[mkhorton]

Problem

Hi team,

With the inclusion of the GNoME data, I've noticed new kwargs like exclude_gnome on certain endpoints. However, it's not always clear to me which methods support this, and which methods might return GNoME data by default or not (for example, MPRester.get_entries).

This introduces a risk that even a user acting in good faith might accidentally retrieve data they cannot use due to licensing restrictions.

Proposed Solution

From a user perspective, I think the ideal option is a global flag, for example:

MPRester(allowed_licenses=("CC-BY", "CC-BY-NC"))`

which would determine what data might be returned globally for the entire MPRester session. For entities or individuals who cannot use CC-BY-NC licensed data (whether from GNoME or any other source), this would give some extra confidence that they might not retrieve this data in error or by accident.

I'm not sure how difficult this would be to implement but seemed like it might be a good idea, and might be a more future-proof solution in the case that MP accepts additional data from another source under CC-BY-NC or even a different license.

Thanks for your consideration on this - I appreciate the maintenance burden that must be involved with handling data available under different licenses in a graceful way.

Alternatives

N/A

[tschaume]

Thanks @mkhorton for this feature request! Definitely a sensible idea.

Just to clarify a couple of things:

• There's no exclude_gnome kwarg available. The summary and materials endpoints only provide search parameters to filter on batch_id or license. At this point, we technically don't need a license field or search parameter at all, since batch_id queries suffice to filter on GNoMe documents.
• The original input structures in the summary/materials collections fall under the GNoME license. MP does not need to impose license restrictions on derived data in other collections.

Whether GNoME data is returned to a user is decided by the API server depending on the API key used. There's an include_gnome kwarg in the mpr.summary.search() client function that allows users to control the inclusion of GNoME data in the response. This flag only makes a difference to users who have accepted the GNoME license. The currently best way to ensure GNoME data is not returned in the API responses is to use a separate account that hasn't accepted the license.

A minor caveat to this is the case where mpr.summary.search() is run without any query parameters (see this forum thread and issue #964). We're working on fixing it and have PR #974 in the pipeline.

So for now, I think our bases are covered since it's only a single batch_id that needs special treatment. Going forward, I agree that adding more transparent handling of licenses might be needed if we end up with more data from different sources with different licenses. HTH.

[mkhorton]

Thanks Patrick! This wasn't immediately obvious to me, I'll take a closer look.

If someone has accepted the GNoME license, are the permissions associated with their API key permanently changed, such that they can't revert? Is there a way to tell via code whether they have or have not accepted the license via an API query?

[tschaume]

If someone has accepted the GNoME license, are the permissions associated with their API key permanently changed, such that they can't revert?

When a user accepts the GNoME license, their account is added to an ACL group internally. As of now, users can't revert that themselves but it would be straight-forward for me to remove the account from the group upon request. However, if I recall correctly from my discussions with Lab counsel, the decision to accept the terms is supposed to be non-reversible.

Is there a way to tell via code whether they have or have not accepted the license via an API query?

The group TERMS:ACCEPT-NC would be indicated on the user's dashboard. We're currently not returning that user information through the API. While not efficient, you could compare counts for include_gnome=True/False. Digging into the mp-api code here and here could get you on the right track. HTH.

[mkhorton]

Thanks, that helps!

[mkhorton]

Sorry a related question on this - the AWS Open Data buckets, do they include the GNoME data or no? And, if so, do GNoME tasks have a special prefix so they can be identified?

I reviewed https://docs.materialsproject.org/downloading-data/aws-opendata for a mention.

Ironically I have not yet accepted the GNoME terms on my own account so cannot investigate by inspection.

[tschaume]

The AWS Open Data buckets, do they include the GNoME data or no?

Yes, the GNoME data is included in the OpenData repository (c.f. this forum thread and issue #964 from my initial response).

And, if so, do GNoME tasks have a special prefix so they can be identified?

GNoMe data will have a dedicated prefix based on batch_id when #974 is resolved. See my comment in the same forum thread for how to identify them. HTH

[mkhorton]

Thanks so much @tschaume!