Skip to content

Conversation

@ItIsJordan
Copy link
Collaborator

@ItIsJordan ItIsJordan commented Aug 13, 2025

Opening a draft PR to trigger AI PR reviewing/test implementations.

Adds observer role/access key functionality to HEPData, allows a user to be sent an access key to allow login-free access to view-only records.

closes #130


The SQL required to add the submissionobserver table to the database is as follows:

create table submissionobserver
(
publication_recid serial,
observer_key varchar(8) not null,
constraint pk_submissionobserver
primary key (publication_recid)
);
alter table submissionobserver
owner to hepdata;


Adds a new SubmissionObserver class to store access keys for observer role in new HEPSubmissions
Adds creation of the SubmissionObserver object upon HEPSubmission creation, as well as deletion when finalising, creating a new version, or removing the specific submission. Also adds key verification function.
Adds functionality for the metadata function, which handles /record/int endpoint to pass given observer_key to the render_record function.
Fixes deletion code for SubmissionObserver, previous code was wrong.
Adds the observer_key data (if exists) to the context in records/api to display a verified access key.
Adds a new clipboard entry for the access key copy button and accompanying HTML, and some modifications to allow the CSS to be extended to function on the new button.
Adds new observer key creation to new version of submission within records/api.py
Updates the submit_post function in submission/views.py (POST handler for /submit) to return observer key and publication_recid of the submission to generate access URL.
Adds the observer key to the post submission screen. Modifies the request triggered by the submit button to receive and display observer key/recid as a response. Adds observer key div to submit.html.
…mission

Add status filtering to observer key creation in get_or_create_hepsubmission to allow only "todo" (to deny creation in sandbox instances)
Adds a basic observer key test to test against the render_record function permission/observer key functionality
Renames access_key to observer_key and refactors the SubmissionObserver object to use publication_recid
Adds an endpoint at /record/coordinator/observer_key/ to retrieve an observer key for a specific record if the user has permission to do so.
Adds a function to get or create a submission observer object for a given recid in submission utils.
…server

Update get_or_create_hepsubmission to use get_or_create_submission_observer
Adds a message to the post-submission widget about the observer key. Also adds default hidden status and code to remove after submission.
Remove unique constraint from observer_key in SubmissionObserver object model.
Updates SubmissionObserver key generation to an 8 character truncated UUID
Fixes old references to access_key and submission.id in test_observer_key. Updated to observer_key and submission.publication_recid.
Moves get_or_create_submission_observer into a more suitable submodule (submission/api). Also handles potential regeneration of submissions during get.
Updates observer_url to use HEPDATA.site_url. No functional change here.
Adds the SubmissionObserver key to the email sent after submission in email/api.py Also modifies HTML to accommodate this.
Switches getting the SubmissionObserver object in creaet_new_version to use get_or_create_submission_observer.
Updates the observer_key endpoint to now return the full URL instead of just the key.
Adds just the observer key URL text to the dashboard "Manage Submission" widget at the bottom. Adds JS to handle the request and setting of this data. Includes HTML/CSS changes.
Moves verify_observer_key function from records/api.py into permissions/api.py, where it is more relevant as a permissions function.
Add missing import from records_test.py for verify_observer_key from permissions.api
Improves observer key selection and use logic. Should now appear when both using (as logged in perms user), and when not logged in with the observer key. Will also not query on every reload (reuses queried val)
Slightly improves observer key logic in the resource widget code. Will check for todo and observer key, instead of just todo.
Modify render_record to pass observer_key to context when logged in so it properly displays when logged in, but not set in request parameter.
Fixes a bug where observer_key does not properly display when logged in for copy/paste.
Adds further URL checks for a logged out observer for URL display
@ItIsJordan ItIsJordan marked this pull request as ready for review November 12, 2025 14:17
Updates get_or_create_submission_observer to use get_latest_hepsubmission instead of a basic query. Previous implementation did not consider versioning, so would not properly select the latest submission.
Having the setup_clipboard function inside of the promise was causing issues where (I think) the clipboard was being set up at a bad time, causing no action when clicking it.
Copy link
Member

@GraemeWatt GraemeWatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I did a bit more testing and unfortunately found a few problems. Hopefully, it is not too much work to address them.

  1. The observer_key access doesn't seem to work for a revised record. When a new version 2 is created (but not finalised), the observer_key is added to the database, but including the observer_key in the URL still returns version 1 not version 2, even if version=2 is included explicitly in the URL.
  2. It should be only the decision of the Coordinator whether or not they want to share the observer_key. It should not be visible to normal Uploaders/Reviewers. Currently, if an Uploader/Reviewer (without Coordinator access) views a record in preparation, the various links (and the JSON format) displayed on the record contain the observer_key. Can the links (and JSON) suppress the observer_key unless it is being used to access the record (for a user without explicit permissions)?
  3. It would be good if the Revision Submission widget could display a link containing the observer_key after clicking the "Revise Submission" button, similar to the link displayed when the original version 1 is created. The create_new_version function should send an email to the Coordinator containing the observer_key using a modification of the notify_submission_created function. Currently, the observer_key of a revised submission can only be obtained from the "Manage Submission" widget on a Coordinator's Dashboard. Sorry, I didn't mention this previously.

Adds a check to ensure only a coordinator can use the get_observer_data endpoint.
Replace == with === in equality check in HEPDATA.get_observer_key_data
Adds a check to ensure that the user is a coordinator before retrieving and setting an observer key in the context.
@ItIsJordan
Copy link
Collaborator Author

1. The `observer_key` access doesn't seem to work for a revised record.  When a new version 2 is created (but not finalised), the `observer_key` is added to the database, but including the `observer_key` in the URL still returns version 1 not version 2, even if `version=2` is included explicitly in the URL.

Without further investigation, I was unsure as to why this was occurring. I will come back to this after I have solved the other outstanding issue.

2. It should be only the decision of the Coordinator whether or not they want to share the `observer_key`.  It should not be visible to normal Uploaders/Reviewers.  Currently, if an Uploader/Reviewer (without Coordinator access) views a record in preparation, the various links (and the JSON format) displayed on the record contain the `observer_key`.  Can the links (and JSON) suppress the `observer_key` unless it is being used to access the record (for a user without explicit permissions)?

I have implemented a check to ensure that the user is a coordinator before allowing the return of any observer key data without the key verification. I need to do some testing for this case still.

3. It would be good if the [Revision Submission widget](https://github.com/HEPData/hepdata/blob/observer-role/hepdata/modules/records/templates/hepdata_records/components/revise-submission-widget.html) could display a link containing the `observer_key` after clicking the "Revise Submission" button, similar to the link displayed when the original version 1 is created.  The [`create_new_version`](https://github.com/HEPData/hepdata/blob/a7cbc23d51fc214aaed671558a45c1b18d730689/hepdata/modules/records/api.py#L549) function should send an email to the Coordinator containing the `observer_key` using a modification of the [`notify_submission_created`](https://github.com/HEPData/hepdata/blob/a7cbc23d51fc214aaed671558a45c1b18d730689/hepdata/modules/email/api.py#L449) function.  Currently, the `observer_key` of a revised submission can only be obtained from the "Manage Submission" widget on a Coordinator's Dashboard. 

I have had some issues implementing the copy/paste boxes on widgets so far, I just need to spend a little more time on this one.

Adds test_observer_create_from_none to test creation of SubmissionObserver objects where a Submission does not already have one, when called for by get_or_create_submission_observer.
Modifies format_submission where the version and version_count values are manipulated as this does not work with the version number of an in progress submission. The version could is incremented instead.
…sion

This allows format_submission to be used to properly handle version number display/access on a todo record.
Implements accessing by inspire ID on /record/ and passes any given key to the render_record function
Fixes bug in get_table_details where version was written over.
Updates tests using render_record where observer_key should be added to use of assert_called_once_with
Adds observer key display functionality to the revision widget with copy button. CSS is still missing.
Adds CSS rules to the revision copy button's IDs, and adds relevant IDs to existing copybutton rules.
Adds flex to data_links for copy buttons so they correctly shift to edge of button container.
Adds logic to default select most recent version in render_record when there is a verified observer key, and no version set. As this is how the keys are generated (without an explicit version number)
…record

Checks for a verified key, rather than use get_or_create_submission_observer as this will always return.
Updates the notify_submission_created function to add version numbers to subject.
Adds a call to send an email using notify_submission_created in create_new_version when a new Submission version is created.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

submission: create new "observer" role with password protection

4 participants