Skip to content

Conversation

@ItIsJordan
Copy link
Collaborator

@ItIsJordan ItIsJordan commented Aug 13, 2025

Opening a draft PR to trigger AI PR reviewing/test implementations.

Adds observer role/access key functionality to HEPData, allows a user to be sent an access key to allow login-free access to view-only records.

closes #130


The SQL required to add the submissionobserver table to the database is as follows:

create table submissionobserver
(
publication_recid serial,
observer_key varchar(8) not null,
constraint pk_submissionobserver
primary key (publication_recid)
);
alter table submissionobserver
owner to hepdata;


Adds a new SubmissionObserver class to store access keys for observer role in new HEPSubmissions
Adds creation of the SubmissionObserver object upon HEPSubmission creation, as well as deletion when finalising, creating a new version, or removing the specific submission. Also adds key verification function.
Adds functionality for the metadata function, which handles /record/int endpoint to pass given observer_key to the render_record function.
Fixes deletion code for SubmissionObserver, previous code was wrong.
Adds the observer_key data (if exists) to the context in records/api to display a verified access key.
Adds a new clipboard entry for the access key copy button and accompanying HTML, and some modifications to allow the CSS to be extended to function on the new button.
Adds new observer key creation to new version of submission within records/api.py
Updates the submit_post function in submission/views.py (POST handler for /submit) to return observer key and publication_recid of the submission to generate access URL.
Adds the observer key to the post submission screen. Modifies the request triggered by the submit button to receive and display observer key/recid as a response. Adds observer key div to submit.html.
…mission

Add status filtering to observer key creation in get_or_create_hepsubmission to allow only "todo" (to deny creation in sandbox instances)
Adds a basic observer key test to test against the render_record function permission/observer key functionality
Renames access_key to observer_key and refactors the SubmissionObserver object to use publication_recid
Adds an endpoint at /record/coordinator/observer_key/ to retrieve an observer key for a specific record if the user has permission to do so.
Adds a function to get or create a submission observer object for a given recid in submission utils.
…server

Update get_or_create_hepsubmission to use get_or_create_submission_observer
Adds a message to the post-submission widget about the observer key. Also adds default hidden status and code to remove after submission.
Remove unique constraint from observer_key in SubmissionObserver object model.
Updates SubmissionObserver key generation to an 8 character truncated UUID
Fixes old references to access_key and submission.id in test_observer_key. Updated to observer_key and submission.publication_recid.
Moves get_or_create_submission_observer into a more suitable submodule (submission/api). Also handles potential regeneration of submissions during get.
Updates observer_url to use HEPDATA.site_url. No functional change here.
Adds the SubmissionObserver key to the email sent after submission in email/api.py Also modifies HTML to accommodate this.
Switches getting the SubmissionObserver object in creaet_new_version to use get_or_create_submission_observer.
Updates the observer_key endpoint to now return the full URL instead of just the key.
Adds just the observer key URL text to the dashboard "Manage Submission" widget at the bottom. Adds JS to handle the request and setting of this data. Includes HTML/CSS changes.
Moves verify_observer_key function from records/api.py into permissions/api.py, where it is more relevant as a permissions function.
Add missing import from records_test.py for verify_observer_key from permissions.api
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 38 out of 38 changed files in this pull request and generated 14 comments.

ItIsJordan and others added 17 commits November 6, 2025 13:35
Use OBSERVER_KEY_LENGTH value in the DB model definition for SubmissionObserver

Co-authored-by: Copilot <[email protected]>
Improves test_logged_out_observer by adding observer key access testing, and an initial check for URL existence on the JSON link at the top right.
Sets OBSERVER_KEY_LENGTH as a const in the javascript, for simpler modification.
Removes extra space before a comment in email_test.py
Removes extra slash in test_dashboard URL to be consistent with other case
Remove extra width value in copybutton.scss for #manager-footer #data_link_container
Correctly refer to dashboard_button by ID in set_observer_key. Adds # to properly refer to the button by ID.
Improves observer key selection and use logic. Should now appear when both using (as logged in perms user), and when not logged in with the observer key. Will also not query on every reload (reuses queried val)
Slightly improves observer key logic in the resource widget code. Will check for todo and observer key, instead of just todo.
Modify render_record to pass observer_key to context when logged in so it properly displays when logged in, but not set in request parameter.
Fixes a bug where observer_key does not properly display when logged in for copy/paste.
Adds further URL checks for a logged out observer for URL display
@ItIsJordan ItIsJordan marked this pull request as ready for review November 12, 2025 14:17
Updates get_or_create_submission_observer to use get_latest_hepsubmission instead of a basic query. Previous implementation did not consider versioning, so would not properly select the latest submission.
Having the setup_clipboard function inside of the promise was causing issues where (I think) the clipboard was being set up at a bad time, causing no action when clicking it.
Copy link
Member

@GraemeWatt GraemeWatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I did a bit more testing and unfortunately found a few problems. Hopefully, it is not too much work to address them.

  1. The observer_key access doesn't seem to work for a revised record. When a new version 2 is created (but not finalised), the observer_key is added to the database, but including the observer_key in the URL still returns version 1 not version 2, even if version=2 is included explicitly in the URL.
  2. It should be only the decision of the Coordinator whether or not they want to share the observer_key. It should not be visible to normal Uploaders/Reviewers. Currently, if an Uploader/Reviewer (without Coordinator access) views a record in preparation, the various links (and the JSON format) displayed on the record contain the observer_key. Can the links (and JSON) suppress the observer_key unless it is being used to access the record (for a user without explicit permissions)?
  3. It would be good if the Revision Submission widget could display a link containing the observer_key after clicking the "Revise Submission" button, similar to the link displayed when the original version 1 is created. The create_new_version function should send an email to the Coordinator containing the observer_key using a modification of the notify_submission_created function. Currently, the observer_key of a revised submission can only be obtained from the "Manage Submission" widget on a Coordinator's Dashboard. Sorry, I didn't mention this previously.

Adds a check to ensure only a coordinator can use the get_observer_data endpoint.
Replace == with === in equality check in HEPDATA.get_observer_key_data
Adds a check to ensure that the user is a coordinator before retrieving and setting an observer key in the context.
@ItIsJordan
Copy link
Collaborator Author

1. The `observer_key` access doesn't seem to work for a revised record.  When a new version 2 is created (but not finalised), the `observer_key` is added to the database, but including the `observer_key` in the URL still returns version 1 not version 2, even if `version=2` is included explicitly in the URL.

Without further investigation, I was unsure as to why this was occurring. I will come back to this after I have solved the other outstanding issue.

2. It should be only the decision of the Coordinator whether or not they want to share the `observer_key`.  It should not be visible to normal Uploaders/Reviewers.  Currently, if an Uploader/Reviewer (without Coordinator access) views a record in preparation, the various links (and the JSON format) displayed on the record contain the `observer_key`.  Can the links (and JSON) suppress the `observer_key` unless it is being used to access the record (for a user without explicit permissions)?

I have implemented a check to ensure that the user is a coordinator before allowing the return of any observer key data without the key verification. I need to do some testing for this case still.

3. It would be good if the [Revision Submission widget](https://github.com/HEPData/hepdata/blob/observer-role/hepdata/modules/records/templates/hepdata_records/components/revise-submission-widget.html) could display a link containing the `observer_key` after clicking the "Revise Submission" button, similar to the link displayed when the original version 1 is created.  The [`create_new_version`](https://github.com/HEPData/hepdata/blob/a7cbc23d51fc214aaed671558a45c1b18d730689/hepdata/modules/records/api.py#L549) function should send an email to the Coordinator containing the `observer_key` using a modification of the [`notify_submission_created`](https://github.com/HEPData/hepdata/blob/a7cbc23d51fc214aaed671558a45c1b18d730689/hepdata/modules/email/api.py#L449) function.  Currently, the `observer_key` of a revised submission can only be obtained from the "Manage Submission" widget on a Coordinator's Dashboard. 

I have had some issues implementing the copy/paste boxes on widgets so far, I just need to spend a little more time on this one.

Adds test_observer_create_from_none to test creation of SubmissionObserver objects where a Submission does not already have one, when called for by get_or_create_submission_observer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

submission: create new "observer" role with password protection

4 participants