Simple globus script

[forsyth2]

Summary

Objectives:

• Distill the Globus logic of zstash into a stand-alone script, for more isolated debugging. (Original script is from Refactor zstash #370).

Select one: This pull request is...

• a bug fix: increment the patch version
• a small improvement: increment the minor version
• a new feature: increment the minor version
• an incompatible (non-backwards compatible) API change: increment the major version

Small Change

• To merge, I will use "Squash and merge". That is, this change should be a single commit.
• Logic: I have visually inspected the entire pull request myself.
• Pre-commit checks: All the pre-commits checks have passed.

[forsyth2]

Based on commits as of b8fe696, I asked Claude if we could get this script to "Authenticate 1x on toy run, 0x on real run". It first suggested using the same client, but I explained we were simulating two command calls. It then said this:

Achieving "1x on toy run, 0x on real run"

This is actually not possible in the current design because:

1. Each command execution creates a new NativeClient instance
2. The NativeClient needs to authenticate at least once per process to get valid tokens
3. Even with cached refresh tokens, the client still needs to perform the initial authentication flow

The closest you can get is Case 2 (1x on toy run, 1x on real run) by setting REQUEST_SCOPES_EARLY=True.

Potential Alternative Approach

If you truly need 0x authentication on the real run, you'd need to implement a more sophisticated token caching mechanism that bypasses the NativeClient.login() entirely on subsequent runs - essentially managing the OAuth flow manually and checking for valid cached tokens before calling login(). However, this would be significantly more complex and might not be worth the effort for most use cases.

The current behavior with REQUEST_SCOPES_EARLY=True is actually quite reasonable for a command-line tool - one authentication prompt per execution is typical.

I then asked "But why can we get Real Run down to no authentications if we authenticate twice in the Toy Run?" To which it summarizes with:

The difference likely comes down to how fair_research_login.client.NativeClient handles the OAuth flow:

• Two-step process: Forces a more thorough consent establishment
• Single-step process: May rely on different token caching behavior

The NativeClient.login() call in the real run might be doing different things depending on what was cached from the previous authentication pattern. In Case 1, it can completely skip user interaction, while in Case 2, it still needs some form of token refresh that requires user confirmation.

This is why the two-authentication approach in the toy run creates a more "persistent" authentication state for subsequent command executions.

which doesn't illuminate terribly much. I guess there is the question of using the fair_research_login NativeClient instead of the Globus one...

[forsyth2]

Results from 3rd commit (572210c):

• Case 1: REQUEST_SCOPES_EARLY: bool = False => authenticate 2x on toy run, 0x on real run
• Case 2: REQUEST_SCOPES_EARLY: bool = True => authenticate 1x on toy run, 1x on real run

Code block	Case 1	Case 2
TOY: check_state_files INI_PATH	/home/ac.forsyth2/.zstash.ini does NOT exist.	Same
TOY: check_state_files GLOBUS_CFG	/home/ac.forsyth2/.globus-native-apps.cfg does NOT exist.	Same
TOY: get_local_endpoint_id	Added local_endpoint to ini file, Got local endpoint from NAME_TO_ENDPOINT_MAP	Same
TOY: native_client_login	Pasting URL brings us to "Allow" screen immediately, paste auth code at command line	Prompt (login NOT required) for Argonne, prompt (login NOT required) for NERSC, "Allow" screen, paste auth code at command line
TOY: transfer_client.submit_transfer try/except	Prompt (login required) for Argonne, prompt (login required) for NERSC, "Allow" screen, paste auth code at command line, Consents added, please re-run the previous command to start transfer	Bypassed 2nd authentication.
For toy_run, skipped_second_auth=	False	True
REAL: check_state_files INI_PATH	/home/ac.forsyth2/.zstash.ini exists.	Same
REAL: check_state_files GLOBUS_CFG	/home/ac.forsyth2/.globus-native-apps.cfg exists.	Same
REAL: get_local_endpoint_id	Got local_endpoint from ini file, Got local endpoint from NAME_TO_ENDPOINT_MAP (implies the value retreived was None...)	Same
REAL: native_client_login	No logins or prompts	Pasting URL brings us to "Allow" screen immediately, paste auth code at command line
REAL: transfer_client.submit_transfer try/except	Bypassed 2nd authentication.	Same
For real_run, skipped_second_auth=	True	Same

[forsyth2]

After extensive back-and-forth with Claude, finally arrived at the changes in 497982a, which give us:

Run: REQUEST_SCOPES_EARLY: bool = True => authenticate 1x on toy run, 0x on real run (i.e., it achieves our goal)

Code block	Run
TOY: check_state_files	INI_PATH: /home/ac.forsyth2/.zstash.ini does NOT exist., GLOBUS_CFG: /home/ac.forsyth2/.globus-native-apps.cfg does NOT exist., TOKEN_FILE: /home/ac.forsyth2/.zstash_globus_tokens.json does NOT exist.
TOY: get_local_endpoint_id	Added local_endpoint to ini file, Got local endpoint from NAME_TO_ENDPOINT_MAP
TOY: get_transfer_client_with_auth	No stored tokens found - starting authentication, paste URL to web browser, Argonne prompt (no login), NERSC prompt (no login), Must add label (no default), "Allow", paste auth code to command line
TOY: transfer_client.submit_transfer try/except block	Used fresh authentication - tokens now stored for next time
For toy_run, skipped_second_auth=	True
REAL: check_state_files	INI_PATH: /home/ac.forsyth2/.zstash.ini exists., GLOBUS_CFG: /home/ac.forsyth2/.globus-native-apps.cfg does NOT exist., TOKEN_FILE: /home/ac.forsyth2/.zstash_globus_tokens.json exists.
REAL: get_local_endpoint_id	Got local_endpoint from ini file, Got local endpoint from NAME_TO_ENDPOINT_MAP (implies the value retreived was None...)
REAL: get_transfer_client_with_auth	Found stored refresh token - using it
REAL: transfer_client.submit_transfer try/except block	Bypassed authentication entirely - used stored tokens!
For real_run, skipped_second_auth	True

After run:

cat /home/ac.forsyth2/.zstash.ini
# [local]
# globus_endpoint_uuid =

cat /home/ac.forsyth2/.globus-native-apps.cfg
# cat: /home/ac.forsyth2/.globus-native-apps.cfg: No such file or directory

cat /home/ac.forsyth2/.zstash_globus_tokens.json
# {
#   "transfer.api.globus.org": {
#     "access_token": "alphanumeric token here>",
#     "refresh_token": "<alphanumeric token here>",
#     "expires_at": <number here>
#   }
# }

[forsyth2]

So it looks like this version trades out ~/.globus-native-apps.cfg for ~/.zstash_globus_tokens.json. For reference, ~/.globus-native-apps.cfg ends up looking something like this:

[<zstash client ID here>]
auth_globus_org__scope = openid
auth_globus_org__access_token = <alphanumeric token 1 here>
auth_globus_org__refresh_token = <alphanumeric token 2 here>
auth_globus_org__token_type = Bearer
auth_globus_org__expires_at_seconds = <number here>
auth_globus_org__resource_server = auth.globus.org
transfer_api_globus_org__scope = urn:globus:auth:scope:transfer.api.globus.org:all
transfer_api_globus_org__access_token = <alphanumeric token 3 here>
transfer_api_globus_org__refresh_token = 
transfer_api_globus_org__token_type = Bearer
transfer_api_globus_org__expires_at_seconds = <number here>
transfer_api_globus_org__resource_server = transfer.api.globus.org

[forsyth2]

I do see the unit tests failing, so it looks like the Globus SDK upgrade might have caused an issue there.

[forsyth2]

I'm going to try to incorporate these changes here in the simple script into the real thing, in #380