-
Notifications
You must be signed in to change notification settings - Fork 32
Addition of a new SSH-based backend for the origin #3077
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
bbockelm
wants to merge
16
commits into
PelicanPlatform:main
Choose a base branch
from
bbockelm:ssh_origin
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
e681bc1
WIP: Addition of a new SSH-based backend for the origin
bbockelm f288712
Working version of the SSH backend post human review
bbockelm 8383196
Add SSH storage to the supported cases
bbockelm 6437e78
Add missing file
bbockelm e01a39d
Fix name of TLS certificate parameter
bbockelm 560a6c4
Fix modest build failures due to rebase
bbockelm 7f42eb2
Remove test environment code: this affects the prune test (requiring …
bbockelm fbf0967
Switch to stdlib for user detection to fix GHA
bbockelm cd6902a
Fix binary transfer detection
bbockelm 5305ec4
Fix sporadic test failure due to Windows timer jitter
bbockelm 4ceffc6
Fix E2E tests for SSH
bbockelm aab3136
Fix trailing slash in unit test for root
bbockelm cc3ccab
Fix pending request with new channel-based design
bbockelm 10bea94
Avoid triggering a redirect (which can cause a second use of one-shot…
bbockelm 48dc836
Fixes from code review
bbockelm 0436f94
Have a separate timeout context for the shutdown and helper run
bbockelm File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -394,7 +394,7 @@ | |
| return | ||
| default: | ||
| if err := tm.executeTransfer(transfer, job.Options); err != nil { | ||
| log.Errorf("Transfer %s failed: %v", transfer.ID, err) | ||
Check failureCode scanning / CodeQL Clear-text logging of sensitive information High Sensitive data returned by an access to S3SecretKeyfile Error loading related location Loading Sensitive data returned by an access to UIPasswordFile Error loading related location Loading Sensitive data returned by an access to PasswordLocation flows to a logging call. |
||
| allSucceeded = false | ||
| anyFailed = true | ||
| } | ||
|
|
@@ -504,7 +504,7 @@ | |
| } | ||
| } | ||
|
|
||
| log.Errorf("Transfer %s failed: %v", transfer.ID, err) | ||
Check failureCode scanning / CodeQL Clear-text logging of sensitive information High Sensitive data returned by an access to S3SecretKeyfile Error loading related location Loading Sensitive data returned by an access to UIPasswordFile Error loading related location Loading Sensitive data returned by an access to PasswordLocation flows to a logging call. |
||
| return err | ||
| } | ||
|
|
||
|
|
@@ -826,10 +826,6 @@ | |
|
|
||
| // Run once on startup after a short delay (use shorter delay for tests) | ||
| startupDelay := 5 * time.Minute | ||
| if tm.maxJobs < 10 { | ||
| // Likely a test environment with low maxJobs, use shorter delay | ||
| startupDelay = 1 * time.Second | ||
| } | ||
|
|
||
| // Use a timer instead of sleep to respect context cancellation | ||
| timer := time.NewTimer(startupDelay) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,203 @@ | ||
| /*************************************************************** | ||
| * | ||
| * Copyright (C) 2026, Pelican Project, Morgridge Institute for Research | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); you | ||
| * may not use this file except in compliance with the License. You may | ||
| * obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| * | ||
| ***************************************************************/ | ||
|
|
||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "encoding/json" | ||
| "fmt" | ||
| "os" | ||
|
|
||
| "github.com/spf13/cobra" | ||
|
|
||
| "github.com/pelicanplatform/pelican/config" | ||
| "github.com/pelicanplatform/pelican/param" | ||
| "github.com/pelicanplatform/pelican/ssh_posixv2" | ||
| ) | ||
|
|
||
| var sshAuthCmd = &cobra.Command{ | ||
| Use: "ssh-auth", | ||
| Short: "SSH authentication tools for the SSH backend", | ||
| Long: `Tools for SSH backend authentication and testing. | ||
|
|
||
| Sub-commands: | ||
| login - Interactive keyboard-interactive authentication via WebSocket | ||
| test - Test SSH connection, binary upload, and helper lifecycle | ||
| status - Check SSH connection status | ||
|
|
||
| For the 'login' and 'status' commands, if --origin is not specified, the command | ||
| will auto-detect the origin URL from the pelican.addresses file (for local | ||
| origins) or from the configuration file. | ||
|
|
||
| Example: | ||
| # Interactive login via WebSocket (auto-detects local origin) | ||
| pelican origin ssh-auth login | ||
|
|
||
| # Interactive login to a specific origin | ||
| pelican origin ssh-auth login --origin https://origin.example.com | ||
|
|
||
| # Check the SSH connection status (auto-detects local origin) | ||
| pelican origin ssh-auth status | ||
|
|
||
| # Test SSH connectivity (similar to ssh command) | ||
| pelican origin ssh-auth test storage.example.com | ||
| pelican origin ssh-auth test pelican@storage.example.com | ||
| pelican origin ssh-auth test pelican@storage.example.com -i ~/.ssh/id_rsa | ||
| `, | ||
| } | ||
|
|
||
| var sshAuthLoginCmd = &cobra.Command{ | ||
| Use: "login", | ||
| Short: "Interactive keyboard-interactive authentication via WebSocket", | ||
| Long: `Connect to an origin's SSH backend via WebSocket to complete | ||
| keyboard-interactive authentication challenges from your terminal. | ||
|
|
||
| This is useful when the origin needs to authenticate to a remote SSH server | ||
| that requires keyboard-interactive authentication (e.g., 2FA, OTP). | ||
|
|
||
| If --origin is not specified, the command will try to determine the origin URL | ||
| from the pelican.addresses file (for local origins) or the configuration. | ||
|
|
||
| Example: | ||
| pelican origin ssh-auth login | ||
| pelican origin ssh-auth login --origin https://origin.example.com | ||
| pelican origin ssh-auth login --origin https://origin.example.com --host storage.internal | ||
| `, | ||
| RunE: runSSHAuthLogin, | ||
| } | ||
|
|
||
| var sshAuthStatusCmd = &cobra.Command{ | ||
| Use: "status", | ||
| Short: "Check SSH connection status of an origin", | ||
| Long: `Query the SSH connection status of an origin's SSH backend. | ||
|
|
||
| If --origin is not specified, the command will try to determine the origin URL | ||
| from the pelican.addresses file (for local origins) or the configuration. | ||
|
|
||
| Example: | ||
| pelican origin ssh-auth status | ||
| pelican origin ssh-auth status --origin https://origin.example.com | ||
| `, | ||
| RunE: runSSHAuthStatus, | ||
| } | ||
|
|
||
| var ( | ||
| sshAuthOrigin string | ||
| sshAuthHost string | ||
| sshAuthToken string | ||
| ) | ||
|
|
||
| func init() { | ||
| // Login command flags | ||
| sshAuthLoginCmd.Flags().StringVar(&sshAuthOrigin, "origin", "", "Origin URL to connect to (auto-detected if not specified)") | ||
| sshAuthLoginCmd.Flags().StringVar(&sshAuthHost, "host", "", "SSH host to authenticate (optional, uses default if not specified)") | ||
| sshAuthLoginCmd.Flags().StringVar(&sshAuthToken, "token", "", "Path to a file containing an admin token (auto-generated if not specified)") | ||
|
|
||
| // Status command uses same origin flag | ||
| sshAuthStatusCmd.Flags().StringVar(&sshAuthOrigin, "origin", "", "Origin URL to check (auto-detected if not specified)") | ||
| sshAuthStatusCmd.Flags().StringVar(&sshAuthToken, "token", "", "Path to a file containing an admin token (auto-generated if not specified)") | ||
|
|
||
| // Add sub-commands | ||
| sshAuthCmd.AddCommand(sshAuthLoginCmd) | ||
| sshAuthCmd.AddCommand(sshAuthStatusCmd) | ||
| } | ||
|
|
||
| // getOriginURL returns the origin URL from the flag, address file, or config | ||
| func getOriginURL() (string, error) { | ||
| // First, check if explicitly provided via flag | ||
| if sshAuthOrigin != "" { | ||
| return sshAuthOrigin, nil | ||
| } | ||
|
|
||
| // Second, try to read from the address file (for local running origins) | ||
| if addrFile, err := config.ReadAddressFile(); err == nil { | ||
| if addrFile.ServerExternalWebURL != "" { | ||
| fmt.Fprintf(os.Stderr, "Using origin URL from address file: %s\n", addrFile.ServerExternalWebURL) | ||
| return addrFile.ServerExternalWebURL, nil | ||
| } | ||
| } | ||
|
|
||
| // Third, try to get from config | ||
| if serverWebUrl := param.Server_ExternalWebUrl.GetString(); serverWebUrl != "" { | ||
| fmt.Fprintf(os.Stderr, "Using origin URL from config: %s\n", serverWebUrl) | ||
| return serverWebUrl, nil | ||
| } | ||
|
|
||
| return "", fmt.Errorf("origin URL not specified and could not be auto-detected; use --origin flag or ensure a local origin is running") | ||
| } | ||
|
|
||
| func runSSHAuthLogin(cmd *cobra.Command, args []string) error { | ||
| ctx := context.Background() | ||
|
|
||
| // Initialize Viper so config/param lookups work (ReadAddressFile, Server_ExternalWebUrl, etc.) | ||
| if err := config.InitClient(); err != nil { | ||
| return fmt.Errorf("failed to initialize client config: %w", err) | ||
| } | ||
|
|
||
| originURL, err := getOriginURL() | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| // Generate or load an admin token for authenticating to the WebSocket endpoint | ||
| tok, err := fetchOrGenerateWebAPIAdminToken(originURL, sshAuthToken) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to obtain admin token: %w", err) | ||
| } | ||
|
|
||
| fmt.Fprintln(os.Stdout, "Starting interactive SSH authentication...") | ||
| fmt.Fprintln(os.Stdout, "Press Ctrl+C to exit.") | ||
| fmt.Fprintln(os.Stdout, "") | ||
|
|
||
| return ssh_posixv2.RunInteractiveAuth(ctx, originURL, sshAuthHost, tok) | ||
| } | ||
|
|
||
| func runSSHAuthStatus(cmd *cobra.Command, args []string) error { | ||
| ctx := context.Background() | ||
|
|
||
| // Initialize Viper so config/param lookups work (ReadAddressFile, Server_ExternalWebUrl, etc.) | ||
| if err := config.InitClient(); err != nil { | ||
| return fmt.Errorf("failed to initialize client config: %w", err) | ||
| } | ||
|
|
||
| originURL, err := getOriginURL() | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| // Generate or load an admin token for authenticating to the status endpoint | ||
| tok, err := fetchOrGenerateWebAPIAdminToken(originURL, sshAuthToken) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to obtain admin token: %w", err) | ||
| } | ||
|
|
||
| status, err := ssh_posixv2.GetConnectionStatus(ctx, originURL, tok) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get status: %w", err) | ||
| } | ||
|
|
||
| // Pretty print the status | ||
| output, err := json.MarshalIndent(status, "", " ") | ||
| if err != nil { | ||
| return fmt.Errorf("failed to format status: %w", err) | ||
| } | ||
|
|
||
| fmt.Println(string(output)) | ||
| return nil | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both
runSSHAuthLoginandrunSSHAuthStatussuffer from a problem where they rely on this function to return the origin's URL, the problem being that there's no guarantee that Viper has been configured, which these functions rely on (indirectly).Or in the words of Copilot:
Emperically, Copilot is not entirely wrong. Where I disagree with it: I think
InitServeris more appropriate.