Skip to content

Conversation

@grondo
Copy link
Contributor

@grondo grondo commented Nov 14, 2025

This PR adds an attach RPC to the subprocess server protocol, which allows a client to attach to an existing background subprocess and convert it to a streaming subprocess.

This should be considered a first cut - happy to take feedback at this point.

Key design decisions:

  • The attach request includes a flags parameter for future extensibility, but no flags are currently supported. Output forwarding behavior is inherited from the original subprocess.

  • An "exec attached" response is sent first to indicate success and communicate the subprocess flags to the client, followed by the same response types used for streaming exec requests.

  • If a waitable subprocess has already terminated, the full response sequence is still sent (attached->output EOFs->finished->ENODATA).

  • A successful attach that receives the finished response reaps the subprocess, preventing it from being waited on or attached to again.

  • If the client disconnects, the subprocess reverts to background mode rather than being terminated.

Error conditions:

  • ENOENT: subprocess doesn't exist or has exited without waitable flag
  • EBUSY: subprocess is not in background mode
  • EPERM: client lacks authorization

@github-actions
Copy link

⚠️ linkcheck failed with status code 2

1 similar comment
@github-actions
Copy link

⚠️ linkcheck failed with status code 2

Copy link
Member

@garlick garlick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one phrase I didnt quite understand.

Also just a thought for later on, not a review comment: once attach is supported after an instance restart, we'll want to think about how to handle EOF and stdin writes if we can't reattach to the stdio streams.

spec_42.rst Outdated
Comment on lines 267 to 268
The :program:`attach` RPC attaches to an existing background subprocess,
converting it to a streaming subprocess.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is meant by "converting it to a streaming subprocess"?

Copy link
Contributor Author

@grondo grondo Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This refers to the "Streaming request: " noted in the exec RPC definition (as opposed to non-streaming request which creates a "background" process)

Streaming request: When initiated with a streaming request, responses
(defined below) SHALL be sent until the process terminates and all output
has been delivered. If the requesting process disconnects, the subprocess
SHALL be terminated.

Probably need a better term here. Also, the behavior defined in the additions in this PR leave the attached process running as opposed to killing it on disconnect (seemed like the correct behavior since the subprocess will still have the background flag, and for our intended future use). Happy to take suggestions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm 🤔 Well, since the process still behaves as a background process when the attach terminates, it seems like the subprocess isn't really converted per se.

Maybe that part of the sentence ("converting it to a streaming subprocess") should just be dropped?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, less words is better.

@garlick
Copy link
Member

garlick commented Dec 10, 2025

Is there a reason this wasn't merged?

@grondo
Copy link
Contributor Author

grondo commented Dec 10, 2025

Oh, we could merge it. I was just waiting to work on a partial implementation to make sure there were no necessary changes. But changes could just be made in a future PR.

Add an attach RPC to the subprocess server protocol, which allows
a client to attach to an existing background subprocess.

Key design decisions:

- The attach request includes a flags parameter for future
  extensibility, but no flags are currently supported. Output forwarding
  behavior is inherited from the original subprocess.

- An "exec attached" response is sent first to indicate success and
  communicate the subprocess flags to the client, followed by the same
  response types used for streaming exec requests.

- If a waitable subprocess has already terminated, the full response
  sequence is still sent (attached->output EOFs->finished->ENODATA).

- A successful attach that receives the finished response reaps the
  subprocess, preventing it from being waited on or attached to again.

- If the client disconnects, the subprocess reverts to background mode
  rather than being terminated.

Error conditions:
- ENOENT: subprocess doesn't exist or has exited without waitable flag
- EBUSY: subprocess is not in background mode
- EPERM: client lacks authorization
@mergify mergify bot added the queued label Dec 10, 2025
@mergify mergify bot merged commit 1cc73a7 into flux-framework:master Dec 10, 2025
7 checks passed
@mergify
Copy link
Contributor

mergify bot commented Dec 10, 2025

Merge Queue Status

✅ The pull request has been merged at 79f83bb

This pull request spent 5 seconds in the queue, with no time running CI.
The checks were run in-place.

Required conditions to merge
  • any of [🛡 GitHub branch protection]:
    • check-success = docs/readthedocs.org:flux-rfc
    • check-neutral = docs/readthedocs.org:flux-rfc
    • check-skipped = docs/readthedocs.org:flux-rfc
  • any of [🛡 GitHub branch protection]:
    • check-success = make check
    • check-neutral = make check
    • check-skipped = make check
  • any of [🛡 GitHub branch protection]:
    • check-success = validate commits
    • check-neutral = validate commits
    • check-skipped = validate commits

@mergify mergify bot removed the queued label Dec 10, 2025
@github-actions
Copy link

⚠️ linkcheck failed with status code 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants