Skip to content

Conversation

@ac-mmi
Copy link

@ac-mmi ac-mmi commented Feb 9, 2025

This PR addresses an issue where streaming responses from the OpenAI API were not closing properly, leading to lingering AgentKeepAlive connections that prevented graceful shutdowns of Node.js processes.

Changes Made:

  1. Modified _iterSSEMessages(): Explicitly close the stream by calling response.body.cancel() in the finally block. This ensures that the body is properly cleaned up, even when the stream completes successfully, not just on errors.
  2. Updated the finally block in fromSSEResponse(): The controller is now aborted only if the stream is not already finished. This prevents redundant abortion calls and ensures smooth handling during normal stream completion.
  3. Added explicit cancellation of the ReadableStream reader in the finally block to ensure the stream is properly closed when the response body is no longer needed. This prevents AgentKeepAlive from holding onto connections unnecessarily.

Fixes #1311

@ac-mmi ac-mmi requested a review from a team as a code owner February 9, 2025 17:34
@RobertCraigie
Copy link
Collaborator

Thanks for the PR! As discussed in the issue thread this was because of an incorrect openai/shims import so I don't think this is necessary and I'd rather avoid adding this extra complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI client prevents process from gracefully terminating

2 participants