-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
feat: keep prompts only in s3 bucket aws #20877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile OverviewGreptile SummaryThis PR adds an The feature is exercised by new/updated tests: a new legacy S3 test that asserts only prompt messages are uploaded, and a new Main issue to address before merge: in the legacy Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/integrations/s3.py | Adds s3_log_prompts_only flag and strips S3 payload down to messages when enabled; however the flag is only read from litellm.s3_callback_params and is ignored for direct constructor usage. |
| litellm/integrations/s3_v2.py | Plumbs s3_log_prompts_only through init params and create_s3_batch_logging_element() to log only messages to S3; also refactors s3_verify/s3_use_ssl param handling and minor formatting. |
| tests/test_litellm/integrations/test_s3.py | New unit test verifies legacy s3 logger writes only {messages: ...} to S3 when s3_log_prompts_only is enabled via litellm.s3_callback_params. |
| tests/test_litellm/integrations/test_s3_v2.py | Adds prompts-only payload test for s3_v2 logger and various formatting tweaks; existing tests still rely on patching asyncio.create_task to avoid background flush tasks. |
Sequence Diagram
sequenceDiagram
participant App as LiteLLM
participant S3 as S3Logger (legacy)
participant S3v2 as S3Logger (v2)
participant AWS as S3-compatible endpoint
App->>S3: log_event(kwargs, start_time, end_time)
S3->>S3: read litellm.s3_callback_params
alt s3_log_prompts_only enabled
S3->>S3: payload_dict = {messages}
else disabled
S3->>S3: payload_dict = full StandardLoggingPayload
end
S3->>AWS: put_object(Body=safe_dumps(payload_dict))
App->>S3v2: async_log_success_event(kwargs, start_time, end_time)
S3v2->>S3v2: _init_s3_params() (reads callback params + ctor args)
S3v2->>S3v2: create_s3_batch_logging_element()
alt s3_log_prompts_only enabled
S3v2->>S3v2: element.payload = {messages}
else disabled
S3v2->>S3v2: element.payload = full StandardLoggingPayload
end
S3v2->>S3v2: queue element for batch flush
S3v2->>AWS: PUT signed request (async_upload_data_to_s3)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
4 files reviewed, 1 comment
Additional Comments (1)
|
Relevant issues
LIT-1899
Pre-Submission checklist
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
✅ Test
Changes
s3_log_prompts_onlybehavior to store only prompt messages (no response/metadata) in S3 logs.s3_v2and legacys3loggers.