fix(proxy): avoid in-place mutation in SpendUpdateQueue aggregation

[emerzon]

Summary

This PR fixes in-place mutation in SpendUpdateQueue._get_aggregated_spend_update_queue_item().

Previously, aggregation stored a direct reference to the first SpendUpdateQueueItem seen for a key and then mutated that dict when combining costs. If any caller retained a reference to the original update object, its response_cost would be overwritten during queue aggregation.

Changes

• In litellm/proxy/db/db_transaction_queue/spend_update_queue.py:
  • Copy the first update per entity key (update.copy()) before storing it in the aggregation map.
  • Continue aggregating into the copied object, preserving original caller-owned dicts.
• In tests/test_litellm/proxy/db/db_transaction_queue/test_spend_update_queue.py:
  • Added regression test test_get_aggregated_spend_update_queue_item_does_not_mutate_original_updates.
  • Verifies:
    • original input update remains unchanged (10.0)
    • aggregated result is correct (30.0)
    • aggregated object is not the same instance as the original input.

Validation

• poetry run pytest -q tests/test_litellm/proxy/db/db_transaction_queue/test_spend_update_queue.py
  • Result: 11 passed

Closes #20875

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 10, 2026 4:49pm

[Copilot]

Pull request overview

This PR fixes a critical bug in the SpendUpdateQueue aggregation logic where the _get_aggregated_spend_update_queue_item() method was modifying original dictionary objects in place. The bug occurred because when multiple spend updates with the same entity key were aggregated, the code stored a direct reference to the first update's dictionary and then mutated it during aggregation, corrupting the original caller-owned data.

Changes:

• Fixed in-place mutation by creating a defensive copy of the first update dictionary before storing it in the aggregation map
• Added a regression test to verify that original update dictionaries remain unchanged after aggregation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
litellm/proxy/db/db_transaction_queue/spend_update_queue.py	Added .copy() call on line 113 to create a defensive copy of update dictionaries before storing them in the aggregation map, preventing in-place mutation of caller-owned objects
tests/test_litellm/proxy/db/db_transaction_queue/test_spend_update_queue.py	Added regression test test_get_aggregated_spend_update_queue_item_does_not_mutate_original_updates that verifies original updates remain unchanged, aggregation is correct, and no object identity is shared

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Consider also verifying that duplicate_key_update is not mutated to make the test more comprehensive. While the current implementation doesn't mutate it (only the copied first update is mutated), adding an assertion like assert duplicate_key_update["response_cost"] == 20.0 would make the test more robust against future refactoring.

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes a correctness bug in SpendUpdateQueue._get_aggregated_spend_update_queue_item() where aggregation previously retained a reference to the first SpendUpdateQueueItem for a given entity key and then mutated it while summing response_cost. The implementation now stores update.copy() as the first aggregated value per key, preventing caller-owned dicts from being modified during queue compaction.

A regression test was added to ensure the original update object remains unchanged, the aggregated total is correct, and the aggregated item is a different instance.

Confidence Score: 4/5

• This PR is largely safe to merge once the new regression test is made robust against aggregation output ordering.
• Core code change is minimal and directly addresses the mutation bug by copying the first aggregated item. The only blocking issue found is an order-dependent assertion in the new test, which can become flaky if aggregation ordering changes.
• tests/test_litellm/proxy/db/db_transaction_queue/test_spend_update_queue.py

Important Files Changed

Filename	Overview
litellm/proxy/db/db_transaction_queue/spend_update_queue.py	Fixes in-place mutation during queue aggregation by copying the first seen update dict before summing response_cost; prevents caller-owned dicts from being modified.
tests/test_litellm/proxy/db/db_transaction_queue/test_spend_update_queue.py	Adds regression test ensuring _get_aggregated_spend_update_queue_item() doesn’t mutate original updates; test currently assumes deterministic ordering of aggregated list output.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant SpendUpdateQueue
    participant InMemMap as AggregationMap
    participant AggList as AggregatedList

    Caller->>SpendUpdateQueue: add_update(update)
    SpendUpdateQueue->>SpendUpdateQueue: update_queue.put(update)

    Caller->>SpendUpdateQueue: aggregate_queue_updates()
    SpendUpdateQueue->>SpendUpdateQueue: flush_all_updates_from_in_memory_queue()
    SpendUpdateQueue->>SpendUpdateQueue: _get_aggregated_spend_update_queue_item(updates)

    loop each update
        SpendUpdateQueue->>InMemMap: compute aggregation key
        alt first update for key
            SpendUpdateQueue->>InMemMap: store shallow copy of update
        else later update for key
            SpendUpdateQueue->>InMemMap: increment stored response_cost
        end
    end

    SpendUpdateQueue->>AggList: append values from map
    SpendUpdateQueue-->>Caller: return aggregated list

    Caller->>SpendUpdateQueue: get_aggregated_db_spend_update_transactions(aggregated)
    SpendUpdateQueue-->>Caller: DBSpendUpdateTransactions

Loading

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[emerzon]

@greptile

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes a correctness issue in spend update queue aggregation where the first SpendUpdateQueueItem for a given (entity_type, entity_id) key was stored by reference and then mutated while combining costs. The fix stores update.copy() for the first-seen item per key, so aggregation mutates only the copied dict while preserving caller-owned input objects. A regression unit test is added to ensure the original update remains unchanged, the aggregated cost is correct, and the aggregated dict is a different instance; the test selects the aggregated entry by key to avoid order dependence.

Change is localized to the proxy’s in-memory spend update batching (litellm/proxy/db/db_transaction_queue/spend_update_queue.py) and its unit tests.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk.
• Change is narrowly scoped (one-line behavioral fix) and aligns with the documented bug; SpendUpdateQueueItem is a flat TypedDict, so a shallow copy correctly prevents caller object mutation without altering aggregation semantics. Test coverage includes an explicit regression assertion for non-mutation and correct aggregation, and the updated test avoids order-dependent assumptions.
• No files require special attention

Important Files Changed

Filename	Overview
litellm/proxy/db/db_transaction_queue/spend_update_queue.py	Fixes in-place mutation during spend update aggregation by storing a shallow copy of the first update per entity key before summing response_cost.
tests/test_litellm/proxy/db/db_transaction_queue/test_spend_update_queue.py	Adds regression test ensuring _get_aggregated_spend_update_queue_item does not mutate caller-owned updates; assertions avoid order-dependence by selecting aggregated entry by key.

Sequence Diagram

sequenceDiagram
    participant Caller as Proxy code (caller)
    participant SUQ as SpendUpdateQueue
    participant AQ as asyncio.Queue

    Caller->>SUQ: add_update(update)
    SUQ->>AQ: put(update)
    alt queue size >= MAX_SIZE_IN_MEMORY_QUEUE
        SUQ->>SUQ: aggregate_queue_updates()
        SUQ->>SUQ: flush_all_updates_from_in_memory_queue()
        SUQ->>SUQ: _get_aggregated_spend_update_queue_item(updates)
        note over SUQ: For each entity key:
        note over SUQ: store update.copy() for first seen
        note over SUQ: then sum response_cost into copy
        SUQ-->>SUQ: aggregated_updates
        loop each aggregated update
            SUQ->>AQ: put(aggregated_update)
        end
    end

    Caller->>SUQ: flush_and_get_aggregated_db_spend_update_transactions()
    SUQ->>SUQ: flush_all_updates_from_in_memory_queue()
    SUQ->>SUQ: get_aggregated_db_spend_update_transactions(updates)
    SUQ-->>Caller: DBSpendUpdateTransactions

Loading

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}