⚡️ Speed up function _get_router_metadata_variable_name by 137%

[codeflash-ai]

📄 137% (1.37x) speedup for _get_router_metadata_variable_name in litellm/router_utils/batch_utils.py

⏱️ Runtime : 2.48 milliseconds → 1.05 milliseconds (best of 17 runs)

📝 Explanation and details

The optimized code achieves a 136% speedup through two key optimizations:

1. Eliminated Repeated Set Construction
The original code recreates the same set of 5 method names on every function call (lines showing 13.3% + 5.8% = 19.1% of total time). The optimization moves this to a module-level frozenset, creating it only once when the module loads.

2. Replaced any() Generator with Early-Return Loop
The original code uses any(method in function_name for method in ROUTER_METHODS...) which creates a generator and has function call overhead (72.2% of total time). The optimization uses a simple for-loop that returns immediately upon finding the first match, eliminating generator overhead.

Performance Benefits by Test Case:

• Short method names like "file" see the largest gains (230-240% faster) because the loop exits quickly on the first iteration
• Non-matching strings benefit significantly (100-150% faster) as the loop efficiently checks all 5 methods without generator overhead
• Long strings with matches still see good improvements (100-110% faster) due to early termination
• None/empty inputs see substantial gains (120-190% faster) by avoiding set construction entirely

The optimization is particularly effective because the method set is small (5 items), making the linear search very fast while eliminating the overhead of set construction and generator expressions that dominated the original performance profile.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	✅ 83 Passed
🌀 Generated Regression Tests	✅ 3058 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	✅ 2 Passed
📊 Tests Coverage	100.0%

⚙️ Existing Unit Tests and Runtime

Test File::Test Function	Original ⏱️	Optimized ⏱️	Speedup
router_unit_tests/test_router_batch_utils.py::test_router_metadata_variable_name	7.68μs	3.74μs	105%✅
router_unit_tests/test_router_helper_utils.py::test_handle_clientside_credential_metadata_variable_name	7.28μs	3.73μs	95.0%✅

🌀 Generated Regression Tests and Runtime

from typing import Optional

# imports
import pytest  # used for our unit tests
from litellm.router_utils.batch_utils import _get_router_metadata_variable_name

# unit tests

# 1. Basic Test Cases

def test_basic_batch_function_name():
    # "batch" is in the set, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("batch") # 2.90μs -> 1.26μs (130% faster)

def test_basic_generic_api_call_function_name():
    # "generic_api_call" is in the set, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("generic_api_call") # 2.47μs -> 1.04μs (137% faster)

def test_basic_file_function_name():
    # "file" is in the set, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("file") # 3.04μs -> 920ns (230% faster)

def test_basic_other_function_name():
    # "other_function" is not in the set, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name("other_function") # 2.69μs -> 1.30μs (108% faster)

def test_basic_none_function_name():
    # None passed, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name(None) # 1.19μs -> 539ns (120% faster)

def test_basic_empty_string_function_name():
    # Empty string passed, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name("") # 1.32μs -> 526ns (150% faster)

# 2. Edge Test Cases

def test_edge_case_partial_match():
    # "batching" contains "batch", so should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("batching") # 2.90μs -> 1.26μs (130% faster)

def test_edge_case_substring_not_in_methods():
    # "bat" is a substring of "batch" but not in the set, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name("bat") # 2.27μs -> 1.12μs (103% faster)

def test_edge_case_multiple_methods_in_name():
    # "batch_generic_api_call_file" contains multiple methods, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("batch_generic_api_call_file") # 2.44μs -> 932ns (162% faster)

def test_edge_case_method_at_end():
    # "my_method_is_batch" contains "batch" at the end, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("my_method_is_batch") # 2.95μs -> 1.42μs (107% faster)

def test_edge_case_method_at_start():
    # "batch_and_more" contains "batch" at the start, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("batch_and_more") # 2.82μs -> 1.32μs (113% faster)

def test_edge_case_case_sensitivity():
    # "Batch" with capital B is not in the set (case-sensitive), should return "metadata"
    codeflash_output = _get_router_metadata_variable_name("Batch") # 2.61μs -> 1.21μs (115% faster)

def test_edge_case_function_name_is_space():
    # " " (space) is not in the set, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name(" ") # 2.39μs -> 1.09μs (120% faster)

def test_edge_case_function_name_is_numeric():
    # "12345" is not in the set, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name("12345") # 2.35μs -> 1.19μs (98.3% faster)

def test_edge_case_function_name_is_special_chars():
    # "!@#$%" is not in the set, should return "metadata"
    codeflash_output = _get_router_metadata_variable_name("!@#$%") # 2.35μs -> 1.15μs (104% faster)

def test_edge_case_function_name_is_method_name_with_extra_spaces():
    # " batch " contains "batch", should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name(" batch ") # 2.76μs -> 1.31μs (111% faster)

def test_edge_case_function_name_is_method_name_embedded():
    # "this_is_a_generic_api_call_method" contains "generic_api_call", should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("this_is_a_generic_api_call_method") # 2.72μs -> 1.20μs (127% faster)

def test_edge_case_function_name_is_method_name_with_underscore():
    # "_acreate_batch" is in the set, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("_acreate_batch") # 2.68μs -> 1.15μs (133% faster)

def test_edge_case_function_name_is_method_name_with_fallbacks():
    # "_ageneric_api_call_with_fallbacks" is in the set, should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("_ageneric_api_call_with_fallbacks") # 2.61μs -> 1.17μs (122% faster)

def test_edge_case_function_name_is_method_name_with_extra_text():
    # "file_upload" contains "file", should return "litellm_metadata"
    codeflash_output = _get_router_metadata_variable_name("file_upload") # 2.77μs -> 863ns (220% faster)

# 3. Large Scale Test Cases

def test_large_scale_many_non_matching_names():
    # Test with 1000 function names not matching any method
    for i in range(1000):
        fn_name = f"function_{i}"
        codeflash_output = _get_router_metadata_variable_name(fn_name) # 765μs -> 317μs (141% faster)

def test_large_scale_many_matching_names():
    # Test with 1000 function names, all containing "batch"
    for i in range(1000):
        fn_name = f"my_batch_function_{i}"
        codeflash_output = _get_router_metadata_variable_name(fn_name) # 834μs -> 349μs (139% faster)

def test_large_scale_mixed_names():
    # Test with 500 matching and 500 non-matching names
    for i in range(500):
        codeflash_output = _get_router_metadata_variable_name(f"generic_api_call_{i}") # 334μs -> 136μs (145% faster)
    for i in range(500, 1000):
        codeflash_output = _get_router_metadata_variable_name(f"random_function_{i}") # 394μs -> 177μs (122% faster)

def test_large_scale_long_function_name():
    # Test with a very long function name containing a method name
    long_name = "x" * 400 + "batch" + "y" * 400
    codeflash_output = _get_router_metadata_variable_name(long_name) # 4.10μs -> 1.98μs (107% faster)

def test_large_scale_long_function_name_no_match():
    # Test with a very long function name not containing any method name
    long_name = "x" * 800
    codeflash_output = _get_router_metadata_variable_name(long_name) # 3.19μs -> 1.74μs (83.7% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
from typing import Optional

# imports
import pytest  # used for our unit tests
from litellm.router_utils.batch_utils import _get_router_metadata_variable_name

# unit tests

# ---------------------------
# 1. Basic Test Cases
# ---------------------------

def test_basic_batch_name():
    # function_name contains "batch"
    codeflash_output = _get_router_metadata_variable_name("batch") # 2.90μs -> 1.23μs (137% faster)

def test_basic_generic_api_call_name():
    # function_name contains "generic_api_call"
    codeflash_output = _get_router_metadata_variable_name("generic_api_call") # 2.60μs -> 1.10μs (137% faster)

def test_basic_file_name():
    # function_name contains "file"
    codeflash_output = _get_router_metadata_variable_name("file") # 2.88μs -> 845ns (241% faster)

def test_basic_ageneric_api_call_with_fallbacks_name():
    # function_name contains "_ageneric_api_call_with_fallbacks"
    codeflash_output = _get_router_metadata_variable_name("_ageneric_api_call_with_fallbacks") # 2.61μs -> 1.17μs (123% faster)

def test_basic_acreate_batch_name():
    # function_name contains "_acreate_batch"
    codeflash_output = _get_router_metadata_variable_name("_acreate_batch") # 2.94μs -> 1.23μs (139% faster)

def test_basic_other_name():
    # function_name does NOT contain any special method
    codeflash_output = _get_router_metadata_variable_name("create_thread") # 2.54μs -> 1.21μs (110% faster)

def test_basic_none_name():
    # function_name is None
    codeflash_output = _get_router_metadata_variable_name(None) # 1.17μs -> 446ns (162% faster)

def test_basic_empty_string():
    # function_name is empty string
    codeflash_output = _get_router_metadata_variable_name("") # 1.27μs -> 436ns (192% faster)

def test_basic_unrelated_string():
    # function_name is unrelated string
    codeflash_output = _get_router_metadata_variable_name("foobar") # 2.88μs -> 1.18μs (144% faster)

def test_basic_partial_match():
    # function_name contains "batch" as substring
    codeflash_output = _get_router_metadata_variable_name("my_batch_processor") # 3.14μs -> 1.36μs (130% faster)

def test_basic_case_sensitive():
    # function_name contains "BATCH" (upper case) - should NOT match
    codeflash_output = _get_router_metadata_variable_name("BATCH") # 2.46μs -> 1.19μs (107% faster)

def test_basic_substring_overlap():
    # function_name contains "batching" - should match since "batch" is substring
    codeflash_output = _get_router_metadata_variable_name("batching") # 2.82μs -> 1.17μs (141% faster)

def test_basic_multiple_methods_in_name():
    # function_name contains multiple method substrings
    codeflash_output = _get_router_metadata_variable_name("batch_generic_api_call_file") # 2.45μs -> 917ns (167% faster)

# ---------------------------
# 2. Edge Test Cases
# ---------------------------

def test_edge_leading_trailing_spaces():
    # function_name has leading/trailing spaces
    codeflash_output = _get_router_metadata_variable_name(" batch ") # 2.80μs -> 1.17μs (140% faster)

def test_edge_special_characters():
    # function_name contains method with special chars around
    codeflash_output = _get_router_metadata_variable_name("file!") # 2.20μs -> 697ns (215% faster)
    codeflash_output = _get_router_metadata_variable_name("!file") # 974ns -> 399ns (144% faster)

def test_edge_method_at_start_end():
    # function_name starts/ends with method substring
    codeflash_output = _get_router_metadata_variable_name("batch_end") # 2.87μs -> 1.24μs (131% faster)
    codeflash_output = _get_router_metadata_variable_name("start_generic_api_call") # 1.21μs -> 687ns (76.1% faster)

def test_edge_method_in_middle():
    # function_name has method substring in the middle
    codeflash_output = _get_router_metadata_variable_name("foo_batch_bar") # 2.92μs -> 1.22μs (140% faster)

def test_edge_method_embedded():
    # function_name has method substring embedded in a word
    codeflash_output = _get_router_metadata_variable_name("superfileprocessor") # 2.77μs -> 806ns (244% faster)

def test_edge_numeric_in_name():
    # function_name contains numbers
    codeflash_output = _get_router_metadata_variable_name("batch123") # 2.56μs -> 1.12μs (128% faster)
    codeflash_output = _get_router_metadata_variable_name("123file456") # 1.18μs -> 430ns (174% faster)

def test_edge_similar_but_not_matching():
    # function_name contains similar but not matching substrings
    codeflash_output = _get_router_metadata_variable_name("batched") # 2.25μs -> 1.11μs (102% faster)
    codeflash_output = _get_router_metadata_variable_name("files") # 1.17μs -> 374ns (214% faster)
    codeflash_output = _get_router_metadata_variable_name("generic_api_calls") # 809ns -> 492ns (64.4% faster)
    codeflash_output = _get_router_metadata_variable_name("acreate_batching") # 1.11μs -> 605ns (83.6% faster)

def test_edge_unicode_characters():
    # function_name contains unicode characters
    codeflash_output = _get_router_metadata_variable_name("batch🚀") # 3.10μs -> 1.60μs (93.3% faster)
    codeflash_output = _get_router_metadata_variable_name("文件file") # 1.39μs -> 637ns (118% faster)
    codeflash_output = _get_router_metadata_variable_name("файлfile") # 972ns -> 336ns (189% faster)
    codeflash_output = _get_router_metadata_variable_name("generic_api_call💡") # 923ns -> 651ns (41.8% faster)

def test_edge_long_string_with_method():
    # function_name is a long string containing a method substring
    long_name = "x" * 100 + "batch" + "y" * 100
    codeflash_output = _get_router_metadata_variable_name(long_name) # 3.31μs -> 1.58μs (109% faster)

def test_edge_long_string_without_method():
    # function_name is a long string NOT containing any method substring
    long_name = "x" * 200
    codeflash_output = _get_router_metadata_variable_name(long_name) # 2.66μs -> 1.32μs (102% faster)

def test_edge_method_substring_overlap():
    # function_name contains overlapping method substrings
    codeflash_output = _get_router_metadata_variable_name("batchfile") # 2.71μs -> 865ns (213% faster)
    codeflash_output = _get_router_metadata_variable_name("filebatch") # 1.14μs -> 423ns (169% faster)

def test_edge_only_method_substring():
    # function_name is exactly one of the method substrings
    codeflash_output = _get_router_metadata_variable_name("batch") # 2.43μs -> 1.06μs (130% faster)
    codeflash_output = _get_router_metadata_variable_name("file") # 1.14μs -> 406ns (182% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
from litellm.router_utils.batch_utils import _get_router_metadata_variable_name

def test__get_router_metadata_variable_name():
    _get_router_metadata_variable_name('\x00')

def test__get_router_metadata_variable_name_2():
    _get_router_metadata_variable_name('file')

🔎 Concolic Coverage Tests and Runtime

Test File::Test Function	Original ⏱️	Optimized ⏱️	Speedup
codeflash_concolic_ualnrfea/tmpudh2n0ld/test_concolic_coverage.py::test__get_router_metadata_variable_name	2.79μs	1.14μs	145%✅
codeflash_concolic_ualnrfea/tmpudh2n0ld/test_concolic_coverage.py::test__get_router_metadata_variable_name_2	2.98μs	944ns	215%✅

To edit these changes git checkout codeflash/optimize-_get_router_metadata_variable_name-mh2n74iw and push.