Skip to content

⚡️ Speed up function qwen_model_profile by 393% #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: try-refinement
Choose a base branch
from

Conversation

codeflash-ai[bot]
Copy link

@codeflash-ai codeflash-ai bot commented Jul 22, 2025

📄 393% (3.93x) speedup for qwen_model_profile in pydantic_ai_slim/pydantic_ai/profiles/qwen.py

⏱️ Runtime : 1.31 milliseconds 265 microseconds (best of 125 runs)

📝 Explanation and details

REFINEMENT Here’s an optimized version of your program. The main inefficiency is that you re-import and re-reference InlineDefsJsonSchemaTransformer each time you call the function, even though it is constant. By caching the constructed ModelProfile object, we can avoid redundant instantiations and speed up execution.

Explanation:

  • This avoids reconstructing the ModelProfile object on every call, saving both time and memory, and removes repeated lookups.
  • The function will return the exact same result as before.
  • Faster, especially if called many times.

If you need the returned ModelProfile to be unique for each call (e.g., if it's mutable), you can’t cache it; but with the code as written, there’s no use of model_name and the transformer is constant, so this optimization is safe.

Correctness verification report:

Test Status
⚙️ Existing Unit Tests 21 Passed
🌀 Generated Regression Tests 6429 Passed
⏪ Replay Tests 🔘 None Found
🔎 Concolic Coverage Tests 1 Passed
📊 Tests Coverage 100.0%
⚙️ Existing Unit Tests and Runtime
Test File::Test Function Original ⏱️ Optimized ⏱️ Speedup
codeflash_concolic_r85i594x/tmp7o6q195b/test_concolic_coverage.py::test_qwen_model_profile 458ns 83ns ✅452%
🌀 Generated Regression Tests and Runtime
import pytest  # used for our unit tests
# function to test
from pydantic_ai.profiles import ModelProfile
from pydantic_ai.profiles._json_schema import InlineDefsJsonSchemaTransformer
from pydantic_ai.profiles.qwen import qwen_model_profile

# unit tests

# 1. BASIC TEST CASES

def test_basic_valid_string_returns_model_profile():
    """Test that a typical Qwen model name returns a ModelProfile with correct transformer."""
    codeflash_output = qwen_model_profile("Qwen-7B-Chat"); profile = codeflash_output # 459ns -> 84ns (446% faster)

def test_basic_another_string_returns_model_profile():
    """Test that another valid string returns a ModelProfile."""
    codeflash_output = qwen_model_profile("Qwen-14B"); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_basic_empty_string_returns_model_profile():
    """Test that an empty string is still treated as a valid model name."""
    codeflash_output = qwen_model_profile(""); profile = codeflash_output # 458ns -> 84ns (445% faster)

def test_basic_whitespace_string_returns_model_profile():
    """Test that a whitespace-only string is treated as valid."""
    codeflash_output = qwen_model_profile("   "); profile = codeflash_output # 416ns -> 83ns (401% faster)

# 2. EDGE TEST CASES

def test_edge_none_input_returns_none():
    """Test that None input returns None."""
    codeflash_output = qwen_model_profile(None); profile = codeflash_output # 458ns -> 84ns (445% faster)

def test_edge_integer_input_returns_none():
    """Test that integer input returns None."""
    codeflash_output = qwen_model_profile(123); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_edge_float_input_returns_none():
    """Test that float input returns None."""
    codeflash_output = qwen_model_profile(1.23); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_edge_list_input_returns_none():
    """Test that list input returns None."""
    codeflash_output = qwen_model_profile(["Qwen-7B-Chat"]); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_edge_dict_input_returns_none():
    """Test that dict input returns None."""
    codeflash_output = qwen_model_profile({"model": "Qwen-7B-Chat"}); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_edge_bool_input_returns_none():
    """Test that boolean input returns None."""
    codeflash_output = qwen_model_profile(True); profile = codeflash_output # 458ns -> 83ns (452% faster)

def test_edge_bytes_input_returns_none():
    """Test that bytes input returns None."""
    codeflash_output = qwen_model_profile(b"Qwen-7B-Chat"); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_edge_object_input_returns_none():
    """Test that arbitrary object input returns None."""
    class Dummy: pass
    codeflash_output = qwen_model_profile(Dummy()); profile = codeflash_output # 417ns -> 83ns (402% faster)

def test_edge_unicode_string_returns_model_profile():
    """Test that a unicode string is accepted."""
    codeflash_output = qwen_model_profile("Qwēn-模型"); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_edge_long_string_returns_model_profile():
    """Test that a very long string is accepted."""
    long_name = "Qwen-" + "X" * 500
    codeflash_output = qwen_model_profile(long_name); profile = codeflash_output # 417ns -> 83ns (402% faster)

# 3. LARGE SCALE TEST CASES

def test_large_scale_many_unique_names():
    """Test that the function can handle a large number of unique model names."""
    # Generate 1000 unique names
    names = [f"Qwen-{i}" for i in range(1000)]
    for name in names:
        codeflash_output = qwen_model_profile(name); profile = codeflash_output # 200μs -> 41.1μs (389% faster)

def test_large_scale_identical_names():
    """Test that repeated calls with the same name return valid profiles."""
    name = "Qwen-7B-Chat"
    for _ in range(1000):
        codeflash_output = qwen_model_profile(name); profile = codeflash_output # 202μs -> 41.1μs (392% faster)

def test_large_scale_mixed_types():
    """Test a large batch of mixed valid and invalid input types."""
    valid_names = [f"Qwen-{i}" for i in range(500)]
    invalid_inputs = [None, 123, 1.23, [], {}, True, b"abc", object()] * 50  # 400 elements
    mixed_inputs = valid_names + invalid_inputs
    for inp in mixed_inputs:
        codeflash_output = qwen_model_profile(inp); profile = codeflash_output # 184μs -> 37.0μs (397% faster)
        if isinstance(inp, str):
            pass
        else:
            pass

def test_large_scale_extremely_long_string():
    """Test with a string of near-maximum allowed length (within practical test limits)."""
    very_long_name = "Qwen-" + "Y" * 999
    codeflash_output = qwen_model_profile(very_long_name); profile = codeflash_output # 417ns -> 83ns (402% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

import pytest  # used for our unit tests
# function to test
from pydantic_ai.profiles import ModelProfile
from pydantic_ai.profiles._json_schema import InlineDefsJsonSchemaTransformer
from pydantic_ai.profiles.qwen import qwen_model_profile

# unit tests

# ---------------------------
# BASIC TEST CASES
# ---------------------------

def test_qwen_model_profile_basic_qwen_model():
    """Test with a typical Qwen model name."""
    codeflash_output = qwen_model_profile("qwen-7b"); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_qwen_model_profile_basic_qwen_uppercase():
    """Test with Qwen model name in uppercase."""
    codeflash_output = qwen_model_profile("QWEN-14B"); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_qwen_model_profile_basic_non_qwen_model():
    """Test with a non-Qwen model name."""
    codeflash_output = qwen_model_profile("gpt-3"); profile = codeflash_output # 375ns -> 83ns (352% faster)

def test_qwen_model_profile_basic_empty_string():
    """Test with an empty string."""
    codeflash_output = qwen_model_profile(""); profile = codeflash_output # 417ns -> 83ns (402% faster)

def test_qwen_model_profile_basic_trailing_spaces():
    """Test with trailing and leading spaces in model name."""
    codeflash_output = qwen_model_profile("  qwen-13b  "); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_qwen_model_profile_basic_qwen_in_middle():
    """Test with 'qwen' not at the start."""
    codeflash_output = qwen_model_profile("openai-qwen-7b"); profile = codeflash_output # 416ns -> 83ns (401% faster)

# ---------------------------
# EDGE TEST CASES
# ---------------------------


def test_qwen_model_profile_edge_similar_name():
    """Test with names that are similar but not Qwen."""
    codeflash_output = qwen_model_profile("qwena-7b") # 458ns -> 125ns (266% faster)
    codeflash_output = qwen_model_profile("qwenner") # 291ns -> 42ns (593% faster)
    codeflash_output = qwen_model_profile("qwenish-7b") # 209ns -> 41ns (410% faster)

def test_qwen_model_profile_edge_case_sensitivity():
    """Test with mixed case model name."""

def test_qwen_model_profile_edge_long_string():
    """Test with a very long string that starts with 'qwen'."""
    name = "qwen" + "x" * 500
    codeflash_output = qwen_model_profile(name); profile = codeflash_output # 459ns -> 83ns (453% faster)

def test_qwen_model_profile_edge_long_non_qwen_string():
    """Test with a very long string that does not start with 'qwen'."""
    name = "x" * 1000
    codeflash_output = qwen_model_profile(name); profile = codeflash_output # 416ns -> 83ns (401% faster)

def test_qwen_model_profile_edge_only_qwen():
    """Test with 'qwen' only."""
    codeflash_output = qwen_model_profile("qwen"); profile = codeflash_output # 458ns -> 83ns (452% faster)

def test_qwen_model_profile_edge_qwen_with_symbols():
    """Test with 'qwen' followed by symbols."""
    codeflash_output = qwen_model_profile("qwen_7b@2024!"); profile = codeflash_output # 417ns -> 83ns (402% faster)

# ---------------------------
# LARGE SCALE TEST CASES
# ---------------------------

def test_qwen_model_profile_large_many_qwen_models():
    """Test performance and correctness with many Qwen model names."""
    for i in range(1000):
        name = f"qwen-model-{i}"
        codeflash_output = qwen_model_profile(name); profile = codeflash_output # 200μs -> 41.1μs (388% faster)

def test_qwen_model_profile_large_many_non_qwen_models():
    """Test performance and correctness with many non-Qwen model names."""
    for i in range(1000):
        name = f"gpt-model-{i}"
        codeflash_output = qwen_model_profile(name); profile = codeflash_output # 205μs -> 41.1μs (399% faster)

def test_qwen_model_profile_large_mixed_models():
    """Test with a mix of Qwen and non-Qwen model names."""
    for i in range(500):
        # Qwen
        name_qwen = f"qwen-{i}"
        codeflash_output = qwen_model_profile(name_qwen); profile_qwen = codeflash_output # 102μs -> 20.6μs (396% faster)
        # Non-Qwen
        name_non_qwen = f"other-{i}"
        codeflash_output = qwen_model_profile(name_non_qwen); profile_non_qwen = codeflash_output # 101μs -> 20.5μs (394% faster)

def test_qwen_model_profile_large_edge_cases():
    """Test with a mix of edge-case names in a large batch."""
    names = ["qwen"] * 100 + ["qwen-7b"] * 100 + ["QWEN-14B"] * 100 + ["gpt-3"] * 100 + [""] * 100
    for name in names:
        codeflash_output = qwen_model_profile(name); profile = codeflash_output # 100μs -> 20.6μs (390% faster)
        if name.lower().startswith("qwen"):
            pass
        else:
            pass
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

from pydantic_ai.profiles._json_schema import InlineDefsJsonSchemaTransformer
from pydantic_ai.profiles.qwen import qwen_model_profile

def test_qwen_model_profile():
    qwen_model_profile('')

To edit these changes git checkout codeflash/optimize-qwen_model_profile-mddvar40 and push.

Codeflash

REFINEMENT Here’s an optimized version of your program. The main inefficiency is that you re-import and re-reference `InlineDefsJsonSchemaTransformer` each time you call the function, even though it is constant. By caching the constructed `ModelProfile` object, we can avoid redundant instantiations and speed up execution.



**Explanation:**  
- This avoids reconstructing the `ModelProfile` object on every call, saving both time and memory, and removes repeated lookups.
- The function will return the exact same result as before.  
- Faster, especially if called many times.

If you need the returned `ModelProfile` to be unique for each call (e.g., if it's mutable), you can’t cache it; but with the code as written, there’s no use of `model_name` and the transformer is constant, so this optimization is safe.
@codeflash-ai codeflash-ai bot added the ⚡️ codeflash Optimization PR opened by Codeflash AI label Jul 22, 2025
@codeflash-ai codeflash-ai bot requested a review from aseembits93 July 22, 2025 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⚡️ codeflash Optimization PR opened by Codeflash AI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants