Skip to content

BUG: tests/test_llm_logs.py fails on Python 3.14 with IndexOutOfRange #1335

@sarahec

Description

@sarahec

While building 0.82 on Python 3.14:

llm> FAILED tests/test_llm_logs.py::test_logs_fragments[fragment_refs0-expected0] - AssertionError: assert [{'name': 'multi_system_fragment', 'prompt_fragments': [], 'system_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6']}] == [{'name': 'single_prompt_fragment', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb'], 'system_fragments': []}, {'name': 'multi_prompt_fragment', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6'], 'system_fragments': []}, {'name': 'multi_system_fragment', 'prompt_fragments': [], 'system_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6']}, {'name': 'both_fragments', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6'], 'system_fragments': ['de2dc9fbffbfd7e3760a19107f13dd6cdcdc78c577b71c58f645d77ec95015e3', '7328249eecee2ec90891237050a7283050bd4560bc5d0bf3643663d340d44cfa']}]
llm>
llm>   At index 0 diff: {'name': 'multi_system_fragment', 'prompt_fragments': [], 'system_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6']} != {'name': 'single_prompt_fragment', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb'], 'system_fragments': []}
llm>   Right contains 3 more items, first extra item: {'name': 'multi_prompt_fragment', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6'], 'system_fragments': []}
llm>
llm>   Full diff:
llm>     [
llm>   -     {
llm>   -         'name': 'single_prompt_fragment',
llm>   -         'prompt_fragments': [
llm>   -             'b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb',
llm>   -         ],
llm>   -         'system_fragments': [],
llm>   -     },
llm>   -     {
llm>   -         'name': 'multi_prompt_fragment',
llm>   -         'prompt_fragments': [
llm>   -             'b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb',
llm>   -             '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6',
llm>   -         ],
llm>   -         'system_fragments': [],
llm>   -     },
llm>         {
llm>             'name': 'multi_system_fragment',
llm>             'prompt_fragments': [],
llm>             'system_fragments': [
llm>                 'b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb',
llm>                 '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6',
llm>             ],
llm>         },
llm>   -     {
llm>   -         'name': 'both_fragments',
llm>   -         'prompt_fragments': [
llm>   -             'b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb',
llm>   -             '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6',
llm>   -         ],
llm>   -         'system_fragments': [
llm>   -             'de2dc9fbffbfd7e3760a19107f13dd6cdcdc78c577b71c58f645d77ec95015e3',
llm>   -             '7328249eecee2ec90891237050a7283050bd4560bc5d0bf3643663d340d44cfa',
llm>   -         ],
llm>   -     },
llm>     ]
llm> FAILED tests/test_llm_logs.py::test_logs_fragments[fragment_refs2-expected2] - AssertionError: assert [] == [{'name': 'both_fragments', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6'], 'system_fragments': ['de2dc9fbffbfd7e3760a19107f13dd6cdcdc78c577b71c58f645d77ec95015e3', '7328249eecee2ec90891237050a7283050bd4560bc5d0bf3643663d340d44cfa']}]
llm>
llm>   Right contains one more item: {'name': 'both_fragments', 'prompt_fragments': ['b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb', '2...760a19107f13dd6cdcdc78c577b71c58f645d77ec95015e3', '7328249eecee2ec90891237050a7283050bd4560bc5d0bf3643663d340d44cfa']}
llm>
llm>   Full diff:
llm>   + []
llm>   - [
llm>   -     {
llm>   -         'name': 'both_fragments',
llm>   -         'prompt_fragments': [
llm>   -             'b4887c00c11d318678012a003fc53203dae1f7a7337e58c982485dbe055d50fb',
llm>   -             '292244e938bf29eca2b7c584a4dab12f479293b2d8ff300343e1b61d30b5efc6',
llm>   -         ],
llm>   -         'system_fragments': [
llm>   -             'de2dc9fbffbfd7e3760a19107f13dd6cdcdc78c577b71c58f645d77ec95015e3',
llm>   -             '7328249eecee2ec90891237050a7283050bd4560bc5d0bf3643663d340d44cfa',
llm>   -         ],
llm>   -     },
llm>   - ]
llm> FAILED tests/test_llm_logs.py::test_expand_fragment_json[-e] - IndexError: list index out of range
llm> FAILED tests/test_llm_logs.py::test_expand_fragment_json[--expand] - IndexError: list index out of range
llm> FAILED tests/test_llm_logs.py::test_expand_fragment_markdown - IndexError: list index out of range
llm> ================ 5 failed, 453 passed, 11 deselected in 17.95s =================

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions