feat(tests): Add per-process test isolation and lean parallel runner by joshuarli · Pull Request #109994 · getsentry/sentry

joshuarli · 2026-03-05T21:45:03Z

todo:

does this break coverage collection?

Per-process test isolation and a lean built-in parallel runner that replaces pytest-xdist.

Motivation

This started as work to make independent pytest sessions safe to run simultaneously — each process needs its own PostgreSQL databases, Redis DB, and Kafka topics so tests don't stomp on each other. Once that isolation layer existed, we realized we could reuse it to reimplement pytest -n in-house with a fraction of the complexity.

pytest-xdist pulls in execnet, uses heavy master-worker IPC, and has a lot of machinery we don't need. Our replacement is ~280 lines with zero new dependencies — it just spawns subprocesses, writes JSONL result files, and replays results through pytest's native reporting hooks.

What changed

src/sentry/testutils/pytest/isolation.py (new) — Worker identity resolution. Each pytest process gets an isolated identity via:

SENTRY_PYTEST_SERIAL=1 → no isolation (old behavior)
SENTRY_TEST_WORKER_ID=N → explicit slot (used by -n)
File-lock slot allocation → auto-isolation for plain pytest

File locks give stable DB names (--reuse-db works) and exclusive Redis DBs (no cross-session collisions). Up to 15 parallel slots (Redis DBs 1–15).

Provides helpers: get_db_suffix(), get_redis_db(), get_kafka_topic(), get_snuba_url().

src/sentry/testutils/pytest/parallel.py (new) — Lean xdist replacement. pytest -n4 distributes tests across 4 workers upfront (partitioned by file, round-robin). The coordinator collects tests once, writes nodeids to temp files, and spawns worker subprocesses.

src/sentry/testutils/pytest/parallel_worker.py (new) — Worker shim. Each worker reads nodeids from its temp file and calls pytest.main(nodeids) directly — skipping the expensive collection walk entirely (~36s saved per worker). Results stream back via JSONL files.

Output is TTY-aware: in-place dot progress with counter on terminals, plain streaming dots on CI pipes. Verbose (-v) shows per-test [wN] nodeid OUTCOME (Xs) lines. Failures always print full tracebacks. Crashed workers dump their captured stdout.

src/sentry/testutils/pytest/sentry.py — Uses isolation helpers for DB names, Redis DBs. Removed pytest_xdist_setupnodes hook and PYTEST_XDIST_TESTRUNUID usage.

Removed reset_snuba fixture — It was a no-op. ClickHouse isolation works via PostgreSQL sequence uniqueness: each test gets fresh project_ids that never collide, so rows from other tests are invisible without any truncation.

Removed all pytest-xdist artifacts — PYTEST_XDIST_WORKER env var checks, looponfailroots config, xdist dependency.

Usage

# Plain pytest — auto-isolates via file locks, safe to run multiple sessions
pytest tests/sentry/foo/

# Parallel — distributes across 4 workers
pytest -n4 tests/sentry/foo/

# Old behavior — no isolation
SENTRY_PYTEST_SERIAL=1 pytest tests/sentry/foo/

Each pytest process now automatically gets isolated PostgreSQL databases, Redis DBs, and Kafka topics via a random hex worker ID — no configuration needed. This started as a way to let independent pytest sessions run simultaneously without interference, then we realized the same isolation machinery could replace pytest-xdist entirely with a much leaner implementation (~280 lines, zero new dependencies). The built-in `-n` flag distributes tests across worker subprocesses upfront, with results replayed through pytest's native reporting hooks. No execnet, no dynamic load balancing, no heavy master-worker IPC — just subprocess spawning and JSONL temp files. Key changes: - `xdist.py`: Per-worker identity via random hex (replaces file locks) - `parallel.py`: Lean xdist replacement with native pytest summary - `sentry.py`: Use xdist helpers for DB names, Redis DBs, Kafka topics - Remove `reset_snuba` fixture (no-op since ClickHouse isolation uses PostgreSQL sequence uniqueness — project IDs never collide) - Remove all pytest-xdist artifacts (PYTEST_XDIST_WORKER, looponfailroots) Co-Authored-By: Claude <noreply@anthropic.com>

Master renamed Region to Cell. Our branch had a partial merge leaving both constructors in the same assignment. Use Cell with the per-worker snowflake_id. Co-Authored-By: Claude <noreply@anthropic.com>

Use a dedicated _SENTRY_PARALLEL_NODEIDS env var for the parallel runner instead of overloading SELECTED_TESTS_FILE, which is meant for selective test execution in CI. Workers now receive exact node IDs, and the partition uses round-robin to preserve collection order. Also adds tests for the partition logic. Co-Authored-By: Claude <noreply@anthropic.com>

Only backend.yml retains its pull_request trigger so that this PR can run backend tests with -n2 without noise from unrelated workflows. Co-Authored-By: Claude <noreply@anthropic.com> (cherry picked from commit 0c01225)

github-actions · 2026-03-05T23:04:08Z

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

- Rename `xdist.py` → `isolation.py` to avoid confusion with pytest-xdist - Replace random hex worker IDs with file-lock slot allocation: fixes `--reuse-db` (stable DB names) and Redis DB collisions between sessions - Raise slot cap from 7 to 15 (Redis DBs 1–15, DB 0 reserved for dev) - Detect TTY via `tw.hasmarkup`: in-place `\r` progress on terminals, plain streaming dots on CI pipes (fixes GHA log spam) - Move parallel test execution docs from AGENTS.md to tests/AGENTS.md Co-Authored-By: Claude <noreply@anthropic.com>

Workers now call pytest.main(nodeids) directly via a worker shim script instead of collecting the full test suite and filtering post-collection. This eliminates ~36s of redundant collection per worker subprocess. - Add parallel_worker.py: standalone script that reads nodeids from a file and calls pytest.main() with them - Remove _SENTRY_PARALLEL_NODEIDS filter from pytest_collection_modifyitems - Coordinator passes non-positional args via _SENTRY_PARALLEL_ARGS env var Co-Authored-By: Claude <noreply@anthropic.com>

github-actions · 2026-03-06T00:08:01Z

Backend Test Failures

Failures on c5eea73 in this run:

tests/sentry/issues/test_suspect_flags.py::TestEAPQueryErrorCounts::test_eap_and_snuba_counts_match_without_group — log

tests/sentry/issues/test_suspect_flags.py:187: in test_eap_and_snuba_counts_match_without_group
    assert snuba_count == eap_count == 7
E   assert 51 == 7

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_one_aggregation_and_environment

— log

tests/sentry/sentry_metrics/querying/data/test_api.py:299: in test_query_with_one_aggregation_and_environment
    assert data[0][0]["series"] == [
E   assert [None, 42000000.0, 28000000.0] == [None, 6000000.0, 4000000.0]
E     
E     At index 1 diff: 42000000.0 != 6000000.0
E     
E     Full diff:
E       [
E           None,
E     -     6000000.0,
E     -     4000000.0,
E     +     42000000.0,
E     ?      +
E     +     28000000.0,
E       ]

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_no_limit_with_series

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1455: in test_no_limit_with_series
    assert group["totals"]["sum(sentry.sessions.session)"] == 4
E   assert 25.0 == 4

tests/sentry/rules/history/test_preview.py::FrequencyConditionTest::test_frequency_conditions — log

tests/sentry/rules/history/test_preview.py:865: in test_frequency_conditions
    result = preview(self.project, conditions, [], *MATCH_ARGS)
src/sentry/rules/history/preview.py:119: in preview
    group_activity = apply_frequency_conditions(
src/sentry/rules/history/preview.py:452: in apply_frequency_conditions
    dataset_map[group],
E   KeyError: 548

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_one_not_in_filter — log

tests/sentry/sentry_metrics/querying/data/test_api.py:702: in test_query_one_not_in_filter
    assert first_query[0]["series"] == [
E   assert [None, 35000000.0, 28000000.0] == [None, 5000000.0, 4000000.0]
E     
E     At index 1 diff: 35000000.0 != 5000000.0
E     
E     Full diff:
E       [
E           None,
E     -     5000000.0,
E     +     35000000.0,
E     ?     +
E     -     4000000.0,
E     ?     ^
E     +     28000000.0,
E     ?     ^^
E       ]

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_one_aggregation_and_latest_release

— log

tests/sentry/sentry_metrics/querying/data/test_api.py:322: in test_query_with_one_aggregation_and_latest_release
    assert data[0][0]["series"] == [
E   assert [None, 78000000.0, 91000000.0] == [None, 6000000.0, 7000000.0]
E     
E     At index 1 diff: 78000000.0 != 6000000.0
E     
E     Full diff:
E       [
E           None,
E     -     6000000.0,
E     -     7000000.0,
E     +     78000000.0,
E     ?      +
E     +     91000000.0,
E       ]

tests/sentry/dynamic_sampling/tasks/test_common.py::TestGetActiveOrgsVolumesMeasureFiltering::test_transactions_measure_volumes

— log

tests/sentry/dynamic_sampling/tasks/test_common.py:364: in test_transactions_measure_volumes
    assert len(found_volumes) == 1
E   assert 12 == 1
E    +  where 12 = len([OrganizationDataVolume(org_id=4557731314794518, total=3.0, indexed=1.0), OrganizationDataVolume(org_id=45577313147945...557731314794515, total=3.0, indexed=1.0), OrganizationDataVolume(org_id=4557731314794514, total=3.0, indexed=1.0), ...])

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_one_aggregation — log

tests/sentry/sentry_metrics/querying/data/test_api.py:217: in test_query_with_one_aggregation
    assert data[0][0]["series"] == [
E   assert [None, 96000000.0, 72000000.0] == [None, 12000000.0, 9000000.0]
E     
E     At index 1 diff: 96000000.0 != 12000000.0
E     
E     Full diff:
E       [
E           None,
E     -     12000000.0,
E     -     9000000.0,
E     +     96000000.0,
E     ?      +
E     +     72000000.0,
E       ]

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_apdex_transaction_threshold

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:136: in test_apdex_transaction_threshold
    assert len(groups) == 2
E   AssertionError: assert 5 == 2
E    +  where 5 = len([{'by': {'transaction_group': '/bar'}, 'totals': {'apdex': None}}, {'by': {'transaction_group': '/baz'}, 'totals': {'a..._transaction'}, 'totals': {'apdex': 1.0}}, {'by': {'transaction_group': 'foo_transaction'}, 'totals': {'apdex': None}}])

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_abnormal_sessions — log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1836: in test_abnormal_sessions
    assert foo_group["totals"] == {"session.abnormal": 4}
E   AssertionError: assert {'session.abnormal': 25.0} == {'session.abnormal': 4}
E     
E     Differing items:
E     {'session.abnormal': 25.0} != {'session.abnormal': 4}
E     
E     Full diff:
E       {
E     -     'session.abnormal': 4,
E     ?                         ^
E     +     'session.abnormal': 25.0,
E     ?                         ^^^^
E       }

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_crash_free_rate_when_no_session_metrics_data_exist

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1730: in test_crash_free_rate_when_no_session_metrics_data_exist
    assert group["totals"]["session.crash_free_rate"] is None
E   assert 1.0 is None

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_staff_crash_free_percentage

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1658: in test_staff_crash_free_percentage
    assert group["totals"]["session.crash_free_rate"] == 0.5
E   assert 0.5538461538461539 == 0.5

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_alias_on_single_entity_derived_metrics

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:936: in test_alias_on_single_entity_derived_metrics
    assert group["totals"] == {"failure_rate_alias": 0.25}
E   AssertionError: assert {'failure_rate_alias': 0.28} == {'failure_rate_alias': 0.25}
E     
E     Differing items:
E     {'failure_rate_alias': 0.28} != {'failure_rate_alias': 0.25}
E     
E     Full diff:
E       {
E     -     'failure_rate_alias': 0.25,
E     ?                              ^
E     +     'failure_rate_alias': 0.28,
E     ?                              ^
E       }

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_count_transaction_with_valid_condition

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:834: in test_count_transaction_with_valid_condition
    assert groups[0]["totals"] == {
E   AssertionError: assert {'count_trans...meterized': 6} == {'count_trans...meterized': 1}
E     
E     Differing items:
E     {'count_transaction_name_is_unparameterized': 6} != {'count_transaction_name_is_unparameterized': 1}
E     {'count_transaction_name_is_null': 36} != {'count_transaction_name_is_null': 2}
E     {'count_transaction_name_has_value': 19} != {'count_transaction_name_has_value': 3}
E     
E     Full diff:
E       {
E     -     'count_transaction_name_has_value': 3,
E     ?                                         ^
E     +     'count_transaction_name_has_value': 19,
E     ?                                         ^^
E     -     'count_transaction_name_is_null': 2,
E     ?                                       ^
E     +     'count_transaction_name_is_null': 36,
E     ?                                       ^^
E     -     'count_transaction_name_is_unparameterized': 1,
E     ?                                                  ^
E     +     'count_transaction_name_is_unparameterized': 6,
E     ?                                                  ^
E       }

tests/sentry/dynamic_sampling/tasks/test_tasks.py::TestBoostLowVolumeProjectsTasks::test_boost_low_volume_projects_with_spans_measure

— log

tests/sentry/dynamic_sampling/tasks/test_tasks.py:246: in test_boost_low_volume_projects_with_spans_measure
    assert generate_rules(proj_a)[0]["samplingValue"] == {
E   AssertionError: assert {'type': 'sam...value': 0.125} == {'type': 'sam...817 ± 1.5e-07}
E     
E     Omitting 1 identical items, use -vv to show
E     Differing items:
E     {'value': 0.125} != {'value': 0.14814814814814817 ± 1.5e-07}
E     
E     Full diff:
E       {
E           'type': 'sampleRate',
E     -     'value': 0.14814814814814817 ± 1.5e-07,
E     +     'value': 0.125,
E       }

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_formula_and_group_by — log

tests/sentry/sentry_metrics/querying/data/test_api.py:1201: in test_query_with_formula_and_group_by
    assert first_query[0]["series"] == [None, 1.0, 2.0]
E   assert [None, 49.0, 98.0] == [None, 1.0, 2.0]
E     
E     At index 1 diff: 49.0 != 1.0
E     
E     Full diff:
E       [
E           None,
E     -     1.0,
E     ?     ^
E     +     49.0,
E     ?     ^^
E     -     2.0,
E     ?     ^
E     +     98.0,
E     ?     ^^
E       ]

tests/sentry/tasks/test_daily_summary.py::DailySummaryTest::test_no_data_summary_doesnt_send — log

tests/sentry/tasks/test_daily_summary.py:561: in test_no_data_summary_doesnt_send
    assert mock_deliver_summary.call_count == 0
E   AssertionError: assert 1 == 0
E    +  where 1 = <MagicMock name='deliver_summary' id='139793750704544'>.call_count

tests/sentry/release_health/test_tasks.py::TestAdoptReleasesPath::test_monitor_release_adoption — log

.venv/lib/python3.13/site-packages/django/db/backends/utils.py:103: in _execute
    return self.cursor.execute(sql)
src/sentry/db/postgres/decorators.py:16: in inner
    return func(self, *args, **kwargs)
src/sentry/db/postgres/base.py:96: in execute
    return self.cursor.execute(sql)
E   psycopg2.errors.ForeignKeyViolation: insert or update on table "sentry_release" violates foreign key constraint "sentry_release_organization_id_45a8c0c8_fk_sentry_or"
E   DETAIL:  Key (organization_id)=(4557742609989648) is not present in table "sentry_organization".

The above exception was the direct cause of the following exception:
src/sentry/testutils/cases.py:408: in _post_teardown
    super()._post_teardown()
.venv/lib/python3.13/site-packages/django/db/backends/postgresql/base.py:482: in check_constraints
    cursor.execute("SET CONSTRAINTS ALL IMMEDIATE")
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:122: in execute
    return super().execute(sql, params)
.venv/lib/python3.13/site-packages/sentry_sdk/utils.py:1870: in runner
    return original_function(*args, **kwargs)
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:79: in execute
    return self._execute_with_wrappers(
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
src/sentry/db/postgres/base.py:70: in _execute__include_sql_in_error
    return execute(sql, params, many, context)
src/sentry/db/postgres/base.py:58: in _execute__clean_params
    return execute(sql, clean_bad_params(params), many, context)
src/sentry/testutils/hybrid_cloud.py:133: in __call__
    return execute(*params)
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:100: in _execute
    with self.db.wrap_database_errors:
.venv/lib/python3.13/site-packages/django/db/utils.py:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:103: in _execute
    return self.cursor.execute(sql)
src/sentry/db/postgres/decorators.py:16: in inner
    return func(self, *args, **kwargs)
src/sentry/db/postgres/base.py:96: in execute
    return self.cursor.execute(sql)
E   django.db.utils.IntegrityError: insert or update on table "sentry_release" violates foreign key constraint "sentry_release_organization_id_45a8c0c8_fk_sentry_or"
E   DETAIL:  Key (organization_id)=(4557742609989648) is not present in table "sentry_organization".
E   
E   SQL: SET CONSTRAINTS ALL IMMEDIATE

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_team_key_transactions_my_teams

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:1570: in test_team_key_transactions_my_teams
    assert data["groups"] == [
E   AssertionError: assert [{'by': {'tea...ns': 0}}, ...] == [{'by': {'tea...actions': 0}}]
E     
E     At index 1 diff: {'by': {'team_key_transactions': 0, 'transaction': '/lorem'}, 'totals': {'team_key_transactions': 0, 'p95': 30.0}} != {'by': {'team_key_transactions': 0, 'transaction': 'bar_transaction'}, 'totals': {'team_key_transactions': 0, 'p95': 1.0}}
E     Left contains 4 more items, first extra item: {'by': {'team_key_transactions': 0, 'transaction': '/bar'}, 'totals': {'p95': 20.0, 'team_key_transactions': 0}}
E     
E     Full diff:
E       [
E           {
E               'by': {
E                   'team_key_transactions': 1,
E                   'transaction': 'foo_transaction',
E               },
E               'totals': {
E                   'p95': 1.0,
E                   'team_key_transactions': 1,
E     +         },
E     +     },
E     +     {
E     +         'by': {
E     +             'team_key_transactions': 0,
E     +             'transaction': '/lorem',
E     +         },
E     +         'totals': {
E     +             'p95': 30.0,
E     +             'team_key_transactions': 0,
E     +         },
E     +     },
E     +     {
E     +         'by': {
E     +             'team_key_transactions': 0,
E     +             'transaction': '<< unparameterized >>',
E     +         },
E     +         'totals': {
E     +             'p95': 27.099999999999998,
E     +             'team_key_transactions': 0,
E     +         },
E     +     },
E     +     {
E     +         'by': {
E     +             'team_key_transactions': 0,
E     +             'transaction': '/bar',
E     +         },
E     +         'totals': {
E     +             'p95': 20.0,
E     +             'team_key_transactions': 0,
E     +         },
E     +     },
E     +     {
... (30 more lines)

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_unparameterized_transactions_in_groupby

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:1686: in test_unparameterized_transactions_in_groupby
    assert sorted(data["groups"], key=lambda group: group["by"]["transaction_name"]) == [
E   AssertionError: assert [{'by': {'tra...nt': 6}}, ...] == [{'by': {'tra...n_count': 2}}]
E     
E     At index 0 diff: {'by': {'transaction_name': '/bar'}, 'totals': {'duration_count': 7}} != {'by': {'transaction_name': '/bar'}, 'totals': {'duration_count': 1}}
E     Left contains 4 more items, first extra item: {'by': {'transaction_name': '<< unparameterized >>'}, 'totals': {'duration_count': 15}}
E     
E     Full diff:
E       [
E           {
E               'by': {
E                   'transaction_name': '/bar',
E               },
E               'totals': {
E     -             'duration_count': 1,
E     ?                               ^
E     +             'duration_count': 7,
E     ?                               ^
E               },
E           },
E           {
E               'by': {
E                   'transaction_name': '/foo',
E     +         },
E     +         'totals': {
E     +             'duration_count': 7,
E     +         },
E     +     },
E     +     {
E     +         'by': {
E     +             'transaction_name': '/lorem',
E               },
E               'totals': {
E                   'duration_count': 1,
E               },
E           },
E           {
E               'by': {
E                   'transaction_name': '<< unparameterized >>',
E               },
E               'totals': {
E     +             'duration_count': 15,
E     +         },
E     +     },
E     +     {
E     +         'by': {
E     +             'transaction_name': 'bar_transaction',
E     +         },
E     +         'totals': {
E     -             'duration_count': 2,
... (22 more lines)

tests/snuba/rules/conditions/test_event_frequency.py::ErrorIssueEventFrequencyPercentConditionTestCase::test_thirty_minutes_no_events

— log

tests/snuba/rules/conditions/test_event_frequency.py:1354: in test_thirty_minutes_no_events
    self._run_test(data=data, minutes=30, passes=True, add_events=True)
tests/snuba/rules/conditions/test_event_frequency.py:1228: in _run_test
    self.assertPasses(rule, self.test_event, is_new=False)
src/sentry/testutils/cases.py:823: in assertPasses
    assert rule.passes(event, state) is True
E   assert False is True
E    +  where False = <bound method BaseEventFrequencyCondition.passes of <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f3fa33dfd90>>(<sentry.services.eventstore.models.GroupEvent object at 0x7f3f4de5c870>, <sentry.rules.base.EventState object at 0x7f3f4dd069e0>)
E    +    where <bound method BaseEventFrequencyCondition.passes of <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f3fa33dfd90>> = <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f3fa33dfd90>.passes

tests/sentry/api/endpoints/test_organization_root_cause_analysis.py::OrganizationRootCauseAnalysisTest::test_returns_change_data_for_regressed_spans

— log

tests/sentry/api/endpoints/test_organization_root_cause_analysis.py:228: in test_returns_change_data_for_regressed_spans
    assert response.data == [
E   AssertionError: assert [{'p95_after'...': None, ...}] == [{'p95_after'...b span', ...}]
E     
E     At index 0 diff: {'span_op': 'db', 'span_group': 'd77d5e503ad1439f', 'span_description': None, 'spm_before': 0.0, 'spm_after': 0.0006944444444444445, 'p95_before': 0.0, 'p95_after': 10000.0, 'score': 6.944444444444445} != {'span_op': 'django.middleware', 'span_group': '2b9cbb96dbf59baa', 'span_description': 'middleware span', 'score': 1.1166666666666667, 'spm_before': 0.00034722222222222224, 'spm_after': 0.0020833333333333333, 'p95_before': 60.0, 'p95_after': 546.0}
E     Left contains one more item: {'p95_after': 60.0, 'p95_before': 60.0, 'score': 0.08333333333333333, 'span_description': None, ...}
E     
E     Full diff:
E       [
E           {
E     +         'p95_after': 10000.0,
E     +         'p95_before': 0.0,
E     +         'score': 6.944444444444445,
E     +         'span_description': None,
E     +         'span_group': 'd77d5e503ad1439f',
E     +         'span_op': 'db',
E     +         'spm_after': 0.0006944444444444445,
E     +         'spm_before': 0.0,
E     +     },
E     +     {
E     -         'p95_after': 546.0,
E     ?                      --
E     +         'p95_after': 600.0,
E     ?                       ++
E               'p95_before': 60.0,
E     -         'score': 1.1166666666666667,
E     +         'score': 4.124999999999999,
E     -         'span_description': 'middleware span',
E     ?                             ^^^^^^ ----------
E     +         'span_description': None,
E     ?                             ^^^
E               'span_group': '2b9cbb96dbf59baa',
E               'span_op': 'django.middleware',
E     -         'spm_after': 0.0020833333333333333,
E     -         'spm_before': 0.00034722222222222224,
E     +         'spm_after': 0.006944444444444444,
E     +         'spm_before': 0.0006944444444444445,
E           },
E           {
E               'p95_after': 60.0,
E               'p95_before': 60.0,
E     -         'score': 0.020833333333333336,
E     ?                    --               ^
E     +         'score': 0.08333333333333333,
E     ?                                   ^^
E     -         'span_description': 'db span',
E     ?                             ^^^^^^^ ^
E     +         'span_description': None,
E     ?                             ^^ ^
E               'span_group': '5ad8c5a1e8d0e5f7',
... (9 more lines)

tests/sentry/tasks/test_weekly_reports.py::WeeklyReportsTest::test_organization_project_issue_status — log

tests/sentry/tasks/test_weekly_reports.py:359: in test_organization_project_issue_status
    assert key_errors == [{"events.group_id": event1.group.id, "count()": 1}]
E   AssertionError: assert [{'count()': ...oup_id': 624}] == [{'count()': ...oup_id': 626}]
E     
E     At index 0 diff: {'events.group_id': 615, 'count()': 1} != {'events.group_id': 626, 'count()': 1}
E     Left contains 2 more items, first extra item: {'count()': 1, 'events.group_id': 626}
E     
E     Full diff:
E       [
E     +     {
E     +         'count()': 1,
E     +         'events.group_id': 615,
E     +     },
E           {
E               'count()': 1,
E               'events.group_id': 626,
E           },
E     +     {
E     +         'count()': 1,
E     +         'events.group_id': 624,
E     +     },
E       ]

tests/snuba/rules/conditions/test_event_frequency.py::PerfIssuePlatformIssueEventFrequencyPercentConditionTestCase::test_five_minutes_no_events

— log

tests/snuba/rules/conditions/test_event_frequency.py:1332: in test_five_minutes_no_events
    self._run_test(data=data, minutes=5, passes=True, add_events=True)
tests/snuba/rules/conditions/test_event_frequency.py:1228: in _run_test
    self.assertPasses(rule, self.test_event, is_new=False)
src/sentry/testutils/cases.py:823: in assertPasses
    assert rule.passes(event, state) is True
E   assert False is True
E    +  where False = <bound method BaseEventFrequencyCondition.passes of <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f55a834f790>>(<sentry.services.eventstore.models.GroupEvent object at 0x7f557433e530>, <sentry.rules.base.EventState object at 0x7f55844d1400>)
E    +    where <bound method BaseEventFrequencyCondition.passes of <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f55a834f790>> = <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f55a834f790>.passes

tests/sentry/release_health/release_monitor/test_metrics.py::MetricFetchProjectsWithRecentSessionsTest::test_monitor_release_adoption_with_filter

— log

tests/sentry/release_health/release_monitor/__init__.py:77: in test_monitor_release_adoption_with_filter
    assert results == {
E   AssertionError

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_staff_filter_by_single_project_slug

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:337: in test_staff_filter_by_single_project_slug
    assert response.data["groups"] == [
E   AssertionError: assert [{'by': {}, '...ion)': 21.0}}] == [{'by': {}, '...ession)': 3}}]
E     
E     At index 0 diff: {'by': {}, 'series': {'sum(sentry.sessions.session)': [0, 21.0]}, 'totals': {'sum(sentry.sessions.session)': 21.0}} != {'by': {}, 'totals': {'sum(sentry.sessions.session)': 3}, 'series': {'sum(sentry.sessions.session)': [0, 3]}}
E     
E     Full diff:
E       [
E           {
E               'by': {},
E               'series': {
E                   'sum(sentry.sessions.session)': [
E                       0,
E     -                 3,
E     ?                 ^
E     +                 21.0,
E     ?                 ^^^^
E                   ],
E               },
E               'totals': {
E     -             'sum(sentry.sessions.session)': 3,
E     ?                                             ^
E     +             'sum(sentry.sessions.session)': 21.0,
E     ?                                             ^^^^
E               },
E           },
E       ]

tests/sentry/rules/history/test_preview.py::FrequencyConditionTest::test_unique_user — log

tests/sentry/rules/history/test_preview.py:936: in test_unique_user
    result = preview(self.project, conditions, [], *MATCH_ARGS)
src/sentry/rules/history/preview.py:119: in preview
    group_activity = apply_frequency_conditions(
src/sentry/rules/history/preview.py:452: in apply_frequency_conditions
    dataset_map[group],
E   KeyError: 209

tests/sentry/utils/test_snowflake.py::SnowflakeUtilsTest::test_generate_correct_ids — log

tests/sentry/utils/test_snowflake.py:39: in test_generate_correct_ids
    region = Region("test-region", 0, "http://testserver", RegionCategory.MULTI_TENANT)
E   NameError: name 'Region' is not defined

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_orderby_percentile_with_many_fields_multiple_entities_with_missing_data

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1215: in test_orderby_percentile_with_many_fields_multiple_entities_with_missing_data
    assert len(groups) == 2
E   AssertionError: assert 5 == 2
E    +  where 5 = len([{'by': {'project_id': 4557733674090513, 'transaction': '/bar/'}, 'series': {'count_unique(transaction.user)': [0], 'p...0000000001]}, 'totals': {'count_unique(transaction.user)': 0, 'p50(transaction.measurements.lcp)': 370.2000000000001}}])

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_series_are_limited_to_total_order_in_case_with_one_field_orderby

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1152: in test_series_are_limited_to_total_order_in_case_with_one_field_orderby
    assert group["series"]["sum(sentry.sessions.session)"] == [1, 2, 3, 4]
E   assert [7.0, 14.0, 21.0, 28.0] == [1, 2, 3, 4]
E     
E     At index 0 diff: 7.0 != 1
E     
E     Full diff:
E       [
E     +     7.0,
E     -     1,
E     +     14.0,
E     ?      +++
E     -     2,
E     +     21.0,
E     ?      +++
E     +     28.0,
E     -     3,
E     -     4,
E       ]

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_validate_include_meta_not_enabled_by_default

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:157: in test_validate_include_meta_not_enabled_by_default
    response = self.get_success_response(
src/sentry/testutils/cases.py:628: in get_success_response
    assert_status_code(response, status.HTTP_200_OK)
src/sentry/testutils/asserts.py:47: in assert_status_code
    assert minimum <= response.status_code < maximum, (
E   AssertionError: (400, b'{"detail":"Unknown string index: 10002"}')
E   assert 400 < 201
E    +  where 400 = <Response status_code=400, "application/json">.status_code

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_errored_rate_sessions_and_users

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:2581: in test_errored_rate_sessions_and_users
    assert group["totals"]["session.errored_rate"] == 0.25
E   assert 0.17857142857142858 == 0.25

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_one_aggregation_and_only_totals

— log

tests/sentry/sentry_metrics/querying/data/test_api.py:248: in test_query_with_one_aggregation_and_only_totals
    assert data[0][0]["totals"] == self.to_reference_unit(21.0)
E   assert 147000000.0 == 21000000.0
E    +  where 21000000.0 = <bound method MetricsAPITestCase.to_reference_unit of <querying.data.test_api.MetricsAPITestCase testMethod=test_query_with_one_aggregation_and_only_totals>>(21.0)
E    +    where <bound method MetricsAPITestCase.to_reference_unit of <querying.data.test_api.MetricsAPITestCase testMethod=test_query_with_one_aggregation_and_only_totals>> = <querying.data.test_api.MetricsAPITestCase testMethod=test_query_with_one_aggregation_and_only_totals>.to_reference_unit

tests/sentry/snuba/metrics/test_metrics_layer/test_release_health.py::ReleaseHealthMetricsLayerTestCase::test_having — log

tests/sentry/snuba/metrics/test_metrics_layer/test_release_health.py:315: in test_having
    assert len(groups) == 1
E   AssertionError: assert 3 == 1
E    +  where 3 = len([{'by': {'release': 'r3'}, 'totals': {'count': 18.0}}, {'by': {'release': 'development'}, 'totals': {'count': 28.0}}, {'by': {'release': 'r1'}, 'totals': {'count': 34.0}}])

tests/sentry/tasks/test_daily_summary.py::DailySummaryTest::test_slack_notification_contents_newline — log

tests/sentry/tasks/test_daily_summary.py:787: in test_slack_notification_contents_newline
    assert '""" Traceback (most recent call las...' in blocks[4]["fields"][0]["text"]
E   KeyError: 'fields'

tests/sentry/utils/test_snowflake.py::SnowflakeUtilsTest::test_generate_correct_ids_with_region_sequence — log

tests/sentry/utils/test_snowflake.py:50: in test_generate_correct_ids_with_region_sequence
    region = Region("test-region", 0, "http://testserver", RegionCategory.MULTI_TENANT)
E   NameError: name 'Region' is not defined

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_groupby_single

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:82: in test_groupby_single
    self.get_success_response(
src/sentry/testutils/cases.py:628: in get_success_response
    assert_status_code(response, status.HTTP_200_OK)
src/sentry/testutils/asserts.py:47: in assert_status_code
    assert minimum <= response.status_code < maximum, (
E   AssertionError: (400, b'{"detail":"Unknown string index: 10002"}')
E   assert 400 < 201
E    +  where 400 = <Response status_code=400, "application/json">.status_code

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_apdex_transactions

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:2270: in test_apdex_transactions
    assert response.data["groups"][0]["totals"] == {"transaction.apdex": 0.6666666666666666}
E   AssertionError: assert {'transaction.apdex': 0.6} == {'transaction...6666666666666}
E     
E     Differing items:
E     {'transaction.apdex': 0.6} != {'transaction.apdex': 0.6666666666666666}
E     
E     Full diff:
E       {
E     -     'transaction.apdex': 0.6666666666666666,
E     ?                             ---------------
E     +     'transaction.apdex': 0.6,
E       }

tests/snuba/api/endpoints/test_organization_events_histogram.py::OrganizationEventsHistogramEndpointTest::test_histogram_exclude_outliers_data_filter

— log

tests/snuba/api/endpoints/test_organization_events_histogram.py:924: in test_histogram_exclude_outliers_data_filter
    assert response.data == self.as_response_data(expected), f"failing for {array_column}"
E   AssertionError: failing for measurements
E   assert {'measurement... 'count': 1}]} == {'measurement... 'count': 4}]}
E     
E     Differing items:
E     {'measurements.foo': [{'bin': 0, 'count': 3}, {'bin': 1000, 'count': 0}, {'bin': 2000, 'count': 0}, {'bin': 3000, 'count': 0}, {'bin': 4000, 'count': 1}]} != {'measurements.foo': [{'bin': 0, 'count': 4}]}
E     
E     Full diff:
E       {
E           'measurements.foo': [
E               {
E                   'bin': 0,
E     -             'count': 4,
E     ?                      ^
E     +             'count': 3,
E     ?                      ^
E     +         },
E     +         {
E     +             'bin': 1000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 2000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 3000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 4000,
E     +             'count': 1,
E               },
E           ],
E       }

tests/sentry/dynamic_sampling/tasks/test_tasks.py::TestBoostLowVolumeProjectsTasks::test_boost_low_volume_projects_simple_with_empty_project

— log

tests/sentry/dynamic_sampling/tasks/test_tasks.py:281: in test_boost_low_volume_projects_simple_with_empty_project
    assert generate_rules(proj_a)[0]["samplingValue"] == {
E   AssertionError: assert {'type': 'sam...2014509230686} == {'type': 'sam...817 ± 1.5e-07}
E     
E     Omitting 1 identical items, use -vv to show
E     Differing items:
E     {'value': 0.2492014509230686} != {'value': 0.14814814814814817 ± 1.5e-07}
E     
E     Full diff:
E       {
E           'type': 'sampleRate',
E     -     'value': 0.14814814814814817 ± 1.5e-07,
E     +     'value': 0.2492014509230686,
E       }

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_limit_above_snuba_limit — log

tests/sentry/sentry_metrics/querying/data/test_api.py:883: in test_query_with_limit_above_snuba_limit
    assert data[0][0]["series"] == [
E   assert [1000000.0, 1....0, 2000000.0] == [None, 1000000.0, 2000000.0]
E     
E     At index 0 diff: 1000000.0 != None
E     
E     Full diff:
E       [
E     -     None,
E     +     1000000.0,
E           1000000.0,
E           2000000.0,
E       ]

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_multiple_aggregations_and_single_group_by

— log

tests/sentry/sentry_metrics/querying/data/test_api.py:757: in test_query_with_multiple_aggregations_and_single_group_by
    assert len(first_query) == 3
E   AssertionError: assert 4 == 3
E    +  where 4 = len([{'by': {'platform': ''}, 'series': [1000000.0, None, None], 'totals': 1000000.0}, {'by': {'platform': 'android'}, 'se...0], 'totals': 3000000.0}, {'by': {'platform': 'windows'}, 'series': [None, 5000000.0, 4000000.0], 'totals': 4000000.0}])

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_query_with_order_by_valid_str_field

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:478: in test_query_with_order_by_valid_str_field
    assert groups[index]["totals"]["count(transaction.duration)"] == expected_count
E   assert 12 == 2

tests/sentry/dynamic_sampling/tasks/test_tasks.py::TestBoostLowVolumeProjectsTasks::test_boost_low_volume_projects_simple_with_sliding_window_org_from_cache

— log

tests/sentry/dynamic_sampling/tasks/test_tasks.py:323: in test_boost_low_volume_projects_simple_with_sliding_window_org_from_cache
    assert generate_rules(proj_a)[0]["samplingValue"] == {
E   AssertionError: assert {'type': 'sam... 'value': 1.0} == {'type': 'sam...817 ± 1.5e-07}
E     
E     Omitting 1 identical items, use -vv to show
E     Differing items:
E     {'value': 1.0} != {'value': 0.14814814814814817 ± 1.5e-07}
E     
E     Full diff:
E       {
E           'type': 'sampleRate',
E     -     'value': 0.14814814814814817 ± 1.5e-07,
E     +     'value': 1.0,
E       }

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_groupby_project

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1402: in test_groupby_project
    assert totals == {"sum(sentry.sessions.session)": expected_count}
E   AssertionError: assert {'sum(sentry....ession)': 6.0} == {'sum(sentry.....session)': 1}
E     
E     Differing items:
E     {'sum(sentry.sessions.session)': 6.0} != {'sum(sentry.sessions.session)': 1}
E     
E     Full diff:
E       {
E     -     'sum(sentry.sessions.session)': 1,
E     ?                                     ^
E     +     'sum(sentry.sessions.session)': 6.0,
E     ?                                     ^^^
E       }

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_orderby_percentile

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:698: in test_orderby_percentile
    assert len(groups) == 2
E   AssertionError: assert 3 == 2
E    +  where 3 = len([{'by': {'tag1': 'value2'}, 'series': {'p50(transaction.measurements.lcp)': [2.0]}, 'totals': {'p50(transaction.measur...''}, 'series': {'p50(transaction.measurements.lcp)': [123.4]}, 'totals': {'p50(transaction.measurements.lcp)': 123.4}}])

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_group_by_and_order_by — log

tests/sentry/sentry_metrics/querying/data/test_api.py:448: in test_query_with_group_by_and_order_by
    assert len(data[0]) == 2
E   AssertionError: assert 3 == 2
E    +  where 3 = len([{'by': {'transaction': '/hello'}, 'series': [None, 42000000.0, 30000000.0], 'totals': 72000000.0}, {'by': {'transacti...0000.0], 'totals': 54000000.0}, {'by': {'transaction': 'foo'}, 'series': [1000000.0, None, None], 'totals': 1000000.0}])

tests/snuba/rules/conditions/test_event_frequency.py::ErrorIssueEventFrequencyPercentConditionTestCase::test_thirty_minutes_with_events

— log

tests/snuba/rules/conditions/test_event_frequency.py:1296: in test_thirty_minutes_with_events
    self._run_test(data=data, minutes=30, passes=True, add_events=True)
tests/snuba/rules/conditions/test_event_frequency.py:1228: in _run_test
    self.assertPasses(rule, self.test_event, is_new=False)
src/sentry/testutils/cases.py:823: in assertPasses
    assert rule.passes(event, state) is True
E   assert False is True
E    +  where False = <bound method BaseEventFrequencyCondition.passes of <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f690c3e6210>>(<sentry.services.eventstore.models.GroupEvent object at 0x7f690c5dc550>, <sentry.rules.base.EventState object at 0x7f690c3d6cf0>)
E    +    where <bound method BaseEventFrequencyCondition.passes of <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f690c3e6210>> = <sentry.rules.conditions.event_frequency.EventFrequencyPercentCondition object at 0x7f690c3e6210>.passes

tests/sentry/dynamic_sampling/tasks/test_boost_low_volume_transactions.py::PrioritiseProjectsSnubaQueryTest::test_fetch_transactions_with_total_volumes

— log

tests/sentry/dynamic_sampling/tasks/test_boost_low_volume_transactions.py:150: in test_fetch_transactions_with_total_volumes
    assert totals["total_num_transactions"] == total_counts
E   assert 42722.0 == 6101

tests/sentry/dynamic_sampling/tasks/test_boost_low_volume_transactions.py::PrioritiseProjectsSnubaQueryTest::test_fetch_transactions_with_total_volumes_large

— log

tests/sentry/dynamic_sampling/tasks/test_boost_low_volume_transactions.py:120: in test_fetch_transactions_with_total_volumes_large
    assert count == self.get_count_for_transaction(idx, name)
E   AssertionError: assert 39000.0 == 3000
E    +  where 3000 = <bound method PrioritiseProjectsSnubaQueryTest.get_count_for_transaction of <tests.sentry.dynamic_sampling.tasks.test_..._low_volume_transactions.PrioritiseProjectsSnubaQueryTest testMethod=test_fetch_transactions_with_total_volumes_large>>(0, 'tl5')
E    +    where <bound method PrioritiseProjectsSnubaQueryTest.get_count_for_transaction of <tests.sentry.dynamic_sampling.tasks.test_..._low_volume_transactions.PrioritiseProjectsSnubaQueryTest testMethod=test_fetch_transactions_with_total_volumes_large>> = <tests.sentry.dynamic_sampling.tasks.test_boost_low_volume_transactions.PrioritiseProjectsSnubaQueryTest testMethod=test_fetch_transactions_with_total_volumes_large>.get_count_for_transaction

tests/sentry/snuba/metrics/test_metrics_layer/test_release_health.py::ReleaseHealthMetricsLayerTestCase::test_anr_rate_operations

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_release_health.py:264: in test_anr_rate_operations
    assert group["totals"]["anr_alias"] == 0.5
E   assert 0.2857142857142857 == 0.5

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_crash_free_percentage

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:1679: in test_crash_free_percentage
    assert group["totals"]["session.crash_free_rate"] == 0.5
E   assert 0.5102040816326531 == 0.5

tests/sentry/releases/endpoints/test_organization_release_health_data.py::DerivedMetricsDataTest::test_healthy_user_sessions

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:2125: in test_healthy_user_sessions
    assert group["totals"]["session.healthy_user"] == 4
E   assert 29 == 4

tests/sentry/sentry_metrics/querying/data/test_api.py::MetricsAPITestCase::test_query_with_group_by_and_order_by_and_only_totals

— log

tests/sentry/sentry_metrics/querying/data/test_api.py:483: in test_query_with_group_by_and_order_by_and_only_totals
    assert data[0][0]["totals"] == self.to_reference_unit(9.0)
E   assert 72000000.0 == 9000000.0
E    +  where 9000000.0 = <bound method MetricsAPITestCase.to_reference_unit of <querying.data.test_api.MetricsAPITestCase testMethod=test_query_with_group_by_and_order_by_and_only_totals>>(9.0)
E    +    where <bound method MetricsAPITestCase.to_reference_unit of <querying.data.test_api.MetricsAPITestCase testMethod=test_query_with_group_by_and_order_by_and_only_totals>> = <querying.data.test_api.MetricsAPITestCase testMethod=test_query_with_group_by_and_order_by_and_only_totals>.to_reference_unit

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py::PerformanceMetricsLayerTestCase::test_rate_epm_hour_rollup

— log

tests/sentry/snuba/metrics/test_metrics_layer/test_metrics_enhanced_performance.py:1135: in test_rate_epm_hour_rollup
    assert data["groups"] == [
E   AssertionError: assert [{'by': {}, '...ion)': 1.85}}] == [{'by': {}, '...tion)': 0.3}}]
E     
E     At index 0 diff: {'by': {}, 'series': {'count(transaction.duration)': [18, 0, 18, 38, 1, 36], 'rate(transaction.duration)': [0.3, 0, 0.3, 0.6333333333333333, 0.016666666666666666, 0.6]}, 'totals': {'count(transaction.duration)': 111, 'rate(transaction.duration)': 1.85}} != {'by': {}, 'series': {'rate(transaction.duration)': [0.05, 0, 0.05, 0.1, 0, 0.1], 'count(transaction.duration)': [3, 0, 3, 6, 0, 6]}, 'totals': {'rate(transaction.duration)': 0.3, 'count(transaction.duration)': 18}}
E     
E     Full diff:
E       [
E           {
E               'by': {},
E               'series': {
E                   'count(transaction.duration)': [
E     -                 3,
E     ?                 ^
E     +                 18,
E     ?                 ^^
E                       0,
E     +                 18,
E     -                 3,
E     +                 38,
E     ?                  +
E     +                 1,
E     -                 6,
E     +                 36,
E     ?                 +
E     -                 0,
E     -                 6,
E                   ],
E                   'rate(transaction.duration)': [
E     -                 0.05,
E     ?                   ^^
E     +                 0.3,
E     ?                   ^
E                       0,
E     -                 0.05,
E     -                 0.1,
E     ?                   ^
E     +                 0.3,
E     ?                   ^
E     -                 0,
E     +                 0.6333333333333333,
E     +                 0.016666666666666666,
E     -                 0.1,
E     ?                   ^
E     +                 0.6,
E     ?                   ^
E                   ],
E               },
E               'totals': {
E     -             'count(transaction.duration)': 18,
... (10 more lines)

tests/sentry/dynamic_sampling/tasks/test_common.py::TestGetActiveOrgsMeasureFiltering::test_transactions_measure_only_counts_transaction_metrics

— log

tests/sentry/dynamic_sampling/tasks/test_common.py:188: in test_transactions_measure_only_counts_transaction_metrics
    assert org2.id not in found_orgs
E   AssertionError: assert 4557731314794514 not in [4557731314794512, 4557731314794514, 4557731314794515, 4557731314794516, 4557731314794517, 4557731314794518, ...]
E    +  where 4557731314794514 = <Organization at 0x7f13cadea410: id=4557731314794514, owner_id=None, name='test-org-2', slug='test-org-2'>.id

tests/sentry/releases/endpoints/test_organization_release_health_data.py::OrganizationReleaseHealthDataTest::test_filter_by_single_project_slug_negation

— log

tests/sentry/releases/endpoints/test_organization_release_health_data.py:405: in test_filter_by_single_project_slug_negation
    assert response.data["groups"] == [
E   AssertionError: assert [{'by': {}, '...ion)': 14.0}}] == [{'by': {}, '...ession)': 2}}]
E     
E     At index 0 diff: {'by': {}, 'series': {'sum(sentry.sessions.session)': [0, 14.0]}, 'totals': {'sum(sentry.sessions.session)': 14.0}} != {'by': {}, 'totals': {'sum(sentry.sessions.session)': 2}, 'series': {'sum(sentry.sessions.session)': [0, 2]}}
E     
E     Full diff:
E       [
E           {
E               'by': {},
E               'series': {
E                   'sum(sentry.sessions.session)': [
E                       0,
E     -                 2,
E     ?                 ^
E     +                 14.0,
E     ?                 ^^^^
E                   ],
E               },
E               'totals': {
E     -             'sum(sentry.sessions.session)': 2,
E     ?                                             ^
E     +             'sum(sentry.sessions.session)': 14.0,
E     ?                                             ^^^^
E               },
E           },
E       ]

The `reset_snuba` fixture was incorrectly removed as a "no-op" — it actually drops and recreates ClickHouse tables between tests, preventing cross-test data contamination. Restore it and all its usages. Fix slot-0 isolation to match historical defaults: - Redis DB base: 9 (was incorrectly 1) - DB suffix: "" for slot 0 (was "_0", breaking --reuse-db) - Kafka topics: base name for slot 0 (was suffixed) - Snowflake ID: 0 for slot 0 (was 1) - Max slots: 7 (Redis DBs 9-15, was 15 which exceeds Redis default) Fix ResourceWarning for unclosed lock file via atexit handler. Co-Authored-By: Claude <noreply@anthropic.com>

…t hack The parallel coordinator now correctly handles pytest-rerunfailures' "rerun" outcome: intermediate reruns don't inflate the progress counter, show as "R" dots (not "F"), and aren't printed in red. Add comment explaining why HAS_PYTEST_HANDLECRASHITEM is disabled. Co-Authored-By: Claude <noreply@anthropic.com>

github-actions · 2026-03-06T00:46:17Z

Backend Test Failures

Failures on 8eee10c in this run:

tests/sentry/utils/test_snowflake.py::SnowflakeUtilsTest::test_generate_correct_ids_with_region_sequence — log

tests/sentry/utils/test_snowflake.py:50: in test_generate_correct_ids_with_region_sequence
    region = Region("test-region", 0, "http://testserver", RegionCategory.MULTI_TENANT)
E   NameError: name 'Region' is not defined

tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_viewed — log

tests/sentry/replays/endpoints/test_organization_replay_index.py:183: in test_get_replays_viewed
    assert response_data["data"][0]["has_viewed"] is False, query
E   AssertionError: 
E   assert True is False

tests/sentry/utils/test_snowflake.py::SnowflakeUtilsTest::test_generate_correct_ids — log

tests/sentry/utils/test_snowflake.py:39: in test_generate_correct_ids
    region = Region("test-region", 0, "http://testserver", RegionCategory.MULTI_TENANT)
E   NameError: name 'Region' is not defined

tests/sentry/tasks/test_daily_summary.py::DailySummaryTest::test_no_data_summary_doesnt_send — log

tests/sentry/tasks/test_daily_summary.py:561: in test_no_data_summary_doesnt_send
    assert mock_deliver_summary.call_count == 0
E   AssertionError: assert 1 == 0
E    +  where 1 = <MagicMock name='deliver_summary' id='140198497969424'>.call_count

The coordinator's `run()` never set `session.testsfailed`, so pytest always exited 0 regardless of worker failures. Now we propagate the failure count from collected results, and also detect worker crashes (non-zero exit without JSONL reports) as a fallback. Co-Authored-By: Claude <noreply@anthropic.com>

When pytest-rerunfailures retries a test, multiple call-phase reports are emitted for the same nodeid (intermediate "rerun" + final outcome). The coordinator was appending all of them to term_reporter.stats, inflating the summary counts. Now we replace the previous report for a nodeid so only the final outcome appears in the summary. Co-Authored-By: Claude <noreply@anthropic.com>

Serial mode (SENTRY_PYTEST_SERIAL=1) uses the same DB, Redis DB, and Kafka topics as auto-allocated slot 0, but previously bypassed the file lock entirely. A concurrent auto-allocated session could claim slot 0 and collide. Now serial mode acquires slot 0's lock as best-effort, preventing overlap. Co-Authored-By: Claude <noreply@anthropic.com>

The old cap of 7 was constrained by Redis DBs 9-15. Now slot 0 stays at DB 9 (historical default), slots 1-8 use DBs 1-8, and slots 9-14 use DBs 10-15 — all 15 DBs except DB 0 (reserved for dev). This allows up to 14 parallel workers with `-n`. Co-Authored-By: Claude <noreply@anthropic.com>

Each `-n` invocation created a /tmp/sentry_parallel_* directory with nodeid lists and JSONL result files that was never cleaned up. Now removed after all results are collected. Co-Authored-By: Claude <noreply@anthropic.com>

…e cleanup Replace the per-test `reset_snuba` fixture (which dropped and recreated all ClickHouse tables before every test) with a session-scoped cleanup that runs once. Within a session, test isolation relies on unique snowflake IDs from PostgreSQL — each test gets fresh org/project IDs that never collide. Key changes: - `reset_snuba` fixture is now session-scoped and autouse. In parallel mode the coordinator resets ClickHouse before spawning workers; worker processes skip the fixture. - Redis `flushdb` in test teardown now preserves `snowflakeid:*` keys. Previously, flushing reset the snowflake sequence counter, and under `@freeze_time` (constant timestamp) subsequent tests regenerated identical snowflake IDs — causing ClickHouse data bleed between tests. - Removed all per-test `reset_snuba` references from SnubaTestCase, ProfilesSnubaTestCase, UptimeCheckSnubaTestCase, MetricsAPIBaseTestCase, ReplayEAPTestCase, and individual test files. - Fixed 15 dynamic_sampling tests that queried ClickHouse without org/project filtering (cross-org discovery queries). These now scope assertions to their own org IDs instead of expecting exact global counts. - Updated tests/AGENTS.md and isolation.py docstring to reflect the new ClickHouse isolation model. Co-Authored-By: Claude <noreply@anthropic.com>

github-actions · 2026-03-06T04:34:55Z

Backend Test Failures

Failures on ee5925c in this run:

tests/sentry/release_health/test_tasks.py::TestAdoptReleasesPath::test_monitor_release_adoption — log

.venv/lib/python3.13/site-packages/django/db/backends/utils.py:103: in _execute
    return self.cursor.execute(sql)
src/sentry/db/postgres/decorators.py:16: in inner
    return func(self, *args, **kwargs)
src/sentry/db/postgres/base.py:96: in execute
    return self.cursor.execute(sql)
E   psycopg2.errors.ForeignKeyViolation: insert or update on table "sentry_release" violates foreign key constraint "sentry_release_organization_id_45a8c0c8_fk_sentry_or"
E   DETAIL:  Key (organization_id)=(4557743650111489) is not present in table "sentry_organization".

The above exception was the direct cause of the following exception:
src/sentry/testutils/cases.py:408: in _post_teardown
    super()._post_teardown()
.venv/lib/python3.13/site-packages/django/db/backends/postgresql/base.py:482: in check_constraints
    cursor.execute("SET CONSTRAINTS ALL IMMEDIATE")
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:122: in execute
    return super().execute(sql, params)
.venv/lib/python3.13/site-packages/sentry_sdk/utils.py:1870: in runner
    return original_function(*args, **kwargs)
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:79: in execute
    return self._execute_with_wrappers(
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
src/sentry/db/postgres/base.py:70: in _execute__include_sql_in_error
    return execute(sql, params, many, context)
src/sentry/db/postgres/base.py:58: in _execute__clean_params
    return execute(sql, clean_bad_params(params), many, context)
src/sentry/testutils/hybrid_cloud.py:133: in __call__
    return execute(*params)
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:100: in _execute
    with self.db.wrap_database_errors:
.venv/lib/python3.13/site-packages/django/db/utils.py:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
.venv/lib/python3.13/site-packages/django/db/backends/utils.py:103: in _execute
    return self.cursor.execute(sql)
src/sentry/db/postgres/decorators.py:16: in inner
    return func(self, *args, **kwargs)
src/sentry/db/postgres/base.py:96: in execute
    return self.cursor.execute(sql)
E   django.db.utils.IntegrityError: insert or update on table "sentry_release" violates foreign key constraint "sentry_release_organization_id_45a8c0c8_fk_sentry_or"
E   DETAIL:  Key (organization_id)=(4557743650111489) is not present in table "sentry_organization".
E   
E   SQL: SET CONSTRAINTS ALL IMMEDIATE

tests/snuba/api/endpoints/test_organization_events_histogram.py::OrganizationEventsHistogramEndpointTest::test_histogram_exclude_outliers_data_filter

— log

tests/snuba/api/endpoints/test_organization_events_histogram.py:924: in test_histogram_exclude_outliers_data_filter
    assert response.data == self.as_response_data(expected), f"failing for {array_column}"
E   AssertionError: failing for measurements
E   assert {'measurement... 'count': 1}]} == {'measurement... 'count': 4}]}
E     
E     Differing items:
E     {'measurements.foo': [{'bin': 0, 'count': 3}, {'bin': 1000, 'count': 0}, {'bin': 2000, 'count': 0}, {'bin': 3000, 'count': 0}, {'bin': 4000, 'count': 1}]} != {'measurements.foo': [{'bin': 0, 'count': 4}]}
E     
E     Full diff:
E       {
E           'measurements.foo': [
E               {
E                   'bin': 0,
E     -             'count': 4,
E     ?                      ^
E     +             'count': 3,
E     ?                      ^
E     +         },
E     +         {
E     +             'bin': 1000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 2000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 3000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 4000,
E     +             'count': 1,
E               },
E           ],
E       }

tests/sentry/release_health/release_monitor/test_metrics.py::MetricFetchProjectsWithRecentSessionsTest::test_monitor_release_adoption_with_filter

— log

tests/sentry/release_health/release_monitor/__init__.py:77: in test_monitor_release_adoption_with_filter
    assert results == {
E   AssertionError

tests/sentry/utils/test_snowflake.py::SnowflakeUtilsTest::test_generate_correct_ids — log

tests/sentry/utils/test_snowflake.py:39: in test_generate_correct_ids
    region = Region("test-region", 0, "http://testserver", RegionCategory.MULTI_TENANT)
E   NameError: name 'Region' is not defined

tests/sentry/utils/test_snowflake.py::SnowflakeUtilsTest::test_generate_correct_ids_with_region_sequence — log

tests/sentry/utils/test_snowflake.py:50: in test_generate_correct_ids_with_region_sequence
    region = Region("test-region", 0, "http://testserver", RegionCategory.MULTI_TENANT)
E   NameError: name 'Region' is not defined

tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_viewed — log

tests/sentry/replays/endpoints/test_organization_replay_index.py:183: in test_get_replays_viewed
    assert response_data["data"][0]["has_viewed"] is False, query
E   AssertionError: 
E   assert True is False

With session-scoped ClickHouse cleanup, data persists across tests within a session. Fix tests that assumed a clean ClickHouse state: - test_snowflake: Update Region→Cell after upstream rename - test_organization_events_histogram: Use dedicated project for outlier test so other tests' events don't affect detection - release_health tests: Scope assertions to test-owned orgs and mock discovery query to prevent IntegrityErrors from stale orgs - test_organization_replay_index: Add project filter to viewed replay query so other tests' replays don't contaminate results Co-Authored-By: Claude <noreply@anthropic.com>

github-actions · 2026-03-06T05:51:04Z

Backend Test Failures

Failures on cf1d7c8 in this run:

tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_viewed — log

tests/sentry/replays/endpoints/test_organization_replay_index.py:184: in test_get_replays_viewed
    assert response_data["data"][0]["has_viewed"] is False, query
E   AssertionError: ?project=4557743967698944
E   assert True is False

tests/snuba/api/endpoints/test_organization_events_histogram.py::OrganizationEventsHistogramEndpointTest::test_histogram_exclude_outliers_data_filter

— log

tests/snuba/api/endpoints/test_organization_events_histogram.py:938: in test_histogram_exclude_outliers_data_filter
    assert response.data == self.as_response_data(expected), f"failing for {array_column}"
E   AssertionError: failing for measurements
E   assert {'measurement... 'count': 1}]} == {'measurement... 'count': 4}]}
E     
E     Differing items:
E     {'measurements.foo': [{'bin': 0, 'count': 3}, {'bin': 1000, 'count': 0}, {'bin': 2000, 'count': 0}, {'bin': 3000, 'count': 0}, {'bin': 4000, 'count': 1}]} != {'measurements.foo': [{'bin': 0, 'count': 4}]}
E     
E     Full diff:
E       {
E           'measurements.foo': [
E               {
E                   'bin': 0,
E     -             'count': 4,
E     ?                      ^
E     +             'count': 3,
E     ?                      ^
E     +         },
E     +         {
E     +             'bin': 1000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 2000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 3000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 4000,
E     +             'count': 1,
E               },
E           ],
E       }

- Histogram test: use self.populate_events with temporary project swap instead of manual reimplementation that dropped events - Replay viewed test: assert by replay ID instead of response position to handle non-deterministic ClickHouse sort order - Exclude slot lock file from unclosed_files fixture to prevent flakes - Pipeline Redis commands in snowflake key preservation to reduce per-test overhead (~150 round trips → 3 per host) Co-Authored-By: Claude <noreply@anthropic.com>

Add release_slot() to close the lock fd and unlink the lock file. Called from pytest_sessionfinish so the slot is freed promptly instead of waiting for process exit / atexit. Co-Authored-By: Claude <noreply@anthropic.com>

github-actions · 2026-03-06T18:05:58Z

Backend Test Failures

Failures on dbac085 in this run:

tests/snuba/api/endpoints/test_organization_events_histogram.py::OrganizationEventsHistogramEndpointTest::test_histogram_exclude_outliers_data_filter

— log

tests/snuba/api/endpoints/test_organization_events_histogram.py:930: in test_histogram_exclude_outliers_data_filter
    assert response.data == self.as_response_data(expected), (
E   AssertionError: failing for measurements
E   assert {'measurement... 'count': 1}]} == {'measurement... 'count': 4}]}
E     
E     Differing items:
E     {'measurements.foo': [{'bin': 0, 'count': 3}, {'bin': 1000, 'count': 0}, {'bin': 2000, 'count': 0}, {'bin': 3000, 'count': 0}, {'bin': 4000, 'count': 1}]} != {'measurements.foo': [{'bin': 0, 'count': 4}]}
E     
E     Full diff:
E       {
E           'measurements.foo': [
E               {
E                   'bin': 0,
E     -             'count': 4,
E     ?                      ^
E     +             'count': 3,
E     ?                      ^
E     +         },
E     +         {
E     +             'bin': 1000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 2000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 3000,
E     +             'count': 0,
E     +         },
E     +         {
E     +             'bin': 4000,
E     +             'count': 1,
E               },
E           ],
E       }

populate_events() uses deepcopy(self.data) which preserves the same span_id across all events. When events share the same (project_id, finish_ts, transaction_name, span_id) sorting key, ClickHouse's ReplacingMergeTree deduplicates them during background merges. This was hidden on master because reset_snuba recreated tables before each test, preventing merges. With session-scoped ClickHouse cleanup, merges happen and rows get deduplicated. Fix: generate a unique span_id per event in populate_events().

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 5, 2026

Merge branch 'master' into enable-parallel-pytest

69e4b5b

vercel bot deployed to Preview March 5, 2026 21:50 View deployment

fix(tests): Resolve Region→Cell merge conflict in test config

7ee4987

Master renamed Region to Cell. Our branch had a partial merge leaving both constructors in the same assignment. Use Cell with the per-worker snowflake_id. Co-Authored-By: Claude <noreply@anthropic.com>

vercel bot deployed to Preview March 5, 2026 22:12 View deployment

vercel bot deployed to Preview March 5, 2026 22:20 View deployment

ci: Disable pull_request triggers on non-backend workflows

c42aae6

Only backend.yml retains its pull_request trigger so that this PR can run backend tests with -n2 without noise from unrelated workflows. Co-Authored-By: Claude <noreply@anthropic.com> (cherry picked from commit 0c01225)

vercel bot deployed to Preview March 5, 2026 22:52 View deployment

github-actions bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Mar 5, 2026

joshuarli added 2 commits March 5, 2026 15:12

skip other backend jobs for now

7ea0fc3

[skip ci] pretty and flush

3ece8ca

vercel bot deployed to Preview March 5, 2026 23:30 View deployment

joshuarli and others added 2 commits March 5, 2026 15:44

docs: Trim parallel test execution section in tests/AGENTS.md

c77806f

getsentry deleted a comment from github-actions bot Mar 5, 2026

vercel bot deployed to Preview March 5, 2026 23:48 View deployment

getsentry deleted a comment from github-actions bot Mar 6, 2026

[skip ci] docs: Update tests/AGENTS.md for worker shim architecture

42d1330

vercel bot deployed to Preview March 6, 2026 00:14 View deployment

vercel bot deployed to Preview March 6, 2026 00:29 View deployment

joshuarli and others added 5 commits March 5, 2026 17:36

fix(tests): Clean up parallel coordinator temp dir after run

ce7f33f

Each `-n` invocation created a /tmp/sentry_parallel_* directory with nodeid lists and JSONL result files that was never cleaned up. Now removed after all results are collected. Co-Authored-By: Claude <noreply@anthropic.com>

[skip ci]

0527e8d

vercel bot deployed to Preview March 6, 2026 02:07 View deployment

vercel bot deployed to Preview March 6, 2026 04:13 View deployment

vercel bot deployed to Preview March 6, 2026 05:25 View deployment

joshuarli and others added 2 commits March 6, 2026 09:39

fix(tests): Release slot lock file at pytest session end

7b25d4f

Add release_slot() to close the lock fd and unlink the lock file. Called from pytest_sessionfinish so the slot is freed promptly instead of waiting for process exit / atexit. Co-Authored-By: Claude <noreply@anthropic.com>

vercel bot deployed to Preview March 6, 2026 17:45 View deployment

vercel bot deployed to Preview March 6, 2026 20:39 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(tests): Add per-process test isolation and lean parallel runner#109994

feat(tests): Add per-process test isolation and lean parallel runner#109994
joshuarli wants to merge 24 commits intomasterfrom
enable-parallel-pytest

joshuarli commented Mar 5, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

joshuarli commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

What changed

Usage

Uh oh!

github-actions bot commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backend Test Failures

Uh oh!

github-actions bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backend Test Failures

Uh oh!

github-actions bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backend Test Failures

Uh oh!

github-actions bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backend Test Failures

Uh oh!

github-actions bot commented Mar 6, 2026

Backend Test Failures

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

joshuarli commented Mar 5, 2026 •

edited

Loading

github-actions bot commented Mar 6, 2026 •

edited

Loading

github-actions bot commented Mar 6, 2026 •

edited

Loading

github-actions bot commented Mar 6, 2026 •

edited

Loading

github-actions bot commented Mar 6, 2026 •

edited

Loading