Skip to content

Commit eb915f8

Browse files
authored
fix: querier integration flaky test
Saw this test fail: https://github.com/cortexproject/cortex/actions/runs/10761255507/job/29840284232 Adding this test option to make the test wait until the metric is available. ``` querier_test.go:231: Error Trace: /home/runner/work/cortex/cortex/integration/querier_test.go:231 Error: Received unexpected error: metric not found github.com/cortexproject/cortex/integration/e2e.init <autogenerated>:1 runtime.doInit1 /opt/hostedtoolcache/go/1.22.5/x64/src/runtime/proc.go:7176 runtime.doInit /opt/hostedtoolcache/go/1.22.5/x64/src/runtime/proc.go:7143 runtime.main /opt/hostedtoolcache/go/1.22.5/x64/src/runtime/proc.go:253 runtime.goexit /opt/hostedtoolcache/go/1.22.5/x64/src/runtime/asm_amd64.s:1695 metric=cortex_bucket_store_blocks_loaded service=store-gateway-1 github.com/cortexproject/cortex/integration/e2e.(*HTTPService).SumMetrics /home/runner/work/cortex/cortex/integration/e2e/service.go:619 github.com/cortexproject/cortex/integration/e2e.(*CompositeHTTPService).SumMetrics /home/runner/work/cortex/cortex/integration/e2e/composite_service.go:78 github.com/cortexproject/cortex/integration/e2e.(*CompositeHTTPService).WaitSumMetricsWithOptions /home/runner/work/cortex/cortex/integration/e2e/composite_service.go:55 github.com/cortexproject/cortex/integration/e2e.(*CompositeHTTPService).WaitSumMetrics /home/runner/work/cortex/cortex/integration/e2e/composite_service.go:44 github.com/cortexproject/cortex/integration.TestQuerierWithBlocksStorageRunningInMicroservicesMode.func1 /home/runner/work/cortex/cortex/integration/querier_test.go:231 testing.tRunner /opt/hostedtoolcache/go/1.22.5/x64/src/testing/testing.go:1689 runtime.goexit /opt/hostedtoolcache/go/1.22.5/x64/src/runtime/asm_amd64.s:1695 Test: TestQuerierWithBlocksStorageRunningInMicroservicesMode/blocks_sharding_disabled,_memcached_index_cache,thanosEngine=false ``` Signed-off-by: Charlie Le <[email protected]>
1 parent bc69e73 commit eb915f8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

integration/querier_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,7 @@ func TestQuerierWithBlocksStorageRunningInMicroservicesMode(t *testing.T) {
228228
// then the metric only appears in one store gateway instance.
229229
require.NoError(t, storeGateways.WaitSumMetricsWithOptions(e2e.Equals(2), []string{"cortex_bucket_store_blocks_loaded"}, e2e.SkipMissingMetrics))
230230
} else {
231-
require.NoError(t, storeGateways.WaitSumMetrics(e2e.Equals(float64(2*storeGateways.NumInstances())), "cortex_bucket_store_blocks_loaded"))
231+
require.NoError(t, storeGateways.WaitSumMetricsWithOptions(e2e.Equals(float64(2*storeGateways.NumInstances())), []string{"cortex_bucket_store_blocks_loaded"}, e2e.WaitMissingMetrics))
232232
}
233233

234234
// Check how many tenants have been discovered and synced by store-gateways.

0 commit comments

Comments
 (0)