Skip to content

Commit aea14fc

Browse files
authored
Seed the non qjit reference run of a test (#1757)
**Context:** The test `pytest/test_measurement_transforms.py::test_measurement_from_samples_with_sample_measurement` has a [non-qjit reference run](https://github.com/PennyLaneAI/catalyst/blob/02af052424ec4fd5ce7d4e846c3410d823e13c55/frontend/test/pytest/test_measurement_transforms.py#L445) that is unseeded and is causing some flaky test noise. **Description of the Change:** Since lightning.qubit does not support seeding, we use default.qubit for this reference run and seed it. **Benefits:** No more flaky CI failures on this test. **Possible Drawbacks:** I don't like using seeding to silence flaky tests. In our [guidance docs for flaky tests](https://github.com/PennyLaneAI/guidance-docs/blob/eae480fd82f88c5f922e4d1b5a324819a4690b8a/development/testing-guidelines.md?plain=1#L222), seeding is only the second suggested strategy. But adopting the first strategy of hypothesis testing would take more time. For now, the band-aid seed solution would have to do.
1 parent 2c47da8 commit aea14fc

File tree

1 file changed

+8
-4
lines changed

1 file changed

+8
-4
lines changed

frontend/test/pytest/test_measurement_transforms.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -438,11 +438,15 @@ def circuit(theta: float):
438438

439439
theta = 2.5
440440
res = qjit(measurements_from_samples(circuit, dev.wires), seed=37)(theta)
441-
441+
# PL flattens N-by-1 2D result arrays into size-N 1D arrays, but Catalyst does not
442442
if len(measurement().wires) == 1:
443-
samples_expected = qjit(circuit, seed=37)(theta)
444-
else:
445-
samples_expected = circuit(theta)
443+
res = res.flatten()
444+
445+
# lightning.qubit does not support seeding.
446+
# To resolve flakiness, we put the non qjit reference run on default.qubit,
447+
# which can be seeded
448+
ref_dev = qml.device("default.qubit", wires=4, shots=3000, seed=42)
449+
samples_expected = qml.qnode(ref_dev)(circuit.func)(theta)
446450

447451
assert res.shape == samples_expected.shape
448452
assert np.allclose(np.mean(res, axis=0), np.mean(samples_expected, axis=0), atol=0.05)

0 commit comments

Comments
 (0)