VRAM usage increase in `main` vs `v0.1.12`

[scarrazza]

After porting the changes to support more than 32 qubits on a single GPU in complex128 (based on mi300x), I realized that I cannot run a 32-qubit complex128 simulation using the main branch due to memory unavailability.

In contrast, v0.1.12 works correctly, and the corresponding changes there allow me to exceed 34 qubits using complex64 (mi300x_v0.1.12).

@renatomello and @BrunoLiegiBastonLiegi, could you please double-check if the backend refactoring or any Clifford-related changes in main since v0.1.12 are duplicating state vectors? It seems like memory is being consumed much faster than expected.

[renatomello]

@scarrazza Do you have a minimum circuit to reproduce the behavior?

[scarrazza]

I am using the default setup for qibojit-benchmarks (QFT 32 qubits).

[renatomello]

@scarrazza could you test your new modifications using the apply_gate branch as base?

[scarrazza]

Thanks @renatomello, indeed, apply_gate consumes the same amount of memory of v0.1.12 so I can simulate 32 qubits in complex128 as is and go up to 34 qubits with my patch. Please open a PR for apply_gate, I will then test it and open my patch on top.

[renatomello]

Thanks @renatomello, indeed, apply_gate consumes the same amount of memory of v0.1.12 so I can simulate 32 qubits in complex128 as is and go up to 34 qubits with my patch. Please open a PR for apply_gate, I will then test it and open my patch on top.

I will. Everything is working with complex128, but there is an issue with cupy and complex64 that needs more investigation. numba is working with both complex128 and complex64.

EDIT: @scarrazza everything is done (qiboteam/qibo#1792 and #243).