[Bug] Lightning GPU and Lightning Tensor support on Google Colab

[DSGuala]

Issue description

Expected behavior:

1. We can install lightning.gpu and lightning.tensor in a GPU-enabled google colab runtime (T4 GPU)
2. PennyLane can create a device using lightning.tensor and lightning.gpu in a GPU-enabled google colab runtime (T4 GPU).
3. The following code runs correctly:

!pip install custatevec_cu12
!pip install pennylane-lightning-gpu pennylane-lightning-tensor

import pennylane as qml

dev = qml.device('lightning.tensor', wires=3)

dev = qml.device('lightning.gpu', wires=3)

Actual behavior:

1. Installation runs without errors
2. Trying to create a device raises ImportError: Pre-compiled binaries for lightning.tensor are not available.

>>> dev = qml.device('lightning.tensor', wires=3)

ImportError: Pre-compiled binaries for lightning.tensor are not available.

dev = qml.device('lightning.gpu', wires=3)

ImportError: Pre-compiled binaries for lightning.tensor are not available.

Reproduces how often:

Every time

System information:

Name: PennyLane
Version: 0.39.0
Summary: PennyLane is a cross-platform Python library for quantum computing, quantum machine learning, and quantum chemistry. Train a quantum computer the same way as a neural network.
Home-page: https://github.com/PennyLaneAI/pennylane
Author:
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, packaging, pennylane-lightning, requests, rustworkx, scipy, toml, typing-extensions
Required-by: PennyLane_Lightning, PennyLane_Lightning_GPU, PennyLane_Lightning_Tensor

Platform info: Linux-6.1.85+-x86_64-with-glibc2.35
Python version: 3.10.12
Numpy version: 1.26.4
Scipy version: 1.13.1
Installed devices:

• default.clifford (PennyLane-0.39.0)
• default.gaussian (PennyLane-0.39.0)
• default.mixed (PennyLane-0.39.0)
• default.qubit (PennyLane-0.39.0)
• default.qutrit (PennyLane-0.39.0)
• default.qutrit.mixed (PennyLane-0.39.0)
• default.tensor (PennyLane-0.39.0)
• null.qubit (PennyLane-0.39.0)
• reference.qubit (PennyLane-0.39.0)
• lightning.gpu (PennyLane_Lightning_GPU-0.39.0)
• lightning.tensor (PennyLane_Lightning_Tensor-0.39.0)
• lightning.qubit (PennyLane_Lightning-0.39.0)

Source code and tracebacks

lightning.tensor full traceback:

/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_tensor/lightning_tensor.py:55: UserWarning: /usr/local/lib/python3.10/dist-packages/pennylane_lightning/../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLog_12_6, version libnvJitLink.so.12
  warn(str(ex), UserWarning)
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-1-b687fb8fe8cf>](https://localhost:8080/#) in <cell line: 3>()
      1 import pennylane as qml
      2 
----> 3 dev = qml.device('lightning.tensor', wires=3)

1 frames
[/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_tensor/lightning_tensor.py](https://localhost:8080/#) in __init__(self, wires, shots, method, c_dtype, **kwargs)
    286     ):
    287         if not self._CPP_BINARY_AVAILABLE:
--> 288             raise ImportError("Pre-compiled binaries for lightning.tensor are not available. ")
    289 
    290         if not accepted_methods(method):

ImportError: Pre-compiled binaries for lightning.tensor are not available. 

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

lightning.gpu full traceback:

/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_gpu/lightning_gpu.py:68: UserWarning: /usr/local/lib/python3.10/dist-packages/pennylane_lightning/../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLog_12_6, version libnvJitLink.so.12
  warn(str(ex), UserWarning)
/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_gpu/_adjoint_jacobian.py:46: UserWarning: /usr/local/lib/python3.10/dist-packages/pennylane_lightning/../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLog_12_6, version libnvJitLink.so.12
  warn(str(ex), UserWarning)
/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_gpu/_measurements.py:35: UserWarning: /usr/local/lib/python3.10/dist-packages/pennylane_lightning/../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLog_12_6, version libnvJitLink.so.12
  warn(str(error_import), UserWarning)
/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_gpu/_state_vector.py:32: UserWarning: /usr/local/lib/python3.10/dist-packages/pennylane_lightning/../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLog_12_6, version libnvJitLink.so.12
  warn(str(ex), UserWarning)
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-2-f85c5c9db4f9>](https://localhost:8080/#) in <cell line: 1>()
----> 1 dev = qml.device('lightning.gpu', wires=3)

1 frames
[/usr/local/lib/python3.10/dist-packages/pennylane_lightning/lightning_gpu/lightning_gpu.py](https://localhost:8080/#) in __init__(self, wires, c_dtype, shots, batch_obs, mpi, mpi_buf_size, use_async)
    330     ):
    331         if not self._CPP_BINARY_AVAILABLE:
--> 332             raise ImportError(
    333                 "Pre-compiled binaries for lightning.gpu are not available. "
    334                 "To manually compile from source, follow the instructions at "

ImportError: Pre-compiled binaries for lightning.gpu are not available. To manually compile from source, follow the instructions at https://docs.pennylane.ai/projects/lightning/en/stable/dev/installation.html.

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

[DSGuala]

CC: @multiphaseCFD

[mlxd]

The issue here is the cusparse and nvjitlink libraries seem to either be improperly versioned against one another, or no longer respect forward compatibility from our build against 12.0

These libraries are now installed by default due to JAX having GPU supported libs in the colab environment.

Downgrading the library versions serves as a temporary workaround from a notebook cell

!python -m pip install pennylane custatevec_cu12 pennylane-lightning-gpu "nvidia-cusparse-cu12~=12.0.0" "nvidia-cublas-cu12~=12.0.0" "nvidia-cuda-runtime-cu12~=12.0.0" "nvidia-nvjitlink-cu12~=12.0.0"

which should work as expected after a kernel restart.

I believe this to be a Colab / Nvidia library issue, rather than an "us" issue for now.