-
Notifications
You must be signed in to change notification settings - Fork 748
Description
Hi,
I've installed the latest version of Boltz from the sources.
I'm experiencing an error related to the triton package:
Exception: Failed to import Triton-based component: triangle_multiplicative_update:
Not Supported
Please make sure to install triton==3.3.0. Other versions may not work!
Predicting DataLoader 0: 0%| | 0/1 [00:01<?, ?it/s]
But inspecting the version of the triton package I receive the following:
(boltz2-env) xxxx@xxxx-Predator-PHN16-71:~/boltz_2_affinity_example$ pip3 list | grep triton
triton 3.3.0
Any suggestion?
Also for the warning:
"You are using a CUDA device ('NVIDIA GeForce RTX 4070 Laptop GPU') that has Tensor Cores. To properly utilize them, you should set torch.set_float32_matmul_precision('medium' | 'high') which will trade-off precision for performance."
How it is possible to make the suggested settings when calling boltz?
Thanks.
Saverio
PS:
The following is the list of the used commands:
(pyenv) xxxx@xxxx-Predator-PHN16-71:$ source boltz2-env/bin/activate$ boltz --help
(boltz2-env) xxxx@xxxx-Predator-PHN16-71:
Usage: boltz [OPTIONS] COMMAND [ARGS]...
Boltz.
Options:
--help Show this message and exit.
Commands:
predict Run predictions with Boltz.
(boltz2-env) xxxx@xxxx-Predator-PHN16-71:$ cd boltz_2_affinity_example//boltz_2_affinity_example$ boltz predict /home/xxxx/sources/boltz-2.1.1/examples/affinity.yaml --use_msa_server
(boltz2-env) xxxx@xxxx-Predator-PHN16-71:
Checking input data.
All inputs are already processed.
Processing 0 inputs with 0 threads.
0it [00:00, ?it/s]
Using bfloat16 Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py:76: Starting from v1.9.0, tensorboardX has been removed as a dependency of the pytorch_lightning package, due to potential conflicts with other packages in the ML ecosystem. For this reason, logger=True will use CSVLogger as the default logger, unless the tensorboard or tensorboardX packages are found. Please pip install lightning[extra] or one of them to enable TensorBoard support by default
Running structure prediction for 1 input.
/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/utilities/migration/utils.py:56: The loaded checkpoint was produced with Lightning v2.5.0.post0, which is newer than your current Lightning version: v2.5.0
You are using a CUDA device ('NVIDIA GeForce RTX 4070 Laptop GPU') that has Tensor Cores. To properly utilize them, you should set torch.set_float32_matmul_precision('medium' | 'high') which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Predicting DataLoader 0: 0%| | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/xxxx/boltz2-env/bin/boltz", line 8, in
sys.exit(cli())
^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/main.py", line 1190, in predict
trainer.predict(
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 859, in predict
return call._call_and_handle_interrupt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/call.py", line 47, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 898, in _predict_impl
results = self._run(model, ckpt_path=ckpt_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 982, in _run
results = self._run_stage()
^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/trainer.py", line 1021, in _run_stage
return self.predict_loop.run()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/loops/utilities.py", line 179, in _decorator
return loop_run(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/loops/prediction_loop.py", line 125, in run
self._predict_step(batch, batch_idx, dataloader_idx, dataloader_iter)
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/loops/prediction_loop.py", line 255, in _predict_step
predictions = call._call_strategy_hook(trainer, "predict_step", *step_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/trainer/call.py", line 323, in _call_strategy_hook
output = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/pytorch_lightning/strategies/strategy.py", line 438, in predict_step
return self.lightning_module.predict_step(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/models/boltz2.py", line 1055, in predict_step
out = self(
^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/models/boltz2.py", line 464, in forward
z = z + template_module(
^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/modules/trunkv2.py", line 498, in forward
v = v + self.pairformer(v, pair_mask, use_kernels=use_kernels)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/layers/pairformer.py", line 331, in forward
z = layer(
^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/layers/pairformer.py", line 245, in forward
z = z + dropout * self.tri_mul_out(
^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/layers/triangular_mult.py", line 92, in forward
return kernel_triangular_mult(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/torch/_dynamo/eval_frame.py", line 838, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/boltz/model/layers/triangular_mult.py", line 23, in kernel_triangular_mult
return triangle_multiplicative_update(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/cuequivariance_torch/primitives/triangle.py", line 223, in triangle_multiplicative_update
return f(
^^
File "/home/xxxx/boltz2-env/lib/python3.12/site-packages/cuequivariance_ops_torch/init.py", line 68, in triangle_multiplicative_update
raise Exception(f"Failed to import Triton-based component: triangle_multiplicative_update:\n{IMPORT_EXCEPTION}\n"
Exception: Failed to import Triton-based component: triangle_multiplicative_update:
Not Supported
Please make sure to install triton==3.3.0. Other versions may not work!
Predicting DataLoader 0: 0%| | 0/1 [00:01<?, ?it/s]
(boltz2-env) xxxx@xxxx-Predator-PHN16-71:~/boltz_2_affinity_example$ pip3 list | grep triton
triton 3.3.0