Floating Point Exception

[jyothsna-phd22]

Hi,

While I was training any model on kitti dataset, I am getting floating point exception at epoch 0 and terminated training. How to resolve this issue??

Kindly help me....

[beijbom]

Same here. I am training cfgs/nuscenes_models/cbgs_pp_multihead.yaml on the nuScenes mini dataset. @jyothsna-phd22 : were you able to resolve the issue?

[beijbom]

Tracked down the offending call. It's tv.from_numpy(points) on this line: https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/datasets/processor/data_processor.py#L55.

[jyothsna-phd22]

Where should we change in this line??

…

On Wed, 7 Aug, 2024, 9:32 am Oscar Beijbom, ***@***.***> wrote: Tracked down the offending call. It's tv.from_numpy(points) on this line: https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/datasets/processor/data_processor.py#L55 . — Reply to this email directly, view it on GitHub <#1642 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6SEC5F3VJVK53IPGBH5JALZQGL5BAVCNFSM6AAAAABLZ5O3FCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZSGU3TAMRUHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[beijbom]

I don't yet know how to fix it. I was just sharing that it seems to be the offending line.

[beijbom]

On digging further, it seems this is due to a mismatch between the binaries of the spconv library and the rest of the environment. I got it to run using the dockerfile in this PR: #1648

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[Sun-Cold]

Same issue here，this is my env info：
python 3.10.14
CUDA 12.1
numpy 2.0.1
spconv-cu120 2.3.6
Here is my solution for reference: downgrade numpy version to 1.26.4
conda install numpy=1.26.4

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[likecaulfield]

Same issue here，this is my env info： python 3.10.14 CUDA 12.1 numpy 2.0.1 spconv-cu120 2.3.6 Here is my solution for reference: downgrade numpy version to 1.26.4 conda install numpy=1.26.4

It worked for me too thank you!