Training IDNet

[AbdessamadElK]

Hello, thank you for this amazing work, it is really interesting.

I am struggling with the training of IDNet and I wonder if you could help me with that. I am trying to reproduce the same results in your paper, so I trained the ID network (id-8x) on the DSEC dataset with the default configuration in "train_id.yaml", except that I trained for 250000 steps (following the paper) instead of 100 epochs.

However, I didn't get the same results as you did, I got an EPE of 0.837 instead of 0.77 on the DSEC test set. Please note that in my experiment there was no warm-starting between different predictions at different timestamps, which I think is the default configuration in your code.

Can you please indicate, if you have any idea, what I might have done wrong.

Thank you in advance for your help.

[yilun-wu]

Thanks for checking out our work! There could be many reasons why the accuracy is not as high as it can be. Some of the things that come to my mind: longer training time, proper data augmentation, using a lr scheduler.
For your reference, below is the loss / validation curve of the corresponding run for that config:

[AbdessamadElK]

Hello, thank you very much for answering my question. I really don't know where could the problem be, I will check again and verify the things that you mentioned. And Thank you for providing the curves, it will be very helpful for comparison.

I also noticed in your code that the deblurring of events is performed on the voxel grid representations instead of raw events. Is that technically the same ?

[yilun-wu]

Technically they are not the same, but warping individual slice of voxel grid is (in this case) a good enough approximation and could benefit from the parallel tensor operations provided by torch cuda.

[AbdessamadElK]

Okay, I see, it is really interesting. Thank you very much for the explanation.
I congratulate you for this amazing work.

[AbdessamadElK]

Hello, I actually ran the training without sanity check.

…

On Wed, 12 Feb 2025, 00:36 JamesYang, ***@***.***> wrote: Thanks for checking out our work! There could be many reasons why the accuracy is not as high as it can be. Some of the things that come to my mind: longer training time, proper data augmentation, using a lr scheduler. For your reference, below is the loss / validation curve of the corresponding run for that config: [image: Image] <https://private-user-images.githubusercontent.com/10345786/411291935-b236c51e-ae8c-4c5f-aeb2-35f6b3c1cd63.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkzMTU1MjIsIm5iZiI6MTczOTMxNTIyMiwicGF0aCI6Ii8xMDM0NTc4Ni80MTEyOTE5MzUtYjIzNmM1MWUtYWU4Yy00YzVmLWFlYjItMzVmNmIzYzFjZDYzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjExVDIzMDcwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTcyNWM2OTU0YzQzZDg5MDQzYWZkYWFjYzIxMGU5NzQ1MmM1ZjMxMDQ3ZjkyOThkZmFkMjc5NzE0MDVkYjljOTcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.md9kOYMdSUIMKunBVAHUinGEUBds5Csz7gElfuulEks> Hello, may I ask if the parameter configuration(id_train.yaml) for this experiment is set as validator.sanity_run_step = 1? — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASEJQDZPY2RQ3HKKVIDOPTL2PKCONAVCNFSM6AAAAABWTKKCHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJSGI4TQOJVGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[JamesYang110043]

Hello, I actually ran the training without sanity check.
…

On Wed, 12 Feb 2025, 00:36 JamesYang, @.***> wrote:
Thanks for checking out our work! There could be many reasons why the
accuracy is not as high as it can be. Some of the things that come to my
mind: longer training time, proper data augmentation, using a lr scheduler.
For your reference, below is the loss / validation curve of the
corresponding run for that config: [image: Image]
https://private-user-images.githubusercontent.com/10345786/411291935-b236c51e-ae8c-4c5f-aeb2-35f6b3c1cd63.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkzMTU1MjIsIm5iZiI6MTczOTMxNTIyMiwicGF0aCI6Ii8xMDM0NTc4Ni80MTEyOTE5MzUtYjIzNmM1MWUtYWU4Yy00YzVmLWFlYjItMzVmNmIzYzFjZDYzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjExVDIzMDcwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTcyNWM2OTU0YzQzZDg5MDQzYWZkYWFjYzIxMGU5NzQ1MmM1ZjMxMDQ3ZjkyOThkZmFkMjc5NzE0MDVkYjljOTcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.md9kOYMdSUIMKunBVAHUinGEUBds5Csz7gElfuulEks

Hello, may I ask if the parameter configuration(id_train.yaml) for this
experiment is set as validator.sanity_run_step = 1?

—
Reply to this email directly, view it on GitHub
<#8 (comment)>, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ASEJQDZPY2RQ3HKKVIDOPTL2PKCONAVCNFSM6AAAAABWTKKCHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJSGI4TQOJVGA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Thank you for your response. Additionally, I’m curious—does the 250k steps in your experiment refer to counting one step at the end of each batch? Also, how long did the entire run take? Thanks!

[AbdessamadElK]

Hi there, yes exactly, I count a step at the end of each batch (3 samples). The original code was a bit slow in dataloading, so to speed things up, I saved the events as voxel grids and loaded them directly. Therefore, it toke me about 30 hours to train on the DSEC train set for 100 epochs using an RTX3090.

[JamesYang110043]

Hi there, yes exactly, I count a step at the end of each batch (3 samples). The original code was a bit slow in dataloading, so to speed things up, I saved the events as voxel grids and loaded them directly. Therefore, it toke me about 30 hours to train on the DSEC train set for 100 epochs using an RTX3090.

Hi @AbdessamadElK , Thank you for your response.
Does storing the event as a voxel grid and loading them directly in mean skipping the event_to_voxel_grid function? Or is it done in another way? Also, is there a similar approach for the ground-truth flow? Looking forward to your response!

[AbdessamadElK]

Hello, yes exactly I used that function to convert events to voxel grids with 15 temporal bins and stored them, then I load them during training directly without the need for that function. Note that by doing this, if you want to change the number of bins, you'll have to do the same operation again. This does not pose a problem if you are using the same number of bins every time, which is the case for me.

For optical flow, I think that loading it from png files using imageio is fast enough so there is no need to save it in another format. However, you can save it as an ".npy" file using numpy but that would be the same anyway, I guess.

[JamesYang110043]

Hello, yes exactly I used that function to convert events to voxel grids with 15 temporal bins and stored them, then I load them during training directly without the need for that function. Note that by doing this, if you want to change the number of bins, you'll have to do the same operation again. This does not pose a problem if you are using the same number of bins every time, which is the case for me.

For optical flow, I think that loading it from png files using imageio is fast enough so there is no need to save it in another format. However, you can save it as an ".npy" file using numpy but that would be the same anyway, I guess.

Thank you for your response, it made things very clear for me. Thanks again!

[AbdessamadElK]

Hello @yilun-wu, I am still unable to reproduce the same results, and it is really important for me to do so before I can move on.

I run the training with the exact same default configuration that is set in the repository, but I get the same result every time. I train for 100 epochs with a batch size of 3 and I save the model after 250k steps. I made sure that the data augmentation is right (crop to 288x384, 50% probability of horizontal flip and 10% probability of vertical flip), and the used scheduler is the OneCycleLR with pct_start of 0.05 and a linear anneal strategy.

I feel like there is something that I am missing. Could you please confirm that using the default configuration in the repository is the right thing to do ?

I thank you in advance for your help.

[yilun-wu]

Sorry that it took you long to reproduce the result. I just checked the wandb log (the one with the training/validation curve I attached above) and for that experiment I used a batch size of 5 and trained for 150 epochs. The rest of the hyper-parameters in idn/config/id_train.yaml are correct.

[AbdessamadElK]

Hello @yilun-wu, thank you very much for your answer. I will run another training using these parameters and see what I will get, hopefully it will be the same.

Thank you for your time.