Normalization during training

[iacopo97]

Good morning, thank you for your amazing work!!! I would like to know when you trained a network on MUSES which kind of normalization you operate for event, lidar and radar data?

[timbroed]

Hi,

We calculate the channel statistics for each projected sensor with this script and use them to normalize the inputs before passing them to the network.

In our case, these were:

cfg.DATASETS.PIXEL_MEAN.CAMERA = [123.675, 116.280, 103.530]
cfg.DATASETS.PIXEL_MEAN.LIDAR = [4.91737, 6.149373, 0.27025607]
cfg.DATASETS.PIXEL_MEAN.RADAR = [4.003564, 9.342789, 0.]
cfg.DATASETS.PIXEL_MEAN.EVENT_CAMERA = [0.12577528, 0.12728328, 0.]

cfg.DATASETS.PIXEL_STD.CAMERA = [58.395, 57.120, 57.375]
cfg.DATASETS.PIXEL_STD.LIDAR = [13.34715295, 17.63751258, 1.23208644]
cfg.DATASETS.PIXEL_STD.RADAR = [14.60685343, 19.29406236, 1.] 
cfg.DATASETS.PIXEL_STD.EVENT_CAMERA = [0.54420582, 0.47625199, 1.]

Best,
Tim

[iacopo97]

Thank you for your answer, have you calculated those statistics after doing the enlargement of events reprojection to rgb and lidar points and radar points?

[iacopo97]

My statistics is different even if I have used your code, It is:
EVENT
{
"mean": [
0.03909787692901231,
0.038359852237654306,
0.0
],
"std": [
0.29815072313316443,
0.2712491760103713,
0.0
]
LIDAR
{
"mean": [
0.3850609064102173,
0.479247123003006,
0.008021799847483635
],
"std": [
3.9262807751879203,
5.091488178740392,
0.3687743626354274
]
}
RADAR
{
"mean": [
0.2816745340824127,
0.4606277346611023,
0.0
],
"std": [
3.6539596201268068,
4.530797783975901,
0.0
]
}

[timbroed]

yes, we did calculate it after the enlargement

[iacopo97]

Sorry but after the projection and the enlargement my event mean lidar mean radar mean and their corresponding standard deviation are different than yours

[iacopo97]

I have done some tests and the kernel size to obtain that values of mean and std that you have shown in the enlargement step should be (3,3) for the event and not (2,2), while the radar is correct with kernel size (10,10), in the case of the lidar the kernel size needed should be (4,4)

[timbroed]

I am sorry for the confusion, I took these numbers from my most recent project, so they might not be the exact ones we used for the original paper submission.

In practice, we did not see large performance differences as long as the normalization parameters were roughly in the same ballpark.

[iacopo97]

Ok, thank you, therefore the enlargement has been operated with (2,2) kernel for events and lidar and with kernel (10,10) for radar