Node crashes when setting lower minimum FOV

[harborhoffer]

I am attempting to use this package by way of using a ZEDx stereo camera with the depthimage-to-laserscan package to obtain a LaserScan which is then used by this package (with laserscan_to_flatscan_node). When setting min_scan_fov_degrees to anything below the default 270.0, I get the following error.

[component_container_mt-2] 2024-04-13 15:50:51.705 ERROR extensions/localization/flatscan_localization/grid_search_localizer.cpp@568: Global localization failed to find a valid sample
[component_container_mt-2] 2024-04-13 15:50:51.705 ERROR gxf/std/entity_executor.cpp@529: Failed to tick codelet grid_search_localizer in entity: QSKPDRWWKJ_global_localization code: GXF_FAILURE
[component_container_mt-2] 2024-04-13 15:50:51.749 WARN  gxf/std/greedy_scheduler.cpp@250: Error while executing entity 37 named 'QSKPDRWWKJ_global_localization': GXF_FAILURE
[component_container_mt-2] 2024-04-13 15:50:52.121 ERROR gxf/std/entity_executor.cpp@210: Entity with eid 52 not found!
[component_container_mt-2] [WARN] [1713037852.121822048] [occupancy_grid_localizer]: [NitrosNode] The heartbeat entity (eid=52) was stopped. The graph may have been terminated.
[component_container_mt-2] [INFO] [1713037852.169526339] [occupancy_grid_localizer]: [NitrosNode] Terminating the running application
[component_container_mt-2] [INFO] [1713037852.169571651] [occupancy_grid_localizer]: [NitrosContext] Interrupting GXF...
[component_container_mt-2] [INFO] [1713037852.169626531] [occupancy_grid_localizer]: [NitrosContext] Waiting on GXF...
[component_container_mt-2] 2024-04-13 15:50:52.169 ERROR gxf/std/program.cpp@556: wait failed. Deactivating...
[component_container_mt-2] |==================================================================================================================================================================|
[component_container_mt-2] |                                           Job Statistics Report (regular)                                                                                        |
[component_container_mt-2] |==================================================================================================================================================================|
[component_container_mt-2] | Name                                               |   Count | Time (Median - 90% - Max) [ms] | Load (%) | Exec(ms) | Variation (Median - 90% - Max) [ns]        |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] |                                    QSKPDRWWKJ_sink |       0 |     0.00 |     0.00 |     0.00 |  100.0 % |      0.0 |            0 |            0 |            0 |
[component_container_mt-2] |            QSKPDRWWKJ_flatscan_beams_tensor_copier |       1 |     0.36 |     0.36 |     0.36 |  100.0 % |      0.4 |            0 |            0 |            0 |
[component_container_mt-2] |                             QSKPDRWWKJ_broadcaster |       1 |     0.05 |     0.05 |     0.05 |  100.0 % |      0.0 |            0 |            0 |            0 |
[component_container_mt-2] |                     QSKPDRWWKJ_global_localization |       0 |     0.00 |     0.00 |     0.00 |  100.0 % |      0.0 |            0 |            0 |            0 |
[component_container_mt-2] |                  QSKPDRWWKJ_localization_to_pose3d |       0 |     0.00 |     0.00 |     0.00 |  100.0 % |      0.0 |            0 |            0 |            0 |
[component_container_mt-2] |==================================================================================================================================================================|
[component_container_mt-2] |==================================================================================================================================================================|
[component_container_mt-2] |                                           Entity Statistics Report (regular)                                                                                     |
[component_container_mt-2] |==================================================================================================================================================================|
[component_container_mt-2] | Entity Name             | Entity State             |   Count | Time (Median - 90% - Max) [ms]                                                                    |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] |         QSKPDRWWKJ_sink  |              StopPending |       1 |  0.00406 |  0.00406 |  0.00406                                                                   |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] | QSKPDRWWKJ_flatscan_bea  |              StopPending |       1 |  0.02176 |  0.02176 |  0.02176                                                                   |
[component_container_mt-2] | QSKPDRWWKJ_flatscan_bea  |                     Idle |       1 | 298.86599 | 298.86599 | 298.86599                                                                   |
[component_container_mt-2] | QSKPDRWWKJ_flatscan_bea  |                  Ticking |       1 |  0.28112 |  0.28112 |  0.28112                                                                   |
[component_container_mt-2] | QSKPDRWWKJ_flatscan_bea  |                  Pending |       1 |  0.04198 |  0.04198 |  0.04198                                                                   |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] |  QSKPDRWWKJ_broadcaster  |              StopPending |       1 |  0.00534 |  0.00534 |  0.00534                                                                   |
[component_container_mt-2] 2024-04-13 15:50:52.223 ERROR gxf/core/runtime.cpp@1408: Graph wait failed with error: GXF_FAILURE
[component_container_mt-2] [ERROR] [1713037852.223302498] [occupancy_grid_localizer]: [NitrosContext] GxfGraphWait Error: GXF_FAILURE
[component_container_mt-2] [INFO] [1713037852.223317506] [occupancy_grid_localizer]: [NitrosNode] Application termination done
[component_container_mt-2] |  QSKPDRWWKJ_broadcaster  |                     Idle |       1 | 298.79629 | 298.79629 | 298.79629                                                                   |
[component_container_mt-2] |  QSKPDRWWKJ_broadcaster  |                  Ticking |       1 |  0.03360 |  0.03360 |  0.03360                                                                   |
[component_container_mt-2] |  QSKPDRWWKJ_broadcaster  |                  Pending |       1 |  0.00838 |  0.00838 |  0.00838                                                                   |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] | QSKPDRWWKJ_global_local  |              StopPending |       1 | 44.07559 | 44.07559 | 44.07559                                                                   |
[component_container_mt-2] | QSKPDRWWKJ_global_local  |                  Ticking |       1 | 254.60102 | 254.60102 | 254.60102                                                                   |
[component_container_mt-2] | QSKPDRWWKJ_global_local  |                  Pending |       1 |  0.00682 |  0.00682 |  0.00682                                                                   |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] | QSKPDRWWKJ_localization  |              StopPending |       1 |  0.00934 |  0.00934 |  0.00934                                                                   |
[component_container_mt-2] |------------------------------------------------------------------------------------------------------------------------------------------------------------------|
[component_container_mt-2] |==================================================================================================================================================================|
[component_container_mt-2] 2024-04-13 15:50:52.266 WARN  external/com_nvidia_isaac_engine/engine/core/allocator/cached_allocator.cpp@132: Potentially lost 158460 bytes of memory (allocated total: 285228)
[INFO] [component_container_mt-2]: process has finished cleanly [pid 76333]

[willdzeng]

Hi Dave,
Thanks for reporting this issue, do you see this printout when you set min_scan_fov_degrees to below 270?

    GXF_LOG_ERROR(
        "Got a scan that has FOV %f smaller than threshold %f, maybe scanner is covered? ignoring "
        "this scan",
        scan_fov_degree, min_scan_fov_degrees_.get());

The actual issue seems related to the global localization search failed to find a valid sample that are within the tolerance of what it expected, probably due to the quality of the flatscan from a depth image being too noisy.
I would try to set max_beam_error to a higher value see if that would help.
I would also suggest you upload a small ros bag that just contains the flatscan and it's corresponding map so we can better debug this issue.

[aditya-roshAI]

Has this issue been resolved?