Shorten boot time

[Lander0606]

Hi all,

I was investigating the boot time and process of an image on the Jetson Orin Nano. On average, the boottime is around 12-13 seconds which is quite long for an arm64 device with 6 cores.

Investigating the different steps and initcalls in the boot process, I came across this:

---

Top 10 initcall durations:
 * tegra_pcie_dw_driver_init -> 4467701us
 * tegra_bpmp_driver_init -> 4193836us
 * clk_disable_unused -> 163179us
 * deferred_probe_initcall -> 120094us
 * tegra_p2u_driver_init -> 69377us
 * init_encrypted -> 60460us
 * optee_core_init -> 59314us
 * arm_smmu_driver_init -> 36234us
 * fusb301_i2c_driver_init -> 34756us
 * tegra_bpmp_thermal_driver_init -> 32982us

---

Top 10 probe durations:
 * bpmp -> 4193443us
 * 141e0000.pcie -> 2047080us
 * 14160000.pcie -> 2046793us
 * 140a0000.pcie -> 246941us
 * 3400000.mmc -> 226069us
 * 14100000.pcie -> 126565us
 * bpmp:thermal -> 61638us
 * firmware:optee -> 59066us
 * 1-0025 -> 52823us
 * 1-2 -> 36737us

---

Failed initcalls/probes:
 * acpi_ac_init -> ret = -19
 * acpi_thermal_init -> ret = -19
 * acpi_battery_init -> ret = -19
 * ptp_kvm_init -> ret = -95
 * tegra_cpufreq_init -> ret = -19
 * set_hardened_usercopy -> ret = -1
 * efi_shutdown_init -> ret = -19

I was able to decrease the boottime by almost 4 seconds down to 8 by just disabling some unused pcie interfaces in the device tree, which eliminates the influence of tegra_pcie_dw_driver_init in the boot. However, to decrease the boot time even more, we need to handle with the bpmp and other initcalls (see above).

I'm not that familiar with the (workings of) BPMP, but I know it's one of the most important and most needed components on the Jetson. Is there a way to decrease the high initcall duration (almost 4 seconds)? Does anyone experience the same?

Another thing could be to disable the initramfs. But, when I tried to disable it with INITRAMFS_IMAGE_${MACHINE} as described in https://github.com/orgs/OE4T/discussions/652, the initramfs just stays active and present in the image. Did something change to the initramfs creation in meta-tegra since that post, or is there another way to disable the initramfs?

I would love to hear your opinions!

[madisongh]

If you haven't already, check the section in the L4T documentation on boot time optimization. The link there is for the latest (36.4.3), but check for the version you're based on, since there would definitely be differences, particularly R32 vs. R35 vs. R36.

For specific questions about the behavior of the drivers, it might be better to ask in the NVIDIA developer forum.

[Lander0606]

Hi @madisongh,
We did indeed follow the boot time optimization guide already. But isn't it crazy that the bpmp kernel driver takes more than 4 seconds to initialize? If I take a deeper look in the bpmp kernel driver source code, I can see that this is caused by a busy-wait for a reaction of the bpmp.

Unfortunately, discussing this on the NVIDIA developer forum didn't bring any extra information: https://forums.developer.nvidia.com/t/jetson-bsp-long-init-times-for-the-bpmp/330713

Is it really better to use the in-tree bpmp driver? Or to just disable the bpmp driver?

[madisongh]

I don't really know if you can disable it altogether, as I expect that would cause trouble with power and thermal management. As to why it's taking so long for the driver to come up, you might want to drop a question about that in the NVIDIA forums, if you haven't already.