Firefox 137 miscompiles with LLVM 20 on x86_64 + musl + LTO #136509
Description
Building Firefox 137 with LTO+PGO configuration equivalent to the upstream builds yields a browser that frequently crashes with the following backtrace:
* thread #1, name = 'firefox', stop reason = signal SIGSEGV: address not mapped to object (fault address=0x3b8)
frame #0: 0x00007fffead0a363 libxul.so`mozilla::dom::BrowsingContext::Top() [inlined] RefPtr<mozilla::dom::WindowContext>::operator bool(this=<unavailable>) const at RefPtr.h:338:45
(lldb) bt
* thread #1, name = 'firefox', stop reason = signal SIGSEGV: address not mapped to object (fault address=0x3b8)
* frame #0: 0x00007fffead0a363 libxul.so`mozilla::dom::BrowsingContext::Top() [inlined] RefPtr<mozilla::dom::WindowContext>::operator bool(this=<unavailable>) const at RefPtr.h:338:45
frame #1: 0x00007fffead0a363 libxul.so`mozilla::dom::BrowsingContext::Top(this=0x0000000000000000) at BrowsingContext.cpp:222:10
frame #2: 0x00007fffebde79da libxul.so`mozilla::dom::BrowserSessionStore::UpdateSessionStore(mozilla::dom::CanonicalBrowsingContext*, mozilla::Maybe<mozilla::dom::sessionstore::FormData> const&, mozilla::Maybe<nsPoint> const&, unsigned int) [inlined] mozilla::dom::CanonicalBrowsingContext::Top(this=0x0000000000000000) at CanonicalBrowsingContext.h:114:66
frame #3: 0x00007fffebde79d2 libxul.so`mozilla::dom::BrowserSessionStore::UpdateSessionStore(mozilla::dom::CanonicalBrowsingContext*, mozilla::Maybe<mozilla::dom::sessionstore::FormData> const&, mozilla::Maybe<nsPoint> const&, unsigned int) [inlined] ShouldUpdateSessionStore(aBrowsingContext=<unavailable>, aEpoch=<unavailable>) at BrowserSessionStore.cpp:71:25
frame #4: 0x00007fffebde79d2 libxul.so`mozilla::dom::BrowserSessionStore::UpdateSessionStore(this=0x00007fffa9894f40, aBrowsingContext=<unavailable>, aFormData=<unavailable>, aScrollPosition=<unavailable>, aEpoch=<unavailable>) at BrowserSessionStore.cpp:245:8
frame #5: 0x00007fffebde8391 libxul.so`mozilla::dom::PSessionStoreParent::OnMessageReceived(IPC::Message const&) [inlined] mozilla::dom::SessionStoreParent::RecvIncrementalSessionStoreUpdate(this=0x00007fffae954800, aBrowsingContext=0x00007fffffffb778, aFormData=<unavailable>, aScrollPosition=<unavailable>, aEpoch=<unavailable>) at SessionStoreParent.cpp:209:20
frame #6: 0x00007fffebde8380 libxul.so`mozilla::dom::PSessionStoreParent::OnMessageReceived(this=0x00007fffae954800, msg__=<unavailable>) at PSessionStoreParent.cpp:344:86
frame #7: 0x00007fffebeaa1d3 libxul.so`mozilla::dom::PContentParent::OnMessageReceived(this=<unavailable>, msg__=0x00007fffe30a3880) at PContentParent.cpp:6738:32
frame #8: 0x00007fffea63fdf6 libxul.so`mozilla::ipc::MessageChannel::DispatchMessage(mozilla::ipc::ActorLifecycleProxy*, mozilla::UniquePtr<IPC::Message, mozilla::DefaultDelete<IPC::Message>>) [inlined] mozilla::ipc::MessageChannel::DispatchAsyncMessage(this=0x00007fff8ce45a80, aProxy=0x00007fffca072a80, aMsg=0x00007fffe30a3880) at MessageChannel.cpp:1789:25
frame #9: 0x00007fffea63fd8d libxul.so`mozilla::ipc::MessageChannel::DispatchMessage(this=0x00007fff8ce45a80, aProxy=0x00007fffca072a80, aMsg=UniquePtr<IPC::Message, mozilla::DefaultDelete<IPC::Message> > @ 0x00007fffffffcdc8) at MessageChannel.cpp:1716:9
frame #10: 0x00007fffea63f71b libxul.so`mozilla::ipc::MessageChannel::MessageTask::Run() [inlined] mozilla::ipc::MessageChannel::RunMessage(this=0x00007fff8ce45a80, aProxy=0x00007fffca072a80, aTask=0x00007fffe304f860) at MessageChannel.cpp:1507:3
frame #11: 0x00007fffea63f640 libxul.so`mozilla::ipc::MessageChannel::MessageTask::Run(this=0x00007fffe304f860) at MessageChannel.cpp:1607:14
frame #12: 0x00007fffea63e269 libxul.so`mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [inlined] mozilla::RunnableTask::Run(this=0x00007fffe3146b20) at TaskController.cpp:703:16
frame #13: 0x00007fffea63e246 libxul.so`mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(mozilla::detail::BaseAutoLock<mozilla::Mutex&> const&) [inlined] mozilla::TaskController::RunTask(aTask=0x00007fffe3146b20) at TaskController.cpp:228:71
frame #14: 0x00007fffea63e246 libxul.so`mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal(this=0x00007ffff38db900, aProofOfLock=<unavailable>) at TaskController.cpp:1250:20
frame #15: 0x00007fffea5b6566 libxul.so`NS_ProcessNextEvent(nsIThread*, bool) [inlined] mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal(this=0x00007ffff38db900, aProofOfLock=0x00007fffffffd130) at TaskController.cpp:1073:15
frame #16: 0x00007fffea5b655b libxul.so`NS_ProcessNextEvent(nsIThread*, bool) [inlined] mozilla::TaskController::ProcessPendingMTTask(this=0x00007ffff38db900, aMayWait=false) at TaskController.cpp:639:36
frame #17: 0x00007fffea5b654f libxul.so`NS_ProcessNextEvent(nsIThread*, bool) [inlined] mozilla::TaskController::TaskController()::$_0::operator()(this=<unavailable>) const at TaskController.cpp:333:37
frame #18: 0x00007fffea5b6540 libxul.so`NS_ProcessNextEvent(nsIThread*, bool) [inlined] mozilla::detail::RunnableFunction<mozilla::TaskController::TaskController()::$_0>::Run(this=<unavailable>) at nsThreadUtils.h:548:5
frame #19: 0x00007fffea5b6540 libxul.so`NS_ProcessNextEvent(nsIThread*, bool) [inlined] nsThread::ProcessNextEvent(this=0x00007ffff38db780, aMayWait=false, aResult=0x00007fffffffcfbd) at nsThread.cpp:1159:16
frame #20: 0x00007fffea5b5f1f libxul.so`NS_ProcessNextEvent(aThread=0x00007ffff38db780, aMayWait=false) at nsThreadUtils.cpp:480:10
frame #21: 0x00007fffea63b484 libxul.so`mozilla::ipc::MessagePump::Run(this=0x00007ffff385b040, aDelegate=0x00007ffff38db180) at MessagePump.cpp:85:21
frame #22: 0x00007fffea5333e1 libxul.so`MessageLoop::Run() [inlined] MessageLoop::RunInternal(this=<unavailable>) at message_loop.cc:369:10
frame #23: 0x00007fffea5333d5 libxul.so`MessageLoop::Run() [inlined] MessageLoop::RunHandler(this=<unavailable>) at message_loop.cc:362:3
frame #24: 0x00007fffea5333d5 libxul.so`MessageLoop::Run(this=<unavailable>) at message_loop.cc:344:3
frame #25: 0x00007fffea63b3a6 libxul.so`nsBaseAppShell::Run(this=0x00007ffff3f31800) at nsBaseAppShell.cpp:148:27
frame #26: 0x00007fffea63cbac libxul.so`nsAppShell::Run(this=<unavailable>) at nsAppShell.cpp:470:33
frame #27: 0x00007fffec86ab11 libxul.so`nsAppStartup::Run(this=0x00007ffff3ef6730) at nsAppStartup.cpp:291:30
frame #28: 0x00007fffea70c197 libxul.so`XREMain::XRE_mainRun(this=<unavailable>) at nsAppRunner.cpp:5866:22
frame #29: 0x00007fffea617c97 libxul.so`XREMain::XRE_main(this=0x00007fffffffd498, argc=<unavailable>, argv=<unavailable>, aConfig=<unavailable>) at nsAppRunner.cpp:6106:8
frame #30: 0x00007fffea617963 libxul.so`XRE_main(argc=1, argv=0x00007fffffffe6f8, aConfig=0x00007fffffffd680) at nsAppRunner.cpp:6179:21
frame #31: 0x00005555555740f1 firefox`main [inlined] do_main(argc=<unavailable>, argv=<unavailable>, envp=<unavailable>) at nsBrowserApp.cpp:232:22
frame #32: 0x000055555557407b firefox`main(argc=<unavailable>, argv=<unavailable>, envp=<unavailable>) at nsBrowserApp.cpp:464:16
frame #33: 0x00007ffff7ed6e3d libc.so
frame #34: 0x000055555558cd9a firefox`_start + 22
With LLVM 19 this did not use to happen (verified the same version of the browser). Trying to follow the logic of the code makes it seem like Top() should never return NULL but here it does. Not sure if this is a miscompilation in the browser caused by a toolchain bug, or whether it's a bug in the Firefox codebase exposed by a newer compiler.
Mozilla bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1961538