Apple M1 / Teleport / Signal stack?

[fzwoch]

Hey peeps,

Here is a technical question I hope someone can shed some light upon, or give back some thoughts or ideas. Here is the current situation encountered so far:

The OBS Teleport plugin is written in Go. It seems to work quite well on Linux amd64, Windows amd64 and macOS amd64.

With the port of OBS to macOS arm64 for the M1/M2 it is discovered that the plugin crashes - sometimes upon loading, sometimes at a later point interacting with properties etc. In particular it is the Go runtime that crashes, not some plugin specific code.

Also the behavior is quite erratic. When OBS is build and run with Xcode's Address Sanitizer the plugin seems to load fine, but may crash at a later point. But also changing around some SVG resources for the status bar seems to have an impact where the crash finally happens.

So it sounds to be some big memory/stack corruption is happening here, but I cannot pinpoint it. It seems more related to the Go runtime and/or how it loads itself than the plugin's code as the issue is reproducible with a very minimal plugin.

Playing around with different things I came across the signal stack. Out of curiosity I ended up with this block of code:

--- a/UI/obs-app.cpp
+++ b/UI/obs-app.cpp
@@ -3055,6 +3055,19 @@ void ctrlc_handler(int s)
 
 int main(int argc, char *argv[])
 {
+#if defined __APPLE__ && defined __arm64__
+    char signalStack[SIGSTKSZ];
+    stack_t ss;
+    memset(&ss, 0, sizeof(ss));
+    ss.ss_sp = signalStack;
+    ss.ss_flags = 0;
+    ss.ss_size = SIGSTKSZ;
+    if (sigaltstack(&ss, NULL) < 0) {
+        perror("sigaltstack");
+        abort();
+    }
+#endif
+
 #ifndef _WIN32
 	signal(SIGPIPE, SIG_IGN);

For some reason this ends up in the most reliable functional plugin yet. But unfortunately the signal stack as well how this interacts with Go is not my strongest expertise - or shall I say, I have no clue at all?

So now I wonder:

• Does this code block actually change something interesting, or does it just modify the memory layout in a way that it just works out of coincidence?
• Do the other platforms simply work out of coincidence, and I have overseen something critical all this time?
• What does the above code change that makes the Go runtime not crash this time around?
• What does the above code change that may break existing things?
• Whelp!

[fzwoch]

There is a way to keep Go out of the tech stack. The above code block:

char signalStack[SIGSTKSZ];
stack_t ss;
memset(&ss, 0, sizeof(ss));
ss.ss_sp = signalStack;
ss.ss_flags = 0;
ss.ss_size = SIGSTKSZ;
if (sigaltstack(&ss, NULL) < 0) {
  perror("sigaltstack");
  abort();
}

can be used to track some weird behaviour. (The Go runtime will use this to allocate an alternate signal stack)

At some point within initing the OBS Qt GUI the use of sigaltstack() will fail with EPERM. I can't make a sense out of it why.

In ui_OBSBasic.h:

statusbar = new OBSBasicStatusBar(OBSBasic);

is the critical point. Before that line the code block succeeds, after it it fails. I think this file gets autogenerated?

I can do the following change to fix the issue:

--- a/UI/window-basic-status-bar.cpp
+++ b/UI/window-basic-status-bar.cpp
@@ -31,7 +31,7 @@ OBSBasicStatusBar::OBSBasicStatusBar(QWidget *parent)
                                          .pixmap(QSize(20, 20))),
          streamingActivePixmap(QIcon(":/res/images/streaming-active.svg")
                                        .pixmap(QSize(20, 20))),
-         streamingInactivePixmap(QIcon(":/res/images/streaming-inactive.svg")
+         streamingInactivePixmap(QIcon(":/res/images/streaming-active.svg")
                                          .pixmap(QSize(20, 20)))
 {
        streamTime->setText(QString("LIVE: 00:00:00"));

Well, it is not a fix, but in runs in this case. Why I have no idea. Is there something wrong with the inactive svg? Is there something wrong with the QtSVG library generated by the obs-build-deps package?

When I build OBS manually with CMake with the local deps from brew I don't see the issue. I only encounter them with the CI build scripts - and therefore probably also with the release versions of OBS.

EDIT: The following patch fixes it for me:

--- a/UI/forms/images/streaming-inactive.svg
+++ b/UI/forms/images/streaming-inactive.svg
@@ -7,7 +7,6 @@
 <style type="text/css">
        .st0{fill:#9a9996;}
 </style>
-<g>
        <path class="st0" d="M6.5,0.8L4.6,2.6C4.4,2.6,4.2,2.5,4,2.5C3.2,2.5,2.5,3.2,2.5,4c0,0.2,0.1,0.4,0.1,0.6L2.2,5.1
                C2,4.8,1.8,4.4,1.8,4c0-0.6,0.3-1.2,0.7-1.6c0.1-0.1,0.1-0.3,0-0.4C2.5,1.9,2.3,1.9,2.2,2C1.6,2.5,1.2,3.2,1.2,4
                c0,0.6,0.2,1.1,0.5,1.5L1.5,5.8C1.1,5.3,0.8,4.7,0.8,4c0-0.8,0.4-1.6,1-2.1c0.1-0.1,0.1-0.3,0-0.4c-0.1-0.1-0.3-0.1-0.4,0
@@ -17,5 +16,4 @@
                c0.1,0,0.1,0,0.2-0.1c0.6-0.5,0.9-1.2,0.9-2C6.8,3.4,6.6,2.9,6.2,2.5z"/>
        <path class="st0" d="M6.9,1.8L6.5,2.2C6.9,2.7,7.2,3.3,7.2,4c0,0.8-0.4,1.6-1,2.1c-0.1,0.1-0.1,0.3,0,0.4c0.1,0.1,0.1,0.1,0.2,0.1
                c0.1,0,0.1,0,0.2-0.1C7.4,5.9,7.8,5,7.8,4C7.8,3.2,7.5,2.4,6.9,1.8z"/>
-</g>
 </svg>

But it is a bogus fix as then, the normal CMake build breaks..