[Metal] Examples not working or crashing on macOS and iOS

[dolphineye74]

Hi,

I have been giving a shot to LLGL and I wanted to test it on macOS and iOS (Metal backend) and a couple of tutorials seem to have issues

Macbook system specs:

• Model: Macbook Pro
• SoC: Apple M1 Max
• RAM: 64 GB
• OS: MacOS Sonoma 14.7

iPhone system specs:

• Model: iPhone X
• OS: iOS 16.7.10

Hello triangle (iOS)

This tutorial shows just a black screen and no triangle.

The app will then start leaking memory by hundreds of megs per second similar to what was reported in #129 on macOS.

Strangely, the Texturing tutorial has no issue at all if that rings a bell :)

Render target (macOS & iOS)

The render target tutorial will crash as follows on macOS (or iOS):

I managed to fix it by changing the default value of ENABLE_DEPTH_TEXTURE to 1.

I could post a PR that adds a platform check to enable depth on Apple platforms by default.

But I can't tell if that's ever supposed to happen in the first place.

Shadow mapping (macOS & iOS)

The shadow map tutorial clearly has issues as shown below:

On iOS, I do have a black screen as well.

Strangely, the HelloGame example seems to have properly working shadow maps 🤷‍♂️

Fonts (macOS & iOS)

Similar issue as with the shadow map it seems:

Similar results on iOS with a black background screen instead of a pink one.

Stencil buffer (macOS & iOS)

The whole screen is pink, and I start to suspect these tutorials all have the same core issue

On iOS, I have just a black screen instead of a pink screen (but no memory leak like in the hello triangle tutorial).

[dolphineye74]

In the meantime I will try to investigate a few of those issues in case if find something.

[dolphineye74]

I made some progress regarding the reported issues.

On macOS, it appears the swap chain's sample count causes issues when > 1.

Using a value of 1x sample count fixes the following tutorials:

• Fonts (macOS & iOS)
• Stencil buffer (macOS only)
• Shadow mapping (macOS only)

[dolphineye74]

On iOS, it appears that disabling depth test for the scene PSO will draw the room & the box.
But obviously depth testing is broken.

Then, I tried to change the number of bits for the swap chain desc (32 bits for depth and 0/8 bits for stencil) but this didn't fix the issue (nothing on screen).

[LukasBanana]

Thanks for the great report!
I haven't been able to test this on an M1-4 Mac until just now as my personal one is still an old Intel Mac, which I hopefully am able to replace soon. I was able to reproduce the described issues as well as your fix with disabling multi-sampling but not the memory leaks you described although I know from previous reports that this is also still a lingering problem.
There also seems to be a problem with the depth-stencil buffer in iOS as you noticed; Not just the StencilBuffer example but the depth test also appears to be disabled in some other examples.
So all of that tells me that the Metal backend is in rough shape right now and needs some love again :)
I'll have more time looking into this in about two weeks when I'm back from traveling.

[LukasBanana]

Strangely, the HelloGame example seems to have properly working shadow maps 🤷‍♂️

I also noticed that the HelloGame example (as one of the more complex examples) works actually pretty decent even in the iOS Simulator, except for some broken input event handling 🤦‍♀️

[dolphineye74]

Thanks for the great report! I haven't been able to test this on an M1-4 Mac until just now as my personal one is still an old Intel Mac, which I hopefully am able to replace soon. I was able to reproduce the described issues as well as your fix with disabling multi-sampling but not the memory leaks you described although I know from previous reports that this is also still a lingering problem. There also seems to be a problem with the depth-stencil buffer in iOS as you noticed; Not just the StencilBuffer example but the depth test also appears to be disabled in some other examples. So all of that tells me that the Metal backend is in rough shape right now and needs some love again :) I'll have more time looking into this in about two weeks when I'm back from traveling.

Thanks for the quick heads up. Metal can be quite a pain to get right. If memory serves me well, we had some weird cases where Metal would endlessly create drawables in some conditions on macOS (especially when switching between virtual desktops). If you cache your drawables, you might rapidly get nasty memleaks.

No rush for a fix, keep us posted whenever you get a chance to work on the Metal backend again. Feel free to ping me whenever you need a round of testing again :)

[LukasBanana]

I found a fix for the multi-sample issue but this will need a revision when I have more time. The render-pass interruptions due to buffer updates caused the multi-sampled attachments to loose their content, same issue with the depth-stencil buffers.
I am still looking into the StencilBuffer issue on iOS.

[dolphineye74]

Thanks for taking a look already!

On macOS, all the samples appear to be working well except for the render target tutorial that needs an explicit depth attachment via ENABLE_DEPTH_TEXTURE (crash issue on both macOS & iOS). We're almost getting there on macOS!

On iOS, this is better alreadt but it seems we're not completely out of the woods yet :

• The stencil tutorial shows something on screen but the stencil buffer isn't working as expected (macOS is fine now).
• The shadows tutorial will now properly render both the cube & the room but shadows won't work at all

To stay on a positive note, the fonts tutorials works as expected on iOS now 🎉

[dolphineye74]

For the Hello Triangle memory leak, I tried to explicitly submit the commands to the command queue, but it didn't seem to change anything. I did a minimal test with an empty command buffer, and just calling Begin and End seem to cause the issue.
I tried a similar setup in the textured cube tutorial and it seems just fine.

I'm starting to speculate the issue is related to the specific setup of the Hello Triangle tutorial.
It doesn't inherit the ExampleBase class and has its own main loop and we are perhaps overwhelming iOS in some ways.