Compatibility checking for iOS Social Apps

[milesgreen]

A huge pain point for many of us (judging by the number of conversations I've had about it) is the lack of AR QuickLook support in many iOS social apps. This presents a real problem when we - or our clients - start sharing content on social, only to find that users following links on that platform can have a (sometimes very) broken experience.

How are people currently working around this?

My thoughts as follows:

Current compatibility testing on iOS when attempting to open a USDZ in AR QuickLook shows:

• Facebook - throws a network error
• Instagram - does nothing
• LinkedIn - displays a page of inline garbage text
• Twitter - works!

Twitter works because the native app is using the newer SFSafariViewController component. Which is the right thing to be using for an in-app web browser. SFSafariViewController is fully compatible with AR QuickLook, so it just works.

The other social apps here are using the WKWebView component. Which is not the right thing to be using for an in-app web browser. WKWebView is not immediately compatible with AR QuickLook and unless app developers specifically implement solutions, it'll break.

See here for Apple's official guidelines of when to use SFSafariViewController vs WKWebView:
https://developer.apple.com/news/?id=trjs0tcd

Still, social apps are using WKWebView, so what are we to do?

One solution would be for Apple to bring AR QuickLook support to WKWebView. A request has been made here (go here to add support):
https://bugs.webkit.org/show_bug.cgi?id=240117

In the meantime, can we just use feature detection? Well...

In theory, it is possible to do feature detection for AR QuickLook support using code like so (as described here: https://webkit.org/blog/8421/viewing-augmented-reality-assets-in-safari-for-ios/):

const a = document.createElement("a");
if (a.relList.supports("ar")) {
  // AR is available.
}

But...

There is a bug with WKWebView where a.relList.supports("ar") always returns true, regardless if there is support or not.

This false positive causes ModelViewer.canActivateAR to return inaccurate results. Issue logged here:
#3526

WKWebView issue is also logged with WebKit and will hopefully be fixed in iOS 16:
https://bugs.webkit.org/show_bug.cgi?id=239135

So what else can be done in the meantime?

It is possible to check if the current page has been opened inside a WKWebView using the following script:

if (window.webkit && window.webkit.messageHandlers) {
   // inside a WKWebView
} else {
   // not inside a WKWebView
}

But...

This will not only be true for the pesky social apps, it'll also be true for all third party browsers on iOS (EG Chrome, Edge, Firefox, Opera, etc), as they too are all built on top of WKWebView.

We don't want to block all browsers, as some do work with viewing USDZs.

Support for viewing USDZ varies from browser to browser. iOS Chrome works well and as of v102 supports auto USDZ generation. Edge, Firefox, Duck Duck Go all support opening USDZ (but not auto USDZ). Opera and Brave fail to open USDZ at all.

I don't know of any other feature detection or indicators to differentiate between those browsers that do or don't support USDZ. Grateful for any suggestions at this point.

Alternatively, it's possible to test the browser agent string and simply opt back in the shortlist of known supported browsers (EG, Chrome, Edge, etc) and attempt to show USDZ for them.

As for the incompatible WebView/App/Browsers, what do we show users?

It's possible to display an error message and encourage them to view the content in a supported browser.

What's not possible is to automatically, or with a single click, transition them over to a compatible browser. Any attempts to open an HTTPS link, even if targeted to a new window, will open within the current WKWebView.

There's no iOS URL scheme to directly open Safari or, better yet, the default web browser. A request has been filed with WebKit (go here to add support):
https://bugs.webkit.org/show_bug.cgi?id=240025

It's possible to provide a 'Copy URL' button to copy URL to the clipboard. But the user still has to manually switch over to their preferred app.

So, there are challenges, with some workarounds, but quite a few missing pieces to the puzzle. That's the extent of my exploration so far.

Any other tips or tricks for navigating the compatibility maze?

[elalish]

@grorg Any tips from the Apple side?