Updated feature for Related Website Sets and Apple app site association #159

tsunoyu · 2025-03-05T16:51:55Z

Enhance Privacy Sandbox Related Website Sets and Apple App Site Association Tracking

This pull request enhances the tracking of Privacy Sandbox Related Website Sets (RWS) features and Apple App Site Association by improving data collection and parsing for:

Privacy Sandbox:
- Related Website Sets: Parses the /.well-known/related-website-set.json file to extract detailed information about website relationships, including primary domain, associated sites, service sites, ccTLDs, and rationale.
Apple App Site Association: Parses the /.well-known/apple-app-site-association file to detect the presence of applinks, webcredentials, activitycontinuation, and appclips services.

Changes:

Updated parseResponse function to handle and parse the following files:
- /.well-known/related-website-set.json
- /.well-known/apple-app-site-association

Rationale:

Privacy Sandbox:
- Related Website Sets: Collecting detailed information about website relationships provides valuable insights into the usage and adoption of this feature.
Apple App Site Association: Detecting the presence of different services in the apple-app-site-association file helps us understand how websites integrate with Apple features like universal links, shared web credentials, and App Clips.

Test websites:

tunetheweb

LGTM. I'm hopefully fixing the linting error in #754 so will merge this after that one.

…ome keys

tsunoyu · 2025-03-06T10:48:42Z

@tunetheweb After reviewing the test results, I noticed that the implementation was collecting more data than intended. I’ve made a new commit to refine the logic and ensure it aligns with the expected behavior. Is Lint Code Base error something I need to further action?

tunetheweb · 2025-03-06T10:52:00Z

Is Lint Code Base error something I need to further action?

Just committed the fix and updated this branch with the last code. Fingers crossed that fixes it!!

tsunoyu · 2025-03-06T11:02:50Z

@tunetheweb Thank you for your review and support. Would it be possible to run one more test with following additonal URLs to check the collected data?

https://mercadolibre.com/
https://google.com/ (instead of https://www.google.com/)

dist/well-known.js

tunetheweb · 2025-03-06T11:59:08Z

Would it be possible to run one more test with following additonal URLs to check the collected data?

Doing that last commit to remove the trailing bank space will trigger the re-test. You can always do a dummy commit (e.g; alter a comment) if you need to do this again.

https://google.com/ (instead of https://www.google.com/)

Let's see how this goes in this latest test run, but suspect the auto-redirect will ruin this here. Maybe the custom metric should also look at domain as well as origin?

tsunoyu · 2025-03-06T12:21:04Z

@tunetheweb is this something you could help? I was reviwing the code but could not figure out what is causing this issue.

I am seeing incorrect "found: false" response when trying to crawl /.well-known/related-website-set.json particularly for https://google.com/ .
Testing the https://google.com/ withi this new logic it should be able to access the https://google.com/.well-known/related-website-set.json path which has following information.

{
   "contact": "[email protected]",
   "primary": "https://google.com",
   "associatedSites": [
      "https://youtube.com",
      "https://android.com"
   ],
   "serviceSites": [
      "https://googleusercontent.com"
   ],
   "rationaleBySite": {
      "https://youtube.com": "Clearly presents an affiliation with Google.com via shared branding, account login, and privacy policy. Used for preserving user journeys related to sign-in / authentication.",
      "https://android.com": "Clearly presents an affiliation with Google.com via shared branding, account login, and privacy policy. Used for preserving user journeys related to sign-in / authentication.",
      "https://googleusercontent.com": "Secure content serving"
   }
}

However the test result shows

{
  "/.well-known/related-website-set.json": {
    "found": false
  }
}

For https://mercadolibre.com it is working as intended with the response below

    "/.well-known/related-website-set.json": {
      "found": true,
      "data": {
        "primary": "https://mercadolibre.com",
        "associatedSites": [
          "https://mercadolivre.com",
          "https://mercadopago.com",
          "https://mercadoshops.com",
          "https://portalinmobiliario.com",
          "https://tucarro.com"
        ]
      }
    }

tunetheweb · 2025-03-06T12:25:43Z

Yes this is what I thought would happen as mentioned about your test is for site https://google.com/ which then redirects to https://www.google.com/. Then you load /.well-known/related-website-set.json. Since that's a relative path, it loads relative to the current URL not the original test URL.

You could try changing it to load $WPT_TEST_URL/.well-known/related-website-set.json (I can't remember if $WPT_TEST_URL is the original test URL but think it is).

However, if testing www.google.com then shouldn't it look at both /.well-known/related-website-set.json and google.com/.well-known/related-website-set.json? So should your custom metric test both URLs?

pmeenan · 2025-03-06T14:47:13Z

FWIW, you should probably parse $WPT_TEST_URL and use the parts to construct the relative url because it won't always be the top-level, it will be whatever URL was tested (and yes, it is the URL that was submitted for the test, not whatever page finally ended up loading).

…ome keys

tsunoyu · 2025-03-07T15:00:09Z

CORS Issue Preventing Data Fetch

I tried running the following code in the browser console:

fetch("https://google.com/.well-known/related-website-set.json")
  .then(response => response.json())
  .then(console.log)
  .catch(console.error);

However, the request was blocked due to CORS policy restrictions:

Access to fetch at 'https://google.com/.well-known/related-website-set.json' 
from origin 'https://www.google.com' has been blocked by CORS policy: 
No 'Access-Control-Allow-Origin' header is present on the requested resource.

Could it be the case that because of this restriction, we are unable to retrieve the required data.

Since this issue prevents us from fetching data, it might be alright just go with the previous commit:

🔗 d1f41d45721607abe7549efc13a8be59742e711b

and try to get the data if it is not blocked. Or are we able to get data even there is a block by CORS policy?

tunetheweb · 2025-03-07T15:37:49Z

Ah CORS. Damn you. Yeah I think you're right so stuck with only measuring same-origin. Can you revert to that version and then let's merge this.

github-actions · 2025-03-07T16:27:09Z

https://almanac.httparchive.org/en/2022/

WPT result details