Ability to declare a browser as part of the OT for testing purposes

[lbdvt]

Hi,

There seems to be a difference between (a) having a Chrome instance part of the OT and (b) running a Chrome instance with flag enabling features in the OT, namely in the fact that in (a) access to the Privacy Sandbox APIs is gated to having OT tokens properly set, whereas in (b) it is not.

It leads to test cases working fine in test environments (using start-up flags), but failing in live traffic, due to incorrect token set-up. While token set-up is most of the time straightforward, it can get complex in case of multiple domains involved, javascript dependencies, etc.

For testing and debugging purposes, it would be nice to be able to declare a Chrome browser instance as part of the OT (through specific start-up flags for instance), which would then behave as any other OT Chrome browser (i.e. require valid tokens for feature activation).

[maudnals]

Hi, thank you for the report, we understand the issue.
Let us check on feasibility / constraints to the proposed solution, and follow up here.

[samdutton]

Hi @lbdvt ! Apologies for the delay in responding.

You can run Chrome with the PrivacySandboxAdsAPIsOverride flag to enable the features supported by the Privacy Sandbox Relevance and Measurement origin trial.

For origin trial participation, take a look at our origin trial checklist for troubleshooting tokens and trial activation.

[lbdvt]

Thanks @samdutton for the reply.

When running Chrome with the PrivacySandboxAdsAPIsOverride flag, OT tokens are not required for accessing Privacy Sandbox origin trial features, correct?

What we are actually looking for is a mode to run Chrome as a user part of the OT: with tokens required for accessing Privacy Sandbox origin trial features.

[samdutton]

Apologies — got delayed responding, again!

When running Chrome with the PrivacySandboxAdsAPIsOverride flag, OT tokens are not required for accessing Privacy Sandbox origin trial features, correct?

That's correct (though look at documentation for individual APIs for more detail about flag usage, such as setting epoch length for Topics). Feature activation by using flags or with a trial token is to allow different types of testing:

* Command-line flags or chrome://flags are used for activating features to allow testing by an individual user, such as a developer or assigned tester.
* Feature activation by using an origin trial token allows testing at scale with end users (who, generally, should not set flags).

What we are actually looking for is a mode to run Chrome as a user part of the OT: with tokens required for accessing Privacy Sandbox origin trial features.

Normally, origin trial features are activated for any user who has an appropriate version of Chrome, on any page that provides a valid token. In this case, the user generally doesn't need to do anything for a trial feature to be made available (though there may be user settings, or other factors, that interfere with feature activation). In other words, they don't need to set flags.

For the Privacy Sandbox Relevance and Measurement, unlike for most trials, the APIs are currently only available for a percentage of users:
• 1% of desktop and Android users.
• 50% of users on Chrome Canary, Dev and Beta.
• APIs are not available on iOS Chrome.

See https://developer.chrome.com/docs/privacy-sandbox/unified-origin-trial/ for updates.

Getting started with Chrome's origin trials has more information about how to enrol in origin trials, and different ways provide tokens to activate trial features.

I'm not sure that this answers your question — let me know if it doesn't!

[lbdvt]

Hi,

For the Privacy Sandbox Relevance and Measurement, unlike for most trials, the APIs are currently only available for a percentage of users:
• 1% of desktop and Android users.
• 50% of users on Chrome Canary, Dev and Beta.
• APIs are not available on iOS Chrome.

Thanks, so I understand that the situation is specific to the Privacy Sandbox origin trial.

So rephrasing the original request: "As a tester of the Privacy Sandbox API, I need on my local machine a Chrome browser that is part of the OT, in order to test the integration I've done for the OT, including token validation. How can I do that?".

[samdutton]

Setting chrome://flags#privacy-sandbox-ads-apis enabled will give you the same experience as having a valid origin trial token provided and being included in the experiment group for the trial. (In other words, there's no need to force inclusion in the group.)

For example, opening https://ps-origin-trial.glitch.me/no-ot-token.html with the flag enabled should show all the APIs as enabled.

You can check in DevTools to validate if a trial token is valid (even if your browser is not part of the experiment group).

Are you seeing a difference between running with the flag enabled and being part of the origin trial?

[lbdvt]

Hi,

Thanks for the reply, but I don't think it answers our use case.

Are you seeing a difference between running with the flag enabled and being part of the origin trial?

Yes, if I'm not mistaken:

• running with the flag enabled: no need to provide a token to access the PS API
• being part of the origin trial: token is required to access the PS API

Our use case is "testing that we provide the token in the right way". Indeed while token set-up is most of the time straightforward, it can get complex in case of multiple domains involved, javascript dependencies, etc.

So what we would need is a flag forcing the browser instance in the experiment group for the trial while still requiring a token to activate the APIs.

[samdutton]

Our use case is "testing that we provide the token in the right way". Indeed while token set-up is most of the time straightforward, it can get complex in case of multiple domains involved, javascript dependencies, etc.

So what we would need is a flag forcing the browser instance in the experiment group for the trial while still requiring a token to activate the APIs.

Got it — thanks for the additional explanation.

Since you would need to use a flag for the testing you describe (i.e. this would be testing with individual users, not testing at scale) could you not just get the user to check DevTools for "testing that we provide the token in the right way" (as described here).

Valid token, browser (Chrome Beta, for me) is in the experimental group:

Valid token, browser (Chrome Stable, for me) is not in the experimental group:

In theory, you could use --force-fieldtrials with the StudyName/GroupName, but the GroupName is not fixed. Also, since you would need to get an individual user (i.e. a tester/developer) to set the flag, you might as well just get them to check DevTools.

[lbdvt]

Sorry for the late reply, and thanks for the answer.

What we'd like to be able to test is the following setup ("As a tester of the Privacy Sandbox API, I need on my local machine a Chrome browser that is part of the OT, in order to test the integration I've done for the OT, including token validation. How can I do that?"):

1. Running a Chrome stable version with PS API enabled
2. Providing an OT token
3. Calling the Privacy Sandbox API

From what I understand:

• Running Chrome with chrome://flags#privacy-sandbox-ads-apis enabled covers 1 & 3
• Token validation through DevTools covers 2

But it would really be helpful to be able to test all in one go, to debug some integration cases (e.g. "providing a valid token but after calling the PS API").

[samdutton]

Once again, apologies for the late reply — just realised I never actually posted my response to this.

If I understand correctly, you're trying to debug situations where you expect the Privacy Sandbox APIs to be enabled, but they're not.

Token validity
As you say, this can be checked from DevTools.

API availability
See How should developers feature detect API support?

There's no way to detect in JavaScript if a valid origin trial token has been provided for the current context (i.e. checking token usage, not feature support). Likewise, it's not possible to check with JavaScript if a user belongs to an experimental group (i.e. if the Privacy Sandbox APIs have been activated for a user in a context for which a valid trial token has been provided). In other words, before calling a Privacy Sandbox trial API, it's not possible to ask 'Has a valid token been provided, and is this browser in the experiment group for which the APIs are activated?'

So, for debugging use cases like the one you describe — a valid token is provided for the current context, but only after a trial API has already been called unsuccessfully — I think the best you can do is to detect if an API is supported and allowed, before calling the API, then log the cases where an API wasn't available, and check trial token code/timing for those contexts.

I'm not sure this answers your question, so feel free to keep asking.

@rowan-m @maudnals @kevinkiklee You may have more/better suggestions!