strip or anonymize non-critical telemetry from requests

[chuckadams]

Given that upstream has shown it's willing to abuse telemetry data for its ongoing vendetta, we need to strip out any data from outgoing requests that isn't essential to the function of the endpoint. We won't use it ourselves, and upstream doesn't deserve to have it. If some of the telemetry items turn out to be required, we should anonymize it as much as possible.

[asirota]

See here.

The repo

Automattic does take credit for this (not WordPress.org mind you).

Some design considerations

• a global AP_KILL_ALL_TELEMETRY flag that covers all telemetry data from the /check endpoints - need to articulate them and monitor which they are and change manage properly (a project all by itself)

• a selective Telemetry picker in the AU configuration in an advanced features tab

• what telemetry would AP never wish to suppress at least in our implementation?

[asirota]

Also do we pass telemtetry if our plugin is not rewriting? Another consideration. We don't pass a UA field at all right now.

[namithj]

Move to Discussions?

[asirota]

@namithj nah -- we can converse here... Then we can propose to discussions -- nobody other than core team uses GitHub anyways. Most stick to slack due to laziness and lack of GitHub knowledge.

[costdev]

I'm not sure that a single constant is the right call for this one.

I think this is very similar to the debugging setup.

• Two constants:
  • (bool) AP_ENABLE_TELEMETRY - Default: false
  • (array) AP_ENABLE_TELEMETRY_TYPES - Default: []
• UI:
  • Enable telemetry - All default to unchecked. Checking this one reveals the others.
    • Send X
    • Send Y
    • Send Z

This gives the user full control while still allowing us to collect telemetry, but only with consent. It also means a shared user and developer experience for the debugging and telemetry features.

For the code that strips each piece of data, it would run by default (i.e. telemetry disabled). When the user/site owner has consented for that piece of data to be sent, the stripping is skipped, but the code would still anonymize whatever isn't anonymized already (such as the URL in the user-agent).

I'm don't think we need to have this behind AP_ENABLE though. Much like the debug types, users should be able to avail of the feature even if they're just using the dotorg API.

[chuckadams]

Let's put together a list of every telemetry item that gets sent to each endpoint, along with an indicator of whether it's necessary at all, whether it's necessary in cleartext (i.e. can be anonymized), and what part of the request it is (GET/POST/Header along with key name). A table would do nicely. Maybe a wiki page so that others can edit it?

I'd start the page, but I don't have write access :)

[costdev]

This quickly generated list might be of use, but we should absolutely verify all the finer details of it before we consider it accurate or complete. If nothingnelse, it might hint at some requests we didn't think of, or it might trigger an idea of other possible requests that it might have missed.

Do not consider the sample values to be accurate representations of what is actually generated. For example, the User Agent sample doesn't include the URL.

[toderash]

You can't realistically use the API without telemetry - how are you going to check if there's an update to version. 4.2 of XYZ plugin without revealing the obvious? You'd have to completely rewrite the whole API in a way that makes it very inefficient.

My take on it is to simplify the whole question by anonymizing the domain before sending it - hashing it will provide a unique (enough) string that we can still get a count of websites using the updater without actually knowing who they are. At that point, nothing we have is traceable to a particular site, and we only need to store it as aggregate per node in order to get the stats we need. Simple approach, privacy question fully resolved, and no possibility of bad actors using site-specific data.

Is there anything in there that's uniquely identifiable besides the domain? I don't think so. IP address doesn't meet that criteria and is important data as-is for knowing network and geography to suggest things like needing another node Europe. We would need to process it for specific purposes before aggregating it, but that could be done before storing it.

I hadn't thought we would need to store data on a per-site basis at all - aggregating everything would be preferred, but to do that we need to determine what we might need to know in the future so we can record it now. For that, we need to think of (a) meaningful stats and (b) managing a distributed network. The first use case I'd throw in there is knowing how many unique sites in a given region are updating from each node. If more than some threshold of sites from idk, Nepal start updating from servers in California and Germany, we want to work out why there isn't a node they can use that has greater network efficiency for them.

This all leads me to 3 observations:

1. Withholding telemetry is not a realistic solution
2. The anonymized data has an important legitimate function
3. We may have a use case for individualizing some of the data, but not in a user-identifiable way (hashed domain)

I think we can resolve every use case we could come up with for having user-identifiable data, but not certain about some individualized data. If we can get what we need with aggregate data only, we should do so. In my example here, if we don't need a count of how many sites are connecting from Nepal, maybe analytics at each node is enough to know how much traffic is being served to certain regions. But what other use cases might there be? We should sort that out before we start turning everything off.

[costdev]

It's not so much about removing all telemetry, but there are absolutely things that aren't needed to determine an appropriate update package, or to check PHP/WP/DB compatibility or plugin dependency fulfilment.

Here's just a few examples of things we can likely strip out:

• blogs => $num_blogs
• users => get_user_count()
• platform_flags => [ 'os', 'bits' ]
• image_support => [ 'gd' => [], 'imagick' => [] ]
• A list of plugins or themes that include an UpdateURI header - which signifies that they are not on the API.
  • For excluding API results that include plugins/themes which have the same slug as one of the non-API ones, we can handle this on the site itself, not on the API server.

There will be more, but that might demonstrate where, like others have been doing recently, we can afford to offer users a choice that they haven't had in the past.

We should evaluate:

1. What's required for the user's site.
2. What may be important for AspirePress to make good tech decisions later.
   • e.g. PHP version usage so we can bump when we feel it's safe.
3. What's nice for marketing, but should absolutely be behind a consent flag.
4. Any other categories.

[toderash]

• num_blogs for a multisite install and user count are maybe ego stats, but do they have a bearing on core dev? Probably not enough of one to warrant collecting anymore, but also has no personal privacy concern associated
• platform flag is relevant to whether or not to bother supporting IIS
• image support is relevant for knowing what libraries are available to process uploaded images, not for an individual site, but as a percentage support thing... what's the lowest common denominator out there, and at what point do newer/better options become widely available for image processing?

I think the approach stands that if you disconnect the data from being identified with a specific site, the issue is resolved without even dropping anything. Not that there aren't things we can drop, just that this is the closest thing to a silver bullet solution that doesn't give away the farm for the value of the aggregate data. It's also way easier to implement than dealing with individual telemetry settings.

[costdev]

num_blogs for a multisite install and user count are maybe ego stats, but do they have a bearing on core dev? Probably not enough of one to warrant collecting anymore, but also has no personal privacy concern associated

No real effect on Core dev. There's a separate "multisite_enabled" for tallying the number of single sites/multisite networks. While in theory it could be used to determine whether to support multisite in the future, that's not going to happen. More for graphs than anything.

I agreed that there's no privacy concern there. However, it may still be something we shouldn't slurp just because we can.

platform flag is relevant to whether or not to bother supporting IIS

That doesn't give enough information to isolate IIS, so I wouldn't use the platform flag alone to determine that. I'd collect the server information (visible in Site Health) instead if I wanted to evaluate my Filesystem API maintenance time allocation in the future. I would put this in the category of evaluating the impact of a bug or lack of existing support rather than whether to continue existing support. Of course, AspirePress isn't WordPress Core, but I do think the original reason for it being collected is relevant to whether we value in continuing to collect it.

image support is relevant for knowing what libraries are available to process uploaded images, not for an individual site, but as a percentage support thing... what's the lowest common denominator out there, and at what point do newer/better options become widely available for image processing?

While this can still be behind a flag of sending anonymized data, I agree with what you say about its relevance and usage.

---

We agree that there's data that's useful for making decisions. However, I don't agree that the issue is necessarily resolved by anonymizing the data. There's a ton of software that has a choice on whether to share anonymous usage data. As you say, it's not that there aren't things we can drop, but I do think we should be clear in how we classify information. That way, we can have the trust of users while also collecting whatever data we feel is necessary or useful.