Ensure that mybinder.org isn't being significantly impacted by bot and crawlers

[choldgraf]

I'm concerned that we have a significant amount of bot traffic creating sessions on mybinder.org. We should confirm that this isn't the case, or fix this issue, if others are worried about this too.

Context

The 2i2c team (thanks @jmunroe!) recently discovered that we may have caught a significant amount of bot and scraper activity leading to launches on a public BinderHub that we run. Binder sessions were being spun up, followed by a period of no activity, before they were shut down. We think it's something like an LLM scraper or a bot that is hitting URLs and causing sessions to spin up.

Why I think we might have a lot of bots spawning Binder sessions

We use plausible for web analytics, and I am pretty sure plausible filters out any known bot activity¹. So we can compare the logs of plausible against the logs of mybinder.org's analytics archive to get an idea of "launches that plausible filtered out".

For example for December 9th:

• Visits to website logged by Plausible: 3,100
• Mybinder launch events: 5,184

December 5th:

• Visits to mybinder.org: 2,500
• Mybinder launch events: 4,398

In both cases it seems like something around 40% of mybinder.org launch events are not being logged by plausible.

Some % of them might be "back-end" launches where nobody touches the browser, but it's plausible that a high percentage of those are bot accounts that are triggering Binder events, but not being logged in Plausible.

Footnotes

1. Plausible is a privacy-first service so we don't have access to the actual user agent names, HTTP requests, etc. That's why we have to kinda infer an outcome here. ↩

[minrk]

Definitely worth investigating, but since any thebe-style launches would not be tracked by plausible, I'm not sure there's a problem. Often a pretty big chunk of our traffic is from those, such as the matplotlib gallery or spacy, or anyone who tries to do a 'pre-build' of images from CI, so 40% is not outlandish. We can try a check with the current launch analytics to see which of the top repos are likely to be associated with 'back-end' launches.

[choldgraf]

maybe we could run a filter for BinderHub pings that checks for things like GPTBot etc? Good point about the thebe launches...

[minrk]

I think we could send plausible stats from the backend for the build/launch requests: https://plausible.io/docs/events-api#endpoints maybe that would help?

[yuvipanda]

Just as a note, the initial investigation that you're referring to @choldgraf isn't a binderhub but a jupyterhub with tmpauthenticator with fundamentally different interactions between bots, HTTP traffic and server spinups. So while we could be seeing bot issues on mybinder, I don't think much from that analysis (which I also haven't dove into much) translates here.

[choldgraf]

It sounds like you both don't think this is likely a big problem, so given this discussion why don't we just close this as wontfix?

[rgaiacs]

Back in 2023, I was looking the analytics of the "old" GESIS server and I noticed that https://github.com/hanlpbot/hanlp-binder was always at the "Top 5 popular repositories (by number of launches in last hour)". This is still the case.

When I investigated further, I noticed that, probably, more than 90% of the launches of https://github.com/hanlpbot/hanlp-binder were coming from the "HanLP: Han Language Processing" website that, at that time, included a mybinder.org embeded launch powered by Thebe.

I think that mybinder.org is being impacted by bot and crawlers but I don't see a way to reduce the impact without add Cloudflare (or similar) to the mybinder.org stack.

[manics]

If we served mybinder.org using Cloudflare we could potentially use it as a load-balancer, which solves the problem of how to run the federation redirector on all nodes:
#3437 (comment)