Skip to content

Conversation

BUZZ1592003
Copy link

Description

Fixes false positive detection issues for two sites that were incorrectly reporting fake usernames as existing users.

Changes Made

Coders Rank

  • Changed detection method from message to status_code with errorCode: 404
  • Previous method looked for "not a registered member" text which was unreliable
  • Site returns HTTP 200 with profile pages for non-existent users

LessWrong

  • Added exclusion due to impossible detection
  • Site returns empty HTTP 200 responses for both real and fake users
  • Cannot reliably distinguish between existing and non-existing users

Testing Evidence

Tested with fake username testuser123:

  • Both sites returned false positives before fix
  • Coders Rank shows profile pages for fake users
  • LessWrong returns empty responses making detection impossible

Contributes to Issue

Addresses #2547 - False Positive Remediation

…e:- Coders Rank: Changed from message to status_code detection - Removed unreliable message detection 'not a registered member' - Added errorCode 404 for better false positive prevention - LessWrong: Added exclusion for false positive prevention - Site returns empty HTTP 200 responses for all users - Cannot distinguish between real and fake users - Excluded to prevent false positive resultsBoth changes contribute to resolving issue sherlock-project#2547 false positive remediation.Tested with fake usernames to verify false positive behavior.
Copy link
Contributor

github-actions bot commented Oct 4, 2025

Automatic validation of changes

Target F+ Check F- Check
LessWrong ✔️   Pass ❌   Fail
Coders Rank ❌   Fail ❌   Fail

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

@BUZZ1592003
Copy link
Author

Coders Rank returns HTTP 200 for fake users making detection extremely difficult

@akh7177
Copy link
Contributor

akh7177 commented Oct 4, 2025

Hello, @BUZZ1592003

Regarding Less Wrong site, Sherlock seems to be blocked by the Vercel security layer. (You can verify that using the --dump-response flag). So to get past that, I believe we have to bring about quite a few changes in Sherlock's working.

I tried with a couple of headers, but Vercel Security Layer seems to block all of them

@BUZZ1592003
Copy link
Author

BUZZ1592003 commented Oct 4, 2025

Hello, @BUZZ1592003

Regarding Less Wrong site, Sherlock seems to be blocked by the Vercel security layer. (You can verify that using the --dump-response flag). So to get past that, I believe we have to bring about quite a few changes in Sherlock's working.

I tried with a couple of headers, but Vercel Security Layer seems to block all of them

Thanks @akhi7177

That explains the empty responses perfectly. Should I keep this open as documentation or would you prefer I close it until the Vercel security layer issues are resolved.......?

@akh7177
Copy link
Contributor

akh7177 commented Oct 4, 2025

Hello, @BUZZ1592003
Regarding Less Wrong site, Sherlock seems to be blocked by the Vercel security layer. (You can verify that using the --dump-response flag). So to get past that, I believe we have to bring about quite a few changes in Sherlock's working.
I tried with a couple of headers, but Vercel Security Layer seems to block all of them

Thanks @akhi7177

That explains the empty responses perfectly. Should I keep this open as documentation or would you prefer I close it until the Vercel security layer issues are resolved.......?

Let's keep this PR open for the time being and have @ppfeister throw some light on this.
There are a couple of more sites Weblate, YandexMusic etc that are also getting blocked in a similar manner by anti-bot checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants