Username Search No Longer Works

[AptAmoeba]

Looks like the Username search does not pull up players anymore. Last I tested this was a few weeks ago and it worked, but recently it doesn't seem to search anymore. Tried checking the commit history but I couldn't immediately find anything that might break the search process.
Maybe GG changed something with their API?

 

Troubleshooting:

• Cleared cache/site data
• Checked on Desktop & Mobile devices

I have tested this on the following Browsers:

• Google Chrome (Version 132.0.6834.196 (Official Build) (64-bit))
• Google Chrome (Mobile version)
• Opera (Current non-beta Build)

 

Thanks for building this by the way. Super cool project. You should put a donate link on the repo :)

[jojo259]

I've experienced this a few times and when I checked the network requests, the leaderboard CSV only had ~100 entries (so it lacked data for the specific player I was viewing).

It seems like the script that updates the CSV fails sometimes e.g. https://github.com/Matt-OP/geoleaderboard/blob/4a84ba09d0074d4911bce997aa06a85ca6edbc86/leaderboard.csv

But other times it will work fine.

[AptAmoeba]

Ah, interesting. So we essentially have to wait for a day where the leaderboard pull script successfully pulls- That's an odd issue, I wonder what's causing it. Maybe the rate limiting elsewhere that was mentioned in a previous commit thread could be happening?

As an update, I tried it today and it did indeed successfully pull my stats!

[jojo259]

https://github.com/Matt-OP/Geoguessr-user-dataset/blob/main/Kaggle-Notebook-Geoguessr-User-Dataset-API-Pull.ipynb

I assume this is the code used to grab the data?

[Matt-OP]

@AptAmoeba and @jojo259, thank you both for your concern. The issue is unrelated to the site's search functionality and instead stems from requesting data from the Geoguessr API.

The code snippet below, which uses ThreadPoolExecutor to request data, has worked reliably for months. However, in recent weeks, the majority of player IDs are being “missed”:

def scrape_player_data(player_ids):
    results = []
    missed_ids = []
    
    with ThreadPoolExecutor(max_workers=4) as executor:
        futures = {executor.submit(fetch_player_data, player_id): player_id for player_id in player_ids}
        for future in tqdm(as_completed(futures), total=len(player_ids)):
            df_row, missed_id = future.result()
            if df_row is not None:
                results.append(df_row)
            if missed_id is not None:
                missed_ids.append(missed_id)
    
    df = pd.concat(results, axis=0, ignore_index=True).rename(columns = rename_dict)
    return df, missed_ids

Suspecting rate limiting, I revised the code to remove ThreadPoolExecutor and introduced a time.sleep(0.3) between requests:

def scrape_player_data(player_ids):
    results = []
    missed_ids = []

    for player_id in tqdm(player_ids):
        time.sleep(0.3)
        df_row, missed_id = fetch_player_data(player_id)
        if df_row is not None:
            results.append(df_row)
        if missed_id is not None:
            missed_ids.append(missed_id)

    df = pd.concat(results, axis=0, ignore_index=True).rename(columns = rename_dict)
    return df, missed_ids

Using this revised function has seen some improvement in day to day runs but occasionally I still see it runs into issues. I recently added a failsafe to prevent a leaderboard update if too many IDs are missed:

if n_missed_ids < 7500:
    # code to push to github repo

I’m still monitoring the situation and evaluating ways to improve consistency.