Data filtering

[DLumi]

Are there any insights as to how HLTB filters the data?
Like, there are obviously false entries, like, Counter Strike 2 having 'main_story' hours, or some purely story-driven game having multiplayer / coop hours listed. On their website they employ some clever tricks to only show the relevant data.
I wonder if anyone figured out how they do it?

[ScrappyCocco]

If you look in the json for counter strike 2 there's a flag called "game_type "multi"", maybe they filter based on that?
Can you check on the single player games you found with multiplayer/coop if they have a similar tag for "single"?
I'd suppose they filter something based on that; or however they filter should be in the response json

[ScrappyCocco]

Even if it's filtered automatically, the fields in the Entry class would be None/0, which must be checked by the user code; at that point I find equally useful having a "iscomp_lvl_combine = true" that the user can check to decide what to show, don't you agree?

[DLumi]

Fair, but if you are gathering a ton of data, and then shove it into pandas / sql or whatever, then handling None / null (not 0, tho) columns is a bit more straightforward, imo. And checking if the actual value is None seems to me a bit more intuitive then looking at some different flag for the same info, but that may be just a preference thing.
So I could see an argument for both, really.

[ScrappyCocco]

Ok so given an entry as

{
            "game_id": 93426,
            "game_name": "Battlefield 2042",
            "game_name_date": 0,
            "game_alias": "",
            "game_type": "multi",
            "game_image": "93426_Battlefield_2042.jpg",
            "comp_lvl_combine": 1,
            "comp_lvl_sp": 1,
            "comp_lvl_co": 1,
            "comp_lvl_mp": 1,
            "comp_main": 170122,
            "comp_plus": 371776,
            "comp_100": 603525,
            "comp_all": 268990,
            "comp_main_count": 18,
            "comp_plus_count": 8,
            "comp_100_count": 6,
            "comp_all_count": 32,
            "invested_co": 2040,
            "invested_mp": 263405,
            "invested_co_count": 1,
            "invested_mp_count": 43,
            "count_comp": 281,
            "count_speedrun": 0,
            "count_backlog": 879,
            "count_review": 199,
            "review_score": 56,
            "count_playing": 97,
            "count_retired": 306,
            "profile_platform": "PC, PlayStation 4, PlayStation 5, Xbox One, Xbox Series X/S",
            "profile_popular": 183,
            "release_world": 2021
        },

Based on

"comp_lvl_combine": 1,
"comp_lvl_sp": 1,
"comp_lvl_co": 1,
"comp_lvl_mp": 1,

Which values would you put to None automatically? Just to understand how you'd like it

[DLumi]

I wouldn't None-ify anything in this particular case. And it's justified, since battlefield 2042 is both single and multiplayer, so both stats are meaningful.

Take Elden Ring - a singleplayer-oriented game which has coop/pvp capabilities. The flags are:

'comp_lvl_combine': 0, 
'comp_lvl_sp': 1, 
'comp_lvl_co': 1, 
'comp_lvl_mp': 1,

Same thing - keep everything. The comp_lvl_combine seems to denote the way the hours are displayed on the website, so I guess, we can ignore this one entirely, unless we want to match website's response 1-1. I'm talking about this big flashy part, since I don't think anybody actually scrolls past it to see the full table down below.

Another example - Witcher 3, purely singleplayer game that has coop hours listed. Bug flags are:

'comp_lvl_combine': 0, 
'comp_lvl_sp': 1, 
'comp_lvl_co': 0, 
'comp_lvl_mp': 0

So we leave the single player hours, and None-ify coop and multiplayer.

That's how I see it, that is

[ScrappyCocco]

Have a look at #47 and tell me if you like or what else would you like to change

[ScrappyCocco]

Feature added in version 1.0.18