Add + Discover Sites

[VlaK0r]

Hello!
Thanks for your work.
Could you please tell me whether the Add + Discover Sites tool is still in development or is it ready to use?

[samuelclay]

Yep, if it's in main and doesn't have a staff-only badge, it's good to go. Story clustering and daily briefing are still being worked on regularly, so they're not quite ready, but I hope you've enjoyed the big daily briefing changes i made last week and this week.

[samuelclay]

I'll mention that I am working on a change to trending feeds, which is in the search tab of the Add and Discover page. It is to basically have a little more consideration of the origin of new feeds so that we don't just show feeds that are imported. We have a bunch of stale feeds that are there because they're popular, and I get it, but they're stale and they're just being imported through OPML and not being manually subscribed to. I am changing that algorithm as we speak.

[VlaK0r]

@samuelclay I’m asking because, besides Reddit and YouTube, not a single tab works properly for me.

[samuelclay]

Ah-ha, you'll want to run docker exec -it newsblur_web ./manage.py bootstrap_popular_feeds --skip-fetch. I just added it to the README in 8c95073.

[VlaK0r]

@samuelclay I didn’t quite understand what that means, but I’ll do it ))

[VlaK0r]

@samuelclay

[samuelclay]

what do you see on make log

[VlaK0r]

@samuelclay I solved the problem with Google News.

And here are the logs for ArtStation.

newsblur_web               | [Mar 06 21:26:21]  ---> POST /webfeed/analyze {'url': ['https://www.artstation.com'], 'request_id': ['wf_1772832381458_4g6vpd42']}
newsblur_web               | [Mar 06 21:26:21]  ---> [------] [VlaKor^] Web Feed: Analyzing https://www.artstation.com
newsblur_celery            | Received task: analyze-webfeed-page[21e1a08d-994f-4f4b-884d-e711907be7a8]  
newsblur_web               | [Mar 06 21:26:21]  ---> [FF    ] [0.01s] [VlaKor^] DB times /webfeed/analyze: sql: 2/0.002s mongo: 0/0.000s redis: user:4/0.002s story:0/0.000s session:0/0.000s pubsub:0/0.000s
newsblur_node              | [2026-03-06 21:26:22]  ---> webfeed:start https://www.artstation.com
newsblur_node              | [2026-03-06 21:26:22]  ---> webfeed:progress https://www.artstation.com: Fetching page...
newsblur_celery            | [Mar 06 21:26:22]  ---> [------] [VlaKor^] Web Feed: Fetching page https://www.artstation.com
newsblur_node              | [2026-03-06 21:26:23]  ---> [webfeed:error https://www.artstation.com: Could not fetch the page. The site may be blocking requests.] undefined
newsblur_celery            | [Mar 06 21:26:23]  ---> [------] [VlaKor^] Web Feed: Fetch failed for https://www.artstation.com

[VlaK0r]

newsblur_web               | [Mar 06 21:31:37]  ---> POST /webfeed/analyze {'url': ['https://store.steampowered.com/'], 'request_id': ['wf_1772832697177_liefxsbq']}
newsblur_web               | [Mar 06 21:31:37]  ---> [------] [VlaKor^] Web Feed: Analyzing https://store.steampowered.com/
newsblur_celery            | Received task: analyze-webfeed-page[f7d7d34e-888b-4450-ba69-f30c9b134e6a]  
newsblur_web               | [Mar 06 21:31:37]  ---> [FF    ] [0.01s] [VlaKor^] DB times /webfeed/analyze: sql: 2/0.002s mongo: 0/0.000s redis: user:4/0.002s story:0/0.000s session:0/0.000s pubsub:0/0.000s
newsblur_node              | [2026-03-06 21:31:37]  ---> webfeed:start https://store.steampowered.com/
newsblur_node              | [2026-03-06 21:31:37]  ---> webfeed:progress https://store.steampowered.com/: Fetching page...
newsblur_celery            | [Mar 06 21:31:37]  ---> [------] [VlaKor^] Web Feed: Fetching page https://store.steampowered.com/
newsblur_celery            | [Mar 06 21:31:38]  ---> [------] [VlaKor^] Web Feed: Fetched 790579 bytes, analyzing with Claude
newsblur_node              | [2026-03-06 21:31:38]  ---> webfeed:progress https://store.steampowered.com/: Preparing page...
newsblur_celery            | [Mar 06 21:31:38]  ---> [------] [VlaKor^] Web Feed: Cleaned HTML from 790579 to 2708 bytes
newsblur_node              | [2026-03-06 21:31:38]  ---> webfeed:progress https://store.steampowered.com/: Finding story patterns...
newsblur_node              | [2026-03-06 21:31:38]  ---> [webfeed:error https://store.steampowered.com/: Unexpected error: 'haiku'] undefined
newsblur_celery            | [Mar 06 21:31:38]  ---> [------] [VlaKor^] Web Feed: Unexpected error - 'haiku'