Cannot process second level country domains (e.g. ".ac.uk")

[hedimdev]

Mastefeeder fails to process feed URLs with a second level country domain, e.g. ".ac.uk", ".co.il" etc.

Two examples to test:
https://cam.ac.uk/news/feed
https://theliberal.co.il/rss

Thanks.

[jehna]

Oh yes, there seems to be a bug! I'll see when I have some time to fix it

[jehna]

The bug is here:

mastofeeder/src/parse-domain.ts

Lines 8 to 9 in 5b35460

	const tld = parts[i];
	if (TLDS.includes(tld)) {

So the matcher now checks just one part at the time. Should probably do a bigger sweep 🤔

[rchurchley]

I think this is a design issue with the replace-slashes-with-dots mechanism.

1. The URLs https://example.co.uk/rss.xml and https://example.co/uk/rss.xml are equally valid feed URLs, but only one of them can be extracted from the username @example.co.uk.

2. The URLs https://example.co.blog/rss.xml and https://example.co/blog/rss.xml are theoretically valid feed URLs, but only one can be extracted from the username @example.co.blog.

In case 1, the first URL is somewhat more likely to be the intended feed, but in case 2 the second URL is almost certainly the right one. But the logic to distinguish between these two cases would be pretty complicated.

This would be a breaking change, but have you considered using a different replacement for slashes? Assuming it's important to not use characters that Mastodon clients may not be familiar with, you could even use a sequence like ._.. Since that's unlikely to occur naturally in feed URLs, this would let you easily and unambiguously translate between usernames and feeds:

• @example.co.uk._.blog refers to https://example.co.uk/blog
• @feeds.yle.fi._.uutiset._.v1._.majorHeadlines._.YLE_UUTISET refers to https://feeds.yle.fi/uutiset/v1/majorHeadlines/YLE_UUTISET.rss

(If you're open to such a solution I'm willing to put together a PR once I have a chance to set up an environment and test!)

[seefood]

Is this issue getting any love? It's a huge show stopper in getting private blogs connected to the fediverse. I'd patch this if I had time to learn TS and refactor this issue....

[jehna]

Hey! Sorry but unfortunately I've had zero time to work on Mastofeeder lately. Will fix this when I have more time to work on open source again!

[seefood]

ok, time for me to learn TS. There's a war going on here in Israel, and I need some feeds on the fediverse to open up conversations. If I manage to merge this I'll make a PR, otherwise I'll just use this branch I guess.