add klassegegenklasse (DE) publisher + parser + tests + tables #809
Conversation
|
Added publisher Klasse gegen Klasse(DE)
|
| Sitemap( | ||
| "https://www.klassegegenklasse.org/wp-sitemap.xml", | ||
| ), |
There was a problem hiding this comment.
It looks like the sitemap includes a lot of unnecessary entries. You can use the sitemap_filter parameter of Sitemap to exclude unwanted sitemap URLs. For example, sitemap_filter=inverse(regex_filter("wp-sitemap-posts-post")) will exclude any sitemap that does not contain the substring wp-sitemap-posts-post.
| nodes = self.precomputed.doc.xpath( | ||
| "//a[@rel='author']/text()" | ||
| " | //span[contains(@class,'author')]//a/text()" | ||
| " | //div[contains(@class,'author')]//a/text()" | ||
| " | //div[contains(@class,'byline')]//a/text()" | ||
| " | //span[contains(@class,'byline')]//a/text()" | ||
| " | //a[contains(@href,'/autor/') or contains(@href,'/author/')]/text()" | ||
| ) |
There was a problem hiding this comment.
Clicking through some of the articles it seems to me that the author can usually be found at the same place. Could you elaborate on your reasoning for adding all the extra selectors?
| @attribute | ||
| def publishing_date(self) -> Optional[datetime]: |
There was a problem hiding this comment.
Same as with authors. It seems that the publishing date of an article is displayed at the same location for the articles i encountered so far. Could you elaborate why you added the extra fall-backs?
2 participants
No description provided.