HTML infiltrating in imported content

[sfatfarma]

Hello,

First of all, congrats for the code!

On some content, HTML code is infiltrating in extracted textual content. Example for this article:

https://www.yahoo.com/entertainment/chris-evans-ignites-celeb-civil-085545038.html

Snippet of the extracted content:

Chris Evans causing people to choose sides in a way that hasn’t been seen since “Captain America: Civil War.”” data-reactid=”16″ type=”text”>There’s an issue that’s divided Twitter almost as much as anything in politics this week, with Chris Evans causing people to choose sides in a way that hasn’t been seen since “Captain America: Civil War.”

As you can see, this is infiltrating in content:

”” data-reactid=”16″ type=”text”>

This is mu current code:

$readConf = new Configuration(); $readConf->setSummonCthulhu(true); $readability = new Readability($readConf); $readability->parse($html_string); $return_me = $readability->getContent();

Any help is appreciated.

Regards,
Szabi.

[andreskrey]

I was able to reproduce this in the latest development version. I'll take a look at it next weeeknd :)

[sfatfarma]

Great, thanks a bunch!

[andreskrey]

Seems that it's a problem with the way you parse elegant quotes (these ones: ”)

If you check that snippet from what Readability returns, you'll see it's valid HTML:

<p content="There’s an issue that’s divided Twitter almost as much as anything in politics this week, with &lt;a href=&quot;v&quot; target=&quot;_blank&quot;>Chris Evans&lt;/a>&amp;nbsp;causing people to choose sides in a way that hasn’t been seen since “&lt;a href=&quot;https://www.huffingtonpost.com/topic/captain-america&quot; rel=&quot;nofollow noopener&quot; target=&quot;_blank&quot;>Captain America: Civil War&lt;/a>.”" data-reactid="16" type="text">
There’s an issue that’s divided Twitter almost as much as anything in politics this week, with <a href="v" target="_blank">Chris Evans</a> causing people to choose sides in a way that hasn’t been seen since “<a href="https://www.huffingtonpost.com/topic/captain-america" rel="nofollow noopener" target="_blank">Captain America: Civil War</a>.”
</p>

Check the end of the line, it ends properly. Although it has the character > inside the content property of the p tag and it's super ugly, it validates in the W3C validator.

May I ask you how are you seeing the "data-reactid" tag? Firefox parses the HTML correctly and doesn't show it. I'm guessing that maybe that fancy quote or the > sign is messing with whatever you're using after Readability.