Bringing back RDFa support to RDFLib

[fathoni]

RDFa support was dropped from RDFLib since RDFLib version 5.0. There have been changes to the RDFLib 5.0 which breaks its compatibility with pyRDFa3, and unfortunately, there is currently no plan on maintaining its compatibility according to the discussion on pyRDFa3's GitHub page. I think It would be great to bring RDFa compatibility back to the current version. Thank you!

[mwatts15]

Can we describe what the incompatibilities are with RDFLib 5+? The only problem I see in the linked issue is that file is no-longer a built-in or a type in Python 3, so there's a reference error (also, pyRrdfa3 is severely lacking in error reporting...though that's a separate issue).

[mwatts15]

I see. Just going from the master branch of pyRDFa3 or the version from PyPI, a number of name reference errors are encountered. It looks like very sloppy work setting up pyRDFa3 in that repo.

[ghost]

I see. Just going from the master branch of pyRDFa3 or the version from PyPI, a number of name reference errors are encountered. It looks like very sloppy work setting up pyRDFa3 in that repo.

Could you be specific about the process in which you are encountering name reference errors? All I'm doing (afaik) is installing pyrdfa3 from PyPi. I did copy over pymicroformats because atm there's no PyPi installation route but that can be elided by specifying the repos details in requirements.txt (not sure how to do the same in setup.py)

[mwatts15]

Yes. I created a virtualenv, cloned pyrdfa3, installed pyrdfa3 in the virtualenv and then ran your code snippet. It failed with

Traceback (most recent call last):
  File "/home/markw/work/pyrdfa3/env/lib/python3.9/site-packages/rdflib/plugin.py", line 124, in get
    p: Plugin[PluginT] = _plugins[(name, kind)]
KeyError: ('text/html', <class 'rdflib.parser.Parser'>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/markw/work/pyrdfa3/temp/discussion-1582.py", line 8, in <module>
    g = Graph().parse(location="https://alistapart.com/article/introduction-to-rdfa/", format="text/html")
  File "/home/markw/work/pyrdfa3/env/lib/python3.9/site-packages/rdflib/graph.py", line 1255, in parse
    parser = plugin.get(format, Parser)()
  File "/home/markw/work/pyrdfa3/env/lib/python3.9/site-packages/rdflib/plugin.py", line 126, in get
    raise PluginException("No plugin registered for (%s, %s)" % (name, kind))
rdflib.plugin.PluginException: No plugin registered

I saw that the entry points were both malformed and missing text/html (my comments added):

        'rdf.plugins.parser': [
            'rdfa = pyRdfa.rdflibparsers:RDFaParser',
            'rdfa1.0 = pyRdfa.rdflibparsers:RDFa10Parser',
            'rdfa1.1 = pyRdfa.rdflibparsers:RDFaParser',
            'application/svg+xml = pyRdfa.rdflibparsers:RDFaParser',
            'application/xhtml+xml = pyRdfa.rdflibparsers:RDFaParser',
            'hturtle = pyRdfa.rdflibparsers:HTurtleParser' # missing trailing comma
            'html = pyRdfa.rdflibparsers:StructuredDataParser'
            'html/text = pyRdfa.rdflibparsers:StructuredDataParser' # correct mime type is text/html
        ],

I wasn't sure what the deal is with html/text, so I just copied that line, changed the name to text/html, and added the missing commas. I reinstalled and ran the code snippet again, then got an error: NameError: name 'warnings' is not defined. I addressed that by adding an import of warnings not behind a conditional. I added a dependency on "requests" to setup.py and reinstalled. I got a NameError on ExecutionContext in rdflibparsers, so added an import. Then, I stopped because I figured someone else may already have done this who actually uses pyrdfa3 but who hasn't gotten it merged to master yet :)

[ghost]

Okay, thanks for the detail, most useful, I'd forgotten about the requirement for requests.

Actually, splitting it out as a separate plugin is fairly trivial, given that RDFLib has a plugin interface. The essential additions are few (adding the plugin registration to plugin.py plus adding the parsers to plugins/parsers) and are detailed in this commit.

In a split-off plugin, none of those changes to RDFLib would persist, they'd all be hived off to the separate plugin code and the registration would be handled by setup.py in exactly the same way that the registration of the parser/serializer for json-ld is handled.

So basically, the plugin code would consist of the two parsers and a setup.py which installs pyrdfa3 from PyPi and pymicrodata from git (happily, there's an example in rdflib-leveldb) and that'd be about it. Oh, and some tests, ofc but they'd be contained in the plugin repos.

I'll re-create it as an RDFLib plugin, commit it to a repos in my personal github space and then post a notification that it's available for tyre-kicking.

[ghost]

I'll re-create it as an RDFLib plugin, commit it to a repos in my personal github space and then post a notification that it's available for tyre-kicking.

Done. Repos is here. The deprecation warnings from pyrdfa3 can safely be ignored.

[ghost]

And, on reflection, the branch could be fairly straightforwardly reworked as a plugin, using the same approach as the plugin stores.

Despite the branch name, I wasn't thinking of proposing it be returned to RDFLib core, I was just checking the feasibility of getting it working at all.

Documentation needs work and it could use some better/more extensive tests.

[ghost]

Despite the branch name, I wasn't thinking of proposing it be returned to RDFLib core, I was just checking the feasibility of getting it working at all.

fwiw, more reflections on strategy/tactics in the draft PR #1503