A plugin system

[cygri]

This might be premature, but here are some ideas.

The goal of a plugin system would be to do vocabulary-specific stuff in the generated HTML pages. For example, if a page has geo:lat and geo:long properties, a map could be shown. Or if a page is a void:Dataset with a void:sparqlEndpoint property, then a query form could be shown.

A plugin could consist of:

• An ASK SPARQL query to be run against the fetched resource description. If the query matches, then the plugin will be active on the page.
• One or more additional SPARQL queries that the plugin asks to run. The plugin can choose whether they are run against the fetched resource description (fast) or against the original data source (i.e. endpoint). The queries can include the ?__this__ variable to refer to the current resource. Results are included as JSON in a <script> block in the generated page.
• One or more Javascript files to be included via <script> tags. Plugin scripts to be added at the end of the <script> sections of the core Pubby page; order of the resulting <script> tags needs to be maintained.
• Zero or more CSS files that are included via <link rel="style"> tags.
• A specific Javascript function that is called on page load, and that will modify the page's DOM structure, perhaps using the provided JSON objects.

This would mean no Java code in the plugin, making them much easier to deploy and share. The whole thing could be bundled as a .zip or .jar, with a Turtle file as manifest that describes the whole thing, and dropped somewhere in WEB-INF. At startup, Pubby could check that directory for plugins.

[psiotwo]

Plugin API would definitely be a step forward. Also the split

• core Pubby - Java
• (visualization) plugins - JS/HTML
  is reasonable.

Here are some remarks regarding the Plugin API:

• Java - the data provided by the initial Pubby query (just first-level property-fillers) might not be sufficient for determining whether the plugin activates (in case the model of the visualization is more complex). So the plugins might have been given an option to augment original pubby queries to fetch other data as well. This would also improve performance, in case lots of data are required by the plugin.

This is also "compliant" with the last point above (A specific JavaScript function ... on page load), as the whole rendering stuff is given to the plugin at the same time as to the core pubby web page.

Nevertheless, speaking about the GEO extension you reference, this functionality is NOT needed. However, these remarks might be applicable to other plugins. Are there any such plugins with these demands ?

• JavaScript - plugin scripts to be added at the end of the <script> sections of the core Pubby page; order of the resulting <script> sections to be maintained.

[cygri]

@psiotwo: Regarding your point about deciding whether a plugin activates: I think this decision needs to be made without asking additional queries to the endpoint. Otherwise, if we install a bunch of plugins, each one will increase the number of queries that are run on every single page, even if the page is about something that is entirely unrelated to the plugin's function. So the activation decision has to be made on something simple that we already know, like the presence of a certain class or property in the resource description.

Based on this, a plugin may decide to be activated, so its additional SPARQL queries will be run, and then the plugin JS code may find out that these queries didn't match any interesting data, and therefore the plugin JS code will decide not to render anything on the page. So even if a plugin is activated, it may decide not to generate visible output.

I've added a comment to the notes above on the placement and ordering of script tags.

[cygri]

On the question of deploying plugins and bundling their resources, in version 1 of the plugin system, a plugin could simply be a directory that is copied to a location like /static/plugins/{my-plugin-name}, plus a block of Turtle that is copied into config.ttl in a similar way how conf:dataset blocks are currently handled. The Turtle block would contain all the information above (queries, stylesheets, custom JS and so on). This would keep things simple for this first version.

[pietercolpaert]

I have started work on a rdf2html javascript library. It takes rdf+turtle from a script tag and visualizes the data in the given HTML tags as the plug-ins specified. Could my work on this be relevant for pubby?

https://github.com/tdt/rdf2html

[cygri]

@pietercolpaert: It could certainly be relevant. I find it difficult to work out what exactly rdf2html does from the README.md. This can run in the browser, right? Do you have a working example somewhere online?

[pietercolpaert]

It's a javascript library that runs in the browser which injects HTML in particular <div>s by reading triples within a <script> tag.

Very early demo:
http://pieter.demo.thedatatank.com/rdf2html/example/

The idea behind writing it this way is that everyone can write their own plugin which generates HTML from an array of triples - https://github.com/tdt/rdf2html/tree/master/plugins

[cygri]

So, the plugin decides which triples (if any) it wants to render, and how to render them; and the HTML skeleton basically just marks the spot where the generated HTML should be injected? So, the rdf2html core itself only really parses the contents of the right script tag, and invokes all the plugins?

[pietercolpaert]

Correct!

[cygri]

I see, @pietercolpaert. So yeah, rdf2html plugins could potentially serve as Pubby's plugin system.

[psiotwo]

@cygri, are there any news regarding the plugin system ? I tried to find its "version 1" mentioned above in the sources, but without success.

[pietercolpaert]

We have been working on RDF2HTML which is now in use in The DataTank.

Cfr. http://demo.thedatatank.com/wikipedia/nmbs/stations?&offset=2000&limit=500

The plug-ins should now be extended: https://github.com/tdt/rdf2html/tree/master/plugins

@cygri are there plans to release a new version of pubby soon with this functionality? Is someone working on that or should it be set up?

[cygri]

@psiotwo, the “version 1” mentioned was hypothetical—if someone were to develop a version 1 of the plugin system, it could be done in such-and-such a way.

[psiotwo]

Ah, I see :-) Thx for explanation. Petr

[cygri]

@pietercolpaert No one is actively working on a plug-in system, although @psiotwo seems interested.

I expect to be able to spend some cycles on Pubby in June and July, so there might be a new release in that timeframe. I don't expect to get around to working on this particular issue.