Update of the `main` branch to work with QLever #2715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

hannahbast wants to merge 206 commits into WDscholia:main from ad-freiburg:qlever

README-apache-config.md

-Original file line number
+Diff line change
@@ -0,0 +1,62 @@
+    # Apache virtual host configuration
+    Currently runs on <https://qlever.scholia.wiki> and
+    <https://wikidata-query-gui.scholia.wiki> with the following
+    Apache configurations.
+    ## qlever.scholia.wiki
+    ```apache
+    <VirtualHost *:443>
+      ServerName qlever.scholia.wiki
+      ServerAdmin webmaster@localhost
+      SSLEngine On
+      SSLCertificateFile /etc/letsencrypt/live/qlever.scholia.wiki/fullchain.pem
+      SSLCertificateKeyFile /etc/letsencrypt/live/qlever.scholia.wiki/privkey.pem
+      # SSLCertificateFile /etc/letsencrypt/live/qlever.cs.uni-freiburg.de/fullchain.pem
+      # SSLCertificateKeyFile /etc/letsencrypt/live/qlever.cs.uni-freiburg.de/privkey.pem
+      Include /etc/letsencrypt/options-ssl-apache.conf
+      ProxyPass / http://tajo.informatik.privat:8100/
+      <Directory />
+        Options Indexes
+        Options FollowSymLinks
+        AllowOverride None
+        Allow from all
+      </Directory>
+      ErrorLog ${APACHE_LOG_DIR}/scholia-error.log
+      CustomLog ${APACHE_LOG_DIR}/scholia-access.log combined
+    </VirtualHost>
+    ```
+    ## wikidata-query-gui.scholia.wiki
+    ```apache
+    <VirtualHost *:443>
+      ServerName wikidata-query-gui.scholia.wiki
+      ServerAdmin webmaster@localhost
+      SSLEngine On
+      SSLCertificateFile /etc/letsencrypt/live/wikidata-query-gui.scholia.wiki/fullchain.pem
+      SSLCertificateKeyFile /etc/letsencrypt/live/wikidata-query-gui.scholia.wiki/privkey.pem
+      Include /etc/letsencrypt/options-ssl-apache.conf
+      ProxyPass / http://tajo.informatik.privat:8080/
+      <Directory />
+        Options Indexes
+        Options FollowSymLinks
+        AllowOverride None
+        Allow from all
+      </Directory>
+      ErrorLog ${APACHE_LOG_DIR}/wikidata-query-gui-error.log
+      CustomLog ${APACHE_LOG_DIR}/wikidata-query-gui.log combined
+    </VirtualHost>
+    ```

runserver.py

-Original file line number
+Diff line change
@@ -1,13 +1,22 @@
     from scholia.app import create_app
     import sys
+    import logging
+    # Enable debug logging
+    logging.basicConfig(level=logging.DEBUG)
     app = create_app(
         text_to_topic_q_text_enabled=False,
         third_parties_enabled=True)
     app.config['APPLICATION_ROOT'] = '/'
     if __name__ == '__main__':
+        if len(sys.argv) == 3:
+            hostIP = port=sys.argv[2]
+        else :
+            hostIP = '0.0.0.0'
         if len(sys.argv) == 2:
-            app.run(debug=True, port=sys.argv[1])
+            app.run(debug=True, host=hostIP, port=sys.argv[1])
         else:
-            app.run(debug=True, port=8100)
+            app.run(debug=True, host=hostIP, port=8100)

scholia/app/__init__.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -5,6 +5,7 @@ @@
     from flask import Flask
     from flask_bootstrap import Bootstrap, StaticCDN
+    from werkzeug.middleware.proxy_fix import ProxyFix
     from ..text import TextToTopicQText
@@ Expand Down Expand Up @@
         """
         app = Flask(__name__)
+        # Configure for proxy deployment
+        app.wsgi_app = ProxyFix(
+            app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_prefix=1
+        )
         Bootstrap(app)
         # Serve assets from wmflabs for privacy reasons
@@ Expand Down @@

scholia/app/static/scholia.js

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -96,69 +96,74 @@ function convertDataTableData(data, columns) {
  
            }

        }

        for (var i = 0 ; i < data.length ; i++) {

    	var convertedRow = {};

    	for (var key in data[i]) {

    	    if (key.slice(-11) == 'Description') {

    		convertedRow[key.slice(0, key.length - 11) + ' description'] = data[i][key];

        var convertedRow = {};

        for (var key in data[i]) {

            if (key.slice(-11) == 'Description') {

            convertedRow[key.slice(0, key.length - 11) + ' description'] = data[i][key];

    	    } else if (

    			key + 'Label' in data[i] &&

            } else if (

                key + 'Label' in data[i] &&

                key + 'Url' in data[i]

    	    ) {

            ) {

                convertedRow[key] = '<a href="' +

    		    data[i][key + 'Url'] +

    		    '">' + data[i][key + 'Label'] + '</a>';

                data[i][key + 'Url'] +

                '">' + data[i][key + 'Label'] + '</a>';

    	    } else if (key.slice(-17) == 'ChemicalStructure') {

            } else if (key.slice(-17) == 'ChemicalStructure') {

                convertedRow[key.slice(0, key.length - 17) + ' structure'] = '<img loading="lazy" src="' +

                'https://cdkdepict.toolforge.org/depict/bow/svg?smi=' +

    		    encodeURIComponent(data[i][key]) +

                encodeURIComponent(data[i][key]) +

                '&abbr=on&hdisp=bridgehead&showtitle=false&zoom=2&annotate=cip' +

    		    '" />';

                '" />';

    	    } else if (

    			key + 'Label' in data[i] &&

    			/^http/.test(data[i][key]) &&

    			data[i][key].length > 30

    	    ) {

    		    convertedRow[key] = '<a href="../' +

    		    data[i][key].slice(31) +

    		    '">' + data[i][key + 'Label'] + '</a>';

            } else if (

                key + 'Label' in data[i] &&

                /^http/.test(data[i][key]) &&

                data[i][key].length > 30

            ) {

                convertedRow[key] = '<a href="../' +

                data[i][key].slice(31) +

                '">' + data[i][key + 'Label'] + '</a>';

            } else if (key.slice(-5) == 'Label') {

    		// pass

    	    } else if (key + 'Url' in data[i]) {

    		    convertedRow[key] = '<a href="' +

    		    data[i][key + 'Url'] +

    		    '">' + data[i][key] + '</a>';

    	    } else if (key.slice(-3) == 'Url') {

    		// pass

    	    } else if (key.slice(-3) == 'url') {

    		// Convert URL to a link

    		    convertedRow[key] = "<a href='" +

    		    data[i][key] + "'>" + 

    		    $("<div>").text(data[i][key]).html() + '</a>';

    	    } else if (key == 'orcid') {

    		// Add link to ORCID website

    		    convertedRow[key] = '<a href="https://orcid.org/' +

    		    data[i][key] + '">' + 

    		    data[i][key] + '</a>';

    	    } else if (key == 'doi') {

    		// Add link to Crossref

    		    convertedRow[key] = '<a href="https://doi.org/' +

    		    encodeURIComponent(data[i][key]) + '">' +

    		    $("<div>").text(data[i][key]).html() + '</a>';

    	    } else {

    		    convertedRow[key] = data[i][key];

    	    }

    	}

    	convertedData.push(convertedRow);

            // pass

            } else if (key + 'Url' in data[i]) {

                convertedRow[key] = '<a href="' +

                data[i][key + 'Url'] +

                '">' + data[i][key] + '</a>';

            } else if (key.slice(-3) == 'Url') {

            // pass

            } else if (key.slice(-3) == 'url') {

            // Convert URL to a link

                convertedRow[key] = "<a href='" +

                data[i][key] + "'>" +

                $("<div>").text(data[i][key]).html() + '</a>';

            } else if (key == 'orcid') {

            // Add link to ORCID website

                convertedRow[key] = '<a href="https://orcid.org/' +

                data[i][key] + '">' +

                data[i][key] + '</a>';

            } else if (key == 'doi') {

            // Add link to Crossref

                convertedRow[key] = '<a href="https://doi.org/' +

                encodeURIComponent(data[i][key]) + '">' +

                $("<div>").text(data[i][key]).html() + '</a>';

            } else {

                var convertedRowValue = data[i][key];

                if (convertedRowValue.startsWith("http://www.wikidata.org/entity/Q")) {

                    var qid = convertedRowValue.slice(31);

                    convertedRowValue = '<a href="../' + qid + '">' + qid + '</a>';

                }

                convertedRow[key] = convertedRowValue;

            }

        }

        convertedData.push(convertedRow);

        }

        return { data: convertedData, columns: convertedColumns };

    }

    @@ -260,7 +265,7 @@ function sparqlToDataTablePost2(url, editURL, sparql, element, filename, options
  
        var paging = (typeof options.paging === 'undefined') ? true : options.paging;

        var sDom = (typeof options.sDom === 'undefined') ? 'lfrtip' : options.sDom;

        var sparqlEndpointName = (typeof options.sparqlEndpointName === 'undefined')

    	? window.jsConfig.sparqlEndpointName : options.sparqlEndpointName;

            ? window.jsConfig.sparqlEndpointName : options.sparqlEndpointName;

        $(element).html("<div class='loader'><div></div><div></div><div></div></div>");

    @@ -277,7 +282,7 @@ function sparqlToDataTablePost2(url, editURL, sparql, element, filename, options
  
                };

                columns.push(column);

            }

            if (convertedData.data.length <= 10) {

              paging = false;

            }

    @@ -299,7 +304,7 @@ function sparqlToDataTablePost2(url, editURL, sparql, element, filename, options
  
                '<caption><span style="float:left; font-size:smaller;"><a href="' + editURL +

                    encodeURIComponent(sparql) +

                    '">' + sparqlEndpointName + '</a></span>' +

                    '<span style="float:right; font-size:smaller;"><a href="https://github.com/WDscholia/scholia/blob/main/scholia/app/templates/' +

                    '<span style="float:right; font-size:smaller;"><a href="https://github.com/ad-freiburg/scholia/blob/qlever/scholia/app/templates/' +

                    filename + '">' +

                    filename.replace("_", ": ") +

                    '</a></span></caption>'

    @@ -359,7 +364,7 @@ function sparqlToDataTable2(url, editURL, sparql, element, filename, options = {
  
        var paging = (typeof options.paging === 'undefined') ? true : options.paging;

        var sDom = (typeof options.sDom === 'undefined') ? 'lfrtip' : options.sDom;

        var sparqlEndpointName = (typeof options.sparqlEndpointName === 'undefined')

    	? window.jsConfig.sparqlEndpointName : options.sparqlEndpointName;

        ? window.jsConfig.sparqlEndpointName : options.sparqlEndpointName;

        // overwrite the central URLs of SPARQL specific URLs are found

        configFromSPARQL = extractConfig(sparql);

    @@ -372,7 +377,7 @@ function sparqlToDataTable2(url, editURL, sparql, element, filename, options = {
  
        const datatableFooter =

            '<caption><span style="float:left; font-size:smaller;"><a href="' + editURL +

            encodeURIComponent(sparql) + '">' + sparqlEndpointName + '</a></span>' +

            '<span style="float:right; font-size:smaller;"><a href="https://github.com/WDscholia/scholia/blob/main/scholia/app/templates/' +

            '<span style="float:right; font-size:smaller;"><a href="https://github.com/ad-freiburg/scholia/blob/qlever/scholia/app/templates/' +

            filename +

            '">' +

            filename.replace('_', ': ') +

    @@ -484,11 +489,15 @@ function sparqlToDataTable2(url, editURL, sparql, element, filename, options = {
  
                    }

                    $(element).append(datatableFooter);

                }

            }).fail(function () {

            }).fail(function (jqXHR, textStatus, errorThrown) {

                $('#' + loaderID).remove(); // remove loader

                $(element).prepend(

                    '<p>This query has timed out, we recommend that you follow the link to the Wikidata Query Service below to modify the query to be less intensive. </p> '

                );

                let error_message = "";

                try {

                    error_message = "QLever execption: " + JSON.parse(jqXHR.responseText).exception;

                } catch (e) {

                    error_message = "getJSON query failed: " + textStatus + " " + errorThrown;

                }

                $(element).prepend("<p style='color:red;'>" + escapeHTML(error_message) + "</p>");

                const reloadButton = document.getElementById(element.slice(1) + '-reload');

                reloadButton.classList.add('btn-secondary');

                reloadButton.classList.remove('btn-outline-secondary');

    @@ -516,6 +525,13 @@ function sparqlToIframe2(url, editURL, embedURL, sparql, element, filename) {
  
        if (!embedURL) embedURL = "https://query.wikidata.org/embed.html#";

        // overwrite the central URLs of SPARQL specific URLs are found

        configFromSPARQL = extractConfig(sparql);

        if (configFromSPARQL["url"]) url = configFromSPARQL["url"];

        if (configFromSPARQL["editURL"]) editURL = configFromSPARQL["editURL"];

        if (configFromSPARQL["embedURL"]) embedURL = configFromSPARQL["embedURL"];

        if (configFromSPARQL["endpointName"]) sparqlEndpointName = configFromSPARQL["endpointName"];

        const wikidata_sparql = url + "?query=" + encodeURIComponent(sparql);

        const wikidata_query = editURL + encodeURIComponent(sparql);

        var url = embedURL + encodeURIComponent(sparql);

    @@ -549,6 +565,7 @@ function sparqlToIframe2(url, editURL, embedURL, sparql, element, filename) {
  
        $.ajax({

            url: wikidata_sparql,

            headers: { Accept: "application/sparql-results+xml" },

            success: function (data) {

                let $xml = $(data);

                let results = $xml.find('results');

    @@ -559,7 +576,7 @@ function sparqlToIframe2(url, editURL, embedURL, sparql, element, filename) {
  
                }

                $iframe.parent().after(

                    '<span style="float:right; font-size:smaller">' +

                        '<a href="https://github.com/WDscholia/scholia/blob/main/scholia/app/templates/' + filename + '">' +

                        '<a href="https://github.com/ad-freiburg/scholia/blob/qlever/scholia/app/templates/' + filename + '">' +

                            filename.replace("_", ": ") +

                        '</a>' +

                    '</span>'

    @@ -796,9 +813,12 @@ function askQuery(panel, askQuery, callback) {
  
    function askQuery2(endpointUrl, panel, askQuery, callback) {

         if (!endpointUrl) endpointUrl = "https://query.wikidata.org/sparql";

         var settings = {

        if (!endpointUrl) endpointUrl = "https://query.wikidata.org/sparql";

        // overwrite the central URLs of SPARQL specific URLs are found

        configFromSPARQL = extractConfig(askQuery);

        if (configFromSPARQL["url"]) endpointUrl = configFromSPARQL["url"];

        var settings = {

           headers: { Accept: 'application/sparql-results+json' },

           data: { query: askQuery },

         };

scholia/app/templates/404-chemical_related.sparql

-Original file line number
+Diff line change
@@ -1,19 +1,16 @@
-    SELECT ?mol ?molLabel ?InChIKey ?CAS ?ChemSpider ?PubChem_CID WITH {
-      SELECT ?mol ?InChIKey WHERE {
-        SERVICE wikibase:mwapi {
-            bd:serviceParam wikibase:endpoint "www.wikidata.org";
-            wikibase:api "Search";
-            mwapi:srsearch "_shortkey_ haswbstatement:P235";
-            mwapi:srlimit "max".
-            ?mol wikibase:apiOutputItem mwapi:title.
-          }
-        ?mol wdt:P235 ?InChIKey .
-        FILTER (regex(str(?InChIKey), "^_shortkey_"))
+    {% import 'sparql-helpers.sparql' as sparql_helpers -%}
+    PREFIX wdt: <http://www.wikidata.org/prop/direct/>
+    SELECT ?mol ?molLabel ?InChIKey ?CAS ?ChemSpider ?PubChem_CID {
+      ?mol wdt:P235 ?InChIKey .
+      FILTER (REGEX(?InChIKey,"^_shortkey_"))
+      OPTIONAL {
+        ?mol wdt:P231 ?CAS
       }
-    } AS %MOLS {
-      INCLUDE %MOLS
-      OPTIONAL { ?mol wdt:P231 ?CAS }
-      OPTIONAL { ?mol wdt:P661 ?ChemSpider }
-      OPTIONAL { ?mol wdt:P662 ?PubChem_CID }
-      SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en" . }
+      OPTIONAL {
+        ?mol wdt:P661 ?ChemSpider
+      }
+      OPTIONAL {
+        ?mol wdt:P662 ?PubChem_CID
+      }
+      {{ sparql_helpers.labels(["?mol"], languages) }}
     }

scholia/app/templates/ask_author_retractions.sparql

-Original file line number
+Diff line change
@@ -1,4 +1,14 @@
+    PREFIX wd: <http://www.wikidata.org/entity/>
+    PREFIX wdt: <http://www.wikidata.org/prop/direct/>
     ASK {
-        ?work wdt:P50 wd:{{q}} . BIND("author-retractions" AS ?aspectsubpage)
-        { ?work wdt:P31 wd:Q45182324 } UNION { ?work wdt:P793 wd:Q7316896 . } UNION { ?work wdt:P5824 [] . }
+      #    ?work wdt:P50 wd:{{q}} . BIND("author-retractions" AS ?aspectsubpage)
+      {
+        ?work wdt:P31 wd:Q45182324
+      }
+      UNION {
+        ?work wdt:P793 wd:Q7316896 .
+      }
+      UNION {
+        ?work wdt:P5824 [] .
+      }
     }

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update of the `main` branch to work with QLever #2715

Uh oh!

Diff view

Diff view

Uh oh!

There are no files selected for viewing

Uh oh!

Update of the main branch to work with QLever #2715

Are you sure you want to change the base?

Uh oh!

Update of the main branch to work with QLever #2715

Uh oh!

Diff view

Diff view

Uh oh!

There are no files selected for viewing

Uh oh!

Update of the `main` branch to work with QLever #2715

Update of the `main` branch to work with QLever #2715