nl to sparql models revisited (and csv to nl!)

[donaldziff]

Hello! My team and I are interested in implementing an end to end KGQA system over (wikibase) for our Master's project. I read this discussion and it was very helpful. Thanks for your description of how you created the Harry Potter model. I agree with the other commenters about your step 2! I love that idea.

We have two open-ended questions:

1. Since more than a year has passed, what model or models would you use as a base for a sparql-generation model if you were implementing it now, Feb 2023? Would you use about the same steps?
2. We're also interested in presenting the results of the Sparql query back to the user in Natural Language. Do you know of pre-trained models for csv (or json) to NL translations?

Thanks in advance for any help you can give.

Donald "Max" Ziff

[julian-risch]

Hello @donaldziff that sounds like a great project idea. You could definitely use the same process that we used two years ago. An alternative is not to train any model but use large language models like GPT-3. For example, you could use Haystack's PromptNode with an OpenAI api key and create a PromptTemplate that instructs the model to translate the NL question into a SPARQL query. Here is our documentation page about PromptNode and the creation of PromptTemplate
It might not work out of the box so you might need to break into into multiple prompts. For example, find entities and relations in the query first, map them to what you have in your knowledge graph and then build the final query. I assume one of the main challenges will be the mapping of entities and relations from natural language questions to the ones in your large wikibase knowledge graph.

Similarly, you could use the PromptNode with a custom template to translate csv or json results into NL answers. For example, the template could be sth like:
"Translate the following json ... into a text answer for the question ..."

Good luck with your Master's project!

[donaldziff]

Thanks very much @julian-risch ! I do have a few follow-ups:

I like the idea you outline as an alternative, partly because I tried to sell my team on that idea, but they didn't bite :-). Maybe we will reconsider it now.

But, if we go the other route, perhaps I'm missing something obvious: what kind of model did you start with? Was it a model that already knew something about sparql generation? Or did you start with some flavor of BERT and the only training it got for generating sparql was the LC-QuAD dataset, and then the fine-tuning on Harry Potter?

BTW, in your answer you refer to a page: https://haystack.deepset.ai/docs/latest/knowledgegraphmd but it doesn't exist.

Thanks agaiun,

Donald (Max) Ziff

[mayankjobanputra]

Maybe try this.
https://docs.haystack.deepset.ai/docs/knowledge_graph

[JasperLS]

If I understand Julian's answer correctly, they used a BART model. You don't necessarily need BART, but you will probably need a seq2seq model. The T5 model might be a fitting update. You may even try a custom pre-trained version like this.