Can Spacy Support other than Natural Language?

[RAJUUBHAI]

Hi,
I would like to know that whether spacy is capable of supporting SQL (Structured Query Language).Instead of extracting the numbers, nouns and verbs, Can spacy extract metadata from SQL i.e., Column and table names. I believe It is possible but just wanted to know like whether if someone has already tired to implement it .

[DuyguA]

For parsing out SQL queries, you don't need to parse natural language, you can do it in an easier way.

SQL is a context free language, which means it's "easier" to parse. Context free languages are more complicated than regular languages and easier than human languages.

Here's a great resource: Hopcroft and Ullmann Automata Theory book . You're looking for Chapter 3, Context Free Languages.

So basically a context free grammar for SQL queries look sth like this:

select_statement = SELECT column_statement FROM general_statement WHERE condition_statement | SELECT operator_statement FROM general_statement ;
general_statement = select_statement | table_name;
column_statement = * | column_name;

Notice that this is a grammar that finely breaks down the SQL query strings into its elements. This is not possible for natural language. Natural languages are hard by nature, this is the reason we use statistical models. Regular and context free languages are represented by easier grammars(which are regular grammars and CF grammars), hence can be parsed with less complicated tools. Statistical models are big hammers, you don't need them for every problem especially when you know the problem already have an easier and well-defined alternative 😉 .

Here's an ANTLR grammar for PostgreSQL: https://github.com/antlr/grammars-v4/tree/master/sql/postgresql

If you want to do it with Python, I recommend Pyparsing lib. This is an easy calculator parser.

Another recommendation is Lark , quite fun to use indeed. Here's a JSON parser example. Another example comes from me, this is a CFG parser I implemented with Lark to parse data end time strings: https://duygua.github.io/blog/2018/03/28/chatbot-nlu-series-datetimeparser/

[RAJUUBHAI]

Thank you