SQL Database with too many tables

[kmirijan]

I'm trying to put together a proof of concept when it comes to looking over a very large database with hundreds of tables. I've went ahead and created an NBA database with the game logs of every player for the season. The issue I run into is that there are about 500 players. So when using a SQL agent or SQL chain, I run into an issue. The system takes all of the table names as observations which is far beyond the 4097 token limit of the default OpenAI model.

So what is a possible work around. Can I pre-train my model to understand my table structure prior to asking a question? So that way it doesn't have to search everything from the start? Or maybe some kind of pre filtering based on the question? I honestly don't know

[willsmithorg]

See #4293 for some ideas, you can supply a list of tables to consider.