Does anyone know what is roughly the database size limit for SQL agent?

[nthanhtin]

I'm just curious would SQLDatabaseChain still work if I have a really large database?

[oddrationale]

Generally speaking, yes. The SQLDatabaseChain will still work if you have a really large database. SQLDatabaseChain doesn't load the entire database into memory. Instead, it formulates a SQL query based on your input and then runs it against the database. The only time you may run into issues is if the SQL query result is too long and exceeds the prompt token limit for your chosen LLM. For example, davinci has a limit of about 4,000 tokens.

If you're using the SQL Database Agent (rather than the chain), be aware that it will start with listing all the tables in the db. This can also potentially cause problems if the number of tables is too large and exceeds the prompt token limit.

Hope that helps!

[nthanhtin]

Thanks, @oddrationale

[naman-gupta99]

Hey, I have a follow-up question. Doesn't the SQL Agent load up the DDL of the database into the context of the current session. If so, then if the database has a large DDL won't that also create an issue?

[ghost]

Yes, it does.

    print(db_chain.run("get me count of all invoices"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 451, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 258, in __call__
    raise e
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 252, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain_experimental/sql/base.py", line 185, in _call
    raise exc
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain_experimental/sql/base.py", line 118, in _call
    sql_cmd = self.llm_chain.predict(
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/llm.py", line 252, in predict
    return self(kwargs, callbacks=callbacks)[self.output_key]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 258, in __call__
    raise e
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 252, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/llm.py", line 92, in _call
    response = self.generate([inputs], run_manager=run_manager)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/chains/llm.py", line 102, in generate
    return self.llm.generate_prompt(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/base.py", line 451, in generate_prompt
    return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/base.py", line 582, in generate
    output = self._generate_helper(
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/base.py", line 488, in _generate_helper
    raise e
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/base.py", line 475, in _generate_helper
    self._generate(
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/openai.py", line 399, in _generate
    response = completion_with_retry(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/openai.py", line 115, in completion_with_retry
    return _completion_with_retry(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
           ^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/langchain/llms/openai.py", line 113, in _completion_with_retry
    return llm.client.create(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/openai/api_resources/completion.py", line 25, in create
    return super().create(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
                           ^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/openai/api_requestor.py", line 230, in request
    resp, got_stream = self._interpret_response(result, stream)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/openai/api_requestor.py", line 624, in _interpret_response
    self._interpret_response_line(
  File "/Users/adityazagade/code/chatgpt/openai-quickstart-python/venv/lib/python3.11/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 2049 tokens, however you requested 66700 tokens (58700 in your prompt; 8000 for the completion). Please reduce your prompt; or completion length.

[starball5]

Related on Stack Overflow: Does the SQLDatabase agent for langchain store the database in memory?

[usersina]

I have a table with 45 rows, 46 column each and run into the limit already. Here's the error I got:

This model's maximum context length is 4097 tokens, however you requested 4693 tokens (4437 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.

[alanwilhelm]

cant you use a chunker to get around this?

[usersina]

A solution worth testing, yes. Following the output though, I see that the agent does the following:

• Attempts to execute the query, fails to do so because of "misspellings" (e.g. SELECT * FROM my_table instead of SELECT * from MySchema.my_table for PostgreSQL, or even a small My_table/my_table misspelling).
• Attempts to fix the query and sends ALL PREVIOUS CONTEXT including the table definitions, and previous failed queries.

The above repeats for 3 to 5 times before I hit the token limit. One thing I didn't attempt yet is to provide context on the exact table names, or in other words extend the toolkit.