SQL Agent with open-source LLM

[hamza-amin-4365]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

llm = HuggingFaceEndpoint(
        huggingfacehub_api_token=API_KEY,
        repo_id="mistralai/Mixtral-8x7B-Instruct-v0.1",
        temperature=0.8,
        max_length=150
    )
    
agent = ZeroShotAgent(
        llm_chain=LLMChain(llm=llm, prompt=prompt.ZeroShotAgent.create_prompt(tools)),
        allowed_tools=[tool.name for tool in tools],
    )

    # Create the agent executor
    agent_executor = AgentExecutor.from_agent_and_tools(
        agent=agent,
        tools=tools,
        verbose=True,
        handle_parsing_errors=True
    )

Description

I'm trying to make an SQL agent with hugging face llm but it seems like the agent settings are only supposed to work with openai.
I've tried too many agents changing the whole toolkits and agent types still I get some errors regarding unexpected argument was passed. If anyone knows how to fix it please help.
Thanks

System Info

Windows 11

[dosubot[bot]]

Hey @hamza-amin-4365! 👋 I'm here to help you out with any bugs, questions, or contributions you might have. I'm a bot designed to assist you while you wait for a human maintainer. Let's squash those bugs together!

To resolve the issue of unexpected arguments being passed when creating an SQL agent using a Hugging Face LLM, ensure that the arguments passed to the createSqlAgent function match the expected parameters defined in the SqlCreatePromptArgs interface. Here is an example of how to set it up correctly:

import { HuggingFaceInference } from "@langchain/core/language_models/llms";
import { SqlToolkit, createSqlAgent } from "@langchain/agents/toolkits/sql";
import { SqlDatabase } from "@langchain/sql_db";

// Initialize the Hugging Face model
const model = new HuggingFaceInference({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
  temperature: 0.8,
  maxTokens: 150,
  apiKey: API_KEY,
});

// Initialize the SQL database and toolkit
const sqlDb = new SqlDatabase(/* your database config */);
const toolkit = new SqlToolkit(sqlDb, model);

// Create the SQL agent
const executor = createSqlAgent(model, toolkit, { topK: 5 });

// Invoke the agent
const result = await executor.invoke({
  input: "List the total sales per country. Which country's customers spent the most?",
});

console.log(`Got output ${result.output}`);

Ensure that when you call createSqlAgent, you pass the correct arguments as per the SqlCreatePromptArgs interface. This setup should help you avoid the unexpected argument errors you are encountering ^[1][2].

To continue talking to Dosu, mention @dosu.

[hamza-amin-4365]

@dosu
please send the code in python

[dosubot[bot]]

Here's the equivalent Python code for creating an SQL agent with a Hugging Face LLM:

from langchain.chains import LLMChain
from langchain.prompts import render_template
from langchain.agents import ZeroShotAgent, AgentExecutor
from langchain.tools.sql import InfoSqlTool, ListTablesSqlTool, QueryCheckerTool, QuerySqlTool
from langchain.sql_db import SqlDatabase
from langchain.core.language_models.base import BaseLanguageModelInterface
from langchain.core.tools import BaseToolkit

SQL_PREFIX = """You are an agent designed to interact with a SQL database.
Given an input question, create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.
Unless the user specifies a specific number of examples they wish to obtain, always limit your query to at most {top_k} results using the LIMIT clause.
You can order the results by a relevant column to return the most interesting examples in the database.
Never query for all the columns from a specific table, only ask for the few relevant columns given the question.
You have access to tools for interacting with the database.
Only use the below tools.
Only use the information returned by the below tools to construct your final answer.
You MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.

DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.

If the question does not seem related to the database, just return "I don't know" as the answer."""
SQL_SUFFIX = """Begin!

Question: {input}
Thought: I should look at the tables in the database to see what I can query.
{agent_scratchpad}"""

class SqlToolkit(BaseToolkit):
    def __init__(self, db: SqlDatabase, llm: BaseLanguageModelInterface = None):
        super().__init__()
        self.db = db
        self.dialect = "sqlite"
        self.tools = [
            QuerySqlTool(db),
            InfoSqlTool(db),
            ListTablesSqlTool(db),
            QueryCheckerTool(llm=llm),
        ]

def create_sql_agent(llm: BaseLanguageModelInterface, toolkit: SqlToolkit, args: dict = None):
    if args is None:
        args = {}
    prefix = args.get("prefix", SQL_PREFIX)
    suffix = args.get("suffix", SQL_SUFFIX)
    input_variables = args.get("inputVariables", ["input", "agent_scratchpad"])
    top_k = args.get("topK", 10)
    
    formatted_prefix = render_template(prefix, "f-string", {
        "dialect": toolkit.dialect,
        "top_k": top_k,
    })
    
    prompt = ZeroShotAgent.create_prompt(toolkit.tools, {
        "prefix": formatted_prefix,
        "suffix": suffix,
        "inputVariables": input_variables,
    })
    
    chain = LLMChain(prompt=prompt, llm=llm)
    agent = ZeroShotAgent(
        llm_chain=chain,
        allowed_tools=[tool.name for tool in toolkit.tools]
    )
    
    return AgentExecutor.from_agent_and_tools(
        agent=agent,
        tools=toolkit.tools,
        return_intermediate_steps=True
    )

# Example usage
from langchain.llms import HuggingFaceEndpoint

llm = HuggingFaceEndpoint(
    huggingfacehub_api_token="your_api_key_here",
    repo_id="mistralai/Mixtral-8x7B-Instruct-v0.1",
    temperature=0.8,
    max_length=150
)

sql_db = SqlDatabase(/* your database config */)
toolkit = SqlToolkit(sql_db, llm)
executor = create_sql_agent(llm, toolkit, { "topK": 5 })

result = executor.invoke({
    "input": "List the total sales per country. Which country's customers spent the most?",
})

print(f"Got output {result.output}")

This code sets up an SQL agent using a Hugging Face LLM in Python, similar to the TypeScript example you provided ^[1][2].

To continue talking to Dosu, mention @dosu.

[hamza-amin-4365]

Anyone please reply to me this bot knows nothing, it is importing wrong libraries who it's gonna give correct code.