Integrating with GOOGLE_API_KEY

[gabegaz]

I want to integrate it with gemini. I changed some the relevant code base but doesn't work. Any plan to also use gemini along with openai?

[NathanTarbert]

Hey @gabegaz, if you are using Copilot Cloud, Gemini is not currently supported.

We plan to expand LLM support to many others including Gemini in the near future.

If you are self-hosting then yes we do have a Gemini adapter.

[awsxuhi]

I changed the model in app/api/copilotkit/route.ts and agent/sample_agent/agent.py to the Claude model supported by AWS Bedrock, and also modified the serviceAdapter in route.ts accordingly to the following:

const serviceAdapter = new LangChainAdapter({
    chainFn: async ({ messages, tools }) => {
        // Filter out empty messages and ensure proper message types
        const validMessages = messages
            .filter(msg => msg.content && msg.content.toString().trim() !== '')
            .map(msg => {
                if (msg._getType() === 'human') {
                    return new HumanMessage({ content: msg.content });
                } else if (msg._getType() === 'ai') {
                    return new AIMessage({ content: msg.content });
                }
                return msg;
            });

        // Ensure the last message is from human
        const lastMessage = validMessages[validMessages.length - 1];
        if (lastMessage && lastMessage._getType() === 'ai') {
            validMessages.pop();
        }

        const model = new ChatBedrockConverse({
            model: "anthropic.claude-3-sonnet-20240229-v1:0",
            temperature: 0,
            region: 'us-east-1',
            maxRetries: 2,
            // credentials: {
            //     secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
            //     accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
            // },
        });

        if (tools && tools.length > 0) {
            return model.bindTools(tools).stream(validMessages);
        } else {
            return model.stream(validMessages);
        }
    },
});

However, during testing, it gets stuck at the "math executing" stage when using tools. Does this mean AWS Bedrock is also not supported?

[NathanTarbert]

Hi @awsxuhi, using the LangChain adapter should work because they support Amazon Bedrock
https://python.langchain.com/docs/integrations/llms/

Let me look into this.