Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .idea/copilot.data.migration.ask2agent.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

138 changes: 125 additions & 13 deletions lib/chatWithCollabifyAI.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,95 @@ import {
GenerationConfig,
} from "@google/generative-ai";

type GeminiModelListResponse = {
models?: Array<{
name?: string;
supportedGenerationMethods?: string[];
}>;
};

const FALLBACK_GEMINI_MODELS = ["gemini-2.5-flash"];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The model name gemini-2.5-flash appears to be a typo. The current flash model from Google is gemini-1.5-flash. Using an incorrect model name will cause the fallback mechanism to fail. Please verify and correct the model name.

Suggested change
const FALLBACK_GEMINI_MODELS = ["gemini-2.5-flash"];
const FALLBACK_GEMINI_MODELS = ["gemini-1.5-flash"];

const MODEL_CACHE_TTL_MS = 5 * 60 * 1000;
let cachedGeminiModels: { models: string[]; fetchedAt: number } | null = null;
let modelRotationIndex = 0;

const isEligibleGeminiModel = (model: {
name?: string;
supportedGenerationMethods?: string[];
}): boolean => {
const name = model.name ?? "";
if (!name.startsWith("models/gemini-")) {
return false;
}

const loweredName = name.toLowerCase();
if (loweredName.includes("embedding") || loweredName.includes("pro")) {
return false;
}

const methods = model.supportedGenerationMethods ?? [];
return methods.length === 0 || methods.includes("generateContent");
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The condition methods.length === 0 optimistically assumes that if the supportedGenerationMethods array is empty or missing, the model is eligible for generateContent. This could be a risky assumption and might lead to using models that don't support content generation. It would be safer to explicitly check for the generateContent method. If the API documentation guarantees that an empty array implies support, please add a comment to clarify this behavior.

Suggested change
return methods.length === 0 || methods.includes("generateContent");
return methods.includes("generateContent");

};

const normalizeModelName = (name: string): string =>
name.replace(/^models\//, "");

const dedupeModels = (models: string[]): string[] => {
const seen = new Set<string>();
const unique: string[] = [];
for (const model of models) {
if (!seen.has(model)) {
seen.add(model);
unique.push(model);
}
}
return unique;
};
Comment on lines +41 to +51
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The dedupeModels function can be made more concise and idiomatic by using a Set to automatically handle uniqueness. This improves readability and reduces the amount of code.

const dedupeModels = (models: string[]): string[] => [...new Set(models)];


const getRotatedModels = (models: string[]): string[] => {
if (models.length === 0) {
return models;
}
const startIndex = modelRotationIndex % models.length;
modelRotationIndex = (modelRotationIndex + 1) % models.length;
Comment on lines +57 to +58
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There is a potential race condition here with modelRotationIndex. Since it's a shared mutable global variable, concurrent requests to getRotatedModels could read the same modelRotationIndex value before it's updated. This would result in them receiving the same model sequence, leading to imperfect rotation under high load. While this might not be critical for the current application, it's an important consideration for concurrent environments.

return [...models.slice(startIndex), ...models.slice(0, startIndex)];
};

const fetchGeminiModels = async (apiKey: string): Promise<string[]> => {
if (
cachedGeminiModels &&
Date.now() - cachedGeminiModels.fetchedAt < MODEL_CACHE_TTL_MS
) {
return cachedGeminiModels.models;
}

const response = await fetch(
`https://generativelanguage.googleapis.com/v1/models?key=${encodeURIComponent(
apiKey,
)}`,
);

if (!response.ok) {
throw new Error(
`Failed to fetch Gemini models: ${response.status} ${response.statusText}`,
);
}

const data = (await response.json()) as GeminiModelListResponse;
const models = (data.models ?? [])
.filter(isEligibleGeminiModel)
.map((model) => normalizeModelName(model.name ?? ""))
.filter(Boolean);

const uniqueModels = dedupeModels(models);
if (uniqueModels.length === 0) {
throw new Error("No eligible Gemini models found.");
}

cachedGeminiModels = { models: uniqueModels, fetchedAt: Date.now() };
return uniqueModels;
};

/**
* Sends a chat message to Gemini AI. This helper preserves conversation history so that the context
* is maintained between messages. The assistant is identified as "Collabify Assistant" and acts as a
Expand Down Expand Up @@ -41,10 +130,6 @@ export async function chatWithCollabifyAI(
`;

const genAI = new GoogleGenerativeAI(apiKey);
const model = genAI.getGenerativeModel({
model: "gemini-1.5-flash",
systemInstruction: defaultSystemInstruction,
});

const generationConfig: GenerationConfig = {
temperature: 1,
Expand Down Expand Up @@ -74,17 +159,44 @@ export async function chatWithCollabifyAI(

history.push({ role: "user", parts: [{ text: message }] });

const chatSession = model.startChat({
generationConfig,
safetySettings,
history,
});
let modelNames = FALLBACK_GEMINI_MODELS;
try {
modelNames = await fetchGeminiModels(apiKey);
} catch {
modelNames = FALLBACK_GEMINI_MODELS;
}
Comment on lines +165 to +167
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The catch block is currently empty, which means any errors from fetchGeminiModels will be silently ignored, and the system will fall back to the default models without any indication of a problem. This can make debugging difficult. It's a good practice to log the error to provide visibility into failures.

} catch (error) {
    console.error("Failed to fetch Gemini models, using fallback:", error);
    modelNames = FALLBACK_GEMINI_MODELS;
  }


const rotatedModels = getRotatedModels(modelNames);
let lastError: unknown = null;

for (const modelName of rotatedModels) {
try {
const model = genAI.getGenerativeModel({
model: modelName,
systemInstruction: defaultSystemInstruction,
});

const chatSession = model.startChat({
generationConfig,
safetySettings,
history,
});

const result = await chatSession.sendMessage(message);
const result = await chatSession.sendMessage(message);

if (!result.response || !result.response.text) {
throw new Error("Failed to get text response from the AI.");
}

return result.response.text();
} catch (error) {
lastError = error;
}
Comment on lines +192 to +194
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In the model rotation loop, you're catching errors but only storing the last one. If multiple models fail, the reasons for the earlier failures are lost, which can make debugging more difficult. It would be beneficial to log each error as it occurs to get a complete picture of what went wrong during the rotation.

    } catch (error) {
      console.warn(`Model ${modelName} failed:`, error);
      lastError = error;
    }

}

if (!result.response || !result.response.text) {
throw new Error("Failed to get text response from the AI.");
if (lastError instanceof Error) {
throw lastError;
}

return result.response.text();
throw new Error("Failed to get text response from the AI.");
}
Loading