Revert Vercel AI SDK adoption and implement clean abstraction for LLM providers

[bhouston]

Revert Vercel AI SDK adoption and implement clean abstraction for LLM providers

The recent adoption of the Vercel AI SDK has introduced reliability issues. We need to revert these changes and implement a cleaner abstraction for LLM providers using the Anthropic SDK directly.

Background

• PR Implement Vercel AI SDK for Anthropic Claude support #55 introduced the Vercel AI SDK for Anthropic Claude support
• Subsequent PRs added support for Ollama, OpenAI, and Grok/XAI using the Vercel AI SDK
• These changes have led to reliability problems

Tasks

1. Remove dependencies on Vercel AI SDK (ai and @ai-sdk/* packages)
2. Create our own message type system with support for:
   • System messages
   • User messages
   • Assistant messages
   • Tool use/result messages
3. Implement a generic generateText() function that:
   • Takes messages, tools, and standard LLM settings
   • Returns a new set of messages
4. Create an Anthropic-specific implementation that:
   • Takes API key and model name
   • Returns a function matching the generic generateText() type
   • Handles conversion between our message format and Anthropic's format

Non-goals for this PR

• Token caching (will be added in a follow-up issue)
• Support for OpenAI, Ollama, and Grok (will be added in follow-up issues)

Benefits

• More reliable LLM integration
• Cleaner abstraction between the application and specific LLM providers
• Easier to add support for other models in the future

[bhouston]

Implementation Plan: Revert Vercel AI SDK and Implement Clean Abstraction

1. Project Structure Changes

We'll create a new directory structure for our LLM abstraction:

packages/agent/src/core/llm/
├── types.ts           # Message types and interfaces
├── anthropic.ts       # Anthropic-specific implementation
├── generateText.ts    # Generic text generation function
└── index.ts           # Main exports

2. Message Types Implementation

Define our own message types in types.ts:

export type MessageRole = 'system' | 'user' | 'assistant' | 'tool';

export interface BaseMessage {
  role: MessageRole;
}

export interface SystemMessage extends BaseMessage {
  role: 'system';
  content: string;
}

export interface UserMessage extends BaseMessage {
  role: 'user';
  content: string;
}

export interface AssistantMessage extends BaseMessage {
  role: 'assistant';
  content: string;
}

export interface ToolCallMessage extends BaseMessage {
  role: 'tool';
  name: string;
  arguments: Record<string, unknown>;
}

export interface ToolResultMessage extends BaseMessage {
  role: 'tool';
  toolCallId: string;
  content: string;
}

export type Message = 
  | SystemMessage 
  | UserMessage 
  | AssistantMessage 
  | ToolCallMessage 
  | ToolResultMessage;

// Tool definitions
export interface ToolDefinition {
  name: string;
  description: string;
  parameters: Record<string, unknown>;
}

// GenerateText function type
export interface GenerateTextOptions {
  messages: Message[];
  tools?: ToolDefinition[];
  model: string;
  maxTokens?: number;
  temperature?: number;
}

export interface GenerateTextResult {
  text: string;
  toolCalls: ToolCallMessage[];
}

export type GenerateTextFunction = (
  options: GenerateTextOptions
) => Promise<GenerateTextResult>;

3. Implementation Steps

Phase 1: Remove Vercel AI SDK Dependencies

1. Update package.json to remove Vercel AI SDK dependencies:

   • Remove ai package
   • Remove @ai-sdk/anthropic, @ai-sdk/mistral, @ai-sdk/openai, @ai-sdk/xai
   • Remove ollama-ai-provider
   • Add direct dependency on @anthropic-ai/sdk

2. Create a new branch from the state before Vercel AI SDK was introduced

   • Use git checkout -b revert-vercel-ai-sdk main
   • Apply only the necessary non-Vercel AI SDK changes since then

Phase 2: Implement New Abstraction

1. Create the message types and interfaces in types.ts

2. Implement the Anthropic adapter in anthropic.ts:

   import Anthropic from '@anthropic-ai/sdk';
   import { 
     Message, 
     GenerateTextFunction, 
     GenerateTextOptions, 
     GenerateTextResult,
     ToolDefinition
   } from './types';
   
   // Conversion functions between our message format and Anthropic's
   function convertToAnthropicMessages(messages: Message[]): Anthropic.Messages.MessageParam[] {
     // Implementation
   }
   
   function convertToolsToAnthropicTools(tools: ToolDefinition[]): Anthropic.Tool[] {
     // Implementation
   }
   
   export function createAnthropicTextGenerator(
     apiKey: string,
     modelName: string
   ): GenerateTextFunction {
     const client = new Anthropic({ apiKey });
     
     return async function generateText(
       options: GenerateTextOptions
     ): Promise<GenerateTextResult> {
       const { messages, tools, maxTokens, temperature } = options;
       
       const anthropicMessages = convertToAnthropicMessages(messages);
       const anthropicTools = tools ? convertToolsToAnthropicTools(tools) : undefined;
       
       const response = await client.messages.create({
         model: modelName,
         messages: anthropicMessages,
         max_tokens: maxTokens || 4096,
         temperature: temperature || 0.7,
         tools: anthropicTools,
       });
       
       // Convert Anthropic response to our format
       return {
         text: response.content[0]?.text || '',
         toolCalls: [], // Convert tool calls from Anthropic format
       };
     };
   }

3. Implement the generic generateText.ts:

   import { GenerateTextFunction, GenerateTextOptions, GenerateTextResult } from './types';
   import { createAnthropicTextGenerator } from './anthropic';
   
   export function createTextGenerator(
     provider: string,
     modelName: string,
     apiKey: string
   ): GenerateTextFunction {
     switch (provider) {
       case 'anthropic':
         return createAnthropicTextGenerator(apiKey, modelName);
       default:
         throw new Error(`Unsupported provider: ${provider}`);
     }
   }

4. Update the toolAgentCore.ts to use our new abstraction instead of Vercel AI SDK

Phase 3: Update Configuration and CLI

1. Update config.ts to use our new model abstraction
2. Update the CLI to pass the correct parameters to our new functions
3. Update the provider configuration in errors.ts if needed

Phase 4: Testing

1. Update tests to use the new abstraction
2. Ensure all tests pass with the new implementation
3. Manually test with Anthropic Claude models

4. Follow-up Issues

Create these additional issues:

1. "Add token caching to the LLM abstraction"
2. "Add OpenAI support to the LLM abstraction"
3. "Add Ollama support to the LLM abstraction"
4. "Add Grok/XAI support to the LLM abstraction"

[bhouston]

This is completed.