On-Device AI Chat App

A Flutter application that demonstrates on-device AI chat capabilities using the Cactus Flutter library. This app runs AI models locally on your device, providing privacy-focused conversational AI without requiring an internet connection for inference.

Features

• 🤖 On-Device AI: Runs AI models locally using Cactus Flutter
• 💬 Chat Interface: Clean, Material 3 chat UI with message bubbles
• 📥 Model Download: Automatic model downloading with progress tracking
• 🎨 Markdown Support: Rich text rendering for AI responses
• 🔄 State Management: Built with Riverpod for reactive state management
• 🧹 Chat Management: Clear conversation history with a single tap

Prerequisites

Before you begin, ensure you have the following installed:

• Flutter SDK: Version 3.10.0 or higher
  • Install Flutter
• Dart SDK: Comes bundled with Flutter
• Platform-specific requirements:
  • iOS: Xcode 14+ and CocoaPods
  • Android: Android Studio with Android SDK
  • macOS/Linux/Windows: Platform-specific development tools

Setup

1. Clone the repository (or navigate to the project directory):

   cd /path/to/on_device_app

2. Install dependencies:

   flutter pub get

3. Generate code (for Riverpod and Freezed):

   flutter pub run build_runner build --delete-conflicting-outputs

4. Platform-specific setup:

   iOS Setup

   cd ios
   pod install
   cd ..

   Android Setup

   No additional setup required beyond having Android SDK installed.

Running the App

1. Check available devices:

   flutter devices

2. Run the app:

   flutter run

   Or specify a device:

   flutter run -d <device-id>

3. Using the app:

   • On first launch, tap "Initialize Model (qwen3-0.6)" to download the AI model
   • Wait for the model download to complete (progress is shown)
   • Once initialized, type your message and press send
   • The AI will respond locally on your device

Project Structure

lib/
├── main.dart                 # App entry point
├── chat/
│   ├── chat_screen.dart      # Main chat UI
│   ├── chat_provider.dart    # Chat state management
│   ├── chat_state.dart       # Chat state model
│   └── message_bubble.dart   # Message UI component
├── core/                     # Core utilities
└── providers/                # Additional providers

Key Dependencies

• cactus: On-device AI inference
• hooks_riverpod: State management and dependency injection
• flutter_hooks: React-like hooks for Flutter
• freezed: Immutable state classes
• flutter_markdown_plus: Markdown rendering for AI responses

Troubleshooting

iOS Build Issues

If you encounter header errors or build failures on iOS:

cd ios
rm -rf Pods Podfile.lock
pod install
cd ..
flutter clean
flutter pub get

Model Download Fails

• Ensure you have a stable internet connection
• Check available storage space on your device
• Try clearing the app data and reinitializing

App Crashes on Launch

• Run flutter clean and rebuild
• Ensure all dependencies are properly installed
• Check device logs: flutter logs

Development

To contribute or modify the app:

1. Watch for code generation changes:

   flutter pub run build_runner watch

2. Run tests:

   flutter test

3. Format code:

   flutter format .

4. Analyze code:

   flutter analyze

License

This project is for educational and demonstration purposes.

Resources

• Cactus Flutter Documentation
• Flutter Documentation
• Riverpod Documentation