PlexusOne PlexusOne Mobile Companion App

Overview

A Flutter-based mobile companion app that connects to the macOS PlexusOne Desktop orchestrator, allowing users to monitor and interact with AI CLI agents (Claude Code, Kiro CLI) from iOS and Android devices.

Goals

Monitor - View real-time output from multiple AI agent sessions
Interact - Respond to TUI prompts, approval requests, and wizards
Control - Start/stop sessions, switch between agents, send commands
Mobility - Work away from desk while agents run tasks

Non-Goals (Phase 1)

Full terminal emulation (no xterm.dart)
Voice input (future feature via omnivoice)
Cloud relay (LAN-first approach)

Architecture

┌─────────────────────────────────────────────────────────────────┐
│  macOS (PlexusOne Desktop Orchestrator)                                     │
│                                                                 │
│  ┌─────────────┐     ┌─────────────────────────────────────┐   │
│  │ tmux        │     │ TUI Parser Wrapper (Go)             │   │
│  │ sessions    │────▶│ - PTY intercept                     │   │
│  │             │◀────│ - ANSI parsing                      │   │
│  │ - coder-1   │     │ - Pattern detection                 │   │
│  │ - coder-2   │     │ - Structured event emission         │   │
│  │ - reviewer  │     │ - Keystroke injection               │   │
│  └─────────────┘     └──────────────┬──────────────────────┘   │
│                                     │                           │
│  ┌─────────────────────────────────┐│                           │
│  │ PlexusOne Desktop Swift App                 ││                           │
│  │ - Session management            │◀───── Local IPC            │
│  │ - SwiftTerm panes               ││                           │
│  │ - State persistence             ││                           │
│  └─────────────────────────────────┘│                           │
│                                     │                           │
│  ┌──────────────────────────────────▼──────────────────────┐   │
│  │ WebSocket Server (Go or embedded in wrapper)            │   │
│  │ - Port 9600 (default)                                   │   │
│  │ - JSON protocol                                         │   │
│  │ - Session multiplexing                                  │   │
│  └──────────────────────────────────┬──────────────────────┘   │
└─────────────────────────────────────┼───────────────────────────┘
                                      │ WebSocket (LAN)
                                      │
┌─────────────────────────────────────▼───────────────────────────┐
│  Mobile App (Flutter)                                           │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │ Session Tabs                                             │   │
│  │ [coder-1 ●] [coder-2 ○] [reviewer ●]                    │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │ Terminal-Style Output View                               │   │
│  │ - Monospace font, dark theme                            │   │
│  │ - Scrollable log                                        │   │
│  │ - Tap to copy                                           │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │ Interactive Prompt Area (context-aware)                  │   │
│  │ - Quick action buttons (Yes/No/Always)                  │   │
│  │ - Virtual D-pad for menu navigation                     │   │
│  │ - Native checkboxes for multi-select                    │   │
│  │ - Text input field                                      │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │ Status Bar                                               │   │
│  │ - Connection status                                     │   │
│  │ - Agent status (running/idle/stuck)                     │   │
│  │ - Token usage (if available)                            │   │
│  └─────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

WebSocket Protocol

Connection

ws://<laptop-ip>:9600/ws

Message Format

All messages are JSON with a type field:

{
  "type": "<message_type>",
  "sessionId": "<tmux_session_name>",
  "timestamp": 1710945600,
  ...
}

Server → Client Messages

`sessions` - Session list update

{
  "type": "sessions",
  "sessions": [
    {
      "id": "coder-1",
      "name": "coder-1",
      "status": "running",
      "lastActivity": 1710945600
    }
  ]
}

`output` - Terminal output chunk

{
  "type": "output",
  "sessionId": "coder-1",
  "text": "Building project...\n",
  "timestamp": 1710945600
}

`prompt` - Interactive prompt detected

{
  "type": "prompt",
  "sessionId": "coder-1",
  "promptType": "yes_no",
  "title": "Tool Approval",
  "message": "Allow Read tool on config.json?",
  "options": ["yes", "no", "always", "never"],
  "defaultOption": "yes",
  "timestamp": 1710945600
}

`menu` - Scrollable menu detected

{
  "type": "menu",
  "sessionId": "coder-1",
  "title": "Select tools to approve",
  "items": [
    {"label": "Read: config.json", "selected": false, "index": 0},
    {"label": "Write: output.txt", "selected": true, "index": 1},
    {"label": "Bash: npm install", "selected": false, "index": 2}
  ],
  "currentIndex": 1,
  "multiSelect": true,
  "timestamp": 1710945600
}

`wizard` - Multi-step wizard detected

{
  "type": "wizard",
  "sessionId": "coder-1",
  "title": "Configure task",
  "currentStep": 2,
  "totalSteps": 4,
  "fields": [
    {"name": "approach", "type": "select", "options": ["Option A", "Option B"]},
    {"name": "confirm", "type": "checkbox", "label": "I understand the changes"}
  ],
  "actions": ["back", "next", "submit"],
  "timestamp": 1710945600
}

`status` - Session status change

{
  "type": "status",
  "sessionId": "coder-1",
  "status": "idle",
  "tokenUsage": {
    "input": 15000,
    "output": 3200
  },
  "timestamp": 1710945600
}

`clear` - Clear terminal buffer

{
  "type": "clear",
  "sessionId": "coder-1"
}

Client → Server Messages

`subscribe` - Subscribe to session(s)

{
  "type": "subscribe",
  "sessionIds": ["coder-1", "coder-2"]
}

`unsubscribe` - Unsubscribe from session(s)

{
  "type": "unsubscribe",
  "sessionIds": ["coder-1"]
}

`input` - Send text input

{
  "type": "input",
  "sessionId": "coder-1",
  "text": "hello world"
}

`key` - Send special key

{
  "type": "key",
  "sessionId": "coder-1",
  "key": "enter"
}

Valid keys: enter, tab, escape, up, down, left, right, space, backspace, y, n, a

`action` - Respond to prompt/menu/wizard

{
  "type": "action",
  "sessionId": "coder-1",
  "action": "select",
  "value": "yes"
}

TUI Parser Wrapper

Overview

A Go binary that wraps tmux sessions, intercepts PTY output, detects TUI patterns, and emits structured events while also forwarding raw output.

Location

github.com/plexusone/plexusone-app/tuiparser/
├── cmd/
│   └── tuiparser/
│       └── main.go
├── internal/
│   ├── parser/
│   │   ├── parser.go       # Main parsing logic
│   │   ├── patterns.go     # Pattern definitions
│   │   └── ansi.go         # ANSI sequence handling
│   ├── pty/
│   │   └── pty.go          # PTY management
│   ├── server/
│   │   └── websocket.go    # WebSocket server
│   └── session/
│       └── manager.go      # Session management
├── pkg/
│   └── protocol/
│       └── messages.go     # Message types
└── go.mod

Pattern Detection

Yes/No Prompts

Patterns:
- "(y/n)" or "[Y/n]" or "[y/N]"
- "? (yes/no)"
- "Allow ... ?"
- "Approve ... ?"

Action: Emit `prompt` message with type `yes_no`

Multi-Select Menus (Claude Code style)

Patterns:
- Lines with "[ ]" or "[x]" or "◯" or "●"
- Highlighted/inverse video line (current selection)
- "Press space to toggle, enter to confirm"

Action: Emit `menu` message with items and selection state

Kiro CLI Tool Approval

Patterns:
- "Tool requests:" header
- Numbered list of tools
- "Enter numbers to approve" or similar

Action: Emit `menu` message with multiSelect: true

Wizard/Questionnaire (Claude Code AskUserQuestion)

Patterns:
- Step indicators "Step 1 of 3" or "[1/3]"
- Multiple choice options with radio buttons
- "Submit" or "Continue" at bottom

Action: Emit `wizard` message with fields and actions

Input Waiting

Patterns:
- Cursor at end of line with no recent output
- ">" or "$" or ":" prompt character
- Blinking cursor (timing-based detection)

Action: Emit `prompt` message with type `input`

ANSI Handling

The parser should:

Strip ANSI codes for clean text extraction
Detect cursor position for menu navigation
Identify inverse/highlight for current selection
Track screen state for multi-line TUI elements

Integration with PlexusOne Desktop

The TUI Parser runs as a daemon alongside PlexusOne Desktop:

# Started by PlexusOne Desktop app or launchd
tuiparser --port 9600 --tmux-socket /tmp/tmux-501/default

PlexusOne Desktop Swift app can:

Launch tuiparser on startup
Query session list via WebSocket
Let mobile app connect directly to tuiparser

Flutter Mobile App

Project Structure

nexus_mobile/
├── lib/
│   ├── main.dart
│   ├── app.dart
│   ├── models/
│   │   ├── session.dart
│   │   ├── message.dart
│   │   └── prompt.dart
│   ├── services/
│   │   ├── websocket_service.dart
│   │   ├── connection_service.dart
│   │   └── notification_service.dart
│   ├── screens/
│   │   ├── home_screen.dart
│   │   ├── session_screen.dart
│   │   └── settings_screen.dart
│   ├── widgets/
│   │   ├── terminal_view.dart
│   │   ├── session_tabs.dart
│   │   ├── prompt_bar.dart
│   │   ├── quick_actions.dart
│   │   ├── menu_selector.dart
│   │   ├── virtual_dpad.dart
│   │   └── status_indicator.dart
│   └── theme/
│       └── terminal_theme.dart
├── pubspec.yaml
└── README.md

Key Dependencies

dependencies:
  flutter:
    sdk: flutter
  web_socket_channel: ^2.4.0
  provider: ^6.1.0
  shared_preferences: ^2.2.0
  flutter_riverpod: ^2.4.0  # Alternative to provider

Terminal-Style View

A custom widget that renders output in a terminal aesthetic:

class TerminalView extends StatelessWidget {
  final List<OutputLine> lines;
  final ScrollController scrollController;

  @override
  Widget build(BuildContext context) {
    return Container(
      color: TerminalTheme.background,
      child: ListView.builder(
        controller: scrollController,
        itemCount: lines.length,
        itemBuilder: (context, index) {
          return TerminalLine(line: lines[index]);
        },
      ),
    );
  }
}

class TerminalLine extends StatelessWidget {
  final OutputLine line;

  @override
  Widget build(BuildContext context) {
    return Padding(
      padding: EdgeInsets.symmetric(horizontal: 8, vertical: 1),
      child: SelectableText(
        line.text,
        style: TextStyle(
          fontFamily: 'JetBrainsMono',  // or 'Menlo', 'Courier'
          fontSize: 12,
          color: _colorForStyle(line.style),
        ),
      ),
    );
  }

  Color _colorForStyle(LineStyle style) {
    switch (style) {
      case LineStyle.error: return TerminalTheme.red;
      case LineStyle.success: return TerminalTheme.green;
      case LineStyle.info: return TerminalTheme.blue;
      default: return TerminalTheme.foreground;
    }
  }
}

Interactive Prompt Bar

Context-aware input area that changes based on detected prompts:

class PromptBar extends StatelessWidget {
  final Prompt? activePrompt;
  final Function(String) onAction;
  final Function(String) onInput;

  @override
  Widget build(BuildContext context) {
    if (activePrompt == null) {
      return TextInputBar(onSubmit: onInput);
    }

    switch (activePrompt!.type) {
      case PromptType.yesNo:
        return QuickActionBar(
          options: activePrompt!.options,
          onSelect: onAction,
        );
      case PromptType.menu:
        return MenuSelector(
          items: activePrompt!.items,
          multiSelect: activePrompt!.multiSelect,
          onConfirm: onAction,
        );
      case PromptType.navigation:
        return VirtualDpad(onKey: onAction);
      default:
        return TextInputBar(onSubmit: onInput);
    }
  }
}

Quick Action Buttons

class QuickActionBar extends StatelessWidget {
  final List<String> options;
  final Function(String) onSelect;

  @override
  Widget build(BuildContext context) {
    return Row(
      mainAxisAlignment: MainAxisAlignment.spaceEvenly,
      children: options.map((option) {
        return ElevatedButton(
          onPressed: () => onSelect(option),
          style: _styleForOption(option),
          child: Text(_labelForOption(option)),
        );
      }).toList(),
    );
  }

  String _labelForOption(String option) {
    switch (option) {
      case 'yes': return 'Yes';
      case 'no': return 'No';
      case 'always': return 'Always';
      case 'never': return 'Never';
      default: return option;
    }
  }
}

Virtual D-Pad

For navigating menus when native UI detection fails:

class VirtualDpad extends StatelessWidget {
  final Function(String) onKey;

  @override
  Widget build(BuildContext context) {
    return Row(
      mainAxisAlignment: MainAxisAlignment.center,
      children: [
        // Arrow keys
        Column(
          children: [
            IconButton(icon: Icon(Icons.arrow_upward), onPressed: () => onKey('up')),
            Row(
              children: [
                IconButton(icon: Icon(Icons.arrow_back), onPressed: () => onKey('left')),
                SizedBox(width: 40),
                IconButton(icon: Icon(Icons.arrow_forward), onPressed: () => onKey('right')),
              ],
            ),
            IconButton(icon: Icon(Icons.arrow_downward), onPressed: () => onKey('down')),
          ],
        ),
        SizedBox(width: 32),
        // Action keys
        Column(
          children: [
            ElevatedButton(onPressed: () => onKey('space'), child: Text('Space')),
            SizedBox(height: 8),
            ElevatedButton(onPressed: () => onKey('enter'), child: Text('Enter')),
          ],
        ),
      ],
    );
  }
}

Implementation Phases

Phase 1: Foundation (Week 1)

TUI Parser (Go):

Basic PTY wrapper for tmux sessions
WebSocket server with session multiplexing
Raw output streaming (no pattern detection yet)
Basic input/keystroke injection

Flutter App:

Phase 2: Pattern Detection (Week 2)

TUI Parser:

ANSI sequence stripping/parsing
Yes/No prompt detection
Simple menu detection (checkbox patterns)
Input waiting detection

Flutter App:

Quick action buttons (Yes/No/Always)
Virtual D-pad widget
Context-aware prompt bar

Phase 3: Advanced Patterns (Week 3)

TUI Parser:

Claude Code wizard detection
Kiro CLI multi-tool approval
Screen state tracking for complex TUIs
Status detection (running/idle/stuck)

Flutter App:

Multi-select menu UI
Wizard/stepper UI
Status indicators
Connection status and reconnection

Phase 4: Polish (Week 4)

Both:

Error handling and edge cases
Performance optimization
Settings (server address, theme)
Notifications for prompt waiting
Testing with Claude Code and Kiro CLI

Future Enhancements

Cloud Relay - For remote access outside LAN
Voice Input - Integration with omnivoice daemon
Multi-Device - Multiple mobile devices connected
Notifications - Push notifications when agent needs input
History - Persistent log storage and search
Themes - Customizable terminal themes

Security Considerations

LAN-Only (Phase 1) - No internet exposure
Optional Auth - Add token-based auth for shared networks
TLS - Use wss:// for encrypted WebSocket (optional)
Input Sanitization - Validate all client input before injection

Open Questions

Should the TUI Parser be a standalone binary or embedded in PlexusOne Desktop Swift app via Go→Swift bridge?
How to handle multiple mobile devices connecting simultaneously?
Should we persist terminal output history on the server for mobile reconnection?
What's the fallback when pattern detection fails? (Default to virtual D-pad?)

FilesExpand file tree

FEAT_MOBILE_PRD.md

Latest commit

History

FEAT_MOBILE_PRD.md

File metadata and controls

PlexusOne PlexusOne Mobile Companion App

Overview

Goals

Non-Goals (Phase 1)

Architecture

WebSocket Protocol

Connection

Message Format

Server → Client Messages

sessions - Session list update

output - Terminal output chunk

prompt - Interactive prompt detected

menu - Scrollable menu detected

wizard - Multi-step wizard detected

status - Session status change

clear - Clear terminal buffer

Client → Server Messages

subscribe - Subscribe to session(s)

unsubscribe - Unsubscribe from session(s)

input - Send text input

key - Send special key

action - Respond to prompt/menu/wizard

TUI Parser Wrapper

Overview

Location

Pattern Detection

Yes/No Prompts

Multi-Select Menus (Claude Code style)

Kiro CLI Tool Approval

Wizard/Questionnaire (Claude Code AskUserQuestion)

Input Waiting

ANSI Handling

Integration with PlexusOne Desktop

Flutter Mobile App

Project Structure

Key Dependencies

Terminal-Style View

Interactive Prompt Bar

Quick Action Buttons

Virtual D-Pad

Implementation Phases

Phase 1: Foundation (Week 1)

Phase 2: Pattern Detection (Week 2)

Phase 3: Advanced Patterns (Week 3)

Phase 4: Polish (Week 4)

Future Enhancements

Security Considerations

Open Questions

`sessions` - Session list update

`output` - Terminal output chunk

`prompt` - Interactive prompt detected

`menu` - Scrollable menu detected

`wizard` - Multi-step wizard detected

`status` - Session status change

`clear` - Clear terminal buffer

`subscribe` - Subscribe to session(s)

`unsubscribe` - Unsubscribe from session(s)

`input` - Send text input

`key` - Send special key

`action` - Respond to prompt/menu/wizard