Skip to content

Add Collaborative Chats#1665

Draft
Josh-XT wants to merge 63 commits intomainfrom
group-chats
Draft

Add Collaborative Chats#1665
Josh-XT wants to merge 63 commits intomainfrom
group-chats

Conversation

@Josh-XT
Copy link
Owner

@Josh-XT Josh-XT commented Feb 12, 2026

Add Collaborative Chats

Summary

Adds full group chat backend infrastructure to AGiXT: channels, threads, participants, reactions, pinning, presence, @mention routing, message search, and comprehensive database migrations. Transforms the conversation system from single-user ownership to multi-user participation with role-based access control.

New Database Models & Schema

New Tables

  • MessageReaction — Emoji reactions on messages (message_id, user_id, emoji, created_at). Unique index on (message_id, user_id, emoji) to prevent duplicates.
  • ConversationParticipant — Tracks users and agents in group conversations. Fields: conversation_id, user_id, agent_id, participant_type (user/agent), role (owner/admin/member/observer), joined_at, last_read_at, status (active/left/removed).

New Columns on Existing Tables

Table New Columns
Company icon_url (server avatar)
User avatar_url, last_seen (presence heartbeat), status_text, status_mode (online/away/dnd/invisible)
Conversation conversation_type (private/group/dm/thread), company_id (FK → Company), parent_id (FK → self, for threads), parent_message_id (FK → Message, thread origin), category (channel grouping), invite_only, description (channel topic)
Message sender_user_id (FK → User), pinned, pinned_at, pinned_by (FK → User)

Migrations (6 total)

  1. migrate_group_chat_tables() — Adds all group chat columns + creates conversation_participant table
  2. migrate_message_reaction_table() — Creates message_reaction table with indexes
  3. migrate_message_pinning() — Adds pinned, pinned_at, pinned_by to Message
  4. migrate_performance_indexes() — Composite indexes on message(conversation_id, timestamp), message(conversation_id, notify), conversation(user_id), conversation(name, user_id), conversation(company_id), conversation_participant(conversation_id, status), user_preferences(user_id, pref_key)
  5. migrate_extract_data_urls_from_messages() — One-time extraction of inline base64 data URLs from existing messages into workspace files (prevents 10MB+ base64 video blobs in DB/DOM)
  6. migrate_backfill_channel_participants() — Backfills missing ConversationParticipant records for company members added before channels existed

New API Endpoints

Group Chat Endpoints

Method Path Description
POST /v1/conversation/group Create group conversation or thread
GET /v1/company/{company_id}/conversations Get group conversations for a company
GET /v1/conversation/{id}/participants Get conversation participants
POST /v1/conversation/{id}/participants Add participant
PATCH /v1/conversation/{id}/participants/{pid} Update participant role
DELETE /v1/conversation/{id}/participants/{pid} Remove participant
POST /v1/conversation/{id}/leave Leave conversation
POST /v1/conversation/{id}/read Mark conversation as read (last_read_at)
GET /v1/conversation/{id}/threads Get threads for a channel
POST /v1/conversation/{id}/threads Create thread from message
PATCH /v1/conversation/{id}/channel Update channel properties

Message Feature Endpoints

Method Path Description
POST /v1/conversation/{id}/message/{mid}/reactions Add/toggle reaction
GET /v1/conversation/{id}/message/{mid}/reactions Get reactions
DELETE /v1/conversation/{id}/message/{mid}/reactions/{emoji} Remove reaction
PUT /v1/conversation/{id}/message/{mid}/pin Toggle pin message
GET /v1/conversation/{id}/pins Get pinned messages
POST /v1/conversations/search Search messages across conversations

Auth & User Endpoints

Method Path Description
POST /v1/user/presence Update presence heartbeat + status
GET /v1/user/status Get user status
GET /v1/companies/{company_id}/members List all company members

New API Models

Request Models

  • CreateGroupConversationModel — name, company_id, type, agents, parent_id, parent_message_id, category, invite_only
  • AddParticipantModel — user_id, agent_id, participant_type, role
  • UpdateParticipantRoleModel — role
  • UpdateChannelModel — category, name, description
  • AddReactionModel — emoji

Response Models

  • ReactionResponse, MessageReactionsResponse
  • ParticipantResponse, GroupConversationResponse, ThreadResponse, ThreadListResponse, GroupConversationListResponse
  • Updated: UserResponse (+avatar_url), CompanyResponse (+icon_url), UserInfo (+username), UpdateCompanyInput (+icon_url)

Core Logic Changes

Conversations.py (+1,753 lines)

New methods:

  • create_group_conversation() — Creates group/dm/thread conversations, auto-adds company members as participants (unless invite-only), adds creator as owner
  • can_speak() — Permission check: observer-role participants cannot send messages
  • add_participant(), remove_participant(), get_participants(), update_participant_role()
  • update_last_read() — Updates last_read_at for unread tracking
  • get_group_conversations_for_company() — Gets all group channels a user participates in
  • get_threads(), get_thread_count() — Thread listing for channels
  • toggle_pin_message(), get_pinned_messages() — Message pinning
  • search_messages() — Full-text search across accessible conversations with type/company filters
  • extract_data_urls_to_workspace() — Extracts base64 data URLs from message content, saves as files, replaces with /outputs/ URLs

Key behavior changes:

  • Multi-user access — All lookup functions (get_conversation_id_by_name, get_conversation, log_interaction, etc.) now check company membership and participant records, so users can access group conversations they didn't create
  • Unread tracking — Shifted from per-message notify flag to last_read_at-based tracking via ConversationParticipant
  • get_conversations() — Includes DMs where user is a participant (not just owner), excludes group/thread types, shows display_name for DMs, adds conversation_type and parent_id
  • get_conversation() — Pre-fetches sender user info and reactions in batch (eliminates N+1 queries), single timezone lookup instead of per-message
  • log_interaction() — Accepts sender_user_id, auto-extracts base64 data URLs before storage
  • delete_conversation() — Cascading deletion through all related tables including reactions, participants, shares, and child threads

WebSocket System

  • Per-connection broadcast tracking — Changed from global broadcasted_message_ids to per-connection tracking via _connection_broadcasted_ids. Prevents race conditions between clients.
  • Typing indicatorsbroadcast_typing_event() broadcasts to all connections except sender. WebSocket handles incoming {"type": "typing"} messages.
  • Participant notificationnotify_conversation_participants_message_added() notifies ALL participants via user-level WebSocket.
  • Message broadcasts — Includes sender user info (avatar, name) in broadcast payloads.
  • Audio transcription — Background task _transcribe_channel_audio() auto-transcribes audio file attachments and updates messages.

@Mention Agent Routing (Completions.py +225 lines)

  • parse_agent_mentions() — Parses @AgentName and @"Agent Name" from messages
  • Case-insensitive, longest-match-wins resolution
  • Applied to /v1/chat/completions and MCP chat completion endpoints
  • Validates agent belongs to conversation's company before routing
  • Strips mention from content and overrides prompt.model to route to mentioned agent

Auth & User Management (MagicalAuth.py +245 lines)

  • add_user_to_company_channels() — Auto-adds users to all non-invite-only group channels when joining a company
  • update_presence() — Heartbeat endpoint updating last_seen, status_text, status_mode
  • get_user_status() — Returns current presence info
  • Username validation: uniqueness, length (3-32 chars), character restrictions
  • Login by username in addition to email
  • Company icon_url and user avatar_url, last_seen, status_text included in all response serializations

File Serving Changes (app.py)

  • Participant-based file accessserve_file checks ConversationParticipant in addition to conversation ownership, so group chat members can access shared files
  • MIME type fix.webm files with "recording" in the filename served as audio/webm instead of video/webm (voice recordings are audio-only)

Bug Fixes

  • Fixed regex catastrophic backtracking in URL detection (conversation content processing)
  • Fixed double-hashing of file URLs causing 404s on uploaded media
  • Fixed workspace directory path resolution for file storage
  • Fixed filename collisions on concurrent uploads (added UUID prefix)
  • Fixed sender_user_id not being passed through conversation endpoints
  • Fixed file serving for group chat participants who aren't conversation owners
  • Fixed audio .webm MIME type causing video player errors on voice recordings

Josh-XT and others added 18 commits February 9, 2026 20:43
Backend changes:
- DB models: Company.icon_url, User.avatar_url, Conversation (conversation_type,
  company_id, parent_id, parent_message_id), Message.sender_user_id,
  ConversationParticipant table
- Migrations in DB.py and run-local.py for new columns/tables
- Group conversation CRUD endpoints (create, list by company, threads)
- Thread creation/listing with message_count and last_message_at
- @mention agent routing in chat completions (both main and MCP)
- sender_user_id tracking on user messages with sender info in responses
- Fixed group conversation access: UserCompany-based company membership
  fallback in get_conversation_name_by_id, get_conversation_id_by_name,
  and get_conversation() for group/dm/thread types
- Fixed duplicate conversation creation: create_if_missing=False in
  Conversations.__init__ for group/thread creation endpoints
- Company icon_url and User avatar_url endpoints
- Pydantic models for group conversations, threads
- WebSocket get_conversation_changes includes sender user info
- Add category column to Conversation model with migration
- Add UpdateChannelModel for PATCH endpoint
- Add PATCH /v1/conversation/{id}/channel endpoint for updating channel name/category
- Pass category param through create_group_conversation
- Include category field in group conversations response
- New MessageReaction DB model with message_id, user_id, emoji fields
- Migration for message_reaction table with unique constraint
- API endpoints: POST/GET/DELETE reactions with toggle behavior
- Reactions included in conversation history data with user details
- Pydantic models for reaction request/response
@Josh-XT Josh-XT marked this pull request as draft February 12, 2026 03:05
Add defense-in-depth guard to /v1/chat/completions and
/v1/mcp/chat/completions endpoints. If the conversation is a DM
with no agent participants, or a thread whose parent DM has no agent
participants, return 400 instead of triggering an agent response.
Also initialize conversation_id = None before the UUID check to
prevent UnboundLocalError when conversation_name is '-'.
… ciphertext

When decrypt_config_value failed (e.g. AGIXT_API_KEY changed), it returned the
raw Fernet-encrypted string (gAAAAABp...) instead of empty string. This caused:

1. STRIPE_API_KEY to leak encrypted ciphertext to Stripe API, producing 500
   errors with 'Invalid API Key provided: gAAAAABp...'
2. stripe_configured to evaluate as True (non-empty string), enabling the
   paywall even though Stripe isn't actually usable
3. New users created with is_active=False and no trial credits if their
   domain was already used

Now returns empty string on failure (matching the original comment intention)
and logs a warning about the encryption key mismatch.
The default limit of 100 could cause message loss on WebSocket reconnection
when a conversation had more than 100 messages. The REST API still uses
limit=100 for pagination, but the WebSocket needs all messages to maintain
accurate state tracking in previous_message_ids.
The query used .order_by(timestamp.asc()).limit(100) which returned the
OLDEST 100 messages. On channels with >100 messages, SWR would show
yesterday's messages for 2-3 seconds until WebSocket connected and
delivered the full history.

Changed to .order_by(timestamp.desc()).limit().reversed() so the API
always returns the most recent messages within the limit.
- Add total message count query to get_conversation() in Conversations.py
- Return total/page/limit in conversation response dict
- Update ConversationHistoryResponse model with Optional total/page/limit fields
- v1 conversation endpoint now passes pagination metadata to response
Thread member lists were only showing the creator and agents because
participants were only copied from the parent channel at thread creation time.
Members who joined the channel later (or threads created before the inheritance
code) would have incomplete participant lists.

Modified get_participants() to detect thread conversations and dynamically
sync missing parent channel participants into the thread. When a thread's
participant list is fetched, any active parent channel members not yet in the
thread are added as 'member' role participants. This is persisted to the DB
so subsequent fetches don't need to re-sync.

This means:
- Existing threads with incomplete participants self-heal on first access
- New channel members automatically appear in all channel threads
- Thread-specific roles (owner) are preserved
…media

- Add /outputs/{agent_id}/{conversation_id}/thumb/{filename} endpoint that
  generates JPEG thumbnails via ffmpeg, cached on disk next to originals
- Add _get_cache_headers() helper: media files get 24hr immutable cache,
  other files retain no-cache behavior
- Thumbnails use scale=640:-2 at JPEG quality 5 (~75%) with fallback
  for very short videos
- Replaces client-side canvas capture which failed silently due to CORS
- Skip recording*.webm files early (audio-only, no video stream)
- Check content_type for audio/* before attempting ffmpeg
- Return 404 instead of 500 when thumbnail generation fails
- Log ffmpeg stderr for debugging failed extractions
last_read_at was returned as naive UTC str() (e.g. '2026-02-13 19:10:37')
while message timestamps used ISO 8601 with timezone offset (e.g.
'2026-02-13T14:10:37-05:00'). String comparison always evaluated messages
as 'newer' because 'T' > ' ' in ASCII.

- Pass last_read_at and joined_at through convert_time() in get_participants()
  so they match message timestamp format (ISO 8601 with user timezone)
- Frontend now uses Date-based numeric comparison as an additional safety net
…read

Two changes to cut DM/channel loading time:

1. WebSocket initial_data: read client-sent ?limit= query param instead of
   hardcoding limit=1000. Clamped to 1-200, defaults to 50. Processing
   1000 messages through timezone conversion, sender lookups, and reaction
   assembly was the primary cause of 5+ second DM load times.

2. Notification mark-read (UPDATE Message SET notify=False): moved AFTER
   the message query instead of before it. The UPDATE+COMMIT was blocking
   the read path — users had to wait for the write to complete before
   seeing any messages. Now messages are returned first, then notifications
   are marked read.
When conversation_id is available (from the v1 endpoint), use the same
access-check pattern as get_conversation: owner → company member →
participant. This replaces the fragile name-based lookup that silently
failed for shared DM conversations where the current user isn't the
conversation owner.

The name-based fallback is preserved for the legacy /api/ endpoint.
…propagation

- Add 'locked' column to Conversation model with migration
- Add PUT /v1/conversation/{id}/lock endpoint for lock/unlock
- Update can_speak() to block non-admin/owner messages in locked threads
- Add locked field to get_threads() and get_conversations_with_detail() responses
- Add ThreadResponse.locked field to Models.py
- Parse <@userid> mentions and [uid:userId] reply targets in messages
- Send targeted 'mention' and 'reply' WebSocket notifications to mentioned/replied users
- Avoid double-notifying users who are both mentioned and replied to
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant