Implement document upload and management system for AI agents #39

Copilot · 2025-08-27T03:22:17Z

This PR implements a comprehensive document upload and management system that allows AI agents to upload relevant documents to devlog entries and save them to the database.

🎯 Problem Solved

Previously, AI agents had no way to attach supporting documents (code files, logs, screenshots, configuration files, etc.) to devlog entries. This limited their ability to provide comprehensive context and reference materials when documenting work.

🚀 Implementation

Core Infrastructure

New Types: Added DevlogDocument interface and DocumentType enum supporting text, markdown, images, PDFs, code files, JSON, CSV, logs, and config files
Database Entity: Created DevlogDocumentEntity with TypeORM mappings for cross-database compatibility (PostgreSQL, MySQL, SQLite)
Service Layer: Implemented DocumentService with singleton pattern matching existing architecture

API Endpoints

Added RESTful endpoints following existing patterns:

POST   /api/projects/[name]/devlogs/[devlogId]/documents           # Upload document
GET    /api/projects/[name]/devlogs/[devlogId]/documents           # List documents  
GET    /api/projects/[name]/devlogs/[devlogId]/documents/[id]      # Get document
DELETE /api/projects/[name]/devlogs/[devlogId]/documents/[id]      # Delete document

MCP Tools for AI Agents

Added 5 new MCP tools with clear naming:

upload_devlog_document - Upload base64-encoded files with metadata
list_devlog_documents - List documents for a devlog entry
get_devlog_document - Retrieve document details and content
delete_devlog_document - Remove document attachments
search_devlog_documents - Search by filename and content

Key Features

Smart Type Detection: Automatic file type classification based on MIME type and extension, with priority for code files over generic text
Content Extraction: Text content extraction for searchable document types (code, markdown, JSON, etc.)
File Validation: 10MB size limit with proper error handling
Project Isolation: Documents are scoped to projects with proper access control
Base64 Support: Handles binary files through base64 encoding for MCP transport

🧪 Testing

Added comprehensive unit tests for document type detection and content extraction
Tests cover edge cases like code files vs plain text detection
All existing tests continue to pass
Built successfully across all packages

🔄 Integration

Updated DevlogService.get() method to optionally include documents alongside notes
Enhanced DevlogApiClient with document upload methods using FormData
Extended MCP adapter with document operation handlers
Maintains backward compatibility - existing functionality unchanged

📝 Usage Example

AI agents can now upload supporting files:

// Upload a code file
await mcp_upload_devlog_document({
  devlogId: 123,
  filename: "auth.ts", 
  content: "Y29uc3QgYXV0aCA9IC4uLg==", // base64 encoded
  mimeType: "text/plain",
  metadata: { purpose: "implementation reference" }
});

// Search across documents
await mcp_search_devlog_documents({
  query: "authentication",
  limit: 10
});

This enables AI agents to provide richer context by attaching relevant code samples, error logs, screenshots, and configuration files directly to their devlog entries.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

vercel · 2025-08-27T03:22:19Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
devlog-web	Ready	Preview	Comment	Aug 27, 2025 3:48am

Co-authored-by: tikazyq <[email protected]>

Copilot

Pull Request Overview

This PR implements a comprehensive document upload and management system that enables AI agents to attach files to devlog entries. The system supports various file types (text, images, PDFs, code files, etc.) with automatic type detection, content extraction for searchable documents, and full CRUD operations through both REST API endpoints and MCP tools.

Key changes include:

New Document Infrastructure: Added DevlogDocument types, DocumentService singleton, and DevlogDocumentEntity with TypeORM mappings
REST API Endpoints: Implemented document upload, listing, retrieval, and deletion endpoints following existing patterns
MCP Tool Integration: Added 5 new MCP tools for AI agents to manage document attachments with proper validation

Reviewed Changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
packages/mcp/src/tools/index.ts	Exported new document tools and updated tool count to 15
packages/mcp/src/tools/document-tools.ts	Added 5 MCP tools for document operations with clear descriptions
packages/mcp/src/schemas/document-schemas.ts	Defined validation schemas for document operations
packages/mcp/src/handlers/tool-handlers.ts	Added document operation handlers with proper validation
packages/mcp/src/api/devlog-api-client.ts	Extended API client with document upload methods using FormData
packages/mcp/src/adapters/mcp-adapter.ts	Implemented document operation adapters with base64 handling
packages/core/src/utils/id-generator.ts	Added utilities for generating unique document IDs
packages/core/src/types/core.ts	Added `DocumentType` enum and `DevlogDocument` interface
packages/core/src/services/document-service.ts	Implemented core document service with CRUD operations and type detection
packages/core/src/services/devlog-service.ts	Enhanced to optionally include documents when retrieving devlogs
packages/core/src/entities/devlog-document.entity.ts	Created TypeORM entity for document storage with JSON metadata support
apps/web/app/api/projects/[name]/devlogs/[devlogId]/documents/route.ts	Implemented document upload and listing REST endpoints
apps/web/app/api/projects/[name]/devlogs/[devlogId]/documents/[documentId]/route.ts	Implemented document retrieval and deletion REST endpoints

Copilot · 2025-08-28T08:22:48Z

packages/mcp/src/adapters/mcp-adapter.ts

+      // For getDocument, we need to find which devlog contains the document
+      // This is a limitation of the current API design - we'll try a simple approach
+      // by searching through recent devlogs
+      const devlogs = await this.apiClient.listDevlogs({ 


The getDocument and deleteDocument methods search through up to 20 devlogs sequentially to find a document. This creates an O(n) search pattern that could become inefficient with many devlogs. Consider adding a direct document lookup API endpoint or maintaining a document-to-devlog mapping.

Copilot · 2025-08-28T08:22:48Z

packages/mcp/src/adapters/mcp-adapter.ts

+        // Search across all recent devlogs
+        const devlogs = await this.apiClient.listDevlogs({ 
+          page: 1, 
+          limit: 10,
+          sortBy: 'updatedAt',
+          sortOrder: 'desc'
+        });
+
+        for (const devlog of devlogs.items || []) {
+          try {
+            const devlogDocuments = await this.apiClient.listDocuments(devlog.id!);
+
+            const matchingDocs = devlogDocuments.filter((doc: any) =>
+              doc.originalName?.toLowerCase().includes(args.query.toLowerCase()) ||
+              (doc.content && doc.content.toLowerCase().includes(args.query.toLowerCase())) ||
+              doc.filename?.toLowerCase().includes(args.query.toLowerCase())
+            );
+
+            documents.push(...matchingDocs);
+          } catch (err) {
+            // Continue with other devlogs if one fails
+            console.warn(`Failed to search documents in devlog ${devlog.id}:`, err);
+          }
+        }


The searchDocuments method fetches documents from up to 10 devlogs sequentially when searching across all devlogs. This could be slow and resource-intensive. Consider implementing a server-side search endpoint that can efficiently query across all documents.

Suggested change

// Search across all recent devlogs

const devlogs = await this.apiClient.listDevlogs({

page: 1,

limit: 10,

sortBy: 'updatedAt',

sortOrder: 'desc'

});

for (const devlog of devlogs.items || []) {

try {

const devlogDocuments = await this.apiClient.listDocuments(devlog.id!);

const matchingDocs = devlogDocuments.filter((doc: any) =>

doc.originalName?.toLowerCase().includes(args.query.toLowerCase()) ||

(doc.content && doc.content.toLowerCase().includes(args.query.toLowerCase())) ||

doc.filename?.toLowerCase().includes(args.query.toLowerCase())

);

documents.push(...matchingDocs);

} catch (err) {

// Continue with other devlogs if one fails

console.warn(`Failed to search documents in devlog ${devlog.id}:`, err);

}

}

// Search across all devlogs using server-side search

const searchResults = await this.apiClient.searchDocuments(args.query, args.limit);

documents = searchResults || [];

Copilot · 2025-08-28T08:22:49Z

packages/core/src/services/document-service.ts

+    }
+
+    // Code files (check before general text types)
+    const codeExtensions = ['.js', '.ts', '.py', '.java', '.cpp', '.c', '.go', '.rs', '.php', '.rb', '.swift', '.kt'];


The code extension list is hardcoded and may become outdated as new languages emerge. Consider moving this to a configuration file or constant that can be easily maintained and extended.

Copilot · 2025-08-28T08:22:49Z

packages/core/src/services/document-service.ts

+    }
+
+    // Config files (check before general text types)
+    const configExtensions = ['.env', '.conf', '.ini', '.yaml', '.yml', '.toml', '.properties'];


Similar to code extensions, the config extension list is hardcoded. Consider consolidating all file type mappings into a centralized configuration structure for better maintainability.

Copilot · 2025-08-28T08:22:49Z

packages/core/src/entities/devlog-document.entity.ts

+  @Column({
+    type: 'varchar',
+    length: 50,
+    enum: ['text', 'markdown', 'image', 'pdf', 'code', 'json', 'csv', 'log', 'config', 'other'],


The enum values are duplicated between the DocumentType type definition and the database column definition. Consider importing the DocumentType enum values to maintain consistency and avoid duplication.

Copilot · 2025-08-28T08:22:50Z

apps/web/app/api/projects/[name]/devlogs/[devlogId]/documents/route.ts

+    }
+
+    // Validate file size (10MB limit)
+    const maxSize = 10 * 1024 * 1024; // 10MB


The 10MB file size limit is hardcoded and duplicated across multiple locations. Consider defining this as a constant in a shared configuration file to ensure consistency and make it easier to modify.

Initial plan

0ca4415

Copilot AI assigned Copilot and tikazyq Aug 27, 2025

Copilot started work on behalf of tikazyq August 27, 2025 03:22 View session

vercel bot deployed to Preview August 27, 2025 03:22 View deployment

Add core document types, entities, service and web API endpoints

faba3a2

Co-authored-by: tikazyq <[email protected]>

vercel bot deployed to Preview August 27, 2025 03:37 View deployment

Add MCP tools and handlers for document management

ce9de87

Co-authored-by: tikazyq <[email protected]>

vercel bot deployed to Preview August 27, 2025 03:43 View deployment

Update DevlogService for document support and add tests

79f05b4

Co-authored-by: tikazyq <[email protected]>

vercel bot deployed to Preview August 27, 2025 03:48 View deployment

Copilot AI requested a review from tikazyq August 27, 2025 03:49

Copilot finished work on behalf of tikazyq August 27, 2025 03:49

tikazyq approved these changes Aug 28, 2025

View reviewed changes

tikazyq marked this pull request as ready for review August 28, 2025 08:19

Copilot AI review requested due to automatic review settings August 28, 2025 08:19

tikazyq merged commit e3732fd into develop Aug 28, 2025
2 of 4 checks passed

tikazyq deleted the copilot/fix-ba15e10c-9577-41cf-820c-2ebb78495a45 branch August 28, 2025 08:19

Copilot AI reviewed Aug 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement document upload and management system for AI agents #39

Implement document upload and management system for AI agents #39

Uh oh!

Copilot AI commented Aug 27, 2025 •

edited

Loading

Uh oh!

vercel bot commented Aug 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 28, 2025

Uh oh!

Copilot AI Aug 28, 2025

Uh oh!

Copilot AI Aug 28, 2025

Uh oh!

Copilot AI Aug 28, 2025

Uh oh!

Copilot AI Aug 28, 2025

Uh oh!

Copilot AI Aug 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Implement document upload and management system for AI agents #39

Implement document upload and management system for AI agents #39

Uh oh!

Conversation

Copilot AI commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 Problem Solved

🚀 Implementation

Core Infrastructure

API Endpoints

MCP Tools for AI Agents

Key Features

🧪 Testing

🔄 Integration

📝 Usage Example

Uh oh!

vercel bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Aug 27, 2025 •

edited

Loading

vercel bot commented Aug 27, 2025 •

edited

Loading