file i/o endpoints

[rgarcia]

• read/write individual files
• list / create directories
• move files
• set file permissions
• watch for changes

idea is to expose most if not all of these in the main API

[mesa-dot-dev]

Mesa Description

TL;DR

Added comprehensive file system I/O and watch capabilities to the API.

Why we made these changes

To expose direct file system interaction, including read/write, directory management, and real-time file change monitoring, through the main API.

What changed?

• server/cmd/api/api/fs.go: Introduced new API endpoints for file and directory operations (read, write, create, delete, list, move, permissions) and real-time file system event watching using fsnotify.
• server/cmd/api/api/api.go: Enhanced ApiService to manage active filesystem watches with concurrency control.
• server/cmd/api/api/fs_test.go: Added extensive unit tests for the new file system management API endpoints.
• server/openapi.yaml: Updated OpenAPI specification to include the newly exposed file I/O and watch endpoints.
• server/go.mod, server/go.sum: Updated Go module dependencies to support the new functionality.

[mesa-dot-dev]

What Changed

This PR introduces a new set of file system I/O endpoints under the /fs/ path. Key features include:

• Basic file and directory operations: read, write, create, delete, list, move, and set permissions.
• A real-time file system watch capability using Server-Sent Events (SSE) to stream changes (create, write, delete, rename) for a given path recursively. This is implemented in server/cmd/api/api/fs.go using the new fsnotify dependency.
• The API service (ApiService) is now equipped to manage these watches.

All new endpoints are documented in server/openapi.yaml and are accompanied by a comprehensive test suite in server/cmd/api/api/fs_test.go.

Risks / Concerns

This is a well-structured and comprehensive PR. The new functionality is thoroughly tested and the API contract is clearly defined in the OpenAPI spec. I don't see any immediate risks. Excellent work!

^{6 files reviewed | 0 comments | Review on Mesa | Edit Reviewer Settings}

[Sayan-]

Overall API + flows lgtm! Some errors I think we should fix but nothing super deep.

The main question I have is whether we should configure basic allow list / deny list shaped configuration so folks don't read/mv/delete things that could cause serious issues. I don't think we can prevent event failure mode but I think we could broadly guard against the cases we care about. What do you think?

[Sayan-]

🔥

[cursor]

Bug: Invalid URLs Cause HTTP Request Failures

The _fetch_remote_downloads and _watch_filesystem functions construct invalid filesystem API URLs (e.g., https://None:444) when the parsed CDP URL lacks a hostname. This causes subsequent HTTP requests to fail.

Additional Locations (1)

• shared/cdp-test/main.py#L375-L377

[Sayan-]

Confirming my allow list / deny list isn't blocking for this pr! Let's go 🚀 !