Skip to content

Commit e4b8399

Browse files
authored
feat: add hypeman cp for file copy to/from running VMs (#45)
* refactor: rename exec-agent to guest-agent and add cp support This commit renames the exec-agent to guest-agent to better reflect its expanding role as a general-purpose guest interaction service. Along with the rename, this adds gRPC support for file copy operations: - Rename lib/exec to lib/guest - Rename lib/system/exec_agent to lib/system/guest_agent - Add CopyToGuest and CopyFromGuest RPCs - Add StatPath RPC for querying guest filesystem metadata - Implement cp handlers with uid/gid preservation support - Update all imports and references The guest-agent continues to listen on vsock port 2222 and handle command execution, with new file copy capabilities. * feat: add WebSocket endpoint for cp operations Add /instances/{id}/cp WebSocket endpoint to enable file copy operations between the local filesystem and running VM instances. Also adds /instances/{id}/stat HTTP endpoint for querying guest path metadata. Features: - Copy files and directories to/from running instances - Streaming file transfer with chunked data - Support for symlinks and directory hierarchies - UID/GID preservation for archive mode - Follow symlinks option * api: add dedicated /instances/{id}/stat endpoint Add a separate HTTP endpoint for querying filesystem path info in guests. This replaces the overloaded 'direction: stat' option in the cp WebSocket endpoint with a cleaner REST API. - Add PathInfo schema with exists, is_dir, is_file, is_symlink, etc. - Add GET /instances/{id}/stat endpoint with path and follow_links params - Add stat method to stainless.yaml for SDK generation The new endpoint: - Uses simple HTTP GET instead of WebSocket overhead - Enables autogenerated SDK methods (client.Instances.Stat()) - Mirrors the guest agent's separate StatPath RPC - Is self-documenting via OpenAPI spec * test: add integration tests for cp operations Add comprehensive integration tests for file copy functionality: - Test copying single files to instance - Test copying directories recursively to instance - Test copying files from instance - Test copying directories from instance - Verify file content, permissions, and metadata preservation Also adds Reset() method to outputBuffer helper used in tests. * fix(cp): preserve directory modification times during copy The mtime was only being applied when currentFile != nil, which is only true for regular files. For directories, currentFile is never set since they are created with os.MkdirAll without opening a file handle. This fix moves the mtime logic outside of the currentFile != nil block so it applies to both files and directories. * chore: remove accidentally committed guest_agent binary Updated .gitignore to cover both guest-agent and guest_agent naming patterns and removed the tracked binary from the repository. * feat(metrics): add cp metrics and OTEL tracing Added metrics for copy operations following the exec metrics pattern: - hypeman_cp_sessions_total (labels: direction, status) - hypeman_cp_duration_seconds - hypeman_cp_bytes_total Also added OTEL span in CpHandler since WebSocket connections bypass the otelchi middleware for HTTP request tracing. * refactor(guest-agent): split main.go into logical files Reorganized the 732-line main.go into a cleaner structure: - main.go: Server setup, listener, gRPC registration (minimal) - exec.go: Exec-related methods (Exec, executeTTY, executeNoTTY, buildEnv) - cp.go: Copy methods (CopyToGuest, CopyFromGuest, helpers) - stat.go: StatPath method This matches the lib/*/README.md pattern used in the server codebase and makes the guest-agent more maintainable as it grows.
1 parent 8328365 commit e4b8399

39 files changed

+4321
-1314
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ lib/vmm/binaries/cloud-hypervisor/*/*/cloud-hypervisor
2020
cloud-hypervisor
2121
cloud-hypervisor/**
2222
lib/system/exec_agent/exec-agent
23+
lib/system/guest_agent/guest-agent
24+
lib/system/guest_agent/guest_agent
2325

2426
# Envoy binaries
2527
lib/ingress/binaries/**

Makefile

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -127,15 +127,23 @@ generate-grpc:
127127
@echo "Generating gRPC code from proto..."
128128
protoc --go_out=. --go_opt=paths=source_relative \
129129
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
130-
lib/exec/exec.proto
130+
lib/guest/guest.proto
131131

132132
# Generate all code
133133
generate-all: oapi-generate generate-vmm-client generate-wire generate-grpc
134134

135135
# Check if CH binaries exist, download if missing
136136
.PHONY: ensure-ch-binaries
137137
ensure-ch-binaries:
138-
@if [ ! -f lib/vmm/binaries/cloud-hypervisor/v48.0/x86_64/cloud-hypervisor ]; then \
138+
@ARCH=$$(uname -m); \
139+
if [ "$$ARCH" = "x86_64" ]; then \
140+
CH_ARCH=x86_64; \
141+
elif [ "$$ARCH" = "aarch64" ] || [ "$$ARCH" = "arm64" ]; then \
142+
CH_ARCH=aarch64; \
143+
else \
144+
echo "Unsupported architecture: $$ARCH"; exit 1; \
145+
fi; \
146+
if [ ! -f lib/vmm/binaries/cloud-hypervisor/v48.0/$$CH_ARCH/cloud-hypervisor ]; then \
139147
echo "Cloud Hypervisor binaries not found, downloading..."; \
140148
$(MAKE) download-ch-binaries; \
141149
fi
@@ -156,27 +164,27 @@ ensure-caddy-binaries:
156164
$(MAKE) build-caddy; \
157165
fi
158166

159-
# Build exec-agent (guest binary) into its own directory for embedding
160-
lib/system/exec_agent/exec-agent: lib/system/exec_agent/main.go
161-
@echo "Building exec-agent..."
162-
cd lib/system/exec_agent && CGO_ENABLED=0 go build -ldflags="-s -w" -o exec-agent .
167+
# Build guest-agent (guest binary) into its own directory for embedding
168+
lib/system/guest_agent/guest-agent: lib/system/guest_agent/main.go
169+
@echo "Building guest-agent..."
170+
cd lib/system/guest_agent && CGO_ENABLED=0 go build -ldflags="-s -w" -o guest-agent .
163171

164172
# Build the binary
165-
build: ensure-ch-binaries ensure-caddy-binaries lib/system/exec_agent/exec-agent | $(BIN_DIR)
173+
build: ensure-ch-binaries ensure-caddy-binaries lib/system/guest_agent/guest-agent | $(BIN_DIR)
166174
go build -tags containers_image_openpgp -o $(BIN_DIR)/hypeman ./cmd/api
167175

168176
# Build all binaries
169177
build-all: build
170178

171179
# Run in development mode with hot reload
172-
dev: ensure-ch-binaries ensure-caddy-binaries lib/system/exec_agent/exec-agent $(AIR)
180+
dev: ensure-ch-binaries ensure-caddy-binaries lib/system/guest_agent/guest-agent $(AIR)
173181
@rm -f ./tmp/main
174182
$(AIR) -c .air.toml
175183

176184
# Run tests (as root for network capabilities, enables caching and parallelism)
177185
# Usage: make test - runs all tests
178186
# make test TEST=TestCreateInstanceWithNetwork - runs specific test
179-
test: ensure-ch-binaries ensure-caddy-binaries lib/system/exec_agent/exec-agent
187+
test: ensure-ch-binaries ensure-caddy-binaries lib/system/guest_agent/guest-agent
180188
@if [ -n "$(TEST)" ]; then \
181189
echo "Running specific test: $(TEST)"; \
182190
sudo env "PATH=$$PATH" "DOCKER_CONFIG=$${DOCKER_CONFIG:-$$HOME/.docker}" go test -tags containers_image_openpgp -run=$(TEST) -v -timeout=180s ./...; \
@@ -194,9 +202,9 @@ clean:
194202
rm -rf $(BIN_DIR)
195203
rm -rf lib/vmm/binaries/cloud-hypervisor/
196204
rm -rf lib/ingress/binaries/
197-
rm -f lib/system/exec_agent/exec-agent
205+
rm -f lib/system/guest_agent/guest-agent
198206

199207
# Prepare for release build (called by GoReleaser)
200208
# Downloads all embedded binaries and builds embedded components
201-
release-prep: download-ch-binaries build-caddy-binaries lib/system/exec_agent/exec-agent
209+
release-prep: download-ch-binaries build-caddy-binaries lib/system/guest_agent/guest-agent
202210
go mod tidy

0 commit comments

Comments
 (0)