Skip to content

Commit 920cfa7

Browse files
MenelionCopilotclaude
authored
Try to reconcile dev with CI and fix tests finally (#32)
* Try to reconcile dev with CI and fix tests finally * Fix CS * Try fixing workflow * More fixes * Meow * Meow2 * Meow3 * Meow4 * Meow5 * meow6 * meow7 * Meow8 * Meow9 * meow11 * Meow12 * meow13 * meow14 * Meow10 * meow15 * meow16 * meow17 * meow18 * meow19 * meow20 * meow21 * meow22 * meow23 * meow23 * meow24 * meow25 * meow26 * meow27 * Fix SFTP test failures in chrooted environments (#33) * Initial plan * Fix SFTP test root path handling for chrooted environments - Handle empty SFTP_TEST_ROOT properly (use relative path instead of /tmp/sharpsync-tests) - When root is empty string (chrooted env), use relative paths for test directories - This fixes "No such file" errors when creating test directories in CI Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> * Improve SFTP chroot detection and path handling - Simplify chroot detection: assume relative paths when RootPath doesn't start with "/" - Fix exception handling to catch SftpPathNotFoundException in addition to permission errors - This should fix issues with chrooted SFTP environments in CI - Separate chroot detection from root path creation logic Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> * Fix whitespace formatting in SftpStorageTests.cs (#34) * Initial plan * Fix whitespace formatting in SftpStorageTests.cs Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> * Fix SFTP integration tests by configuring writable chroot directory (#35) * Initial plan * Fix SFTP test configuration for chrooted environment - Change SFTP container to use command array format for proper parsing - Set SFTP_TEST_ROOT to "upload" directory which is writable by test user - Remove volume mount that was causing permission issues - The atmoz/sftp image chroots users and only specified directories are writable - This fixes "No such file" errors when SFTP tests try to create directories Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> * Fix WebDAV GetRelativePath to handle full URLs from server responses (#36) * Initial plan * Fix WebDavStorage GetRelativePath to correctly strip root path from WebDAV resource URLs Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> * Fix WebDavStorage GetRelativePath to handle full URLs from WebDAV server Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Menelion <595597+Menelion@users.noreply.github.com> * Fix CS * Fix WebDAV CreateDirectoryAsync to ensure root path exists before creating subdirectories CreateDirectoryAsync was missing the call to EnsureRootPathExistsAsync that WriteFileAsync already had. This caused integration tests to fail because each test uses a unique root path (containing a GUID for isolation), and attempting to create subdirectories like "test/subdir" would fail with 409 Conflict when the root path itself didn't exist yet. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Remove garbage --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 0ab18b1 commit 920cfa7

File tree

8 files changed

+581
-353
lines changed

8 files changed

+581
-353
lines changed

‎.github/workflows/dotnet.yml‎

Lines changed: 190 additions & 117 deletions
Original file line numberDiff line numberDiff line change
@@ -1,117 +1,190 @@
1-
# This workflow will build a .NET project
2-
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net
3-
4-
name: .NET
5-
6-
on:
7-
push:
8-
branches: [ "master" ]
9-
pull_request:
10-
branches: [ "master" ]
11-
12-
jobs:
13-
build:
14-
15-
runs-on: ubuntu-latest
16-
17-
services:
18-
sftp:
19-
image: atmoz/sftp:latest
20-
ports:
21-
- 2222:22
22-
options: >-
23-
--health-cmd "pgrep sshd"
24-
--health-interval 10s
25-
--health-timeout 5s
26-
--health-retries 5
27-
env:
28-
SFTP_USERS: testuser:testpass:1001:100:upload
29-
30-
ftp:
31-
image: fauria/vsftpd:latest
32-
ports:
33-
- 21:21
34-
- 21000-21010:21000-21010
35-
options: >-
36-
--health-cmd "pgrep vsftpd"
37-
--health-interval 10s
38-
--health-timeout 5s
39-
--health-retries 5
40-
env:
41-
FTP_USER: testuser
42-
FTP_PASS: testpass
43-
PASV_ADDRESS: localhost
44-
PASV_MIN_PORT: 21000
45-
PASV_MAX_PORT: 21010
46-
47-
localstack:
48-
image: localstack/localstack:latest
49-
ports:
50-
- 4566:4566
51-
options: >-
52-
--health-cmd "curl -f http://localhost:4566/_localstack/health || exit 1"
53-
--health-interval 10s
54-
--health-timeout 5s
55-
--health-retries 5
56-
env:
57-
SERVICES: s3
58-
DEBUG: 0
59-
EDGE_PORT: 4566
60-
61-
62-
steps:
63-
- uses: actions/checkout@v6
64-
65-
- name: Start WebDAV server
66-
run: |
67-
echo "=== Starting WebDAV server ==="
68-
docker run -d --name webdav-server -p 8080:8080 \
69-
eclipse-temurin:11-jre bash -c "
70-
apt-get update && apt-get install -y curl wget &&
71-
wget -O webdav-server.jar https://repo1.maven.org/maven2/io/github/atetzner/webdav-embedded-server/0.2.1/webdav-embedded-server-0.2.1.jar &&
72-
mkdir -p /webdav &&
73-
java -jar webdav-server.jar --port 8080 --directory /webdav
74-
"
75-
76-
- name: Wait for services to be ready
77-
run: |
78-
echo "=== Waiting for services to be ready ==="
79-
sleep 15
80-
echo "=== Testing WebDAV server ==="
81-
curl -f http://localhost:8080/ || echo "WebDAV not ready yet, continuing..."
82-
83-
- name: Setup .NET
84-
uses: actions/setup-dotnet@v5
85-
with:
86-
dotnet-version: 8.0.x
87-
- name: Restore dependencies
88-
run: dotnet restore
89-
- name: Check format
90-
run: dotnet format --verify-no-changes
91-
- name: Build
92-
run: dotnet build --no-restore
93-
- name: Create S3 test bucket
94-
run: |
95-
docker exec $(docker ps -q -f ancestor=localstack/localstack:latest) awslocal s3 mb s3://test-bucket
96-
- name: Test
97-
run: dotnet test --no-build --verbosity normal
98-
env:
99-
SFTP_TEST_HOST: localhost
100-
SFTP_TEST_PORT: 2222
101-
SFTP_TEST_USER: testuser
102-
SFTP_TEST_PASS: testpass
103-
SFTP_TEST_ROOT: "upload"
104-
FTP_TEST_HOST: localhost
105-
FTP_TEST_PORT: 21
106-
FTP_TEST_USER: testuser
107-
FTP_TEST_PASS: testpass
108-
FTP_TEST_ROOT: ""
109-
S3_TEST_BUCKET: test-bucket
110-
S3_TEST_ACCESS_KEY: test
111-
S3_TEST_SECRET_KEY: test
112-
S3_TEST_ENDPOINT: http://localhost:4566
113-
S3_TEST_PREFIX: sharpsync-tests
114-
WEBDAV_TEST_URL: http://localhost:8080/
115-
WEBDAV_TEST_USER: ""
116-
WEBDAV_TEST_PASS: ""
117-
WEBDAV_TEST_ROOT: ""
1+
# This workflow will build a .NET project
2+
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net
3+
4+
name: .NET
5+
6+
on:
7+
push:
8+
branches: [ "master" ]
9+
pull_request:
10+
branches: [ "master" ]
11+
12+
jobs:
13+
build:
14+
15+
runs-on: ubuntu-latest
16+
17+
steps:
18+
- uses: actions/checkout@v6
19+
20+
- name: Cleanup previous test services
21+
run: |
22+
echo "=== Cleaning up any previous test services ==="
23+
docker compose -f docker-compose.test.yml down -v --remove-orphans || true
24+
sudo rm -rf /tmp/localstack || true
25+
echo "=== Removing stale Docker volumes ==="
26+
docker volume rm sharp-sync_sftp-data || true
27+
docker volume rm sharp-sync_ftp-data || true
28+
docker volume rm sharp-sync_webdav-data || true
29+
30+
- name: Start test services
31+
run: |
32+
echo "=== Starting test services with docker-compose ==="
33+
docker compose -f docker-compose.test.yml up -d
34+
35+
- name: Wait for services to be ready
36+
run: |
37+
echo "=== Waiting for services to be healthy ==="
38+
docker compose -f docker-compose.test.yml ps
39+
40+
echo "=== Waiting for SFTP server ==="
41+
for i in {1..18}; do
42+
nc -z localhost 2222 && echo "SFTP ready" && break
43+
echo "SFTP not ready, retrying in 5 seconds... ($i/18)"
44+
sleep 5
45+
done
46+
47+
echo "=== Waiting for FTP server ==="
48+
for i in {1..18}; do
49+
nc -z localhost 21 && echo "FTP ready" && break
50+
echo "FTP not ready, retrying in 5 seconds... ($i/18)"
51+
sleep 5
52+
done
53+
54+
echo "=== Waiting for LocalStack ==="
55+
for i in {1..30}; do
56+
curl -sf http://localhost:4566/_localstack/health && echo "LocalStack ready" && break
57+
echo "LocalStack not ready, retrying in 10 seconds... ($i/30)"
58+
sleep 10
59+
done
60+
61+
echo "=== Waiting for WebDAV server ==="
62+
WEBDAV_BASIC_READY=false
63+
for i in {1..30}; do
64+
if curl -sf -u testuser:testpass http://localhost:8080/ > /dev/null 2>&1; then
65+
echo "WebDAV responding to basic requests"
66+
WEBDAV_BASIC_READY=true
67+
break
68+
fi
69+
echo "WebDAV not responding, retrying in 5 seconds... ($i/30)"
70+
sleep 5
71+
done
72+
73+
if [ "$WEBDAV_BASIC_READY" = "false" ]; then
74+
echo "ERROR: WebDAV server not responding after 30 attempts"
75+
docker compose -f docker-compose.test.yml logs webdav
76+
exit 1
77+
fi
78+
79+
echo "=== Checking WebDAV full functionality (MKCOL) ==="
80+
WEBDAV_FULLY_READY=false
81+
for i in {1..40}; do
82+
# Show verbose output for debugging
83+
echo "Attempt $i: Testing MKCOL operation..."
84+
MKCOL_RESULT=$(curl -s -w "%{http_code}" -u testuser:testpass -X MKCOL http://localhost:8080/_health-check-dir/ -o /dev/null 2>&1)
85+
echo "MKCOL response code: $MKCOL_RESULT"
86+
87+
if [ "$MKCOL_RESULT" = "201" ] || [ "$MKCOL_RESULT" = "405" ]; then
88+
echo "WebDAV is fully operational (MKCOL working)"
89+
# Cleanup test directory
90+
curl -sf -u testuser:testpass -X DELETE http://localhost:8080/_health-check-dir/ > /dev/null 2>&1 || true
91+
WEBDAV_FULLY_READY=true
92+
break
93+
fi
94+
95+
# Show container status and logs on failures
96+
if [ $((i % 5)) -eq 0 ]; then
97+
echo "--- WebDAV container status ---"
98+
docker compose -f docker-compose.test.yml ps webdav
99+
echo "--- Recent WebDAV logs ---"
100+
docker compose -f docker-compose.test.yml logs --tail=10 webdav
101+
fi
102+
103+
echo "MKCOL not working yet, retrying in 5 seconds..."
104+
sleep 5
105+
done
106+
107+
if [ "$WEBDAV_FULLY_READY" = "false" ]; then
108+
echo "ERROR: WebDAV MKCOL not working after 40 attempts"
109+
echo "=== Full WebDAV logs ==="
110+
docker compose -f docker-compose.test.yml logs webdav
111+
exit 1
112+
fi
113+
114+
echo "=== Final service status ==="
115+
docker compose -f docker-compose.test.yml ps
116+
117+
- name: Setup .NET
118+
uses: actions/setup-dotnet@v5
119+
with:
120+
dotnet-version: 8.0.x
121+
- name: Restore dependencies
122+
run: dotnet restore
123+
- name: Check format
124+
run: dotnet format --verify-no-changes
125+
- name: Build
126+
run: dotnet build --no-restore
127+
- name: Create S3 test bucket
128+
run: |
129+
docker exec sharp-sync-localstack-1 awslocal s3 mb s3://test-bucket
130+
131+
- name: Debug WebDAV setup
132+
run: |
133+
echo "=== WebDAV Container Status ==="
134+
docker compose -f docker-compose.test.yml ps webdav
135+
echo ""
136+
echo "=== WebDAV Container Logs ==="
137+
docker compose -f docker-compose.test.yml logs webdav
138+
echo ""
139+
echo "=== Testing WebDAV Operations ==="
140+
echo "PROPFIND (list root):"
141+
curl -s -w "\nHTTP Status: %{http_code}\n" -u testuser:testpass -X PROPFIND http://localhost:8080/ -H "Depth: 1" | head -30
142+
echo ""
143+
echo "PUT (write test):"
144+
echo "test content" | curl -s -w "\nHTTP Status: %{http_code}\n" -u testuser:testpass -X PUT http://localhost:8080/_debug-test.txt -d @-
145+
echo ""
146+
echo "DELETE (cleanup):"
147+
curl -s -w "\nHTTP Status: %{http_code}\n" -u testuser:testpass -X DELETE http://localhost:8080/_debug-test.txt
148+
149+
- name: Prepare WebDAV test root
150+
run: |
151+
echo "=== Creating WebDAV test root directory ==="
152+
# Delete existing test root if present
153+
curl -sf -u testuser:testpass -X DELETE http://localhost:8080/ci-root/ --output /dev/null 2>&1 || true
154+
# Create fresh test root
155+
curl -sf -u testuser:testpass -X MKCOL http://localhost:8080/ci-root/
156+
echo "WebDAV test root created successfully"
157+
158+
- name: Test
159+
run: dotnet test --no-build --verbosity normal
160+
env:
161+
SFTP_TEST_HOST: localhost
162+
SFTP_TEST_PORT: 2222
163+
SFTP_TEST_USER: testuser
164+
SFTP_TEST_PASS: testpass
165+
SFTP_TEST_ROOT: upload
166+
FTP_TEST_HOST: localhost
167+
FTP_TEST_PORT: 21
168+
FTP_TEST_USER: testuser
169+
FTP_TEST_PASS: testpass
170+
FTP_TEST_ROOT: ""
171+
S3_TEST_BUCKET: test-bucket
172+
S3_TEST_ACCESS_KEY: test
173+
S3_TEST_SECRET_KEY: test
174+
S3_TEST_ENDPOINT: http://localhost:4566
175+
S3_TEST_PREFIX: sharpsync-tests
176+
WEBDAV_TEST_URL: http://localhost:8080/
177+
WEBDAV_TEST_USER: testuser
178+
WEBDAV_TEST_PASS: testpass
179+
WEBDAV_TEST_ROOT: "ci-root"
180+
181+
- name: Dump container logs
182+
if: failure()
183+
run: |
184+
echo "=== Container logs for debugging ==="
185+
docker compose -f docker-compose.test.yml logs
186+
187+
- name: Stop test services
188+
if: always()
189+
run: |
190+
docker compose -f docker-compose.test.yml down -v --remove-orphans || true

0 commit comments

Comments
 (0)