Skip to content

Files are removed from s3 during syncΒ #9

@mefju

Description

@mefju

I have directory with 5 files:

ls -al tests
total 28
drwxrwxr-x 2 user user 4096 sty  5 13:43 .
drwxrwxr-x 4 user user 4096 sty  5 14:00 ..
-rw-rw-r-- 1 user user   36 sty  5 13:56 1
-rw-rw-r-- 1 user user   36 sty  5 13:43 2
-rw-rw-r-- 1 user user   36 sty  5 13:43 3
-rw-rw-r-- 1 user user   36 sty  5 13:43 4
-rw-rw-r-- 1 user user   36 sty  5 13:43 5

when I run sync (AWS_SECRET_KEY, AWS_ACCESS_KEY configured), files are copied to bucket::

./s3-cli --skip-existing -v sync tests s3://my_bucket_name/
Copy file://tests/2 -> s3://my_bucket_name/tests/2
Copy file://tests/1 -> s3://my_bucket_name/tests/1
0 B /Copy file://tests/3 -> s3://my_bucket_name/tests/3
Copy file://tests/4 -> s3://my_bucket_name/tests/4
0 B / 16 files to consider - 180 bytes
0 B / 180 B (0.0%)   0 B/secRemove s3://my_bucket_name/tests/
Copy file://tests/5 -> s3://my_bucket_name/tests/5
180 B / 180 B (100.0%)   86 B/sec
./s3-cli -v ls s3://my_bucket_name/tests/
2018-01-05 15:43        36   s3://my_bucket_name/tests/1
2018-01-05 15:43        36   s3://my_bucket_name/tests/2
2018-01-05 15:43        36   s3://my_bucket_name/tests/3
2018-01-05 15:43        36   s3://my_bucket_name/tests/4
2018-01-05 15:43        36   s3://my_bucket_name/tests/5

When I try sync second time (without changing anything in tests directory or bucket), all files are removed from bucked:

./s3-cli --skip-existing -v sync tests s3://my_bucket_name/
Copy file://tests/4 -> s3://my_bucket_name/tests/4
Copy file://tests/3 -> s3://my_bucket_name/tests/3
0 B Copy file://tests/5 -> s3://my_bucket_name/tests/5
0 B / 108 BCopy file://tests/1 -> s3://my_bucket_name/tests/1
0 B / 180 B (0.0%)   0 B/sec10 files to consider - 180 bytes
Remove s3://my_bucket_name/tests/5
Remove s3://my_bucket_name/tests/1
Remove s3://my_bucket_name/tests/2
Remove s3://my_bucket_name/tests/3
Remove s3://my_bucket_name/tests/4
Copy file://tests/2 -> s3://my_bucket_name/tests/2
180 B / 180 B (100.0%)   144 B/sec

if between first and second sync I add something to directory, files in bucket are randomly deleted

./s3-cli -v ls s3://my_bucket_name/tests/
[no output] - files deleted from bucket
Ubuntu 16.04.3 LTS
s3-cli version 0.2.0
go version go1.6.2 linux/amd64

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions