Skip to content

Commit 3e7de74

Browse files
committed
feat: impliment some basic s3 operations of s6cmd
Signed-off-by: LinPr <[email protected]>
1 parent 0447a49 commit 3e7de74

File tree

32 files changed

+2886
-2
lines changed

32 files changed

+2886
-2
lines changed

.github/workflows/release.yml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v*'
7+
branches:
8+
- master
9+
10+
jobs:
11+
release:
12+
permissions:
13+
contents: write
14+
15+
runs-on: ubuntu-latest
16+
if: startsWith(github.ref, 'refs/tags/')
17+
18+
steps:
19+
- name: Checkout code
20+
uses: actions/checkout@v5
21+
with:
22+
fetch-depth: 0
23+
24+
- name: Set up Go
25+
uses: actions/setup-go@v6
26+
with:
27+
go-version: '1.25.0'
28+
- run: go version
29+
30+
- name: Get version
31+
id: get_version
32+
run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
33+
34+
- name: Build for Linux AMD64
35+
run: |
36+
GOOS=linux GOARCH=amd64 CGO_ENABLE=0 go build -ldflags "-s -w" -o s6cmd-linux-amd64 .
37+
38+
- name: Build for Linux ARM64
39+
run: |
40+
GOOS=linux GOARCH=arm64 CGO_ENABLE=0 go build -ldflags "-s -w" -o s6cmd-linux-arm64 .
41+
42+
- name: Build for Windows AMD64
43+
run: |
44+
GOOS=windows GOARCH=amd64 CGO_ENABLE=0 go build -ldflags "-s -w" -o s6cmd-windows-amd64.exe .
45+
46+
- name: Build for Windows ARM64
47+
run: |
48+
GOOS=windows GOARCH=arm64 CGO_ENABLE=0 go build -ldflags "-s -w" -o s6cmd-windows-arm64.exe .
49+
50+
- name: Build for macOS AMD64
51+
run: |
52+
GOOS=darwin GOARCH=amd64 CGO_ENABLE=0 go build -ldflags "-s -w" -o s6cmd-darwin-amd64 .
53+
54+
- name: Build for macOS ARM64
55+
run: |
56+
GOOS=darwin GOARCH=arm64 CGO_ENABLE=0 go build -ldflags "-s -w" -o s6cmd-darwin-arm64 .
57+
58+
- name: Create checksums
59+
run: |
60+
sha256sum s6cmd-* > checksums.txt
61+
62+
- name: Create Release
63+
uses: softprops/action-gh-release@v1
64+
with:
65+
tag_name: ${{ steps.get_version.outputs.VERSION }}
66+
name: Release ${{ steps.get_version.outputs.VERSION }}
67+
draft: false
68+
prerelease: false
69+
generate_release_notes: true
70+
files: |
71+
s6cmd-linux-amd64
72+
s6cmd-linux-arm64
73+
s6cmd-windows-amd64.exe
74+
s6cmd-windows-arm64.exe
75+
s6cmd-darwin-amd64
76+
s6cmd-darwin-arm64
77+
checksums.txt
78+
env:
79+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,3 +30,7 @@ go.work.sum
3030
# Editor/IDE
3131
# .idea/
3232
# .vscode/
33+
34+
s6cmd
35+
Makefile
36+
vendor/

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@
186186
same "printed page" as the copyright notice for easier
187187
identification within third-party archives.
188188

189-
Copyright [yyyy] [name of copyright owner]
189+
Copyright 2025 LinPr
190190

191191
Licensed under the Apache License, Version 2.0 (the "License");
192192
you may not use this file except in compliance with the License.

README.md

Lines changed: 150 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,150 @@
1-
# s6cmd
1+
# s6cmd
2+
3+
[![Release](https://img.shields.io/github/v/release/LinPr/s6cmd)](https://github.com/LinPr/s6cmd/releases)
4+
[![Go Version](https://img.shields.io/github/go-mod/go-version/LinPr/s6cmd)](https://golang.org/)
5+
[![License](https://img.shields.io/github/license/LinPr/s6cmd)](LICENSE)
6+
7+
*s6cmd is stiall under developing, please do not use it in your production environment.*
8+
9+
s6cmd is a command-line tool for Amazon S3, using [aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2), since the aws has announced the [aws-sdk-go](https://github.com/aws/aws-sdk-go) is deprecated. It's inspired by the popular [s3cmd](https://s3tools.org/s3cmd) tool and aims to provide similar functionality with improved performance and modern Go architecture.
10+
11+
## Features
12+
13+
s6cmd currently supports the following S3 operations:
14+
15+
- **Bucket Operations**
16+
- `mb` - Make bucket (create S3 bucket)
17+
- `rb` - Remove bucket (delete S3 bucket)
18+
- `ls` - List buckets and objects
19+
20+
- **Object Operations**
21+
- `put` - Upload files to S3
22+
- `get` - Download files from S3
23+
- `cp` - Copy objects within S3
24+
- `mv` - Move/rename objects in S3
25+
- `rm` - Remove objects from S3
26+
- `stat` - Display object metadata
27+
- `du` - Display disk usage for objects
28+
29+
30+
31+
## Installation
32+
33+
### Download Pre-built Binaries
34+
35+
Download the latest release for your platform from the [releases page](https://github.com/LinPr/s6cmd/releases):
36+
37+
```bash
38+
# Linux AMD64
39+
wget https://github.com/LinPr/s6cmd/releases/latest/download/s6cmd-linux-amd64
40+
chmod +x s6cmd-linux-amd64
41+
sudo mv s6cmd-linux-amd64 /usr/local/bin/s6cmd
42+
43+
# macOS ARM64 (Apple Silicon)
44+
wget https://github.com/LinPr/s6cmd/releases/latest/download/s6cmd-darwin-arm64
45+
chmod +x s6cmd-darwin-arm64
46+
sudo mv s6cmd-darwin-arm64 /usr/local/bin/s6cmd
47+
```
48+
49+
### Build from Source
50+
51+
```bash
52+
git clone https://github.com/LinPr/s6cmd.git
53+
cd s6cmd
54+
go build -o s6cmd .
55+
```
56+
57+
## Configuration
58+
59+
s6cmd uses AWS credentials and configuration, similar to the AWS CLI. You can configure it using:
60+
61+
### Environment Variables
62+
```bash
63+
export export AWS_ENDPOINT_URL_S3=your-object-storage-service-endpoint
64+
export AWS_ACCESS_KEY_ID=your-access-key
65+
export AWS_SECRET_ACCESS_KEY=your-secret-key
66+
export AWS_REGION=your-object-storage-region
67+
```
68+
69+
### AWS Credentials File
70+
```bash
71+
aws configure
72+
```
73+
74+
### Configuration File
75+
Create a configuration file at `config/s6cmd.yaml`:
76+
77+
78+
79+
## Usage
80+
81+
### Basic Examples
82+
83+
```bash
84+
# List all buckets
85+
s6cmd ls
86+
87+
# List objects in a bucket
88+
s6cmd ls s3://my-bucket/
89+
90+
# Upload a file
91+
s6cmd put local-file.txt s3://my-bucket/remote-file.txt
92+
93+
# Download a file
94+
s6cmd get s3://my-bucket/remote-file.txt local-file.txt
95+
96+
# Copy objects within S3
97+
s6cmd cp s3://source-bucket/file.txt s3://dest-bucket/file.txt
98+
99+
# Sync local directory with S3
100+
s6cmd sync ./local-dir/ s3://my-bucket/prefix/
101+
102+
# Display bucket structure
103+
s6cmd tree s3://my-bucket/
104+
105+
# Create a new bucket
106+
s6cmd mb s3://my-new-bucket
107+
108+
# Remove objects
109+
s6cmd rm s3://my-bucket/file.txt
110+
111+
# Show object statistics
112+
s6cmd stat s3://my-bucket/file.txt
113+
```
114+
115+
### Global Flags
116+
117+
```bash
118+
# Dry run mode (show what would be done without executing)
119+
s6cmd put -n local-file.txt s3://my-bucket/file.txt
120+
121+
# Get help for any command
122+
s6cmd help
123+
s6cmd put --help
124+
```
125+
126+
## Architecture
127+
128+
s6cmd is built using:
129+
130+
- **[AWS SDK for Go v2](https://github.com/aws/aws-sdk-go-v2)** - Modern, efficient AWS SDK
131+
- **[Cobra](https://github.com/spf13/cobra)** - CLI framework for Go
132+
- **[Viper](https://github.com/spf13/viper)** - Configuration management
133+
134+
135+
## Development Status
136+
137+
s6cmd is actively under development. While it already provides essential S3 operations, we're continuously adding new features and improvements.
138+
139+
140+
141+
## License
142+
143+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
144+
145+
## Acknowledgments
146+
147+
- Inspired by [s3cmd](https://s3tools.org/s3cmd) - The original S3 command-line tool
148+
- Built with the excellent [AWS SDK for Go v2](https://github.com/aws/aws-sdk-go-v2)
149+
150+

cmd/cp/cp.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
package cp

cmd/du/du.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
package cmd

cmd/get/get.go

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
package get
2+
3+
import (
4+
"context"
5+
"encoding/json"
6+
"fmt"
7+
"os"
8+
9+
"github.com/LinPr/s6cmd/storage"
10+
"github.com/LinPr/s6cmd/storage/uri"
11+
"github.com/go-playground/validator/v10"
12+
"github.com/spf13/cobra"
13+
)
14+
15+
func NewGetCmd() *cobra.Command {
16+
o := newOptions()
17+
cmd := cobra.Command{
18+
Use: "get [flags] <target>",
19+
Short: "get buckets and objects",
20+
Args: cobra.ExactArgs(2),
21+
Run: func(cmd *cobra.Command, args []string) {
22+
if len(args) > 0 {
23+
o.S3Uri = args[0]
24+
o.FsPath = args[1]
25+
}
26+
if err := o.complete(); err != nil {
27+
fmt.Fprintf(os.Stderr, "err: %v\n", err)
28+
return
29+
}
30+
if err := o.validate(); err != nil {
31+
fmt.Fprintf(os.Stderr, "err: %v\n", err)
32+
return
33+
}
34+
if err := o.run(); err != nil {
35+
fmt.Fprintf(os.Stderr, "err: %v\n", err)
36+
return
37+
}
38+
},
39+
}
40+
41+
cmd.Flags().BoolVarP(&o.DryRun, "dryRun", "n", false, "show what would be transferred")
42+
43+
return &cmd
44+
}
45+
46+
type Args struct {
47+
S3Uri string `validate:"omitempty"`
48+
FsPath string `validate:"omitempty"`
49+
}
50+
type Flags struct {
51+
DryRun bool `json:"DryRun" yaml:"DryRun"`
52+
}
53+
54+
type Options struct {
55+
Args
56+
Flags
57+
}
58+
59+
func newOptions() *Options {
60+
return &Options{}
61+
}
62+
63+
func (o *Options) complete() error {
64+
// 使用 viper 获取到最终生效的配置 flag > env > config > default
65+
return nil
66+
}
67+
68+
func (o *Options) validate() error {
69+
if err := validator.New().Struct(o); err != nil {
70+
return err
71+
}
72+
73+
return nil
74+
}
75+
76+
func (o *Options) run() error {
77+
j, _ := json.Marshal(o)
78+
fmt.Fprintf(os.Stdout, "options: %s\n", string(j))
79+
// return nil
80+
81+
parsedUri, err := uri.ParseS3Url(o.S3Uri)
82+
if err != nil {
83+
return err
84+
}
85+
86+
store, err := storage.NewStorage(context.TODO())
87+
if err != nil {
88+
return err
89+
}
90+
91+
return store.DownloadFile(context.TODO(), parsedUri.GetBucket(), parsedUri.GetKey(), o.FsPath)
92+
}

0 commit comments

Comments
 (0)