Skip to content
This repository was archived by the owner on Sep 3, 2025. It is now read-only.

Commit 8dcd8de

Browse files
sadoyansadoyan
committed
Added JWT, ApiKey authentication methods
1 parent 78709b2 commit 8dcd8de

File tree

14 files changed

+402
-236
lines changed

14 files changed

+402
-236
lines changed

‎README.md‎

Lines changed: 94 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,23 @@
11
**WebRados**
22
---------
33

4-
**WebRados** is a simple and high performance HTTP service for **CEPH** distributed file system.
4+
**WebRados** is a simple and high performance HTTP service for **CEPH** distributed file system.
55
The main goal of this tool is to provide simple HTTP interface for **Ceph's** bare **RADOS** layer.
6-
**WebRados** is not a replacement for **RadosGW** as is does not have all reach APIs and features of RadosGW(S3, Swift, etc ...), instead it stands for simplicity and performance.
6+
**WebRados** is not a replacement for **RadosGW** as is does not have all reach APIs and features of RadosGW(S3, Swift, etc ...), instead it stands for simplicity and performance.
77

8-
WebRados relies on HTTP methods to interact with RADOS object, thus it can provide access to objects store in RADOS directly from internet browser .
9-
The ide is to have web accessible storage for millions of relatively small files, which can be accessed from browser directly.
8+
WebRados relies on HTTP methods to interact with RADOS object, thus it can provide access to objects store in RADOS directly from internet browser .
9+
The ide is to have web accessible storage for millions of relatively small files, which can be accessed from browser directly.
1010

11-
WebRados relies on C bindings of **Ceph** so in order to run this program you need to install Ceph packages.
12-
Running Ceph services on computer which hosts WebRados is not required, it's even better to have a dedicated server or server for running WebRados
11+
WebRados relies on C bindings of **Ceph** so in order to run this program you need to install Ceph packages.
12+
Running Ceph services on computer which hosts WebRados is not required, it's even better to have a dedicated server or server for running WebRados
1313

1414
### **Download and install**
1515
---------
1616

17-
You can build WebRados from source or download precompiled binaries. If you already have installed Cephs packages and want to make things easy ,
18-
just download te WebRados binary, make it executeable, and you are ready to run .
17+
You can build WebRados from source or download precompiled binaries. If you already have installed Cephs packages and want to make things easy ,
18+
just download te WebRados binary, make it executeable, and you are ready to run .
1919

20-
Building from a source is also easy.
20+
Building from a source is also easy.
2121

2222
```shell
2323
git clone https://github.com/sadoyan/go-webrados.git
@@ -27,128 +27,126 @@ go mod tidy
2727
do build .
2828
```
2929

30-
3130
### **Configuration**
3231
---------
3332

3433
Configuration paramaters are stored in ```config.ini``` file, which should be in running directory.
35-
Sample config file, with reasoneable defaulr ships with source code.
36-
37-
```ini
38-
[main]
39-
listen : 0.0.0.0:8080
40-
dispatchers : 20
41-
serveruser : admin
42-
serverpass : 261a5983599fd57a016122ec85599ec4
43-
dangerzone : yes
44-
readonly : no
45-
authread : no
46-
authwrite : yes
47-
radoconns : 25
48-
logfile : no
49-
logpath: /opt/webrados.log
50-
allpools: no
51-
poollist: bublics, donuts, images
52-
53-
[cache]
54-
shards: 1024
55-
lifewindow: 10
56-
cleanwindow: 1
57-
maxrntriesinwindow: 600000
58-
maxentrysize: 5000
59-
maxcachemb: 1024
60-
61-
[database]
62-
type : ceph
63-
server: 127.0.0.1:6379
64-
username : none
65-
password : none
66-
database : 0
67-
68-
[monitoring]
69-
enabled : true
70-
url: 127.0.0.1:9090
71-
user: admin
72-
pass: admin
34+
Sample config file, with reasoneable defaulr ships with source code.
35+
36+
```yaml
37+
main:
38+
listen: 0.0.0.0:8080
39+
dispatchers: 20
40+
serveruser: admin
41+
serverpass: 261a5983599fd57a016122ec85599ec4
42+
dangerzone: yes
43+
readonly: no
44+
authread: no
45+
authwrite: yes
46+
radoconns: 25
47+
logfile: no
48+
logpath: /opt/webrados.log
49+
allpools: no
50+
poollist:
51+
- bublics
52+
- donuts
53+
- images
54+
usersfile: users.txt
55+
authtype: jwt # apikey , basic, jwt, none
56+
cache:
57+
shards: 1024
58+
lifewindow: 10
59+
cleanwindow: 1
60+
maxrntriesinwindow: 600000
61+
maxentrysize: 5000
62+
maxcachemb: 1024
63+
monitoring:
64+
enabled: yes
65+
url: 127.0.0.1:9090
66+
user: admin
67+
pass: admin
68+
7369
```
7470

7571
### **API**
7672
---------
7773

78-
| **Name** | **Description** |
79-
| ------------- | ------------- |
80-
|**Read File**|HTTP ```GET:``` http://{BINDADDRESS}/{POOLNAME}/{FILENAME}|
81-
|**Upload File**|HTTP ```POST, PUT:``` http://{BINDADDRESS}/{POOLNAME}/{FILENAME}|
82-
|**Remove File**|HTTP ```DELETE:``` http://{BINDADDRESS}/{POOLNAME}/{FILENAME}|
74+
| **Name** | **Description** |
75+
|-----------------|------------------------------------------------------------------|
76+
| **Read File** | HTTP ```GET:``` http://{BINDADDRESS}/{POOLNAME}/{FILENAME} |
77+
| **Upload File** | HTTP ```POST, PUT:``` http://{BINDADDRESS}/{POOLNAME}/{FILENAME} |
78+
| **Remove File** | HTTP ```DELETE:``` http://{BINDADDRESS}/{POOLNAME}/{FILENAME} |
8379

84-
Configuration file is pretty simple and intuitive.
80+
Configuration file is pretty simple and intuitive.
8581

8682
### **Section main**
8783
---------
8884

89-
| **Name** | **Description** |
90-
|-------------------|--------------------------------------------------------------------------------------------------------------------------------------|
91-
| **listen** | IP port to bind. |
92-
| **dispatchers** | Number of threads for webserver. |
93-
| **serveruser** | Static user. |
94-
| **serverpass** | MD5 hash of password for static user. It can be the output of `echo -n SecretPaSs md5sum |awk '{print $1}'`| |
95-
| **dangerzone** | Enable destructive methods and commands (DELETE). |
96-
| **readonly** | Enable readonly mode. If 'yes' only GET is allowed. |
97-
| **authread** | Require authentication for GET only. |
98-
| **authwrite** | Require authentication for POST/PUT/DELETE. |
99-
| **radoconns** | Number of connection to CEPH. |
100-
| **logfile** | Log to file, if 'no' logs are sent to stdout. |
101-
| **logpath** | Path for log file. |
102-
| **uploadmaxpart** | Maximum file chunk size (Sbould be amaller or erqual ro `osd max object size`). |
103-
| **allpools:** | yes/no . If yes program will scan ceph and enable access via web to all pool. |
104-
| **poollist:** | Works only if **allpools** is set to **no**. Coma separated list of pools which should be accesible via webrados program. |
105-
| **database** | Name of database server for storing metadata of fliles (Ceph, Redis, MySQL). MySQL and Redis are deprecated and will be removed soon |
85+
| **Name** | **Description** |
86+
|-------------------|----------------------------------------------------------------------------------------------------------------------------|
87+
| **listen** | IP port to bind. |
88+
| **dispatchers** | Number of threads for webserver. |
89+
| **serveruser** | Static user. |
90+
| **serverpass** | MD5 hash of password for static user. | |
91+
| **dangerzone** | Enable destructive methods and commands (DELETE). |
92+
| **readonly** | Enable readonly mode. If 'yes' only GET is allowed. |
93+
| **authread** | Require authentication for GET only. |
94+
| **authwrite** | Require authentication for POST/PUT/DELETE. |
95+
| **radoconns** | Number of connection to CEPH. |
96+
| **logfile** | Log to file, if 'no' logs are sent to stdout. |
97+
| **logpath** | Path for log file. |
98+
| **uploadmaxpart** | Maximum file chunk size (Sbould be amaller or erqual ro `osd max object size`). |
99+
| **allpools:** | yes/no . If yes program will scan ceph and enable access via web to all pool. |
100+
| **poollist:** | Works only if **allpools** is set to **no**. |
101+
| **usersfile** | Path for file containing list of users with `username passwordhash` format separated by new line. |
102+
| **authtype** | Authentication methods. ***apikey*** (X-API-KEY Header) , **basic** (HTTP Basic Auth), **jwt** (https://jwt.io/), **none** |
106103

107104
### **Section cache**
108105
---------
109-
| **Name** | **Description** |
110-
| ------------- |---------------------------------------------------------------------------------------------------|
111-
|**shards**| Number of shards (must be a power of 2) |
112-
|**lifewindow**| Time after which entry can be evicted |
113-
|**cleanwindow**| Interval between removing expired entries (clean up). If set to <= 0 then no action is performed. |
114-
|**maxrntriesinwindow**| rps * lifeWindow, used only in initial memory allocation |
115-
|**maxentrysize**| max entry size in bytes, used only in initial memory allocation |
116-
|**maxcachemb**| Cache will not allocate more memory than this limit, value in MB. 0 value means no size limit |
117106

107+
| **Name** | **Description** |
108+
|------------------------|---------------------------------------------------------------------------------------------------|
109+
| **shards** | Number of shards (must be a power of 2) |
110+
| **lifewindow** | Time after which entry can be evicted |
111+
| **cleanwindow** | Interval between removing expired entries (clean up). If set to <= 0 then no action is performed. |
112+
| **maxrntriesinwindow** | rps * lifeWindow, used only in initial memory allocation |
113+
| **maxentrysize** | max entry size in bytes, used only in initial memory allocation |
114+
| **maxcachemb** | Cache will not allocate more memory than this limit, value in MB. 0 value means no size limit |
118115

119116
### **Section monitoring**
120117
---------
121-
| **Name** | **Description** |
122-
| ------------- | ------------- |
123-
|**enabled**|Enable/Disable monitoring.|
124-
|**url**|IP address and port for minitoring interface.|
125-
|**user**|Monitoring user.|
126-
|**pass**|Password for monitoring user.|
118+
119+
| **Name** | **Description** |
120+
|-------------|-----------------------------------------------|
121+
| **enabled** | Enable/Disable monitoring. |
122+
| **url** | IP address and port for minitoring interface. |
123+
| **user** | Monitoring user. |
124+
| **pass** | Password for monitoring user. |
127125

128126
### **users.txt file**
129127

130-
Webrados can dynamically update users from ```users.txt``` file .
128+
Webrados can dynamically update users from ```users.txt``` file .
131129
```users.txt``` should contain user and md5hash of password divided by space in each line.
132-
`echo -n SecretPaSs | md5sum |awk '{print $1}'` on Linux systemd will output md5hash for using it as password in `users.txt` file
133-
Webrados will periodically read ```uesrs.txt``` file and automatically update users in memory.
130+
```echo -n SecretPaSs | md5sum |awk '{print $1}'``` on Linux systemd will output md5hash for using it as password in ```users.txt``` file
131+
Webrados will periodically read ```uesrs.txt``` file and automatically update users in memory.
134132

135133
### **Large files**
136134

137-
In order to be able to store large file in RADOS directly files needs to be split to smaller chunks.
138-
WebRados will automatically set maximum chunk size to **OSDMaxObjectSize** of Ceph and split files in accordance to that.
135+
In order to be able to store large file in RADOS directly files needs to be split to smaller chunks.
136+
WebRados will automatically set maximum chunk size to **OSDMaxObjectSize** of Ceph and split files in accordance to that.
139137

140138
### **Special commands**
141139

142-
**HTTP GET** http://{BINDADDRESS}/{POOLNAME}/{FILENAME}?info
143-
Return information about requested file in json format.
140+
**HTTP GET** http://{BINDADDRESS}/{POOLNAME}/{FILENAME}?info
141+
Return information about requested file in json format.
144142

145143
```curl -s http://ceph1:8080/bublics/katana.mp4?info | python -mjson.tool```
146144

147145
```json
148146
{
149-
"name": "katana.mp4",
150-
"pool": "bublics",
151-
"segments": "11",
152-
"size": "471861144"
147+
"name": "katana.mp4",
148+
"pool": "bublics",
149+
"segments": "11",
150+
"size": "471861144"
153151
}
154152
```

0 commit comments

Comments
 (0)