Skip to content

Commit 6cf6971

Browse files
tpodowdtpodowd
committed
Add New Object Storage Provider Plugin for Cloudian HyperStore
- Allow the CloudStack administrator to connect to Cloudian HyperStore object storage. - Once connected, CloudStack Accounts can create buckets that are managed by and belong to their own Account. - IAM Credentials are available for each bucket such that Accounts can use the buckets either from 3rd party S3 applications or from the CloudStack Bucket Browser UI Feature. - The plugin supports all the current CloudStack bucket operations such as Object Lock, Versioning, Encryption and policy settings. - The plugin currently does not support setting a bucket quota as HyperStore does not currently support that functionality. - Bucket usage is supported. More Details: - See plugins/storage/object/cloudian/README.md for details UI Changes - Add Object Storage for Cloudian HyperStore: - Cloudian HyperStore Object Storage requires more fields than Minio, Ceph and Simulator so when the Cloudian HyperStore provider is selected, the GUI adjusts and offers the extra fields that the provider requires.
1 parent 528bead commit 6cf6971

File tree

17 files changed

+2374
-11
lines changed

17 files changed

+2374
-11
lines changed

‎client/pom.xml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -642,6 +642,11 @@
642642
<artifactId>cloud-plugin-storage-object-ceph</artifactId>
643643
<version>${project.version}</version>
644644
</dependency>
645+
<dependency>
646+
<groupId>org.apache.cloudstack</groupId>
647+
<artifactId>cloud-plugin-storage-object-cloudian</artifactId>
648+
<version>${project.version}</version>
649+
</dependency>
645650
<dependency>
646651
<groupId>org.apache.cloudstack</groupId>
647652
<artifactId>cloud-plugin-storage-object-simulator</artifactId>

‎plugins/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@
139139
<module>storage/volume/primera</module>
140140
<module>storage/object/minio</module>
141141
<module>storage/object/ceph</module>
142+
<module>storage/object/cloudian</module>
142143
<module>storage/object/simulator</module>
143144

144145

‎plugins/storage/object/cloudian/README.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# Cloudian HyperStore Object Storage Plugin
2+
3+
## Plugin Purpose
4+
5+
This plugin implements the Object Storage DataStore for Cloudian HyperStore.
6+
7+
## About Cloudian HyperStore
8+
9+
Cloudian HyperStore is a fully AWS-S3 compatible Object Storage solution. The following services are used by this plugin.
10+
11+
| Service | HTTP Port | HTTPS Port | Description |
12+
|:-------:|----------:|-----------:|:-----------------------|
13+
| Admin | | 19443 | User Management etc. |
14+
| S3 | 80 | 443 | AWS-S3 compatible API |
15+
| IAM | 16080 | 16443 | AWS-IAM compatible API |
16+
17+
## Configuration
18+
19+
### HyperStore Configuration
20+
21+
1. Enable Bucket Usage Statistics
22+
23+
Bucket Level QoS settings must be set to true. On HyperStore 8+, this can be done as follows. Earlier versions require puppet configuration not documented here.
24+
25+
```shell
26+
hsh$ hsctl config set s3.qos.bucketLevel=true
27+
hsh$ hsctl config apply s3
28+
hsh$ hsctl service restart --nodes=ALL
29+
```
30+
31+
2. The Admin API Username and Password
32+
33+
The connector requires an ADMIN API username and password to connect to the Admin service and create and manage HyperStore resources such as HyperStore Users and Groups. Please review your HyperStore Admin Guide and the settings under the `admin.auth` namespace.
34+
35+
3. Enable Object Lock via License
36+
37+
HyperStore fully supports S3 Object Lock. However, Object Lock is currently only available with a special Object Lock License from Cloudian. If the connected HyperStore system does not have an Object Lock license, it will only allow creating regular buckets. Contact Cloudian Support to request an Object Lock license if required.
38+
39+
### CloudStack Configuration
40+
41+
A new `Cloudian HyperStore` Object Store can be added by the CloudStack `admin` user via the UI -> Infrastructure -> Object Storage -> Add Object Storage button.
42+
43+
![Add Cloudian HyperStore Object Storage](add_cloudian_hyperstore.png)
44+
45+
These configuration parameters are delivered to the LifeCycle class as a map with the following keys and values.
46+
47+
```text
48+
DataStoreInfo MAP
49+
++++++++++++++++++++++++++++++++++++++
50+
| Key | Value |
51+
|-------------|----------------------|
52+
|name | <user`s choice> |
53+
|providerName | Cloudian HyperStore |
54+
|url | <ADMIN endpoint URL> |
55+
|details | <MAP> ===========|=====+
56+
++++++++++++++++++++++++++++++++++++++ v
57+
v
58+
　+======================================+
59+
V
60+
Details MAP
61+
++++++++++++++++++++++++++++++++++
62+
| Key | Value |
63+
|-------------|------------------|
64+
| validateSSL | true/false |
65+
| accesskey | Admin Username |
66+
| secretkey | Admin Password |
67+
| s3Url | S3 endpoint URL |
68+
| iamUrl | IAM endpoint URL |
69+
++++++++++++++++++++++++++++++++++
70+
```
71+
72+
The following "details" map entries are all required.
73+
- validateSSL : The ADMIN API is internal and may not have a proper SSL Certificate.
74+
- accesskey : Reuse of a shared configuration parameter to pass the Admin Username.
75+
- secretkey : Reuse of a shared configuration parameter to pass the Admin password.
76+
- s3Url : The HyperStore S3 endpoint URL. HTTPS is preferred when the service has a proper SSL Certificate which should be true in production.
77+
- iamUrl : The HyperStore IAM endpoint URL. Again HTTPS is preferred.
78+
79+
The LifeCycle initialize() method should validate connectivity to the different services.
80+
81+
## CloudStack Account Mappings
82+
83+
| CloudStack | HyperStore | Name Assigned |
84+
|:-----------|:-----------------|:---------------------|
85+
| Domain | HyperStore Group | Domain UUID |
86+
| Account | HyperStore User | Account UUID |
87+
| Project | HyperStore User | Project Account UUID |
88+
89+
When a CloudStack Account user creates a bucket under their account for the first time a new HyperStore User is allocated under the HyperStore Group that is mapped to the CloudStack Domain. A new HyperStore Group is also allocated if one does not already exist.
90+
91+
## HyperStore User Resources
92+
93+
The following additional resources are also created for each HyperStore User.
94+
95+
| Resource | Description |
96+
|-----------|--------------|
97+
| Root Credential Pair | These credentials have full access to the HyperStore User account. They are used to manage the IAM user resources listed below as well as to perform any top level bucket actions such as creating buckets, updating policies, enabling versioning etc. |
98+
| IAM User "CloudStack" | The "CloudStack" IAM user is created with an inline policy as-per below. The IAM user is used by the CloudStack Bucket Browser UI to manage bucket contents.|
99+
| IAM User Policy | This inline IAM user policy grants the "CloudStack" IAM user permission to any S3 action except `s3:createBucket` and `s3:deleteBucket`. This is mostly to ensure that all Buckets remain under CloudStack control as well as to restrict control over IAM actions.|
100+
| IAM User Credential Pair | The "CloudStack" IAM user credentials are also managed by the plugin and are made available to the user under the "Bucket Details" page. They are additionally used by the CloudStack Bucket Browser UI. They are restricted by the aforementioned user policy.
101+
102+
## Bucket Management
103+
104+
The following are note worthy.
105+
106+
### Bucket Quota is Unsupported
107+
108+
This operation is not supported by this plugin. Cloudian HyperStore does not currently support restricting the size of a bucket to a particular quota.
109+
110+
### Bucket Usage
111+
112+
HyperStore does not collect bucket usage statistics by default. They must be enabled by the HyperStore Administrator. On systems where this has not been enabled, bucket usage is reported as 0 bytes.
113+
114+
See the configuration section above for more details.
115+
116+
### Supported Bucket Policies
117+
118+
Two "policies" are configurable using the CloudStack interface.
119+
120+
- Private : Objects are only accessible to the bucket owner. This is the equivalent of no bucket policy (and is implemented that way).
121+
- Public : Objects are readable to everyone. Listing of all bucket objects is not granted so the object name must be known in order to access it.
122+
```json
123+
{
124+
"Version": "2012-10-17",
125+
"Statement": [
126+
{
127+
"Sid": "PublicReadForObjects",
128+
"Effect": "Allow",
129+
"Principal": "*",
130+
"Action": "s3:GetObject",
131+
"Resource": "arn:aws:s3:::BUCKET/*"
132+
}
133+
]
134+
}
135+
```
136+
137+
### Additional Bucket CORS Settings
138+
139+
Buckets created by the CloudStack plugin are additionally created with a Cross-Origin Resource Sharing (CORS) configuration. A permissive CORS setting on buckets is required by the CloudStack Bucket Browser UI functionality as it is written in JavaScript and runs in the end user's browser.
140+
141+
```xml
142+
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
143+
<CORSRule>
144+
<ID>AllowAny</ID>
145+
<AllowedOrigin>*</AllowedOrigin>
146+
<AllowedMethod>GET</AllowedMethod>
147+
<AllowedMethod>HEAD</AllowedMethod>
148+
<AllowedMethod>PUT</AllowedMethod>
149+
<AllowedMethod>POST</AllowedMethod>
150+
<AllowedMethod>DELETE</AllowedMethod>
151+
<AllowedHeader>*</AllowedHeader>
152+
</CORSRule>
153+
</CORSConfiguration>
154+
```
155+
156+
### Visibility of other Buckets under the same HyperStore User
157+
158+
While the "CloudStack" IAM user cannot create other buckets under the HyperStore User account, there are other reasons that buckets can exist under the HyperStore user but not be known by the CloudStack. These include network connectivity issues between creating a bucket and updating the database. Note that this can usually be rectified by retrying the create bucket operation.
159+
160+
While a bucket is not visible to CloudStack, a 3rd party application using the same IAM credentials will be able to see and operate on the bucket.
161+
162+
## Interoperability with Existing HyperStore Plugin
163+
164+
This plugin is mostly interoperable with the existing HyperStore Infrastructure plugin. However, it is recommended to use one or the other but __not both__ plugins.
165+
166+
The purpose of the older HyperStore infrastructure plugin is to grant full access to the HyperStore User that is mapped to the CloudStack Account. As such it grants the logged in CloudStack Account Single-Sign-On (SSO) into the Cloudian Management Console (CMC) as the Root User of the HyperStore User. This would allow the CloudStack Account to create and delete HyperStore User resources (credentials/IAM users/federated logins/buckets/etc) outside of CloudStack control.
167+
168+
In comparison, this plugin attempts to restrict HyperStore User level, IAM and Bucket level actions by providing CloudStack Account access via IAM credentials.
169+
170+
## Known Issues
171+
172+
1. Currently there is no way to edit the Object Storage Configuration for any of the parameters configured in the "details" map. It seems that other Object Storage providers have the same issue.
173+
2. The Bucket Browser UI feature may not work correctly on HyperStore versions older than 8.2 due to some bugs in the CORS implementation. However, everything else will still function correctly.
174+
3. Object metadata is not correctly displayed in the CloudStack Bucket Browser. This is due to the javascript client using a MinIO only (non-s3 compatible) extension call that collects the metadata as part of the bucket listing. To fix this for non-MinIO S3 Object Stores, Object Metadata should be collected using the S3 standard headObject operation.
175+
4. CloudStack does not yet have a deleteUser API for Object Stores so when a CloudStack Account is deleted, the mapped HyperStore User is not currently cleaned up.

‎plugins/storage/object/cloudian/add_cloudian_hyperstore.png‎

135 KB
Loading

‎plugins/storage/object/cloudian/pom.xml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
20+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
21+
<modelVersion>4.0.0</modelVersion>
22+
<artifactId>cloud-plugin-storage-object-cloudian</artifactId>
23+
<name>Apache CloudStack Plugin - Cloudian HyperStore object storage provider</name>
24+
<parent>
25+
<groupId>org.apache.cloudstack</groupId>
26+
<artifactId>cloudstack-plugins</artifactId>
27+
<version>4.20.0.0-SNAPSHOT</version>
28+
<relativePath>../../../pom.xml</relativePath>
29+
</parent>
30+
<dependencies>
31+
<dependency>
32+
<groupId>org.apache.cloudstack</groupId>
33+
<artifactId>cloud-engine-storage</artifactId>
34+
<version>${project.version}</version>
35+
</dependency>
36+
<dependency>
37+
<groupId>org.apache.cloudstack</groupId>
38+
<artifactId>cloud-engine-storage-object</artifactId>
39+
<version>${project.version}</version>
40+
</dependency>
41+
<dependency>
42+
<groupId>org.apache.cloudstack</groupId>
43+
<artifactId>cloud-engine-schema</artifactId>
44+
<version>${project.version}</version>
45+
</dependency>
46+
<dependency>
47+
<groupId>org.apache.cloudstack</groupId>
48+
<artifactId>cloud-plugin-integrations-cloudian-connector</artifactId>
49+
<version>${project.version}</version>
50+
</dependency>
51+
<dependency>
52+
<groupId>com.amazonaws</groupId>
53+
<artifactId>aws-java-sdk-core</artifactId>
54+
</dependency>
55+
<dependency>
56+
<groupId>com.amazonaws</groupId>
57+
<artifactId>aws-java-sdk-iam</artifactId>
58+
</dependency>
59+
<dependency>
60+
<groupId>com.amazonaws</groupId>
61+
<artifactId>aws-java-sdk-s3</artifactId>
62+
</dependency>
63+
</dependencies>
64+
</project>

0 commit comments

Comments
 (0)