Skip to content

Commit 3d53298

Browse files
andrewazoresandrewazores
authored
feat(storage): add toggle for presigned file transfers (#269)
1 parent 22258a3 commit 3d53298

File tree

5 files changed

+67
-0
lines changed

5 files changed

+67
-0
lines changed

‎charts/cryostat/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,8 @@ certificate issuance and rotation.
185185
| `storage.storageSecretName` | Name of the secret containing the object storage secret access key. This secret must contain a STORAGE_ACCESS_KEY secret which is the object storage secret access key. It must not be updated across chart upgrades, or else the connection between Cryostat components and object storage will not be able to initialize. If using an external S3 provider requiring authentication then this **must** be provided. It is recommended that the secret should be marked as immutable to avoid accidental changes to secret's data. More details: [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable) | `""` |
186186
| `storage.provider.url` | URL to the S3 object storage provider instance. This can be an in-cluster self-hosted instance with a hostname like s3.storage.local, or it can be an external commercial service. This should include scheme, host, and port. User authenication information should be provided using a *Secret* and *storage.storageSecretName*. If this is not specified then a managed [cryostat-storage](https://github.com/cryostatio/cryostat-storage) instance will be automatically deployed and configured. If an unmanaged S3 instance is specified here then other storage configuration settings (such as at-rest encryption, Pod annotations, Service configurations) do not apply. Production installations of Cryostat should not rely on `cryostat-storage` | `""` |
187187
| `storage.provider.usePathStyleAccess` | whether path-style accesses are used for ex. object buckets. If path style access is not used then DNS subdomain resolution will be used. This is *true* by default for broader compatibility for low-footprint storage container installations, but subdomain resolution generally offers better performance if it is available and may be required for use with commercial storage providers. | `true` |
188+
| `storage.provider.usePresignedRecordingTransfers` | whether object storage presigned GET URLs should be used for transferring files between Cryostat components (ex. for automated analysis report generation). If this is disabled then Cryostat will act as a "network pipe" between other components and handle streaming file contents. This is *true* by default to reduce network utilization and request latency | `true` |
189+
| `storage.provider.usePresignedDownloads` | whether object storage presigned GET URLs should be used for downloading files via the user's browser. If this is disabled then Cryostat will act as a "network pipe" between storage and the user's browser and handle streaming file contents. If the object storage URLs are not accessible from the user's network location then this must be disabled, otherwise enabling it will reduce network utilization and request latency. This is *false* by default | `false` |
188190
| `storage.provider.region` | S3 object storage provider region. This may be used by the storage provider to geolocate the physical storage in a particular region for regulatory, performance, or cost reasons | `""` |
189191
| `storage.provider.authentication.credentialsType` | configuration for how the S3 client will locate credentials for the S3 service. See: [Quarkus S3 client](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-s3.html#) | `default` |
190192
| `storage.provider.tls.trustAll` | enable this to disable TLS certificate verification on the S3 client | `false` |

‎charts/cryostat/templates/cryostat_deployment.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,10 @@ spec:
114114
value: {{ default (printf "http://%s-storage:8333" $fullName) .Values.storage.provider.url }}
115115
- name: QUARKUS_S3_PATH_STYLE_ACCESS
116116
value: "{{ .Values.storage.provider.usePathStyleAccess }}"
117+
- name: STORAGE_PRESIGNED_TRANSFERS_ENABLED
118+
value: "{{ .Values.storage.provider.usePresignedRecordingTransfers }}"
119+
- name: STORAGE_PRESIGNED_DOWNLOADS_ENABLED
120+
value: "{{ .Values.storage.provider.usePresignedDownloads }}"
117121
- name: QUARKUS_S3_AWS_REGION
118122
# if an external provider URL is supplied then a region must also be supplied.
119123
# Otherwise we are deploying a managed storage instance and can set a default value

‎charts/cryostat/tests/cryostat_deployment_test.yaml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,9 @@ tests:
132132
- equal:
133133
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='QUARKUS_S3_PATH_STYLE_ACCESS')].value
134134
value: "true"
135+
- equal:
136+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='STORAGE_PRESIGNED_TRANSFERS_ENABLED')].value
137+
value: "true"
135138
- equal:
136139
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='QUARKUS_S3_AWS_REGION')].value
137140
value: "us-east-1"
@@ -200,6 +203,50 @@ tests:
200203
- notExists:
201204
path: spec.template.spec.contains[?(@.name=='cryostat')].volumeMounts
202205

206+
- it: should allow configuration of external object storage provider
207+
set:
208+
storage:
209+
provider:
210+
url: 'https://s3.example.com:1234'
211+
usePathStyleAccess: true
212+
usePresignedRecordingTransfers: false
213+
region: 'a-b1'
214+
metadata:
215+
storageMode: bucket
216+
asserts:
217+
- equal:
218+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='QUARKUS_S3_ENDPOINT_OVERRIDE')].value
219+
value: "https://s3.example.com:1234"
220+
- equal:
221+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='QUARKUS_S3_PATH_STYLE_ACCESS')].value
222+
value: "true"
223+
- equal:
224+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='STORAGE_PRESIGNED_TRANSFERS_ENABLED')].value
225+
value: "false"
226+
- equal:
227+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='STORAGE_PRESIGNED_DOWNLOADS_ENABLED')].value
228+
value: "false"
229+
- equal:
230+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='QUARKUS_S3_AWS_REGION')].value
231+
value: "a-b1"
232+
- equal:
233+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='STORAGE_METADATA_STORAGE_MODE')].value
234+
value: "bucket"
235+
236+
- it: should allow configuration of presigned downloads separately from presigned transfers
237+
set:
238+
storage:
239+
provider:
240+
usePresignedRecordingTransfers: true
241+
usePresignedDownloads: false
242+
asserts:
243+
- equal:
244+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='STORAGE_PRESIGNED_TRANSFERS_ENABLED')].value
245+
value: "true"
246+
- equal:
247+
path: spec.template.spec.containers[?(@.name=='cryostat')].env[?(@.name=='STORAGE_PRESIGNED_DOWNLOADS_ENABLED')].value
248+
value: "false"
249+
203250
- it: should allow overriding S3 bucket names
204251
set:
205252
storage:

‎charts/cryostat/values.schema.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -676,6 +676,16 @@
676676
"description": "whether path-style accesses are used for ex. object buckets. If path style access is not used then DNS subdomain resolution will be used. This is *true* by default for broader compatibility for low-footprint storage container installations, but subdomain resolution generally offers better performance if it is available and may be required for use with commercial storage providers.",
677677
"default": true
678678
},
679+
"usePresignedRecordingTransfers": {
680+
"type": "boolean",
681+
"description": "whether object storage presigned GET URLs should be used for transferring files between Cryostat components (ex. for automated analysis report generation). If this is disabled then Cryostat will act as a \"network pipe\" between other components and handle streaming file contents. This is *true* by default to reduce network utilization and request latency",
682+
"default": true
683+
},
684+
"usePresignedDownloads": {
685+
"type": "boolean",
686+
"description": "whether object storage presigned GET URLs should be used for downloading files via the user's browser. If this is disabled then Cryostat will act as a \"network pipe\" between storage and the user's browser and handle streaming file contents. If the object storage URLs are not accessible from the user's network location then this must be disabled, otherwise enabling it will reduce network utilization and request latency. This is *false* by default",
687+
"default": false
688+
},
679689
"region": {
680690
"type": "string",
681691
"description": "S3 object storage provider region. This may be used by the storage provider to geolocate the physical storage in a particular region for regulatory, performance, or cost reasons",

‎charts/cryostat/values.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,10 @@ storage:
279279
url: ""
280280
## @param storage.provider.usePathStyleAccess whether path-style accesses are used for ex. object buckets. If path style access is not used then DNS subdomain resolution will be used. This is *true* by default for broader compatibility for low-footprint storage container installations, but subdomain resolution generally offers better performance if it is available and may be required for use with commercial storage providers.
281281
usePathStyleAccess: true
282+
## @param storage.provider.usePresignedRecordingTransfers whether object storage presigned GET URLs should be used for transferring files between Cryostat components (ex. for automated analysis report generation). If this is disabled then Cryostat will act as a "network pipe" between other components and handle streaming file contents. This is *true* by default to reduce network utilization and request latency
283+
usePresignedRecordingTransfers: true
284+
## @param storage.provider.usePresignedDownloads whether object storage presigned GET URLs should be used for downloading files via the user's browser. If this is disabled then Cryostat will act as a "network pipe" between storage and the user's browser and handle streaming file contents. If the object storage URLs are not accessible from the user's network location then this must be disabled, otherwise enabling it will reduce network utilization and request latency. This is *false* by default
285+
usePresignedDownloads: false
282286
## @param storage.provider.region S3 object storage provider region. This may be used by the storage provider to geolocate the physical storage in a particular region for regulatory, performance, or cost reasons
283287
region: ''
284288
authentication:

0 commit comments

Comments
 (0)