Skip to content

Commit ca37826

Browse files
josecorellajosecorella
committed
updates for february read:
1 parent e95b0f2 commit ca37826

File tree

3 files changed

+335
-21
lines changed

3 files changed

+335
-21
lines changed

‎changes/2025-09-12_introduce-metrics-interface/background.md‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,14 @@
55

66
## Definitions
77

8+
- Metrics: Throughout this document and other related documents the word, "metrics" is used extensively.
9+
For Crypto Tools' libraries metrics means two things.
10+
11+
1. Measuring application performance, (e.g. api requests, cache performance, latency).
12+
1. Collecting application information, (e.g. algorithm suite choice, cmm configuration, keyring configuration),
13+
and either transmitting that information to a specific user defined location or transmitting this
14+
information with Crypto Tools.
15+
816
### Conventions used in this document
917

1018
The key words
@@ -20,7 +28,7 @@ versions of these libraries have no logging or metrics publishing
2028
to either a local application or to an observability service like AWS CloudWatch.
2129

2230
As client side encryption libraries emitting metrics must be done carefully as
23-
to avoid accidentally [leaking](https://github.com/aws/aws-encryption-sdk-python/pull/105/files) any information related to the plaintext that could lead to a
31+
to avoid accidentally leaking any information related to the plaintext that could lead to a
2432
loss of customer trust.
2533

2634
A popular feature request has been for in depth insights into CT libraries. Many customers
@@ -38,7 +46,7 @@ a better customer experience.
3846
### Issue 1: What will be the default behavior?
3947

4048
As a client-side encryption library CT should be as cautious as possible.
41-
Customers of CT libraries should be on the driver seat and determine for
49+
Customers of CT libraries should be in the driver's seat and determine for
4250
themselves if their application could benefit from emitting metrics.
4351
Making that decision for customers can erode customer trust.
4452

@@ -61,7 +69,7 @@ will not change. Perhaps a bold implication this is ultimately what the customer
6169
will feel like; getting no choice on the matter and opting to not upgrade.
6270
Going from never emitting metrics to always emitting them says to customers
6371
that their application no matter its use case will always benefit from metrics.
64-
Without letting customers make that choice, CT looses hard earned customer trust.
72+
Without letting customers make that choice, CT loses hard earned customer trust.
6573

6674
This also forces customers to make a choice, start collecting metrics and pick up
6775
additional updates CT provides or get stuck in a version of the library that will
@@ -100,6 +108,19 @@ Keeping in line with the rest of CT features, a well defined interface with out
100108
of the box implementations should satisfy the feature request.
101109

102110
Out of the box implementations should cover publishing metrics to an
103-
existing observability service like AWS CloudWatch and to the local file system.
111+
existing observability service like AWS CloudWatch and to the local file system using
112+
a hardened framework for that particular language implementation.
104113
These implementations should offer customers a guide into implementing their own
105114
if they wish to do so.
115+
116+
### Issue 4: Should out of the box solutions be custom implementations
117+
118+
#### No (recommended)
119+
120+
There is no need to reinvent the wheel. Other metric frameworks have solved many
121+
of the issues that are described above, (e.g. handling failing requests, perform
122+
blocking requests to CT libraries, use a separate thread/thread pool that handles
123+
these request). By just providing a wrapper, customers can use the framework they
124+
are most comfortable with.
125+
126+
#### Yes

‎changes/2025-09-12_introduce-metrics-interface/change.md‎

Lines changed: 300 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ in this document are to be interpreted as described in
4646
## Summary
4747

4848
Existing users of Crypto Tools (CT) libraries do no have any insights as to
49-
how the librar(y/ies) behave(s) in their application.
49+
how the libraries behaves in their application.
5050
This can lead to frustrating debugging sessions where users
5151
are required to have explicit tests to assert they are using a particular feature
5252
correctly, or if customers are using any of the KMS keyrings users have to have
@@ -105,6 +105,13 @@ level Metrics Agent; this agent will get plumbed throughout CT's stack.
105105

106106
For example, in the ESDK for Java this would look like:
107107

108+
**This example uses a simple metrics agent that the mpl provides to customers**
109+
110+
**This simple interface is simply a wrapper around a battle tested logging framework.**
111+
112+
_Note: The following code snippets show changes to the ESDK.
113+
Please see the [Appendix](#appendix) for proposed changes to other libraries._
114+
108115
```java
109116
final AwsCrypto crypto = AwsCrypto.builder().build();
110117
// Create a Keyring
@@ -147,7 +154,7 @@ structure EncryptInput {
147154

148155
frameLength: FrameLength,
149156

150-
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
157+
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
151158
}
152159
...
153160
structure DecryptInput {
@@ -164,6 +171,296 @@ structure DecryptInput {
164171
//# - [Encryption Context](#encryption-context)
165172
encryptionContext: aws.cryptography.materialProviders#EncryptionContext,
166173

167-
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
174+
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
175+
}
176+
```
177+
178+
## Issues and Alternatives
179+
180+
### Issue 0: Should Metric Agents be supported on client construction?
181+
182+
#### Yes
183+
184+
#### No
185+
186+
## Appendix
187+
188+
The following API changes are not complete. These changes assume that the new optional parameter is supplied to any additional downstream consumer that may or may not be listed here.
189+
190+
### MPL
191+
192+
#### CMM Operations
193+
194+
```diff
195+
namespace aws.cryptography.materialProviders
196+
197+
use aws.polymorph#reference
198+
use aws.polymorph#positional
199+
use aws.polymorph#extendable
200+
use aws.polymorph#javadoc
201+
...
202+
structure GetEncryptionMaterialsInput {
203+
//= aws-encryption-sdk-specification/framework/cmm-interface.md#encryption-materials-request
204+
//= type=implication
205+
//# The encryption materials request MUST include the following:
206+
//#
207+
//# - [Encryption Context](structures.md#encryption-context)
208+
//# - The encryption context provided MAY be empty.
209+
//# - [Commitment Policy](./commitment-policy.md#supported-commitment-policy-enum)
210+
211+
@required
212+
encryptionContext: EncryptionContext,
213+
214+
@required
215+
commitmentPolicy: CommitmentPolicy,
216+
217+
//= aws-encryption-sdk-specification/framework/cmm-interface.md#encryption-materials-request
218+
//= type=implication
219+
//# The encryption request MAY include the following:
220+
//#
221+
//# - [Algorithm Suite Id](algorithm-suites.md#algorithm-suite-id)
222+
//# - Required Encryption Context Keys - a set of strings.
223+
//# - Max Plaintext Length
224+
//# - This value represents the maximum length of the plaintext to be encrypted
225+
//# using the returned materials.
226+
//# The length of the plaintext to be encrypted MUST not be larger than this value.
227+
228+
algorithmSuiteId: AlgorithmSuiteId,
229+
230+
maxPlaintextLength: Long,
231+
232+
requiredEncryptionContextKeys: EncryptionContextKeys,
233+
234+
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
235+
}
236+
...
237+
structure DecryptMaterialsInput {
238+
//= aws-encryption-sdk-specification/framework/cmm-interface.md#decrypt-materials-request
239+
//= type=implication
240+
//# The decrypt materials request MUST include the following:
241+
//#
242+
//# - [Algorithm Suite Id](algorithm-suites.md#algorithm-suite-id)
243+
//# - [Commitment Policy](./commitment-policy.md#supported-commitment-policy-enum)
244+
//# - [Encrypted Data Keys](structures.md#encrypted-data-keys)
245+
//# - [Encryption Context](structures.md#encryption-context)
246+
//# - The encryption context provided MAY be empty.
247+
248+
@required
249+
algorithmSuiteId: AlgorithmSuiteId,
250+
251+
@required
252+
commitmentPolicy: CommitmentPolicy,
253+
254+
@required
255+
encryptedDataKeys: EncryptedDataKeyList,
256+
257+
@required
258+
encryptionContext: EncryptionContext,
259+
260+
//= aws-encryption-sdk-specification/framework/cmm-interface.md#decrypt-materials-request
261+
//= type=implication
262+
//# The decrypt materials request MAY include the following:
263+
//#
264+
//# - [Reproduced Encryption Context](structures.md#encryption-context)
265+
266+
reproducedEncryptionContext: EncryptionContext,
267+
268+
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
269+
}
270+
```
271+
272+
#### Keyring Operations
273+
274+
```diff
275+
namespace aws.cryptography.materialProviders
276+
277+
use aws.polymorph#reference
278+
use aws.polymorph#positional
279+
use aws.polymorph#extendable
280+
use aws.polymorph#javadoc
281+
282+
use com.amazonaws.kms#EncryptionAlgorithmSpec
283+
use aws.cryptography.materialProviders#CacheType
284+
285+
@extendable
286+
resource Keyring {
287+
operations: [OnEncrypt, OnDecrypt]
288+
}
289+
290+
// Keyring Structures
291+
292+
@reference(resource: Keyring)
293+
structure KeyringReference {}
294+
...
295+
// Keyring Operations
296+
297+
//= aws-encryption-sdk-specification/framework/keyring-interface.md#onencrypt
298+
//= type=implication
299+
//# This interface MUST take [encryption materials](structures.md#encryption-materials) as input.
300+
operation OnEncrypt {
301+
input: OnEncryptInput,
302+
output: OnEncryptOutput,
303+
}
304+
305+
structure OnEncryptInput {
306+
@required
307+
materials: EncryptionMaterials,
308+
309+
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
310+
}
311+
312+
structure OnEncryptOutput {
313+
@required
314+
materials: EncryptionMaterials
315+
}
316+
317+
//= aws-encryption-sdk-specification/framework/keyring-interface.md#ondecrypt
318+
//= type=implication
319+
//# This interface MUST take [decryption materials](structures.md#decryption-materials) and
320+
//# a list of [encrypted data keys](structures.md#encrypted-data-key) as input.
321+
operation OnDecrypt {
322+
input: OnDecryptInput,
323+
output: OnDecryptOutput,
324+
}
325+
326+
structure OnDecryptInput {
327+
@required
328+
materials: DecryptionMaterials,
329+
330+
@required
331+
encryptedDataKeys: EncryptedDataKeyList,
332+
333+
+ metricsAgent: aws.cryptography.materialProviders#MetricsAgentReference
334+
}
335+
336+
structure OnDecryptOutput {
337+
@required
338+
materials: DecryptionMaterials
339+
}
340+
...
341+
```
342+
343+
### S3EC
344+
345+
The S3EC does not have a smithy model, but its Java implementation
346+
is the canonical implementation that the other language implementations
347+
should base their implementations off of.
348+
349+
#### S3EncryptionClient
350+
351+
```diff
352+
@Override
353+
public PutObjectResponse putObject(PutObjectRequest putObjectRequest, RequestBody requestBody)
354+
throws AwsServiceException, SdkClientException {
355+
...
356+
}
357+
358+
+ public PutObjectResponse putObject(PutObjectRequest putObjectRequest, RequestBody requestBody, IMetricsAgent metricsAgent)
359+
+ throws AwsServiceException, SdkClientException {
360+
+ ...
361+
+ ## Pass the IMetricsAgent reference to the appropriate downstream consumers
362+
+}
363+
...
364+
365+
@Override
366+
public <T> T getObject(GetObjectRequest getObjectRequest,
367+
ResponseTransformer<GetObjectResponse, T> responseTransformer)
368+
throws AwsServiceException, SdkClientException {
369+
...
370+
}
371+
372+
+public <T> T getObject(GetObjectRequest getObjectRequest,
373+
+ ResponseTransformer<GetObjectResponse, T> responseTransformer,
374+
+ IMetricsAgent metricsAgent)
375+
+ throws AwsServiceException, SdkClientException {
376+
+ ...
377+
+ ## Pass the IMetricsAgent reference to the appropriate downstream consumers
378+
+}
379+
```
380+
381+
#### GetEncryptedObjectPipeline
382+
383+
```diff
384+
...
385+
public class GetEncryptedObjectPipeline {
386+
private final S3AsyncClient _s3AsyncClient;
387+
private final CryptographicMaterialsManager _cryptoMaterialsManager;
388+
private final boolean _enableLegacyUnauthenticatedModes;
389+
private final boolean _enableDelayedAuthentication;
390+
private final long _bufferSize;
391+
private final InstructionFileConfig _instructionFileConfig;
392+
private final CommitmentPolicy _commitmentPolicy;
393+
+ private final IMetricsAgent _metricsAgent;
394+
395+
public static Builder builder() {
396+
return new Builder();
397+
}
398+
399+
private GetEncryptedObjectPipeline(Builder builder) {
400+
this._s3AsyncClient = builder._s3AsyncClient;
401+
this._cryptoMaterialsManager = builder._cryptoMaterialsManager;
402+
this._enableLegacyUnauthenticatedModes = builder._enableLegacyUnauthenticatedModes;
403+
this._enableDelayedAuthentication = builder._enableDelayedAuthentication;
404+
this._bufferSize = builder._bufferSize;
405+
this._instructionFileConfig = builder._instructionFileConfig;
406+
this._commitmentPolicy = builder._commitmentPolicy;
407+
+ this._metricsAgent = builder._metricsAgentl
408+
}
409+
...
410+
}
411+
```
412+
413+
#### PutEncryptedObjectPipeline
414+
415+
```diff
416+
...
417+
public class PutEncryptedObjectPipeline {
418+
final private S3AsyncClient _s3AsyncClient;
419+
final private CryptographicMaterialsManager _cryptoMaterialsManager;
420+
final private AsyncContentEncryptionStrategy _asyncContentEncryptionStrategy;
421+
final private ContentMetadataEncodingStrategy _contentMetadataEncodingStrategy;
422+
final private AlgorithmSuite _encryptionAlgorithm;
423+
+ final private IMetricsAgent _metricsAgent;
424+
425+
public static Builder builder() {
426+
return new Builder();
427+
}
428+
429+
private PutEncryptedObjectPipeline(Builder builder) {
430+
this._s3AsyncClient = builder._s3AsyncClient;
431+
this._cryptoMaterialsManager = builder._cryptoMaterialsManager;
432+
this._asyncContentEncryptionStrategy = builder._asyncContentEncryptionStrategy;
433+
this._contentMetadataEncodingStrategy = builder._contentMetadataEncodingStrategy;
434+
this._encryptionAlgorithm = builder._encryptionAlgorithm;
435+
+ this._metricsAgent = builder._metricsAgent;
436+
}
437+
...
438+
}
439+
```
440+
441+
#### Keyring
442+
443+
```diff
444+
package software.amazon.encryption.s3.materials;
445+
446+
import java.util.List;
447+
public interface Keyring {
448+
EncryptionMaterials onEncrypt(final EncryptionMaterials materials);
449+
+ EncryptionMaterials onEncrypt(final EncryptionMaterials materials, final IMetricsAgent metricsAgent);
450+
DecryptionMaterials onDecrypt(final DecryptionMaterials materials, final List<EncryptedDataKey> encryptedDataKeys);
451+
+ DecryptionMaterials onDecrypt(final DecryptionMaterials materials, final List<EncryptedDataKey> encryptedDataKeys, final IMetricsAgent metricsAgent);
452+
}
453+
```
454+
455+
#### CryptographicMaterialsManager
456+
457+
```diff
458+
package software.amazon.encryption.s3.materials;
459+
460+
public interface CryptographicMaterialsManager {
461+
EncryptionMaterials getEncryptionMaterials(EncryptionMaterialsRequest request);
462+
+ EncryptionMaterials getEncryptionMaterials(EncryptionMaterialsRequest request, IMetricsAgent metricsAgent);
463+
DecryptionMaterials decryptMaterials(DecryptMaterialsRequest request);
464+
+ DecryptionMaterials decryptMaterials(DecryptMaterialsRequest request, IMetricsAgent metricsAgent);
168465
}
169466
```

0 commit comments

Comments
 (0)