Skip to content

Support S3 Server Side Encryption with custom key#3192

Open
tuxgasy wants to merge 4 commits intomotioneye-project:devfrom
tuxgasy:s3-encryption
Open

Support S3 Server Side Encryption with custom key#3192
tuxgasy wants to merge 4 commits intomotioneye-project:devfrom
tuxgasy:s3-encryption

Conversation

@tuxgasy
Copy link

@tuxgasy tuxgasy commented Aug 31, 2025

Add a supports of Server Side Encryption known as SSE-C for compatible S3 bucket provider.

The encryption is performed on the S3 server with the provided key. The key is not stored on the S3 server. To download encrypted file, you must therefore provide the same key.

The encryption key must be a base64 encoded string with a length of 32 bytes. You can generate one as following:

openssl rand 32 | base64

@tuxgasy
Copy link
Author

tuxgasy commented Aug 31, 2025

MD5 hash of the encryption key is required. How to ignore the failing check ?

s3.upload_file(
filename,
self._bucket,
filename[(len(target_dir)+1) :],
Copy link
Member

@MichaIng MichaIng Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The +1 was added to remove the last slash as well? Did this always appear in the resulting key? But what happens if the target_dir is passed with trailing slash? I am currently not sure whether it is striped somewhere, otherwise we could either target_dir.rstrip('/') or alternatively filename[len(target_dir) :].lstrip('/') to achieve the same effect.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh, it seems to be actually solved better here: https://github.com/motioneye-project/motioneye/pull/2760/changes
Let me merge this first.

Signed-off-by: MichaIng <micha@dietpi.com>
@MichaIng
Copy link
Member

MichaIng commented Feb 4, 2026

@tuxgasy
Can you please test it with the SSECustomerKeyMD5 argument removed? As said, according to boto3 docs, it is not needed, but calculated automatically if missing. And see my above question regarding +1 removed characters for the key name.

@tuxgasy
Copy link
Author

tuxgasy commented Feb 4, 2026

@tuxgasy Can you please test it with the SSECustomerKeyMD5 argument removed? As said, according to boto3 docs, it is not needed, but calculated automatically if missing. And see my above question regarding +1 removed characters for the key name.

Without SSECustomerKeyMD5, I get the following error message :

boto3.exceptions.S3UploadFailedError: Failed to upload /var/lib/motioneye/Camera1/2026-02-04/18-07-57.jpg to me-salon/2026-02-04/18-07-57.jpg:
An error occurred (InvalidArgument) when calling the PutObject operation: The secret key was invalid for the specified algorithm.

@MichaIng
Copy link
Member

MichaIng commented Feb 4, 2026

This is latest boto3? Weird, the docs say it is not needed: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/upload_part.html
Though this is upload_part and a bunch of other methods: https://boto3.amazonaws.com/v1/documentation/api/latest/search.html?q=SSECustomerKeyMD5
But the upload_file page does not explicitly mention it: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/upload_file.html

However, if it does not work reliably without, I'll add it back in.

Since I want to fix/merge #2760 first, can you confirm that using slashes in the filename creates subdirectories on the S3 server?

@tuxgasy
Copy link
Author

tuxgasy commented Feb 5, 2026

Since I want to fix/merge #2760 first, can you confirm that using slashes in the filename creates subdirectories on the S3 server?

The subdirectories are created on S3 server. But when I disable Include Subdirectories, I get the following error :

motioneye-1  |    ERROR: failed to upload file "/var/lib/motioneye/Camera1/2026-02-05/06-46-43.jpg" with service s3: object of type 'bool' has no len()
motioneye-1  | Traceback (most recent call last):
motioneye-1  |   File "/usr/local/lib/python3.13/dist-packages/motioneye/uploadservices.py", line 1324, in upload_media_file
motioneye-1  |     service.upload_file(target_dir, filename, camera_name)
motioneye-1  |     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
motioneye-1  |   File "/usr/local/lib/python3.13/dist-packages/motioneye/uploadservices.py", line 1242, in upload_file
motioneye-1  |     filename[(len(target_dir) + 1) :],
motioneye-1  |               ~~~^^^^^^^^^^^^
motioneye-1  | TypeError: object of type 'bool' has no len()

We need to check if target_dir is a boolean (False ?). Do you want to fix it with the other PR ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

2 participants