Skip to content

Conversation

@giancarloromeo
Copy link
Contributor

@giancarloromeo giancarloromeo commented Jan 24, 2025

What do these changes do?

This PR enhances the security of API keys by hashing API secrets using Blowfish (with 10 iterations) and refactors API key creation/deletion flows across multiple services and tests.

  • Updates API key creation and deletion RPC interfaces and repository functions.
  • Adjusts related unit tests and migration scripts to support the new security standards.
  • Refactors director-v2 and dynamic sidecar modules to use the updated API key logic.

Store a hashed version (actually blowfish with 10 iterations) of the api_secret.

When storing:

  • Generate a random API key/secret and hash the secret.
  • Store the hashed secret in the database.
  • Return the "plain" secret back to the caller for subsequent verification.

When verifying:

  • Provide the "plain" secret as part of the request.
  • Validate using the same function used when hashing.

All existing API keys already stored in the DB must be hashed & updated.

image

Related issue/s

How to test

Dev-ops checklist

@giancarloromeo giancarloromeo self-assigned this Jan 24, 2025
@giancarloromeo giancarloromeo added this to the Singularity milestone Jan 24, 2025
@giancarloromeo giancarloromeo added t:maintenance Some planned maintenance work security Pull requests that address a security vulnerability a:webserver webserver's codebase. Assigning the area is particularly useful for bugs labels Jan 24, 2025
@codecov
Copy link

codecov bot commented Jan 24, 2025

Codecov Report

Attention: Patch coverage is 88.23529% with 14 lines in your changes missing coverage. Please review.

Project coverage is 87.60%. Comparing base (d0f485b) to head (ef6493d).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7085   +/-   ##
=======================================
  Coverage   87.60%   87.60%           
=======================================
  Files        1758     1758           
  Lines       68099    68116   +17     
  Branches     1124     1124           
=======================================
+ Hits        59660    59676   +16     
- Misses       8130     8131    +1     
  Partials      309      309           
Flag Coverage Δ
integrationtests 65.17% <68.00%> (+0.14%) ⬆️
unittests 86.79% <86.55%> (+<0.01%) ⬆️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library 93.91% <ø> (ø)
pkg_dask_task_models_library 97.09% <ø> (ø)
pkg_models_library 92.68% <62.50%> (-0.10%) ⬇️
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.18% <ø> (ø)
pkg_service_integration 69.98% <ø> (ø)
pkg_service_library 72.78% <0.00%> (-0.03%) ⬇️
pkg_settings_library 90.84% <ø> (ø)
pkg_simcore_sdk 85.40% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 91.23% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.46% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.29% <ø> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.78% <ø> (ø)
director_v2 91.37% <95.12%> (+0.01%) ⬆️
dynamic_scheduler 97.40% <ø> (ø)
dynamic_sidecar 90.11% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.23% <ø> (ø)
storage 87.58% <ø> (+0.07%) ⬆️
webclient ∅ <ø> (∅)
webserver 86.05% <94.91%> (+0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0f485b...ef6493d. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@giancarloromeo giancarloromeo changed the title 🔒️ Hash API Keys secret WIP: 🔒️ Hash API Keys secret Jan 27, 2025
@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 5, 2025

@sanderegg sanderegg modified the milestones: Singularity, The Awakening Feb 24, 2025
@giancarloromeo giancarloromeo changed the title WIP: 🔒️ Hash API Keys secret 🔒️ Hash API Keys secret Mar 12, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances the security of API keys by hashing API secrets using Blowfish (with 10 iterations) and refactors API key creation/deletion flows across multiple services and tests.

  • Updates API key creation and deletion RPC interfaces and repository functions.
  • Adjusts related unit tests and migration scripts to support the new security standards.
  • Refactors director-v2 and dynamic sidecar modules to use the updated API key logic.

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
services/web/server/tests/unit/with_dbs/01/test_api_keys_rpc.py Update test calls to use the new delete_api_key_by_key and convert api_key ids with IDStr.
services/web/server/tests/unit/with_dbs/01/test_api_keys.py Remove legacy test for get_or_create_api_key.
services/web/server/src/simcore_service_webserver/api_keys/_service.py Refactor API key creation/deletion functions to use new helper functions.
services/web/server/src/simcore_service_webserver/api_keys/_repository.py Update API key insertion to hash the secret and add new deletion and listing functions.
services/web/server/src/simcore_service_webserver/api_keys/_controller_rpc.py Update exposed RPC endpoints to reflect new method names.
services/director-v2/** Refactor API key creation and deletion logic to use new functions in substitutions, _api_auth, and _api_auth_rpc.
services/director-v2/src/simcore_service_director_v2/modules/dynamic_sidecar/scheduler/_core/_events_utils.py Adjust dynamic sidecar cleanup to generate and delete API keys using new helpers.
services/api-server/tests/unit/_with_db/conftest.py Update fake API key creation to hash API secrets; use new column filtering.
packages/service-library/, packages/postgres-database/, packages/models-library/** Update models, migration scripts, and RPC interfaces for consistent API key/security handling.

@giancarloromeo giancarloromeo requested a review from Copilot April 22, 2025 07:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the security of API keys by switching to Blowfish-based hashing with 10 iterations and refactoring key creation and deletion flows across multiple services and tests. Key changes include:

  • Updating API key creation to use a new hashing function and refactoring the corresponding RPC and REST endpoints.
  • Modifying repository operations and migration scripts to store hashed API secrets and add an index on the api_key column.
  • Adjusting related test fixtures and director-v2 modules to use the updated API key logic.

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
services/web/server/tests/unit/with_dbs/01/test_api_keys.py Removed redundant test for idempotent API key creation.
services/web/server/src/simcore_service_webserver/api_keys/plugin.py Updated API key controller route imports.
services/web/server/src/simcore_service_webserver/api_keys/_service.py Refactored API key creation/deletion flows and replaced legacy key generation.
services/web/server/src/simcore_service_webserver/api_keys/_repository.py Changed API secret storage to use Blowfish hashing and updated conflict handling.
services/director-v2/... Renamed API auth functions and updated tests to reflect the new behavior.
services/api-server/... Adjusted test fixtures and repository queries to match the updated API key schema.
packages/postgres-database/... Added migration scripts to index api_key and hash existing API secrets.
packages/models-library/... Introduced functions for generating unique API keys and secrets.
Comments suppressed due to low confidence (1)

services/director-v2/src/simcore_service_director_v2/modules/dynamic_sidecar/scheduler/_core/_events_utils.py:330

  • The comment contains a typo ('debuug'); please correct it to 'debug'.
      # used for debuug, normally sleeps 0

@giancarloromeo giancarloromeo marked this pull request as ready for review April 22, 2025 07:25
@giancarloromeo giancarloromeo requested a review from pcrespov April 22, 2025 10:23
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx! I was looking forward to this change since long time!

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some suggestions. Thx

@sonarqubecloud
Copy link

@giancarloromeo giancarloromeo merged commit 5e1323e into ITISFoundation:master Apr 23, 2025
94 checks passed
@giancarloromeo giancarloromeo deleted the is6880/hash-api-key-secret branch April 23, 2025 06:04
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 8, 2025
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:apiserver api-server service a:director-v2 issue related with the director-v2 service a:webserver webserver's codebase. Assigning the area is particularly useful for bugs security Pull requests that address a security vulnerability t:maintenance Some planned maintenance work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

api key: upgrade encryption scheme of key-secret in db

5 participants