Skip to content

Conversation

@pcrespov
Copy link
Member

@pcrespov pcrespov commented Apr 23, 2025

What do these changes do?

Use the json_loads function from common_library.json_serialization instead of the built-in json.dumps and json.loads for serialization and deserialization. These functions are built on top of orjson and are significantly more reliable. Avoid using json.dumps and json.loads.

NOTE: json_dumps by default has no spaces compared to `json.dumps"

>>> from common_library.json_serialization import *
>>> import json

>>> data = {"x": 1, "y": "foo", "z":True }

>>> json_dumps(data)
'{"x":1,"y":"foo","z":true}'  # <-- NOTE that this string has no spaces
>>> json.dumps(data)
'{"x": 1, "y": "foo", "z": true}' # <-- NOTE that this string has spaces after `:` and ,

When working with Pydantic models, use their built-in methods for serialization/deserialization—for example, model.model_dump_json()—instead of external utilities.

This PR updates this practice across all the repo code-base (avoided scripts and tests)

Related issue/s

Maintenance

How to test

Dev-ops

@pcrespov pcrespov self-assigned this Apr 23, 2025
@codecov
Copy link

codecov bot commented Apr 23, 2025

Codecov Report

Attention: Patch coverage is 83.13253% with 14 lines in your changes missing coverage. Please review.

Project coverage is 87.55%. Comparing base (2d68a47) to head (c9fc66a).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7569      +/-   ##
==========================================
- Coverage   87.63%   87.55%   -0.08%     
==========================================
  Files        1759     1698      -61     
  Lines       68188    66440    -1748     
  Branches     1125     1037      -88     
==========================================
- Hits        59755    58174    -1581     
+ Misses       8124     7977     -147     
+ Partials      309      289      -20     
Flag Coverage Δ
integrationtests 65.12% <71.05%> (+0.02%) ⬆️
unittests 86.71% <75.90%> (-0.11%) ⬇️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library 97.10% <100.00%> (+0.01%) ⬆️
pkg_models_library 92.70% <100.00%> (ø)
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.18% <100.00%> (ø)
pkg_service_integration 69.92% <50.00%> (-0.06%) ⬇️
pkg_service_library 72.92% <75.00%> (ø)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.72% <100.00%> (+0.28%) ⬆️
agent 96.46% <ø> (ø)
api_server 91.27% <ø> (-0.02%) ⬇️
autoscaling 96.08% <100.00%> (ø)
catalog 92.64% <100.00%> (-0.01%) ⬇️
clusters_keeper 99.25% <ø> (ø)
dask_sidecar 91.29% <100.00%> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.80% <66.66%> (+0.02%) ⬆️
director_v2 91.38% <78.57%> (-0.01%) ⬇️
dynamic_scheduler 97.40% <100.00%> (ø)
dynamic_sidecar 90.15% <55.55%> (+<0.01%) ⬆️
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.12% <ø> (ø)
storage 87.58% <100.00%> (+0.07%) ⬆️
webclient ∅ <ø> (∅)
webserver 86.04% <100.00%> (+<0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d68a47...c9fc66a. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pcrespov pcrespov force-pushed the mai/json-serializatino branch from abd3323 to 49f71b1 Compare April 23, 2025 14:05
@pcrespov pcrespov added this to the Pauwel Kwak milestone Apr 23, 2025
@pcrespov pcrespov added the t:maintenance Some planned maintenance work label Apr 23, 2025
@pcrespov pcrespov changed the title WIP: Mai/json serializatino Replaces built-in JSON serialization with common_library utilities Apr 23, 2025
@pcrespov pcrespov changed the title Replaces built-in JSON serialization with common_library utilities 🎨 Replaces built-in JSON serialization with common_library utilities Apr 23, 2025
@pcrespov pcrespov marked this pull request as ready for review April 23, 2025 14:06
@pcrespov pcrespov requested a review from bisgaard-itis April 23, 2025 14:07
@pcrespov pcrespov force-pushed the mai/json-serializatino branch from 724236a to 892456b Compare April 23, 2025 14:10
@pcrespov pcrespov added the 🤖-automerge marks PR as ready to be merged for Mergify label Apr 23, 2025
@pcrespov
Copy link
Member Author

@mergify queue

@mergify
Copy link
Contributor

mergify bot commented Apr 23, 2025

queue

🟠 Waiting for conditions to match

  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
      • #approved-reviews-by>=2
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved=0
      • -conflict
      • -draft
      • base=master
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=🤖-do-not-merge
      • label=🤖-automerge
      • any of: [🛡 GitHub branch protection]
        • check-skipped = deploy to dockerhub
        • check-neutral = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of: [🛡 GitHub branch protection]
        • check-success = system-tests
        • check-neutral = system-tests
        • check-skipped = system-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = unit-tests
        • check-neutral = unit-tests
        • check-skipped = unit-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = check OAS' are up to date
        • check-neutral = check OAS' are up to date
        • check-skipped = check OAS' are up to date
      • any of: [🛡 GitHub branch protection]
        • check-success = integration-tests
        • check-neutral = integration-tests
        • check-skipped = integration-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = [build] docker images (excluding frontend) (3.11, ubuntu-24.04)
        • check-neutral = [build] docker images (excluding frontend) (3.11, ubuntu-24.04)
        • check-skipped = [build] docker images (excluding frontend) (3.11, ubuntu-24.04)

@pcrespov pcrespov force-pushed the mai/json-serializatino branch from ed09b5e to 566452d Compare April 23, 2025 21:19
@pcrespov pcrespov enabled auto-merge (squash) April 23, 2025 21:19
@pcrespov pcrespov force-pushed the mai/json-serializatino branch from 0f9d431 to b53900d Compare April 24, 2025 08:45
Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@pcrespov pcrespov force-pushed the mai/json-serializatino branch from b53900d to c9fc66a Compare April 24, 2025 11:12
@sonarqubecloud
Copy link

@pcrespov pcrespov merged commit bb45023 into ITISFoundation:master Apr 24, 2025
93 of 94 checks passed
@pcrespov pcrespov deleted the mai/json-serializatino branch April 28, 2025 12:04
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 8, 2025
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖-automerge marks PR as ready to be merged for Mergify t:maintenance Some planned maintenance work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants