Skip to content

Comments

[media] build / bugfix : upgrade infra , 3rd party libraries, move build test flow to docker#54

Merged
metalalive merged 41 commits intomasterfrom
media-dev/build/upgrade-infra-docker
Nov 2, 2025
Merged

[media] build / bugfix : upgrade infra , 3rd party libraries, move build test flow to docker#54
metalalive merged 41 commits intomasterfrom
media-dev/build/upgrade-infra-docker

Conversation

@metalalive
Copy link
Owner

@metalalive metalalive commented Nov 1, 2025

  • CMake and docker config
    • build executable dev and test apps in docker build
    • move the commands that launch executable dev and test apps to docker compose
    • move database migration, certificate renewal, and other helper tools to docker compose
  • run unit test / integration test through docker compose
  • use separate docker images for helper tools and executable server applications
  • standalone health check program for app server
  • upgrade mariadb , rabbitmq, gcc, and essential 3rd-party libraries

bugfix :

  • pass environment variables to development and test apps
    • allow system users to specify essential environment variables for running applications in different OS environment
    • specify database host and port by environment variable instead of secret data file.
  • add macro which concatenates 2 path segments then run user-specific code block with the final concatenated string, the purpose is to ensure full path of a file in local / remote storage.
  • fix base path issues for external files
    • associated with callable symbol lookup in ELF file , config file parsing, persisted user-asset storage, and image / video transcoding modules.
    • system base path will be handled in storage layer, remove system base path on API endpoints of app server and RPC consumer
  • for temp buffer in local transcoding RPC consumer
    • replace path of temp buffer, with dedicated storage setup in configuration.
    • remove confusing storage alias localfs, insead using persist_usr_asset for user media file persistence, and local_tmpbuf as temporary buffer files when transcoding videos / images locally in app server
  • for integration test
    • modify URL verification during HLS element lookup in relevant test cases
  • fix errors in mock auth servers

others :

  • reformat python modules for build and fix path issues
  • update README and build help doc
  • update github action workflow, run test with docker build / compose
  • probe object inside json array using standalone json_renderer.py for test

…Make config

- compile applications and test programs during docker build
- move database migration to docker compose
- run unit test through docker compose
- TODO, finish integration test setup with docker compose

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- allow system users to specify essential environment variables for
  running applications in different OS environment
- specify database host and port by environment variable instead of
  secret data file.
- add macro which concatenates 2 path segments then run user-specific
  code block with the final concatenated string, the purpose is to
  ensure full path of a file in local / remote storage.
- fix base path issues for external files such as image mask pattern,
  callable symbol lookup in ELF file and config file parsing.
- TODO, fix base path issues for persisted storage files and transcoding
  / streaming media segment files

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- fix base path issues for persisted storage files
- modify unit test cases

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- fix base path issues for file procssing parts in transcoder
- modify unit test cases

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
… poll

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- unit test will be run by docker compose with valgrind memory check, and
  no longer be part of cmake script
- update docker commands to README

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…h issues

- ensure correct path to certificates in docker container when running
  app server

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- separate docker images for helper tools and executable applications

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- correct base path issues in config modules
- URL to AMQP broker is HTTP instead of HTTPS for directly connecting to
  user-management Django development server.
  - TODO, consider to support HTTPS endpoints on user-management app server
- modify host / port of AMQP broker in RPC config module

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- this is part of integration test flow
- support HTTP2 secure connection which will interact with media test
  app server

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…r compose

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…ver for integration test

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…cker compose

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- system base path will be handled in storage layer, remove system base
  path on API endpoints of  app server and RPC consumer

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
… filesystem

- replace path of temp buffer, with dedicated storage setup in
  configuration.
- remove confusing storage alias `localfs`, insead using `persist_usr_asset`
  for user media file persistence, and `local_tmpbuf` as temporary
  buffer files when transcoding videos / images locally in app server
- remove field `tmp_buf.path` from `app_cfg_t`

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- fix base paths against ffmpeg library functions for HLS file
  generation
- modify URL verification during HLS element lookup in relevant
  test cases

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- ensure correct base paths against ffmpeg library functions for
  image input / output file generation
- fix incorrect external path to mask file during image filtering
  process

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…in transcode / storage module

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…ation test

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
- chore , modify CORS config files and gitignore
- modify corresponding app setting file for dev env

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…mpose

- update doc , unit test can be running without `interpolation-test.env`
  in docker compose

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
@metalalive metalalive self-assigned this Nov 1, 2025
@metalalive metalalive added the bug Something isn't working label Nov 1, 2025
@metalalive metalalive added build issues related to build tools devlopment environment media test labels Nov 1, 2025
@gitguardian
Copy link

gitguardian bot commented Nov 1, 2025

⚠️ GitGuardian has uncovered 4 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
19482685 Triggered Generic Password 87a32c1 services/media/infra/container-sample.env View secret
19482686 Triggered Generic Password 87a32c1 services/media/infra/container-sample.env View secret
10609120 Triggered Generic Password 367e0eb services/media/src/models/cfg_parser.c View secret
19482687 Triggered Generic Password 87a32c1 services/media/infra/container-sample.env View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…tion

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
…nderer.py

Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
Signed-off-by: T.H. <7197142+metalalive@users.noreply.github.com>
@metalalive metalalive merged commit beef058 into master Nov 2, 2025
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working build issues related to build tools devlopment environment media test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant