Skip to content

madengine v2 with unified framework for local and distribution #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
coketaste wants to merge 261 commits into
base: main
Choose a base branch
from
Open

madengine v2 with unified framework for local and distribution #57

coketaste wants to merge 261 commits into from
+47,030 −6,167

Conversation

@coketaste
Copy link
Contributor

@coketaste coketaste commented Dec 5, 2025

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

coketaste added 30 commits July 4, 2025 22:20
@coketaste
Fix the perf.csv generation in distributed execution
3e2a44c
@coketaste
Fixed the data which update to perf.csv
8a359ac
@coketaste
Fixed the columns in perf.csv due to parsing issue
ac32cbe
@coketaste
Fix the incorrect regex escaping in the container runner that prevent…
bb6d3fc 
…ed proper performance metric extraction
@coketaste
Update the patterns of performance and metric
a255c50
@coketaste
Fixed the issue of docker_image column in perf.csv
8988508
@coketaste
Improve the interface and reduce erro in registry flow
f0a10a7
@coketaste
Updated the flow of run phase, fix the docker pull, fix the creds ver…
8caae5c 
…ify before docker login
@coketaste
updated the tagged name for docker image and add a docker_image_tagge…
942e666 
…d field to the build manifest
@coketaste
Updated the sequence of operations in build phas
a7baa17
@coketaste
Fixed the registry_image
2e613ca
@coketaste
Update the registry_image
b4e7d22
@coketaste
Updated the process of run phase
d1ecb97
@coketaste
Refactored the file structure of package for distributed_cli
799cce7
@coketaste
Fixed the errors in unit tests
9875fda
@coketaste
Fix the error in unit test of distributed cli
168ffe5
@coketaste
Refactored constants to make design as best practices
756d82a
@coketaste
Cleanup:
9431d7f 
Simplified - No confusing multiple configuration files
Modern - Follows current Python packaging standards (PEP 621)
Maintainable - Single source of truth
Compatible - Works with all modern Python tools
@coketaste
Fixed the regex pattern
cf50f13
@coketaste
Fix ensures that distributed_cli logs will now contain the same detai…
c909a93 
…led system environment information as standard madengine runs, making the logs consistent and comprehensive for debugging and analysis purposes.
@coketaste
Implemented new test cases for pre/post scripts and profiling cases
c04435d
@coketaste
Debug the test cases
72bc7bc
@coketaste
Fixed the test cases in distributed integration
c0dd6ca
@coketaste
Refactor context class to make it work on usages of build-only on cpu…
92db9fb 
… node, run on gpu node; and legecy run with madengine
@coketaste
Update the validation function and GPU detection in additional context
9628a01
@coketaste
tests now automatically detect machine capabilities and skip GPU-depe…
68e19fb 
…ndent tests on CPU-only machines, while avoiding mock context failures on build-only nodes
@coketaste
Create a new madengine CLI application
5dfa775
@coketaste
Fixed the test cases of distrubted integration and profiling
901c12b
@coketaste
Fix the python version compatible issue
6caf244
@coketaste
Fixed the error of model dict
d87e9b0
@coketaste
Updated docs of project refer to recent changes
2fe8ab0
@coketaste
Updated README of project
b28b4f7
@coketaste
Fixed the stack tools for tracing
f3878bc
@coketaste
Fixed the tools stack for gpu info power and gpu info vram profilers
e5094ed
@coketaste
Fixed the error of regex pattern mismatch
3e056cd
@coketaste
Added a model to test theRock and its performance on PyTorch framewor…
123439f 
…k; Added tool of detect_therock
@coketaste
Fixed the error handling to validate the exact bug
404f0ce
@coketaste
Updated docs refer to recent changes
e88b368
@coketaste
Fixed the module combining logs of building and running
04e459a
@coketaste
Fixed the trace shell script
d2bdce5
@coketaste
Updated trace script
2cce1fd
@coketaste
Fixed the SLURM job setup HIP_VISIBLE_DEVICES for Ray/vLLM
70e38c0
@coketaste
Debug the vllm multinode on SLURM
95cd871
@coketaste
Fixed the superset upload to mongodb
ddf7b07
@coketaste
Fixed the unit tests checking hardward capability
c5673e9
@coketaste
Fixed the superset data generation and uploading
4d48e9c
@coketaste
Added new opt to clean perf intermediated files
4587cbf
@coketaste
Merge pull request #61 from ROCm/coketaste/refactor-dis-db
1568d33 
madengine v2 database
@coketaste coketaste changed the title New madengine CLI with unified interface for local and distribution madengine v2 with unified interface for local and distribution Jan 8, 2026
@coketaste coketaste changed the title madengine v2 with unified interface for local and distribution madengine v2 with unified framework for local and distribution Jan 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gargrahul gargrahul Awaiting requested review from gargrahul

@amathews-amd amathews-amd Awaiting requested review from amathews-amd

At least 2 approving reviews are required to merge this pull request.

Assignees

@coketaste coketaste

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@coketaste @Boss2002n