Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 10 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,22 @@
## Unified and cross-platform CM interface for DevOps, MLOps and MLPerf
# Legacy CM4MLOps repository with DevOps, MLOps and MLPerf automations

[![License](https://img.shields.io/badge/License-Apache%202.0-green)](LICENSE.md)
[![Powered by CM](https://img.shields.io/badge/Powered_by-MLCommons%20CM-blue)](https://pypi.org/project/cmind).

[![CM script automation features test](https://github.com/mlcommons/cm4mlops/actions/workflows/test-cm-script-features.yml/badge.svg)](https://github.com/mlcommons/cm4mlops/actions/workflows/test-cm-script-features.yml)
[![MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)](https://github.com/mlcommons/cm4mlops/actions/workflows/test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml/badge.svg)](https://github.com/mlcommons/cm4mlops/actions/workflows/test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml)
[![MLPerf inference MLCommons C++ ResNet50](https://github.com/mlcommons/cm4mlops/actions/workflows/test-mlperf-inference-mlcommons-cpp-resnet50.yml/badge.svg)](https://github.com/mlcommons/cm4mlops/actions/workflows/test-mlperf-inference-mlcommons-cpp-resnet50.yml)
[![MLPerf inference ABTF POC Test](https://github.com/mlcommons/cm4mlops/actions/workflows/test-mlperf-inference-abtf-poc.yml/badge.svg)](https://github.com/mlcommons/cm4mlops/actions/workflows/test-mlperf-inference-abtf-poc.yml)

# CM4MLOps repository

This repository is powered by the [Collective Mind workflow automation framework](https://github.com/mlcommons/ck/tree/master/cm).

The latest sources are available in [this repository](https://github.com/mlcommons/ck/tree/master/cm4mlops).

Two key automations developed using CM are **Script** and **Cache**, which streamline machine learning (ML) workflows,
including managing Docker runs. Both Script and Cache automations are part of the **cmx4mlops** repository.
including managing Docker runs. Both Script and Cache automations are part of the **cm4mlops** repository.

The [CM scripts](https://access.cknowledge.org/playground/?action=scripts),
also housed in this repository, consist of hundreds of modular Python-wrapped scripts accompanied
by `yaml` metadata, enabling the creation of robust and flexible ML workflows.

- **CM Scripts Documentation**: [https://docs.mlcommons.org/cm4mlops/](https://docs.mlcommons.org/cm4mlops/)
- **CM CLI Documentation**: [https://docs.mlcommons.org/ck/specs/cm-cli/](https://docs.mlcommons.org/ck/specs/cm-cli/)
- **CM Scripts Documentation**: [Browse](https://access.cknowledge.org/playground/?action=scripts)
- **CM CLI Documentation**: [https://docs.mlcommons.org/ck/specs/cm-cli/](https://docs.mlcommons.org/ck/specs/cm-cli)

## License

Expand All @@ -38,31 +34,22 @@ Grigori Fursin, the cTuning foundation and OctoML donated the CK and CM projects

## Author

[Grigori Fursin](https://cKnowledge.org/gfursin)
[Grigori Fursin](https://cKnowledge.org/gfursin).

We sincerely appreciate all [contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTORS.md)
We thank all [contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTORS.md)
for their invaluable feedback and support!

## Concepts

Check our [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339) and the [white paper](https://arxiv.org/abs/2406.16791).

## Test image classification and MLPerf R-GAT inference benchmark via CMX PYPI package

```bash
pip install cmind
pip install cmx4mlops
cmx run script "python app image-classification onnx" --quiet
cmx run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="MLCommons" --adr.inference-src.tags=_branch.dev --pull_changes=yes --pull_inference_changes=yes --submitter="MLCommons" --hw_name=ubuntu-latest_x86 --model=rgat --implementation=python --backend=pytorch --device=cpu --scenario=Offline --test_query_count=500 --adr.compiler.tags=gcc --category=datacenter --quiet --v --target_qps=1
```

## Test image classification and MLPerf R-GAT inference benchmark via CMX GitHub repo

```bash
pip uninstall cmx4mlops
pip install cmind
cmx pull repo mlcommons@ck --dir=cmx4mlops/cmx4mlops
cmx pull repo mlcommons@ck --dir=cm4mlops/cm4mlops
cmx run script "python app image-classification onnx" --quiet
cmx run script "run-mlperf inference _performance-only _short" --model=resnet50 --precision=float32 --backend=onnxruntime --scenario=Offline --device=cpu --env.CM_SUDO_USER=no --quiet
cmx run script --tags=run,mlperf,inference,generate-run-cmds,_submission,_short --submitter="MLCommons" --adr.inference-src.tags=_branch.dev --pull_changes=yes --pull_inference_changes=yes --submitter="MLCommons" --hw_name=ubuntu-latest_x86 --model=rgat --implementation=python --backend=pytorch --device=cpu --scenario=Offline --test_query_count=500 --adr.compiler.tags=gcc --category=datacenter --quiet --v --target_qps=1
```

Expand Down
2 changes: 2 additions & 0 deletions automation/script/module.py
Original file line number Diff line number Diff line change
Expand Up @@ -6558,7 +6558,9 @@ def dump_repro(repro_prefix, rr, run_state):
cm_output['version_info'] = version_info

if rr['return'] == 0:

# See https://cTuning.org/ae

cm_output['acm_ctuning_repro_badge_available'] = True
cm_output['acm_ctuning_repro_badge_functional'] = True

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def main():
path_to_model_pth = os.environ['CM_ML_MODEL_FILE_WITH_PATH']

model = models.resnet50(pretrained=False)
model.load_state_dict(torch.load(path_to_model_pth))
model.load_state_dict(torch.load(path_to_model_pth, weights_only=False))

model.eval()

Expand Down
Loading