Skip to content

Commit fbef40e

Browse files
anandhu-enganandhu-eng
committed
add documentation page for mlcflow
1 parent f157c0c commit fbef40e

File tree

9 files changed

+425
-0
lines changed

9 files changed

+425
-0
lines changed

‎.github/workflows/publish.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: Publish MLCFlow documentation site
2+
3+
4+
on:
5+
release:
6+
types: [published]
7+
push:
8+
branches:
9+
- main
10+
- dev
11+
12+
jobs:
13+
14+
publish:
15+
name: Publish the site
16+
runs-on: ubuntu-latest
17+
18+
steps:
19+
- name: Checkout repository normally
20+
uses: actions/checkout@v3
21+
22+
- name: Set up Python
23+
uses: actions/setup-python@v4
24+
with:
25+
python-version: "3.11"
26+
27+
- name: Install Mkdocs
28+
run: pip install -r docs/requirements.txt
29+
30+
- name: Run Mkdocs deploy
31+
run: mkdocs gh-deploy --force

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../README.md

‎docs/img/logo_v2.svg‎

Lines changed: 6 additions & 0 deletions
Loading

‎docs/install/install.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
hide:
3+
- toc
4+
---
5+
6+
# Installation
7+
8+
MLCFlow needs `python>=3.7`, `python3-pip` installed on your system.
9+
10+
## Activate a Virtual ENV for MLCFlow (Optional)
11+
This step is not mandatory. But the latest `pip` install requires this or else will need the `--break-system-packages` flag while installing.
12+
13+
```bash
14+
python3 -m venv mlcflow
15+
source mlcflow/bin/activate
16+
```
17+
18+
## Install MLCFLow
19+
=== "Use the latest stable release"
20+
```bash
21+
pip install mlcflow
22+
```
23+
=== "To use mlcflow with latest changes"
24+
Fetches the latest changes developers are working on from the `dev` branch.
25+
```bash
26+
pip install git+https://github.com/mlcommons/mlcflow.git@dev
27+
```
28+
29+
Now, you are ready to use the `mlc` commands. Currently, `mlc` is being used to automating the benchmark runs for:
30+
31+
* [MLCOMMONS MLPerf Inference](https://docs.mlcommons.org/inference/)
32+
* [MLCOMMONS MLPerf Automotive]()
33+
* [MLCOMMONS MLPerf Training]()

‎docs/requirements.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
mkdocs-material
2+
swagger-markdown
3+
mkdocs-macros-plugin
4+
ruamel.yaml
5+
mkdocs-redirects
6+
mkdocs-site-urls

‎docs/targets/cache/index.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Cache
2+
3+
Currently, the following actions are supported for cache:
4+
5+
## Find
6+
7+
`find` action is used to list the path of the cache generated while running scripts through MLC.
8+
9+
**Syntax**
10+
11+
```bash
12+
mlc find cache --tags=<list_of_tags_used_while_running_script>
13+
```
14+
15+
Examples of `find` action for `cache` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
16+
17+
18+
19+
## Show
20+
21+
`show` action is used to list the path and meta data of the cache generated while running scripts through MLC.
22+
23+
**Syntax**
24+
25+
```bash
26+
mlc show cache --tags=<list_of_tags_used_while_running_script>
27+
```
28+
29+
Examples of `show` action for `cache` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
30+
31+
## Rm
32+
33+
`rm` action is used to remove one/more caches generated while running scripts through MLC.
34+
35+
**Syntax**
36+
37+
```bash
38+
mlc rm cache --tags=<list_of_tags_used_while_running_script>
39+
```
40+
41+
A user could delete the entire generated caches through the following command:
42+
43+
```bash
44+
mlc rm cache
45+
```
46+
47+
`-f` could be used to force remove caches. Without `-f`, user would be prompted for confirmation to delete a cache.
48+
49+
Examples of `rm` action for `cache` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
50+

‎docs/targets/repo/index.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Repo
2+
3+
Currently, the following actions are supported for repos:
4+
5+
## Add
6+
7+
`add` action is used to create a new MLC repo and register in MLCFlow. The newly created repo folder would be present inside the `repos` folder present inside the parent `MLC` folder.
8+
9+
**Syntax**
10+
11+
```bash
12+
mlc add repo <repo_name/github_link>
13+
```
14+
15+
Examples of `add` action for `repo` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
16+
17+
## Find
18+
19+
`find` action is used to get the path of a particular repository registered in MLCFlow.
20+
21+
**Syntax**
22+
23+
```bash
24+
mlc find repo <repo_owner@repo_name>
25+
```
26+
27+
OR
28+
29+
```bash
30+
mlc find repo <repo_url>
31+
```
32+
33+
OR
34+
35+
```bash
36+
mlc find repo <repo_uid>
37+
```
38+
39+
OR
40+
41+
```bash
42+
mlc find repo <repo_alias>
43+
```
44+
45+
OR
46+
47+
```bash
48+
mlc find repo <repo_alias>,<repo_uid>
49+
```
50+
51+
Examples of `find` action for `repo` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
52+
53+
## Pull
54+
55+
`pull` action is used to clone a MLC repo and register in MLC.
56+
57+
**Syntax**
58+
59+
```bash
60+
mlc pull repo <repo_owner>@<repo_name>
61+
```
62+
63+
OR
64+
65+
```bash
66+
mlc pull repo <repo_urll>
67+
```
68+
69+
`--checkout` could be added if an user needs to checkout to a specific branch after cloning.
70+
71+
Examples of `pull` action for `repo` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
72+
73+
## List
74+
75+
`list` action is used to list the alias and path of the MLC repos registered in MLC.
76+
77+
**Syntax**
78+
79+
```bash
80+
mlc list repo
81+
```
82+
Example of `list` action for `repo` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
83+
84+
## Rm (Remove)
85+
86+
`rm` action is used to remove the specified repo registered in MLC.
87+
88+
**Syntax**
89+
90+
```bash
91+
mlc rm repo <repo_owner>@<repo_name>
92+
```
93+
Example of `rm` action for `repo` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).

‎docs/targets/script/index.md‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# Script
2+
3+
Currently, the following actions are supported for script:
4+
5+
## Find
6+
7+
`find` action is used to list the path of partiicular scripts present in MLC repos registered in MLC.
8+
9+
**Syntax**
10+
11+
```bash
12+
mlc find script --tags=<list_of_tags_matching_to_particular_script>
13+
```
14+
15+
OR
16+
17+
```bash
18+
mlc find script <script_alias>
19+
```
20+
21+
OR
22+
23+
```bash
24+
mlc find script <script_uid>
25+
```
26+
27+
OR
28+
29+
```bash
30+
mlc find script <script_alias>,<script_uid>
31+
```
32+
33+
Examples of `find` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
34+
35+
## Show
36+
37+
`show` action is used to list the path and meta data of partiicular scripts present in MLC repos registered in MLC.
38+
39+
**Syntax**
40+
41+
```bash
42+
mlc show script --tags=<list_of_tags_matching_to_particular_script>
43+
```
44+
45+
OR
46+
47+
```bash
48+
mlc show script <script_alias>
49+
```
50+
51+
OR
52+
53+
```bash
54+
mlc show script <script_uid>
55+
```
56+
57+
OR
58+
59+
```bash
60+
mlc show script <script_alias>,<script_uid>
61+
```
62+
63+
Examples of `show` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
64+
65+
## Rm
66+
67+
`rm` action is used to remove one/more scripts present in repos which are registered in MLC.
68+
69+
**Syntax**
70+
71+
```bash
72+
mlc rm script --tags=<list_of_tags_matching_to_particular_script>
73+
```
74+
75+
OR
76+
77+
```bash
78+
mlc rm script <script_alias>
79+
```
80+
81+
OR
82+
83+
```bash
84+
mlc rm script <script_uid>
85+
```
86+
87+
`-f` could be used to force remove scripts. Without `-f`, user would be prompted for confirmation to delete a script.
88+
89+
Examples of `rm` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
90+
91+
## Add
92+
93+
`add` script is used to add a new script to any of the registered MLC repos.
94+
95+
**Syntax**
96+
97+
```bash
98+
mlc add script <registered_mlc_repo_name>:<new_script_name> --tags=<set_of_tags> --template=<set_of_tags>
99+
```
100+
101+
* `--tags` contains set of tags to identify the newly created script.
102+
* `--template` contains set of tags of the template script from which we are creating the new script. If not specified, default [template](https://github.com/mlcommons/mlperf-automations/tree/main/script/template-script) would be considered.
103+
* `registered_mlc_repo_name` is of the format `repo_owner`@`repo_name`.
104+
105+
Examples of `add` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
106+
107+
## Mv
108+
109+
`mv` script is used to move a script from source repo to destination repo.
110+
111+
**Syntax**
112+
113+
```bash
114+
mlc mv script <registered_mlc_source_repo_name>:<source_script_name> <registered_mlc_target_repo_name>:<source_script_name>
115+
```
116+
117+
* `registered_mlc_source/target_repo_name` is of the format `repo_owner`@`repo_name`.
118+
119+
Examples of `mv` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
120+
121+
## Cp
122+
123+
`cp` script is used to copy a script from source repo to destination repo.
124+
125+
**Syntax**
126+
127+
```bash
128+
mlc cp script <registered_mlc_source_repo_name>:<source_script_name> <registered_mlc_target_repo_name>:<source_script_name>
129+
```
130+
131+
* `registered_mlc_source/target_repo_name` is of the format `repo_owner`@`repo_name`.
132+
133+
Examples of `c=` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).
134+
135+
## Run
136+
137+
`run` script is used to run scripts from any of the repos registered in MLC.
138+
139+
**Syntax**
140+
141+
```bash
142+
mlc run script --tags=<list_of_tags_matching_to_particular_script> <input_flags>
143+
```
144+
145+
OR
146+
147+
```bash
148+
mlcr <list_of_tags_matching_to_particular_script> <input_flags>
149+
```
150+
151+
* `input_flags` are the additional input that could be given to a particular script. They are specified in the format `--<name_of_input_flag>=<value>`. Some of the examples could be found in run commands from inference documentation [here](https://docs.mlcommons.org/inference/benchmarks/language/gpt-j/).
152+
153+
Examples of `c=` action for `script` target could be found inside the GitHub action workflow [here](https://github.com/mlcommons/mlcflow/blob/d0269b47021d709e0ffa7fe0db8c79635bfd9dff/.github/workflows/test-mlc-core-actions.yaml).

0 commit comments

Comments
 (0)