Skip to content

Commit 3059d77

Browse files
authored
Merge pull request #171 from mlcommons/dev
Sync Dev
2 parents 9d42227 + c0ba5ea commit 3059d77

File tree

13 files changed

+212
-27
lines changed

13 files changed

+212
-27
lines changed

.github/pull_request_template.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
## ✅ PR Checklist
2+
3+
- [ ] Target branch is `dev`
4+
5+
📌 Note: PRs must be raised against `dev`. Do not commit directly to `main`.
6+
7+
### ✅ Testing & CI
8+
- [ ] Have tested the changes in my local environment, else have properly conveyed in the PR description
9+
- [ ] The change includes a GitHub Action to test the script(if it is possible to be added).
10+
- [ ] No existing GitHub Actions are failing because of this change.
11+
12+
### 📚 Documentation
13+
- [ ] README or help docs are updated for new features or changes.
14+
- [ ] CLI help messages are meaningful and complete.
15+
16+
### 📁 File Hygiene & Output Handling
17+
- [ ] No unintended files (e.g., logs, cache, temp files, __pycache__, output folders) are committed.
18+
19+
### 🛡️ Safety & Security
20+
- [ ] No secrets or credentials are committed.
21+
- [ ] Paths, shell commands, and environment handling are safe and portable.
22+
23+
### 🙌 Contribution Hygiene
24+
- [ ] PR title and description are concise and clearly state the purpose of the change.
25+
- [ ] Related issues (if any) are properly referenced using `Fixes #` or `Closes #`.
26+
- [ ] All reviewer feedback has been addressed.

.github/workflows/codeql.yml

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# For most projects, this workflow file will not need changing; you simply need
2+
# to commit it to your repository.
3+
#
4+
# You may wish to alter this file to override the set of languages analyzed,
5+
# or to provide custom queries or build logic.
6+
#
7+
# ******** NOTE ********
8+
# We have attempted to detect the languages in your repository. Please check
9+
# the `language` matrix defined below to confirm you have the correct set of
10+
# supported CodeQL languages.
11+
#
12+
name: "CodeQL Advanced"
13+
14+
on:
15+
push:
16+
branches: [ "main" ]
17+
pull_request:
18+
branches: [ "main" ]
19+
schedule:
20+
- cron: '43 6 * * 0'
21+
22+
jobs:
23+
analyze:
24+
name: Analyze (${{ matrix.language }})
25+
# Runner size impacts CodeQL analysis time. To learn more, please see:
26+
# - https://gh.io/recommended-hardware-resources-for-running-codeql
27+
# - https://gh.io/supported-runners-and-hardware-resources
28+
# - https://gh.io/using-larger-runners (GitHub.com only)
29+
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
30+
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
31+
permissions:
32+
# required for all workflows
33+
security-events: write
34+
35+
# required to fetch internal or private CodeQL packs
36+
packages: read
37+
38+
# only required for workflows in private repositories
39+
actions: read
40+
contents: read
41+
42+
strategy:
43+
fail-fast: false
44+
matrix:
45+
include:
46+
- language: actions
47+
build-mode: none
48+
- language: python
49+
build-mode: none
50+
# CodeQL supports the following values keywords for 'language': 'actions', 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'rust', 'swift'
51+
# Use `c-cpp` to analyze code written in C, C++ or both
52+
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
53+
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
54+
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
55+
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
56+
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
57+
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
58+
steps:
59+
- name: Checkout repository
60+
uses: actions/checkout@v4
61+
62+
# Add any setup steps before running the `github/codeql-action/init` action.
63+
# This includes steps like installing compilers or runtimes (`actions/setup-node`
64+
# or others). This is typically only required for manual builds.
65+
# - name: Setup runtime (example)
66+
# uses: actions/setup-example@v1
67+
68+
# Initializes the CodeQL tools for scanning.
69+
- name: Initialize CodeQL
70+
uses: github/codeql-action/init@v3
71+
with:
72+
languages: ${{ matrix.language }}
73+
build-mode: ${{ matrix.build-mode }}
74+
# If you wish to specify custom queries, you can do so here or in a config file.
75+
# By default, queries listed here will override any specified in a config file.
76+
# Prefix the list here with "+" to use these queries and those in the config file.
77+
78+
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
79+
# queries: security-extended,security-and-quality
80+
81+
# If the analyze step fails for one of the languages you are analyzing with
82+
# "We were unable to automatically build your code", modify the matrix above
83+
# to set the build mode to "manual" for that language. Then modify this step
84+
# to build your code.
85+
# ℹ️ Command-line programs to run using the OS shell.
86+
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
87+
- if: matrix.build-mode == 'manual'
88+
shell: bash
89+
run: |
90+
echo 'If you are using a "manual" build mode for one or more of the' \
91+
'languages you are analyzing, replace this with the commands to build' \
92+
'your code, for example:'
93+
echo ' make bootstrap'
94+
echo ' make release'
95+
exit 1
96+
97+
- name: Perform CodeQL Analysis
98+
uses: github/codeql-action/analyze@v3
99+
with:
100+
category: "/language:${{matrix.language}}"

.github/workflows/mlperf-inference-bert.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ name: MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)
22

33
on:
44
pull_request:
5-
branches: [ "main", "dev" ]
5+
branches: [ "main_off", "dev_off" ]
66
paths:
77
- '.github/workflows/test-mlperf-inference-bert-deepsparse-tf-onnxruntime-pytorch.yml'
88
- '**'

VERSION

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.0.8
1+
1.1.0

docs/install/index.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,11 @@ This step is not mandatory. But the latest `pip` install requires this or else w
4848

4949
=== "Windows"
5050
```bash
51-
python3 -m venv mlcflow
51+
python -m venv mlcflow
5252
mlcflow\Scripts\activate.bat
53+
```
54+
Run as Administrator
55+
```bash
5356
git config --system core.longpaths true
5457
```
5558

@@ -60,9 +63,14 @@ If you are not using virtual ENV for installation, the latest `pip` install requ
6063
```bash
6164
pip install mlcflow
6265
```
63-
66+
## Pull the Automation Repo
67+
```bash
68+
mlc pull repo mlcommons@mlperf-automations
69+
```
70+
* If you are forking https://github.com/mlcommons/mlperf-automations you can substitute the above command by `mlc pull repo <your_github_username>@mlperf-automations`.
71+
6472
!!! tip
65-
If you want to pull the latest changes (recommended), please do `mlc pull repo` after the installation.
73+
If you want to pull the latest changes (recommended), please do `mlc pull repo` periodically.
6674

6775

6876
Now, you are ready to use the `mlc` commands. Currently, `mlc` is being used to automate the benchmark runs for:

docs/targets/script/execution-flow.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,14 +36,14 @@ When we run an MLC script we can also pass inputs to it and any input added in `
3636
We can use `skip_if_env` dictionary inside any `deps`, `prehook_deps`, `posthook_deps` or `post_deps` to make its execution conditional
3737

3838
### Versions
39-
We can specify any specific version of a script using `version`. `version_max` and `version_min` are also possible options.
39+
We can specify any specific version of a script using `version`. `version_max`, `version_min` and `version_max_usable` are also possible options.
4040

41-
* When `version_min` is given, any version above this if present in the cache or detected in the system can be chosen. If nothing is detected `default_version` if present and if above `version_min` will be used for installation. Otherwise `version_min` will be used as `version`.
41+
* When `version_min` is given, any version above this if present in the cache or detected in the system can be chosen. If nothing is detected `default_version` if present and if above `version_min` will be used for installation. Otherwise `version_max_usable` if present or else `version_min` will be used as `version`.
4242

4343
* When `version_max` is given, any version below this if present in the cache or detected in the system can be chosen. If nothing is detected `default_version` if present and if below `version_max` will be used for installation. Otherwise `version_max_usable` (additional needed input for `version_max`) will be used as `version`.
44-
44+
4545
### Variations
46-
* Variations are used to customize MLC script and each unique combination of variations uses a unique cache entry. Each variation can turn on `env` keys also any other meta including dependencies specific to it. Variations are turned on like tags but with a `_` prefix. For example, if a script is having tags `"get,myscript"`, to call the variation `"test"` inside it, we have to use tags `"get,myscript,_test"`.
46+
* Variations are used to customize MLC script and each set of variations uses a unique cache entry. A variation can turn on `env` keys and any other meta including dependencies specific to it. Variations are turned on like tags but with a `_` prefix. For example, if a script is having tags `"get,myscript"`, to call the variation `"test"` inside it, we can use tags `"get,myscript,_test"`.
4747

4848
#### Variation groups
4949
`group` is a key to map variations into a group and at any time only one variation from a group can be used in the variation tags. For example, both `cpu` and `cuda` can be two variations under the `device` group, but user can at any time use either `cpu` or `cuda` as variation tags but not both.
@@ -63,6 +63,7 @@ Sometimes it is difficult to add all variations needed for a script like say `ba
6363
* Similarly we can avoid any env key from being passed to a given dependency by adding the prefix of the key in the `clean_env_keys` list of the concerned dependency.
6464
* `--input` is automatically converted to `MLC_INPUT` env key
6565
* `version` is converted to `MLC_VERSION`, ``version_min` to `MLC_VERSION_MIN` and `version_max` to `MLC_VERSION_MAX`
66+
* `mlc
6667
* If `env['MLC_GH_TOKEN']=TOKEN_VALUE` is set then git URLs (specified by `MLC_GIT_URL`) are changed to add this token.
6768
* If `env['MLC_GIT_SSH']=yes`, then git URLs are changed to SSH from HTTPS.
6869

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
site_name: MLCFlow Documentation
22
repo_url: https://github.com/mlcommons/mlcflow
3+
site_url: https://docs.mlcommons.org/mlcflow
34
theme:
45
name: material
56
logo: img/logo_v2.svg

mlc/action.py

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,6 @@ class Action:
2323
logger = None
2424
local_repo = None
2525
current_repo_path = None
26-
#mlc = None
2726
repos = [] #list of Repo objects
2827

2928
# Main access function to simulate a Python interface for CLI
@@ -167,7 +166,7 @@ def load_repos(self):
167166

168167

169168
def __init__(self):
170-
setup_logging(log_path=os.getcwd(),log_file='mlc-log.txt')
169+
setup_logging(log_path=os.getcwd(), log_file='.mlc-log.txt')
171170
self.logger = logger
172171

173172
temp_repo = os.environ.get('MLC_REPOS','').strip()
@@ -654,6 +653,7 @@ def search(self, i):
654653
uid = i.get("uid")
655654
alias = i.get("alias")
656655
item_repo = i.get('item_repo')
656+
exact_tags_match = i.get('exact_tags_match', False)
657657
fetch_all = True if i.get('fetch_all') else False
658658

659659
# For targets like cache, sometimes user would need to clear the entire cache folder present in the system
@@ -707,13 +707,13 @@ def search(self, i):
707707
for res in target_index:
708708
if os.path.basename(res["path"]) == folder_name:
709709
it = Item(res['path'], res['repo'])
710-
#result.append(it)
710+
result.append(it)
711711
else:
712712
tags = i.get("tags")
713713
if tags:
714714
tags_split = tags.split(",")
715715
else:
716-
return {"return":1, "error": f"Tags are not specifeid for completing the specific action"}
716+
return {"return":1, "error": f"Tags are not specified for completing the requested action"}
717717
if target == "script":
718718
non_variation_tags = [t for t in tags_split if not t.startswith("_")]
719719
tags_to_match = non_variation_tags
@@ -726,7 +726,7 @@ def search(self, i):
726726
p_tags = list(set(tags_to_match) - set(n_tags_))
727727
for res in target_index:
728728
c_tags = res["tags"]
729-
if set(p_tags).issubset(set(c_tags)) and set(n_tags).isdisjoint(set(c_tags)):
729+
if (exact_tags_match and set(p_tags) == set(c_tags)) or (not exact_tags_match and set(p_tags).issubset(set(c_tags)) and set(n_tags).isdisjoint(set(c_tags))):
730730
it = Item(res['path'], res['repo'])
731731
result.append(it)
732732
return {'return': 0, 'list': result}

mlc/logger.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ def format(self, record):
2121

2222

2323
# Set up logging configuration
24-
def setup_logging(log_path = os.getcwd(),log_file = 'mlc-log.txt'):
24+
def setup_logging(log_path = os.getcwd(), log_file = '.mlc-log.txt'):
2525

2626
if not logger.hasHandlers():
2727
logFormatter = ColoredFormatter('[%(asctime)s %(filename)s:%(lineno)d %(levelname)s] - %(message)s')

mlc/main.py

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -64,11 +64,12 @@ def search(self, i):
6464
uid = item_split[1]
6565
else:
6666
uid = item_split[0]
67-
for res in target_index:
68-
c_tags = res["tags"]
69-
if set(p_tags).issubset(set(c_tags)) and set(n_tags).isdisjoint(set(c_tags)) and (not uid or uid == res['uid']) and (not alias or alias == res['alias']):
70-
it = Item(res['path'], res['repo'])
71-
result.append(it)
67+
if tags or uid or i.get('all'):
68+
for res in target_index:
69+
c_tags = res["tags"]
70+
if set(p_tags).issubset(set(c_tags)) and set(n_tags).isdisjoint(set(c_tags)) and (not uid or uid == res['uid']) and (not alias or alias == res['alias']):
71+
it = Item(res['path'], res['repo'])
72+
result.append(it)
7273
#logger.info(result)
7374
return {'return': 0, 'list': result}
7475
#indices
@@ -152,7 +153,7 @@ def main():
152153
action_parser.add_argument('extra', nargs=argparse.REMAINDER, help='Extra options (e.g., -v)')
153154

154155
# Script specific subcommands
155-
for action in ['docker', 'experiment']:
156+
for action in ['docker', 'experiment', 'doc', 'lint']:
156157
action_parser = subparsers.add_parser(action, help=f'{action.capitalize()} a target.')
157158
action_parser.add_argument('target', choices=['script', 'run'], help='Target type (script).')
158159
# the argument given after target and before any extra options like --tags will be stored in "details"
@@ -247,7 +248,7 @@ def main():
247248
if args.target == "repo":
248249
run_args['repo'] = args.details
249250

250-
if args.command in ['docker', 'experiment']:
251+
if args.command in ['docker', 'experiment', 'doc', 'lint']:
251252
if args.target == "run":
252253
run_args['target'] = 'script' #allowing run to be used for docker run instead of docker script
253254
args.target = "script"

0 commit comments

Comments
 (0)