Skip to content

Commit e4be590

Browse files
konabutakonabuta
authored
Merge pull request #44 from Azure/code-check
code quality check GHA
2 parents 95936c6 + 3c57780 commit e4be590

23 files changed

+275
-32
lines changed

‎.github/workflows/code-quality.yml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: Code Quality (linter, formatter, pre-commit)
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
branches:
8+
- main
9+
schedule:
10+
- cron: "0 0 * * *"
11+
workflow_dispatch:
12+
13+
14+
jobs:
15+
lint-python:
16+
name: Python Lint
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout repository
20+
uses: actions/checkout@v3
21+
- name: Setup python
22+
uses: actions/setup-python@v4.2.0
23+
with:
24+
python-version: "3.9"
25+
- name: Set up python
26+
run: |
27+
pip install -r environments/code_quality.txt
28+
pip list
29+
- name: Lint with flake8
30+
run: |
31+
flake8 .
32+
# - name: Tyepcheck with mypy
33+
# run: |
34+
# mypy .
35+
36+
37+
format-python:
38+
name: Python Format
39+
runs-on: ubuntu-latest
40+
steps:
41+
- name: Checkout repository
42+
uses: actions/checkout@v3
43+
- name: Setup python
44+
uses: actions/setup-python@v4.2.0
45+
with:
46+
python-version: "3.9"
47+
- name: Set up python
48+
run: |
49+
pip install -r environments/code_quality.txt
50+
pip list
51+
- name: Lint with isort
52+
run: |
53+
isort . --check --diff
54+
- name: Lint with black
55+
run: |
56+
black . --check --diff
57+
58+
59+
pre-commit:
60+
name: pre-commit
61+
runs-on: ubuntu-latest
62+
steps:
63+
- uses: actions/checkout@v3
64+
- uses: actions/setup-python@v3
65+
- uses: pre-commit/action@v3.0.0

‎.github/workflows/codeql-analysis.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,10 @@ name: "CodeQL"
1313

1414
on:
1515
push:
16-
branches: [ "dev" ]
16+
branches: [ "main" ]
1717
pull_request:
1818
# The branches below must be a subset of the branches above
19-
branches: [ "dev" ]
19+
branches: [ "main" ]
2020
schedule:
2121
- cron: '45 21 * * 5'
2222

‎.github/workflows/microsoft-security-devops-analysis.yml‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: Microsoft Security DevOps Analysis
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
branches:
8+
- main
9+
schedule:
10+
- cron: "0 0 * * *"
11+
workflow_dispatch:
12+
13+
jobs:
14+
scan-code:
15+
name: Microsoft Security DevOps Analysis
16+
17+
# MSDO runs on windows-latest.
18+
# ubuntu-latest and macos-latest supporting coming soon
19+
runs-on: ubuntu-latest
20+
21+
steps:
22+
23+
# Checkout your code repository to scan
24+
- uses: actions/checkout@v3
25+
26+
# Install dotnet, used by MSDO
27+
- uses: actions/setup-dotnet@v3
28+
with:
29+
dotnet-version: |
30+
3.1.x
31+
5.0.x
32+
6.0.x
33+
34+
# Run analyzers
35+
- name: Run Microsoft Security DevOps Analysis
36+
uses: microsoft/security-devops-action@preview
37+
id: msdo
38+
39+
# Upload alerts to the Security tab
40+
- name: Upload alerts to Security tab
41+
uses: github/codeql-action/upload-sarif@v2
42+
with:
43+
sarif_file: ${{ steps.msdo.outputs.sarifFile }}
44+
45+
# Upload alerts file as a workflow artifact
46+
- name: Upload alerts file as a workflow artifact
47+
uses: actions/upload-artifact@v3
48+
with:
49+
name: alerts
50+
path: ${{ steps.msdo.outputs.sarifFile }}

‎.github/workflows/smoke-testing-azureml.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Smoke Testing for Azure ML
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
branches:
8+
- main
9+
schedule:
10+
- cron: "0 0 * * *"
11+
workflow_dispatch:
12+
13+
14+
jobs:
15+
smoke-testing-azureml-training:
16+
name: Smoke Testing for Azure ML (Training)
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout repository
20+
uses: actions/checkout@v3
21+
- name: Install az ml extension
22+
run: az extension add -n ml -y
23+
- name: Azure login
24+
uses: azure/login@v1
25+
with:
26+
creds: ${{secrets.AZURE_CREDENTIALS}}
27+
- name: Configure default azureml workspace
28+
run: |
29+
az configure --defaults group=${{secrets.RESOURCE_GROUP}} workspace=${{secrets.WORKSPACE_NAME}} location=${{secrets.LOCATION}}
30+
- name: Job for model training
31+
run: |
32+
az ml job create -f train.yml --stream
33+
working-directory: cli/jobs

‎.github/workflows/smoke-testing-notebook.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: Smoke Testing for Notebook
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
branches:
8+
- main
9+
schedule:
10+
- cron: "0 0 * * *"
11+
workflow_dispatch:
12+
13+
14+
jobs:
15+
smoke-testing-notebook:
16+
name: Smoke Testing for Notebook
17+
runs-on: ubuntu-latest
18+
steps:
19+
- name: Checkout repository
20+
uses: actions/checkout@v3
21+
- name: Setup python
22+
uses: actions/setup-python@v4.2.0
23+
with:
24+
python-version: "3.9"
25+
- name: Create conda environment
26+
uses: conda-incubator/setup-miniconda@v2
27+
with:
28+
activate-environment: mlops-train
29+
environment-file: environments/conda_train.yml
30+
- name : Kernel configuration
31+
run: |
32+
python -m ipykernel install --user --name mlops-train
33+
shell: bash -el {0}
34+
- name: Run Notebook for experiment
35+
run: |
36+
papermill train-experiment.ipynb output.ipynb -k mlops-train
37+
working-directory: notebooks
38+
shell: bash -el {0}
39+
- name: Run Notebook for mlflow
40+
run: |
41+
papermill train-mlflow-local.ipynb output.ipynb -k mlops-train
42+
working-directory: notebooks
43+
shell: bash -el {0}
44+
- name: Run Notebook for responsible ai debugging
45+
run: |
46+
papermill train-model-debugging.ipynb output.ipynb -k mlops-train
47+
working-directory: notebooks
48+
shell: bash -el {0}

‎.github/workflows/smoke-testing-python-script.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Smoke Testing for Python Script
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
branches:
8+
- main
9+
schedule:
10+
- cron: "0 0 * * *"
11+
workflow_dispatch:
12+
13+
14+
jobs:
15+
smoke-testing-python-script:
16+
name: Smoke Testing for Python Script
17+
runs-on: ubuntu-latest
18+
env:
19+
INPUT_DATA: './data/raw/nyc_taxi_dataset.csv'
20+
steps:
21+
- name: Checkout repository
22+
uses: actions/checkout@v3
23+
- name: Setup python
24+
uses: actions/setup-python@v4.2.0
25+
with:
26+
python-version: "3.9"
27+
- name: Create conda environment
28+
uses: conda-incubator/setup-miniconda@v2
29+
with:
30+
activate-environment: mlops-train
31+
environment-file: environments/conda_train.yml
32+
- name : Kernel configuration
33+
run: |
34+
python -m ipykernel install --user --name mlops-train
35+
shell: bash -el {0}
36+
- name: Run Python script
37+
run: |
38+
python src/model/train.py --input_data $INPUT_DATA
39+
shell: bash -el {0}
40+
env:
41+
AZUREML_RUN_ID : $GITHUB_RUN_ID

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,4 +135,4 @@ config.json
135135
mlruns/
136136
mlartifacts/
137137
outputs/
138-
mlruns.db
138+
mlruns.db

‎.pre-commit-config.yaml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
repos:
44
# サンプルで生成されるもの (pre-commit sample-config > .pre-commit-config.yaml)
55
- repo: https://github.com/pre-commit/pre-commit-hooks
6-
rev: v4.3.0
6+
rev: v4.4.0
77
hooks:
88
- id: trailing-whitespace
99
- id: no-commit-to-branch
@@ -14,22 +14,22 @@ repos:
1414

1515
# Python 自動フォーマッター
1616
- repo: https://github.com/ambv/black
17-
rev: 22.8.0
17+
rev: 23.1.0
1818
hooks:
1919
- id: black
2020
- id: black-jupyter
2121
language_version: python3
2222

2323
# import 並び替え
2424
- repo: https://github.com/pycqa/isort
25-
rev: 5.10.1
25+
rev: 5.12.0
2626
hooks:
2727
- id: isort
2828
name: isort (python)
2929
args: ["--profile", "black"] # black との競合回避 (他には .isort.cfg にて [tools.isort] profile="black" とする方法もある)
3030

3131
# Python 静的解析ツール
3232
- repo: https://github.com/pycqa/flake8
33-
rev: 5.0.4
33+
rev: 6.0.0
3434
hooks:
3535
- id: flake8

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@ This project welcomes contributions and suggestions. Most contributions require
44

55
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
66

7-
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
7+
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

‎README.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<div align="center">
22
<h1>
3-
<img width="30", src="./docs/images/azureml-icon.svg">
3+
<img width="30", src="./docs/images/azureml-icon.svg">
44
&nbsp;
55
MLOps with Azure Machine Learning
66
</h1>
@@ -16,7 +16,7 @@ Azure Machine Learning + GitHub を利用した MLOps 実装サンプルコー
1616
---
1717

1818
## 👋 概要
19-
本リポジトリは、MLOps のサンプルコードを素早く利用できることを目的に作成されました。Azure Machine Learning と GitHub Actions を利用する想定です。
19+
本リポジトリは、MLOps のサンプルコードを素早く利用できることを目的に作成されました。Azure Machine Learning と GitHub Actions を利用する想定です。
2020

2121

2222
## 🚀 使い方
@@ -84,7 +84,13 @@ Azure Machine Learning + GitHub を利用した MLOps 実装サンプルコー
8484
|Create Online Endpoint (custom) |[cli/endpoints/online_deployment.yml](cli/endpoints/online_deployment.yml)|[scripts/endpoints/deploy-online-endpoint-custom.sh](scripts/endpoints/deploy-online-endpoint-custom.sh)|カスタム型モデルのオンラインエンドポイントへのデプロイ|
8585
|Create Online Endpoint (mlflow) |[cli/endpoints/online_deployment_mlflow.yml](cli/endpoints/online_deployment_mlflow.yml)|[scripts/endpoints/deploy-online-endpoint-mlflow.sh](scripts/endpoints/deploy-online-endpoint-mlflow.sh)|MLflow 型モデルのオンラインエンドポイントへのデプロイ|
8686

87+
88+
### CI/CD Pipeline
89+
90+
>TODO
91+
8792
## 🗒️ ドキュメンテーション
93+
- [クイックスタート](./docs/quickstart.md)
8894
- [Coding Guideline](./docs/coding-guidelines.md)
8995

9096
## 📄 ディレクトリ構造

0 commit comments

Comments
 (0)