Skip to content

Commit a2f4027

Browse files
jgchnjgchn
committed
Pr pipeline 2 (#5)
* Cleaning up run examples pipeline Signed-off-by: Jing Chen <[email protected]> Rm run examples composite Signed-off-by: Jing Chen <[email protected]> Halt early if no file patch Signed-off-by: Jing Chen <[email protected]> Better msg Signed-off-by: Jing Chen <[email protected]> Modify some PDL files Signed-off-by: Jing Chen <[email protected]> Revert revisions Signed-off-by: Jing Chen <[email protected]> Modify some PDL programs Signed-off-by: Jing Chen <[email protected]> Fix if Signed-off-by: Jing Chen <[email protected]> Composite run examples Signed-off-by: Jing Chen <[email protected]> Checkout before using composite Test if Test if again Rm bash test Fix unfound path Update if check Delete a folder and see if test runs Bring back deleted example. Expects test to run Test if examples config is patched Mv yaml patch to composite action Try another patch strategy Change some files Checkout again in run examples Modify files Typo in composite action Use json in check parse Try to escape json Use changes instead in yaml action Bring back Ollama set up Gather new results from GH env Clean up workflows Fix lint Modify chatbot example Fix run examples config file * Add possible results for react examples * Preserve trailing white space * See if a commit is pushed on my behalf * Rename config file * Diff pdl file * Check if commit happens despite pytet failures * Allow pytest failure and continue run * github-actions[bot]: Updated results file when running examples on Mon Apr 7 23:24:42 UTC 2025 Signed-off-by: github-actions[bot] <[email protected]> * Set max parallel to 1 to prevent race conditions * See if yaml action is triggered * Change master branch * Fix pytest * Set update results to true * Test yaml actio again * Update result to true * Reverting update_results to False. If you wish to update again, change locally and push with change. * WRite permissions * Reverting update_results to False. If you wish to update again, change locally and push with change. * Don't commit using yaml action * Set update results to true * github-actions[bot]: Updated results file when running examples on Tue Apr 8 20:11:04 UTC 2025 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Reverting update_results to False. Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Try pytest failure * Test pytest behavior again * Test set +e * Test again * Set update to true to see if changed-files can capture results * PR test * github-actions[bot]: Updated results file when running examples Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * github-actions[bot]: Reverting update_result to false Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Test only with PR, not push * github-actions[bot]: Updated results file when running examples Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * github-actions[bot]: Reverting update_result to false Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * No pdl file change * Clean up and fix run examples nightly Signed-off-by: Jing Chen <[email protected]> * Fix empty parse * Fix * Bring back more models on ollama set up * Contrib readme updates --------- Signed-off-by: github-actions[bot] <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Jing Chen <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <75508867+github-actions[bot][email protected]> Get new results for failing progs github-actions[bot]: Run examples: updated result files on your behalf Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> github-actions[bot]: Run examples: reverting update_result to false on your behalf Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> See if ollama is cached Try caching ollama Set up Ollama outside composite action Revert changes Update runexamples config
1 parent 73b2b7d commit a2f4027

39 files changed

+859
-319
lines changed

‎.github/actions/ollama/action.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: 'Ollama Setup'
2+
description: 'Composte action for Ollama set up in GH environment'
3+
runs:
4+
using: 'composite'
5+
steps:
6+
- name: Remove unnecessary files
7+
shell: bash
8+
run: |
9+
sudo rm -rf /usr/share/dotnet
10+
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
11+
12+
# Set up Ollama
13+
- name: Install Ollama and start server
14+
shell: bash
15+
run: |
16+
curl -fsSL https://ollama.com/install.sh | sudo -E sh
17+
18+
- name: Pull models in examples/
19+
shell: bash
20+
run: |
21+
ollama pull granite3.2:2b
22+
ollama pull granite3.2:8b
23+
ollama pull mxbai-embed-large
24+
ollama list
25+
26+
- name: Check that all required models are available
27+
shell: bash
28+
run: |
29+
models=("mxbai-embed-large" "granite3.2:2b" "granite3.2:8b")
30+
missing=0
31+
for model in "${models[@]}"; do
32+
if ! ollama list | awk 'NR>1 {print $1}' | grep -q "$model"; then
33+
echo "❌ Model $model is missing!"
34+
missing=1
35+
fi
36+
done
37+
38+
if [ "$missing" -eq 1 ]; then
39+
exit 1
40+
else
41+
echo "✅ All expected models are available."
42+
fi
43+
44+
- name: Wait for Ollama server
45+
shell: bash
46+
run: |
47+
sleep 5
48+
time curl -i http://localhost:11434

‎.github/actions/run-examples/action.yml‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
name: 'Ollama Setup'
2+
description: 'Composte action to set up Run Examples'
3+
inputs:
4+
python-version:
5+
description: 'Python version'
6+
required: true
7+
default: '3.11'
8+
runner-os:
9+
description: 'Runner OS'
10+
required: true
11+
head-ref:
12+
description: 'Head ref of the repo'
13+
required: true
14+
update-results:
15+
description: 'Whether to update the results for this run. Must be false for nightly runs'
16+
required: true
17+
check:
18+
description: 'Files to patch tests/test_examples_run.yaml with. These are the PDL files that the test will run against. Defaults to all PDL files.'
19+
required: false
20+
default: '[]'
21+
runs:
22+
using: 'composite'
23+
steps:
24+
# # Set up Ollama
25+
- uses: ./.github/actions/ollama
26+
27+
# Configure Run Examples environment
28+
- uses: actions/checkout@v4
29+
with:
30+
ref: ${{ inputs.head-ref }}
31+
fetch-depth: 0 # Full history needed for fetch/rebase
32+
- name: Patch tests/test_examples_run.yaml check with modified files
33+
uses: fjogeleit/yaml-update-action@main
34+
with:
35+
changes: |
36+
{
37+
"tests/test_examples_run.yaml": {
38+
"check": ${{ inputs.check }}
39+
}
40+
}
41+
commitChange: false
42+
- name: Print test Run Examples config
43+
shell: bash
44+
run: cat tests/test_examples_run.yaml
45+
46+
# Run tests
47+
- name: Set up Python ${{ inputs.python-version }}
48+
uses: actions/setup-python@v5
49+
with:
50+
python-version: ${{ inputs.python-version }}
51+
- name: Cache pip
52+
uses: actions/cache@v4
53+
with:
54+
# This path is specific to Ubuntu
55+
path: ${{ env.pythonLocation }}
56+
# Look to see if there is a cache hit for the setup file
57+
key: ${{ inputs.runner-os }}-pip-new3-${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}
58+
restore-keys: |
59+
${{ inputs.runner-os }}-pip-new3
60+
${{ inputs.runner-os }}-new3
61+
- name: Install dependencies
62+
shell: bash
63+
run: pip install --upgrade --upgrade-strategy eager .[all]
64+
- name: Pip list packages
65+
shell: bash
66+
run: pip list
67+
- name: Run Pytest
68+
shell: bash
69+
run: |
70+
cat tests/test_examples_run.yaml
71+
(
72+
set +e
73+
py.test -v --capture=tee-sys -rfE -s tests/test_examples_run.py --disable-pytest-warnings
74+
EXIT_CODE=$?
75+
76+
if [ $EXIT_CODE -eq 0 ]; then
77+
echo "TEST_RESULT=PASSED" >> $GITHUB_ENV
78+
else
79+
echo "TEST_RESULT=FAILED" >> $GITHUB_ENV
80+
fi
81+
)
82+
83+
# Commit the results if update results
84+
- name: Push new results to branch
85+
shell: bash
86+
if: ${{ inputs.update-results == 'true' }}
87+
run: |
88+
git config --local user.name github-actions[bot]
89+
git config --local user.email 41898282+github-actions[bot]@users.noreply.github.com
90+
git status
91+
git pull
92+
git add tests/results/
93+
git diff --cached --quiet || git commit -s -m "github-actions[bot]: Run examples: updated result files on your behalf"
94+
git push
95+
96+
- name: Check if pytest passed
97+
shell: bash
98+
run: |
99+
if [ "$TEST_RESULT" == "PASSED" ]; then
100+
exit 0
101+
else
102+
exit 1
103+
fi

‎.github/workflows/run-examples-prep.yml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
name: Run examples on modified PDL files
3+
on: [pull_request]
4+
jobs:
5+
tests:
6+
name: Execution tests
7+
runs-on: ubuntu-latest
8+
permissions:
9+
contents: write
10+
strategy:
11+
fail-fast: false
12+
max-parallel: 1
13+
matrix:
14+
python-version: ['3.11', '3.12', '3.13']
15+
steps:
16+
# Detect modified PDL files, includes Add, Modified, but not Deleted
17+
- uses: actions/checkout@v4
18+
with:
19+
fetch-depth: 0
20+
- name: Detect all PDL files that were changed or added
21+
id: changed-pdl-files
22+
uses: tj-actions/changed-files@6cb76d07bee4c9772c6882c06c37837bf82a04d3 # v46
23+
with:
24+
files: |
25+
**.pdl
26+
json: "true"
27+
escape_json: "false"
28+
- name: List PDL files that were modified or added and append to test_examples_run
29+
env:
30+
MODIFIED_PDL_FILES: ${{ steps.changed-pdl-files.outputs.all_changed_files }}
31+
run: echo "$MODIFIED_PDL_FILES"
32+
33+
# Run tests if there are modified PDL files
34+
- uses: actions/checkout@v4
35+
if: ${{ steps.changed-pdl-files.outputs.all_changed_files_count > 0 }}
36+
with:
37+
ref: ${{ github.head_ref }}
38+
fetch-depth: 0
39+
40+
- name: Check if update results automatically by GitHub Actions Bot
41+
id: parse-run-examples-config
42+
shell: bash
43+
run: |
44+
value=$(yq '.update_results' tests/test_examples_run.yaml | tr '[:upper:]' '[:lower:]')
45+
echo "value=$value"
46+
echo "update_results=$value" >> $GITHUB_OUTPUT
47+
48+
- uses: ./.github/actions/run-examples
49+
if: ${{ steps.changed-pdl-files.outputs.all_changed_files_count > 0 }}
50+
with:
51+
python-version: ${{ matrix.python-version }}
52+
runner-os: ${{ runner.os }}
53+
update-results: ${{ steps.parse-run-examples-config.outputs.update_results }}
54+
check: ${{ steps.changed-pdl-files.outputs.all_changed_files }}
55+
head-ref: ${{ github.head_ref }}
56+
57+
# Patch config with update_results to false
58+
# Fetch the latest branch
59+
- uses: actions/checkout@v4
60+
if: ${{ steps.parse-run-examples-config.outputs.update_results && matrix.python-version == '3.13' }}
61+
with:
62+
ref: ${{ github.head_ref }}
63+
fetch-depth: 0
64+
- name: Patch tests/test_examples_run.yaml with update_results to False, only for the last matrix version
65+
if: ${{ steps.parse-run-examples-config.outputs.update_results && matrix.python-version == '3.13' }}
66+
uses: fjogeleit/yaml-update-action@main
67+
with:
68+
changes: |
69+
{
70+
"tests/test_examples_run.yaml": {
71+
"update_results": false
72+
}
73+
}
74+
commitChange: false
75+
- name: Push the new run-examples yaml config
76+
if: ${{ steps.parse-run-examples-config.outputs.update_results && matrix.python-version == '3.13' }}
77+
shell: bash
78+
run: |
79+
git config --local user.name github-actions[bot]
80+
git config --local user.email 41898282+github-actions[bot]@users.noreply.github.com
81+
git status
82+
git pull
83+
git add tests/test_examples_run.yaml
84+
git diff --cached --quiet || git commit -s -m "github-actions[bot]: Run examples: reverting update_result to false on your behalf"
85+
git push
86+
87+
88+
89+
90+

‎.github/workflows/run-examples.yml‎

Lines changed: 10 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ on:
66
- cron: '0 1 * * *'
77
workflow_dispatch:
88

9-
109
jobs:
1110
tests:
1211
name: Execution tests
@@ -15,93 +14,14 @@ jobs:
1514
fail-fast: false
1615
matrix:
1716
python-version: ['3.11', '3.12', '3.13']
18-
1917
steps:
20-
21-
# Free up some disk space
22-
- name: Remove unnecessary files
23-
run: |
24-
sudo rm -rf /usr/share/dotnet
25-
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
26-
27-
# Set up Ollama
28-
- name: Install Ollama and start server
29-
shell: bash
30-
run: |
31-
curl -fsSL https://ollama.com/install.sh | sudo -E sh
32-
33-
- name: Pull models in examples/
34-
shell: bash
35-
run: |
36-
ollama pull granite3.2:2b
37-
ollama pull granite3.2:8b
38-
ollama pull mxbai-embed-large
39-
ollama list
40-
41-
- name: Check that all required models are available
42-
shell: bash
43-
run: |
44-
models=("mxbai-embed-large" "granite3.2:2b" "granite3.2:8b")
45-
missing=0
46-
for model in "${models[@]}"; do
47-
if ! ollama list | awk 'NR>1 {print $1}' | grep -q "$model"; then
48-
echo "❌ Model $model (or substring) is missing!"
49-
missing=1
50-
fi
51-
done
52-
53-
if [ "$missing" -eq 1 ]; then
54-
exit 1
55-
else
56-
echo "✅ All expected models are available."
57-
fi
58-
59-
- name: Wait for Ollama server
60-
shell: bash
61-
run: |
62-
sleep 10
63-
time curl -i http://localhost:11434
64-
65-
# Run tests
66-
- uses: actions/checkout@v4
67-
with:
68-
ref: ${{ github.head_ref }}
69-
- name: Set up Python ${{ matrix.python-version }}
70-
uses: actions/setup-python@v5
71-
with:
72-
python-version: ${{ matrix.python-version }}
73-
- name: Cache pip
74-
uses: actions/cache@v4
75-
with:
76-
# This path is specific to Ubuntu
77-
path: ${{ env.pythonLocation }}
78-
# Look to see if there is a cache hit for the setup file
79-
key: ${{ runner.os }}-pip-new3-${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}
80-
restore-keys: |
81-
${{ runner.os }}-pip-new3
82-
${{ runner.os }}-new3
83-
- name: Install dependencies
84-
run: pip install --upgrade --upgrade-strategy eager .[all]
85-
- name: pip list packages
86-
run: pip list
87-
- name: show pip dependencies
88-
run: |
89-
pip install pipdeptree
90-
pipdeptree -fl
91-
- name: run tests
92-
env:
93-
WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
94-
WATSONX_APIKEY: ${{ secrets.WATSONX_APIKEY }}
95-
WATSONX_URL: ${{ secrets.WATSONX_URL }}
96-
REPLICATE_API_TOKEN: ${{ secrets.REPLICATE_API_TOKEN }}
97-
OLLAMA_GHACTIONS_RESULTS: true
98-
run: py.test -v --capture=tee-sys -rfE -s tests/test_examples_run.py
99-
- name: Update example result files (if any) generated from Ollama running on GH Actions
100-
if: matrix.python-version == '3.11'
101-
run: |
102-
git config --local user.name github-actions[bot]
103-
git config --local user.email "${{ github.actor_id }}+${{ github.actor }}@users.noreply.github.com"
104-
git status
105-
git add tests/results/
106-
git diff --cached --quiet || git commit -S -s -m "github-actions[bot]: Updated results file when running examples on $(date)"
107-
git push
18+
- uses: actions/checkout@v4
19+
with:
20+
ref: ${{ github.head_ref }}
21+
- uses: ./.github/actions/run-examples
22+
with:
23+
python-version: ${{ matrix.python-version }}
24+
runner-os: ${{ runner.os }}
25+
head-ref: ${{ github.head_ref }}
26+
update-results: 'false'
27+
check: '[]' # Empty list means run against all PDL programs

0 commit comments

Comments
 (0)