Skip to content

Commit 309d9ac

Browse files
general-kroll-4-lifegeneral-kroll-4-life
authored
demo-md-walkthrough-testing (#493)
Summary: - Support for simple testing of walkthroughs written in markdown, with annotations. - Whole table match scenario. - Nonempty table match scenario. - Skip testing on files named `README.md`. - Runnable from CI, with tag of form: `scenario-<stackql_build_run_number>-<anything>`.
1 parent 6825e6d commit 309d9ac

File tree

6 files changed

+562
-0
lines changed

6 files changed

+562
-0
lines changed

‎.github/workflows/scenario.yaml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
name: scenario
2+
3+
on:
4+
push:
5+
tags:
6+
- 'scenario*'
7+
8+
env:
9+
TAG_NAME: ${{ github.ref_name }}
10+
11+
jobs:
12+
scenario-testing:
13+
runs-on: ubuntu-latest
14+
timeout-minutes: ${{ vars.DEFAULT_JOB_TIMEOUT_MIN == '' && 120 || vars.DEFAULT_JOB_TIMEOUT_MIN }}
15+
steps:
16+
- name: Ref Parse
17+
run: |
18+
{
19+
echo "runID=$(echo -n '${{ github.ref_name }}' | cut -d '-' -f 2)"
20+
} >> "${GITHUB_ENV}"
21+
22+
- name: Check out code into the Go module directory
23+
uses: actions/[email protected]
24+
25+
- name: Download Artifact
26+
uses: actions/[email protected]
27+
with:
28+
name: stackql_linux_amd64
29+
path: build
30+
github-token: ${{ secrets.CI_STACKQL_PACKAGE_DOWNLOAD_TOKEN }}
31+
repository: stackql/stackql
32+
run-id: ${{ env.runID }}
33+
34+
- name: Stackql permissions
35+
run: |
36+
sudo chmod a+rwx build/stackql
37+
ls -al build/stackql
38+
{
39+
echo "$(pwd)/build"
40+
} >> "${GITHUB_PATH}"
41+
42+
- name: Check Stackql Version
43+
run: |
44+
stackql --version
45+
46+
- name: Persist secrets
47+
run: |
48+
echo "$GCP_RO_SECRET" >> cicd/keys/testing/google-ro-credentials.json
49+
shell: bash
50+
env:
51+
GCP_RO_SECRET: ${{ secrets.CI_SCENARIO_GCP_RO_SECRET }}
52+
53+
- name: Install Python dependencies
54+
run: |
55+
pip3 install -r cicd/requirements.txt
56+
57+
- name: Run Walkthrough Scenarios
58+
run: |
59+
python3 test/python/markdown_testing/markdown_testing.py 2>&1 | tee cicd/log/markdown-testing-results.log
60+
61+
- name: Upload Test Results
62+
uses: actions/[email protected]
63+
with:
64+
name: scenario_test_results
65+
path: cicd/log/markdown-testing-results.log
66+
67+
- name: Cleanup
68+
if: always()
69+
run: |
70+
rm -rf cicd/keys/testing
71+

‎cicd/requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
Flask==3.0.3
2+
mistune==3.0.2
23
psycopg2-binary>=2.9.9
34
psycopg[binary]>=3.1.16
45
PyYaml>=6.0.1

‎docs/walkthroughs/README.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
2+
# `stackql` walkthroughs
3+
4+
All markdown documents hereunder, execept those named `README.md`,
5+
are **provably working** examples of `stackql` in action.
6+
These materials serve as useful examples and reference materials for
7+
using `stackql`. If you have some use case that you would like to see added here; please let us know!
8+
9+
10+
All walkthrough files are testable with CI, leveraging annotations (eg: code block info strings)
11+
in order to setup, run, verify and tear down testing scenarios. The tests *can* be run:
12+
13+
- Locally and manually, on your own machine. That's the whole idea; please follow the instructions, mix and match, and let us know any ideas that occur.
14+
- Directly from CI. Reports are generated and archived.
15+
- From test harnesses, such as robot framework. This has not yet been implemented.
16+
17+
## Running from CI
18+
19+
The canonical, **ruleset-protected** tag form is `scenario-<run_number>-<anything>`. At this stage, `run_number` must refer to a `stackql` run for which a `linux` `amd64` stackql binary archive is present at the time the tag is run.
20+
21+
22+

‎docs/walkthroughs/get-google-accelerator-types.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
2+
## Setup
3+
4+
First, create a google service account key using the GCP Console, per [the GCP documentation](https://cloud.google.com/iam/docs/keys-create-delete). Grant the service account at least `Viewer` role equivalent privileges, per [the GCP dumentation](https://cloud.google.com/iam/docs/create-service-agents#grant-roles).
5+
6+
Then, do this in bash:
7+
8+
```bash setup stackql-shell credentials-path=cicd/keys/testing/google-ro-credentials.json app-root-path=./test/tmp/.get-google-accel.stackql
9+
10+
export GOOGLE_CREDENTIALS="$(cat <credentials-path>)";
11+
12+
stackql shell --approot=<app-root-path>
13+
```
14+
15+
## Method
16+
17+
Do this in the `stackql` shell, replacing `<project>` with your GCP project name, and `<zone>` as desired, eg: `australia-southeast1-a`:
18+
19+
```sql stackql-shell input required project=stackql-demo zone=australia-southeast1-a
20+
21+
registry pull google;
22+
23+
select
24+
name,
25+
kind
26+
FROM google.compute.accelerator_types
27+
WHERE
28+
project = '<project>'
29+
AND zone = '<zone>'
30+
ORDER BY
31+
name desc
32+
;
33+
34+
```
35+
36+
## Result
37+
38+
39+
You will see something very much like this included in the output, presuming you have one VM (if you have zero, only the headers should appper, more VMs means more rows):
40+
41+
```sql expectation stdout-contains-all
42+
|---------------------|-------------------------|
43+
| name | kind |
44+
|---------------------|-------------------------|
45+
| nvidia-tesla-t4-vws | compute#acceleratorType |
46+
|---------------------|-------------------------|
47+
| nvidia-tesla-t4 | compute#acceleratorType |
48+
|---------------------|-------------------------|
49+
| nvidia-tesla-p4-vws | compute#acceleratorType |
50+
|---------------------|-------------------------|
51+
| nvidia-tesla-p4 | compute#acceleratorType |
52+
|---------------------|-------------------------|
53+
```
54+
55+
<!--- EXPECTATION
56+
google\ provider,\ version\ 'v24.11.00274'\ successfully\ installed
57+
goodbye
58+
-->
59+
60+
<x-expectation style="display: none;">
61+
<stdout-contains-nonempty-table></stdout-contains-nonempty-table>
62+
</x-expectation>
63+
64+
## Cleanup
65+
66+
```bash teardown best-effort app-root-path=./test/tmp/.get-google-accel.stackql
67+
68+
rm -rf <app-root-path>
69+
70+
```

‎docs/walkthroughs/get-google-vms.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
2+
## Setup
3+
4+
First, create a google service account key using the GCP Console, per [the GCP documentation](https://cloud.google.com/iam/docs/keys-create-delete). Grant the service account at least `Viewer` role equivalent privileges, per [the GCP dumentation](https://cloud.google.com/iam/docs/create-service-agents#grant-roles).
5+
6+
Then, do this in bash:
7+
8+
```bash setup stackql-shell credentials-path=cicd/keys/testing/google-ro-credentials.json app-root-path=./test/tmp/.get-google-vms.stackql
9+
10+
export GOOGLE_CREDENTIALS="$(cat <credentials-path>)";
11+
12+
stackql shell --approot=<app-root-path>
13+
```
14+
15+
## Method
16+
17+
Do this in the `stackql` shell, replacing `<project>` with your GCP project name, and `<zone>` as desired, eg: `australia-southeast1-a`:
18+
19+
```sql stackql-shell input required project=stackql-demo zone=australia-southeast1-a
20+
21+
registry pull google;
22+
23+
select
24+
name,
25+
id
26+
FROM google.compute.instances
27+
WHERE
28+
project = '<project>'
29+
AND zone = '<zone>'
30+
;
31+
32+
```
33+
34+
## Result
35+
36+
37+
You will see something very much like this included in the output, presuming you have one VM (if you have zero, only the headers should appper, more VMs means more rows):
38+
39+
```sql stackql stdout expectation stdout-table-contains-data
40+
|--------------------------------------------------|---------------------|
41+
| name | id |
42+
|--------------------------------------------------|---------------------|
43+
| any-compute-cluster-1-default-abcd-00000001-0001 | 1000000000000000001 |
44+
|--------------------------------------------------|---------------------|
45+
```
46+
47+
<!--- EXPECTATION
48+
google\ provider,\ version\ 'v24.11.00274'\ successfully\ installed
49+
goodbye
50+
-->
51+
52+
<x-expectation style="display: none;">
53+
<stdout-contains-nonempty-table></stdout-contains-nonempty-table>
54+
</x-expectation>
55+
56+
## Cleanup
57+
58+
```bash teardown best-effort app-root-path=./test/tmp/.get-google-vms.stackql
59+
60+
rm -rf <app-root-path>
61+
62+
```

0 commit comments

Comments
 (0)