Skip to content

Commit a25a726

Browse files
authored
chore: move testing infra code out of core (PolusAI#317)
* Move testing infra code out of core * rearrange test code
1 parent 9ffe7f5 commit a25a726

File tree

7 files changed

+83
-76
lines changed

7 files changed

+83
-76
lines changed

.github/workflows/fuzzy_compile_weekly.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ jobs:
9797

9898
- name: Generate Sophios Python API Workflows (*.py -> *.wic)
9999
if: always()
100-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
100+
run: cd sophios/ && pytest -k test_compile_python_workflows
101101

102102
- name: Generate Sophios Validation Jsonschema
103103
if: always()
@@ -108,7 +108,7 @@ jobs:
108108
# WIC Python API workflows as well as the WIC Python API itself.
109109
- name: Validate Sophios Python API Workflows (*.py -> *.wic)
110110
if: always()
111-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
111+
run: cd sophios/ && pytest -k test_compile_python_workflows
112112

113113
# Since a randomly chosen subschema is used every time, repeat 10X for more coverage.
114114

.github/workflows/lint_and_test.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ jobs:
179179

180180
- name: Generate Sophios Python API Workflows (*.py -> *.wic)
181181
if: always()
182-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
182+
run: cd sophios/ && pytest -k test_compile_python_workflows
183183

184184
- name: Generate Sophios Validation Jsonschema
185185
if: always()
@@ -190,7 +190,7 @@ jobs:
190190
# Sophios Python API workflows as well as the Sophios Python API itself.
191191
- name: Validate sophios Python API Workflows (*.py -> *.wic)
192192
if: always()
193-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
193+
run: cd sophios/ && pytest -k test_compile_python_workflows
194194

195195
- name: Build Documentation
196196
if: always()

.github/workflows/lint_and_test_macos.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ jobs:
103103

104104
- name: Generate Sophios Python API Workflows (*.py -> *.wic)
105105
if: always()
106-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
106+
run: cd sophios/ && pytest -k test_compile_python_workflows
107107

108108
- name: Generate Sophios Validation Jsonschema
109109
if: always()
@@ -114,7 +114,7 @@ jobs:
114114
# Sophios Python API workflows as well as the Sophios Python API itself.
115115
- name: Validate Sophios Python API Workflows (*.py -> *.wic)
116116
if: always()
117-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
117+
run: cd sophios/ && pytest -k test_compile_python_workflows
118118

119119
- name: Build Documentation
120120
if: always()

.github/workflows/run_workflows.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,7 @@ jobs:
157157

158158
- name: Generate Sophios Python API Workflows (*.py -> *.wic)
159159
if: always()
160-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
160+
run: cd sophios/ && pytest -k test_compile_python_workflows
161161

162162
- name: Generate Sophios Validation Jsonschema
163163
if: always()
@@ -168,7 +168,7 @@ jobs:
168168
# WIC Python API workflows as well as the WIC Python API itself.
169169
- name: Validate Sophios Python API Workflows (*.py -> *.wic)
170170
if: always()
171-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
171+
run: cd sophios/ && pytest -k test_compile_python_workflows
172172

173173
- name: cwl-docker-extract (i.e. recursively docker pull)
174174
if: always()

.github/workflows/run_workflows_weekly.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ jobs:
120120

121121
- name: Generate Sophios Python API Workflows (*.py -> *.wic)
122122
if: always()
123-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
123+
run: cd sophios/ && pytest -k test_compile_python_workflows
124124

125125
- name: Generate Sophios Validation Jsonschema
126126
if: always()
@@ -131,7 +131,7 @@ jobs:
131131
# Sophios Python API workflows as well as the WIC Python API itself.
132132
- name: Validate Sophios Python API Workflows (*.py -> *.wic)
133133
if: always()
134-
run: cd sophios/ && python -c 'import sophios; import sophios.plugins; sophios.plugins.blindly_execute_python_workflows()'
134+
run: cd sophios/ && pytest -k test_compile_python_workflows
135135

136136
- name: cwl-docker-extract (i.e. recursively docker pull)
137137
if: always()

src/sophios/plugins.py

Lines changed: 0 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -466,69 +466,3 @@ def get_yml_paths(config: Json) -> Dict[str, Dict[str, Path]]:
466466

467467
def get_py_paths(config: Json) -> Dict[str, Dict[str, Path]]:
468468
return get_workflow_paths(config, 'py')
469-
470-
471-
def blindly_execute_python_workflows() -> None:
472-
"""This function imports (read: blindly executes) all python files in 'search_paths_wic'
473-
The python files are assumed to have a top-level workflow() function
474-
which returns a sophios.api.pythonapi.Workflow object.
475-
The python files should NOT call the .run() method!
476-
(from any code path that is automatically executed on import)
477-
"""
478-
# I hope u like Remote Code Execution vulnerabilities!
479-
# See https://en.wikipedia.org/wiki/Arithmetical_hierarchy
480-
from sophios.api import pythonapi # pylint: disable=C0415:import-outside-toplevel
481-
# Since this is completely different test path we have to copy
482-
# default .txt files to default global_config.json
483-
config_file = Path().home()/'wic'/'global_config.json'
484-
global_config = io.read_config_from_disk(config_file)
485-
pythonapi.global_config = get_tools_cwl(global_config) # Use path fallback in the CI
486-
paths = get_py_paths(global_config)
487-
# Above we are assuming that config is default
488-
paths_tuples = [(path_str, path)
489-
for namespace, paths_dict in paths.items()
490-
for path_str, path in paths_dict.items()]
491-
any_import_errors = False
492-
for path_stem, path in paths_tuples:
493-
if 'mm-workflows' in str(path) or 'docs/tutorials/' in str(path):
494-
# Exclude paths that only contain 'regular' python files.
495-
continue
496-
# NOTE: Use anything (unique?) for the python_module_name.
497-
try:
498-
module = import_python_file(path_stem, path)
499-
# Let's require all python API files to define a function, say
500-
# def workflow() -> Workflow
501-
# so we can programmatically call it here:
502-
retval: pythonapi.Workflow = module.workflow() # no arguments
503-
# which allows us to programmatically call Workflow methods:
504-
compiler_info = retval.compile() # hopefully retval is actually a Workflow object!
505-
# But since this is python (i.e. not Haskell) that in no way eliminates
506-
# the above security considerations.
507-
508-
# This lets us use path.parent to write a *.wic file in the
509-
# auto-discovery path, and thus reuse the existing wic CI
510-
retval.write_ast_to_disk(path.parent)
511-
512-
# Programmatically blacklist subworkflows from running in config_ci.json
513-
# (Again, because subworkflows are missing inputs and cannot run.)
514-
config_ci = path.parent / 'config_ci.json'
515-
json_contents = {}
516-
if config_ci.exists():
517-
with open(config_ci, mode='r', encoding='utf-8') as r:
518-
json_contents = json.load(r)
519-
run_blacklist: list[str] = json_contents.get('run_blacklist', [])
520-
# Use [1:] for proper subworkflows only
521-
subworkflows: list[pythonapi.Workflow] = retval.flatten_subworkflows()[1:]
522-
run_blacklist += [wf.process_name for wf in subworkflows]
523-
json_contents['run_blacklist'] = run_blacklist
524-
with open(config_ci, mode='w', encoding='utf-8') as f:
525-
json.dump(json_contents, f)
526-
527-
except Exception as e:
528-
any_import_errors = True
529-
if sys.version_info >= (3, 10):
530-
traceback.print_exception(type(e), value=e, tb=None)
531-
else:
532-
traceback.print_exception(etype=type(e), value=e, tb=None)
533-
if any_import_errors:
534-
sys.exit(1) # Make sure the CI fails
Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
import json
2+
import sys
3+
import traceback
4+
from pathlib import Path
5+
6+
import sophios
7+
import sophios.plugins
8+
from sophios import input_output as io
9+
from sophios.python_cwl_adapter import import_python_file
10+
11+
12+
def test_compile_python_workflows() -> None:
13+
"""This function imports (read: blindly executes) all python files in 'search_paths_wic'
14+
The python files are assumed to have a top-level workflow() function
15+
which returns a sophios.api.pythonapi.Workflow object.
16+
The python files should NOT call the .run() method!
17+
(from any code path that is automatically executed on import)
18+
"""
19+
from sophios.api import pythonapi # pylint: disable=C0415:import-outside-toplevel
20+
# Since this is completely different test path we have to copy
21+
# default .txt files to default global_config.json
22+
config_file = Path().home()/'wic'/'global_config.json'
23+
global_config = io.read_config_from_disk(config_file)
24+
pythonapi.global_config = sophios.plugins.get_tools_cwl(global_config) # Use path fallback in the CI
25+
paths = sophios.plugins.get_py_paths(global_config)
26+
# Above we are assuming that config is default
27+
paths_tuples = [(path_str, path)
28+
for namespace, paths_dict in paths.items()
29+
for path_str, path in paths_dict.items()]
30+
any_import_errors = False
31+
for path_stem, path in paths_tuples:
32+
if 'mm-workflows' in str(path) or 'docs/tutorials/' in str(path):
33+
# Exclude paths that only contain 'regular' python files.
34+
continue
35+
# NOTE: Use anything (unique?) for the python_module_name.
36+
try:
37+
module = import_python_file(path_stem, path)
38+
# Let's require all python API files to define a function, say
39+
# def workflow() -> Workflow
40+
# so we can programmatically call it here:
41+
retval: pythonapi.Workflow = module.workflow() # no arguments
42+
# which allows us to programmatically call Workflow methods:
43+
compiler_info = retval.compile() # hopefully retval is actually a Workflow object!
44+
# But since this is python (i.e. not Haskell) that in no way eliminates
45+
# the above security considerations.
46+
47+
# This lets us use path.parent to write a *.wic file in the
48+
# auto-discovery path, and thus reuse the existing wic CI
49+
retval.write_ast_to_disk(path.parent)
50+
51+
# Programmatically blacklist subworkflows from running in config_ci.json
52+
# (Again, because subworkflows are missing inputs and cannot run.)
53+
config_ci = path.parent / 'config_ci.json'
54+
json_contents = {}
55+
if config_ci.exists():
56+
with open(config_ci, mode='r', encoding='utf-8') as r:
57+
json_contents = json.load(r)
58+
run_blacklist: list[str] = json_contents.get('run_blacklist', [])
59+
# Use [1:] for proper subworkflows only
60+
subworkflows: list[pythonapi.Workflow] = retval.flatten_subworkflows()[1:]
61+
run_blacklist += [wf.process_name for wf in subworkflows]
62+
json_contents['run_blacklist'] = run_blacklist
63+
with open(config_ci, mode='w', encoding='utf-8') as f:
64+
json.dump(json_contents, f)
65+
66+
except Exception as e:
67+
any_import_errors = True
68+
if sys.version_info >= (3, 10):
69+
traceback.print_exception(type(e), value=e, tb=None)
70+
else:
71+
traceback.print_exception(etype=type(e), value=e, tb=None)
72+
if any_import_errors:
73+
sys.exit(1) # Make sure the CI fails

0 commit comments

Comments
 (0)