[cueman] Add comprehensive test infrastructure and update documentation for Cueman #1954

Aniketsy · 2025-09-18T07:25:14Z

Link the Issue(s) this Pull Request is related to.

Summarize your change.

1) Integration tests

Implement integration tests that cover complete Cueman workflows from argument parsing through execution, simulating real-world usage scenarios.

Created /cueman/tests/test_integration_workflows.py with end-to-end tests for:

Complete job management workflow (find -> pause -> modify -> resume)
Frame management workflow (query -> filter -> retry/kill)
Batch operation workflow (multiple jobs with confirmation)
Error recovery scenarios
Complex filter combination workflows
Help text and version display
Invalid command handling from start to finish
Use mock opencue API but test real command line parsing and execution flow. Verify proper cleanup and error states.

2) Lint and Code quality

Fix pylint issues in test_integration_workflows.py
Add module docstring and fix import style
Remove trailing whitespace and add proper spacing
Achieve 10/10 pylint rating

3) Test fixes

Fix SystemExit test failures in integration tests
Update test_batch_operation_workflow to use comma-separated job format
Mock displayLayers function properly in workflow tests
Remove incorrect SystemExit expectations
All tests now pass successfully

4) Test infrastructure

Add pytest and test dependencies to pyproject.toml:
- pytest, pytest-cov, pytest-mock, mock, pyfakefs
- dev dependencies: pylint, black, isort
Configure pytest.ini_options with discovery and execution settings
Add coverage configuration with HTML/XML reports
Include test markers (unit, integration, slow)
Add run_tests.sh script for coverage options
Add tox.ini for multi-Python testing and lint checks
Include GitHub Actions workflow example in pyproject.toml comments

5) Documentation updates

Enhance README with detailed testing section
Add test infrastructure description and development workflow
Update docs/reference/tools/cueman.md with dev and testing info
Enhance docs/tutorials/cueman-tutorial.md with contribution guidelines

6) Developer experience

Document coverage reporting capabilities
Provide clear testing and development guidelines
Include CI/CD integration documentation
Add code quality tool usage instructions

All tests pass with comprehensive coverage reporting and lint-free code.

Co-authored-by: Ramon Figueiredo [email protected]
Co-authored-by: Aniket Singh Yadav [email protected]

Aniketsy · 2025-09-18T10:49:02Z

@ramonfigueiredo Could you please suggest what changes are needed to resolve these check failures? I think I might be missing something.

ramonfigueiredo · 2025-09-18T18:33:46Z

@ramonfigueiredo Could you please suggest what changes are needed to resolve these check failures? I think I might be missing something.

Hi @Aniketsy, the lint issues are straightforward:

Remove trailing spaces on lines 10 and 22.

Add a module docstring at the top of tests/test_integration_workflows.py.

Change the import to from cueman import main.

For the test failures: the CLI is raising SystemExit because of argparse and a protobuf request is built incorrectly (JobGetLayersRequest needs keyword args/dict). You’ll need to fix the request init and/or mock the job-layers call in your tests so the CLI doesn’t exit with error codes.

…on for cueman - Fix pylint issues in `test_integration_workflows.py` * Add module docstring and fix import style * Remove trailing whitespace and add proper spacing * Achieve 10/10 pylint rating - Fix SystemExit test failures in integration tests: * Update test_batch_operation_workflow to use comma-separated job format * Mock displayLayers function properly in workflow tests * Remove incorrect SystemExit expectations where not applicable * All tests now pass successfully - Add pytest and test dependencies to pyproject.toml: * Add test optional dependencies: pytest, pytest-cov, pytest-mock, mock, pyfakefs * Add dev optional dependencies including pylint, black, isort * Configure pytest.ini_options with test discovery and execution settings * Add coverage configuration with HTML/XML report generation * Include test markers for unit, integration, and slow tests - Create test infrastructure tools: * Add `run_tests.sh` convenience script for running tests with coverage options * Add `tox.ini` for multi-Python version testing and code quality checks * Include GitHub Actions workflow example in pyproject.toml comments - Update documentation comprehensively: * Enhance README.md with detailed testing section covering tests * Add test infrastructure description and development workflow * Update docs/reference/tools/cueman.md with development and testing section * Enhance docs/tutorials/cueman-tutorial.md with contribution information - Improve development experience: * Document coverage reporting capabilities * Provide clear testing and development guidelines * Include CI/CD integration documentation * Add code quality tool usage instructions All tests pass with comprehensive coverage reporting and lint-free code.

ramonfigueiredo · 2025-09-18T18:59:35Z

Hi @Aniketsy

I fixed. I will submit new changes soon. Please way for the final version. Thanks

ramonfigueiredo · 2025-09-18T19:08:43Z

Hi @Aniketsy

I submitted the fixes for pylint issues, resolves SystemExit failures in integration tests, and adds full test infrastructure (pytest, tox, run_tests.sh, coverage). It also updates documentation (README, reference, tutorials) with testing and contribution guidelines.

All tests now pass with 10/10 pylint and comprehensive coverage. Ready for merge soon.

ramonfigueiredo · 2025-09-18T19:31:26Z

All tests now passed!

cd cueman/

1) Running Cueman tests

pytest

cueman % ./run_tests.sh 
Running: python -m pytest tests/
----------------------------------------
=============================================================================================================================================== test session starts ===============================================================================================================================================
configfile: pyproject.toml
plugins: pyfakefs-5.9.3
collected 42 items                                                                                                                                                                                                                                                                                                

tests/test_integration_workflows.py::TestCuemanIntegrationWorkflows::test_batch_operation_workflow PASSED                                                                                                                                                                                                   [  2%]
tests/test_integration_workflows.py::TestCuemanIntegrationWorkflows::test_complex_filter_combination_workflows PASSED                                                                                                                                                                                       [  4%]
tests/test_integration_workflows.py::TestCuemanIntegrationWorkflows::test_error_recovery_scenarios PASSED                                                                                                                                                                                                   [  7%]
tests/test_integration_workflows.py::TestCuemanIntegrationWorkflows::test_frame_management_workflow PASSED                                                                                                                                                                                                  [  9%]
tests/test_integration_workflows.py::TestCuemanIntegrationWorkflows::test_help_text_and_version_display PASSED                                                                                                                                                                                              [ 11%]
tests/test_integration_workflows.py::TestCuemanIntegrationWorkflows::test_invalid_command_handling PASSED                                                                                                                                                                                                   [ 14%]
tests/test_main.py::TestCuemanMain::test_buildFrameSearch_default PASSED                                                                                                                                                                                                                                    [ 16%]
tests/test_main.py::TestCuemanMain::test_buildFrameSearch_with_layer PASSED                                                                                                                                                                                                                                 [ 19%]
tests/test_main.py::TestCuemanMain::test_buildFrameSearch_with_memory_range PASSED                                                                                                                                                                                                                          [ 21%]
tests/test_main.py::TestCuemanMain::test_buildFrameSearch_with_page PASSED                                                                                                                                                                                                                                  [ 23%]
tests/test_main.py::TestCuemanMain::test_buildFrameSearch_with_range PASSED                                                                                                                                                                                                                                 [ 26%]
tests/test_main.py::TestCuemanMain::test_buildFrameSearch_with_state PASSED                                                                                                                                                                                                                                 [ 28%]
tests/test_main.py::TestCuemanMain::test_format_nargs_input_comma_separated PASSED                                                                                                                                                                                                                          [ 30%]
tests/test_main.py::TestCuemanMain::test_format_nargs_input_single_job PASSED                                                                                                                                                                                                                               [ 33%]
tests/test_main.py::TestCuemanMain::test_format_nargs_input_with_spaces PASSED                                                                                                                                                                                                                              [ 35%]
tests/test_main.py::TestCuemanMain::test_reorderJob_with_invalid_order PASSED                                                                                                                                                                                                                               [ 38%]
tests/test_main.py::TestCuemanMain::test_reorderJob_with_valid_order PASSED                                                                                                                                                                                                                                 [ 40%]
tests/test_main.py::TestCuemanMain::test_staggerJob_with_layers PASSED                                                                                                                                                                                                                                      [ 42%]
tests/test_main.py::TestCuemanMain::test_staggerJob_without_layers PASSED                                                                                                                                                                                                                                   [ 45%]
tests/test_main.py::TestCuemanMain::test_terminateJobs PASSED                                                                                                                                                                                                                                               [ 47%]
tests/test_main.py::TestCuemanHandleArgs::test_handleArgs_info PASSED                                                                                                                                                                                                                                       [ 50%]
tests/test_main.py::TestCuemanHandleArgs::test_handleArgs_list_frames PASSED                                                                                                                                                                                                                                [ 52%]
tests/test_main.py::TestCuemanHandleArgs::test_handleArgs_no_command PASSED                                                                                                                                                                                                                                 [ 54%]
tests/test_main.py::TestCuemanHandleArgs::test_handleArgs_pause PASSED                                                                                                                                                                                                                                      [ 57%]
tests/test_suite.py::TestCuemanMain::test_buildFrameSearch_default PASSED                                                                                                                                                                                                                                   [ 59%]
tests/test_suite.py::TestCuemanMain::test_buildFrameSearch_with_layer PASSED                                                                                                                                                                                                                                [ 61%]
tests/test_suite.py::TestCuemanMain::test_buildFrameSearch_with_memory_range PASSED                                                                                                                                                                                                                         [ 64%]
tests/test_suite.py::TestCuemanMain::test_buildFrameSearch_with_page PASSED                                                                                                                                                                                                                                 [ 66%]
tests/test_suite.py::TestCuemanMain::test_buildFrameSearch_with_range PASSED                                                                                                                                                                                                                                [ 69%]
tests/test_suite.py::TestCuemanMain::test_buildFrameSearch_with_state PASSED                                                                                                                                                                                                                                [ 71%]
tests/test_suite.py::TestCuemanMain::test_format_nargs_input_comma_separated PASSED                                                                                                                                                                                                                         [ 73%]
tests/test_suite.py::TestCuemanMain::test_format_nargs_input_single_job PASSED                                                                                                                                                                                                                              [ 76%]
tests/test_suite.py::TestCuemanMain::test_format_nargs_input_with_spaces PASSED                                                                                                                                                                                                                             [ 78%]
tests/test_suite.py::TestCuemanMain::test_reorderJob_with_invalid_order PASSED                                                                                                                                                                                                                              [ 80%]
tests/test_suite.py::TestCuemanMain::test_reorderJob_with_valid_order PASSED                                                                                                                                                                                                                                [ 83%]
tests/test_suite.py::TestCuemanMain::test_staggerJob_with_layers PASSED                                                                                                                                                                                                                                     [ 85%]
tests/test_suite.py::TestCuemanMain::test_staggerJob_without_layers PASSED                                                                                                                                                                                                                                  [ 88%]
tests/test_suite.py::TestCuemanMain::test_terminateJobs PASSED                                                                                                                                                                                                                                              [ 90%]
tests/test_suite.py::TestCuemanHandleArgs::test_handleArgs_info PASSED                                                                                                                                                                                                                                      [ 92%]
tests/test_suite.py::TestCuemanHandleArgs::test_handleArgs_list_frames PASSED                                                                                                                                                                                                                               [ 95%]
tests/test_suite.py::TestCuemanHandleArgs::test_handleArgs_no_command PASSED                                                                                                                                                                                                                                [ 97%]
tests/test_suite.py::TestCuemanHandleArgs::test_handleArgs_pause PASSED                                                                                                                                                                                                                                     [100%]

=============================================================================================================================================== 42 passed in 0.45s ================================================================================================================================================

2) Install with development dependencies

pip install -e ".[dev]"

.
.
.
Successfully installed astroid-3.3.11 black-25.1.0 click-8.3.0 coverage-7.10.6 dill-0.4.0 isort-6.0.1 mccabe-0.7.0 mypy-extensions-1.1.0 opencue_cueman-1.11.79+f3eb2488 pathspec-0.12.1 platformdirs-4.4.0 pylint-3.3.8 pytest-cov-7.0.0 pytest-mock-3.15.1 tomlkit-0.13.3

3) Run the test suite

pytest --cov=cueman --cov-report=term-missing

=============================================================================================================================================== test session starts ===============================================================================================================================================
configfile: pyproject.toml
testpaths: tests
plugins: mock-3.15.1, pyfakefs-5.9.3, cov-7.0.0
collected 42 items                                                                                                                                                                                                                                                                                                

.
.
.

================================================================================================================================================= tests coverage ==================================================================================================================================================
________________________________________________________________________________________________________________________________ coverage: platform darwin, python 3.13.2-final-0 _________________________________________________________________________________________________________________________________

Name                 Stmts   Miss   Cover   Missing
---------------------------------------------------
cueman/__init__.py      12      8  33.33%   22-31
cueman/__main__.py      11     11   0.00%   19-37
cueman/main.py         377    221  41.38%   40-41, 49-55, 193, 200-201, 206-210, 244-249, 251-261, 266-271, 276-281, 295, 300-457, 469-513, 545, 584, 591, 601-620
---------------------------------------------------
TOTAL                  400    240  40.00%
=============================================================================================================================================== 42 passed in 0.22s ================================================================================================================================================

4) Format and lint code

black cueman tests && isort cueman tests

All done! ✨ 🍰 ✨
7 files left unchanged.

pylint cueman tests

.
.
.
==tests.test_main:[285:301]
            "lf",
            "lp",
            "ll",
            "info",
            "pause",
            "resume",
            "term",
            "eat",
            "kill",
            "retry",
            "done",
            "stagger",
            "reorder",
            "retries",
            "autoeaton",
            "autoeatoff", (duplicate-code)
.
.
.
-----------------------------------
Your code has been rated at 8.76/10

…on for cueman - Fix pylint issues in `test_integration_workflows.py` * Add module docstring and fix import style * Remove trailing whitespace and add proper spacing * Achieve 10/10 pylint rating - Fix SystemExit test failures in integration tests: * Update test_batch_operation_workflow to use comma-separated job format * Mock displayLayers function properly in workflow tests * Remove incorrect SystemExit expectations where not applicable * All tests now pass successfully - Add pytest and test dependencies to pyproject.toml: * Add test optional dependencies: pytest, pytest-cov, pytest-mock, mock, pyfakefs * Add dev optional dependencies including pylint, black, isort * Configure pytest.ini_options with test discovery and execution settings * Add coverage configuration with HTML/XML report generation * Include test markers for unit, integration, and slow tests - Create test infrastructure tools: * Add `run_tests.sh` convenience script for running tests with coverage options * Add `tox.ini` for multi-Python version testing and code quality checks * Include GitHub Actions workflow example in pyproject.toml comments - Update documentation comprehensively: * Enhance README.md with detailed testing section covering tests * Add test infrastructure description and development workflow * Update docs/reference/tools/cueman.md with development and testing section * Enhance docs/tutorials/cueman-tutorial.md with contribution information - Improve development experience: * Document coverage reporting capabilities * Provide clear testing and development guidelines * Include CI/CD integration documentation * Add code quality tool usage instructions All tests pass with comprehensive coverage reporting and lint-free code.

Signed-off-by: Ramon Figueiredo <[email protected]>

lithorus · 2025-09-22T21:59:46Z

@Aniketsy
The updated dependencies in this PR for the tests in cueman are not in line with the main dependencies and breaks the coverage reports.
Please create another PR one brining them in line.

Aniketsy · 2025-09-22T22:16:22Z

@lithorus Sure I will do that .

Aniketsy · 2025-09-23T16:00:50Z

@ramonfigueiredo Just want to confirm before I proceed and make the changes as per suggestions.

Aniketsy added 5 commits September 17, 2025 16:01

Add comprehensive unit tests for terminateJobs logic

63ade91

Add integration tests for complete cueman workflows

b3af81a

Add integration tests for complete cueman workflows

e6be31a

Add integration tests for complete cueman workflows

69dc6c9

Add integration tests for complete cueman workflows

06f5862

Aniketsy requested review from DiegoTavares, lithorus and ramonfigueiredo as code owners September 18, 2025 07:25

Aniketsy added 2 commits September 18, 2025 13:08

Add integration tests for complete cueman workflows

4008509

Add integration tests for complete cueman workflows

798c20b

Aniketsy marked this pull request as draft September 18, 2025 08:43

Aniketsy added 4 commits September 18, 2025 14:38

Add integration tests for complete cueman workflows

00e85e3

Add integration tests for complete cueman workflows

8863f51

Add integration tests for complete cueman workflows

9223b6c

Add integration tests for complete cueman workflows

8da624c

Aniketsy marked this pull request as ready for review September 18, 2025 10:46

ramonfigueiredo marked this pull request as draft September 18, 2025 18:34

Aniketsy and others added 2 commits September 19, 2025 00:27

Add integration tests for complete cueman workflows

658002d

ramonfigueiredo force-pushed the cueman-integration-tests branch 3 times, most recently from faba875 to 7bd5455 Compare September 18, 2025 19:18

ramonfigueiredo changed the title ~~Add Integration Tests for Complete Cueman Workflows~~ [cueman] Add integration tests for complete cueman workflows Sep 18, 2025

ramonfigueiredo changed the title ~~[cueman] Add integration tests for complete cueman workflows~~ [cueman] Add comprehensive test infrastructure and update documentation for cueman Sep 18, 2025

ramonfigueiredo changed the title ~~[cueman] Add comprehensive test infrastructure and update documentation for cueman~~ [cueman] Add comprehensive test infrastructure and update documentation for Cueman Sep 18, 2025

ramonfigueiredo marked this pull request as ready for review September 18, 2025 19:27

ramonfigueiredo approved these changes Sep 18, 2025

View reviewed changes

ramonfigueiredo force-pushed the cueman-integration-tests branch 4 times, most recently from 2e7af24 to f3eb248 Compare September 18, 2025 19:41

ramonfigueiredo force-pushed the cueman-integration-tests branch from f3eb248 to 0a31c6e Compare September 18, 2025 19:59

AcademySoftwareFoundation deleted a comment from Aniketsy Sep 18, 2025

ramonfigueiredo marked this pull request as draft September 18, 2025 20:06

ramonfigueiredo marked this pull request as ready for review September 18, 2025 20:07

ramonfigueiredo marked this pull request as draft September 18, 2025 20:13

Merge branch 'master' into cueman-integration-tests

327192f

Signed-off-by: Ramon Figueiredo <[email protected]>

ramonfigueiredo assigned ramonfigueiredo and Aniketsy Sep 18, 2025

ramonfigueiredo marked this pull request as ready for review September 18, 2025 20:39

ramonfigueiredo merged commit 90c3c15 into AcademySoftwareFoundation:master Sep 18, 2025
20 checks passed

ramonfigueiredo added the devdays25 label Oct 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[cueman] Add comprehensive test infrastructure and update documentation for Cueman #1954

[cueman] Add comprehensive test infrastructure and update documentation for Cueman #1954

Uh oh!

Aniketsy commented Sep 18, 2025 •

edited by ramonfigueiredo

Loading

Uh oh!

Aniketsy commented Sep 18, 2025

Uh oh!

ramonfigueiredo commented Sep 18, 2025

Uh oh!

ramonfigueiredo commented Sep 18, 2025 •

edited

Loading

Uh oh!

ramonfigueiredo commented Sep 18, 2025

Uh oh!

ramonfigueiredo commented Sep 18, 2025 •

edited

Loading

Uh oh!

Uh oh!

lithorus commented Sep 22, 2025

Uh oh!

Aniketsy commented Sep 22, 2025

Uh oh!

Aniketsy commented Sep 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[cueman] Add comprehensive test infrastructure and update documentation for Cueman #1954

[cueman] Add comprehensive test infrastructure and update documentation for Cueman #1954

Uh oh!

Conversation

Aniketsy commented Sep 18, 2025 • edited by ramonfigueiredo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Aniketsy commented Sep 18, 2025

Uh oh!

ramonfigueiredo commented Sep 18, 2025

Uh oh!

ramonfigueiredo commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ramonfigueiredo commented Sep 18, 2025

Uh oh!

ramonfigueiredo commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1) Running Cueman tests

2) Install with development dependencies

3) Run the test suite

4) Format and lint code

Uh oh!

Uh oh!

lithorus commented Sep 22, 2025

Uh oh!

Aniketsy commented Sep 22, 2025

Uh oh!

Aniketsy commented Sep 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Aniketsy commented Sep 18, 2025 •

edited by ramonfigueiredo

Loading

ramonfigueiredo commented Sep 18, 2025 •

edited

Loading

ramonfigueiredo commented Sep 18, 2025 •

edited

Loading