diff --git a/cueadmin/README.md b/cueadmin/README.md index 6567bdb07..cffb3f0d5 100644 --- a/cueadmin/README.md +++ b/cueadmin/README.md @@ -2,3 +2,199 @@ CueAdmin is a command-line administration tool for OpenCue that provides full control over jobs, layers, frames, and hosts. It allows administrators to perform advanced management tasks such as setting priorities, killing jobs, or managing resource allocation. It's written in Python and provides a thin layer over the OpenCue Python API. +## Table of Contents + +- [Installation](#installation) +- [Usage](#usage) +- [Running Tests](#running-tests) +- [Contributing](#contributing) + +## Installation + +Install CueAdmin with: + +```bash +pip install opencue-cueadmin +``` + +For development: + +```bash +# Clone repository and install with test dependencies +git clone https://github.com/AcademySoftwareFoundation/OpenCue.git +cd OpenCue/cueadmin +pip install -e ".[dev]" +``` + +## Usage + +Basic CueAdmin commands: + +```bash +# List jobs +cueadmin -lj + +# List hosts +cueadmin -lh + +# Kill a job +cueadmin -kill-job JOB_NAME + +# Set job priority +cueadmin -priority JOB_NAME 100 +``` + +For full documentation, see the [OpenCue Documentation](https://opencue.io/docs/). + +## Running Tests + +CueAdmin includes a comprehensive test suite covering allocation management, output formatting, and core functionality. + +### Quick Start + +```bash +# Install with test dependencies +pip install -e ".[test]" + +# Run all tests +pytest + +# Run with coverage +pytest --cov=cueadmin --cov-report=term-missing +``` + +### Test Infrastructure + +**Test Dependencies:** +- `pytest>=8.0.0` - Modern test framework +- `pytest-cov>=4.0.0` - Coverage reporting +- `pytest-mock>=3.10.0` - Enhanced mocking +- `mock>=4.0.0` - Core mocking library +- `pyfakefs>=5.2.3` - Filesystem mocking + +**Test Types:** +- **Unit tests** - Function-level testing (`tests/test_*.py`) +- **Integration tests** - Command workflow testing +- **Allocation tests** - Allocation management functionality (`tests/test_allocation_commands.py`) + +### Running Tests + +```bash +# Basic test run +pytest tests/ + +# Verbose output +pytest -v + +# Run specific test file +pytest tests/test_allocation_commands.py + +# Run with coverage and HTML report +pytest --cov=cueadmin --cov-report=html --cov-report=term-missing + +# Use the convenience script +./run_tests.sh --coverage --html +``` + +### Coverage Reporting + +```bash +# Terminal coverage report +pytest --cov=cueadmin --cov-report=term-missing + +# HTML coverage report (generates htmlcov/ directory) +pytest --cov=cueadmin --cov-report=html + +# XML coverage for CI/CD +pytest --cov=cueadmin --cov-report=xml +``` + +### Development Testing + +**For contributors:** + +```bash +# Install development dependencies +pip install -e ".[dev]" + +# Run all tests with linting +pytest && pylint cueadmin tests + +# Run tests across Python versions (requires tox) +tox + +# Format code +black cueadmin tests +isort cueadmin tests +``` + +**CI/CD Integration:** + +```bash +# In OpenCue root directory +./ci/run_python_tests.sh # Includes cueadmin tests +./ci/run_python_lint.sh # Includes cueadmin linting + +# Run cueadmin tests specifically +cd cueadmin && python -m pytest tests/ +``` + +### Test Configuration + +Tests are configured via `pyproject.toml`: +- **pytest.ini_options** - Test discovery and execution +- **coverage settings** - Coverage reporting configuration +- **markers** - Test categorization (unit, integration, slow) + +### Continuous Integration + +The test suite is integrated into: +- **GitHub Actions** - Automated testing on PRs +- **Docker builds** - Container-based testing +- **Lint pipeline** - Code quality checks + +## Contributing + +We welcome contributions to CueAdmin! The project includes comprehensive development infrastructure: + +### Development Setup + +```bash +# Clone and setup +git clone https://github.com/AcademySoftwareFoundation/OpenCue.git +cd OpenCue/cueadmin + +# Install with development dependencies +pip install -e ".[dev]" +``` + +### Testing and Quality + +```bash +# Run comprehensive test suite +pytest --cov=cueadmin --cov-report=term-missing + +# Code formatting and linting +black cueadmin tests && isort cueadmin tests +pylint cueadmin tests + +# Multi-environment testing +tox +``` + +### Project Quality + +- **Comprehensive test coverage** with unit and integration tests +- **Modern testing infrastructure** using pytest, coverage, and CI/CD +- **Code quality tools** including pylint, black, and isort +- **Multi-Python version support** via tox +- **Docker support** for containerized development + +For detailed contribution guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md). + +### Get Involved + +- **Report Issues**: [GitHub Issues](https://github.com/AcademySoftwareFoundation/OpenCue/issues) +- **Contribute Code**: Submit pull requests with tests and documentation +- **Improve Documentation**: Help enhance tutorials and reference docs +- **Share Use Cases**: Contribute real-world examples and workflows diff --git a/cueadmin/cueadmin/__main__.py b/cueadmin/cueadmin/__main__.py index 2a354f3a2..3130b9bf4 100644 --- a/cueadmin/cueadmin/__main__.py +++ b/cueadmin/cueadmin/__main__.py @@ -16,15 +16,12 @@ """Entrypoint for CueAdmin tool.""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function import logging import cueadmin.common - logger = logging.getLogger("opencue.tools.cueadmin") @@ -41,5 +38,5 @@ def main(): cueadmin.common.handleParserException(args, e) -if __name__ == '__main__': +if __name__ == "__main__": main() diff --git a/cueadmin/cueadmin/common.py b/cueadmin/cueadmin/common.py index 8b38a2837..3126c1ccc 100644 --- a/cueadmin/cueadmin/common.py +++ b/cueadmin/cueadmin/common.py @@ -16,16 +16,13 @@ """Main CueAdmin code.""" -from __future__ import print_function -from __future__ import absolute_import -from __future__ import division +from __future__ import absolute_import, division, print_function -from builtins import str -from builtins import object import argparse import logging import sys import traceback +from builtins import object, str import opencue import opencue.wrappers.job @@ -34,29 +31,30 @@ import cueadmin.output import cueadmin.util - logger = logging.getLogger("opencue.tools.cueadmin") -__ALL__ = ["testServer", - "handleCommonArgs", - "handleParserException", - "handFloatCriterion", - "getCommonParser", - "setCommonQueryArgs", - "handleCommonQueryArgs", - "resolveJobNames", - "resolveHostNames", - "resolveShowNames", - "confirm", - "formatTime", - "formatDuration", - "formatLongDuration", - "formatMem", - "cutoff", - "ActionUtil", - "DependUtil", - "Convert", - "AllocUtil"] +__ALL__ = [ + "testServer", + "handleCommonArgs", + "handleParserException", + "handFloatCriterion", + "getCommonParser", + "setCommonQueryArgs", + "handleCommonQueryArgs", + "resolveJobNames", + "resolveHostNames", + "resolveShowNames", + "confirm", + "formatTime", + "formatDuration", + "formatLongDuration", + "formatMem", + "cutoff", + "ActionUtil", + "DependUtil", + "Convert", + "AllocUtil", +] # pylint: disable=broad-except @@ -67,200 +65,375 @@ def handleParserException(args, e): traceback.print_exc(file=sys.stderr) raise e except ValueError as ex: - print("Error: %s. Try the -verbose or -h flags for more info." % ex, file=sys.stderr) + print( + "Error: %s. Try the -verbose or -h flags for more info." % ex, + file=sys.stderr, + ) except Exception as ex: print("Error: %s." % ex, file=sys.stderr) def getParser(): """Constructs and returns the CueAdmin argument parser.""" - parser = argparse.ArgumentParser(description="CueAdmin OpenCue Administrator Tool", - formatter_class=argparse.RawDescriptionHelpFormatter) + parser = argparse.ArgumentParser( + description="CueAdmin OpenCue Administrator Tool", + formatter_class=argparse.RawDescriptionHelpFormatter, + ) general = parser.add_argument_group("General Options") - general.add_argument("-server", action='store', nargs="+", metavar='HOSTNAME', - help='Specify cuebot addres(s).') - general.add_argument("-facility", action='store', metavar='CODE', - help='Specify the facility code.') - general.add_argument("-verbose", "-v", action='store_true', - help='Turn on verbose logging.') - general.add_argument("-force", action='store_true', - help='Force operations that usually require confirmation.') + general.add_argument( + "-server", + action="store", + nargs="+", + metavar="HOSTNAME", + help="Specify cuebot addres(s).", + ) + general.add_argument( + "-facility", action="store", metavar="CODE", help="Specify the facility code." + ) + general.add_argument( + "-verbose", "-v", action="store_true", help="Turn on verbose logging." + ) + general.add_argument( + "-force", + action="store_true", + help="Force operations that usually require confirmation.", + ) query = parser.add_argument_group("Query Options") - query.add_argument("-lj", "-laj", action=QueryAction, nargs="*", metavar="SUBSTR", - help="List jobs with optional name substring match.") - - query.add_argument("-lji", action=QueryAction, nargs="*", metavar="SUBSTR", - help="List job info with optional name substring match.") + query.add_argument( + "-lj", + "-laj", + action=QueryAction, + nargs="*", + metavar="SUBSTR", + help="List jobs with optional name substring match.", + ) + + query.add_argument( + "-lji", + action=QueryAction, + nargs="*", + metavar="SUBSTR", + help="List job info with optional name substring match.", + ) query.add_argument("-ls", action="store_true", help="List shows.") query.add_argument("-la", action="store_true", help="List allocations.") - query.add_argument("-lb", action="store", nargs="+", help="List subscriptions.", metavar="SHOW") - query.add_argument("-lba", action="store", metavar="ALLOC", - help="List all subscriptions to a specified allocation.") - - query.add_argument("-lp", "-lap", action="store", nargs="*", - metavar="[SHOW ...] [-host HOST ...] [-alloc ...] [-job JOB ...] " - "[-memory ...] [-limit ...]", - help="List running procs. Optionally filter by show, show, memory, alloc. " - "Use -limit to limit the results to N procs.") - - query.add_argument("-ll", "-lal", action="store", nargs="*", - metavar="[SHOW ...] [-host HOST ...] [-alloc ...] [-job JOB ...] " - "[-memory ...] [-limit ...]", - help="List running frame log paths. Optionally filter by show, show, memory" - ", alloc. Use -limit to limit the results to N logs.") - - query.add_argument("-lh", action=QueryAction, nargs="*", - metavar="[SUBSTR ...] [-state STATE] [-alloc ALLOC]", - help="List hosts with optional name substring match.") - - query.add_argument("-lv", action="store", nargs="*", metavar="[SHOW]", - help="List default services.") - - query.add_argument("-query", "-q", nargs="+", action="store", default=[], - help=argparse.SUPPRESS) + query.add_argument( + "-lb", action="store", nargs="+", help="List subscriptions.", metavar="SHOW" + ) + query.add_argument( + "-lba", + action="store", + metavar="ALLOC", + help="List all subscriptions to a specified allocation.", + ) + + query.add_argument( + "-lp", + "-lap", + action="store", + nargs="*", + metavar="[SHOW ...] [-host HOST ...] [-alloc ...] [-job JOB ...] " + "[-memory ...] [-limit ...]", + help="List running procs. Optionally filter by show, show, memory, alloc. " + "Use -limit to limit the results to N procs.", + ) + + query.add_argument( + "-ll", + "-lal", + action="store", + nargs="*", + metavar="[SHOW ...] [-host HOST ...] [-alloc ...] [-job JOB ...] " + "[-memory ...] [-limit ...]", + help="List running frame log paths. Optionally filter by show, show, memory" + ", alloc. Use -limit to limit the results to N logs.", + ) + + query.add_argument( + "-lh", + action=QueryAction, + nargs="*", + metavar="[SUBSTR ...] [-state STATE] [-alloc ALLOC]", + help="List hosts with optional name substring match.", + ) + + query.add_argument( + "-lv", + action="store", + nargs="*", + metavar="[SHOW]", + help="List default services.", + ) + + query.add_argument( + "-query", "-q", nargs="+", action="store", default=[], help=argparse.SUPPRESS + ) # # Filter # filter_grp = parser.add_argument_group("Filter Options") - filter_grp.add_argument("-job", nargs="+", metavar="JOB", action="store", default=[], - help="Filter proc or log search by job") - - filter_grp.add_argument("-alloc", nargs="+", metavar="ALLOC", action="store", default=[], - help="Filter host or proc search by allocation") - - filter_grp.add_argument("-memory", action="store", - help="Filters a list of procs by the amount of reserved memory. " - "Memory can be specified in one of 3 ways. As a range, " - "-. Less than, lt. Greater than, gt. " - "Values should be specified in GB.") - - filter_grp.add_argument("-duration", action="store", - help="Show procs that have been running longer than the specified " - "number of hours or within a specific time frame. Ex. -time 1.2 " - "or -time 3.5-4.5. Waiting frames are automatically filtered " - "out.") - - filter_grp.add_argument("-limit", action="store", default=0, - help="Limit the result of a proc search to N rows") - filter_grp.add_argument("-state", nargs="+", metavar="STATE", action="store", default=[], - #choices=["UP", "DOWN", "REPAIR"], type=str.upper, - help="Filter host search by hardware state, up or down.") + filter_grp.add_argument( + "-job", + nargs="+", + metavar="JOB", + action="store", + default=[], + help="Filter proc or log search by job", + ) + + filter_grp.add_argument( + "-alloc", + nargs="+", + metavar="ALLOC", + action="store", + default=[], + help="Filter host or proc search by allocation", + ) + + filter_grp.add_argument( + "-memory", + action="store", + help="Filters a list of procs by the amount of reserved memory. " + "Memory can be specified in one of 3 ways. As a range, " + "-. Less than, lt. Greater than, gt. " + "Values should be specified in GB.", + ) + + filter_grp.add_argument( + "-duration", + action="store", + help="Show procs that have been running longer than the specified " + "number of hours or within a specific time frame. Ex. -time 1.2 " + "or -time 3.5-4.5. Waiting frames are automatically filtered " + "out.", + ) + + filter_grp.add_argument( + "-limit", + action="store", + default=0, + help="Limit the result of a proc search to N rows", + ) + filter_grp.add_argument( + "-state", + nargs="+", + metavar="STATE", + action="store", + default=[], + # choices=["UP", "DOWN", "REPAIR"], type=str.upper, + help="Filter host search by hardware state, up or down.", + ) # # Show # show = parser.add_argument_group("Show Options") - show.add_argument("-create-show", action="store", metavar="SHOW", - help="create a new show") - - show.add_argument("-delete-show", action="store", metavar="SHOW", - help="delete specified show") - - show.add_argument("-disable-show", action="store", metavar="SHOW", - help="Disable the specified show") - - show.add_argument("-enable-show", action="store", metavar="SHOW", - help="Enable the specified show") - - show.add_argument("-dispatching", action="store", nargs=2, metavar="SHOW ON|OFF", - help="Enables frame dispatching on the specified show.") - - show.add_argument("-booking", action="store", nargs=2, metavar="SHOW ON|OFF", - help="Booking is new proc assignment. If booking is disabled " - "procs will continue to run on new jobs but no new jobs will " - "be booked.") - - show.add_argument("-default-min-cores", action="store", nargs=2, metavar="SHOW CORES", - help="The default min core value for all jobs before " - "any min core filers are applied.") - - show.add_argument("-default-max-cores", action="store", nargs=2, metavar="SHOW CORES", - help="The default max core value for all jobs before " - "any max core filters are applied.") + show.add_argument( + "-create-show", action="store", metavar="SHOW", help="create a new show" + ) + + show.add_argument( + "-delete-show", action="store", metavar="SHOW", help="delete specified show" + ) + + show.add_argument( + "-disable-show", + action="store", + metavar="SHOW", + help="Disable the specified show", + ) + + show.add_argument( + "-enable-show", action="store", metavar="SHOW", help="Enable the specified show" + ) + + show.add_argument( + "-dispatching", + action="store", + nargs=2, + metavar="SHOW ON|OFF", + help="Enables frame dispatching on the specified show.", + ) + + show.add_argument( + "-booking", + action="store", + nargs=2, + metavar="SHOW ON|OFF", + help="Booking is new proc assignment. If booking is disabled " + "procs will continue to run on new jobs but no new jobs will " + "be booked.", + ) + + show.add_argument( + "-default-min-cores", + action="store", + nargs=2, + metavar="SHOW CORES", + help="The default min core value for all jobs before " + "any min core filers are applied.", + ) + + show.add_argument( + "-default-max-cores", + action="store", + nargs=2, + metavar="SHOW CORES", + help="The default max core value for all jobs before " + "any max core filters are applied.", + ) # # Allocation # alloc = parser.add_argument_group("Allocation Options") - alloc.add_argument("-create-alloc", action="store", nargs=3, metavar="FACILITY ALLOC TAG", - help="Create a new allocation.") - alloc.add_argument("-delete-alloc", action="store", metavar="NAME", - help="Delete an allocation. It must be empty.") - alloc.add_argument("-rename-alloc", action="store", nargs=2, metavar="OLD NEW", - help="Rename allocation. New name must not contain facility prefix.") - alloc.add_argument("-transfer", action="store", nargs=2, metavar="OLD NEW", - help="Move all hosts from src alloc to dest alloc") - alloc.add_argument("-tag-alloc", action="store", nargs=2, metavar="ALLOC TAG", - help="Tag allocation.") + alloc.add_argument( + "-create-alloc", + action="store", + nargs=3, + metavar="FACILITY ALLOC TAG", + help="Create a new allocation.", + ) + alloc.add_argument( + "-delete-alloc", + action="store", + metavar="NAME", + help="Delete an allocation. It must be empty.", + ) + alloc.add_argument( + "-rename-alloc", + action="store", + nargs=2, + metavar="OLD NEW", + help="Rename allocation. New name must not contain facility prefix.", + ) + alloc.add_argument( + "-transfer", + action="store", + nargs=2, + metavar="OLD NEW", + help="Move all hosts from src alloc to dest alloc", + ) + alloc.add_argument( + "-tag-alloc", + action="store", + nargs=2, + metavar="ALLOC TAG", + help="Tag allocation.", + ) # # Subscription # sub = parser.add_argument_group("Subscription Options") - sub.add_argument("-create-sub", action="store", nargs=4, - help="Create new subcription.", metavar="SHOW ALLOC SIZE BURST") - sub.add_argument("-delete-sub", action="store", nargs=2, metavar="SHOW ALLOC", - help="Delete subscription") - sub.add_argument("-size", action="store", nargs=3, metavar="SHOW ALLOC SIZE", - help="Set the guaranteed number of cores.") - sub.add_argument("-burst", action="store", nargs=3, metavar="SHOW ALLOC BURST", - help="Set the number of burst cores for a subscription passing: " - " show allocation value" - "Use the percent sign in value to indicate a percentage " - "of the subscription size instead of a hard size.") + sub.add_argument( + "-create-sub", + action="store", + nargs=4, + help="Create new subcription.", + metavar="SHOW ALLOC SIZE BURST", + ) + sub.add_argument( + "-delete-sub", + action="store", + nargs=2, + metavar="SHOW ALLOC", + help="Delete subscription", + ) + sub.add_argument( + "-size", + action="store", + nargs=3, + metavar="SHOW ALLOC SIZE", + help="Set the guaranteed number of cores.", + ) + sub.add_argument( + "-burst", + action="store", + nargs=3, + metavar="SHOW ALLOC BURST", + help="Set the number of burst cores for a subscription passing: " + " show allocation value" + "Use the percent sign in value to indicate a percentage " + "of the subscription size instead of a hard size.", + ) # # Host # host = parser.add_argument_group("Host Options") - host.add_argument("-host", action="store", nargs="+", metavar="HOSTNAME", - help="Specify the host names to operate on") - host.add_argument("-hostmatch", "-hm", action="store", nargs="+", metavar="SUBSTR", - help="Specify a list of substring matches to match groups of hosts.") + host.add_argument( + "-host", + action="store", + nargs="+", + metavar="HOSTNAME", + help="Specify the host names to operate on", + ) + host.add_argument( + "-hostmatch", + "-hm", + action="store", + nargs="+", + metavar="SUBSTR", + help="Specify a list of substring matches to match groups of hosts.", + ) host.add_argument("-lock", action="store_true", help="lock hosts") host.add_argument("-unlock", action="store_true", help="unlock hosts") - host.add_argument("-move", action="store", metavar="ALLOC", - help="move hosts into a new allocation") + host.add_argument( + "-move", + action="store", + metavar="ALLOC", + help="move hosts into a new allocation", + ) host.add_argument("-delete-host", action="store_true", help="delete hosts") - host.add_argument("-safe-reboot", action="store_true", help="lock and reboot hosts when idle") - host.add_argument("-repair", action="store_true", help="Sets hosts into the repair state.") + host.add_argument( + "-safe-reboot", action="store_true", help="lock and reboot hosts when idle" + ) + host.add_argument( + "-repair", action="store_true", help="Sets hosts into the repair state." + ) host.add_argument("-fixed", action="store_true", help="Sets hosts into Up state.") - host.add_argument("-thread", action="store", help="Set the host's thread mode.", - choices=[ - mode.lower() for mode in list(opencue.api.host_pb2.ThreadMode.keys())]) + host.add_argument( + "-thread", + action="store", + help="Set the host's thread mode.", + choices=[mode.lower() for mode in list(opencue.api.host_pb2.ThreadMode.keys())], + ) return parser class QueryAction(argparse.Action): """Sets various query modes if arguments are detected.""" + def __call__(self, parser, namespace, values, option_string=None): - if option_string == '-lh': + if option_string == "-lh": namespace.lh = True namespace.query = values - elif option_string in ('-lj', '-laj'): + elif option_string in ("-lj", "-laj"): namespace.lj = True namespace.query = values - elif option_string == '-lji': + elif option_string == "-lji": namespace.lji = True namespace.query = values def handleFloatCriterion(mixed, convert=None): """handleFloatCriterion - returns the proper subclass of FloatSearchCriterion based on - input from the user. There are a few formats which are accepted. - - float/int - GreaterThanFloatSearchCriterion - string - - gt - GreaterThanFloatSearchCriterion - lt - LessThanFloatSearchCriterion - min-max - InRangeFloatSearchCriterion + returns the proper subclass of FloatSearchCriterion based on + input from the user. There are a few formats which are accepted. + + float/int - GreaterThanFloatSearchCriterion + string - + gt - GreaterThanFloatSearchCriterion + lt - LessThanFloatSearchCriterion + min-max - InRangeFloatSearchCriterion """ + def _convert(val): if not convert: return float(val) @@ -269,24 +442,31 @@ def _convert(val): criterions = [ opencue.api.criterion_pb2.GreaterThanFloatSearchCriterion, opencue.api.criterion_pb2.LessThanFloatSearchCriterion, - opencue.api.criterion_pb2.InRangeFloatSearchCriterion] + opencue.api.criterion_pb2.InRangeFloatSearchCriterion, + ] if isinstance(mixed, (float, int)): - result = opencue.api.criterion_pb2.GreaterThanFloatSearchCriterion(value=_convert(mixed)) + result = opencue.api.criterion_pb2.GreaterThanFloatSearchCriterion( + value=_convert(mixed) + ) elif isinstance(mixed, str): if mixed.startswith("gt"): result = opencue.api.criterion_pb2.GreaterThanFloatSearchCriterion( - value=_convert(mixed[2:])) + value=_convert(mixed[2:]) + ) elif mixed.startswith("lt"): result = opencue.api.criterion_pb2.LessThanFloatSearchCriterion( - value=_convert(mixed[2:])) + value=_convert(mixed[2:]) + ) elif mixed.find("-") > -1: min_value, max_value = mixed.split("-", 1) - result = opencue.api.criterion_pb2.InRangeFloatSearchCriterion(min=_convert(min_value), - max=_convert(max_value)) + result = opencue.api.criterion_pb2.InRangeFloatSearchCriterion( + min=_convert(min_value), max=_convert(max_value) + ) else: result = opencue.api.criterion_pb2.GreaterThanFloatSearchCriterion( - value=_convert(mixed)) + value=_convert(mixed) + ) # pylint: disable=use-a-generator elif any([isinstance(mixed.__class__, crit_cls) for crit_cls in criterions]): result = mixed @@ -300,15 +480,16 @@ def _convert(val): def handleIntCriterion(mixed, convert=None): """handleIntCriterion - returns the proper subclass of IntSearchCriterion based on - input from the user. There are a few formats which are accepted. - - float/int - GreaterThanFloatSearchCriterion - string - - gt - GreaterThanFloatSearchCriterion - lt - LessThanFloatSearchCriterion - min-max - InRangeFloatSearchCriterion + returns the proper subclass of IntSearchCriterion based on + input from the user. There are a few formats which are accepted. + + float/int - GreaterThanFloatSearchCriterion + string - + gt - GreaterThanFloatSearchCriterion + lt - LessThanFloatSearchCriterion + min-max - InRangeFloatSearchCriterion """ + def _convert(val): if not convert: return int(val) @@ -317,24 +498,31 @@ def _convert(val): criterions = [ opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion, opencue.api.criterion_pb2.LessThanIntegerSearchCriterion, - opencue.api.criterion_pb2.InRangeIntegerSearchCriterion] + opencue.api.criterion_pb2.InRangeIntegerSearchCriterion, + ] if isinstance(mixed, (float, int)): - result = opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=_convert(mixed)) + result = opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion( + value=_convert(mixed) + ) elif isinstance(mixed, str): if mixed.startswith("gt"): result = opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion( - value=_convert(mixed[2:])) + value=_convert(mixed[2:]) + ) elif mixed.startswith("lt"): result = opencue.api.criterion_pb2.LessThanIntegerSearchCriterion( - value=_convert(mixed[2:])) + value=_convert(mixed[2:]) + ) elif mixed.find("-") > -1: min_value, max_value = mixed.split("-", 1) result = opencue.api.criterion_pb2.InRangeIntegerSearchCriterion( - min=_convert(min_value), max=_convert(max_value)) + min=_convert(min_value), max=_convert(max_value) + ) else: result = opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion( - value=_convert(mixed)) + value=_convert(mixed) + ) # pylint: disable=use-a-generator elif any([isinstance(mixed.__class__, crit_cls) for crit_cls in criterions]): result = mixed @@ -353,12 +541,17 @@ def resolveHostNames(names=None, substr=None): items = opencue.search.HostSearch.byName(names) logger.debug("found %d of %d supplied hosts", len(items), len(names)) if len(names) != len(items) and items: - logger.warning("Unable to match all host names with valid hosts on the cue.") logger.warning( - "Operations executed for %s", set(names).intersection([i.data.name for i in items])) + "Unable to match all host names with valid hosts on the cue." + ) logger.warning( - "Operations NOT executed for %s", set(names).difference( - [i.data.name for i in items])) + "Operations executed for %s", + set(names).intersection([i.data.name for i in items]), + ) + logger.warning( + "Operations NOT executed for %s", + set(names).difference([i.data.name for i in items]), + ) elif substr: items = opencue.search.HostSearch.byMatch(substr) logger.debug("matched %d hosts using patterns %s", len(items), substr) @@ -378,10 +571,14 @@ def resolveShowNames(names): logger.debug("found %d of %d supplied shows", len(items), len(names)) if len(names) != len(items) and items: logger.warning("Unable to match all show names with active shows.") - logger.warning("Operations executed for %s", set(names).intersection( - [i.data.name for i in items])) - logger.warning("Operations NOT executed for %s", set(names).difference( - [i.data.name for i in items])) + logger.warning( + "Operations executed for %s", + set(names).intersection([i.data.name for i in items]), + ) + logger.warning( + "Operations NOT executed for %s", + set(names).difference([i.data.name for i in items]), + ) if not items: raise ValueError("no valid shows") return items @@ -418,7 +615,9 @@ def dropAllDepends(job, layer=None, frame=None): logger.debug("dropping all depends on: %s", job) depend_er_job = opencue.api.findJob(job) for depend in depend_er_job.getWhatThisDependsOn(): - logger.debug("dropping depend %s %s", depend.data.type, opencue.id(depend)) + logger.debug( + "dropping depend %s %s", depend.data.type, opencue.id(depend) + ) depend.satisfy() @@ -522,21 +721,25 @@ def getValue(a): def setValue(act, value): """Sets an action's value.""" if act.type == opencue.api.filter_pb2.MOVE_JOB_TO_GROUP: - act.groupValue = opencue.proxy(value, 'Group') + act.groupValue = opencue.proxy(value, "Group") act.valueType = opencue.api.filter_pb2.GROUP_TYPE elif act.type == opencue.api.filter_pb2.PAUSE_JOB: act.booleanValue = value act.valueType = opencue.api.filter_pb2.BOOLEAN_TYPE - elif act.type in (opencue.api.filter_pb2.SET_JOB_PRIORITY, - opencue.api.filter_pb2.SET_ALL_RENDER_LAYER_MEMORY): + elif act.type in ( + opencue.api.filter_pb2.SET_JOB_PRIORITY, + opencue.api.filter_pb2.SET_ALL_RENDER_LAYER_MEMORY, + ): act.integerValue = int(value) act.valueType = opencue.api.filter_pb2.INTEGER_TYPE - elif act.type in (opencue.api.filter_pb2.SET_JOB_MIN_CORES, - opencue.api.filter_pb2.SET_JOB_MAX_CORES, - opencue.api.filter_pb2.SET_ALL_RENDER_LAYER_CORES): + elif act.type in ( + opencue.api.filter_pb2.SET_JOB_MIN_CORES, + opencue.api.filter_pb2.SET_JOB_MAX_CORES, + opencue.api.filter_pb2.SET_ALL_RENDER_LAYER_CORES, + ): act.floatValue = float(value) act.valueType = opencue.api.filter_pb2.FLOAT_TYPE @@ -580,19 +783,28 @@ def handleArgs(args): alloc=args.alloc, job=args.job, memory=handleIntCriterion(args.memory, Convert.gigsToKB), - duration=handleIntCriterion(args.duration, Convert.hoursToSeconds)) + duration=handleIntCriterion(args.duration, Convert.hoursToSeconds), + ) if isinstance(args.ll, list): - print("\n".join( - [opencue.wrappers.proc.Proc(proc).data.log_path for proc in result.procs.procs])) + print( + "\n".join( + [ + opencue.wrappers.proc.Proc(proc).data.log_path + for proc in result.procs.procs + ] + ) + ) else: cueadmin.output.displayProcs( - [opencue.wrappers.proc.Proc(proc) for proc in result.procs.procs]) + [opencue.wrappers.proc.Proc(proc) for proc in result.procs.procs] + ) return if args.lh: states = [Convert.strToHardwareState(s) for s in args.state] cueadmin.output.displayHosts( - opencue.api.getHosts(match=args.query, state=states, alloc=args.alloc)) + opencue.api.getHosts(match=args.query, state=states, alloc=args.alloc) + ) return if args.lba: @@ -615,8 +827,11 @@ def handleArgs(args): if args.lji: cueadmin.output.displayJobs( - [opencue.wrappers.job.Job(job) - for job in opencue.search.JobSearch.byMatch(args.query).jobs.jobs]) + [ + opencue.wrappers.job.Job(job) + for job in opencue.search.JobSearch.byMatch(args.query).jobs.jobs + ] + ) return if args.la: @@ -625,7 +840,9 @@ def handleArgs(args): if args.lb: for show in resolveShowNames(args.lb): - cueadmin.output.displaySubscriptions(show.getSubscriptions(), show.data.name) + cueadmin.output.displaySubscriptions( + show.getSubscriptions(), show.data.name + ) return if args.ls: @@ -647,11 +864,18 @@ def handleArgs(args): fac, name, tag = args.create_alloc confirm( "Create new allocation %s.%s, with tag %s" % (fac, name, tag), - args.force, createAllocation, fac, name, tag) + args.force, + createAllocation, + fac, + name, + tag, + ) elif args.delete_alloc: allocation = opencue.api.findAllocation(args.delete_alloc) - confirm("Delete allocation %s" % args.delete_alloc, args.force, allocation.delete) + confirm( + "Delete allocation %s" % args.delete_alloc, args.force, allocation.delete + ) elif args.rename_alloc: old, new = args.rename_alloc @@ -662,43 +886,72 @@ def handleArgs(args): confirm( "Rename allocation from %s to %s" % (old, new), - args.force, opencue.api.findAllocation(old).setName, new) + args.force, + opencue.api.findAllocation(old).setName, + new, + ) elif args.transfer: src = opencue.api.findAllocation(args.transfer[0]) dst = opencue.api.findAllocation(args.transfer[1]) confirm( "Transfer hosts from from %s to %s" % (src.data.name, dst.data.name), - args.force, dst.reparentHosts, src.getHosts()) + args.force, + dst.reparentHosts, + src.getHosts(), + ) elif args.tag_alloc: alloc, tag = args.tag_alloc - confirm("Re-tag allocation %s with %s" % (alloc, tag), - args.force, opencue.api.findAllocation(alloc).setTag, tag) + confirm( + "Re-tag allocation %s with %s" % (alloc, tag), + args.force, + opencue.api.findAllocation(alloc).setTag, + tag, + ) # # Shows # elif args.create_show: - confirm("Create new show %s" % args.create_show, - args.force, opencue.api.createShow, args.create_show) + confirm( + "Create new show %s" % args.create_show, + args.force, + opencue.api.createShow, + args.create_show, + ) elif args.delete_show: - confirm("Delete show %s" % args.delete_show, - args.force, opencue.api.findShow(args.delete_show).delete) + confirm( + "Delete show %s" % args.delete_show, + args.force, + opencue.api.findShow(args.delete_show).delete, + ) elif args.disable_show: - confirm("Disable show %s" % args.disable_show, - args.force, opencue.api.findShow(args.disable_show).setActive, False) + confirm( + "Disable show %s" % args.disable_show, + args.force, + opencue.api.findShow(args.disable_show).setActive, + False, + ) elif args.enable_show: - confirm("Enable show %s" % args.enable_show, - args.force, opencue.api.findShow(args.enable_show).setActive, True) + confirm( + "Enable show %s" % args.enable_show, + args.force, + opencue.api.findShow(args.enable_show).setActive, + True, + ) elif args.dispatching: show = opencue.api.findShow(args.dispatching[0]) enabled = Convert.stringToBoolean(args.dispatching[1]) if not enabled: - confirm("Disable dispatching on %s" % opencue.rep(show), - args.force, show.enableDispatching, enabled) + confirm( + "Disable dispatching on %s" % opencue.rep(show), + args.force, + show.enableDispatching, + enabled, + ) else: show.enableDispatching(True) @@ -706,22 +959,30 @@ def handleArgs(args): show = opencue.api.findShow(args.booking[0]) enabled = Convert.stringToBoolean(args.booking[1]) if not enabled: - confirm("Disable booking on %s" % opencue.rep(show), - args.force, show.enableBooking, False) + confirm( + "Disable booking on %s" % opencue.rep(show), + args.force, + show.enableBooking, + False, + ) else: show.enableBooking(True) elif args.default_min_cores: - confirm("Set the default min cores to: %s" % - args.default_min_cores[1], args.force, - opencue.api.findShow(args.default_min_cores[0]).setDefaultMinCores, - float(int(args.default_min_cores[1]))) + confirm( + "Set the default min cores to: %s" % args.default_min_cores[1], + args.force, + opencue.api.findShow(args.default_min_cores[0]).setDefaultMinCores, + float(int(args.default_min_cores[1])), + ) elif args.default_max_cores: - confirm("Set the default max cores to: %s" % - args.default_max_cores[1], args.force, - opencue.api.findShow(args.default_max_cores[0]).setDefaultMaxCores, - float(int(args.default_max_cores[1]))) + confirm( + "Set the default max cores to: %s" % args.default_max_cores[1], + args.force, + opencue.api.findShow(args.default_max_cores[0]).setDefaultMaxCores, + float(int(args.default_max_cores[1])), + ) # # Hosts are handled a bit differently than the rest # of the entities. To specify a host or hosts the user @@ -752,8 +1013,13 @@ def moveHosts(hosts_, dst_): logger.debug("moving %s to %s", opencue.rep(host_), opencue.rep(dst_)) host_.setAllocation(dst_) - confirm("Move %d hosts to %s" % (len(hosts), args.move), - args.force, moveHosts, hosts, opencue.api.findAllocation(args.move)) + confirm( + "Move %d hosts to %s" % (len(hosts), args.move), + args.force, + moveHosts, + hosts, + opencue.api.findAllocation(args.move), + ) elif args.delete_host: if not hosts: @@ -772,11 +1038,17 @@ def deleteHosts(hosts_): def safeReboot(hosts_): for host_ in hosts_: - logger.debug("locking host and rebooting when idle %s", opencue.rep(host_)) + logger.debug( + "locking host and rebooting when idle %s", opencue.rep(host_) + ) host_.rebootWhenIdle() - confirm("Lock and reboot %d hosts when idle" % len(hosts), - args.force, safeReboot, hosts) + confirm( + "Lock and reboot %d hosts when idle" % len(hosts), + args.force, + safeReboot, + hosts, + ) elif args.thread: if not hosts: @@ -787,8 +1059,13 @@ def setThreadMode(hosts_, mode): logger.debug("setting host %s to thread mode %s", host_.data.name, mode) host_.setThreadMode(Convert.strToThreadMode(mode)) - confirm("Set %d hosts to thread mode %s" % (len(hosts), args.thread), args.force, - setThreadMode, hosts, args.thread) + confirm( + "Set %d hosts to thread mode %s" % (len(hosts), args.thread), + args.force, + setThreadMode, + hosts, + args.thread, + ) elif args.repair: if not hosts: @@ -799,8 +1076,12 @@ def setRepairState(hosts_): logger.debug("setting host into the repair state %s", host_.data.name) host_.setHardwareState(opencue.api.host_pb2.REPAIR) - confirm("Set %d hosts into the Repair state?" % len(hosts), - args.force, setRepairState, hosts) + confirm( + "Set %d hosts into the Repair state?" % len(hosts), + args.force, + setRepairState, + hosts, + ) elif args.fixed: if not hosts: @@ -811,21 +1092,33 @@ def setUpState(hosts_): logger.debug("setting host into the repair state %s", host_.data.name) host_.setHardwareState(opencue.api.host_pb2.UP) - confirm("Set %d hosts into the Up state?" % len(hosts), - args.force, setUpState, hosts) + confirm( + "Set %d hosts into the Up state?" % len(hosts), + args.force, + setUpState, + hosts, + ) elif args.create_sub: show = opencue.api.findShow(args.create_sub[0]) alloc = opencue.api.findAllocation(args.create_sub[1]) - confirm("Create subscription for %s on %s" % (opencue.rep(show), opencue.rep(alloc)), - args.force, show.createSubscription, - alloc.data, float(args.create_sub[2]), float(args.create_sub[3])) + confirm( + "Create subscription for %s on %s" + % (opencue.rep(show), opencue.rep(alloc)), + args.force, + show.createSubscription, + alloc.data, + float(args.create_sub[2]), + float(args.create_sub[3]), + ) elif args.delete_sub: sub_name = "%s.%s" % (args.delete_sub[1], args.delete_sub[0]) - confirm("Delete %s's subscription to %s" % - (args.delete_sub[0], args.delete_sub[1]), - args.force, opencue.api.findSubscription(sub_name).delete) + confirm( + "Delete %s's subscription to %s" % (args.delete_sub[0], args.delete_sub[1]), + args.force, + opencue.api.findSubscription(sub_name).delete, + ) elif args.size: sub_name = "%s.%s" % (args.size[1], args.size[0]) diff --git a/cueadmin/cueadmin/format.py b/cueadmin/cueadmin/format.py index 6321c7c33..0169dd1fe 100644 --- a/cueadmin/cueadmin/format.py +++ b/cueadmin/cueadmin/format.py @@ -16,9 +16,7 @@ """Functions for formatting text output.""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function import time @@ -64,10 +62,12 @@ def formatDuration(sec): :param sec: duration in seconds :rtype: str :return: duration formatted in HH:MM:SS format.""" + def splitTime(seconds): minutes, seconds = divmod(seconds, 60) hour, minutes = divmod(minutes, 60) return hour, minutes, seconds + return "%02d:%02d:%02d" % splitTime(sec) @@ -78,11 +78,13 @@ def formatLongDuration(sec): :param sec: duration in seconds :rtype: str :return: duration formatted in days:hours format.""" + def splitTime(seconds): minutes, seconds = divmod(seconds, 60) hour, minutes = divmod(minutes, 60) days, hour = divmod(hour, 24) return days, hour + return "%02d:%02d" % splitTime(sec) @@ -112,6 +114,6 @@ def cutoff(s, length): :param length: max number of characters :rtype: str :return: truncated string""" - if len(s) < length-2: + if len(s) < length - 2: return s - return "%s.." % s[0:length-2] + return "%s.." % s[0 : length - 2] diff --git a/cueadmin/cueadmin/output.py b/cueadmin/cueadmin/output.py index 1c6d8d943..dc38b5c11 100644 --- a/cueadmin/cueadmin/output.py +++ b/cueadmin/cueadmin/output.py @@ -16,10 +16,8 @@ """Functions for displaying output to the terminal.""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import -from __future__ import unicode_literals +from __future__ import (absolute_import, division, print_function, + unicode_literals) import time @@ -39,16 +37,23 @@ def displayProcs(procs): proc_format = "%-10s %-7s %-24s %-30s / %-30s %-12s %-12s" print(proc_format % ("Host", "Cores", "Memory", "Job", "Frame", "Start", "Runtime")) for proc in procs: - print(proc_format % (proc.data.name.split("/")[0], - "%0.2f" % proc.data.reserved_cores, - "%s of %s (%0.2f%%)" % ( - cueadmin.format.formatMem(proc.data.used_memory), - cueadmin.format.formatMem(proc.data.reserved_memory), - (proc.data.used_memory / float(proc.data.reserved_memory) * 100)), - cueadmin.format.cutoff(proc.data.job_name, 30), - cueadmin.format.cutoff(proc.data.frame_name, 30), - cueadmin.format.formatTime(proc.data.dispatch_time), - cueadmin.format.formatDuration(time.time() - proc.data.dispatch_time))) + print( + proc_format + % ( + proc.data.name.split("/")[0], + "%0.2f" % proc.data.reserved_cores, + "%s of %s (%0.2f%%)" + % ( + cueadmin.format.formatMem(proc.data.used_memory), + cueadmin.format.formatMem(proc.data.reserved_memory), + (proc.data.used_memory / float(proc.data.reserved_memory) * 100), + ), + cueadmin.format.cutoff(proc.data.job_name, 30), + cueadmin.format.cutoff(proc.data.frame_name, 30), + cueadmin.format.formatTime(proc.data.dispatch_time), + cueadmin.format.formatDuration(time.time() - proc.data.dispatch_time), + ) + ) def displayHosts(hosts): @@ -56,26 +61,56 @@ def displayHosts(hosts): @type hosts: list @param hosts: Hosts to display information about """ - host_format = "%-15s %-4s %-5s %-8s %-8s %-9s %-5s %-5s %-16s %-8s %-8s %-6s %-9s %-10s %-7s" - print(host_format % ("Host", "Load", "NIMBY", "freeMem", "freeSwap", "freeMcp", "Cores", "Mem", - "Idle", "Os", "Uptime", "State", "Locked", "Alloc", "Thread")) + host_format = ( + "%-15s %-4s %-5s %-8s %-8s %-9s %-5s %-5s %-16s %-8s %-8s %-6s %-9s %-10s %-7s" + ) + print( + host_format + % ( + "Host", + "Load", + "NIMBY", + "freeMem", + "freeSwap", + "freeMcp", + "Cores", + "Mem", + "Idle", + "Os", + "Uptime", + "State", + "Locked", + "Alloc", + "Thread", + ) + ) for host in sorted(hosts, key=lambda v: v.data.name): - print(host_format % (host.data.name, host.data.load, - host.data.nimby_enabled, - cueadmin.format.formatMem(host.data.free_memory), - cueadmin.format.formatMem(host.data.free_swap), - cueadmin.format.formatMem(host.data.free_mcp), - host.data.cores, - cueadmin.format.formatMem(host.data.memory), - "[ %0.2f / %s ]" % (host.data.idle_cores, - cueadmin.format.formatMem(host.data.idle_memory)), - host.data.os, - cueadmin.format.formatLongDuration( - int(time.time()) - host.data.boot_time), - opencue.api.host_pb2.HardwareState.Name(host.data.state), - opencue.api.host_pb2.LockState.Name(host.data.lock_state), - host.data.alloc_name, - opencue.api.host_pb2.ThreadMode.Name(host.data.thread_mode))) + print( + host_format + % ( + host.data.name, + host.data.load, + host.data.nimby_enabled, + cueadmin.format.formatMem(host.data.free_memory), + cueadmin.format.formatMem(host.data.free_swap), + cueadmin.format.formatMem(host.data.free_mcp), + host.data.cores, + cueadmin.format.formatMem(host.data.memory), + "[ %0.2f / %s ]" + % ( + host.data.idle_cores, + cueadmin.format.formatMem(host.data.idle_memory), + ), + host.data.os, + cueadmin.format.formatLongDuration( + int(time.time()) - host.data.boot_time + ), + opencue.api.host_pb2.HardwareState.Name(host.data.state), + opencue.api.host_pb2.LockState.Name(host.data.lock_state), + host.data.alloc_name, + opencue.api.host_pb2.ThreadMode.Name(host.data.thread_mode), + ) + ) def displayShows(shows): @@ -84,15 +119,29 @@ def displayShows(shows): @param shows: A list of show objects """ show_format = "%-8s %-6s %15s %15s %15s %15s" - print(show_format % ("Show", "Active", "ReservedCores", "RunningFrames", "PendingFrames", - "PendingJobs")) + print( + show_format + % ( + "Show", + "Active", + "ReservedCores", + "RunningFrames", + "PendingFrames", + "PendingJobs", + ) + ) for show in shows: - print(show_format % (show.data.name, - show.data.active, - "%0.2f" % show.data.show_stats.reserved_cores, - show.data.show_stats.running_frames, - show.data.show_stats.pending_frames, - show.data.show_stats.pending_jobs)) + print( + show_format + % ( + show.data.name, + show.data.active, + "%0.2f" % show.data.show_stats.reserved_cores, + show.data.show_stats.running_frames, + show.data.show_stats.pending_frames, + show.data.show_stats.pending_jobs, + ) + ) def displayServices(services): @@ -101,13 +150,20 @@ def displayServices(services): @param services: A list of Server objects """ service_format = "%-20s %-12s %-20s %-15s %-36s" - print(service_format % ("Name", "Can Thread", "Min Cores Units", "Min Memory", "Tags")) + print( + service_format % ("Name", "Can Thread", "Min Cores Units", "Min Memory", "Tags") + ) for srv in services: - print(service_format % (srv.data.name, - srv.data.threadable, - srv.data.min_cores, - "%s MB" % srv.data.min_memory, - " | ".join(srv.data.tags))) + print( + service_format + % ( + srv.data.name, + srv.data.threadable, + srv.data.min_cores, + "%s MB" % srv.data.min_memory, + " | ".join(srv.data.tags), + ) + ) def displayAllocations(allocations): @@ -116,18 +172,35 @@ def displayAllocations(allocations): @param allocations: A list of allocation objects """ alloc_format = "%-25s %15s %8s %8s %8s %8s %8s %8s %-8s" - print(alloc_format % ("Name", "Tag", "Running", "Avail", "Cores", "Hosts", "Locked", "Down", - "Billable")) + print( + alloc_format + % ( + "Name", + "Tag", + "Running", + "Avail", + "Cores", + "Hosts", + "Locked", + "Down", + "Billable", + ) + ) for alloc in sorted(allocations, key=lambda v: v.data.facility): - print(alloc_format % (alloc.data.name, - alloc.data.tag, - "%0.2f" % alloc.data.stats.running_cores, - "%0.2f" % alloc.data.stats.available_cores, - alloc.data.stats.cores, - alloc.data.stats.hosts, - alloc.data.stats.locked_hosts, - alloc.data.stats.down_hosts, - alloc.data.billable)) + print( + alloc_format + % ( + alloc.data.name, + alloc.data.tag, + "%0.2f" % alloc.data.stats.running_cores, + "%0.2f" % alloc.data.stats.available_cores, + alloc.data.stats.cores, + alloc.data.stats.hosts, + alloc.data.stats.locked_hosts, + alloc.data.stats.down_hosts, + alloc.data.billable, + ) + ) def displaySubscriptions(subscriptions, show): @@ -141,20 +214,25 @@ def displaySubscriptions(subscriptions, show): sub_format = "%-30s %-12s %6s %8s %8s %8s" print(sub_format % ("Allocation", "Show", "Size", "Burst", "Run", "Used")) for s in subscriptions: - size = s.data.size/100 - burst = s.data.burst/100 - run = s.data.reserved_cores/100 + size = s.data.size / 100 + burst = s.data.burst / 100 + run = s.data.reserved_cores / 100 if s.data.size: perc = float(s.data.reserved_cores) / s.data.size * 100.0 else: perc = run - print(sub_format % (s.data.allocation_name, - s.data.show_name, - size, - burst, - "%0.2f" % run, - "%0.2f%%" % perc)) + print( + sub_format + % ( + s.data.allocation_name, + s.data.show_name, + size, + burst, + "%0.2f" % run, + "%0.2f%%" % perc, + ) + ) def displayJobs(jobs): @@ -162,31 +240,42 @@ def displayJobs(jobs): @type jobs: list @param jobs: All objects must have a .name parameter""" job_format = "%-56s %-15s %5s %7s %8s %5s %8s %8s" - print(job_format % ("Job", "Group", "Booked", "Cores", "Wait", "Pri", "MinCores", "MaxCores")) + print( + job_format + % ("Job", "Group", "Booked", "Cores", "Wait", "Pri", "MinCores", "MaxCores") + ) for job in jobs: - name = job.data.name + (' [paused]' if job.data.is_paused else '') - print(job_format % (cueadmin.format.cutoff(name, 52), - cueadmin.format.cutoff(job.data.group, 15), - job.data.job_stats.running_frames, - "%0.2f" % job.data.job_stats.reserved_cores, - job.data.job_stats.waiting_frames, - job.data.priority, - "%0.2f" % job.data.min_cores, - "%0.2f" % job.data.max_cores)) + name = job.data.name + (" [paused]" if job.data.is_paused else "") + print( + job_format + % ( + cueadmin.format.cutoff(name, 52), + cueadmin.format.cutoff(job.data.group, 15), + job.data.job_stats.running_frames, + "%0.2f" % job.data.job_stats.reserved_cores, + job.data.job_stats.waiting_frames, + job.data.priority, + "%0.2f" % job.data.min_cores, + "%0.2f" % job.data.max_cores, + ) + ) def displayJobInfo(job): """Displays the job's information in cueman format. @type job: Job @param job: Job to display""" - print("-"*60) + print("-" * 60) print("job: %s\n" % job.data.name) print("%13s: %s" % ("start time", cueadmin.format.formatTime(job.data.start_time))) print("%13s: %s" % ("state", "PAUSED" if job.data.is_paused else job.data.state)) print("%13s: %s" % ("type", "N/A")) print("%13s: %s" % ("architecture", "N/A")) print("%13s: %s" % ("services", "N/A")) - print("%13s: %0.2f / %0.2f" % ("Min/Max cores", job.data.min_cores, job.data.max_cores)) + print( + "%13s: %0.2f / %0.2f" + % ("Min/Max cores", job.data.min_cores, job.data.max_cores) + ) print("") print("%22s: %s" % ("total number of frames", job.data.job_stats.total_frames)) print("%22s: %s" % ("done", job.data.job_stats.succeeded_frames)) @@ -199,11 +288,17 @@ def displayJobInfo(job): layers = job.getLayers() print("this is a OpenCue job with %d layers\n" % len(layers)) for layer in layers: - print("%s (%d frames, %d done)" % (layer.data.name, layer.data.layer_stats.total_frames, - layer.data.layer_stats.succeeded_frames)) + print( + "%s (%d frames, %d done)" + % ( + layer.data.name, + layer.data.layer_stats.total_frames, + layer.data.layer_stats.succeeded_frames, + ) + ) print(" average frame time: %s" % "N/A") print(" average ram usage: %s" % "N/A") - print(" tags: %s\n" % ' | '.join(layer.data.tags)) + print(" tags: %s\n" % " | ".join(layer.data.tags)) def displayFrames(frames): @@ -212,7 +307,16 @@ def displayFrames(frames): @param frames: List of frames to display""" framesFormat = "%-35s %-11s %-15s %-13s %-12s %-9s %5s %7s %5s" header = framesFormat % ( - "Frame", "Status", "Host", "Start", "End", "Runtime", "Mem", "Retry", "Exit") + "Frame", + "Status", + "Host", + "Start", + "End", + "Runtime", + "Mem", + "Retry", + "Exit", + ) print(header + "\n" + "-" * len(header)) for frame in frames: @@ -221,24 +325,33 @@ def displayFrames(frames): if frame.data.start_time: duration = cueadmin.format.formatDuration( - cueadmin.format.findDuration(frame.data.start_time, frame.data.stop_time)) + cueadmin.format.findDuration( + frame.data.start_time, frame.data.stop_time + ) + ) else: duration = "" memory = cueadmin.format.formatMem(frame.data.max_rss) exitStatus = frame.data.exit_status - print(framesFormat % ( - cueadmin.format.cutoff(frame.data.name, 35), - job_pb2.FrameState.Name(frame.data.state), - frame.data.last_resource, - startTime, - stopTime, - duration, - memory, - frame.data.retry_count, - exitStatus)) + print( + framesFormat + % ( + cueadmin.format.cutoff(frame.data.name, 35), + job_pb2.FrameState.Name(frame.data.state), + frame.data.last_resource, + startTime, + stopTime, + duration, + memory, + frame.data.retry_count, + exitStatus, + ) + ) if len(frames) == 1000: - print("Warning: Only showing first 1000 matches. See frame query options to " - "limit your results.") + print( + "Warning: Only showing first 1000 matches. See frame query options to " + "limit your results." + ) diff --git a/cueadmin/cueadmin/util.py b/cueadmin/cueadmin/util.py index bb51e6033..a423b3925 100644 --- a/cueadmin/cueadmin/util.py +++ b/cueadmin/cueadmin/util.py @@ -16,21 +16,16 @@ """Utility functions for CueAdmin""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function -from builtins import input import logging import sys import time +from builtins import input import opencue - -__ALL__ = ["enableDebugLogging", - "promptYesNo", - "waitOnJobName"] +__ALL__ = ["enableDebugLogging", "promptYesNo", "waitOnJobName"] def enableDebugLogging(): diff --git a/cueadmin/pyproject.toml b/cueadmin/pyproject.toml index 0f6a72965..b4be64e26 100644 --- a/cueadmin/pyproject.toml +++ b/cueadmin/pyproject.toml @@ -27,15 +27,80 @@ cueadmin = "cueadmin.__main__:main" # --- Pytest configuration --- [tool.pytest.ini_options] -minversion = "6.0" # Set to required pytest version +minversion = "8.0" # Set to required pytest version testpaths = ["tests"] # Relative path(s) where tests are located python_files = ["test_*.py", "*_tests.py"] # Default test file pattern python_functions = ["test_*"] # Default test function pattern +addopts = [ + "-v", # Verbose output + "--strict-markers", # Markers must be registered + "--tb=short", # Shorter traceback format +] +markers = [ + "unit: Unit tests", + "integration: Integration tests", + "slow: Slow running tests", +] # --- Optional Test Dependencies --- [project.optional-dependencies] test = [ - "mock==2.0.0", - "pyfakefs==5.2.3", - "pytest" + "pytest>=8.0.0", + "pytest-cov>=4.0.0", + "pytest-mock>=3.10.0", + "mock>=4.0.0", + "pyfakefs>=5.2.3" +] + +dev = [ + "pytest>=8.0.0", + "pytest-cov>=4.0.0", + "pytest-mock>=3.10.0", + "mock>=4.0.0", + "pyfakefs>=5.2.3", + "pylint>=3.0.0", + "black>=23.0.0", + "isort>=5.12.0" +] + +# --- Coverage configuration --- +[tool.coverage.run] +source = ["cueadmin"] +omit = [ + "*/tests/*", + "*/__pycache__/*", + "*/test_*.py", +] + +[tool.coverage.report] +exclude_lines = [ + "pragma: no cover", + "def __repr__", + "if self.debug:", + "if __name__ == .__main__.:", + "raise AssertionError", + "raise NotImplementedError", + "if 0:", + "if False:", ] +show_missing = true +precision = 2 + +[tool.coverage.html] +directory = "htmlcov" + +# --- GitHub Actions Example --- +# To run tests in GitHub Actions, add this to .github/workflows/test.yml: +# +# - name: Run cueadmin tests +# run: | +# cd cueadmin +# pip install -e ".[test]" +# pytest --cov=cueadmin --cov-report=xml +# +# - name: Upload coverage to Codecov +# uses: codecov/codecov-action@v3 +# with: +# file: ./cueadmin/coverage.xml +# flags: cueadmin +# name: cueadmin-coverage diff --git a/cueadmin/run_tests.sh b/cueadmin/run_tests.sh new file mode 100755 index 000000000..36f0bfe41 --- /dev/null +++ b/cueadmin/run_tests.sh @@ -0,0 +1,179 @@ +#!/bin/bash +# Copyright Contributors to the OpenCue Project +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Script to run cueadmin tests with various options + +set -e + +# Default values +COVERAGE=false +HTML=false +VERBOSE=false +TESTS="" +QUICK=false + +# Color codes for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +# Function to print colored output +print_status() { + echo -e "${GREEN}[INFO]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}[WARNING]${NC} $1" +} + +# Function to display help +show_help() { + echo "Usage: ./run_tests.sh [OPTIONS] [TEST_PATHS]" + echo "" + echo "Run cueadmin tests with various options." + echo "" + echo "Options:" + echo " -c, --coverage Enable coverage reporting" + echo " -h, --html Generate HTML coverage report (implies --coverage)" + echo " -v, --verbose Run tests in verbose mode" + echo " -q, --quick Run tests quickly (no coverage, parallel execution)" + echo " --help Show this help message and exit" + echo "" + echo "Examples:" + echo " ./run_tests.sh # Run all tests" + echo " ./run_tests.sh --coverage # Run with coverage" + echo " ./run_tests.sh --html # Run with HTML coverage report" + echo " ./run_tests.sh tests/test_output.py # Run specific test file" + echo " ./run_tests.sh -v tests/ # Run all tests verbosely" + exit 0 +} + +# Parse command line arguments +while [[ $# -gt 0 ]]; do + case $1 in + --help) + show_help + ;; + --coverage|-c) + COVERAGE=true + shift + ;; + --html|-h) + HTML=true + COVERAGE=true + shift + ;; + --verbose|-v) + VERBOSE=true + shift + ;; + --quick|-q) + QUICK=true + COVERAGE=false + shift + ;; + *) + TESTS="$TESTS $1" + shift + ;; + esac +done + +# Check if pytest is installed +if ! python -m pytest --version &> /dev/null; then + print_error "pytest is not installed. Please install it with: pip install pytest" + exit 1 +fi + +# Check if pytest-cov is installed when coverage is requested +if [ "$COVERAGE" = true ]; then + if ! python -c "import pytest_cov" &> /dev/null; then + print_warning "pytest-cov is not installed. Installing it now..." + pip install pytest-cov + fi +fi + +# Build the pytest command +CMD="python -m pytest" + +# Add verbose flag if requested +if [ "$VERBOSE" = true ]; then + CMD="$CMD -vv" +else + CMD="$CMD -v" +fi + +# Add quick mode flags +if [ "$QUICK" = true ]; then + CMD="$CMD -n auto --dist loadgroup" + # Check if pytest-xdist is installed for parallel execution + if ! python -c "import xdist" &> /dev/null; then + print_warning "pytest-xdist not installed, falling back to sequential execution" + CMD="python -m pytest -v" + fi +fi + +# Add coverage flags if requested +if [ "$COVERAGE" = true ]; then + CMD="$CMD --cov=cueadmin --cov-report=term-missing" + + if [ "$HTML" = true ]; then + CMD="$CMD --cov-report=html" + print_status "HTML coverage report will be generated in htmlcov/" + fi + + CMD="$CMD --cov-report=xml" +fi + +# Add test paths if specified, otherwise run all tests +if [ -n "$TESTS" ]; then + CMD="$CMD $TESTS" +else + CMD="$CMD tests/" +fi + +# Display test environment info +print_status "Python version: $(python --version 2>&1)" +print_status "pytest version: $(python -m pytest --version 2>&1 | head -1)" +print_status "Test directory: $(pwd)" + +# Run the tests +echo "" +print_status "Running: $CMD" +echo "==========================================" +$CMD +TEST_EXIT_CODE=$? + +# Print summary +echo "==========================================" +if [ $TEST_EXIT_CODE -eq 0 ]; then + print_status "All tests passed successfully!" + + if [ "$COVERAGE" = true ]; then + echo "" + print_status "Coverage reports generated:" + [ -f coverage.xml ] && echo " - XML: coverage.xml" + [ -d htmlcov ] && echo " - HTML: htmlcov/index.html" + fi +else + print_error "Some tests failed. Exit code: $TEST_EXIT_CODE" +fi + +exit $TEST_EXIT_CODE diff --git a/cueadmin/tests/test_common.py b/cueadmin/tests/test_common.py index 435acb25d..a1b6b9154 100644 --- a/cueadmin/tests/test_common.py +++ b/cueadmin/tests/test_common.py @@ -18,36 +18,32 @@ """Tests for cueadmin.common.""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function -from builtins import str import unittest +from builtins import str import mock - -import opencue_proto.facility_pb2 -import opencue_proto.host_pb2 -import opencue_proto.job_pb2 -import opencue_proto.service_pb2 -import opencue_proto.show_pb2 -import opencue_proto.subscription_pb2 import opencue.wrappers.allocation import opencue.wrappers.host import opencue.wrappers.proc import opencue.wrappers.service import opencue.wrappers.show import opencue.wrappers.subscription +import opencue_proto.facility_pb2 +import opencue_proto.host_pb2 +import opencue_proto.job_pb2 +import opencue_proto.service_pb2 +import opencue_proto.show_pb2 +import opencue_proto.subscription_pb2 import cueadmin.common - -TEST_SHOW = 'test_show' -TEST_FACILITY = 'some-non-default-facility' -TEST_ALLOC = 'test_alloc' -TEST_HOST = 'some_host' -TEST_JOB = 'my_random_job_name' +TEST_SHOW = "test_show" +TEST_FACILITY = "some-non-default-facility" +TEST_ALLOC = "test_alloc" +TEST_HOST = "some_host" +TEST_JOB = "my_random_job_name" class CommonArgTests(unittest.TestCase): @@ -55,49 +51,49 @@ class CommonArgTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() - @mock.patch('cueadmin.util.enableDebugLogging') + @mock.patch("cueadmin.util.enableDebugLogging") def testVerboseLogging(self, enableDebugLoggingMock): - args = self.parser.parse_args(['-verbose']) + args = self.parser.parse_args(["-verbose"]) cueadmin.common.handleArgs(args) enableDebugLoggingMock.assert_called_with() - @mock.patch('opencue.Cuebot.setHosts') + @mock.patch("opencue.Cuebot.setHosts") def testSetServer(self, setHostsMock): - serverName = 'someRandomServer01' - args = self.parser.parse_args(['-server', serverName]) + serverName = "someRandomServer01" + args = self.parser.parse_args(["-server", serverName]) cueadmin.common.handleArgs(args) setHostsMock.assert_called_with([serverName]) - @mock.patch('opencue.Cuebot.setHostWithFacility') + @mock.patch("opencue.Cuebot.setHostWithFacility") def testSetFacility(self, setFacilityMock): - args = self.parser.parse_args(['-facility', TEST_FACILITY]) + args = self.parser.parse_args(["-facility", TEST_FACILITY]) cueadmin.common.handleArgs(args) setFacilityMock.assert_called_with(TEST_FACILITY) -@mock.patch('opencue.api.findShow') -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.api.findShow") +@mock.patch("opencue.cuebot.Cuebot.getStub") class ShowTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() - @mock.patch('opencue.api.createShow') + @mock.patch("opencue.api.createShow") def testCreateShow(self, createShowMock, getStubMock, findShowMock): - args = self.parser.parse_args(['-create-show', TEST_SHOW, '-force']) + args = self.parser.parse_args(["-create-show", TEST_SHOW, "-force"]) cueadmin.common.handleArgs(args) createShowMock.assert_called_with(TEST_SHOW) def testDeleteShow(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-delete-show', TEST_SHOW, '-force']) + args = self.parser.parse_args(["-delete-show", TEST_SHOW, "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -106,7 +102,7 @@ def testDeleteShow(self, getStubMock, findShowMock): showMock.delete.assert_called_with() def testDisableShow(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-disable-show', TEST_SHOW, '-force']) + args = self.parser.parse_args(["-disable-show", TEST_SHOW, "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -115,7 +111,7 @@ def testDisableShow(self, getStubMock, findShowMock): showMock.setActive.assert_called_with(False) def testEnableShow(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-enable-show', TEST_SHOW, '-force']) + args = self.parser.parse_args(["-enable-show", TEST_SHOW, "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -124,7 +120,7 @@ def testEnableShow(self, getStubMock, findShowMock): showMock.setActive.assert_called_with(True) def testEnableBooking(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-booking', TEST_SHOW, 'on', '-force']) + args = self.parser.parse_args(["-booking", TEST_SHOW, "on", "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -133,7 +129,7 @@ def testEnableBooking(self, getStubMock, findShowMock): showMock.enableBooking.assert_called_with(True) def testDisableBooking(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-booking', TEST_SHOW, 'off', '-force']) + args = self.parser.parse_args(["-booking", TEST_SHOW, "off", "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -142,7 +138,7 @@ def testDisableBooking(self, getStubMock, findShowMock): showMock.enableBooking.assert_called_with(False) def testEnableDispatch(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-dispatching', TEST_SHOW, 'on', '-force']) + args = self.parser.parse_args(["-dispatching", TEST_SHOW, "on", "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -151,7 +147,7 @@ def testEnableDispatch(self, getStubMock, findShowMock): showMock.enableDispatching.assert_called_with(True) def testDisableDispatch(self, getStubMock, findShowMock): - args = self.parser.parse_args(['-dispatching', TEST_SHOW, 'off', '-force']) + args = self.parser.parse_args(["-dispatching", TEST_SHOW, "off", "-force"]) showMock = mock.Mock() findShowMock.return_value = showMock @@ -162,7 +158,8 @@ def testDisableDispatch(self, getStubMock, findShowMock): def testDefaultMinCores(self, getStubMock, findShowMock): arbitraryCoreCount = 873 args = self.parser.parse_args( - ['-default-min-cores', TEST_SHOW, str(arbitraryCoreCount), '-force']) + ["-default-min-cores", TEST_SHOW, str(arbitraryCoreCount), "-force"] + ) showMock = mock.Mock() findShowMock.return_value = showMock @@ -173,7 +170,8 @@ def testDefaultMinCores(self, getStubMock, findShowMock): def testDefaultMaxCores(self, getStubMock, findShowMock): arbitraryCoreCount = 9349 args = self.parser.parse_args( - ['-default-max-cores', TEST_SHOW, str(arbitraryCoreCount), '-force']) + ["-default-max-cores", TEST_SHOW, str(arbitraryCoreCount), "-force"] + ) showMock = mock.Mock() findShowMock.return_value = showMock @@ -181,21 +179,22 @@ def testDefaultMaxCores(self, getStubMock, findShowMock): showMock.setDefaultMaxCores.assert_called_with(arbitraryCoreCount) - @mock.patch('opencue.api.getShows') + @mock.patch("opencue.api.getShows") def testListShows(self, getShowsMock, getStubMock, findShowMock): - args = self.parser.parse_args(['-ls']) + args = self.parser.parse_args(["-ls"]) getShowsMock.return_value = [ opencue.wrappers.show.Show( opencue_proto.show_pb2.Show( - name='testing', + name="testing", active=True, show_stats=opencue_proto.show_pb2.ShowStats( reserved_cores=265, running_frames=100, pending_frames=248, - pending_jobs=29 - ) - )) + pending_jobs=29, + ), + ) + ) ] cueadmin.common.handleArgs(args) @@ -203,19 +202,22 @@ def testListShows(self, getShowsMock, getStubMock, findShowMock): getShowsMock.assert_called_with() -@mock.patch('opencue.api.findAllocation') -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.api.findAllocation") +@mock.patch("opencue.cuebot.Cuebot.getStub") class AllocTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() - @mock.patch('opencue.api.createAllocation') - @mock.patch('opencue.api.getFacility') - def testCreateAlloc(self, getFacilityMock, createAllocMock, getStubMock, findAllocMock): - tagName = 'random-tag' + @mock.patch("opencue.api.createAllocation") + @mock.patch("opencue.api.getFacility") + def testCreateAlloc( + self, getFacilityMock, createAllocMock, getStubMock, findAllocMock + ): + tagName = "random-tag" args = self.parser.parse_args( - ['-create-alloc', TEST_FACILITY, TEST_ALLOC, tagName, '-force']) + ["-create-alloc", TEST_FACILITY, TEST_ALLOC, tagName, "-force"] + ) facMock = mock.Mock() getFacilityMock.return_value = facMock @@ -226,7 +228,8 @@ def testCreateAlloc(self, getFacilityMock, createAllocMock, getStubMock, findAll def testDeleteAlloc(self, getStubMock, findAllocMock): args = self.parser.parse_args( - ['-delete-alloc', '%s.%s' % (TEST_FACILITY, TEST_ALLOC), '-force']) + ["-delete-alloc", "%s.%s" % (TEST_FACILITY, TEST_ALLOC), "-force"] + ) allocMock = mock.Mock() findAllocMock.return_value = allocMock @@ -235,10 +238,12 @@ def testDeleteAlloc(self, getStubMock, findAllocMock): allocMock.delete.assert_called_with() def testRenameAlloc(self, getStubMock, findAllocMock): - oldFullName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) - newName = 'some_new_alloc_name' - newFullName = '%s.%s' % (TEST_FACILITY, newName) - args = self.parser.parse_args(['-rename-alloc', oldFullName, newFullName, '-force']) + oldFullName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) + newName = "some_new_alloc_name" + newFullName = "%s.%s" % (TEST_FACILITY, newName) + args = self.parser.parse_args( + ["-rename-alloc", oldFullName, newFullName, "-force"] + ) allocMock = mock.Mock() findAllocMock.return_value = allocMock @@ -248,9 +253,11 @@ def testRenameAlloc(self, getStubMock, findAllocMock): allocMock.setName.assert_called_with(newName) def testInvalidRenameAlloc(self, getStubMock, findAllocMock): - oldFullName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) - invalidNewName = 'invalid_alloc_name' - args = self.parser.parse_args(['-rename-alloc', oldFullName, invalidNewName, '-force']) + oldFullName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) + invalidNewName = "invalid_alloc_name" + args = self.parser.parse_args( + ["-rename-alloc", oldFullName, invalidNewName, "-force"] + ) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -258,9 +265,9 @@ def testInvalidRenameAlloc(self, getStubMock, findAllocMock): findAllocMock.assert_not_called() def testTagAlloc(self, getStubMock, findAllocMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) - tagName = 'new_tag' - args = self.parser.parse_args(['-tag-alloc', allocName, tagName, '-force']) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) + tagName = "new_tag" + args = self.parser.parse_args(["-tag-alloc", allocName, tagName, "-force"]) allocMock = mock.Mock() findAllocMock.return_value = allocMock @@ -270,36 +277,40 @@ def testTagAlloc(self, getStubMock, findAllocMock): allocMock.setTag.assert_called_with(tagName) def testReparentHosts(self, getStubMock, findAllocMock): - srcAllocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) - dstAllocName = '%s.some_other_alloc' % TEST_FACILITY - args = self.parser.parse_args(['-transfer', srcAllocName, dstAllocName, '-force']) + srcAllocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) + dstAllocName = "%s.some_other_alloc" % TEST_FACILITY + args = self.parser.parse_args( + ["-transfer", srcAllocName, dstAllocName, "-force"] + ) srcAllocMock = mock.Mock() dstAllocMock = mock.Mock() findAllocMock.side_effect = [srcAllocMock, dstAllocMock] - hostList = ['some', 'arbitrary', 'list', 'of', 'hosts'] + hostList = ["some", "arbitrary", "list", "of", "hosts"] srcAllocMock.getHosts.return_value = hostList cueadmin.common.handleArgs(args) - findAllocMock.assert_has_calls([mock.call(srcAllocName), mock.call(dstAllocName)]) + findAllocMock.assert_has_calls( + [mock.call(srcAllocName), mock.call(dstAllocName)] + ) dstAllocMock.reparentHosts.assert_called_with(hostList) - @mock.patch('opencue.api.getAllocations') + @mock.patch("opencue.api.getAllocations") def testListAllocs(self, getAllocsMock, getStubMock, findAllocMock): - args = self.parser.parse_args(['-la']) + args = self.parser.parse_args(["-la"]) getAllocsMock.return_value = [ opencue.wrappers.allocation.Allocation( opencue_proto.facility_pb2.Allocation( - name='local.desktop', - tag='desktop', + name="local.desktop", + tag="desktop", billable=False, stats=opencue_proto.facility_pb2.AllocationStats( running_cores=100, available_cores=125, cores=600, locked_hosts=25, - down_hosts=3 - ) + down_hosts=3, + ), ) ) ] @@ -309,16 +320,16 @@ def testListAllocs(self, getAllocsMock, getStubMock, findAllocMock): getAllocsMock.assert_called_with() def testListSubscriptionsForAlloc(self, getStubMock, findAllocMock): - args = self.parser.parse_args(['-lba', TEST_ALLOC]) + args = self.parser.parse_args(["-lba", TEST_ALLOC]) allocMock = mock.Mock() allocMock.getSubscriptions.return_value = [ opencue.wrappers.subscription.Subscription( opencue_proto.subscription_pb2.Subscription( - allocation_name='local.general', - show_name='showName', + allocation_name="local.general", + show_name="showName", size=1000, burst=1500, - reserved_cores=500 + reserved_cores=500, ) ) ] @@ -330,15 +341,15 @@ def testListSubscriptionsForAlloc(self, getStubMock, findAllocMock): allocMock.getSubscriptions.assert_called_with() -@mock.patch('opencue.search.HostSearch') -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.search.HostSearch") +@mock.patch("opencue.cuebot.Cuebot.getStub") class HostTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() def testSetRepairState(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-repair', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-repair", "-host", TEST_HOST, "-force"]) hostMock = mock.Mock() hostSearchMock.byName.return_value = [hostMock] @@ -348,7 +359,7 @@ def testSetRepairState(self, getStubMock, hostSearchMock): hostMock.setHardwareState.assert_called_with(opencue.api.host_pb2.REPAIR) def testInvalidSetRepairState(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-repair', '-force']) + args = self.parser.parse_args(["-repair", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -356,7 +367,7 @@ def testInvalidSetRepairState(self, getStubMock, hostSearchMock): hostSearchMock.byName.assert_not_called() def testLockHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-lock', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-lock", "-host", TEST_HOST, "-force"]) hostMock = mock.Mock() hostSearchMock.byName.return_value = [hostMock] @@ -366,7 +377,7 @@ def testLockHost(self, getStubMock, hostSearchMock): hostMock.lock.assert_called_with() def testInvalidLockHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-lock', '-force']) + args = self.parser.parse_args(["-lock", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -374,7 +385,7 @@ def testInvalidLockHost(self, getStubMock, hostSearchMock): hostSearchMock.byName.assert_not_called() def testUnlockHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-unlock', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-unlock", "-host", TEST_HOST, "-force"]) hostMock = mock.Mock() hostSearchMock.byName.return_value = [hostMock] @@ -384,17 +395,19 @@ def testUnlockHost(self, getStubMock, hostSearchMock): hostMock.unlock.assert_called_with() def testInvalidUnlockHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-unlock', '-force']) + args = self.parser.parse_args(["-unlock", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) hostSearchMock.byName.assert_not_called() - @mock.patch('opencue.api.findAllocation') + @mock.patch("opencue.api.findAllocation") def testMoveHost(self, findAllocMock, getStubMock, hostSearchMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) - args = self.parser.parse_args(['-move', allocName, '-host', TEST_HOST, '-force']) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) + args = self.parser.parse_args( + ["-move", allocName, "-host", TEST_HOST, "-force"] + ) host = opencue.wrappers.host.Host(opencue_proto.host_pb2.Host()) host.setAllocation = mock.Mock() hostSearchMock.byName.return_value = [host] @@ -408,22 +421,31 @@ def testMoveHost(self, findAllocMock, getStubMock, hostSearchMock): host.setAllocation.assert_called_with(alloc) def testInvalidMoveHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-move', TEST_ALLOC, '-force']) + args = self.parser.parse_args(["-move", TEST_ALLOC, "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) hostSearchMock.byName.assert_not_called() - @mock.patch('opencue.api.getHosts') + @mock.patch("opencue.api.getHosts") def testListHosts(self, getHostsMock, getStubMock, hostSearchMock): - arbitraryMatchString = 'arbitraryMatchString' + arbitraryMatchString = "arbitraryMatchString" args = self.parser.parse_args( - ['-lh', arbitraryMatchString, '-state', 'up', 'repair', '-alloc', TEST_ALLOC]) + [ + "-lh", + arbitraryMatchString, + "-state", + "up", + "repair", + "-alloc", + TEST_ALLOC, + ] + ) getHostsMock.return_value = [ opencue.wrappers.host.Host( opencue_proto.host_pb2.Host( - name='host1', + name="host1", load=25, nimby_enabled=False, free_memory=3500000, @@ -433,12 +455,12 @@ def testListHosts(self, getHostsMock, getStubMock, hostSearchMock): memory=4500000, idle_cores=5, idle_memory=3000000, - os='Linux', + os="Linux", boot_time=1556836762, state=1, lock_state=1, - alloc_name='alloc01', - thread_mode=1 + alloc_name="alloc01", + thread_mode=1, ) ) ] @@ -446,11 +468,13 @@ def testListHosts(self, getHostsMock, getStubMock, hostSearchMock): cueadmin.common.handleArgs(args) getHostsMock.assert_called_with( - alloc=[TEST_ALLOC], match=[arbitraryMatchString], - state=[opencue.api.host_pb2.UP, opencue.api.host_pb2.REPAIR]) + alloc=[TEST_ALLOC], + match=[arbitraryMatchString], + state=[opencue.api.host_pb2.UP, opencue.api.host_pb2.REPAIR], + ) def testDeleteHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-delete-host', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-delete-host", "-host", TEST_HOST, "-force"]) hostMock1 = mock.Mock() hostMock2 = mock.Mock() hostSearchMock.byName.return_value = [hostMock1, hostMock2] @@ -462,7 +486,7 @@ def testDeleteHost(self, getStubMock, hostSearchMock): hostMock2.delete.assert_called_with() def testInvalidDeleteHost(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-delete-host', '-force']) + args = self.parser.parse_args(["-delete-host", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -470,7 +494,7 @@ def testInvalidDeleteHost(self, getStubMock, hostSearchMock): hostSearchMock.byName.assert_not_called() def testSafeReboot(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-safe-reboot', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-safe-reboot", "-host", TEST_HOST, "-force"]) hostMock1 = mock.Mock() hostMock2 = mock.Mock() hostSearchMock.byName.return_value = [hostMock1, hostMock2] @@ -482,7 +506,7 @@ def testSafeReboot(self, getStubMock, hostSearchMock): hostMock2.rebootWhenIdle.assert_called_with() def testInvalidSafeReboot(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-safe-reboot', '-force']) + args = self.parser.parse_args(["-safe-reboot", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -490,7 +514,7 @@ def testInvalidSafeReboot(self, getStubMock, hostSearchMock): hostSearchMock.byName.assert_not_called() def testSetThreadMode(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-thread', 'all', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-thread", "all", "-host", TEST_HOST, "-force"]) hostMock1 = mock.Mock() hostMock2 = mock.Mock() hostSearchMock.byName.return_value = [hostMock1, hostMock2] @@ -502,7 +526,7 @@ def testSetThreadMode(self, getStubMock, hostSearchMock): hostMock2.setThreadMode.assert_called_with(opencue.api.host_pb2.ALL) def testInvalidSetThreadMode(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-thread', 'all', '-force']) + args = self.parser.parse_args(["-thread", "all", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -510,7 +534,7 @@ def testInvalidSetThreadMode(self, getStubMock, hostSearchMock): hostSearchMock.byName.assert_not_called() def testSetFixed(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-fixed', '-host', TEST_HOST, '-force']) + args = self.parser.parse_args(["-fixed", "-host", TEST_HOST, "-force"]) hostMock = mock.Mock() hostSearchMock.byName.return_value = [hostMock] @@ -520,7 +544,7 @@ def testSetFixed(self, getStubMock, hostSearchMock): hostMock.setHardwareState.assert_called_with(opencue.api.host_pb2.UP) def testInvalidSetFixed(self, getStubMock, hostSearchMock): - args = self.parser.parse_args(['-fixed', '-force']) + args = self.parser.parse_args(["-fixed", "-force"]) with self.assertRaises(ValueError): cueadmin.common.handleArgs(args) @@ -528,21 +552,29 @@ def testInvalidSetFixed(self, getStubMock, hostSearchMock): hostSearchMock.byName.assert_not_called() -@mock.patch('opencue.api.findSubscription') -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.api.findSubscription") +@mock.patch("opencue.cuebot.Cuebot.getStub") class SubscriptionTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() - @mock.patch('opencue.api.findAllocation') - @mock.patch('opencue.api.findShow') + @mock.patch("opencue.api.findAllocation") + @mock.patch("opencue.api.findShow") def testCreateSub(self, findShowMock, findAllocMock, getStubMock, findSubMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) numCores = 125 burstCores = 236 args = self.parser.parse_args( - ['-create-sub', TEST_SHOW, allocName, str(numCores), str(burstCores), '-force']) + [ + "-create-sub", + TEST_SHOW, + allocName, + str(numCores), + str(burstCores), + "-force", + ] + ) showMock = mock.Mock() findShowMock.return_value = showMock allocMock = mock.Mock() @@ -552,20 +584,22 @@ def testCreateSub(self, findShowMock, findAllocMock, getStubMock, findSubMock): findShowMock.assert_called_with(TEST_SHOW) findAllocMock.assert_called_with(allocName) - showMock.createSubscription.assert_called_with(allocMock.data, numCores, burstCores) + showMock.createSubscription.assert_called_with( + allocMock.data, numCores, burstCores + ) - @mock.patch('opencue.api.findShow') + @mock.patch("opencue.api.findShow") def testListSubs(self, findShowMock, getStubMock, findSubMock): - args = self.parser.parse_args(['-lb', TEST_SHOW]) + args = self.parser.parse_args(["-lb", TEST_SHOW]) showMock = mock.Mock() showMock.getSubscriptions.return_value = [ opencue.wrappers.subscription.Subscription( opencue_proto.subscription_pb2.Subscription( - allocation_name='cloud.desktop', - show_name='showName', + allocation_name="cloud.desktop", + show_name="showName", size=0, burst=1500, - reserved_cores=50 + reserved_cores=50, ) ), ] @@ -577,9 +611,9 @@ def testListSubs(self, findShowMock, getStubMock, findSubMock): showMock.getSubscriptions.assert_called_with() def testDeleteSub(self, getStubMock, findSubMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) - args = self.parser.parse_args(['-delete-sub', TEST_SHOW, allocName, '-force']) - subName = '%s.%s' % (allocName, TEST_SHOW) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) + args = self.parser.parse_args(["-delete-sub", TEST_SHOW, allocName, "-force"]) + subName = "%s.%s" % (allocName, TEST_SHOW) subMock = mock.Mock() findSubMock.return_value = subMock @@ -589,10 +623,12 @@ def testDeleteSub(self, getStubMock, findSubMock): subMock.delete.assert_called_with() def testSetSize(self, getStubMock, findSubMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) newSize = 200 - args = self.parser.parse_args(['-size', TEST_SHOW, allocName, str(newSize), '-force']) - subName = '%s.%s' % (allocName, TEST_SHOW) + args = self.parser.parse_args( + ["-size", TEST_SHOW, allocName, str(newSize), "-force"] + ) + subName = "%s.%s" % (allocName, TEST_SHOW) subMock = mock.Mock() findSubMock.return_value = subMock @@ -602,10 +638,12 @@ def testSetSize(self, getStubMock, findSubMock): subMock.setSize.assert_called_with(newSize) def testSetBurst(self, getStubMock, findSubMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) newBurstSize = 847 - args = self.parser.parse_args(['-burst', TEST_SHOW, allocName, str(newBurstSize), '-force']) - subName = '%s.%s' % (allocName, TEST_SHOW) + args = self.parser.parse_args( + ["-burst", TEST_SHOW, allocName, str(newBurstSize), "-force"] + ) + subName = "%s.%s" % (allocName, TEST_SHOW) subMock = mock.Mock() findSubMock.return_value = subMock @@ -615,11 +653,13 @@ def testSetBurst(self, getStubMock, findSubMock): subMock.setBurst.assert_called_with(newBurstSize) def testSetBurstPercentage(self, getStubMock, findSubMock): - allocName = '%s.%s' % (TEST_FACILITY, TEST_ALLOC) + allocName = "%s.%s" % (TEST_FACILITY, TEST_ALLOC) originalSize = 120 - newBurstPerc = '20%' - args = self.parser.parse_args(['-burst', TEST_SHOW, allocName, newBurstPerc, '-force']) - subName = '%s.%s' % (allocName, TEST_SHOW) + newBurstPerc = "20%" + args = self.parser.parse_args( + ["-burst", TEST_SHOW, allocName, newBurstPerc, "-force"] + ) + subName = "%s.%s" % (allocName, TEST_SHOW) subMock = mock.Mock() subMock.data.size = originalSize findSubMock.return_value = subMock @@ -627,25 +667,27 @@ def testSetBurstPercentage(self, getStubMock, findSubMock): cueadmin.common.handleArgs(args) findSubMock.assert_called_with(subName) - subMock.setBurst.assert_called_with(originalSize * (1 + float(newBurstPerc[:-1]) / 100)) + subMock.setBurst.assert_called_with( + originalSize * (1 + float(newBurstPerc[:-1]) / 100) + ) -@mock.patch('opencue.search.JobSearch') -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.search.JobSearch") +@mock.patch("opencue.cuebot.Cuebot.getStub") class JobTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() def testListJobs(self, getStubMock, jobSearchMock): - args = self.parser.parse_args(['-lj', TEST_JOB]) + args = self.parser.parse_args(["-lj", TEST_JOB]) jobSearchMock.byMatch.return_value = opencue_proto.job_pb2.JobGetJobsResponse( jobs=opencue_proto.job_pb2.JobSeq( jobs=[ opencue_proto.job_pb2.Job( - name='d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d', + name="d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d", is_paused=False, - group='u0uMmB1O0z3ZkvreFYzP', + group="u0uMmB1O0z3ZkvreFYzP", job_stats=opencue_proto.job_pb2.JobStats( running_frames=5, reserved_cores=5, @@ -653,7 +695,7 @@ def testListJobs(self, getStubMock, jobSearchMock): ), priority=89, min_cores=1, - max_cores=1 + max_cores=1, ) ] ) @@ -664,14 +706,14 @@ def testListJobs(self, getStubMock, jobSearchMock): jobSearchMock.byMatch.assert_called_with([TEST_JOB]) def testListJobInfo(self, getStubMock, jobSearchMock): - args = self.parser.parse_args(['-lji', TEST_JOB]) + args = self.parser.parse_args(["-lji", TEST_JOB]) jobSearchMock.byMatch.return_value = opencue_proto.job_pb2.JobGetJobsResponse( jobs=opencue_proto.job_pb2.JobSeq( jobs=[ opencue_proto.job_pb2.Job( - name='d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d', + name="d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d", is_paused=False, - group='u0uMmB1O0z3ZkvreFYzP', + group="u0uMmB1O0z3ZkvreFYzP", job_stats=opencue_proto.job_pb2.JobStats( running_frames=5, reserved_cores=5, @@ -679,7 +721,7 @@ def testListJobInfo(self, getStubMock, jobSearchMock): ), priority=89, min_cores=1, - max_cores=1 + max_cores=1, ) ] ) @@ -690,112 +732,161 @@ def testListJobInfo(self, getStubMock, jobSearchMock): jobSearchMock.byMatch.assert_called_with([TEST_JOB]) -@mock.patch('opencue.search.ProcSearch') -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.search.ProcSearch") +@mock.patch("opencue.cuebot.Cuebot.getStub") class ProcTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() def testListProcs(self, getStubMock, procSearchMock): - resultsLimit = '54' + resultsLimit = "54" args = self.parser.parse_args( - ['-lp', TEST_SHOW, '-alloc', TEST_ALLOC, '-duration', '1.5', '-host', TEST_HOST, - '-job', TEST_JOB, '-limit', resultsLimit, '-memory', '128']) - procSearchMock.byOptions.return_value = \ + [ + "-lp", + TEST_SHOW, + "-alloc", + TEST_ALLOC, + "-duration", + "1.5", + "-host", + TEST_HOST, + "-job", + TEST_JOB, + "-limit", + resultsLimit, + "-memory", + "128", + ] + ) + procSearchMock.byOptions.return_value = ( opencue_proto.host_pb2.ProcGetProcsResponse( procs=opencue_proto.host_pb2.ProcSeq( procs=[ opencue_proto.host_pb2.Proc( - name='proc1', + name="proc1", reserved_cores=28, used_memory=44, reserved_memory=120, - job_name='mms2oazed2bbcjk60gho_w11licymr63s66bw1b3s', - frame_name='y0ihh3fxrstz6ub7ut2k', - dispatch_time=1556845762 + job_name="mms2oazed2bbcjk60gho_w11licymr63s66bw1b3s", + frame_name="y0ihh3fxrstz6ub7ut2k", + dispatch_time=1556845762, ) ] ) ) + ) cueadmin.common.handleArgs(args) procSearchMock.byOptions.assert_called_with( alloc=[TEST_ALLOC], - duration=[opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=5400)], - host=[TEST_HOST], job=[TEST_JOB], limit=resultsLimit, - memory=[opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=134217728)], - show=[TEST_SHOW]) + duration=[ + opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=5400) + ], + host=[TEST_HOST], + job=[TEST_JOB], + limit=resultsLimit, + memory=[ + opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion( + value=134217728 + ) + ], + show=[TEST_SHOW], + ) def testListFrameLogPaths(self, getStubMock, procSearchMock): - resultsLimit = '54' + resultsLimit = "54" args = self.parser.parse_args( - ['-ll', TEST_SHOW, '-alloc', TEST_ALLOC, '-duration', '1.5', - '-job', TEST_JOB, '-limit', resultsLimit, '-memory', '128']) - procSearchMock.byOptions.return_value = \ + [ + "-ll", + TEST_SHOW, + "-alloc", + TEST_ALLOC, + "-duration", + "1.5", + "-job", + TEST_JOB, + "-limit", + resultsLimit, + "-memory", + "128", + ] + ) + procSearchMock.byOptions.return_value = ( opencue_proto.host_pb2.ProcGetProcsResponse( procs=opencue_proto.host_pb2.ProcSeq( procs=[ opencue_proto.host_pb2.Proc( - name='proc1', + name="proc1", reserved_cores=28, used_memory=44, reserved_memory=120, - job_name='mms2oazed2bbcjk60gho_w11licymr63s66bw1b3s', - frame_name='y0ihh3fxrstz6ub7ut2k', - dispatch_time=1556845762 + job_name="mms2oazed2bbcjk60gho_w11licymr63s66bw1b3s", + frame_name="y0ihh3fxrstz6ub7ut2k", + dispatch_time=1556845762, ) ] ) ) + ) cueadmin.common.handleArgs(args) procSearchMock.byOptions.assert_called_with( alloc=[TEST_ALLOC], - duration=[opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=5400)], - host=[], job=[TEST_JOB], limit=resultsLimit, - memory=[opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=134217728)], - show=[TEST_SHOW]) + duration=[ + opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion(value=5400) + ], + host=[], + job=[TEST_JOB], + limit=resultsLimit, + memory=[ + opencue.api.criterion_pb2.GreaterThanIntegerSearchCriterion( + value=134217728 + ) + ], + show=[TEST_SHOW], + ) -@mock.patch('opencue.cuebot.Cuebot.getStub') +@mock.patch("opencue.cuebot.Cuebot.getStub") class ServiceTests(unittest.TestCase): def setUp(self): self.parser = cueadmin.common.getParser() - @mock.patch('opencue.api.getDefaultServices') + @mock.patch("opencue.api.getDefaultServices") def testListDefaultServices(self, getDefaultServicesMock, getStubMock): - args = self.parser.parse_args(['-lv']) + args = self.parser.parse_args(["-lv"]) getDefaultServicesMock.return_value = [ opencue.wrappers.service.Service( opencue_proto.service_pb2.Service( - name='maya', + name="maya", threadable=False, min_cores=100, min_memory=2097152, - tags=['general', 'desktop'] - )) + tags=["general", "desktop"], + ) + ) ] cueadmin.common.handleArgs(args) getDefaultServicesMock.assert_called_with() - @mock.patch('opencue.api.findShow') + @mock.patch("opencue.api.findShow") def testListShowServices(self, findShowMock, getStubMock): - args = self.parser.parse_args(['-lv', TEST_SHOW]) + args = self.parser.parse_args(["-lv", TEST_SHOW]) showMock = mock.Mock() showMock.getServiceOverrides.return_value = [ opencue.wrappers.service.Service( opencue_proto.service_pb2.Service( - name='maya', + name="maya", threadable=False, min_cores=100, min_memory=2097152, - tags=['general', 'desktop'] + tags=["general", "desktop"], ) ) ] @@ -807,5 +898,5 @@ def testListShowServices(self, findShowMock, getStubMock): showMock.getServiceOverrides.assert_called_with() -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/cueadmin/tests/test_format.py b/cueadmin/tests/test_format.py index 2cff81dda..63bea3cf1 100644 --- a/cueadmin/tests/test_format.py +++ b/cueadmin/tests/test_format.py @@ -18,13 +18,11 @@ """Tests for cueadmin.format.""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function -import unittest -import time import sys +import time +import unittest from unittest import mock import cueadmin.format as fmt diff --git a/cueadmin/tests/test_output.py b/cueadmin/tests/test_output.py index dca71dd7b..291135d79 100644 --- a/cueadmin/tests/test_output.py +++ b/cueadmin/tests/test_output.py @@ -18,27 +18,20 @@ """Tests for cueadmin.output.""" -from __future__ import print_function -from __future__ import division -from __future__ import absolute_import +from __future__ import absolute_import, division, print_function # pylint: disable=wrong-import-order,wrong-import-position from future import standard_library + standard_library.install_aliases() import contextlib -import mock import io import sys import time import unittest -import opencue_proto.facility_pb2 -import opencue_proto.host_pb2 -import opencue_proto.job_pb2 -import opencue_proto.service_pb2 -import opencue_proto.show_pb2 -import opencue_proto.subscription_pb2 +import mock import opencue.wrappers.allocation import opencue.wrappers.frame import opencue.wrappers.host @@ -48,10 +41,15 @@ import opencue.wrappers.service import opencue.wrappers.show import opencue.wrappers.subscription +import opencue_proto.facility_pb2 +import opencue_proto.host_pb2 +import opencue_proto.job_pb2 +import opencue_proto.service_pb2 +import opencue_proto.show_pb2 +import opencue_proto.subscription_pb2 import cueadmin.output - # pylint: disable=line-too-long @@ -66,36 +64,39 @@ def captured_output(): sys.stdout, sys.stderr = old_out, old_err -@mock.patch('opencue.cuebot.Cuebot.getStub') -@mock.patch('time.time', mock.MagicMock(return_value=1556846762+time.altzone)) +@mock.patch("opencue.cuebot.Cuebot.getStub") +@mock.patch("time.time", mock.MagicMock(return_value=1556846762 + time.altzone)) class OutputTests(unittest.TestCase): def testDisplayProcs(self, getStubMock): procs = [ opencue.wrappers.proc.Proc( opencue_proto.host_pb2.Proc( - name='proc1', + name="proc1", reserved_cores=28, used_memory=44, reserved_memory=120, - job_name='mms2oazed2bbcjk60gho_w11licymr63s66bw1b3s', - frame_name='y0ihh3fxrstz6ub7ut2k', - dispatch_time=1556845762+time.altzone - ))] + job_name="mms2oazed2bbcjk60gho_w11licymr63s66bw1b3s", + frame_name="y0ihh3fxrstz6ub7ut2k", + dispatch_time=1556845762 + time.altzone, + ) + ) + ] with captured_output() as (out, err): cueadmin.output.displayProcs(procs) self.assertEqual( - 'Host Cores Memory Job / Frame Start Runtime \n' - 'proc1 28.00 44K of 120K (36.67%) mms2oazed2bbcjk60gho_w11licy.. / y0ihh3fxrstz6ub7ut2k 05/03 01:09 00:16:40 \n', - out.getvalue()) + "Host Cores Memory Job / Frame Start Runtime \n" + "proc1 28.00 44K of 120K (36.67%) mms2oazed2bbcjk60gho_w11licy.. / y0ihh3fxrstz6ub7ut2k 05/03 01:09 00:16:40 \n", + out.getvalue(), + ) def testDisplayHosts(self, getStubMock): hosts = [ opencue.wrappers.host.Host( opencue_proto.host_pb2.Host( - name='host1', + name="host1", load=25, nimby_enabled=False, free_memory=3500000, @@ -105,125 +106,139 @@ def testDisplayHosts(self, getStubMock): memory=4500000, idle_cores=5, idle_memory=3000000, - os='Linux', - boot_time=1556836762+time.altzone, + os="Linux", + boot_time=1556836762 + time.altzone, state=1, lock_state=1, - alloc_name='alloc01', - thread_mode=1 - ))] + alloc_name="alloc01", + thread_mode=1, + ) + ) + ] with captured_output() as (out, err): cueadmin.output.displayHosts(hosts) self.assertEqual( - 'Host Load NIMBY freeMem freeSwap freeMcp Cores Mem Idle Os Uptime State Locked Alloc Thread \n' - 'host1 25 False 3.3G 1015M 80.9G 6.0 4.3G [ 5.00 / 2.9G ] Linux 00:02 DOWN LOCKED alloc01 ALL \n', - out.getvalue()) + "Host Load NIMBY freeMem freeSwap freeMcp Cores Mem Idle Os Uptime State Locked Alloc Thread \n" + "host1 25 False 3.3G 1015M 80.9G 6.0 4.3G [ 5.00 / 2.9G ] Linux 00:02 DOWN LOCKED alloc01 ALL \n", + out.getvalue(), + ) def testDisplayShows(self, getStubMock): shows = [ opencue.wrappers.show.Show( opencue_proto.show_pb2.Show( - name='testing', + name="testing", active=True, show_stats=opencue_proto.show_pb2.ShowStats( reserved_cores=265, running_frames=100, pending_frames=248, - pending_jobs=29 - ) - ))] + pending_jobs=29, + ), + ) + ) + ] with captured_output() as (out, err): cueadmin.output.displayShows(shows) self.assertEqual( - 'Show Active ReservedCores RunningFrames PendingFrames PendingJobs\n' - 'testing True 265.00 100 248 29\n', - out.getvalue()) + "Show Active ReservedCores RunningFrames PendingFrames PendingJobs\n" + "testing True 265.00 100 248 29\n", + out.getvalue(), + ) def testDisplayServices(self, getStubMock): services = [ opencue.wrappers.service.Service( opencue_proto.service_pb2.Service( - name='maya', + name="maya", threadable=False, min_cores=100, min_memory=2097152, - tags=['general', 'desktop'] - )) + tags=["general", "desktop"], + ) + ) ] with captured_output() as (out, err): cueadmin.output.displayServices(services) self.assertEqual( - 'Name Can Thread Min Cores Units Min Memory Tags \n' - 'maya False 100 2097152 MB general | desktop \n', - out.getvalue()) + "Name Can Thread Min Cores Units Min Memory Tags \n" + "maya False 100 2097152 MB general | desktop \n", + out.getvalue(), + ) def testDisplayAllocations(self, getStubMock): allocs = [ opencue.wrappers.allocation.Allocation( opencue_proto.facility_pb2.Allocation( - name='local.desktop', - tag='desktop', + name="local.desktop", + tag="desktop", billable=False, stats=opencue_proto.facility_pb2.AllocationStats( running_cores=100, available_cores=125, cores=600, locked_hosts=25, - down_hosts=3 - ) - ))] + down_hosts=3, + ), + ) + ) + ] with captured_output() as (out, err): cueadmin.output.displayAllocations(allocs) self.assertEqual( - 'Name Tag Running Avail Cores Hosts Locked Down Billable\n' - 'local.desktop desktop 100.00 125.00 600.0 0 25 3 False \n', - out.getvalue()) + "Name Tag Running Avail Cores Hosts Locked Down Billable\n" + "local.desktop desktop 100.00 125.00 600.0 0 25 3 False \n", + out.getvalue(), + ) def testDisplaySubscriptions(self, getStubMock): subs = [ opencue.wrappers.subscription.Subscription( opencue_proto.subscription_pb2.Subscription( - allocation_name='local.general', - show_name='showName', + allocation_name="local.general", + show_name="showName", size=1000, burst=1500, - reserved_cores=500 - )), + reserved_cores=500, + ) + ), opencue.wrappers.subscription.Subscription( opencue_proto.subscription_pb2.Subscription( - allocation_name='cloud.desktop', - show_name='showName', + allocation_name="cloud.desktop", + show_name="showName", size=0, burst=1500, - reserved_cores=50 - )), + reserved_cores=50, + ) + ), ] with captured_output() as (out, err): - cueadmin.output.displaySubscriptions(subs, 'showName') + cueadmin.output.displaySubscriptions(subs, "showName") self.assertEqual( - 'Subscriptions for showName\n' - 'Allocation Show Size Burst Run Used\n' - 'local.general showName 10.0 15.0 5.00 50.00%\n' - 'cloud.desktop showName 0.0 15.0 0.50 0.50%\n', - out.getvalue()) + "Subscriptions for showName\n" + "Allocation Show Size Burst Run Used\n" + "local.general showName 10.0 15.0 5.00 50.00%\n" + "cloud.desktop showName 0.0 15.0 0.50 0.50%\n", + out.getvalue(), + ) def testDisplayJobs(self, getStubMock): jobs = [ opencue.wrappers.job.Job( opencue_proto.job_pb2.Job( - name='d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d', + name="d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d", is_paused=False, - group='u0uMmB1O0z3ZkvreFYzP', + group="u0uMmB1O0z3ZkvreFYzP", job_stats=opencue_proto.job_pb2.JobStats( running_frames=5, reserved_cores=5, @@ -231,13 +246,14 @@ def testDisplayJobs(self, getStubMock): ), priority=89, min_cores=1, - max_cores=1 - )), + max_cores=1, + ) + ), opencue.wrappers.job.Job( opencue_proto.job_pb2.Job( - name='mlSCNFWWwksH8i0rb8UE-v5u1bh5jfixzXG7', + name="mlSCNFWWwksH8i0rb8UE-v5u1bh5jfixzXG7", is_paused=True, - group='u0uMmB1O0z3ZkvreFYzP', + group="u0uMmB1O0z3ZkvreFYzP", job_stats=opencue_proto.job_pb2.JobStats( running_frames=2300, reserved_cores=1000, @@ -245,25 +261,27 @@ def testDisplayJobs(self, getStubMock): ), priority=95, min_cores=6, - max_cores=None - )), + max_cores=None, + ) + ), ] with captured_output() as (out, err): cueadmin.output.displayJobs(jobs) self.assertEqual( - 'Job Group Booked Cores Wait Pri MinCores MaxCores\n' - 'd7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHm.. u0uMmB1O0z3Zk.. 5 5.00 182 89 1.00 1.00\n' - 'mlSCNFWWwksH8i0rb8UE-v5u1bh5jfixzXG7 [paused] u0uMmB1O0z3Zk.. 2300 1000.00 0 95 6.00 0.00\n', - out.getvalue()) + "Job Group Booked Cores Wait Pri MinCores MaxCores\n" + "d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHm.. u0uMmB1O0z3Zk.. 5 5.00 182 89 1.00 1.00\n" + "mlSCNFWWwksH8i0rb8UE-v5u1bh5jfixzXG7 [paused] u0uMmB1O0z3Zk.. 2300 1000.00 0 95 6.00 0.00\n", + out.getvalue(), + ) - @mock.patch('opencue.wrappers.job.Job.getLayers') + @mock.patch("opencue.wrappers.job.Job.getLayers") def testDisplayJobInfo(self, getLayersMock, getStubMock): job = opencue.wrappers.job.Job( opencue_proto.job_pb2.Job( - name='d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d', - start_time=1556836762+time.altzone, + name="d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d", + start_time=1556836762 + time.altzone, is_paused=True, min_cores=1, max_cores=6, @@ -281,86 +299,85 @@ def testDisplayJobInfo(self, getLayersMock, getStubMock): getLayersMock.return_value = [ opencue.wrappers.layer.Layer( opencue_proto.job_pb2.Layer( - name='preflight', - tags=['preflightTag', 'general'], + name="preflight", + tags=["preflightTag", "general"], layer_stats=opencue_proto.job_pb2.LayerStats( - total_frames=2, - succeeded_frames=1 - ) + total_frames=2, succeeded_frames=1 + ), ) ), opencue.wrappers.layer.Layer( opencue_proto.job_pb2.Layer( - name='render', - tags=['renderPool'], + name="render", + tags=["renderPool"], layer_stats=opencue_proto.job_pb2.LayerStats( - total_frames=2598, - succeeded_frames=149 - ) + total_frames=2598, succeeded_frames=149 + ), ) - ) + ), ] with captured_output() as (out, err): cueadmin.output.displayJobInfo(job) self.assertEqual( - '------------------------------------------------------------\n' - 'job: d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d\n' - '\n' - ' start time: 05/02 22:39\n' - ' state: PAUSED\n' - ' type: N/A\n' - ' architecture: N/A\n' - ' services: N/A\n' - 'Min/Max cores: 1.00 / 6.00\n' - '\n' - 'total number of frames: 2600\n' - ' done: 150\n' - ' running: 2300\n' - ' waiting (ready): 100\n' - ' waiting (depend): 50\n' - ' failed: 0\n' - ' total frame retries: N/A\n' - '\n' - 'this is a OpenCue job with 2 layers\n' - '\n' - 'preflight (2 frames, 1 done)\n' - ' average frame time: N/A\n' - ' average ram usage: N/A\n' - ' tags: preflightTag | general\n' - '\n' - 'render (2598 frames, 149 done)\n' - ' average frame time: N/A\n' - ' average ram usage: N/A\n' - ' tags: renderPool\n' - '\n', - out.getvalue()) + "------------------------------------------------------------\n" + "job: d7HXvMXDNMKyfzLumwsY-P3CNG1w4pa452dGcqOyf_qVK5PbHmCZafkv4rEF8d\n" + "\n" + " start time: 05/02 22:39\n" + " state: PAUSED\n" + " type: N/A\n" + " architecture: N/A\n" + " services: N/A\n" + "Min/Max cores: 1.00 / 6.00\n" + "\n" + "total number of frames: 2600\n" + " done: 150\n" + " running: 2300\n" + " waiting (ready): 100\n" + " waiting (depend): 50\n" + " failed: 0\n" + " total frame retries: N/A\n" + "\n" + "this is a OpenCue job with 2 layers\n" + "\n" + "preflight (2 frames, 1 done)\n" + " average frame time: N/A\n" + " average ram usage: N/A\n" + " tags: preflightTag | general\n" + "\n" + "render (2598 frames, 149 done)\n" + " average frame time: N/A\n" + " average ram usage: N/A\n" + " tags: renderPool\n" + "\n", + out.getvalue(), + ) def testDisplayFrames(self, getStubMock): frames = [ opencue.wrappers.frame.Frame( opencue_proto.job_pb2.Frame( - name='rFNQafSvWkCQA3O7SaJw-tWa1L92CjGM0syBsxMwp-8sk6X0thFbCFaL06wAPc', + name="rFNQafSvWkCQA3O7SaJw-tWa1L92CjGM0syBsxMwp-8sk6X0thFbCFaL06wAPc", state=opencue_proto.job_pb2.SUCCEEDED, - start_time=1556836762+time.altzone, - stop_time=1556846762+time.altzone, + start_time=1556836762 + time.altzone, + stop_time=1556846762 + time.altzone, max_rss=927392, exit_status=0, - last_resource='render-host-01', - retry_count=1 + last_resource="render-host-01", + retry_count=1, ) ), opencue.wrappers.frame.Frame( opencue_proto.job_pb2.Frame( - name='XjWPTN6CsAujCmgKHfyA-u2wFSQg2MNu', + name="XjWPTN6CsAujCmgKHfyA-u2wFSQg2MNu", state=opencue_proto.job_pb2.WAITING, start_time=None, stop_time=None, max_rss=None, exit_status=None, last_resource=None, - retry_count=0 + retry_count=0, ) ), ] @@ -369,12 +386,13 @@ def testDisplayFrames(self, getStubMock): cueadmin.output.displayFrames(frames) self.assertEqual( - 'Frame Status Host Start End Runtime Mem Retry Exit\n' - '------------------------------------------------------------------------------------------------------------------------\n' - 'rFNQafSvWkCQA3O7SaJw-tWa1L92CjGM0.. SUCCEEDED render-host-01 05/02 22:39 05/03 01:26 02:46:40 905M 1 0\n' - 'XjWPTN6CsAujCmgKHfyA-u2wFSQg2MNu WAITING --/-- --:-- --/-- --:-- 0K 0 0\n', - out.getvalue()) + "Frame Status Host Start End Runtime Mem Retry Exit\n" + "------------------------------------------------------------------------------------------------------------------------\n" + "rFNQafSvWkCQA3O7SaJw-tWa1L92CjGM0.. SUCCEEDED render-host-01 05/02 22:39 05/03 01:26 02:46:40 905M 1 0\n" + "XjWPTN6CsAujCmgKHfyA-u2wFSQg2MNu WAITING --/-- --:-- --/-- --:-- 0K 0 0\n", + out.getvalue(), + ) -if __name__ == '__main__': +if __name__ == "__main__": unittest.main() diff --git a/cueadmin/tests/test_suite.py b/cueadmin/tests/test_suite.py index 898b03515..e7d295d61 100644 --- a/cueadmin/tests/test_suite.py +++ b/cueadmin/tests/test_suite.py @@ -15,13 +15,15 @@ import unittest + def create_test_suite(): loader = unittest.TestLoader() - start_dir = '.' # Specify the directory where your test files reside - suite = loader.discover(start_dir, pattern='test_*.py') + start_dir = "." # Specify the directory where your test files reside + suite = loader.discover(start_dir, pattern="test_*.py") return suite -if __name__ == '__main__': + +if __name__ == "__main__": runner = unittest.TextTestRunner() test_suite = create_test_suite() runner.run(test_suite) diff --git a/cueadmin/tox.ini b/cueadmin/tox.ini new file mode 100644 index 000000000..7373a9e9b --- /dev/null +++ b/cueadmin/tox.ini @@ -0,0 +1,45 @@ +[tox] +envlist = py{38,39,310,311,312,313}, lint, coverage +isolated_build = true + +[testenv] +deps = + pytest>=8.0.0 + pytest-mock>=3.10.0 + mock>=4.0.0 + pyfakefs>=5.2.3 +commands = + pytest {posargs:tests} + +[testenv:lint] +deps = + pylint>=3.0.0 +commands = + pylint --rcfile=../ci/pylintrc_main cueadmin + pylint --rcfile=../ci/pylintrc_test tests + +[testenv:coverage] +deps = + pytest>=8.0.0 + pytest-cov>=4.0.0 + pytest-mock>=3.10.0 + mock>=4.0.0 + pyfakefs>=5.2.3 +commands = + pytest --cov=cueadmin --cov-report=term-missing --cov-report=html --cov-report=xml tests + +[testenv:format] +deps = + black>=23.0.0 + isort>=5.12.0 +commands = + black cueadmin tests + isort cueadmin tests + +[testenv:format-check] +deps = + black>=23.0.0 + isort>=5.12.0 +commands = + black --check cueadmin tests + isort --check cueadmin tests \ No newline at end of file diff --git a/docs/_docs/reference/tools/cueadmin.md b/docs/_docs/reference/tools/cueadmin.md index d2db93e11..160529bee 100644 --- a/docs/_docs/reference/tools/cueadmin.md +++ b/docs/_docs/reference/tools/cueadmin.md @@ -680,6 +680,49 @@ CueAdmin returns the following exit codes: - `1`: General error or operation failed - `2`: Invalid arguments or command syntax +## Development and Testing + +### Running Tests + +CueAdmin includes a comprehensive test suite with 16+ tests: + +```bash +# Install with test dependencies +pip install -e ".[test]" + +# Run all tests +pytest + +# Run with coverage +pytest --cov=cueadmin --cov-report=term-missing +``` + +### Test Types + +- **Unit Tests** - Function-level testing of core functionality +- **Integration Tests** - End-to-end workflow testing +- **Coverage Testing** - Code coverage analysis and reporting + +### Development Setup + +```bash +# Install development dependencies +pip install -e ".[dev]" + +# Run tests with linting +pytest && pylint cueadmin tests + +# Format code +black cueadmin tests && isort cueadmin tests +``` + +### Continuous Integration + +The test suite is integrated into: +- GitHub Actions for automated testing +- Docker builds for container-based testing +- Lint pipeline for code quality checks + ## Additional Resources - [CueAdmin Tutorial](/docs/tutorials/cueadmin-tutorial/) - Step-by-step tutorial with practical examples diff --git a/docs/_docs/tutorials/cueadmin-tutorial.md b/docs/_docs/tutorials/cueadmin-tutorial.md index 21f257691..187c4516d 100644 --- a/docs/_docs/tutorials/cueadmin-tutorial.md +++ b/docs/_docs/tutorials/cueadmin-tutorial.md @@ -624,5 +624,27 @@ Remember to always: ## Next Steps - Explore the [CueAdmin Reference](/docs/reference/tools/cueadmin/) for complete command documentation -- Learn about [Cueman](/docs/reference/tools/cueman/) for job management -- Practice with test shows and allocations before working on production \ No newline at end of file +- Practice with test shows and allocations before working on production +- Learn about [CueAdmin development and testing](/docs/reference/tools/cueadmin/#development-and-testing) if you want to contribute +- Continue to the [Developer Guide](/docs/developer-guide/) to learn about contributing to OpenCue + +## Development and Contributing + +CueAdmin is actively developed with: +- **Comprehensive test suite** with tests covering unit and integration scenarios +- **Modern testing infrastructure** using pytest, coverage reporting, and CI/CD integration +- **Development tools** including linting, formatting, and multi-Python version testing + +To contribute or run tests locally: + +```bash +# Install with development dependencies +pip install -e ".[dev]" + +# Run the test suite +pytest --cov=cueadmin --cov-report=term-missing + +# Format and lint code +black cueadmin tests && isort cueadmin tests +pylint cueadmin tests +```