feat(cli): add CLI with CI convenience functions (#394)

* feat(cli): add CLI with CI convenience functions for NiFi automation

- Add nipyapi CLI with 'ci' subcommand for NiFi CI/CD operations
- Profile auto-resolution: switch() without args auto-detects env vars or profile file
- CI functions: deploy_flow, start_flow, stop_flow, get_status, cleanup, configure_params
- New functions: configure_inherited_params, purge_flowfiles, upload_asset
- get_status defaults to root process group when no PG specified
- Log control via NIFI_LOG_LEVEL and NIFI_LOG_ON_ERROR env vars
- Suppress SSL warnings to keep CI output clean
- Convert snake_case to kebab-case for GitHub Actions output format
- resolve_git_ref to resolve tags/branches to SHAs

* feat: add layout module, CLI enhancements, and improve test infrastructure

Layout Module:
- New nipyapi/layout.py with intelligent canvas positioning
- Process group grid alignment (align_pg_grid, suggest_pg_position)
- Flow layout functions (below, above, right_of, left_of, fork, new_flow)
- Component movement with self-loop bend handling
- Flow transposition and de-overlap capabilities
- Spine/branch detection for flow restructuring (find_flow_spine, get_side_branches)
- Automated flow layout (suggest_flow_layout)

Canvas Enhancements:
- get_flow_components() for graph traversal of connected components
- Automatic bend calculation for self-loop connections
- Bends parameter support in create_connection()

CLI Improvements:
- Expose layout functions via CLI (align_pg_grid, suggest_pg_position, etc.)
- SafeModule wrapper for structured error handling
- Custom serializer for formatted output

Client Generation:
- Add octet-stream handler to rest.mustache template
- Regenerate NiFi and Registry clients with binary upload support

Profile System:
- Fix get_default_profile_name() to honor NIPYAPI_PROFILES_FILE env var
- Improve profile resolution order for testing isolation

CI Module:
- Fix purge_flowfiles.py to use detail='all' for process group status

Test Infrastructure:
- Add NIPYAPI_PROFILES_FILE to Makefile for test isolation
- Update conftest.py to use env var for profiles path
- Add comprehensive tests for layout, CLI, CI, canvas, and parameters
- 332 tests passing, lint clean (10.00/10)

* fix: remove switch() from CI functions to prevent profile override

- CI functions no longer call profiles.switch() internally
- Profile configuration is now solely the responsibility of the caller
- CLI main() handles switch() with --profile arg or auto-resolution
- Fixes bug where --profile flag was ignored by CI commands

Also includes:
- Add --profile CLI option for explicit profile selection
- Add NIPYAPI_PROFILE env var support in switch() auto-resolution
- Update profiles.rst documentation
- Add tests for new profile resolution behavior

* fix: regenerate clients with basicAuth and fix Python 3.9 test

- Regenerate NiFi and Registry clients from augmented OpenAPI specs
- Restore basicAuth in Registry configuration.py and access_api.py
- Fix mock patch path in test_ci.py for Python 3.9 compatibility
  (use @patch('requests.get') instead of module path)

* fix(ci): select token env var based on provider

The ensure_registry function was resolving the token before determining
the provider, always preferring GH_REGISTRY_TOKEN regardless of whether
gitlab was specified. This caused GitLab deployments to fail when both
tokens were set.

Now resolves provider first, then selects the appropriate token:
- gitlab: GL_REGISTRY_TOKEN first, fallback to GH_REGISTRY_TOKEN
- github: GH_REGISTRY_TOKEN first, fallback to GL_REGISTRY_TOKEN

* style(tests): use urlparse for hostname assertion

Replace string 'in' check with urlparse().hostname for cleaner URL validation.

* test: improve coverage for cli, profiles, and ci modules

CLI coverage: 62% -> 81%
- Add tests for list serialization, heredoc format, _to_dict methods
- Add tests for _parse_profile_arg function
- Add tests for SafeModule error handling and log level inclusion

Profiles coverage: 68% -> 74%
- Add tests for unicode error handling in properties files
- Add tests for get_default_profile_name edge cases

CI coverage: 50% -> 58%
- ensure_registry: 17% -> 95% (validation + token priority tests)
- deploy_flow: 17% -> 61% (validation tests)
- configure_params: 27% -> 49% (JSON validation tests)

Overall coverage: 66% -> 69%

* docs: add comprehensive CLI and CI documentation

CLI documentation (cli.rst):
- Installation options (pip, uv, uvx)
- Configuration (env vars, profiles, priority)
- Discovering commands with --help
- All available modules with API reference links
- Output formatting (JSON, GitHub Actions, GitLab)
- Note that low-level nifi/registry APIs not exposed

CI documentation (ci.rst):
- All 14 CI operations with full parameter tables
- configure_inherited_params for inheritance-aware updates
- resolve_git_ref utility function
- Environment variable reference
- Complete workflow examples (shell, GitHub, GitLab, Python)

Other changes:
- Fix test_safe_module_error_handling for CI environments
- Fix security.rst title (was incorrectly 'NiPyAPI 2')
- Add layout module to core_modules docs
- Update dependencies list

* test: improve coverage for CI, parameters, layout modules and fix test isolation

Add integration tests against real NiFi for better coverage:

test_ci.py:
- configure_inherited_params: 10 tests (validation + integration)
- upload_asset: 8 tests (file path, PG resolution, param linking)

test_parameters.py:
- get_parameter_context_hierarchy: 3 tests
- get_parameter_ownership_map: 2 tests
- update_parameter_in_context: 4 tests

test_layout.py:
- move_port: 3 tests (input, output, no-refresh)
- move_component with port: 1 test
- snap_position, get_pg_grid_position, left_of, check_overlap: 5 tests

test_versioning.py:
- Fix test isolation: wait for async revert to complete in
  test_revert_flow_ver_wait_false before test ends, preventing
  revision conflicts in subsequent tests using same fixture

* feat(ci): add version control CI functions and helpers

New CI functions:
- commit_flow: save flow to version control (initial or subsequent)
- get_flow_versions: list version history for a flow
- detach_flow: remove version control (for forking workflows)
- get_flow_diff: get local modifications to a versioned flow

Renames for consistency:
- change_version -> change_flow_version
- get_versions -> list_flows

New versioning helper:
- get_local_modifications: helper for fetching local changes

Improvements:
- list_flows: use nipyapi.utils.getenv_bool for env var parsing
- list_flows: use nipyapi.canvas.get_flow helper instead of direct API
- save_git_flow_ver: accept string process group ID

Tests:
- Added comprehensive tests for all new CI functions
- Fixed flaky versioning tests with proper state handling
- Improved test isolation in shared fixtures

* feat(ci): add export_flow_definition and import_flow_definition commands

- Add export_flow_definition CI function for exporting process groups to files
- Add import_flow_definition CI function for importing flow definitions from files
- Support JSON and YAML formats for export
- Add include_referenced_services option to include external controller services
- Use smart positioning (suggest_pg_position) when import location not provided
- Add 10 tests covering validation, roundtrip export/import, and file operations

* feat(extensions): add NAR management with defensive lifecycle handling

- Add nipyapi.extensions module for NAR upload, delete, download
- Add Python processor initialization status and wait functions
- Add processor bundle version management for multi-version NARs
- Implement defensive delete_nar with initialization guard and cleanup wait
- Implement two-phase upload_nar wait (install + processor discovery)
- Fix canvas.create_processor to include bundle for multi-version support
- Add programmatic NAR generation for testing (create_test_nar)
- Add comprehensive tests for extension operations
- Add extensions operations guide (docs/extensions.rst) covering NAR lifecycle,
  Python processor initialization, version management, and troubleshooting

* fix(cli): quote dotenv values containing special characters

Values with spaces, pipes, brackets, and other shell special characters
are now properly quoted in dotenv output format. This prevents export
failures when values contain characters like | [ ] that break shell parsing.

Embedded double quotes are escaped with backslash.

* feat(parameters): enhance get_parameter_context_hierarchy with bindings and optional parameters

- Add include_bindings parameter to include bound_process_groups for each context
- Add include_parameters parameter to optionally exclude parameter details
- Add description field to parameter output for better context
- Add tests for new functionality including transitive binding behavior
- Useful for cleanup workflows and parameter inspection during deployment

* feat(cli): add --version and verbosity flags

- Add --version/-V flags to display version and exit
- Add -v/-vv/-vvv flags to control log verbosity
- Refactor _parse_profile_arg to _parse_cli_flags for combined parsing
- Add _apply_verbosity to set NIFI_LOG_LEVEL from verbosity count
- Add comprehensive unit tests for flag parsing
- Add integration tests for version output

* feat(canvas): allow create_process_group to accept pg_id string

- Accept either a string ID or ProcessGroupEntity for parent_pg parameter
- Maintains backward compatibility with existing code using entities
- Raises TypeError with clear message for invalid types
- Supports 'root' as parent_pg for creating PGs on root canvas
- Add tests for string ID, UUID string, and invalid type cases

* feat: add list_registry_flows CI function and improve registry client handling

Registry client improvements:
- versioning.get_registry_client: add greedy parameter (default True for compat)
- ci.deploy_flow: accept registry client name or ID, add greedy param
- ci.list_registry_flows: new function to list flows in registry bucket

Canvas improvements:
- create_process_group: accept string ID or ProcessGroupEntity for parent_pg
- delete_remote_process_group: add force param to stop transmission before delete
- set_remote_process_group_transmission: wait for state change before returning

Test fixes:
- conftest: use force=True when cleaning up RPGs
- Add tests for greedy parameter and list_registry_flows validation

* fix: JSON-serialize lists in GitHub/GitLab CI output formats

Previously, lists nested within dict results were converted via str()
which produces Python repr format with single quotes. This is not valid
JSON and breaks downstream parsing.

Now lists and dicts at leaf positions are JSON-serialized with json.dumps()
producing valid JSON with double quotes.

Fixes output for: list_flows, list_registry_flows, get_flow_versions,
get_flow_diff, and any future CI functions returning list values.

Added tests for nested list serialization in both GitHub and dotenv formats.

* feat(ci): add is_uuid() utility and auto-detect registry client ID vs name

- Add is_uuid() function to nipyapi/utils.py for UUID format detection
- Update deploy_flow and list_registry_flows to auto-detect if registry_client is UUID (ID) or name
- Add comprehensive tests for is_uuid() in test_utils.py
- Update docstrings to document auto-detection behavior

* feat(layout): add connection management and improved bend handling

- Add get_connection() to fetch connection by ID or entity
- Add update_connection() with name/bends parameters (bends=[] clears)
- Add clear_flow_bends() to clear bends before flow reorganization
- Update get_flow_components() to return FlowSubgraph(components, connections)
- Add include_retry parameter to move_component/move_processor
- Refactor transpose_flow to handle all bends in single pass
- Add comprehensive tests for new functionality

* feat: add config verification functions and test optimizations

- Add nipyapi.canvas.verify_controller() for controller service verification
- Add nipyapi.canvas.verify_processor() for processor verification
- Add nipyapi.ci.verify_config() for batch verification of process groups
- Add refresh parameter to delete_controller and update_controller
- Improve delete_controller to accept ID or object
- Add PYTEST_ARGS support to Makefile test target
- Add fix_multi_version_nars module-scoped fixture for test optimization
- Remove useless @pytest.mark.usefixtures() decorator
- Extension tests now ~27% faster with shared NAR fixtures

* feat: add orphaned context cleanup and bulletin filtering

- Add list_orphaned_contexts() to parameters module for detecting
  parameter contexts not bound to any process groups
- Enhance get_bulletin_board() with pg_id, source_name, message,
  and limit filters for scoped bulletin retrieval
- Add delete_orphaned_contexts option to ci.cleanup() for safe
  removal of unused parameter contexts after PG deletion
- Add comprehensive tests for all new functionality

Note: NiFi does not provide an API to clear bulletins in 2.6.0.
Clear bulletins API was introduced in 2.7.0 - upgrade planned separately.

* feat(bulletins): add bulletin management module with NiFi 2.7.0+ clearing

New nipyapi.bulletins module:
- get_bulletins(): controller-level bulletins
- get_bulletin_board(): filtered bulletins returning BulletinDTO directly
- clear_*_bulletins(): component-specific clearing (10 functions)
- clear_all_bulletins(): high-level helper for CI

Additional changes:
- nipyapi.profiles: add list_profiles, show, current commands
- nipyapi.utils: add format_timestamp utility
- nipyapi.ci.get_status: fix to capture controller service bulletins
- nipyapi.canvas: bulletin functions now alias to bulletins module
- Makefile: intelligent uv/pip detection
- docs: updated contributing guide, module documentation

* feat(canvas): add component state management functions

- Add get_processor_state and clear_processor_state for processor state inspection
- Add get_controller_state and clear_controller_state for controller service state
- All functions accept entity objects or ID strings
- Fix CLI --help flag handling in SafeModule wrapper
- Remove unused LogCapture.all_records attribute
- Add fix_state_flow test fixture with MapCacheServer and ListFile
- Add comprehensive tests for state operations

* feat(ci): add parameter export/import and VCS parameter handling

- Add export_parameters CI command for exporting parameter values
- Add --parameters_file option to configure_inherited_params for file-based import
- Add parameter_context_handling option to deploy_git_registry_flow for VCS
- Add deploy_flow CI command parameter_context_handling support
- Fix test fixture isolation to prevent shared fixtures being deleted
- Fix parameter context cleanup version check logic
- Add ACTIVE_PROFILES_PATH for consistent profile file handling
- Add parameter inheritance tests for VCS deployments

* test: add comprehensive CLI JSON input/output serialization tests

Add 43 new tests for CLI handling of complex JSON objects:

- 19 JSON input parsing tests (quotes, newlines, backslashes, nested
  structures, Unicode, null/empty values, mixed types)
- 4 roundtrip tests (serialize -> parse verification)
- 8 CLI subprocess tests (verify JSON passes through CLI correctly)
- 12 CI function tests for configure_params complex inputs

These tests verify AI agents can use the nipyapi CLI directly for
common operations without needing to write Python code for complex
parameter structures.

* fix(tests): force JSON output format in CLI subprocess tests

In GitHub Actions CI, GITHUB_ACTIONS=true causes the CLI to
auto-detect 'github' output format (key=value), but the tests
expect JSON output for parsing. Force NIFI_OUTPUT_FORMAT=json
in the subprocess environment to ensure consistent JSON output.

Fixes JSONDecodeError in configure_params CLI tests.

* docs: comprehensive docstring and documentation revision

- Create AGENTS.md for AI agent guidance (consuming/contributing sections)
- Update contributing.rst with docstring standards and discovery patterns
- Fix all 51 Sphinx docstring warnings (nested lists, Example formatting)
- Standardize Example:: format across 91 instances with required blank lines
- Fix RST title underlines in ci.rst, security.rst
- Add cross-reference notation guidance for Sphinx
- Add module intent comments to __init__.py

* feat(canvas): add get_processor_docs() and enhance schedule_processor()

New features:
- get_processor_docs(processor): retrieve full processor documentation including
  property_descriptors, tags, use cases. Accepts ProcessorEntity, DocumentedTypeDTO,
  or processor type name string.

- schedule_processor() enhancements:
  - Now accepts processor ID string or ProcessorEntity object
  - Supports all processor states: RUNNING, STOPPED, DISABLED, RUN_ONCE
  - Uses direct processor API for reliable state transitions
  - Backwards compatible: bool values still work (True=RUNNING, False=STOPPED)

Client fixes:
- Add nifi_processor_state_enum.py augmentation to fix upstream NiFi OpenAPI bug
  where ProcessorDTO.state enum was missing transitional states

Testing improvements:
- Add pytest filterwarnings config to suppress InsecureRequestWarning in tests
- Add comprehensive tests for new functionality

Docs:
- Minor AGENTS.md improvement: venv activation reminder

* Add FlowFile inspection helpers and enhance fs_write for binary mode

- nipyapi.canvas: Add list_flowfiles, get_flowfile_details, get_flowfile_content, peek_flowfiles
- get_flowfile_content supports decode options (auto/text/bytes) and output_file saving
- nipyapi.utils.fs_write: Add binary=False parameter for raw byte writing (backwards compatible)
- Tests: Add test_fs_write_binary, expand test_get_flowfile_content coverage

* Address Copilot review feedback

- Extract NiFi Registry functions to nipyapi.nifi_registry module
  with backwards-compatible shims in versioning.py
- Replace global cache with @lru_cache in conftest.py
- Clarify switch() usage in profiles.py error message
- Document UUID format-only validation in utils.py
- Fix URL validation in test_ci.py (use urlparse for hostname check)

* Fix cluster_node_id handling, remove fail_on_error, improve CLI docstrings

Bug fixes:
- canvas.py: Add _resolve_flowfile_cluster_node() helper for clustered NiFi
- canvas.py: get_flowfile_details/get_flowfile_content auto-resolve cluster_node_id
- canvas.py: peek_flowfiles preserves cluster_node_id from FlowFileSummaryDTO
- canvas.py: create_processor accepts string ID or ProcessGroupEntity
- verify_config.py: Remove fail_on_error param (always return result dict)

CLI improvements:
- cli.py: Use functools.wraps for proper signature preservation in fire
- canvas.py, layout.py, utils.py: Inline docstring format for CLI compatibility

Documentation:
- contributing.rst: Add CLI-specific docstring guidance

Tests:
- test_canvas.py: Add tests for cluster_node_id handling
- test_ci.py: Update tests for verify_config changes

* docs: Add CLI boolean parameter gotcha to contributing guide

* fix: Use bulk disable instead of delete for controllers in force delete_process_group

Previously, force deletion would delete each controller service individually,
which was slow and caused version control to show local modifications.
Now uses schedule_all_controllers() to bulk disable, which is faster and
non-destructive.

* fix: Disable controllers in fix_state_flow cleanup before PG deletion

MapCacheServer binds to port 4557 and won't release it until disabled.
Without explicit cleanup, subsequent tests fail with 'Address already in use'
when trying to enable their own MapCacheServer.

Added finalizer to disable both server and client controllers before
fix_pg's finalizer runs to delete the process group.

* fix: schedule_all_controllers now waits for controllers to reach target state

Previously the function returned immediately after the API call, leaving
controllers in transitional states (ENABLING/DISABLING). This caused race
conditions in slower environments (e.g. Python 3.9) where callers would
check controller state before transitions completed.

Now uses wait_to_complete to poll until all controllers in the PG reach
the target state, consistent with schedule_controller behavior.

* docs: Prepare 1.2.0 release - CLI documentation and history

Author: Chaffelson

AGENTS.md

@@ -0,0 +1,205 @@
+# AGENTS.md
+
+Agent guidance for NiPyAPI - Apache NiFi Python Client SDK.
+
+**Which workflow applies?**
+- Using nipyapi in another project → [Consuming nipyapi](#consuming-nipyapi)
+- Working in the nipyapi repository → [Contributing to nipyapi](#contributing-to-nipyapi)
+
+---
+
+## Consuming nipyapi
+
+For agents using nipyapi to perform NiFi operations.
+
+### Install
+
+```bash
+uv pip install "nipyapi[cli]"    # Recommended: includes CLI
+pip install nipyapi              # Alternative: library only
+```
+
+### Connect
+
+```bash
+# Option 1: Profile (recommended)
+export NIPYAPI_PROFILE=my-profile
+# Create profile in ~/.nipyapi/profiles.yml - see examples/profiles.yml
+
+# Option 2: Environment variables
+export NIFI_API_ENDPOINT=https://nifi.example.com/nifi-api
+export NIFI_USERNAME=admin
+export NIFI_PASSWORD=secret
+```
+
+### Documentation
+
+| If you need to... | See |
+|-------------------|-----|
+| Install nipyapi | `README.rst` → "Quick Start" |
+| Configure profiles | `examples/profiles.yml`, `docs/profiles.rst` |
+| Authenticate (Basic, mTLS, LDAP, OIDC) | `docs/security.rst` |
+| Migrate from 0.x | `docs/migration.rst` |
+| Full API reference | https://nipyapi.readthedocs.io |
+| CLI usage and options | `nipyapi --help`, `nipyapi <module> --help` |
+| Function signatures and behavior | Read module docstrings directly (comprehensive) |
+
+### Modules
+
+Prefer in this order: `nipyapi.ci` → helper modules → low-level API.
+
+For function discovery: `nipyapi <module> --help` (CLI) or `help(nipyapi.canvas)` (Python).
+
+| If you need to... | Use |
+|-------------------|-----|
+| Deploy flows in CI/CD | `nipyapi.ci` |
+| Manage processors, process groups, connections | `nipyapi.canvas` |
+| Import/export versioned flows | `nipyapi.versioning` |
+| Authenticate, manage users/policies | `nipyapi.security` |
+| Work with parameter contexts | `nipyapi.parameters` |
+| Switch connection profiles | `nipyapi.profiles` |
+| Position components on canvas | `nipyapi.layout` |
+| Get system/cluster info | `nipyapi.system` |
+| Retrieve/filter/clear bulletins | `nipyapi.bulletins` |
+| Manage NiFi extensions (NARs) | `nipyapi.extensions` |
+| File operations, filtering, common utilities | `nipyapi.utils` |
+| Access/modify endpoint configuration | `nipyapi.config` |
+| Direct NiFi API access (when helpers insufficient) | `nipyapi.nifi.*` |
+| Direct Registry API access (when helpers insufficient) | `nipyapi.registry.*` |
+
+### CLI Examples
+
+```bash
+nipyapi ci ensure_registry --name my-registry --uri https://github.com/org/repo
+nipyapi ci deploy_flow --registry_client my-registry --bucket flows --flow my-flow
+nipyapi ci start_flow --pg_id <process-group-id>
+nipyapi canvas list_all_process_groups
+```
+
+---
+
+## Contributing to nipyapi
+
+For agents modifying the nipyapi codebase.
+
+### Project Context
+
+This project is nearly 10 years old with a substantial user base. Approach changes with care:
+
+- **Existing patterns may handle edge cases** you're not aware of - ask before "simplifying"
+- **Some code predates modern Python** (originated in Python 2 era) - refactoring requires testing
+- **Changes can have broad impact** - prefer surgical modifications over sweeping rewrites
+- **When in doubt, discuss first** - the PLAN→ACT→REVIEW workflow exists for good reason
+
+### Setup
+
+```bash
+git clone https://github.com/Chaffelson/nipyapi.git && cd nipyapi
+make dev-install    # Uses uv if available, pip otherwise
+make help           # See all available targets
+```
+
+**Important**:
+- Always activate venv first: `source .venv/bin/activate` before running Python/make commands
+- Prefer make commands: Check `make help` for available targets before using generic tooling (pip, pytest, sphinx, etc.)
+
+### Intent Routing
+
+| If you need to... | See |
+|-------------------|-----|
+| Set up dev environment | `docs/contributing.rst` → "Get Started!" |
+| Understand code organization | `docs/contributing.rst` → "Generated vs Maintained Code" |
+| Run tests | `docs/contributing.rst` → "Make Targets Quick Reference" |
+| Add a new module | `docs/contributing.rst` → "Adding New Core Modules" |
+| Submit a PR | `docs/contributing.rst` → "Pull Request Guidelines" |
+| Understand profiles system | `docs/profiles.rst` |
+| Migrate from 0.x | `docs/migration.rst` |
+
+### Before Writing Code
+
+**Discovery pattern** - check before implementing new helpers:
+
+1. Read `nipyapi/__init__.py` for module intent mapping (comments describe each module's purpose)
+2. Check `__all__` at top of relevant module (`utils.py`, `canvas.py`, etc.)
+3. Grep function names that might serve your purpose
+4. Read docstrings to understand intent and edge cases handled
+5. For tests: check `tests/conftest.py` for fixtures, read an example test file first
+
+See `docs/contributing.rst` → "Reuse Existing Code" for detailed guidance.
+
+### Critical Rules
+
+**Will cause failures if ignored:**
+
+1. **Profile required for tests**: `make test NIPYAPI_PROFILE=<profile>` - never bare pytest
+2. **Docker readiness**: Always `make wait-ready NIPYAPI_PROFILE=<profile>` before testing
+3. **Certificates before containers**: Run `make down` before `make certs`
+4. **Zsh bracket quoting**: Use `make dev-install` not bare `pip install -e .[dev]`
+
+### Code Organization
+
+**Maintained** (where to contribute):
+- `nipyapi/*.py` - Core modules
+- `tests/` - Test suite
+- `examples/` - Usage examples
+- `docs/` - Documentation
+
+**Generated** (do not directly modify):
+- `nipyapi/nifi/` - NiFi API client (generated from OpenAPI specs)
+- `nipyapi/registry/` - Registry API client (generated from OpenAPI specs)
+- `nipyapi/_version.py` - Git version (generated by setuptools-scm)
+
+**To modify generated clients**: Edit templates in `resources/client_gen/`, then run `make gen-clients`.
+See `docs/contributing.rst` → "Regenerating Clients" and "Augmentation System" for the full workflow.
+
+### Testing Workflow
+
+```bash
+make certs                              # Generate certificates (once)
+make up NIPYAPI_PROFILE=single-user     # Start Docker services
+make wait-ready NIPYAPI_PROFILE=single-user
+make test NIPYAPI_PROFILE=single-user   # Run tests
+make down                               # Stop services
+```
+
+Profiles: `single-user`, `secure-ldap`, `secure-mtls`, `secure-oidc`
+
+**Profile file behavior**: Make targets default to `examples/profiles.yml` (local Docker setup).
+To use your own profiles (e.g., `~/.nipyapi/profiles.yml`), set `NIPYAPI_PROFILES_FILE`.
+
+### Code Style
+
+- **Line length**: 100 chars
+- **Formatting**: black + isort
+- **Linting**: flake8 + pylint
+- **Docstrings**: Google style
+- **Python**: 3.9-3.12
+
+**Before committing**: Run `make pre-commit` (auto-fixes formatting, then lints).
+This avoids commit failures where black rewrites files. Use `make lint` for check-only.
+
+### Workflow
+
+1. **PLAN** - Investigate codebase, discuss approach, propose changes. No file edits until user authorizes.
+2. **ACT** - Execute agreed plan with surgical, minimal changes. Follow discovery pattern. Use established patterns.
+3. **REVIEW**:
+   - Validate implementation solves the stated problem
+   - Run `make pre-commit` to fix formatting and check lint
+   - Run tests if code was changed (`make test NIPYAPI_PROFILE=...`)
+   - Update documentation if functionality was added
+   - Present changes for user review and feedback
+   - Reflect: any lessons learned or patterns discovered to carry forward?
+
+---
+
+## References
+
+| Resource | Location |
+|----------|----------|
+| Full documentation | https://nipyapi.readthedocs.io |
+| Contributing guide | `docs/contributing.rst` |
+| Profiles guide | `docs/profiles.rst` |
+| Security guide | `docs/security.rst` |
+| Migration 0.x→1.x | `docs/migration.rst` |
+| Example scripts | `examples/` |
+| Issues | https://github.com/Chaffelson/nipyapi/issues |

Makefile

@@ -5,13 +5,30 @@
 NIFI_VERSION ?= 2.7.2
 
 # Load .env file if it exists (for secrets like GH_REGISTRY_TOKEN)
+# WARNING: All variables from .env are exported and can override profile settings.
+# If tests fail with unexpected URLs, check if .env contains NIFI_API_ENDPOINT
+# or other NIFI_*/REGISTRY_* variables that conflict with examples/profiles.yml.
+# For development/testing, either remove .env or ensure it doesn't contain
+# conflicting endpoint URLs.
 -include .env
 export
 
 # Python command for cross-platform compatibility
 # Defaults to 'python' for conda/venv users, override with PYTHON=python3 for system installs
 PYTHON ?= python
 
+# Package installer: prefer uv if available, fall back to pip
+# Users with uv get faster installs; users without uv work unchanged
+UV := $(shell command -v uv 2>/dev/null)
+ifdef UV
+    PIP := uv pip
+else
+    PIP := pip
+endif
+
+# Profiles file for development/testing (explicitly use examples file, not user's ~/.nipyapi/profiles.yml)
+NIPYAPI_PROFILES_FILE ?= examples/profiles.yml
+
 # Paths and docker compose helpers (avoid cd by using -f)
 COMPOSE_DIR := $(abspath resources/docker)
 COMPOSE_FILE := $(COMPOSE_DIR)/compose.yml
@@ -107,8 +124,8 @@ clean-build: ## remove build artifacts
 	rm -fr build/
 	rm -fr dist/
 	rm -fr .eggs/
-	find . -name '*.egg-info' -exec rm -fr {} +
-	find . -name '*.egg' -exec rm -f {} +
+	find . -path ./.venv -prune -o -name '*.egg-info' -exec rm -fr {} +
+	find . -path ./.venv -prune -o -name '*.egg' -exec rm -rf {} +
 
 clean-pyc: ## remove Python file artifacts
 	find . -name '*.pyc' -exec rm -f {} +
@@ -141,21 +158,21 @@ clean-docker: clean-act ## comprehensive Docker cleanup: act + containers + volu
 	@echo "Comprehensive Docker cleanup complete"
 
 install: clean ## install the package to the active Python's site-packages
-	pip install .
+	$(PIP) install .
 
 dev-install: ## install dev extras for local development
-	pip install -e ".[dev]"
+	$(PIP) install -e ".[dev]"
 
 docs-install: ## install docs extras
-	pip install -e ".[docs]"
+	$(PIP) install -e ".[docs]"
 
 coverage: ensure-certs ## run pytest with coverage and generate report (set coverage-min=NN to enforce; requires infrastructure)
 	@echo "Running coverage analysis (single-user profile)..."
 	@echo "Ensuring single-user infrastructure is ready..."
 	$(MAKE) up NIPYAPI_PROFILE=single-user
 	$(MAKE) wait-ready NIPYAPI_PROFILE=single-user
 	@echo "Running pytest with coverage..."
-	NIPYAPI_PROFILE=single-user PYTHONPATH=$(PWD):$$PYTHONPATH pytest --cov=nipyapi --cov-report=term-missing --cov-report=html
+	NIPYAPI_PROFILE=single-user NIPYAPI_PROFILES_FILE=$(NIPYAPI_PROFILES_FILE) PYTHONPATH=$(PWD):$$PYTHONPATH pytest --cov=nipyapi --cov-report=term-missing --cov-report=html
 	@if [ -n "$(coverage-min)" ]; then coverage report --fail-under=$(coverage-min); fi
 	@echo "Coverage analysis complete. See htmlcov/index.html for detailed report."
 
@@ -181,7 +198,7 @@ flake8: ## run flake8 linter on core nipyapi files
 pylint: ## run pylint on core nipyapi files
 	pylint nipyapi/ --rcfile=pylintrc --ignore=nifi,registry,_version.py
 
-pre-commit: ## run pre-commit hooks on all files
+pre-commit: ## run pre-commit hooks on all files (black, isort, flake8, pylint)
 	pre-commit run --all-files
 
 #################################################################################
@@ -244,7 +261,7 @@ down: ## bring down all docker services
 wait-ready: ## wait for readiness using profile configuration; requires NIPYAPI_PROFILE=single-user|secure-ldap|secure-mtls|secure-oidc|github-cicd
 	@if [ -z "$(NIPYAPI_PROFILE)" ]; then echo "ERROR: NIPYAPI_PROFILE is required"; exit 1; fi
 	@echo "Waiting for $(NIPYAPI_PROFILE) infrastructure to be ready..."
-	@NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) $(PYTHON) resources/scripts/wait_ready.py
+	@NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) NIPYAPI_PROFILES_FILE=$(NIPYAPI_PROFILES_FILE) $(PYTHON) resources/scripts/wait_ready.py
 
 # API & Client generation
 fetch-openapi-base: ## refresh base OpenAPI specs for current NIFI_VERSION (always overwrite base)
@@ -272,9 +289,9 @@ gen-clients: ## generate NiFi and Registry clients from specs (use wv_spec_varia
 
 # Individual testing
 
-test: ## run pytest with provided NIPYAPI_PROFILE; config resolved by tests/conftest.py
+test: ## run pytest with provided NIPYAPI_PROFILE; use PYTEST_ARGS for extra options (e.g., make test NIPYAPI_PROFILE=single-user PYTEST_ARGS="-k verify -v")
 	@if [ -z "$(NIPYAPI_PROFILE)" ]; then echo "NIPYAPI_PROFILE is required (single-user|secure-ldap|secure-mtls|secure-oidc|github-cicd)"; exit 1; fi; \
-	NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) PYTHONPATH=$(PWD):$$PYTHONPATH pytest -q
+	NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) NIPYAPI_PROFILES_FILE=$(NIPYAPI_PROFILES_FILE) PYTHONPATH=$(PWD):$$PYTHONPATH pytest -q $(PYTEST_ARGS)
 
 test-su: ## shortcut: NIPYAPI_PROFILE=single-user pytest
 	NIPYAPI_PROFILE=single-user $(MAKE) test
@@ -288,10 +305,11 @@ test-mtls: ## shortcut: NIPYAPI_PROFILE=secure-mtls pytest
 test-oidc: check-certs ## shortcut: NIPYAPI_PROFILE=secure-oidc pytest (requires: make sandbox NIPYAPI_PROFILE=secure-oidc)
 	NIPYAPI_PROFILE=secure-oidc $(MAKE) test
 
-test-specific: ## run specific pytest with provided NIPYAPI_PROFILE and TEST_ARGS
-	@if [ -z "$(NIPYAPI_PROFILE)" ]; then echo "NIPYAPI_PROFILE is required (single-user|secure-ldap|secure-mtls|secure-oidc|github-cicd)"; exit 1; fi; \
-	if [ -z "$(TEST_ARGS)" ]; then echo "TEST_ARGS is required (e.g., tests/test_utils.py::test_dump -v)"; exit 1; fi; \
-	NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) PYTHONPATH=$(PWD):$$PYTHONPATH pytest -q $(TEST_ARGS)
+test-specific: ## DEPRECATED: use 'make test PYTEST_ARGS="..."' instead
+	@echo "DEPRECATED: Use 'make test NIPYAPI_PROFILE=... PYTEST_ARGS=\"...\"' instead"
+	@if [ -z "$(NIPYAPI_PROFILE)" ]; then echo "NIPYAPI_PROFILE is required"; exit 1; fi; \
+	if [ -z "$(TEST_ARGS)" ]; then echo "TEST_ARGS is required"; exit 1; fi; \
+	NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) NIPYAPI_PROFILES_FILE=$(NIPYAPI_PROFILES_FILE) PYTHONPATH=$(PWD):$$PYTHONPATH pytest -q $(TEST_ARGS)
 
 
 # Build & Documentation
@@ -345,7 +363,7 @@ sandbox: ensure-certs ## create isolated environment with sample objects: make s
 	@echo "=== 2/4: Waiting for readiness ==="
 	$(MAKE) wait-ready NIPYAPI_PROFILE=$(NIPYAPI_PROFILE)
 	@echo "=== 3/4: Setting up authentication and sample objects ==="
-	@NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) $(PYTHON) examples/sandbox.py $(NIPYAPI_PROFILE)
+	@NIPYAPI_PROFILE=$(NIPYAPI_PROFILE) NIPYAPI_PROFILES_FILE=$(NIPYAPI_PROFILES_FILE) $(PYTHON) examples/sandbox.py $(NIPYAPI_PROFILE)
 
 rebuild-all: ## comprehensive rebuild: clean -> certs -> extract APIs -> gen clients -> test all -> build -> validate -> docs
 	@echo "Starting comprehensive rebuild from clean slate..."