Skip to content

Commit 595f1c4

Browse files
docktermjdocktermj
committed
Merge main
2 parents 710d71b + b559342 commit 595f1c4

35 files changed

+687
-209
lines changed

‎.claude/CLAUDE.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
This repository contains a Senzing Governor plugin that monitors PostgreSQL transaction ID (XID) age. When XID age exceeds a high watermark, it pauses processing threads until the age drops below a low watermark (achieved via PostgreSQL VACUUM).
8+
9+
## Build and Development Commands
10+
11+
This project uses `pyproject.toml` with setuptools and requires Python >= 3.10.
12+
13+
```bash
14+
# Install dependencies (using uv, pip, or similar)
15+
uv sync --group all # All dependency groups
16+
uv sync --group development # Development dependencies only
17+
uv sync --group lint # Linting tools only
18+
uv sync --group test # Test dependencies only
19+
20+
# Run unit tests
21+
python -m pytest src/senzing_governor_unit_test.py
22+
python -m unittest src/senzing_governor_unit_test.py
23+
24+
# Run linters (configured in pyproject.toml)
25+
black --check src/
26+
isort --check src/
27+
flake8 src/
28+
pylint src/
29+
mypy src/
30+
bandit -r src/
31+
32+
# Build package
33+
python -m build
34+
```
35+
36+
## Code Architecture
37+
38+
The entire implementation is in `src/senzing_governor.py`:
39+
40+
- **`Governor` class**: The main class implementing the governor pattern
41+
- `__init__()`: Configures watermarks, intervals, and database connections from environment variables or parameters
42+
- `govern()`: Called by worker threads; checks XID age periodically and returns wait time (uses thread locking)
43+
- `close()`: Closes database connections
44+
- Supports Python context manager protocol (`with` statement)
45+
46+
- **Watermark behavior**: Uses a stepped wait-time function based on ratio between current XID and watermarks. When above high watermark, returns -1.0 to signal system trouble.
47+
48+
- **Database URL parsing**: Custom parsing handles complex PostgreSQL URLs with special characters; extracts from `SENZING_GOVERNOR_DATABASE_URLS`, `SENZING_DATABASE_URL`, or `SENZING_ENGINE_CONFIGURATION_JSON`.
49+
50+
## Key Environment Variables
51+
52+
- `SENZING_GOVERNOR_DATABASE_URLS`: PostgreSQL connection URLs (comma-separated for multiple DBs)
53+
- `SENZING_GOVERNOR_POSTGRESQL_HIGH_WATERMARK`: Default 1,500,000,000
54+
- `SENZING_GOVERNOR_POSTGRESQL_LOW_WATERMARK`: Default 1,200,000,000
55+
- `SENZING_GOVERNOR_INTERVAL`: Check interval (default 100,000 records)
56+
- `SENZING_GOVERNOR_LIST_SEPARATOR`: URL list separator (default comma)
57+
58+
## Testing
59+
60+
- `src/senzing_governor_unit_test.py`: Unit tests for `get_wait_time()` method
61+
- `src/senzing_governor_tester.py`: Integration test with multi-threaded example
62+
- `src/senzing_governor_tester_context_manager.py`: Context manager usage example
63+
64+
## Code Style
65+
66+
- Line length: 120 characters (black, flake8)
67+
- Import sorting: isort with black profile
68+
- Type checking: mypy with several error codes disabled (see pyproject.toml)

‎.claude/commands/senzing-code-review.md‎

Lines changed: 0 additions & 3 deletions
This file was deleted.

‎.claude/commands/senzing.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Senzing
2+
3+
- Perform the steps specified by <https://raw.githubusercontent.com/senzing-factory/claude/refs/tags/v1/commands/senzing.md>

.claude/settings.local.json renamed to .claude/settings.json

File renamed without changes.

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Default code owner
1+
# Default code owner
22

33
* @Senzing/senzing-community
44

‎.github/dependabot.yml‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,15 @@
33

44
version: 2
55
updates:
6-
- package-ecosystem: "github-actions"
7-
directory: "/"
6+
- package-ecosystem: github-actions
7+
cooldown:
8+
default-days: 21
9+
directory: /
810
schedule:
9-
interval: "daily"
10-
- package-ecosystem: "pip"
11-
directory: "/"
11+
interval: daily
12+
- package-ecosystem: pip
13+
cooldown:
14+
default-days: 21
15+
directory: /
1216
schedule:
13-
interval: "daily"
17+
interval: daily

‎.github/linters/.jscpd.json‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
11
{
2-
"ignore": [
3-
"**/src/**"
4-
],
2+
"ignore": ["**/src/**"],
53
"threshold": 15
6-
}
4+
}

‎.github/linters/README.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Linters
2+
3+
## .checkov.yaml
4+
5+
- [.checkov.yaml]
6+
- Used by [lint-workflows.yaml]
7+
- [checkov]
8+
- [checkov configuration]
9+
10+
## .jscpd.json
11+
12+
- [.jscpd.json]
13+
- Used by [lint-workflows.yaml]
14+
- [jscpd]
15+
- [jscpd configuration]
16+
- Example:
17+
18+
```json
19+
{
20+
"ignore": ["**/*.py,**/python-test*.yaml"],
21+
"threshold": 10
22+
}
23+
```
24+
25+
## .yaml-lint.yml
26+
27+
- [.yaml-lint.yml]
28+
- Used by [lint-workflows.yaml]
29+
- [yaml-lint]
30+
- [yaml-lint configuration]
31+
32+
## bearer.yml
33+
34+
- [bearer.yml]
35+
- Used by [bearer.yaml]
36+
- [bearer]
37+
- [bearer repository]
38+
- [bearer configuration]
39+
40+
## zizmor.yaml
41+
42+
- [zizmor.yaml]
43+
44+
[.checkov.yaml]: .checkov.yaml
45+
[.jscpd.json]: .jscpd.json
46+
[.yaml-lint.yml]: .yaml-lint.yml
47+
[bearer configuration]: https://docs.bearer.com/reference/config/
48+
[bearer repository]: https://github.com/Bearer/bearer/tree/main
49+
[bearer.yaml]: ../workflows/README.md#beareryaml
50+
[bearer.yml]: bearer.yml
51+
[bearer]: https://docs.bearer.com/
52+
[checkov configuration]: https://www.checkov.io/2.Basics/CLI%20Command%20Reference.html
53+
[checkov]: https://www.checkov.io/
54+
[jscpd configuration]: https://github.com/kucherenko/jscpd/tree/master/apps/jscpd#options
55+
[jscpd]: https://github.com/kucherenko/jscpd
56+
[lint-workflows.yaml]: ../workflows/README.md#lint-workflowsyaml
57+
[yaml-lint configuration]: https://yamllint.readthedocs.io/en/stable/configuration.html
58+
[yaml-lint]: https://github.com/adrienverge/yamllint
59+
[zizmor.yaml]: zizmor.yaml

‎.github/linters/bearer.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
rule:
2+
skip-rule: []

‎.github/workflows/README.md‎

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# Workflows
2+
3+
## add-labels-standardized.yaml
4+
5+
When issues are opened,
6+
this action adds appropriate labels to the issue.
7+
(e.g. "triage", "customer-submission")
8+
9+
- [Add Labels Standardized GitHub Action]
10+
- Uses: [senzing-factory/build-resources/.../add-labels-to-issue.yaml]
11+
12+
## add-to-project-garage-dependabot.yaml
13+
14+
When a Dependabot Pull Request (PR) is made against `main` branch,
15+
this action adds the PR to the "Garage" project board as "In Progress".
16+
17+
- [Add to Project Garage Dependabot GitHub Action]
18+
- Uses: [senzing-factory/build-resources/.../add-to-project-dependabot.yaml]
19+
20+
## add-to-project-garage.yaml
21+
22+
When an issue is created,
23+
this action adds the issue to the "Garage" board as "Backlog".
24+
25+
- [Add to Project Garage GitHub Action]
26+
- Uses: [senzing-factory/build-resources/.../add-to-project.yaml]
27+
28+
## bandit.yaml
29+
30+
When a Pull Request (PR) is made against `main` branch,
31+
this action runs [Bandit] to detect security issues.
32+
33+
- [bandit.yaml]
34+
- Uses: [lukehinds/bandit-action]
35+
36+
## black.yaml
37+
38+
When a change is committed to GitHub or a Pull Request is made against the `main` branch,
39+
this action runs the [Black] code formatter.
40+
41+
- [black.yaml]
42+
43+
## dependabot-approve-and-merge.yaml
44+
45+
When a Dependabot Pull Request (PR) is made against the `main` branch,
46+
this action determines if it should be automatically approved and merged into the `main` branch.
47+
Once this action occurs [move-pr-to-done-dependabot.yaml] moves the PR on the "Garage" project board to "Done".
48+
49+
- [Dependabot Approve and Merge GitHub Action]
50+
- Uses: [senzing-factory/build-resources/.../dependabot-approve-and-merge.yaml]
51+
52+
## dependency-scan.yaml
53+
54+
When a Pull Request is made against the `main` branch,
55+
this action runs [Fast Python Vulnerability Scanner] and [pip-audit].
56+
57+
- [dependency-scan.yaml]
58+
- Uses:
59+
- [Fast Python Vulnerability Scanner]
60+
- [pypa/gh-action-pip-audit]
61+
62+
## docker-build-container.yaml
63+
64+
When a Pull Request is made against the `main` branch,
65+
this action verifies that the `Dockerfile` can be successfully built.
66+
67+
_Note:_ The Docker image is **not** pushed to [DockerHub].
68+
69+
- [Docker Build Container GitHub Action]
70+
- Uses: [senzing-factory/github-action-docker-buildx-build]
71+
72+
## docker-push-containers-to-dockerhub.yaml
73+
74+
After a [Semantic Version] release is created,
75+
this action builds Docker images on multiple architectures and pushes the Docker images to [DockerHub].
76+
77+
- [Docker Push Containers to DockerHub GitHub Action]
78+
- Uses: [senzing-factory/github-action-docker-buildx-build]
79+
80+
## flake8.yaml
81+
82+
When a change is committed to GitHub or a Pull Request is made against the `main` branch,
83+
this action runs [flake8] for Python style enforcement.
84+
85+
- [flake8.yaml]
86+
- Uses: [py-actions/flake8]
87+
88+
## isort.yaml
89+
90+
When a change is committed to GitHub or a Pull Request is made against the `main` branch,
91+
this action runs [isort] to sort the Python import statements
92+
93+
- [isort.yaml]
94+
- Uses: [isort/isort-action]
95+
96+
## lint-workflows.yaml
97+
98+
When a change is committed to GitHub or a Pull Request is made against the `main` branch,
99+
this action runs [super-linter] to run multiple linters against the code.
100+
101+
- [Lint Workflows GitHub Action]
102+
- Configuration:
103+
- [.checkov.yaml]
104+
- [.jscpd.json]
105+
- [.yaml-lint.yml]
106+
- Uses: [senzing-factory/build-resources/.../lint-workflows.yaml]
107+
108+
## move-pr-to-done-dependabot.yaml
109+
110+
When a Pull Request is merged into the `main` branch,
111+
this action moves the PR on the "Garage" project board to "Done".
112+
113+
- [Move PR to Done Dependabot GitHub Action]
114+
- Uses: [senzing-factory/build-resources/.../move-pr-to-done-dependabot.yaml]
115+
116+
## mypy.yaml
117+
118+
When a change is committed to GitHub or a Pull Request is made against the `main` branch,
119+
this action runs [mypy] to perform static type checking.
120+
121+
- [mypy.yaml]
122+
123+
## pylint.yaml
124+
125+
When a change is committed to GitHub,
126+
this action runs [pylint] to perform static code analysis.
127+
128+
- [pylint.yaml]
129+
130+
## pytest-darwin.yaml
131+
132+
When a Pull Request is merged into the `main` branch,
133+
this action runs [pytest] on the Darwin/macOS platform to perform unit tests and code coverage.
134+
135+
- [pytest-darwin.yaml]
136+
- Uses:
137+
- [actions/checkout]
138+
- [actions/setup-python]
139+
- [senzing-factory/github-action-install-senzing-sdk]
140+
- [pytest]
141+
- [actions/upload-artifact]
142+
143+
## pytest-linux.yaml
144+
145+
When a change is committed to GitHub or a Pull Request is made against the `main` branch,
146+
this action runs [pytest] on the Linux platform to perform unit tests and code coverage.
147+
148+
- [pytest-linux.yaml]
149+
- Uses:
150+
- [actions/checkout]
151+
- [actions/setup-python]
152+
- [senzing-factory/github-action-install-senzing-sdk]
153+
- [pytest]
154+
- [actions/upload-artifact]
155+
156+
## pytest-windows.yaml
157+
158+
When a Pull Request is merged into the `main` branch,
159+
this action runs [pytest] on the Windows platform to perform unit tests and code coverage.
160+
161+
- [pytest-windows.yaml]
162+
- Uses:
163+
- [actions/checkout]
164+
- [actions/setup-python]
165+
- [senzing-factory/github-action-install-senzing-sdk]
166+
- [pytest]
167+
- [actions/upload-artifact]
168+
169+
[.checkov.yaml]: ../linters/README.md#checkovyaml
170+
[.jscpd.json]: ../linters/README.md#jscpdjson
171+
[.yaml-lint.yml]: ../linters/README.md#yaml-lintyml
172+
[actions/checkout]: https://github.com/actions/checkout
173+
[actions/setup-python]: https://github.com/actions/setup-python
174+
[actions/upload-artifact]: https://github.com/actions/upload-artifact
175+
[Add Labels Standardized GitHub Action]: add-labels-standardized.yaml
176+
[Add to Project Garage Dependabot GitHub Action]: add-to-project-garage-dependabot.yaml
177+
[Add to Project Garage GitHub Action]: add-to-project-garage.yaml
178+
[bandit.yaml]: bandit.yaml
179+
[Bandit]: https://bandit.readthedocs.io/en/latest/
180+
[black.yaml]: black.yaml
181+
[Black]: https://github.com/psf/black
182+
[Dependabot Approve and Merge GitHub Action]: dependabot-approve-and-merge.yaml
183+
[dependency-scan.yaml]: dependency-scan.yaml
184+
[Docker Build Container GitHub Action]: docker-build-container.yaml
185+
[Docker Push Containers to DockerHub GitHub Action]: docker-push-containers-to-dockerhub.yaml
186+
[DockerHub]: https://hub.docker.com/
187+
[Fast Python Vulnerability Scanner]: https://github.com/vanschelven/fpvs/
188+
[flake8.yaml]: flake8.yaml
189+
[flake8]: https://flake8.pycqa.org/en/latest/
190+
[isort.yaml]: isort.yaml
191+
[isort]: https://pycqa.github.io/isort/
192+
[isort/isort-action]: https://github.com/isort/isort-action
193+
[Lint Workflows GitHub Action]: lint-workflows.yaml
194+
[lukehinds/bandit-action]: https://github.com/lukehinds/bandit-action
195+
[Move PR to Done Dependabot GitHub Action]: move-pr-to-done-dependabot.yaml
196+
[move-pr-to-done-dependabot.yaml]: move-pr-to-done-dependabotyaml
197+
[mypy.yaml]: mypy.yaml
198+
[mypy]: https://mypy-lang.org/
199+
[pip-audit]: https://github.com/pypa/pip-audit
200+
[py-actions/flake8]: https://github.com/py-actions/flake8
201+
[pylint.yaml]: pylint.yaml
202+
[pylint]: https://pypi.org/project/pylint/
203+
[pypa/gh-action-pip-audit]: https://github.com/pypa/gh-action-pip-audit
204+
[pytest-darwin.yaml]: pytest-darwin.yaml
205+
[pytest-linux.yaml]: pytest-linux.yaml
206+
[pytest-windows.yaml]: pytest-windows.yaml
207+
[pytest]: https://docs.pytest.org/en/stable/
208+
[Semantic Version]: https://semver.org/
209+
[senzing-factory/build-resources/.../add-labels-to-issue.yaml]: https://github.com/senzing-factory/build-resources/blob/main/.github/workflows/add-labels-to-issue.yaml
210+
[senzing-factory/build-resources/.../add-to-project-dependabot.yaml]: https://github.com/senzing-factory/build-resources/blob/main/.github/workflows/add-to-project-dependabot.yaml
211+
[senzing-factory/build-resources/.../add-to-project.yaml]: https://github.com/senzing-factory/build-resources/blob/main/.github/workflows/add-to-project.yaml
212+
[senzing-factory/build-resources/.../dependabot-approve-and-merge.yaml]: https://github.com/senzing-factory/build-resources/blob/main/.github/workflows/dependabot-approve-and-merge.yaml
213+
[senzing-factory/build-resources/.../lint-workflows.yaml]: https://github.com/senzing-factory/build-resources/blob/main/.github/workflows/lint-workflows.yaml
214+
[senzing-factory/build-resources/.../move-pr-to-done-dependabot.yaml]: https://github.com/senzing-factory/build-resources/blob/main/.github/workflows/move-pr-to-done-dependabot.yaml
215+
[senzing-factory/github-action-docker-buildx-build]: https://github.com/senzing-factory/github-action-docker-buildx-build
216+
[senzing-factory/github-action-install-senzing-sdk]: https://github.com/senzing-factory/github-action-install-senzing-sdk
217+
[super-linter]: https://github.com/super-linter/super-linter

0 commit comments

Comments
 (0)