Merge pull request #17 from MapColonies/MAPCO-9803/implement-cleaner-worker-skeleton

feat: implement cleaner worker skeleton with error handling, validation, and strategy pattern (MAPCO-9803)

Author: almog8k

.github/copilot-instructions.md

@@ -4,7 +4,7 @@
 
 **Cleaner** is a TypeScript-based worker service built on the MapColonies Jobnik SDK framework. It is a distributed task processing worker that handles cleanup operations as part of the raster processing pipeline.
 
-This project is based on the `jobnik-worker-boilerplate` template and follows the MapColonies ecosystem conventions for observability, configuration, and deployment.
+This project is based on the `cleaner` template and follows the MapColonies ecosystem conventions for observability, configuration, and deployment.
 
 ## Quick Reference
 
@@ -55,15 +55,22 @@ npm run lint:fix      # Auto-fix linting issues
 
 ## Important Notes for AI Agents
 
-1. **Remove Demo Code**: The `logistics/` folder and `seeder.ts` are examples. Replace with actual cleaner implementation.
+1. **Incremental Commits & PRs**: Always implement changes in small commits grouped into reviewable PRs. After implementing each part:
+   - Stage files with `git add`
+   - Stop and wait for user confirmation before committing
+   - Only commit after explicit user approval
+   - Use conventional commits: `<type>(<scope>): <description>`
+   - When enough commits are accumulated (3-5 related commits), STOP and tell user to create PR before continuing
 
-2. **Package Metadata**: Update `package.json` name from `jobnik-worker-boilerplate` to `cleaner`.
+2. **Remove Demo Code**: The `logistics/` folder and `seeder.ts` are examples. Replace with actual cleaner implementation.
 
-3. **Type Safety**: Define job/stage types in `src/cleaner/types.ts` and update `worker.ts` with correct generic parameters.
+3. **Package Metadata**: Update `package.json` name from `cleaner` to `cleaner`.
 
-4. **Path Aliases**: Use `@common/` for imports from `src/common/`.
+4. **Type Safety**: Define job/stage types in `src/cleaner/types.ts` and update `worker.ts` with correct generic parameters.
 
-5. **Endpoints**: `/metrics` (Prometheus), `/liveness` (health check).
+5. **Path Aliases**: Use `@common/` for imports from `src/common/`.
+
+6. **Endpoints**: `/metrics` (Prometheus), `/liveness` (health check).
 
 ## Common Tasks
 

.gitignore

@@ -77,6 +77,13 @@ typings/
 .env.test
 config.json
 
+# Local configuration overrides
+config/local.json
+helm/local.yaml
+
+# Local documentation (can be regenerated)
+ai-docs/implementation-plan.md
+
 # parcel-bundler cache (https://parceljs.org/)
 .cache
 

AGENTS.md

@@ -4,7 +4,7 @@
 
 **Cleaner** is a TypeScript-based worker service built on the MapColonies Jobnik SDK framework. It is a distributed task processing worker that handles cleanup operations as part of the raster processing pipeline.
 
-This project is based on the `jobnik-worker-boilerplate` template and follows the MapColonies ecosystem conventions for observability, configuration, and deployment.
+This project is based on the `cleaner` template and follows the MapColonies ecosystem conventions for observability, configuration, and deployment.
 
 ## Quick Reference
 
@@ -55,15 +55,22 @@ npm run lint:fix      # Auto-fix linting issues
 
 ## Important Notes for AI Agents
 
-1. **Remove Demo Code**: The `logistics/` folder and `seeder.ts` are examples. Replace with actual cleaner implementation.
+1. **Incremental Commits & PRs**: Always implement changes in small commits grouped into reviewable PRs. After implementing each part:
+   - Stage files with `git add`
+   - Stop and wait for user confirmation before committing
+   - Only commit after explicit user approval
+   - Use conventional commits: `<type>(<scope>): <description>`
+   - When enough commits are accumulated (3-5 related commits), STOP and tell user to create PR before continuing
 
-2. **Package Metadata**: Update `package.json` name from `jobnik-worker-boilerplate` to `cleaner`.
+2. **Remove Demo Code**: The `logistics/` folder and `seeder.ts` are examples. Replace with actual cleaner implementation.
 
-3. **Type Safety**: Define job/stage types in `src/cleaner/types.ts` and update `worker.ts` with correct generic parameters.
+3. **Package Metadata**: Update `package.json` name from `cleaner` to `cleaner`.
 
-4. **Path Aliases**: Use `@common/` for imports from `src/common/`.
+4. **Type Safety**: Define job/stage types in `src/cleaner/types.ts` and update `worker.ts` with correct generic parameters.
 
-5. **Endpoints**: `/metrics` (Prometheus), `/liveness` (health check).
+5. **Path Aliases**: Use `@common/` for imports from `src/common/`.
+
+6. **Endpoints**: `/metrics` (Prometheus), `/liveness` (health check).
 
 ## Common Tasks
 

README.md

@@ -156,7 +156,7 @@ export const workerBuilder: FactoryFunction<IWorker> = (container: DependencyCon
 
 ### 5. Rename Throughout the Project
 
-Update references to `jobnik-worker-boilerplate` in:
+Update references to `cleaner` in:
 - `package.json` - name, description, author
 - `helm/Chart.yaml` - name, description
 - `helm/values.yaml` - mclabels, configManagement.name, image.repository

ai-docs/common-patterns.md

@@ -54,17 +54,19 @@ In `common/constants.ts`:
 
 ```typescript
 export const SERVICES = {
-  CONFIG: Symbol.for('Config'),
-  LOGGER: Symbol.for('Logger'),
-  TRACER: Symbol.for('Tracer'),
-  METRICS: Symbol.for('Metrics'),
-  JOBNIK_SDK: Symbol.for('JobnikSDK'),
-  WORKER: Symbol.for('Worker'),
+  CONFIG: Symbol('Config'),
+  LOGGER: Symbol('Logger'),
+  TRACER: Symbol('Tracer'),
+  METRICS: Symbol('Metrics'),
+  JOBNIK_SDK: Symbol('JobnikSDK'),
+  WORKER: Symbol('Worker'),
   // Add new tokens here
-  CLEANER_SERVICE: Symbol.for('CleanerService'),
-} as const;
+  CLEANER_SERVICE: Symbol('CleanerService'),
+} satisfies Record<string, symbol>;
 ```
 
+**Note**: Strategy classes are registered using their task type string directly (e.g., `'tiles-deletion'`) rather than Symbol tokens. This eliminates the need for an additional mapping layer.
+
 ## Task Handling
 
 ### Task Handler Structure
@@ -170,13 +172,81 @@ export class MyService {
 
 ```
 config/
-  default.json      # Base config (always loaded)
-  development.json  # Merged when NODE_ENV=development
-  production.json   # Merged when NODE_ENV=production
-  test.json         # Merged when NODE_ENV=test
-  local.json        # Local overrides (gitignored)
+  default.json                   # Base config (always loaded)
+  development.json               # Merged when NODE_ENV=development
+  production.json                # Merged when NODE_ENV=production
+  test.json                      # Merged when NODE_ENV=test
+  local.json                     # Local overrides (gitignored)
+  custom-environment-variables.json  # Environment variable mappings
+helm/
+  values.yaml                    # Helm chart default values
+  local.yaml                     # Local Helm overrides (gitignored)
+  templates/configmap.yaml       # ConfigMap template
 ```
 
+### Adding New Configuration Values
+
+**IMPORTANT**: When adding new configuration values, you MUST update ALL configuration files to maintain sync across all deployment levels:
+
+1. **`config/default.json`** - Add the configuration with default values
+2. **`config/custom-environment-variables.json`** - Add environment variable mappings
+3. **`helm/values.yaml`** - Add Helm chart values
+4. **`helm/values-local.yaml`** - Add local development values (if needed)
+5. **`helm/templates/configmap.yaml`** - Add to ConfigMap template
+
+**Example:**
+
+Adding a new `queue.jobManagerBaseUrl` configuration:
+
+**1. config/default.json**
+
+```json
+{
+  "queue": {
+    "jobManagerBaseUrl": "http//localhost:8080"
+  }
+}
+```
+
+**2. config/custom-environment-variables.json**
+
+```json
+{
+  "queue": {
+    "jobManagerBaseUrl": "QUEUE_JOB_MANAGER_BASE_URL"
+  }
+}
+```
+
+**3. helm/values.yaml**
+
+```yaml
+env:
+  queue:
+    jobManagerBaseUrl: 'http//localhost:8080'
+```
+
+**4. helm/local.yaml** (for local development)
+
+```yaml
+env:
+  queue:
+    jobManagerBaseUrl: 'http://localhost:8080'
+```
+
+**5. helm/templates/configmap.yaml**
+
+```yaml
+data:
+  QUEUE_JOB_MANAGER_BASE_URL: { { .Values.env.queue.jobManagerBaseUrl | quote } }
+```
+
+This ensures configuration works correctly in:
+
+- Local development (default.json, local.json)
+- CI/CD environments (environment variables)
+- Kubernetes deployments (Helm charts)
+
 ## Error Handling
 
 ### In Task Handlers