Merge pull request #121 from hua7450/eval-2-skill-improvements

Eval-2: eligibility completeness and childcare subsidy patterns

Author: hua7450

agents/country-models/rules-engineer.md

@@ -43,6 +43,14 @@ Creates parameter YAML files and variable Python files for government benefit pr
 2. Read the scope decision (if provided)
 3. Search for 3+ similar parameter files AND 3+ variable files from reference implementations
 4. Learn their folder structure, naming, description patterns, and code patterns
+5. **List every eligibility variable** in the reference implementation's eligibility folder. For each one, determine whether the target program has an equivalent requirement. Common eligibility types to check:
+   - Income eligibility
+   - Asset/resource eligibility
+   - Activity/work requirement eligibility
+   - Immigration/citizenship eligibility
+   - Demographic eligibility (age, household composition)
+
+   If the reference has an eligibility type that's not in the spec, check the regulation — the spec may be incomplete.
 
 ### Step 2: Create Parameters
 
@@ -104,16 +112,25 @@ Create Python variable files following `policyengine-variable-patterns` and `pol
   # But ALWAYS verify with the state's legal code — follow the law, not the pattern.
   ```
 
-### Step 4: Parameter-to-Variable Mapping (CRITICAL)
+### Step 4: Spec-to-Implementation Completeness Check (CRITICAL)
+
+After creating both parameters and variables, perform TWO verification passes:
+
+**Pass 1: Spec coverage** — Go through EVERY requirement in the spec/working_references, line by line:
+- [ ] Each requirement has at least one parameter AND one variable implementing it
+- [ ] Requirements listed as bullet points or in "other requirements" sections are NOT informational — they need implementation too
+- [ ] If the spec mentions employment/work hours → create `{prefix}_activity_eligible` or `{prefix}_work_eligible`
+- [ ] If the spec mentions citizenship/immigration → create `{prefix}_immigration_eligible` or use existing federal variable
+- [ ] If the spec mentions assets/resources → create `{prefix}_resource_eligible`
 
-After creating both parameters and variables, verify completeness:
+**Pass 2: Parameter-to-variable mapping** — List every parameter file you created:
 - [ ] Every parameter has at least one variable using it
 - [ ] All eligibility parameters have corresponding `_eligible` variables
 - [ ] All calculation parameters have corresponding calculation variables
 - [ ] Main eligibility variable combines ALL eligibility checks
 - [ ] No parameters are orphaned (created but never used)
 
-**RED FLAG:** If you created a `resources/limit` parameter but no `resource_eligible` variable!
+**RED FLAG:** If you created a parameter but no variable uses it — e.g., `min_work_hours.yaml` exists but no `work_eligible` variable!
 
 ### Step 5: Simplified vs Full TANF
 

changelog.d/eval-2-improvements.changed.md

@@ -0,0 +1 @@
+Eval-2 driven improvements to rules-engineer agent, code-organization skill, and variable-patterns skill

skills/technical-patterns/policyengine-code-organization-skill/SKILL.md

@@ -69,6 +69,7 @@ Use these standard suffixes with your variable prefix:
 {prefix}_eligible              # Final eligibility (combines all checks)
 {prefix}_income_eligible       # Income eligibility
 {prefix}_resource_eligible     # Resource/asset eligibility
+{prefix}_activity_eligible     # Work/activity requirement eligibility
 {prefix}_categorical_eligible  # Categorical eligibility
 {prefix}_demographic_eligible  # Age, household composition eligibility
 {prefix}_immigration_eligible  # Immigration status eligibility
@@ -196,6 +197,41 @@ policyengine_us/tests/policy/baseline/gov/states/{state}/{agency}/{program}/
 
 ---
 
+## Parameter Folder Organization
+
+Parameter folders follow similar principles to variable folders — group by function.
+
+### Keep folders focused on their purpose
+
+Each subfolder should contain parameters that serve the same function. Don't use a folder as a catch-all.
+
+```
+# ❌ BAD — eligibility/ contains rate dimension boundaries that aren't eligibility criteria
+ccap/
+├── eligibility/
+│   ├── income_limit.yaml           # ✅ Eligibility
+│   ├── child_age_threshold.yaml    # ✅ Eligibility
+│   ├── center_infant_max_months.yaml  # ❌ Rate table age group boundary
+│   ├── preschool_max_years.yaml       # ❌ Rate table age group boundary
+│   └── time_category.yaml            # ❌ Authorization concept
+└── rates/
+
+# ✅ GOOD — each folder contains parameters that match its purpose
+ccap/
+├── age_groups/                     # Age group boundaries for rate lookups
+│   ├── center_infant_max_months.yaml
+│   └── preschool_max_years.yaml
+├── eligibility/                    # Actual eligibility criteria
+│   ├── income_limit.yaml
+│   └── child_age_threshold.yaml
+├── rates/
+└── time_category.yaml              # At root — doesn't fit neatly in a subfolder
+```
+
+**The key question**: "Is this parameter an eligibility rule, a rate-table dimension boundary, a benefit calculation input, or something else?" Put it in the folder that matches its function.
+
+---
+
 ## Common Mistakes to Avoid
 
 ### Variable Names

skills/technical-patterns/policyengine-variable-patterns-skill/SKILL.md

@@ -207,6 +207,29 @@ class some_variable(Variable):
     reference = "https://example.gov/rules.pdf#page=10"  # USE THIS
 ```
 
+### Scoping `defined_for` to the Right Level
+
+`defined_for` controls which entities run the formula. Use the most specific scope that fits:
+
+```python
+# ❌ TOO BROAD — calculates rates for all RI residents (adults, ineligible children)
+class ri_ccap_licensed_center_rate(Variable):
+    entity = Person
+    defined_for = StateCode.RI
+
+# ✅ CORRECT — only calculates rates for children eligible for CCAP
+class ri_ccap_licensed_center_rate(Variable):
+    entity = Person
+    defined_for = "ri_ccap_eligible_child"
+```
+
+**Guidelines:**
+- **SPMUnit-level benefit variables** → `defined_for = "state_program_eligible"` (the main eligibility variable)
+- **Person-level variables within a program** (rates, per-child amounts) → `defined_for = "state_program_eligible_child"` or the person-level eligibility variable
+- **Eligibility check variables themselves** → `defined_for = StateCode.XX` (they determine eligibility, so they can't depend on it)
+
+This avoids unnecessary computation and makes the variable's scope clear from its definition.
+
 **Reference format:**
 ```python
 # Single reference:
@@ -1127,6 +1150,7 @@ When you create parameters, you MUST create corresponding variables:
 |---------------|---------------------|
 | resources/limit | `state_program_resources_eligible` |
 | income/limit | `state_program_income_eligible` |
+| min_work_hours or activity_requirements | `state_program_activity_eligible` |
 | payment_standard | `state_program_maximum_benefit` |
 | income/disregard | `state_program_countable_earned_income` |
 | categorical/requirements | `state_program_categorically_eligible` |
@@ -1147,8 +1171,41 @@ class state_program_eligible(Variable):
 
 **Common Implementation Failures:**
 - ❌ Created resource limit parameter but no resource_eligible variable
-- ❌ Main eligible variable only checks income, ignores resources
+- ❌ Created min_work_hours parameter but no activity_eligible variable
+- ❌ Main eligible variable only checks income, ignores resources/work/immigration
 - ❌ Parameters created but never referenced in any formula
+- ❌ Spec lists requirements (work hours, citizenship) but no variables implement them
+
+---
+
+## Childcare Subsidy Benefit Calculation
+
+Childcare subsidy programs (CCAP, CCFA, etc.) share a common pattern: the benefit is capped at actual childcare expenses, not just the provider reimbursement rate.
+
+PolicyEngine has an existing federal variable `pre_subsidy_childcare_expenses` (Person, YEAR) that represents what families actually pay for childcare. **Always use this variable** — don't calculate the benefit from provider rates alone.
+
+```python
+# ❌ BAD — subsidy based only on provider rate (ignores actual expenses)
+class ri_ccap(Variable):
+    def formula(spm_unit, period, parameters):
+        total_weekly_rate = spm_unit.sum(weekly_rate * is_eligible_child)
+        return max_(total_weekly_rate - family_share, 0)
+
+# ✅ GOOD — subsidy capped at actual expenses (matches MA CCFA, CO CCAP pattern)
+class ri_ccap(Variable):
+    def formula(spm_unit, period, parameters):
+        actual_expenses = spm_unit(
+            "spm_unit_pre_subsidy_childcare_expenses", period.this_year
+        )
+        copay = spm_unit("ri_ccap_family_share", period)
+        max_reimbursement = add(spm_unit, period, ["ri_ccap_weekly_provider_rate"])
+        uncapped_benefit = max_(actual_expenses - copay, 0)
+        return min_(uncapped_benefit, max_reimbursement)
+```
+
+The subsidy is the lesser of:
+1. Actual expenses minus co-payment
+2. Maximum reimbursement (provider rate based on type, quality, age, time)
 
 ---
 

skills/technical-patterns/policyengine-vectorization-skill/SKILL.md

@@ -70,6 +70,37 @@ else:
 )
 ```
 
+### Pattern 2b: Enum Dispatch → List ALL Conditions
+
+When dispatching on an Enum variable, list all values explicitly in `select()`. Set the `default` to match the Enum's `default_value` so unexpected values get consistent behavior.
+
+```python
+# ❌ BAD — hides one option in default, unclear which is the fallback
+provider_type = person("ri_ccap_provider_type", period)
+types = provider_type.possible_values
+return select(
+    [
+        provider_type == types.LICENSED_CENTER,
+        provider_type == types.LICENSED_FAMILY,
+    ],
+    [center_rate, family_rate],
+    default=exempt_rate,  # Why exempt as fallback?
+)
+
+# ✅ GOOD — all options listed, default matches Enum default_value
+provider_type = person("ri_ccap_provider_type", period)
+types = provider_type.possible_values
+return select(
+    [
+        provider_type == types.LICENSED_CENTER,
+        provider_type == types.LICENSED_FAMILY,
+        provider_type == types.LICENSE_EXEMPT,
+    ],
+    [center_rate, family_rate, exempt_rate],
+    default=center_rate,  # Matches default_value = LICENSED_CENTER
+)
+```
+
 ### Pattern 3: Boolean Operations
 
 ```python