feat: perm ledger expiry for testnet

.claude/commands/codex.md

@@ -1,147 +1,70 @@
 # Codex Second Opinion
 
-Get a second opinion from OpenAI Codex on code reviews, debugging, and analysis.
+Get a second opinion from OpenAI Codex on any question about this codebase — debugging, analysis, architecture, code review, or anything else.
 
 ## Arguments
 
-The user's input is: $ARGUMENTS
+The user's request is: $ARGUMENTS
 
 ## Instructions
 
-Parse the arguments and determine which mode to use, then gather context and execute the appropriate codex command.
+Your job is to gather the right context, build a prompt that gives Codex enough understanding of this codebase to be useful, and then run `codex exec` with that prompt. Codex has zero knowledge of this codebase, so the context you provide is everything.
 
-### Mode Detection
+### Step 1: Gather Context
 
-Analyze `$ARGUMENTS` to determine the mode:
+Read the Architecture Overview section of `CLAUDE.md` for a high-level understanding. Then, based on what the user is asking about, selectively read the files that are most relevant. Use these heuristics:
 
-1. **Branch review** — arguments match `review`, `review branch`, `review --base`, or no arguments at all (default mode). Reviews the current branch against `master`.
-2. **PR review** — arguments match patterns like `review PR #42`, `review pr 42`, `pr 42`, `pr #42`
-3. **Uncommitted changes review** — arguments match `review uncommitted`, `uncommitted`, `wip`
-4. **Commit review** — arguments match `review commit <SHA>`, `commit <SHA>`
-5. **Free-form** — anything else (e.g. `debug why test_foo fails`, `explain the block validation flow`)
+- **If actor services are involved**: read the service's message types and the `ServiceSenders`/`ServiceReceivers` pattern
+- **If types crate is involved**: check wire formats (`BlockBody`, `BlockHeader`, `DataTransactionHeader`)
+- **If consensus/mining is involved**: read the VDF + PoA section and shadow transaction patterns
+- **If p2p is involved**: check gossip protocol routes and circuit breaker usage
+- **If storage/packing is involved**: check chunk size constants and XOR packing invariants
+- **If reth integration is involved**: check CL/EL boundary and payload building flow
 
-### Context Gathering (All Modes)
+If the user's request references specific files, diffs, or branches, read those too. Keep context focused — aim for 3-5 key files maximum.
 
-Before running codex, you must build a targeted review prompt. This is the most important step — codex has no knowledge of this codebase, so you need to give it the right context.
+### Step 2: Build the Prompt
 
-#### Step 1: Identify touched areas
+Construct a single prompt string that includes:
 
-Get the diff for the relevant mode:
+1. **Architecture summary** — 2-3 sentences describing the relevant components and how they fit together
+2. **Key conventions** — patterns that apply (e.g., "custom Tokio channel-based actor system, not Actix", "crypto crates compiled with opt-level=3")
+3. **Relevant code** — inline the key snippets or file contents that Codex needs to see
+4. **The user's request** — what they actually want Codex to analyze
 
-| Mode | Command |
-|---|---|
-| Branch review | `git diff master...HEAD --stat` then `git diff master...HEAD` |
-| PR review | `gh pr diff <N>` |
-| Uncommitted | `git diff HEAD --stat` then `git diff HEAD` (includes staged + unstaged) |
-| Commit | `git show <SHA> --stat` then `git show <SHA>` |
+### Step 3: Run Codex
 
-From the `--stat` output, identify which crates and modules are touched (e.g., `crates/actors/src/block_tree/`, `crates/types/src/`).
-
-#### Step 2: Read relevant context
-
-Based on the touched areas, read the relevant sections of `CLAUDE.md` (specifically the Architecture Overview section). Then selectively read files that provide context for the review:
-
-- **If actor services are touched**: read the service's message types and the `ServiceSenders`/`ServiceReceivers` pattern
-- **If types crate is touched**: check for breaking changes to wire formats (`BlockBody`, `BlockHeader`, `DataTransactionHeader`)
-- **If consensus/mining is touched**: read the VDF + PoA section and shadow transaction patterns
-- **If p2p is touched**: check gossip protocol routes and circuit breaker usage
-- **If storage/packing is touched**: check chunk size constants and XOR packing invariants
-- **If reth integration is touched**: check CL/EL boundary and payload building flow
-
-Keep the context focused — only read files directly relevant to the diff. Aim for 3-5 key files maximum.
-
-#### Step 3: Build the review prompt
-
-Construct a prompt that includes:
-
-1. **Architecture summary** — a 2-3 sentence description of what the touched components do and how they fit together, derived from your reading
-2. **Key conventions** — the specific patterns that apply (e.g., "This codebase uses a custom Tokio channel-based actor system, not Actix" or "Crypto crates must compile with opt-level=3")
-3. **Review focus areas** — what to pay attention to based on the diff:
-   - Unsafe code and memory safety (especially in packing/crypto crates)
-   - Correctness of `Arc`/`Clone` patterns in actor message passing
-   - Wire format backward compatibility (types crate changes)
-   - Concurrency bugs (deadlocks, race conditions in channel-based services)
-   - Error handling (are errors propagated correctly, or silently swallowed?)
-   - Off-by-one errors in chunk/partition/offset calculations
-
-### Execution by Mode
-
-#### Mode 1: Branch Review (Default)
-
-Run codex review with the base flag and your constructed prompt:
 ```bash
-codex review --base master "<constructed review prompt>"
+codex exec --sandbox read-only "<constructed prompt>"
 ```
-Run this in the background with a 300s timeout.
-
-#### Mode 2: PR Review
-
-1. Extract the PR number from the arguments.
-2. Gather PR metadata:
-   ```bash
-   gh pr view <N> --json title,body,labels,baseRefName
-   ```
-3. Get the diff via `gh pr diff <N>`.
-4. Include the PR title and description in the constructed prompt.
-5. Run codex with the combined context:
-   ```bash
-   echo "<constructed prompt including PR context and diff>" | codex exec --sandbox read-only -o /tmp/codex-review-$$.txt -
-   ```
-   Run this in the background with a 300s timeout.
 
-#### Mode 3: Uncommitted Changes Review
-
-```bash
-codex review --uncommitted "<constructed review prompt>"
-```
-Run in background with 300s timeout.
-
-#### Mode 4: Commit Review
-
-Extract the commit SHA from the arguments. Run:
-```bash
-codex review --commit <SHA> "<constructed review prompt>"
-```
-Run in background with 300s timeout.
-
-#### Mode 5: Free-form
-
-Pass the arguments as a prompt to codex exec, prefixed with the architecture context you gathered:
-```bash
-codex exec --sandbox read-only -o /tmp/codex-review-$$.txt "<architecture context>\n\nUser request: $ARGUMENTS"
-```
-Run in background with 300s timeout.
+Run this in the background with a 300s timeout.
 
-### Progress Monitoring
+### Step 4: Monitor Progress
 
 After launching codex in the background:
 
 1. Wait ~30 seconds, then check the background task output using `TaskOutput` with `block: false`.
-2. If there is new output, give the user a brief progress update (e.g. "Codex is analyzing the diff..." or quote a snippet of what it's working on).
+2. If there is new output, give the user a brief progress update.
 3. Repeat every ~30 seconds.
-4. If no new output appears for 60+ seconds and the task hasn't completed, warn the user that codex may be stuck and offer to kill the process.
-5. When the task completes, proceed to output presentation.
+4. If no new output appears for 60+ seconds and the task hasn't completed, warn the user that codex may be stuck and offer to kill it.
 
-### Output Presentation
+### Step 5: Present Results
 
 Once codex finishes:
 
-1. **Summary**: Present a concise summary of key findings organized by category:
+1. **Summary**: Concise summary of key findings, organized by category (only include categories with findings):
    - Bugs and logic errors
    - Security concerns
    - Concurrency / actor system issues
-   - Code quality and style suggestions
+   - Code quality and style
    - Performance considerations
-   Only include categories that have findings.
 
-2. **Raw output**: Include the complete codex output in a fenced code block so the user can read the full analysis.
+2. **Raw output**: Complete codex output in a fenced code block.
 
-3. **Counterpoints**: If you (Claude) disagree with any of codex's findings or think something was missed, add a "Claude's take" section noting your perspective. This is especially valuable when codex lacks the architectural context to understand why something was done a certain way. Only include this if you have a meaningful counterpoint — don't add it just for the sake of it.
+3. **Counterpoints**: If you (Claude) disagree with any findings or think something was missed, add a "Claude's take" section. Only include this if you have a meaningful counterpoint.
 
 ### Error Handling
 
 - If `codex` is not found, tell the user to install it: `npm install -g @openai/codex`
-- If `gh` is not found (PR mode only), tell the user to install GitHub CLI
 - If codex times out (5 minutes), show whatever partial output was captured and note the timeout
-- If the PR number doesn't exist, report the gh error clearly
-- If the current branch IS master (branch review mode), tell the user and suggest using `uncommitted` or `commit <SHA>` mode instead

crates/actors/src/block_producer/ledger_expiry.rs

@@ -95,6 +95,14 @@ pub async fn calculate_expired_ledger_fees(
     db: &DatabaseProvider,
     expect_txs_to_be_promoted: bool,
 ) -> eyre::Result<LedgerExpiryBalanceDelta> {
+    // Fee distribution is only implemented for Submit ledger. Publish expiry
+    // simply resets partitions without fee redistribution.
+    debug_assert_ne!(
+        ledger_type,
+        DataLedger::Publish,
+        "fee distribution not supported for Publish ledger"
+    );
+
     // Step 1: Collect expired partitions
     let expired_slots =
         collect_expired_partitions(parent_epoch_snapshot, block_height, ledger_type)?;
@@ -288,44 +296,42 @@ fn collect_expired_partitions(
     );
 
     for expired_partition in expired_partition_info {
-        let partition = partition_assignments
-            .get_assignment(expired_partition.partition_hash)
-            .ok_or_eyre("could not get expired partition")?;
-
         let ledger_id = expired_partition.ledger_id;
         let slot_index = SlotIndex::new(expired_partition.slot_index as u64);
 
-        // Only process partitions for the target ledger type
-        if ledger_id == target_ledger_type {
-            // Verify this ledger type can expire
-            if ledger_id == DataLedger::Publish {
-                eyre::bail!("publish ledger cannot expire");
-            }
-
-            tracing::info!(
-                "Found expired partition for {:?} ledger at slot_index={}, miner={:?}",
-                ledger_id,
-                slot_index.0,
-                partition.miner_address
-            );
-
-            // Store miner_address (not reward_address) to preserve unique miner identities
-            // for correct fee distribution. Reward address resolution is deferred to
-            // aggregate_balance_deltas to ensure pooled miners (sharing a reward address)
-            // are counted individually for fee splitting.
-            expired_ledger_slot_indexes
-                .entry(slot_index)
-                .and_modify(|miners: &mut Vec<IrysAddress>| {
-                    miners.push(partition.miner_address);
-                })
-                .or_insert(vec![partition.miner_address]);
-        } else {
+        // Filter by ledger type FIRST — before any lookup that could fail.
+        // This prevents a Publish partition state inconsistency from blocking
+        // Submit fee distribution (or vice versa).
+        if ledger_id != target_ledger_type {
             tracing::debug!(
                 "Skipping partition with ledger_id={:?} (looking for {:?})",
                 ledger_id,
                 target_ledger_type
             );
+            continue;
         }
+
+        let partition = partition_assignments
+            .get_assignment(expired_partition.partition_hash)
+            .ok_or_eyre("could not get expired partition")?;
+
+        tracing::info!(
+            "Found expired partition for {:?} ledger at slot_index={}, miner={:?}",
+            ledger_id,
+            slot_index.0,
+            partition.miner_address
+        );
+
+        // Store miner_address (not reward_address) to preserve unique miner identities
+        // for correct fee distribution. Reward address resolution is deferred to
+        // aggregate_balance_deltas to ensure pooled miners (sharing a reward address)
+        // are counted individually for fee splitting.
+        expired_ledger_slot_indexes
+            .entry(slot_index)
+            .and_modify(|miners: &mut Vec<IrysAddress>| {
+                miners.push(partition.miner_address);
+            })
+            .or_insert(vec![partition.miner_address]);
     }
 
     Ok(expired_ledger_slot_indexes)

crates/actors/tests/epoch_snapshot_tests.rs

@@ -185,6 +185,7 @@ async fn add_slots_test() {
             num_blocks_in_epoch: 100,
             num_capacity_partitions: Some(123),
             submit_ledger_epoch_length: 5,
+            publish_ledger_epoch_length: None,
         },
         ..ConsensusConfig::testing()
     };
@@ -281,6 +282,7 @@ async fn unique_addresses_per_slot_test() {
             num_blocks_in_epoch: 100,
             num_capacity_partitions: Some(123),
             submit_ledger_epoch_length: 5,
+            publish_ledger_epoch_length: None,
         },
         ..ConsensusConfig::testing()
     };
@@ -430,6 +432,7 @@ async fn partition_expiration_and_repacking_test() {
             submit_ledger_epoch_length: 2,
             num_blocks_in_epoch: 5,
             num_capacity_partitions: Some(123),
+            publish_ledger_epoch_length: None,
         },
         ..ConsensusConfig::testing()
     };
@@ -964,6 +967,7 @@ async fn partitions_assignment_determinism_test() {
             num_blocks_in_epoch: 100,
             submit_ledger_epoch_length: 2,
             num_capacity_partitions: None,
+            publish_ledger_epoch_length: None,
         },
         ..ConsensusConfig::testing()
     };

crates/chain-tests/src/lib.rs

@@ -21,6 +21,8 @@ mod packing;
 #[cfg(test)]
 mod partition_assignments;
 #[cfg(test)]
+mod perm_ledger_expiry;
+#[cfg(test)]
 mod programmable_data;
 #[cfg(test)]
 mod promotion;