Skip to content

feat(recomp): discover packed jal-only entries via cross-unit manifests and thread-entry scan#150

Open
smmathews wants to merge 1 commit into
ran-j:mainfrom
smmathews:feature/03-recompiler-packed-jal-entries
Open

feat(recomp): discover packed jal-only entries via cross-unit manifests and thread-entry scan#150
smmathews wants to merge 1 commit into
ran-j:mainfrom
smmathews:feature/03-recompiler-packed-jal-entries

Conversation

@smmathews

Copy link
Copy Markdown
Contributor

Discover packed jal-only entry points across compilation units and in thread-param data

Problem

A callee reached only by a jal/j immediate from a caller that lies outside the byte
range the analyzer grouped into one function unit gets disassembled as unreachable tail
code inside whichever unit physically contains it, but receives no callable entry of its
own. At runtime lookupFunction() reports the address not-found on the real call and the
runtime's recovery path silently substitutes broken behavior (recovering with stale
registers / a bogus recover pc) instead of failing loudly. The bug hides behind "it boots."

The production post-pass discoverAdditionalEntryPoints() already registers
same-invocation cross-function jal/j targets via resume mapping (dispatching into the
owner unit at the target pc). Two forms survive and still produce silent failures:

  1. Cross-compilation-unit / overlay callers. When a title is recompiled in more than
    one invocation (e.g. a main ELF plus a separately compiled overlay), the analyzer for
    one unit never sees the jal that lives in the other unit, so the callee stays entryless.
  2. Data-embedded thread entries. The callee address is never computed by any
    instruction; it sits in a pre-initialized ThreadParam struct in .data and is passed
    to CreateThread via a struct pointer. No jal exists anywhere in the code text, so no
    instruction scan can ever see it.

Part A — cross-compilation-unit / overlay jal targets

  • Every recompile invocation now emits a call-target manifest external_call_targets.txt
    into the output directory: sorted, de-duplicated jal/j targets that lie in an
    executable section but outside all of that invocation's recompiled function ranges
    (candidate cross-unit calls). The selection is factored into a public static
    CollectExternalCallTargets for direct unit testing.
  • New config option [general] external_call_target_manifests = [ ... ] ingests
    manifests from other invocations
    (blank lines and # comments allowed; a graceful
    warning is logged on an unreadable file). During discoverAdditionalEntryPoints(), an
    ingested target that lands strictly inside one of this invocation's function ranges is
    registered through the existing m_resumeEntryTargetsByOwner /
    registerFunction(target, ownerName) path, so lookupFunction(target) dispatches into
    the owner unit at the correct pc.
  • Resume mapping (not a standalone slice) is used deliberately: entries whose real control
    flow includes backward branches into shared parent-unit code work without cloning,
    because the owner unit retains its full declared range.

This makes the fix a build-graph change (invocations exchange call-target manifests), not a
per-game data edit — generic to any multi-ELF / overlay title.

Part B — data-embedded thread-entry pointers

New static analysis helper DiscoverDataEmbeddedThreadEntries(decodedFunctions, isValidAddress, readWord):

  1. Identifies CreateThread (syscall 0x20) wrapper functions (addiu $v1,$zero,0x20
    followed by syscall in the prologue) and direct inline syscall sites (with a
    $v1-clobber-aware back-scan).
  2. Recovers the static ThreadParam pointer in $a0 via a bounded forward
    lui/addiu/ori constant-propagation walk over the call window, including the jal
    delay slot; $gp-relative forms are conservatively skipped.
  3. Reads the entry function pointer from ELF data at param + 4 (PS2-ABI ThreadParam
    layout, matching the runtime's CreateThread decode) and registers in-range entries
    through the same resume mapping.

The pass is wired into discoverAdditionalEntryPoints() and guarded against a data-less
section (e.g. BSS) throwing on read.

Scope: Part B detects the CreateThread-with-immediate-ThreadParam case only. It does
not cover StartThread (syscall 0x22) call sites, thread entries whose param pointer is
computed at runtime rather than link-time-constant, or handler-registration variants
(AddIntcHandler / AddDmacHandler). Those remain out of scope here.

Configuration

[general]
external_call_target_manifests = [
    "overlay_out/external_call_targets.txt",
    "main_out/external_call_targets.txt",
]

Each invocation writes its own external_call_targets.txt; sibling invocations list the
others' manifests to close cross-unit calls. Documented in the README configuration section.

Tests

Full build (Ninja, Debug, Linux/GCC) is clean; the ps2x_tests suite passes 313/313,
including 17 new tests:

  • Manifest parsing (sort / dedup / comments / blanks, empty input) and the external-target
    selection helper (CollectExternalCallTargets: local-range exclusion,
    non-executable-target exclusion, sort/dedup across jal and j, skipped/stub ranges still
    emitted, conditional branches ignored).
  • Thread-entry discovery unit tests: jal-to-wrapper with delay-slot $a0, direct inline
    syscall, clobbered $a0 negative, wrong syscall number negative, zero-entry filtering,
    dedup across call sites, sign-extended lo16 arithmetic, and a regression for a decoder
    quirk where a J-type instruction's raw target bits alias register fields and must not
    count as a register write.
  • Two end-to-end regression tests that drive a real PS2Recompiler instance
    (initialize()recompile()generateOutput()) over generated ELF fixtures and
    inspect the actual emitted register_functions.cpp:
    • Cross-unit manifest: with a manifest naming a mid-body address of the only
      function in the ELF (no jal anywhere targets it), the address is registered and maps
      to the owning unit's function — and the same run without the manifest leaves it
      unregistered, reproducing the original bug. The test also asserts the ingested entry
      resolves to the exact same owner as the containing function's head, i.e. it dispatches
      into the owning unit (which retains its full declared range) rather than being sliced
      into a truncated standalone function; the slicer path's end-boundary behavior is
      covered by the pre-existing boundary tests.
    • Data-embedded thread entry: an ELF with a caller that loads a static
      ThreadParam pointer and jals a CreateThread wrapper, plus a real .data section
      holding the struct whose second word points mid-body into a third function — the
      entry pointer is read through the real ELF parser and the entry registers to the
      containing function.

Limitations

  • Part B is CreateThread-immediate only (see Scope above); StartThread / computed-pointer
    entries are not handled.
  • The inline-syscall back-scan can, in principle, double-attribute one addiu $v1 write to
    two nearby syscalls; this is harmless (a spurious in-range candidate that is de-duplicated
    downstream) and is annotated in the code.

…ts and thread-entry scan

A callee reached only by a jal/j immediate from a caller outside the byte
range grouped into one function unit is disassembled as unreachable tail
code inside whichever unit contains it, but gets no callable entry of its
own. At runtime lookupFunction() reports it not-found and the recovery path
silently substitutes broken behavior instead of failing loudly. The existing
post-pass discoverAdditionalEntryPoints() already handles same-invocation
cross-function targets via resume mapping; two forms survived.

Part A - cross-compilation-unit / overlay callers: each invocation now emits
external_call_targets.txt (jal/j targets in an executable section but outside
its own recompiled ranges) into the output directory, and a new [general]
external_call_target_manifests config array ingests manifests produced by
other invocations. Ingested targets that land inside this invocation's
function ranges are registered through the existing resume-entry mapping
(m_resumeEntryTargetsByOwner), so lookupFunction(target) dispatches into the
owner unit at the right pc. The emit-selection logic is a public static
(CollectExternalCallTargets) for unit testing.

Part B - data-embedded thread entries: a new analysis pass locates
CreateThread (syscall 0x20) call sites (direct syscall or jal to a libkernel
wrapper), recovers the static ThreadParam pointer in $a0 via a bounded
lui/addiu/ori constant walk (including the jal delay slot), reads the entry
function pointer from ELF data at param+4, and registers entries that fall
inside recompiled function ranges via the same resume mapping. Scope is
CreateThread-with-immediate-ThreadParam only: StartThread (0x22),
runtime-computed param pointers, and handler-registration variants are not
covered.

Both discovery paths feed the existing registration machinery; no runtime or
emitter behavior changes. Includes unit tests for manifest parsing, the
external-target selection helper, and the thread-entry discovery helper, plus
end-to-end regression tests that drive the real recompile pipeline and assert
registration through the emitted function table.
@smmathews smmathews force-pushed the feature/03-recompiler-packed-jal-entries branch from 1b4571f to a5e4a6a Compare July 7, 2026 17:49
@smmathews smmathews marked this pull request as ready for review July 7, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant