feat(recomp): discover packed jal-only entries via cross-unit manifests and thread-entry scan#150
Open
smmathews wants to merge 1 commit into
Open
Conversation
f96656a to
1b4571f
Compare
…ts and thread-entry scan A callee reached only by a jal/j immediate from a caller outside the byte range grouped into one function unit is disassembled as unreachable tail code inside whichever unit contains it, but gets no callable entry of its own. At runtime lookupFunction() reports it not-found and the recovery path silently substitutes broken behavior instead of failing loudly. The existing post-pass discoverAdditionalEntryPoints() already handles same-invocation cross-function targets via resume mapping; two forms survived. Part A - cross-compilation-unit / overlay callers: each invocation now emits external_call_targets.txt (jal/j targets in an executable section but outside its own recompiled ranges) into the output directory, and a new [general] external_call_target_manifests config array ingests manifests produced by other invocations. Ingested targets that land inside this invocation's function ranges are registered through the existing resume-entry mapping (m_resumeEntryTargetsByOwner), so lookupFunction(target) dispatches into the owner unit at the right pc. The emit-selection logic is a public static (CollectExternalCallTargets) for unit testing. Part B - data-embedded thread entries: a new analysis pass locates CreateThread (syscall 0x20) call sites (direct syscall or jal to a libkernel wrapper), recovers the static ThreadParam pointer in $a0 via a bounded lui/addiu/ori constant walk (including the jal delay slot), reads the entry function pointer from ELF data at param+4, and registers entries that fall inside recompiled function ranges via the same resume mapping. Scope is CreateThread-with-immediate-ThreadParam only: StartThread (0x22), runtime-computed param pointers, and handler-registration variants are not covered. Both discovery paths feed the existing registration machinery; no runtime or emitter behavior changes. Includes unit tests for manifest parsing, the external-target selection helper, and the thread-entry discovery helper, plus end-to-end regression tests that drive the real recompile pipeline and assert registration through the emitted function table.
1b4571f to
a5e4a6a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Discover packed jal-only entry points across compilation units and in thread-param data
Problem
A callee reached only by a
jal/jimmediate from a caller that lies outside the byterange the analyzer grouped into one function unit gets disassembled as unreachable tail
code inside whichever unit physically contains it, but receives no callable entry of its
own. At runtime
lookupFunction()reports the address not-found on the real call and theruntime's recovery path silently substitutes broken behavior (recovering with stale
registers / a bogus recover pc) instead of failing loudly. The bug hides behind "it boots."
The production post-pass
discoverAdditionalEntryPoints()already registerssame-invocation cross-function jal/j targets via resume mapping (dispatching into the
owner unit at the target pc). Two forms survive and still produce silent failures:
one invocation (e.g. a main ELF plus a separately compiled overlay), the analyzer for
one unit never sees the
jalthat lives in the other unit, so the callee stays entryless.instruction; it sits in a pre-initialized
ThreadParamstruct in.dataand is passedto
CreateThreadvia a struct pointer. Nojalexists anywhere in the code text, so noinstruction scan can ever see it.
Part A — cross-compilation-unit / overlay jal targets
external_call_targets.txtinto the output directory: sorted, de-duplicated
jal/jtargets that lie in anexecutable section but outside all of that invocation's recompiled function ranges
(candidate cross-unit calls). The selection is factored into a public static
CollectExternalCallTargetsfor direct unit testing.[general] external_call_target_manifests = [ ... ]ingestsmanifests from other invocations (blank lines and
#comments allowed; a gracefulwarning is logged on an unreadable file). During
discoverAdditionalEntryPoints(), aningested target that lands strictly inside one of this invocation's function ranges is
registered through the existing
m_resumeEntryTargetsByOwner/registerFunction(target, ownerName)path, solookupFunction(target)dispatches intothe owner unit at the correct pc.
flow includes backward branches into shared parent-unit code work without cloning,
because the owner unit retains its full declared range.
This makes the fix a build-graph change (invocations exchange call-target manifests), not a
per-game data edit — generic to any multi-ELF / overlay title.
Part B — data-embedded thread-entry pointers
New static analysis helper
DiscoverDataEmbeddedThreadEntries(decodedFunctions, isValidAddress, readWord):CreateThread(syscall0x20) wrapper functions (addiu $v1,$zero,0x20followed by
syscallin the prologue) and direct inline syscall sites (with a$v1-clobber-aware back-scan).ThreadParampointer in$a0via a bounded forwardlui/addiu/oriconstant-propagation walk over the call window, including the jaldelay slot;
$gp-relative forms are conservatively skipped.param + 4(PS2-ABIThreadParamlayout, matching the runtime's
CreateThreaddecode) and registers in-range entriesthrough the same resume mapping.
The pass is wired into
discoverAdditionalEntryPoints()and guarded against a data-lesssection (e.g. BSS) throwing on read.
Scope: Part B detects the
CreateThread-with-immediate-ThreadParamcase only. It doesnot cover
StartThread(syscall0x22) call sites, thread entries whose param pointer iscomputed at runtime rather than link-time-constant, or handler-registration variants
(
AddIntcHandler/AddDmacHandler). Those remain out of scope here.Configuration
Each invocation writes its own
external_call_targets.txt; sibling invocations list theothers' manifests to close cross-unit calls. Documented in the README configuration section.
Tests
Full build (Ninja, Debug, Linux/GCC) is clean; the
ps2x_testssuite passes 313/313,including 17 new tests:
selection helper (
CollectExternalCallTargets: local-range exclusion,non-executable-target exclusion, sort/dedup across jal and j, skipped/stub ranges still
emitted, conditional branches ignored).
$a0, direct inlinesyscall, clobbered
$a0negative, wrong syscall number negative, zero-entry filtering,dedup across call sites, sign-extended lo16 arithmetic, and a regression for a decoder
quirk where a J-type instruction's raw target bits alias register fields and must not
count as a register write.
PS2Recompilerinstance(
initialize()→recompile()→generateOutput()) over generated ELF fixtures andinspect the actual emitted
register_functions.cpp:function in the ELF (no jal anywhere targets it), the address is registered and maps
to the owning unit's function — and the same run without the manifest leaves it
unregistered, reproducing the original bug. The test also asserts the ingested entry
resolves to the exact same owner as the containing function's head, i.e. it dispatches
into the owning unit (which retains its full declared range) rather than being sliced
into a truncated standalone function; the slicer path's end-boundary behavior is
covered by the pre-existing boundary tests.
ThreadParampointer and jals aCreateThreadwrapper, plus a real.datasectionholding the struct whose second word points mid-body into a third function — the
entry pointer is read through the real ELF parser and the entry registers to the
containing function.
Limitations
CreateThread-immediate only (see Scope above);StartThread/ computed-pointerentries are not handled.
addiu $v1write totwo nearby syscalls; this is harmless (a spurious in-range candidate that is de-duplicated
downstream) and is annotated in the code.