From f46b4a44a48fc6c6a6c6fcdf539beeaef9825f93 Mon Sep 17 00:00:00 2001 From: SDD Date: Fri, 16 Oct 2026 10:48:02 +0300 Subject: [PATCH] docs: root-cause Ghidra TMS34010 CALLA bug to addr32 & 0xFFFFF (empty call graph) --- docs/ghidra-tms34010-calla-bug.md | 358 ++++++++++++++++++++++++++++++ 1 file changed, 358 insertions(+) create mode 100644 docs/ghidra-tms34010-calla-bug.md diff --git a/docs/ghidra-tms34010-calla-bug.md b/docs/ghidra-tms34010-calla-bug.md new file mode 100644 index 0000000..2f40490 --- /dev/null +++ b/docs/ghidra-tms34010-calla-bug.md @@ -0,0 +1,358 @@ +# Ghidra TMS34010 module corrupts `CALLA` targets (`& 0xFFFFF`), so the call graph is empty + +**Status:** root-caused to a single SLEIGH expression. Fix is a one-token change in the +processor module source; no Ghidra core fix is needed. +**Owner of the defect:** the third-party processor module, not this project and not stock Ghidra. +**Signed-off investigation:** all findings below were reproduced live against the open program +(`vt1200_v22.bin`, language `tms34010:LE:32:default`) with the Ghidra MCP server. + +--- + +## 1. Summary + +Ghidra's TMS34010 processor module decodes the `CALLA` (call absolute) operand with this +SLEIGH field definition: + +``` +addr32: addy is imm32 [addy = (imm32 / 8) & 0xfffff;] {export *:4 addy;} +``` + +`imm32 / 8` is the correct bit-address → byte-address conversion for this firmware (the ROM's +stored pointers are TMS34010 **bit** addresses; `bit / 8` is exactly the Ghidra byte address). +The trailing `& 0xfffff` is spurious and destroys the top 12 bits of the 32-bit address. For +every call target in this ROM it strips the `0x07F00000` nibbles and leaves a target such as +`0x07FBC018` rendered as `0x000BC018`. + +`0x000BC018` lies **outside every defined memory block** (the ROM block is +`0x07E40000`–`0x07FFFFFF`; the lowest defined block is `0x00800000`). Ghidra therefore creates +an `UNCONDITIONAL_CALL` reference to an unmapped address with no function, so +`function_callees` / `function_callers` never link it. The call graph for this ROM is empty. + +The **same** transform is applied to `ADDI`'s 32-bit immediate (a second site, line 185), where +even `/ 8` is wrong; see §7. + +--- + +## 2. Impact + +* `function_callers` / `function_callees` return **0** for functions that plainly call others. + Verified: `FUN_rom_07fbd000` (the level-1 error dispatcher) returns **0 callers, 0 callees**, + while it calls the printer routine and a helper many times. +* The decompiler renders those calls as calls to unknown externals + (`func_0x000bc018(...)`, `func_0x000bbf96()`), never as calls to real functions. +* Any analysis built on Ghidra's call graph for this ROM is unsound: reachability, + "is this function dead/unreachable", caller lookups, and any "0 callers" conclusion are all + artefacts of this bug. One such "0 callers" result was already recorded and retracted + (`harness/mame/BRINGUP_LOG.md`, around line 920). +* Data/pointer cross-references are **not** affected, because they are added by a separate + script (`ghidra_scripts/FixPtrConstants.java`) that converts bit→byte with `value >>> 3` and + adds `DATA` references. This is why `MOVI …` constants resolve but `CALLA` targets do not. + +--- + +## 3. Environment / versions (for whoever fixes this) + +| item | value | +|---|---| +| Ghidra | `12.1.2`, release `DEV`, build `2026-Jun-29 1434 GMT` (`/opt/ghidra/Ghidra/application.properties`) | +| Language | `tms34010:LE:32:default` | +| Processor module | third-party `ZPaul2Fresh8/Ghidra_TMS34010`, vendored in `tools/ghidra-tms34010/` | +| Upstream commit | `f4818e2738a1803849fabe25f826faba231c2610` ("Added ImportSymbols script for Python version 3") — still contains the defect | +| Installed spec | `/opt/ghidra/Ghidra/Processors/TMS34010/data/languages/tms34010.sinc` line **73** | +| Compiled spec | `/opt/ghidra/Ghidra/Processors/TMS34010/data/languages/tms34010.sla` | +| Installer | `tools/install-ghidra-plugin.sh` (copies vendored source to `/opt/ghidra`, recompiles `.sla`) | +| Program | `vt1200_v22.bin`, image base `rom:07e40000`, blocks: ROM `0x07E40000`–`0x07FFFFFF` (execute), I/O windows `0x00800000`–`0x00D00FFF` | + +The project's local patch (`tools/ghidra-tms34010-local.patch`) only changes the **pspec** +(`data_space` → `rom`, and removal of the original Midway board memory map). The `.sinc` — where +this defect lives — is byte-for-byte identical to the vendored module and to the installed copy. + +--- + +## 4. Exact reproduction + +### 4.1 Open the program + +``` +tools.ghidra.project_program_open_existing + project_location = /home/arti/code/k-space/vt1200/ghidra + project_name = vt1200 + program_name = vt1200_v22.bin + read_only = false +-> session_id = d05dfb7f99fe446ebb801c0609d2da8a +``` + +### 4.2 The instruction and its rendering + +``` +tools.ghidra.listing_code_unit_at { address: "0x07fbd020" } +-> { bytes: "5f0dc000de3f", mnemonic: "CALLA", text: "CALLA 0x000bc018" } +``` + +The raw operand is correct in memory: opcode `0x0D5F`, then 32-bit little-endian immediate +`c0 00 de 3f` = **`0x3FDE00C0`**. `smart34010` renders the same instruction as +`calla 0x3fde00c0` (see `reference/dis34-a-v22.txt.gz`, line 628112). + +### 4.3 The transformation, computed from the raw bytes + +``` +python3 - <<'EOF' +d = open('vt1200_v22.bin','rb').read() # same bytes the Ghidra program holds +BASE = 0x07E40000 # Ghidra image base +def le32(o): return int.from_bytes(d[o:o+4],'little') +for foff in (0x17d020, 0x17d044, 0x17d062, 0x17d226, 0x17c000): + imm = le32(foff+2) + print(f"ghidra={BASE+foff:#010x} op={le32(foff):#06x} imm32={imm:#010x} " + f"imm/8={imm//8:#010x} masked={imm//8 & 0xfffff:#010x}") +EOF +``` +Output: +``` +ghidra=0x07fbd020 op=0x0d5f imm32=0x3fde00c0 imm/8=0x07fbc018 masked=0x000bc018 +ghidra=0x07fbd044 op=0x0d5f imm32=0x3fde00c0 imm/8=0x07fbc018 masked=0x000bc018 +ghidra=0x07fbd062 op=0x0d5f imm32=0x3fddfcb0 imm/8=0x07fbbf96 masked=0x000bbf96 +ghidra=0x07fbd226 op=0x0d5f imm32=0x3f208660 imm/8=0x07e410cc masked=0x000410cc +ghidra=0x07fbc000 op=0x0d5f imm32=0x3fdf97f0 imm/8=0x07fbf2fe masked=0x000bf2fe +``` +Every `masked` value equals the operand Ghidra prints. + +### 4.4 Why the target does not resolve + +``` +tools.ghidra.function_at { address: "0x000bc018" } +-> ERR "no function found at rom:000bc018" # rendered (truncated) address +tools.ghidra.function_at { address: "0x07fbc018" } +-> ERR "no function found at rom:07fbc018" # correct address: currently a data byte +tools.ghidra.function_at { address: "0x07fbbf96" } +-> FUN_rom_07fbbf96 (exists; the helper) +``` + +* `0x000BC018` is below the lowest defined block (`0x00800000`), so it is **unmapped**; no + function can ever exist there. `reference_to 0x000bc018` returns a capped list of **100 + `UNCONDITIONAL_CALL` references** (e.g. `from rom:07fbd020`), proving Ghidra *did* build call + references — just to the wrong, unmapped address. +* `0x07FBC018` (the correct address) is inside the ROM block but Ghidra currently types it as + `DataDB` byte `0x8F` (`DAT_rom_07fbc018`). Those bytes are a valid function prologue + (`8F 09 74 01` = `MMTM SP,0x0174`); the code was never disassembled there precisely because + no correct call reference ever reached it. + +### 4.5 The empty call graph (the consequence that matters) + +``` +tools.ghidra.function_callers { function_start: "0x07fbd000" } +-> { count: 0, items: [] } +tools.ghidra.function_callees { function_start: "0x07fbd000" } +-> { count: 0, items: [] } +``` + +And the p-code emitted for the instruction calls the truncated address: + +``` +tools.ghidra.pcode_op_at { address: "0x07fbd020" } +-> CALL (rom, 0xbc018, 4) # should be (rom, 0x7fbc018, 4) +``` + +The decompiler output for the same function makes the damage visible: + +```c +void FUN_rom_07fbd000(undefined4 param_1,undefined4 *param_2) +{ + func_0x000bbf96(); /* real function is FUN_rom_07fbbf96 */ + if (param_2 == 0) func_0x000bc018(0x3fdf5338,param_1); /* printer, target 0x3fde00c0 */ + ... +} +``` + +--- + +## 5. Derived transformation (exact arithmetic) + +Let `I` = the raw 32-bit immediate little-endian stored after opcode `0x0D5F`. + +``` +Ghidra operand shown = (I / 8) AND 0xFFFFF # addr32, line 73 +correct Ghidra address = I / 8 # because 0x3F200000 / 8 == 0x07E40000 +``` + +where `0x07E40000` is the image base and `0x3F200000` is the firmware's bit-address base. +(The identity `0x3F200000 / 8 = 0x07E40000` is the whole reason `I / 8` is the right answer.) + +The fault is **`& 0xFFFFF`**: a flat 20-bit truncation of the address. It is not a shift error, +not a signedness error, and not a relative-vs-absolute mix-up. The `/ 8` is correct for `CALLA`. + +### Worked examples (opcode word `0x0D5F`, raw imm32 = stored value = `smart34010` rendering) + +| Ghidra addr of instr | raw bytes | raw imm32 (= `smart34010`) | correct Ghidra target (`I/8`) | Ghidra shows | target meaning | +|---|---|---|---|---|---| +| `0x07FBD020` | `5f0d c000de3f` | `0x3FDE00C0` | `0x07FBC018` | `0x000BC018` | printer routine (`MMTM SP,0x174`) | +| `0x07FBD062` | `5f0d b0fcdd3f` | `0x3FDDFCB0` | `0x07FBBF96` | `0x000BBF96` | `FUN_rom_07fbbf96` | +| `0x07FBD226` | `5f0d 6086203f` | `0x3F208660` | `0x07E410CC` | `0x000410CC` | low-ROM routine | +| `0x07FBC000` | `5f0d f097df3f` | `0x3FDF97F0` | `0x07FBF2FE` | `0x000BF2FE` | routine | +| `0x07E40146` (`JAEQ`) | `80ca a024203f` | `0x3F2024A0` | `0x07E40494` | `0x00040494` | conditional absolute jump | + +`smart34010` cross-checks (from `reference/dis34-a-v22.txt.gz`, file offset `f` shown as base-0 +bit address `f*8`): `00be8100: calla 0x3fde00c0`, `00be8310: calla 0x3fddfcb0`, +`00be9130: calla 0x3f208660`, `00000a30: jaeq 0x3f2024a0`. All match the "raw imm32" column. + +--- + +## 6. Scope + +The defect is confined to the `addr32` field, which is used by exactly two instruction forms: + +| instruction | field | affected? | evidence | +|---|---|---|---| +| `CALLA addr32` | `addr32` | **yes** | 16 939 `calla` sites / 1 767 distinct targets in `reference/dis34-a-v22.txt.gz`; all truncated | +| `JA^cc addr32` (conditional absolute jump) | `addr32` | **yes** | e.g. Ghidra `0x07E40146 JAEQ 0x00040494`; `smart34010` `jaeq 0x3f2024a0`. 103 sites / 82 distinct targets | +| `ADDI link,Rd` | `(imm32/8)&0xfffff` (line 185) | **yes** (separate site) | `0x07FBD128`: raw imm `0x00800000`; Ghidra `ADDI 0x0,A9`; `smart34010` `addi 0x800000, a9` | +| `CALL Rd` | register indirect | no | no immediate target | +| `JUMP Rd` | register indirect | no | no immediate target | +| `CALLR addr16` | `addr16` = `inst_next + s16*2` | no | relative; 0 absolute targets in reference | +| `JR^cc addr16` / `JR^cc addr8` | `addr8`/`addr16` | no (for this bug) | relative; 0 absolute targets in reference. NB the spec header itself notes a separate, unrelated "JRcc Long" arithmetic TODO | + +Because the ROM's live code all sits at byte `≥ 0x07E40000`, **every** `addr32` absolute +target in this image is corrupted (all would need the `0x07` nibble that the mask deletes). +The defect is effectively total for absolute calls and jumps in this firmware. + +The commented-out `MOVI link` variant at line 940 shows the module author applying the same +`(imm32 / 8) & 0xfffff` idea; the active `MOVI imm32,Rd` (line 942) does **not** use it, which is +why `MOVI 0x3fdca990`-style operands render correctly. + +--- + +## 7. Root cause + +* **Where:** `tms34010.sinc`, the `addr32` field. + * Vendored source: `tools/ghidra-tms34010/tms34010.sinc` line **73** + * Installed copy: `/opt/ghidra/Ghidra/Processors/TMS34010/data/languages/tms34010.sinc` line **73** + * Compiled into: `…/tms34010.sla` +* **Exact text:** + ``` + addr32: addy is imm32 [addy = (imm32 / 8) & 0xfffff;] {export *:4 addy;} + ``` +* **Origin:** the third-party module `ZPaul2Fresh8/Ghidra_TMS34010`, at upstream HEAD + `f4818e2738a1803849fabe25f826faba231c2610`; the `.sinc` is unmodified by this project. This is + **not** a stock-Ghidra module (Ghidra 12.1.2 has no in-tree TMS34010 SLEIGH source), so it is + not a stock-Ghidra regression. +* **Why the mask exists (inference):** the module was originally written against a Midway + arcade board map (the unpatched pspec maps ROM at `rom:0xff800000`, GFX at `rom:0x02000000`, + etc.). A flat `0xFFFFF` looks like a leftover assumption that the CPU address space is 20 bits; + the `addr32` value is otherwise a full 32-bit bit address. Whatever its intent, it discards the + high bits of every absolute target and cannot be correct for any machine whose code lives above + byte `0x0FFFFF`. The `pspec` in this project was already changed to `space="rom"` precisely + because the shipped board map did not describe the VT1200. +* **Known upstream issue?** No fix is present at the upstream commit above. The project treats + `tools/ghidra-tms34010/tms34010.sinc` as the source of truth and installs it with + `tools/install-ghidra-plugin.sh`. + +--- + +## 8. Recommended fix + +**Change belongs in the SLEIGH processor spec** (`addr32`), then recompile to `.sla`. In +`tms34010.sinc`: + +```diff +-addr32: addy is imm32 [addy = (imm32 / 8) & 0xfffff;] {export *:4 addy;} ++addr32: addy is imm32 [addy = imm32 / 8;] {export *:4 addy;} +``` + +(equivalently `imm32 >> 3`). If a bound is genuinely wanted it must be the full 32-bit range, +but there is no reason to mask a 32-bit address. All bit addresses in this ROM are +`< 0x80000000`, so signed/unsigned division does not differ here. + +Steps for the project (not performed by this investigation, per scope): + +1. Edit `tools/ghidra-tms34010/tms34010.sinc` as above (and, if desired, the matching + `ADDI` site at line 185 and the commented line 940 — but note `/8` itself is wrong for + `ADDI`'s arithmetic immediate; that operand should be rendered as the raw imm32). +2. Reinstall: `tools/install-ghidra-plugin.sh` (needs `sudo`; copies sources into + `/opt/ghidra/Ghidra/Processors/TMS34010` and recompiles `tms34010.sla`). +3. Re-run auto-analysis over the ROM (or `listing_disassemble_seed` from each corrected call + target) so functions are created at the now-correct targets; the existing project's + references will remain pointing at the old truncated addresses until they are cleared. +4. Upstream: the same change should be offered to `ZPaul2Fresh8/Ghidra_TMS34010`. + +**Consistency check:** with the mask removed, `0x3FDE00C0 / 8 = 0x07FBC018`, which is inside the +ROM block and holds the printer prologue `MMTM SP,0x0174`; the call reference and the `DATA` +reference created by `FixPtrConstants.java` (`value >>> 3`) would then agree, instead of +disagreeing (`CALL 0x0BC018` vs `DATA 0x7FBC018`). + +--- + +## 9. Workaround (until the module is fixed) + +**Use `smart34010` for call targets; use Ghidra for structure and decompilation.** Concretely: + +* Treat the raw `CALLA` immediate as the truth. It is the firmware's bit address; the Ghidra + byte address is `imm32 / 8` (drop the `& 0xFFFFF`). The operand bytes are intact in the listing + (`listing_code_unit_at` returns them), so the correct target is recoverable by hand/script. +* `reference/dis34-a-v22.txt.gz` already contains the authoritative `calla`/`ja` targets + (16 939 `calla` sites, 1 767 distinct targets). `reference/dis34-a-v1.txt.gz` for v2.1. +* Do **not** build any conclusion on Ghidra `function_callers` / `function_callees`, + `function_report`'s `callers`/`callees`, or decompiler `func_0x…` names for this ROM. +* What still works in Ghidra: mnemonic/instruction structure, `MOVI` constants (raw bit + addresses), data xrefs created by `FixPtrConstants.java`, and decompilation *body* analysis + (the `func_0x…` call placeholders are the only call-graph damage). +* Caveat on `smart34010`: its own **analyze mode does not follow `CALLA` targets either** — the + printer prologue at base-0 bit address `0x00BE00C0` is emitted as `.word 0x098F` rather than + disassembled. So "use `smart34010` for call targets" means *use its correctly-rendered operand + values*, not its auto-discovered call graph. Both tools need the manual `bit/8` mapping. +* Also beware the separate `ADDI` corruption (e.g. `ADDI 0x0,A9` where the real immediate is + `0x800000`); read `ADDI`/`MOVI` immediates from the raw bytes when they matter. + +--- + +## 10. Verified vs inferred + +**Verified against the live Ghidra program and the raw image:** +* The exact rendering `CALLA 0x000bc018` and raw bytes `5f0dc000de3f` at `0x07FBD020`. +* The formula `(imm32 / 8) & 0xFFFFF` reproduces every observed operand (5 `CALLA` sites, 1 + `JAEQ` site, 1 `ADDI` site), and the exact source line 73 produces it. +* `function_callers` and `function_callees` are 0 for `FUN_rom_07fbd000`; `pcode_op_at` emits + `CALL (rom, 0xbc018, 4)`; `reference_to 0x000bc018` returns `UNCONDITIONAL_CALL` refs from the + call sites. +* `0x000BC018` is outside all defined memory blocks; `0x07FBC018` and `0x07FBBF96` are inside the + ROM block; `FUN_rom_07fbbf96` exists; `0x07FBC018` is currently a `DataDB` whose bytes are the + `MMTM` printer prologue. +* `smart34010` targets for all sampled sites (from the committed reference disassembly). +* Module origin/commit, Ghidra version, and that the vendored/installed `.sinc` is unmodified. + +**Inferred (not directly executed, because this task must not modify the install):** +* That removing `& 0xFFFFF` and re-running analysis will populate the call graph. This is + strongly supported: the helper target `0x07FBBF96` already exists as a function, the printer + bytes are a valid prologue, and Ghidra already generates the call references — only the + address is wrong. +* The original rationale for `& 0xFFFFF` (a 20-bit-address assumption / Midway-board leftover). +* The exact post-fix behaviour of `function_callees` (whether Ghidra auto-creates the printer + function at `0x07FBC018` on re-analysis) was not run. + +--- + +## 11. Appendix — raw evidence + +Raw instruction bytes (`vt1200_v22.bin`, little-endian; `xxd -s 0x17c000`): + +``` +0017c000: 5f0d f097 df3f 8a11 2b14 efc5 484d eeb7 <- CALLA @0x07FBC000, imm=0x3fdf97f0 +0017c010: 2001 af09 a83e 6209 8f09 7401 cd4d a9b7 <- 0x07FBC018 = 8f 09 74 01 = MMTM SP,0x0174 +``` + +Spec excerpt (`/opt/ghidra/Ghidra/Processors/TMS34010/data/languages/tms34010.sinc`): + +``` +73 :addr32: addy is imm32 [addy = (imm32 / 8) & 0xfffff;] {export *:4 addy;} +... +294::CALLA addr32 is word=0x0D5F; addr32 +296: call addr32; +... +560::JA^cc addr32 is cc & bits_12_15=0b1100 & bits_0_7s=0b10000000; addr32 +185::ADDI link,Rd is bits_5_15=0b00001011001 & Rd; imm32 [link=(imm32 / 8) & 0xfffff;] +940:#:MOVI link,Rd is bits_5_15=0b00001001111 & Rd; imm32 [link=(imm32 / 8) & 0xfffff;] (commented out) +942::MOVI imm32,Rd is bits_5_15=0b00001001111 & Rd; imm32 (correct: raw) +``` + +MCP calls used (all with `session_id` from §4.1): `listing_code_unit_at`, +`listing_disassemble_function`, `listing_disassemble_range`, `pcode_op_at`, `function_at`, +`function_callers`, `function_callees`, `function_report`, `reference_to`, `reference_from`, +`memory_blocks_list`.