diff --git a/docs/superpowers/specs/2026-10-31-badapple-60p-design.md b/docs/superpowers/specs/2026-10-31-badapple-60p-design.md new file mode 100644 index 0000000..7abee5c --- /dev/null +++ b/docs/superpowers/specs/2026-10-31-badapple-60p-design.md @@ -0,0 +1,514 @@ +# Bad Apple at 60 fps, exact — design + +Date: 2026-10-31 +Status: **rev 1** — first draft, awaiting approval. +Author: (session). Decisions marked "**call:**" are the builder's proposals, not +something the human partner has agreed to. Decisions marked "**human:**" were +given in this session and are not open. + +Supersedes the tile-codec half of +[`2026-10-30-vt1200-video-streaming-design.md`](2026-10-30-vt1200-video-streaming-design.md) +(§8.3 scheme C, §8.9 OS interference, §10 M3–M5). The transport there (§8.1), the +back-channel (§8.4) and the delivery path (`harness/badapple/`) stand. + +--- + +## 1. Intent + +Play **Bad Apple!!** on a VT1200 from +`/tmp/opencode/badapple-real/hd60.mp4` at the source's **60 fps**, upscaled to +fill the panel, with the picture **no longer quantised through a 256-entry tile +dictionary** — because that dictionary, not the resolution, is what makes the +diagonals jagged. + +The ROM stays byte-identical (`md5 b46a3e442ccb86134d2356bd70d22fd2`). The OS is +allowed to die (§8.9 of this document). + +## 2. Grounding: the source + +`ffprobe /tmp/opencode/badapple-real/hd60.mp4` (human-supplied, this session): + +| field | value | +|---|---| +| video | 960x720, SAR 1:1, DAR 4:3, **60 fps**, progressive, yuv420p, h264 Main | +| duration | 00:03:39.52 → **≈13,171 frames** at 60 fps | +| audio | aac LC stereo (ignored — no audio path in this harness) | + +960×720 is 4:3 exactly, and 1280×960 is 4:3 exactly, so an upscale to +1280×960 is a clean 1.3333× with no crop and no aspect error. 960 is also a +multiple of 32, so `clip_from_video.py`'s default `--height 960` is already +legal (its `height % 32` check at line 98 rejects 720, not 960). + +**human:** deliver it upscaled to fill the panel rather than 1:1 with borders. + +**call:** the target geometry is **1280×960 at the top of the screen**, rows +960–1023 a permanently black bar. Stretching 960 rows to 1024 (a further 6.7 % +in Y) is a one-flag alternative if the bars look wrong on the unit; it changes +no other number in this document except the bar tiles. + +## 3. Grounding: what exists today + +All paths below are in this repository. + +| piece | where | what it does | +|---|---|---| +| payload | `harness/udpstream/vidstream.c` | takes the LANCE over, arms a 32-descriptor RX ring in two 48 KB slots, parses VS frames, dispatches to `paint` / `paint_tiles` / `paint_delta` / `dict_chunk`, sends a status frame every 64 accepted frames | +| tile primitive | `harness/udpstream/vram_tile.asm` | one `PIXBLT B,XY` of a 32×32 dictionary entry; ~35 instructions of prologue per call | +| strip primitive | `harness/udpstream/vram_b.asm` | one `PIXBLT B,XY` of a row with `SPTCH=0` vertical stretch | +| fill primitive | `harness/udpstream/vram.asm` | `FILL L` of the whole screen (fixed geometry) | +| wire format | `harness/udpstream/frame.py` + `frame_parse.h` | 16-byte `VS` header, header-only CRC-16, kinds 1–5 | +| tile wire form | `harness/udpstream/tiledict.py` | `TILES_FULL` = 1,280 index bytes; `TILES_DELTA` = `u16 count` + `count×u16 pos` + `count×u8 index`; `KEYFRAME_EVERY = 60` | +| dictionary build | `harness/udpstream/tilecost.py` | Lloyd/k-means over 32×32 tiles, Hamming distance, 256 entries × 128 B | +| encoder | `harness/udpstream/make_tiles.py` | clip → dictionary + indices + stream plan + manifest | +| sender | `harness/udpstream/vidsend.py` | paces at `--fps`, reads status frames, forces a keyframe when `drops` rises | +| checker | `harness/mame/tile_check.py` | decodes `stream.bin` independently, renders it, compares against dumped VRAM | +| demo | `harness/badapple/demo.sh` | waits for the OS's stack, loads the payload over rdebug, sends the clip | + +Measured behaviour of today's codec, all from `harness/mame/NETWORK_REPORT.md`: + +| quantity | value | where | +|---|---|---| +| synthetic clip, 30 frames | PSNR **29.47 dB**, 1.16 bits/tile mean error, 97.0 % of tiles exact | §76.4 | +| **real clip** (3,292 frames, 15 fps, 1280×960) | **PSNR 24.4 dB**, mean delta **496 B**, 55 keyframes, 1.63 MB stream | `harness/badapple/README.md` | +| one full tile repaint in MAME | `decode_worst` **37,612 REFCNT ticks** (1,280 tiles) | §76.5.1 | + +## 4. The diagnosis + +**The jaggies are the dictionary, and they can be counted.** 24.4 dB is a +bit-error rate of 0.0036, i.e. **≈4,760 wrong pixels per frame** out of +1,310,720. A 32×32 tile has on the order of 2³² distinct edge placements and the +dictionary has 256 centroids, so the errors concentrate on edges — which is +exactly what a viewer sees as ragged diagonals. At 1280 wide there is no +resolution left to buy; the fragment is the *edge vocabulary*. + +Three other causes look identical on screen and must be ruled out before the +codec is blamed. All are cheap: + +1. `clip_from_video.py` line 49/64: `scale=...:flags=lanczos` then `gray > + threshold`. Hard-thresholding a downscaled frame makes its own staircase. + Dump one `.bin` as an image and look. +2. `VT1200_MONITOR=VR150` shears the picture (`harness/mame/README.md`). +3. A source frame whose edges are already stepped (e.g. a 4:3 crop of a + differently-scaled master). + +**And the speed is not the constraint.** At the real clip's rate of change a +delta is ~165 tiles at 15 fps, so ~41 tiles per frame at 60 fps (motion per +frame divides by four). Against the budgets in §6 that is roughly **1/20th of +one frame**. There is no speed problem to solve in the delta path; there is +spare budget to spend on exactness. + +## 5. What the hardware actually charges for + +The single most important number in this document, measured on the unit and +recorded in the 2026-10-30 design (§2, §6): + +> `FILL L` costs **0.40 µs per destination word**; `PIXBLT L,XY` costs +> **0.62 µs/word**; each blit/fill *call* costs **2.6–3.3 µs** on top. + +A destination word is 16 pixels at 1 bpp. So **the pixel engine's cost is +proportional to the number of destination words written, and essentially +independent of which primitive writes them and of how the writes are grouped.** +Consequences, all of which this design depends on: + +* A 32×32 tile is 32 rows × 2 words = **64 destination words**. +* A 48×48 tile is 48 × 3 = 144 words; a 64×64 is 256. Per *pixel* there is no + gain — the extra call overhead saved (0.8 ms vs 3.1 ms over a full screen) is + under 10 % of a repaint. **Tile size is a wire decision, not a CPU decision.** +* `FILL` and `PIXBLT` cost the same per word, so collapsing a run of solid tiles + into one `FILL` saves **call overhead and wire bytes only**, not blitter time. +* The whole 1280×960 picture is 960 × 80 = **76,800 destination words**. + At 0.40 µs/word that is **≈31 ms** — *more than one 60 fps frame*, whatever + scheme drives it. + +Note the contradiction this exposes: MAME reports a full tile repaint at +~12 ms (37,612 REFCNT ticks, §3) while the unit's `FILL` rate says the same +pixels cost ~31 ms. `PIXBLT B,XY` at "0.32 µs/word" is a **MAME benchmark and +has never been measured on the unit** (the 2026-10-30 doc says so in its own +risk table). Every per-frame CPU number in this document is therefore labelled +either **unit** or **MAME**, and the first milestone measures the gap. + +## 6. Budget + +| quantity | value | source | +|---|---|---| +| frame period at 60 fps | **16.67 ms** | derived | +| display refresh | 72 Hz (13.9 ms) | 2026-10-30 §2 | +| destination words affordable per frame | **41,700** at 0.40 µs/word | derived from the unit rate | +| full picture (1280×960) | 76,800 words ≈ **31 ms** = **1.84 frames** | derived, unit rate | +| link | 10 Mbit half-duplex, **1.27 MB/s** measured | 2026-10-30 §2 | +| wire per frame at 60 fps | **21 KB** | derived | +| RX ring | 32 descriptors × 1526 B = 48,832 B (one slot) | `vidstream.c` | +| `payload_len` | u16 → a frame's payload is ≤ 65,535 B | `frame_parse.h` | +| heap | ~2.7 MB free at takeover | status frames | + +Two hard limits follow: **a frame's payload cannot exceed 65,535 B** +(`payload_len` is 16 bits) and **cannot exceed 32 datagrams** (~45.9 KB of +payload) without deepening the RX ring. The binding one is the ring. + +**The central finding: at 60 fps a full-screen repaint fits neither budget.** +It costs 1.84 frames of CPU and ~7 frames of wire (154 KB at 1.27 MB/s = +121 ms). Everything in the design below exists to make sure the full screen is +never repainted in one frame, and that ordinary frames are exact. + +## 7. Approaches considered + +**A. Keep the dictionary; widen the index to 10–12 bits.** Gets PSNR up by +increasing entries (256 → 1,024+). Wire: 2 B/tile. Cost: a 92-datagram upload +(the current 32-descriptor ring holds 23), plus the dictionary still cannot +represent every edge. **Rejected as the primary path**: it improves the same +approximation that causes the complaint, and it keeps the k-means encoder and +the 32 KB upload. + +**B. Keep the dictionary; add an 8×8 residual patch pass.** Exactness with +bounded cost (patches only where error > 0). **Not rejected, deferred**: it is +the best option if raw bitmaps turn out too expensive on the wire, and it reuses +everything. Kept as the fallback in §12. + +**C. Raw tiles — send the changed tiles' actual bits, with a solid tag.** +No dictionary at all. Exact by construction. Wire: 1 bit per changed pixel plus +a 1-byte tag per changed tile; a solid tile costs 1 byte instead of 128. +**Chosen.** + +**D. Spans / vectors (`FILL XY` per scanline run).** Exact and the same +per-word cost as everything else, so its only advantage is that it *writes fewer +words* on flat content and its only cost is that it turns dither into runs of +length 1. Bad Apple is full of dithered texture (grass, hair, checkerboard), so +a span encoding explodes exactly where tiles are cheap. **Rejected for this +source**; it stays the right answer for a flat-shaded stylisation, which is a +different film. + +**Decision (this document): C.** It is also the *simplest* option — it deletes +more code than it adds (§8.6). + +## 8. Design + +### 8.1 Geometry + +1280×960 at the top of the panel: **40 × 30 tiles of 32×32** = 1,200 picture +tiles, plus 64 permanently black bar tiles (grid rows 30–31) that are painted +once and never listed again. The grid, `TILES_SCREEN`, the row-major position +encoding, `tile_check.py`'s renderer and `vram_tile.asm`'s `DYDX=(32,32)` are all +unchanged — the smallest possible diff to the geometry. + +32×32 rather than 64×64 because tile size affects CPU cost by under 10 % (§5) +and affects wire cost a great deal: a solid 32×32 costs one tag, and a changed +32×32 patch costs 128 B against 512 B for 64×64. Finer tiles waste less overdraw +on an edge. **call:** 32×32. + +### 8.2 Wire form + +`KIND_TILES_FULL` and `KIND_TILES_DELTA` keep their numbers (2, 3) so the kind +table and the existing tests stay meaningful; their **payloads change**, and +`VERSION` goes to 2 so an old sender cannot be mis-dispatched. + +``` +payload: + offset size field + 0 2 count, u16 little-endian, 1..1,280 + 2 2*count position[k], u16, row-major tile index, STRICTLY ASCENDING + 2+2c count tag[k], u8: + 0x00 = this tile is solid black + 0x01 = this tile is solid white + 0x02 = a 128-byte bitmap follows + 2+3c m*128 bitmaps for the tags equal to 0x02, in k order + payload_len == 4 + 3*count + 128*m +``` + +Grouped rather than interleaved (positions, then tags, then bitmaps) so the +receiver can validate **the whole list without reading a single bitmap**: +count in range, length consistent with the count, every position on the grid and +strictly ascending, every tag in {0,1,2}, and the tail length exactly +128 × (number of `0x02` tags). That preserves the property the harness's +off-grid negative control exists to protect — *one bad entry paints nothing* — +and it is `O(count)` with one cursor and no division. + +`TILES_FULL` is the same form with `count == 1,200` (all picture tiles); the bar +tiles are never sent. + +**Positions refer to the 40×30 picture grid**: 0..1,199, row-major, with +`ty = pos / 40` avoided by incrementing as today. That is a *change* to the wire +constant `FP_TILES_SCREEN`/`TILES_PER_SCREEN` (1,280 → 1,200) and to +`tile_check.py`'s renderer, and the bar rows are outside the wire entirely — +which is the one place this design is not a minimal diff, so it is stated rather +than implied. + +**Expected wire, estimated from the one real-clip measurement** (496 B mean +delta at 15 fps ≈ 165 changed tiles → ~41 tiles per frame at 60 fps): if a third +of the changed tiles need a bitmap, ≈1.9 KB/frame ≈ **115 KB/s at 60 fps**, i.e. +9 % of the link. If every one needs a bitmap, 5.3 KB/frame ≈ 320 KB/s, 25 % of +the link. **The measurement in §11 replaces both numbers before any code is +written.** + +### 8.3 Decoder + +Per frame, unchanged from today's structure (§76's `paint_delta`): two passes, +validate then paint, inside one burst. + +Paint per tag: +* `0x00` / `0x01` → one `PIXBLT B,XY` from a **64-word** all-zero or all-ones + constant in the payload's pool, `DYDX=(32,32)`. (A `FILL` is the same cost — + §5 — and `PIXBLT` keeps one primitive and one set of implied operands.) +* `0x02` → one `PIXBLT B,XY` from the bitmap in the frame itself (it is already + in a slot, bit address `rx_frag_ptr + offset*8`; no copy). + +**Optimisation carried over from the earlier analysis, and it is the only one +that matters:** hoist `PSIZE`, `CONVSP`, `CONVDP`, `PMASK`, `CONTROL`, `B1`, +`B3`, `B7`, `B8`, `B9` out of the per-tile primitive into a per-frame (really +per-run) setup, and delete the eight push/pop pairs in `vram_tile.asm`. Only +`B0` (SADDR) and `B2` (DADDR) vary per tile, and `PIXBLT` destroys both. This is +a small constant on top of the per-word cost, so it is worth doing once and not +worth designing around. + +**Decoder state — the encoder's shadow screen.** Every delta must be computed +against *what the decoder currently shows*, not against the source's previous +frame, because a rolling refresh (§8.4) makes those two different: after a cut +the decoder holds the old picture with a few hundred tiles replaced. So +`vidsend.py` keeps a **shadow index buffer** of the decoder's screen, updated as +each frame is sent and marked "everything stale" when a refresh begins, and +computes every delta against it. Today's `plan()` diffs `previous` and assumes it +arrived; that is already false after a drop and would be routinely false under a +rolling refresh. Small change in the sender, a *precondition* for a correct +picture. + +### 8.4 Refresh + +A full refresh is **not** one frame. Split into sub-frames of at most **354 +bitmaps**: a frame must fit 32 datagrams of 1,450-byte payloads, so +`4 + 3c + 128m ≤ 32 × 1,450 = 46,400` with `m = c` gives `c ≤ 354` — fewer when +the tags make it smaller. (The u16 `payload_len` would allow 500; the ring binds +first.) The sub-frames go out across consecutive frames. + +**Bitmaps must not straddle a datagram.** A 128-byte bitmap lands mid-datagram +for most payload sizes, and a split bitmap would need bit-level reassembly across +fragments — per-byte CPU work, which is an anti-goal. **call:** the sender ends +each datagram at a tile boundary, wasting at most 127 bytes of a 1,450-byte +payload (≤ 9 % of the wire) and leaving the decoder a single fragment arithmetic +per bitmap. + +**The first refresh is a boot sequence, not a repair.** Before it the picture is +undefined, so painting it progressively is a wipe-in on a black screen. Every +later refresh replaces a picture already on screen, which is why §12 treats the +cut wipe as a real risk. On the wire a refresh +is ~154 KB worst case = **121 ms**, so a refresh occupies ~7 frame periods +whatever the decoder does; the decoder's own share (1,200 × 64 words = 76,800 = +31 ms) is spread by the same split. + +**call:** the fixed `KEYFRAME_EVERY = 60` cadence goes away. A refresh is forced +(a) at the first frame, (b) when the status frame's `drops` rises, and (c) on a +slow safety cadence of 3,600 frames (60 s at 60 fps). The reason to remove the +fixed cadence is arithmetic: one 31 ms refresh every second is 1.9 frames of +budget per second *and* 7 frames of wire, i.e. a dropped frame every 60 — the +hitch a viewer would notice. `keyframe_due` keeps its `drops` trigger, which is +the only repair the terminal can actually ask for. + +On a hard cut the whole screen changes, so the cut is a refresh and the picture +wipes over ~7 frame periods (117 ms). That is the price of dropping the +dictionary and it is stated here rather than discovered later. + +### 8.5 Pacing + +**call:** replace the fixed `--fps` clock with **wire-capacity pacing**: send a +frame whenever the ring has room and the previous frame's bytes have left, and +never send faster than 60 fps. Normal frames (1.9–5.3 KB) then arrive at 60 fps +with the link at 9–25 % duty; a refresh (or a cut) temporarily lowers the +delivered rate rather than overflowing the ring or dropping frames. `--fps` +stays as a ceiling so the demo can still be slowed for inspection. + +This matters because the failure mode of the fixed clock is silent: at 60 fps a +7-frame refresh cannot be sent, the ring fills, the chip sets `MISS`, the status +frame reports a rise in `drops`, and `keyframe_due` responds by… scheduling +another refresh. Capacity pacing removes the loop. + +### 8.6 What gets deleted + +* `tilecost.build_dictionary`, `quantise`, `_frame_tiles` and the whole k-means + path; the dictionary, its 32 KB, its 23-datagram upload, `DICT_READY`, + `DICT_BYTES`, `DICT_BAD`, `dict_chunk()`, `frag_copy()` (its only caller), + and the `DICT_CHUNK` kind. +* `OS_TOUCH` and the VRAM sentinel word (`SENT_ADDR`/`SENT_VALUE`), because the + OS is gone (§8.9). +* `TILES_DROP` survives, but now means "a frame failed validation" rather than + "no dictionary yet". +* The per-frame `vid_dint_` / `vid_eint_` pair: one `DINT` at startup, never + re-enabled. +* `CONVSP`-for-source (`PIXBLT B,XY` here has a constant source pitch of 32 + bits). `CONVDP`, `PSIZE` and `PMASK` still matter. + +Net effect: `vidstream.c` and `make_tiles.py` both get *smaller*, and the +encoder loses its only slow step. + +### 8.7 Encoder pipeline + +One pass, no intermediate frames: `ffmpeg` rawvideo → numpy → per-frame tile +diff → stream records. Reasons: `tilecost`/`make_tiles` are pure Python and +13,171 frames × 691,200 pixels is 9.1 G pixels (hours in the current code), and +materialising 13,171 × 153,600 B of `*.bin` frames is 2.0 GB of disk. + +Bit order stays **LSB-first within each byte** (`clip_from_video.py` and +`vidsrc.pack_row` both spell out why: an MSB-first packer is self-consistent and +mirrors every eight pixels on the display). + +**No dither.** A hard threshold is what makes exact deltas cheap; ordered or +error-diffused dither would make nearly every tile dirty every frame and the +wire cost would leave the budget. This is a design constraint, not a preference. + +`clip_from_video.py` stays the reference for the extraction front end +(upscale in grey, then threshold); its `--height` check already accepts 960. + +### 8.8 Failure behaviour + +* Bad header / bad length / descending positions / off-grid position / unknown + tag: refuse the frame, paint nothing, count it (`RX_BAD` / `TILES_DROP`), keep + the picture. Unchanged from today. +* Ring overrun: `MISS` is reported; capacity pacing (§8.5) is the prevention. +* A refresh interrupted (drops mid-refresh): the next `drops` rise restarts it; + the picture is mixed during the window and self-heals on completion. Say so in + the status frame — a `refresh_in_progress` bit is cheap and makes the mixed + window explainable instead of mysterious. + +### 8.9 The payload owns the machine + +**human:** the resident OS may die; it is already losing its network stack to +the LANCE takeover. + +What that removes: `OS_TOUCH` and the sentinel, the `dint`/`eint` burst +discipline, and any question of interleaving. + +What it obliges, and this is the part that must not be skipped: + +* **One `DINT` at startup, never re-enabled.** With `IE = 0` nothing maskable is + taken, so the OS's old ISRs are never entered against dead state. NMI remains + maskable-only-by-the-host (it needs a `HSTCTL` write) and is not a concern. +* **Video registers are inherited, verified and then owned.** Nothing will + re-program `DPYCTL`/`DPYSTRT`/`DPYADR`/`DPYINT`/`DUDATE`/`HTOTAL`… once the OS + is gone, and the TMS34010's own screen-refresh machinery runs on those values + forever. So the payload reads them at startup, publishes them in the marker + page for diagnosis, and re-asserts them only if a sanity check fails. It does + **not** hardcode a mode: the values that matter are the ones the OS just + programmed for the modelled monitor (`VT1200_MONITOR`, default VR319). + Consequence to state plainly: **a payload loaded from a machine that never + configured video has no picture**, and the demo's boot path is therefore + "boot the OS, wait for it, then take over" — as `demo.sh` already does. +* **Operational cost:** a payload that hangs with the OS dead takes the machine + down to a power cycle — no rdebug, no console. This matches the existing + "one demo per boot" note in `harness/badapple/README.md` and should be + repeated in it. +* The bar rows are painted once by the payload (2 × 1,280 × 32 pixels = 5,120 + destination words ≈ 2 ms) so the picture is not sitting on whatever the OS + left there. + +## 9. Budgets + +Unit rates for the blit path, MAME-labelled where the unit has never been +measured. `B,XY` is shown at both ends of its uncertainty. + +| case | tiles | dest words | CPU @0.32 µs/word | CPU @0.62 µs/word | wire | +|---|---|---|---|---|---| +| typical 60 fps delta (≈41 tiles) | 41 | 2,624 | 0.9 ms | 1.7 ms | 1.9–5.3 KB | +| delta, busy frame (≈150 tiles) | 150 | 9,600 | 3.2 ms | 6.1 ms | ~7–19 KB | +| refresh sub-frame (354 bitmaps) | 354 | 22,656 | 7.3 ms | 14.0 ms | ~46.4 KB | +| full picture (1280×960) | 1,200 | 76,800 | 24.6 ms | 47.6 ms | ~154 KB | + +Frame budgets: **41,700 words** and **21 KB**. So a typical delta uses ~6 % of +the CPU budget and ~10–25 % of the wire; a refresh sub-frame uses 18–35 % of the +CPU but **>100 % of one frame's wire**, which is why refresh is paced by the wire +and not by the clock (§8.5). + +## 10. Verification + +* **Pixel verdict, unchanged in spirit:** `tile_check.py` decodes the new + payloads independently and renders them against the payload's dumped VRAM. It + must show **exact** agreement frame for frame — with no dictionary there is no + reconstruction error to report, so a single differing bit is a bug rather than + a PSNR. +* **Host tests:** `harness/udpstream/tests/` keeps its role — the kind table is + mirrored in `frame.py` and `frame_parse.h` and `test_frame.py` compares them; + the new payload needs its own negative controls (count 0, count > 1,200, + descending position, off-grid position, unknown tag, tail length not + 128 × m). +* **MAME:** the no-env regression baseline + (`PC=3FDCE400 samples=199 (2 distinct PCs)`, `harness/mame/README.md`) must not + move; the tile modes (`VT1200_MODE=tiles|delta|replay`) are re-pointed at the + new payloads. +* **Unit:** `bench-sweep.sh` methodology plus the 3.5-minute soak; the new + `decode_worst` samples are the instrument. **The `B,XY` per-word rate must be + measured on the unit in the first milestone** — every CPU number above is an + extrapolation from `FILL`/`L,XY` until then. + +## 11. First task: measure before building + +Before any payload change, one host-side pass over `hd60.mp4` with numpy: + +1. extract at **1280×960, 60 fps, hard threshold** (the proposed front end); +2. per frame, diff against its predecessor and record **changed tiles of the + 40×30 picture grid**, **changed pixels**, **tags-only bytes**, **bitmap + bytes**, and the solid/bitmap split; +3. report mean / p50 / p95 / max wire per frame, the frames where wire exceeds + the 21 KB budget, and where the cuts are. + +This decides three things the design currently *assumes*: +* whether the solid-tag split makes the wire budget comfortable or marginal; +* whether 32×32 stays the tile size (a bitmap-heavy distribution would argue for + a smaller tile, a tag-heavy one for a larger); +* how many refreshes the clip actually needs, i.e. how visible the cut wipes are. + +It also settles the extractor question (§4, cause 1) by producing the frames +that the picture will actually contain. + +## 12. Risks + +| risk | why it matters | mitigation | +|---|---|---| +| `B,XY` rate is a MAME number | every CPU estimate here; the 12 ms MAME repaint contradicts the unit's 31 ms | measure `B,XY` on the unit first (§10); the design is wire-bound on deltas so a 2× CPU surprise costs little | +| refresh is 7 frames of wire | cuts wipe for ~117 ms | capacity pacing; keep the dictionary as a *refresh-only* accelerator (approach B) if it looks bad | +| a cut needs a full refresh | all 1,200 bitmaps, 154 KB | tags collapse the mostly-black ones; if not, approach B | +| killing the OS is irreversible | hang = power cycle; no video if video was never configured | keep the boot path `demo.sh` uses; publish the video registers; never hardcode a mode | +| `payload_len` u16 and a 32-descriptor ring cap a frame at ~355 bitmaps | a naive full-frame refresh cannot be expressed | sub-frames (§8.4); optional 64-descriptor ring if it ever binds | +| 60 fps on a 72 Hz panel | every frame is 1.2 refresh periods, the tear line drifts | accepted (§8.7 of the 2026-10-30 doc); revisit with a start-of-frame delay | +| encoder is pure Python today | 9.1 G pixels | numpy or C, one pass, no intermediate frames (§8.7) | +| the jaggies are not the dictionary | rebuilding the codec would not fix them | §4's three cheap checks run first | + +## 13. Left to the builder + +* Ring depth: 32 descriptors with 4-way refresh split, or 64 with a wider slot. +* Whether solid tiles use a constant bitmap and `PIXBLT`, or `FILL`. +* Exact `CONTROL` mask and implied-operand setup; how much of it is hoisted and + where the hoist lives (a `_tile_init_` in assembly, or C writing the I/O + registers once). +* The status body's new fields (a `refresh_in_progress` bit, and whether + `decode_worst` should now be reported per kind). +* Whether the encoder keeps `make_tiles.py`'s cache files + (`dictionary.bin`/`indices.bin`/`stream.bin`/`manifest.json`) or moves to a + single stream file. +* How `tile_check.py` reports "exact" (a bit count of zero, or a hash). +* Whether the bars are bottom-aligned (960 rows at the top, §2) or split 32/32. +* Whether the old strip path (`KIND_STRIP`, `paint()`, `vram_b.asm`) is kept as a + black-screen/bring-up test or deleted with the rest of the tile machinery. + Keeping it costs nothing and it is already verified; deleting it removes a + second decoder path. +* `vidsend.py`'s `min_chunk` check exists to stop a frame being split across more + than 32 descriptors; it must be recomputed from the new maximum frame size. + +## 14. Decisions from the human partner (this session) + +* **The output geometry is upscaled to fill the panel**, not 1:1 with borders. +* **The resident OS may die** — it already loses its network stack to the + LANCE takeover. +* **60 fps is the target**, and the source in `/tmp/opencode/badapple-real/` is + the one to use. +* "We can change everything" — the wire format, the encoder and the payload are + all in scope. + +Carried over from 2026-10-30 §14 and still binding: *"30 fps is just best case if +there is enough budget. Real fps can drop."* So a rate that falls below 60 fps in +busy sections is acceptable as long as the picture stays legible and the drop is +a consequence of the content, not of a stall. + +## 15. Open questions + +1. Is the panel's pixel grid square? If it is not, 1280×960 is a subtly wrong + aspect and the 1024-row stretch becomes the right answer. +2. Does the terminal's Ethernet path ever deliver at more than the measured + 1.27 MB/s — is the 10 Mbit figure the NIC or the hub? +3. Should the demo keep any input at all (a stop key), which would argue for + keeping one ISR alive rather than `DINT` forever?