arti/vt1200
1
0 Fork 1
mirror of https://git.k-space.ee/arti/vt1200.git synced 2026-11-01 12:14:56 +02:00
Code Wiki Activity

spec: Bad Apple at 60 fps exact — raw tagged tiles, no dictionary

Browse Source
This commit is contained in:
arti committed 2026-10-31 21:54:48 +02:00
1 parent d6aa6f46c1
commit 650ec9a0cd
1 file changed
+514
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/specs/2026-10-31-badapple-60p-design.md
+514
View File
@@ -0,0 +1,514 @@
# Bad Apple at 60 fps, exact — design
Date: 2026-10-31
Status: **rev 1** — first draft, awaiting approval.
Author: (session). Decisions marked "**call:**" are the builder's proposals, not
something the human partner has agreed to. Decisions marked "**human:**" were
given in this session and are not open.
Supersedes the tile-codec half of
[`2026-10-30-vt1200-video-streaming-design.md`](2026-10-30-vt1200-video-streaming-design.md)
(§8.3 scheme C, §8.9 OS interference, §10 M3–M5). The transport there (§8.1), the
back-channel (§8.4) and the delivery path (`harness/badapple/`) stand.
---
## 1. Intent
Play **Bad Apple!!** on a VT1200 from
`/tmp/opencode/badapple-real/hd60.mp4` at the source's **60 fps**, upscaled to
fill the panel, with the picture **no longer quantised through a 256-entry tile
dictionary** — because that dictionary, not the resolution, is what makes the
diagonals jagged.
The ROM stays byte-identical (`md5 b46a3e442ccb86134d2356bd70d22fd2`). The OS is
allowed to die (§8.9 of this document).
## 2. Grounding: the source
`ffprobe /tmp/opencode/badapple-real/hd60.mp4` (human-supplied, this session):
| field | value |
|---|---|
| video | 960x720, SAR 1:1, DAR 4:3, **60 fps**, progressive, yuv420p, h264 Main |
| duration | 00:03:39.52 → **≈13,171 frames** at 60 fps |
| audio | aac LC stereo (ignored — no audio path in this harness) |
960×720 is 4:3 exactly, and 1280×960 is 4:3 exactly, so an upscale to
1280×960 is a clean 1.3333× with no crop and no aspect error. 960 is also a
multiple of 32, so `clip_from_video.py`'s default `--height 960` is already
legal (its `height % 32` check at line 98 rejects 720, not 960).
**human:** deliver it upscaled to fill the panel rather than 1:1 with borders.
**call:** the target geometry is **1280×960 at the top of the screen**, rows
960–1023 a permanently black bar. Stretching 960 rows to 1024 (a further 6.7 %
in Y) is a one-flag alternative if the bars look wrong on the unit; it changes
no other number in this document except the bar tiles.
## 3. Grounding: what exists today
All paths below are in this repository.
| piece | where | what it does |
|---|---|---|
| payload | `harness/udpstream/vidstream.c` | takes the LANCE over, arms a 32-descriptor RX ring in two 48 KB slots, parses VS frames, dispatches to `paint` / `paint_tiles` / `paint_delta` / `dict_chunk`, sends a status frame every 64 accepted frames |
| tile primitive | `harness/udpstream/vram_tile.asm` | one `PIXBLT B,XY` of a 32×32 dictionary entry; ~35 instructions of prologue per call |
| strip primitive | `harness/udpstream/vram_b.asm` | one `PIXBLT B,XY` of a row with `SPTCH=0` vertical stretch |
| fill primitive | `harness/udpstream/vram.asm` | `FILL L` of the whole screen (fixed geometry) |
| wire format | `harness/udpstream/frame.py` + `frame_parse.h` | 16-byte `VS` header, header-only CRC-16, kinds 1–5 |
| tile wire form | `harness/udpstream/tiledict.py` | `TILES_FULL` = 1,280 index bytes; `TILES_DELTA` = `u16 count` + `count×u16 pos` + `count×u8 index`; `KEYFRAME_EVERY = 60` |
| dictionary build | `harness/udpstream/tilecost.py` | Lloyd/k-means over 32×32 tiles, Hamming distance, 256 entries × 128 B |
| encoder | `harness/udpstream/make_tiles.py` | clip → dictionary + indices + stream plan + manifest |
| sender | `harness/udpstream/vidsend.py` | paces at `--fps`, reads status frames, forces a keyframe when `drops` rises |
| checker | `harness/mame/tile_check.py` | decodes `stream.bin` independently, renders it, compares against dumped VRAM |
| demo | `harness/badapple/demo.sh` | waits for the OS's stack, loads the payload over rdebug, sends the clip |
Measured behaviour of today's codec, all from `harness/mame/NETWORK_REPORT.md`:
| quantity | value | where |
|---|---|---|
| synthetic clip, 30 frames | PSNR **29.47 dB**, 1.16 bits/tile mean error, 97.0 % of tiles exact | §76.4 |
| **real clip** (3,292 frames, 15 fps, 1280×960) | **PSNR 24.4 dB**, mean delta **496 B**, 55 keyframes, 1.63 MB stream | `harness/badapple/README.md` |
| one full tile repaint in MAME | `decode_worst` **37,612 REFCNT ticks** (1,280 tiles) | §76.5.1 |
## 4. The diagnosis
**The jaggies are the dictionary, and they can be counted.** 24.4 dB is a
bit-error rate of 0.0036, i.e. **≈4,760 wrong pixels per frame** out of
1,310,720. A 32×32 tile has on the order of 2³² distinct edge placements and the
dictionary has 256 centroids, so the errors concentrate on edges — which is
exactly what a viewer sees as ragged diagonals. At 1280 wide there is no
resolution left to buy; the fragment is the *edge vocabulary*.
Three other causes look identical on screen and must be ruled out before the
codec is blamed. All are cheap:
1. `clip_from_video.py` line 49/64: `scale=...:flags=lanczos` then `gray >
threshold`. Hard-thresholding a downscaled frame makes its own staircase.
Dump one `.bin` as an image and look.
2. `VT1200_MONITOR=VR150` shears the picture (`harness/mame/README.md`).
3. A source frame whose edges are already stepped (e.g. a 4:3 crop of a
differently-scaled master).
**And the speed is not the constraint.** At the real clip's rate of change a
delta is ~165 tiles at 15 fps, so ~41 tiles per frame at 60 fps (motion per
frame divides by four). Against the budgets in §6 that is roughly **1/20th of
one frame**. There is no speed problem to solve in the delta path; there is
spare budget to spend on exactness.
## 5. What the hardware actually charges for
The single most important number in this document, measured on the unit and
recorded in the 2026-10-30 design (§2, §6):
> `FILL L` costs **0.40 µs per destination word**; `PIXBLT L,XY` costs
> **0.62 µs/word**; each blit/fill *call* costs **2.6–3.3 µs** on top.
A destination word is 16 pixels at 1 bpp. So **the pixel engine's cost is
proportional to the number of destination words written, and essentially
independent of which primitive writes them and of how the writes are grouped.**
Consequences, all of which this design depends on:
* A 32×32 tile is 32 rows × 2 words = **64 destination words**.
* A 48×48 tile is 48 × 3 = 144 words; a 64×64 is 256. Per *pixel* there is no
gain — the extra call overhead saved (0.8 ms vs 3.1 ms over a full screen) is
under 10 % of a repaint. **Tile size is a wire decision, not a CPU decision.**
* `FILL` and `PIXBLT` cost the same per word, so collapsing a run of solid tiles
into one `FILL` saves **call overhead and wire bytes only**, not blitter time.
* The whole 1280×960 picture is 960 × 80 = **76,800 destination words**.
At 0.40 µs/word that is **≈31 ms** — *more than one 60 fps frame*, whatever
scheme drives it.
Note the contradiction this exposes: MAME reports a full tile repaint at
~12 ms (37,612 REFCNT ticks, §3) while the unit's `FILL` rate says the same
pixels cost ~31 ms. `PIXBLT B,XY` at "0.32 µs/word" is a **MAME benchmark and
has never been measured on the unit** (the 2026-10-30 doc says so in its own
risk table). Every per-frame CPU number in this document is therefore labelled
either **unit** or **MAME**, and the first milestone measures the gap.
## 6. Budget
| quantity | value | source |
|---|---|---|
| frame period at 60 fps | **16.67 ms** | derived |
| display refresh | 72 Hz (13.9 ms) | 2026-10-30 §2 |
| destination words affordable per frame | **41,700** at 0.40 µs/word | derived from the unit rate |
| full picture (1280×960) | 76,800 words ≈ **31 ms** = **1.84 frames** | derived, unit rate |
| link | 10 Mbit half-duplex, **1.27 MB/s** measured | 2026-10-30 §2 |
| wire per frame at 60 fps | **21 KB** | derived |
| RX ring | 32 descriptors × 1526 B = 48,832 B (one slot) | `vidstream.c` |
| `payload_len` | u16 → a frame's payload is ≤ 65,535 B | `frame_parse.h` |
| heap | ~2.7 MB free at takeover | status frames |
Two hard limits follow: **a frame's payload cannot exceed 65,535 B**
(`payload_len` is 16 bits) and **cannot exceed 32 datagrams** (~45.9 KB of
payload) without deepening the RX ring. The binding one is the ring.
**The central finding: at 60 fps a full-screen repaint fits neither budget.**
It costs 1.84 frames of CPU and ~7 frames of wire (154 KB at 1.27 MB/s =
121 ms). Everything in the design below exists to make sure the full screen is
never repainted in one frame, and that ordinary frames are exact.
## 7. Approaches considered
**A. Keep the dictionary; widen the index to 10–12 bits.** Gets PSNR up by
increasing entries (256 → 1,024+). Wire: 2 B/tile. Cost: a 92-datagram upload
(the current 32-descriptor ring holds 23), plus the dictionary still cannot
represent every edge. **Rejected as the primary path**: it improves the same
approximation that causes the complaint, and it keeps the k-means encoder and
the 32 KB upload.
**B. Keep the dictionary; add an 8×8 residual patch pass.** Exactness with
bounded cost (patches only where error > 0). **Not rejected, deferred**: it is
the best option if raw bitmaps turn out too expensive on the wire, and it reuses
everything. Kept as the fallback in §12.
**C. Raw tiles — send the changed tiles' actual bits, with a solid tag.**
No dictionary at all. Exact by construction. Wire: 1 bit per changed pixel plus
a 1-byte tag per changed tile; a solid tile costs 1 byte instead of 128.
**Chosen.**
**D. Spans / vectors (`FILL XY` per scanline run).** Exact and the same
per-word cost as everything else, so its only advantage is that it *writes fewer
words* on flat content and its only cost is that it turns dither into runs of
length 1. Bad Apple is full of dithered texture (grass, hair, checkerboard), so
a span encoding explodes exactly where tiles are cheap. **Rejected for this
source**; it stays the right answer for a flat-shaded stylisation, which is a
different film.
**Decision (this document): C.** It is also the *simplest* option — it deletes
more code than it adds (§8.6).
## 8. Design
### 8.1 Geometry
1280×960 at the top of the panel: **40 × 30 tiles of 32×32** = 1,200 picture
tiles, plus 64 permanently black bar tiles (grid rows 30–31) that are painted
once and never listed again. The grid, `TILES_SCREEN`, the row-major position
encoding, `tile_check.py`'s renderer and `vram_tile.asm`'s `DYDX=(32,32)` are all
unchanged — the smallest possible diff to the geometry.
32×32 rather than 64×64 because tile size affects CPU cost by under 10 % (§5)
and affects wire cost a great deal: a solid 32×32 costs one tag, and a changed
32×32 patch costs 128 B against 512 B for 64×64. Finer tiles waste less overdraw
on an edge. **call:** 32×32.
### 8.2 Wire form
`KIND_TILES_FULL` and `KIND_TILES_DELTA` keep their numbers (2, 3) so the kind
table and the existing tests stay meaningful; their **payloads change**, and
`VERSION` goes to 2 so an old sender cannot be mis-dispatched.
```
payload:
offset size field
0 2 count, u16 little-endian, 1..1,280
2 2*count position[k], u16, row-major tile index, STRICTLY ASCENDING
2+2c count tag[k], u8:
0x00 = this tile is solid black
0x01 = this tile is solid white
0x02 = a 128-byte bitmap follows
2+3c m*128 bitmaps for the tags equal to 0x02, in k order
payload_len == 4 + 3*count + 128*m
```
Grouped rather than interleaved (positions, then tags, then bitmaps) so the
receiver can validate **the whole list without reading a single bitmap**:
count in range, length consistent with the count, every position on the grid and
strictly ascending, every tag in {0,1,2}, and the tail length exactly
128 × (number of `0x02` tags). That preserves the property the harness's
off-grid negative control exists to protect — *one bad entry paints nothing* —
and it is `O(count)` with one cursor and no division.
`TILES_FULL` is the same form with `count == 1,200` (all picture tiles); the bar
tiles are never sent.
**Positions refer to the 40×30 picture grid**: 0..1,199, row-major, with
`ty = pos / 40` avoided by incrementing as today. That is a *change* to the wire
constant `FP_TILES_SCREEN`/`TILES_PER_SCREEN` (1,280 → 1,200) and to
`tile_check.py`'s renderer, and the bar rows are outside the wire entirely —
which is the one place this design is not a minimal diff, so it is stated rather
than implied.
**Expected wire, estimated from the one real-clip measurement** (496 B mean
delta at 15 fps ≈ 165 changed tiles → ~41 tiles per frame at 60 fps): if a third
of the changed tiles need a bitmap, ≈1.9 KB/frame ≈ **115 KB/s at 60 fps**, i.e.
9 % of the link. If every one needs a bitmap, 5.3 KB/frame ≈ 320 KB/s, 25 % of
the link. **The measurement in §11 replaces both numbers before any code is
written.**
### 8.3 Decoder
Per frame, unchanged from today's structure (§76's `paint_delta`): two passes,
validate then paint, inside one burst.
Paint per tag:
* `0x00` / `0x01` → one `PIXBLT B,XY` from a **64-word** all-zero or all-ones
constant in the payload's pool, `DYDX=(32,32)`. (A `FILL` is the same cost —
§5 — and `PIXBLT` keeps one primitive and one set of implied operands.)
* `0x02` → one `PIXBLT B,XY` from the bitmap in the frame itself (it is already
in a slot, bit address `rx_frag_ptr + offset*8`; no copy).
**Optimisation carried over from the earlier analysis, and it is the only one
that matters:** hoist `PSIZE`, `CONVSP`, `CONVDP`, `PMASK`, `CONTROL`, `B1`,
`B3`, `B7`, `B8`, `B9` out of the per-tile primitive into a per-frame (really
per-run) setup, and delete the eight push/pop pairs in `vram_tile.asm`. Only
`B0` (SADDR) and `B2` (DADDR) vary per tile, and `PIXBLT` destroys both. This is
a small constant on top of the per-word cost, so it is worth doing once and not
worth designing around.
**Decoder state — the encoder's shadow screen.** Every delta must be computed
against *what the decoder currently shows*, not against the source's previous
frame, because a rolling refresh (§8.4) makes those two different: after a cut
the decoder holds the old picture with a few hundred tiles replaced. So
`vidsend.py` keeps a **shadow index buffer** of the decoder's screen, updated as
each frame is sent and marked "everything stale" when a refresh begins, and
computes every delta against it. Today's `plan()` diffs `previous` and assumes it
arrived; that is already false after a drop and would be routinely false under a
rolling refresh. Small change in the sender, a *precondition* for a correct
picture.
### 8.4 Refresh
A full refresh is **not** one frame. Split into sub-frames of at most **354
bitmaps**: a frame must fit 32 datagrams of 1,450-byte payloads, so
`4 + 3c + 128m ≤ 32 × 1,450 = 46,400` with `m = c` gives `c ≤ 354` — fewer when
the tags make it smaller. (The u16 `payload_len` would allow 500; the ring binds
first.) The sub-frames go out across consecutive frames.
**Bitmaps must not straddle a datagram.** A 128-byte bitmap lands mid-datagram
for most payload sizes, and a split bitmap would need bit-level reassembly across
fragments — per-byte CPU work, which is an anti-goal. **call:** the sender ends
each datagram at a tile boundary, wasting at most 127 bytes of a 1,450-byte
payload (≤ 9 % of the wire) and leaving the decoder a single fragment arithmetic
per bitmap.
**The first refresh is a boot sequence, not a repair.** Before it the picture is
undefined, so painting it progressively is a wipe-in on a black screen. Every
later refresh replaces a picture already on screen, which is why §12 treats the
cut wipe as a real risk. On the wire a refresh
is ~154 KB worst case = **121 ms**, so a refresh occupies ~7 frame periods
whatever the decoder does; the decoder's own share (1,200 × 64 words = 76,800 =
31 ms) is spread by the same split.
**call:** the fixed `KEYFRAME_EVERY = 60` cadence goes away. A refresh is forced
(a) at the first frame, (b) when the status frame's `drops` rises, and (c) on a
slow safety cadence of 3,600 frames (60 s at 60 fps). The reason to remove the
fixed cadence is arithmetic: one 31 ms refresh every second is 1.9 frames of
budget per second *and* 7 frames of wire, i.e. a dropped frame every 60 — the
hitch a viewer would notice. `keyframe_due` keeps its `drops` trigger, which is
the only repair the terminal can actually ask for.
On a hard cut the whole screen changes, so the cut is a refresh and the picture
wipes over ~7 frame periods (117 ms). That is the price of dropping the
dictionary and it is stated here rather than discovered later.
### 8.5 Pacing
**call:** replace the fixed `--fps` clock with **wire-capacity pacing**: send a
frame whenever the ring has room and the previous frame's bytes have left, and
never send faster than 60 fps. Normal frames (1.9–5.3 KB) then arrive at 60 fps
with the link at 9–25 % duty; a refresh (or a cut) temporarily lowers the
delivered rate rather than overflowing the ring or dropping frames. `--fps`
stays as a ceiling so the demo can still be slowed for inspection.
This matters because the failure mode of the fixed clock is silent: at 60 fps a
7-frame refresh cannot be sent, the ring fills, the chip sets `MISS`, the status
frame reports a rise in `drops`, and `keyframe_due` responds by… scheduling
another refresh. Capacity pacing removes the loop.
### 8.6 What gets deleted
* `tilecost.build_dictionary`, `quantise`, `_frame_tiles` and the whole k-means
path; the dictionary, its 32 KB, its 23-datagram upload, `DICT_READY`,
`DICT_BYTES`, `DICT_BAD`, `dict_chunk()`, `frag_copy()` (its only caller),
and the `DICT_CHUNK` kind.
* `OS_TOUCH` and the VRAM sentinel word (`SENT_ADDR`/`SENT_VALUE`), because the
OS is gone (§8.9).
* `TILES_DROP` survives, but now means "a frame failed validation" rather than
"no dictionary yet".
* The per-frame `vid_dint_` / `vid_eint_` pair: one `DINT` at startup, never
re-enabled.
* `CONVSP`-for-source (`PIXBLT B,XY` here has a constant source pitch of 32
bits). `CONVDP`, `PSIZE` and `PMASK` still matter.
Net effect: `vidstream.c` and `make_tiles.py` both get *smaller*, and the
encoder loses its only slow step.
### 8.7 Encoder pipeline
One pass, no intermediate frames: `ffmpeg` rawvideo → numpy → per-frame tile
diff → stream records. Reasons: `tilecost`/`make_tiles` are pure Python and
13,171 frames × 691,200 pixels is 9.1 G pixels (hours in the current code), and
materialising 13,171 × 153,600 B of `*.bin` frames is 2.0 GB of disk.
Bit order stays **LSB-first within each byte** (`clip_from_video.py` and
`vidsrc.pack_row` both spell out why: an MSB-first packer is self-consistent and
mirrors every eight pixels on the display).
**No dither.** A hard threshold is what makes exact deltas cheap; ordered or
error-diffused dither would make nearly every tile dirty every frame and the
wire cost would leave the budget. This is a design constraint, not a preference.
`clip_from_video.py` stays the reference for the extraction front end
(upscale in grey, then threshold); its `--height` check already accepts 960.
### 8.8 Failure behaviour
* Bad header / bad length / descending positions / off-grid position / unknown
tag: refuse the frame, paint nothing, count it (`RX_BAD` / `TILES_DROP`), keep
the picture. Unchanged from today.
* Ring overrun: `MISS` is reported; capacity pacing (§8.5) is the prevention.
* A refresh interrupted (drops mid-refresh): the next `drops` rise restarts it;
the picture is mixed during the window and self-heals on completion. Say so in
the status frame — a `refresh_in_progress` bit is cheap and makes the mixed
window explainable instead of mysterious.
### 8.9 The payload owns the machine
**human:** the resident OS may die; it is already losing its network stack to
the LANCE takeover.
What that removes: `OS_TOUCH` and the sentinel, the `dint`/`eint` burst
discipline, and any question of interleaving.
What it obliges, and this is the part that must not be skipped:
* **One `DINT` at startup, never re-enabled.** With `IE = 0` nothing maskable is
taken, so the OS's old ISRs are never entered against dead state. NMI remains
maskable-only-by-the-host (it needs a `HSTCTL` write) and is not a concern.
* **Video registers are inherited, verified and then owned.** Nothing will
re-program `DPYCTL`/`DPYSTRT`/`DPYADR`/`DPYINT`/`DUDATE`/`HTOTAL`… once the OS
is gone, and the TMS34010's own screen-refresh machinery runs on those values
forever. So the payload reads them at startup, publishes them in the marker
page for diagnosis, and re-asserts them only if a sanity check fails. It does
**not** hardcode a mode: the values that matter are the ones the OS just
programmed for the modelled monitor (`VT1200_MONITOR`, default VR319).
Consequence to state plainly: **a payload loaded from a machine that never
configured video has no picture**, and the demo's boot path is therefore
"boot the OS, wait for it, then take over" — as `demo.sh` already does.
* **Operational cost:** a payload that hangs with the OS dead takes the machine
down to a power cycle — no rdebug, no console. This matches the existing
"one demo per boot" note in `harness/badapple/README.md` and should be
repeated in it.
* The bar rows are painted once by the payload (2 × 1,280 × 32 pixels = 5,120
destination words ≈ 2 ms) so the picture is not sitting on whatever the OS
left there.
## 9. Budgets
Unit rates for the blit path, MAME-labelled where the unit has never been
measured. `B,XY` is shown at both ends of its uncertainty.
| case | tiles | dest words | CPU @0.32 µs/word | CPU @0.62 µs/word | wire |
|---|---|---|---|---|---|
| typical 60 fps delta (≈41 tiles) | 41 | 2,624 | 0.9 ms | 1.7 ms | 1.9–5.3 KB |
| delta, busy frame (≈150 tiles) | 150 | 9,600 | 3.2 ms | 6.1 ms | ~7–19 KB |
| refresh sub-frame (354 bitmaps) | 354 | 22,656 | 7.3 ms | 14.0 ms | ~46.4 KB |
| full picture (1280×960) | 1,200 | 76,800 | 24.6 ms | 47.6 ms | ~154 KB |
Frame budgets: **41,700 words** and **21 KB**. So a typical delta uses ~6 % of
the CPU budget and ~10–25 % of the wire; a refresh sub-frame uses 18–35 % of the
CPU but **>100 % of one frame's wire**, which is why refresh is paced by the wire
and not by the clock (§8.5).
## 10. Verification
* **Pixel verdict, unchanged in spirit:** `tile_check.py` decodes the new
payloads independently and renders them against the payload's dumped VRAM. It
must show **exact** agreement frame for frame — with no dictionary there is no
reconstruction error to report, so a single differing bit is a bug rather than
a PSNR.
* **Host tests:** `harness/udpstream/tests/` keeps its role — the kind table is
mirrored in `frame.py` and `frame_parse.h` and `test_frame.py` compares them;
the new payload needs its own negative controls (count 0, count > 1,200,
descending position, off-grid position, unknown tag, tail length not
128 × m).
* **MAME:** the no-env regression baseline
(`PC=3FDCE400 samples=199 (2 distinct PCs)`, `harness/mame/README.md`) must not
move; the tile modes (`VT1200_MODE=tiles|delta|replay`) are re-pointed at the
new payloads.
* **Unit:** `bench-sweep.sh` methodology plus the 3.5-minute soak; the new
`decode_worst` samples are the instrument. **The `B,XY` per-word rate must be
measured on the unit in the first milestone** — every CPU number above is an
extrapolation from `FILL`/`L,XY` until then.
## 11. First task: measure before building
Before any payload change, one host-side pass over `hd60.mp4` with numpy:
1. extract at **1280×960, 60 fps, hard threshold** (the proposed front end);
2. per frame, diff against its predecessor and record **changed tiles of the
40×30 picture grid**, **changed pixels**, **tags-only bytes**, **bitmap
bytes**, and the solid/bitmap split;
3. report mean / p50 / p95 / max wire per frame, the frames where wire exceeds
the 21 KB budget, and where the cuts are.
This decides three things the design currently *assumes*:
* whether the solid-tag split makes the wire budget comfortable or marginal;
* whether 32×32 stays the tile size (a bitmap-heavy distribution would argue for
a smaller tile, a tag-heavy one for a larger);
* how many refreshes the clip actually needs, i.e. how visible the cut wipes are.
It also settles the extractor question (§4, cause 1) by producing the frames
that the picture will actually contain.
## 12. Risks
| risk | why it matters | mitigation |
|---|---|---|
| `B,XY` rate is a MAME number | every CPU estimate here; the 12 ms MAME repaint contradicts the unit's 31 ms | measure `B,XY` on the unit first (§10); the design is wire-bound on deltas so a 2× CPU surprise costs little |
| refresh is 7 frames of wire | cuts wipe for ~117 ms | capacity pacing; keep the dictionary as a *refresh-only* accelerator (approach B) if it looks bad |
| a cut needs a full refresh | all 1,200 bitmaps, 154 KB | tags collapse the mostly-black ones; if not, approach B |
| killing the OS is irreversible | hang = power cycle; no video if video was never configured | keep the boot path `demo.sh` uses; publish the video registers; never hardcode a mode |
| `payload_len` u16 and a 32-descriptor ring cap a frame at ~355 bitmaps | a naive full-frame refresh cannot be expressed | sub-frames (§8.4); optional 64-descriptor ring if it ever binds |
| 60 fps on a 72 Hz panel | every frame is 1.2 refresh periods, the tear line drifts | accepted (§8.7 of the 2026-10-30 doc); revisit with a start-of-frame delay |
| encoder is pure Python today | 9.1 G pixels | numpy or C, one pass, no intermediate frames (§8.7) |
| the jaggies are not the dictionary | rebuilding the codec would not fix them | §4's three cheap checks run first |
## 13. Left to the builder
* Ring depth: 32 descriptors with 4-way refresh split, or 64 with a wider slot.
* Whether solid tiles use a constant bitmap and `PIXBLT`, or `FILL`.
* Exact `CONTROL` mask and implied-operand setup; how much of it is hoisted and
where the hoist lives (a `_tile_init_` in assembly, or C writing the I/O
registers once).
* The status body's new fields (a `refresh_in_progress` bit, and whether
`decode_worst` should now be reported per kind).
* Whether the encoder keeps `make_tiles.py`'s cache files
(`dictionary.bin`/`indices.bin`/`stream.bin`/`manifest.json`) or moves to a
single stream file.
* How `tile_check.py` reports "exact" (a bit count of zero, or a hash).
* Whether the bars are bottom-aligned (960 rows at the top, §2) or split 32/32.
* Whether the old strip path (`KIND_STRIP`, `paint()`, `vram_b.asm`) is kept as a
black-screen/bring-up test or deleted with the rest of the tile machinery.
Keeping it costs nothing and it is already verified; deleting it removes a
second decoder path.
* `vidsend.py`'s `min_chunk` check exists to stop a frame being split across more
than 32 descriptors; it must be recomputed from the new maximum frame size.
## 14. Decisions from the human partner (this session)
* **The output geometry is upscaled to fill the panel**, not 1:1 with borders.
* **The resident OS may die** — it already loses its network stack to the
LANCE takeover.
* **60 fps is the target**, and the source in `/tmp/opencode/badapple-real/` is
the one to use.
* "We can change everything" — the wire format, the encoder and the payload are
all in scope.
Carried over from 2026-10-30 §14 and still binding: *"30 fps is just best case if
there is enough budget. Real fps can drop."* So a rate that falls below 60 fps in
busy sections is acceptable as long as the picture stays legible and the drop is
a consequence of the content, not of a stall.
## 15. Open questions
1. Is the panel's pixel grid square? If it is not, 1280×960 is a subtly wrong
aspect and the 1024-row stretch becomes the right answer.
2. Does the terminal's Ethernet path ever deliver at more than the measured
1.27 MB/s — is the 10 Mbit figure the NIC or the hub?
3. Should the demo keep any input at all (a stop key), which would argue for
keeping one ISR alive rather than `DINT` forever?