forked from arti/vt1200
plan: tile codec (scheme C) — dictionary, tile blit, deltas, bad apple
This commit is contained in:
opencode.energia.arti.ee committed
1 parent
22c539bf74
commit
cd4dbe2317
1 file changed
+253
@@ -0,0 +1,253 @@
|
||||
# VT1200 video streaming — tile codec Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace row strips with scheme C — a 256-entry dictionary of 32×32 1-bpp tiles and a stream of tile indices plus deltas — so a full-screen frame costs ~1.3 KB on the wire and only changed tiles are redrawn.
|
||||
|
||||
**Architecture:** The host quantises video frames into 32×32 tiles, uploads the dictionary once in chunks, then sends index bytes (full frames) and changed-tile lists (deltas). The terminal stores the dictionary in its malloc'd pool and paints each accepted tile with one `PIXBLT B,XY` straight into VRAM, reusing the transport, parser, slot ring and status frames that plan 1 built.
|
||||
|
||||
**Tech Stack:** Python 3 host encoder; C (1987 TI TMS34010 SDK, K&R) + TMS34010 asm payload; MAME 0.289 + Lua; the DOSBox build wrapper from plan 1.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-10-30-vt1200-video-streaming-design.md` (§7 C, §8.3, §8.4)
|
||||
|
||||
**Depends on:** `docs/superpowers/plans/2026-10-30-vt1200-stream-transport-and-row-strips.md` (complete)
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- ROM byte-identical (`md5 b46a3e442ccb86134d2356bd70d22fd2`); payloads arrive over the network or the debugger.
|
||||
- Bit addresses everywhere in payload code; `malloc` takes bytes, returns a bit pointer.
|
||||
- Payload C is K&R: no ANSI prototypes, no `//`, no declarations after statements.
|
||||
- Frame CRC covers the **12 header bytes only** (ruling R14); body integrity is the Ethernet FCS.
|
||||
- Tiles are **32×32 1 bpp**: 32 px = **two** 16-bit words per row, `SPTCH=32` bits, one `PIXBLT B,XY` per tile with `DYDX=(32,32)`.
|
||||
- A full-screen repaint is 1,280 tiles; the ring caps one frame at ~144 rows of strip data, so tile frames must stay ≤ 24 KB (they are ~1.3 KB).
|
||||
- No CPU copy of pixel data; the blitter moves every output pixel.
|
||||
- The no-env MAME baseline must not move: `PROG_FRAMES=200` → `PC=3FDCE400 samples=198 (3 distinct PCs)`.
|
||||
- Shared repo: stage only your own paths (`git add <path>`), never `git add -A`.
|
||||
- The unit run needs the physical terminal; it is human-assisted and recorded as such.
|
||||
|
||||
## Review Focus
|
||||
|
||||
- A tile index above 255 (or a dictionary that was never uploaded) must not paint garbage or read outside the pool.
|
||||
- A partial delta must never leave a region showing stale content from two different frames without a keyframe to repair it.
|
||||
- A keyframe must arrive after N frames or after a reported drop, whichever comes first, or a lost delta desynchronises the screen forever.
|
||||
- The tile blit must not write outside its 32×32 destination (a wrong `DYDX` or `SPTCH` smears a row across the screen).
|
||||
- The dictionary upload must be atomic from the decoder's point of view: no frame may be painted against a half-uploaded dictionary.
|
||||
|
||||
---
|
||||
|
||||
## File structure
|
||||
|
||||
| path | responsibility |
|
||||
|---|---|
|
||||
| `harness/udpstream/tilecost.py` | k-means tile dictionary builder + frame quantiser (host) |
|
||||
| `harness/udpstream/tests/test_tilecost.py` | dictionary determinism, index-range, reconstruction error |
|
||||
| `harness/udpstream/tiledict.py` | dictionary upload protocol (chunking, offsets) and delta frame encoding |
|
||||
| `harness/udpstream/tests/test_tiledict.py` | chunk layout, delta list, keyframe cadence |
|
||||
| `harness/udpstream/vram_tile.asm` | `PIXBLT B,XY` 32×32 helper, `SPTCH=32` |
|
||||
| `harness/udpstream/vidstream.c` | dictionary storage, `KIND_DICT_CHUNK`/`TILES_FULL`/`TILES_DELTA` dispatch |
|
||||
| `harness/mame/vidstream.lua` | `dict` and `tiles` modes |
|
||||
| `harness/mame/tile_check.py` | compare a VRAM dump against a host-rendered reference frame |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Dictionary builder and quantiser
|
||||
|
||||
**Files:**
|
||||
- Create: `harness/udpstream/tilecost.py`, `harness/udpstream/tests/test_tilecost.py`
|
||||
|
||||
**Interfaces:**
|
||||
- `build_dictionary(frames: list[bytes], k: int = 256, seed: int = 0) -> bytes` — returns `k` tiles of 128 bytes each (32×32 1 bpp, row-major, two bytes per row), ordered deterministically.
|
||||
- `quantise(frame: bytes, dictionary: bytes) -> bytes` — returns 1,280 index bytes (40×32 grid, row major), choosing for each tile the dictionary entry with the smallest Hamming distance.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
import random
|
||||
from harness.udpstream.tilecost import build_dictionary, quantise
|
||||
|
||||
def _frame(seed):
|
||||
r = random.Random(seed)
|
||||
return bytes(r.randrange(256) for _ in range(128 * 1280 // 8))
|
||||
|
||||
def test_dictionary_is_deterministic_and_indexable():
|
||||
frames = [_frame(i) for i in range(4)]
|
||||
a = build_dictionary(frames, k=8, seed=7)
|
||||
b = build_dictionary(frames, k=8, seed=7)
|
||||
assert a == b and len(a) == 8 * 128
|
||||
|
||||
def test_quantise_returns_1280_indices_within_range():
|
||||
frames = [_frame(i) for i in range(4)]
|
||||
d = build_dictionary(frames, k=8, seed=7)
|
||||
idx = quantise(frames[0], d)
|
||||
assert len(idx) == 1280 and max(idx) < 8
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run it to verify it fails**
|
||||
|
||||
Run: `python3 -m pytest harness/udpstream/tests/test_tilecost.py -v`
|
||||
Expected: FAIL — no module `tilecost`.
|
||||
|
||||
- [ ] **Step 3: Implement `tilecost.py`**
|
||||
|
||||
Tiles are extracted from each frame in 32×32 blocks (40×32 per frame), k-means over Hamming distance with a fixed seed and a deterministic tie-break (lowest tile id), at most 8 iterations.
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it passes**
|
||||
|
||||
Run: `python3 -m pytest harness/udpstream/tests/test_tilecost.py -v`
|
||||
Expected: PASS (2 passed)
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add harness/udpstream/tilecost.py harness/udpstream/tests/test_tilecost.py
|
||||
git commit -m "feat(tiles): deterministic dictionary builder and quantiser"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Dictionary upload
|
||||
|
||||
**Files:**
|
||||
- Create: `harness/udpstream/tiledict.py`, `harness/udpstream/tests/test_tiledict.py`
|
||||
- Modify: `harness/udpstream/frame.py` (add `KIND_DICT_CHUNK = 4`, `KIND_TILES_FULL = 2`, `KIND_TILES_DELTA = 3`)
|
||||
|
||||
**Interfaces:**
|
||||
- `dict_chunks(dictionary: bytes, offset_base: int = 0) -> list[bytes]` — each packet carries `offset u16` then ≤1450 bytes of dictionary.
|
||||
- `payload_offset(packet: bytes) -> int`, `payload_body(packet: bytes) -> bytes`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
from harness.udpstream.tiledict import dict_chunks, payload_offset, payload_body
|
||||
|
||||
def test_chunks_cover_the_dictionary_once_in_order():
|
||||
d = bytes(range(256)) * 128 # 32 KB
|
||||
pk = dict_chunks(d)
|
||||
assert b"".join(payload_body(p) for p in pk) == d
|
||||
offs = [payload_offset(p) for p in pk]
|
||||
assert offs == sorted(offs) and offs[0] == 0
|
||||
assert offs[-1] + len(payload_body(pk[-1])) == len(d)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run it to verify it fails**, then implement (Step 3), then verify it passes (Step 4).
|
||||
|
||||
Commands: `python3 -m pytest harness/udpstream/tests/test_tiledict.py -v`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add harness/udpstream/tiledict.py harness/udpstream/tests/test_tiledict.py harness/udpstream/frame.py
|
||||
git commit -m "feat(tiles): chunked dictionary upload protocol"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Dictionary storage and dispatch in the payload
|
||||
|
||||
**Files:**
|
||||
- Modify: `harness/udpstream/vidstream.c`, `harness/udpstream/frame_parse.h` (kinds), `harness/mame/vidstream.lua` (`dict` mode)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `fp_parse`, the slot ring, the pool (`POOL_PTR`), markers.
|
||||
- Produces: `DICT_READY` (cell on the marker page), `DICT_BYTES` (bytes accepted), `DICT_BAD` (out-of-order or oversize chunks).
|
||||
|
||||
- [ ] **Step 1: Write the failing MAME check**
|
||||
|
||||
`VT1200_MODE=dict`: inject a dictionary in 23 chunks, then a `TILES_FULL` frame. Assert `DICT_READY == 1`, `DICT_BYTES == 32768`, a chunk with a wrong offset increments `DICT_BAD`, and a `TILES_FULL` frame that arrives before `DICT_READY` is rejected without painting.
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** — `VT1200_MODE=dict harness/mame/run-vidstream.sh` (the kind is unknown, nothing is stored).
|
||||
|
||||
- [ ] **Step 3: Implement.** Dictionary lives at `POOL_PTR + slot bytes`; chunks must arrive in ascending offset with no gaps; `TILES_FULL`/`TILES_DELTA` are rejected until `DICT_READY`.
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes.**
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add harness/udpstream/vidstream.c harness/udpstream/frame_parse.h harness/mame/vidstream.lua
|
||||
git commit -m "feat(tiles): dictionary storage and kind dispatch in the payload"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Tile blit
|
||||
|
||||
**Files:**
|
||||
- Create: `harness/udpstream/vram_tile.asm`
|
||||
- Modify: `harness/udpstream/vidstream.c`, `harness/udpstream/build-stream.sh`, `harness/mame/vidstream.lua` (`tiles` mode)
|
||||
- Create: `harness/mame/tile_check.py`
|
||||
|
||||
**Interfaces:**
|
||||
- `tile_blit(tile_bitaddr, tx, ty)` — one `PIXBLT B,XY`, `SPTCH=32`, `DYDX=(32,32)`, destination XY `(ty*32, tx*32)`; reloads all implied operands per call (ruling R13 style: values published in `.bss`, wrapper is the only entry).
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
`VT1200_MODE=tiles`: upload a dictionary whose entry *i* is a known pattern, send a `TILES_FULL` frame with index `i` in every position for four corners, dump VRAM, and assert each 32×32 block equals `dictionary[i]` bit-for-bit (`tile_check.py` renders the expected screen from the indices).
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails** (screen unchanged).
|
||||
|
||||
- [ ] **Step 3: Implement** the asm helper and the 1,280-call loop; `dint` around the burst like the strip path.
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes** (each corner tile matches; a deliberately wrong `SPTCH` must fail the check — run that as a negative control).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add harness/udpstream/vram_tile.asm harness/udpstream/vidstream.c harness/udpstream/build-stream.sh harness/mame/vidstream.lua harness/mame/tile_check.py
|
||||
git commit -m "feat(tiles): 32x32 PIXBLT B,XY tile blit and full-frame paint"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Deltas and keyframes
|
||||
|
||||
**Files:**
|
||||
- Modify: `harness/udpstream/tiledict.py`, `harness/udpstream/vidstream.c`, `harness/mame/vidstream.lua`
|
||||
|
||||
**Interfaces:**
|
||||
- `delta_frame(prev: bytes, cur: bytes) -> bytes | None` — `None` when nothing changed; otherwise a run-length list of changed tile indices followed by their new values.
|
||||
- Payload: `KIND_TILES_DELTA` paints only the listed tiles; a `TILES_FULL` keyframe is sent when `frame_no % KEYFRAME_EVERY == 0` (default 60, ~2 s at 30 fps) or when the status frame reports a drop.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Host: `test_tiledict.py` — a delta of two changed tiles lists exactly those two indices and no others; identical frames produce `None`; a keyframe is due at frame 60 and 120.
|
||||
MAME: `VT1200_MODE=delta` — paint a full frame, then a delta touching one tile, and assert only that 32×32 block changed (compare dumps before/after).
|
||||
|
||||
- [ ] **Step 2: Run to verify it fails.**
|
||||
|
||||
- [ ] **Step 3: Implement** encoder and decoder; deltas must not paint a tile index above 255 (guard) and must not run past the frame length.
|
||||
|
||||
- [ ] **Step 4: Run to verify it passes.**
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add harness/udpstream/tiledict.py harness/udpstream/vidstream.c harness/mame/vidstream.lua harness/udpstream/tests/test_tiledict.py
|
||||
git commit -m "feat(tiles): delta frames and periodic keyframes"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Bad Apple end to end
|
||||
|
||||
**Files:**
|
||||
- Create: `harness/udpstream/make_tiles.py` (video → dictionary + per-frame index/delta stream, cached on disk)
|
||||
- Modify: `harness/udpstream/vidsend.py` (tile mode), `harness/mame/NETWORK_REPORT.md` (new section)
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — a 30-frame synthetic clip quantised and replayed in MAME must reproduce each frame's rendered bitmap within the dictionary's error bound (report both PSNR and the per-frame tile-change count).
|
||||
- [ ] **Step 2: Run to verify it fails.**
|
||||
- [ ] **Step 3: Implement** `make_tiles.py` and `vidsend.py --tiles`; print per-frame bytes, changed tiles, and the keyframe schedule.
|
||||
- [ ] **Step 4: Run the MAME replay; record the numbers.**
|
||||
- [ ] **Step 5: Unit run (human-assisted).** `sudo ./test-stream.sh <iface> <mac> --tiles` for 300 frames, then the 6,300-frame soak. Record fps, drops, `OS_TOUCH`, and whether the OS repainted.
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add harness/udpstream/make_tiles.py harness/udpstream/vidsend.py harness/mame/NETWORK_REPORT.md
|
||||
git commit -m "feat(tiles): bad apple tile stream end to end"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Execution handoff
|
||||
|
||||
Plan 1's unit measurements (fps for `L,XY` and `B,XY`, heap headroom, `OS_TOUCH`) size this plan's expectations; nothing here depends on them being good, only on them existing.
|
||||
Reference in new issue
Block a user