mirror of
https://git.k-space.ee/arti/vt1200.git
synced 2026-11-01 12:14:56 +02:00
plan: Bad Apple 30s vector demo implementation
This commit is contained in:
arti committed
1 parent
bc3b20113c
commit
3d586106bf
1 file changed
+782
@@ -0,0 +1,782 @@
|
||||
# Bad Apple 30 s Vector Demo Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Play the first 30 s of `hd60.mp4` full-screen on a VT1200 as quantised black-and-white polygon art, loaded once and played from the terminal's 4 MB of RAM with `FILL` rectangles.
|
||||
|
||||
**Architecture:** Host pipeline turns video into a 640×512 binary picture (2×2 screen blocks), traces it into simplified straight-edged polygons, rasterises them back, and packs only changed 32×32-tile rectangles into a delta stream. A TMS34010 payload decodes that stream and paints dirty tiles with `FILL L`. A small `rdebug` bootstrap kills the OS, receives the stream over LANCE, verifies it, and jumps to the player.
|
||||
|
||||
**Tech Stack:** Python 3 + numpy + Pillow + ffmpeg + potrace + pytest; TMS34010 C/assembly built with the 1987 TI SDK under DOSBox; MAME's `vt1200` driver and Lua debugger for verification.
|
||||
|
||||
**Spec:** [`docs/superpowers/specs/2026-10-31-bad-apple-vector-design.md`](../specs/2026-10-31-bad-apple-vector-design.md)
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Source is `/tmp/opencode/badapple-real/hd60.mp4`; first 30 s; 1,800 frames at 60 fps.
|
||||
- Output is full screen, 1280×1024, 1 bpp, stretched from 4:3 to 5:4.
|
||||
- Logical picture is 640×512; each logical pixel is a 2×2 screen block.
|
||||
- Tiles are 32×32 logical (64×64 screen); 20×16 = 320 tiles.
|
||||
- Movie data must be `<= 3.0 MB` and must never exceed the hard data area `0x010000..0x317FFF` (3.03 MB).
|
||||
- Runtime uses `FILL L` only; `LINE` is debug-only.
|
||||
- Incremental dirty-tile painting is accepted; a brief wipe is not a failure.
|
||||
- The OS dies before the first byte lands in low RAM; recovery is a power cycle.
|
||||
- The payload never skips a record. If the unit is slower than the encoder estimate, the movie stretches.
|
||||
- Every image is LSB-first: bit 0 of a byte is the leftmost pixel.
|
||||
- Do not edit `harness/` or the ROM. Stage only paths under `bad-apple/` and the two design/plan docs.
|
||||
- The 1987 C compiler is K&R/C89: no `//`, no declarations after statements, no `stdint.h`, no `for (int ...)`.
|
||||
|
||||
## Review Focus
|
||||
|
||||
- **Malformed or truncated movie data** must be refused with an error block and halt, never read past the buffer or paint a partial movie. Tests: `test_movie_truncated_record`, `test_player_rejects_bad_header`.
|
||||
- **Tile rectangle edge cases**: `w=32`/`h=32` must survive the 5-bit `w-1`/`h-1` encoding, and a tile with more than 127 rectangles must encode and decode without truncation. Tests: `test_rect_full_tile_bias`, `test_more_than_127_rects`.
|
||||
- **Data cap**: the encoder must fail loudly above 3.03 MB, not truncate. Test: `test_write_movie_refuses_over_cap`.
|
||||
- **Delta chain**: no runtime record skipping; a slow unit stretches. Test: `test_select_records_holds_time_not_records`.
|
||||
- **Partial network load**: the player must not jump into a movie whose checksum failed. Test: `test_loader_rejects_bad_crc`.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
bad-apple/
|
||||
├── README.md
|
||||
├── REPORT.md
|
||||
├── .gitignore
|
||||
├── pytest.ini
|
||||
├── conftest.py
|
||||
├── pipeline/
|
||||
│ ├── __init__.py
|
||||
│ ├── params.json
|
||||
│ ├── frames.py video -> gray -> temporal median -> blur -> binary
|
||||
│ ├── vectorize.py potrace + RDP + even-odd rasteriser
|
||||
│ ├── movie.py header, frame records, pack/unpack, cap check
|
||||
│ ├── tilepack.py frame diff -> per-tile background + rectangles
|
||||
│ ├── cost.py 13.9 ms cost model and record selection
|
||||
│ ├── build.py CLI: video -> movie.bin + manifest.json
|
||||
│ └── preview.py side-by-side PNG and MP4 previews
|
||||
├── player/
|
||||
│ ├── movie.h
|
||||
│ ├── movie.c pure decoder, host-testable
|
||||
│ ├── main.c hardware setup, playback loop, pacing
|
||||
│ ├── fill.asm fill_rect() over FILL L
|
||||
│ ├── entry.asm _start + ROM thunks (copied from udpstream)
|
||||
│ ├── build.sh DOSBox SDK build
|
||||
│ └── tests/
|
||||
│ ├── test_movie_decode.c
|
||||
│ └── fixture.bin generated by tests/make_fixture.py
|
||||
├── tools/
|
||||
│ ├── load_mame.lua debugger load for one frame / 5 s / full 30 s
|
||||
│ ├── check_vram.py VRAM dump -> logical frame -> exact compare
|
||||
│ ├── send_movie.py raw-Ethernet sender
|
||||
│ ├── load.sh rdebug bootstrap + network load
|
||||
│ └── tests/
|
||||
│ ├── test_check_vram.py
|
||||
│ └── test_send_movie.py
|
||||
└── tests/
|
||||
├── test_frames.py
|
||||
├── test_vectorize.py
|
||||
├── test_movie.py
|
||||
├── test_tilepack.py
|
||||
├── test_cost.py
|
||||
├── test_build.py
|
||||
└── test_c_decoder.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Scaffold and frame extraction primitives
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/README.md`
|
||||
- Create: `bad-apple/.gitignore`
|
||||
- Create: `bad-apple/pytest.ini`
|
||||
- Create: `bad-apple/conftest.py`
|
||||
- Create: `bad-apple/pipeline/__init__.py`
|
||||
- Create: `bad-apple/pipeline/params.json`
|
||||
- Create: `bad-apple/pipeline/frames.py`
|
||||
- Test: `bad-apple/tests/test_frames.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `temporal_median3(frames: Sequence[np.ndarray]) -> np.ndarray`
|
||||
- `box_blur3(gray: np.ndarray) -> np.ndarray`
|
||||
- `binarize(gray: np.ndarray, threshold: int = 127) -> np.ndarray` (True = black)
|
||||
- `pack_lsb(mask: np.ndarray) -> bytes`
|
||||
- `extract_gray(video: str, start: float, duration: float, fps: int, width: int, height: int) -> Iterator[np.ndarray]`
|
||||
- `extract_binary(video: str, start: float, duration: float, fps: int, width: int, height: int, params: dict) -> Iterator[np.ndarray]`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_frames.py
|
||||
import numpy as np
|
||||
from pipeline.frames import temporal_median3, box_blur3, binarize, pack_lsb
|
||||
|
||||
def test_temporal_median_removes_single_frame_flash():
|
||||
a = np.zeros((4, 4), dtype=np.uint8)
|
||||
b = np.zeros((4, 4), dtype=np.uint8); b[1, 1] = 255
|
||||
c = np.zeros((4, 4), dtype=np.uint8)
|
||||
out = temporal_median3([a, b, c])
|
||||
assert out[1, 1] == 0
|
||||
|
||||
def test_pack_lsb_is_leftmost_pixel_bit_zero():
|
||||
mask = np.zeros((1, 8), dtype=bool)
|
||||
mask[0, 0] = True
|
||||
assert pack_lsb(mask) == b"\x01"
|
||||
mask[0, 7] = True
|
||||
assert pack_lsb(mask) == b"\x81"
|
||||
|
||||
def test_binarize_black_is_true():
|
||||
gray = np.array([[0, 255]], dtype=np.uint8)
|
||||
assert binarize(gray).tolist() == [[True, False]]
|
||||
|
||||
def test_box_blur_softens_a_single_pixel():
|
||||
gray = np.zeros((3, 3), dtype=np.uint8); gray[1, 1] = 9
|
||||
out = box_blur3(gray)
|
||||
assert out[1, 1] == 1
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests to verify they fail**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_frames.py -q`
|
||||
Expected: FAIL with `ModuleNotFoundError: No module named 'pipeline'` or missing names.
|
||||
|
||||
- [ ] **Step 3: Implement the primitives**
|
||||
|
||||
`temporal_median3` returns `np.median(np.stack(frames), axis=0).astype(np.uint8)`. `box_blur3` is a 3×3 mean with edge clamping (do not `np.roll`, which wraps). `binarize` is `gray < threshold` (black is True). `pack_lsb` uses `np.packbits(mask, axis=1, bitorder="little")`. `extract_gray` runs:
|
||||
|
||||
```
|
||||
ffmpeg -v error -ss <start> -t <duration> -i <video>
|
||||
-vf fps=<fps>,scale=<w>:<h>:flags=lanczos,format=gray
|
||||
-f rawvideo -
|
||||
```
|
||||
|
||||
and yields `(h, w)` uint8 arrays. `extract_binary` buffers three gray frames, applies `temporal_median3`, `box_blur3`, `binarize`, then yields; it duplicates the first and last frame at the boundaries so exactly `duration*fps` frames are produced. Add `pipeline/params.json` with the values from the spec:
|
||||
|
||||
```json
|
||||
{
|
||||
"start": 0.0, "duration": 30.0, "fps": 60,
|
||||
"width": 640, "height": 512, "threshold": 127,
|
||||
"temporal_median": 3, "blur": 3,
|
||||
"tile": 32, "scale": 2,
|
||||
"rdp_epsilon": 1.5, "min_segment": 2, "min_component": 8
|
||||
}
|
||||
```
|
||||
|
||||
`bad-apple/conftest.py` inserts the `bad-apple/` directory on `sys.path`. `.gitignore` ignores `data/*` but keeps `!data/manifest.json`, plus `preview/`, `player/build/`, `__pycache__/`, `.pytest_cache/`. `pytest.ini` sets `testpaths = tests`; the two `tools/tests` files are run explicitly.
|
||||
|
||||
- [ ] **Step 4: Run the tests to verify they pass**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_frames.py -q`
|
||||
Expected: `4 passed`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/README.md bad-apple/.gitignore bad-apple/pytest.ini \
|
||||
bad-apple/conftest.py bad-apple/pipeline/__init__.py \
|
||||
bad-apple/pipeline/params.json bad-apple/pipeline/frames.py \
|
||||
bad-apple/tests/test_frames.py
|
||||
git commit -m "feat(badapple): frame extraction primitives"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Polygon tracing, simplification and rasterisation
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/pipeline/vectorize.py`
|
||||
- Test: `bad-apple/tests/test_vectorize.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `rdp(points: list[tuple[float, float]], epsilon: float) -> list[tuple[float, float]]`
|
||||
- `parse_potrace_geojson(text: str) -> list[list[list[tuple[float, float]]]]`
|
||||
- `trace(mask: np.ndarray, params: dict) -> list[list[list[tuple[float, float]]]]`
|
||||
- `rasterize(polygons, width: int, height: int) -> np.ndarray`
|
||||
- `vectorize_frame(mask: np.ndarray, params: dict) -> np.ndarray`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_vectorize.py
|
||||
import numpy as np
|
||||
from pipeline.vectorize import rdp, rasterize, trace, vectorize_frame
|
||||
|
||||
def test_rdp_drops_collinear_point():
|
||||
pts = [(0, 0), (1, 0), (2, 0)]
|
||||
assert rdp(pts, 0.5) == [(0, 0), (2, 0)]
|
||||
|
||||
def test_rasterize_square():
|
||||
poly = [[(1, 1), (4, 1), (4, 4), (1, 4), (1, 1)]]
|
||||
out = rasterize([poly], 6, 6)
|
||||
assert out[2, 2]
|
||||
assert not out[0, 0]
|
||||
|
||||
def test_rasterize_square_with_hole():
|
||||
outer = [(1, 1), (5, 1), (5, 5), (1, 5), (1, 1)]
|
||||
hole = [(2, 2), (2, 4), (4, 4), (4, 2), (2, 2)]
|
||||
out = rasterize([[outer, hole]], 7, 7)
|
||||
assert out[3, 1] # left wall
|
||||
assert not out[3, 3] # hole
|
||||
|
||||
def test_trace_simple_shape_and_roundtrip():
|
||||
mask = np.zeros((16, 16), dtype=bool)
|
||||
mask[4:12, 4:12] = True
|
||||
polys = trace(mask, {"rdp_epsilon": 1.0, "min_segment": 2})
|
||||
assert len(polys) >= 1
|
||||
roundtrip = rasterize(polys, 16, 16)
|
||||
assert roundtrip[6, 6]
|
||||
assert not roundtrip[0, 0]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests to verify they fail**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_vectorize.py -q`
|
||||
Expected: FAIL with missing module or names.
|
||||
|
||||
- [ ] **Step 3: Implement the vectoriser**
|
||||
|
||||
`trace` writes the black mask to a temporary PBM (`P4`, 1 = black), runs:
|
||||
|
||||
```
|
||||
potrace -b geojson -a 0 -t 4 -u 1 -o <json> <pbm>
|
||||
```
|
||||
|
||||
and parses `FeatureCollection` → `Polygon`/`MultiPolygon` rings. Apply `rdp` to each closed ring (keep the first/last point). Drop rings with fewer than 3 distinct points or a longest segment below `min_segment`. `rasterize` collects every ring edge, intersects it with each scanline at `y + 0.5`, sorts the x values, and fills pixel centres between pairs (even-odd). `vectorize_frame` is `rasterize(trace(mask, params), width, height)`.
|
||||
|
||||
- [ ] **Step 4: Run the tests to verify they pass**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_vectorize.py -q`
|
||||
Expected: `4 passed`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/pipeline/vectorize.py bad-apple/tests/test_vectorize.py
|
||||
git commit -m "feat(badapple): polygon tracing and rasterisation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Tile diff and movie format
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/pipeline/movie.py`
|
||||
- Create: `bad-apple/pipeline/tilepack.py`
|
||||
- Test: `bad-apple/tests/test_tilepack.py`
|
||||
- Test: `bad-apple/tests/test_movie.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `Rect(x0, y0, w, h)` frozen dataclass, `TileRecord(tile_index, bg, rects)`, `FrameRecord(ticks, flags, tiles)`, `MovieHeader(...)`
|
||||
- `decompose_tile(tile: np.ndarray) -> tuple[int, list[Rect]]`
|
||||
- `diff_tiles(prev: np.ndarray, cur: np.ndarray, tile: int = 32) -> list[TileRecord]`
|
||||
- `apply_record(canvas: np.ndarray, record: FrameRecord, tile: int = 32) -> None`
|
||||
- `encode_frame(record: FrameRecord) -> bytes`
|
||||
- `decode_frame(data: bytes, offset: int) -> tuple[FrameRecord, int]`
|
||||
- `write_movie(path: str, frames: Sequence[FrameRecord], header: MovieHeader) -> int`
|
||||
- `read_movie(path: str) -> tuple[MovieHeader, list[FrameRecord]]`
|
||||
- `MovieFormatError` exception
|
||||
- `MAX_DATA_BYTES = 0x308000`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_tilepack.py
|
||||
import numpy as np
|
||||
from pipeline.tilepack import decompose_tile, diff_tiles, apply_record
|
||||
from pipeline.movie import Rect, TileRecord, FrameRecord
|
||||
|
||||
def test_decompose_solid_tiles():
|
||||
black = np.ones((32, 32), dtype=bool)
|
||||
bg, rects = decompose_tile(black)
|
||||
assert bg == 1 and rects == []
|
||||
white = np.zeros((32, 32), dtype=bool)
|
||||
bg, rects = decompose_tile(white)
|
||||
assert bg == 0 and rects == []
|
||||
|
||||
def test_decompose_block_is_one_rect():
|
||||
tile = np.zeros((32, 32), dtype=bool)
|
||||
tile[4:6, 7:10] = True
|
||||
bg, rects = decompose_tile(tile)
|
||||
assert bg == 0
|
||||
assert rects == [Rect(7, 4, 3, 2)]
|
||||
|
||||
def test_diff_only_reports_changed_tiles():
|
||||
a = np.zeros((64, 64), dtype=bool)
|
||||
b = a.copy(); b[0:32, 0:32] = True
|
||||
recs = diff_tiles(a, b, tile=32)
|
||||
assert len(recs) == 1 and recs[0].tile_index == 0
|
||||
|
||||
def test_apply_record_reconstructs():
|
||||
a = np.zeros((32, 32), dtype=bool)
|
||||
b = a.copy(); b[4:9, 3:12] = True
|
||||
recs = diff_tiles(a, b, tile=32)
|
||||
canvas = a.copy()
|
||||
for rec in recs:
|
||||
apply_record(canvas, FrameRecord(1, 0, [rec]), tile=32)
|
||||
assert np.array_equal(canvas, b)
|
||||
```
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_movie.py
|
||||
import numpy as np
|
||||
from pipeline.movie import (Rect, TileRecord, FrameRecord, MovieHeader,
|
||||
encode_frame, decode_frame, write_movie, read_movie,
|
||||
MovieFormatError)
|
||||
|
||||
def test_rect_full_tile_bias_roundtrip():
|
||||
rec = FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 32, 32)])])
|
||||
back, off = decode_frame(encode_frame(rec), 0)
|
||||
assert back.tiles[0].rects[0] == Rect(0, 0, 32, 32)
|
||||
assert off == len(encode_frame(rec))
|
||||
|
||||
def test_more_than_127_rects_roundtrip():
|
||||
rects = [Rect(i % 32, i // 32, 1, 1) for i in range(200)]
|
||||
rec = FrameRecord(1, 0, [TileRecord(3, 0, rects)])
|
||||
back, _ = decode_frame(encode_frame(rec), 0)
|
||||
assert len(back.tiles[0].rects) == 200
|
||||
|
||||
def test_truncated_record_raises():
|
||||
data = encode_frame(FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 4, 4)])]))
|
||||
try:
|
||||
decode_frame(data[:-1], 0)
|
||||
except MovieFormatError:
|
||||
return
|
||||
assert False, "truncated record was accepted"
|
||||
|
||||
def test_write_movie_refuses_over_cap(tmp_path):
|
||||
frames = [FrameRecord(1, 0, []) for _ in range(10)]
|
||||
header = MovieHeader(640, 512, 2, 32, 20, 16, 10, 60, 72)
|
||||
# 10 empty records are tiny; the cap is exercised by a patched constant.
|
||||
import pipeline.movie as m
|
||||
old = m.MAX_DATA_BYTES
|
||||
m.MAX_DATA_BYTES = 8
|
||||
try:
|
||||
try:
|
||||
write_movie(str(tmp_path / "x.bin"), frames, header)
|
||||
except MovieFormatError:
|
||||
return
|
||||
assert False, "cap not enforced"
|
||||
finally:
|
||||
m.MAX_DATA_BYTES = old
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests to verify they fail**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_tilepack.py tests/test_movie.py -q`
|
||||
Expected: FAIL with missing modules.
|
||||
|
||||
- [ ] **Step 3: Implement tile decomposition and the format**
|
||||
|
||||
`decompose_tile` chooses `bg` by majority, makes `fg = tile != bool(bg)`, extracts horizontal runs per row, and merges identical `(x0, x1)` runs across consecutive rows into `Rect(x0, y0, x1-x0+1, height)`. `diff_tiles` compares `prev` and `cur` tile by tile and calls `decompose_tile(cur_tile)` for changed tiles. `apply_record` fills the whole tile with `bg`, then fills each rect with `1-bg`.
|
||||
|
||||
Header layout (36 bytes, little-endian): `magic "BAPL"`, `version u16=1`, `width u16`, `height u16`, `scale u16`, `tile u16`, `tiles_x u16`, `tiles_y u16`, `frame_count u32`, `index_offset u32`, `data_offset u32`, `source_fps u16`, `display_hz u16`, `reserved u16=0`. Index entry is `<IBB` (offset, ticks, flags). Record is `<H` dirty count, then per tile `<HH` (tile_index, `(bg << 15) | rect_count`), then 3-byte rects packed as `x0 | (y0 << 5) | ((w-1) << 10) | ((h-1) << 15)`.
|
||||
|
||||
`decode_frame` validates counts against the remaining bytes before allocating/reading. `write_movie` builds the header + index + records in memory, raises `MovieFormatError` if the total exceeds `MAX_DATA_BYTES`, and writes atomically via a temporary file.
|
||||
|
||||
- [ ] **Step 4: Run the tests to verify they pass**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_tilepack.py tests/test_movie.py -q`
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/pipeline/movie.py bad-apple/pipeline/tilepack.py \
|
||||
bad-apple/tests/test_tilepack.py bad-apple/tests/test_movie.py
|
||||
git commit -m "feat(badapple): tile delta format and movie packer"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Cost model, record selection and build CLI
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/pipeline/cost.py`
|
||||
- Create: `bad-apple/pipeline/build.py`
|
||||
- Create: `bad-apple/pipeline/preview.py`
|
||||
- Test: `bad-apple/tests/test_cost.py`
|
||||
- Test: `bad-apple/tests/test_build.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `CostConstants(word_us=0.40, call_us=3.0, rect_us=1.0)`
|
||||
- `estimate_record_us(record: FrameRecord, constants: CostConstants, scale=2, tile=32) -> float`
|
||||
- `select_records(frames: Sequence[np.ndarray], constants: CostConstants, budget_us: float = 13900.0, max_skip: int = 2) -> list[FrameRecord]`
|
||||
- `build_movie(video: str, out_dir: str, params: dict) -> dict`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests**
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_cost.py
|
||||
import numpy as np
|
||||
from pipeline.cost import CostConstants, estimate_record_us, select_records
|
||||
from pipeline.movie import FrameRecord, TileRecord, Rect
|
||||
|
||||
C = CostConstants()
|
||||
|
||||
def test_one_solid_tile_cost():
|
||||
rec = FrameRecord(1, 0, [TileRecord(0, 1, [])])
|
||||
# 64x64 screen tile = 256 words; 256*0.40 + 3us call
|
||||
assert abs(estimate_record_us(rec, C) - (256 * 0.40 + 3.0)) < 0.01
|
||||
|
||||
def test_full_screen_is_over_budget():
|
||||
recs = [TileRecord(i, 1, []) for i in range(320)]
|
||||
assert estimate_record_us(FrameRecord(1, 0, recs), C) > 13900.0
|
||||
|
||||
def test_select_records_holds_time_not_records():
|
||||
quiet = np.zeros((32, 32), dtype=bool)
|
||||
busy = np.ones((32, 32), dtype=bool)
|
||||
out = select_records([quiet, busy, quiet, quiet], C, budget_us=2000.0)
|
||||
assert all(r.ticks >= 1 for r in out)
|
||||
assert sum(r.ticks for r in out) == 4
|
||||
```
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_build.py
|
||||
import json
|
||||
import numpy as np
|
||||
import pipeline.build as build
|
||||
from pipeline.movie import read_movie
|
||||
|
||||
def test_build_tiny_movie(tmp_path, monkeypatch):
|
||||
frames = [np.zeros((64, 64), dtype=bool) for _ in range(3)]
|
||||
frames[1][8:24, 8:24] = True
|
||||
monkeypatch.setattr(build, "extract_binary", lambda *a, **k: iter(frames))
|
||||
monkeypatch.setattr(build, "vectorize_frame", lambda m, p: m)
|
||||
out = tmp_path / "data"
|
||||
manifest = build.build_movie("unused.mp4", str(out), {"duration": 3, "fps": 60})
|
||||
header, recs = read_movie(str(out / "movie.bin"))
|
||||
assert header.frame_count == 3
|
||||
assert len(recs) == 3
|
||||
assert (out / "manifest.json").exists()
|
||||
assert json.loads((out / "manifest.json").read_text())["frame_count"] == 3
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the tests to verify they fail**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_cost.py tests/test_build.py -q`
|
||||
Expected: FAIL with missing modules.
|
||||
|
||||
- [ ] **Step 3: Implement the cost model and build CLI**
|
||||
|
||||
`estimate_record_us` sums, per dirty tile, one background fill of 64×64 screen pixels (256 words) plus each rect's `ceil(w*scale/16) * h*scale` words, all at `word_us`, plus `call_us` per fill and `rect_us` per rect. `select_records` walks the source frames, computes the record from the last **kept** frame; if it fits the budget, emit it with `ticks=1`; otherwise remember it and skip up to `max_skip` frames, accumulating ticks, then emit the best candidate even if over budget (a cut). It never emits a record from a skipped frame. `build_movie` streams `extract_binary`, applies `vectorize_frame`, computes records, calls `select_records`, writes `movie.bin`, `manifest.json` (data size, dirty-tile percentiles, cost percentiles, dropped count), and previews for frames 0, 60, 300, 900, 1799. `preview.py` writes PNGs and an optional MP4 via ffmpeg.
|
||||
|
||||
- [ ] **Step 4: Run the tests to verify they pass**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_cost.py tests/test_build.py -q`
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/pipeline/cost.py bad-apple/pipeline/build.py \
|
||||
bad-apple/pipeline/preview.py bad-apple/tests/test_cost.py \
|
||||
bad-apple/tests/test_build.py
|
||||
git commit -m "feat(badapple): cost model, frame selection and build CLI"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Host-testable player decoder
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/player/movie.h`
|
||||
- Create: `bad-apple/player/movie.c`
|
||||
- Create: `bad-apple/player/tests/test_movie_decode.c`
|
||||
- Create: `bad-apple/tests/make_fixture.py`
|
||||
- Test: `bad-apple/tests/test_c_decoder.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces C:
|
||||
- `struct movie_header { unsigned int width, height, scale, tile, tiles_x, tiles_y, frame_count, index_offset, data_offset, source_fps, display_hz; };`
|
||||
- `int movie_open(const unsigned char *data, unsigned long len, struct movie_header *h);`
|
||||
- `int movie_decode_frame(const unsigned char *data, unsigned long len, unsigned int frame, const struct movie_header *h, void (*fill)(int x, int y, int w, int h, int color, void *ctx), void *ctx);`
|
||||
- `int movie_checksum(const unsigned char *data, unsigned long len);`
|
||||
|
||||
- [ ] **Step 1: Generate the fixture and write the failing tests**
|
||||
|
||||
`bad-apple/tests/make_fixture.py` writes `player/tests/fixture.bin` using `pipeline.movie` with two frames: frame 0 a white tile with one black `Rect(0,0,32,32)`; frame 1 a black tile with `Rect(1,1,2,3)`. `tests/test_c_decoder.py` compiles `movie.c` + the C test with `gcc -std=c99 -Wall -Werror` into a temp binary, runs it, and compares its printed lines to the Python decoder.
|
||||
|
||||
```python
|
||||
# bad-apple/tests/test_c_decoder.py
|
||||
import subprocess, sys, tempfile
|
||||
from pathlib import Path
|
||||
from pipeline.movie import read_movie
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
def test_c_decoder_matches_python(tmp_path):
|
||||
subprocess.run([sys.executable, str(ROOT / "tests" / "make_fixture.py")], check=True)
|
||||
exe = tmp_path / "decode"
|
||||
subprocess.run(["gcc", "-std=c99", "-Wall", "-Werror",
|
||||
str(ROOT / "player" / "movie.c"),
|
||||
str(ROOT / "player" / "tests" / "test_movie_decode.c"),
|
||||
"-o", str(exe)], check=True)
|
||||
out = subprocess.run([str(exe), str(ROOT / "player" / "tests" / "fixture.bin")],
|
||||
check=True, capture_output=True, text=True).stdout
|
||||
assert "frames=2" in out
|
||||
assert "0 0 0 0 0 32 32" in out
|
||||
assert "1 1 1 1 1 2 3" in out
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_c_decoder.py -q`
|
||||
Expected: FAIL because `player/movie.c` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the decoder and the C test**
|
||||
|
||||
`movie.c` uses only C89-compatible declarations (K&R style where needed) and no libc beyond what the 1987 compiler provides. It reads little-endian words with byte shifts, validates magic/version, bounds-checks every count against `len`, and calls `fill` with screen coordinates already scaled: `sx = (tile_x*tile + x0)*scale`, `sy = (tile_y*tile + y0)*scale`, `sw = (w+1)*scale`, `sh = (h+1)*scale`. The C test prints `frames=N`, then one line per rect `frame tile bg x y w h`.
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it passes**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tests/test_c_decoder.py -q`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/player/movie.h bad-apple/player/movie.c \
|
||||
bad-apple/player/tests/test_movie_decode.c bad-apple/tests/make_fixture.py \
|
||||
bad-apple/tests/test_c_decoder.py bad-apple/player/tests/fixture.bin
|
||||
git commit -m "feat(badapple): host-tested movie decoder"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: FILL primitive, payload build and one exact frame in MAME
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/player/fill.asm`
|
||||
- Create: `bad-apple/player/entry.asm`
|
||||
- Create: `bad-apple/player/main.c`
|
||||
- Create: `bad-apple/player/build.sh`
|
||||
- Create: `bad-apple/tools/load_mame.lua`
|
||||
- Create: `bad-apple/tools/check_vram.py`
|
||||
- Test: `bad-apple/tools/tests/test_check_vram.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `void fill_rect(int x, int y, int w, int h, int color);` (C-callable, published-word form)
|
||||
- `main.c` mode `ONE`: open the movie, paint frame 0, write `FRAME=0` to marker bit `0x01810000`, spin.
|
||||
- `load_mame.lua` env: `BA_MODE=one`, `BA_MOVIE`, `BA_PAYLOAD`, `BA_AT`, `BA_ENTRY`.
|
||||
- `check_vram.py --vram <dump> --movie <movie.bin> --frame <n> --scale 2` exits 0 on exact match.
|
||||
|
||||
- [ ] **Step 1: Write the failing checker test**
|
||||
|
||||
```python
|
||||
# bad-apple/tools/tests/test_check_vram.py
|
||||
import numpy as np
|
||||
from tools.check_vram import vram_to_logical
|
||||
|
||||
def test_vram_downsamples_2x2_lsb_first():
|
||||
vram = bytearray(256 * 1024)
|
||||
# logical pixel (0,0) black means screen pixels x=0..1,y=0..1 set.
|
||||
vram[0] = 0x03 # bits 0 and 1 of row 0
|
||||
vram[256] = 0x03 # bits 0 and 1 of row 1
|
||||
logical = vram_to_logical(bytes(vram), width=640, height=512, scale=2)
|
||||
assert logical[0, 0]
|
||||
assert not logical[0, 1]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tools/tests/test_check_vram.py -q`
|
||||
Expected: FAIL because `tools/check_vram.py` does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the FILL primitive, payload, loader and checker**
|
||||
|
||||
`fill.asm` follows `harness/udpstream/vram.asm`: a no-argument entry reads published globals `fill_x`, `fill_y`, `fill_w`, `fill_h`, `fill_color`; sets `B2 = 0x04000000 + y*2048 + x`, `B3 = 0x800`, `B7 = (h << 16) | w`, `B9 = color`, programs `PSIZE=1`, `PMASK=0`, `CONTROL` replace once, then `FILL L`. The C wrapper `fill_rect` publishes the globals and calls the asm entry.
|
||||
|
||||
`entry.asm` is copied from `harness/udpstream/entry.asm`, with `_start` setting `SP` and calling `_main`, then spinning (no return). `build.sh` copies `harness/udpstream/build-stream.sh`, changes `SRC` to `player/main.c`, copies `movie.c` as `MOVIE.C`, links `MOVIE.obj`, `FILL.obj`, `MAIN.obj`, `ENTRY.obj`, and links `.text` at bit `0x018C0000`, `.bss` at bit `0x01C00000`. It writes `build/player.bin` and `build/player.env` with `ENTRY_PTR`.
|
||||
|
||||
`load_mame.lua` loads `player.bin` at bit `0x018C0000` and `movie.bin` at bit `0x00080000`, verifies a readback, sets a breakpoint at the OS idle PC (default `0x3F579970`, as `vidstream.lua`), and jumps. `check_vram.py` reads the first `256*1024` bytes of the dump, extracts bit `x` of row `y` (LSB-first), downsamples 2×2 by majority, and compares to `apply_record` over the movie frames.
|
||||
|
||||
- [ ] **Step 4: Run the checker test and the MAME one-frame check**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tools/tests/test_check_vram.py -q`
|
||||
Expected: PASS.
|
||||
|
||||
Run a one-frame movie in MAME with `VT1200_SKIP_LEVEL1=1`, `BA_MODE=one`, `VT1200_DUMP_VRAM_FRAMES=<after paint>`, then:
|
||||
`python3 tools/check_vram.py --vram <dump> --movie data/movie.bin --frame 0`
|
||||
Expected: `EXACT` and exit 0.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/player/fill.asm bad-apple/player/entry.asm \
|
||||
bad-apple/player/main.c bad-apple/player/build.sh \
|
||||
bad-apple/tools/load_mame.lua bad-apple/tools/check_vram.py \
|
||||
bad-apple/tools/tests/test_check_vram.py
|
||||
git commit -m "feat(badapple): FILL primitive and one-frame MAME check"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Playback, pacing and the 5 s slice
|
||||
|
||||
**Files:**
|
||||
- Modify: `bad-apple/player/main.c`
|
||||
- Modify: `bad-apple/tools/load_mame.lua`
|
||||
- Create: `bad-apple/tools/run_mame.sh`
|
||||
- Test: manual/MAME (no host unit test)
|
||||
|
||||
**Interfaces:**
|
||||
- `main.c` mode `PLAY`: open the movie, clear the screen white, loop over records, call `movie_decode_frame`, pace on `INTPEND` DI, and write `FRAME`, `TICKS`, `LAST_BYTES`, `PAINT_LOOPS`, `ERROR` to the marker page.
|
||||
- `load_mame.lua` env `BA_MODE=play`, `BA_FRAME` (start frame), `BA_DUMP_FRAMES`.
|
||||
|
||||
- [ ] **Step 1: Add the playback loop and marker page**
|
||||
|
||||
Implement `play()` in `main.c`: `DINT`, initial full-screen white `FILL`, then for each frame call the decoder. Before each record, read `INTPEND` (bit `TMS34010_DI = 0x0400`) and wait for the next vblank; program `DPYINT` if the inherited value does not produce the pending bit. Maintain a 16.16 `now`/`next` accumulator with `now += 60/72` per vsync; if `now >= next`, paint and `next += ticks`; if `now - next > 1.0`, set `next = now` (slow down, never skip). Publish the marker page at bit `0x01810000`.
|
||||
|
||||
- [ ] **Step 2: Run the 5 s slice in MAME and compare sampled frames**
|
||||
|
||||
Use the encoder to pack frames 0..300 into `data/movie5.bin`. Run `tools/run_mame.sh` with `BA_MODE=play`, dump VRAM at MAME frames corresponding to movie frames 0, 60, 180, 300, and compare each with `check_vram.py`.
|
||||
Expected: every sample prints `EXACT`.
|
||||
|
||||
- [ ] **Step 3: Record the paint-cost histogram and the first visual clip**
|
||||
|
||||
Capture a windowed run with `x11grab` (or `tools/run_mame.sh`), inspect for stale tiles, double-applied deltas, and a half-painted cut. Record the measured `PAINT_LOOPS` / `LAST_BYTES` distribution in `bad-apple/REPORT.md` (created in Task 9; a temporary note is fine).
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/player/main.c bad-apple/tools/load_mame.lua \
|
||||
bad-apple/tools/run_mame.sh
|
||||
git commit -m "feat(badapple): playback loop and vblank pacing"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Full 30 s encode and MAME playback
|
||||
|
||||
**Files:**
|
||||
- Modify: `bad-apple/pipeline/params.json`
|
||||
- Modify: `bad-apple/pipeline/build.py` (only if tuning reveals a bug)
|
||||
- Test: MAME sampled frames
|
||||
|
||||
- [ ] **Step 1: Run the real pipeline**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pipeline.build --video /tmp/opencode/badapple-real/hd60.mp4 --out data --duration 30`
|
||||
Expected: `data/movie.bin`, `data/manifest.json`, preview frames; manifest reports `data_bytes <= 3.0e6` and p95 cost `<= 13900`.
|
||||
|
||||
- [ ] **Step 2: If the gate fails, tune in this order**
|
||||
|
||||
Increase `rdp_epsilon` (1.5 → 2.0 → 3.0), increase `min_component` (8 → 16), then raise `tile` from 32 to 64 (the format supports any power-of-two tile; the player uses the header). Do **not** shorten the clip. Re-run and re-check the gate after each change.
|
||||
|
||||
- [ ] **Step 3: Play the full 30 s in MAME**
|
||||
|
||||
Run `tools/run_mame.sh` with `BA_MODE=play`. Dump VRAM at movie frames 0, 60, 300, 600, 900, 1200, 1500, 1799 and every scene cut. Compare each with `check_vram.py`.
|
||||
Expected: every sample `EXACT`; the last frame is frame 1799.
|
||||
|
||||
- [ ] **Step 4: Commit the tuned parameters and manifest summary**
|
||||
|
||||
```bash
|
||||
git add bad-apple/pipeline/params.json bad-apple/data/manifest.json
|
||||
git commit -m "feat(badapple): tune the 30s stream to the memory budget"
|
||||
```
|
||||
|
||||
(`data/movie.bin` and previews stay gitignored; the manifest is the record.)
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Network loader and real-unit delivery
|
||||
|
||||
**Files:**
|
||||
- Create: `bad-apple/player/loader.c` or extend `main.c` with `MODE=LOAD`
|
||||
- Create: `bad-apple/tools/send_movie.py`
|
||||
- Create: `bad-apple/tools/load.sh`
|
||||
- Test: `bad-apple/tools/tests/test_send_movie.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Wire packet: `u32 offset`, `u16 length`, `u16 seq`, `u16 crc`, then `length` bytes.
|
||||
- `send_movie(iface, host_mac, path, chunk=1450) -> int` returns 0 after the final ACK.
|
||||
- Loader writes to `0x010000`, ACKs the highest contiguous offset every 32 datagrams, verifies the whole-file CRC, then jumps.
|
||||
|
||||
- [ ] **Step 1: Write the failing sender test**
|
||||
|
||||
```python
|
||||
# bad-apple/tools/tests/test_send_movie.py
|
||||
from tools.send_movie import pack_packet, parse_ack
|
||||
|
||||
def test_pack_packet_header():
|
||||
pkt = pack_packet(0x1234, 1, b"abc")
|
||||
assert pkt[0:4] == (0x1234).to_bytes(4, "little")
|
||||
assert pkt[4:6] == (3).to_bytes(2, "little")
|
||||
assert pkt[6:8] == (1).to_bytes(2, "little")
|
||||
assert pkt[10:] == b"abc"
|
||||
|
||||
def test_parse_ack():
|
||||
assert parse_ack(b"BAK1" + (0x1000).to_bytes(4, "little")) == 0x1000
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tools/tests/test_send_movie.py -q`
|
||||
Expected: FAIL because the module does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the sender and loader**
|
||||
|
||||
Copy the raw-Ethernet framing from `harness/udpstream` / `harness/badapple` (ethertype and tap/interface selection). Send 1,450-byte chunks, wait for ACKs, retransmit from the first gap, and require a final `BAK!` ACK. The loader uses the verified LANCE init/ring sequence from `harness/udpstream/vidstream.c`, receives into `0x010000`, and checks the whole-file CRC before jumping. If the real chip truncates a high RX buffer address, fall back to a low 32 KB staging ring and copy each datagram up.
|
||||
|
||||
- [ ] **Step 4: Run the sender unit test and the MAME tap test**
|
||||
|
||||
Run: `cd bad-apple && python3 -m pytest tools/tests/test_send_movie.py -q`
|
||||
Expected: PASS.
|
||||
|
||||
Run `harness/mame/run.sh net`, then `tools/load.sh --host 192.168.99.2 --iface vt1200 --movie data/movie.bin`.
|
||||
Expected: final ACK, then the 30 s movie plays.
|
||||
|
||||
- [ ] **Step 5: Real unit**
|
||||
|
||||
Run the same `load.sh` against the unit's address/interface, after confirming `rdebug` is reachable. Record a video. Recovery is a power cycle.
|
||||
Expected: the same picture, no host attached after the load.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/player/loader.c bad-apple/tools/send_movie.py \
|
||||
bad-apple/tools/load.sh bad-apple/tools/tests/test_send_movie.py
|
||||
git commit -m "feat(badapple): network loader and delivery"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Documentation, report and regression
|
||||
|
||||
**Files:**
|
||||
- Create/modify: `bad-apple/README.md`
|
||||
- Create: `bad-apple/REPORT.md`
|
||||
- Test: `cd /home/arti/code/k-space/vt1200 && M=/home/arti/code/k-space/mame0289/vt1200 && ... progress.lua`
|
||||
|
||||
- [ ] **Step 1: Write the README**
|
||||
|
||||
Cover: one-command build pipeline, preview command, MAME run, real-unit load, recovery warning, data budget numbers, and the fact that the OS dies.
|
||||
|
||||
- [ ] **Step 2: Write the report**
|
||||
|
||||
Record: measured data size, dirty-tile and rect percentiles, paint-cost histogram, dropped frames, MAME exact-match evidence, unit observations, and the remaining gaps (potrace temporal jitter, LANCE high-address uncertainty, incremental wipes).
|
||||
|
||||
- [ ] **Step 3: Run the regression baseline**
|
||||
|
||||
Run the standard no-env `progress.lua` at 200 frames and confirm the line is still `PC=3FDCE400 samples=199 (2 distinct PCs)`.
|
||||
Expected: unchanged.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add bad-apple/README.md bad-apple/REPORT.md
|
||||
git commit -m "docs(badapple): usage, measurements and gaps"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: geometry (§5), vectorisation (§7.1), tile format (§7.2), frame adaptation (§7.3), cuts (§7.4), memory layout (§8.1), takeover (§8.2), loader (§8.3), rendering (§8.4), presentation (§8.5), pacing (§8.6), verification (§9), milestones (§10), risks (§11) each map to a task above.
|
||||
- Type consistency: `Rect`, `TileRecord`, `FrameRecord`, `MovieHeader`, `MAX_DATA_BYTES` are used with the same names in Tasks 3–10.
|
||||
- Review Focus: truncated data (Task 3/5), rect bias and >127 count (Task 3), cap (Task 3), delta chain (Task 4), partial load (Task 9) are pinned by tests or MAME checks.
|
||||
- Proportion: the plan names files, signatures and tests; it does not transcribe the decoder or the encoder bodies.
|
||||
Reference in new issue
Block a user