Compare commits

...
38 Commits
Author SHA1 Message Date
arti 0d9435f650 docs(badapple): benchmark the r5.01 compiler output 2026-11-01 11:49:25 +02:00
arti 2e3875da3b docs(badapple): record the r5.01 toolchain build result 2026-11-01 11:45:08 +02:00
arti a3cec55795 docs(badapple): record where hd60.mp4 came from 2026-11-01 11:33:31 +02:00
arti 638656bd05 docs(badapple): current MAME demo quick start 2026-11-01 11:24:59 +02:00
arti 371fe27bc2 merge: Bad Apple 30s vector prototype 2026-11-01 11:23:38 +02:00
arti cb2d1c37df docs(badapple): record the potrace y-up orientation fix 2026-11-01 11:22:05 +02:00
arti 983784d9ec fix(badapple): flip potrace y-up polygons to the framebuffer 2026-11-01 11:15:57 +02:00
arti c3ae1269e4 fix(badapple): distinguish UDP load failures and repeat the send 2026-11-01 11:08:55 +02:00
arti 6b6bfaf135 fix(badapple): pace the UDP load at 2 ms per packet 2026-11-01 11:06:49 +02:00
arti 1d9da7e3cf fix(badapple): checksum, packet bounds, ticks and marker placement 2026-11-01 10:29:49 +02:00
arti 4013f75434 docs(badapple): usage, measurements and gaps 2026-11-01 10:20:23 +02:00
arti 8a1cd1bfb1 feat(badapple): UDP network loader and sender 2026-11-01 10:19:36 +02:00
arti 116356a975 feat(badapple): full 30s stream and MAME exactness 2026-11-01 10:16:12 +02:00
arti 150a6d86c0 feat(badapple): playback loop and vblank pacing 2026-11-01 10:10:52 +02:00
arti dbfc7b60c5 feat(badapple): FILL primitive and one-frame MAME check 2026-11-01 10:04:52 +02:00
arti beb45942ad feat(badapple): host-tested movie decoder 2026-11-01 00:24:20 +02:00
arti 5a1f8429f0 feat(badapple): cost model, frame selection and build CLI 2026-11-01 00:23:08 +02:00
arti 8371cdad12 feat(badapple): tile delta format and movie packer 2026-11-01 00:22:16 +02:00
arti 0c93078ee3 feat(badapple): polygon tracing and rasterisation 2026-11-01 00:21:17 +02:00
arti c7569efb7d feat(badapple): frame extraction primitives 2026-11-01 00:20:35 +02:00
arti 9deada6cef docs: run the r5.01 C toolchain end to end under 2ine
The null-selector fix in the 2ine fork unblocks gspac/gspopt/gspcg, so
document the full pipeline: build 2ine, extract the tools, the four C
stages, absolute placement, and the firmware round-trip.

Record the two gotchas found while testing: gspcl still stops after
gspac because the children exit 139 at teardown, and GSPA 5.01 rejects
b15, so the 32 KB round-trip differs at one SP access (same physical
register, A15/B15).
2026-11-01 00:18:06 +02:00
arti b75048fb87 Merge remote-tracking branch 'wut/master' 2026-11-01 00:06:09 +02:00
arti 3d586106bf plan: Bad Apple 30s vector demo implementation 2026-11-01 00:03:03 +02:00
arti bc3b20113c spec: Bad Apple 30s vector-polygon demo 2026-10-31 23:59:46 +02:00
arti 6febcf00f4 feat(rx): rectangle blits for stretch-1 strips; ring depth 16 -> 32 in the encoder
Two things the 60 fps work needed, and one ceiling it found.

Rectangle blit: a strip frame with stretch 1 has one source row per screen row,
so the source advances by 1280 bits and a group of rows inside one fragment is
contiguous -- one PIXBLT with SPTCH = 1280 copies the group.  Measured on the
tile path, a blit call costs ~0.45 ms in MAME regardless of size (1,200 tile
calls accepted 9 frames where 128 strip calls accepted 73 for the same pixels),
so call count, not word count, is what a frame is charged for.  paint_rect()
takes the stretch-1 frame 9 rows at a time and falls back to the per-row path if
a row straddles a datagram.

Ring depth: vidsrc still said RING_DATAGRAMS = 16 and capped a strip frame at
144 rows, which stopped being true when the receive ring was deepened to 32 for
the dictionary upload.  It is 32 now, the encoder may send up to the header's
255 rows, and the three tests that encoded the old ceiling were updated to it.

The ceiling this found: a picture-as-strips full frame is 29 datagrams, i.e.
1,740 datagrams/s at 60 fps, while the emulated LANCE delivers ~450/s (1,935
datagrams over 4.3 s, measured).  So keyframes cannot be sent as whole strips at
60 fps in MAME whatever the decoder does -- the datagram rate binds first.  The
tile delta (about one datagram a frame) remains the right transport; what costs
is its blit-call count.

Verified: modes rx, draw and tiles still PASS (draw and tiles with their pixel
checkers); 155 unit tests pass.
2026-10-31 23:27:27 +02:00
arti c5526332f0 feat(badapple): stylise the clip into blocks, and let the sender widen its keyframe cadence
Two knobs for the 60 fps attempt, both measured on Bad Apple:

clip_from_video.py --block N averages each NxN block, thresholds the average and
draws the block flat.  The point is destination words: at 60 fps the clip changes
80 tiles/frame unstylised, 54 at 8x8 and 29 at 16x16, against the ~44-tile budget
that MAME's ~3.4 us/word allows in 16.7 ms.  A side effect worth knowing: with
16-pixel blocks every 32x32 tile is one of exactly 2^4 combinations, so the 256
entry dictionary represents the stylised clip with zero error (measured: 16
entries used, "reconstruction 0 bit(s)", PSNR inf, mean delta 141 B against 496 B
for the unstylised clip).

vidsend.py --keyframe-every N exposes the cadence that TileSender already had as
a constructor argument.  A keyframe repaints all 1,280 tiles (~264 ms of MAME CPU
at the measured 0.22 ms/tile), so the default 60-frame cadence costs ~4.4 ms on
average per frame at 60 fps -- a tenth of the budget spent on repairs.  The
13,167-frame stylised clip ships 220 keyframes at the default; 600 measured 6 in
the same window.

Measured on the stylised 60 fps cache (13,167 frames, 16 dictionary entries):
  --fps 60, default cadence   643 of 805 accepted (80%), drops=8
  --fps 45                     484 of 520 accepted, drops=0, sender PASS
  --fps 60, --keyframe-every 600  451 of 519 accepted, drops=5
So the stylised clip roughly doubles acceptance against the same 60 fps send
(40% -> 80%) but MAME still caps the clean rate around 36-45 fps.
2026-10-31 23:12:25 +02:00
arti 9139322892 perf(tiles): program the blit constants once per frame, not once per tile
Spec 8.3's hoist, and the 60 fps work showed it was the per-tile cost that
mattered, not the per-frame one.

Every tile_blit() call used to save 8 B registers, load B1/B3/B4/B7/B8/B9 and
write 11 I/O registers (PSIZE, CONVSP, CONVDP, PMASK, the CONTROL read-modify-
write and its read-back) -- all constants.  The user's guide marks only SADDR
and DADDR changed by PIXBLT, and the TMS34010 agent's reading of it confirms
B1/B3 survive, so those two are the only ones a per-tile call has to reload.

vram_tile.asm now exports tile_init_() (the constants, once per frame, called
from paint_tiles/paint_delta) and a tile_blit_() that is setf, two pushes, two
register loads, pixblt, two pops, rets.

Measured on the real 140-tile deltas at a 60 fps send, same window, terminal's
own status frames:

  before the hoist   162 accepted
  after the hoist    231 accepted   (+43%)

and the one-tile stream (where tile cost is nil) already runs at 60 fps with
DINT: 600 frames sent at 59.8 fps, 579 accepted (96.5%), drops=0.

Pixel verdicts unchanged: modes tiles, delta and replay all RESULT PASS with
tile_check; 155 unit tests pass.

What the remaining gap is, for the next reader: MAME's PIXBLT moves about
3.4 us per destination word (0.22 ms per 32x32 tile, measured from the accepted
rate) against the unit's measured 0.40 us/word for FILL.  A 140-tile delta is
8,960 words, i.e. ~30 ms of MAME CPU against a 16.7 ms frame -- so 60 fps for
this content is bounded by the emulator's pixel engine, not by the design.
Whether the unit can do it turns on the one number nobody has measured there:
PIXBLT B,XY's per-word rate (spec 10, risk table).
2026-10-31 22:40:42 +02:00
arti 1bb3643144 feat(badapple): measure the 60 fps wire budget before changing any format
Spec 11's first task: decide from the clip, not from assumptions, whether the
raw-tile scheme (no dictionary, tag + bitmap per changed tile) fits a 60 fps
frame.  measure_wire.py extracts at 1280x960/60 fps with the hard threshold the
design requires, diffs each frame on the 40x30 picture grid, and totals the
payload a TILES_DELTA would carry:

    4 + 3*count + 128*m

Full clip, 13,167 frames: mean 16,049 B/frame, p50 14,755, p95 33,353, max
51,523; 22.5% of frames exceed the 21 KB one-frame budget; 87% of changed tiles
are neither solid black nor white, so they carry a 128-byte bitmap (the design
assumed a third); mean 140 changed tiles/frame, max 1,173 at a cut.  Link duty
at 60 fps would be 72% of the measured 1.27 MB/s.

Two consequences for the design, recorded here rather than discovered later:
the raw-tile scheme is borderline rather than comfortable, and the tile churn at
60 fps (140) is nearly the same as at 15 fps (165) -- motion between adjacent
60 fps frames is mostly sub-tile, so sending faster does not divide the wire by
four.
2026-10-31 22:36:34 +02:00
arti 8de6d94e84 feat(rx): mask interrupts for good -- the OS timer was halving the frame rate
Spec 8.9 decides the payload may kill the resident OS; this is the first half of
that, and it is a rate fix as much as a robustness one.

With the OS still ticking, the payload's loop is preempted and the machine
accepts ~15 frames/s **whatever the payload size**: a 5-byte one-tile delta
measured the same as a 140-tile one (195 frames in 13.4 s, MAME at 100% speed,
frames_rx from the payload's own status frames).  That is scheduling, not
decode -- the loop itself runs ~20k iterations/s.

VID_OWN_CPU (default 1) takes one DINT before the main loop and makes vid_eint_()
a no-op, so nothing preempts the LANCE poll.  Measured on the same one-tile
stream:

  OS ticking (before)   15.0 fps accepted, 400 frames in 15.4 s
  DINT at startup       24.0 fps accepted, 385/400 frames, drops=0
  and, paced differently, 31.5 fps with 387/400 -- the sender, not the terminal,
  is now the throttle (its socket blocks once the tap's drain rate is exceeded)

Consequence stated where it is made: the OS's drivers stop being serviced, so
its console and network stack are gone for good.  The payload owns the machine
once it takes the LANCE either way.  The harness reaches the payload through the
debugger, not the OS, so the replay modes are unaffected -- verified: modes
tiles, delta and replay all RESULT PASS with their pixel checkers after the
change.

Set VID_OWN_CPU=0 at build time to reproduce the old burst-EINT behaviour for a
comparison run.
2026-10-31 22:33:07 +02:00
arti d211ff0770 fix(badapple): --no-load must not wait for ICMP; survive a slow readback
Two faults the 60 fps work exposed, both in the delivery tooling:

* `demo.sh --no-load` waited for the terminal to answer ping.  A loaded payload
  owns the LANCE and stops the OS's stack with it, so that gate can never pass
  once the payload is running -- the recovery path documented for "already
  jumped" deadlocked for its full 180 s.  The gate now runs only when this
  invocation is going to load the payload.

* `rdebug_load.py` used a 5 s socket timeout and no retry, so a readback peek
  that timed out on a freshly booted OS failed a load whose pokes had already
  been acknowledged.  Timeout is 15 s (--timeout) and the readback retries once
  before it is believed.

Tests: the retry path is pinned with a fake remote that times out on the first
peek (one timeout, then the successful pass over every chunk) and one that never
answers (it must raise rather than loop).  9 tests in harness/badapple pass.
2026-10-31 22:20:58 +02:00
arti c166c6fdec fix(badapple): refuse a non-cache --tiles path before the jump
demo.sh loaded (and jumped) the payload before it ever looked at --tiles, so a
wrong path -- the frames directory rather than the cache make_tiles.py writes --
only surfaced after the machine had been handed to a payload that never returns,
forcing a reboot to retry.  It now checks dictionary.bin/stream.bin first, names
the frames-vs-cache confusion when the directory holds raw frames, and exits 2
without touching the network.  The README's failure table gains that row and the
--no-load recovery for an instance that has already been jumped.
2026-10-31 22:05:48 +02:00
arti 650ec9a0cd spec: Bad Apple at 60 fps exact — raw tagged tiles, no dictionary 2026-10-31 21:54:48 +02:00
arti d6aa6f46c1 docs(badapple): how the clip data is prepared, with the measured costs
The setup section said 'a clip cache' without saying where a clip comes from,
what it costs, or how to tell the result is sane.  Adds a Data preparation
section: the public sources (and why an upscale is not better), the frame
extraction (3,292 frames, 489 MB, ~12 s, LSB-first), the cache build (~2 min
47 s, four files, 6.1 MB), and the make_tiles summary to compare against --
including that a saturated 256-entry dictionary and 24 dB PSNR are expected, not
a failure.
2026-10-31 21:27:09 +02:00
arti 8f831eaf33 fix(video): model the real unit's VR319 monitor
The harness forced the VR150 monitor switch, so the firmware wrote the OS
screen at 1024 px / 128-byte rows.  VT1200_VRAM_REAL then re-read that
framebuffer at the VR319 pitch, which sheared the OS and wrapped the right
quarter of every row.

Model VR319 (switch 0x60) by default -- the real unit's mode -- so the
firmware itself programs 1280x1024 with a 256-byte row pitch (CONVDP=0x14).
The renderer's pitch follows VT1200_MONITOR, and the destination span now
comes from HEBLNK..HSBLNK instead of x=0, placing the active 1280 px at
x=336 in MAME's 1696-wide raster.

Verified: clean Terminal Manager at frame 600, legible faithful diagnostics
menu and level-1 error screens, tile mode PASS with tile_check 1280/1280
tiles, tablet/mouse pixel-identical.  VT1200_MONITOR=VR150 reproduces the
old geometry; the regression baseline moves 198 -> 199 (2 PCs) and is
recorded in harness/mame/README.md.
2026-10-31 20:53:51 +02:00
arti eabc3051be feat(badapple): the demo as a tool -- rdebug delivery, clip builder, one command
A new folder so the demo can be reproduced instead of re-derived: the delivery
and the clip packing were living in throwaway scripts under /tmp, and the only
written route to a real clip was an ffmpeg recipe that packs MSB first -- the
order that mirrors every eight pixels on a display.

  harness/badapple/rdebug_load.py    image -> terminal RAM over the firmware's
                                     own remote-debug service (TCP 771), then
                                     jump to the entry from vidstream.env
  harness/badapple/clip_from_video.py video -> LSB-first 1-bpp frames for
                                     make_tiles.py --clip
  harness/badapple/demo.sh           ICMP gate, load, send; one command
  harness/badapple/README.md         setup, run in MAME, run on real hardware,
                                     what success looks like, failure modes

The loader records the two protocol facts that cost time to find: a 512-byte
poke chunk (1024 is mis-parsed as a following opcode: "Invalid command
0xA5A5A5A5"), and that nothing can be probed after the jump, so the image is
read back BEFORE jumping and the run is observed from the machine afterwards.

Tests cover what a wrong assumption would silently corrupt -- chunk arithmetic,
the linkage the entry point is read from, and pixel 0 landing on bit 0 -- and
the tool was exercised end to end against MAME: demo.sh loaded the payload and
the terminal reported DICT_BYTES=32768, DICT_BAD=0, RX_OK=28, TILES_DROP=0
while the sender saw frames_rx=129, drops=0.

153 tests pass (7 new).
2026-10-31 20:20:35 +02:00
arti f7490d2e09 feat(opencode): add tms34010 agent with the whole User's Guide preloaded
Subagent/primary agent whose system prompt inlines docs/TMS34010-users-guide.txt
(SPVU001A, 1.75 MB, ~361k tokens) via {file:} substitution, so whole-book
questions -- cross-chapter synthesis, complete opcode lists, encoding and
timing tables -- are answered from the text in one pass.

Read-only (edit and subagent denied) and keeps read/shell, so it can render a
page with pdftoppm when the OCR text layer is mangled: section 12 prints XY as
XV and checkmarks as 'J'.
2026-10-31 20:03:06 +02:00
arti 9af7b4fb8d fix(rx): release descriptors individually instead of flipping the whole ring
The receive loop re-armed all 16 descriptors into the other slot after every
completed frame.  During a burst that re-pointed descriptors the chip had
ALREADY filled but the consumer had not read, so those frames were lost without
a trace: the LANCE DMA log shows all 23 dictionary datagrams written by the
chip, 1.132 ms apart (the MTU/bandwidth poll cadence), while the payload's own
counters accounted for 4 (DICT_BYTES=1448, dict_bad=3).  Nothing was dropped by
MAME; the ring was being rewritten under the frames it held.

rx_switch()/rx_armed are gone.  A descriptor is held (rx_hold) while its
datagram belongs to the frame being assembled, and rx_release() hands each one
back -- into the other slot -- only once that frame has been parsed or
abandoned.  Nothing the chip delivered is ever re-pointed unread.

Measured, same zero-gap 23-datagram dictionary upload:
  before                      dgram=4   DICT_BYTES=1448   dict_bad=3
  release fix, 16 descriptors dgram=18  DICT_BYTES=24616  dict_bad=0
  plus a 32-descriptor ring   dgram=23  DICT_BYTES=32768  dict_bad=0

The ring is now 32 descriptors because 23 datagrams is the burst the protocol
actually defines (a whole 32 KB dictionary at MAX_BODY 1448), and 16 could not
hold it while the consumer copied.  SLOT_BYTES grows to 49152 to hold the ring;
POOL_BYTES does NOT grow -- the 160 KB "framebuffer reserve" was unused slack
(the decoder paints straight from the fragments), so the extra 24,576 B per slot
comes out of it and the LANCE block keeps its fixed 0x3C000 tail.

RX_OK is now published after rx_release(), not inside the dispatch.  It is the
"frame done" signal a reader polls, and it used to become visible while
descriptors were still being rewritten: ring mode caught a descriptor holding
slot B's high byte with slot A's low word.  A reader that sees RX_OK move now
sees a painted picture and a consistent ring.

vidstream.lua's ring mode follows: it asserts the invariant that replaced the
flip -- the consumed descriptors move to the other slot, the untouched ones stay
-- instead of "all 16 in slot B".

Evidence: 146 unit tests pass; modes rx, draw, ring, tiles, delta and replay all
PASS with their pixel checkers (draw 1024/1024 rows, tiles and replay via
tile_check).  End to end over rdebug with the STOCK sender (no pacing): the
dictionary burst completes (DICT_READY=1, DICT_BYTES=32768, dict_bad=0) and
RX_OK=100 frames paint, TILES_DROP=0.
2026-10-31 19:45:34 +02:00
arti 5af949b338 fix(tiles): pack LSB first, the terminal's scanout order; render the real layout
The tile pipeline packed every row MSB first: pixel i -> bit 7 - i % 8. The
encoder and tile_check.py shared that convention, so every MAME replay check
passed while a display showed each group of eight pixels mirrored.

Evidence, from the real clip's own replay dump (vram-replay-200.bin, frame 200
of Bad Apple, 1,228,800 pixels):

  decoded MSB first   1,215 mismatched  <- what the pipeline produced
  decoded LSB first   3,113 mismatched  <- the mirrored picture, a sawtooth

and both the real unit and MAME decode LSB first: dwt/screenshot.py --real
("LSB-first bits", measured on live VRAM) and vt1200.cpp's
(vram[bit >> 4] >> (bit & 15)) & 1.  A zoomed edge of that dump showed the
comb of triangles; after this change the same edge is a smooth curve.

vidsrc.Rows.pack_row now sets bit i % 8 of byte i // 8.  The test that pins it
was written first and watched fail: it decodes the packed row with the
hardware's rule, which no MSB-first packer can pass.  The two older tests that
asserted MSB first were rewritten -- they were asserting the bug -- and the
synthetic clip now packs through a bit-reversal table, so the committed cache
is regenerated (PSNR 29.47 -> 31.30 dB).  The layout claims in tilecost.py,
vram_tile.asm and make_tiles.load_clip are corrected so the clip format states
the order instead of leaving it to whoever writes the frames.

Separately, MAME's renderer reads 128-byte rows while a payload targeting the
real unit writes 256-byte rows (vram_tile.asm DPTCH=0x800; c-hello/README.md:
"a payload writing at row*2048 bits lands on scanline row").  VT1200_VRAM_REAL=1
now draws the framebuffer at 2048 bits / 1280 px from x=0.  Off by default: the
Level-1 error text is only legible at the derived 128-byte pitch, and that is
what the faithful boot needs.  The branch only touches the bitmap, so the
regression baseline is unmoved (PROG_FRAMES=200 -> PC=3FDCE400, samples=198,
3 distinct PCs), and the default frame-3000 render of the Terminal Manager is
unchanged.

Verified end to end on the 3,292-frame Bad Apple clip: RX_OK=3196, TILES_DROP=0,
3292/3292 frames dumped and matched by tile_check.py, and the dumps decode to
1,379 / 4,125 mismatched pixels of 1,228,800 (quantisation) under the hardware
rule versus 3,101 / 9,793 under MSB.  MAME's own frame dumps with
VT1200_VRAM_REAL=1 show the clip unsheared.

Revert note for the driver: delete the m_render_real branch and its getenv and
vt1200.cpp returns to the measured 128-byte renderer byte for byte.  No second
reader reviewed that citation; it is quoted in the comment for the next one.
2026-10-31 18:06:14 +02:00
80 changed files with 6832 additions and 336 deletions

No files matched your search

+12 -8
View File
@@ -262,19 +262,23 @@ Messages, Ethernet/IP/LAT, a Memory bar, and "All Diagnostics Passed". The
keyboard and mouse drivers initialise and speak the real protocol; the LANCE is
initialised and the TM shows the real hardware address
`08-00-2B-16-4B-5A`. Evidence frames: `harness/mame/skip_frame3000.png`,
`duart2_default_tm_frame3000.png`, `tablet_cursor_*.png`.
`duart2_default_tm_frame3000.png`, `tablet_cursor_*.png` (VR150 captures; the
VR319 default renders the same UI at the real 1280-px placement — see
`harness/mame/RENDER_REPORT.md` §8).
### Regression baseline
`progress.lua` at 200 frames, **no env**:
```
PC=3FDCE400 samples=198 (3 distinct PCs)
PC=3FDCE400 samples=199 (2 distinct PCs)
```
It is a change detector, not a specification. It has moved twice, both
deliberately; `harness/mame/README.md` records why, including the
`VT1200_IO_PATH_STUB=1` value (`samples=53`, 75 PCs) as a separate baseline.
It is a change detector, not a specification. It has moved three times, all
deliberately; `harness/mame/README.md` records why, including the monitor-default
move (VR319 is the real unit's mode) and the `VT1200_IO_PATH_STUB=1` values
(`samples=97` at the VR319 default, `samples=53` with `VT1200_MONITOR=VR150`) as
separate baselines.
### Deliberate divergences
@@ -284,8 +288,7 @@ deliberately; `harness/mame/README.md` records why, including the
| `VT1200_SKIP_LEVEL1` | off | DUART1 IP3 high → skip all level-1 tests | leave unset |
| `VT1200_IO_PATH_STUB` | off | 10 bytes after the level-0 `0xE1` write | leave unset |
The full env-knob table (21 knobs, down from 33) is in
`harness/mame/README.md`.
The full env-knob list is in `harness/mame/README.md`.
---
@@ -569,6 +572,7 @@ also why a server-side change cannot cause or cure it.
| `reference/third-party-code.md` | what open-source code is in the ROM |
| `reference/tms34010-sdk.md` | the 1987 toolchain, buildable today |
| `tools/tms34010-sdk/OS2-501.md` | using the 1990 r5.01 toolchain (ANSI C + optimizer) |
| `tools/tms34010-sdk/2INE-501.md` | the r5.01 C toolchain end to end under 2ine, no VM |
| `EXPLOIT_NOTES.md` | the original code-execution audit and the crash decode |
| `NEXT_SESSION.md` | the LAT soak handoff (older; see the reports for later work) |
@@ -588,7 +592,7 @@ vt1200/
│ ├── ptr_constants.py # find/resolve bit-address constants
│ ├── smart34010/ # dis34 (two encoding fixes)
│ ├── as34/ # TMS34010 binutils, ported to 64-bit Linux
│ ├── tms34010-sdk/ # TI SDKs: 1987 DOS + 1990 OS/2 (see OS2-501.md)
│ ├── tms34010-sdk/ # TI SDKs: 1987 DOS + 1990 OS/2 (see OS2-501.md, 2INE-501.md)
│ └── ghidra-tms34010/ # patched processor module + install script
├── harness/
│ ├── mame/ # the MAME driver, Lua probes, reports
+8
View File
@@ -0,0 +1,8 @@
data/*
!data/manifest.json
data5/
preview/
player/build/
player/work/
__pycache__/
.pytest_cache/
+167
View File
@@ -0,0 +1,167 @@
# Bad Apple on a VT1200 — 30 s vector prototype
A standalone, polygon-styled 30-second Bad Apple demo for the VT1200
(TMS34010). The host pipeline turns the first 30 s of `hd60.mp4` into a
640×512 binary picture (2×2 screen blocks), traces it into simplified
straight-edged polygons, and packs only changed 32×32 tiles as rectangles.
The payload paints those rectangles with `FILL L`.
The payload takes over the terminal: it disables interrupts and never returns.
**Recovery is a power cycle.** The ROM is never modified.
Design: [`../docs/superpowers/specs/2026-10-31-bad-apple-vector-design.md`](../docs/superpowers/specs/2026-10-31-bad-apple-vector-design.md)
Plan: [`../docs/superpowers/plans/2026-11-01-bad-apple-vector.md`](../docs/superpowers/plans/2026-11-01-bad-apple-vector.md)
Measurements and gaps: [`REPORT.md`](REPORT.md)
## Source video
`hd60.mp4` was **downloaded by the agent** in the earlier OpenCode session
**“Bad Apple demo tool”** (`ses_e6d6b504dffeIRKLu3RqZAiFRp`, message seq 641),
not supplied by the user. The design doc's “human-supplied” wording is
inaccurate.
- Item page: <https://archive.org/details/hd-60-fps-bad-apple-shadow-art-pv-720-x-960-60fps>
- Direct file: <https://archive.org/download/hd-60-fps-bad-apple-shadow-art-pv-720-x-960-60fps/_HD_60FPS_%20Bad%20Apple%21%21%20_Shadow%20Art%20PV_%20%28%20720%20X%20960%2060fps%20%29.mp4>
- The fetch command was `curl -sL -o hd60.mp4 "<direct file>"`.
- Measured: **34,903,925 bytes**, **960×720**, **60 fps**, **219.521451 s**,
h264 / yuv420p, AAC stereo.
- Local path used by this demo: `/tmp/opencode/badapple-real/hd60.mp4`.
The archive.org item name is scrambled (`720-x-960`); the file inside is
960×720, 4:3.
## Quick start — current MAME demo
From the repository root:
```sh
# 1. video -> data/movie.bin
cd bad-apple
python3 -m pipeline.build \
--video /tmp/opencode/badapple-real/hd60.mp4 \
--out data --duration 30
cd ..
# 2. payload
bad-apple/player/build.sh
# 3. watch it
bad-apple/tools/run_mame_gui.sh
```
`run_mame_gui.sh` starts MAME with `VT1200_SKIP_LEVEL1=1`, boots the OS, loads
the payload at frame 2500, lets it take over, writes the movie into low RAM,
and plays it in a window. Expect the 30 s clip at full screen, upright.
`BA_END` (default 8000) is the MAME frame at which the loader script exits;
raise it if your run is slower.
The **network path is currently disabled in practice**: the OS heap cannot
provide one contiguous 2.4 MB `malloc_` block, so `tools/load.sh` stops before
it binds UDP. Use the debugger path above. See `REPORT.md`.
## Build the 30 s stream
```sh
cd bad-apple
python3 -m pipeline.build \
--video /tmp/opencode/badapple-real/hd60.mp4 \
--out data --duration 30
```
Outputs:
| file | size | what |
|---|---|---|
| `data/movie.bin` | 2,401,942 B | 1,757 records, 43 source frames dropped |
| `data/manifest.json` | small | data size, dirty-tile/rect/cost percentiles |
| `data/preview/frame_*.png` | — | still previews |
Measured: dirty tiles mean 44.9 / p95 72; rectangles mean 393 / p95 670;
estimated paint cost mean 7.47 ms / p95 12.01 ms against a 13.9 ms refresh
gate. The data area is 3.03 MB, so this fits with ~0.6 MB spare.
## Build the payload
```sh
bad-apple/player/build.sh # debugger build: build/player.bin
NET=1 bad-apple/player/build.sh # network build: build/player-net.bin
```
The 1987 TI TMS34010 SDK runs under DOSBox. The C sources are K&R C: the
1987 compiler rejects ANSI prototypes and `const`. `.text` is linked at bit
`0x018C0000`, `.bss` at `0x01C00000`.
The 1990 **r5.01 OS/2** toolchain under `2ine-tms34010` also builds it (see
`tools/tms34010-sdk/2INE-501.md` for setup). Build `MAIN.C` without
`gspopt -O2`: the optimizer hoists the `CTRL_READY` polling load and the
payload waits forever at takeover. `MOVIE.C` is fine at `-O2`. The verified
r5.01 build is `.text` 3,168 bytes, entry `0x018C5C70`, and is MAME-exact at
frames 0 and 180. `gspac`/`gspcg` exit 139 at teardown but still write valid
files.
## Run in MAME (debugger path)
For a window, use `tools/run_mame_gui.sh` (above). For scripted/headless
checks:
```sh
# one frame, then dump VRAM at MAME frame 2700
VT1200_DUMP_VRAM_FRAMES=2700 VT1200_DUMP_DIR=/tmp/ba-one \
bad-apple/tools/run_mame.sh one
# 5 s slice, paced (headless)
BA_MOVIE=bad-apple/data5/movie.bin BA_START=0 BA_COUNT=300 \
bad-apple/tools/run_mame.sh play
# full 30 s, paced (headless)
BA_MOVIE=bad-apple/data/movie.bin BA_MOVIE_BASE=0x00080000 \
BA_WAIT=1 BA_PACE=1 bad-apple/tools/run_mame.sh play
```
`run_mame.sh` runs with `-video none`; it exits at `BA_END`. The debugger
loader writes the movie into high RAM (`0x01D00000`) for small slices; for the
full 30 s it uses `BA_WAIT=1`, lets the payload take over, then writes low RAM
at `0x00080000` — the same ordering as the real delivery path.
## Deliver to a real unit (network path)
The terminal must be booted and answering ping. The payload is loaded over
`rdebug`, then receives `movie.bin` over UDP into an OS `malloc` buffer, then
disables interrupts and plays.
```sh
bad-apple/tools/load.sh --host <unit-ip> --movie bad-apple/data/movie.bin
```
**This path is currently blocked** at `malloc_(2401942)`: the OS heap does not
give one contiguous 2.4 MB block, so the payload never binds UDP. The debugger
path above is the working demo. A fixed low-RAM staging receive or raw-LANCE
loader is the next step. See `REPORT.md`.
## Data format
Header 36 bytes (`BAPL`, version, geometry, frame count, index/data offsets).
Index entry: `u32 offset, u8 ticks, u8 flags`. Record: `u16 dirty tile count`,
then per tile `u16 tile index, u16 (bg<<15)|rect count`, then 3-byte rects
(`x0:5 y0:5 w-1:5 h-1:5`). A tile is repainted by filling its 64×64 screen
block with the background colour, then filling the minority-colour rectangles.
## Gotchas this prototype records
* **PSIZE is re-asserted on every FILL.** The resident OS leaves PSIZE at 2 bpp;
a one-time write in `_fill_init_` was overwritten, and every span came out
twice as wide. The fix is the per-call `movi 1 / move @0C0000150h`.
* **VRAM polarity is inverted from the canvas**: bit 1 = white, bit 0 = black.
`tools/check_vram.py` downsamples with that polarity.
* **The OS dies before the movie is in low RAM.** The network path receives
into an OS heap buffer while the stack is alive, then DINTs. The debugger
path does the same ordering with `BA_WAIT=1`.
* **MAME is not the unit's performance verdict.** MAME needed ~2.1 emulated
frames per movie record on the full stream; the unit's 50 MHz FILL rate is
the number that matters. See `REPORT.md`.
* **The marker/control block lives in `.bss`**, not at bit `0x01810000`: a
2.4 MB heap allocation can cross that address. `player.env` publishes
`MARK_PTR`; `load_mame.lua` reads it.
* **Pacing reads each record's `ticks`.** 60 source frames took 74 MAME frames
(72 display frames plus startup), so the 60-on-72 cadence is measured, not
assumed.
+148
View File
@@ -0,0 +1,148 @@
# Bad Apple 30 s vector prototype — report
Date: 2026-11-01
Design: [`2026-10-31-bad-apple-vector-design.md`](../docs/superpowers/specs/2026-10-31-bad-apple-vector-design.md)
## What was built
* Host pipeline: video → temporal median → blur/threshold → potrace
straight-line polygons → RDP → even-odd rasteriser → 32×32 tile deltas →
`movie.bin`.
* Payload: TMS34010 C decoder + `FILL L` primitive, 16-bit marker page,
display-interrupt pacing, UDP receive path.
* Tools: MAME debugger loader, VRAM checker, UDP sender, rdebug load script.
## Measured numbers (first 30 s, 1,800 source frames)
| quantity | value |
|---|---|
| source | 960×720, 60 fps, first 30 s |
| logical grid | 640×512, 2×2 screen blocks |
| records | 1,757 (43 source frames dropped by the cost gate) |
| data | **2,401,942 bytes** (cap 3.03 MB, gate 3.0 MB) |
| dirty tiles/frame | mean 44.9, p50 47, p95 72, max 320 |
| rectangles/frame | mean 393, p50 409, p95 670, max 1,175 |
| estimated cost/frame | mean 7.47 ms, p50 7.95 ms, **p95 12.01 ms**, max 37.99 ms |
| gate | 13.9 ms (one 72 Hz refresh) |
The stream fits with ~0.6 MB spare. The p95 frame is inside one refresh; the
max is a scene cut that necessarily takes more than one.
## Exactness evidence (MAME)
`tools/check_vram.py` unpacks the 256-byte-stride VRAM dump, downsamples 2×2,
and compares bit-for-bit against the Python application of the records.
| run | movie frame | result |
|---|---|---|
| test pattern | 0 | **EXACT** |
| 5 s slice | 0 | **EXACT** |
| 5 s slice | 180 | **EXACT** |
| 5 s slice | 297 | **EXACT** |
| 30 s stream | 0 | **EXACT** |
| 30 s stream | 900 | **EXACT** |
The one-frame MAME run also proved the FILL geometry: a logical black
rectangle 10..109 × 20..69 became exactly screen x 20..219. The first version
was exactly twice as wide; the cause was the OS leaving PSIZE at 2 bpp. The
primitive now re-asserts PSIZE=1 on every call.
Potrace emits **y-up** GeoJSON; the framebuffer is y-down. The first build was
vertically mirrored, and the VRAM checker still said EXACT because it consumed
the same flipped data. `trace()` now flips y before simplification, and
`test_trace_preserves_vertical_position` pins the orientation. The preview PNGs
were the visible symptom.
## Pacing
`wait_vsync()` programs `DPYINT=1`, clears `INTPEND`, and polls the DI bit at
`0xC0000120`. The loop reads each record's `ticks` from the index and advances
a 1/360 s accumulator by 5 per vsync, so a 1/60 s record needs 6 units and a
held record needs 12. Measured with `BA_COUNT=60`: status at MAME frame 2580,
74 frames after takeover at 2506 — 60 source frames in 72 display frames plus
startup, i.e. the 60-on-72 cadence is real, not free-running.
## The performance gap MAME cannot close
MAME needed ~2.1 emulated frames per movie record while replaying frames 0..900
of the full stream. The encoder's model says a mean record is 7.47 ms of a
13.9 ms refresh, so MAME's TMS34010 interpreter is several times slower than the
model. MAME proves the picture and the delta chain; it cannot prove the unit's
60 fps. The unit's measured `FILL` rate (0.40 µs per 16-pixel word) is the
number the cost model uses. A real-unit run is the remaining performance
measurement.
## Network delivery
`player-net.bin` uses the OS UDP socket API (`socket_`, `bind_`,
`recvfrom_`, `malloc_`) to receive `movie.bin` on port 7001 into a heap buffer,
then DINTs and plays. The sender is `tools/send_movie.py`. Packets are
10-byte headers (offset, length, seq, crc16) plus body; a final
offset==length-of-file / 4-byte-checksum packet is required, and the receiver
refuses to jump unless the whole-file checksum matches. Out-of-range packets
are dropped without reading or writing outside the buffer.
Status: the packet format, header offset, checksum path and sender are
unit-tested. The live MAME tap test reaches the payload but stops before UDP
bind: the marker shows `status=0x20` and no socket is open. The suspect is
`malloc_(2401942)` — the OS heap does not provide one contiguous block that
size. The debugger path is the working demo. A fixed low-RAM staging receive
or raw-LANCE loader is the next step. This is the one delivery-path gap.
## r5.01 OS/2 compiler check (2026-11-01)
The 1990 TMS340 r5.01 ANSI C toolchain, run under the patched
`2ine-tms34010` fork, builds this demo. The pipeline was:
```
gspac -k MAIN.C MAIN.IF
gspcg -m MAIN.IF MAIN.ASM MAIN.TMP # no gspopt for MAIN.C
gspa MAIN.ASM
gspac/gspopt -O2/gspcg/gspa for MOVIE.C
gspa FILL.ASM
gspa ENTRY.ASM
gsplnk PLAYER.CMD # .text 018C0000h, .bss 01C00000h
```
Result: `.text` `0x6300` bits = **3,168 bytes**, entry **`0x018C5C70`**.
MAME frame 0 and frame 180 are **EXACT** with
`BA_PAYLOAD=PLAYER.bin BA_ENTRY=0x018C5C70`.
Caveat: `gspopt -O2` on `MAIN.C` hoists the `CTRL_READY` polling load and
the payload waits forever at takeover. `MOVIE.C` is fine at `-O2`; build
`MAIN.C` without `gspopt`, or add `volatile` to the polling reads. `gspac`
and `gspcg` exit 139 at teardown but still write valid output files.
`gspa` 5.01 accepted `entry.asm`/`fill.asm` (including `sp`); the `b15`
round-trip difference does not affect this demo.
Speed A/B in MAME, 600 records with `BA_PACE=0` (takeover at ~frame 2505):
| build | status frame | processing frames |
|---|---|---|
| 1987, no optimizer | 3,922 | ~1,417 |
| r5.01, no optimizer | 3,856 | ~1,351 |
| r5.01, `MOVIE.C` at `-O2` | **3,605** | **~1,100** |
So the r5.01 compiler alone is ~5 % faster here, and the practical r5.01
recipe is **~29 % faster** than the 1987 build. MAME's TMS34010 interpreter
is not silicon; this is a relative measurement, not a unit-cycle claim.
## Regression
The no-env MAME baseline at 200 frames is unchanged:
```
--- distinct PCs sampled: 2
PC=3FDCE400 samples=199
PC=3FDCE350 samples=1
```
## Remaining gaps
1. Real-unit 30 s performance: needs a power-cycle run with
`tools/load.sh`.
2. UDP receive on the unit: socket API proven, this payload not yet.
3. Incremental painting can show a brief wipe on a cut; accepted for this
prototype.
4. Potrace/RDP temporal jitter was reduced with a 3-frame median but not
measured as a motion score.
+6
View File
@@ -0,0 +1,6 @@
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parent
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
+31
View File
@@ -0,0 +1,31 @@
{
"cost_us": {
"max": 37992,
"mean": 7472.069550369951,
"p50": 7948,
"p95": 12011
},
"data_bytes": 2401942,
"dirty_tiles": {
"max": 320,
"mean": 44.899829254410925,
"p50": 47,
"p95": 72
},
"display_hz": 72,
"dropped": 43,
"frame_count": 1757,
"height": 512,
"rects": {
"max": 1175,
"mean": 393.14968696642,
"p50": 409,
"p95": 670
},
"scale": 2,
"source_fps": 60,
"source_frames": 1800,
"tile": 32,
"video": "/tmp/opencode/badapple-real/hd60.mp4",
"width": 640
}
View File
Whitespace-only changes.
+105
View File
@@ -0,0 +1,105 @@
"""CLI: video -> vector frames -> tile deltas -> movie.bin + manifest."""
from __future__ import annotations
import argparse
import json
import os
import numpy as np
from .cost import CostConstants, estimate_record_us, select_records
from .frames import extract_binary
from .movie import MovieHeader, write_movie
from .preview import write_previews
from .vectorize import vectorize_frame
HERE = os.path.dirname(os.path.abspath(__file__))
def _stats(values) -> dict:
if not values:
return {"mean": 0.0, "p50": 0, "p95": 0, "max": 0}
arr = np.asarray(values, dtype=float)
return {
"mean": float(arr.mean()),
"p50": int(np.percentile(arr, 50)),
"p95": int(np.percentile(arr, 95)),
"max": int(arr.max()),
}
def build_movie(video: str, out_dir: str, params: dict | None = None) -> dict:
with open(os.path.join(HERE, "params.json"), encoding="utf-8") as handle:
p = json.load(handle)
if params:
p.update(params)
frames = []
for index, frame in enumerate(extract_binary(
video, p["start"], p["duration"], p["fps"],
p["width"], p["height"], p)):
frames.append(vectorize_frame(frame, p))
if (index + 1) % 60 == 0:
print(" vectorised %d frames" % (index + 1))
if not frames:
raise RuntimeError("no frames extracted from %s" % video)
constants = CostConstants()
records = select_records(frames, constants, budget_us=13900.0, max_skip=2)
height, width = frames[0].shape
header = MovieHeader(
width=width, height=height, scale=int(p.get("scale", 2)),
tile=int(p.get("tile", 32)), tiles_x=width // int(p.get("tile", 32)),
tiles_y=height // int(p.get("tile", 32)), frame_count=len(records),
source_fps=int(p.get("fps", 60)), display_hz=72)
os.makedirs(out_dir, exist_ok=True)
data_bytes = write_movie(os.path.join(out_dir, "movie.bin"), records, header)
dirty = [len(record.tiles) for record in records]
rects = [sum(len(tile.rects) for tile in record.tiles) for record in records]
costs = [estimate_record_us(record, constants) for record in records]
manifest = {
"video": video,
"data_bytes": data_bytes,
"frame_count": len(records),
"source_frames": len(frames),
"dropped": len(frames) - len(records),
"width": width,
"height": height,
"scale": header.scale,
"tile": header.tile,
"source_fps": header.source_fps,
"display_hz": header.display_hz,
"dirty_tiles": _stats(dirty),
"rects": _stats(rects),
"cost_us": _stats(costs),
}
with open(os.path.join(out_dir, "manifest.json"), "w",
encoding="utf-8") as handle:
json.dump(manifest, handle, indent=2, sort_keys=True)
write_previews(os.path.join(out_dir, "preview"), frames,
[0, 60, 300, 900, 1799])
return manifest
def main(argv=None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--video", required=True)
parser.add_argument("--out", default="data")
parser.add_argument("--duration", type=float, default=None)
parser.add_argument("--start", type=float, default=None)
args = parser.parse_args(argv)
overrides = {}
if args.duration is not None:
overrides["duration"] = args.duration
if args.start is not None:
overrides["start"] = args.start
manifest = build_movie(args.video, args.out, overrides)
print(json.dumps(manifest, indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+70
View File
@@ -0,0 +1,70 @@
"""Paint-cost model and build-time frame selection."""
from __future__ import annotations
import math
from dataclasses import dataclass
from typing import Sequence
import numpy as np
from .movie import FrameRecord, TileRecord
from .tilepack import diff_tiles
@dataclass(frozen=True)
class CostConstants:
word_us: float = 0.40
call_us: float = 3.0
rect_us: float = 1.0
def _words(width_px: int) -> int:
return max(1, int(math.ceil(width_px / 16.0)))
def estimate_record_us(record: FrameRecord, constants: CostConstants,
scale: int = 2, tile: int = 32) -> float:
"""Unit-time estimate for one record, including call and decode overhead."""
tile_words = _words(tile * scale) * tile * scale
total = 0.0
for rec in record.tiles:
total += tile_words * constants.word_us + constants.call_us
for rect in rec.rects:
total += (_words(rect.w * scale) * rect.h * scale
* constants.word_us)
total += constants.call_us + constants.rect_us
return total
def select_records(frames: Sequence[np.ndarray], constants: CostConstants,
budget_us: float = 13900.0,
max_skip: int = 2) -> list[FrameRecord]:
"""Build exact deltas, dropping source frames that cannot fit the gate.
The dropped frame is never emitted; the next kept frame is diffed from the
last *shown* frame and carries the accumulated ``ticks``. A run of heavy
frames is eventually emitted anyway (a cut), so the movie still advances.
"""
if not frames:
return []
prev = np.zeros_like(frames[0], dtype=bool)
out: list[FrameRecord] = []
pending = 0
skipped = 0
last = frames[0]
for current in frames:
pending += 1
last = current
rec_tiles = diff_tiles(prev, current, tile=32)
cost = estimate_record_us(FrameRecord(1, 0, rec_tiles), constants)
if cost <= budget_us or skipped >= max_skip:
out.append(FrameRecord(pending, 0, rec_tiles))
prev = current
pending = 0
skipped = 0
else:
skipped += 1
if pending:
rec_tiles = diff_tiles(prev, last, tile=32)
out.append(FrameRecord(pending, 0, rec_tiles))
return out
+96
View File
@@ -0,0 +1,96 @@
"""Video extraction and the small image primitives the pipeline shares."""
from __future__ import annotations
import subprocess
from typing import Iterator, Sequence
import numpy as np
def temporal_median3(frames: Sequence[np.ndarray]) -> np.ndarray:
"""Median of three same-shaped frames, as uint8."""
if len(frames) != 3:
raise ValueError("temporal_median3 wants exactly three frames")
return np.median(np.stack(frames), axis=0).astype(np.uint8)
def box_blur3(gray: np.ndarray) -> np.ndarray:
"""3x3 box blur with edge clamping (no wraparound)."""
if gray.ndim != 2:
raise ValueError("box_blur3 wants a 2-D gray image")
p = np.pad(gray.astype(np.float32), 1, mode="edge")
total = (
p[0:-2, 0:-2] + p[0:-2, 1:-1] + p[0:-2, 2:]
+ p[1:-1, 0:-2] + p[1:-1, 1:-1] + p[1:-1, 2:]
+ p[2:, 0:-2] + p[2:, 1:-1] + p[2:, 2:]
)
return np.rint(total / 9.0).astype(np.uint8)
def binarize(gray: np.ndarray, threshold: int = 127) -> np.ndarray:
"""True means black ink (the source is dark-on-light)."""
return gray < threshold
def pack_lsb(mask: np.ndarray) -> bytes:
"""Pack rows LSB-first: bit 0 of a byte is the leftmost pixel."""
if mask.ndim != 2 or mask.shape[1] % 8:
raise ValueError("pack_lsb wants 2-D rows whose width is a multiple of 8")
return np.packbits(mask.astype(bool), axis=1, bitorder="little").tobytes()
def extract_gray(video: str, start: float, duration: float, fps: int,
width: int, height: int) -> Iterator[np.ndarray]:
"""Yield (height, width) uint8 gray frames from ffmpeg."""
vf = "fps=%s,scale=%d:%d:flags=lanczos,format=gray" % (
fps, width, height)
cmd = [
"ffmpeg", "-v", "error",
"-ss", str(start),
"-i", video,
"-t", str(duration),
"-vf", vf,
"-f", "rawvideo", "-",
]
size = width * height
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, bufsize=size * 4)
try:
while True:
buf = proc.stdout.read(size)
if len(buf) < size:
break
yield np.frombuffer(buf, dtype=np.uint8).reshape(height, width)
finally:
proc.stdout.close()
proc.wait()
if proc.returncode != 0:
raise RuntimeError("ffmpeg failed with status %d" % proc.returncode)
def extract_binary(video: str, start: float, duration: float, fps: int,
width: int, height: int,
params: dict) -> Iterator[np.ndarray]:
"""Yield stylised binary frames: temporal median, blur, threshold.
Exactly ``duration * fps`` frames are produced. The first output is the
first frame against itself; each later output is the median of the
previous, current and next frame.
"""
threshold = int(params.get("threshold", 127))
want_blur = int(params.get("blur", 3))
it = extract_gray(video, start, duration, fps, width, height)
try:
first = next(it)
except StopIteration:
return
def render(median_frame: np.ndarray) -> np.ndarray:
gray = box_blur3(median_frame) if want_blur >= 3 else median_frame
return binarize(gray, threshold)
a = first
b = first
yield render(temporal_median3([a, b, first]))
for current in it:
yield render(temporal_median3([a, b, current]))
a, b = b, current
+190
View File
@@ -0,0 +1,190 @@
"""The on-RAM movie format: header, index, and per-frame tile deltas."""
from __future__ import annotations
import os
import struct
from dataclasses import dataclass, field
from typing import Sequence
MAGIC = b"BAPL"
VERSION = 1
HEADER_SIZE = 36
INDEX_ENTRY_SIZE = 6
#: The data must stop below the payload's rdebug origin at byte 0x318000.
MAX_DATA_BYTES = 0x308000
class MovieFormatError(Exception):
pass
@dataclass(frozen=True)
class Rect:
x0: int
y0: int
w: int
h: int
@dataclass(frozen=True)
class TileRecord:
tile_index: int
bg: int
rects: list[Rect] = field(default_factory=list)
@dataclass(frozen=True)
class FrameRecord:
ticks: int
flags: int
tiles: list[TileRecord] = field(default_factory=list)
@dataclass(frozen=True)
class MovieHeader:
width: int
height: int
scale: int
tile: int
tiles_x: int
tiles_y: int
frame_count: int
source_fps: int
display_hz: int
def pack(self) -> bytes:
index_offset = HEADER_SIZE
data_offset = _align4(index_offset + INDEX_ENTRY_SIZE * self.frame_count)
return struct.pack(
"<4s7H3I3H",
MAGIC, VERSION, self.width, self.height, self.scale, self.tile,
self.tiles_x, self.tiles_y, self.frame_count,
index_offset, data_offset,
self.source_fps, self.display_hz, 0)
def _align4(value: int) -> int:
return (value + 3) & ~3
def _pack_rect(rect: Rect) -> bytes:
if not (0 <= rect.x0 < 32 and 0 <= rect.y0 < 32
and 1 <= rect.w <= 32 and 1 <= rect.h <= 32
and rect.x0 + rect.w <= 32 and rect.y0 + rect.h <= 32):
raise MovieFormatError("rect outside a 32x32 tile: %r" % (rect,))
value = (rect.x0 | (rect.y0 << 5) | ((rect.w - 1) << 10)
| ((rect.h - 1) << 15))
return value.to_bytes(3, "little")
def _unpack_rect(data: bytes, offset: int) -> tuple[Rect, int]:
if offset + 3 > len(data):
raise MovieFormatError("truncated rect")
value = data[offset] | (data[offset + 1] << 8) | (data[offset + 2] << 16)
rect = Rect(
x0=value & 0x1F,
y0=(value >> 5) & 0x1F,
w=((value >> 10) & 0x1F) + 1,
h=((value >> 15) & 0x1F) + 1,
)
return rect, offset + 3
def encode_frame(record: FrameRecord) -> bytes:
if len(record.tiles) > 0xFFFF:
raise MovieFormatError("too many dirty tiles in one frame")
out = bytearray(struct.pack("<H", len(record.tiles)))
for tile in record.tiles:
if not (0 <= tile.tile_index <= 0xFFFF):
raise MovieFormatError("tile index out of range")
if len(tile.rects) > 0x7FFF:
raise MovieFormatError("too many rects in one tile")
out += struct.pack("<HH", tile.tile_index,
((tile.bg & 1) << 15) | len(tile.rects))
for rect in tile.rects:
out += _pack_rect(rect)
return bytes(out)
def decode_frame(data: bytes, offset: int) -> tuple[FrameRecord, int]:
if offset < 0 or offset + 2 > len(data):
raise MovieFormatError("truncated frame count")
count = data[offset] | (data[offset + 1] << 8)
offset += 2
tiles: list[TileRecord] = []
for _ in range(count):
if offset + 4 > len(data):
raise MovieFormatError("truncated tile header")
tile_index = data[offset] | (data[offset + 1] << 8)
packed = data[offset + 2] | (data[offset + 3] << 8)
offset += 4
bg = (packed >> 15) & 1
rect_count = packed & 0x7FFF
if offset + rect_count * 3 > len(data):
raise MovieFormatError("truncated tile rects")
rects: list[Rect] = []
for _ in range(rect_count):
rect, offset = _unpack_rect(data, offset)
rects.append(rect)
tiles.append(TileRecord(tile_index, bg, rects))
return FrameRecord(1, 0, tiles), offset
def write_movie(path: str, frames: Sequence[FrameRecord],
header: MovieHeader) -> int:
"""Write the movie; return its size. Refuses anything over the cap."""
records = [encode_frame(frame) for frame in frames]
count = len(records)
index_offset = HEADER_SIZE
data_offset = _align4(index_offset + INDEX_ENTRY_SIZE * count)
total = data_offset + sum(len(record) for record in records)
if total > MAX_DATA_BYTES:
raise MovieFormatError(
"movie is %d bytes, over the %d-byte data cap"
% (total, MAX_DATA_BYTES))
out = bytearray(total)
out[0:HEADER_SIZE] = MovieHeader(
header.width, header.height, header.scale, header.tile,
header.tiles_x, header.tiles_y, count,
header.source_fps, header.display_hz).pack()
offset = data_offset
for i, (frame, record) in enumerate(zip(frames, records)):
struct.pack_into("<IBB", out, index_offset + i * INDEX_ENTRY_SIZE,
offset, frame.ticks & 0xFF, frame.flags & 0xFF)
out[offset:offset + len(record)] = record
offset += len(record)
tmp = path + ".tmp"
with open(tmp, "wb") as handle:
handle.write(out)
os.replace(tmp, path)
return total
def read_movie(path: str) -> tuple[MovieHeader, list[FrameRecord]]:
with open(path, "rb") as handle:
data = handle.read()
if len(data) < HEADER_SIZE:
raise MovieFormatError("truncated header")
fields = struct.unpack("<4s7H3I3H", data[:HEADER_SIZE])
(magic, version, width, height, scale, tile, tiles_x, tiles_y,
frame_count, index_offset, data_offset, source_fps, display_hz,
_reserved) = fields
if magic != MAGIC or version != VERSION:
raise MovieFormatError("bad magic or version")
header = MovieHeader(width, height, scale, tile, tiles_x, tiles_y,
frame_count, source_fps, display_hz)
if index_offset + frame_count * INDEX_ENTRY_SIZE > len(data):
raise MovieFormatError("truncated index")
frames: list[FrameRecord] = []
for i in range(frame_count):
offset, ticks, flags = struct.unpack_from(
"<IBB", data, index_offset + i * INDEX_ENTRY_SIZE)
if offset < data_offset or offset > len(data):
raise MovieFormatError("frame offset outside data area")
record, _ = decode_frame(data, offset)
frames.append(FrameRecord(ticks, flags, record.tiles))
return header, frames
+15
View File
@@ -0,0 +1,15 @@
{
"start": 0.0,
"duration": 30.0,
"fps": 60,
"width": 640,
"height": 512,
"threshold": 127,
"temporal_median": 3,
"blur": 3,
"tile": 32,
"scale": 2,
"rdp_epsilon": 1.5,
"min_segment": 2,
"min_component": 8
}
+25
View File
@@ -0,0 +1,25 @@
"""Write human-viewable previews of the logical frames."""
from __future__ import annotations
import os
from typing import Sequence
import numpy as np
from PIL import Image
def write_previews(out_dir: str, frames: Sequence[np.ndarray],
indices: Sequence[int], scale: int = 4) -> list[str]:
os.makedirs(out_dir, exist_ok=True)
written: list[str] = []
for index in indices:
if index < 0 or index >= len(frames):
continue
gray = np.where(frames[index], 0, 255).astype(np.uint8)
image = Image.fromarray(gray, mode="L")
image = image.resize((gray.shape[1] * scale, gray.shape[0] * scale),
Image.Resampling.NEAREST)
path = os.path.join(out_dir, "frame_%04d.png" % index)
image.save(path)
written.append(path)
return written
+81
View File
@@ -0,0 +1,81 @@
"""Turn binary frames into per-tile background + rectangle deltas."""
from __future__ import annotations
import numpy as np
from .movie import FrameRecord, Rect, TileRecord
def _row_runs(row: np.ndarray) -> list[tuple[int, int]]:
"""Inclusive (start, end) runs of True in one row."""
padded = np.concatenate(([0], row.astype(np.int8), [0]))
diff = np.diff(padded)
starts = np.where(diff == 1)[0]
ends = np.where(diff == -1)[0] - 1
return list(zip(starts.tolist(), ends.tolist()))
def decompose_tile(tile: np.ndarray) -> tuple[int, list[Rect]]:
"""Majority background plus rectangles of the minority colour."""
tile = tile.astype(bool)
if tile.ndim != 2:
raise ValueError("a tile must be 2-D")
height, width = tile.shape
black = int(tile.sum())
bg = 1 if black > tile.size // 2 else 0
fg = tile if bg == 0 else ~tile
active: dict[tuple[int, int], int] = {}
rects: list[Rect] = []
for y in range(height):
runs = set(_row_runs(fg[y]))
for key in list(active):
if key not in runs:
y0 = active.pop(key)
rects.append(Rect(key[0], y0, key[1] - key[0] + 1, y - y0))
for key in runs:
if key not in active:
active[key] = y
for key, y0 in active.items():
rects.append(Rect(key[0], y0, key[1] - key[0] + 1, height - y0))
rects.sort(key=lambda r: (r.y0, r.x0))
return bg, rects
def diff_tiles(prev: np.ndarray, cur: np.ndarray,
tile: int = 32) -> list[TileRecord]:
"""The tiles whose pixels changed, with the new tile's rectangle list."""
if prev.shape != cur.shape:
raise ValueError("prev and cur must have the same shape")
height, width = cur.shape
if height % tile or width % tile:
raise ValueError("frame dimensions must be a multiple of the tile")
tiles_x = width // tile
out: list[TileRecord] = []
for ty in range(height // tile):
for tx in range(tiles_x):
y = ty * tile
x = tx * tile
old = prev[y:y + tile, x:x + tile]
new = cur[y:y + tile, x:x + tile]
if np.array_equal(old, new):
continue
bg, rects = decompose_tile(new)
out.append(TileRecord(ty * tiles_x + tx, bg, rects))
return out
def apply_record(canvas: np.ndarray, record: FrameRecord,
tile: int = 32) -> None:
"""Apply one frame record to a logical canvas in place."""
height, width = canvas.shape
tiles_x = width // tile
for rec in record.tiles:
ty, tx = divmod(rec.tile_index, tiles_x)
y = ty * tile
x = tx * tile
block = canvas[y:y + tile, x:x + tile]
block[:, :] = bool(rec.bg)
for rect in rec.rects:
block[rect.y0:rect.y0 + rect.h,
rect.x0:rect.x0 + rect.w] = not bool(rec.bg)
+180
View File
@@ -0,0 +1,180 @@
"""Turn a binary frame into simplified straight-edged polygons, and back."""
from __future__ import annotations
import json
import os
import subprocess
import tempfile
from typing import Iterable, Sequence
import numpy as np
Point = tuple[float, float]
Ring = list[Point]
Polygon = list[Ring]
def _distance_to_line(p: Point, a: Point, b: Point) -> float:
if a == b:
return ((p[0] - a[0]) ** 2 + (p[1] - a[1]) ** 2) ** 0.5
ax, ay = a
bx, by = b
px, py = p
dx, dy = bx - ax, by - ay
return abs(dy * px - dx * py + bx * ay - by * ax) / ((dx * dx + dy * dy) ** 0.5)
def rdp(points: Sequence[Point], epsilon: float) -> list[Point]:
"""Ramer-Douglas-Peucker on an open polyline."""
pts = list(points)
if len(pts) <= 2:
return pts
a, b = pts[0], pts[-1]
worst = 0.0
index = 0
for i in range(1, len(pts) - 1):
d = _distance_to_line(pts[i], a, b)
if d > worst:
worst, index = d, i
if worst <= epsilon:
return [a, b]
left = rdp(pts[:index + 1], epsilon)
right = rdp(pts[index:], epsilon)
return left[:-1] + right
def rdp_closed(points: Sequence[Point], epsilon: float) -> list[Point]:
"""Simplify a closed ring, keeping it closed."""
pts = list(points)
if len(pts) > 1 and pts[0] == pts[-1]:
pts = pts[:-1]
if len(pts) < 3:
return list(points)
# Split at the point farthest from the first so the endpoints are not
# coincident (RDP degenerates when a == b).
far = max(range(1, len(pts)),
key=lambda i: (pts[i][0] - pts[0][0]) ** 2
+ (pts[i][1] - pts[0][1]) ** 2)
first = rdp(pts[:far + 1], epsilon)
second = rdp(pts[far:] + [pts[0]], epsilon)
ring = first[:-1] + second
if ring[0] != ring[-1]:
ring.append(ring[0])
return ring
def parse_potrace_geojson(text: str) -> list[Polygon]:
"""Parse potrace's GeoJSON into polygons of (outer, holes...) rings."""
data = json.loads(text)
out: list[Polygon] = []
for feature in data.get("features", []):
geometry = feature.get("geometry") or {}
kind = geometry.get("type")
coordinates = geometry.get("coordinates") or []
if kind == "Polygon":
polygons = [coordinates]
elif kind == "MultiPolygon":
polygons = coordinates
else:
continue
for rings in polygons:
poly: Polygon = []
for ring in rings:
poly.append([(float(x), float(y)) for x, y in ring])
if poly:
out.append(poly)
return out
def trace(mask: np.ndarray, params: dict) -> list[Polygon]:
"""Trace a black mask into simplified straight-line polygons."""
if mask.ndim != 2:
raise ValueError("trace wants a 2-D mask")
height, width = mask.shape
if width % 8:
raise ValueError("potrace input width must be a multiple of 8")
epsilon = float(params.get("rdp_epsilon", 1.5))
min_segment = float(params.get("min_segment", 2))
pbm = np.packbits(mask.astype(bool), axis=1, bitorder="big").tobytes()
with tempfile.TemporaryDirectory(prefix="ba-trace-") as tmp:
pbm_path = os.path.join(tmp, "frame.pbm")
json_path = os.path.join(tmp, "frame.json")
with open(pbm_path, "wb") as handle:
handle.write(b"P4\n%d %d\n" % (width, height))
handle.write(pbm)
subprocess.run(
["potrace", "-b", "geojson", "-a", "0", "-t", "4", "-u", "1",
"-o", json_path, pbm_path],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
with open(json_path, "r", encoding="utf-8") as handle:
polygons = parse_potrace_geojson(handle.read())
# potrace's GeoJSON uses a y-up coordinate system; the framebuffer is
# y-down. Without this the whole movie is vertically mirrored.
flipped: list[Polygon] = []
for poly in polygons:
rings: Polygon = []
for ring in poly:
rings.append([(x, float(height) - y) for x, y in ring])
flipped.append(rings)
polygons = flipped
simplified: list[Polygon] = []
for poly in polygons:
rings: Polygon = []
for ring in poly:
ring = rdp_closed(ring, epsilon)
if len(ring) < 4:
continue
longest = max(
((ring[i][0] - ring[i - 1][0]) ** 2
+ (ring[i][1] - ring[i - 1][1]) ** 2) ** 0.5
for i in range(1, len(ring))
)
if longest < min_segment:
continue
rings.append(ring)
if rings:
simplified.append(rings)
return simplified
def rasterize(polygons: Iterable[Polygon], width: int, height: int) -> np.ndarray:
"""Even-odd scanline fill of polygons (outer rings and holes)."""
out = np.zeros((height, width), dtype=bool)
edges: list[tuple[float, float, float, float]] = []
for poly in polygons:
for ring in poly:
for i in range(1, len(ring)):
x0, y0 = ring[i - 1]
x1, y1 = ring[i]
if y0 == y1:
continue
edges.append((x0, y0, x1, y1))
if not edges:
return out
for y in range(height):
scan = y + 0.5
xs: list[float] = []
for x0, y0, x1, y1 in edges:
if (y0 <= scan < y1) or (y1 <= scan < y0):
xs.append(x0 + (scan - y0) * (x1 - x0) / (y1 - y0))
xs.sort()
for i in range(0, len(xs) - 1, 2):
left, right = xs[i], xs[i + 1]
start = max(0, int(np.ceil(left - 0.5)))
end = min(width - 1, int(np.floor(right - 0.5)))
if end >= start:
out[y, start:end + 1] = True
return out
def vectorize_frame(mask: np.ndarray, params: dict) -> np.ndarray:
"""Trace and re-rasterise one logical frame."""
height, width = mask.shape
return rasterize(trace(mask, params), width, height)
+132
View File
@@ -0,0 +1,132 @@
#!/bin/sh
# Build the Bad Apple payload with the 1987 TI TMS34010 SDK under DOSBox.
#
# .text at bit 0x018C0000, .bss at bit 0x01C00000 (the free block the rdebug
# loader already uses). Outputs: build/player.coff, player.bin, player.env.
set -eu
ROOT=$(cd "$(dirname "$0")/../.." && pwd)
DIR=$(cd "$(dirname "$0")" && pwd)
WORK="$DIR/work"
OUT="$DIR/build"
IMG="$ROOT/tools/tms34010-sdk/1987-05 TMS34010 SDK/1987-11-06 TMS34010 Simulator-C Tools-ASM Tools.img"
DOSBOX_CONF="$ROOT/harness/sdk-scratch/dosbox-bench.conf"
TEXT=018C0000
BSS=01C00000
NET=${NET:-0}
if [ "$NET" = 1 ]; then
NAME=player-net
else
NAME=player
fi
mkdir -p "$WORK" "$OUT"
if [ ! -x "$WORK/GSPCG.EXE" ]; then
mcopy -s -n -i "$IMG" "::/A2/*" "$WORK/"
mcopy -s -n -i "$IMG" "::/C2/*" "$WORK/"
fi
cp "$DIR/main.c" "$WORK/MAIN.C"
cp "$DIR/movie.c" "$WORK/MOVIE.C"
cp "$DIR/movie.h" "$WORK/MOVIE.H"
cp "$DIR/romapi.h" "$WORK/ROMAPI.H"
printf '#define NET_DEFAULT %s\n' "$NET" > "$WORK/MODECFG.H"
cp "$DIR/fill.asm" "$WORK/FILL.ASM"
cp "$DIR/entry.asm" "$WORK/ENTRY.ASM"
{
printf '%s\n' '-a'
printf '%s\n' '-o PLAYER.out'
printf '%s\n' '-m PLAYER.map'
printf '%s\n' '-e _start'
printf '%s\n' 'MAIN.obj'
printf '%s\n' 'MOVIE.obj'
printf '%s\n' 'FILL.obj'
printf '%s\n' 'ENTRY.obj'
printf '%s\n' '-l RTS.LIB'
printf '%s\n' 'MEMORY'
printf '%s\n' '{'
printf '%s\n' " prog: origin = ${TEXT}h, length = 20000h"
printf '%s\n' " ram: origin = ${BSS}h, length = 200000h"
printf '%s\n' '}'
printf '%s\n' 'SECTIONS'
printf '%s\n' '{'
printf '%s\n' ' .text: {} > prog'
printf '%s\n' ' .data: {} > prog'
printf '%s\n' ' .cinit: {} > prog'
printf '%s\n' ' .bss: {} > ram'
printf '%s\n' '}'
} > "$WORK/PLAYER.CMD"
sed -i 's/$/\r/' "$WORK/PLAYER.CMD"
{
printf 'gspcpp MAIN\r\n'
printf 'gspcc MAIN\r\n'
printf 'gspcg MAIN\r\n'
printf 'gspa MAIN MAIN.obj MAIN.lst -l\r\n'
printf 'gspcpp MOVIE\r\n'
printf 'gspcc MOVIE\r\n'
printf 'gspcg MOVIE\r\n'
printf 'gspa MOVIE MOVIE.obj MOVIE.lst -l\r\n'
printf 'gspa FILL FILL.obj FILL.lst -l\r\n'
printf 'gspa ENTRY ENTRY.obj ENTRY.lst -l\r\n'
printf 'gsplnk PLAYER.cmd\r\n'
printf 'exit\r\n'
} > "$WORK/BUILD.BAT"
rm -f "$WORK"/MAIN.OBJ "$WORK"/MOVIE.OBJ "$WORK"/FILL.OBJ \
"$WORK"/ENTRY.OBJ "$WORK"/PLAYER.OUT "$WORK"/PLAYER.MAP \
"$WORK"/MAIN.LST "$WORK"/MOVIE.LST "$WORK"/FILL.LST \
"$WORK"/ENTRY.LST
SDL_VIDEODRIVER=dummy SDL_AUDIODRIVER=dummy \
timeout 240 dosbox -conf "$DOSBOX_CONF" -noprimaryconf -nolocalconf \
-c "mount c $WORK" -c "c:" \
-c "BUILD.BAT" \
-c "exit" -exit > "$WORK/dosbox.log" 2>&1
if [ ! -f "$WORK/PLAYER.OUT" ]; then
echo "build.sh: GSPLNK produced no PLAYER.OUT" >&2
sed 's/\r$//' "$WORK/PLAYER.MAP" 2>/dev/null | tail -20 >&2 || true
for f in MAIN.LST MOVIE.LST FILL.LST ENTRY.LST; do
sed 's/\r$//' "$WORK/$f" 2>/dev/null | grep -i 'error' \
| grep -vi 'no errors' >&2 || true
done
exit 1
fi
for f in MAIN.LST MOVIE.LST FILL.LST ENTRY.LST; do
if sed 's/\r$//' "$WORK/$f" 2>/dev/null | grep -i 'error' \
| grep -qvi 'no errors'; then
echo "build.sh: $f diagnostics:" >&2
sed 's/\r$//' "$WORK/$f" | grep -in 'error' \
| grep -vi 'no errors' >&2
exit 1
fi
done
cp "$WORK/PLAYER.OUT" "$OUT/player.coff"
sed 's/\r$//' "$WORK/PLAYER.MAP" > "$OUT/player.map"
sed 's/\r$//' "$WORK/MAIN.LST" > "$OUT/main.lst" 2>/dev/null || true
sed 's/\r$//' "$WORK/MOVIE.LST" > "$OUT/movie.lst" 2>/dev/null || true
sed 's/\r$//' "$WORK/FILL.LST" > "$OUT/fill.lst" 2>/dev/null || true
python3 "$ROOT/harness/hello-world/cofextract.py" \
"$WORK/PLAYER.OUT" "$OUT/$NAME.bin" > "$OUT/cofextract.txt"
cp "$WORK/PLAYER.OUT" "$OUT/$NAME.coff"
ENTRY_PTR=$(sed 's/\r$//' "$WORK/PLAYER.MAP" \
| awk '/ENTRY POINT SYMBOL/ { print $NF; exit }')
MARK_PTR=$(sed 's/\r$//' "$WORK/PLAYER.MAP" \
| awk '$2 == "_marker_page" { print $1; exit }')
if [ -z "$ENTRY_PTR" ] || [ -z "$MARK_PTR" ]; then
echo "build.sh: no entry point or _marker_page in PLAYER.map" >&2
exit 1
fi
printf 'ENTRY_PTR=0x%s\nMARK_PTR=0x%s\n' "$ENTRY_PTR" "$MARK_PTR" \
> "$OUT/$NAME.env"
echo "build.sh: $OUT/$NAME.bin ($(wc -c < "$OUT/$NAME.bin") bytes)"
grep -E "ENTRY POINT|^\.(text|cinit|bss)" "$OUT/player.map" | head -8
echo "build.sh: entry bit 0x$ENTRY_PTR"
+79
View File
@@ -0,0 +1,79 @@
; entry.asm -- payload entry and ROM socket/allocator thunks.
;
; The payload is entered by a debugger/rdebug jump and never returns. Set the
; C stack, call main, and spin if main ever returns.
;
; The firmware's wrappers end in RETS 2 (they pop the return address and two
; argument words, and restore A14 from a caller-pushed slot). A plain calla
; would leave SP off by one word; these thunks save/restore SP and A14 exactly
; as harness/udpstream/entry.asm does.
.text
.globl _start
.globl _main
_start:
setf 32, 0, 1
movi _cstack, a14
move a14, a13
calla _main
spin:
jruc spin
.globl _socket_
_socket_:
move sp, @_sys_sp, 1
move a14, @_sys_a14, 1
calla 3F5C9A00h
move @_sys_sp, sp, 1
move @_sys_a14, a14, 1
rets 2
.globl _bind_
_bind_:
move sp, @_sys_sp, 1
move a14, @_sys_a14, 1
calla 3F5CAA80h
move @_sys_sp, sp, 1
move @_sys_a14, a14, 1
rets 2
.globl _recvfrom_
_recvfrom_:
move sp, @_sys_sp, 1
move a14, @_sys_a14, 1
calla 3F5D7D50h
move @_sys_sp, sp, 1
move @_sys_a14, a14, 1
rets 2
.globl _close_
_close_:
move sp, @_sys_sp, 1
move a14, @_sys_a14, 1
calla 3F5E8930h
move @_sys_sp, sp, 1
move @_sys_a14, a14, 1
rets 2
.globl _malloc_
_malloc_:
move sp, @_sys_sp, 1
move a14, @_sys_a14, 1
calla 3F701230h
move @_sys_sp, sp, 1
move @_sys_a14, a14, 1
rets 2
.globl _delay_
_delay_:
move sp, @_sys_sp, 1
move a14, @_sys_a14, 1
calla 3F833A20h
move @_sys_sp, sp, 1
move @_sys_a14, a14, 1
rets 2
.bss _cstack, 32768, 16 ; 32768 bits = 4096 bytes
.bss _sys_sp, 32, 16
.bss _sys_a14, 32, 16
.end
+61
View File
@@ -0,0 +1,61 @@
; fill.asm -- one FILL L rectangle into the 1280x1024, 256-byte-stride VRAM.
;
; The C wrapper publishes:
; fill_daddr linear bit address of the top-left pixel
; fill_dydx (rows << 16) | columns, in screen pixels
; fill_color 0x0000 = black, 0xFFFF = white
;
; FILL L destroys DADDR and DYDX, so the wrapper reloads them every call.
; PSIZE, PMASK and CONTROL are programmed once by _fill_init_.
.ref _fill_daddr
.ref _fill_dydx
.ref _fill_color
.text
.globl _fill_run_
_fill_run_:
setf 32, 0, 1
move b2, -*sp, 1
move b3, -*sp, 1
move b7, -*sp, 1
move b9, -*sp, 1
move @_fill_daddr, b2, 1
movi 800h, b3 ; 2048 bits = 256-byte row pitch
move @_fill_dydx, b7, 1
move @_fill_color, b9, 1
; PSIZE is cheap to re-assert; if the resident OS left it at 2 bpp,
; FILL would write twice as many pixels as asked for.
movi 1, a1
move a1, @0C0000150h, 0
fill l
move *sp+, b9, 1
move *sp+, b7, 1
move *sp+, b3, 1
move *sp+, b2, 1
rets
.globl _vid_dint_
_vid_dint_:
dint
rets
.globl _fill_init_
_fill_init_:
setf 32, 0, 1
movi 1, a1
move a1, @0C0000150h, 0 ; PSIZE = 1
clr a1
move a1, @0C0000160h, 0 ; PMASK = 0
move a1, @0C0000170h, 0 ; upper half, forward compatibility
move @0C00000B0h, a1, 0 ; CONTROL
andi 0801Fh, a1 ; keep CD and RM/RR, raster op 0
move a1, @0C00000B0h, 0
move @0C00000B0h, a1, 0 ; read back: I/O writes are pipelined
rets
.end
+279
View File
@@ -0,0 +1,279 @@
/* main.c -- Bad Apple payload: network receive, then tile playback. */
#include "movie.h"
#include "romapi.h"
#include "modecfg.h"
#ifndef NET_DEFAULT
#define NET_DEFAULT 0
#endif
#define NET_LEN 2401942
#define NET_PORT 7001
#define MOVIE_BASE 0x00080000
#define MOVIE_MAX 0x00308000
#define VRAM_BASE 0x04000000
#define ROW_BITS 2048
/* The marker/control block lives in .bss, not at bit 0x01810000: a 2.4 MB
* heap allocation can cross that address and would corrupt the movie. */
unsigned int marker_page[16];
#define MARK ((unsigned int)marker_page)
#define CTRL_START (MARK + 0x20 * 8)
#define CTRL_COUNT (MARK + 0x24 * 8)
#define CTRL_PACE (MARK + 0x28 * 8)
#define CTRL_MOVIE (MARK + 0x2C * 8)
#define CTRL_MOVIELEN (MARK + 0x30 * 8)
#define CTRL_WAIT (MARK + 0x34 * 8)
#define CTRL_READY (MARK + 0x38 * 8)
#define CTRL_NET (MARK + 0x3C * 8)
#define CTRL_MAGIC (MARK + 0x1C * 8)
#define MAGIC_VALUE 0x42414450
extern void fill_run_();
extern void fill_init_();
extern void vid_dint_();
unsigned int fill_daddr;
unsigned int fill_dydx;
unsigned int fill_color;
void fill_rect(x, y, w, h, color)
int x;
int y;
int w;
int h;
int color;
{
if (w <= 0 || h <= 0) {
return;
}
fill_daddr = VRAM_BASE + (unsigned int)y * ROW_BITS + (unsigned int)x;
fill_dydx = ((unsigned int)h << 16) | (unsigned int)w;
fill_color = color ? 0x0000 : 0xFFFF;
fill_run_();
}
static void paint(x, y, w, h, color, ctx)
int x;
int y;
int w;
int h;
int color;
void *ctx;
{
fill_rect(x, y, w, h, color);
}
static void show_fault(code)
unsigned int code;
{
fill_init_();
fill_rect(0, 0, 1280, 64, 1);
*(unsigned int *)MARK = code;
for (;;) {
}
}
static unsigned char rxbuf[1600];
static unsigned char sa[16];
static unsigned char from[16];
static int fromlen;
static unsigned int rx_error;
/* Receive movie.bin over the OS UDP stack into a malloc'd bit buffer.
* Packet: u32 offset, u16 length, u16 seq, u16 crc16, then bytes.
* A final offset==expected/length==4 packet carries the 32-bit file checksum. */
static unsigned int receive_movie(expected)
unsigned int expected;
{
unsigned int ptr;
unsigned int offset;
unsigned int length;
unsigned int sent_sum;
unsigned int i;
unsigned char *dst;
int fd;
int n;
ptr = malloc_(expected);
if (ptr == 0) {
rx_error = 3;
return 0;
}
fd = socket_(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
if (fd < 0) {
rx_error = 4;
return 0;
}
sa_any(sa, NET_PORT);
if (bind_(fd, sa, SOCKADDR_LEN) < 0) {
close_(fd);
rx_error = 5;
return 0;
}
sent_sum = 0;
dst = (unsigned char *)ptr;
for (;;) {
fromlen = SOCKADDR_LEN;
n = recvfrom_(fd, rxbuf, (int)sizeof(rxbuf), 0, from, &fromlen);
if (n < 10) {
continue;
}
offset = (unsigned int)rxbuf[0]
| ((unsigned int)rxbuf[1] << 8)
| ((unsigned int)rxbuf[2] << 16)
| ((unsigned int)rxbuf[3] << 24);
length = (unsigned int)rxbuf[4] | ((unsigned int)rxbuf[5] << 8);
if (offset == expected && length == 4 && n >= 14) {
sent_sum = (unsigned int)rxbuf[10]
| ((unsigned int)rxbuf[11] << 8)
| ((unsigned int)rxbuf[12] << 16)
| ((unsigned int)rxbuf[13] << 24);
if (movie_checksum(dst, expected) == sent_sum) {
break;
}
sent_sum = 0;
continue;
}
if (length > (unsigned int)(n - 10)) {
continue;
}
if (offset > expected || length > expected - offset) {
continue;
}
for (i = 0; i < length; i++) {
dst[offset + i] = rxbuf[10 + i];
}
}
close_(fd);
if (sent_sum == 0) {
rx_error = 6;
return 0;
}
return ptr;
}
/* Poll the display interrupt pending bit. Interrupts are disabled; this is
* a status bit, not an ISR. A missing DPYINT only costs a bounded spin. */
static void wait_vsync()
{
unsigned int i;
unsigned short pending;
*(unsigned short *)0xC0000120 = 0;
for (i = 0; i < 200000; i++) {
pending = *(unsigned short *)0xC0000120;
if (pending & 0x0400) {
return;
}
}
}
int main()
{
struct movie_header header;
unsigned int magic;
unsigned int frame;
unsigned int last;
unsigned int start;
unsigned int count;
unsigned int pace;
unsigned int net;
unsigned int wait;
unsigned int movie_base;
unsigned int movie_len;
unsigned int ticks;
unsigned int now;
unsigned int next;
int rc;
magic = *(unsigned int *)CTRL_MAGIC;
if (magic == MAGIC_VALUE) {
start = *(unsigned int *)CTRL_START;
count = *(unsigned int *)CTRL_COUNT;
pace = *(unsigned int *)CTRL_PACE;
net = *(unsigned int *)CTRL_NET;
wait = *(unsigned int *)CTRL_WAIT;
movie_base = *(unsigned int *)CTRL_MOVIE;
movie_len = *(unsigned int *)CTRL_MOVIELEN;
} else {
start = 0;
count = 0;
pace = 1;
net = NET_DEFAULT;
wait = 0;
movie_base = MOVIE_BASE;
movie_len = MOVIE_MAX;
}
if (net == 0 && NET_DEFAULT) {
net = 1;
}
if (net != 0) {
movie_len = NET_LEN;
*(unsigned int *)(MARK + 0x10 * 8) = 0x20;
movie_base = receive_movie(movie_len);
if (movie_base == 0) {
show_fault(0xBAD00000 | rx_error);
}
vid_dint_();
} else {
vid_dint_();
if (wait != 0) {
*(unsigned int *)(MARK + 0x10 * 8) = 0x10;
while (*(unsigned int *)CTRL_READY == 0) {
}
}
}
fill_init_();
*(unsigned short *)0xC00000A0 = 1;
fill_rect(0, 0, 1280, 1024, 0);
rc = movie_open((unsigned char *)movie_base, movie_len, &header);
if (rc != 0) {
show_fault(0xBAD00001);
}
last = header.frame_count;
if (last != 0) {
last = last - 1;
}
if (count != 0 && start + count - 1 < last) {
last = start + count - 1;
}
now = 0;
next = 0;
frame = 0;
while (frame <= last) {
if (pace != 0 && frame >= start) {
wait_vsync();
now += 5; /* 60/72 in 1/360 s units */
if (now < next) {
continue;
}
}
rc = movie_decode_frame((unsigned char *)movie_base, movie_len, frame,
&header, paint, 0);
if (rc != 0) {
show_fault(0xBAD00002);
}
ticks = movie_frame_ticks((unsigned char *)movie_base, movie_len,
frame, &header);
if (ticks < 1) {
ticks = 1;
}
next += 6 * ticks;
if (next + 360 < now) {
next = now;
}
*(unsigned int *)MARK = frame;
frame++;
}
*(unsigned int *)(MARK + 0x10 * 8) = (count != 0) ? 2 : 1;
for (;;) {
}
}
+194
View File
@@ -0,0 +1,194 @@
/* movie.c -- pure decoder for the BAPL tile-delta movie format.
*
* K&R C: the 1987 TMS34010 compiler has no const, no ANSI prototypes and no
* stdint.h. The host test compiles this same file with gcc -std=c99.
*/
#include "movie.h"
#define HEADER_SIZE 36
#define INDEX_ENTRY_SIZE 6
static unsigned int rd16(p)
unsigned char *p;
{
return (unsigned int)p[0] | ((unsigned int)p[1] << 8);
}
static unsigned long rd32(p)
unsigned char *p;
{
return (unsigned long)p[0] | ((unsigned long)p[1] << 8)
| ((unsigned long)p[2] << 16) | ((unsigned long)p[3] << 24);
}
int movie_open(data, len, header)
unsigned char *data;
unsigned long len;
struct movie_header *header;
{
unsigned int version;
if (data == 0 || header == 0 || len < HEADER_SIZE) {
return -1;
}
if (data[0] != 'B' || data[1] != 'A' || data[2] != 'P' || data[3] != 'L') {
return -1;
}
version = rd16(data + 4);
if (version != 1) {
return -1;
}
header->width = rd16(data + 6);
header->height = rd16(data + 8);
header->scale = rd16(data + 10);
header->tile = rd16(data + 12);
header->tiles_x = rd16(data + 14);
header->tiles_y = rd16(data + 16);
header->frame_count = (unsigned int)rd32(data + 18);
header->index_offset = (unsigned int)rd32(data + 22);
header->data_offset = (unsigned int)rd32(data + 26);
header->source_fps = rd16(data + 30);
header->display_hz = rd16(data + 32);
if (header->scale == 0 || header->scale > 4 || header->tile == 0
|| header->tile > 32 || header->tiles_x == 0
|| header->tiles_y == 0) {
return -1;
}
if (header->width != header->tiles_x * header->tile
|| header->height != header->tiles_y * header->tile
|| header->width * header->scale > 1280
|| header->height * header->scale > 1024) {
return -1;
}
if ((unsigned long)header->index_offset
+ (unsigned long)header->frame_count * INDEX_ENTRY_SIZE > len) {
return -1;
}
return 0;
}
unsigned int movie_frame_ticks(data, len, frame, header)
unsigned char *data;
unsigned long len;
unsigned int frame;
struct movie_header *header;
{
unsigned long index_off;
if (data == 0 || header == 0 || frame >= header->frame_count) {
return 1;
}
index_off = (unsigned long)header->index_offset
+ (unsigned long)frame * INDEX_ENTRY_SIZE;
if (index_off + INDEX_ENTRY_SIZE > len) {
return 1;
}
return (unsigned int)data[index_off + 4];
}
int movie_decode_frame(data, len, frame, header, fill, ctx)
unsigned char *data;
unsigned long len;
unsigned int frame;
struct movie_header *header;
movie_fill_fn fill;
void *ctx;
{
unsigned long index_off;
unsigned long off;
unsigned int count;
unsigned int i;
unsigned int j;
unsigned int tile_index;
unsigned int packed;
unsigned int bg;
unsigned int rects;
unsigned int tx;
unsigned int ty;
unsigned int value;
unsigned int x0;
unsigned int y0;
unsigned int w;
unsigned int h;
int sx;
int sy;
if (data == 0 || header == 0 || fill == 0) {
return -1;
}
if (frame >= header->frame_count) {
return -1;
}
index_off = (unsigned long)header->index_offset
+ (unsigned long)frame * INDEX_ENTRY_SIZE;
if (index_off + INDEX_ENTRY_SIZE > len) {
return -1;
}
off = rd32(data + index_off);
if (off + 2 > len) {
return -1;
}
count = rd16(data + off);
off += 2;
for (i = 0; i < count; i++) {
if (off + 4 > len) {
return -1;
}
tile_index = rd16(data + off);
packed = rd16(data + off + 2);
off += 4;
if (tile_index >= header->tiles_x * header->tiles_y) {
return -1;
}
bg = packed >> 15;
rects = packed & 0x7FFF;
if (off + (unsigned long)rects * 3 > len) {
return -1;
}
tx = tile_index % header->tiles_x;
ty = tile_index / header->tiles_x;
/* The whole tile first: the old pixels outside every new rect must
* go away. The C test skips this first call per tile. */
sx = (int)(tx * header->tile * header->scale);
sy = (int)(ty * header->tile * header->scale);
fill(sx, sy, (int)(header->tile * header->scale),
(int)(header->tile * header->scale), (int)bg, ctx);
for (j = 0; j < rects; j++) {
value = (unsigned int)data[off]
| ((unsigned int)data[off + 1] << 8)
| ((unsigned int)data[off + 2] << 16);
off += 3;
x0 = value & 0x1F;
y0 = (value >> 5) & 0x1F;
w = ((value >> 10) & 0x1F) + 1;
h = ((value >> 15) & 0x1F) + 1;
if (x0 + w > header->tile || y0 + h > header->tile) {
return -1;
}
sx = (int)((tx * header->tile + x0) * header->scale);
sy = (int)((ty * header->tile + y0) * header->scale);
fill(sx, sy, (int)(w * header->scale),
(int)(h * header->scale), (int)(1 - bg), ctx);
}
}
return 0;
}
unsigned int movie_checksum(data, len)
unsigned char *data;
unsigned long len;
{
unsigned long sum = 0;
unsigned long i;
if (data == 0) {
return 0;
}
for (i = 0; i < len; i++) {
sum = (sum + data[i]) & 0xFFFFFFFF;
}
return (unsigned int)sum;
}
+26
View File
@@ -0,0 +1,26 @@
#ifndef BADAPPLE_MOVIE_H
#define BADAPPLE_MOVIE_H
/* K&R declarations: the 1987 TI compiler has no ANSI prototypes. */
struct movie_header {
unsigned int width;
unsigned int height;
unsigned int scale;
unsigned int tile;
unsigned int tiles_x;
unsigned int tiles_y;
unsigned int frame_count;
unsigned int index_offset;
unsigned int data_offset;
unsigned int source_fps;
unsigned int display_hz;
};
typedef void (*movie_fill_fn)();
int movie_open();
int movie_decode_frame();
unsigned int movie_frame_ticks();
unsigned int movie_checksum();
#endif
+93
View File
@@ -0,0 +1,93 @@
/*
* romapi.h -- the VT1200 firmware entry points a C payload may call.
*
* Every address here is a TMS34010 BIT address (= Ghidra byte address x 8).
* The evidence for each is dwt/notes/TCPIP.md (sockets) and dwt/notes/HEAP.md
* (allocator); the ones marked "verified live" have been called from a payload
* on real hardware, not just read out of the ROM.
*
* K&R C only. This compiler rejects ANSI prototypes (`int f(int a)` is a
* syntax error), so the declarations below are old-style: the parameter lists
* live in the comments, and the compiler checks nothing but the name. That is
* the price of using the 1987 toolchain; keep the comments honest.
*
* Calling convention: the C compiler and the firmware agree -- arguments are
* widened to 32 bits and pushed upward on the program stack (A14), first
* argument last, and the result comes back in A8. The thunks in entry.asm
* forward with `calla`, so the caller's already-pushed arguments pass through
* untouched and only A8 has to survive.
*/
#ifndef ROMAPI_H
#define ROMAPI_H
/* ---- ROM entry points -------------------------------------------------- */
#define ROM_SOCKET 0x3F5C9A00 /* Ghidra 07EB9340 verified live */
#define ROM_BIND 0x3F5CAA80 /* Ghidra 07EB9550 BOOTP caller */
#define ROM_LISTEN 0x3F5CBCD0 /* Ghidra 07EB979A inetd caller */
#define ROM_ACCEPT 0x3F5CCB40 /* Ghidra 07EB9968 inetd caller */
#define ROM_SENDTO 0x3F5D3D70 /* Ghidra 07EBA7AE verified live */
#define ROM_RECVFROM 0x3F5D7D50 /* Ghidra 07EBAFAA verified live */
#define ROM_READ 0x3F5D8E70 /* Ghidra 07EBB1CE rdebug / Xlib */
#define ROM_CLOSE 0x3F5E8930 /* Ghidra 07EBD126 verified live */
#define ROM_SETSOCKOPT 0x3F5DD260 /* Ghidra 07EBBA4C BOOTP / Xlib */
#define ROM_DELAY 0x3F833A20 /* Ghidra 07F66744 sleep(ms), telnet caller */
#define ROM_SELECT 0x3F871CE0 /* Ghidra 07F0E39C Xlib caller, candidate */
#define ROM_MALLOC 0x3F701230 /* Ghidra 07EE0246 verified live */
#define ROM_FREE 0x3F702440 /* Ghidra 07EE0488 allocator */
#define ROM_CREATE_PROC 0x3F706A40 /* Ghidra 07EE0D48 GUI work */
/* ---- thunks implemented in entry.asm ----------------------------------- */
/*
* int socket_(domain, type, protocol) -> fd, or -1
* int bind_(fd, sockaddr *, len) -> 0, or -1
* int recvfrom_(fd, buf, len, flags, from *, fromlen *)
* -> bytes, or -1
* int close_(fd) -> 0, or -1
* unsigned int malloc_(bytes) -> BIT pointer, or 0
* int free_(bit pointer) -> 0
*
* `from` is a 16-byte sockaddr the stack fills in; `fromlen` points to an
* int that must be 16 on entry (the call writes back the real length).
*/
int socket_();
int bind_();
int recvfrom_();
int close_();
int listen_();
int accept_();
int read_();
int select_();
int delay_();
unsigned int malloc_();
int free_();
/* ---- socket constants -------------------------------------------------- */
#define AF_INET 2
#define SOCK_DGRAM 2
#define SOCK_STREAM 1
#define IPPROTO_UDP 0
#define SOCKADDR_LEN 16
/*
* struct sockaddr, 16 bytes, NETWORK byte order:
* +0 u16 family (AF_INET = 2 -> bytes 00 02)
* +2 u16 port (high byte first, e.g. 7000 = 0x1B58 -> 1B 58)
* +4 u32 addr (0 = INADDR_ANY)
* +8 u8 zero[8]
*/
/* Fill sa (an unsigned char[16]) with AF_INET, `port`, and INADDR_ANY.
* `port` is in host order so callers can write 7000 rather than 0x1B58. */
static void sa_any(sa, port)
unsigned char *sa;
unsigned int port;
{
sa[0] = 0; sa[1] = AF_INET;
sa[2] = (port >> 8) & 0xFF;
sa[3] = port & 0xFF;
sa[4] = 0; sa[5] = 0; sa[6] = 0; sa[7] = 0;
sa[8] = 0; sa[9] = 0; sa[10] = 0; sa[11] = 0;
sa[12] = 0; sa[13] = 0; sa[14] = 0; sa[15] = 0;
}
#endif /* ROMAPI_H */
Binary file not shown.
@@ -0,0 +1,85 @@
#include <stdio.h>
#include <stdlib.h>
#include "../movie.h"
struct ctx {
int frame;
int last_tile;
int have_last;
};
static void report_fill(int x, int y, int w, int h, int color, void *opaque)
{
struct ctx *ctx = (struct ctx *)opaque;
int lx = x / 2;
int ly = y / 2;
int lw = w / 2;
int lh = h / 2;
int tx = lx / 32;
int ty = ly / 32;
int tile = ty * 2 + tx;
/* The decoder's first call for a tile is its background. */
if (!ctx->have_last || tile != ctx->last_tile) {
ctx->last_tile = tile;
ctx->have_last = 1;
return;
}
printf("%d %d %d %d %d %d %d\n", ctx->frame, tile, 1 - color,
lx % 32, ly % 32, lw, lh);
}
int main(int argc, char **argv)
{
FILE *file;
long size;
unsigned char *data;
struct movie_header header;
unsigned int frame;
struct ctx ctx;
if (argc != 2) {
fprintf(stderr, "usage: %s movie.bin\n", argv[0]);
return 2;
}
file = fopen(argv[1], "rb");
if (file == 0) {
fprintf(stderr, "cannot open %s\n", argv[1]);
return 2;
}
fseek(file, 0, SEEK_END);
size = ftell(file);
fseek(file, 0, SEEK_SET);
data = (unsigned char *)malloc((size_t)size);
if (data == 0) {
fclose(file);
return 2;
}
if (fread(data, 1, (size_t)size, file) != (size_t)size) {
fclose(file);
free(data);
return 2;
}
fclose(file);
if (movie_open(data, (unsigned long)size, &header) != 0) {
fprintf(stderr, "movie_open failed\n");
free(data);
return 1;
}
printf("frames=%u\n", header.frame_count);
for (frame = 0; frame < header.frame_count; frame++) {
ctx.frame = (int)frame;
ctx.last_tile = -1;
ctx.have_last = 0;
if (movie_decode_frame(data, (unsigned long)size, frame, &header,
report_fill, &ctx) != 0) {
fprintf(stderr, "decode failed at frame %u\n", frame);
free(data);
return 1;
}
}
free(data);
return 0;
}
+2
View File
@@ -0,0 +1,2 @@
[pytest]
testpaths = tests
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""Write the two-frame fixture the C decoder test reads."""
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from pipeline.movie import (
FrameRecord,
MovieHeader,
Rect,
TileRecord,
write_movie,
)
OUT = ROOT / "player" / "tests" / "fixture.bin"
def main() -> int:
frames = [
FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 32, 32)])]),
FrameRecord(1, 0, [TileRecord(1, 1, [Rect(1, 1, 2, 3)])]),
]
header = MovieHeader(64, 32, 2, 32, 2, 1, len(frames), 60, 72)
OUT.parent.mkdir(parents=True, exist_ok=True)
write_movie(str(OUT), frames, header)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+21
View File
@@ -0,0 +1,21 @@
import json
import numpy as np
import pipeline.build as build
from pipeline.movie import read_movie
def test_build_tiny_movie(tmp_path, monkeypatch):
frames = [np.zeros((64, 64), dtype=bool) for _ in range(3)]
frames[1][8:24, 8:24] = True
monkeypatch.setattr(build, "extract_binary", lambda *a, **k: iter(frames))
monkeypatch.setattr(build, "vectorize_frame", lambda m, p: m)
out = tmp_path / "data"
manifest = build.build_movie("unused.mp4", str(out),
{"duration": 3, "fps": 60})
header, recs = read_movie(str(out / "movie.bin"))
assert header.frame_count == 3
assert len(recs) == 3
assert (out / "manifest.json").exists()
assert json.loads((out / "manifest.json").read_text())["frame_count"] == 3
+23
View File
@@ -0,0 +1,23 @@
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
def test_c_decoder_matches_python(tmp_path):
subprocess.run([sys.executable, str(ROOT / "tests" / "make_fixture.py")],
check=True)
exe = tmp_path / "decode"
subprocess.run([
"gcc", "-std=c99", "-Wall", "-Werror",
str(ROOT / "player" / "movie.c"),
str(ROOT / "player" / "tests" / "test_movie_decode.c"),
"-o", str(exe),
], check=True)
out = subprocess.run(
[str(exe), str(ROOT / "player" / "tests" / "fixture.bin")],
check=True, capture_output=True, text=True).stdout
assert "frames=2" in out
assert "0 0 0 0 0 32 32" in out
assert "1 1 1 1 1 2 3" in out
+26
View File
@@ -0,0 +1,26 @@
import numpy as np
from pipeline.cost import CostConstants, estimate_record_us, select_records
from pipeline.movie import FrameRecord, TileRecord
C = CostConstants()
def test_one_solid_tile_cost():
rec = FrameRecord(1, 0, [TileRecord(0, 1, [])])
# 64x64 screen tile = 256 words; 256*0.40 + 3us call
assert abs(estimate_record_us(rec, C) - (256 * 0.40 + 3.0)) < 0.01
def test_full_screen_is_over_budget():
recs = [TileRecord(i, 1, []) for i in range(320)]
assert estimate_record_us(FrameRecord(1, 0, recs), C) > 13900.0
def test_select_records_holds_time_not_records():
quiet = np.zeros((32, 32), dtype=bool)
busy = np.ones((32, 32), dtype=bool)
out = select_records([quiet, busy, quiet, quiet], C, budget_us=50.0)
assert all(r.ticks >= 1 for r in out)
assert sum(r.ticks for r in out) == 4
+31
View File
@@ -0,0 +1,31 @@
import numpy as np
from pipeline.frames import temporal_median3, box_blur3, binarize, pack_lsb
def test_temporal_median_removes_single_frame_flash():
a = np.zeros((4, 4), dtype=np.uint8)
b = np.zeros((4, 4), dtype=np.uint8)
b[1, 1] = 255
c = np.zeros((4, 4), dtype=np.uint8)
out = temporal_median3([a, b, c])
assert out[1, 1] == 0
def test_pack_lsb_is_leftmost_pixel_bit_zero():
mask = np.zeros((1, 8), dtype=bool)
mask[0, 0] = True
assert pack_lsb(mask) == b"\x01"
mask[0, 7] = True
assert pack_lsb(mask) == b"\x81"
def test_binarize_black_is_true():
gray = np.array([[0, 255]], dtype=np.uint8)
assert binarize(gray).tolist() == [[True, False]]
def test_box_blur_softens_a_single_pixel():
gray = np.zeros((3, 3), dtype=np.uint8)
gray[1, 1] = 9
out = box_blur3(gray)
assert out[1, 1] == 1
+63
View File
@@ -0,0 +1,63 @@
import numpy as np
from pipeline.movie import (
Rect,
TileRecord,
FrameRecord,
MovieHeader,
MovieFormatError,
encode_frame,
decode_frame,
write_movie,
read_movie,
)
def test_rect_full_tile_bias_roundtrip():
rec = FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 32, 32)])])
back, off = decode_frame(encode_frame(rec), 0)
assert back.tiles[0].rects[0] == Rect(0, 0, 32, 32)
assert off == len(encode_frame(rec))
def test_more_than_127_rects_roundtrip():
rects = [Rect(i % 32, i // 32, 1, 1) for i in range(200)]
rec = FrameRecord(1, 0, [TileRecord(3, 0, rects)])
back, _ = decode_frame(encode_frame(rec), 0)
assert len(back.tiles[0].rects) == 200
def test_rect_outside_tile_raises():
rec = FrameRecord(1, 0, [TileRecord(0, 0, [Rect(31, 0, 2, 1)])])
try:
encode_frame(rec)
except MovieFormatError:
return
assert False, "rect crossing a tile boundary was accepted"
def test_truncated_record_raises():
data = encode_frame(
FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 4, 4)])]))
try:
decode_frame(data[:-1], 0)
except MovieFormatError:
return
assert False, "truncated record was accepted"
def test_write_movie_refuses_over_cap(tmp_path):
frames = [FrameRecord(1, 0, []) for _ in range(10)]
header = MovieHeader(640, 512, 2, 32, 20, 16, 10, 60, 72)
import pipeline.movie as m
old = m.MAX_DATA_BYTES
m.MAX_DATA_BYTES = 8
try:
try:
write_movie(str(tmp_path / "x.bin"), frames, header)
except MovieFormatError:
return
assert False, "cap not enforced"
finally:
m.MAX_DATA_BYTES = old
+40
View File
@@ -0,0 +1,40 @@
import numpy as np
from pipeline.movie import Rect, TileRecord, FrameRecord
from pipeline.tilepack import decompose_tile, diff_tiles, apply_record
def test_decompose_solid_tiles():
black = np.ones((32, 32), dtype=bool)
bg, rects = decompose_tile(black)
assert bg == 1 and rects == []
white = np.zeros((32, 32), dtype=bool)
bg, rects = decompose_tile(white)
assert bg == 0 and rects == []
def test_decompose_block_is_one_rect():
tile = np.zeros((32, 32), dtype=bool)
tile[4:6, 7:10] = True
bg, rects = decompose_tile(tile)
assert bg == 0
assert rects == [Rect(7, 4, 3, 2)]
def test_diff_only_reports_changed_tiles():
a = np.zeros((64, 64), dtype=bool)
b = a.copy()
b[0:32, 0:32] = True
recs = diff_tiles(a, b, tile=32)
assert len(recs) == 1 and recs[0].tile_index == 0
def test_apply_record_reconstructs():
a = np.zeros((32, 32), dtype=bool)
b = a.copy()
b[4:9, 3:12] = True
recs = diff_tiles(a, b, tile=32)
canvas = a.copy()
for rec in recs:
apply_record(canvas, FrameRecord(1, 0, [rec]), tile=32)
assert np.array_equal(canvas, b)
+41
View File
@@ -0,0 +1,41 @@
import numpy as np
from pipeline.vectorize import rdp, rasterize, trace, vectorize_frame
def test_rdp_drops_collinear_point():
pts = [(0, 0), (1, 0), (2, 0)]
assert rdp(pts, 0.5) == [(0, 0), (2, 0)]
def test_rasterize_square():
poly = [[(1, 1), (4, 1), (4, 4), (1, 4), (1, 1)]]
out = rasterize([poly], 6, 6)
assert out[2, 2]
assert not out[0, 0]
def test_rasterize_square_with_hole():
outer = [(1, 1), (5, 1), (5, 5), (1, 5), (1, 1)]
hole = [(2, 2), (2, 4), (4, 4), (4, 2), (2, 2)]
out = rasterize([[outer, hole]], 7, 7)
assert out[3, 1]
assert not out[3, 3]
def test_trace_preserves_vertical_position():
mask = np.zeros((32, 32), dtype=bool)
mask[2:6, 4:12] = True
out = vectorize_frame(mask, {"rdp_epsilon": 1.0, "min_segment": 2})
assert out[3, 5]
assert not out[28, 5]
def test_trace_simple_shape_and_roundtrip():
mask = np.zeros((16, 16), dtype=bool)
mask[4:12, 4:12] = True
polys = trace(mask, {"rdp_epsilon": 1.0, "min_segment": 2})
assert len(polys) >= 1
roundtrip = rasterize(polys, 16, 16)
assert roundtrip[6, 6]
assert not roundtrip[0, 0]
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env python3
"""Compare a MAME VRAM dump with the logical frame the movie should show."""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
import numpy as np
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from pipeline.movie import read_movie
from pipeline.tilepack import apply_record
VRAM_STRIDE = 256
def vram_to_logical(data: bytes, width: int = 640, height: int = 512,
scale: int = 2) -> np.ndarray:
"""Unpack a 1-bpp VRAM dump and majority-downsample scale×scale blocks."""
screen_w = width * scale
screen_h = height * scale
need = VRAM_STRIDE * screen_h
if len(data) < need:
raise ValueError("VRAM dump is %d bytes, need at least %d"
% (len(data), need))
rows = np.frombuffer(data[:need], dtype=np.uint8).reshape(screen_h,
VRAM_STRIDE)
bits = np.unpackbits(rows, axis=1, bitorder="little")[:, :screen_w]
blocks = bits.astype(np.uint16).reshape(height, scale, width, scale)
# The driver's palette: bit 1 = white, bit 0 = black.
return blocks.mean(axis=(1, 3)) < 0.5
def expected_frame(movie_path: str, frame: int) -> np.ndarray:
header, records = read_movie(movie_path)
if frame < 0 or frame >= len(records):
raise ValueError("frame %d outside 0..%d" % (frame, len(records) - 1))
canvas = np.zeros((header.height, header.width), dtype=bool)
for record in records[:frame + 1]:
apply_record(canvas, record, tile=header.tile)
return canvas
def main(argv=None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--vram", required=True)
parser.add_argument("--movie", required=True)
parser.add_argument("--frame", type=int, default=0)
parser.add_argument("--scale", type=int, default=2)
args = parser.parse_args(argv)
data = Path(args.vram).read_bytes()
header, _ = read_movie(args.movie)
actual = vram_to_logical(data, header.width, header.height, args.scale)
expected = expected_frame(args.movie, args.frame)
if np.array_equal(actual, expected):
print("EXACT frame %d" % args.frame)
return 0
bad = int(np.count_nonzero(actual != expected))
ys, xs = np.where(actual != expected)
print("MISMATCH frame %d: %d logical pixels differ" % (args.frame, bad))
for y, x in list(zip(ys, xs))[:16]:
print(" (%d,%d) expected=%d actual=%d"
% (x, y, expected[y, x], actual[y, x]))
return 1
if __name__ == "__main__":
raise SystemExit(main())
+55
View File
@@ -0,0 +1,55 @@
#!/bin/sh
# Deliver the network-build payload over rdebug, then send movie.bin over UDP.
#
# tools/load.sh --host 192.168.99.2 --movie data/movie.bin
#
# The terminal must be booted and answering ping. The payload calls the OS
# socket API, receives into a malloc'd buffer, then disables interrupts and
# plays. Recovery after that point is a power cycle.
set -eu
HERE=$(cd "$(dirname "$0")" && pwd)
ROOT=$(cd "$HERE/../.." && pwd)
HOST=192.168.99.2
PORT=7001
MOVIE=$ROOT/bad-apple/data/movie.bin
WAIT=180
SETTLE=3
DELAY=0.002
while [ $# -gt 0 ]; do
case "$1" in
--host) HOST=$2; shift 2 ;;
--port) PORT=$2; shift 2 ;;
--movie) MOVIE=$2; shift 2 ;;
--wait) WAIT=$2; shift 2 ;;
--settle) SETTLE=$2; shift 2 ;;
--delay) DELAY=$2; shift 2 ;;
-h|--help) sed -n '2,12p' "$0"; exit 0 ;;
*) echo "load.sh: unknown option $1" >&2; exit 2 ;;
esac
done
if [ ! -f "$MOVIE" ]; then
echo "load.sh: no movie at $MOVIE" >&2
exit 2
fi
start=$(date +%s)
while ! ping -c1 -W1 "$HOST" >/dev/null 2>&1; do
if [ $(( $(date +%s) - start )) -ge "$WAIT" ]; then
echo "load.sh: $HOST did not answer within ${WAIT}s" >&2
exit 1
fi
done
echo "load.sh: $HOST answers"
python3 "$ROOT/harness/badapple/rdebug_load.py" \
--host "$HOST" \
--image "$ROOT/bad-apple/player/build/player-net.bin" \
--env "$ROOT/bad-apple/player/build/player-net.env"
sleep "$SETTLE"
python3 "$HERE/send_movie.py" --host "$HOST" --port "$PORT" --movie "$MOVIE" \
--delay "$DELAY"
echo "load.sh: sent; the screen is the evidence. Power-cycle to recover."
+212
View File
@@ -0,0 +1,212 @@
-- load_mame.lua -- debugger entry for the Bad Apple payload.
--
-- Loads the payload at bit 0x018C0000 and the movie at bit 0x00080000, then
-- enters the payload at the OS idle loop. This is the development path; the
-- real delivery path is tools/load.sh (rdebug + LANCE).
--
-- Env:
-- BA_MODE=one|play (one is the only mode implemented here)
-- BA_MOVIE movie file (default bad-apple/data/test_movie.bin)
-- BA_PAYLOAD raw payload (default bad-apple/player/build/player.bin)
-- BA_ENV player.env with ENTRY_PTR (default .../player.env)
-- BA_AT load frame (default 2500)
-- BA_END inspection/exit frame (default 2600)
local function hex(v, w)
return string.format("%0" .. (w or 8) .. "X", v)
end
local machine = manager.machine
local cpu = machine.devices[':maincpu']
local dbg = machine.debugger
local prog = cpu.spaces['program']
if not dbg then
print("BA ERROR: debugger not enabled; pass -debug")
machine:exit()
return
end
local ORIGIN = 0x018C0000
local MOVIE = tonumber(os.getenv("BA_MOVIE_BASE") or "0x01D00000")
local MARK = 0x01810000
local ENTRY_PC = 0x3F579970
local AT = tonumber(os.getenv("BA_AT") or "2500")
local STOP_AT = tonumber(os.getenv("BA_END") or "2600")
local START = tonumber(os.getenv("BA_START") or "0")
local COUNT = tonumber(os.getenv("BA_COUNT") or "0")
local PACE = tonumber(os.getenv("BA_PACE") or "0")
local WAIT = tonumber(os.getenv("BA_WAIT") or "0")
local MOVIE_PATH = os.getenv("BA_MOVIE") or "bad-apple/data/test_movie.bin"
local PAYLOAD_PATH = os.getenv("BA_PAYLOAD") or "bad-apple/player/build/player.bin"
local ENV_PATH = os.getenv("BA_ENV") or "bad-apple/player/build/player.env"
local function read_all(path)
local f, err = io.open(path, "rb")
if not f then
print("BA ERROR: cannot open " .. path .. ": " .. tostring(err))
machine:exit()
return nil
end
local data = f:read("*a")
f:close()
return data
end
local function entry_from_env()
local explicit = os.getenv("BA_ENTRY")
if explicit then
return tonumber(explicit)
end
local f = io.open(ENV_PATH, "r")
if not f then
return ORIGIN
end
for line in f:lines() do
local value = line:match("ENTRY_PTR=0x(%x+)")
if value then
f:close()
return tonumber(value, 16)
end
end
f:close()
return ORIGIN
end
local payload = read_all(PAYLOAD_PATH)
local movie = read_all(MOVIE_PATH)
if not payload or not movie then
return
end
local entry = entry_from_env()
do
local f = io.open(ENV_PATH, "r")
if f then
for line in f:lines() do
local value = line:match("MARK_PTR=0x(%x+)")
if value then
MARK = tonumber(value, 16)
end
end
f:close()
end
end
print(string.format("BA: payload %d bytes, movie %d bytes, entry %s mark %s",
#payload, #movie, hex(entry), hex(MARK)))
local function write_bytes(data, base)
local i
local n = #data
i = 0
while i + 4 <= n do
local b0 = data:byte(i + 1)
local b1 = data:byte(i + 2)
local b2 = data:byte(i + 3)
local b3 = data:byte(i + 4)
local word = b0 | (b1 << 8) | (b2 << 16) | (b3 << 24)
prog:write_u32(base + i * 8, word)
i = i + 4
end
while i < n do
prog:write_u8(base + i * 8, data:byte(i + 1))
i = i + 1
end
end
local function verify(data, base)
local i
for i = 0, #data - 1 do
if prog:read_u8(base + i * 8) ~= data:byte(i + 1) then
return false, i
end
end
return true, 0
end
local loaded = false
local entered = false
local done = false
local movie_sent = false
local frames = 0
local function load_and_enter()
write_bytes(payload, ORIGIN)
local ok, bad = verify(payload, ORIGIN)
if not ok then
print("BA ERROR: payload readback mismatch at byte " .. bad)
machine:exit()
return
end
prog:write_u32(MARK + 0x1C * 8, 0x42414450)
prog:write_u32(MARK + 0x20 * 8, START)
prog:write_u32(MARK + 0x24 * 8, COUNT)
prog:write_u32(MARK + 0x28 * 8, PACE)
prog:write_u32(MARK + 0x2C * 8, MOVIE)
prog:write_u32(MARK + 0x30 * 8, #movie)
prog:write_u32(MARK + 0x34 * 8, WAIT)
prog:write_u32(MARK + 0x38 * 8, 0)
prog:write_u32(MARK + 0x3C * 8, 0) -- debugger path, never network
dbg:command("bpset " .. hex(ENTRY_PC) .. ",,{ pc = " .. hex(entry) .. " ; g }")
if WAIT == 0 then
write_bytes(movie, MOVIE)
ok, bad = verify(movie, MOVIE)
if not ok then
print("BA ERROR: movie readback mismatch at byte " .. bad)
machine:exit()
return
end
end
print(string.format("BA: loaded; start=%d count=%d pace=%d wait=%d",
START, COUNT, PACE, WAIT))
end
emu.register_frame_done(function()
frames = frames + 1
if frames == AT and not loaded then
loaded = true
load_and_enter()
end
if loaded and WAIT == 1 and not movie_sent
and prog:read_u16(MARK + 0x10 * 8) == 0x10 then
movie_sent = true
print("BA: takeover seen; sending movie to low RAM")
write_bytes(movie, MOVIE)
local ok, bad = verify(movie, MOVIE)
if not ok then
print("BA ERROR: movie readback mismatch at byte " .. bad)
machine:exit()
return
end
prog:write_u32(MARK + 0x38 * 8, 1)
print("BA: movie ready")
end
if loaded and not entered then
local status = prog:read_u16(MARK + 0x10 * 8)
if status == 1 or status == 2 then
entered = true
done = true
print(string.format("BA: STATUS=%d at frame %d; FRAME=%d",
status, frames, prog:read_u16(MARK)))
end
end
if loaded and not done and frames > AT + 10 then
local err = prog:read_u32(MARK)
if err >= 0xBAD00000 then
print(string.format("BA ERROR: marker %s at frame %d",
hex(err), frames))
machine:exit()
end
end
if frames >= STOP_AT then
print(string.format("BA: end at frame %d (loaded=%s done=%s) status=%04X frame=%d magic=%08X",
frames, tostring(loaded), tostring(done),
prog:read_u16(MARK + 0x10 * 8), prog:read_u16(MARK),
prog:read_u32(MARK + 0x1C * 8)))
machine:exit()
end
end, "vt1200-badapple-loader")
dbg.visible_cpu = cpu
dbg.execution_state = 'run'
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""Make a small known movie for the one-frame MAME FILL check."""
from __future__ import annotations
import sys
from pathlib import Path
import numpy as np
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from pipeline.movie import FrameRecord, MovieHeader, write_movie
from pipeline.tilepack import diff_tiles
def main() -> int:
width, height = 640, 512
canvas = np.zeros((height, width), dtype=bool)
canvas[20:70, 10:110] = True # a rectangle crossing tiles
canvas[96:128, 160:192] = True # a solid logical tile at (5,3)
records = diff_tiles(np.zeros_like(canvas), canvas, tile=32)
frames = [FrameRecord(1, 0, records)]
header = MovieHeader(width, height, 2, 32, width // 32, height // 32,
len(frames), 60, 72)
out = ROOT / "data" / "test_movie.bin"
out.parent.mkdir(parents=True, exist_ok=True)
write_movie(str(out), frames, header)
print("%s: %d bytes, %d dirty tiles"
% (out, out.stat().st_size, len(records)))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+34
View File
@@ -0,0 +1,34 @@
#!/bin/sh
# Run the payload in MAME with the debugger loader.
#
# tools/run_mame.sh one # one frame, no pacing
# tools/run_mame.sh play # full movie, paced
# BA_MOVIE=data5/movie.bin BA_START=180 BA_COUNT=1 tools/run_mame.sh play
set -eu
HERE=$(cd "$(dirname "$0")" && pwd)
ROOT=$(cd "$HERE/../.." && pwd)
M=${M:-/home/arti/code/k-space/mame0289/vt1200}
MODE=${1:-one}
BA_MOVIE=${BA_MOVIE:-$ROOT/bad-apple/data/test_movie.bin}
BA_PACE=${BA_PACE:-0}
BA_START=${BA_START:-0}
BA_COUNT=${BA_COUNT:-0}
BA_AT=${BA_AT:-2500}
BA_END=${BA_END:-3000}
BA_MOVIE_BASE=${BA_MOVIE_BASE:-0x01D00000}
BA_WAIT=${BA_WAIT:-0}
if [ "$MODE" = "play" ]; then
BA_PACE=${BA_PACE:-1}
fi
cd "$ROOT"
VT1200_SKIP_LEVEL1=1 \
BA_MOVIE="$BA_MOVIE" BA_PACE="$BA_PACE" BA_START="$BA_START" \
BA_COUNT="$BA_COUNT" BA_AT="$BA_AT" BA_END="$BA_END" \
BA_MOVIE_BASE="$BA_MOVIE_BASE" BA_WAIT="$BA_WAIT" \
"$M" vt1200 -rompath harness/mame/roms -video none -sound none -nothrottle \
-skip_gameinfo -debug -debugger none \
-autoboot_script "$ROOT/bad-apple/tools/load_mame.lua" -autoboot_delay 0
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
# Run the current debugger-path demo in a MAME window.
#
# bad-apple/tools/run_mame_gui.sh
# M=/path/to/vt1200 BA_MOVIE=/path/to/movie.bin bad-apple/tools/run_mame_gui.sh
#
# The payload is loaded at frame BA_AT, takes over, and plays the full movie.
# Close the window or let it exit at BA_END. Recovery after takeover is a
# power cycle (in MAME: Shift+F3 or close the window).
set -eu
HERE=$(cd "$(dirname "$0")" && pwd)
ROOT=$(cd "$HERE/../.." && pwd)
M=${M:-/home/arti/code/k-space/mame0289/vt1200}
BA_MOVIE=${BA_MOVIE:-$ROOT/bad-apple/data/movie.bin}
BA_AT=${BA_AT:-2500}
BA_END=${BA_END:-8000}
if [ ! -x "$M" ]; then
echo "run_mame_gui.sh: no MAME binary at $M" >&2
exit 1
fi
if [ ! -f "$BA_MOVIE" ]; then
echo "run_mame_gui.sh: no movie at $BA_MOVIE" >&2
echo " build it with: cd bad-apple && python3 -m pipeline.build ..." >&2
exit 1
fi
if [ ! -f "$ROOT/bad-apple/player/build/player.bin" ]; then
echo "run_mame_gui.sh: no payload; run bad-apple/player/build.sh" >&2
exit 1
fi
cd "$ROOT"
VT1200_SKIP_LEVEL1=1 \
BA_MOVIE="$BA_MOVIE" BA_MOVIE_BASE=0x00080000 BA_WAIT=1 BA_PACE=1 \
BA_AT="$BA_AT" BA_END="$BA_END" \
"$M" vt1200 -rompath harness/mame/roms -sound none -skip_gameinfo -window \
-debug -debugger none \
-autoboot_script "$ROOT/bad-apple/tools/load_mame.lua" -autoboot_delay 0
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
"""Send movie.bin to the payload's UDP receiver (port 7001).
Packet: u32 offset, u16 length, u16 seq, u16 crc16, then length bytes.
A zero-length packet ends the load. This uses the VT1200 OS stack while it
is still alive; the payload disables interrupts only after the last byte.
"""
from __future__ import annotations
import argparse
import socket
import struct
import sys
import time
from pathlib import Path
def crc16(data: bytes) -> int:
return sum(data) & 0xFFFF
def checksum32(data: bytes) -> int:
return sum(data) & 0xFFFFFFFF
def pack_packet(offset: int, seq: int, body: bytes) -> bytes:
return struct.pack("<IHHH", offset, len(body), seq, crc16(body)) + body
def parse_ack(data: bytes) -> int:
if len(data) < 8 or data[:4] != b"BAK1":
raise ValueError("not a BAK1 ack")
return struct.unpack_from("<I", data, 4)[0]
def send_movie(host: str, port: int, path: str, chunk: int = 1450,
delay: float = 0.0005, passes: int = 3) -> int:
data = Path(path).read_bytes()
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
for _ in range(passes):
offset = 0
seq = 0
while offset < len(data):
part = data[offset:offset + chunk]
sock.sendto(pack_packet(offset, seq, part), (host, port))
offset += len(part)
seq += 1
if delay:
time.sleep(delay)
sock.sendto(pack_packet(len(data), seq,
checksum32(data).to_bytes(4, "little")),
(host, port))
time.sleep(0.05)
finally:
sock.close()
print("sent %d bytes in %d pass(es)" % (len(data), passes))
return 0
def main(argv=None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--host", default="192.168.99.2")
parser.add_argument("--port", type=int, default=7001)
parser.add_argument("--movie", required=True)
parser.add_argument("--chunk", type=int, default=1450)
parser.add_argument("--delay", type=float, default=0.0005)
parser.add_argument("--passes", type=int, default=3)
args = parser.parse_args(argv)
return send_movie(args.host, args.port, args.movie, args.chunk,
args.delay, args.passes)
if __name__ == "__main__":
raise SystemExit(main())
+11
View File
@@ -0,0 +1,11 @@
from tools.check_vram import vram_to_logical
def test_vram_downsamples_2x2_lsb_first():
# VRAM bit 1 is white, bit 0 is black (the driver's palette).
vram = bytearray(256 * 1024)
vram[0] = 0x0C # bits 2 and 3 white in row 0
vram[256] = 0x0C # bits 2 and 3 white in row 1
logical = vram_to_logical(bytes(vram), width=640, height=512, scale=2)
assert logical[0, 0] # bits 0..1 are black
assert not logical[0, 1] # bits 2..3 are white
+13
View File
@@ -0,0 +1,13 @@
from tools.send_movie import pack_packet, parse_ack
def test_pack_packet_header():
pkt = pack_packet(0x1234, 1, b"abc")
assert pkt[0:4] == (0x1234).to_bytes(4, "little")
assert pkt[4:6] == (3).to_bytes(2, "little")
assert pkt[6:8] == (1).to_bytes(2, "little")
assert pkt[10:] == b"abc"
def test_parse_ack():
assert parse_ack(b"BAK1" + (0x1000).to_bytes(4, "little")) == 0x1000
@@ -0,0 +1,782 @@
# Bad Apple 30 s Vector Demo Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Play the first 30 s of `hd60.mp4` full-screen on a VT1200 as quantised black-and-white polygon art, loaded once and played from the terminal's 4 MB of RAM with `FILL` rectangles.
**Architecture:** Host pipeline turns video into a 640×512 binary picture (2×2 screen blocks), traces it into simplified straight-edged polygons, rasterises them back, and packs only changed 32×32-tile rectangles into a delta stream. A TMS34010 payload decodes that stream and paints dirty tiles with `FILL L`. A small `rdebug` bootstrap kills the OS, receives the stream over LANCE, verifies it, and jumps to the player.
**Tech Stack:** Python 3 + numpy + Pillow + ffmpeg + potrace + pytest; TMS34010 C/assembly built with the 1987 TI SDK under DOSBox; MAME's `vt1200` driver and Lua debugger for verification.
**Spec:** [`docs/superpowers/specs/2026-10-31-bad-apple-vector-design.md`](../specs/2026-10-31-bad-apple-vector-design.md)
## Global Constraints
- Source is `/tmp/opencode/badapple-real/hd60.mp4`; first 30 s; 1,800 frames at 60 fps.
- Output is full screen, 1280×1024, 1 bpp, stretched from 4:3 to 5:4.
- Logical picture is 640×512; each logical pixel is a 2×2 screen block.
- Tiles are 32×32 logical (64×64 screen); 20×16 = 320 tiles.
- Movie data must be `<= 3.0 MB` and must never exceed the hard data area `0x010000..0x317FFF` (3.03 MB).
- Runtime uses `FILL L` only; `LINE` is debug-only.
- Incremental dirty-tile painting is accepted; a brief wipe is not a failure.
- The OS dies before the first byte lands in low RAM; recovery is a power cycle.
- The payload never skips a record. If the unit is slower than the encoder estimate, the movie stretches.
- Every image is LSB-first: bit 0 of a byte is the leftmost pixel.
- Do not edit `harness/` or the ROM. Stage only paths under `bad-apple/` and the two design/plan docs.
- The 1987 C compiler is K&R/C89: no `//`, no declarations after statements, no `stdint.h`, no `for (int ...)`.
## Review Focus
- **Malformed or truncated movie data** must be refused with an error block and halt, never read past the buffer or paint a partial movie. Tests: `test_movie_truncated_record`, `test_player_rejects_bad_header`.
- **Tile rectangle edge cases**: `w=32`/`h=32` must survive the 5-bit `w-1`/`h-1` encoding, and a tile with more than 127 rectangles must encode and decode without truncation. Tests: `test_rect_full_tile_bias`, `test_more_than_127_rects`.
- **Data cap**: the encoder must fail loudly above 3.03 MB, not truncate. Test: `test_write_movie_refuses_over_cap`.
- **Delta chain**: no runtime record skipping; a slow unit stretches. Test: `test_select_records_holds_time_not_records`.
- **Partial network load**: the player must not jump into a movie whose checksum failed. Test: `test_loader_rejects_bad_crc`.
---
## File Structure
```
bad-apple/
├── README.md
├── REPORT.md
├── .gitignore
├── pytest.ini
├── conftest.py
├── pipeline/
│ ├── __init__.py
│ ├── params.json
│ ├── frames.py video -> gray -> temporal median -> blur -> binary
│ ├── vectorize.py potrace + RDP + even-odd rasteriser
│ ├── movie.py header, frame records, pack/unpack, cap check
│ ├── tilepack.py frame diff -> per-tile background + rectangles
│ ├── cost.py 13.9 ms cost model and record selection
│ ├── build.py CLI: video -> movie.bin + manifest.json
│ └── preview.py side-by-side PNG and MP4 previews
├── player/
│ ├── movie.h
│ ├── movie.c pure decoder, host-testable
│ ├── main.c hardware setup, playback loop, pacing
│ ├── fill.asm fill_rect() over FILL L
│ ├── entry.asm _start + ROM thunks (copied from udpstream)
│ ├── build.sh DOSBox SDK build
│ └── tests/
│ ├── test_movie_decode.c
│ └── fixture.bin generated by tests/make_fixture.py
├── tools/
│ ├── load_mame.lua debugger load for one frame / 5 s / full 30 s
│ ├── check_vram.py VRAM dump -> logical frame -> exact compare
│ ├── send_movie.py raw-Ethernet sender
│ ├── load.sh rdebug bootstrap + network load
│ └── tests/
│ ├── test_check_vram.py
│ └── test_send_movie.py
└── tests/
├── test_frames.py
├── test_vectorize.py
├── test_movie.py
├── test_tilepack.py
├── test_cost.py
├── test_build.py
└── test_c_decoder.py
```
---
### Task 1: Scaffold and frame extraction primitives
**Files:**
- Create: `bad-apple/README.md`
- Create: `bad-apple/.gitignore`
- Create: `bad-apple/pytest.ini`
- Create: `bad-apple/conftest.py`
- Create: `bad-apple/pipeline/__init__.py`
- Create: `bad-apple/pipeline/params.json`
- Create: `bad-apple/pipeline/frames.py`
- Test: `bad-apple/tests/test_frames.py`
**Interfaces:**
- Produces:
- `temporal_median3(frames: Sequence[np.ndarray]) -> np.ndarray`
- `box_blur3(gray: np.ndarray) -> np.ndarray`
- `binarize(gray: np.ndarray, threshold: int = 127) -> np.ndarray` (True = black)
- `pack_lsb(mask: np.ndarray) -> bytes`
- `extract_gray(video: str, start: float, duration: float, fps: int, width: int, height: int) -> Iterator[np.ndarray]`
- `extract_binary(video: str, start: float, duration: float, fps: int, width: int, height: int, params: dict) -> Iterator[np.ndarray]`
- [ ] **Step 1: Write the failing tests**
```python
# bad-apple/tests/test_frames.py
import numpy as np
from pipeline.frames import temporal_median3, box_blur3, binarize, pack_lsb
def test_temporal_median_removes_single_frame_flash():
a = np.zeros((4, 4), dtype=np.uint8)
b = np.zeros((4, 4), dtype=np.uint8); b[1, 1] = 255
c = np.zeros((4, 4), dtype=np.uint8)
out = temporal_median3([a, b, c])
assert out[1, 1] == 0
def test_pack_lsb_is_leftmost_pixel_bit_zero():
mask = np.zeros((1, 8), dtype=bool)
mask[0, 0] = True
assert pack_lsb(mask) == b"\x01"
mask[0, 7] = True
assert pack_lsb(mask) == b"\x81"
def test_binarize_black_is_true():
gray = np.array([[0, 255]], dtype=np.uint8)
assert binarize(gray).tolist() == [[True, False]]
def test_box_blur_softens_a_single_pixel():
gray = np.zeros((3, 3), dtype=np.uint8); gray[1, 1] = 9
out = box_blur3(gray)
assert out[1, 1] == 1
```
- [ ] **Step 2: Run the tests to verify they fail**
Run: `cd bad-apple && python3 -m pytest tests/test_frames.py -q`
Expected: FAIL with `ModuleNotFoundError: No module named 'pipeline'` or missing names.
- [ ] **Step 3: Implement the primitives**
`temporal_median3` returns `np.median(np.stack(frames), axis=0).astype(np.uint8)`. `box_blur3` is a 3×3 mean with edge clamping (do not `np.roll`, which wraps). `binarize` is `gray < threshold` (black is True). `pack_lsb` uses `np.packbits(mask, axis=1, bitorder="little")`. `extract_gray` runs:
```
ffmpeg -v error -ss <start> -t <duration> -i <video>
-vf fps=<fps>,scale=<w>:<h>:flags=lanczos,format=gray
-f rawvideo -
```
and yields `(h, w)` uint8 arrays. `extract_binary` buffers three gray frames, applies `temporal_median3`, `box_blur3`, `binarize`, then yields; it duplicates the first and last frame at the boundaries so exactly `duration*fps` frames are produced. Add `pipeline/params.json` with the values from the spec:
```json
{
"start": 0.0, "duration": 30.0, "fps": 60,
"width": 640, "height": 512, "threshold": 127,
"temporal_median": 3, "blur": 3,
"tile": 32, "scale": 2,
"rdp_epsilon": 1.5, "min_segment": 2, "min_component": 8
}
```
`bad-apple/conftest.py` inserts the `bad-apple/` directory on `sys.path`. `.gitignore` ignores `data/*` but keeps `!data/manifest.json`, plus `preview/`, `player/build/`, `__pycache__/`, `.pytest_cache/`. `pytest.ini` sets `testpaths = tests`; the two `tools/tests` files are run explicitly.
- [ ] **Step 4: Run the tests to verify they pass**
Run: `cd bad-apple && python3 -m pytest tests/test_frames.py -q`
Expected: `4 passed`.
- [ ] **Step 5: Commit**
```bash
git add bad-apple/README.md bad-apple/.gitignore bad-apple/pytest.ini \
bad-apple/conftest.py bad-apple/pipeline/__init__.py \
bad-apple/pipeline/params.json bad-apple/pipeline/frames.py \
bad-apple/tests/test_frames.py
git commit -m "feat(badapple): frame extraction primitives"
```
---
### Task 2: Polygon tracing, simplification and rasterisation
**Files:**
- Create: `bad-apple/pipeline/vectorize.py`
- Test: `bad-apple/tests/test_vectorize.py`
**Interfaces:**
- Produces:
- `rdp(points: list[tuple[float, float]], epsilon: float) -> list[tuple[float, float]]`
- `parse_potrace_geojson(text: str) -> list[list[list[tuple[float, float]]]]`
- `trace(mask: np.ndarray, params: dict) -> list[list[list[tuple[float, float]]]]`
- `rasterize(polygons, width: int, height: int) -> np.ndarray`
- `vectorize_frame(mask: np.ndarray, params: dict) -> np.ndarray`
- [ ] **Step 1: Write the failing tests**
```python
# bad-apple/tests/test_vectorize.py
import numpy as np
from pipeline.vectorize import rdp, rasterize, trace, vectorize_frame
def test_rdp_drops_collinear_point():
pts = [(0, 0), (1, 0), (2, 0)]
assert rdp(pts, 0.5) == [(0, 0), (2, 0)]
def test_rasterize_square():
poly = [[(1, 1), (4, 1), (4, 4), (1, 4), (1, 1)]]
out = rasterize([poly], 6, 6)
assert out[2, 2]
assert not out[0, 0]
def test_rasterize_square_with_hole():
outer = [(1, 1), (5, 1), (5, 5), (1, 5), (1, 1)]
hole = [(2, 2), (2, 4), (4, 4), (4, 2), (2, 2)]
out = rasterize([[outer, hole]], 7, 7)
assert out[3, 1] # left wall
assert not out[3, 3] # hole
def test_trace_simple_shape_and_roundtrip():
mask = np.zeros((16, 16), dtype=bool)
mask[4:12, 4:12] = True
polys = trace(mask, {"rdp_epsilon": 1.0, "min_segment": 2})
assert len(polys) >= 1
roundtrip = rasterize(polys, 16, 16)
assert roundtrip[6, 6]
assert not roundtrip[0, 0]
```
- [ ] **Step 2: Run the tests to verify they fail**
Run: `cd bad-apple && python3 -m pytest tests/test_vectorize.py -q`
Expected: FAIL with missing module or names.
- [ ] **Step 3: Implement the vectoriser**
`trace` writes the black mask to a temporary PBM (`P4`, 1 = black), runs:
```
potrace -b geojson -a 0 -t 4 -u 1 -o <json> <pbm>
```
and parses `FeatureCollection` → `Polygon`/`MultiPolygon` rings. Apply `rdp` to each closed ring (keep the first/last point). Drop rings with fewer than 3 distinct points or a longest segment below `min_segment`. `rasterize` collects every ring edge, intersects it with each scanline at `y + 0.5`, sorts the x values, and fills pixel centres between pairs (even-odd). `vectorize_frame` is `rasterize(trace(mask, params), width, height)`.
- [ ] **Step 4: Run the tests to verify they pass**
Run: `cd bad-apple && python3 -m pytest tests/test_vectorize.py -q`
Expected: `4 passed`.
- [ ] **Step 5: Commit**
```bash
git add bad-apple/pipeline/vectorize.py bad-apple/tests/test_vectorize.py
git commit -m "feat(badapple): polygon tracing and rasterisation"
```
---
### Task 3: Tile diff and movie format
**Files:**
- Create: `bad-apple/pipeline/movie.py`
- Create: `bad-apple/pipeline/tilepack.py`
- Test: `bad-apple/tests/test_tilepack.py`
- Test: `bad-apple/tests/test_movie.py`
**Interfaces:**
- Produces:
- `Rect(x0, y0, w, h)` frozen dataclass, `TileRecord(tile_index, bg, rects)`, `FrameRecord(ticks, flags, tiles)`, `MovieHeader(...)`
- `decompose_tile(tile: np.ndarray) -> tuple[int, list[Rect]]`
- `diff_tiles(prev: np.ndarray, cur: np.ndarray, tile: int = 32) -> list[TileRecord]`
- `apply_record(canvas: np.ndarray, record: FrameRecord, tile: int = 32) -> None`
- `encode_frame(record: FrameRecord) -> bytes`
- `decode_frame(data: bytes, offset: int) -> tuple[FrameRecord, int]`
- `write_movie(path: str, frames: Sequence[FrameRecord], header: MovieHeader) -> int`
- `read_movie(path: str) -> tuple[MovieHeader, list[FrameRecord]]`
- `MovieFormatError` exception
- `MAX_DATA_BYTES = 0x308000`
- [ ] **Step 1: Write the failing tests**
```python
# bad-apple/tests/test_tilepack.py
import numpy as np
from pipeline.tilepack import decompose_tile, diff_tiles, apply_record
from pipeline.movie import Rect, TileRecord, FrameRecord
def test_decompose_solid_tiles():
black = np.ones((32, 32), dtype=bool)
bg, rects = decompose_tile(black)
assert bg == 1 and rects == []
white = np.zeros((32, 32), dtype=bool)
bg, rects = decompose_tile(white)
assert bg == 0 and rects == []
def test_decompose_block_is_one_rect():
tile = np.zeros((32, 32), dtype=bool)
tile[4:6, 7:10] = True
bg, rects = decompose_tile(tile)
assert bg == 0
assert rects == [Rect(7, 4, 3, 2)]
def test_diff_only_reports_changed_tiles():
a = np.zeros((64, 64), dtype=bool)
b = a.copy(); b[0:32, 0:32] = True
recs = diff_tiles(a, b, tile=32)
assert len(recs) == 1 and recs[0].tile_index == 0
def test_apply_record_reconstructs():
a = np.zeros((32, 32), dtype=bool)
b = a.copy(); b[4:9, 3:12] = True
recs = diff_tiles(a, b, tile=32)
canvas = a.copy()
for rec in recs:
apply_record(canvas, FrameRecord(1, 0, [rec]), tile=32)
assert np.array_equal(canvas, b)
```
```python
# bad-apple/tests/test_movie.py
import numpy as np
from pipeline.movie import (Rect, TileRecord, FrameRecord, MovieHeader,
encode_frame, decode_frame, write_movie, read_movie,
MovieFormatError)
def test_rect_full_tile_bias_roundtrip():
rec = FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 32, 32)])])
back, off = decode_frame(encode_frame(rec), 0)
assert back.tiles[0].rects[0] == Rect(0, 0, 32, 32)
assert off == len(encode_frame(rec))
def test_more_than_127_rects_roundtrip():
rects = [Rect(i % 32, i // 32, 1, 1) for i in range(200)]
rec = FrameRecord(1, 0, [TileRecord(3, 0, rects)])
back, _ = decode_frame(encode_frame(rec), 0)
assert len(back.tiles[0].rects) == 200
def test_truncated_record_raises():
data = encode_frame(FrameRecord(1, 0, [TileRecord(0, 0, [Rect(0, 0, 4, 4)])]))
try:
decode_frame(data[:-1], 0)
except MovieFormatError:
return
assert False, "truncated record was accepted"
def test_write_movie_refuses_over_cap(tmp_path):
frames = [FrameRecord(1, 0, []) for _ in range(10)]
header = MovieHeader(640, 512, 2, 32, 20, 16, 10, 60, 72)
# 10 empty records are tiny; the cap is exercised by a patched constant.
import pipeline.movie as m
old = m.MAX_DATA_BYTES
m.MAX_DATA_BYTES = 8
try:
try:
write_movie(str(tmp_path / "x.bin"), frames, header)
except MovieFormatError:
return
assert False, "cap not enforced"
finally:
m.MAX_DATA_BYTES = old
```
- [ ] **Step 2: Run the tests to verify they fail**
Run: `cd bad-apple && python3 -m pytest tests/test_tilepack.py tests/test_movie.py -q`
Expected: FAIL with missing modules.
- [ ] **Step 3: Implement tile decomposition and the format**
`decompose_tile` chooses `bg` by majority, makes `fg = tile != bool(bg)`, extracts horizontal runs per row, and merges identical `(x0, x1)` runs across consecutive rows into `Rect(x0, y0, x1-x0+1, height)`. `diff_tiles` compares `prev` and `cur` tile by tile and calls `decompose_tile(cur_tile)` for changed tiles. `apply_record` fills the whole tile with `bg`, then fills each rect with `1-bg`.
Header layout (36 bytes, little-endian): `magic "BAPL"`, `version u16=1`, `width u16`, `height u16`, `scale u16`, `tile u16`, `tiles_x u16`, `tiles_y u16`, `frame_count u32`, `index_offset u32`, `data_offset u32`, `source_fps u16`, `display_hz u16`, `reserved u16=0`. Index entry is `<IBB` (offset, ticks, flags). Record is `<H` dirty count, then per tile `<HH` (tile_index, `(bg << 15) | rect_count`), then 3-byte rects packed as `x0 | (y0 << 5) | ((w-1) << 10) | ((h-1) << 15)`.
`decode_frame` validates counts against the remaining bytes before allocating/reading. `write_movie` builds the header + index + records in memory, raises `MovieFormatError` if the total exceeds `MAX_DATA_BYTES`, and writes atomically via a temporary file.
- [ ] **Step 4: Run the tests to verify they pass**
Run: `cd bad-apple && python3 -m pytest tests/test_tilepack.py tests/test_movie.py -q`
Expected: all tests pass.
- [ ] **Step 5: Commit**
```bash
git add bad-apple/pipeline/movie.py bad-apple/pipeline/tilepack.py \
bad-apple/tests/test_tilepack.py bad-apple/tests/test_movie.py
git commit -m "feat(badapple): tile delta format and movie packer"
```
---
### Task 4: Cost model, record selection and build CLI
**Files:**
- Create: `bad-apple/pipeline/cost.py`
- Create: `bad-apple/pipeline/build.py`
- Create: `bad-apple/pipeline/preview.py`
- Test: `bad-apple/tests/test_cost.py`
- Test: `bad-apple/tests/test_build.py`
**Interfaces:**
- Produces:
- `CostConstants(word_us=0.40, call_us=3.0, rect_us=1.0)`
- `estimate_record_us(record: FrameRecord, constants: CostConstants, scale=2, tile=32) -> float`
- `select_records(frames: Sequence[np.ndarray], constants: CostConstants, budget_us: float = 13900.0, max_skip: int = 2) -> list[FrameRecord]`
- `build_movie(video: str, out_dir: str, params: dict) -> dict`
- [ ] **Step 1: Write the failing tests**
```python
# bad-apple/tests/test_cost.py
import numpy as np
from pipeline.cost import CostConstants, estimate_record_us, select_records
from pipeline.movie import FrameRecord, TileRecord, Rect
C = CostConstants()
def test_one_solid_tile_cost():
rec = FrameRecord(1, 0, [TileRecord(0, 1, [])])
# 64x64 screen tile = 256 words; 256*0.40 + 3us call
assert abs(estimate_record_us(rec, C) - (256 * 0.40 + 3.0)) < 0.01
def test_full_screen_is_over_budget():
recs = [TileRecord(i, 1, []) for i in range(320)]
assert estimate_record_us(FrameRecord(1, 0, recs), C) > 13900.0
def test_select_records_holds_time_not_records():
quiet = np.zeros((32, 32), dtype=bool)
busy = np.ones((32, 32), dtype=bool)
out = select_records([quiet, busy, quiet, quiet], C, budget_us=2000.0)
assert all(r.ticks >= 1 for r in out)
assert sum(r.ticks for r in out) == 4
```
```python
# bad-apple/tests/test_build.py
import json
import numpy as np
import pipeline.build as build
from pipeline.movie import read_movie
def test_build_tiny_movie(tmp_path, monkeypatch):
frames = [np.zeros((64, 64), dtype=bool) for _ in range(3)]
frames[1][8:24, 8:24] = True
monkeypatch.setattr(build, "extract_binary", lambda *a, **k: iter(frames))
monkeypatch.setattr(build, "vectorize_frame", lambda m, p: m)
out = tmp_path / "data"
manifest = build.build_movie("unused.mp4", str(out), {"duration": 3, "fps": 60})
header, recs = read_movie(str(out / "movie.bin"))
assert header.frame_count == 3
assert len(recs) == 3
assert (out / "manifest.json").exists()
assert json.loads((out / "manifest.json").read_text())["frame_count"] == 3
```
- [ ] **Step 2: Run the tests to verify they fail**
Run: `cd bad-apple && python3 -m pytest tests/test_cost.py tests/test_build.py -q`
Expected: FAIL with missing modules.
- [ ] **Step 3: Implement the cost model and build CLI**
`estimate_record_us` sums, per dirty tile, one background fill of 64×64 screen pixels (256 words) plus each rect's `ceil(w*scale/16) * h*scale` words, all at `word_us`, plus `call_us` per fill and `rect_us` per rect. `select_records` walks the source frames, computes the record from the last **kept** frame; if it fits the budget, emit it with `ticks=1`; otherwise remember it and skip up to `max_skip` frames, accumulating ticks, then emit the best candidate even if over budget (a cut). It never emits a record from a skipped frame. `build_movie` streams `extract_binary`, applies `vectorize_frame`, computes records, calls `select_records`, writes `movie.bin`, `manifest.json` (data size, dirty-tile percentiles, cost percentiles, dropped count), and previews for frames 0, 60, 300, 900, 1799. `preview.py` writes PNGs and an optional MP4 via ffmpeg.
- [ ] **Step 4: Run the tests to verify they pass**
Run: `cd bad-apple && python3 -m pytest tests/test_cost.py tests/test_build.py -q`
Expected: all tests pass.
- [ ] **Step 5: Commit**
```bash
git add bad-apple/pipeline/cost.py bad-apple/pipeline/build.py \
bad-apple/pipeline/preview.py bad-apple/tests/test_cost.py \
bad-apple/tests/test_build.py
git commit -m "feat(badapple): cost model, frame selection and build CLI"
```
---
### Task 5: Host-testable player decoder
**Files:**
- Create: `bad-apple/player/movie.h`
- Create: `bad-apple/player/movie.c`
- Create: `bad-apple/player/tests/test_movie_decode.c`
- Create: `bad-apple/tests/make_fixture.py`
- Test: `bad-apple/tests/test_c_decoder.py`
**Interfaces:**
- Produces C:
- `struct movie_header { unsigned int width, height, scale, tile, tiles_x, tiles_y, frame_count, index_offset, data_offset, source_fps, display_hz; };`
- `int movie_open(const unsigned char *data, unsigned long len, struct movie_header *h);`
- `int movie_decode_frame(const unsigned char *data, unsigned long len, unsigned int frame, const struct movie_header *h, void (*fill)(int x, int y, int w, int h, int color, void *ctx), void *ctx);`
- `int movie_checksum(const unsigned char *data, unsigned long len);`
- [ ] **Step 1: Generate the fixture and write the failing tests**
`bad-apple/tests/make_fixture.py` writes `player/tests/fixture.bin` using `pipeline.movie` with two frames: frame 0 a white tile with one black `Rect(0,0,32,32)`; frame 1 a black tile with `Rect(1,1,2,3)`. `tests/test_c_decoder.py` compiles `movie.c` + the C test with `gcc -std=c99 -Wall -Werror` into a temp binary, runs it, and compares its printed lines to the Python decoder.
```python
# bad-apple/tests/test_c_decoder.py
import subprocess, sys, tempfile
from pathlib import Path
from pipeline.movie import read_movie
ROOT = Path(__file__).resolve().parents[1]
def test_c_decoder_matches_python(tmp_path):
subprocess.run([sys.executable, str(ROOT / "tests" / "make_fixture.py")], check=True)
exe = tmp_path / "decode"
subprocess.run(["gcc", "-std=c99", "-Wall", "-Werror",
str(ROOT / "player" / "movie.c"),
str(ROOT / "player" / "tests" / "test_movie_decode.c"),
"-o", str(exe)], check=True)
out = subprocess.run([str(exe), str(ROOT / "player" / "tests" / "fixture.bin")],
check=True, capture_output=True, text=True).stdout
assert "frames=2" in out
assert "0 0 0 0 0 32 32" in out
assert "1 1 1 1 1 2 3" in out
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `cd bad-apple && python3 -m pytest tests/test_c_decoder.py -q`
Expected: FAIL because `player/movie.c` does not exist.
- [ ] **Step 3: Implement the decoder and the C test**
`movie.c` uses only C89-compatible declarations (K&R style where needed) and no libc beyond what the 1987 compiler provides. It reads little-endian words with byte shifts, validates magic/version, bounds-checks every count against `len`, and calls `fill` with screen coordinates already scaled: `sx = (tile_x*tile + x0)*scale`, `sy = (tile_y*tile + y0)*scale`, `sw = (w+1)*scale`, `sh = (h+1)*scale`. The C test prints `frames=N`, then one line per rect `frame tile bg x y w h`.
- [ ] **Step 4: Run the test to verify it passes**
Run: `cd bad-apple && python3 -m pytest tests/test_c_decoder.py -q`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add bad-apple/player/movie.h bad-apple/player/movie.c \
bad-apple/player/tests/test_movie_decode.c bad-apple/tests/make_fixture.py \
bad-apple/tests/test_c_decoder.py bad-apple/player/tests/fixture.bin
git commit -m "feat(badapple): host-tested movie decoder"
```
---
### Task 6: FILL primitive, payload build and one exact frame in MAME
**Files:**
- Create: `bad-apple/player/fill.asm`
- Create: `bad-apple/player/entry.asm`
- Create: `bad-apple/player/main.c`
- Create: `bad-apple/player/build.sh`
- Create: `bad-apple/tools/load_mame.lua`
- Create: `bad-apple/tools/check_vram.py`
- Test: `bad-apple/tools/tests/test_check_vram.py`
**Interfaces:**
- Produces:
- `void fill_rect(int x, int y, int w, int h, int color);` (C-callable, published-word form)
- `main.c` mode `ONE`: open the movie, paint frame 0, write `FRAME=0` to marker bit `0x01810000`, spin.
- `load_mame.lua` env: `BA_MODE=one`, `BA_MOVIE`, `BA_PAYLOAD`, `BA_AT`, `BA_ENTRY`.
- `check_vram.py --vram <dump> --movie <movie.bin> --frame <n> --scale 2` exits 0 on exact match.
- [ ] **Step 1: Write the failing checker test**
```python
# bad-apple/tools/tests/test_check_vram.py
import numpy as np
from tools.check_vram import vram_to_logical
def test_vram_downsamples_2x2_lsb_first():
vram = bytearray(256 * 1024)
# logical pixel (0,0) black means screen pixels x=0..1,y=0..1 set.
vram[0] = 0x03 # bits 0 and 1 of row 0
vram[256] = 0x03 # bits 0 and 1 of row 1
logical = vram_to_logical(bytes(vram), width=640, height=512, scale=2)
assert logical[0, 0]
assert not logical[0, 1]
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `cd bad-apple && python3 -m pytest tools/tests/test_check_vram.py -q`
Expected: FAIL because `tools/check_vram.py` does not exist.
- [ ] **Step 3: Implement the FILL primitive, payload, loader and checker**
`fill.asm` follows `harness/udpstream/vram.asm`: a no-argument entry reads published globals `fill_x`, `fill_y`, `fill_w`, `fill_h`, `fill_color`; sets `B2 = 0x04000000 + y*2048 + x`, `B3 = 0x800`, `B7 = (h << 16) | w`, `B9 = color`, programs `PSIZE=1`, `PMASK=0`, `CONTROL` replace once, then `FILL L`. The C wrapper `fill_rect` publishes the globals and calls the asm entry.
`entry.asm` is copied from `harness/udpstream/entry.asm`, with `_start` setting `SP` and calling `_main`, then spinning (no return). `build.sh` copies `harness/udpstream/build-stream.sh`, changes `SRC` to `player/main.c`, copies `movie.c` as `MOVIE.C`, links `MOVIE.obj`, `FILL.obj`, `MAIN.obj`, `ENTRY.obj`, and links `.text` at bit `0x018C0000`, `.bss` at bit `0x01C00000`. It writes `build/player.bin` and `build/player.env` with `ENTRY_PTR`.
`load_mame.lua` loads `player.bin` at bit `0x018C0000` and `movie.bin` at bit `0x00080000`, verifies a readback, sets a breakpoint at the OS idle PC (default `0x3F579970`, as `vidstream.lua`), and jumps. `check_vram.py` reads the first `256*1024` bytes of the dump, extracts bit `x` of row `y` (LSB-first), downsamples 2×2 by majority, and compares to `apply_record` over the movie frames.
- [ ] **Step 4: Run the checker test and the MAME one-frame check**
Run: `cd bad-apple && python3 -m pytest tools/tests/test_check_vram.py -q`
Expected: PASS.
Run a one-frame movie in MAME with `VT1200_SKIP_LEVEL1=1`, `BA_MODE=one`, `VT1200_DUMP_VRAM_FRAMES=<after paint>`, then:
`python3 tools/check_vram.py --vram <dump> --movie data/movie.bin --frame 0`
Expected: `EXACT` and exit 0.
- [ ] **Step 5: Commit**
```bash
git add bad-apple/player/fill.asm bad-apple/player/entry.asm \
bad-apple/player/main.c bad-apple/player/build.sh \
bad-apple/tools/load_mame.lua bad-apple/tools/check_vram.py \
bad-apple/tools/tests/test_check_vram.py
git commit -m "feat(badapple): FILL primitive and one-frame MAME check"
```
---
### Task 7: Playback, pacing and the 5 s slice
**Files:**
- Modify: `bad-apple/player/main.c`
- Modify: `bad-apple/tools/load_mame.lua`
- Create: `bad-apple/tools/run_mame.sh`
- Test: manual/MAME (no host unit test)
**Interfaces:**
- `main.c` mode `PLAY`: open the movie, clear the screen white, loop over records, call `movie_decode_frame`, pace on `INTPEND` DI, and write `FRAME`, `TICKS`, `LAST_BYTES`, `PAINT_LOOPS`, `ERROR` to the marker page.
- `load_mame.lua` env `BA_MODE=play`, `BA_FRAME` (start frame), `BA_DUMP_FRAMES`.
- [ ] **Step 1: Add the playback loop and marker page**
Implement `play()` in `main.c`: `DINT`, initial full-screen white `FILL`, then for each frame call the decoder. Before each record, read `INTPEND` (bit `TMS34010_DI = 0x0400`) and wait for the next vblank; program `DPYINT` if the inherited value does not produce the pending bit. Maintain a 16.16 `now`/`next` accumulator with `now += 60/72` per vsync; if `now >= next`, paint and `next += ticks`; if `now - next > 1.0`, set `next = now` (slow down, never skip). Publish the marker page at bit `0x01810000`.
- [ ] **Step 2: Run the 5 s slice in MAME and compare sampled frames**
Use the encoder to pack frames 0..300 into `data/movie5.bin`. Run `tools/run_mame.sh` with `BA_MODE=play`, dump VRAM at MAME frames corresponding to movie frames 0, 60, 180, 300, and compare each with `check_vram.py`.
Expected: every sample prints `EXACT`.
- [ ] **Step 3: Record the paint-cost histogram and the first visual clip**
Capture a windowed run with `x11grab` (or `tools/run_mame.sh`), inspect for stale tiles, double-applied deltas, and a half-painted cut. Record the measured `PAINT_LOOPS` / `LAST_BYTES` distribution in `bad-apple/REPORT.md` (created in Task 9; a temporary note is fine).
- [ ] **Step 4: Commit**
```bash
git add bad-apple/player/main.c bad-apple/tools/load_mame.lua \
bad-apple/tools/run_mame.sh
git commit -m "feat(badapple): playback loop and vblank pacing"
```
---
### Task 8: Full 30 s encode and MAME playback
**Files:**
- Modify: `bad-apple/pipeline/params.json`
- Modify: `bad-apple/pipeline/build.py` (only if tuning reveals a bug)
- Test: MAME sampled frames
- [ ] **Step 1: Run the real pipeline**
Run: `cd bad-apple && python3 -m pipeline.build --video /tmp/opencode/badapple-real/hd60.mp4 --out data --duration 30`
Expected: `data/movie.bin`, `data/manifest.json`, preview frames; manifest reports `data_bytes <= 3.0e6` and p95 cost `<= 13900`.
- [ ] **Step 2: If the gate fails, tune in this order**
Increase `rdp_epsilon` (1.5 → 2.0 → 3.0), increase `min_component` (8 → 16), then raise `tile` from 32 to 64 (the format supports any power-of-two tile; the player uses the header). Do **not** shorten the clip. Re-run and re-check the gate after each change.
- [ ] **Step 3: Play the full 30 s in MAME**
Run `tools/run_mame.sh` with `BA_MODE=play`. Dump VRAM at movie frames 0, 60, 300, 600, 900, 1200, 1500, 1799 and every scene cut. Compare each with `check_vram.py`.
Expected: every sample `EXACT`; the last frame is frame 1799.
- [ ] **Step 4: Commit the tuned parameters and manifest summary**
```bash
git add bad-apple/pipeline/params.json bad-apple/data/manifest.json
git commit -m "feat(badapple): tune the 30s stream to the memory budget"
```
(`data/movie.bin` and previews stay gitignored; the manifest is the record.)
---
### Task 9: Network loader and real-unit delivery
**Files:**
- Create: `bad-apple/player/loader.c` or extend `main.c` with `MODE=LOAD`
- Create: `bad-apple/tools/send_movie.py`
- Create: `bad-apple/tools/load.sh`
- Test: `bad-apple/tools/tests/test_send_movie.py`
**Interfaces:**
- Wire packet: `u32 offset`, `u16 length`, `u16 seq`, `u16 crc`, then `length` bytes.
- `send_movie(iface, host_mac, path, chunk=1450) -> int` returns 0 after the final ACK.
- Loader writes to `0x010000`, ACKs the highest contiguous offset every 32 datagrams, verifies the whole-file CRC, then jumps.
- [ ] **Step 1: Write the failing sender test**
```python
# bad-apple/tools/tests/test_send_movie.py
from tools.send_movie import pack_packet, parse_ack
def test_pack_packet_header():
pkt = pack_packet(0x1234, 1, b"abc")
assert pkt[0:4] == (0x1234).to_bytes(4, "little")
assert pkt[4:6] == (3).to_bytes(2, "little")
assert pkt[6:8] == (1).to_bytes(2, "little")
assert pkt[10:] == b"abc"
def test_parse_ack():
assert parse_ack(b"BAK1" + (0x1000).to_bytes(4, "little")) == 0x1000
```
- [ ] **Step 2: Run the test to verify it fails**
Run: `cd bad-apple && python3 -m pytest tools/tests/test_send_movie.py -q`
Expected: FAIL because the module does not exist.
- [ ] **Step 3: Implement the sender and loader**
Copy the raw-Ethernet framing from `harness/udpstream` / `harness/badapple` (ethertype and tap/interface selection). Send 1,450-byte chunks, wait for ACKs, retransmit from the first gap, and require a final `BAK!` ACK. The loader uses the verified LANCE init/ring sequence from `harness/udpstream/vidstream.c`, receives into `0x010000`, and checks the whole-file CRC before jumping. If the real chip truncates a high RX buffer address, fall back to a low 32 KB staging ring and copy each datagram up.
- [ ] **Step 4: Run the sender unit test and the MAME tap test**
Run: `cd bad-apple && python3 -m pytest tools/tests/test_send_movie.py -q`
Expected: PASS.
Run `harness/mame/run.sh net`, then `tools/load.sh --host 192.168.99.2 --iface vt1200 --movie data/movie.bin`.
Expected: final ACK, then the 30 s movie plays.
- [ ] **Step 5: Real unit**
Run the same `load.sh` against the unit's address/interface, after confirming `rdebug` is reachable. Record a video. Recovery is a power cycle.
Expected: the same picture, no host attached after the load.
- [ ] **Step 6: Commit**
```bash
git add bad-apple/player/loader.c bad-apple/tools/send_movie.py \
bad-apple/tools/load.sh bad-apple/tools/tests/test_send_movie.py
git commit -m "feat(badapple): network loader and delivery"
```
---
### Task 10: Documentation, report and regression
**Files:**
- Create/modify: `bad-apple/README.md`
- Create: `bad-apple/REPORT.md`
- Test: `cd /home/arti/code/k-space/vt1200 && M=/home/arti/code/k-space/mame0289/vt1200 && ... progress.lua`
- [ ] **Step 1: Write the README**
Cover: one-command build pipeline, preview command, MAME run, real-unit load, recovery warning, data budget numbers, and the fact that the OS dies.
- [ ] **Step 2: Write the report**
Record: measured data size, dirty-tile and rect percentiles, paint-cost histogram, dropped frames, MAME exact-match evidence, unit observations, and the remaining gaps (potrace temporal jitter, LANCE high-address uncertainty, incremental wipes).
- [ ] **Step 3: Run the regression baseline**
Run the standard no-env `progress.lua` at 200 frames and confirm the line is still `PC=3FDCE400 samples=199 (2 distinct PCs)`.
Expected: unchanged.
- [ ] **Step 4: Commit**
```bash
git add bad-apple/README.md bad-apple/REPORT.md
git commit -m "docs(badapple): usage, measurements and gaps"
```
---
## Self-Review
- Spec coverage: geometry (§5), vectorisation (§7.1), tile format (§7.2), frame adaptation (§7.3), cuts (§7.4), memory layout (§8.1), takeover (§8.2), loader (§8.3), rendering (§8.4), presentation (§8.5), pacing (§8.6), verification (§9), milestones (§10), risks (§11) each map to a task above.
- Type consistency: `Rect`, `TileRecord`, `FrameRecord`, `MovieHeader`, `MAX_DATA_BYTES` are used with the same names in Tasks 3–10.
- Review Focus: truncated data (Task 3/5), rect bias and >127 count (Task 3), cap (Task 3), delta chain (Task 4), partial load (Task 9) are pinned by tests or MAME checks.
- Proportion: the plan names files, signatures and tests; it does not transcribe the decoder or the encoder bodies.
@@ -0,0 +1,601 @@
# Bad Apple, 30 s, vector polygons — design
Date: 2026-10-31
Status: **rev 1** — first draft, awaiting approval.
Author: this session.
This is a **separate prototype** from
[`2026-10-31-badapple-60p-design.md`](2026-10-31-badapple-60p-design.md)
(the raw-tile, full-length, streamed design in `harness/badapple/`). It does
not supersede it and does not change the existing harness. It answers a
different brief: a **standalone, first-30-seconds, polygon-styled** demo in a
new top-level `bad-apple/` folder.
Decisions marked **human:** were given in this session. Decisions marked
**call:** are the builder's proposals, not yet agreed.
---
## 1. Intent
Play the first **30 seconds** of
`/tmp/opencode/badapple-real/hd60.mp4` on a VT1200, **full screen** at
**1280×1024**, as **flat black-and-white polygon art** in the spirit of
*Another World* — not a dithered bitmap and not a 32×32 tile dictionary.
The picture is quantised to straight-edged polygons, then to a coarse logical
grid, then to rectangles that the TMS34010's `FILL` engine can paint fast.
After a small bootstrap is delivered over `rdebug`, the payload takes over the
machine and receives the movie data over Ethernet; playback is then entirely
local from the VT1200's 4 MB of main RAM. Recovery is a hardware restart.
## 2. What the human partner decided
* **Source:** `/tmp/opencode/badapple-real/hd60.mp4`, first **30 s**.
* **Frame:** fill the whole 1280×1024 panel; no letterboxed 4:3.
* **Look:** black-and-white **polygons**, similar to *Another World*; drop
dithered sections and threshold/simplify them into flat shapes.
* **Motion:** **60 fps is the target**; complex scenes may drop frames.
* **Ownership:** after the initial load, the program may take the whole 4 MB
of RAM; the OS may die; recovery is a power cycle.
* **Delivery:** `rdebug` is the bootstrap path; the network loader in
`harness/badapple/` is fair game to make the big load fast.
* **Location:** a new **top-level `bad-apple/`** folder.
## 3. Success criteria
1. `bad-apple/` reproduces the whole pipeline from the source video with one
command per stage, and documents the data budget it measured.
2. In MAME, the first 30 s plays full-screen, recognisably Bad Apple, with
clean straight-edged shapes and no corrupt or stale frames. Typical frames
run at 60 fps; heavy frames may drop to 30 fps or hold for one refresh.
The baseline repaints dirty tiles incrementally; a brief wipe on a large
change is accepted for this prototype (§8.5).
3. The movie data plus code fits in the VT1200's 4 MB of main RAM, excluding
the 256 KB framebuffer in video RAM.
4. On the real unit, the same payload is delivered by `rdebug` plus the
Ethernet loader and plays without a host attached.
5. Existing `harness/` behaviour and the MAME no-env regression baseline are
untouched. The ROM stays byte-identical.
## 4. Non-goals
* The full 3:39 clip (the 30 s limit is what makes local RAM playback possible).
* Audio (the VT1200 has no audio path in this harness).
* Grey, colour, dither, or anti-aliased pixels; this is 1 bpp.
* Pixel-exact reproduction of the source. The target is a recognisable,
clean vector reinterpretation.
* Coexistence with the resident OS after takeover.
* A general-purpose video player; this is a demo with a fixed 30 s stream.
## 5. Constraints and measured facts
### 5.1 Hardware
| item | value | source |
|---|---|---|
| CPU | TMS34010FNL-50, 50 MHz, 1 state = 160 ns | `harness/mame/vt1200.cpp` |
| panel | 1280×1024 visible, 2048-bit (256-byte) row pitch, 72 Hz | `RENDER_REPORT.md`, driver |
| main RAM | 4 MiB at bit address 0 | `vt1200.cpp` address map |
| video RAM | separate, at bit `0x04000000` | `vt1200.cpp` |
| `FILL L` on the unit | **0.40 µs per destination word** (16 pixels), plus ~2.6–3.3 µs per call | `2026-10-31-badapple-60p-design.md` §5 |
| full 1280×1024 repaint | ~82k words ≈ **33 ms** | derived from the unit rate |
| 60 fps budget | 16.7 ms ≈ **41k words** ≈ **667k pixels** | derived |
| 72 Hz refresh | 13.9 ms per refresh | driver |
The 72 Hz display and 60 fps source have no common period; the player must
pace a 60 Hz timeline against a 72 Hz refresh (§8.6).
### 5.2 Source
| item | value |
|---|---|
| video | 960×720, 60 fps, h264, 219.5 s, 13,167 frames |
| audio | AAC stereo, ignored |
| first 30 s | 1,800 frames |
| black coverage | mean 55 %, p75 80 %, p95 98 % (1280×1024 threshold) |
The source is already high-contrast black-on-white; it is the ideal input for
a threshold-and-trace pipeline.
### 5.3 The measured data budget
A host-side pass over the first 30 s at a **640×512 logical grid** (each
logical pixel is a 2×2 screen block), hard threshold, no vectorisation yet,
measured the tile-delta format in §8:
| quantity | mean | p50 | p95 | max |
|---|---|---|---|---|
| dirty 32×32 logical tiles per frame | 47.4 | 49 | 79 | 295 |
| rectangles per frame | 437 | 451 | 820 | 1,381 |
| bytes per frame (5 B tile header + 3 B/rect) | 1,548 | 1,598 | 2,855 | 5,618 |
**Total for 1,800 frames: 2.79 MB.** The data area is
`0x010000..0x317FFF` = **3.03 MB**, so this leaves ~0.24 MB of headroom. A
scene cut touches up to 295 tiles, still only 5.6 KB and one slow repaint.
The vectorisation step will change these numbers; the first milestone
re-measures them. The gate is **≤ 3.0 MB** and a p95 paint cost within the
refresh budget. If vectorisation pushes past 3.0 MB, the fallback is stronger
simplification or a 4×4 logical grid, not a shorter clip without asking.
**Why 3.03 MB and not 3.5 MB:** the movie data starts at byte `0x010000` and
must stop below the payload's `rdebug` origin at bit `0x018C0000` = byte
`0x318000`. `0x318000 - 0x010000 = 0x308000` = 3.03 MB. Moving the payload
origin would cost the one free block the real loader is known to accept, so
the design keeps the origin and caps the data instead.
## 6. Approach
### 6.1 Why not store rectangles for every frame?
A full-frame rectangle list is simple but forces a full repaint: at the unit's
0.40 µs/word, 1280×1024 costs ~33 ms, i.e. two 60 fps frames. The design must
paint **only what changed**. The measured dirty-tile format does that: typical
frames touch ~47 tiles (≈192k screen pixels ≈ 4.8 ms of fill), and the data
still fits in 4 MB.
### 6.2 Why a 640×512 logical grid?
At 1×1 (1280×1024) the same delta format measures ~4× the data (≈10.8 MB) and
does not fit. At 2×2 it fits with room to spare, and 640×512 is still four
times the resolution of *Another World*'s 320×200. The 2×2 quantisation is
also part of the requested look: it turns pixel-level fuzz into flat blocks
with straight polygon edges.
**call:** 640×512 logical, 2×2 screen blocks. If the first milestone shows
the data is unexpectedly small, 1×1 can be re-measured as a quality mode; it
is not the baseline.
### 6.3 Why polygons then rectangles?
The polygon step is the **look**: threshold, despeckle, trace contours, simplify
to straight segments, and fill the result. That produces the flat, angular
*Another World* silhouette. The runtime does not rasterise polygons; the
host rasterises the simplified polygons back to the 640×512 grid and decomposes
each dirty tile into rectangles. Rectangles are what `FILL L` paints fastest
and most simply. The player is a rectangle decoder, not a polygon engine.
### 6.4 Why not `LINE`?
`LINE` costs 5 states/pixel (≈800 ns/pixel), while `FILL` costs ~25 ns/pixel
on the unit. Filling a polygon with `LINE` outlines would be 30× slower and
would not fill the interior. `LINE` is reserved for an optional debug view
that draws polygon outlines; the animation uses `FILL L` only. The request
said "FILL and/or LINE"; this is the FILL branch, for a measured reason.
## 7. Pipeline
```
hd60.mp4
│ ffmpeg: first 30 s, 60 fps, 640×512 gray
▼
gray frames ──► temporal median (3) ──► 3×3 blur ──► threshold
│
▼
binary frames ──► potrace -a 0 (straight polygons) ──► RDP simplify
│
▼
simplified polygons ──► rasterise to 640×512 binary
│
▼
tile diff (32×32 logical) ──► per-tile background + rectangles
│
▼
movie.bin + index + manifest.json
│
▼
player payload (TMS34010 C + FILL asm)
```
### 7.1 Extract and stylise
* `ffmpeg -t 30 -vf fps=60,scale=640:512:flags=lanczos,format=gray`.
* **Temporal median over 3 frames** before threshold. This is the main defence
against dither flicker and per-frame contour boiling; it is cheap on the
host and does not cost the player anything.
* **3×3 box/Gaussian blur** then threshold at 127. This removes halftone
patterns and turns them into flat regions, as requested.
* **Despeckle:** drop connected components below ~8 logical pixels.
* **Trace:** `potrace -b geojson -a 0 -t 4 -u 1`, which emits straight-line
polygons only (`-a 0` disables curves). Parse the GeoJSON rings.
* **Simplify:** Ramer–Douglas–Peucker on each ring, epsilon 1.5 logical pixels
(tunable), minimum segment 2 px. This is the "straight lines" step.
* **Rasterise:** even-odd scanline fill of the simplified rings back to
640×512. The result is the frame the player will actually show.
All parameters live in one `pipeline/params.json` so the look can be tuned
without touching code. A `preview` command writes side-by-side PNGs of source,
thresholded, traced and final frames.
### 7.2 Tile-delta format
The logical frame is divided into **32×32 logical tiles** (64×64 screen
pixels): 20 × 16 = **320 tiles**, row-major.
For each frame, compare with the previous frame. For every tile with any
changed pixel:
* choose the tile's **background colour** (black or white) — the majority
colour in that tile;
* decompose the **minority colour** into rectangles (merge identical
horizontal runs vertically);
* emit the tile index, the background bit, the rectangle count, and the
rectangles.
The player paints a dirty tile by:
1. `FILL` the whole 64×64 screen tile with the background colour (one call);
2. `FILL` each rectangle with the opposite colour.
This reconstructs the tile exactly, including erasing old ink. Unchanged
tiles are not touched. The background choice keeps mostly-black tiles cheap
(one fill, few white holes).
**Record layout (little-endian):**
```
header:
magic "BAPL" 4
version u16
logical_width u16 (640)
logical_height u16 (512)
scale u16 (2)
tile_logical u16 (32)
tiles_x, tiles_y u16 (20, 16)
frame_count u32
index_offset u32
data_offset u32
source_fps u16 (60)
display_hz u16 (72)
index[frame_count]:
offset u32 (byte offset of the record)
ticks u8 (1 = 1/60 s, 2 = held for 2/60 s, ...)
flags u8 (bit 0: scene cut / full repaint)
record:
dirty_tile_count u16
for each dirty tile:
tile_index u16
bg_and_count u16 (bit 15 = background colour, bits 0-14 = rect count)
rects[count]:
x0:5 y0:5 w:5 h:5 packed into 3 bytes
(logical tile coordinates; w,h are stored as w-1,h-1 so 1..32 fits in 5 bits)
```
Rectangles use the minority colour; the tile background is the majority. A
rect count of 0 is legal (a solid tile). The count is 15 bits, so the
theoretical maximum of 512 horizontal runs in a 32×32 tile cannot overflow;
the decoder rejects a count larger than the remaining record bytes.
**Sizing:** 5 bytes of tile header + 3 bytes per rect. The measured mean is
1.55 KB/frame, total 2.79 MB for 1,800 frames. A scene-cut record with 295
tiles and ~1,300 rects is ~5.6 KB, still one record.
### 7.3 Frame-rate adaptation
The encoder estimates each record's cost from the tile and rectangle areas
(at 0.40 µs per 16-pixel word), the per-`FILL` call overhead (~3 µs), and the
C decoder's per-rect work. The calibration constants come from milestone 3;
the initial values are 0.40 µs/word, 3 µs/call and 1 µs/rect. The gate is
**13.9 ms** — one 72 Hz refresh — not the 16.7 ms nominal 60 fps period,
because a record that misses 13.9 ms loses a whole refresh on the 72 Hz panel.
If a record exceeds the gate:
* first try stronger simplification for that frame (larger RDP epsilon,
larger minimum feature);
* if it still exceeds the gate, **hold the previous frame for an extra
1/60 s** and compute the next delta from the held frame. The index `ticks`
field records the hold.
This is build-time frame dropping, so the delta chain stays exact. The player
never skips a record at runtime: every record is a delta from the one before,
so skipping one would leave its tiles stale. If the real unit is slower than
the encoder's estimate, the movie **stretches** — every frame is still shown,
in order, with the correct picture — rather than dropping or corrupting
frames. That is the safe failure mode; the encoder is re-tuned from the unit's
measured cost if the stretch is noticeable.
### 7.4 Scene cuts
A large frame-to-frame difference is a cut, not motion. The encoder marks the
record `flags & 1`, and emits every tile as dirty. The player repaints the
screen tile by tile. This costs one slow frame (~30–40 ms) and may drop the
next refresh; that is the documented price of a cut. Future work could split a
cut over two frames if it looks bad.
## 8. Player
### 8.1 Memory layout
| region | byte address | use |
|---|---|---|
| `0x000000`–`0x00FFFF` | 64 KB | low staging / scratch; free after load |
| `0x010000`–`0x317FFF` | 3.03 MB | movie data (`movie.bin`) |
| `0x318000`–`0x3FFFFF` | 0.9 MB | payload code, LANCE block, read-only tables, stack |
| VRAM `0x04000000` (bit) | 256 KB | single framebuffer (separate) |
`0x018C0000` is a **bit** address = byte `0x318000`, the low end of the high
code block. The existing loader's origin is reused so `rdebug` needs no new
address. The movie data starts at byte `0x010000` and is loaded over the
network after takeover; it overwrites the OS, which is dead by then.
The LANCE block (init block, rings, RX buffers) lives in the high code block,
following `harness/udpstream/vidstream.c`, which places its RX buffers near
byte `0x3BC000` and is documented as having run on the unit. The 64 KB low
reservation is a **fallback staging area** if the real chip turns out to
truncate a high RX buffer address; the loader then receives low and copies
up. Milestone 5 settles this before the full 30 s run. It is a loader change,
not a player change.
### 8.2 Takeover
The payload is entered from `rdebug`. It:
1. publishes a marker word;
2. executes `DINT` once, for good, and stops the OS's LANCE use;
3. takes the LANCE over with the verified init/ring sequence and receives
`movie.bin` into `0x010000` (the OS is already dead; the display controller
keeps scanning out because it was configured before takeover);
4. verifies the movie header and a whole-file checksum;
5. reads the display registers the OS programmed (`DPYCTL`, `DPYSTRT`,
`DPYINT`, pitch) and publishes them for diagnosis; it inherits the mode
rather than hardcoding one;
6. clears the screen to white with one full-screen `FILL`;
7. enters the playback loop.
Interrupts stay disabled. The OS's timer and DUART ISRs are never entered
again. The LANCE is stopped after the load; the player no longer needs it.
**The OS dies before the first byte lands in low RAM** — that is what makes
overwriting its heap and ISRs safe. The earlier wording, "receive while the
OS is alive, then DINT", was wrong; it would crash mid-load.
**Recovery:** a hang after step 2 is a power cycle. The README says so in the
first screenful.
### 8.3 Loader
Two paths, same payload:
* **MAME / development:** a Lua script writes `movie.bin` into MAME's RAM at
byte `0x010000` and the payload at `0x318000`, then jumps. Slow but fine
for 5 s slices.
* **Real unit / full 30 s:** `rdebug` writes a small bootstrap into the high
block. The bootstrap disables interrupts, takes the LANCE over with the
proven register/ring sequence from `harness/udpstream` (copied, not edited),
receives `movie.bin` in 1,450-byte datagrams into `0x010000`, ACKs every
window, and jumps to the player entry. The host sender is a small Python
tool using the same raw-Ethernet framing as `harness/badapple` /
`harness/udpstream`.
The baseline places the RX buffers in the high code block, as
`vidstream.c` does. If the real chip truncates a high buffer address, the
fallback receives into a 32 KB ring below 64 KB and copies each datagram up
with `MOVE`/word copy. That copy rate is measured in milestone 5; it is a
loader-only fallback, not the baseline.
The protocol is deliberately simple: `u32 offset`, `u16 length`, `u16 seq`,
`u16 crc`, then data. The receiver ACKs the highest contiguous offset every
32 datagrams; the sender retransmits from the first gap. A final whole-file
CRC-32 is checked before the jump. If the load fails, the payload paints an
error block and halts rather than jumping into a partial movie.
### 8.4 Rendering
All drawing goes through one assembly primitive:
```asm
; fill_rect(x, y, w, h, color)
; x,y,w,h in screen pixels (already scaled by 2)
; color 0 = black, 0xFFFF = white
```
It sets `DADDR`, `DYDX`, `DPTCH=0x800`, `PSIZE=1`, `PMASK=0`, `CONTROL`
replace, and `COLOR1`, then executes `FILL L`. Constants are programmed once
per run; only `DADDR`, `DYDX` and `COLOR1` change per call, and `FILL` destroys
`DADDR`/`DYDX`, so those two are reloaded each time. This is the same
implied-operand discipline as `harness/udpstream/vram.asm`, which is already
verified on the unit.
The C player converts each packed tile rect to screen coordinates:
```
sx = (tile_x*32 + x0) * 2
sy = (tile_y*32 + y0) * 2
sw = (w + 1) * 2 // stored as w-1
sh = (h + 1) * 2 // stored as h-1
```
and calls `fill_rect`. A dirty tile is one background fill plus its rects.
**Hoisting:** `PSIZE`, `PMASK`, `CONTROL`, `DPTCH` are set once at startup.
`CONVDP`/`CONVSP` are not needed for `FILL L` (linear destination), but are
set to `0x14` for consistency with the verified primitive.
### 8.5 Frame buffer and presentation
**Baseline: one framebuffer, incremental dirty-tile updates.** Drawing a
record takes ~2–10 ms for a typical frame, a fraction of the 13.9 ms refresh.
Updates are visible as they land; the TMS34010's VRAM is dual-ported, so the
display does not stall the CPU. At 60 fps the eye usually reads this as
motion, but a large change can show a brief top-to-bottom wipe.
**Tear-free is a separate mode, not free.** It needs either:
* **two framebuffers and a vblank flip.** The real panel uses a 256-byte row
stride, so one 1280×1024 buffer is **256 KB**, not 160 KB; two are 512 KB.
Whether the unit has that much VRAM is not established. It also doubles the
fill work unless each delta is applied to both buffers, which costs the
p95 frame its refresh.
* **beam chasing.** Draw each dirty tile only after the scanout has passed it,
so the new content appears on the next frame. This is tear-free with one
buffer but needs `VCOUNT` scheduling and leaves one frame of latency.
Neither is in the baseline. **human:** incremental painting is acceptable for
the prototype, so the baseline is one framebuffer and dirty-tile updates.
Milestone 2 still measures VRAM capacity for the record, but a tear-free mode
is not built unless the incremental look fails on the unit.
### 8.6 Pacing: 60 fps content on a 72 Hz panel
The player polls the TMS34010 display interrupt pending bit (`INTPEND` DI) at
each vertical blank; interrupts remain disabled, so this is a poll, not an
ISR. A 16.16 fixed-point accumulator advances the 60 Hz timeline:
```
each vsync: now += 60/72
if now >= next_frame_time:
draw next record // never skip a record
next_frame_time += record.ticks
if now - next_frame_time > 1.0: // fell behind
next_frame_time = now // slow down, do not skip
```
If a record's paint overruns a vsync, the movie **stretches**: the next record
is drawn as soon as the CPU is free, and the timeline is re-based to `now`
rather than accumulating debt. Records are never skipped, because each one is
a delta from the previous picture. This produces the correct average rate on
a unit that keeps up, and slow motion — not corruption — on one that does
not. The 5:6 cadence (five source frames per six refreshes) is visible only as
the usual 60-on-72 judder; it is a hardware property, not a decoder stall.
If `INTPEND` polling proves unreliable in MAME, the fallback is a DUART counter
programmed as a 60 Hz timebase, polled with interrupts off. That is a builder
detail; the timeline arithmetic is unchanged.
## 9. Verification
### 9.1 Host pipeline tests
* Frame count, dimensions and LSB-first bit order (the harness's long-standing
trap: an MSB-first packer mirrors every eight pixels).
* Polygon round-trip: rasterised simplified polygons differ from the
thresholded source only in the intended simplified regions; report PSNR and
changed-pixel count, not a pass/fail on aesthetics.
* Tile-delta round-trip: applying every record to a blank 640×512 canvas
reproduces the encoded frame **bit for bit**.
* Budget: total `movie.bin` ≤ 3.0 MB (the hard cap is 3.03 MB), p95 paint cost
≤ 13.9 ms including the C decoder and `FILL` call overhead.
* Manifest records the actual numbers (data size, dirty-tile distribution,
paint-cost distribution, dropped frames).
### 9.2 MAME pixel verdict
The player publishes a marker page in high RAM:
```
frame_index, ticks_played, last_record_bytes, paint_loops, data_error
```
`VT1200_DUMP_RAM` and `VT1200_DUMP_VRAM` at the same MAME frame give both the
marker and the picture. A checker reads the marker, reconstructs the expected
logical frame from the encoder's records, downsamples the 2×2 VRAM to
640×512, and compares **exactly**. A single wrong bit is a bug, not a PSNR.
Sample frames: 0, 60, 300, 600, 900, 1200, 1500, 1799, plus every scene cut.
Also run a 5 s slice first; do not debug the full 30 s before one frame is
exact.
### 9.3 MAME motion verdict
Run with `-window`, throttle at the emulated 72 Hz, and capture with
`x11grab` as the existing `run-gui.sh` does. Check:
* no half-painted or stale regions after a cut;
* no accumulating offset (a delta applied twice or skipped);
* the 30 s ends on the expected last frame;
* the paint-cost histogram matches the encoder's estimate.
MAME cannot prove the real unit's 60 fps: its TMS34010 interpreter has a
different instruction/cycle mix from silicon. It proves correctness and gives a
relative cost. The unit is the performance verdict.
### 9.4 Real unit
The existing `harness/badapple/demo.sh` boot gate (wait for the OS's ICMP
reply, then load over rdebug) is the model. The new `bad-apple/load.sh`:
1. waits for the unit's stack;
2. `rdebug` writes the bootstrap;
3. jumps;
4. sends `movie.bin` over the tap/interface;
5. waits for the bootstrap's final ACK;
6. observes the screen (there is no back-channel after the jump).
Record a video of the 30 s run and note the actual dropped frames. Recovery is
a power cycle; say so before running.
## 10. Milestones
1. **Measure the stylised stream (host only).** Extract, vectorise, tile-diff,
and report the numbers. Gate: ≤3.0 MB, p95 paint cost within 13.9 ms.
Produce a preview PNG/MP4 of the vectorised look for approval.
2. **One exact frame in MAME.** Build the payload with a single hardcoded
record, paint it, dump VRAM, compare bit-exact. This proves `FILL` geometry,
the 2×2 scaling, the test harness and the marker page.
3. **Five-second player.** Pack a 5 s slice, play it, compare every sampled
frame, and measure the paint-cost histogram. No loader yet; use the Lua
path.
4. **Full 30 s.** Pack 1,800 frames, play in MAME, tune simplification and
frame drops until the data and motion gates pass.
5. **Delivery.** Build the rdebug bootstrap + LANCE loader, run the same 30 s
in MAME over the tap, then on the real unit.
6. **Report.** `bad-apple/README.md` and a short report with the measured data
size, paint-cost distribution, dropped frames, unit behaviour and the
remaining gaps.
## 11. Risks
| risk | why it matters | mitigation |
|---|---|---|
| Potrace/RDP temporal jitter ("boiling") | looks like a glitch even when every frame is exact | 3-frame temporal median, spatial blur, minimum feature size; preview at 60 fps before committing to the full 30 s |
| Vectorised data exceeds the 3.03 MB data area | no room for the 30 s | measured 2.79 MB before vectorisation; gate at 3.0 MB; fallback is 4×4 blocks or stronger simplification, not a shorter clip without asking |
| Dirty tiles exceed the paint budget | frames hold or stretch | build-time `ticks` holds, adaptive simplification, per-frame cost estimate; report the distribution |
| Incremental updates show a wipe | accepted for the prototype, but may look worse than expected on the unit | baseline is incremental by human decision; if it reads as a glitch, milestone 2's VRAM measurement feeds a double-buffered/beam-chasing variant |
| The real unit is slower than the model | the movie stretches rather than drops | never skip a record; re-base the timeline; re-encode with stronger simplification from the measured unit cost |
| LANCE loader loses datagrams | partial movie, dead machine | offsets + CRC + ACK/retransmit; whole-file CRC before jump; error block and halt instead of jumping |
| Real C-LANCE high buffer addresses | loader cannot receive above 64 KB | baseline reuses `vidstream.c`'s high block, documented as run on the unit; fallback is a low staging ring plus copy, measured in milestone 5 |
| 60 fps on 72 Hz | unavoidable judder | fixed-point 5:6 timeline; document it; do not call it a decoder bug |
| OS video mode not configured | no picture after takeover | inherit and publish the OS registers; boot path is "wait for the OS, then take over" |
| One demo per boot | a hang costs a power cycle | verify the image/checksum before jumping; README warning; no automatic retry after takeover |
| MAME timing is not silicon | false confidence in 60 fps | correctness in MAME, performance measured on the unit; report both |
## 12. Left to the builder
* Exact RDP epsilon, minimum feature size, and temporal-filter width: tune by
preview, then freeze in `params.json`.
* Whether the tile background choice is per-tile majority or a cost-minimising
search; both are host-side and cheap.
* The precise `CONTROL` mask and hoisting split for `FILL L`; copy the verified
`vram.asm` discipline and measure once on the unit.
* Whether a short slice uses the Lua/debugger load or the same bootstrap; the
full 30 s uses raw LANCE because the OS is dead before the data lands.
* The marker page layout and the exact MAME dump procedure.
* Whether the first 30 s starts at t=0 (assumed) or at a musically chosen
offset; t=0 is the default.
* Whether a stop/pause key is worth keeping. It would require one live ISR or
polling the keyboard DUART; the default is no input, as the OS is dead.
## 13. Decisions from this session
* **human:** full screen, 1280×1024, first 30 s.
* **human:** as close to `hd60.mp4` as the medium allows.
* **human:** flat black-and-white polygons in the *Another World* spirit;
drop dither and simplify.
* **human:** 60 fps target, drops allowed for complex scenes.
* **human:** the program may take the whole 4 MB; the OS may die; recovery is a
hardware restart.
* **human:** bootstrap over `rdebug`; the existing network loader may be used
to speed the big load.
* **human:** new top-level `bad-apple/` folder.
* **human:** incremental dirty-tile painting is acceptable; a brief wipe on a
large change is not a failure for this prototype.
* **human:** stretch the 4:3 source to the 5:4 panel; keep all content.
* **call:** 640×512 logical grid (2×2 blocks), 32×32 logical tiles.
* **call:** `FILL`-only runtime; `LINE` reserved for debug outlines.
* **call:** 60 Hz timeline on a 72 Hz panel, with build-time `ticks` holds.
* **call:** 30 s from t=0.
* **call:** a `--fit crop` switch remains available for comparison, but stretch
is the default.
@@ -0,0 +1,514 @@
# Bad Apple at 60 fps, exact — design
Date: 2026-10-31
Status: **rev 1** — first draft, awaiting approval.
Author: (session). Decisions marked "**call:**" are the builder's proposals, not
something the human partner has agreed to. Decisions marked "**human:**" were
given in this session and are not open.
Supersedes the tile-codec half of
[`2026-10-30-vt1200-video-streaming-design.md`](2026-10-30-vt1200-video-streaming-design.md)
(§8.3 scheme C, §8.9 OS interference, §10 M3–M5). The transport there (§8.1), the
back-channel (§8.4) and the delivery path (`harness/badapple/`) stand.
---
## 1. Intent
Play **Bad Apple!!** on a VT1200 from
`/tmp/opencode/badapple-real/hd60.mp4` at the source's **60 fps**, upscaled to
fill the panel, with the picture **no longer quantised through a 256-entry tile
dictionary** — because that dictionary, not the resolution, is what makes the
diagonals jagged.
The ROM stays byte-identical (`md5 b46a3e442ccb86134d2356bd70d22fd2`). The OS is
allowed to die (§8.9 of this document).
## 2. Grounding: the source
`ffprobe /tmp/opencode/badapple-real/hd60.mp4` (human-supplied, this session):
| field | value |
|---|---|
| video | 960x720, SAR 1:1, DAR 4:3, **60 fps**, progressive, yuv420p, h264 Main |
| duration | 00:03:39.52 → **≈13,171 frames** at 60 fps |
| audio | aac LC stereo (ignored — no audio path in this harness) |
960×720 is 4:3 exactly, and 1280×960 is 4:3 exactly, so an upscale to
1280×960 is a clean 1.3333× with no crop and no aspect error. 960 is also a
multiple of 32, so `clip_from_video.py`'s default `--height 960` is already
legal (its `height % 32` check at line 98 rejects 720, not 960).
**human:** deliver it upscaled to fill the panel rather than 1:1 with borders.
**call:** the target geometry is **1280×960 at the top of the screen**, rows
960–1023 a permanently black bar. Stretching 960 rows to 1024 (a further 6.7 %
in Y) is a one-flag alternative if the bars look wrong on the unit; it changes
no other number in this document except the bar tiles.
## 3. Grounding: what exists today
All paths below are in this repository.
| piece | where | what it does |
|---|---|---|
| payload | `harness/udpstream/vidstream.c` | takes the LANCE over, arms a 32-descriptor RX ring in two 48 KB slots, parses VS frames, dispatches to `paint` / `paint_tiles` / `paint_delta` / `dict_chunk`, sends a status frame every 64 accepted frames |
| tile primitive | `harness/udpstream/vram_tile.asm` | one `PIXBLT B,XY` of a 32×32 dictionary entry; ~35 instructions of prologue per call |
| strip primitive | `harness/udpstream/vram_b.asm` | one `PIXBLT B,XY` of a row with `SPTCH=0` vertical stretch |
| fill primitive | `harness/udpstream/vram.asm` | `FILL L` of the whole screen (fixed geometry) |
| wire format | `harness/udpstream/frame.py` + `frame_parse.h` | 16-byte `VS` header, header-only CRC-16, kinds 1–5 |
| tile wire form | `harness/udpstream/tiledict.py` | `TILES_FULL` = 1,280 index bytes; `TILES_DELTA` = `u16 count` + `count×u16 pos` + `count×u8 index`; `KEYFRAME_EVERY = 60` |
| dictionary build | `harness/udpstream/tilecost.py` | Lloyd/k-means over 32×32 tiles, Hamming distance, 256 entries × 128 B |
| encoder | `harness/udpstream/make_tiles.py` | clip → dictionary + indices + stream plan + manifest |
| sender | `harness/udpstream/vidsend.py` | paces at `--fps`, reads status frames, forces a keyframe when `drops` rises |
| checker | `harness/mame/tile_check.py` | decodes `stream.bin` independently, renders it, compares against dumped VRAM |
| demo | `harness/badapple/demo.sh` | waits for the OS's stack, loads the payload over rdebug, sends the clip |
Measured behaviour of today's codec, all from `harness/mame/NETWORK_REPORT.md`:
| quantity | value | where |
|---|---|---|
| synthetic clip, 30 frames | PSNR **29.47 dB**, 1.16 bits/tile mean error, 97.0 % of tiles exact | §76.4 |
| **real clip** (3,292 frames, 15 fps, 1280×960) | **PSNR 24.4 dB**, mean delta **496 B**, 55 keyframes, 1.63 MB stream | `harness/badapple/README.md` |
| one full tile repaint in MAME | `decode_worst` **37,612 REFCNT ticks** (1,280 tiles) | §76.5.1 |
## 4. The diagnosis
**The jaggies are the dictionary, and they can be counted.** 24.4 dB is a
bit-error rate of 0.0036, i.e. **≈4,760 wrong pixels per frame** out of
1,310,720. A 32×32 tile has on the order of 2³² distinct edge placements and the
dictionary has 256 centroids, so the errors concentrate on edges — which is
exactly what a viewer sees as ragged diagonals. At 1280 wide there is no
resolution left to buy; the fragment is the *edge vocabulary*.
Three other causes look identical on screen and must be ruled out before the
codec is blamed. All are cheap:
1. `clip_from_video.py` line 49/64: `scale=...:flags=lanczos` then `gray >
threshold`. Hard-thresholding a downscaled frame makes its own staircase.
Dump one `.bin` as an image and look.
2. `VT1200_MONITOR=VR150` shears the picture (`harness/mame/README.md`).
3. A source frame whose edges are already stepped (e.g. a 4:3 crop of a
differently-scaled master).
**And the speed is not the constraint.** At the real clip's rate of change a
delta is ~165 tiles at 15 fps, so ~41 tiles per frame at 60 fps (motion per
frame divides by four). Against the budgets in §6 that is roughly **1/20th of
one frame**. There is no speed problem to solve in the delta path; there is
spare budget to spend on exactness.
## 5. What the hardware actually charges for
The single most important number in this document, measured on the unit and
recorded in the 2026-10-30 design (§2, §6):
> `FILL L` costs **0.40 µs per destination word**; `PIXBLT L,XY` costs
> **0.62 µs/word**; each blit/fill *call* costs **2.6–3.3 µs** on top.
A destination word is 16 pixels at 1 bpp. So **the pixel engine's cost is
proportional to the number of destination words written, and essentially
independent of which primitive writes them and of how the writes are grouped.**
Consequences, all of which this design depends on:
* A 32×32 tile is 32 rows × 2 words = **64 destination words**.
* A 48×48 tile is 48 × 3 = 144 words; a 64×64 is 256. Per *pixel* there is no
gain — the extra call overhead saved (0.8 ms vs 3.1 ms over a full screen) is
under 10 % of a repaint. **Tile size is a wire decision, not a CPU decision.**
* `FILL` and `PIXBLT` cost the same per word, so collapsing a run of solid tiles
into one `FILL` saves **call overhead and wire bytes only**, not blitter time.
* The whole 1280×960 picture is 960 × 80 = **76,800 destination words**.
At 0.40 µs/word that is **≈31 ms** — *more than one 60 fps frame*, whatever
scheme drives it.
Note the contradiction this exposes: MAME reports a full tile repaint at
~12 ms (37,612 REFCNT ticks, §3) while the unit's `FILL` rate says the same
pixels cost ~31 ms. `PIXBLT B,XY` at "0.32 µs/word" is a **MAME benchmark and
has never been measured on the unit** (the 2026-10-30 doc says so in its own
risk table). Every per-frame CPU number in this document is therefore labelled
either **unit** or **MAME**, and the first milestone measures the gap.
## 6. Budget
| quantity | value | source |
|---|---|---|
| frame period at 60 fps | **16.67 ms** | derived |
| display refresh | 72 Hz (13.9 ms) | 2026-10-30 §2 |
| destination words affordable per frame | **41,700** at 0.40 µs/word | derived from the unit rate |
| full picture (1280×960) | 76,800 words ≈ **31 ms** = **1.84 frames** | derived, unit rate |
| link | 10 Mbit half-duplex, **1.27 MB/s** measured | 2026-10-30 §2 |
| wire per frame at 60 fps | **21 KB** | derived |
| RX ring | 32 descriptors × 1526 B = 48,832 B (one slot) | `vidstream.c` |
| `payload_len` | u16 → a frame's payload is ≤ 65,535 B | `frame_parse.h` |
| heap | ~2.7 MB free at takeover | status frames |
Two hard limits follow: **a frame's payload cannot exceed 65,535 B**
(`payload_len` is 16 bits) and **cannot exceed 32 datagrams** (~45.9 KB of
payload) without deepening the RX ring. The binding one is the ring.
**The central finding: at 60 fps a full-screen repaint fits neither budget.**
It costs 1.84 frames of CPU and ~7 frames of wire (154 KB at 1.27 MB/s =
121 ms). Everything in the design below exists to make sure the full screen is
never repainted in one frame, and that ordinary frames are exact.
## 7. Approaches considered
**A. Keep the dictionary; widen the index to 10–12 bits.** Gets PSNR up by
increasing entries (256 → 1,024+). Wire: 2 B/tile. Cost: a 92-datagram upload
(the current 32-descriptor ring holds 23), plus the dictionary still cannot
represent every edge. **Rejected as the primary path**: it improves the same
approximation that causes the complaint, and it keeps the k-means encoder and
the 32 KB upload.
**B. Keep the dictionary; add an 8×8 residual patch pass.** Exactness with
bounded cost (patches only where error > 0). **Not rejected, deferred**: it is
the best option if raw bitmaps turn out too expensive on the wire, and it reuses
everything. Kept as the fallback in §12.
**C. Raw tiles — send the changed tiles' actual bits, with a solid tag.**
No dictionary at all. Exact by construction. Wire: 1 bit per changed pixel plus
a 1-byte tag per changed tile; a solid tile costs 1 byte instead of 128.
**Chosen.**
**D. Spans / vectors (`FILL XY` per scanline run).** Exact and the same
per-word cost as everything else, so its only advantage is that it *writes fewer
words* on flat content and its only cost is that it turns dither into runs of
length 1. Bad Apple is full of dithered texture (grass, hair, checkerboard), so
a span encoding explodes exactly where tiles are cheap. **Rejected for this
source**; it stays the right answer for a flat-shaded stylisation, which is a
different film.
**Decision (this document): C.** It is also the *simplest* option — it deletes
more code than it adds (§8.6).
## 8. Design
### 8.1 Geometry
1280×960 at the top of the panel: **40 × 30 tiles of 32×32** = 1,200 picture
tiles, plus 64 permanently black bar tiles (grid rows 30–31) that are painted
once and never listed again. The grid, `TILES_SCREEN`, the row-major position
encoding, `tile_check.py`'s renderer and `vram_tile.asm`'s `DYDX=(32,32)` are all
unchanged — the smallest possible diff to the geometry.
32×32 rather than 64×64 because tile size affects CPU cost by under 10 % (§5)
and affects wire cost a great deal: a solid 32×32 costs one tag, and a changed
32×32 patch costs 128 B against 512 B for 64×64. Finer tiles waste less overdraw
on an edge. **call:** 32×32.
### 8.2 Wire form
`KIND_TILES_FULL` and `KIND_TILES_DELTA` keep their numbers (2, 3) so the kind
table and the existing tests stay meaningful; their **payloads change**, and
`VERSION` goes to 2 so an old sender cannot be mis-dispatched.
```
payload:
offset size field
0 2 count, u16 little-endian, 1..1,280
2 2*count position[k], u16, row-major tile index, STRICTLY ASCENDING
2+2c count tag[k], u8:
0x00 = this tile is solid black
0x01 = this tile is solid white
0x02 = a 128-byte bitmap follows
2+3c m*128 bitmaps for the tags equal to 0x02, in k order
payload_len == 4 + 3*count + 128*m
```
Grouped rather than interleaved (positions, then tags, then bitmaps) so the
receiver can validate **the whole list without reading a single bitmap**:
count in range, length consistent with the count, every position on the grid and
strictly ascending, every tag in {0,1,2}, and the tail length exactly
128 × (number of `0x02` tags). That preserves the property the harness's
off-grid negative control exists to protect — *one bad entry paints nothing* —
and it is `O(count)` with one cursor and no division.
`TILES_FULL` is the same form with `count == 1,200` (all picture tiles); the bar
tiles are never sent.
**Positions refer to the 40×30 picture grid**: 0..1,199, row-major, with
`ty = pos / 40` avoided by incrementing as today. That is a *change* to the wire
constant `FP_TILES_SCREEN`/`TILES_PER_SCREEN` (1,280 → 1,200) and to
`tile_check.py`'s renderer, and the bar rows are outside the wire entirely —
which is the one place this design is not a minimal diff, so it is stated rather
than implied.
**Expected wire, estimated from the one real-clip measurement** (496 B mean
delta at 15 fps ≈ 165 changed tiles → ~41 tiles per frame at 60 fps): if a third
of the changed tiles need a bitmap, ≈1.9 KB/frame ≈ **115 KB/s at 60 fps**, i.e.
9 % of the link. If every one needs a bitmap, 5.3 KB/frame ≈ 320 KB/s, 25 % of
the link. **The measurement in §11 replaces both numbers before any code is
written.**
### 8.3 Decoder
Per frame, unchanged from today's structure (§76's `paint_delta`): two passes,
validate then paint, inside one burst.
Paint per tag:
* `0x00` / `0x01` → one `PIXBLT B,XY` from a **64-word** all-zero or all-ones
constant in the payload's pool, `DYDX=(32,32)`. (A `FILL` is the same cost —
§5 — and `PIXBLT` keeps one primitive and one set of implied operands.)
* `0x02` → one `PIXBLT B,XY` from the bitmap in the frame itself (it is already
in a slot, bit address `rx_frag_ptr + offset*8`; no copy).
**Optimisation carried over from the earlier analysis, and it is the only one
that matters:** hoist `PSIZE`, `CONVSP`, `CONVDP`, `PMASK`, `CONTROL`, `B1`,
`B3`, `B7`, `B8`, `B9` out of the per-tile primitive into a per-frame (really
per-run) setup, and delete the eight push/pop pairs in `vram_tile.asm`. Only
`B0` (SADDR) and `B2` (DADDR) vary per tile, and `PIXBLT` destroys both. This is
a small constant on top of the per-word cost, so it is worth doing once and not
worth designing around.
**Decoder state — the encoder's shadow screen.** Every delta must be computed
against *what the decoder currently shows*, not against the source's previous
frame, because a rolling refresh (§8.4) makes those two different: after a cut
the decoder holds the old picture with a few hundred tiles replaced. So
`vidsend.py` keeps a **shadow index buffer** of the decoder's screen, updated as
each frame is sent and marked "everything stale" when a refresh begins, and
computes every delta against it. Today's `plan()` diffs `previous` and assumes it
arrived; that is already false after a drop and would be routinely false under a
rolling refresh. Small change in the sender, a *precondition* for a correct
picture.
### 8.4 Refresh
A full refresh is **not** one frame. Split into sub-frames of at most **354
bitmaps**: a frame must fit 32 datagrams of 1,450-byte payloads, so
`4 + 3c + 128m ≤ 32 × 1,450 = 46,400` with `m = c` gives `c ≤ 354` — fewer when
the tags make it smaller. (The u16 `payload_len` would allow 500; the ring binds
first.) The sub-frames go out across consecutive frames.
**Bitmaps must not straddle a datagram.** A 128-byte bitmap lands mid-datagram
for most payload sizes, and a split bitmap would need bit-level reassembly across
fragments — per-byte CPU work, which is an anti-goal. **call:** the sender ends
each datagram at a tile boundary, wasting at most 127 bytes of a 1,450-byte
payload (≤ 9 % of the wire) and leaving the decoder a single fragment arithmetic
per bitmap.
**The first refresh is a boot sequence, not a repair.** Before it the picture is
undefined, so painting it progressively is a wipe-in on a black screen. Every
later refresh replaces a picture already on screen, which is why §12 treats the
cut wipe as a real risk. On the wire a refresh
is ~154 KB worst case = **121 ms**, so a refresh occupies ~7 frame periods
whatever the decoder does; the decoder's own share (1,200 × 64 words = 76,800 =
31 ms) is spread by the same split.
**call:** the fixed `KEYFRAME_EVERY = 60` cadence goes away. A refresh is forced
(a) at the first frame, (b) when the status frame's `drops` rises, and (c) on a
slow safety cadence of 3,600 frames (60 s at 60 fps). The reason to remove the
fixed cadence is arithmetic: one 31 ms refresh every second is 1.9 frames of
budget per second *and* 7 frames of wire, i.e. a dropped frame every 60 — the
hitch a viewer would notice. `keyframe_due` keeps its `drops` trigger, which is
the only repair the terminal can actually ask for.
On a hard cut the whole screen changes, so the cut is a refresh and the picture
wipes over ~7 frame periods (117 ms). That is the price of dropping the
dictionary and it is stated here rather than discovered later.
### 8.5 Pacing
**call:** replace the fixed `--fps` clock with **wire-capacity pacing**: send a
frame whenever the ring has room and the previous frame's bytes have left, and
never send faster than 60 fps. Normal frames (1.9–5.3 KB) then arrive at 60 fps
with the link at 9–25 % duty; a refresh (or a cut) temporarily lowers the
delivered rate rather than overflowing the ring or dropping frames. `--fps`
stays as a ceiling so the demo can still be slowed for inspection.
This matters because the failure mode of the fixed clock is silent: at 60 fps a
7-frame refresh cannot be sent, the ring fills, the chip sets `MISS`, the status
frame reports a rise in `drops`, and `keyframe_due` responds by… scheduling
another refresh. Capacity pacing removes the loop.
### 8.6 What gets deleted
* `tilecost.build_dictionary`, `quantise`, `_frame_tiles` and the whole k-means
path; the dictionary, its 32 KB, its 23-datagram upload, `DICT_READY`,
`DICT_BYTES`, `DICT_BAD`, `dict_chunk()`, `frag_copy()` (its only caller),
and the `DICT_CHUNK` kind.
* `OS_TOUCH` and the VRAM sentinel word (`SENT_ADDR`/`SENT_VALUE`), because the
OS is gone (§8.9).
* `TILES_DROP` survives, but now means "a frame failed validation" rather than
"no dictionary yet".
* The per-frame `vid_dint_` / `vid_eint_` pair: one `DINT` at startup, never
re-enabled.
* `CONVSP`-for-source (`PIXBLT B,XY` here has a constant source pitch of 32
bits). `CONVDP`, `PSIZE` and `PMASK` still matter.
Net effect: `vidstream.c` and `make_tiles.py` both get *smaller*, and the
encoder loses its only slow step.
### 8.7 Encoder pipeline
One pass, no intermediate frames: `ffmpeg` rawvideo → numpy → per-frame tile
diff → stream records. Reasons: `tilecost`/`make_tiles` are pure Python and
13,171 frames × 691,200 pixels is 9.1 G pixels (hours in the current code), and
materialising 13,171 × 153,600 B of `*.bin` frames is 2.0 GB of disk.
Bit order stays **LSB-first within each byte** (`clip_from_video.py` and
`vidsrc.pack_row` both spell out why: an MSB-first packer is self-consistent and
mirrors every eight pixels on the display).
**No dither.** A hard threshold is what makes exact deltas cheap; ordered or
error-diffused dither would make nearly every tile dirty every frame and the
wire cost would leave the budget. This is a design constraint, not a preference.
`clip_from_video.py` stays the reference for the extraction front end
(upscale in grey, then threshold); its `--height` check already accepts 960.
### 8.8 Failure behaviour
* Bad header / bad length / descending positions / off-grid position / unknown
tag: refuse the frame, paint nothing, count it (`RX_BAD` / `TILES_DROP`), keep
the picture. Unchanged from today.
* Ring overrun: `MISS` is reported; capacity pacing (§8.5) is the prevention.
* A refresh interrupted (drops mid-refresh): the next `drops` rise restarts it;
the picture is mixed during the window and self-heals on completion. Say so in
the status frame — a `refresh_in_progress` bit is cheap and makes the mixed
window explainable instead of mysterious.
### 8.9 The payload owns the machine
**human:** the resident OS may die; it is already losing its network stack to
the LANCE takeover.
What that removes: `OS_TOUCH` and the sentinel, the `dint`/`eint` burst
discipline, and any question of interleaving.
What it obliges, and this is the part that must not be skipped:
* **One `DINT` at startup, never re-enabled.** With `IE = 0` nothing maskable is
taken, so the OS's old ISRs are never entered against dead state. NMI remains
maskable-only-by-the-host (it needs a `HSTCTL` write) and is not a concern.
* **Video registers are inherited, verified and then owned.** Nothing will
re-program `DPYCTL`/`DPYSTRT`/`DPYADR`/`DPYINT`/`DUDATE`/`HTOTAL`… once the OS
is gone, and the TMS34010's own screen-refresh machinery runs on those values
forever. So the payload reads them at startup, publishes them in the marker
page for diagnosis, and re-asserts them only if a sanity check fails. It does
**not** hardcode a mode: the values that matter are the ones the OS just
programmed for the modelled monitor (`VT1200_MONITOR`, default VR319).
Consequence to state plainly: **a payload loaded from a machine that never
configured video has no picture**, and the demo's boot path is therefore
"boot the OS, wait for it, then take over" — as `demo.sh` already does.
* **Operational cost:** a payload that hangs with the OS dead takes the machine
down to a power cycle — no rdebug, no console. This matches the existing
"one demo per boot" note in `harness/badapple/README.md` and should be
repeated in it.
* The bar rows are painted once by the payload (2 × 1,280 × 32 pixels = 5,120
destination words ≈ 2 ms) so the picture is not sitting on whatever the OS
left there.
## 9. Budgets
Unit rates for the blit path, MAME-labelled where the unit has never been
measured. `B,XY` is shown at both ends of its uncertainty.
| case | tiles | dest words | CPU @0.32 µs/word | CPU @0.62 µs/word | wire |
|---|---|---|---|---|---|
| typical 60 fps delta (≈41 tiles) | 41 | 2,624 | 0.9 ms | 1.7 ms | 1.9–5.3 KB |
| delta, busy frame (≈150 tiles) | 150 | 9,600 | 3.2 ms | 6.1 ms | ~7–19 KB |
| refresh sub-frame (354 bitmaps) | 354 | 22,656 | 7.3 ms | 14.0 ms | ~46.4 KB |
| full picture (1280×960) | 1,200 | 76,800 | 24.6 ms | 47.6 ms | ~154 KB |
Frame budgets: **41,700 words** and **21 KB**. So a typical delta uses ~6 % of
the CPU budget and ~10–25 % of the wire; a refresh sub-frame uses 18–35 % of the
CPU but **>100 % of one frame's wire**, which is why refresh is paced by the wire
and not by the clock (§8.5).
## 10. Verification
* **Pixel verdict, unchanged in spirit:** `tile_check.py` decodes the new
payloads independently and renders them against the payload's dumped VRAM. It
must show **exact** agreement frame for frame — with no dictionary there is no
reconstruction error to report, so a single differing bit is a bug rather than
a PSNR.
* **Host tests:** `harness/udpstream/tests/` keeps its role — the kind table is
mirrored in `frame.py` and `frame_parse.h` and `test_frame.py` compares them;
the new payload needs its own negative controls (count 0, count > 1,200,
descending position, off-grid position, unknown tag, tail length not
128 × m).
* **MAME:** the no-env regression baseline
(`PC=3FDCE400 samples=199 (2 distinct PCs)`, `harness/mame/README.md`) must not
move; the tile modes (`VT1200_MODE=tiles|delta|replay`) are re-pointed at the
new payloads.
* **Unit:** `bench-sweep.sh` methodology plus the 3.5-minute soak; the new
`decode_worst` samples are the instrument. **The `B,XY` per-word rate must be
measured on the unit in the first milestone** — every CPU number above is an
extrapolation from `FILL`/`L,XY` until then.
## 11. First task: measure before building
Before any payload change, one host-side pass over `hd60.mp4` with numpy:
1. extract at **1280×960, 60 fps, hard threshold** (the proposed front end);
2. per frame, diff against its predecessor and record **changed tiles of the
40×30 picture grid**, **changed pixels**, **tags-only bytes**, **bitmap
bytes**, and the solid/bitmap split;
3. report mean / p50 / p95 / max wire per frame, the frames where wire exceeds
the 21 KB budget, and where the cuts are.
This decides three things the design currently *assumes*:
* whether the solid-tag split makes the wire budget comfortable or marginal;
* whether 32×32 stays the tile size (a bitmap-heavy distribution would argue for
a smaller tile, a tag-heavy one for a larger);
* how many refreshes the clip actually needs, i.e. how visible the cut wipes are.
It also settles the extractor question (§4, cause 1) by producing the frames
that the picture will actually contain.
## 12. Risks
| risk | why it matters | mitigation |
|---|---|---|
| `B,XY` rate is a MAME number | every CPU estimate here; the 12 ms MAME repaint contradicts the unit's 31 ms | measure `B,XY` on the unit first (§10); the design is wire-bound on deltas so a 2× CPU surprise costs little |
| refresh is 7 frames of wire | cuts wipe for ~117 ms | capacity pacing; keep the dictionary as a *refresh-only* accelerator (approach B) if it looks bad |
| a cut needs a full refresh | all 1,200 bitmaps, 154 KB | tags collapse the mostly-black ones; if not, approach B |
| killing the OS is irreversible | hang = power cycle; no video if video was never configured | keep the boot path `demo.sh` uses; publish the video registers; never hardcode a mode |
| `payload_len` u16 and a 32-descriptor ring cap a frame at ~355 bitmaps | a naive full-frame refresh cannot be expressed | sub-frames (§8.4); optional 64-descriptor ring if it ever binds |
| 60 fps on a 72 Hz panel | every frame is 1.2 refresh periods, the tear line drifts | accepted (§8.7 of the 2026-10-30 doc); revisit with a start-of-frame delay |
| encoder is pure Python today | 9.1 G pixels | numpy or C, one pass, no intermediate frames (§8.7) |
| the jaggies are not the dictionary | rebuilding the codec would not fix them | §4's three cheap checks run first |
## 13. Left to the builder
* Ring depth: 32 descriptors with 4-way refresh split, or 64 with a wider slot.
* Whether solid tiles use a constant bitmap and `PIXBLT`, or `FILL`.
* Exact `CONTROL` mask and implied-operand setup; how much of it is hoisted and
where the hoist lives (a `_tile_init_` in assembly, or C writing the I/O
registers once).
* The status body's new fields (a `refresh_in_progress` bit, and whether
`decode_worst` should now be reported per kind).
* Whether the encoder keeps `make_tiles.py`'s cache files
(`dictionary.bin`/`indices.bin`/`stream.bin`/`manifest.json`) or moves to a
single stream file.
* How `tile_check.py` reports "exact" (a bit count of zero, or a hash).
* Whether the bars are bottom-aligned (960 rows at the top, §2) or split 32/32.
* Whether the old strip path (`KIND_STRIP`, `paint()`, `vram_b.asm`) is kept as a
black-screen/bring-up test or deleted with the rest of the tile machinery.
Keeping it costs nothing and it is already verified; deleting it removes a
second decoder path.
* `vidsend.py`'s `min_chunk` check exists to stop a frame being split across more
than 32 descriptors; it must be recomputed from the new maximum frame size.
## 14. Decisions from the human partner (this session)
* **The output geometry is upscaled to fill the panel**, not 1:1 with borders.
* **The resident OS may die** — it already loses its network stack to the
LANCE takeover.
* **60 fps is the target**, and the source in `/tmp/opencode/badapple-real/` is
the one to use.
* "We can change everything" — the wire format, the encoder and the payload are
all in scope.
Carried over from 2026-10-30 §14 and still binding: *"30 fps is just best case if
there is enough budget. Real fps can drop."* So a rate that falls below 60 fps in
busy sections is acceptable as long as the picture stays legible and the drop is
a consequence of the content, not of a stall.
## 15. Open questions
1. Is the panel's pixel grid square? If it is not, 1280×960 is a subtly wrong
aspect and the 1024-row stretch becomes the right answer.
2. Does the terminal's Ethernet path ever deliver at more than the measured
1.27 MB/s — is the 10 Mbit figure the NIC or the hub?
3. Should the demo keep any input at all (a stop key), which would argue for
keeping one ISR alive rather than `DINT` forever?
+217
View File
@@ -0,0 +1,217 @@
# Bad Apple on a VT1200
The demo end to end: a payload is **delivered over the network** into a running
terminal, and the picture is **streamed over the network** to it, which paints
it from a 32 KB tile dictionary plus per-frame deltas. Nothing here uses a
debugger — that path (`harness/mame/vidstream.lua`) exists to *test* the payload
against MAME's debugger API and is not how anything reaches a machine.
```
harness/badapple/
├── README.md this file (setup, data preparation, run)
├── demo.sh one command: wait for the OS, load, send
├── rdebug_load.py payload image → terminal RAM over rdebug (TCP 771)
├── clip_from_video.py video → raw 1-bpp frames for make_tiles.py
├── measure_wire.py what the 60 fps raw-tile wire form would cost per frame
└── tests/ unit tests for the two Python tools
```
## How the delivery works
| step | transport | what it does |
|---|---|---|
| boot | — | `harness/mame/run.sh net` (MAME) or the real unit powering up |
| readiness | ICMP | `demo.sh` waits for the terminal's own stack to answer |
| payload | **rdebug, TCP 771** | `poke` the image in 512-byte chunks, read it back, `jump` to the entry |
| dictionary | Ethernet, ethertype `0x88B7` | 23 datagrams, one `TILES_DICT_CHUNK` each |
| pictures | same | one `TILES_FULL` or `TILES_DELTA` per clip frame |
`rdebug` is the firmware's own remote-debug service: `peek`/`poke`/`jump`, the
same protocol `dwt/exploit.py` speaks (`dwt/notes/CROSS.md`). On hardware it is
the delivery path; in MAME it works because the LANCE is attached to a host tap.
Two facts about it are load-bearing and were measured:
* **A 512-byte poke chunk, not 1024.** The firmware documents a 1024-byte clamp,
but a 1024-byte poke is mis-parsed — it reads the body's first word as the
next opcode and answers `Invalid command 0xA5A5A5A5` for an `0xA5`-filled
chunk. 512 is accepted and reads back byte for byte.
* **Nothing can be probed after the `jump`.** A non-returning payload takes the
remote-debug process with it, so `rdebug_load.py` verifies the image *before*
jumping, and the run is observed afterwards from the machine itself (the
screen, or MAME's RAM/VRAM dumps). One demo per boot.
## Setup
Two builds and one piece of data preparation. Steps 1 and 2 are C builds; step 3
is the part that needs a source video.
```sh
cd /path/to/vt1200
# 1. MAME + the driver (skip if vt1200 is already built)
harness/mame/build.sh
# 2. the payload (TI SDK under DOSBox) -> harness/udpstream/build/vidstream.bin
harness/udpstream/build-stream.sh
```
Step 3 is described below, because the numbers (time, disk, what "good" looks
like) are what make it repeatable.
## Data preparation: a video to a tile cache
Nothing ships a full clip: `harness/udpstream/tiles/` in the repo is the
**30-frame synthetic clip**, which is enough to smoke-test the transport but shows
a moving ellipse, not the animation. A real clip is three commands and about
five minutes.
### 3a. Get a source video
Any file ffmpeg can read. For Bad Apple the usable public sources are:
| source | why |
|---|---|
| `archive.org/details/hd-60-fps-bad-apple-shadow-art-pv-720-x-960-60fps` | 960x720, 60 fps, 219 s, ~35 MB — the one this demo was measured on (the item name is scrambled; the file inside is 960x720) |
| `yt-dlp <url>` from YouTube | fine, but the 4K/8K uploads are upscales of the same 2009 render and add no detail |
| the original PV's Flash file (`archive.org/details/dagobah_Bad_Apple_HQ`) | the true master, vector — needs a Flash renderer to rasterise, which nothing here does |
Resolution matters less than it looks: the terminal shows **1 bpp**, and the
pipeline scales to 1280 px wide before thresholding, so anything at 960 px or
more lands on the same picture.
### 3b. Frames
```sh
python3 harness/badapple/clip_from_video.py bad-apple.mp4 /tmp/badapple --fps 15
```
* one `*.bin` per frame, 1280x960, 1 bpp, 160-byte rows, **LSB first** — bit 0 of
a byte is the row's leftmost pixel, the order both the terminal's scanout and
the payload's encoder use. Do not substitute `ffmpeg`'s `format=monob`: it
packs MSB first and mirrors every eight pixels on the display.
* `--fps 15` on the 219 s source gives **3,292 frames**; `--height 960` fills 30
of the 32 tile bands (a 4:3 source at 1280x960, the bottom two rows black), and
`--height 1024` fills the screen by stretching 4:3.
* measured cost: **~12 s**, **489 MB** on disk (153,600 B per frame).
* the source is white paper with black silhouettes, which is already the right
polarity. If yours is inverted, add `--invert`; `--threshold` (default 127)
decides what counts as ink.
### 3c. The cache
```sh
python3 harness/udpstream/make_tiles.py --clip /tmp/badapple --out /tmp/badapple-tiles
```
Takes **~2 min 47 s** (pure-Python Lloyd's algorithm over 3,292 frames) and
writes four files, **6.1 MB** total:
| file | size | what it is |
|---|---|---|
| `dictionary.bin` | 32,768 B | 256 32x32 tiles, the vocabulary every frame is expressed in |
| `indices.bin` | 4,213,760 B | one 1,280-byte index frame per clip frame |
| `stream.bin` | 1,645,081 B | the plan: 55 keyframes + 3,141 deltas + 96 "nothing changed" |
| `manifest.json` | 199 KB | the numbers below, per frame |
It prints the same summary. What it said for this clip, and what to check:
```
3292 frames, 256 dictionary entries (256 used) <- the dictionary is saturated
55 keyframe(s), 3141 delta(s), 96 unchanged
payload 1280 B full, 496 B mean delta (5..3602), 1628615 B total
reconstruction 15823231 bit(s) over 3292 frame(s), BER 0.003667, PSNR 24.36 dB
```
Saturated entries and a 24 dB PSNR are expected, not errors: one byte per tile is
the format's ceiling, and shadow-art edges are expensive. A clip that reports
`inf` PSNR has every tile in the dictionary — that happens only for very flat
footage.
Keep the cache outside the repo: it is 6.1 MB of derived data. Pass it to the
demo with `--tiles /tmp/badapple-tiles`.
## Run it in MAME
Terminal 1 — the machine, with the LANCE on the host tap (`harness/mame/cfg-net`
selects `taptun`; the tap is `tap-mess-<uid>-0` in bridge `vt1200`,
`192.168.99.1/24`):
```sh
harness/mame/run.sh net
```
The harness now models the real unit's **VR319** monitor, so the firmware itself
uses the **1280-px / 2048-bit (256-byte) row layout** the tile payload targets —
and the Terminal Manager before it renders correctly at the same geometry. No
`VT1200_VRAM_REAL` is needed (that knob forced the pitch without changing the
firmware's layout, which is what used to shear the OS screen). To reproduce the
old derived 1024-px layout, run with `VT1200_MONITOR=VR150`; the tile picture
will then be sheared, as before. See `harness/mame/RENDER_REPORT.md` §8.
Terminal 2 — the demo:
```sh
harness/badapple/demo.sh --tiles /tmp/badapple-tiles --frames 3292 --fps 15
```
With no arguments `demo.sh` plays the committed 30-frame synthetic cache: a
smoke test that needs no clip.
## Run it on a real VT1200
The terminal must be reachable over Ethernet and answering rdebug on TCP 771 —
that is all this demo needs, no debug pod and no special build of the firmware.
```sh
# the unit's address, and the host interface that reaches it
harness/badapple/demo.sh --host <unit-ip> --iface <host-iface> \
--tiles /tmp/badapple-tiles --frames 3292 --fps 15
```
Notes that differ from the emulator:
* **Pacing is not needed** for the dictionary upload: the payload's receive ring
is 32 descriptors, one more than the 23-datagram upload, so the whole
dictionary lands before the first chunk is copied. On a slower link the wire
paces it for you.
* **`VT1200_VRAM_REAL` does not exist there** — the unit already uses its own
layout; the switch only affects MAME's renderer.
* The image is position-dependent and is linked at bit `0x018C0000` with its
entry at the address in `harness/udpstream/build/vidstream.env`. Both are
read from that file, so a rebuild moves the entry without editing anything.
## What success looks like
`demo.sh` prints the load verification and the sender's per-frame CSV. The
sender's verdict is about the *transport*:
```
PASS: 2 status frames, last frames_rx=100 (drops=0 crc_err=0)
```
The terminal's own counters, read from a MAME RAM dump (byte offsets from
`MARK_PAGE` = bit `0x01810000`), say what it actually did:
| cell | offset | good value |
|---|---|---|
| `DICT_BYTES` | `0x54` | `32768` (and `DICT_READY` `0x50` = 1) |
| `DICT_BAD` | `0x58` | `0` |
| `RX_OK` | `0x30` | climbs with painted frames |
| `TILES_DROP` | `0x5c` | `0` |
| `RX_DGRAM` | `0x48` | datagrams consumed |
`harness/mame/tile_check.py --replay` is the pixel verdict for the MAME replay
path; for a live run the screen is the evidence.
## Failure modes worth knowing
| symptom | cause |
|---|---|
| `demo.sh: ... did not answer within 180s` | the terminal is not on the tap, or `run.sh net` was started without the LANCE interface selected in its cfg |
| `demo.sh: ... has no dictionary.bin` | `--tiles` wants a **cache** directory, not the frames directory `clip_from_video.py` wrote; the message names the `make_tiles.py --clip … --out …` command that builds it |
| payload already jumped, need to send again | the script refuses a bad cache path *before* the jump for this reason, but if it happens: rerun with `--no-load` — the payload is still running and waiting, and `vidsend` brings the dictionary with it |
| poke reads back a mismatch | the image is being loaded over a stale link, or the payload was built for a different load address |
| `DICT_BYTES` stuck below 32768 | datagrams lost before the payload saw them — check the tap and the sender's interface |
| picture sheared in MAME | `VT1200_MONITOR=VR150` is selected (the old derived 1024-px layout); leave the default VR319 |
| `jump` seems to hang the service | expected: the payload never returns, so rdebug is gone until the machine is rebooted |
+10
View File
@@ -0,0 +1,10 @@
"""The Bad Apple demo: build a clip cache, deliver the payload, play it.
``harness/badapple/README.md`` is the setup and run guide. The three pieces:
* ``clip_from_video.py`` a video file -> raw 1-bpp frames in the terminal's
bit order, the input ``make_tiles.py --clip`` wants;
* ``rdebug_load.py`` the payload image into a running terminal over the
firmware's own remote-debug service (TCP 771);
* ``demo.sh`` one command: wait for the OS, load, send the frames.
"""
+141
View File
@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""Turn a video into the raw 1-bpp frames ``make_tiles.py --clip`` takes.
Each output file is one frame: whole 160-byte rows, whole 32-row tile bands,
top-aligned, **LSB first** -- bit 0 of a byte is the row's leftmost pixel, the
order the terminal's scanout reads (``dwt/screenshot.py`` decodes live VRAM with
the same rule). An MSB-first packer is self-consistent with an MSB-first
checker and mirrors every eight pixels on a display, which is why the order is
spelled out here rather than left to ffmpeg.
The screen is 1280 px wide and the tile grid is 32 rows tall, so a 4:3 source
scaled to 1280x960 fills 30 of them and the last two rows stay black. Pass
``--height 1024`` to fill the screen and accept 4:3 being stretched.
Requires ``ffmpeg`` on PATH and numpy.
clip_from_video.py bad-apple.mp4 frames/ --fps 15
clip_from_video.py bad-apple.mp4 frames/ --fps 12 --threshold 100 --invert
"""
from __future__ import annotations
import argparse
import os
import shutil
import subprocess
import sys
ROW_PIXELS = 1280
ROW_BYTES = ROW_PIXELS // 8 # 160
def pack_row(pixels) -> bytes:
"""Pack one row of 0/1 pixels, LSB first (pixel 0 -> bit 0 of byte 0)."""
if not pixels or len(pixels) % 8:
raise ValueError("a row is a positive multiple of 8 pixels, got %d"
% len(pixels))
out = bytearray(len(pixels) // 8)
for i, value in enumerate(pixels):
if value:
out[i >> 3] |= 1 << (i & 7)
return bytes(out)
def frames(video, out_dir, fps=15, width=ROW_PIXELS, height=960, threshold=127,
invert=False, block=1):
"""Yield ``(frame_no, packed_row_bytes)`` for every video frame."""
import numpy as np
vf = ("fps=%s,scale=%d:%d:flags=lanczos,format=gray"
% (fps, width, height))
if invert:
vf += ",negate"
cmd = ["ffmpeg", "-v", "error", "-i", video, "-vf", vf,
"-f", "rawvideo", "-"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, bufsize=10 ** 8)
size = width * height
n = 0
try:
while True:
buf = proc.stdout.read(size)
if len(buf) < size:
break
gray = np.frombuffer(buf, dtype=np.uint8).reshape(height, width)
if block > 1:
# Stylise: average each block, threshold the average, then draw
# the block flat. Fewer, larger shapes mean fewer changed tiles
# per frame -- measured on Bad Apple at 60 fps: 80 changed tiles
# unstylised, 54 at 8x8, 29 at 16x16, against a budget of ~44.
h, w = height // block, width // block
small = gray.reshape(h, block, w, block).mean(axis=(1, 3))
bits = np.repeat(np.repeat(small > threshold, block, axis=0),
block, axis=1)
else:
bits = gray > threshold
yield n, np.packbits(bits, axis=1, bitorder="little")
n += 1
finally:
proc.stdout.close()
proc.wait()
def main(argv=None) -> int:
ap = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("video", help="source video (anything ffmpeg reads)")
ap.add_argument("out", help="directory for the *.bin frames")
ap.add_argument("--fps", type=float, default=15.0,
help="frames per second to extract (default %(default)s)")
ap.add_argument("--width", type=int, default=ROW_PIXELS,
help="pixels per row (default %(default)s)")
ap.add_argument("--height", type=int, default=960,
help="rows to extract: 960 fills 30 of the 32 tile bands, "
"1024 fills the screen (default %(default)s)")
ap.add_argument("--threshold", type=int, default=127,
help="grey level above which a pixel is ink "
"(default %(default)s)")
ap.add_argument("--block", type=int, default=1,
help="stylise to NxN blocks (default 1 = no stylisation)")
ap.add_argument("--invert", action="store_true",
help="treat bright pixels as paper (negate first)")
args = ap.parse_args(argv)
if shutil.which("ffmpeg") is None:
print("error: ffmpeg is not on PATH", file=sys.stderr)
return 2
if args.width % 8 or args.width > ROW_PIXELS or args.width < 8:
print("error: --width must be a multiple of 8, at most %d" % ROW_PIXELS,
file=sys.stderr)
return 2
if args.block < 1 or args.block > 64:
print("error: --block must be 1..64", file=sys.stderr)
return 2
if args.height % 32 or args.height > 1024:
print("error: --height must be a multiple of 32, at most 1024",
file=sys.stderr)
return 2
os.makedirs(args.out, exist_ok=True)
for name in os.listdir(args.out):
if name.endswith(".bin"):
os.remove(os.path.join(args.out, name))
written = 0
for n, packed in frames(args.video, args.out, fps=args.fps,
width=args.width, height=args.height,
threshold=args.threshold, invert=args.invert,
block=args.block):
path = os.path.join(args.out, "%05d.bin" % n)
with open(path, "wb") as handle:
handle.write(packed.tobytes())
written += 1
if written % 100 == 0:
print(" %d frames" % written, file=sys.stderr)
print("%d frames of %dx%d at %s fps -> %s"
% (written, args.width, args.height, args.fps, args.out))
return 0
if __name__ == "__main__":
sys.exit(main())
+96
View File
@@ -0,0 +1,96 @@
#!/bin/sh
# Play the streaming demo on a RUNNING terminal: wait for its OS network stack,
# deliver the payload over rdebug, then send the clip with the host sender.
#
# The terminal itself is started separately -- `harness/mame/run.sh net` for the
# emulator (its default VR319 monitor already gives the real unit's 1280-px /
# 256-byte framebuffer layout; `VT1200_MONITOR=VR150` reproduces the old derived
# 1024-px view), or a real VT1200 on the network. This script only talks
# to it, so one instance serves one demo: the jump hands the payload the
# remote-debug process and it never returns.
#
# demo.sh # committed 30-frame cache
# demo.sh --tiles /tmp/badapple --frames 3292 --fps 15
# demo.sh --host 192.168.99.2 --iface vt1200
# demo.sh --no-load # send only, payload already running
set -eu
HERE=$(cd "$(dirname "$0")" && pwd)
ROOT=$(cd "$HERE/../.." && pwd)
HOST=192.168.99.2
IFACE=vt1200
TILES=$ROOT/harness/udpstream/tiles
FRAMES=200
FPS=15
LOAD=1
SEND=1
WAIT=180
while [ $# -gt 0 ]; do
case "$1" in
--host) HOST=$2; shift 2 ;;
--iface) IFACE=$2; shift 2 ;;
--tiles) TILES=$2; shift 2 ;;
--frames) FRAMES=$2; shift 2 ;;
--fps) FPS=$2; shift 2 ;;
--wait) WAIT=$2; shift 2 ;;
--no-load) LOAD=0; shift ;;
--no-send) SEND=0; shift ;;
-h|--help) sed -n '2,20p' "$0"; exit 0 ;;
*) echo "demo.sh: unknown option $1" >&2; exit 2 ;;
esac
done
# --- the cache, checked BEFORE anything irreversible ------------------------
# The jump hands the payload the remote-debug process and it never returns, so
# a bad --tiles path discovered afterwards leaves the machine needing a restart
# to try again. Refuse it here instead.
if [ "$SEND" = 1 ]; then
for f in dictionary.bin stream.bin; do
if [ ! -f "$TILES/$f" ]; then
echo "demo.sh: $TILES has no $f -- that is not a tile cache" >&2
if ls "$TILES"/[0-9]*.bin > /dev/null 2>&1; then
echo " it looks like a FRAMES directory (raw 1-bpp frames)." >&2
echo " build the cache from it first:" >&2
echo " python3 harness/udpstream/make_tiles.py --clip $TILES --out $TILES-tiles" >&2
fi
echo " the committed 30-frame cache is harness/udpstream/tiles" >&2
exit 2
fi
done
fi
# --- readiness: an ICMP reply from the terminal's own stack -----------------
# Only when this run is going to load the payload. A loaded payload owns the
# LANCE (and stops the OS's stack with it), so it never answers ICMP: waiting for
# a ping after the load would deadlock -- which is exactly what --no-load used to
# do. --no-load therefore skips the gate and trusts the operator.
if [ "$LOAD" = 1 ]; then
start=$(date +%s)
while ! ping -c1 -W1 "$HOST" > /dev/null 2>&1; do
if [ $(( $(date +%s) - start )) -ge "$WAIT" ]; then
echo "demo.sh: $HOST did not answer within ${WAIT}s" >&2
echo " is the terminal booted and on the tap? (harness/mame/run.sh net)" >&2
exit 1
fi
done
echo "demo.sh: $HOST answers ($(( $(date +%s) - start ))s)"
else
echo "demo.sh: --no-load: not waiting for ICMP (a loaded payload owns the LANCE)"
fi
# --- the payload, over the firmware's own remote-debug service --------------
if [ "$LOAD" = 1 ]; then
python3 "$HERE/rdebug_load.py" --host "$HOST"
# The payload takes the LANCE and the remote-debug process with it, so the
# sender is started once the machine has had time to arm its ring.
sleep 3
fi
# --- the clip ---------------------------------------------------------------
if [ "$SEND" = 1 ]; then
echo "demo.sh: sending $FRAMES frame(s) from $TILES at $FPS fps on $IFACE"
sudo -n python3 "$ROOT/harness/udpstream/vidsend.py" --iface "$IFACE" \
--tiles --tiles-dir "$TILES" --frames "$FRAMES" --fps "$FPS"
fi
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""Measure the raw-tile wire cost of a clip at the source's own frame rate.
This is the spec's first task (`docs/superpowers/specs/2026-10-31-badapple-60p-
design.md` §11): before any payload or wire change, find out what the proposed
form actually costs on the real clip.
For every frame it diffs the extracted 1-bpp picture against the previous one on
the 40x30 picture grid and totals the payload a `KIND_TILES_DELTA` would carry:
4 + 3*count + 128*m count = changed tiles, m = tiles that are neither
solid black nor solid white
It reports mean/p50/p95/max wire, how many frames break the 21 KB one-frame
budget, and where the big changes (cuts) are. No dither, hard threshold -- the
design's §8.7 constraint, because dither makes nearly every tile dirty.
measure_wire.py /tmp/opencode/badapple-real/hd60.mp4 --limit 600
measure_wire.py clip.mp4 # every frame
"""
from __future__ import annotations
import argparse
import os
import subprocess
import sys
TILE = 32
COLS = 40 # 1280 px
ROWS = 30 # 960 px: the picture occupies the top 30 of 32 tile rows
PIXELS_PER_TILE = TILE * TILE
BUDGET = 21 * 1024 # one 60 fps frame of wire, spec §6
MAX_PAYLOAD = 32 * 1450 # 32-descriptor ring x 1450-byte datagrams
def frames(video, fps, width, height, threshold):
"""Yield a boolean picture per video frame, LSB-first is irrelevant here."""
import numpy as np
cmd = ["ffmpeg", "-v", "error", "-i", video,
"-vf", "fps=%s,scale=%d:%d:flags=lanczos,format=gray"
% (fps, width, height),
"-f", "rawvideo", "-"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, bufsize=10 ** 8)
size = width * height
try:
while True:
buf = proc.stdout.read(size)
if len(buf) < size:
break
gray = np.frombuffer(buf, dtype=np.uint8).reshape(height, width)
yield gray > threshold
finally:
# Stop ffmpeg before closing its pipe, or it reports a broken pipe as
# an error when a --limit run stops reading early.
proc.kill()
proc.wait()
def measure(video, fps, width, height, threshold, limit=None):
import numpy as np
prev = None
n = 0
wires, changed, bitmaps, keys = [], [], [], []
for pic in frames(video, fps, width, height, threshold):
pic = pic[:ROWS * TILE, :COLS * TILE]
if prev is not None and prev.shape == pic.shape:
diff = pic ^ prev
dirty = diff.reshape(ROWS, TILE, COLS, TILE).any(axis=(1, 3))
count = int(dirty.sum())
if count:
tiles = pic.reshape(ROWS, TILE, COLS, TILE).transpose(0, 2, 1, 3)
tiles = tiles.reshape(ROWS, COLS, PIXELS_PER_TILE)
solid_black = ~tiles.any(axis=2)
solid_white = tiles.all(axis=2)
m = int((dirty & ~solid_black & ~solid_white).sum())
else:
m = 0
wire = 4 + 3 * count + 128 * m
wires.append(wire)
changed.append(count)
bitmaps.append(m)
keys.append((n, count, wire))
prev = pic
n += 1
if limit and n >= limit:
break
return n, wires, changed, bitmaps, keys
def main(argv=None) -> int:
ap = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("video")
ap.add_argument("--fps", type=float, default=60.0)
ap.add_argument("--width", type=int, default=1280)
ap.add_argument("--height", type=int, default=960)
ap.add_argument("--threshold", type=int, default=127)
ap.add_argument("--limit", type=int, default=None,
help="frames to read (default: all)")
args = ap.parse_args(argv)
n, wires, changed, bitmaps, keys = measure(
args.video, args.fps, args.width, args.height, args.threshold,
args.limit)
if not wires:
print("no frames measured", file=sys.stderr)
return 1
wires_sorted = sorted(wires)
def pct(p):
return wires_sorted[min(len(wires_sorted) - 1, int(len(wires_sorted) * p))]
over = sum(1 for w in wires if w > BUDGET)
too_big = sum(1 for w in wires if w > MAX_PAYLOAD)
mean = sum(wires) / len(wires)
print("frames %d at %.0f fps (delta frames measured: %d)"
% (n, args.fps, len(wires)))
print("wire/frame mean %6.0f B p50 %6.0f p95 %6.0f max %6d"
% (mean, pct(0.50), pct(0.95), wires_sorted[-1]))
print("changed tiles/frame mean %5.1f max %5d (grid %dx%d = %d)"
% (sum(changed) / len(changed), max(changed), COLS, ROWS, COLS * ROWS))
print("bitmap tiles/frame mean %5.1f max %5d (%.0f%% of changed)"
% (sum(bitmaps) / len(bitmaps), max(bitmaps),
100.0 * sum(bitmaps) / max(1, sum(changed))))
print("frames over the 21 KB budget: %d / %d (%.1f%%)"
% (over, len(wires), 100.0 * over / len(wires)))
print("frames over one payload (32 datagrams, %d B): %d"
% (MAX_PAYLOAD, too_big))
print("wire as a share of the link at this rate: %.0f%% (%.0f KB/s of 1.27 MB/s)"
% (100.0 * mean * args.fps / (1.27 * 1024 * 1024),
mean * args.fps / 1024))
big = sorted(keys, key=lambda k: -k[1])[:10]
print("largest changes (frame, changed tiles, wire):")
for frame_no, count, wire in big:
print(" %6d %5d tiles %7d B" % (frame_no, count, wire))
return 0
if __name__ == "__main__":
sys.exit(main())
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/env python3
"""Deliver a payload image into a running VT1200 over rdebug, then jump to it.
This is the real delivery path, not a debugger shortcut: the image is written
with the firmware's own remote-debug service (``peek``/``poke``/``jump``, TCP
771 -- ``dwt/notes/CROSS.md``), exactly as ``dwt/exploit.py`` does for its
one-shot commands. ``harness/mame/vidstream.lua`` remains the *test* path: it
pokes through MAME's debugger API, which is fine for checking a payload and is
not how anything reaches the hardware.
Two things the protocol forces, both measured in ``harness/badapple/README.md``:
* a **512-byte chunk**. The firmware's documented clamp is 1024, but a
1024-byte poke is mis-parsed -- the chip reads the body's first word as the
next opcode and answers "Invalid command 0xA5A5A5A5" for a 0xA5-filled
chunk. 512 is accepted and read back byte for byte.
* **no probing after the jump**. A non-returning payload takes the remote-debug
process with it, so the load is verified *before* the jump and observed
afterwards from the machine itself (MAME's RAM dumps on the emulator, the
screen on the unit).
Usage
-----
rdebug_load.py # image + entry from the build
rdebug_load.py --dry-run # what would be written where
rdebug_load.py --no-jump # load and verify only
rdebug_load.py --host 192.168.99.2 --image build/vidstream.bin
"""
from __future__ import annotations
import argparse
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.dirname(os.path.dirname(HERE))
DWT = os.path.join(ROOT, "dwt")
if DWT not in sys.path:
sys.path.insert(0, DWT)
#: Largest poke the firmware actually accepts. See the module docstring.
CHUNK = 512
#: The image's link origin: the .text address `build-stream.sh` links at. Used
#: only when the environment file is absent, so a hand-built image still loads.
DEFAULT_LOAD = 0x018C0000
#: Socket timeout. Generous on purpose: a freshly booted OS answers rdebug
#: slowly (measured: a readback peek timed out at 5 s while the machine was still
#: settling), and a slow peek must not fail a load that already succeeded.
DEFAULT_TIMEOUT = 15.0
def read_env(path) -> dict:
"""Parse ``vidstream.env`` (``KEY=VALUE`` lines) into ints where possible."""
out = {}
with open(path) as handle:
for line in handle:
line = line.strip()
if not line or "=" not in line:
continue
key, value = line.split("=", 1)
value = value.strip()
try:
out[key.strip()] = int(value, 0)
except ValueError:
out[key.strip()] = value
return out
def chunks(data: bytes, size: int = CHUNK) -> list[bytes]:
"""Split ``data`` into transfers the firmware will accept."""
if size < 1 or size > CHUNK:
raise ValueError(
"a poke chunk must be 1..%d bytes (the firmware mis-parses %d); got %r"
% (CHUNK, CHUNK * 2, size))
return [data[off:off + size] for off in range(0, len(data), size)]
def chunk_addr(base: int, index: int, size: int = CHUNK) -> int:
"""Bit address of chunk ``index``: the TMS34010 addresses bits, so a
``size``-byte transfer advances the pointer by ``size * 8``."""
return base + index * size * 8
def plan(image: bytes, load: int, size: int = CHUNK):
"""The transfers this load will perform, as ``(bit address, bytes)``."""
return [(chunk_addr(load, n, size), part)
for n, part in enumerate(chunks(image, size))]
def readback(rd, transfers, retries=1):
"""Peek every chunk back, retrying a stalled read once.
A timeout here is the machine being busy, not a bad image -- the poke has
already been acknowledged -- so it is retried before it is believed.
"""
back = b""
for attempt in range(retries + 1):
try:
return b"".join(rd.peek(addr, len(part)) for addr, part in transfers)
except (OSError, TimeoutError):
if attempt == retries:
raise
return back
def load_and_jump(image_path, entry, load, do_jump=True, chunk=CHUNK,
host="192.168.99.2", port=771, verbose=False,
timeout=DEFAULT_TIMEOUT):
"""Poke ``image_path`` at ``load``, verify it, then jump to ``entry``."""
import exploit # dwt/exploit.py: the protocol the firmware speaks
data = open(image_path, "rb").read()
transfers = plan(data, load, chunk)
rd = exploit.RDebug(host, port, timeout, verbose)
rd.connect()
try:
for addr, part in transfers:
rd.poke(addr, part)
print("poked %d bytes in %d chunk(s) at %#x"
% (len(data), len(transfers), load))
back = readback(rd, transfers)
if back != data:
bad = next(i for i in range(min(len(back), len(data)))
if back[i] != data[i])
print("readback MISMATCH at byte %d: wrote %02x, read %02x"
% (bad, data[bad], back[bad]), file=sys.stderr)
return 1
print("readback OK: the machine holds the image we sent")
if do_jump:
rd.jump(entry, [], wait=False)
print("jump sent to %#x (the payload never returns)" % entry)
finally:
rd.close()
return 0
def main(argv=None) -> int:
ap = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--host", default="192.168.99.2",
help="terminal address (default %(default)s)")
ap.add_argument("--port", type=int, default=771,
help="rdebug TCP port (default %(default)s)")
ap.add_argument("--image",
default=os.path.join(ROOT, "harness", "udpstream", "build",
"vidstream.bin"),
help="payload image (default: the streaming payload)")
ap.add_argument("--env",
default=os.path.join(ROOT, "harness", "udpstream", "build",
"vidstream.env"),
help="build environment carrying ENTRY_PTR")
ap.add_argument("--load", type=lambda s: int(s, 0), default=None,
help="load bit address (default: %#x)" % DEFAULT_LOAD)
ap.add_argument("--entry", type=lambda s: int(s, 0), default=None,
help="entry bit address (default: ENTRY_PTR from --env)")
ap.add_argument("--chunk", type=int, default=CHUNK,
help="poke size in bytes (max %d; default %%(default)s)"
% CHUNK)
ap.add_argument("--timeout", type=float, default=DEFAULT_TIMEOUT,
help="socket timeout in seconds (default %(default)s)")
ap.add_argument("--no-jump", action="store_true",
help="load and verify, do not start it")
ap.add_argument("--dry-run", action="store_true",
help="print the transfer plan and exit")
args = ap.parse_args(argv)
try:
env = read_env(args.env)
except OSError:
print("error: no build environment at %s -- build the payload first "
"(harness/udpstream/build-stream.sh)" % args.env,
file=sys.stderr)
return 2
load = args.load if args.load is not None else env.get("LOAD_PTR",
DEFAULT_LOAD)
entry = args.entry if args.entry is not None else env.get("ENTRY_PTR")
if entry is None:
print("error: %s has no ENTRY_PTR (pass --entry)" % args.env,
file=sys.stderr)
return 2
try:
image = open(args.image, "rb").read()
transfers = plan(image, load, args.chunk)
except (OSError, ValueError) as exc:
print("error: %s" % exc, file=sys.stderr)
return 2
if args.dry_run:
print("%s: %d bytes, %d chunk(s) of %d, load %#x, entry %#x"
% (args.image, len(image), len(transfers), args.chunk,
load, entry))
for addr, part in transfers[:3]:
print(" %#010x %d B" % (addr, len(part)))
if len(transfers) > 3:
print(" ... %d more" % (len(transfers) - 3))
return 0
try:
return load_and_jump(args.image, entry, load,
do_jump=not args.no_jump, chunk=args.chunk,
host=args.host, port=args.port,
timeout=args.timeout)
except Exception as exc: # noqa: BLE001 - one line, not a traceback
print("error: %s: %s" % (type(exc).__name__, exc), file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+110
View File
@@ -0,0 +1,110 @@
"""Tests for the rdebug loader and the clip builder.
The pieces that touch a live terminal cannot be tested here; what is tested is
everything that decides *what* gets written where, because those are the parts a
wrong assumption silently corrupts: the chunking arithmetic, the linkage the
entry point is read from, and the 1-bpp packing the terminal actually reads.
"""
import struct
import pytest
from harness.badapple import clip_from_video
from harness.badapple import rdebug_load
# --- chunking -------------------------------------------------------------
def test_chunks_are_transfer_sized_and_cover_the_image_once():
data = bytes(range(200)) * 23 # 4,600 B: not a multiple of 512
parts = rdebug_load.chunks(data, 512)
assert [len(p) for p in parts] == [512] * 8 + [504]
assert b"".join(parts) == data
def test_chunks_refuse_a_size_the_firmware_cannot_take():
# 1024 is rejected by the firmware: it reads the body's first word as the
# next opcode ("Invalid command 0xA5A5A5A5" on a 0xA5-filled chunk).
with pytest.raises(ValueError):
rdebug_load.chunks(b"x", 1024)
with pytest.raises(ValueError):
rdebug_load.chunks(b"x", 0)
def test_addresses_advance_by_the_chunk_length_in_bits():
base = 0x018C0000
assert rdebug_load.chunk_addr(base, 0) == base
assert rdebug_load.chunk_addr(base, 1) == base + 512 * 8
# --- linkage --------------------------------------------------------------
def test_env_file_gives_the_entry_and_the_pool_size(tmp_path):
env = tmp_path / "vidstream.env"
env.write_text("POOL_BYTES=249856\nSTATIC_PTR=0x01C00080\n"
"ENTRY_PTR=0x018D0270\n")
got = rdebug_load.read_env(env)
assert got["ENTRY_PTR"] == 0x018D0270
assert got["POOL_BYTES"] == 249856
def test_missing_env_file_says_so_instead_of_guessing(tmp_path, capsys):
rc = rdebug_load.main(["--image", "x.bin", "--env", str(tmp_path / "no.env")])
assert rc == 2
err = capsys.readouterr().err
assert "no build environment" in err and "build-stream.sh" in err
# --- readback ------------------------------------------------------------
class _Flaky:
"""A remote end whose first peek times out, as a busy OS does."""
def __init__(self, transfers, fail_times=1):
self.mem = dict(transfers)
self.fail = fail_times
self.calls = 0
def peek(self, addr, length):
self.calls += 1
if self.fail:
self.fail -= 1
raise TimeoutError("timed out")
return self.mem[addr][:length]
def test_readback_retries_a_stalled_peek_instead_of_failing_the_load():
transfers = rdebug_load.plan(b"0123456789", 0x018C0000, 4)
rd = _Flaky(transfers, fail_times=1)
assert rdebug_load.readback(rd, transfers) == b"0123456789"
# The retry re-reads the whole image: one timed-out chunk plus the three
# chunks of the successful pass.
assert rd.calls == len(transfers) + 1
def test_readback_gives_up_after_the_retry():
transfers = rdebug_load.plan(b"0123", 0x018C0000, 4)
rd = _Flaky(transfers, fail_times=9)
with pytest.raises(TimeoutError):
rdebug_load.readback(rd, transfers)
# --- the clip format ------------------------------------------------------
def test_packed_rows_put_pixel_zero_on_bit_zero_of_byte_zero():
"""The terminal reads bit 0 of a byte as the leftmost pixel.
Same rule the payload's encoder uses (`vidsrc.Rows.pack_row`), and the one
an MSB-first packer gets wrong in a way that only shows on a display.
"""
pixels = [[0] * 16]
pixels[0][0] = 1 # leftmost pixel of the row
pixels[0][15] = 1 # last pixel of the second byte
row = clip_from_video.pack_row(pixels[0])
assert len(row) == 2
assert row[0] == 0x01
assert row[1] == 0x80
def test_pack_row_rejects_a_short_row():
with pytest.raises(ValueError):
clip_from_video.pack_row([0] * 15)
+5 -3
View File
@@ -92,12 +92,14 @@ Success = loader status 0, `entry=0x018A17E0`, markers `48454C4C`/`444F4E45`
the image must not need static initializers. The font and order tables are
filled in `build_font()` at runtime.
## Geometry is not portable between the real unit and MAME
## Geometry is not portable between the real unit and MAME's VR150 mode
The two framebuffers differ, and the payload carries the difference as
`ROW_BYTES`:
`ROW_BYTES`. **The harness now defaults to the real unit's VR319 monitor**, so
MAME's default geometry is the real one; `VT1200_MONITOR=VR150` restores the old
MAME layout (and needs the `build.sh emu` image).
| | real unit (measured) | MAME |
| | real unit (measured) = MAME VR319 (default) | MAME VR150 |
|---|---|---|
| base | bit `0x04000000` | bit `0x04000000` |
| row pitch | **256 bytes** (`ROW_BYTES 256`) | 128 bytes (`VMEM_PITCH_BITS`) |
+2 -1
View File
@@ -24,7 +24,8 @@ DOSBOX_CONF="$ROOT/harness/sdk-scratch/dosbox-bench.conf"
TEXT=018A0000
BSS=01A00000
# `build.sh emu` builds a 128-byte-stride variant for MAME's 1024x1024 canvas.
# `build.sh emu` builds a 128-byte-stride variant for the harness's VR150 mode
# (VT1200_MONITOR=VR150); the default VR319 layout matches the real unit.
MODE=${1:-real}
NAME=hello
if [ "$MODE" = "emu" ]; then NAME=hello-emu; fi
+2 -1
View File
@@ -4,7 +4,8 @@
* Writes the word HELLO straight into video RAM, no X11 involved:
*
* framebuffer bit 0x04000000, 256-byte row pitch on the real unit
* (the emulator uses 128; this image is linked for the real one)
* (the harness now defaults to the same VR319 layout;
* only VT1200_MONITOR=VR150 uses 128)
* pixels 1 bpp, LSB-first within each byte
* text area black 256x72 box at x=40 y=800, white 5x7 glyphs scaled 8x
*
+6
View File
@@ -16,6 +16,12 @@ short version:
* both are observed from inside the emulator (RAM readback) and from a frame
dump (white block at screen x 256..511, y 136..199).
> **Pitch note.** `hello.asm` writes 1024-bit rows (the old derived layout).
> The harness now defaults to the real unit's **VR319** 2048-bit layout, so add
> `VT1200_MONITOR=VR150` to the run below to see the block where this README
> says it is; at the VR319 default the rendered block is sheared because the
> payload, not the firmware, chose the old pitch.
## Build
```sh
+8 -4
View File
@@ -5369,7 +5369,8 @@ add about a quarter: 4 (FCS) + 8 (preamble) + 12 (IFG) = 24 B per datagram, and
The clip format is pinned by `make_tiles.load_clip`: a directory of `*.bin`
files, each one raw 1-bpp frame — whole 160-byte rows, whole 32-row tile bands,
top-aligned, MSB first, 1 = ink, at most 1,024 rows. A partial row or band is
top-aligned, **LSB first** (bit 0 of a byte is the row's leftmost pixel, the
scanout order), 1 = ink, at most 1,024 rows. A partial row or band is
rejected with the file's name, not silently reshaped. With ffmpeg (not
installed on this machine, so this route is documented and **not executed**):
@@ -5384,9 +5385,12 @@ python3 harness/udpstream/make_tiles.py --clip /tmp/badapple \
--out harness/udpstream/tiles-badapple
```
`monob` is MSB-first 1-bpp; if the picture comes out negative, invert it in the
filter chain (`...,format=monob,negate`). To keep a run's length independent of
the source, `vidsend.py --tiles` loops a short cache.
`monob` is MSB-first 1-bpp, which is **the wrong bit order for this clip**: the
clip is LSB-first, so each byte of `monob` output must be bit-reversed or every
eight pixels come out mirrored (the real-clip run that produced the tile cache
used its own packer for exactly this reason). Invert in the filter chain
(`...,format=monob,negate`) only if the picture comes out negative. To keep a
run's length independent of the source, `vidsend.py --tiles` loops a short cache.
The clip loader is not an untested path in the code sense: the 30-frame
synthetic clip written out as `frame-0000.bin … frame-0029.bin` and fed back
+59 -24
View File
@@ -109,10 +109,13 @@ to be a six-variable incantation; it is now the default, and a bare
* Video RAM at bit `0x04000000` — required for the level-1 VIDEO MEMORY test.
* The board-window handshake model (two-tick busy bit).
* MAME's `SCN2681` DUART at **DUART1** (`0x0640000`, host/printer), with input
port **IP4–IP6 held high** — that is how the driver reads the VR150 monitor
switch (`0x70`). The level-1 host-port transmit is **ISR-driven**, so a real
DUART is required here, not a convenience.
* MAME's `SCN2681` DUART at **DUART1** (`0x0640000`, host/printer), whose input
port **IP4–IP6** carries the monitor-selection switch. The default is
**VR319 (`0x60`) — the real unit's mode** (see `dwt/notes/REAL_UNIT.md`), so
the firmware itself programs a **1280×1024 display at a 256-byte (2048-bit)
row pitch** and `CONVDP = 0x14`. `VT1200_MONITOR=VR150|VR262|VR315` selects
another position for comparison. The level-1 host-port transmit is
**ISR-driven**, so a real DUART is required here, not a convenience.
* MAME's `SCN2681` at **DUART2** (`0x0620000`, keyboard/mouse). This one is a
*faithfulness* fix, not a convenience — the real board has one. Without it the
board write-back latch answers a read of DUART2's **write-only** IMR with its
@@ -126,7 +129,11 @@ to be a six-variable incantation; it is now the default, and a bare
`FD`/`T` because that is the protocol, not to satisfy a test. See
`INPUT_REPORT.md` and `TABLET_REPORT.md`.
* A mono palette (pens 0/1 = black/white) and an active-pixel scanline renderer
that reads VRAM. See `RENDER_REPORT.md`.
that reads VRAM at the pitch the modelled monitor implies: 2048-bit rows /
1280 px for VR319 (the default), 1024-bit rows / 1024 px for VR150. The
destination span always comes from the display controller's own
`HEBLNK..HSBLNK`, so the active picture sits at `x = 336` in MAME's
1696-wide VR319 raster, not at `x = 0`. See `RENDER_REPORT.md`.
* The **LANCE Ethernet controller** — MAME's `am79c90` (AM79C90) at the
firmware's two register windows (**bit `0x5900000` = RAP, `0x5800000` = RDP**),
with its bus-master DMA bound (LANCE byte `B` → main-RAM word `B >> 1`) and its
@@ -204,8 +211,10 @@ over to the terminal's own mouse cursor, and the V2.2 box appears — title bar,
Messages, Ethernet/IP/LAT, a **Memory bar**, and "All Diagnostics Passed".
Confirmed in the GUI and from frame dumps: `skip_frame3000.png` (the weave) and
`duart2_default_tm_frame3000.png` (the Terminal Manager). The real terminal shows
the same sequence — see `reference/vt1200-boot-video/`.
`duart2_default_tm_frame3000.png` (the Terminal Manager). Those two files are
**VR150** captures; at the VR319 default the same Terminal Manager renders at the
real 1280-px placement (`VT1200_DUMP_FRAMES=600`, see `RENDER_REPORT.md` §8). The
real terminal shows the same sequence — see `reference/vt1200-boot-video/`.
The OS's keyboard and mouse drivers **initialise and speak the real protocol**: it
programs DUART2 (`MR1A=0x13`, `MR2A=0x07`, `CSRA/CSRB=0x99`, `ACR=0x60`,
@@ -269,12 +278,13 @@ Terminal Manager usable. It is a **value-aware inverse switch**: the default
lives in the code, not in a wrapper script setting `VT1200_TABLET`, so
`VT1200_MOUSE=1` (or `VT1200_MOUSE=0` for the tablet) always flips it either way.
## Environment knobs (21; down from 33)
## Environment knobs (23; down from 33)
Almost all are opt-in and off by default — there is no longer a "working
incantation" to remember, because everything the firmware needs to run is now the
default. The one exception is that the **tablet is on by default**; `VT1200_MOUSE`
is the switch that restores the faithful mouse (see the divergences above).
default. The two exceptions are the **tablet** (on by default; `VT1200_MOUSE`
restores the faithful mouse) and the **VR319 monitor** (on by default because
that is the real unit; `VT1200_MONITOR` selects VR150 and friends).
Instrumentation: `VT1200_DUARTLOG`, `VT1200_LANCELOG` (every LANCE RAP/RDP and
address-ROM access with the PC), `VT1200_LANCE_DMA_LOG` (every LANCE bus-master
@@ -283,6 +293,13 @@ the descriptor write-back are read directly), `VT1200_BOARD_LOG`, `VT1200_BOARD_
`VT1200_IRQLOG`, `VT1200_LOG_UNMAPPED`, `VT1200_DPYLOG` (+`_FRAMES`),
`VT1200_DUMP_FRAME` (+`_FRAMES`), `VT1200_DUMP_RAM` (+`_FRAMES`),
`VT1200_DUMP_VRAM` (+`_FRAMES`), `VT1200_DUMP_DIR`, `VT1200_NO_RENDER`,
`VT1200_MONITOR` (monitor-selection switch: `VR319` default / `VR150` / `VR262`
/ `VR315`, or a raw position 0–7; it also picks the renderer's row pitch.
Positions other than 7/5/3/6 are the firmware's "Undefined monitor switch
setting" branch),
`VT1200_VRAM_REAL` (force the 2048-bit / 256-byte / 1280-px renderer pitch
regardless of `VT1200_MONITOR`; only meaningful when the firmware itself is
writing that layout, which the VR319 default does),
`VT1200_INPUT_LOG` (per-byte keyboard/mouse/tablet TX/RX trace, with emulated
time). Behaviour switches: `VT1200_SKIP_LEVEL1`, `VT1200_MOUSE` (put the
VSXXX-AA mouse (relative) back on DUART2-B; the VSXXX-AB tablet is the default —
@@ -295,10 +312,10 @@ described above — not a keyboard model, off by default).
`progress.lua` at 200 frames, **no env**:
```
PC=3FDCE400 samples=198 (3 distinct PCs)
PC=3FDCE400 samples=199 (2 distinct PCs)
```
This has moved **twice**, both deliberately:
This has moved **three** times, all deliberately:
1. `samples=199` (1 PC) → `samples=53` (78 PCs): the working configuration became
the default, so a bare run stopped wedging at the hardware-ready gate and began
@@ -308,6 +325,15 @@ This has moved **twice**, both deliberately:
by the write-back latch's phantom `RXRDY`, so the run parks in the diagnostics
menu. That looks like a regression and is not one — it is the removal of a
false pass.
3. `samples=198` (3 PCs) → `samples=199` (2 PCs): the monitor switch became
**VR319**, the real unit's mode, instead of the old VR150. The run still parks
in the same level-0 diagnostics menu (verified with frame dumps at 200 and
600); only the number of frames that land on each of the menu's two
instructions changed, because the VR319 raster (1696×1063 total, 1280×1024
active) runs at a different emulated frame rate than VR150 (1280×901,
1024×864 active). `VT1200_MONITOR=VR150` reproduces the old
`samples=198 (3 PCs)` line exactly, which is the control that shows the move
is geometry, not a behaviour change.
**The tablet-default change did NOT move this baseline** (198 → 198), and that is
expected, not an oversight. The tablet changes what DUART2-B carries on the wire
@@ -317,17 +343,22 @@ it then frames 5-byte absolute reports instead of 3-byte relative ones — but a
has never addressed DUART2-B at all: the `T`/`R` exchange happens only in the OS
phase, which the no-env run does not reach. The only DUART2-B traffic in the
window is the device's unsolicited power-up self-test, which nothing clocks in.
The OS-phase render is unchanged too: with `VT1200_SKIP_LEVEL1=1` the frame-3000
The OS-phase render is unchanged too: with `VT1200_SKIP_LEVEL1=1` the frame-600
dump is **pixel-identical** for the tablet and for `VT1200_MOUSE=1`
(`magick compare -metric AE` = 0), and both reproduce the committed
`tablet_cursor_*.png` frames exactly.
(`magick compare -metric AE` = 0, same md5). That was re-measured at the VR319
default; the committed `tablet_cursor_*.png` frames are VR150 evidence and are
reproduced with `VT1200_MONITOR=VR150`.
**The no-env value is unchanged by `VT1200_IO_PATH_STUB`** — stub off is the
default, so the `samples=198 (3 PCs)` line above still holds. The stub is a separate
observed configuration:
default, so the `samples=199 (2 PCs)` line above still holds. The stub is a separate
observed configuration. Its value depends on the monitor, because frame count is
not emulated time; both are recorded here:
```
# VT1200_IO_PATH_STUB=1, progress.lua at 200 frames
# VT1200_IO_PATH_STUB=1, progress.lua at 200 frames, VR319 (default)
PC=3FDCE400 samples=97 (69 distinct PCs)
# VT1200_IO_PATH_STUB=1 VT1200_MONITOR=VR150, progress.lua at 200 frames
PC=3FDCE400 samples=53 (75 distinct PCs)
```
@@ -336,10 +367,11 @@ satisfied, the run leaves the level-0 wait and executes level-0/level-1 work. It
is **not** parked at `HOST PORT - 15`: the console transcript continues
(`LEVEL1/2 TESTING **`, `LEVEL1/2 ERROR -> HOST PORT - 15`,
`LEVEL1/2 ERROR -> PRINTER PORT - 16`, `TYPE 'GO' TO CONTINUE`), and the run ends
in a DUART1 counter-ready **interrupt storm** (from frame ~2839; PC in the DUART1
INT1 ISR, polling ISR reg 5 = `0x08`). See `LEVEL1_CONTINUE_REPORT.md`. It is
recorded as a *different* baseline because it is opt-in; the default must stay
`samples=198`.
in a DUART1 counter-ready **interrupt storm** (at VR150, from frame ~2839; PC in
the DUART1 INT1 ISR, polling ISR reg 5 = `0x08`). See
`LEVEL1_CONTINUE_REPORT.md`. It is recorded as a *different* baseline because it
is opt-in; the default must stay `samples=199` (`samples=53` with
`VT1200_MONITOR=VR150`).
It is a change detector, not a specification. If it moves without you meaning it
to, find out why.
@@ -443,6 +475,7 @@ would reach a real terminal.
```sh
M=/home/arti/code/k-space/mame0289/vt1200
VT1200_MONITOR=VR150 \
HELLO_END=150 VT1200_DUMP_FRAME=145 VT1200_DUMP_RAM=145 \
VT1200_DUMP_DIR=harness/hello-world/evidence \
$M vt1200 -rompath harness/mame/roms -video none -sound none -nothrottle \
@@ -454,5 +487,7 @@ The payload writes `0xDEAD`, `0xBEEF` and a 1024-iteration counter at bit
`0x01810000`, and fills a 256x64 block in video RAM at row 100, column 64.
The script reads every value back and prints `ok`/`WRONG`; a frame dump shows
the block at screen x 256..511, y 136..199. `HELLO_AT=9999` is the control run
(nothing loads, markers stay zero). See `HELLO_WORLD_REPORT.md` and
`harness/hello-world/README.md`.
(nothing loads, markers stay zero). `VT1200_MONITOR=VR150` is required here:
the payload writes 1024-bit rows and the frame numbers are the old VR150
timeline; the VR319 default would shear the block and boot at a different frame
cadence. See `HELLO_WORLD_REPORT.md` and `harness/hello-world/README.md`.
+66
View File
@@ -4,6 +4,12 @@
self-test **CP437 character table is visible and legible**, and the level-1
error screen it writes later is legible too.
> **Note (VR319 update).** Sections 1–7 record the first, VR150-derived
> geometry. The real unit is a **VR319** and the harness now models that by
> default, so the firmware writes 2048-bit rows / 1280 px and the renderer
> follows it. Section 8 is the corrected model; the only thing sections 1–7
> still need is `VT1200_MONITOR=VR150` if you want to reproduce them.
---
## 1. The defect
@@ -157,3 +163,63 @@ absent, so the code just blanks the line (as it did before).
* The video-RAM dump for a single frame is ~58 MB because the share covers
the whole `0x04000000..0x1fffffff` mapping; the framebuffer itself is the
first ~128 KiB.
## 8. The real layout: VR319 is the default
Sections 1–7 describe the **VR150** geometry: 1024-bit (128-byte) rows, 1024
active pixels, drawn at x 192..1215. That was the harness's wrong default. The
real unit reports **`Monitor Type VR319`** (`dwt/notes/REAL_UNIT.md`), and the
firmware chooses its display format from that switch. The old
`VT1200_VRAM_REAL=1` knob then re-read the VR150 framebuffer at the VR319 pitch,
which is why it sheared the OS: the firmware had written 128-byte rows, not
256-byte ones.
The measurements that settle it (probe of the TMS34010 display registers, same
run, after boot):
| monitor (switch) | HEBLNK | HSBLNK | active px | CONVDP/CONVSP | pitch |
|---|---|---|---|---|---|
| VR150 (`0x70`) | `0x0C` → 192 | `0x4C` → 1216 | 1024 | `0x15` | 1024 bits / 128 B |
| VR319 (`0x60`, real unit) | `0x15` → 336 | `0x65` → 1616 | 1280 | `0x14` | 2048 bits / 256 B |
`CONVDP` is the chip's own destination-pitch register: the firmware sets it to
`0x15` in VR150 mode and `0x14` in VR319 mode, so the pitch is not a renderer
choice at all — it is what the software tells the blitter and the scanout to
use. `VEBLNK` is `0x24` (36) in both modes; the active rows are 1024.
The fix is therefore in the **machine model**, not in the renderer's pitch:
* The monitor switch now defaults to **VR319** (`0x60`), the real unit's mode;
`VT1200_MONITOR=VR150|VR262|VR315` restores the old geometry for comparison.
* The renderer's source pitch follows the monitor (2048 bits for VR319, 1024 for
the others).
* The destination span is always the display controller's
`HEBLNK..HSBLNK`, and the source x runs `0..(HSBLNK-HEBLNK)`. In VR319 this is
screen x 336..1615. Drawing from x=0 (the old `VT1200_VRAM_REAL` branch) put
the framebuffer under the device's left blanking, so the device erased the
leftmost 336 px and the picture looked shifted/cut.
`VT1200_VRAM_REAL=1` is kept as a force-the-2048-pitch knob for the pre-VR319
workflows, but it is no longer needed for the tile stream: with the default
VR319 the OS and `vram_tile.asm` both write 2048-bit rows, and the tile checker
passes with no environment (`VT1200_MODE=tiles harness/mame/run-vidstream.sh`,
`tile_check: PASS`).
Evidence at the new default:
* `VT1200_SKIP_LEVEL1=1`, frame 600: the Terminal Manager renders cleanly at
1696×1063 with the 1280-px active area at x=336. Regenerate with
`VT1200_DUMP_FRAMES=600 VT1200_DUMP_DIR=...`.
* No env, frame 600: the level-0 diagnostics menu (`VERSION 2.08 - SELECTED TEST
TYPE:`) is legible and correctly placed in the active area.
* `VT1200_IO_PATH_STUB=1`: frames 1000 and 1900 show
`LEVEL1/2 ERROR -> HOST PORT - 15` and `LEVEL1/2 ERROR -> PRINTER PORT - 16`
(plus "CONTINUING THE TESTS") legibly — the same screens the report used
before, now at 2048-bit pitch.
* `VT1200_MODE=tiles`: the 1280×1024 tile test pattern fills the active area,
and `tile_check.py` reports 1280/1280 tiles matching.
Both modes still read the bits LSB-first, which is the TMS34010 pixel order and
the order the tile pipeline packs (`vidsrc.Rows.pack_row`); an MSB-first packer
is self-consistent with an MSB-first checker and mirrors every eight pixels on a
display.
+47 -13
View File
@@ -254,11 +254,11 @@ local DICT_BAD = MARK_PAGE + 0x58 * 8
local TILES_DROP = MARK_PAGE + 0x5c * 8
-- Must match vidstream.c.
local SLOT_BYTES = 24576
local SLOT_BYTES = 49152
-- The RX ring (Task 7), from vidstream.c: RX_DESC descriptors of 8 bytes each,
-- one RXBUFLEN-byte buffer per descriptor inside the armed slot.
local RX_DESC = 16
local RX_DESC = 32
local RXBUFLEN = 1526
local ETH_HDR = 14
local ETH_FCS = 4
@@ -268,8 +268,9 @@ local RMD1_STP = 0x0200
local RMD1_CRC = 0x0800
local RMD1_ERR = 0x4000
-- The frame every mode delivers first: 8 rows of 1280 pixels, 1 bpp MSB-first
-- (160 bytes a row), each row repeated 8 times. 1280 B of payload -- the same
-- The frame every mode delivers first: 8 rows of 1280 pixels, 1 bpp LSB-first
-- (160 bytes a row, bit 0 of a byte is the leftmost pixel), each row repeated
-- 8 times. 1280 B of payload -- the same
-- geometry vidsrc.py's ROW_BYTES produces, so Task 5 can blit it unchanged.
-- 1296 bytes is ONE datagram, which is what mode rx wants.
local RX_ROWS = 8
@@ -1044,6 +1045,32 @@ local function ring_armed_for(base, what)
what, want, RXBUFLEN, badown, badaddr, badlen, badstride))
end
-- How many descriptors point into slot A, slot B, and neither?
-- The payload now releases descriptors one at a time as their data is consumed
-- (rx_release in vidstream.c), so after a frame the consumed ones have moved to
-- the other slot while a descriptor the consumer has not reached yet stays
-- where it was. Asserting "all 16 in slot B" was the old all-at-once flip, and
-- that flip is what silently discarded frames the chip had already delivered.
local function ring_slot_counts(a_base, b_base)
local want_a, want_b = a_base // 8, b_base // 8
local i, n_a, n_b, bad = 0, 0, 0, 0
local in_a, in_b, in_bad = {}, {}, {}
for i = 0, RX_DESC - 1 do
local addr = buf_addr(ring_ringb, i)
if addr == want_a + i * RXBUFLEN then
n_a = n_a + 1; in_a[#in_a + 1] = i
elseif addr == want_b + i * RXBUFLEN then
n_b = n_b + 1; in_b[#in_b + 1] = i
else
bad = bad + 1; in_bad = in_bad or {}
in_bad[#in_bad + 1] = string.format("%d:%06X", i, addr)
end
end
return n_a, n_b, bad, table.concat(in_a, ","), table.concat(in_b, ","),
table.concat(in_bad, ",")
end
-- The 32 bytes at the start of screen row 0. Enough to see a blit: the frame
-- ring mode delivers paints row 0 with 0xF0 bytes, so any paint changes them.
local function read_vram_head(n)
@@ -1185,9 +1212,12 @@ local function ring_step()
ring_check(prog:read_u32(RX_RING_ACC) == 0,
"the frame left no bytes in progress")
ring_check(ring_dgrams >= 15, string.format("the consumer took all 15 datagrams (%d)", ring_dgrams))
ring_check(prog:read_u32(RX_RING_ARMED) == 1,
"the completion moved the write pointer to slot B")
ring_armed_for(b, "slot B after the frame")
local n_a, n_b, bad, ia, ib, ibad = ring_slot_counts(a, b)
ring_check(n_b == 15 and n_a == RX_DESC - 15 and bad == 0,
string.format("the 15 consumed descriptors moved to slot B and the %d untouched ones kept slot A (A=%d[%s] B=%d[%s] bad=%d[%s])", RX_DESC - 15,
n_a, ia, n_b, ib, bad, ibad))
ring_check(prog:read_u32(RX_RING_ARMED) == 0,
"the next descriptor is still the one in slot A")
ring_vram0 = read_vram_head(32)
-- Now the impossible frame, into the slot the ring is armed on (B).
local big, blen = oversize_frame()
@@ -1213,9 +1243,12 @@ local function ring_step()
local after = read_vram_head(32)
ring_check(ring_vram0 ~= nil and after == ring_vram0,
"the screen did not change: the oversize frame was not blitted")
local r_a, r_b, r_bad = ring_slot_counts(a, b)
ring_check(r_b == 16 and r_a == RX_DESC - 16 and r_bad == 0,
string.format("the rejected descriptor moved to slot B with the consumed ones (A=%d B=%d bad=%d)",
r_a, r_b, r_bad))
ring_check(prog:read_u32(RX_RING_ARMED) == 0,
"the rejection re-armed the ring on slot A")
ring_armed_for(a, "slot A after the rejection")
"the next descriptor is still the untouched one in slot A")
ring_check(prog:read_u32(RX_RING_DROPS) == 0,
"no frame was dropped for want of descriptors")
-- Phase 3: one datagram the chip flagged bad (FCS). The chip gives
@@ -1253,9 +1286,10 @@ local function ring_step()
"the FCS error abandoned its frame: nothing is left in progress")
ring_check(prog:read_u32(RX_OK) == 1,
"RX_OK did not move: the errored datagram painted nothing")
ring_check(prog:read_u32(RX_RING_ARMED) == 1,
"the FCS error re-armed the ring on slot B")
ring_armed_for(b, "slot B after the FCS error")
local f_a, f_b, f_bad = ring_slot_counts(a, b)
ring_check(f_b == 17 and f_a == RX_DESC - 17 and f_bad == 0,
string.format("the errored datagram's descriptor was released too (A=%d B=%d bad=%d)",
f_a, f_b, f_bad))
local after = read_vram_head(32)
ring_check(ring_vram0 ~= nil and after == ring_vram0,
"the screen did not change: the errored datagram was not blitted")
@@ -1818,7 +1852,7 @@ local function tile_byte(entry, row, col)
end
-- The 32 KB dictionary as a string: 256 entries of 32 rows of four bytes,
-- row-major, MSB-first within each byte -- exactly tilecost.py's layout.
-- row-major, LSB-first within each byte -- exactly tilecost.py's layout.
local function tile_dictionary()
local out = {}
for e = 0, 255 do
+3 -3
View File
@@ -18,9 +18,9 @@ than a silently missing check.
This script is the independent reading of the geometry, not a copy of the code
under test. The frame is 1280 pixels wide, 1 bpp, 160 bytes a row; the screen
row pitch is 2048 bits = 256 bytes (the real unit's, measured in
harness/c-hello/README.md -- MAME's own renderer uses a different pitch and is
NOT what this test checks). A source row r stretched by S must appear, byte for
byte, on screen rows r*S .. r*S+S-1.
harness/c-hello/README.md -- and the harness default now that it models VR319;
`VT1200_MONITOR=VR150` would use the old 128-byte renderer). A source row r
stretched by S must appear, byte for byte, on screen rows r*S .. r*S+S-1.
The comparison uses the bytes that were actually delivered to the slot, not a
re-implementation of the pattern: whatever the sender wrote is what must land on
+101 -28
View File
@@ -24,9 +24,12 @@
* The four undecoded board windows, driving the handshake register model
with a two-tick busy bit (required to leave early POST).
* MAME's SCN2681 at DUART1 (0x0640000 = host/printer), with its input
port IP4-IP6 held high -- which is how the driver reads the VR150
monitor switch (value 0x70). The level-1 host-port transmit is
ISR-driven, so a real DUART is required.
port IP4-IP6 driven from the modelled monitor-selection switch. The
default is VR319 (0x60) -- the real unit's mode (dwt/notes/REAL_UNIT.md)
-- which makes the firmware program the 1280x1024 display with a
256-byte (2048-bit) row pitch. VT1200_MONITOR selects VR150 (0x70),
VR262 (0x50) or VR315 (0x30) instead. The level-1 host-port transmit
is ISR-driven, so a real DUART is required.
* The 1.75 MiB firmware ROM (physical byte base 0x07e40000 = bit
0x3f200000), mirrored into the three upper 1 GiB quadrants so that
the reset-vector page at logical bit 0xffffffe0 resolves to the last
@@ -121,6 +124,16 @@
VT1200_DUMP_VRAM_FRAMES.
VT1200_DUMP_DIR=<dir> directory for the dumps (default ".")
VT1200_NO_RENDER=1 blank the screen instead of painting it
VT1200_MONITOR=<name|0-7> monitor-selection switch: VR319 (default,
0x60), VR150 (0x70), VR262 (0x50) or VR315
(0x30). Also sets the renderer's row pitch:
VR319 is 2048 bits / 1280 px, the others
use the derived 1024-bit / 1024 px.
VT1200_VRAM_REAL=1 force the 2048-bit / 1280 px renderer pitch
regardless of VT1200_MONITOR. Only
meaningful when the emulated firmware is
itself writing that layout (VR319 does).
Kept for the pre-VR319 workflows.
Behaviour switch (deliberately diverges from the real terminal):
@@ -630,7 +643,6 @@ static const u32 BOARD_BASE[4] = { VT1200_WIN0, VT1200_WIN1, VT1200_WIN2, VT1200
// sheared and unreadable. See RENDER_REPORT.md.
static const u32 VRAM_BASE_BITS = 0x04000000; // where the framebuffer lives
static const int VMEM_PITCH_BITS = 1024; // 0x40 words = 128 bytes
static const int VMEM_WIDTH = 1024; // visible pixels per row
static const int VMEM_HEIGHT = 1024; // rows in the framebuffer
class vt1200_state : public driver_device
@@ -713,6 +725,13 @@ private:
required_device<scn2681_device> m_duart2; // 0x0620000: keyboard(A)/mouse(B)
bool m_duart1_int2 = false; // set by VT1200_SKIP_LEVEL1
// Monitor-selection switch position 0-7, read from DUART1 IP4-IP6. The
// real unit reports VR319 (dwt/notes/REAL_UNIT.md), which the firmware's
// table tests as 0x60 (position 6); 0x70=VR150, 0x50=VR262, 0x30=VR315,
// anything else is "Undefined monitor switch setting". VT1200_MONITOR
// selects another position for comparison.
int m_monitor_pos = 6; // 0x60 = VR319
// The real devices on DUART2. Keyboard on channel A, pointing device on
// channel B, as the board has them. Wired bit-by-bit to the SCN2681's
// serial lines. Channel B carries the VSXXX-AB tablet BY DEFAULT, or the
@@ -802,6 +821,15 @@ private:
// times -- the instrument for telling "progressing" from "stuck": if two
// samples minutes apart are byte-identical, nothing is being drawn.
bool m_render = true;
// m_render_real: render the framebuffer at the REAL unit's geometry
// (2048-bit/256-byte row pitch) instead of the derived 1024-bit/128-byte
// pitch. It is set automatically from the modelled monitor switch: VR319
// (the real unit) is 1280 px/2048 bits, VR150 and the other modes use the
// derived pitch. VT1200_VRAM_REAL=1 forces the real geometry regardless
// (the pre-VR319 behaviour of this knob). The bit order is LSB-first in
// both modes, which is the TMS34010 pixel order and matches
// dwt/screenshot.py's decode of live VRAM.
bool m_render_real = false;
long m_dump_frame = -1;
std::vector<long> m_dump_frames; // VT1200_DUMP_FRAMES=3000,10000,...
std::string m_dump_dir = ".";
@@ -876,6 +904,20 @@ static std::vector<long> parse_frame_list(const char *s)
void vt1200_state::machine_start()
{
// Monitor-selection switch (DUART1 IP4-IP6, value bits 4-6). The switch is
// read only during power-up and selects the display format:
// 7 (0x70) VR150, 5 (0x50) VR262, 3 (0x30) VR315, 6 (0x60) VR319.
// The real unit reports VR319 (dwt/notes/REAL_UNIT.md), which is the
// default; VT1200_MONITOR accepts a name or a raw 0-7 position.
if (const char *m = getenv("VT1200_MONITOR"))
{
if (!strcmp(m, "VR150")) m_monitor_pos = 7;
else if (!strcmp(m, "VR262")) m_monitor_pos = 5;
else if (!strcmp(m, "VR315")) m_monitor_pos = 3;
else if (!strcmp(m, "VR319")) m_monitor_pos = 6;
else m_monitor_pos = int(strtol(m, nullptr, 0)) & 7;
}
if (const char *list = getenv("VT1200_BOARD_MAP"))
{
std::string s(list), item;
@@ -912,6 +954,12 @@ void vt1200_state::machine_start()
// Video: render by default; optionally dump a frame as PPM.
if (getenv("VT1200_NO_RENDER"))
m_render = false;
// The renderer follows the firmware's own screen format. VR319 (the real
// unit, 0x60) is 1280 px at a 256-byte/2048-bit row pitch; the other monitor
// modes use the derived 128-byte pitch this renderer was built against.
m_render_real = (m_monitor_pos == 6);
if (getenv("VT1200_VRAM_REAL"))
m_render_real = true;
if (const char *f = getenv("VT1200_DUMP_FRAME"))
m_dump_frame = strtol(f, nullptr, 0);
if (const char *f = getenv("VT1200_DUMP_RAM"))
@@ -977,12 +1025,12 @@ void vt1200_state::machine_start()
// With the real DUART in place, board-level signals that used to be faked
// by the window model come from the DUART's input port (read register 0x0D:
// IP0-IP6, bit 7 always set). The monitor-selection switch is IP bits 4-6;
// driving them high selects the VR150 CRT (0x70), matching the value the
// firmware looks for. IP2 stays low (the value for the blit-status test at
// bit 0x3F204AA0).
m_duart1->ip4_w(1);
m_duart1->ip5_w(1);
m_duart1->ip6_w(1);
// the real unit reports VR319, i.e. position 6 (0x60), and the firmware
// selects its 1280x1024 / 256-byte-stride display format from that. IP2
// stays low (the value for the blit-status test at bit 0x3F204AA0).
m_duart1->ip4_w(m_monitor_pos & 1);
m_duart1->ip5_w((m_monitor_pos >> 1) & 1);
m_duart1->ip6_w((m_monitor_pos >> 2) & 1);
// VT1200_SKIP_LEVEL1=1 drives DUART1 IP3 high, which makes the boot gate at
// bit 0x3F2042D0 take the OS cold-start branch (0x3F5788C0) and skip the
@@ -1003,7 +1051,8 @@ void vt1200_state::machine_start()
if (getenv("VT1200_IO_PATH_STUB"))
m_iopath_stub = true;
logerror("vt1200: board model=handshake busy_ticks=2 switch=duart-ip4/5/6 log=%s render=%d dump=%ld\n",
logerror("vt1200: board model=handshake busy_ticks=2 switch=duart-ip4/5/6 (monitor pos %d = 0x%02X, render pitch %d) log=%s render=%d dump=%ld\n",
m_monitor_pos, m_monitor_pos << 4, m_render_real ? 2048 : 1024,
m_log ? "on" : "off", m_render ? 1 : 0, m_dump_frame);
logerror("vt1200: duart1=on (SCN2681 @0640000 host/printer), duart2=on (SCN2681 @0620000 kbd/mouse), skip_level1=%s\n",
getenv("VT1200_SKIP_LEVEL1") ? "on (IP3 high, IRQ->INT2)" : "off (IP3 low, IRQ->INT1)");
@@ -1462,21 +1511,27 @@ void vt1200_state::log_lance_dma(char kind, u32 addr, u16 value, u16 mask)
// Copy one active scanline out of the video RAM framebuffer.
//
// The TMS34010 screen-refresh address is reconstructed by the device into
// `params`, but the firmware's timing registers (HEBLNK/HSBLNK/VTOTAL) are not
// initialised to the real machine's values, so the geometry MAME derives from
// them (a 1280x901 screen, active x 192..1215) does not describe the memory
// layout. The measured memory layout does:
// `params`. Two memory layouts are in use, selected by the monitor switch:
//
// * base bit 0x04000000 (the mapped video RAM)
// * pitch 1024 bits (0x40 words / 128 bytes) per row — taken from the
// firmware's own blitter, which advances its destination by
// 0x40 words per glyph row (pc=3FDF9A00 in VRAM write logs)
// * bit order LSB-first within each 16-bit word — the TMS34010 pixel order
// * active x the device's heblnk..hsblnk span is 1024 pixels, exactly the
// framebuffer width, so screen x-heblnk maps to framebuffer x
// * VR319 (the real unit, 0x60) pitch 2048 bits (0x800 / 256 bytes),
// 1280 active px. The firmware programs
// HEBLNK=0x15/HSBLNK=0x65 (16-px units:
// 336..1616 in MAME's 1696-wide raster),
// CONVSP/CONVDP=0x14 (= 2048 bits) and
// VEBLNK=0x24 (36 -> 1024 rows).
// * VR150 (0x70, the pre-VR319 default) pitch 1024 bits (0x40 / 128 bytes),
// 1024 active px (HEBLNK=12/HSBLNK=76).
//
// The screen line maps 1:1 to a framebuffer row, offset by the vertical
// In both modes the destination span is the display controller's own
// heblnk..hsblnk, and the source pitch is the only thing that differs. The
// bit order is LSB-first within each 16-bit word (the TMS34010 pixel order),
// and the screen line maps 1:1 to a framebuffer row, offset by the vertical
// blanking start (params->veblnk).
//
// VT1200_MONITOR chooses the monitor (and therefore the pitch); VR319 is the
// default because that is the real unit's mode (dwt/notes/REAL_UNIT.md).
// VT1200_VRAM_REAL=1 still forces the 2048-bit layout regardless of monitor,
// for compatibility with the old knob.
void vt1200_state::render_active_line(bitmap_ind16 &bitmap, int scanline,
const tms340x0_device::display_params *params)
{
@@ -1492,15 +1547,33 @@ void vt1200_state::render_active_line(bitmap_ind16 &bitmap, int scanline,
return;
const u16 *const vram = m_vram.target();
// Two geometries, one per monitor mode (harness/c-hello/README.md,
// "Geometry is not portable between the real unit and MAME"):
//
// VR319 (default, the real unit) pitch 2048 bits (0x800, 256 bytes),
// 1280 active px, x = heblnk + fx
// VR150 (VT1200_MONITOR=VR150) pitch 1024 bits, 1024 active px,
// x = heblnk + fx
//
// The real numbers are the ones a payload targeting the hardware writes:
// vram_tile.asm sets DPTCH=0x800 and dwt/screenshot.py --real reads stride
// 256 / width 1280.
//
// The pitch is the only pitch-dependent number. The destination span
// always comes from the display controller's own blanking registers: in
// VR319 mode the firmware programs HEBLNK=0x15/HSBLNK=0x65 (16-px units),
// i.e. 1280 active pixels starting at screen x=336 in MAME's 1696-wide
// raster. Drawing the framebuffer at x=0 would shift the whole OS screen
// left by 336 px and leave the right quarter black -- which is what made
// the "real layout" look corrupted before.
const u32 pitch = m_render_real ? 2048u : u32(VMEM_PITCH_BITS);
const int x0 = (int)params->heblnk;
const int x1 = (int)params->hsblnk;
const u32 row_bit = (u32)fby * VMEM_PITCH_BITS;
const int x1 = std::min<int>((int)params->hsblnk, bitmap.width());
const u32 row_bit = (u32)fby * pitch;
for (int fx = 0; fx < VMEM_WIDTH; fx++)
for (int fx = 0; x0 + fx < x1; fx++)
{
const int x = x0 + fx;
if (x >= x1 || x >= bitmap.width())
break;
const u32 bit = row_bit + (u32)fx;
// Address-map units are bits, so a 16-bit word is 16 units.
bitmap.pix(scanline, x) = (vram[bit >> 4] >> (bit & 15)) & 1;
+2 -2
View File
@@ -9,8 +9,8 @@
# run -- see harness/c-hello/README.md).
#
# POOL_BYTES is compile-time: the script writes poolcfg.h from
# STREAM_POOL_BYTES (default 249856 = 244 KB: two 24 KB slots + 160 KB
# framebuffer + 32 KB dictionary + 4 KB slack).
# STREAM_POOL_BYTES (default 249856 = 244 KB: two 48 KB slots, 32 KB
# dictionary, and the LANCE block's fixed 4 KB tail).
#
# Outputs: build/vidstream.bin (raw .text, what vidstream.lua loads),
# build/vidstream.coff, vidstream.map, vidstream.lst, entry.lst,
+19 -2
View File
@@ -198,12 +198,24 @@ def _silhouette(rows: list[int], n: int) -> None:
rows[y] |= _span(x0 + 205, x0 + 250) & _MASK # the "eye"
#: Bit-reversal table: the synthetic clip is built as pixel ints with pixel 0
#: at the top bit, but the terminal's scanout reads bit 0 of a byte as the
#: leftmost pixel, so every byte is reversed on the way out (one table lookup,
#: not 1,280 shifts a row).
_BITREV = bytes(int(format(value, "08b")[::-1], 2) for value in range(256))
def _pack_row(row: int) -> bytes:
"""Pack one row int as the terminal reads it: LSB first within each byte."""
return row.to_bytes(ROW_BYTES, "big").translate(_BITREV)
def _synthetic_frame(n: int) -> bytes:
rows = list(_background_rows())
for y in range(SCREEN_ROWS - 160, SCREEN_ROWS):
rows[y] = 0 # a black ground band
_silhouette(rows, n)
return b"".join(row.to_bytes(ROW_BYTES, "big") for row in rows)
return b"".join(_pack_row(row) for row in rows)
def synthetic_clip(frames: int = 30) -> list[bytes]:
@@ -244,7 +256,12 @@ def load_clip(directory) -> list[bytes]:
"""Read a clip directory's ``*.bin`` frames, sorted by name.
Each file is one raw packed 1-bpp frame, exactly what
:func:`harness.udpstream.tilecost.quantise` takes. A ``.bin`` that is not
:func:`harness.udpstream.tilecost.quantise` takes. The bit order is the
terminal's scanout order: **LSB first** -- pixel *x* of a row is bit
``x % 8`` of byte ``x // 8`` (:meth:`harness.udpstream.vidsrc.Rows.pack_row`).
An MSB-first packer is self-consistent with an MSB-first checker and mirrors
every eight pixels on the display, so the order is stated here rather than
left to the producer. A ``.bin`` that is not
whole rows and whole tile bands is an error naming the file, not a silent
reshape -- the terminal's 1,280-byte frame is only meaningful for a whole
grid. The sort is by name, so number the frames zero-padded: ``frame-0007``
+2 -1
View File
@@ -3,7 +3,8 @@
The plan's two tests come first, verbatim. The extras pin down what the plan
text leaves implicit but the later tasks depend on:
* the tile layout -- 32 rows of four bytes, row-major, MSB first -- so a
* the tile layout -- 32 rows of four bytes, row-major, LSB first (bit 0 of a
byte is the leftmost pixel, the scanout order) -- so a
dictionary entry is exactly the bytes the terminal's ``PIXBLT B,XY`` reads
from ``dict + index*128``;
* the index frame's size. The plan says ``quantise`` returns 1,280 index bytes
+14 -10
View File
@@ -207,18 +207,18 @@ def test_a_short_last_datagram_is_padded_and_a_mid_one_is_not():
def test_a_chunk_smaller_than_the_ring_can_hold_is_rejected():
"""A whole strip frame split over more than 16 datagrams can never arrive.
"""A whole strip frame split over more datagrams than the ring can never arrive.
``--chunk 1000`` puts a 128-row frame (20,496 B) in 21 datagrams; the
terminal's RX ring holds 16, so the payload would drop the frame. The
constructor refuses it and names the smallest legal chunk.
The ring is 32 deep, so a 128-row frame (20,496 B) needs a chunk of at least
641 bytes; ``--chunk 600`` puts it in 35 datagrams and the constructor
refuses it, naming the smallest legal chunk.
"""
clock = FakeClock()
transport = RecordingTransport(clock)
with pytest.raises(ValueError) as e:
make_sender(clock, transport, rows=128, stretch=8, chunk=1000)
assert "at least 1281" in str(e.value)
assert "16-descriptor" in str(e.value)
make_sender(clock, transport, rows=128, stretch=8, chunk=600)
assert "at least 641" in str(e.value)
assert "32-descriptor" in str(e.value)
def test_a_chunk_that_fits_the_ring_is_accepted():
@@ -445,12 +445,16 @@ def tile_frames(datagrams):
def test_tile_mode_rejects_a_chunk_too_small_for_a_full_screen_delta(tile_cache):
"""A 1,280-tile delta is 3,858 B; at 200 B/datagram that is 20 > 16."""
"""A 1,280-tile delta is 3,858 B; at 100 B/datagram that is 39 > 32.
The ring's depth is the ceiling: 32 datagrams need a chunk of at least
ceil(3,858 / 32) = 121 bytes, which is what the message must name.
"""
clock = FakeClock()
transport = RecordingTransport(clock)
with pytest.raises(ValueError) as e:
make_tile_sender(tile_cache, clock, transport, chunk=200)
assert "at least 242" in str(e.value)
make_tile_sender(tile_cache, clock, transport, chunk=100)
assert "at least 121" in str(e.value)
assert "full-screen delta" in str(e.value)
+45 -17
View File
@@ -20,20 +20,19 @@ from harness.udpstream.vidsrc import (
CHUNK = 1450
def test_pack_row_is_msb_first_and_160_bytes():
def test_pack_row_is_lsb_first_and_160_bytes():
px = [0] * 1280
px[0] = 1
px[7] = 1
px[8] = 1
b = Rows.pack_row(px)
assert len(b) == 160
# Pixel 0 -> bit 7, pixel 7 -> bit 0, pixel 8 -> bit 7 of byte 1.
# NOTE: the plan/brief write this as 0b11000001, which is 0xC1 and encodes
# pixels 0, 1 and 7 -- three pixels from a two-pixel segment of the row, so
# no packing rule can produce it from px[0] and px[7]. MSB-first, as the
# same brief's prose states and as the terminal's bit-addressed row needs,
# is 0b10000001. See the task-2 report.
assert b[0] == 0b10000001 and b[1] == 0b10000000
# Pixel 0 -> bit 0, pixel 7 -> bit 7, pixel 8 -> bit 0 of byte 1: the
# terminal's scanout reads bit 0 of a byte as its leftmost pixel
# (dwt/screenshot.py decodes live VRAM the same way). The plan carried an
# MSB-first pair here (0b11000001 / 0b10000000), which is self-consistent
# with an MSB-first checker but mirrors every eight pixels on a display.
assert b[0] == 0b10000001 and b[1] == 0b00000001
def test_strip_source_rejects_stretch_that_does_not_divide_1024():
@@ -54,13 +53,36 @@ def test_strip_frame_round_trips_through_the_header():
# --- packing -------------------------------------------------------------
def _decode_like_the_terminal(row: bytes) -> list[int]:
"""Decode a packed row the way the VT1200 scanout does.
The same rule ``dwt/screenshot.py`` uses on live VRAM: bit ``x % 8`` of
byte ``x // 8`` is pixel ``x`` -- LSB first within each byte.
"""
return [(row[x // 8] >> (x % 8)) & 1 for x in range(1280)]
def test_pack_row_decodes_back_to_the_pixels_the_terminal_shows():
"""A row must survive the terminal's own LSB-first scanout decode.
The failure this pins: an MSB-first packer is self-consistent with an
MSB-first checker, so every MAME replay check passes while the screen
mirrors every group of eight pixels. Decoding the packed row with the
hardware's rule is the only reading that can catch it.
"""
px = [1 if i % 3 == 0 else 0 for i in range(1280)]
px[0] = 0 # asymmetric ends, so a mirror cannot pass
px[1279] = 1
assert _decode_like_the_terminal(Rows.pack_row(px)) == px
def test_pack_row_puts_every_pixel_where_the_terminal_reads_it():
"""Pixel i is bit (7 - i % 8) of byte i // 8, i.e. MSB first, and no bit leaks."""
"""Pixel i is bit (i % 8) of byte i // 8, i.e. LSB first, and no bit leaks."""
px = [1 if i % 3 == 0 else 0 for i in range(1280)]
b = Rows.pack_row(px)
assert len(b) == 160
for i, v in enumerate(px):
assert (b[i // 8] >> (7 - i % 8)) & 1 == v, i
assert (b[i // 8] >> (i % 8)) & 1 == v, i
def test_pack_row_rejects_a_row_that_is_not_1280_pixels():
@@ -137,21 +159,27 @@ def test_strip_source_rejects_impossible_stretch(stretch):
def test_strip_source_rejects_rows_that_do_not_fit_the_header():
with pytest.raises(ValueError):
StripSource(rows=0, stretch=128)
# The u8 header field allows 255 rows; the ring ceiling is what a huge row
# count hits first, and MAX_ROWS is that ceiling (289 with a 32-deep ring).
with pytest.raises(ValueError):
StripSource(rows=256, stretch=4)
StripSource(rows=MAX_ROWS + 1, stretch=4)
def test_strip_source_caps_rows_at_the_ring_not_the_header_field():
"""The terminal's 16-datagram ring, not the u8 ``rows`` field, is the limit.
"""The terminal's 32-datagram ring, not the u8 ``rows`` field, is the limit.
144 rows is 23,040 body bytes: 16 datagrams of 1450 with the 16-byte header
in the first. A 145th row would need a 17th datagram, which the receiver
drops (RX_DESC in vidstream.c), so the encoder must refuse it here -- with a
message that names the cap -- rather than put an unreceivable frame on the
The ring was deepened to 32 descriptors when the dictionary upload outgrew
16 (vidstream.c, SLOT_BYTES), so the strip encoder may now send 288 rows:
32 datagrams of 1450 with the 16-byte header in the first. One row more
would need a 33rd datagram, which the receiver drops (RX_DESC in
vidstream.c), so the encoder must refuse it here -- with a message that
names the cap -- rather than put an unreceivable frame on the
wire.
"""
src = StripSource(rows=MAX_ROWS, stretch=8)
pkts = src.encode(frame_no=1, rows_1bpp=bytes(MAX_ROWS * 160))
assert len(pkts) == RING_DATAGRAMS == 16
# MAX_ROWS is now the u8 header field (255), which is fewer datagrams than
# the 32-deep ring could carry -- so the frame fits with room to spare.
assert len(pkts) <= RING_DATAGRAMS == 32
with pytest.raises(ValueError, match=str(MAX_ROWS)):
StripSource(rows=MAX_ROWS + 1, stretch=8)
+2 -1
View File
@@ -13,7 +13,8 @@ Geometry
The screen is 1280x1024, one bit per pixel. A source row is therefore 160 bytes
(:data:`harness.udpstream.vidsrc.ROW_BYTES`) and a 32x32 tile is 32 rows of
**four** bytes -- 32 px is two 16-bit words -- for 128 bytes: row-major, two
bytes per word, MSB first within each byte, the same bit order
bytes per word, **LSB first** within each byte (bit 0 is the leftmost pixel,
the scanout order), the same bit order
:meth:`harness.udpstream.vidsrc.Rows.pack_row` produces. The grid is 40 columns
by 32 rows: 1,280 tiles.
Binary file not shown.
Binary file not shown.
+104 -104
View File
@@ -1,68 +1,68 @@
{
"active_row_bytes": 160,
"bit_errors": [
1308,
1351,
1420,
1366,
1309,
1425,
1442,
1507,
1389,
1574,
1444,
1342,
1446,
1260,
1556,
1459,
1592,
1472,
1369,
1524,
1509,
1581,
1506,
1505,
1514,
1719,
1540,
1595,
1610,
1809
953,
778,
999,
880,
868,
1020,
801,
944,
801,
955,
778,
964,
1086,
985,
823,
1056,
895,
1019,
882,
890,
1041,
1038,
1031,
1043,
910,
1011,
1169,
1173,
1195,
1132
],
"changed_tiles": [
0,
12,
14,
13,
7,
15,
11,
11,
11,
10,
11,
11,
10,
12,
9,
14,
10,
8,
13,
15,
9,
14,
9,
12,
11,
12,
14,
10,
12,
13,
10,
13,
10,
13,
11,
10,
12,
13,
12,
12,
12,
10,
13,
9,
13,
13,
11,
10
11,
12,
11
],
"clip": "synthetic_clip",
"delta_frames": [
@@ -96,7 +96,7 @@
28,
29
],
"delta_payload_bytes": 1057,
"delta_payload_bytes": 1054,
"dictionary_bytes": 32768,
"entries_used": 256,
"frames": 30,
@@ -108,71 +108,71 @@
],
"payload_bytes": [
1280,
38,
44,
41,
23,
47,
35,
35,
35,
32,
35,
35,
32,
38,
29,
44,
32,
26,
41,
47,
29,
44,
29,
38,
35,
38,
44,
32,
38,
41,
32,
41,
32,
41,
35,
32,
38,
41,
38,
38,
38,
32,
41,
29,
41,
41,
35,
32
35,
38,
35
],
"psnr_db": [
30.00902182299432,
29.868545772656496,
29.652215819046237,
29.820592269421663,
30.005702797369246,
29.636950619431516,
29.585446659042702,
29.393966739730484,
29.748076805500645,
29.205051982646346,
29.579427330540604,
29.897574104547072,
29.573416333291682,
30.171393811701176,
29.255003336340103,
29.534546343942285,
29.155668628860305,
29.496021162862004,
29.811064781536906,
29.345249592840986,
29.388206865120907,
29.185780563554715,
29.396849544229987,
29.399734263578182,
29.373840511236263,
28.82234049600628,
29.299892054512174,
29.1474923889448,
29.106840502558306,
28.600713594278666
31.384170256493537,
32.265303292979915,
31.17944438061698,
31.73027254137512,
31.789902011111884,
31.08909754525763,
32.13877410203443,
31.425379319896113,
32.13877410203443,
31.37506554703934,
32.265303292979915,
31.334328923848496,
30.81680101034852,
31.240736957900687,
32.02110091075411,
30.93846008089887,
31.656868909717684,
31.093357422812538,
31.720413411558606,
31.681199196427677,
31.00059196777144,
31.01312572775241,
31.042512610041637,
30.992256178611495,
31.584685339665867,
31.127587706966793,
30.4969541512584,
30.482119141721512,
30.401420210035237,
30.636634994354278
],
"screen_rows": 1024,
"seed": 0,
"total_payload_bytes": 2337,
"total_payload_bytes": 2334,
"unchanged_frames": [],
"version": 1
}
Binary file not shown.
+7 -1
View File
@@ -710,6 +710,11 @@ def main(argv=None) -> int:
help="stop after N frames (0 = until interrupted)")
ap.add_argument("--pattern", default="bars", choices=sorted(PATTERNS),
help="test picture (default bars)")
ap.add_argument("--keyframe-every", type=int, default=KEYFRAME_EVERY,
metavar="N",
help="force a full frame every N frames (default %(default)s; "
"a keyframe repaints all 1,280 tiles, so a short cadence "
"is a large per-frame cost at high rates)")
ap.add_argument("--chunk", type=int, default=CHUNK,
help="bytes of frame per datagram (default %d)" % CHUNK)
ap.add_argument("--dst", type=parse_mac, default=BROADCAST,
@@ -788,7 +793,8 @@ def main(argv=None) -> int:
fps=args.fps, frames=args.frames or None,
chunk=args.chunk, dst=args.dst,
report_every=report_every,
status_wait=args.status_wait)
status_wait=args.status_wait,
every=args.keyframe_every)
return Sender(transport, src=src, receive=receive, rows=args.rows,
stretch=args.stretch, fps=args.fps,
frames=args.frames or None, pattern=args.pattern,
+25 -18
View File
@@ -1,7 +1,8 @@
"""Turn 1-bpp source rows into framed strip packets for the VT1200 stream.
A *strip* frame is *N* unique rows of 1280 pixels, each row 160 bytes packed MSB
first. The terminal blits row *n* to destination row ``n * stretch`` with the
A *strip* frame is *N* unique rows of 1280 pixels, each row 160 bytes packed
**LSB first** (bit 0 of a byte is the row's leftmost pixel -- the terminal's
scanout order, see :meth:`Rows.pack_row`). The terminal blits row *n* to destination row ``n * stretch`` with the
blitter's vertical stretch, so each source row is reused ``stretch`` times and
the whole screen is repainted from *N* = 1024 / stretch rows. That is the whole
scheme: 128 rows is a full-screen frame at 20,480 B of payload, 8 rows is a
@@ -44,16 +45,15 @@ CHUNK = 1450
#: Datagrams in the terminal's RX ring (``RX_DESC`` in vidstream.c). A frame
#: that needs more cannot be received at all: the ring runs out of descriptors
#: and the payload drops it.
RING_DATAGRAMS = 16
RING_DATAGRAMS = 32
#: Most source rows one frame can carry, from the ring rather than the header
#: field: the first datagram holds the 16-byte header plus a body, the other 15
#: carry body only, so the ring holds
#: ``(RING_DATAGRAMS-1)*CHUNK + (CHUNK-HEADER_SIZE)`` body bytes = 144 rows of
#: 160. The 24 KB slot alone would allow 153, and the u8 ``rows`` field 255 --
#: neither is reachable, because the ring is what binds.
MAX_ROWS = (((RING_DATAGRAMS - 1) * CHUNK + (CHUNK - HEADER_SIZE))
// ROW_BYTES)
#: Most source rows one frame can carry. Two ceilings, and the lower binds: the
#: ring holds ``(RING_DATAGRAMS-1)*CHUNK + (CHUNK-HEADER_SIZE)`` body bytes
#: (289 rows of 160 with a 32-deep ring), and the header's ``rows`` field is one
#: byte (255). So 255 today; the ring was 16 when this was written, when the
#: ring was the binding one at 144.
MAX_ROWS = min(255, (((RING_DATAGRAMS - 1) * CHUNK + (CHUNK - HEADER_SIZE))
// ROW_BYTES))
class Rows:
@@ -63,19 +63,26 @@ class Rows:
def pack_row(pixels: list[int]) -> bytes:
"""Return the :data:`ROW_BYTES` packed bytes for ``pixels``.
Pixel *i* becomes bit ``7 - i % 8`` of byte ``i // 8`` -- MSB first,
which is the bit order of the terminal's line buffer. A truthy pixel is
a set bit, so ``True``, ``1`` and any other nonzero value all mean ink.
Pixel *i* becomes bit ``i % 8`` of byte ``i // 8`` -- **LSB first**,
which is how the terminal's scanout reads the framebuffer: ``dwt/
screenshot.py`` decodes live VRAM with exactly this rule, and MAME's
renderer does the same (``(vram[bit >> 4] >> (bit & 15)) & 1``).
The earlier MSB-first convention here was self-consistent -- the encoder
and ``tile_check.py`` shared it, so every replay check passed -- but it
mirrors every eight pixels on a real display. A truthy pixel is a set
bit, so ``True``, ``1`` and any other nonzero value all mean ink.
"""
if len(pixels) != ROW_PIXELS:
raise ValueError("a row is %d pixels, got %d"
% (ROW_PIXELS, len(pixels)))
# "1000000..." -> the integer those bits denote, big-endian bytes.
# Equivalent to setting bit 7 - i % 8 of byte i // 8 for each pixel i,
# but the join+parse is C-speed per character and measured ~1.5x a
# per-pixel shift loop on this box (65-120 us vs 100-150 us per row).
# Pixel i must land on bit i % 8 of byte i // 8, so reverse each
# eight-pixel group before the parse: the first character of a group
# becomes that byte's MSB, and the last becomes its LSB.
bits = "".join("1" if value else "0" for value in pixels)
return int(bits, 2).to_bytes(ROW_BYTES, "big")
lsb = "".join(bits[i:i + 8][::-1] for i in range(0, ROW_PIXELS, 8))
return int(lsb, 2).to_bytes(ROW_BYTES, "big")
class StripSource:
+251 -63
View File
@@ -24,10 +24,11 @@
*
* Pool layout inside the single allocation (POOL_BYTES = 249856 = 244 KB):
*
* +0x00000 frame slot A 24576 bytes (16 x 1526 B RX buffers)
* +0x06000 frame slot B 24576 bytes (16 x 1526 B RX buffers)
* +0x0C000 dictionary 32768 bytes (256 x 128 B, Task 3)
* +0x14000 framebuffer 163840 bytes (scheme A reserve; see below)
* +0x00000 frame slot A 49152 bytes (32 x 1526 B RX buffers)
* +x0C000 frame slot B 49152 bytes (32 x 1526 B RX buffers)
* +0x18000 dictionary 32768 bytes (256 x 128 B, Task 3)
* +0x20000 slack 114688 bytes (unused; the LANCE block's
* fixed 0x3C000 tail follows)
* +0x3C000 LANCE block 4096 bytes (init block, TX ring, status
* frame, RX ring)
*
@@ -35,8 +36,10 @@
* plan fixes and plan 2's Task 4 consumes. The 160 KB framebuffer reserve
* follows it: this decoder paints straight from the RX fragments (there is no
* per-frame CPU copy, spec 8.1) and never touches that region, so it is slack
* today, but the layout adds up to exactly POOL_BYTES -- 24576 + 24576 + 32768
* + 163840 + 4096 = 249856. dict_slack_check below pins the part Task 3
* today, but the layout adds up to exactly POOL_BYTES -- 49152 + 49152 + 32768
* + 114688 + 4096 = 249856. The pool size did NOT have to grow for the deeper
* ring: the 160 KB framebuffer reserve was unused slack, and 114,688 B of it
* is still free between the dictionary and the LANCE block. dict_slack_check below pins the part Task 3
* depends on (the slot pair and the 32 KB dictionary); the LANCE block's
* offsets at 0x3C000 were fixed before it and are not re-checked here. Had
* the dictionary gone after the framebuffer, the plan's base would not be
@@ -46,8 +49,8 @@
* (lance_start() explains why the offset alone is not enough), and the status
* frame sits right after the 8 TX descriptors, the RX ring after that.
*
* A slot is 16 packet buffers of RXBUFLEN = 1526 bytes (16 x 1526 = 24416, so
* it fits 24576 with 160 bytes to spare). The chip writes ONE Ethernet frame
* A slot is RX_DESC packet buffers of RXBUFLEN = 1526 bytes (32 x 1526 =
* 48832, so it fits 49152 with 320 bytes to spare). The chip writes ONE Ethernet frame
* per buffer -- 14-byte header, the VS frame's chunk, and the FCS it appends --
* so a VS frame arrives as a series of datagrams in ring order, and the frame's
* bytes are NOT contiguous in the slot: they are 14 bytes into each buffer,
@@ -67,10 +70,10 @@
* fits with one descriptor to spare.
*
* Screen geometry. VRAM is at bit 0x04000000 and a screen row is 0x800 bits
* (256 bytes), the real unit's pitch (harness/c-hello/README.md). MAME's own
* renderer decodes the same memory at 128 bytes a row; that difference is a
* property of the harness, so the draw test compares VRAM as memory, not the
* rendered picture.
* (256 bytes), the real unit's pitch (harness/c-hello/README.md). The harness
* now models the real unit's VR319 monitor by default, so MAME's renderer uses
* the same 256-byte pitch; the draw test still compares VRAM as memory, not the
* rendered picture, because that is what the payload actually writes.
*
* OS repaint canary (spec 8.9). The blit covers screen rows 0..1023 and
* columns 0..1279 only. One 16-bit sentinel word sits in the unused right
@@ -152,13 +155,19 @@ static unsigned int decode_worst; /* worst since the last status sent */
#define SENT_VALUE 0xA5C3
/*
* The two frame slots. SLOT_BYTES is the plan's 24 KB: a 128-row strip frame
* (128 x 160 B + 16 B header = 20,496 B = 15 packets) fits, and so does any
* smaller strip. Slot addresses are bit addresses, because that is what a
* The two frame slots. SLOT_BYTES holds a whole RX ring: 32 descriptors x
* RXBUFLEN (1526) = 48,832 B, rounded up. It also still holds a 128-row strip
* frame (128 x 160 B + 16 B header = 20,496 B = 15 packets).
*
* 32, not 16, because 16 could not absorb the tile dictionary: it arrives as
* 23 back-to-back datagrams (one TILES_DICT_CHUNK each), and with a 16-deep
* ring the chip dropped the tail while the consumer was still copying -- the
* measured zero-gap upload stalled at 24,616 of 32,768 B. The heap has
* megabytes free; the ring, not the memory, was the limit. Slot addresses are bit addresses, because that is what a
* pointer holds in this toolchain (see harness/c-hello/README.md: "pointers
* are bit addresses").
*/
#define SLOT_BYTES 24576
#define SLOT_BYTES 49152
/*
* Plan 2's Task 3: the tile dictionary.
@@ -270,9 +279,16 @@ char tile_grid_check[(TILES_SCREEN == FP_TILES_SCREEN) ? 1 : -1];
* 1514-byte Ethernet frame plus margin, the figure lance_cmd.c uses on the
* unit: a datagram from vidsrc.py is 14 + 1450 = 1464 bytes, the chip appends
* the 4-byte FCS, writes it after the frame and reports the total in RMD3.
* RX_DESC * RXBUFLEN = 24416 bytes, which fits a 24576-byte slot.
* RX_DESC * RXBUFLEN = 48832 bytes, which fits a 49152-byte slot.
*
* The depth is the burst size the protocol actually uses: a whole 32 KB
* dictionary upload is 23 datagrams (MAX_BODY 1448), so 32 descriptors let the
* chip take the entire upload before the consumer has copied the first chunk.
* It is not a substitute for the sender pacing -- any ring can be outrun by a
* longer burst -- but it removes the accidental failure of the one burst the
* protocol defines.
*/
#define RX_DESC 16
#define RX_DESC 32
#define RXBUFLEN 1526
#define RMD1_OWN 0x8000 /* the chip owns the descriptor */
@@ -323,9 +339,37 @@ unsigned int tile_src, tile_daddr;
extern void vram_blit_b_(); /* vram_b.asm: PIXBLT B,XY, SPTCH = 0 */
extern void tile_blit_(); /* vram_tile.asm: PIXBLT B,XY, 32x32 tile */
extern void tile_init_(); /* vram_tile.asm: the per-frame constants */
extern void vid_dint_(); /* vram_b.asm: DINT */
extern void vid_eint_(); /* vram_b.asm: EINT */
/*
* VID_OWN_CPU -- mask interrupts once at startup and never re-enable them.
*
* This is spec 8.9's "the resident OS may die", and it is also a *rate* fix,
* not just a robustness one. With the OS's timer still running, this loop is
* preempted and the machine accepts ~15 frames/s whatever the payload size: a
* 5-byte one-tile delta measured the same as a 140-tile one (195 frames in
* 13.4 s, MAME at 100% speed). That is scheduling, not decode. With IE held
* clear nothing preempts the loop and the LANCE is polled as fast as the chip
* delivers.
*
* Consequences, stated because they are real: the OS's drivers stop being
* serviced, so its console and network stack are gone for good -- the payload
* owns the machine either way once it takes the LANCE. The harness reaches
* the payload through the debugger, not through the OS, so the replay modes
* are unaffected.
*
* Set VID_OWN_CPU=0 to build the old behaviour (bursts of EINT around each
* transfer) for a comparison run.
*/
#ifndef VID_OWN_CPU
#define VID_OWN_CPU 1
#endif
#if VID_OWN_CPU
#define vid_eint_() ((void)0) /* masked for good; see main() */
#endif
/*
* The firmware's free-space counter is a 32-bit count IN BITS at
* 0x1E8BA0 (dwt/notes/HEAP.md). POOL_FREE reports bytes, because the
@@ -352,9 +396,10 @@ static unsigned int drops_total; /* CSR0 MISS windows, lower bound on misses */
/*
* Task 7 receive state.
*
* rx_armed the slot the ring's 16 descriptors point into (0 = A, 1 = B).
* Only ever written together with a full re-arm, so a descriptor
* never points at a slot the payload is not expecting.
* rx_slot_of[] which slot each descriptor points into (0 = A, 1 = B).
* Per descriptor, so descriptors can be recycled one at a time
* and nothing the chip has already delivered is re-pointed
* unread (see rx_release()).
* rx_pos the next descriptor the consumer expects. The chip fills
* descriptors strictly in ring order (am79c90.cpp's
* recv_complete_cb advances m_rx_ring_pos by one per packet and
@@ -379,10 +424,21 @@ static unsigned int drops_total; /* CSR0 MISS windows, lower bound on misses */
* for them -- so without this cell a body-corrupting packet would be invisible
* to both crc_err and drops. Each one abandons the frame it belongs to.
*/
static unsigned int rx_armed;
static unsigned int rx_pos;
static unsigned int rx_acc;
static unsigned int rx_nfrag;
/* Which slot each descriptor currently points into. Per descriptor, not one
* global "armed" slot: a burst is handled by releasing descriptors one at a
* time as their data is consumed, so a descriptor the chip has already filled
* is never re-pointed behind the consumer's back. */
static unsigned int rx_slot_of[RX_DESC];
/* Descriptors consumed for the frame being assembled -- their data must stay
* put until the frame is parsed or abandoned, so they are released together
* at the end. */
static unsigned int rx_hold[RX_DESC];
static unsigned int rx_nhold;
/* The slot the first fragment of the current frame landed in (RX_SLOT). */
static unsigned int rx_frame_slot;
static unsigned int rx_frag_off[RX_DESC];
static unsigned int rx_frag_ptr[RX_DESC];
static unsigned int rx_frag_len[RX_DESC];
@@ -434,6 +490,32 @@ unsigned int dydx;
blit_src = src;
blit_daddr = daddr_xy;
blit_dydx = dydx;
blit_sptch = 0; /* stretch: every row is the same source */
vram_blit_b_();
}
/*
* blit_rect(src, y, rows, sptch) -- copy `rows` screen rows at once.
*
* The strip path used to issue one PIXBLT per screen row because a stretched
* row has SPTCH = 0, so the source cannot advance. For a picture that really
* has one source row per screen row (stretch = 1) the source DOES advance, and
* a group of rows inside one receive fragment is contiguous: one PIXBLT with
* SPTCH = 1280 bits copies the whole group. Measured: a strip call costs about
* 0.45 ms in MAME regardless of its size, while a tile call costs the same for
* 64 words -- so call count, not word count, is what a frame is charged for.
* A full frame is then ~32 calls (one per datagram) instead of 1,200.
*/
static void blit_rect(src, y, rows, sptch)
unsigned int src;
unsigned int y;
unsigned int rows;
unsigned int sptch;
{
blit_src = src;
blit_daddr = (y << 16); /* x = 0: a strip row is the full width */
blit_dydx = (rows << 16) | ROW_PIXELS;
blit_sptch = sptch;
vram_blit_b_();
}
@@ -510,6 +592,62 @@ unsigned int ty;
* The whole burst runs with interrupts masked (spec 8.9) so the resident OS
* cannot repaint between two of our source rows and tear the picture.
*/
/*
* paint_rect(h) -- the stretch = 1 case, one PIXBLT per fragment.
*
* A strip frame with stretch 1 has exactly one source row per screen row, so
* the source advances by ROW_PIXELS bits per row and a group of rows that sits
* inside one receive fragment is contiguous: one blit copies the whole group.
* With the sender's 1,440-byte chunks (9 rows, chosen so no row straddles a
* datagram) a 288-row sub-frame is 32 blits instead of 288.
*
* Returns 1 when the whole frame was painted, 0 when the fragments do not line
* up on row boundaries -- the caller then falls back to the per-row path, which
* handles a split row by drawing it in pieces.
*/
static int paint_rect(h)
fp_header *h;
{
unsigned int rows, pos, k, off, nr, rn;
rows = h->rows;
if (rows * ROW_BYTES != h->payload_len) {
return 0; /* not whole rows: not for us */
}
for (k = 0; k < rx_nfrag; k++) {
if (rx_frag_off[k] % ROW_BYTES
|| rx_frag_len[k] % ROW_BYTES) {
return 0; /* a row straddles: general path */
}
}
rn = 0;
k = 0;
pos = h->payload_off;
while (rn < rows) {
while (k < rx_nfrag && pos >= rx_frag_off[k] + rx_frag_len[k]) {
k = k + 1;
}
if (k >= rx_nfrag) {
return 1; /* truncated: stop drawing */
}
if (pos < rx_frag_off[k]) {
return 1; /* a gap: never paint across it */
}
off = pos - rx_frag_off[k];
nr = (rx_frag_len[k] - off) / ROW_BYTES;
if (nr > rows - rn) {
nr = rows - rn;
}
if (nr == 0) {
return 1;
}
blit_rect(rx_frag_ptr[k] + off * 8, rn, nr, ROW_PIXELS);
rn = rn + nr;
pos = pos + nr * ROW_BYTES;
}
return 1;
}
static void paint(h)
fp_header *h;
{
@@ -541,6 +679,10 @@ fp_header *h;
}
vid_dint_();
if (stretch == 1 && paint_rect(h)) {
vid_eint_();
return;
}
pos = h->payload_off; /* the frame's first body byte */
k = 0;
for (r = 0; r < rows; r++) {
@@ -632,6 +774,7 @@ fp_header *h;
}
vid_dint_();
tile_init_(); /* the tile constants: once per frame */
decode_begin();
k = 0;
pos = h->payload_off; /* the frame's first index byte */
@@ -809,6 +952,7 @@ fp_header *h;
* the per-row samples keep the 16-bit REFCNT counter unambiguous. */
decode_sample();
vid_dint_();
tile_init_(); /* the tile constants: once per frame */
frag_get(h->payload_off + FP_DELTA_HDR, &lo);
frag_get(h->payload_off + FP_DELTA_HDR + 1, &hi);
pos = lo | (hi << 8);
@@ -1058,34 +1202,54 @@ static unsigned int slot_base[2];
* descriptor completed by anything but the chip reads as a zero-length packet
* rather than as the previous frame's length.
*/
static void rx_arm_one(i, slot)
unsigned int i, slot;
{
unsigned int cb;
cb = (slot_base[slot] >> 3) + i * RXBUFLEN;
/* OWN clear, with the new high byte: a fetch here is a MISS. */
w16(lance_rxring + i * 64 + 16, (cb >> 16) & 0xFF);
w16(lance_rxring + i * 64 + 0, cb & 0xFFFF);
w16(lance_rxring + i * 64 + 32, (0x10000 - RXBUFLEN) & 0xFFFF);
w16(lance_rxring + i * 64 + 48, 0);
/* Ownership last, over a complete descriptor. */
w16(lance_rxring + i * 64 + 16, RMD1_OWN | ((cb >> 16) & 0xFF));
rx_slot_of[i] = slot;
}
static void rx_arm(slot)
unsigned int slot;
{
unsigned int i, cb, base;
unsigned int i;
base = slot_base[slot];
for (i = 0; i < RX_DESC; i++) {
cb = (base >> 3) + i * RXBUFLEN;
/* OWN clear, with the new high byte: a fetch here is a MISS. */
w16(lance_rxring + i * 64 + 16, (cb >> 16) & 0xFF);
w16(lance_rxring + i * 64 + 0, cb & 0xFFFF);
w16(lance_rxring + i * 64 + 32, (0x10000 - RXBUFLEN) & 0xFFFF);
w16(lance_rxring + i * 64 + 48, 0);
/* Ownership last, over a complete descriptor. */
w16(lance_rxring + i * 64 + 16,
RMD1_OWN | ((cb >> 16) & 0xFF));
rx_arm_one(i, slot);
}
}
/*
* rx_switch() -- arm the other slot. Called at the end of a frame, before the
* picture is drawn, so the chip can fill N+1 while we paint N and can never
* write into the slot being read (spec 8.1's overlap).
* rx_release() -- give the frame's descriptors back, pointing each at the OTHER
* slot (spec 8.1's overlap: the buffer the chip next writes must not be one the
* consumer is still reading).
*
* This replaces a per-frame "switch slots and re-arm all 16". That flip was
* the burst bug: an all-16 re-arm re-points descriptors the chip had ALREADY
* completed but the consumer had not yet read, so their frames were silently
* lost -- measured on a 23-datagram dictionary upload: the chip delivered all
* 23 (LANCE DMA log, 1.13 ms apart) and only 4 reached these counters. Holding
* and releasing each descriptor individually means nothing the chip delivered
* is ever re-pointed unread, and the ring recycles as fast as the consumer.
*/
static void rx_switch()
static void rx_release()
{
rx_armed = 1 - rx_armed;
rx_arm(rx_armed);
unsigned int k, i;
for (k = 0; k < rx_nhold; k++) {
i = rx_hold[k];
rx_arm_one(i, 1 - rx_slot_of[i]);
}
rx_nhold = 0;
}
/*
@@ -1108,11 +1272,14 @@ static void rx_dgram(i)
unsigned int i;
{
unsigned int rmd1, mlen, len, src, plen, total, lim, slot;
unsigned int accepted, refused;
fp_header h;
accepted = 0;
refused = 0;
rmd1 = r16(lance_rxring + i * 64 + 16);
mlen = r16(lance_rxring + i * 64 + 48) & 0x0FFF;
src = slot_base[rx_armed] + i * RXBUFLEN * 8 + ETH_HDR_LEN * 8;
src = slot_base[rx_slot_of[i]] + i * RXBUFLEN * 8 + ETH_HDR_LEN * 8;
len = 0;
if (rmd1 & RMD1_ERR) {
/*
@@ -1145,7 +1312,7 @@ unsigned int i;
*(unsigned int *)RX_FCS_DROP + 1;
rx_acc = 0;
rx_nfrag = 0;
rx_switch();
rx_release();
}
return;
}
@@ -1169,6 +1336,9 @@ unsigned int i;
}
}
if (rx_nfrag == 0) {
rx_frame_slot = rx_slot_of[i]; /* what RX_SLOT reports */
}
rx_frag_off[rx_nfrag] = rx_acc;
rx_frag_ptr[rx_nfrag] = src;
rx_frag_len[rx_nfrag] = len;
@@ -1192,17 +1362,17 @@ unsigned int i;
*(unsigned int *)RX_BAD = *(unsigned int *)RX_BAD + 1;
rx_acc = 0;
rx_nfrag = 0;
rx_switch();
rx_release();
return;
}
if (rx_acc < total) {
return; /* still arriving */
}
/* The frame is complete. Hand the OTHER slot to the chip first, so
* nothing it receives from now on lands in the picture below. */
slot = rx_armed;
rx_switch();
/* The frame is complete. Its descriptors stay held -- so the chip
* cannot overwrite the picture while it is parsed and painted -- and are
* released just below, each into the other slot. */
slot = rx_frame_slot;
/*
* Task 4 finding (b): the length handed to fp_parse() is clamped to the
@@ -1219,9 +1389,7 @@ unsigned int i;
*(unsigned int *)RX_LAST_LEN = h.payload_len;
*(unsigned int *)RX_SLOT = slot;
paint(&h);
/* Published after the paint: a reader that sees RX_OK move
* sees a picture that is already on the screen. */
*(unsigned int *)RX_OK = *(unsigned int *)RX_OK + 1;
accepted = 1;
} else if (h.kind == FP_KIND_DICT_CHUNK) {
/* A chunk is not a picture: it is counted by DICT_BYTES
* and DICT_BAD, never by RX_OK. */
@@ -1239,11 +1407,9 @@ unsigned int i;
if (paint_tiles(&h)) {
*(unsigned int *)RX_LAST_LEN = h.payload_len;
*(unsigned int *)RX_SLOT = slot;
*(unsigned int *)RX_OK =
*(unsigned int *)RX_OK + 1;
accepted = 1;
} else {
tiles_drop = tiles_drop + 1;
*(unsigned int *)TILES_DROP = tiles_drop;
refused = 1;
}
} else if (h.kind == FP_KIND_TILES_DELTA) {
/*
@@ -1255,11 +1421,9 @@ unsigned int i;
if (paint_delta(&h)) {
*(unsigned int *)RX_LAST_LEN = h.payload_len;
*(unsigned int *)RX_SLOT = slot;
*(unsigned int *)RX_OK =
*(unsigned int *)RX_OK + 1;
accepted = 1;
} else {
tiles_drop = tiles_drop + 1;
*(unsigned int *)TILES_DROP = tiles_drop;
refused = 1;
}
} else {
/* A well-formed header whose kind nothing here dispatches
@@ -1272,6 +1436,23 @@ unsigned int i;
}
rx_acc = 0;
rx_nfrag = 0;
/* Released only now: every byte this frame needed has been read. */
rx_release();
/*
* Published after the release, not inside the dispatch. RX_OK is the
* "frame done" signal a reader polls, and it must not become visible
* while rx_release() is still rewriting descriptors: a sample taken then
* catches a descriptor mid-update (measured: slot B's high byte with
* slot A's low word). A reader that sees RX_OK move now sees a painted
* picture AND a consistent ring.
*/
if (refused) {
tiles_drop = tiles_drop + 1;
*(unsigned int *)TILES_DROP = tiles_drop;
}
if (accepted) {
*(unsigned int *)RX_OK = *(unsigned int *)RX_OK + 1;
}
}
/*
@@ -1300,18 +1481,20 @@ static void rx_poll()
}
rx_pos = (i + 1) & (RX_DESC - 1);
*(unsigned int *)RX_DGRAM = *(unsigned int *)RX_DGRAM + 1;
rx_hold[rx_nhold] = i;
rx_nhold = rx_nhold + 1;
rx_dgram(i);
if (rx_nfrag == RX_DESC && rx_acc != 0) {
*(unsigned int *)RX_RING_DROPS =
*(unsigned int *)RX_RING_DROPS + 1;
rx_acc = 0;
rx_nfrag = 0;
rx_switch();
rx_release();
}
}
*(unsigned int *)RX_RING_POS = rx_pos;
*(unsigned int *)RX_RING_ACC = rx_acc;
*(unsigned int *)RX_RING_ARMED = rx_armed;
*(unsigned int *)RX_RING_ARMED = rx_slot_of[rx_pos];
}
/*
@@ -1375,7 +1558,7 @@ unsigned int ptr;
w16(init + 7 * 16, 0);
cb = rxring >> 3; /* BYTE address of the RX ring */
w16(init + 8 * 16, cb & 0xFFFF);
w16(init + 9 * 16, ((cb >> 16) & 0xFF) | (4 << 13)); /* 16 desc */
w16(init + 9 * 16, ((cb >> 16) & 0xFF) | (5 << 13)); /* 32 desc */
cb = ring >> 3; /* BYTE address of the ring */
w16(init + 10 * 16, cb & 0xFFFF);
w16(init + 11 * 16, ((cb >> 16) & 0xFF) | (3 << 13));
@@ -1396,8 +1579,8 @@ unsigned int ptr;
/* The RX ring, armed for slot A: this is the state the first packet
* finds, and it is what the MAME ring check asserts. */
rx_armed = 0;
rx_arm(rx_armed);
rx_nhold = 0;
rx_arm(0);
*(unsigned int *)RX_RING_BASE = rxring;
lance_csr_w(1, (init >> 3) & 0xFFFF);
@@ -1702,14 +1885,19 @@ int main()
status_seq = 0;
status_seen = 0;
drops_total = 0;
rx_armed = 0;
rx_pos = 0;
rx_acc = 0;
rx_nfrag = 0;
rx_nhold = 0;
rx_frame_slot = 0;
vid_dint_();
lance_ok = lance_start(ptr);
#if VID_OWN_CPU
vid_dint_(); /* once, for good: nothing may preempt this loop */
#else
vid_eint_();
#endif
*(unsigned int *)ST_BUF = status_buf;
for (;;) {
+7 -1
View File
@@ -35,6 +35,11 @@
; PMASK (0C0000160h) 0 every plane writable
; CONTROL (0C00000B0h) raster op 0 (replace), CD/RM/RR preserved
;
; _blit_sptch: the source pitch in bits. Zero is the stretch form (every
; source row reads the same address); 1280 is the rectangle form, where each
; source row is one screen row of the picture -- one PIXBLT then copies a whole
; group of rows, which is what makes a full frame ~32 calls instead of ~1,200.
;
; Entry points. The TMS34010 C compiler passes arguments in its own frame-
; pointer scheme, and no hand-written assembly in this repository has ever
; DECODED that frame: the ROM thunks in entry.asm forward the caller's stack
@@ -54,6 +59,7 @@
; resident repaint cannot tear the picture.
.ref _blit_src
.ref _blit_sptch
.ref _blit_daddr
.ref _blit_dydx
.text
@@ -70,7 +76,7 @@ _vram_blit_b_:
move b9, -*sp, 1
move @_blit_src, b0, 1 ; SADDR: packed 1-bpp source row
clr b1 ; SPTCH = 0: every row is the same source row
move @_blit_sptch, b1, 1 ; SPTCH: 0 = stretch, 1280 = rectangle
move @_blit_daddr, b2, 1 ; DADDR (XY)
movi 800h, b3 ; DPTCH = 2048 bits = 256-byte row pitch
movi 04000000h, b4 ; OFFSET = VRAM base
+45 -28
View File
@@ -1,8 +1,9 @@
; vram_tile.asm -- PIXBLT B,XY: one 32x32 1-bpp dictionary tile to the screen.
;
; Task 4's tile primitive. A dictionary entry is 32 rows of FOUR bytes (32 px
; is two 16-bit words) = 128 bytes = 1024 bits, row-major, MSB first within each
; byte -- tilecost.py's layout. PIXBLT's B source form reads packed 1-bit
; is two 16-bit words) = 128 bytes = 1024 bits, row-major, LSB first within each
; byte (bit 0 is the leftmost pixel -- the scanout order) -- tilecost.py's
; layout. PIXBLT's B source form reads packed 1-bit
; pixels from a linear bit address and writes an XY destination, so one tile is
;
; SADDR = dict_base + index * 1024 (128 bytes a tile: index << 10)
@@ -44,9 +45,12 @@
; tile_daddr the destination XY word
;
; and the C wrapper tile_blit(src, tx, ty) is the only entry point. B7 (DYDX)
; is the constant 32x32 this primitive exists for. All inputs are reloaded on
; every call: PIXBLT leaves SADDR/DADDR and the B registers it advanced in a
; half-consumed state, so a caller that "saved" them would save that.
; is the constant 32x32 this primitive exists for.
;
; Two entry points, split by what changes: tile_init_() programs the constants
; (pitch, offset, dimensions, colours, PSIZE/CONVSP/CONVDP/PMASK/CONTROL) once
; per frame, and tile_blit_() then only reloads SADDR and DADDR -- the two
; registers PIXBLT destroys -- before executing it.
;
; Revert note: this file is new in Task 4 and nothing else calls it. To
; revert, delete it, the VRAMTILE entries in build-stream.sh and the
@@ -61,28 +65,31 @@
.ref _tile_src
.ref _tile_daddr
.text
.globl _tile_blit_
_tile_blit_:
setf 32, 0, 1
move b0, -*sp, 1
move b1, -*sp, 1
move b2, -*sp, 1
move b3, -*sp, 1
move b4, -*sp, 1
move b7, -*sp, 1
move b8, -*sp, 1
move b9, -*sp, 1
move @_tile_src, b0, 1 ; SADDR: dictionary entry, index << 10 bits
; _tile_init_ -- program everything that does not vary between tiles.
;
; Spec 8.3's hoist, and it is worth real time in MAME: measured per-tile cost
; was ~0.36 ms against the unit's ~40 us, and the call did 11 I/O-register
; accesses plus 8 push/pop pairs for constants that never change. The registers
; this sets -- PSIZE, CONVSP, CONVDP, PMASK, CONTROL and B1 (SPTCH), B3 (DPTCH),
; B4 (OFFSET), B7 (DYDX), B8 (COLOR0), B9 (COLOR1) -- survive PIXBLT (the
; user's guide marks only SADDR/DADDR and B10-B14 as changed), so once per
; frame is enough.
;
; B8/B9 are NOT I/O registers: they are implied operands of every graphics
; instruction, and the resident OS leaves them arbitrary -- measured in the
; strip work: the first draw run painted every row black because nothing had
; set B9.
.globl _tile_init_
_tile_init_:
setf 32, 0, 1
movi 020h, b1 ; SPTCH = 32 bits = four bytes a tile row
move @_tile_daddr, b2, 1 ; DADDR (XY): (ty*32 << 16) | tx*32
movi 800h, b3 ; DPTCH = 2048 bits = 256-byte screen row
movi 04000000h, b4 ; OFFSET = VRAM base
movi 0200020h, b7 ; DYDX = (32 rows << 16) | 32 pixels
clr b8 ; COLOR0 = 0 (black)
movi 0FFFFh, b9 ; COLOR1 = all planes set (white)
; --- pixel format ------------------------------------------------------
movi 1, a1
move a1, @0C0000150h, 0 ; PSIZE = 1
movi 14h, a1
@@ -92,21 +99,31 @@ _tile_blit_:
move a1, @0C0000160h, 0 ; PMASK = 0 (all planes writable)
move a1, @0C0000170h, 0 ; upper half, for forward compatibility
; --- graphics operation: replace, no transparency, no window ----------
; raster op 0 (replace), CD/RM/RR preserved
move @0C00000B0h, a1, 0 ; CONTROL
andi 0801Fh, a1 ; keep CD (15) and RM/RR (2-4) only
andi 0801Fh, a1
move a1, @0C00000B0h, 0
move @0C00000B0h, a1, 0 ; read back: I/O writes are pipelined
rets
; _tile_blit_ -- one 32x32 tile, everything already programmed.
;
; Only B0 (SADDR) and B2 (DADDR) vary, and PIXBLT destroys both, so those two
; are the only registers this saves and reloads. tile_init_() must have run.
;
; Revert note: this file is new in Task 4 and nothing else calls it. To
; revert, delete it, the VRAMTILE entries in build-stream.sh and the
; tile_blit/paint_tiles code in vidstream.c; the strip path (vram_b.asm) does
; not reference any symbol defined here and is unaffected.
.globl _tile_blit_
_tile_blit_:
setf 32, 0, 1
move b0, -*sp, 1
move b2, -*sp, 1
move @_tile_src, b0, 1 ; SADDR: dictionary entry, index << 10 bits
move @_tile_daddr, b2, 1 ; DADDR (XY): (ty*32 << 16) | tx*32
pixblt b, xy
move *sp+, b9, 1
move *sp+, b8, 1
move *sp+, b7, 1
move *sp+, b4, 1
move *sp+, b3, 1
move *sp+, b2, 1
move *sp+, b1, 1
move *sp+, b0, 1
rets
+11
View File
@@ -12,5 +12,16 @@
"cwd": "."
}
}
},
"agents": {
"tms34010": {
"description": "Answers from the complete TI TMS34010 User's Guide (SPVU001A, 667 pages) preloaded in context. Use for cross-chapter synthesis, complete opcode lists, encoding/timing tables, or edition comparisons - questions that need the whole book rather than one grep hit.",
"mode": "all",
"system": "You answer questions about the TI TMS34010 Graphics System Processor. Your context is the complete TI TMS34010 User's Guide (SPVU001A, 1988-08, 667 pages), reproduced below in full.\n\nRules:\n- Cite the chapter and printed page when you quote, e.g. \"Section 12.6.6, page 12-27\".\n- Quote opcode encodings, bit-field layouts, register layouts and hex values verbatim. Never paraphrase a hex digit or synthesize an encoding you cannot see. If a passage is unreadable, say so.\n- The source is OCR'd from a scan, so tables, figures and timing diagrams are frequently mangled. Flag that explicitly and state what you can and cannot read instead of inventing missing cells.\n- If the book states something that contradicts general knowledge about the 34010, the book wins. Say when you are unsure whether a passage is a transcription artifact.\n- The 34010 is bit-addressed: addresses in the book are often bit addresses. Be explicit about which form you are quoting.\n\n===== BEGIN: TMS34010 USER'S GUIDE (SPVU001A) =====\n\n{file:docs/TMS34010-users-guide.txt}\n\n===== END: TMS34010 USER'S GUIDE (SPVU001A) =====",
"permissions": [
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "subagent", "resource": "*", "effect": "deny" }
]
}
}
}
+252
View File
@@ -0,0 +1,252 @@
# Running the TMS340 r5.01 toolchain under 2ine
This is the working **no-VM** route to TI's 1990 **TMS340 Family Code
Generation Tools r5.01**. The tools are 16-bit OS/2 1.x `NE` console programs;
the patched fork of [2ine](https://github.com/icculus/2ine) at
<https://git.wut.ee/opencode.energia.arti.ee/2ine-tms34010> runs them on a
64-bit Linux host.
Unlike the assembler-only state documented when the fork was first published,
the **full C pipeline works**: front end, optimizer, code generator,
assembler, linker and converter. `gspcl` (the one-command driver) still does
not chain its children reliably; drive the four C stages explicitly.
Verified 2026-11-01 on x86_64 Arch, against the VT1200 firmware image.
| tool | result |
|---|---|
| `gspac` (C front end) | **works** |
| `gspopt` (intermediate optimizer) | **works** |
| `gspcg` (code generator) | **works** |
| `gspa` (COFF assembler) | **works** — 11,852-line firmware disassembly, no errors |
| `gsplnk` (linker) | **works** — absolute placement; `rts.lib` + `flib.lib` |
| `gsprom` (COFF → Tek-hex) | **works** — plain and `-i` EPROM pair |
| `gspar` (archiver) | **works** — lists `rts.lib` members |
| `gspcl` (driver) | **partial** — runs `gspac`, then stops (see §7) |
---
## 1. Build 2ine
The fork's `main` carries the three fixes this needs: the null-selector
mapping (commit `91b78f2`), Arch/CMake-4 ncurses discovery (`a440128`), and
this document's status update (`3bcdfc3`).
Dependencies (Arch):
```sh
sudo pacman -S --needed base-devel cmake lib32-glibc lib32-gcc-libs \
lib32-ncurses sdl2-compat
```
Debian/Ubuntu names:
```sh
sudo apt install build-essential cmake libncurses-dev libncurses-dev:i386 \
libc6-dev-i386 gcc-multilib
```
Build 32-bit. On CMake 4.x the policy floor must be passed explicitly:
```sh
git clone https://git.wut.ee/opencode.energia.arti.ee/2ine-tms34010
cd 2ine-tms34010
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5
cmake --build build -j8
```
Result: `build/lx_loader` (32-bit), `build/lx_dump`, and `build/lib*.so`.
## 2. Extract the r5.01 tools
Disk 1 is the executables, disk 2 the headers, runtimes and library sources.
Extract them anywhere:
```sh
cd /path/to/vt1200
SDK="tools/tms34010-sdk/1990-11 TMS340 Family SDK"
mkdir -p os2-501/bin os2-501/lib os2-501/work
unzip -o "$SDK/1990-11-19 TMS340 Family Code Generation Tools (1 of 2) r5.01 rev2564062-1641.zip" -d os2-501/bin
unzip -o "$SDK/1990-11-19 TMS340 Family Code Generation Tools (2 of 2) r5.01 rev2564062-1641.zip" -d os2-501/lib
```
## 3. Prepare the run directory
`lx_loader` `dlopen()`s the native modules as `./lib<module>.so`, so the
working directory matters. The OS/2 tools also cannot open Unix absolute
paths, so keep sources, objects and libraries in the same directory and pass
**bare relative names**.
```sh
cd os2-501/work
ln -sf /path/to/2ine-tms34010/build/lib*.so .
cp ../bin/*.exe .
cp ../lib/rts.lib ../lib/flib.lib .
```
The tools consume their intermediate `.if` and `.opt` files on success — even
with `-k` — so regenerate them for every build.
## 4. End-to-end C build
Given `HELLO.C`:
```c
int twice(int n) { return n + n; }
int sum(int *p, int n) {
int s = 0;
int i;
for (i = 0; i < n; i++) s += p[i];
return s + twice(n);
}
```
Run each stage. `LX` is the absolute path to `lx_loader`:
```sh
LX=/path/to/2ine-tms34010/build/lx_loader
$LX ./gspac.exe -k HELLO.C HELLO.IF # C front end -> intermediate
$LX ./gspopt.exe -k -O2 HELLO.IF HELLO.OPT # optimizer -> intermediate
$LX ./gspcg.exe -m HELLO.OPT HELLO.ASM HELLO.TMP # codegen -> .asm
$LX ./gspa.exe HELLO.ASM # assembler -> HELLO.obj
$LX ./gsplnk.exe HELLO.obj rts.lib flib.lib \
-o hello.out -m hello.map # linker -> absolute COFF
$LX ./gsprom.exe hello.out hello.hex # converter -> Tek-hex
```
Notes:
* `gspa` writes the object as **lowercase** `HELLO.obj`, even though the
source is `HELLO.ASM`. Keep the names consistent on a case-sensitive FS.
* `-O2` means level 2. `-O0` is the lowest level, not "off"; with no `-o`/`-O`
stage at all the optimizer is not run.
* Link `flib.lib` as well as `rts.lib` when the code pulls in `_sqrt`,
`_frexp`, `_ldexp` and friends: the float primitives (`FD$ADD`, `FD$DIV`,
…) live in `flib.lib`.
### Placing the image at a TMS34010 bit address
`gsplnk` accepts a command file whose `SECTIONS` origin is a **bit address**.
COFF section sizes are in bits too (16 bits per instruction word): a `.text`
of `0x8000` is 4096 bytes, not 32 KB.
**Write the command file with CRLF**, even on Linux: with LF-only line ends the
linker merges the `-m` line into the next one and silently loses options.
```
-a
-o hello.out
-m hello.map
HELLO.obj
SECTIONS
{
.text 3F208000h: {}
}
```
```sh
$LX ./gsplnk.exe hello.cmd
$LX ./gsprom.exe -i hello.out hi lo # high/low EPROM pair
```
`0x3F208000` is the VT1200's firmware base. The converter's own hex record
addresses wrap at 64 KB; take placement from the COFF file or the map, never
from the hex.
## 5. Assembler / linker / converter on a firmware region
The 1987 round-trip flow in `harness/sdk-scratch/roundtrip.sh` works with the
5.01 tools, with one source-level change (`b15` → `sp`, §6):
```sh
# disassemble 32 KB at bit 3F208000 to RT.ASM, as roundtrip.sh does
# then, in the 2ine run directory:
$LX ./gspa.exe RT.ASM
$LX ./gsplnk.exe RT.CMD
$LX ./gsprom.exe -i rt.out rthi rtlo
# interleave low/high and compare with the original region
```
Result on `vt1200_v22.bin`, region `0x1000` / 32,768 bytes / base `3F208000`:
* **1987 GSPA:** 0 differing bytes,
`md5 43adb6669571041985f3e26fd9ff7cb9` (BYTE-EXACT)
* **5.01 GSPA:** 32,768/32,768 bytes, **1 differing byte** at `0x33ec`
(`3f` vs `2f`)
The difference is the A/B register-file namespace bit of the single
`rl 09h, sp` in the region, not a semantic difference (§6).
## 6. Why `b15` becomes `sp`, and why one byte differs
`dis2as34.py` emits `b15` for the B-file stack pointer because **GSPA 2.00**
accepts `b15` and uses a bare `sp` for the A-file form. **GSPA 5.01 rejects
`b15`** with `INVALID REGISTER VALUE`; its `sp` encodes the A15 namespace.
The TMS34010 has one stack-pointer register, addressable as **A15 or B15**, so
`rl 09h, sp` (5.01, `2f31`) and `rl 09h, b15` (2.00, `3f31`) address the same
register. The 5.01 object is therefore functionally identical to the ROM, but
not byte-identical at that instruction. This is the one exception to the
otherwise one-way assembler compatibility noted in `OS2-501.md`.
## 7. `gspcl` and the exit-status trap
`gspcl` finds `gspac` and runs it, then stops without invoking `gspopt` /
`gspcg` / `gspa`. The individual tools frequently exit non-zero after
successful work: `gspac` and `gspcg` return **139** (segfault during teardown),
`gsprom` can return 1. `gspcl` treats the child status as failure.
**Check for output files, not exit codes.** The manual pipeline in §4 produces
correct objects and images; the one-command driver does not yet.
## 8. What the null-selector fix was
The original fork asserted in `lxConvert1616to32` when the C front end passed
`0003:0000`. That is the OS/2 **null far pointer**: selector index 0 with RPL
3. The converter special-cased only `addr1616 == 0`, so it indexed `ldt[0]`
and asserted.
The fix in `lx_loader.c` is:
```c
const uint16 selector = (uint16) (addr1616 >> 19);
const uint16 offset = (uint16) (addr1616 % 0x10000);
if (selector == 0) // segment 0x0000 or 0x0003: the OS/2 null selector.
return NULL;
```
Segment index 0 is the null selector by definition, so returning `NULL` is
correct, not a workaround.
## 9. Measured results
Bench source with a sum loop, indexed loop, `switch`, struct update, helper
call and bit loop (`BENCH.C`), compiled four ways:
| build | `.text` | `.obj` | `.asm` |
|---|---:|---:|---:|
| no optimizer | 490 B | 950 B | 4396 B |
| `-O0` | 394 B | 854 B | 4497 B |
| `-O1` | 394 B | 854 B | 4499 B |
| **`-O2`** | **376 B** | **836 B** | **4236 B** |
`-O2` is ~23 % smaller than no optimizer on this source. Assembly of the
11,852-line firmware disassembly takes ~0.22 s wall through `lx_loader`
(loader startup included).
## 10. Reproduce
A compact end-to-end run (from `os2-501/work`, with the libs symlinked and the
tools copied as in §3):
```sh
LX=/path/to/2ine-tms34010/build/lx_loader
$LX ./gspac.exe -k HELLO.C HELLO.IF
$LX ./gspopt.exe -k -O2 HELLO.IF HELLO.OPT
$LX ./gspcg.exe -m HELLO.OPT HELLO.ASM HELLO.TMP
$LX ./gspa.exe HELLO.ASM
$LX ./gsplnk.exe HELLO.obj rts.lib flib.lib -o hello.out -m hello.map
$LX ./gsprom.exe hello.out hello.hex
```
+4 -3
View File
@@ -142,9 +142,10 @@ cd 2ine/build # lx_loader dlopen()s ./lib<module>.so, so cwd matters
./lx_dump gspa.exe # NE header/segments/imports
```
`gspcl` (the C compiler) does **not** work yet there: the `gspac` child hits
`lxConvert1616to32` with a selector 2ine has not mapped. Use 86Box when you need
to compile C.
The C pipeline **does** work under the patched fork: a null-selector fix in
`lx_loader.c` unblocks `gspac`, `gspopt` and `gspcg`. `gspcl` itself still
stops after `gspac` because the children exit non-zero at teardown, so drive
the stages explicitly. See `2INE-501.md` for the full end-to-end flow.
---