fix(tiles): early status on dictionary failure, decode-time field, chunk bound

This commit is contained in:
opencode.energia.arti.ee committed 2026-10-31 03:35:14 +00:00
1 parent b995c2caaa
commit a84f5f2811
11 files changed
+527 -68

No files matched your search

@@ -182,8 +182,9 @@ frame start, so a frame never straddles the ring wrap and slot ownership is expl
* **Dictionary**: 256 × 32×32 1-bpp (128 B each = 32 KB), built offline by k-means
over the video's tiles, uploaded in chunks, held in the startup allocation.
* **Frame**: header + 1,280 index bytes (40×32 grid, row major). `kind = TILES_FULL`.
* **Delta**: `kind = TILES_DELTA` + a changed-tile list (run-length list of tile
indices, 1–2 bytes per entry) and their new indices. Deltas are the normal case.
* **Delta**: `kind = TILES_DELTA` + a changed-tile list: `count u16`, then
`count` × (`pos u16`, `index u8`) little-endian, where `pos` is the row-major
tile position and `index` its new dictionary entry. Deltas are the normal case.
* Decoder: one cached loop; `PIXBLT B,XY` from `dict + index*128`, `DYDX=(32,32)`,
`SPTCH=32` (a 32-px row is **two** words), destination XY `(ty*32, tx*32)`.
* **Repair**: a `TILES_FULL` keyframe every 2 s (and on the host's first frame), so
+57 -2
View File
@@ -5274,9 +5274,64 @@ So the pixel verdict is not a rubber stamp; the host tests carry the rest of
the negative controls (one flipped dump bit, a stream that paints something
else, a missing dump, and each bad status counter).
### 76.5.1 The status body after the review: `dict_state` and `decode_worst`
The Task 6 review extended the status body from 12 to 16 bytes
(`frame.py`'s layout block and `send_status()` in `vidstream.c` are the two
halves of the contract). The first ten bytes are unchanged; the new fields are
| offset | field | meaning |
|---|---|---|
| 12 | `dict_state` u8 | bit 0 `DICT_READY` (the 32 KB is complete), bit 1 `DICT_BAD` (a chunk was refused); both can be set |
| 13 | reserved u8 | zero |
| 14 | `decode_worst` u16 | worst per-frame decode since the last status actually sent, in **REFCNT ticks** |
Two out-of-cadence reports now exist. The **first accepted frame** sends one,
so a host sees the payload running (and the dictionary state) before 64 frames
have arrived. **Every refused dictionary chunk** sends one too: before this,
one lost chunk made every later chunk fail, no frame was ever accepted, the
64-frame rule never fired, and the host could only report "payload not
running" instead of "dictionary incomplete". The cadence is measured from the
last report, so after the early report at frame 1 the periodic reports are 65,
129, 193 rather than 64, 128, 192.
`decode_worst` is the instrument for the plan's §12 risk (`B,XY` at
0.32 µs/word is MAME-only). It is **not** microseconds: `REFCNT` is the
TMS34010's own refresh counter, the one the firmware's POST reads and requires
to move; MAME ticks it every 16 CPU cycles with the low two bits masked, and
the real part's rate has not been measured on the unit, so turning ticks into
frame time is still M2 work. The payload samples it once per tile row (32
times a full frame), which is what keeps the 16-bit counter unambiguous across
a repaint longer than its wrap period.
Evidence from `VT1200_MODE=dict`:
```
VID: dict PASS: the refusal's early status reports frames_rx=0 (got 0)
VID: dict PASS: the refusal's early status sets DICT_BAD (dict_state=0x02)
VID: dict PASS: the refusal's early status does not claim DICT_READY (dict_state=0x02)
VID: dict PASS: the early status reports dict_state=0x03 (DICT_READY|DICT_BAD = 3)
VID: dict PASS: the early status reports the painted frame's decode time (decode_worst=37612 REFCNT ticks)
VID: dict PASS: a DICT_CHUNK after DICT_READY is refused and counted (DICT_BAD=4, expected 4)
```
and from `VT1200_MODE=stream`:
```
VID: status 1: frames_rx=1 crc_err=0 drops=0 csr0=01B3 heap_free_kb=2787 dict_state=00 decode_worst=0 kind=5 payload_len=16
VID: status 2: frames_rx=65 crc_err=0 drops=0 csr0=03B3 heap_free_kb=2787 dict_state=00 decode_worst=0 kind=5 payload_len=16
VID: status 5: frames_rx=200 crc_err=0 drops=0 csr0=03B3 heap_free_kb=2787 dict_state=00 decode_worst=0 kind=5 payload_len=16
vidstream_status_check: 5/5 status frames decoded, last reports frames_rx=200 drops=0 crc_err=0 heap_free_kb=2787
```
`decode_worst=0` in the stream run is correct, not missing: that mode delivers
strip frames, and only the tile paths (`paint_tiles`/`paint_delta`) carry the
instrument.
The other MAME modes still pass after the replay mode was added: `alloc`, `rx`,
`ring`, `dict`, `tiles` (1280/1280), `delta`, `stream` (4/4 status frames), and
the 200-frame boot baseline is unchanged (`samples=198`, 3 distinct PCs).
`ring`, `dict`, `tiles` (1280/1280), `delta`, `stream` (5/5 status frames,
including the early report on the first accepted frame), and the 200-frame boot
baseline is unchanged (`samples=198`, 3 distinct PCs).
### 76.6 The host sender in tile mode
+120 -18
View File
@@ -65,9 +65,11 @@
-- KIND_DICT_CHUNK (4) upload in 23 chunks with a duplicate-offset chunk, a
-- gapped-offset chunk and an overrunning tail chunk mixed in; then it reads the
-- 32 KB back from POOL_PTR + 2 * SLOT_BYTES and compares it with what was sent.
-- DICT_READY/DICT_BYTES/DICT_BAD are asserted after every chunk, so a
-- premature "ready" fails at the chunk that caused it. Verdict and evidence:
-- the printed checks and $DUMP_DIR/dict-status.txt.
-- One more chunk *after* DICT_READY must be refused (ruling R1: the dictionary
-- is fixed for the clip), and the 32 KB is compared again to show the refusal
-- wrote nothing. DICT_READY/DICT_BYTES/DICT_BAD are asserted after every
-- chunk, so a premature "ready" fails at the chunk that caused it. Verdict and
-- evidence: the printed checks and $DUMP_DIR/dict-status.txt.
--
-- Mode "tiles" (Task 4) is the tile blit's pixel check. It uploads a 32 KB
-- dictionary whose every 128-byte entry is a distinct, row-varying pattern,
@@ -174,8 +176,8 @@ elseif MODE == "ring" and not os.getenv("VID_END") then
-- Three phases, each with its own bounded wait (VID_RING_WAIT, 300).
DEFAULT_END = AT + 3 * tonumber(os.getenv("VID_RING_WAIT") or "300") + 120
elseif MODE == "dict" and not os.getenv("VID_END") then
-- Five phases, each with its own bounded wait (VID_DICT_WAIT, 300).
DEFAULT_END = AT + 5 * tonumber(os.getenv("VID_DICT_WAIT") or "300") + 120
-- Six phases, each with its own bounded wait (VID_DICT_WAIT, 300).
DEFAULT_END = AT + 6 * tonumber(os.getenv("VID_DICT_WAIT") or "300") + 120
elseif MODE == "tiles" and not os.getenv("VID_END") then
-- One wait for the ring, 23 for the dictionary chunks, one for the tile
-- frame, each bounded (VID_TILE_WAIT, 300); the first timeout dumps and
@@ -906,12 +908,14 @@ local function capture_status()
drops = le16(t, STATUS_BODY + 6),
csr0 = le16(t, STATUS_BODY + 8),
heap_kb = le16(t, STATUS_BODY + 10),
dict_state = t[STATUS_BODY + 12],
decode_worst = le16(t, STATUS_BODY + 14),
}
-- Only a well-formed VS status frame counts. A payload without the
-- status path leaves ST_SENT/ST_BUF as whatever RAM held, so the first
-- "capture" would otherwise be garbage presented as a report -- and
-- "no status frames" is exactly what a Task 5 tree must say.
if not rec.magic_ok or rec.kind ~= 5 or rec.payload_len ~= 12 then
if not rec.magic_ok or rec.kind ~= 5 or rec.payload_len ~= 16 then
bad_captures = bad_captures + 1
print(string.format("VID: status capture %d is not a VS status frame (magic=%s kind=%d payload_len=%d); ignored",
bad_captures, rec.magic_ok and "VS" or "WRONG", rec.kind,
@@ -919,9 +923,10 @@ local function capture_status()
return
end
captures[#captures + 1] = rec
print(string.format("VID: status %d: frames_rx=%d crc_err=%d drops=%d csr0=%04X heap_free_kb=%d kind=%d payload_len=%d",
print(string.format("VID: status %d: frames_rx=%d crc_err=%d drops=%d csr0=%04X heap_free_kb=%d dict_state=%02X decode_worst=%d kind=%d payload_len=%d",
#captures, rec.frames_rx, rec.crc_err, rec.drops, rec.csr0,
rec.heap_kb, rec.kind, rec.payload_len))
rec.heap_kb, rec.dict_state, rec.decode_worst, rec.kind,
rec.payload_len))
end
-- Dump the captured status frames and the run's counters, then end the run.
@@ -951,6 +956,9 @@ local function dump_stream(timed_out)
lines[#lines + 1] = string.format("LAST_DROPS=%d", last.drops)
lines[#lines + 1] = string.format("LAST_CSR0=%d", last.csr0)
lines[#lines + 1] = string.format("LAST_HEAP_KB=%d", last.heap_kb)
lines[#lines + 1] = string.format("LAST_DICT_STATE=%d", last.dict_state)
lines[#lines + 1] = string.format("LAST_DECODE_WORST=%d",
last.decode_worst)
lines[#lines + 1] = string.format("LAST_KIND=%d", last.kind)
lines[#lines + 1] = string.format("LAST_PAYLOAD_LEN=%d", last.payload_len)
end
@@ -1606,6 +1614,33 @@ local function dict_step()
dict_check(prog:read_u32(DICT_READY) == dict_last.ready,
string.format("%s: DICT_READY=%d (expected %d)",
dict_last.label, prog:read_u32(DICT_READY), dict_last.ready))
-- The first refusal (the duplicate-offset chunk) must produce an
-- out-of-cadence status: no frame has been accepted, so the
-- 64-frame rule would never fire and the host would see silence
-- instead of "dictionary incomplete". ST_SENT moves only after
-- send_status() hands the frame to the chip.
if dict_last.bad == 1 and prog:read_u32(ST_SENT) == 0 then
if frames >= dict_phase_at + DICT_WAIT then
dict_check(false,
"a refused dictionary chunk produced an early status (ST_SENT=0)")
dict_dump(true)
return true
end
return false
end
if dict_last.bad == 1 then
after = select(2, read_bytes(prog:read_u32(ST_BUF)
+ STATUS_BODY * 8, 16))
dict_check(le32(after, 0) == 0,
string.format("the refusal's early status reports frames_rx=0 (got %d)",
le32(after, 0)))
dict_check((after[12] & 2) ~= 0,
string.format("the refusal's early status sets DICT_BAD (dict_state=0x%02X)",
after[12]))
dict_check((after[12] & 1) == 0,
string.format("the refusal's early status does not claim DICT_READY (dict_state=0x%02X)",
after[12]))
end
dict_last = nil
end
if dict_at >= #dict_plan then
@@ -1648,22 +1683,89 @@ local function dict_step()
end
if dict_phase == 4 then
if not dict_settled() then
if dict_last ~= nil then
if not dict_settled() then
if frames >= dict_phase_at + DICT_WAIT then
dict_check(false, "the post-dictionary TILES_FULL frame was consumed")
dict_dump(true)
return true
end
return false
end
dict_check(prog:read_u32(DICT_READY) == 1
and prog:read_u32(DICT_BYTES) == DICT_SIZE,
"a TILES_FULL after DICT_READY leaves the dictionary record alone")
dict_lines[#dict_lines + 1] = string.format(
"INFO post-dictionary TILES_FULL: TILES_DROP=%d RX_OK=%d (Task 4 paints it; the pixels are tiles mode's check)",
prog:read_u32(TILES_DROP), prog:read_u32(RX_OK))
print(string.format("VID: dict: post-dictionary TILES_FULL: TILES_DROP=%d RX_OK=%d (Task 4 paints it; pixels are tiles mode's check)",
prog:read_u32(TILES_DROP), prog:read_u32(RX_OK)))
dict_last = nil
end
-- The first accepted frame forces an early status report. Waiting
-- for ST_FRAMES to say 1 (send_status publishes it after the chip
-- takes the frame) is also the check on that path, and the body now
-- carries dict_state and the worst decode time -- the latter must be
-- nonzero because a TILES_FULL was just painted.
if prog:read_u32(ST_FRAMES) ~= 1 then
if frames >= dict_phase_at + DICT_WAIT then
dict_check(false, "the post-dictionary TILES_FULL frame was consumed")
dict_check(false, string.format(
"the first accepted frame produced an early status (ST_FRAMES=%d, RX_OK=%d)",
prog:read_u32(ST_FRAMES), prog:read_u32(RX_OK)))
dict_dump(true)
return true
end
return false
end
dict_check(prog:read_u32(DICT_READY) == 1
and prog:read_u32(DICT_BYTES) == DICT_SIZE,
"a TILES_FULL after DICT_READY leaves the dictionary record alone")
dict_lines[#dict_lines + 1] = string.format(
"INFO post-dictionary TILES_FULL: TILES_DROP=%d RX_OK=%d (Task 4 paints it; the pixels are tiles mode's check)",
prog:read_u32(TILES_DROP), prog:read_u32(RX_OK))
print(string.format("VID: dict: post-dictionary TILES_FULL: TILES_DROP=%d RX_OK=%d (Task 4 paints it; pixels are tiles mode's check)",
prog:read_u32(TILES_DROP), prog:read_u32(RX_OK)))
after = select(2, read_bytes(prog:read_u32(ST_BUF)
+ (STATUS_BODY + 12) * 8, 4))
dict_check(after[0] == 3,
string.format("the early status reports dict_state=0x%02X (DICT_READY|DICT_BAD = 3)",
after[0]))
dict_check((after[2] | (after[3] << 8)) > 0,
string.format("the early status reports the painted frame's decode time (decode_worst=%d REFCNT ticks)",
after[2] | (after[3] << 8)))
-- Plan 2 ruling R1: the dictionary is fixed for the clip, so a chunk
-- that arrives after DICT_READY is refused, not applied. This is the
-- case Task 3's review carried: a late or duplicated upload must not
-- silently rewrite the picture's vocabulary. dict_at is no longer
-- needed as the plan cursor, so it holds the pre-refusal DICT_BAD.
dict_at = prog:read_u32(DICT_BAD)
a = { f = dict_chunk_frame(0, DICT_ENTRY), n = 16 + 2 + DICT_ENTRY,
label = "chunk at offset 0 after DICT_READY (must be refused)" }
if not dict_take(a) then
dict_check(false, "the ring accepted the post-DICT_READY DICT_CHUNK")
dict_dump(true)
return true
end
dict_phase = 5
dict_phase_at = frames
return false
end
if dict_phase == 5 then
if not dict_settled() then
if frames >= dict_phase_at + DICT_WAIT then
dict_check(false, "the post-DICT_READY DICT_CHUNK was consumed")
dict_dump(true)
return true
end
return false
end
dict_check(prog:read_u32(DICT_BAD) == dict_at + 1,
string.format("a DICT_CHUNK after DICT_READY is refused and counted (DICT_BAD=%d, expected %d)",
prog:read_u32(DICT_BAD), dict_at + 1))
dict_check(prog:read_u32(DICT_BYTES) == DICT_SIZE,
string.format("the refused late chunk left DICT_BYTES alone (DICT_BYTES=%d)",
prog:read_u32(DICT_BYTES)))
dict_check(prog:read_u32(DICT_READY) == 1,
"the refused late chunk left DICT_READY 1")
-- The bytes, not just the counters: a refused chunk that wrote first
-- and counted after would show up here.
okv, first, why = dict_verify()
dict_check(okv, why or string.format(
"the refused late chunk did not rewrite the dictionary (first bad byte %d)",
first))
dict_dump(false)
return true
end
+30 -14
View File
@@ -21,10 +21,13 @@ cannot drift apart without a failure here. It checks:
the stream's ethertype 0x88B7);
* the VS header (magic, version, KIND_STATUS, rows/stretch zero, payload_len
12, header CRC valid);
* the 12-byte body (frames_rx, crc_err, drops, csr0, heap_free_kb);
* the cadence and the verdict the brief asks for: at least three status
frames, the periodic ones at 64, 128, 192, and the last reporting
frames_rx == 200 with drops == 0 and crc_err == 0;
* the 16-byte body (frames_rx, crc_err, drops, csr0, heap_free_kb,
dict_state, decode_worst);
* the cadence and the verdict the brief asks for: the early report on the
first accepted frame (frames_rx == 1), then the periodic ones 64 accepted
frames later (65, 129, 193 -- the cadence is measured from the last
report), and the last reporting frames_rx == 200 with drops == 0 and
crc_err == 0;
* that the transmit descriptor actually completed (TMD1's OWN bit clear).
TMD1's ERR bit is expected to be set here and is not a failure: the LANCE
has no network medium in MAME (ruling R11), so the chip reports loss of
@@ -216,9 +219,10 @@ def main():
"what decides whether the chip ever took it" % at_send)
# --- the bytes -------------------------------------------------------
if not check(len(records) >= 3,
"%d status frames, expected at least 3 (every %d accepted "
"frames)" % (len(records), STATUS_EVERY)):
if not check(len(records) >= 4,
"%d status frames, expected at least 4 (the first accepted "
"frame plus three periodic reports every %d accepted frames)"
% (len(records), STATUS_EVERY)):
print("vidstream_status_check: FAIL")
return 1
@@ -229,20 +233,32 @@ def main():
continue
bodies.append(body)
print(" frame %d: frame_no=%d frames_rx=%d crc_err=%d drops=%d "
"csr0=0x%04X heap_free_kb=%d"
"csr0=0x%04X heap_free_kb=%d dict_state=0x%02X "
"decode_worst=%d"
% (i, body["frame_no"], body["frames_rx"], body["crc_err"],
body["drops"], body["csr0"], body["heap_free_kb"]))
body["drops"], body["csr0"], body["heap_free_kb"],
body["dict_state"], body["decode_worst"]))
if len(bodies) != len(records):
print("vidstream_status_check: FAIL (%d of %d frames decoded)"
% (len(bodies), len(records)))
return 1
# --- the cadence -----------------------------------------------------
for i, want in enumerate((STATUS_EVERY, 2 * STATUS_EVERY, 3 * STATUS_EVERY)):
# The first accepted frame forces an early report, so the periodic ones
# start at index 1. That early frame is also the only place the stream
# path can show dict_state: no dictionary is uploaded here, so it is 0.
check(bodies[0]["frames_rx"] == 1,
"the first status reports frames_rx=%d, expected the early report "
"on the first accepted frame (1)" % (bodies[0]["frames_rx"],))
check(bodies[0]["dict_state"] == 0,
"the early status reports dict_state=0x%02X, expected 0 (no "
"dictionary in a strip stream)" % (bodies[0]["dict_state"],))
for i, want in enumerate((STATUS_EVERY + 1, 2 * STATUS_EVERY + 1,
3 * STATUS_EVERY + 1), start=1):
if i < len(bodies):
check(bodies[i]["frames_rx"] == want,
"status %d reports frames_rx=%d, expected %d (a report "
"every %d accepted frames)"
"every %d accepted frames after the one at frame 1)"
% (i + 1, bodies[i]["frames_rx"], want, STATUS_EVERY))
for i in range(1, len(bodies)):
check(bodies[i]["frames_rx"] > bodies[i - 1]["frames_rx"],
@@ -250,9 +266,9 @@ def main():
"the previous %d" % (i + 1, bodies[i]["frames_rx"],
bodies[i - 1]["frames_rx"]))
# After RX_EOF the terminal reports each further accepted frame, so the
# reports after the third are consecutive. That is how the final count
# (200) is reached without waiting for 256.
after = bodies[3:]
# reports after the third periodic one are consecutive. That is how the
# final count (200) is reached without waiting for 256.
after = bodies[4:]
for i in range(1, len(after)):
check(after[i]["frames_rx"] == after[i - 1]["frames_rx"] + 1,
"post-EOF report %d jumps from %d to %d; the terminal reports "
+53 -9
View File
@@ -38,7 +38,7 @@ The status body
A ``KIND_STATUS`` frame is the terminal's report to the host, and it is an
ordinary frame: the same 16-byte header, the same header-only CRC, with
``rows`` and ``stretch`` zero (a status has no geometry). Its body is 12 bytes:
``rows`` and ``stretch`` zero (a status has no geometry). Its body is 16 bytes:
offset size field
------ ---- ------------------------------------------------------
@@ -54,6 +54,31 @@ ordinary frame: the same 16-byte header, the same header-only CRC, with
8 2 csr0, u16 little-endian (the chip's status word when the
report was built)
10 2 heap_free_kb, u16 little-endian (the terminal's free heap)
12 1 dict_state, u8: bit 0 ``DICT_READY`` (the 32 KB dictionary
is complete), bit 1 ``DICT_BAD`` (at least one chunk was
refused). Both can be set: a chunk refused after the
dictionary became ready leaves it usable.
13 1 reserved, zero
14 2 decode_worst, u16 little-endian (the worst per-frame decode
time since the last status frame that was actually sent,
in TMS34010 REFCNT ticks -- *not* microseconds; the unit
conversion is still to be measured on the real machine)
The out-of-cadence reports
----------------------------
A status is sent every 64 accepted frames, but two early reports exist so the
host is never left with silence:
* the **first** accepted frame -- it proves the payload is running and carries
``dict_state`` before the first 64 frames have arrived;
* **immediately after a refused dictionary chunk** -- one lost chunk makes
every later chunk fail and no frame is ever accepted, so the cadence would
never fire and a host would report "payload not running" instead of
"dictionary incomplete".
``decode_worst`` is reset only when a status frame has actually been handed to
the chip, so a failed transmit does not lose the measurement.
Both ends put that frame in an Ethernet frame: 6 bytes destination, 6 bytes
source, the ethertype below, then the ``VS`` frame. The terminal does not
@@ -68,6 +93,7 @@ __all__ = [
"HEADER_SIZE", "MAGIC", "VERSION", "CRC_OFFSET",
"KINDS", "KIND_STRIP", "KIND_TILES_FULL", "KIND_TILES_DELTA",
"KIND_DICT_CHUNK", "KIND_STATUS",
"DICT_STATE_READY", "DICT_STATE_BAD",
"ETHERTYPE", "BROADCAST", "TERMINAL_MAC", "MIN_ETHERNET_PAYLOAD",
"STATUS_SIZE",
"crc16", "pack_header", "seal_header", "unpack_header",
@@ -130,14 +156,19 @@ TERMINAL_MAC = b"\x08\x00\x2b\x16\x4b\x5a"
MIN_ETHERNET_PAYLOAD = 46
#: Size of the status body (see the module docstring).
STATUS_SIZE = 12
STATUS_SIZE = 16
#: ``dict_state`` bits: the upload completed / a chunk was refused.
DICT_STATE_READY = 0x01
DICT_STATE_BAD = 0x02
#: Everything before the CRC field, little-endian.
_PREFIX = struct.Struct("<2sBBHHBBH")
#: The CRC field and the two padding bytes behind it.
_TAIL = struct.Struct("<HH")
#: The status body: frames_rx u32, crc_err u16, drops u16, csr0 u16, heap u16.
_STATUS = struct.Struct("<IHHHH")
#: The status body: frames_rx u32, four u16, dict_state u8, reserved u8,
#: decode_worst u16.
_STATUS = struct.Struct("<IHHHHBBH")
def _make_crc_table():
@@ -280,35 +311,48 @@ def unpack_header(buf: bytes, payload: bytes | None = None) -> dict:
# --- the status body and the Ethernet envelope ---------------------------
def pack_status(frames_rx: int, crc_err: int, drops: int, csr0: int,
heap_free_kb: int) -> bytes:
"""Build the 12-byte status body the terminal sends (see the docstring).
heap_free_kb: int, dict_state: int = 0,
decode_worst: int = 0) -> bytes:
"""Build the 16-byte status body the terminal sends (see the docstring).
The field widths are the terminal's: ``frames_rx`` is a 32-bit counter that
must not wrap in any run, the rest are 16-bit quantities whose limits
(65,535 missed packets, a 64 MB heap) are far above anything a VT1200
reaches, so an out-of-range value is a bug here rather than a wrap on the
wire. ``send_status()`` in vidstream.c writes these same offsets.
``dict_state`` is the two-bit set :data:`DICT_STATE_READY` /
:data:`DICT_STATE_BAD`; ``decode_worst`` is in REFCNT ticks, not
microseconds (the terminal-side comment says why).
"""
frames_rx = _checked("frames_rx", frames_rx, 0xFFFFFFFF)
crc_err = _checked("crc_err", crc_err, 0xFFFF)
drops = _checked("drops", drops, 0xFFFF)
csr0 = _checked("csr0", csr0, 0xFFFF)
heap_free_kb = _checked("heap_free_kb", heap_free_kb, 0xFFFF)
return _STATUS.pack(frames_rx, crc_err, drops, csr0, heap_free_kb)
dict_state = _checked("dict_state", dict_state, 0xFF)
decode_worst = _checked("decode_worst", decode_worst, 0xFFFF)
return _STATUS.pack(frames_rx, crc_err, drops, csr0, heap_free_kb,
dict_state, 0, decode_worst)
def unpack_status(body: bytes) -> dict:
"""Decode a status body, or raise ValueError if it is not 12 bytes."""
"""Decode a status body, or raise ValueError if it is not 16 bytes."""
if len(body) != STATUS_SIZE:
raise ValueError("a status body is %d bytes, got %d"
% (STATUS_SIZE, len(body)))
frames_rx, crc_err, drops, csr0, heap_free_kb = _STATUS.unpack_from(body, 0)
(frames_rx, crc_err, drops, csr0, heap_free_kb, dict_state, _reserved,
decode_worst) = _STATUS.unpack_from(body, 0)
return {
"frames_rx": frames_rx,
"crc_err": crc_err,
"drops": drops,
"csr0": csr0,
"heap_free_kb": heap_free_kb,
"dict_state": dict_state,
"dict_ready": bool(dict_state & DICT_STATE_READY),
"dict_bad": bool(dict_state & DICT_STATE_BAD),
"decode_worst": decode_worst,
}
+3 -2
View File
@@ -300,8 +300,9 @@
#define ST_INIT MARK_CELL(0x8c)
#define ST_TMLAST MARK_CELL(0x90)
/* The status body (Task 6): frames_rx u32, then four u16, little-endian. */
#define FP_STATUS_SIZE 12
/* The status body (Task 6): frames_rx u32, four u16, dict_state u8,
* reserved u8, decode_worst u16, little-endian. */
#define FP_STATUS_SIZE 16
/* What a valid frame told us. payload_off is FP_HEADER_SIZE on success. */
typedef struct {
+23 -6
View File
@@ -21,7 +21,8 @@ import pytest
from harness.udpstream.frame import (CRC_OFFSET, HEADER_SIZE, KIND_STATUS,
KIND_STRIP, KINDS, MAGIC, VERSION,
STATUS_SIZE,
STATUS_SIZE, DICT_STATE_READY,
DICT_STATE_BAD,
BROADCAST, ETHERTYPE, TERMINAL_MAC,
crc16, ethernet_frame, pack_header,
pack_status, seal_header, unpack_header,
@@ -204,30 +205,46 @@ def test_out_of_range_fields_raise(kwargs):
# --- the status body that travels back (Task 6) --------------------------
def test_status_body_is_twelve_bytes_little_endian():
def test_status_body_is_sixteen_bytes_little_endian():
"""The terminal's status body, field by field, at the offsets the C writes.
The C side (vidstream.c, send_status()) writes the same five fields with
The C side (vidstream.c, send_status()) writes the same seven fields with
byte stores, so the offsets below are the contract between the two ends.
"""
b = pack_status(frames_rx=0x12345678, crc_err=0x0102, drops=0x0304,
csr0=0x20F2, heap_free_kb=0x0506)
assert len(b) == STATUS_SIZE == 12
csr0=0x20F2, heap_free_kb=0x0506, dict_state=0x03,
decode_worst=0x0A0B)
assert len(b) == STATUS_SIZE == 16
assert b[0:4] == b"\x78\x56\x34\x12" # frames_rx, u32 LE
assert b[4:6] == b"\x02\x01" # crc_err, u16 LE
assert b[6:8] == b"\x04\x03" # drops, u16 LE
assert b[8:10] == b"\xF2\x20" # csr0, u16 LE
assert b[10:12] == b"\x06\x05" # heap_free_kb, u16 LE
assert b[12:13] == b"\x03" # dict_state, u8
assert b[13:14] == b"\x00" # reserved
assert b[14:16] == b"\x0B\x0A" # decode_worst, u16 LE
assert unpack_status(b) == {
"frames_rx": 0x12345678, "crc_err": 0x0102, "drops": 0x0304,
"csr0": 0x20F2, "heap_free_kb": 0x0506,
"dict_state": 0x03, "dict_ready": True, "dict_bad": True,
"decode_worst": 0x0A0B,
}
def test_status_dict_state_is_a_two_bit_set():
"""DICT_READY and DICT_BAD can both be set: a late refusal is not a loss."""
ready = unpack_status(pack_status(1, 0, 0, 0, 0,
dict_state=DICT_STATE_READY))
assert ready["dict_ready"] is True and ready["dict_bad"] is False
bad = unpack_status(pack_status(1, 0, 0, 0, 0, dict_state=DICT_STATE_BAD))
assert bad["dict_ready"] is False and bad["dict_bad"] is True
@pytest.mark.parametrize("kwargs", [
dict(frames_rx=0x100000000), dict(frames_rx=-1),
dict(crc_err=0x10000), dict(drops=-1),
dict(csr0=0x10000), dict(heap_free_kb=-1),
dict(dict_state=0x100), dict(decode_worst=-1),
])
def test_status_fields_out_of_range_raise(kwargs):
base = dict(frames_rx=1, crc_err=0, drops=0, csr0=0, heap_free_kb=0)
@@ -236,7 +253,7 @@ def test_status_fields_out_of_range_raise(kwargs):
pack_status(**base)
def test_status_body_must_be_exactly_twelve_bytes():
def test_status_body_must_be_exactly_sixteen_bytes():
b = pack_status(1, 0, 0, 0, 0)
with pytest.raises(ValueError):
unpack_status(b[:-1])
+35
View File
@@ -206,6 +206,31 @@ def test_a_short_last_datagram_is_padded_and_a_mid_one_is_not():
assert frame[1296:] == bytes(5) # the pad, and only at the end
def test_a_chunk_smaller_than_the_ring_can_hold_is_rejected():
"""A whole strip frame split over more than 16 datagrams can never arrive.
``--chunk 1000`` puts a 128-row frame (20,496 B) in 21 datagrams; the
terminal's RX ring holds 16, so the payload would drop the frame. The
constructor refuses it and names the smallest legal chunk.
"""
clock = FakeClock()
transport = RecordingTransport(clock)
with pytest.raises(ValueError) as e:
make_sender(clock, transport, rows=128, stretch=8, chunk=1000)
assert "at least 1281" in str(e.value)
assert "16-descriptor" in str(e.value)
def test_a_chunk_that_fits_the_ring_is_accepted():
"""The boundary: 1,281 B * 16 = 20,496, exactly the 128-row frame."""
clock = FakeClock()
transport = RecordingTransport(clock)
sender = make_sender(clock, transport, rows=128, stretch=8, chunk=1281,
frames=1)
sender.run()
assert len(transport.datagrams) == 16
def test_frame_numbers_wrap_at_16_bits_and_bodies_differ_per_frame():
"""frame_no is a u16 field; the pattern must move or a stream is not one."""
assert vidsend.bars(8, 0) != vidsend.bars(8, 1)
@@ -419,6 +444,16 @@ def tile_frames(datagrams):
return [d[14:] for d in datagrams]
def test_tile_mode_rejects_a_chunk_too_small_for_a_full_screen_delta(tile_cache):
"""A 1,280-tile delta is 3,858 B; at 200 B/datagram that is 20 > 16."""
clock = FakeClock()
transport = RecordingTransport(clock)
with pytest.raises(ValueError) as e:
make_tile_sender(tile_cache, clock, transport, chunk=200)
assert "at least 242" in str(e.value)
assert "full-screen delta" in str(e.value)
def test_tile_mode_uploads_the_dictionary_before_the_first_frame(tile_cache):
clock = FakeClock()
transport = RecordingTransport(clock)
+8 -3
View File
@@ -269,9 +269,14 @@ def keyframe_due(frame_no: int, drops: int = 0, seen_drops: int = 0,
cumulative, so any increase means at least one frame never reached the
screen and the host's idea of the screen is stale.
``crc_err`` is deliberately not a trigger: a refused frame never painted,
so it cannot have desynchronised anything, and the cadence repairs a drop
the terminal never counted.
``crc_err`` is deliberately not a trigger. A refused delta *does* leave
the terminal's screen behind the host's -- the host has moved to the next
frame and the terminal never painted that one -- so the reason is not that
nothing was desynchronised. It is that a fixed 60-frame cadence already
bounds how long that desync lasts, and a refused frame carries no screen
state a keyframe could recover sooner than the schedule does. ``drops`` is
different: it counts frames the terminal *accepted* and then lost, which is
the counter the terminal raises for the same repair.
"""
if (not isinstance(frame_no, int) or isinstance(frame_no, bool)
or frame_no < 0):
+57 -2
View File
@@ -62,9 +62,10 @@ from harness.udpstream.frame import ( # noqa: E402
BROADCAST, ETHERTYPE, HEADER_SIZE, KIND_STATUS, MAGIC, STATUS_SIZE,
ethernet_frame, pack_header, unpack_header, unpack_status)
from harness.udpstream.tiledict import ( # noqa: E402
KEYFRAME_EVERY, delta_frame, dict_chunks, keyframe_due)
DELTA_HDR, KEYFRAME_EVERY, TILES_PER_SCREEN, delta_frame,
dict_chunks, keyframe_due)
from harness.udpstream.vidsrc import ( # noqa: E402
CHUNK, MAX_ROWS, ROW_BYTES, StripSource)
CHUNK, MAX_ROWS, RING_DATAGRAMS, ROW_BYTES, StripSource)
__all__ = [
"CSV_HEADER", "CSV_HEADER_TILES", "Sender", "TileSender", "bars", "split",
@@ -127,6 +128,24 @@ def split(rows: int, frame_no: int) -> bytes:
PATTERNS = {"bars": bars, "split": split}
#: The largest VS frame the tile path can put on the wire: a delta that names
#: every tile is ``count u16`` plus 3 bytes per entry, the most the format can
#: carry. The dictionary goes out as one-datagram chunks and never uses
#: ``--chunk``.
MAX_TILE_FRAME = HEADER_SIZE + DELTA_HDR + 3 * TILES_PER_SCREEN
def min_chunk(frame_bytes: int) -> int:
"""Smallest ``--chunk`` whose frame fits the terminal's RX ring.
The ring holds :data:`RING_DATAGRAMS` descriptors and cannot receive a frame
that needs more; the payload drops it and counts ``RX_RING_DROPS``. One
frame of ``frame_bytes`` split at ``chunk`` needs ``ceil(frame_bytes /
chunk)`` datagrams, so the smallest legal chunk is the ceiling of the frame
size over the ring size.
"""
return (frame_bytes + RING_DATAGRAMS - 1) // RING_DATAGRAMS
# --- the status back-channel ---------------------------------------------
@@ -184,6 +203,14 @@ class Sender:
if fps < 0:
raise ValueError("fps must be >= 0 (0 = unpaced): %r" % (fps,))
self.source = StripSource(rows, stretch)
need = min_chunk(HEADER_SIZE + rows * ROW_BYTES)
if chunk < need:
raise ValueError(
"chunk %d would split a %d-row frame (%d bytes) across more "
"than the terminal's %d-descriptor RX ring; chunk must be at "
"least %d"
% (chunk, rows, HEADER_SIZE + rows * ROW_BYTES,
RING_DATAGRAMS, need))
self.pattern = pattern
self.rows = rows
self.stretch = stretch
@@ -373,6 +400,13 @@ class TileSender:
every=KEYFRAME_EVERY, out=None, err=None):
if chunk < HEADER_SIZE + 1:
raise ValueError("chunk %d is too small for a frame" % chunk)
need = min_chunk(MAX_TILE_FRAME)
if chunk < need:
raise ValueError(
"chunk %d would split a full-screen delta (%d bytes) across "
"more than the terminal's %d-descriptor RX ring; chunk must "
"be at least %d"
% (chunk, MAX_TILE_FRAME, RING_DATAGRAMS, need))
if fps < 0:
raise ValueError("fps must be >= 0 (0 = unpaced): %r" % (fps,))
if not cache.indices:
@@ -727,6 +761,27 @@ def main(argv=None) -> int:
report_every = (args.report_every if args.report_every is not None
else REPORT_EVERY)
# A frame split across more datagrams than the terminal's RX ring holds can
# never be received: the ring runs out of descriptors and the payload drops
# the partial frame. This is worst in tile mode (a delta naming every tile
# is 3,858 bytes) and for a wide strip frame at a small --chunk; reject it
# here, where the limit can be named, rather than on the wire.
if cache is not None:
frame_bytes = MAX_TILE_FRAME
what = "a full-screen delta"
else:
frame_bytes = HEADER_SIZE + args.rows * ROW_BYTES
what = "a %d-row frame" % args.rows
need = min_chunk(frame_bytes)
if args.chunk < need:
print("vidsend.py: --chunk %d would split %s (%d bytes) across %d "
"datagrams; the terminal's RX ring holds %d, so --chunk must be "
"at least %d"
% (args.chunk, what, frame_bytes,
(frame_bytes + args.chunk - 1) // args.chunk,
RING_DATAGRAMS, need), file=sys.stderr)
return 2
def build_sender(transport, receive, src):
if cache is not None:
return TileSender(cache, transport, src=src, receive=receive,
+138 -10
View File
@@ -107,6 +107,32 @@
#define HEAP_FREE 0x1E8BA0 /* firmware free-byte counter, IN BITS */
#define MARK 0x01A00000 /* .text must end before this bit address */
/*
* The decode-time instrument (plan 2, the 12 risk: B,XY at 0.32 us/word is
* MAME-only). REFCNT is the TMS34010's own refresh counter, an I/O register
* at bit 0xC00001F0 -- the same one the firmware's POST reads twice and
* requires to have moved (harness/mame/BRINGUP_LOG.md, the REFCNT fix). It is
* a real register, not a harness invention, and reading it costs a MOVE.
*
* What it is NOT: microseconds. MAME returns (total_cycles()/16) & 0xfffc,
* i.e. a tick every 64 CPU cycles with the low two bits masked, and the real
* part's DRAM-refresh rate has not been measured on the unit. The status
* field therefore carries REFCNT ticks, and the unit conversion is still the
* M2 measurement the plan asks for. The counter is 16 bits and wraps; every
* sample below extends the difference modulo 65,536. Stock MAME advances it
* by 4 every 64 cycles (a 21 ms wrap at 50 MHz); the documented local REFCNT
* patch advances it every cycle (a 1.3 ms wrap). A tile row is well under one
* period in either build, so sampling once per row -- and every 64 entries in
* the delta validation pass -- keeps the extension unambiguous. If a future
* payload cannot read REFCNT, it must report that instead of substituting a
* made-up clock.
*/
#define REFCNT_REG 0xC00001F0 /* TMS34010 refresh counter, 16 bits */
static unsigned int decode_mark; /* REFCNT at the last sample */
static unsigned int decode_ticks; /* this frame so far, wrap-extended */
static unsigned int decode_worst; /* worst since the last status sent */
#define STATIC_BYTES 249856 /* the designed working set, 244 KB */
/* Screen memory and the strip geometry (see the header comment). */
@@ -382,6 +408,13 @@ static unsigned int dict_bad;
static unsigned int dict_ready;
static unsigned int tiles_drop;
/* Set by dict_refuse(); main() turns it into an out-of-cadence status frame.
* That early report is the difference between the host seeing "dictionary
* incomplete" and seeing nothing at all: after one lost chunk every later
* chunk is refused, no frame is ever accepted, and the 64-frame cadence never
* fires. See the note on the report loop in main(). */
static unsigned int status_now;
/*
* blit_row(src, daddr_xy, dydx)
*
@@ -404,6 +437,39 @@ unsigned int dydx;
vram_blit_b_();
}
/*
* The decode-time instrument around a paint (REFCNT_REG above).
*
* decode_begin() marks the start of one frame's decode, decode_sample()
* extends the elapsed count by however many REFCNT ticks have passed since the
* last sample (mod 65,536 -- the counter is 16 bits), and decode_end() folds
* the frame's total into decode_worst, which send_status() reports and main()
* clears on a successful send. The per-row sample is what makes the wrapped
* counter unambiguous over a full repaint.
*/
static void decode_begin()
{
decode_mark = *(unsigned short *)REFCNT_REG;
decode_ticks = 0;
}
static void decode_sample()
{
unsigned int now;
now = *(unsigned short *)REFCNT_REG;
decode_ticks = decode_ticks + ((now - decode_mark) & 0xFFFF);
decode_mark = now;
}
static void decode_end()
{
decode_sample();
if (decode_ticks > decode_worst) {
decode_worst = decode_ticks;
}
}
/*
* tile_blit(src, tx, ty)
*
@@ -566,6 +632,7 @@ fp_header *h;
}
vid_dint_();
decode_begin();
k = 0;
pos = h->payload_off; /* the frame's first index byte */
for (ty = 0; ty < TILE_ROWS; ty++) {
@@ -576,10 +643,12 @@ fp_header *h;
}
if (k >= rx_nfrag) {
vid_eint_();
decode_end();
return 1; /* truncated: stop drawing */
}
if (pos < rx_frag_off[k]) {
vid_eint_();
decode_end();
return 1; /* a gap: never paint across it */
}
idx = (unsigned int)*(unsigned char *)
@@ -587,8 +656,10 @@ fp_header *h;
tile_blit(dict_base + (idx << TILE_SHIFT), tx, ty);
pos = pos + 1;
}
decode_sample();
}
vid_eint_();
decode_end();
return 1;
}
@@ -677,6 +748,7 @@ fp_header *h;
unsigned int count, k, pos, val, prev, lo, hi, sent;
unsigned int tile, tx, ty;
decode_begin();
if (!dict_ready || dict_base == 0) {
return 0;
}
@@ -698,6 +770,9 @@ fp_header *h;
/* Pass 1: the whole list, before anything is painted. */
prev = 0;
for (k = 0; k < count; k++) {
if ((k & 0x3F) == 0) {
decode_sample();
}
if (!frag_get(h->payload_off + FP_DELTA_HDR + k * 2, &lo)
|| !frag_get(h->payload_off + FP_DELTA_HDR + k * 2 + 1,
&hi)) {
@@ -729,7 +804,10 @@ fp_header *h;
}
/* Pass 2: walk the grid, paint the named tiles. `pos` holds the next
* named position; `tile` is the grid position the loop is on. */
* named position; `tile` is the grid position the loop is on. The
* checkpoint here ends the validation stretch (which is O(count)) and
* the per-row samples keep the 16-bit REFCNT counter unambiguous. */
decode_sample();
vid_dint_();
frag_get(h->payload_off + FP_DELTA_HDR, &lo);
frag_get(h->payload_off + FP_DELTA_HDR + 1, &hi);
@@ -754,8 +832,10 @@ fp_header *h;
}
tile = tile + 1;
}
decode_sample();
}
vid_eint_();
decode_end();
return 1;
}
@@ -805,12 +885,16 @@ unsigned int n;
return 1;
}
/* One refused dictionary chunk: count it and leave the accepted prefix alone,
* so the sender's next good chunk still lands. */
/* One refused dictionary chunk: count it, ask main() for an immediate status
* report, and leave the accepted prefix alone so the sender's next good chunk
* still lands. The early report is diagnostic: without it a lost chunk makes
* every later chunk fail and no frame is ever accepted, so the 64-frame report
* never fires and the host only sees silence. */
static void dict_refuse()
{
dict_bad = dict_bad + 1;
*(unsigned int *)DICT_BAD = dict_bad;
status_now = 1;
}
/*
@@ -1350,11 +1434,22 @@ unsigned int ptr;
* terminal's own address from its Ethernet ROM, the stream's ethertype 0x88B7
* (frame.py ETHERTYPE), then the same 16-byte VS header every other frame
* carries, with rows and stretch zero because a status has no geometry. The
* body is 12 bytes, LITTLE-endian as frame.py's pack_status() writes it:
* frames_rx u32, crc_err u16, drops u16, csr0 u16, heap_free_kb u16. (The
* command protocol in lance_cmd.c is big-endian; this one is not, and getting
* it backwards reads 64 frames as 0x40000000 -- which is how the Task 6
* MAME run first found it.)
* body is 16 bytes, LITTLE-endian as frame.py's pack_status() writes it:
* frames_rx u32, crc_err u16, drops u16, csr0 u16, heap_free_kb u16,
* dict_state u8, reserved u8, decode_worst u16. (The command protocol in
* lance_cmd.c is big-endian; this one is not, and getting it backwards reads
* 64 frames as 0x40000000 -- which is how the Task 6 MAME run first found it.)
*
* dict_state is two bits: 0x01 DICT_READY (the 32 KB dictionary is complete)
* and 0x02 DICT_BAD (at least one chunk was refused since the payload started).
* Both can be set -- a chunk refused after the dictionary was ready leaves the
* dictionary usable -- which is why it is a bit set rather than an enum.
*
* decode_worst is the worst decode time, in REFCNT ticks, since the last
* status frame that was actually sent (decode_begin() above says why ticks and
* not microseconds). It is the instrument for the plan's 12 risk: whether the
* B,XY tile path is fast enough on the unit. A report sent before any frame
* was painted carries 0.
*
* crc_err is RX_BAD: the frames fp_parse() rejected (header CRC, magic,
* version or length). Since ruling R14 a corrupted *body* cannot appear
@@ -1383,7 +1478,7 @@ static int send_status(frames_rx)
unsigned int frames_rx;
{
unsigned int i, crc, csr0, tmd1, off, hb, miss, bad, rdrops, fdrops,
free_kb;
free_kb, dstate;
hb = status_buf;
@@ -1453,6 +1548,14 @@ unsigned int frames_rx;
wb(off + 9 * 8, (csr0 >> 8) & 0xFF);
wb(off + 10 * 8, free_kb & 0xFF);
wb(off + 11 * 8, (free_kb >> 8) & 0xFF);
/* The two Task 6 review fields: dictionary state and the worst decode
* since the last report (frame.py's pack_status() writes the same
* offsets; both bits and the tick unit are in the docstring above). */
dstate = (dict_ready ? 0x01 : 0) | (dict_bad ? 0x02 : 0);
wb(off + 12 * 8, dstate & 0xFF);
wb(off + 13 * 8, 0); /* reserved: keep the u16 aligned */
wb(off + 14 * 8, decode_worst & 0xFF);
wb(off + 15 * 8, (decode_worst >> 8) & 0xFF);
/* Arm the next descriptor: TMD3, then the length, then ownership last,
* so the chip cannot take a descriptor whose length is still stale
@@ -1538,6 +1641,10 @@ int main()
dict_bytes = 0;
dict_bad = 0;
tiles_drop = 0;
status_now = 0;
decode_mark = 0;
decode_ticks = 0;
decode_worst = 0;
/* Arm the VRAM canary before any frame is accepted, so a repaint
* between now and the first blit is counted rather than mistaken for
@@ -1613,18 +1720,39 @@ int main()
* harness says the sender has stopped -- for every further
* accepted frame, so a finite run ends with a report whose
* frames_rx is the final count rather than the last multiple
* of 64 (RX_EOF is harness-only; see frame_parse.h). */
* of 64 (RX_EOF is harness-only; see frame_parse.h).
*
* Two out-of-cadence reports exist so the host is never left
* with silence:
* - the first accepted frame, so a host that is sending
* sees that the payload is running before 64 frames have
* arrived (and gets the dictionary state with it);
* - status_now, set by dict_refuse(). A lost dictionary
* chunk makes every later chunk fail, no frame is ever
* accepted, and the 64-frame rule would never fire: the
* host would report "PAYLOAD NOT RUNNING" instead of
* "dictionary incomplete". These are one-shot in the
* sense that status_seen advances; further refusals set
* the flag again, which is wanted while an upload is
* broken.
*/
frames = *(unsigned int *)RX_OK;
if (!lance_ok) {
continue;
}
if (frames - status_seen >= STATUS_EVERY
|| status_now
|| (frames != 0 && status_seen == 0)
|| (*(unsigned int *)RX_EOF != 0 && frames != status_seen)) {
vid_dint_();
sent = send_status(frames);
vid_eint_();
if (sent) {
status_seen = frames;
status_now = 0;
/* "since the last report": reset only after the
* frame really went to the chip. */
decode_worst = 0;
}
}
}