From 56d6a71cb7302dd7fd4a9235f7600bb21f557f42 Mon Sep 17 00:00:00 2001 From: arti Date: Wed, 11 Nov 2026 09:53:23 +0000 Subject: [PATCH] docs: M1 minimal backend and test convention --- README.md | 209 +++++++++++++++++++++++++++++++----------- scripts/build-llvm.sh | 96 +++++++++++++++---- 2 files changed, 233 insertions(+), 72 deletions(-) diff --git a/README.md b/README.md index 455a326..b9d8765 100644 --- a/README.md +++ b/README.md @@ -3,13 +3,14 @@ A modern C toolchain for the DEC VT1200 / TI TMS34010, built as a small fork of upstream LLVM. -**Status: M0.** Clang understands the `tms34010-unknown-none` target, the TI -data model, and bit-addressed pointers. There is **no machine-code backend -yet**: M0 stops at LLVM IR. The backend, TI ABI, ROM calls, Xlib and real -clients are M1–M5 and are separate plans (see the design spec). +**Status: M1.** `llc -march=tms34010` emits GSPA-compatible assembly for a +small set of leaf functions, and that assembly assembles, links and runs in +MAME. M1 is an **assembly-only, test-only** snapshot: it uses an A0–A3/A0 +register convention, not the TI ABI, and has no object writer, calls or runtime. - Design: [`docs/superpowers/specs/2026-11-10-tms34010-llvm-design.md`](docs/superpowers/specs/2026-11-10-tms34010-llvm-design.md) - M0 plan: [`docs/superpowers/plans/2026-11-10-tms34010-m0-fork-and-bit-address-semantics.md`](docs/superpowers/plans/2026-11-10-tms34010-m0-fork-and-bit-address-semantics.md) +- M1 plan: [`docs/superpowers/plans/2026-11-11-tms34010-m1-minimal-backend.md`](docs/superpowers/plans/2026-11-11-tms34010-m1-minimal-backend.md) - Spike: [`spike/REPORT-23.md`](spike/REPORT-23.md) (current, LLVM 23.1.5); [`spike/REPORT.md`](spike/REPORT.md) (historical, 19.1.7) @@ -17,42 +18,67 @@ clients are M1–M5 and are separate plans (see the design spec). - **Compiler fork:** (branch `tms34010`) — the full buildable LLVM 23.1.5 tree with the TMS34010 - target and clang bit-address changes. + target, clang bit-address changes and the M1 backend. - **This project:** — plans/specs, VT1200 integration tooling, `tms-bitaddr`, `tms-clang`, - symbols and the spike. + symbols, the M1 harness and the spike. -The compiler fork is the canonical source for LLVM changes. `patches/llvm` -here is a lightweight mirror used by `scripts/build-llvm.sh`, so the project -can be built without cloning 3 GB of LLVM; keep it in sync with the fork's -`tms34010` branch. +The compiler fork is the canonical source for all target code. `patches/llvm` +here is a **frozen M0 mirror** used by the default upstream+patches build; M1 +does not change it. -## What M0 provides +## What M1 provides -- `clang --target=tms34010-unknown-none`: 32-bit bit-addressed pointers, - 8-bit signed `char`, 16-bit `short`, 32-bit `int`/`long`, packed structs, - `sizeof(struct { char; int; void*; }) == 9`. -- Pointer arithmetic scaled by 8; pointer difference divided by 8 - (`char *` → `/8`, `int *` → `/32`), while `(int)(p+1) - (int)p` stays an - integer subtraction. -- `tools/tms-bitaddr`: GEP `--scale` and late `--normalize` passes, including - constant-expression GEPs. -- `scripts/tms-clang`: the M0 pipeline - `clang -O0` → `tms-bitaddr --scale` → `opt -O2` → `tms-bitaddr --normalize`. -- Lit tests for the triple, data model and pointer difference; an integration - test over a real Xlib client IR. +M1 fills in the M0 target skeleton and stops at assembly: + +- `llc -march=tms34010 -filetype=asm` for leaf 32-bit functions: + integer/pointer arguments in `A0`–`A3`, result in `A0`, return via `RETS 0`. +- `ADD` for `i32`, `MOVI` for constants, `MOVE ..., 1` (FS1) for 32-bit + load/store, `MOVE ..., 0` (FS0) for `i16`, and sign-extending `MOVB` for + `i8` load/store. +- Constant global addresses as `MOVI _sym+bits`; a surviving GEP offset is a + bit offset (`ADDI 24`, not `ADDI 192`). +- Hard errors, not silent fallbacks, for calls (`M1 has no calls; M2 adds the + TI ABI`), stack frames (`M1 has no stack frame`), more than four register + arguments, and unsupported types. +- Nine `llvm-lit`/FileCheck tests under + `llvm/test/CodeGen/TMS34010/m1-*.ll` in the compiler fork. +- An end-to-end path: `scripts/tms-clang` IR → `llc -O0 -filetype=asm` → + **1987 GSPA/GSPLNK 2.00 under DOSBox** → COFF → the VT1200 loader in MAME. + The run checks `add(20, 22) == 42`, `load(&word) == 0x12345678`, + `second() == &arr[1]` (a `symbol+32` bit relocation), `loadb` of a byte + `0x81` returns the sign-extended `0xFFFFFF81`, and `storeb` leaves `0x81` + in memory. +- A `dis34` round-trip that reads the linked `.text` from the COFF and checks + the instructions GSPA encoded at their linked bit addresses. + +## M1 is not the TI ABI + +| | M1 test convention | TI ABI (M2) | +|---|---|---| +| Integer/pointer arguments | `A0`–`A3`, four maximum | program stack (via `A14` STK) | +| Return value | `A0` | `A8` | +| Frame | none; leaf functions only | `A13` FP, `A14` STK, `A15` system stack | +| Return instruction | `RETS 0` | `RETS 2` | +| Calls, callee-saves, stack args | rejected | required | +| Field select | harness sets FS1/FS0 before the leaf | `FS1 = 32` preserved across calls | + +M1 assembly is not ABI-compatible with TI's compiler or the ROM Xlib. M2 +replaces the convention; do not build on the M1 register assignment. ## Repository layout ```text -build/llvm-src/ local fork (ignored; created by build-llvm.sh or manually) -build/llvm-build/ Ninja build of the fork (ignored) -build/llvm-clean*/ clean CI clone/build (ignored) -patches/llvm/ the complete fork delta against llvmorg-23.1.5 +build/llvm-clean/ compiler fork checkout used for M1 (ignored) +build/llvm-clean-build/ its Ninja build tree (ignored) +build/llvm-src/ M0 working fork (ignored) +patches/llvm/ frozen M0 patch series against llvmorg-23.1.5 tools/tms-bitaddr/ GEP scale/normalize tool -scripts/tms-clang M0 pipeline wrapper +scripts/tms-clang IR pipeline wrapper (clang + tms-bitaddr + opt) +scripts/m1/ M1 harness: C inputs, entry.asm, link.cmd, checks +scripts/tests/m1/ M1 gates (run.sh, dis-roundtrip.sh) scripts/tests/m0/ M0 integration tests -scripts/build-llvm.sh clean clone + patches + build +scripts/build-llvm.sh fork or upstream+patches build scripts/fetch-vt1200.sh fetch the VT1200 RE tree at its pinned commit spike/ measurement corpus and reports symbols/ per-ROM import tables (v2.2 subset) @@ -63,10 +89,13 @@ docs/ design and plans ## Prerequisites `cmake`, `ninja-build`, `ccache`, a C++17 host compiler (GCC 14 is used here), -Python 3, and about 20 GB of disk for the LLVM source and build. On Debian: +Python 3, and about 20 GB of disk for the LLVM source and build. The M1 gates +additionally need `dosbox`, `mtools` (`mcopy`), the VT1200 SDK image, the +`dwt` COFF loader, and a built MAME `vt1200` binary +(`build/mame-src/vt1200`). On Debian: ```sh -sudo apt-get install -y cmake ninja-build ccache +sudo apt-get install -y cmake ninja-build ccache dosbox mtools ``` ## Fetch the VT1200 assets @@ -78,54 +107,122 @@ Xlib call sequence, `as34`, `dis34`, SDKs, manuals). It is not vendored: ./scripts/fetch-vt1200.sh # -> vendor/vt1200, pinned 17dfd24... ``` -## Build the pinned fork +## Build the compiler + +### Compiler fork (M1, the normal path) + +The fork checkout already exists at `build/llvm-clean` on `tms34010`. Build it +in place: + +```sh +LLVM_SRC=$PWD/build/llvm-clean LLVM_BRANCH=tms34010 ./scripts/build-llvm.sh +# binaries: build/llvm-clean-build/bin/{llc,clang,opt,llvm-config,FileCheck} +# lit driver: build/llvm-clean-build/bin/llvm-lit +``` + +To clone the fork first instead of using an existing checkout: + +```sh +LLVM_REPO=gitea@git.wut.ee:opencode.siil.arti.ee/llvm-tms34010.git \ +LLVM_BRANCH=tms34010 ./scripts/build-llvm.sh +``` + +Fork mode **never** checks out, resets, cleans or patches the source tree; it +only runs CMake and Ninja. Local commits on the branch are safe. Use +`LLVM_BUILD=...` to keep a separate build directory. It builds `llc`, clang, +opt, `llvm-config`, FileCheck and the test helpers with X86 (host) and +TMS34010; a first build takes roughly 20–40 minutes. + +### Upstream + patches (frozen M0 path) ```sh ./scripts/build-llvm.sh -# binaries: build/llvm-clean-build/bin/{clang,opt,llvm-config,FileCheck,llvm-lit} ``` -The script clones `llvmorg-23.1.5`, applies `patches/llvm/*.patch`, and builds -`clang`/`opt` with X86 (host) and TMS34010. A first build takes roughly -20–40 minutes; rebuilds are incremental. +This mode clones `llvmorg-23.1.5`, applies `patches/llvm/*.patch`, and builds +`build/llvm-clean-build`. It owns and resets only a plain `llvm-project` clone: +if `build/llvm-clean` has some other origin (for example the M1 fork), it +refuses rather than destroying it. Use fork mode for that checkout. -For development, `build/llvm-src` is the working fork and `build/llvm-build` is -its build tree. `scripts/export-patches.sh` regenerates the committed patch -series with `git diff` (including new files via `git add -N`). +`scripts/export-patches.sh` regenerates the committed M0 patch series with +`git diff` (including new files via `git add -N`); it is not used for M1. -## Build the M0 tool and run the tests +## Build the M0 support tools + +The M1 pipeline still uses `tms-bitaddr` and `tms-clang`: ```sh LLVM_CONFIG=$PWD/build/llvm-clean-build/bin/llvm-config \ tools/tms-bitaddr/build.sh +``` +M0 regression test (optional in M1): + +```sh TMS_CLANG=$PWD/build/llvm-clean-build/bin/clang \ TMS_OPT=$PWD/build/llvm-clean-build/bin/opt \ TMS_BITADDR=$PWD/tools/tms-bitaddr/tms-bitaddr \ ./scripts/tests/m0/run.sh # -> M0 pipeline: PASS - -build/llvm-clean-build/bin/llvm-lit -v \ - build/llvm-clean-build/tools/clang/test --filter='TMS34010' -# -> 3 passed ``` -## Use +## Run the M1 gates + +These are the documented commands; all three must pass: ```sh -scripts/tms-clang -O2 -I spike -o /tmp/client.ll spike/xclient.c +# 1. LLVM codegen tests: 9 passed +build/llvm-clean-build/bin/llvm-lit -v \ + build/llvm-clean-build/test/CodeGen/TMS34010 --filter='m1-' + +# 2. Compile, assemble, link and run in MAME +./scripts/tests/m1/run.sh +# -> M1 MAME: PASS + +# 3. Rebuild the COFF and check the GSPA encodings with dis34 +./scripts/tests/m1/dis-roundtrip.sh +# -> M1 dis34 round-trip: PASS ``` -The output is LLVM IR in which every address is a bit address. It is not -assembly or an object file; `llc` support is M1. +`run.sh` compiles five C inputs with `scripts/tms-clang -O2`, runs +`llc -march=tms34010 -O0 -filetype=asm` on the normalized IR, concatenates +the result with `scripts/m1/entry.asm` and `scripts/m1/char-test.asm`, +assembles/links with GSPA/GSPLNK 2.00 under DOSBox, then loads the COFF in +MAME. `dis-roundtrip.sh` runs the same build and then disassembles the linked +`.text` at the symbol addresses from the COFF symbol table. Paths are +overridable with `TMS_CLANG`, `TMS_OPT`, `TMS_BITADDR`, `TMS_LLC`, `MAME`, +`DOSBOX`, `SDK_IMG`, `M1_WORK`, `M1_COFF` and `M1_SDK`. -## What is deliberately missing +## Emit assembly by hand -- TMS34010 instruction selection, registers, frame lowering, asm/MC output. -- The TI ABI: A8 return, A13/A14/A15 dual stack, `RETS 2`, `FS1 = 32`. -- ROM import symbols beyond the subset in `symbols/`. -- Xlib headers/clients on the target, `memcpy`/`byval`/varargs/float/bitfield - lowering, native COFF. +```sh +TMS_CLANG=$PWD/build/llvm-clean-build/bin/clang \ +TMS_OPT=$PWD/build/llvm-clean-build/bin/opt \ +TMS_BITADDR=$PWD/tools/tms-bitaddr/tms-bitaddr \ + scripts/tms-clang -O2 -o /tmp/add.ll scripts/m1/add.c -These are M1–M5 in the design spec, written as separate plans because their -tasks depend on what M0/M1 measure. +build/llvm-clean-build/bin/llc -march=tms34010 -O0 -filetype=asm \ + /tmp/add.ll -o /tmp/add.s +``` + +Use `-O0` on the IR that `tms-clang` already optimized and scaled; feeding +raw clang IR to `llc` double-scales GEP offsets. The lit tests use +`llc -march=tms34010 -O2 -filetype=asm` on hand-written IR to exercise the +selector pipeline directly. Output is assembly only: `llc` cannot emit an +object file or a usable COFF for this target. + +## Not yet supported (M2 and later) + +- **Object emission:** no `MCCodeEmitter`, object writer, `llc -filetype=obj`, + `clang -c`, or native COFF. GSPA assembles and GSPLNK links; the existing + COFF loader loads. +- **TI ABI:** `A8` return, `A13`/`A14`/`A15` dual stack, `RETS 2`, `FS1 = 32` + across calls, callee-saves, stack arguments, more than four arguments. +- **C features:** calls, locals/`alloca`, structures/`byval`, floats, varargs, + bitfields, 64-bit arithmetic, `memcpy`/`alloca` lowering. +- **Runtime and clients:** ROM/Xlib imports, `strlen`/`malloc`, X11 headers, + real VT1200 clients, debug info. +- **2ine:** M1 uses the proven 1987 GSPA/GSPLNK under DOSBox, not the external + 2ine r5.01 tools; 2ine is an M2 speed-up. + +These are M2–M5 in the design spec, each with its own plan. diff --git a/scripts/build-llvm.sh b/scripts/build-llvm.sh index 57e2230..499571b 100755 --- a/scripts/build-llvm.sh +++ b/scripts/build-llvm.sh @@ -1,28 +1,92 @@ #!/bin/sh -# Build the pinned LLVM 23.1.5 fork from a clean clone and the committed -# patch series. Intended for CI and for verifying that patches/llvm is the -# complete fork delta. +# Build the TMS34010 LLVM compiler. +# +# Two modes: +# +# upstream + patches (M0, default) +# ./scripts/build-llvm.sh +# Clones llvmorg-23.1.5 into build/llvm-clean, applies patches/llvm and +# builds it. This mode owns its clone and resets it, but refuses to touch +# a checkout whose origin is not llvm-project. +# +# compiler fork (M1 and later) +# LLVM_SRC=$PWD/build/llvm-clean LLVM_BRANCH=tms34010 \ +# ./scripts/build-llvm.sh +# LLVM_REPO=gitea@git.wut.ee:opencode.siil.arti.ee/llvm-tms34010.git \ +# LLVM_BRANCH=tms34010 ./scripts/build-llvm.sh +# Build the fork branch in place. Fork mode never checks out, resets, +# cleans or patches the source tree; it only configures and builds it. +# +# Environment: +# LLVM_SRC source tree (default build/llvm-clean) +# LLVM_BUILD build tree (default build/llvm-clean-build) +# LLVM_REPO git URL to clone when LLVM_SRC does not exist +# LLVM_BRANCH branch to clone/build; setting this (or LLVM_REPO) selects +# fork mode set -eu ROOT=$(cd "$(dirname "$0")/.." && pwd) cd "$ROOT" mkdir -p build SRC=${LLVM_SRC:-build/llvm-clean} BUILD=${LLVM_BUILD:-build/llvm-clean-build} +REPO=${LLVM_REPO:-} +BRANCH=${LLVM_BRANCH:-} TAG=llvmorg-23.1.5 -if [ ! -d "$SRC/.git" ]; then - git clone --depth 1 --branch "$TAG" \ - https://github.com/llvm/llvm-project.git "$SRC" -fi -git -C "$SRC" checkout --detach "$TAG" -git -C "$SRC" reset --hard -git -C "$SRC" clean -fdx +origin_url() { + git -C "$1" config --get remote.origin.url 2>/dev/null || true +} -PATCHROOT=$(pwd)/patches/llvm -for p in "$PATCHROOT"/*.patch; do - [ -e "$p" ] || continue - git -C "$SRC" apply "$p" -done +if [ -n "$REPO" ] || [ -n "$BRANCH" ]; then + echo "== compiler fork mode: $SRC${BRANCH:+ (branch $BRANCH)}" + if [ ! -e "$SRC/.git" ]; then + [ -n "$REPO" ] || { + echo "build-llvm: $SRC is not a git checkout; set LLVM_REPO to clone it" >&2 + exit 1 + } + if [ -n "$BRANCH" ]; then + git clone --branch "$BRANCH" "$REPO" "$SRC" + else + git clone "$REPO" "$SRC" + fi + fi + if [ -n "$BRANCH" ]; then + CURRENT=$(git -C "$SRC" rev-parse --abbrev-ref HEAD) + if [ "$CURRENT" != "$BRANCH" ]; then + echo "build-llvm: $SRC is on '$CURRENT', not '$BRANCH'" >&2 + echo "build-llvm: check the branch out first; fork mode never" >&2 + echo "build-llvm: modifies the source tree" >&2 + exit 1 + fi + fi +else + echo "== upstream + patches mode: $SRC (tag $TAG)" + if [ -e "$SRC/.git" ]; then + URL=$(origin_url "$SRC") + case "$URL" in + *llvm-project*) ;; + *) + echo "build-llvm: refusing to reset $SRC: origin is" >&2 + echo "build-llvm: '${URL:-}', not llvm-project." >&2 + echo "build-llvm: use LLVM_REPO/LLVM_BRANCH for a compiler fork" >&2 + echo "build-llvm: checkout (default mode must not reset it)" >&2 + exit 1 + ;; + esac + else + git clone --depth 1 --branch "$TAG" \ + https://github.com/llvm/llvm-project.git "$SRC" + fi + git -C "$SRC" checkout --detach "$TAG" + git -C "$SRC" reset --hard + git -C "$SRC" clean -fdx + + PATCHROOT=$(pwd)/patches/llvm + for p in "$PATCHROOT"/*.patch; do + [ -e "$p" ] || continue + git -C "$SRC" apply "$p" + done +fi cmake -S "$SRC/llvm" -B "$BUILD" -G Ninja \ -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON \ @@ -30,4 +94,4 @@ cmake -S "$SRC/llvm" -B "$BUILD" -G Ninja \ -DLLVM_TARGETS_TO_BUILD="X86;TMS34010" \ -DLLVM_INCLUDE_TESTS=ON -DLLVM_INCLUDE_EXAMPLES=OFF \ -DLLVM_INCLUDE_BENCHMARKS=OFF -ninja -C "$BUILD" clang opt llvm-config FileCheck count not +ninja -C "$BUILD" clang opt llc llvm-config FileCheck count not