LCARS Challenge Fall 2026

ModelmacOS collector, native M4 run (each segment is 0.5 of 5)What broke on real hardware

9 / 9compiled from a clean snapshot with zero Linux warnings

45 / 45exits restored the terminal (5 exit paths × 9 monitors)

4 of 7native macOS collectors read process CPU in the wrong unit

1 of 9macOS collectors fully correct on real hardware

Standings

Final score = human visual score (out of 30) + blind engineering score (out of 70) − honesty deduction. Engineering was graded and locked before model names or visual scores were revealed. Select a row to open its dossier.

Visual, human-judged (/30) Engineering, blind (/70) Honesty deduction

GPT-6.1 Sol and GPT-6 Astra tie on 91.5. Sol leads on engineering (67.5 vs 66.0); Astra leads on visuals (25.5 vs 24.0). The tie stands.

Conflict of interest

The evaluator and all nine blind grading agents ran on Claude Opus 5.5, which is also the winning contestant. Claude Fable 5.1, in second place, comes from the same lab.

The safeguards: snapshots were renamed S01 to S09 by a seeded shuffle, a grep found no model or vendor names in any of them, and the mapping stayed sealed until every scorecard was locked. Build, lint, test, terminal lifecycle, performance and the native macOS runs all come from scripts, not judgement. Every point in Opus's scorecard cites probe output or a file:line. A model might still recognise its own style, so weigh the judgement-based items with that in mind.

Dossiers

macOS audit

Eight of nine READMEs said plainly that the author never ran the macOS build; Gemini's says nothing either way. We built each one natively on an Apple M4 (macOS 27.2) and compared its screen against sysctl, vm_stat, top and df taken at the same moment.

Model	Approach	What broke on real hardware	Score /5

The Mach-tick trap

On Intel Macs, proc_taskinfo.pti_total_user and pti_total_system are nanoseconds. On Apple Silicon they are Mach absolute-time ticks from a 24 MHz timebase, so they have to be scaled. Skip the scaling and process CPU reads about 40 times too low.

ns = ticks × timebase.numer / timebase.denom = ticks × 125 / 3 # M4: one tick ≈ 41.7 ns

Got it wrong: Fable 5.1, GPT-6.1 Sol, GPT-6 Astra, Grok 4.7. Scaled correctly: Opus 5.5, Gemini 4.

The PID-count trap

proc_listallpids(NULL, 0) returns how many PIDs exist, not how many bytes to allocate. Treat the count as a byte size and the buffer ends up too small, so the process list is silently cut short and the top-5 misses busy processes.

n = proc_listallpids(NULL, 0) # a count buf = calloc(n + slack, sizeof(pid_t)) # not n bytes

Got it wrong: Gemini 4 (297 of 915 processes), Grok 4.7 (74 of 916).

Neither bug shows up in a cross-compile, go vet, clippy, or a code review that trusts the man page. Both appear only when the binary runs on Apple Silicon.

Scoring ladder, after blind calibration

5.0Complete and verified natively: Opus 5.5.

3.5One materially wrong metric: Fable, Sol and Astra (Mach ticks), Kimi (memory available), DeepSeek (disk 256× from Darwin's 1 MiB f_bsize).

3.0Process enumeration truncated: Gemini 4.

2.5Enumeration truncated and CPU unscaled, so every top-5 row reads 0%: Grok 4.7.

1.0Most metrics N/A or wrong, from bad struct offsets and a sysctl name that does not exist on macOS: Muse Wonder.

Breakdown

Engineering points by rubric category, after calibration. Cell shade shows the share of the category maximum; brighter is closer to full marks.

Blind calibration

Nine independent graders read the same rubric bands nine ways. After all nine scorecards were locked, and before anything was unblinded, every card was compared item by item. Each change is recorded as an addendum inside the affected scorecard.

Model	Change	Reason
Grok 4.7	64.0 → 63.5	Arithmetic: the subtotals summed to 63.5.
GPT-6 Astra	66.5 → 66.0	Arithmetic: the subtotals summed to 66.0.
Grok 4.7	macOS 3.0 → 2.5	Enumeration bug plus unscaled CPU is strictly worse than Gemini's enumeration bug alone.
Kimi K3.1 Pro	macOS 4.5 → 3.5	Memory available is materially wrong (19.8 G against about 10.2 GiB), the same band as the other single-metric faults.
Gemini 4	macOS 2.5 → 3.0	One bug affecting count and top-5. The deprecation warnings are already penalised under build and lint.

Ranking effects: Astra fell from sole third (92.0) into a tie with Sol at 91.5, and DeepSeek moved ahead of Grok (86.5 → 85.5). The top two and bottom two did not move.

Performance

Linux, Ryzen 9 7900X, 120×36 PTY. Startup is the median of 15 launches; steady state is an 8-second window after 2 seconds of warm-up. Lower is better in every column, and each bar is scaled to its own column.

Model	First byte	First frame	Idle CPU	Peak RSS	Output	Per frame	Binary

All nine are far inside the 250 ms startup target, none sent a full-screen clear during steady-state refresh, and memory stayed flat or bounded over a further 60 seconds. CPU resolution is 0.125% (100 Hz ticks over 8 s), so 0.0% means less than one tick. Grok rewrites every changed row in full, sending close to a full screen each second. Muse is the only entry above 1% idle CPU. Kimi and DeepSeek pay for sysinfo's initial scan at startup.

Lifecycle

Each monitor was stopped five ways inside a 120×36 PTY, with the terminal's termios state recorded before launch and after exit. A pass means it exited, restored termios exactly, left the alternate screen and showed the cursor again. The same run was repeated natively on macOS.

Every monitor survived 80×24, 40×10 and 20×5, and a live resize from 120×36 to 80×24 to 30×8 and back, without a panic.
Gemini and Grok build with panic = "abort" but install a panic hook that restores the terminal first.
The one latent risk: GPT-6 Astra's sampler goroutine has no recover, so a collector panic would skip restoration.
Kimi and DeepSeek send no colour reset after the quit key, but their last frame already ends in one. DeepSeek also renames the window title to "Terminal" on exit.
All nine detect redirected output and exit with a message that says what to do.

Honesty

Three deductions in total. Two are for the same thing: an in-process microbenchmark presented as whole-program startup time.

−1.0Gemini 4 Argon

“Measured cold startup + first-frame render is ~0.37 ms (nearly 700× faster than the 250 ms target)”

Claimed0.37 ms Measured30.25 ms

The number comes from bench.rs, which skips exec, terminal setup, a 25 ms warm-up sleep and the terminal write. The README never says so.

−1.0Muse Wonder

“Verified the full brief”

The main view never shows memory available, which the brief requires. Per-core CPU is cut off at 120×36 and the frame breaks at 40×10.

−0.5Claude Fable 5.1

“starts in well under a millisecond after exec (measured 0.4–0.5 ms…)”

Claimed0.5 ms Measured4.48 ms

Timed from the top of main. The README admits further down that the figure excludes exec, which contradicts the headline. That partial disclosure is why the deduction is half of Gemini's.

Patterns

Looks and engineering mostly disagree

Gemini tied Opus for the best-looking screen (9.5) yet scored 63 of 70, the only entry to fail cargo fmt --check. GPT-6.1 Sol scored a modest 8.0 visually but 67.5 on engineering, from the smallest codebase in the field. The two axes agree only at the ends: Opus led both, Muse trailed both.

`sysinfo` is not free

The two entries that used the crate started 40 to 90 ms slower than the rest, and on Linux both listed threads as processes: about 1,630 against roughly 505 real ones, with duplicate thread IDs in the top-5.

No TUI frameworks, nine renderers

Every entry hand-rolled a cell buffer and a diff renderer. Eight diff at the cell or span level; Grok diffs whole rows. Toolchain hygiene was nearly universal: eight of nine were fmt- and lint-clean on Linux and on the darwin target.

Language did not decide it

	Rust (6)	Go (3)
Mean engineering	64.8	62.8 (66.75 without Muse)
Binary size	0.47–0.82 MB	1.97–2.11 MB
Idle RSS	2.7–10.6 MiB	5.0–8.5 MiB
macOS approach	4 hand-written FFI, 2 `sysinfo`	2 cgo, 1 pure syscall

Method

SnapshotCopied each original into S01–S09 with a seeded shuffle, stripping git history, binaries, cost files and the submitted screenshots. The originals were never touched.
ProbeScripted release builds, tests, fmt and lint; PTY sessions with five exit paths, four sizes, live resize and seven colour modes; startup and steady-state performance.
Run on a MacNative build, tests and the same PTY probes on an M4, compared against live sysctl, vm_stat, top and df.
Grade blindOne isolated agent per snapshot, with access only to the brief, the rubric, its own snapshot and its own probe output.
Calibrate blindCompared all nine scorecards item by item, fixed two arithmetic errors and put macOS scores on one severity ladder.
Lock, then unblindMade every scorecard read-only, then opened the model mapping and added the human visual scores.

Hosts

Linux	AMD Ryzen 9 7900X, 93 GiB, Ubuntu 26.04.1, kernel 7.0. All scored runtime and performance measurements.
macOS	Apple M4, 24 GiB, macOS 27.2 (Darwin 27.2.0 arm64). Native collector cross-check.
Toolchains	Rust 1.99.1, Go 1.27.3, Python 3.14 harness with pyte.

Caveats

The Mac's toolchain had no clippy, so macOS code paths were linted from Linux against aarch64-apple-darwin.
The capture renderer drops bright-magenta backgrounds in the 16-colour images. Graders checked the raw streams; no score depends on the images.
The screens in the dossiers are the evaluator's fresh captures on the Linux host. Before publication, hostnames, process names and network identifiers in every capture and log were replaced with stand-ins such as linux, mac-host and svc07.
One host per OS. Absolute timings will differ on other hardware; the ratios should hold.

Cost and effort

Not scored. Reported verbatim from each run's cost.txt. Harnesses measure cost differently, so these figures are not comparable with each other.

Model	Reported cost	Session span

Session span is an upper bound from file timestamps, from the moment the brief was placed to the newest file.