Testing Strategy and Acceptance-Test Matrix
Status: M0/C0 (CosmoOS deliverable 11). No test in this document has been executed yet. Every row is proposed. When a row is first executed, it is marked with the date and result. Tests that were performed are never mixed with proposals.
1. Test kinds
| Kind | Location | Oracle |
|---|---|---|
| Unit | tests/Nova.Compiler.Tests | xUnit assertions per pass |
| parse/check pass/fail | tests/corpus/* | // error: Nxxxx @line:col annotations |
| compile-fail (ownership) | tests/corpus/check-fail | // error[Nxxxx] annotations, as for the other check-fail files |
| Fault-injection sweep (M3) | tests/Nova.Compiler.Tests/FaultInjectionTests.cs | every run-pass program that allocates, built with ASan+UBSan, is run once per allocation attempt k with attempt k failing. Each run must return from main with live=0, or panic with "unwrapped an Err value"; no sanitizer report |
| Mutation suite (M2) | tests/Nova.Compiler.Tests/MutationTests.cs | mutants of seed programs; the expected (code, line) diagnostics come from an independent oracle, not from the checker |
| run-pass / run-fail | tests/corpus/run-* | // expect-stdout:, // expect-panic: overflow, exit status |
| IR snapshots | tests/corpus/*/ + .nir files | reviewed text dumps |
| Differential | runner | clang×gcc×O0/O2×sanitizers agree |
| Fault injection | runner + FailingAllocator | no leaks or double frees (ASan), expected AllocError paths |
| Fuzz (M7) | tests/Nova.Fuzz (§5) | generator + NIR interpreter + native agreement |
| CosmoOS boot | integrations/cosmoos (S0), later run_boot_test.py markers (S1) | NOVATEST: PASS marker |
| Kernel selftest (C5+) | CosmoOS selftest framework (owner-approved unit) | selftest pass + lockdep clean |
Sanitizer results are supporting evidence, never proof of language safety (master prompt §18).
3. Corpus format (implemented in increment 1a)
Runner: tests/Nova.Compiler.Tests/CorpusTests.cs, run by dotnet test.
tests/corpus/parse-pass/*.nova: must parse with no diagnostics. The AST dump (novac parse --dump-ast) must equal the sibling.astsnapshot. To create or update snapshots, runNOVA_BLESS=1 dotnet test. Every blessed snapshot must be reviewed by hand before it is committed. A snapshot is an expected value, not a recording.tests/corpus/parse-fail/*.nova: annotate each expected diagnostic on the line where it is reported, as// error[N0100], or// error[N0100]@14to also pin the column. The test fails on any missing or extra diagnostic, so it also checks error recovery.tests/corpus/check-pass/*.nova: must type-check with no diagnostics. The HIR dump (novac check --dump-hir) must equal the sibling.hirsnapshot (also blessed and reviewed).tests/corpus/check-fail/*.nova: must parse cleanly and produce exactly the annotated semantic diagnostics.A file with a line
// profile: freestandingis checked in the freestanding profile (cosmoos/profiles.md §4), and one with// option: no-floatas if built with--no-float(type-system.md §35.9).Every parse-pass file is also run through the type checker to check that it never crashes (semantic errors are expected there).
tests/corpus/run-pass/*.novaandrun-fail/*.nova(RunCorpusTests) are compiled to C, built with every available C configuration with strict warnings and-Werror, then run. Directives:// expect-stdout: text— one per output line, in order.// expect-stdout:alone is an empty line.// expect-exit: N— the exit status (default 0).// expect-panic: reason— placed on the line where the panic must be reported. The test requires exit status 134 (SIGABRT) andnova panic: <reason> at <file>:<line>:on stderr.
C configurations (
CToolchains), each built with-Werror:- clang
-O0; - clang
-O2; - clang
-O2 -DNV_NO_BUILTIN_OVERFLOW(the portable fallback); - clang
-O1with ASan+UBSan and-fno-sanitize-recover; - clang
-O2with the C split into a unit per Nova file (--split, c-backend.md §2.4); - clang
-O1with ThreadSanitizer, only for programs that start threads (the generated C includesnova_rt_thread.c), where it links (CToolchains.ThreadSanitizer; reported missing otherwise); - the same three without sanitizers for a real GCC when one is found (
gcc,gcc-12…gcc-15), plusNOVA_TEST_CCif set.
Every configuration must match the expectations, so the run corpus is also a differential test.
- clang
The examples in the repository's
README.mdare corpus files namedreadme-*(run-pass, run-fail, check-fail).ReadmeTestsrequires each one's code, without its header comment and test annotations, to appear verbatim in the README, so the README cannot show code the compiler does not accept, or output it does not print.FormatterTests(M10 T1,formatting.md§6): one case per style rule, and every parseable.novafile ofbenchmarks/,integrations/,lib/,programs/andtests/formatted twice (a fixed point). Each formatting re-parses its own output and fails unless the tree, the tokens (commas aside) and the comments are the original's. The repository's own code and thereadme-*files must already be formatted.DocTests(M10 T2,tooling.md§1): the page and the--missinglines of a package with every kind of item, anddocs/api/core.mdup to date withlib/core.PackageTests(M10 T5, type-system §2) andPackageManagerTests(M10 T6,tooling.md§4): names across packages, with a two-package program built and run; manifests, resolution, the lock, and path and git dependencies (git from repositories the test makes; nothing when git is absent).SiteTests: every Nova example of the documentation site's guide (site/guide) type-checks, runs with the output the page shows, or fails with exactly the errors the page shows (site/README.md).DebuggerTests(M10 T4,tooling.md§3.4): a-gbuild under lldb (macOS) or gdb (Linux CI installs it), with breakpoints at Nova lines and onnova_panicand theeditors/debugcommands; nothing when neither is installed.editors/vscodehas its own tests, run with Vite+ (vp test,tooling.md§2.4): the grammar undervscode-textmate, andnovac lspthroughvscode-jsonrpc. They are not part ofdotnet test.LspTests(M10 T3,tooling.md§2): the language server through in-memory streams (diagnostics, hover, definition, symbols, formatting, errors), and the VS Code grammar's keywords against the compiler's.tests/corpus/emit-c/*.novahave reviewed.csnapshots of the generatedmain.c.RuntimeSemanticsTestsgenerates a C harness that calls every checked helper innova/rt.hon edge values of every integer type (~9,000 cases) and compares results and panic kinds withIntegerSemantics, under every C configuration. Planting a bug innv_rem_*was checked to fail this test (2026-10-22).BackendTests: generated C is byte-identical across runs and contains no host paths. It also compiles freestanding (-ffreestanding -nostdinc) forx86_64-andaarch64-unknown-none-elf.CollectionPropertyTests(M6f; M6 acceptance criterion 2) are model-based property tests of the built-in collections. For each collection (aVectorof items whosedropprints,String,SlotMapwith storedHandles,StableVector) and each of a fixed set of seeds, a generator writes a Nova program of random operations (System.Randomwith the seed, so the program is the same on every machine). A C# model of the collection predicts the program's exact output: values read back, lengths, capacities (the growth rule oftype-system.md§23.1), stale-handle results, slices at character boundaries, and the order of everydrop. The program then runs under every C configuration (as run-pass, including ASan+UBSan) and must print exactly that, and goes through the fault-injection sweep. Any disagreement is reported with the seed, so it can be reproduced.
5. Random program generator (M7d)
tests/Nova.Fuzz is a Csmith-style generator of Nova programs. It is a library and a campaign tool: dotnet run --project tests/Nova.Fuzz -c Release -- [--seed S] [--count N | --minutes M] [--jobs J] [--out DIR]. FuzzTests runs a fixed set of seeds in the test suite.
Programs. Each seed gives one program, the same on every machine (System.Random with a seed). Every program is meant to be:
- well-typed and borrow-correct, so the checker accepts it;
- terminating:
- calls go only to later-declared functions (a call DAG, no recursion);
- loops have constant trip counts (
for (i in 0..N), countedwhile); - the total iteration budget is bounded;
- deterministic: no I/O except standard output, no addresses printed.
It uses:
- integers of every width (
i8–i64,u8–u64,usize) andbool; - literals at the edges of each range;
as(lossless only),truncate<T>, and every operator;- structs (Copy, and with a
dropblock that prints), with field reads and writes; - enums with payloads,
matchon enums, integers and booleans; - fixed arrays (indexing,
forover them) and slices; borrow/borrow mutparameters;Box,Vector(push, indexing,length,pop), optionals (== null,!), andResultwith?and!;if/else,for,while,loopwithbreak/continue, and earlyreturn;- statics (type-system §39; M14 K7): immutable scalars, arrays and structs (some built by a
@constantconstructor), read anywhere like variables; astatic mutread and changed inunsafeblocks, and static atomics changed by two threads at once (Thread.both). The last two are used only inmain, so every run prints the same.
Arithmetic. Most operations go through guarded helper functions written in Nova, as Csmith does (its safe_add and friends):
add,sub: compare against the limits first;mul: widen narrow types; reduce 64-bit operands below 2³² (unsigned) or 3·10⁹ (signed);div,rem: skip a zero divisor andMIN / -1;- shifts: amount
% width; neg: skipMIN.
A guard returns its left operand instead of panicking. A small fraction of operations, bounds and unwraps are raw. Those may panic, so the panic path (message, location, exit status) is compared too. The generated programs print running checksums of their variables throughout, and main returns a value derived from them.
Oracle. For each program:
novacmust accept it. A rejection is a finding: of the generator, or a checker bug.- The NIR interpreter's outcome (standard output, standard error, exit status) is the reference. An interpreter fault is a finding.
- Every C configuration must compile it warning-free (
-Werror, strict warnings) and give the same outcome: clang-O0, clang-O2, clang-O2flat form, and, where a real GCC exists, GCC-O0and-O2. A program that starts threads (Thread.both) is also built with clang's ThreadSanitizer where it links, so a race in the generated C is a finding. A run is limited to 10 s.
Kernel-style programs (M14 K7; --freestanding, GenerateDual). The generator writes a program with only what the freestanding profile has (no system heap, Console, Process, threads or floats), printing through out_* functions, in two variants with the same lines: hosted, whose main and printers use Console; and freestanding, whose entry is @export("fuzz_main") and whose printers call a C harness (Oracle.HarnessSource), which also defines nova_panic to print what the hosted runtime prints and exit as it does. The hosted variant, interpreted, is the reference; the freestanding variant must be accepted with --profile freestanding --no-float and, built with the harness, agree in every C configuration. FuzzTests checks 16 seeds; the nightly campaign spends a quarter of its time here. lib/kernel's guards need CosmoOS's C, so they are not generated: k3.py covers them.
Each finding keeps the program and a report (DIR/finding-<seed>.nova, .txt). A finding that reveals a compiler bug is fixed, and its program, reduced by hand, joins the corpus.
4. Execution log
| Date | What ran | Result |
|---|---|---|
| 2026-10-30 | M14 K1's negative cases: check-fail kernel-static-writes.nova (a static written, a static lock's data and a per-CPU value reached without a guard), kernel-percpu.nova (sleeping while a per-CPU value is lent), kernel-percpu-guard.nova (one guard lending twice) | check-fail corpus 128 passed |
| 2026-10-29 | CosmoOS nova-kcore with novac 0.3.0 (a static Mutex, an ObjectCache), gmake test on aarch64 | FAIL: ObjectCache.setUp passed Tracked's alignment, 8, and CosmoOS's cache_setup refuses one below KMALLOC_MIN_ALIGN (16). Fixed in lib/kernel (0 asks for the minimum); with the fix, nova-kcore ok and 523 self-tests PASS. MUTEX_INIT_AT matched mutex_init member by member |
| 2026-10-29 | M14 K7 (kernel-style programs, freestanding): FuzzTests (16 freestanding seeds, every configuration); a campaign of 300 programs from seed 500000001; 40 seeds of the normal generator compared with the previous commit's | 17 passed; 0 findings (26 panicked, each with the same message and status); the normal generator's programs unchanged |
| 2026-10-29 | ADR 0027 (moves through raw pointers, layoutOf<T>(), here, a static Mutex, ObjectCache<T>): local suite on macOS (clang) with run-pass raw-moves.nova and layout-of.nova, check-fail raw-moves-c-state.nova, layout-of.nova, static-here.nova and the changed raw-pointer-types.nova, emit-c static-here.nova (new .c, reviewed); 200 generated programs (seeds 400000001..400000200); k3.py with k3/statics.nova against CosmoOS main, both architectures, one file and split, with MUTEX_INIT_AT from k3/mutex_init_at.h | 3,563 passed; 0 findings; k3.py all PASS, undefined symbols only kernel exports, CosmoOS tree unchanged |
| 2026-10-29 | Enum constructors for pointer payloads (c-backend §2, ADR 0025 §7): local suite on macOS (clang); 200 generated programs (seeds 300000001..300000200); CosmoOS's kcore_test.nova under clang --analyze with its kernel flags, both architectures, against novac 0.2.0's C as the control; benchmarks/suite (Nova, clang and gcc, two interleaved rounds, before and after) on the Ryzen box, then memory-copy alone with and without -falign-loops=32 | 3,524 passed; 0 findings; 0 analyzer reports (control: 4); within noise but memory-copy/clang 1.10× (1.03× aligned); an earlier all-payload version made gcc string-building 1.8× slower and was dropped |
| 2026-10-29 | M14 K3 in a booted kernel: CosmoOS's self-test nova-kcore (branch nova-kcore, PR #489, novac 0.2.0), gmake test on aarch64 and x86_64 (debug, lockdep), with lockdep's class table at 512 and at 640; host-test, check-docs, nova-check | at 512: aarch64 PASS, x86-64 FAIL (class table full, a later lock); at 640: both PASS (197.5 s, 193.6 s), nova-kcore ok, 523 self-tests, peak 79% |
| 2026-10-29 | M14 K5 (inline atomics, ADR 0026): local suite on macOS (clang); atomic-orderings, statics and threads built and interpreted (identical output); k3.py with the new k3/atomics.nova against CosmoOS main, both architectures, one file and split; the AArch64 object disassembled; benchmarks/atomics on the Ryzen box (ADR 0026 §6) | 3,524 passed (after emit-c and doc fixes); k3.py all PASS, the only undefined symbol panic; ldaxr/stlxr, ldxr/stxr, ldar, stlr inline; ordered ring 0.215 s (clang) and 0.195 s (gcc), was 0.366 s and 0.442 s |
| 2026-10-29 | M14 K7 (statics in the random program generator): --accept 400; a campaign of 120 programs from seed 200000001 found 21 C compile failures, all one bug (an unused static's library type not emitted); after the fix, 200 programs from that seed and FuzzTests | 400 of 400 accepted; 21 findings, then 0 of 200; 33 passed |
| 2026-10-29 | M14 K3 (lib/kernel, ADR 0025): local suite on macOS (clang) with check-pass/kernel-library.nova (new .hir, reviewed), check-fail kernel-library.nova and kernel-library-constant.nova, KernelLibraryTests; then integrations/cosmoos/k3.py against CosmoOS main on aarch64 and x86_64, one file and split, after the static Mutex was removed (kernel-library, corpus and formatter tests re-run) | 3,508 passed; 1,553 passed on the re-run; k3.py all PASS, CosmoOS tree unchanged |
| 2026-10-29 | M14 K1 (statics, ADR 0024): local suite on macOS (clang) with run-pass/statics.nova (every C configuration, split and TSan included, and the interpreter), check-fail statics.nova, static-moves.nova, static-initializers.nova, parse-pass/statics.nova (new .ast, reviewed); parse-fail/region-static-parameter.nova's cascade annotation updated (recovery now stops at static); check-pass/effects-of.hir re-blessed (closure ids only) | 3,487 passed after the re-bless |
| 2026-10-29 | M14 K2a (--split, ADR 0022): local suite on macOS (clang) with every run program also built split (clang -O2 split); two split builds of a 202,004-line synthetic program compared; that program over 21 files built with make -j10 and after one change; vector-growth and string-building one-file against split | 3,463 passed; byte-identical; 14.0 s (33.8 s as one file), 2.4 s after one change, same output; split as fast as one file |
| 2026-10-29 | ADR 0021 (atomic orderings) and ADR 0020 (@effectsOf): local suite on macOS (clang) with run-pass/atomic-orderings.nova (every C configuration, TSan included, and the interpreter), check-pass/effects-of.nova (new .hir, reviewed), check-fail/effects-of.nova, effects-of-attribute.nova and the changed closure-method-effects.nova; closure-calls/known.calls re-blessed (lib/core line numbers only). benchmarks/atomics on the Ryzen box and the Mac (ADR 0021 §1, §6). CI run 40829078442 (Linux, Windows MSVC) | 3,463 passed after the two snapshots; CI green |
| 2026-10-29 | Shared-path soundness fix (check-fail/shared-path-mutation.nova): local suite on macOS (clang). Then the ThreadSanitizer configuration: the run corpus (thread-both, threads built with -fsanitize=thread), FuzzTests, and 40 generated programs (seeds 100000001..100000040, TSan in the oracle); a racy C control under the same flags | 3,438 passed; run corpus 188 passed; fuzz tests 33 passed; 40 programs, 0 findings; the control is reported as a data race |
| 2026-10-22 | dotnet test on macOS arm64 (.NET 10.0.401): lexer and parser unit tests, 20 parse-pass and 18 parse-fail corpus files | 151 passed, 0 failed |
| 2026-10-22 | same, plus 11 check-pass and 19 check-fail files and the checker crash test (increment 1b) | 201 passed, 0 failed |
| 2026-10-22 | Linux CI run 39792099920 (ubuntu-24.04, clang 18.1.3, GCC 13.3.0): every corpus and run test passed under both compilers. The runtime harness failed only on GCC -Wclobbered (harness-only; fixed) | 379/380 |
| 2026-10-26 | M10 acceptance: C from M11's compiler (b8b0907) and M10's for 222 programs (run-pass, run-fail, emit-c, benchmarks, nwc); vp test in editors/vscode; Nova.Fuzz 400 programs (seeds 9400000..9400399); c4.py both arches, c6.py, c7.py --no-boot, c8.py --no-boot; programs/nwc built as a package | 221 byte-identical, 1 differing by one source line (a documented lib/core function moved); 19 passed; 0 findings; every CosmoOS check passes, tree unchanged; nwc counts as before |
| 2026-10-26 | M10 T6 (packages): local suite on macOS with PackageManagerTests (29, git from local repositories) and the package-aware LspTests case; by hand: novac new, a build before fetch (N0324), fetch and builds with a path and a git (tag) dependency, an edited checkout (N0325), --offline with and without the cache | 2,993 passed; every manual case as specified |
| 2026-10-26 | M10 T5 (names across packages, ADR 0013): local suite on macOS with PackageTests (a two-package program compiled with clang and run; 9 negative cases, N0313, N0314); the first run found check-fail/systems-capabilities.nova no longer reporting a redeclared lib/core name (N0201), fixed; c4.py both arches, c6.py, c7.py --no-boot, c8.py --no-boot | 2,962 of 2,963 passed, then the failing file and PackageTests 117 of 117 after the fix; every CosmoOS check passes, tree unchanged |
| 2026-10-26 | M10 T2–T4: local suite on macOS with DocTests, LspTests (21) and, before them, the T2+T3 state; DebuggerTests under lldb (breakpoints, nova-locals, nova-bt, nova_panic); editors/vscode with Vite+ v1.1.0: vp test (grammar 16, server through vscode-jsonrpc 3), vp lint, vp run package (a 119 KB .vsix); the real novac lsp over pipes by hand | 2,942 passed (T2+T3), then DebuggerTests 6 passed; vp test 19 passed; lint clean |
| 2026-10-26 | M10 T1 (formatter): local suite on macOS with FormatterTests (26 style cases, every parseable repository file formatted twice, the repository's own code formatted); the repository's code and README examples reformatted; c4.py both arches, c6.py, c7.py --no-boot, c8.py --no-boot on the reformatted kernel sources (c5.py boots QEMU, not run; its change is whitespace, which the formatter's own check guarantees) | 2,916 passed; every CosmoOS check passes, tree unchanged |
| 2026-10-26 | M11 acceptance: the gaps' probes rebuilt (clang and --interp); local suite on macOS (clang) with run-pass/io-error-results.nova (the probes' backend finding, fixed) and run-pass/readme-closures.nova; c4.py both arches, c6.py, c7.py --no-boot; on the shared Ryzen (m11.sh): the whole suite in every language under clang and gcc, LZ4 and fbvalid, then a 60-minute campaign (seeds from 9300000); m11-ab.sh/m11-ab2.sh: 9f's compiler and the close-out's interleaved | 8 of 8 probes run; 2,401 passed; every CosmoOS check passes, tree unchanged; Nova/C 0.89–2.45 (clang), 0.96–3.50 (gcc); campaign 8,502 programs, 0 findings; old and new compilers equal (milestone-11.md, Acceptance) |
| 2026-10-26 | M11 G8c campaign finding fixed (constants decided exactly in overflow-check elimination): local suite with run-pass/dead-constant-overflow.nova; G7d nbody checksums in all eight builds (M1: nova, c, cpp, rust, rust-checked; Ryzen: all, clang and gcc) | 2,378 passed; checksums equal |
| 2026-10-26 | M11 G8c (library methods taking closures): local suite on macOS (clang) with run-pass/closure-methods.nova (every C configuration and --interp; same output), one run-fail and three check-fail files; the generator's closures accepted (600/600 type-check) and local programs: a first 200 found the const const capture bug (fixed), then 500 (seeds 9100000..9100499), then a 60-minute campaign on the shared Debian box (seeds from 9200000, 10 jobs) | 2,367 passed; local 500 programs, 0 findings; campaign: 8,517 programs, 1 finding (older than G8, fixed: milestone-11.md G8c) |
| 2026-10-26 | M11 G8b′ (effect lists on function types): local suite on macOS (clang) with run-pass/effect-lists.nova (every C configuration and under --interp; same output), parse-pass/effect-lists.nova (new .ast, reviewed), one parse-fail and two check-fail files; no existing snapshot changed | 2,347 passed |
| 2026-10-26 | M11 G8b (closures): local suite on macOS (clang) with run-pass/closures.nova (every C configuration, ASan+UBSan included, and under --interp; same output), parse-pass/closures.nova (new .ast, reviewed) and three check-fail files. G7c's campaign on the shared box (seeds from 8200000, 60 minutes, float expressions) | 2,331 passed; campaign 8,588 programs, 0 findings |
| 2026-10-26 | M11 G8a (function values): local suite on macOS (clang) with run-pass/function-values.nova (compiled and under --interp, same output), parse-pass/function-types.nova (new .ast, reviewed) and four check-fail files; G7c on the shared box (a440608): the full suite under gcc and clang | 2,315 passed after two older tests were updated to the new rules; 2,298 passed on the box |
| 2026-10-26 | M11 G7c with CosmoOS (75e1e15f), no boots: c4.py --arch x86_64 and --arch aarch64, c6.py --component all --arch x86_64, c7.py --arch x86_64 --no-boot, each now calling novac with --no-float | every check passes; CosmoOS working tree unchanged |
| 2026-10-26 | M11 G7c (--no-float, math methods, floats in the fuzzer): local suite on macOS (clang) with run-pass/float-math.nova (compiled and interpreted, same output), check-fail/no-float.nova, new check-fail/floats.nova cases; Nova.Fuzz with float expressions, seeds 9300000–9300299: first 259 findings, all one type-checker bug (flexible float branches typed as i32), then 0 after the fix | 2,298 passed; fuzz 300 programs, 0 findings after the fix |
| 2026-10-26 | M11 G7b offline, on the shared Debian box (/root/nova-ci/g7b/run.sh, gcc, glibc): every f32 through exhaust32 (the text reads back with strtof; no decimal one digit shorter next to the value reads back), and 5,010,501 doubles (boundaries and random) against Python repr. CI for 0b4d487: Windows passed; Linux failed on the test harness, a C string without a final newline (-Wnewline-eof, newer clang), fixed in the test | 4,278,190,082 f32 values (all but the 16,777,214 NaNs), 0 bad; 5,010,501 doubles, 0 mismatches |
| 2026-10-25 | M11 G7b (printing floats): local suite on macOS (clang) with FloatFormatTests (C Schubfach against the C# exact search: 75,000 values per C configuration) and run-pass/float-printing.nova (compiled and interpreted; expected text from Python repr and numpy); offline, the C formatter against Python repr (f64) and numpy (f32) on 412,042 boundary and random values | 2,286 passed; 0 mismatches. Found on the way: .NET 10's shortest double formatting does not read back for 2^-25 |
| 2026-10-25 | G7a fix: c4.py --arch aarch64 and --arch x86_64 (no boot) after G7a. aarch64 passed; x86_64 failed: rt.h's new FLT_EVAL_METHOD #error fired under CosmoOS's -mgeneral-regs-only (no SSE). After the fix (NV_FLOATS, only in programs with floats), both pass; FreestandingTests' cross objects now build with -mgeneral-regs-only too, and fail without the fix | x86_64 FAIL, then PASS on both arches; 2,259 tests passed |
| 2026-10-25 | M11 G7a (f32/f64): local suite on macOS (clang) with run-pass/floats.nova (compiled and under --interp, same output; values cross-checked with Python's struct and numpy float32), two run-fail tests, check-fail/floats.nova and the emit-c/floats.nova snapshot (reviewed); rt.h's float helpers in a C test under clang -Wconversion -Wsign-conversion -Werror at -O0/-O2, with UBSan float-cast-overflow, and with -DNV_NO_BUILTIN_OVERFLOW | 2,259 passed; helper test passed in every configuration |
| 2026-10-25 | M11 G4b–G5 on the shared Debian box (tree at 63230c5, /root/nova-ci/g5.sh): the full suite (gcc and clang), then a 60-minute Nova.Fuzz campaign generating overflow methods and if/match expressions (seeds from 8100000, 10 jobs) | 2,220 passed; 11,059 programs, 0 findings |
| 2026-10-25 | M11 G6 (references to optionals): local suite on macOS (clang) with run-pass/optional-references.nova (compiled and under --interp, same output), check-fail/optional-references.nova and parse-pass/parenthesised-types.nova (new .ast snapshot, reviewed). CI for G5 (63230c5): Linux and Windows MSVC | 2,234 passed; CI green |
| 2026-10-25 | M11 G5 (if and match expressions): local suite on macOS (clang) with run-pass/branch-expressions.nova (compiled and under --interp, same output), parse-pass/parse-fail/branch-expressions.nova (new .ast snapshot, reviewed) and two check-fail files; Nova.Fuzz generating the expressions, seeds 9200000–9200299. CI for G4c (03e39d7): Linux and Windows MSVC. The G1 campaign on the shared Debian box (seeds from 8000000, 60 minutes, 10 jobs) | 2,220 tests, all passing after one annotation fix; 300 programs, 0 findings; CI green; campaign: 11,550 programs, 0 findings |
| 2026-10-25 | M11 G4c (String.appendNumber): local suite on macOS (clang) with run-pass/append-number.nova (compiled and under --interp, same output; in the fault-injection sweep) and check-fail/append-number.nova. CI for G4b (4b015c9): Linux gcc/clang and Windows MSVC | 2,204 passed; CI green |
| 2026-10-25 | M11 G4b (wrapping*, checked*, saturating*): local suite on macOS (clang) with run-pass/overflow-methods.nova (compiled and under --interp, same output) and check-fail/overflow-methods.nova; rt.h's helpers in a C check program under clang -Wconversion -Wsign-conversion -Werror, with the builtins and with -DNV_NO_BUILTIN_OVERFLOW (the fallback); Nova.Fuzz with the methods generated, seeds 9100000–9100299 | 2,192 passed; the check program passed both ways; 300 programs, 0 findings |
| 2026-10-25 | C8: c8.py --arch aarch64 --cycles 1000 and --arch x86_64 --cycles 1000 (CosmoOS 41368f9e, debug build, QEMU -smp 4; C8b in a copy of CosmoOS with proposals/module-cycle/ applied): 3 negative cases, the host test, the module loaded at boot, 1,000 cycles, the zombie control. Earlier runs: an unbounded watcher thread (one sched-balance-pull selftest failure), a zombie control whose object was made outside the module's live window (no zombie: CosmoOS finding 1) | both arches: negative cases exact; host test PASS; C8a PASS (507/507 selftests, clean shutdown); C8b 1,000 cycles PASS, every statistic back (lockdep classes +7, finding 2); zombie control: 3 of 3 unloads -EBUSY |
| 2026-10-25 | C7e: c7.py --arch aarch64 and --arch x86_64 (CosmoOS 5cfd1ffe, debug build, QEMU -smp 4): the stress module (4 threads × 2,000 rounds) and the inversion control, plus all of C7c–C7d's checks; a baseline boot without the module; a repeat of the AArch64 boot | both arches: every total exact (8,000 ×4, KRef count 1, 252 deferred frees), 4 workers overlapping on 4 CPUs, unlocked control lost updates, no lockdep report, 507/507 CosmoOS selftests, clean shutdown; lockdep reports the inversion. One earlier AArch64 run failed CosmoOS's blk-irq-budget timing selftest once; the rerun and the baseline passed. Earlier runs' cosmofs selftest failures came from stale scratch disks (Nova harness, fixed) |
| 2026-10-25 | C7d: c7.py --arch aarch64 and --arch x86_64 (CosmoOS 5cfd1ffe, macOS host): 11 negative cases, the host test against CosmoOS's object.c, both objects under MODULE_CFLAGS; local suite with ffi/raw-boxes.nova and check-fail/callbacks.nova | 11/11 negative cases exact; host test PASS (release once at the last put, deferred free at the grace period, no block live); both objects pass; 1,975 tests |
| 2026-10-25 | M9f (acceptance): benchmarks/suite/run.py with every language under clang 18 and gcc 12 on the Ryzen; three alternated rounds of Nova by default and with --no-bce, --no-oce, --no-placement and all three off, on the suite, LZ4 and fbvalid; benchmarks/suite/run_msvc.py, three rounds, on the Windows VM (MSVC 19.51, ARM64); MsvcTests on the Windows VM | every row printed its checksum (MSVC: 90 timed runs); MsvcTests 164 passed; results in milestone-9.md, 9f |
| 2026-10-25 | M9e′ (niche optionals, Box.create in place): local suite on macOS (clang) with run-pass/niche-optionals.nova, NicheLayoutTests and emit-c/boxes-in-place.nova; CI (Linux gcc/clang, Windows MSVC); c2.py --arch aarch64 --ssh (CosmoOS VM, v1.0.13); c4.py both arches; a 60-minute fuzz campaign on the Ryzen (10 jobs, seeds from 5000000); benchmarks/suite/run.py on the six workloads whose C changes, three alternated rounds of 9e and 9e′ under both compilers | 2,130 passed; CI failed once on MSVC C4101 (an unused temporary, fixed) then passed; C2: 160 programs, sweep 30 programs / 251 runs PASS; C4 PASS; fuzz: 11,603 programs, 0 findings; gains up to 3.49× (large-objects, gcc), peak heaps equal to C's |
| 2026-10-25 | M9e (placement by default, loop sites): local suite on macOS (clang) and on the Ryzen (clang 18, gcc 12 -Werror), with placement/loops.nova, emit-c/placed-boxes.nova, the two duplicated-drop mutation tests and the clang -O2 no-placement configuration; c2.py --arch aarch64 --ssh (the CosmoOS VM, v1.0.13); a 60-minute fuzz campaign on the Ryzen (10 jobs, seeds from 4000000); benchmarks/suite/run.py --workloads heap-alloc, five alternated rounds with and without --placement, both compilers | 2,117 passed locally and on the Ryzen; C2: 159 programs in the guest, guest sweep 29 programs / 242 runs, PASS; fuzz: 11,614 programs, 0 findings; heap-alloc 1.30× (clang) and 1.16× (gcc) faster, 1 allocation instead of 10,000,001; the other fourteen workloads' C unchanged |
| 2026-10-25 | M9d (overflow-check elimination, per-site check locations, zero-initialisation where needed): local suite on macOS (clang) and on the Ryzen (clang 18, gcc 12 -Werror), with overflow-checks/*.oce, run-pass/payload-reads.nova and the clang -O2 no-elim configuration; c4.py --arch x86_64 and --arch aarch64; c6.py --component all; two 60-minute fuzz campaigns on the Ryzen (10 jobs, seeds from 3000000); benchmarks/suite/run.py, benchmarks/lz4/run.py and benchmarks/fbvalid/run.py, three alternated rounds of 9c, --no-oce and 9d under both compilers | 2,016 passed locally and on the Ryzen; C4: both arches pass, location tables relocated with R_X86_64_64/R_AARCH64_ABS64; C6 PASS; first campaign: 4 findings in 400 programs, all one gcc -O0 -Wmaybe-uninitialized case (fixed, then stopped); second: 11,581 programs, 0 findings; benchmarks: every row printed its checksum (results in milestone-9.md, 9d) |
| 2026-10-25 | M9c (bounds-check elimination): local suite on macOS with BoundsCheckEliminationTests and the clang -O2 no-bce configuration; c6.py --component all; benchmarks/suite/run.py and benchmarks/lz4/run.py on the Ryzen with and without --no-bce under clang and gcc; 60-minute fuzz campaign on the Ryzen (10 jobs, seeds 2000000–2011301) | 2,000 passed; C6 PASS; LZ4 14 of 21 checks removed, gcc decompression 1.32–1.33×, clang binaries for data-oriented identical; fuzz: 11,302 programs, 0 findings |
| 2026-10-25 | M9b (bulk slice operations): local suite on macOS; c6.py --component lz4 (host test, 20,000 fuzz runs, differential, aarch64 module object); benchmarks/suite/run.py and benchmarks/lz4/run.py on the Ryzen under clang and gcc | 1,995 passed; LZ4: both host tests PASS, 0 mismatches in 20,000 cases, object meets C4's checks with memmove undefined; every benchmark row printed its checksum |
| 2026-10-24 | C7c: c7.py --arch aarch64 and --arch x86_64 (CosmoOS 5cfd1ffe, macOS host): 6 negative cases, the host test over CosmoOS's spinlock shim, the package under MODULE_CFLAGS with C4's checks; a negative control (a guard that does not unlock); local suite with PrivateFieldTests | 6/6 negative cases give exactly their errors; host test PASS; the control is caught (re-acquired (deadlock)); both objects pass (frames 64 / 56 B, only kernel exports undefined); 1,972 tests |
| 2026-10-24 | C7b: C state embedded by value (type-system §33): local suite on macOS, with ffi/embedded.nova (every C configuration), check-fail/c-state-types.nova, check-fail/c-state-moves.nova, the @placed effect case in check-fail/effects.nova, and the interpreter fault; emit-c snapshots unchanged, so no guest runs | 1,967 passed; self-pointing C list heads survive in their boxes, placement order 1–4; 14 type-level and 5 move-level rejections, nothing else |
| 2026-10-24 | C7a: context effects (type-system §32): local suite on macOS, with check-fail/effects.nova, check-fail/effect-attributes.nova and run-pass/effects.nova (every C configuration, interpreter, fault-injection sweep); generated C unchanged (emit-c snapshots), so no guest runs | 13 of 13 effect violations and 9 of 9 attribute misuses reported, nothing else; the positive program runs everywhere |
| 2026-10-24 | C6b: imported C struct types (type-system §31): local suite (ffi/imports.nova in every C configuration, check-fail/imported-c-types.nova, the interpreter fault); three hand-built mismatched declarations. c6.py --arch aarch64 and --arch x86_64 (both components; CosmoOS 98040b9b, macOS host): test_fbvalid and fuzz_fbvalid with fbvalid.c and with c6/fbvalid.nova, the differential test, the object under MODULE_CFLAGS; two negative controls. benchmarks/fbvalid/run.py on the M1 (Apple clang 21) and the Ryzen 5 PRO 3600 (clang 18, gcc 12), twice each | each mismatch is a C compile error naming the field; both validators pass the host test and 20,000 fuzz mutations; 20,000 differential cases, all 11 answers, 0 mismatches; kernel checks pass on both arches (frames 48 / 40 B, only panic undefined); C6a unchanged; a dropped check fails all three tests; the C-style wrap check panics in Nova; CosmoOS's host test does not reach the wrap check; 0.98 / 1.03 / 1.76 of C's time |
| 2026-10-24 | C6a: c6.py --arch aarch64 and --arch x86_64 (CosmoOS 98040b9b, macOS host): CosmoOS's test_lz4 and fuzz_lz4 built with its own HOST_CFLAGS/FUZZ_CFLAGS, once with lz4.c and once with c6/lz4.nova; the differential test; the Nova object under MODULE_CFLAGS with C4's checks. Negative control: an off-by-one planted in each codec. benchmarks/lz4/run.py on the Mac (Apple clang 21) and on a Ryzen 5 PRO 3600 (clang 18, gcc 12) | both codecs pass the host test and 20,000 fuzz mutations; 20,000 differential cases, 0 mismatches; kernel checks pass on both arches (largest frame 160 / 136 B, only panic undefined); the planted bug panics in Nova (lz4.nova:169:20) and is silent in C without ASan; throughput in benchmarks/lz4/README.md |
| 2026-10-24 | C5: c5.py --arch x86_64 and --arch aarch64 (CosmoOS 98040b9b, QEMU): the out-of-tree module nova_c5 (KernelHeap, the selftest) built with CosmoOS's rules, signed with its developer key, checked with check-module-elf.py and C4's checks, loaded at boot; local suite; c2.py on the VM (AArch64) and in QEMU (x86-64) with run-pass/user-allocators.nova | every check passes on both arches; the module loads and its 5 in-kernel tests pass with 0 blocks live; 15 boot modules, 0 failed; 1,945 tests; 147/147 programs and 230/230 sweep runs on both arches |
| 2026-10-24 | C4: c4.py --arch x86_64 and --arch aarch64 (CosmoOS 98040b9b): kdemo.nova built with CosmoOS's MODULE_CFLAGS and ld.lld -r, checked against modelf.c's ELF rules, modreloc.c's relocation types, the kernel's exports, the 2048-byte frame limit and check-fpregs.sh; 8 negative controls. Also c2.py on the VM (AArch64) and in QEMU (x86-64), since the core runtime changed (nova_panic_reason) | every check passes on both arches; 8/8 controls rejected; 1,932 tests; 146/146 programs and 230/230 sweep runs on both arches |
| 2026-10-24 | M8e (M8 acceptance): local suite on macOS. FreestandingTests checks the cross objects' sections and symbols (llvm-objdump -h, llvm-nm); a hand-built object with unwind tables and a mutable global was seen to show .eh_frame and .data, which the checks reject. Hosted output did not change (only freestanding flags did), so the M8d guest runs stand | 1,932 passed |
| 2026-10-24 | M8d (lib/core, volatile accesses, C function pointers, @align, @noncopy): local suite on macOS; c2.py on the CosmoOS VM over ssh (AArch64) and in QEMU (x86-64), with run-pass/mmio.nova and function-pointers.nova running in the guests | 1,932 passed; 146/146 programs and 230/230 sweep runs on both arches |
| 2026-10-24 | M8c (profiles, freestanding objects; the core runtime split by NV_FREESTANDING): local suite on macOS; c2.py and c3.py on the CosmoOS VM over ssh (AArch64), c2.py in QEMU (x86-64). Checked by hand on the demo's cross objects: no .init_array, .tdata, .tbss or .eh_frame (llvm-objdump -h) | 1,902 passed; 144/144 programs and 230/230 sweep runs on both arches, 26/26 nwc cases, c3-io |
| 2026-10-24 | M8b code on the CosmoOS VM, now at 10.211.55.15 (v1.0.12, AArch64): c2.py and c3.py over ssh | 144/144 programs, 230/230 sweep runs, 26/26 nwc cases, c3-io pass; tree unchanged |
| 2026-10-24 | M8b (C FFI): local suite on macOS; c2.py in QEMU on x86-64 and AArch64 (the CosmoOS VM still did not answer). run-pass/ffi-libc.nova calls CosmoOS libc's puts, strlen, abs, memcmp, memcpy and memset in the guests | 1,897 passed; 144/144 programs and 230/230 sweep runs on both arches |
| 2026-10-24 | M8a (unsafe and raw pointers): local suite on macOS; c2.py in QEMU on x86-64 and AArch64 (the CosmoOS VM did not answer ssh or ping). On x86-64, the check "CosmoOS working tree unchanged" failed: four files under CosmoOS tests/boot and tests/host changed during the run. That work is outside Nova (c2.py only builds into out/; the latest CosmoOS commits are in that area). The AArch64 run, after it, kept the tree unchanged | 1,880 passed; 143/143 programs and 230/230 sweep runs on both arches |
| 2026-10-23 to 24 | M7e 24-hour fuzz campaign (tests/Nova.Fuzz, commit 973ea0e) on Debian 12 x86-64, 12 jobs: clang 18.1.8 -O0/-O2/flat, GCC 12.2 -O0/-O2, interpreter. All findings were one kind, clang -Warray-bounds on a constant slice index, fixed after the first three; all 24 seeds re-run on 01aa4e0 give 0 findings | 423,773 programs, no miscompile; 24 findings, one fixed kind |
| 2026-10-23 | After the CosmoOS VM was reinstalled (now v1.0.11-rc1, AArch64): c2.py and c3.py over ssh on the current code (M5 and the campaign fix included) | 141/141 programs, 230/230 sweep runs, 26/26 nwc cases, c3-io pass |
| 2026-10-23 | M7e campaign, interim: 3 findings in the first 25,600 programs (constant slice index, -Warray-bounds); fixed, CI run 39953749461 green; c2.py in QEMU (x86-64) passes (141/141, 230/230). The CosmoOS AArch64 VM was not reachable (ssh refused), so AArch64 was not repeated | fixed |
| 2026-10-23 | M5 (placement): local suite on macOS with PlacementTests, the clang -O2 placement run configuration, and 500 fuzz programs with a placement configuration (milestone-5.md) | 1843 passed; 0 fuzz findings |
| 2026-10-23 | M7e MSVC: MsvcTests on a Windows 11 ARM64 VM (MSVC 19.51) and on x64 in CI (test-windows, run 39944590259) | 145/145 on both |
| 2026-10-23 | M7d CI run 39936293723 failed: FuzzTests under GCC found -Wtype-limits warnings in generated C (fixed, finding 7 in milestone-7.md); then 1,500 fuzz programs on the Debian x86-64 box with GCC 12.2 and clang 18.1.8 | 0 findings after the fix |
| 2026-10-23 | M7d (random program generator): local suite on macOS with FuzzTests (32 seeds through the whole oracle); campaigns of 2,000 and 500 programs (0 findings after the fixes listed in milestone-7.md: a C name collision with the runtime, a false N0272, four kinds of warnings on valid programs); c2.py/c3.py on both arches (generated C changed: nvF_ names) | 1272 passed; 141/141 programs and 230/230 sweep runs in CosmoOS, 26/26 nwc cases, on both arches |
| 2026-10-23 | M7c (NIR reference interpreter): local suite on macOS with InterpreterTests, interpreter agreement in the fault sweep, the property tests and the I/O programs (milestone-7.md) | 1216 passed |
| 2026-10-23 | M7b (structured C, now the default): local suite on macOS with the clang -O2 flat oracle configuration, -Wimplicit-fallthrough added to the strict set, and StructuredCTests; c2.py and c3.py on x86-64 (QEMU) and AArch64 (VM). Found during the work: a one-pass transfer elimination left a goto where an empty exit block's own goto was removed later (now iterated to a fixpoint) | 1058 passed; 137/137 programs, 228/228 sweep runs, 26/26 nwc cases on both arches |
| 2026-10-23 | M7a (source maps): local suite on macOS with the new clang -O0 -g line-directives run configuration and SourceMapTests; manual lldb session on novac build -g output (milestone-7.md). Default C output unchanged (all earlier snapshots equal), so guest runs were not repeated | 951 passed |
| 2026-10-23 | C3 (files, sockets, processes, nwc): local suite on macOS; c3.py in QEMU (x86-64) and on the CosmoOS VM (AArch64); c2.py on both. The first nwc matched BusyBox on 24/26 cases: BusyBox skips non-ASCII and control bytes for words and -L, and ends lines at \v, \f, \r too; fixed, then 26/26. The QEMU harness needed kernel log lines filtered and the inputs under bin/ | suite passes; 26/26 cases and c3-io on both arches; c2 137/137 programs, 228/228 sweep runs |
| 2026-10-23 | M6x (region parameters, OD-3): local suite on macOS; c2.py on the CosmoOS VM (AArch64) and in QEMU (x86-64). During the work, the probes found and fixed: reference fields assigned through (now re-pointed), loans attached to holders before their creation (now reachability-limited), field stores of fresh borrows creating no loan (now spilled to a temporary), and struct emission order for slice and collection fields | 815 passed; 136/136 programs and 228/228 sweep runs on both arches |
| 2026-10-23 | M6f (property tests, invalidation suite, OD-3 design; M6 complete): local suite on macOS; c2.py on the CosmoOS VM (AArch64) and in QEMU (x86-64). Property tests checked against planted bugs: minimum capacity 8 instead of 4 (8 of 24 model cases fail), no generation bump on slot-map removal (4 fail); both reverted | 785 passed; 133/133 programs and 217/217 sweep runs pass |
| 2026-10-23 | CI stalls investigated. Two more runs (42b102f) ended in SIGTERM after a slowdown, and one runner died without uploading logs. Not reproduced on a real x86-64 Debian 12 machine (GCC 12, clang 18.1.8, .NET 10.0.402): the committed suite passes with 12 cores (1.3 min), pinned to 4 and to 2 cores, and on 2 cores under a 7 GB memory cap (2.8 min). Under x86-64 emulation (Docker on arm64) the test host deadlocked with unreaped child processes and threads in rt_mutex_schedule; the 6d baseline was not compared there, so that may be an emulation artefact (inferred). Hardening: tools run on at most one per core, and their output is read on dedicated threads instead of the thread pool. Root cause (found with resource samples streamed into the CI log, run 39879716478): every panic aborts, and the runner pipes each abort to apport/systemd-coredump. Over hundreds of aborting runs (run-fail tests, fault-injection sweeps, ASan builds) the load average rose to 224 and free memory to 116 MB, then the job was killed. The workflow now disables crash reports (kernel.core_pattern=core, ulimit -c 0, apport stopped) | suite green on real x86-64 in every configuration; CI fixed (see the next run) |
| 2026-10-23 | Linux CI run 39860779514 (M6e commit): the first attempt stalled after run-pass/generic-containers.nova and hit the 30-minute job limit with no test failing or timing out; the re-run passed every test in 7 min. The tool runner bounded only process exit, not the end of its output, so a process left holding the pipes could block forever without a diagnosis (inferred cause); both are now bounded | transient; re-run green |
| 2026-10-23 | M6e (SlotMap, Handle, StableVector): local suite on macOS; c2.py on the Parallels VM over ssh (aarch64) and in QEMU on x86-64. Found and fixed two older bugs: borrow-checker liveness ignored Unpack definitions (a false N0240/N0241 before a consuming match binding a reference payload; regression test check-pass/scoped-payload-liveness.nova), and a discarded Copy e! result crashed the C emitter. The sweeps now detect stable-vector chunks (nova_svec_slot(). Not tested: generation retirement after 2³¹ reuses of one slot (impractical in the corpus; proposed as a runtime unit test) | 736 passed; 133/133 programs and 217/217 sweep runs (19 allocating programs) pass |
| 2026-10-23 | M6d (Vector, String; Ex9, Ex10): local suite on macOS; c2.py on the Parallels VM over ssh (aarch64) and in QEMU on x86-64. The sweeps now also detect vector growth (nova_vec_grow() | 724 passed; 131/131 programs and 204/204 sweep runs (17 allocating programs) pass |
| 2026-10-23 | M6c (optionals, StringView): local suite on macOS; c2.py in QEMU on x86-64 and on the Parallels VM over ssh (aarch64). Two lowering bugs were found and fixed: a match on a value binding a reference payload took its address, and a string constant used as a place dropped its statement (now an internal-error invariant) | 703 passed; 125/125 programs and 181/181 sweep runs pass in both |
| 2026-10-22 | M6b (interfaces): local suite on macOS; c2.py in QEMU on x86-64 and AArch64, and on the Parallels VM over ssh (aarch64, new --ssh mode). A first QEMU x86-64 run hung in firmware while another (non-Nova) QEMU boot test shared the host; it was stopped and re-run | 685 passed; 120/120 programs and 179/179 sweep runs pass in all three |
| 2026-10-22 | M6a (generics): local suite on macOS; c2.py on x86-64 and AArch64 | 677 passed; 119/119 programs and 179/179 guest sweep runs pass on both arches. The live-allocation report caught a leak (outer instances marked Copy while inner ones were made); fixed |
| 2026-10-22 | M4c (Pool, PoolBox, var scoped values; M4 complete): local suite on macOS; c2.py on x86-64 and AArch64 | 663 passed; 117/117 programs and 170/170 guest sweep runs (11 allocating programs) pass on both arches |
| 2026-10-22 | M4b (FixedBufferAllocator, scratch frames, pinned using): local suite on macOS; c2.py on x86-64 and AArch64 | 653 passed; 115/115 programs and 166/166 guest sweep runs (10 allocating programs) pass on both arches |
| 2026-10-22 | M4a (scoped results, Arena): local suite on macOS; c2.py on x86-64 and AArch64. The first run's sweep missed region allocations (detection only looked for nova_alloc(); fixed, then re-run | 645 passed; 113/113 programs and 161/161 guest sweep runs (9 allocating programs, arena programs included) pass on both arches |
| 2026-10-22 | C2: c2.py on x86-64 and AArch64 (CosmoOS c4ac584b): corpus plus guest fault-injection sweep | 111/111 programs and 123/123 sweep runs pass on both arches; attempt counts equal the host's; local suite 633 passed |
| 2026-10-22 | M3c (moves out of boxes, consuming match, tail-iterative drops): local suite on macOS; c1.py --corpus on x86-64 (CosmoOS d45235c3) and AArch64 (c4ac584b). The first AArch64 run reported a changed CosmoOS tree because the owner committed c4ac584b during the run; the re-run was clean | 629 passed; 110/110 corpus programs pass inside CosmoOS on both arches |
| 2026-10-22 | M3b (Box, allocators, fault-injection sweep): local suite on macOS (the sweep runs every allocating run-pass program under ASan+UBSan); c1.py --corpus on x86-64 and AArch64 | 610 passed; 106/106 corpus programs (heap programs included, CosmoOS libc malloc) pass inside CosmoOS on both arches |
| 2026-10-22 | M3a (results, ?, !, early-exit temporaries): local suite on macOS; c1.py --corpus on x86-64 and AArch64 | 515 passed; 102/102 corpus programs pass inside CosmoOS on both arches |
| 2026-10-22 | M2d (enums, arrays, slices, for, mutation suite): local suite on macOS; c1.py --corpus on x86-64 and AArch64 | 504 passed (1,152 mutants checked against the oracle); 99/99 corpus programs pass inside CosmoOS on both arches |
| 2026-10-22 | M2c (moves, drops, using, defer): local suite on macOS; c1.py --corpus on x86-64 and AArch64 | 451 passed; 87/87 corpus programs (drop order included) pass inside CosmoOS on both arches |
| 2026-10-22 | M2b (references + borrow checker): local suite on macOS; c1.py --corpus on x86-64 and AArch64 | 428 passed; 81/81 corpus programs pass inside CosmoOS on both arches |
| 2026-10-22 | M2a (structs): local suite on macOS; c1.py --corpus on x86-64 and AArch64 | 405 passed; 77/77 corpus programs pass inside CosmoOS on both arches |
| 2026-10-22 | Linux CI run 39797265501 (after the -Wclobbered harness fix) | 380 passed, 0 failed (~10 min) |
| 2026-10-22 | integrations/cosmoos/c1.py --corpus on x86-64 and AArch64 (CosmoOS 30d34b29, QEMU TCG): nova-hello, nova-panic and all 70 run programs inside CosmoOS | all pass on both arches (cosmoos/c1-report.md) |
| 2026-10-22 | increment 1c: plus 51 run-pass and 19 run-fail programs under 4 clang configurations (Apple clang 21: -O0, -O2, portable overflow, ASan+UBSan), runtime/semantics agreement harness, determinism, freestanding cross-compiles, 3 C snapshots. No real GCC on this host, so the GCC configurations did not run | 379 passed, 0 failed |
2. Acceptance-test matrix
Legend: ● required for acceptance, ○ optional / best-effort, — not applicable.
Profile A (userland) and hosted
| Test group | Milestone | macOS arm64 host (hosted) | Linux x86-64 CI (hosted) | CosmoOS x86-64 QEMU | CosmoOS AArch64 QEMU |
|---|---|---|---|---|---|
| Parser/type corpus | M1 | ● | ● | — | — |
| Run-pass (clang/gcc, O0/O2) | M1 | ● | ● | — | — |
| Cross-compile with USER_CFLAGS | M1 | ● | ● | — | — |
Hello World + .note.cosmo + nm -u check | C1 | — | — | ● | ● |
| Ownership compile-fail suite | M2 | ● | ● | — | — |
| Drop order / move run tests | M2 | ● | ● | ○ | ○ |
| Heap + fault-injection sweep | M3/C2 | ● | ● | ● | ● |
| Arena/scratch escape compile-fail | M4 | ● | ● | — | — |
| Placement equivalence | M5 | ● | ● | ○ | ○ |
| Collection property tests | M6 | ● | ● | ○ | ○ |
| Utility vs reference output | C3 | ○ | ○ | ● | ● |
| Fuzz campaign (24 h) | M7 | ○ | ● | — | — |
| Reproducible build (byte-identical C and ELF) | C1+ | ● | ● | ● | ● |
Profile B (kernel)
| Test group | Milestone | Kernel build x86-64 | Kernel build AArch64 | QEMU x86-64 | QEMU AArch64 |
|---|---|---|---|---|---|
| Freestanding object: symbols / relocs / sections allowlist | C4 | ● | ● | — | — |
| Frame ≤ 2 KiB, call-chain ≤ budget | C4 | ● | ● | — | — |
| No FP registers (disassembly) | C4 | ● | ● | — | — |
| Kernel allocator adapter selftest (incl. exhaustion) | C5 | ● | ● | ● | ● |
| Component parity vs C original | C6 | ● | ● | ● | ● |
| Effect compile-fail suite | C7 | ● | ● | — | — |
| Lockdep-clean stress (debug build) | C7 | — | — | ● | ● |
| Module load/unload ×1000, no zombie, signed | C8 | ● | ● | ● | ● |
Preservation requirement for all C milestones (prompt §11): existing CosmoOS make test on both arches, host tests, sanitizer tests and the analyzer gate stay green, and CosmoOS's own build targets are unchanged unless an approved unit changes them.