Milestone 10 — Developer tooling (plan and report)

Status: complete (2026-10-26; the acceptance is the last section). The owner put M11 (language gaps) before this milestone. On 2026-10-26 the owner chose the order (tools first, modules and packages last), git dependencies as well as path ones, a VS Code extension, and a formatter with one canonical style and no options.

Why

The master prompt's Milestone 10 asks for a formatter, a package manager, a language server, debugging integration, a documentation generator and build-system improvements. Today:

• Formatting: none. The lexer discards comments (Lexer.SkipWhitespaceAndComments), and no document states a style. The repository's 465 .nova files (14,902 lines) are written in one style: 4-space indents, no tabs, braces on the line that opens them, } else on one line; 33 lines are longer than 120 columns.
• Editors: nothing. The parser recovers after an error (grammar.md §3), and diagnostics have spans, so a language server can report them as the user types; the AST has spans for every node, and the HIR has types.
• Packages: none. A compilation is one package, every file in one item scope; namespace is parsed and recorded but does not qualify names, and import is reserved but not implemented (type-system.md §1, language-spec.md §5). OD-11 proposes nova.toml with path dependencies.
• Debugging: -g and --line-directives map the generated C back to Nova lines (M7a), so gdb and lldb step through Nova source. Names in the debugger are the C names (nvF_…).
• Documentation: /// doc comments are part of the grammar (§1.2) and attach to items, but nothing reads them.
• Building: novac build takes files on the command line; there is no project file.

Plan

#	Increment	Spec	Status
T1	Formatter: novac fmt (rewrite files, or --check), one canonical style. The lexer keeps comments; the formatter prints the source's tokens and decides the whitespace between them, with a 120-column limit. Every parseable file in the repository must format to the same AST, keep every comment, and be a fixed point	formatting.md (new)	done
T2	Documentation generator: novac doc writes a Markdown page of a package's public items, their signatures as novac fmt prints them and their /// comments; --missing lists the undocumented ones. docs/api/core.md is generated from lib/core	tooling.md (new) §1	done
T3	Language server: novac lsp over stdio (LSP 3.17, JSON-RPC by hand, no new dependency). Diagnostics as you type, hover with types, go to definition, document symbols, formatting through T1. A VS Code extension in editors/vscode: the TextMate grammar and the client that starts novac lsp	tooling.md §2	done
T4	Debugging integration: a debugging guide for lldb and gdb (how Nova names and lines appear), commands that show locals and the backtrace by their Nova names (editors/debug), and a test that runs a real debugger on a -g build	tooling.md §3	done
T5	Names across packages: a dependency's public items as geometry.Point, import geometry; / import geometry.Point;, public/internal enforced between packages, package-prefixed C names	ADR 0013 (accepted 2026-10-26), type-system §2	done
T6	Packages and building: nova.toml, path and git dependencies (a commit pinned in nova.lock), novac new, novac fetch, and build/check/run/doc/fmt from a manifest; the language server checks a document with its package	ADR 0013, tooling.md §4	done

T5 and T6 change the language and the build model; they got a design and an ADR for the owner before code, as G7 and G8 did: ADR 0013, accepted on 2026-10-26 with every recommendation (package-level names, namespace retired, src/ by default, git rev/tag/branch pinned in the lock, dependencies' C names prefixed).

Each increment: spec first, positive and negative tests, the full test suite, the docs, then commit and CI.

Results

T1: the formatter

• novac fmt file.nova… rewrites files (or every .nova file under a directory), --check lists the files that would change and exits 1, --stdout prints one. A file with a syntax error is reported and left alone.

• Design (src/Nova.Compiler/Formatting/Formatter.cs): the lexer now records every comment's span (LexedSource.Comments). The formatter prints the file's own tokens in order and decides only the whitespace between them, and the trailing commas of formatting.md §3.3. A walk over the AST marks:

  • spacing roles: a call's (, generic < >, unary and postfix operators, pointer *;
  • containers that are laid out on one line or broken: blocks, bodies and match arms one per line, lists with the author's line breaks;
  • continuations: the places after =>, = or a binary operator, and before a chained ., where a long line may break.

  It renders, breaks the first construct of the first line over 120 columns, and renders again until every line fits or nothing is left to break.

• Printing the source's tokens is what makes the result safe. The formatter cannot misspell a literal or drop a parenthesis, and it checks its own output anyway. It re-parses the result and refuses to return text whose tree, tokens (commas aside) or comments differ from the original's: an internal error, never a changed program.

• Decisions made while implementing, now in the spec:

  • the blocks of an if/else chain break together;
  • attributes keep the author's line break;
  • trailing comments align over consecutive lines only;
  • a trailing comment does not count toward the width;
  • the order of the kinds to break (§3.4).

• The repository:

  • Of its 465 .nova files, 27 are parse-fail tests, which the formatter refuses as it should. Of the other 438, 253 were already in canonical form.
  • The 11 files outside the test corpus that were not, and the README's examples (the ten readme-* files), are now formatted, and the README's copies with them. The changes are long lines broken, else joined to its }, { } written {}, and hand-aligned trailing comments.
  • The corpus is not reformatted: its annotations name lines and columns.

• Tests: FormatterTests, 26 style cases (each also a fixed point), CRLF and final-newline handling, a syntax error, every parseable repository file formatted twice (697 at first, including git-ignored fuzz findings; then the tracked source directories only), and the repository's own code already formatted.

T2: the documentation generator

• novac doc path… [-o file] [--title text] writes one Markdown page for the files' public items (tooling.md §1), made from the syntax alone, so code that does not type-check yet can be documented. --missing lists the public items and methods without a doc comment, path:line: kind name, and exits 1 if there is any.
• Design (src/Nova.Compiler/Documentation/DocGenerator.cs):
  • Each file is parsed and also formatted. The two token streams are the same apart from commas, so each item's signature is cut from the formatted text: a long signature is broken as novac fmt breaks it.
  • Doc comments are read from the comments before an item's first token, attributes included, keeping the indentation after /// . The lexer's own doc text strips every leading space, which would flatten nested Markdown lists.
• docs/api/core.md is generated from lib/core. Seven public methods there had no doc comment (PhysAddr/VirtAddr of, bits, offset; MmioPtr.address); they have one now, and lib/core passes --missing.
• Tests: DocTests:
  • a package with every kind of item, public and not: the expected page, which keeps a nested list and shows a signature broken by the formatter;
  • the --missing lines;
  • a syntax error;
  • repeated headings getting distinct anchors;
  • docs/api/core.md up to date, with nothing missing in lib/core.

T3: the language server and the VS Code extension

• novac lsp serves LSP 3.17 over standard input and output (tooling.md §2). JSON-RPC and its framing are written by hand on System.Text.Json, so novac gained no dependency (src/Nova.Compiler/Tooling/LanguageServer.cs).

• Checking: each open document is checked as its own package with lib/core, from the editor's text, on every change. Its diagnostics are published with their codes, and notes become related information.

• Requests:

  • hover: a local's let/var and type, a function's signature, a field's type, any expression's type;
  • definition: locals, parameters, functions and methods, fields, and struct, enum and interface names;
  • document symbols;
  • formatting, through T1.

• The index (SemanticIndex.cs) answers hover and definition from the typed tree (HIR) and the syntax tree. It takes the innermost thing at the position. The HIR has 57 node kinds, so it is visited through its records' properties, by reflection, rather than by a switch that would have to follow every new kind. A document that does not parse has no typed tree: hover and definition then answer null, and the diagnostics say why.

• editors/vscode: the TextMate grammar, the language configuration, and a client that starts novac lsp through vscode-languageclient (the nova.server.path setting). It is built with Vite+ (vp, at the owner's suggestion), outside the .NET build. vp pack bundles the client into dist/extension.cjs: vsce could not package pnpm's node_modules layout, and one file is smaller anyway. vp run package makes the .vsix: 7 files, 119 KB.

  • vp test, run 2026-10-26: 19 passed. grammar.test.mjs (16) tokenizes Nova lines with vscode-textmate, as VS Code does, and checks each token's scope. server.test.mjs (3) drives the built novac lsp through vscode-jsonrpc, the library under VS Code's own client, so the hand-written framing is known to work with it.
  • vp lint is clean, and vp fmt keeps the JavaScript and JSON in its default style.
  • The extension has not been installed in a running VS Code.

• Tests: LspTests, through in-memory streams:

  • initialize;
  • diagnostics on open, re-checked on change and cleared on close;
  • hover and definition at the positions above, and null where nothing is;
  • symbols and formatting;
  • a document that does not parse;
  • an unknown method, and exit without shutdown;
  • the grammar's keywords equal to the compiler's.

  Also, by hand, the real novac lsp process over pipes: initialize, diagnostics, hover, shutdown, exit status 0.

T4: debugging

• Nothing needed to change in the compiler. -g already writes #line directives and compiles with -g -O0 (M7a), so lldb and gdb stop and step at Nova lines, and a breakpoint on nova_panic stops at a failed check (tooling.md §3.1). What was missing was knowing what the C names are, and seeing Nova names.
• tooling.md §3 is the guide: the commands for both debuggers, and the C name of each Nova name (c-backend.md §2.1).
• editors/debug/nova_lldb.py and nova_gdb.py add two commands, with the naming rules shared in nova_names.py:
  • nova-locals prints the frame's locals and parameters by their Nova names (p = { x: 3, y: 4 }), without the compiler's temporaries;
  • nova-bt prints the backtrace as Counter.bump at demo.nova:6.
• Tests: DebuggerTests.
  • It builds a program with -g and runs lldb (macOS) or gdb (elsewhere) in batch mode: breakpoints at two Nova lines, a method and a function, and then nova-locals and nova-bt. Also a breakpoint on nova_panic, whose backtrace must reach the overflowing function and its caller at their Nova lines. The demangling rules are checked through nova_names.py itself.
  • Run 2026-10-26 under lldb on macOS: passed.
  • gdb is in neither the Mac nor the Debian box. The Linux CI job installs it: the first run (898d0b8, gdb 15.1 on Ubuntu 24.04) passed both debugger tests through nova_gdb.py.

T5: names across packages (ADR 0013)

• Compilation.Create(PackageInfo root, dependencies, …) compiles a package with its dependencies, as one program (ADR 0013 D4). The old entry point is a package with no dependencies, so every existing program is unchanged.
• Qualified names (PackageNames.cs): before type checking, geometry.X with geometry a dependency becomes one dotted name, in types, expressions, patterns, and struct literals (which the parser joins, p.T { … }). A dependency's name cannot name anything else in the package (N0315, a rule added in the spec), so the rewrite needs no scopes, and every place in the checker that dispatches on the shape of a callee works unchanged.
• The checker keeps one item table per package (and one for lib/core and the built-in items) instead of one for everything. A lookup tries the package's own items, then a dotted name's dependency, then the file's imports, then lib/core.
  • A dependency's items are named with their package (geometry.Point) in messages and in the HIR.
  • The C backend writes the . as _ (nvT_geometry_Point, nvF_geometry_area), so two packages may declare the same name. Imported C structs keep their C names.
• Checks:
  • N0310–N0313 are reported syntactically, before any declaration. The checker's own "cannot find" for the same name is left out, so each mistake is reported once.
  • N0312 also covers fields and methods.
  • N0314 runs once types are resolved.
  • N0109 is an import after an item.
• Tests: PackageTests.
  • A program using a dependency through qualified names, an import, a generic struct and function of the dependency, an interface bound, an enum in a pattern, and the same names declared in both packages. It is compiled with clang under the strict warnings and run: the right output, and both nvT_Point and nvT_geometry_Point in the C.
  • Nine negative cases, one per check.
  • N0313 with two imports.
  • N0314.
  • An import bringing public names only.
  • A dependency that cannot see its dependent.

T6: packages, the lock file, the commands

• src/Nova.Compiler/Packages/:
  • Toml.cs, the TOML subset of manifests and locks, with N0320 at the place of the mistake;
  • Manifest.cs, nova.toml and its checks (N0321);
  • Workspace.cs, resolution, the lock, and the git cache. Nothing new is a dependency of novac: git is run as the git executable.
• Resolution (tooling.md §4.2–§4.3):
  • Dependencies are found breadth-first from the package being built. A dependency is known by its own manifest's name (N0322), and must be a library (N0321).
  • One name through two sources is N0326, naming both dependents; cycles are N0327.
  • novac fetch resolves tags and branches to commits in a bare clone of each repository ($NOVA_HOME/git/db), checks each commit out once ($NOVA_HOME/git/checkouts), and writes nova.lock: sorted by name, git entries with their commit and a SHA-256 of the checkout's manifest and sources.
  • Every other command resolves through the lock. A manifest the lock does not match is N0324, and an edited checkout N0325. A missing checkout is made again from the cached repository, even with --offline; with no cache, --offline is N0323.
• Commands:
  • novac new <name> [--lib];
  • novac fetch [--offline];
  • parse, check, build and run without files use the nearest manifest's package with its dependencies. The manifest's profile, float, kind and C files are defaults the command line can override, and an executable goes to out/<name>;
  • doc and fmt without files take the package's own sources.
• The language server checks a document of a package with that package and its locked dependencies, never fetching. Other documents are checked alone, as before.
• Run by hand, 2026-10-26:
  • novac new app and novac new geometry --lib;
  • a build before fetch (N0324);
  • fetch, then a build that prints 84 (a path dependency);
  • the same with geometry from a local git repository by tag;
  • a checkout edited in the cache (N0325);
  • --offline with the checkout deleted (rebuilt from the cache), and with no cache (N0323).
• Tests:

  • PackageManagerTests, 29:

    • every manifest field, and 13 bad manifests, each with its one code;
    • a path dependency locked (the lock's exact text) and compiled;
    • a dependency added after fetch;
    • a conflict, a diamond reaching one package, two cycles, a missing, a misnamed and an executable dependency;
    • the hash;
    • git by tag, by branch and by commit, where the lock keeps the commit when the branch moves;
    • an unknown tag, an edited checkout, and offline with and without the cache.

    The git cases use a repository the test makes, so the suite needs no network.

  • LspTests gains a document checked with its package: no diagnostics, and hover on the dependency's function, where the same text alone is an error.

• Risk R-12 (fetched dependencies) is new in risks.md.

Acceptance (2026-10-26)

The criterion (roadmap.md): each tool has its spec, its tests, and the repository's own code kept passing it. Each increment above records its spec and tests; the close-out checked the whole.

#	Spec	Tests	The repository
T1 novac fmt	formatting.md	FormatterTests: 27 style cases, every parseable file formatted twice	its own code and the README's examples are formatted, which the suite checks
T2 novac doc	tooling.md §1	DocTests	docs/api/core.md generated from lib/core, current, nothing missing
T3 novac lsp, editors/vscode	tooling.md §2	LspTests (22); vp test (19: grammar, and novac lsp through vscode-jsonrpc)	the grammar's keywords are the compiler's
T4 debugging	tooling.md §3	DebuggerTests, under lldb locally and gdb in CI	—
T5 names across packages	ADR 0013, type-system §2	PackageTests	—
T6 packages	ADR 0013, tooling.md §4	PackageManagerTests (30)	programs/nwc is a package (nova.toml, sources = ["."]): novac build there writes out/nwc, and a test resolves and checks it

Nothing else moved.

• Generated C. Every run-pass, run-fail and emit-c corpus program, the benchmark programs and nwc (222 programs) were compiled to C with M11's compiler (b8b0907) and with M10's. 221 are byte-identical. One differs by a single source location: run-pass/mmio.nova reaches PhysAddr.offset, which moved from line 10 to line 15 of lib/core/address.nova when T2 documented it.
  • T5 rewrote the type checker's name lookup, so this was the risk. Since M11's acceptance measured every benchmark, the identical C means its performance figures still hold, without rerunning them.
• CosmoOS, no boots: c4.py on both arches, c6.py, c7.py --no-boot, c8.py --no-boot: every check passes, and CosmoOS's tree is unchanged.
• Fuzzing: 400 programs locally (seeds 9400000–9400399): no findings.
• CI: every M10 commit passed on Linux and Windows. Windows ran the git tests with Windows paths; Linux ran the debugger tests under gdb.

Not done, and recorded:

• The VS Code extension was built (a .vsix), its client library tested, and installed in the owner's VS Code 1.143.1 (code --install-extension, 2026-10-26). Its behaviour in the editor has not been observed by the agent.
• namespace declarations are accepted with no effect; making them an error is left for later (ADR 0013 D2).
• Builds stay whole-program, with no cache (ADR 0013 D4): to be decided after measuring build times.
• Monomorphised closures, owning closures and effect-polymorphic functions remain open (OD-9a, OD-9b, OD-9c).

Result: M10 is complete.