Skip to content

Track 1.0.0: self-hosted compiler with verified bootstrap fixpoint - #1

Merged
dev-dami merged 8 commits into
mainfrom
bootstrap-v1
Oct 31, 2026
Merged

dev-dami merged 8 commits into
mainfrom
bootstrap-v1

Conversation

@dev-dami

@dev-dami dev-dami commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Summary

Track is now self-hosting. trackc (compiler/src/) is written entirely in Track and compiles itself to a fixed point: the Rust-built and Track-built compilers emit byte-identical C (20,125 lines), and the stage 2 and stage 3 executables are byte-identical.

Self-hosted compiler

  • Lexer, full-grammar parser, whole-program module loader, ownership checker (same rules as src/checker/), and a portable C99 backend.
  • CLI: trackc build | run | check | emit-c | tokens | --version, with error[Exxxx] diagnostics and source frames.
  • runtime/track_runtime.c: one word-ABI runtime shared by both compilers (embedded into trackc by scripts/embed_runtime.sh).

Bugs fixed in the Rust compiler

  • Cranelift backend moved to a uniform 64-bit word ABI (cross-module bool returns had garbage upper bits; C int results lost their sign).
  • argv never reached os_args_count/os_arg.
  • &&/|| were bitwise and didn't short-circuit.
  • return inside match arms / while bodies / nested ifs panicked the backend.
  • if/match values, dangling returned tuples, pointer-compared string patterns, lost union payloads, ignored enum discriminants.
  • Cross-module constants compiled to 0; constants used before declaration were rejected.
  • Block bindings leaked into the enclosing scope; let mut leaked across functions; assignment to undeclared variables was accepted.
  • Vec/math_* runtime ABI bugs; O(n) str_substr/str_char_at; open-ended slices failed to parse.
  • yard reused stale objects after compiler changes or imported-constant changes.

New language support (both compilers)

  • Structs (literals, field read/update, struct patterns) and bounds-checked arrays with slicing and array_len.

Verification

  • scripts/bootstrap.sh: stage 0 → 3 with the fixed-point gate (now in CI).
  • tests/selfhost_tests.rs: fixed point, example parity (stdout + exit code), 29 shared accept/reject verdicts, diagnostic frames, native suite under trackc.
  • tests/codegen_semantics_tests.rs: run-and-compare regressions for each backend fix.
  • Locally: 115 Rust tests pass, yard test passes, cargo clippy -- -D warnings and cargo fmt --check clean.

Known limitations

  • Stage 0 still needs the Rust compiler; yard, the track CLI, and the LSP remain in Rust.
  • Neither compiler frees linear values at scope exit yet (docs corrected; on the roadmap).
  • Verified on Linux x86-64 only.

See docs/bootstrap.md and CHANGELOG.md.


Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Released Track 1.0 with a self-hosted compiler that can check programs, emit C, build executables, and run programs.
    • Added support for structs and fixed-length arrays, including field access, pattern matching, indexing, and slicing.
    • Added a shared runtime with bounds-checked arrays and expanded process, file, string, and networking capabilities.
    • Improved compiler diagnostics and ownership and mutability checks.
  • Documentation

    • Updated the language specification, examples, and guides for Track 1.0, compiler commands, runtime behavior, and bootstrap verification.
  • Bug Fixes

    • Improved compiler behavior for control flow, values, and array bounds; added broader compiler and runtime verification.

… valued if/match, tuple returns, union payloads

The Cranelift backend lowered values with mixed I8/I32/I64 types and
declared cross-module imports with guessed signatures, which produced
garbage upper bits on bool returns and sign-lost C int results. All
values are now 64-bit words, matching the self-hosted C backend.

Also fixes:
- os_args_count/os_arg always saw 0 args (argv never reached runtime)
- && and || were bitwise (1 && 2 == 0) and did not short-circuit
- return inside match arms / while bodies / nested ifs panicked
  ("block already filled")
- if/match used as expressions read values from non-dominating blocks
- returning a tuple-literal local returned a dangling stack pointer
- string literal patterns compared pointers
- union payload variants lost their payload; enum explicit
  discriminants were ignored; variant lookup used bare names
- runtime functions returning int lost the sign of -1 results
- str_substr/str_char_at scanned the entire string per call

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
Move the runtime out of a Rust string into runtime/track_runtime.c so
the self-hosted compiler can link the exact same implementation. Every
exported function now takes and returns 64-bit words, which fixes
Vec (a 16-byte struct return was truncated to its data pointer) and
math_* (declared with double parameters but called with integers).
Adds collection pop/truncate, a string builder, whole-file read/write,
unformatted output, and track_str_bytes for O(1) source access.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
trackc is written entirely in Track and compiles itself to a fixed
point: stage1 (built by the Rust compiler) and stage2 (built by stage1)
emit byte-identical C, and stage2/stage3 executables are identical.

- base.trk: word vectors, string builders, interning, context, diagnostics
- lexer.trk / parser.trk: complete Track grammar into a node pool
- loader.trk: whole-program loading of imported modules
- symbols.trk / checker.trk: ownership checker mirroring LinearChecker
- cgen.trk: portable C99 backend on the shared word-ABI runtime
- runtime_c.trk: generated by scripts/embed_runtime.sh

Host fixes found along the way: constants used before their
declaration, and constants used across modules compiled to 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
…ache staleness

- scripts/bootstrap.sh runs stage0 -> stage3 and gates on identical C
  and identical stage2/stage3 executables
- tests/selfhost_tests.rs: fixpoint, example parity (stdout + exit code),
  22 accept/reject verdicts shared by both checkers, diagnostic frames,
  and the native suite under trackc
- native tests for the new lexer, parser, checker, and C emitter

Host fixes found by the parity tests:
- codegen: bindings inside if/while/for/match/with blocks leaked into
  the enclosing scope (shadowing changed the outer variable)
- checker: `let mut x` made every later `x` mutable, in any function;
  mutability is now lexically scoped and params/loop vars are immutable
- checker: assignment to an undeclared variable was silently accepted
- yard: the build cache keyed objects only on their own source, so
  compiler upgrades and changes to imported constants reused stale objects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
Bump track, yard, and trackc to 1.0.0. CI now checks the embedded
runtime is current, builds and tests compiler/ with yard, and gates on
the stage0 -> stage3 bootstrap fixed point.

Docs: docs/bootstrap.md (pipeline, word ABI, C backend contract,
runtime embedding), roadmap through v1.0.0, CHANGELOG, README, and SPEC
sections on mutability and the runtime value model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
Struct literals, field reads, and arrays parsed but evaluated to 0.
Both backends now lower them to heap cells: structs are
[name hash, fields in declaration order] (a field name shared by
several structs dispatches on the tag) and arrays are
[length, elements...] with bounds-checked get/set and slice copies.

- p.x = v, t.0 = v, and a[i] = v update in place; the root variable must
  be let mut or a lens alias
- struct patterns match and bind by field
- array_len(&a); a[s..e] and a[s..] slices
- unknown struct fields and unknown field names are compile errors

Also fixes open-ended slices (a[2..]) failing to parse and generic calls
inside index/slice expressions skipping monomorphization. The bootstrap
fixed point still holds (20125 lines of identical C).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
Neither compiler inserts frees at scope exit; the docs claimed both.
State what v1.0 does (manual frees rejected, memory reclaimed at exit)
and list scope-exit deallocation on the roadmap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQbdbWq7kNpFj8roT4CZSq
@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

The change adds a Track-written compiler with a full frontend, ownership checker, C backend, and command-line interface. It adds a shared C runtime and bootstrap verification. The Rust compiler, tests, release metadata, examples, and documentation are updated for version 1.0.0 and the compiler changes.

Changes: Track compiler and runtime

Layer / File(s) Summary
Native compiler frontend
compiler/src/base.trk, compiler/src/ast.trk, compiler/src/lexer.trk, compiler/src/parser.trk, compiler/tests/*
The Track compiler adds shared context utilities, a node-pool AST, a token-based lexer, and a recursive-descent parser. Native tests cover lexer, parser, AST, and backend behavior.
Module loading and program checking
compiler/src/loader.trk, compiler/src/symbols.trk, compiler/src/checker.trk
The compiler loads modules, collects symbols, and checks ownership, borrowing, types, assignments, and control flow.
Shared runtime and native C backend
runtime/track_runtime.c, compiler/src/runtime_c.trk, compiler/src/cgen.trk, scripts/embed_runtime.sh
A C runtime provides process, memory, I/O, collection, array, and socket functions. The Track backend emits C for functions, control flow, patterns, structs, arrays, tuples, and runtime calls.
Compiler commands and bootstrap
compiler/src/main.trk, scripts/bootstrap.sh, .github/workflows/ci.yml, .gitignore
The trackc CLI adds check, build, run, C-emission, token-dump, help, and version commands. The bootstrap script builds three compiler stages and compares generated C and executables. CI checks compiler sources and the bootstrapped version.
Rust compiler checker metadata
src/ast/mod.rs, src/checker/mod.rs
The Rust compiler adds struct expressions, stores type metadata for code generation, checks struct fields, and tracks mutability across scopes and control flow.
Rust compiler backend and integration
src/codegen/mod.rs, src/lib.rs, src/mono.rs, src/parser/*, src/yard/builder.rs, tests/codegen_semantics_tests.rs
The Rust backend uses a word ABI and adds code generation for structs, arrays, unions, and tuple returns. The compiler passes checker metadata to code generation, updates parsing and source traversal, and fingerprints package inputs for build caching. Runtime tests cover execution results and bounds-checked array access.
Release, documentation, and parity tests
Cargo.toml, compiler/Track.toml, CHANGELOG.md, README.md, SPEC.md, docs/*, examples/*, tests/*
Release metadata and documentation describe version 1.0.0, runtime and language behavior, and bootstrap stages. Tests compare host and self-hosted compiler results, diagnostics, and bootstrap outputs. Examples demonstrate slice, struct, and array operations.

Estimated code review effort: 5 (Critical) | ~120 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant Trackc as trackc CLI
  participant FrontEnd as front_end
  participant Loader as load_program
  participant Parser as parse_file
  participant Checker as check_program
  participant Backend as generate_c
  participant Runtime as runtime_source
  User->>Trackc: run build or emit-c
  Trackc->>FrontEnd: compile source path
  FrontEnd->>Loader: load root and imported modules
  Loader->>Parser: parse each source file
  FrontEnd->>Checker: collect symbols and check program
  Trackc->>Backend: generate C from checked context
  Backend->>Runtime: include runtime implementation
Loading
Merge Risk: High · up to da1ee

The new compiler can silently miscompile valid programs, producing dangling tuple pointers, unchecked struct field access that can corrupt memory, constants read as 0, and wrong array iteration. These should be fixed before merge, or the limitations explicitly accepted.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dev-dami
dev-dami merged commit 5bcd5b2 into main Oct 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants