diff --git a/CODE_OF_CONDUCT.adoc b/CODE_OF_CONDUCT.adoc new file mode 100644 index 0000000..bd2a83c --- /dev/null +++ b/CODE_OF_CONDUCT.adoc @@ -0,0 +1,24 @@ +== Contributor Covenant Code of Conduct + +=== Our Pledge + +We pledge to make participation a harassment-free experience for +everyone. + +=== Our Standards + +*Positive behavior:* * Using welcoming language * Being respectful of +differing viewpoints * Accepting constructive criticism * Focusing on +what is best for the community + +*Unacceptable behavior:* * Harassment, trolling, or personal attacks * +Publishing private information without permission + +=== Enforcement + +Report issues to the maintainers. All complaints will be reviewed. + +=== Attribution + +Adapted from https://www.contributor-covenant.org/[Contributor Covenant] +v2.1. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md deleted file mode 100644 index caeda1c..0000000 --- a/CODE_OF_CONDUCT.md +++ /dev/null @@ -1,27 +0,0 @@ - -# Contributor Covenant Code of Conduct - -## Our Pledge - -We pledge to make participation a harassment-free experience for everyone. - -## Our Standards - -**Positive behavior:** -* Using welcoming language -* Being respectful of differing viewpoints -* Accepting constructive criticism -* Focusing on what is best for the community - -**Unacceptable behavior:** -* Harassment, trolling, or personal attacks -* Publishing private information without permission - -## Enforcement - -Report issues to the maintainers. All complaints will be reviewed. - -## Attribution - -Adapted from [Contributor Covenant](https://www.contributor-covenant.org/) v2.1. - diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc new file mode 100644 index 0000000..dd089ae --- /dev/null +++ b/CONTRIBUTING.adoc @@ -0,0 +1,71 @@ +== Contributing + +Thank you for your interest in contributing! We follow a "`Dual-Track`" +architecture where human-readable documentation lives in the root and +machine-readable policies live in `+.machine_readable/+`. + +=== How to Contribute + +We welcome contributions in many forms: + +* *Code:* Improving the core stack or extensions +* *Documentation:* Enhancing docs or AI manifests +* *Testing:* Adding property-based tests or formal proofs +* *Bug reports:* Filing clear, reproducible issues + +=== Getting Started + +[arabic] +. *Read the AI Manifest:* Start with `+0-AI-MANIFEST.a2ml+` (if present) +to understand the repository structure. +. *Environment:* Use `+guix develop+` or `+direnv allow+` to set up your +tools. +. *Task Runner:* Use `+just+` to see available commands +(`+just --list+`). + +=== Development Workflow + +==== Branch Naming + +.... +docs/short-description # Documentation +test/what-added # Test additions +feat/short-description # New features +fix/issue-number-description # Bug fixes +refactor/what-changed # Code improvements +security/what-fixed # Security fixes +.... + +==== Commit Messages + +We follow https://www.conventionalcommits.org/[Conventional Commits]: + +.... +(): + +[optional body] + +[optional footer] +.... + +Types: `+feat+`, `+fix+`, `+docs+`, `+test+`, `+refactor+`, `+ci+`, +`+chore+`, `+security+` + +=== Reporting Bugs + +Before reporting: 1. Search existing issues 2. Check if it’s already +fixed in `+main+` + +When reporting, include: - Clear, descriptive title - Environment +details (OS, versions, toolchain) - Steps to reproduce - Expected vs +actual behaviour + +=== Code of Conduct + +All contributors are expected to adhere to our +link:CODE_OF_CONDUCT.md[Code of Conduct]. + +=== License + +By contributing, you agree that your contributions will be licensed +under the same license as the project (see LICENSE). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 90e87dc..0000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,66 +0,0 @@ - -# Contributing - -Thank you for your interest in contributing! We follow a "Dual-Track" architecture where human-readable documentation lives in the root and machine-readable policies live in `.machine_readable/`. - -## How to Contribute - -We welcome contributions in many forms: - -- **Code:** Improving the core stack or extensions -- **Documentation:** Enhancing docs or AI manifests -- **Testing:** Adding property-based tests or formal proofs -- **Bug reports:** Filing clear, reproducible issues - -## Getting Started - -1. **Read the AI Manifest:** Start with `0-AI-MANIFEST.a2ml` (if present) to understand the repository structure. -2. **Environment:** Use `guix develop` or `direnv allow` to set up your tools. -3. **Task Runner:** Use `just` to see available commands (`just --list`). - -## Development Workflow - -### Branch Naming - -``` -docs/short-description # Documentation -test/what-added # Test additions -feat/short-description # New features -fix/issue-number-description # Bug fixes -refactor/what-changed # Code improvements -security/what-fixed # Security fixes -``` - -### Commit Messages - -We follow [Conventional Commits](https://www.conventionalcommits.org/): - -``` -(): - -[optional body] - -[optional footer] -``` - -Types: `feat`, `fix`, `docs`, `test`, `refactor`, `ci`, `chore`, `security` - -## Reporting Bugs - -Before reporting: -1. Search existing issues -2. Check if it's already fixed in `main` - -When reporting, include: -- Clear, descriptive title -- Environment details (OS, versions, toolchain) -- Steps to reproduce -- Expected vs actual behaviour - -## Code of Conduct - -All contributors are expected to adhere to our [Code of Conduct](CODE_OF_CONDUCT.md). - -## License - -By contributing, you agree that your contributions will be licensed under the same license as the project (see [LICENSE](LICENSE)). diff --git a/README.adoc b/README.adoc new file mode 100644 index 0000000..d9ef274 --- /dev/null +++ b/README.adoc @@ -0,0 +1,163 @@ +https://github.com/sponsors/hyperpolymath[image:https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github[Sponsor]] + +== What Is This? + +Chapeliser is a *general-purpose Chapel acceleration framework* that +lets developers scale single-machine applications to distributed +clusters without learning Chapel. + +You describe your workload in a manifest (`+chapeliser.toml+`), point +Chapeliser at your code, and it generates the distributed scaffolding — +Chapel `+coforall+` loops, data partitioning, result gathering, and the +ABI/FFI bridge between your application and the Chapel runtime. + +== The Problem + +Chapel is one of the most powerful parallel programming languages ever +built. It can distribute computation across thousands of nodes with +elegant syntax. But almost nobody uses it because: + +[arabic] +. *Steep learning curve* — you must rewrite your application in Chapel +or deeply understand its interop model +. *No incremental adoption path* — it’s all-or-nothing +. *Build system complexity* — integrating Chapel with existing +Rust/C/Zig projects is non-trivial + +Chapeliser solves all three. + +== How It Works + +.... +Your application (Rust, C, Zig) + │ + ▼ +chapeliser.toml ──► Chapeliser CLI + │ │ + │ ┌───────────┴───────────┐ + │ │ │ + ▼ ▼ ▼ + Idris2 ABI Zig FFI Chapel wrapper + (formal proof of (C-ABI bridge (coforall + data + data layout + to Chapel distribution + + partition safety) runtime) gather/reduce) + │ │ │ + └────────┬───────┘ │ + ▼ │ + generated/abi/*.h ◄───────────────────┘ + │ + ▼ + Your app, now distributed +.... + +=== The Manifest + +[source,toml] +---- +[workload] +name = "my-scanner" +entry = "src/batch.rs::scan_all" # function to distribute +partition = "per-item" # split strategy +gather = "merge" # combine strategy + +[data] +input-type = "Vec" # what gets distributed +output-type = "Vec" # what comes back +serialization = "bincode" # wire format + +[scaling] +min-nodes = 1 # runs locally if alone +max-nodes = 256 # scales to cluster +grain-size = 50 # items per Chapel task +---- + +You write *zero Chapel code*. Chapeliser generates everything. + +=== Partition Strategies + +[width="100%",cols="19%,34%,47%",options="header",] +|=== +|Strategy |Description |Best For +|`+per-item+` |One item per task |File scanning, image processing +|`+chunk+` |Fixed-size chunks |Data pipelines, ETL +|`+adaptive+` |Dynamic load balancing |Heterogeneous workloads +|`+spatial+` |Domain decomposition |Simulation, matrices +|`+keyed+` |Group by key |Map-reduce, aggregation +|=== + +=== Gather Strategies + +[cols=",",options="header",] +|=== +|Strategy |Description +|`+merge+` |Concatenate all results +|`+reduce+` |Apply reduction function (sum, max, min, custom) +|`+tree-reduce+` |Logarithmic reduction for associative ops +|`+stream+` |Results stream back as they complete +|`+first+` |Return first successful result (search) +|=== + +== Architecture + +Chapeliser follows the hyperpolymath ABI-FFI standard: + +* *Idris2 ABI* (`+src/interface/abi/+`) — Dependent-type *proof +obligations*, machine-checked in CI +(`+.github/workflows/provable.yml+`); the Rust mirror in `+src/abi/+` +carries the matching runtime types and checks: +** Partition functions produce complete, non-overlapping splits +** Gather functions preserve all results +** Serialization round-trips are identity +** Memory layouts are consistent across the FFI boundary + +* *Zig FFI* (`+src/interface/ffi/+`) — C-ABI bridge between: +** The user’s application (any language with C FFI) +** The Chapel runtime (`+chpl_*+` functions) +** Memory management across the boundary + +* *Chapel codegen* (`+src/codegen/+`) — Generates: +** `+coforall+` distribution loops +** Locale-aware data placement +** Communication primitives (GET/PUT/AMO) +** Fault tolerance (retry, checkpoint, redistribute) + +* *Rust CLI* (`+src/+`) — The `+chapeliser+` command: +** Parses `+chapeliser.toml+` +** Validates workload description +** Generates Chapel + Zig + C header scaffolding +** Builds and links everything +** Provides `+chapeliser+` `+run+` for execution + +== Quick Start + +[source,bash] +---- +# Install (once published to crates.io — see ROADMAP Phase 3) +cargo install chapeliser + +# In your project directory, create chapeliser.toml (see above) +chapeliser init # generates scaffold from manifest +chapeliser build # compiles Chapel wrapper + FFI bridge +chapeliser run # executes locally (1 node) +chapeliser run -n 8 # distributes across 8 nodes +chapeliser run --cluster my-cluster.toml # full cluster +---- + +== First Consumer: panic-attacker + +The first application to be Chapelised is +https://github.com/hyperpolymath/panic-attacker[panic-attacker]’s +`+mass-panic+` (assemblyline) mode — distributing static analysis across +hundreds of repositories on a compute cluster. + +== Status + +*Pre-alpha.* The Rust CLI and code generator are implemented and tested +(63 tests). The Idris2 proofs, Zig FFI, and a golden Chapel +compile-and-run are wired into CI as the verification gate +(`+.github/workflows/provable.yml+`) — see `+ROADMAP.adoc+` Phase 1b for +their live (green/red) status. Not yet released. + +== License + +SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/README.md b/README.md deleted file mode 100644 index 83c9e95..0000000 --- a/README.md +++ /dev/null @@ -1,183 +0,0 @@ - - -[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github)](https://github.com/sponsors/hyperpolymath) - -# What Is This? - -Chapeliser is a **general-purpose Chapel acceleration framework** that -lets developers scale single-machine applications to distributed -clusters without learning Chapel. - -You describe your workload in a manifest (`chapeliser.toml`), point -Chapeliser at your code, and it generates the distributed scaffolding — -Chapel `coforall` loops, data partitioning, result gathering, and the -ABI/FFI bridge between your application and the Chapel runtime. - -# The Problem - -Chapel is one of the most powerful parallel programming languages ever -built. It can distribute computation across thousands of nodes with -elegant syntax. But almost nobody uses it because: - -1. **Steep learning curve** — you must rewrite your application in - Chapel or deeply understand its interop model - -2. **No incremental adoption path** — it’s all-or-nothing - -3. **Build system complexity** — integrating Chapel with existing - Rust/C/Zig projects is non-trivial - -Chapeliser solves all three. - -# How It Works - - Your application (Rust, C, Zig) - │ - ▼ - chapeliser.toml ──► Chapeliser CLI - │ │ - │ ┌───────────┴───────────┐ - │ │ │ - ▼ ▼ ▼ - Idris2 ABI Zig FFI Chapel wrapper - (formal proof of (C-ABI bridge (coforall + data - data layout + to Chapel distribution + - partition safety) runtime) gather/reduce) - │ │ │ - └────────┬───────┘ │ - ▼ │ - generated/abi/*.h ◄───────────────────┘ - │ - ▼ - Your app, now distributed - -## The Manifest - -```toml -[workload] -name = "my-scanner" -entry = "src/batch.rs::scan_all" # function to distribute -partition = "per-item" # split strategy -gather = "merge" # combine strategy - -[data] -input-type = "Vec" # what gets distributed -output-type = "Vec" # what comes back -serialization = "bincode" # wire format - -[scaling] -min-nodes = 1 # runs locally if alone -max-nodes = 256 # scales to cluster -grain-size = 50 # items per Chapel task -``` - -You write **zero Chapel code**. Chapeliser generates everything. - -## Partition Strategies - -| Strategy | Description | Best For | -|------------|------------------------|---------------------------------| -| `per-item` | One item per task | File scanning, image processing | -| `chunk` | Fixed-size chunks | Data pipelines, ETL | -| `adaptive` | Dynamic load balancing | Heterogeneous workloads | -| `spatial` | Domain decomposition | Simulation, matrices | -| `keyed` | Group by key | Map-reduce, aggregation | - -## Gather Strategies - -| Strategy | Description | -|---------------|--------------------------------------------------| -| `merge` | Concatenate all results | -| `reduce` | Apply reduction function (sum, max, min, custom) | -| `tree-reduce` | Logarithmic reduction for associative ops | -| `stream` | Results stream back as they complete | -| `first` | Return first successful result (search) | - -# Architecture - -Chapeliser follows the hyperpolymath ABI-FFI standard: - -- **Idris2 ABI** (`src/interface/abi/`) — Dependent-type **proof - obligations**, machine-checked in CI - (`.github/workflows/provable.yml`); the Rust mirror in `src/abi/` - carries the matching runtime types and checks: - - - Partition functions produce complete, non-overlapping splits - - - Gather functions preserve all results - - - Serialization round-trips are identity - - - Memory layouts are consistent across the FFI boundary - - - -- **Zig FFI** (`src/interface/ffi/`) — C-ABI bridge between: - - - The user’s application (any language with C FFI) - - - The Chapel runtime (`chpl_*` functions) - - - Memory management across the boundary - - - -- **Chapel codegen** (`src/codegen/`) — Generates: - - - `coforall` distribution loops - - - Locale-aware data placement - - - Communication primitives (GET/PUT/AMO) - - - Fault tolerance (retry, checkpoint, redistribute) - - - -- **Rust CLI** (`src/`) — The `chapeliser` command: - - - Parses `chapeliser.toml` - - - Validates workload description - - - Generates Chapel + Zig + C header scaffolding - - - Builds and links everything - - - Provides `chapeliser` `run` for execution - -# Quick Start - -```bash -# Install (once published to crates.io — see ROADMAP Phase 3) -cargo install chapeliser - -# In your project directory, create chapeliser.toml (see above) -chapeliser init # generates scaffold from manifest -chapeliser build # compiles Chapel wrapper + FFI bridge -chapeliser run # executes locally (1 node) -chapeliser run -n 8 # distributes across 8 nodes -chapeliser run --cluster my-cluster.toml # full cluster -``` - -# First Consumer: panic-attacker - -The first application to be Chapelised is -[panic-attacker](https://github.com/hyperpolymath/panic-attacker)’s -`mass-panic` (assemblyline) mode — distributing static analysis across -hundreds of repositories on a compute cluster. - -# Status - -**Pre-alpha.** The Rust CLI and code generator are implemented and -tested (63 tests). The Idris2 proofs, Zig FFI, and a golden Chapel -compile-and-run are wired into CI as the verification gate -(`.github/workflows/provable.yml`) — see `ROADMAP.adoc` Phase 1b for -their live (green/red) status. Not yet released. - -# License - -SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/SECURITY.md b/SECURITY.adoc similarity index 58% rename from SECURITY.md rename to SECURITY.adoc index 9ff5183..9df6ba7 100644 --- a/SECURITY.md +++ b/SECURITY.adoc @@ -1,21 +1,24 @@ -# Security Policy +== Security Policy -## Supported Versions +=== Supported Versions -| Version | Supported | -|---------|-----------| -| 0.1.x | ✅ | +[cols=",",options="header",] +|=== +|Version |Supported +|0.1.x |✅ +|=== -## Reporting a Vulnerability +=== Reporting a Vulnerability Please report security vulnerabilities to: j.d.a.jewell@open.ac.uk Do NOT open a public issue for security vulnerabilities. -## Response Time +=== Response Time -We aim to respond within 48 hours and provide a fix within 7 days for critical issues. +We aim to respond within 48 hours and provide a fix within 7 days for +critical issues. -## Scope +=== Scope This policy covers the chapeliser CLI tool and its generated artifacts. diff --git a/TEST-NEEDS.adoc b/TEST-NEEDS.adoc new file mode 100644 index 0000000..eec58fd --- /dev/null +++ b/TEST-NEEDS.adoc @@ -0,0 +1,34 @@ +== TEST-NEEDS.md — chapeliser + +=== CRG Grade: C — ACHIEVED 2026-04-04 + +=== Current Test State + +[cols=",,",options="header",] +|=== +|Category |Count |Notes +|Test directories |2 |Location(s): /tests, /verification/tests +|CI workflows |22 |Running tests on GitHub Actions +|Unit tests |Built-in |Rust/cargo test framework +|Integration tests |Configured |Via integration/ directory +|=== + +=== What’s Covered + +* [x] Rust unit test suite (cargo test) +* [x] Documentation tests +* [x] Example programs with tests + +=== Still Missing (for CRG B+) + +* [ ] Code coverage reports (codecov integration) +* [ ] Detailed test documentation in CONTRIBUTING.md +* [ ] Integration tests beyond unit tests +* [ ] Performance benchmarking suite + +=== Run Tests + +[source,bash] +---- +cargo test +---- diff --git a/TEST-NEEDS.md b/TEST-NEEDS.md deleted file mode 100644 index 6783144..0000000 --- a/TEST-NEEDS.md +++ /dev/null @@ -1,31 +0,0 @@ -# TEST-NEEDS.md — chapeliser - -## CRG Grade: C — ACHIEVED 2026-04-04 - -## Current Test State - -| Category | Count | Notes | -|----------|-------|-------| -| Test directories | 2 | Location(s): /tests, /verification/tests | -| CI workflows | 22 | Running tests on GitHub Actions | -| Unit tests | Built-in | Rust/cargo test framework | -| Integration tests | Configured | Via integration/ directory | - -## What's Covered - -- [x] Rust unit test suite (cargo test) -- [x] Documentation tests -- [x] Example programs with tests - -## Still Missing (for CRG B+) - -- [ ] Code coverage reports (codecov integration) -- [ ] Detailed test documentation in CONTRIBUTING.md -- [ ] Integration tests beyond unit tests -- [ ] Performance benchmarking suite - -## Run Tests - -```bash -cargo test -``` diff --git a/TOPOLOGY.md b/TOPOLOGY.adoc similarity index 70% rename from TOPOLOGY.md rename to TOPOLOGY.adoc index 780f8bd..7550e60 100644 --- a/TOPOLOGY.md +++ b/TOPOLOGY.adoc @@ -1,14 +1,18 @@ - - -# TOPOLOGY.md — chapeliser +== TOPOLOGY.md — chapeliser -## Purpose +=== Purpose -chapeliser is a general-purpose Chapel acceleration framework that distributes any workload across Chapel clusters without the user writing Chapel code. It reads a `chapeliser.toml` manifest describing entry points, data partition strategies, and gather strategies, then generates Chapel wrapper code, a Zig FFI bridge, and C headers. chapeliser is priority #2 in the -iser family, targeting any compute workload that benefits from data-parallel or distributed execution. +chapeliser is a general-purpose Chapel acceleration framework that +distributes any workload across Chapel clusters without the user writing +Chapel code. It reads a `+chapeliser.toml+` manifest describing entry +points, data partition strategies, and gather strategies, then generates +Chapel wrapper code, a Zig FFI bridge, and C headers. chapeliser is +priority #2 in the -iser family, targeting any compute workload that +benefits from data-parallel or distributed execution. -## Module Map +=== Module Map -``` +.... chapeliser/ ├── src/ │ ├── main.rs # CLI entry point (clap): init, validate, generate, build, run, info @@ -20,11 +24,11 @@ chapeliser/ ├── verification/ # Proof harnesses ├── container/ # Stapeln container ecosystem └── .machine_readable/ # A2ML metadata -``` +.... -## Data Flow +=== Data Flow -``` +.... chapeliser.toml manifest │ ┌────▼────┐ @@ -43,4 +47,4 @@ chapeliser.toml manifest ┌────▼────┐ │ Chapel │ compile + distribute across Chapel cluster └─────────┘ -``` +.... diff --git a/llm-warmup-dev.adoc b/llm-warmup-dev.adoc new file mode 100644 index 0000000..69e7769 --- /dev/null +++ b/llm-warmup-dev.adoc @@ -0,0 +1,19 @@ +== LLM Warmup — chapeliser (Developer) + +=== What is chapeliser? + +See README.adoc for overview. + +=== Key Commands + +* `+just setup+` — set up development environment +* `+just build+` — build the project +* `+just test+` — run tests +* `+just doctor+` — diagnose issues +* `+just heal+` — attempt auto-repair + +=== Quick Context + +* License: MPL-2.0 +* Part of hyperpolymath ecosystem +* See EXPLAINME.adoc for architecture diff --git a/llm-warmup-dev.md b/llm-warmup-dev.md deleted file mode 100644 index 8e58a2f..0000000 --- a/llm-warmup-dev.md +++ /dev/null @@ -1,16 +0,0 @@ -# LLM Warmup — chapeliser (Developer) - -## What is chapeliser? -See README.adoc for overview. - -## Key Commands -- `just setup` — set up development environment -- `just build` — build the project -- `just test` — run tests -- `just doctor` — diagnose issues -- `just heal` — attempt auto-repair - -## Quick Context -- License: MPL-2.0 -- Part of hyperpolymath ecosystem -- See EXPLAINME.adoc for architecture diff --git a/llm-warmup-user.adoc b/llm-warmup-user.adoc new file mode 100644 index 0000000..57cce42 --- /dev/null +++ b/llm-warmup-user.adoc @@ -0,0 +1,19 @@ +== LLM Warmup — chapeliser (User) + +=== What is chapeliser? + +See README.adoc for overview. + +=== Key Commands + +* `+just setup+` — set up development environment +* `+just build+` — build the project +* `+just test+` — run tests +* `+just doctor+` — diagnose issues +* `+just heal+` — attempt auto-repair + +=== Quick Context + +* License: MPL-2.0 +* Part of hyperpolymath ecosystem +* See EXPLAINME.adoc for architecture diff --git a/llm-warmup-user.md b/llm-warmup-user.md deleted file mode 100644 index 1f0ff11..0000000 --- a/llm-warmup-user.md +++ /dev/null @@ -1,16 +0,0 @@ -# LLM Warmup — chapeliser (User) - -## What is chapeliser? -See README.adoc for overview. - -## Key Commands -- `just setup` — set up development environment -- `just build` — build the project -- `just test` — run tests -- `just doctor` — diagnose issues -- `just heal` — attempt auto-repair - -## Quick Context -- License: MPL-2.0 -- Part of hyperpolymath ecosystem -- See EXPLAINME.adoc for architecture