Status: a way of working, written down after doing it on this repository with release 2.1.0. It uses
the step commands (reference) and deconvolute analyze. No released command finds the
scattered state for you; that part is reading, and a research program that helps with it is described
at the end.
The running example is real: the batch runner of lesscode-core itself, step batch-progress. Its
declaration, verdict and both versions are in examples/batch-progress.
If the idea is new to you, read the made-up opener first.
State is scattered when one fact about where the program stands is kept in several places, and different functions set and ask them. The usual signs:
- Two or three flags, locks or counters that sit next to each other in one struct or class.
- One function sets a value, another function asks it, and the two never mention each other.
- A rule you can only state by naming two of them together: "take a job only while the run has not stopped".
In the batch runner it looked like this. Workers shared three locked fields:
struct Shared<'a> {
queue: Mutex<VecDeque<Job>>,
results: Mutex<Vec<MutantResult>>,
/// A failed restoration: the run stops and is void.
fatal: Mutex<Option<String>>,
// ...
}and each worker asked two of them in turn before every job:
loop {
if self.fatal.lock().unwrap().is_some() {
break;
}
let Some(job) = self.queue.lock().unwrap().pop_front() else {
break;
};
// ...
}Say in words what the states are and what moves between them. If you cannot, you have not found the machine yet.
For the batch runner: a batch is running or stopped. While it runs, a worker takes the next job, and puts it back if it could not run it. Once it has stopped, no worker takes another job. At the end the runner asks what is left, what was done, and why it stopped.
That is one value with three parts and one question, "the next job, unless the run has stopped".
Take one machine per step. Make a branch step/<name> and a small spec that says what stays the same
and what gets simpler. Keep the change small enough to review in a minute.
The declaration names the file, the functions and the project's own test command (fields).
- Put every function that sets or asks the state into
focus. Their planted faults are run even where your change does not touch them. Here:one,worker,run_batch. - Set
max_token_growthbefore you write the change, and do not raise it afterwards. - Name everything the tests start. The tests run in a fresh copy of your repository, and only what the commands in the declaration build exists there.
The last point is the one that goes wrong. Our first declaration named two test suites:
"test": ["cargo", "nextest", "run", "--locked", "--manifest-path", "native/Cargo.toml",
"--test", "driver_run", "--test", "driver_interrupt"]With a --test filter Cargo builds those two test programs and nothing else. The suites start a
small example program, and it was not there. The fix is a build command:
"build": ["cargo", "build", "--locked", "--manifest-path", "native/Cargo.toml", "--examples"]build runs before the tests for every planted fault, so the example is rebuilt with the fault in
it. Use setup only for things that do not depend on the code under test, such as installing
dependencies; setup runs once per copy.
Before any fault is planted, the tests run once on the base. If they do not pass there, the step is refused and nothing else runs:
step batch-progress: refused
the project's tests do not pass on the base
That took one minute and was right. Release 2.1.0 does not say why the tests failed; run the test
command yourself in the copy under the work directory (base-run/w1/tree) to see it. After 2.1.0 the
reason names the step that failed and the end of its output.
With 2.1.0, do not commit a refused verdict. We did, by adding the whole spec folder, and
step record then answered "not recorded: the verdict is refused" for the accepted verdict on disk,
until that one was committed. After 2.1.0 step record judges the verdict on disk, which is the one
it writes down.
The check plants small faults in the functions of the step and counts which ones the tests catch. Before our change:
| Function | Faults noticed | Faults unnoticed |
|---|---|---|
one |
1 | 5 |
worker |
8 | 5 |
run_batch |
0 | 3 |
An unnoticed fault is a behaviour that a change can alter without any test failing. One of ours is a transition of the machine itself: dropping the statement that puts a job back into the queue after a failed run goes unnoticed.
This is where a pin belongs: a test that fails on that fault, committed by itself before the change
(step pin). We did not write one for this step, and say
so: the put-back is as unprotected after the step as it was before.
/// How far the batch is.
struct Progress {
queue: VecDeque<Job>,
results: Vec<MutantResult>,
/// A failed restoration or a stop from outside: the run ends and is void.
fatal: Option<String>,
}
impl Progress {
/// The next mutant to run; none once the run has stopped.
fn next(&mut self) -> Option<Job> {
if self.fatal.is_some() {
return None;
}
self.queue.pop_front()
}
}The worker's loop asks once:
loop {
let Some(job) = self.progress().next() else {
break;
};
// ...
}Run your formatter, lints and tests yourself first. The check runs the tests again in copies.
lesscode-core step check --declaration .kiro/specs/batch-progress/step.json --work ../workstep batch-progress: accepted
It took 14 minutes here. What the verdict says:
| Check | Result |
|---|---|
| Changes stay in scope | pass |
| Size | 2,761 tokens before, 2,757 after; allowed growth was 80 |
| Tests on the base and on the step | pass |
| Planted faults | in no function are more faults unnoticed after the step than before |
| Commands tied to the spec's criteria | pass |
| Function | Noticed before | Noticed after | Unnoticed before | Unnoticed after |
|---|---|---|---|---|
one |
1 | 1 | 5 | 5 |
worker |
8 | 7 | 5 | 5 |
run_batch |
0 | 0 | 3 | 3 |
next (new) |
1 | 0 |
The one fault that left worker is the one now noticed in next: the question moved, and its
protection moved with it.
If the check refuses, read the reasons and fix the step. Do not raise the limit, narrow the tests or move a function out of reach to get past it; that is the owner's decision, not the author's.
step check counts the tokens of the file. It does not look at single routines. For that, this
repository uses kiro-cli from OpenIDEA Kiro, with the
limits declared in .kiro/code-rules.toml:
kiro-cli code check --changed <base>It compares every changed routine with itself at the base and blocks one that is above a limit and got worse. On our step it did:
runner.rs: ERROR CODE_RATCHET routine `run_batch`: code_lines rose from 60 to 61, limit 60 (default)
We had already merged. run_batch was shortened afterwards. Run this before the record.
Expect the routine that now holds the decisions to go up. That is the trade: several small deciding routines for one larger one, and the limit per routine keeps it honest.
lesscode-core step record --declaration .kiro/specs/batch-progress/step.json
git add <verdict> <evidence> <ledger> && git commit
lesscode-core step guard --branch step/batch-progressThe step is done when the guard exits 0 on the committed branch. Merging is a person's decision.
| Measure | Before | After |
|---|---|---|
| Locked values the workers share | 3 | 1 |
| Questions a worker asks before a job | 2, under two locks | 1, under one lock |
deconvolute analyze: branch conditions in the file |
13 | 13 |
deconvolute analyze: declared gaps in the file |
3 | 1 |
kiro-cli: decisions in the file (cyclomatic sum minus routines) |
24 | 24 |
kiro-cli: cyclomatic of worker |
7 | 6 |
The conditions did not get fewer. The check on "stopped" moved from the loop into next. What
changed is that "stopped" and "nothing left" can no longer be asked apart.
- That the two versions behave the same. Planted faults measure how sharp the tests are.
- Behaviour under real concurrency. Before, a worker could see "not stopped", and another worker could stop the run before the first one took its job. One lock closes that gap. No test here observes it, in either version.
- Whether the step was worth taking.
A research program in this repository lists values of the kinds that hold shared state in Rust
(Mutex, Cell, atomics and a few more), who writes and reads them, and which conditions depend on
them. On this repository it named queue and fatal as two machines and results as shared. On a
remote desktop client it had never read, it found eight of nine machines a person had listed
beforehand. It does not see state in plain fields, and after our step it no longer sees this machine
either, because the state is in plain fields behind one lock now. It is not part of a release. See
the research log, R156 to R161.