Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions changelog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@ description: "Product changes and upgrade notes."
agentsfleet is in **stealth-mode testing** and pre-production. APIs and agent behavior may change between releases without long deprecation windows. Email [agentsfleet@agentmail.to](mailto:agentsfleet@agentmail.to) if you want a hand calibrating an agent or to join as a design partner.
</Tip>

<Update label="Nov 18, 2026" tags={["Upgrading", "Bug fixes"]}>
## Enable checkpoint resumption after testing

<Note>This feature is coming soon.</Note>

Checkpoint resumption now starts disabled until the deployment enables it.

## Upgrading

Set `runner.recovery.resume_from_checkpoint = true` in the daemon configuration after testing, then restart every daemon replica.
This setting needs the matching server version; it requires no separate command-line interface (CLI) change.

## Bug fixes

- **Recovery default** — Environment-only startup now uses the same disabled default as the deployment files.

See [runner recovery](/runners/recovery#controlling-checkpoint-resumption) for rollout behavior.
</Update>

<Update label="Nov 18, 2026" tags={["What's new", "Breaking", "CLI", "UI"]}>
## Recovery after a runner fails

Expand Down
86 changes: 85 additions & 1 deletion runners/install.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Install a runner
description: Put the runner binary and its signed toolbox on a Linux host, check the host, and start the runner as a service.
type: how-to
audience: operator
verified: 2026-11-09
verified: 2026-11-19
product_version: 0.75.0
executable: false
---
Expand Down Expand Up @@ -268,6 +268,90 @@ To remove the runner for good, revoke it under **Admin → Runners**.
Deleting `/var/lib/agentsfleet-runner` removes its toolbox, repository mirrors and sandbox files.
Unacknowledged results exist only in the running process and can be lost when that process stops.

## Runner-local settings

<Note>
Runner profiles are coming with the next checkpoint recovery release.
</Note>

The managed host deploy installs `/etc/agentsfleet/agentsfleet-runner.toml` and selects it through `AGENTSFLEET_RUNNER_CONFIG_FILE`.

For a manually installed service:

1. Create `/etc/agentsfleet/agentsfleet-runner.toml` using the settings below.
2. Add `AGENTSFLEET_RUNNER_CONFIG_FILE=/etc/agentsfleet/agentsfleet-runner.toml` to `/etc/default/agentsfleet-runner`.
3. Restart the runner to load the profile:

```bash
systemctl restart agentsfleet-runner
```

```text
(no output)
```

Repeat the restart after later file changes.

Without a selected file, the runner uses these defaults:

```toml
[inference]
connect_timeout_seconds = 10
stream_idle_timeout_seconds = 300

[provider_retries]
max_attempts = 3
max_stream_open_attempts = 3
initial_backoff_ms = 1000
max_retry_after_seconds = 60

[supervisor]
daemon_max_attempts = 5
daemon_initial_backoff_ms = 250
daemon_max_backoff_ms = 30_000
renewal_interval_ms = 5000
authoring_renewal_interval_ms = 10_000
authoring_renewal_retry_ms = 250
max_concurrent_file_operations = 2
```

| Table | Setting | Accepted values | Effect |
| --- | --- | --- | --- |
| `inference` | `connect_timeout_seconds` | 1–120 | Limits establishing a provider connection. |
| `inference` | `stream_idle_timeout_seconds` | 1–3600 | Limits silence between transport reads or parsed provider events. |
| `provider_retries` | `max_attempts` | 1–10 | Caps HTTP sends within each stream opening. |
| `provider_retries` | `max_stream_open_attempts` | 1–10 | Caps eligible openings before visible output or known spend. |
| `provider_retries` | `initial_backoff_ms` | 1–60,000 | Sets the first jittered exponential retry pause. |
| `provider_retries` | `max_retry_after_seconds` | 1–300 | Ends an attempt when the provider requests a longer wait. |
| `supervisor` | `daemon_max_attempts` | 1–10 | Caps sends for a bounded daemon operation. |
| `supervisor` | `daemon_initial_backoff_ms` | 1–`daemon_max_backoff_ms` | Sets the first jittered daemon retry pause. |
| `supervisor` | `daemon_max_backoff_ms` | 1–30,000 | Caps the exponential component before jitter. |
| `supervisor` | `renewal_interval_ms` | 100–5000 | Sets the batched host renewal interval. |
| `supervisor` | `authoring_renewal_interval_ms` | 100–10,000 | Sets the private authoring renewal interval. |
| `supervisor` | `authoring_renewal_retry_ms` | 1–250 | Sets the pause after a retryable authoring renewal failure. |
| `supervisor` | `max_concurrent_file_operations` | 1–16 | Caps admitted file captures and restores across this runner. |

Attempt counts include the first send.
Provider HTTP retries and eligible stream reopenings have separate limits, allowing at most nine generation-request sends per turn with these defaults.
Native token-count requests are separate and send once per opening when needed.

Known spend, visible output, and single-attempt requests prevent automatic stream reopening.

Daemon connection recovery and owed-report delivery retry until cancellation or their deadline.
They use the configured backoff without a retry-count limit.
Earlier lease and turn deadlines still stop work.

Jitter can extend a pause beyond `daemon_max_backoff_ms`.
Connection recovery also applies a compiled 30-second final pause limit.

Required file restores wait for a runner-wide admission slot while lease renewal continues.
Optional captures skip a busy boundary and preserve the previous committed checkpoint.
Each admitted operation retains the compiled four-transfer window.

The runner refuses unreadable files, unknown settings, invalid bounds, and renewal intervals longer than the supported safety windows.
Keep tokens in the environment file.
Worker count remains assigned per runner by the daemon.

## Related pages

- [Runners](/runners)
Expand Down
7 changes: 5 additions & 2 deletions runners/recovery.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,11 @@ The daemon deployment selects a Tom's Obvious Minimal Language (TOML) configurat
resume_from_checkpoint = false
```

Environment-only startup defaults to resumption enabled.
Set this value explicitly when testing a staged rollout.
Resumption defaults to disabled, including environment-only startup.
Enable it after testing recovery with an eligible replacement runner.
Comment thread
indykish marked this conversation as resolved.
Resolved

For environment-only startup, set `AGENTSFLEET_RESUME_FROM_CHECKPOINT=true` on each daemon replica and restart it.
When a selected TOML file supplies this setting, its value takes precedence over the environment.

With this setting disabled, an interrupted ordinary run with a complete checkpoint
waits after its lease expires.
Expand Down
Loading