Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,11 @@ restating commands, and `pages.yml` calls `site`, so the local and CI commands a
same text; `test-fast` is local-only. Arm wall times on the host that
last refreshed them are generated in
[docs/_generated/gate-measurements.md](docs/_generated/gate-measurements.md); never
hand-edit those numbers. There is no types arm: `mypy` is configured in
`pyproject.toml` but has never been a gate and reports 89 errors (mostly missing
third-party stubs). See [development](docs/development.md).
hand-edit those numbers. There is no types arm: use the version-bound
`python scripts/check_types.py` diagnostic, which retains all configured mypy
rules and package modules. See [development](docs/development.md) for its scope.

`pytest` enforces the 80% coverage gate declared in `pyproject.toml`. Live-provider
`pytest` enforces the 90.01% coverage gate declared in `pyproject.toml`. Live-provider
tests are opt-in via `uv run python -m pytest --run-real-api -m real_api` and need
credentials. A checkout whose `.venv` has not been synced cannot run the suite
offline; see [`TODO.md`](TODO.md).
Expand Down Expand Up @@ -88,8 +88,8 @@ conventions to follow when changing Symposium.

- Unit tests for functions, integration tests for end-to-end workflows, and
live-provider tests only behind the opt-in `real_api` marker.
- Coverage regression gate: `--cov-fail-under=80` in `pyproject.toml`; raising
it is tracked in `TODO.md`.
- Coverage regression gate: `fail_under = 90.01` with `precision = 2` in
`pyproject.toml`; retain historical measurements without restamping them.
- Mocking is expected where it keeps tests deterministic: realistic fixtures
and edge cases instead of live calls.
- `ruff check` and `ruff format --check` are the lint and format gates; CI runs
Expand Down
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,33 @@

All notable changes to the Symposium package.

## [Unreleased] — 2026-11-13 (offline correctness qualification)

- Treat HTTP 402 as terminal before OpenRouter's retry-text classification.
- Count presenter/profile/project completion only after all requested reports are
persisted; retain completed/partial files and fail incomplete or empty CLI work.
- Consolidate configured provider creation in `APIClient.from_config`, preserving
explicit config selection, overrides and Perplexity background workflows.
- Preserve ISO timestamp instants and reject invalid timestamps in the legacy
calendar exporter; publish through an owned temporary file and atomic replacement
without deleting an existing destination on failure.
- Reject non-object JSON report/data roots and malformed provider options/keys;
retain the existing `Participant` record schema and private credential boundaries.
- Reject public PDF pages without indirect identity and invalid snapshot timestamp
types before publication output creation.
- Qualify the 90.01% coverage floor with precision 2 using retained coverage from
902 passing offline tests (5,013/5,547 statements, 90.37%); keep the original
fractional-precision failure and dated generated measurements as historical evidence.
- Replace the future branch-publication reset/amend recipe with an ordinary
fetched-parent commit. Disposable local proof covers staged bytes, CRLF,
identical-tree no-op and stale-push rejection; no hosted publication occurred.
- Qualify all 51 package modules with version-bound local typing while preserving
all 64 original runtime versions; no parser pin or scientific artifact changed.
- Add an explicitly version-bound complete-package typing command, consumed
plotting declarations and temporary CSR/community signature corrections.
Preserve installed package files, numerical versions, checked modules and
error rules; use WordCloud's actual array conversion at the plotting boundary.

## [Unreleased] — 2026-11-12 (gate and documentation)

### Participant figure legibility
Expand Down
68 changes: 35 additions & 33 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,25 +285,26 @@ The three originally-deferred Majors (M1 coverage, M2 run.py orchestration, M3 l
were substantially addressed in the 2026-08-01 completion pass (see Completed/Closed below).
What remains below are the residual / aspirational follow-ups not fully closed.

### R1 — Push coverage from 89.76% to the documented >90% target
- **Status:** the full arm recorded 776 passed, 9 skipped and 89.76% coverage at `b43415b`
([gate measurements](docs/_generated/gate-measurements.md)); the regression gate in
`pyproject.toml` is still `--cov-fail-under=80`. The per-module figures (~74%
`cli/analyze.py` etc.) predate the coverage uplift and are not reproduced by any current
record, so they are no longer restated here.
- **Suggested fix:** add the remaining edge-case tests (error paths, empty inputs) to climb
above 90%, then raise `--cov-fail-under` to match. Owner: maintainer; no external blocker,
only unmeasured headroom.

### R2 — Consolidate the `APIClient.create` wiring that now lives in the CLI
- **Status:** the menu duplication is gone — every `run.py` operation delegates to
`SymposiumRunner.create_api_client` / `load_config` (`run.py:44-56`). The same
`Config(...)` + `APIClient.create(...)` construction (provider plus
`get_api_config`/`get_api_key`) is now repeated six times:
`src/symposium/cli/analyze.py:40,75,118,153` and `src/symposium/cli/generate.py:19,56`.
- **Suggested fix:** one helper (on `Config`, or a small factory beside `APIClient.create`)
that both the CLI commands and `run.py` call, so provider/key handling cannot drift between
the two entry points. Owner: maintainer.
### R1 — Greater-than-90% coverage target (qualified 2026-11-13)
- **Measured:** the admitted full offline run passed 902 tests, deselected nine
paid-provider tests, and covered 5,013 of 5,547 statements (90.373174689%).
Its initial exit was 1 because the fractional 90.01 floor used zero display
precision. The separate source-bound reporting run reused that coverage database
without rerunning tests and exited 0 with precision 2 and floor 90.01.
- **Guard:** `[tool.coverage.report]` now declares `fail_under = 90.01` and
`precision = 2`. The existing generated 776-test/89.76% record remains historical;
it was not rewritten. Later focused annotation checks are not a fresh full-suite
coverage measurement. This closes the measured target/guard obligation, not
scientific, hosted or release acceptance.

### R2 — Configured-provider construction (qualified 2026-11-13)
- **Completed:** `APIClient.from_config` serves all six CLI construction sites and
the interactive runner; the redundant runner factory is removed.
- The admitted offline suite covers explicit config-file selection, environment
precedence, explicit key/options overrides, conflicting provider options and
forced Perplexity for participant background work. See
[the API contract](docs/api_guide.md). No paid-provider authentication or
availability is claimed.

---

Expand Down Expand Up @@ -397,20 +398,21 @@ All items below were implemented in this pass (commits `669643f`, `98ae63d`, `a6
✓ `82ea73a`

### Open / deferred
- None from this pass. (Pre-existing R1/R2 residuals in the Major section above
remain open; R3 is done.)
- [ ] **Mypy is still far from clean** (re-measured 2026-11-12):
`uv run --no-sync mypy src/symposium` reports **89 errors in 26 files**. Most are
missing third-party stubs (`pandas-stubs`, `types-networkx`, and no `py.typed` for
`sklearn.feature_extraction.text`), which `mypy --install-types` or dev-dependency
additions would settle. The rest are real annotations, e.g.
`src/symposium/core/api.py:108` (a `str | None` returned as `str`),
`src/symposium/core/__init__.py:40` (a `str | Path` passed to a `Path` parameter)
and `src/symposium/generation/profiles.py:157` (unannotated `results`). Add
`mypy src/symposium` to the `lint` arm only after it passes. Owner: maintainer.
- Heavy suites (Julia/LaTeX) do not apply — this is a Python repo; the current full
suite result is the generated gate measurement (776 passed, 9 skipped, 89.76%,
gate 80%), not the 188-test figure from this pass.
- None from this documentation pass. R1/R2 now have separate 2026-11-13 offline
qualification records above; R3 remains done.
- [ ] **Complete package typing remains open.** The 2026-11-12 baseline was 89
errors in 26 files. Subsequent source repairs and a separately authorized Python
3.12 typing environment preserve all 64 existing dependency versions; the first
post-stub run reported 27 errors in seven files across all 51 package modules.
This is a diagnostic result, not a passing gate. `adjustText` and `wordcloud`
remain untyped, and the selected sklearn/SciPy and NetworkX stubs disagree at
sparse-matrix and community-partition boundaries. Do not change scientific
computations, fabricate stubs or suppress imports to obtain a pass.
Add `mypy src/symposium` to `lint` only after complete qualification with a
compatible reproducible dependency set. Owner: maintainer.
- Heavy suites (Julia/LaTeX) do not apply to this Python repo. The generated
776-test/89.76%/80%-gate record and the 188-test figure remain historical, not
current qualification. See the later source-bound R1 observation above.

---

Expand Down
53 changes: 43 additions & 10 deletions docs/api_guide.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Provider API and configuration

The public organizer workflow requires no LLM API. The optional research workflows
use `APIClient.create(provider, api_key=None, **kwargs)` from
[`core.api`](../src/symposium/core/api.py). Supported provider identifiers are
use `APIClient.from_config(config, provider=None, api_key=None, **kwargs)` from
[`core.api`](../src/symposium/core/api.py). `APIClient.create` remains the direct
factory for callers supplying provider settings themselves. Supported identifiers are
`perplexity` and `openrouter`; returned providers expose
`get_response(prompt, system_prompt=None, **kwargs)`. Calling that method can incur
charges and sends its supplied content to the provider.
Expand Down Expand Up @@ -46,17 +47,49 @@ print(config.get("api.provider"))
```

`load_env=False` skips the dotenv file; it does not disable existing environment
overrides. `get_api_config(provider)` and `get_api_key(provider)` supply the factory
inputs. Never dump a complete configuration or environment into public logs.
overrides. Construct a configured provider without duplicating that resolution:

```python
from symposium.core.api import APIClient
from symposium.core.config import Config

client = APIClient.from_config(Config(config_file="config.json"))
```

An explicit `provider` selects the provider; an explicit non-`None` `api_key` and
keyword options override resolved settings. The factory copies provider options,
removes their credential entry, and rejects a conflicting `provider` option rather
than allowing it to override selection. CLI handlers and the interactive runner use
this boundary. Participant background research explicitly selects Perplexity,
including combined background/curriculum workflows.

`get_api_config(provider)` requires an object with string keys.
`get_api_key(provider)` accepts strings or null; malformed values raise a
field-identifying `ValueError` without disclosing their contents. Empty/null keys
retain the existing lookup precedence. Malformed legacy key files retain the
warning-and-`None` behavior. Never dump complete configuration or credentials.

JSON report/data readers require an object at the top level; arrays, null and scalar
roots raise `ValueError`. Participant loaders return the existing
`dict[str, Participant]` schema, including integer `row_index`, without copying the
records. Read-only participant helpers accept `Mapping[str, object]`.

## Errors and limits

Both providers map HTTP 402 to `PaymentRequiredError`. Callers must retain completed
work and stop payment-failed processing. OpenRouter's application retry decorator
retries errors whose text includes `rate` or `timeout`, bounded by five attempts and
300 seconds. Its stored `max_retries` and `retry_delay` settings do not alter those
decorator bounds. Perplexity has no equivalent application backoff wrapper; underlying
SDK behavior is separate. There is no automatic provider failover.
Both providers map HTTP 402 to `PaymentRequiredError`. OpenRouter treats that
exception as terminal before inspecting retry text, including messages containing
`rate` or `timeout`. Other matching errors retain the application retry bound of
five attempts and 300 seconds. Stored `max_retries` and `retry_delay` do not change
those decorator bounds. Perplexity has no equivalent application backoff wrapper;
underlying SDK behavior is separate. There is no automatic provider failover.

Presenter, profile and project batch mappings contain only records whose requested
reports were fully persisted. Nonpayment failures are accumulated while other
eligible records are attempted, then raise `BatchIncompleteError` with `requested`,
`completed`, `failures` and `output_dir` evidence. Payment failure stops immediately.
Completed and partial files remain on disk; partial records are not successes.
Affected CLI requests fail for incomplete or empty work rather than reporting
completion.

Token estimates and truncation are local heuristics, not proof that a provider's
context constraints are met. Returned prose and citations are unverified model
Expand Down
42 changes: 39 additions & 3 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,45 @@ benchmark. Refresh with `uv run --no-sync python scripts/gate_metrics.py refresh
re-renders the prose without measuring.

There is no types arm. `pyproject.toml` configures mypy and the dev extra ships it,
but no gate has ever run it: `uv run --no-sync mypy src/symposium` reports 89 errors
in 26 files, mostly missing third-party stubs (verified 2026-11-12). Adding the arm
requires clearing those first; until then a passing type check is not claimed.
but it has not joined a gate. The 2026-11-12 observation was 89 errors in 26 files.
On 2026-11-13, source repairs and separately provisioned third-party stubs reduced
the first post-stub diagnostic run to 27 errors in seven files, checking all 51
package modules. That run still had missing `adjustText`/`wordcloud` typing and
incompatible sparse/community declarations; it did not establish a passing check.
That repository-specific diagnostic environment preserved all 64 existing
dependency versions and is not a new locked development requirement.
Add mypy to lint only after complete, reproducible qualification.

`python scripts/check_types.py` checks the complete package using the existing
mypy rules. It requires the exact diagnostic dependency versions recorded in
`typings/bindings.json`; it never installs packages or changes runtime versions.
Local `adjustText` and `wordcloud` declarations describe only the consumed APIs.
The word-cloud renderer passes the real `to_array()` result to Matplotlib rather
than pretending that WordCloud implements NumPy's dtype-taking array protocol.

Upstream declaration corrections are generated in a temporary directory and
passed through mypy's `--shadow-file`: CSR array/matrix vectorizer results,
sparse-array consumer inputs and NetworkX mutable/frozen community sets.
Installed package files remain unchanged.
Both package versions and exact upstream declaration hashes must match; a mismatch
requires new API review and qualification, not automatic rebinding. Temporary
projections preserve the installed declarations and license text outside these
narrow corrections. This diagnostic is not a new CI gate or scientific proof.

On 2026-11-15, the complete-package command reported no issues in all 51 source
files while preserving all 64 original runtime distribution versions. Real
synthetic API checks exercised both sparse interfaces through LSA, NMF and LDA,
mutable/frozen community sets, empty/nonempty label adjustment and the actual
word-cloud renderer. Wrong-version and changed-declaration controls refused
before projection. Earlier typing and formatting failures remain historical
evidence; these software checks do not approve research inputs or publication.

The coverage guard is `fail_under = 90.01` with `precision = 2`. The admitted
2026-11-13 offline suite passed 902 tests with nine paid-provider tests deselected
and measured 5,013/5,547 statements (90.37%). Its zero-precision threshold failure
remains recorded; a separate reporting-only check reused the database and passed
with the corrected precision. Do not rerun successful tests just to relabel that
historical failure or hand-edit the generated gate measurements.

## Continuous integration

Expand Down
47 changes: 37 additions & 10 deletions docs/organizing_publication.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,11 @@ November 12, 2026 the repository's Pages source is the `gh-pages` branch at `/`
`main`, plus `.nojekyll` and a `.gitattributes` of `* -text` so the CRLF calendar and
every other file are committed byte for byte.

Actual publication requires separate approval. Bind `PUBLISH_REMOTE` to the reviewed
publication remote and `SOURCE_COMMIT` to the reviewed revision used for the approved
build. The following continuation requires an existing `gh-pages` branch and a fresh
private site directory; it is not a branch-initialization or force-push recipe.

```bash
SITE="$SYMPOSIUM_PRIVATE_DIR/pages-site" # must not exist yet
uv run python - "$SITE" <<'PY'
Expand All @@ -241,18 +246,40 @@ root = Path("data/public/2026")
build_site(verify_public_tree(root), Path(sys.argv[1]), assets_dir=root, artifacts_dir=root)
PY
touch "$SITE/.nojekyll"; printf '* -text\n' > "$SITE/.gitattributes"
git -C "$SITE" init -q -b gh-pages && git -C "$SITE" add -A
git -C "$SITE" commit -q -m "Publish the approved site built from main $(git rev-parse --short HEAD)"
git -C "$SITE" fetch -q https://github.com/ActiveInferenceInstitute/Symposium.git gh-pages
git -C "$SITE" reset --soft FETCH_HEAD && git -C "$SITE" commit -q --amend --no-edit
git -C "$SITE" push https://github.com/ActiveInferenceInstitute/Symposium.git gh-pages
(
set -eu
test ! -e "$SITE/.git"
git -C "$SITE" init -q -b gh-pages
git -C "$SITE" add -A
STAGED_TREE="$(git -C "$SITE" write-tree)"
git -C "$SITE" fetch -q "$PUBLISH_REMOTE" refs/heads/gh-pages
PUBLISHED="$(git -C "$SITE" rev-parse --verify 'FETCH_HEAD^{commit}')"
git -C "$SITE" update-ref refs/heads/gh-pages "$PUBLISHED" ""
test "$(git -C "$SITE" write-tree)" = "$STAGED_TREE"
if test "$STAGED_TREE" = "$(git -C "$SITE" rev-parse "$PUBLISHED^{tree}")"; then
printf '%s\n' 'Approved site unchanged; no commit or push required.'
else
git -C "$SITE" commit -q -m "Publish approved site built from main $SOURCE_COMMIT"
test "$(git -C "$SITE" rev-parse 'HEAD^')" = "$PUBLISHED"
test "$(git -C "$SITE" rev-parse 'HEAD^{tree}')" = "$STAGED_TREE"
git -C "$SITE" merge-base --is-ancestor "$PUBLISHED" HEAD
git -C "$SITE" push "$PUBLISH_REMOTE" HEAD:refs/heads/gh-pages
fi
)
```

The fetch and soft reset make the new snapshot a descendant of the published branch,
so the push is an ordinary fast-forward. After the Pages build reports `built`, compare
every published file with the local output byte for byte. To return to the workflow,
set the Pages source back to GitHub Actions; the `pages.yml` deploy then replaces the
branch-built site.
The fresh branch is anchored at the fetched tip without changing the staged index.
An ordinary new commit therefore has that tip as its parent and the captured staged
tree as its tree. Identical trees require neither a commit nor a push. A concurrently
advanced remote must reject an ordinary stale push; do not force, reset or amend
around it.

A disposable-local-remote proof on 2026-11-13 exercised changed and identical trees,
exact parent/ancestry, committed CRLF calendar bytes, and stale-push rejection. It
did not publish or verify a hosted site. After an authorized Pages build reports
`built`, compare published files with the approved local output byte for byte.
Returning Pages to GitHub Actions remains a separate owner-approved deployment
change.

## Command-line workflow

Expand Down
Loading
Loading