docs: add docs index, challenges.md and page-sorts.md; fix shell-unsafe command examples

AGENTS.md

@@ -67,6 +67,19 @@ src/
 └── index.ts             # package entry point
 ```
 
+## Docs Lookup
+
+Topic docs live flat under `docs/`; [docs/README.md](docs/README.md) is the index. Before touching an area, read its doc:
+
+| Touching                                                                 | Read                                       |
+| ------------------------------------------------------------------------ | ------------------------------------------ |
+| `src/cli/commands/challenge/`, `src/challenge-packages/`                 | [docs/challenges.md](docs/challenges.md)   |
+| `src/cli/commands/page-sort/`, `src/page-sort-packages/`                 | [docs/page-sorts.md](docs/page-sorts.md)   |
+| `src/common-utils/package-registry.ts`, package reload endpoints in `src/webui/daemon-server.ts` | both of the above |
+| `settings.challenges` / `settings.pages` examples in `src/cli/commands/community/edit.ts` | the matching doc above |
+
+When a change alters behavior a doc describes (commands, flags, paths, endpoints, output), update the doc in the same PR.
+
 ## GitHub Issues
 
 When the user gives a prompt to implement a feature or bug fix in this repository's codebase:
@@ -99,6 +112,10 @@ Routine live operations are not repository feature work:
 
 - **MUST** add a test when you add a feature or fix a bug.
 
+### Docs
+
+- **MUST** add an entry to `docs/README.md` when adding a topic doc (any markdown file under `docs/` other than `docs/README.md` itself and the generated `docs/CHANGELOG.md`), and a row to the Docs Lookup table above if the doc maps to source paths.
+
 ## Core SHOULD Rules
 
 - **SHOULD** keep changes focused — avoid unrelated refactors in the same commit.

README.md

@@ -698,12 +698,12 @@ EXAMPLES
 
     $ bitsocial community edit mycommunity.bso '--roles["esteban.bso"].role' admin
 
-  Add two challenges to the community. The first challenge will be a question and answer, and the second will be an
-  image captcha
+  Add two challenges to the community. The first challenge will be a question and answer, and the second will be a
+  math question
 
-    $ bitsocial community edit mycommunity.bso --settings.challenges[0].name question \
-      --settings.challenges[0].options.question "what is the password?" --settings.challenges[0].options.answer \
-      thepassword --settings.challenges[1].name captcha-canvas-v3
+    $ bitsocial community edit mycommunity.bso '--settings.challenges[0].name' question \
+      '--settings.challenges[0].options.question' "what is the password?" '--settings.challenges[0].options.answer' \
+      thepassword '--settings.challenges[1].name' text-math
 
   Use an installed page sort package (bitsocial page-sort install @pkcprotocol/active-page-sort) as the preloaded
   posts sort, where replies containing 'sage' do not bump a thread
@@ -718,7 +718,7 @@ EXAMPLES
 
   Remove a role from a moderator/admin/owner
 
-    $ bitsocial community edit bitsocial.bso --roles['rinse12.bso'] null
+    $ bitsocial community edit bitsocial.bso '--roles["rinse12.bso"]' null
 
   Enable settings.fetchThumbnailUrls to fetch the thumbnail of url submitted by authors
 
@@ -931,7 +931,7 @@ EXAMPLES
 
   $ bitsocial daemon --pkcOptions.dataPath /tmp/bitsocial-datapath/
 
-  $ bitsocial daemon --pkcOptions.kuboRpcClientsOptions[0] https://remoteipfsnode.com
+  $ bitsocial daemon '--pkcOptions.kuboRpcClientsOptions[0]' https://remoteipfsnode.com
 
   $ bitsocial daemon --chainProviderUrls https://mainnet.infura.io/v3/YOUR_KEY
 

docs/README.md

@@ -0,0 +1,11 @@
+# bitsocial-cli docs
+
+Topic docs for agents and operators. The per-command reference (every flag, alias and example) is generated by oclif into the [README's Commands section](../README.md#commands); these docs cover what `--help` does not: where things live, how the daemon picks changes up, and the gotchas.
+
+When you add a topic doc under `docs/`, add a row here.
+
+| Doc | Covers |
+| --- | --- |
+| [challenges.md](challenges.md) | Challenge packages: `bitsocial challenge install/list/remove`, where they are installed, how the daemon loads and hot-reloads them, shadowing a pkc-js built-in, configuring them with `community edit` |
+| [page-sorts.md](page-sorts.md) | Page sort packages: `bitsocial page-sort install/list/remove`, `sortName` vs package name and the upgrade warning, the in-use refusal on remove, configuring `settings.pages` with `community edit` |
+| [CHANGELOG.md](CHANGELOG.md) | Release notes (generated on release; do not edit by hand) |

docs/challenges.md

@@ -0,0 +1,107 @@
+# Challenge packages
+
+A challenge package is an npm package whose default export is a pkc-js challenge factory. Installing one registers it into `PKC.challenges` under its **package name**, so a community can name it in `settings.challenges[i].name`, the same way it names a pkc-js built-in such as `question`.
+
+Page sort packages work the same way; see [page-sorts.md](page-sorts.md). Both kinds share `src/common-utils/package-registry.ts`.
+
+## Commands
+
+```bash
+bitsocial challenge install <package>   # aliases: i, add
+bitsocial challenge list                # alias: ls; -q prints names only
+bitsocial challenge remove <name>       # aliases: uninstall, rm, un
+```
+
+`install` accepts anything `npm pack` accepts:
+
+```bash
+bitsocial challenge install @bitsocial/mintpass-challenge
+bitsocial challenge install @bitsocial/mintpass-challenge@1.0.0
+bitsocial challenge install github:user/repo
+bitsocial challenge install https://example.com/my-challenge-1.0.0.tar.gz
+bitsocial challenge install ./my-local-challenge
+```
+
+All three commands take `--pkcOptions.dataPath` (default: the platform data dir from `env-paths("bitsocial")`, e.g. `~/.local/share/bitsocial` on Linux). `install` and `remove` also take `--pkcRpcUrl` (default `ws://localhost:9138`) to find the daemon to reload.
+
+`remove` takes the package name, not the spec you installed from: `bitsocial challenge remove @bitsocial/mintpass-challenge`.
+
+## Where packages live
+
+```
+<dataPath>/challenges/<name>/            # unscoped
+<dataPath>/challenges/@scope/<name>/     # scoped
+```
+
+Each directory is a fully installed package (its own `node_modules`). `list` reads this directory and prints an npm-ls-style tree:
+
+```
+/home/me/.local/share/bitsocial/challenges
+└── @bitsocial/mintpass-challenge@1.0.0
+```
+
+## What install does
+
+1. `npm pack <spec>` into a temp dir under the data path, then extract it.
+2. Read `package.json`. The `name` must be a valid npm name (`pkg` or `@scope/pkg`), because it becomes the install directory.
+3. `npm install` inside the temp dir, so a failed install never touches a working one.
+4. Check that native modules match this Node's ABI.
+5. Import the entry point and run pkc-js `validateChallengeFileFactory` on it. A page sort package, or anything else that is not a challenge factory, is refused here.
+6. Swap the package into `<dataPath>/challenges/<name>`. An existing install is moved aside first and restored if the swap fails. Output is `added <name>@<version>` or `changed <name>@<version>` (replaced an existing install).
+7. Ask a running daemon to reload (below).
+
+`npm` must be on `PATH`.
+
+## How the daemon loads them
+
+- **At start:** `bitsocial daemon` loads every package under `<dataPath>/challenges/` into `PKC.challenges` before starting the RPC server, and logs `Loaded challenge packages: name@version, ...`. A package that fails to import is logged and skipped. It does not stop the daemon.
+- **Hot reload:** after `install` or `remove`, the CLI POSTs to the daemon's reload endpoint, which re-reads the directory and notifies settings subscribers. If it worked, `install` prints `reloaded <name>@<version> in the daemon at <url>`. Running communities pick up the change on their next cycle, and nothing is restarted.
+  - `POST http://<host>:<port>/api/challenges/reload` is local only (403 for remote connections).
+  - `POST http://<host>:<port>/<authKey>/api/challenges/reload` works remotely. The CLI uses this one when `--pkcRpcUrl` includes the auth key path (`ws://host:9138/<authKey>`).
+  - The reload URL is derived from `--pkcRpcUrl`, and a wildcard host (`0.0.0.0`, `::`) is dialed as `127.0.0.1`.
+- **No daemon running:** the reload is best-effort. Install and remove still succeed, and the daemon picks the change up at its next start. The CLI waits up to 30s for a daemon that accepts the connection but does not answer.
+
+Upgrading in place (`install` of a newer version) takes effect on reload. The entry module is imported under a URL keyed on the package contents, so the old module is not reused. Only the entry module is re-evaluated, so packages should ship a bundled entry point.
+
+If a reinstalled package fails to import on reload, the daemon keeps the previously loaded factory under that name instead of dropping it.
+
+## Shadowing a built-in
+
+pkc-js ships built-in challenges: `text-math`, `fail`, `blacklist`, `whitelist`, `question`, `publication-match`. A package with the same name as a built-in replaces it in `PKC.challenges` for this daemon. `remove` puts the built-in back on the next reload instead of deleting the key. A package name that was never a built-in is simply deleted from the registry.
+
+## Using a challenge in a community
+
+Name the package in `settings.challenges`:
+
+```bash
+bitsocial community edit mycommunity.bso \
+  '--settings.challenges[0].name' @bitsocial/mintpass-challenge
+```
+
+Options go under `settings.challenges[i].options`:
+
+```bash
+bitsocial community edit mycommunity.bso \
+  '--settings.challenges[0].name' question \
+  '--settings.challenges[0].options.question' "what is the password?" \
+  '--settings.challenges[0].options.answer' thepassword
+```
+
+Quote the flags in the shell: the `[0]` brackets are glob characters, and zsh refuses to run the command when they match no file.
+
+For larger changes, write the settings to a file and pass `--jsonFile <file>`; CLI flags take priority over the file.
+
+## Removing
+
+`challenge remove` deletes the directory and reloads the daemon. It does **not** check whether a community still names the package in `settings.challenges`. Check first with `bitsocial community get <address>`. (`page-sort remove` does refuse in that case; see [page-sorts.md](page-sorts.md#removing).)
+
+## Troubleshooting
+
+- **Install fails with "No valid package.json found in the archive":** the spec did not produce an npm package tarball.
+- **Install fails validation:** the default export is not a challenge factory. A page sort package belongs under `bitsocial page-sort install`.
+- **Daemon log `Failed to load challenge "<name>"`:** the package is installed but its entry point throws on import. It is left out of the registry until fixed. A previously loaded version keeps serving.
+- **`install` printed no `reloaded ...` line:** no daemon answered at `--pkcRpcUrl`. Pass the URL the daemon actually listens on, including the auth key path for a remote daemon, or restart the daemon.
+
+## Writing a challenge package
+
+The factory contract (arguments, `getChallenge`, result shape) is defined by pkc-js; see [challenge-flow.md in pkc-js](https://github.com/pkcprotocol/pkc-js/blob/master/docs/protocol/challenge-flow.md). The package's `main` (default `index.js`) must default-export the factory. Test it locally with `bitsocial challenge install ./path/to/package`.

docs/page-sorts.md

@@ -0,0 +1,125 @@
+# Page sort packages
+
+A page sort package is an npm package whose default export is a pkc-js page sort factory. Installing one registers it into `PKC.pageSorts` under its **package name**, so a community can name it in `settings.pages.posts[i].name` or `settings.pages.replies[i].name`, the same way it names a pkc-js built-in sort such as `hot`.
+
+Install, storage, loading and hot reload work exactly as for challenge packages ([challenges.md](challenges.md)); this doc repeats the essentials and covers what is specific to page sorts: `sortName` and the in-use check on remove.
+
+## Commands
+
+```bash
+bitsocial page-sort install <package>   # aliases: i, add
+bitsocial page-sort list                # alias: ls; -q prints names only
+bitsocial page-sort remove <name>       # aliases: uninstall, rm, un; --force skips the in-use check
+```
+
+```bash
+bitsocial page-sort install @pkcprotocol/active-page-sort
+bitsocial page-sort install @pkcprotocol/active-page-sort@1.0.0
+bitsocial page-sort install ./my-local-page-sort
+```
+
+`install` accepts anything `npm pack` accepts (name, name@version, git URL, tarball URL, local path). All commands take `--pkcOptions.dataPath`; `install` and `remove` take `--pkcRpcUrl` (default `ws://localhost:9138`).
+
+## Where packages live
+
+```
+<dataPath>/page-sorts/<name>/
+<dataPath>/page-sorts/@scope/<name>/
+```
+
+## Package name vs `sortName`
+
+Two names matter:
+
+- **Package name** (`@pkcprotocol/active-page-sort`): the key in `PKC.pageSorts`, and what `settings.pages.posts[i].name` refers to.
+- **`sortName`** (e.g. `active`): declared by the package itself. It is the key the community publishes pages under in `pageCids`, so it is what clients and frontends ask for.
+
+They need not match. `list` shows both:
+
+```
+/home/me/.local/share/bitsocial/page-sorts
+└── @pkcprotocol/active-page-sort@1.0.0 (sortName: active)
+```
+
+A package that fails to import shows `(failed to load: <error>)` in place of its `sortName`. The rest of the listing still prints.
+
+### Upgrades that change `sortName`
+
+`install` reads the `sortName` of the version being replaced before swapping in the new one. If it differs, the install still goes through, and it warns:
+
+```
+Warning: @scope/my-sort changed its sortName from "active" to "activity". Communities using it now publish
+these pages under "activity", and clients asking for "active" will not find them.
+```
+
+pkc-js accepts the new name, but every community using the package switches the `pageCids` key on its next update, so frontends hard-coded to the old key stop finding the pages. Pin the previous version (`install <name>@<old version>`) if that is not intended. A previous install that no longer loads is not compared and does not block the upgrade.
+
+## What install validates
+
+The new package is imported and checked with pkc-js `validatePageSortFileFactory` **before** it replaces anything. A challenge package, or anything else that is not a page sort factory, is refused and leaves the existing install untouched. Steps otherwise match [challenges.md › What install does](challenges.md#what-install-does).
+
+## How the daemon loads them
+
+Same mechanism as challenges ([details](challenges.md#how-the-daemon-loads-them)):
+
+- At start, `bitsocial daemon` logs `Loaded page sort packages: name@version, ...`.
+- After `install`/`remove`, the CLI POSTs `/api/page-sorts/reload` (local) or `/<authKey>/api/page-sorts/reload` (remote). It is best-effort: with no daemon running the change applies at the next start.
+- Running communities pick up the new registry on their next cycle; nothing is restarted.
+
+### Shadowing a built-in
+
+pkc-js built-in sorts: `hot`, `new`, `old`, `best`, `top`, `topHour`, `topDay`, `topWeek`, `topMonth`, `topYear`, `topAll`, `active`, `controversial`, `newFlat`, `oldFlat`. A package with the same name as a built-in replaces it for this daemon. `remove` puts the built-in back on the next reload.
+
+## Using a page sort in a community
+
+`settings.pages` lists which sorts to generate per scope (`posts`, `replies`). Leaving it unset generates the pkc-js default sorts. Any change regenerates every CommentUpdate.
+
+```bash
+bitsocial community edit mycommunity.bso \
+  '--settings.pages.posts[0].name' @pkcprotocol/active-page-sort \
+  '--settings.pages.posts[0].options.noBumpKeywords' sage \
+  '--settings.pages.posts[0].preloaded' true
+```
+
+Fields of each entry:
+
+| Field | Meaning |
+| --- | --- |
+| `name` | Key in `PKC.pageSorts`: a built-in or an installed package name |
+| `path` | Alternative to `name`: path to a page sort JS file |
+| `options` | Passed to the sort. `maxAge` (`"7d"`, `"2w"`), `pinnedFirst` and the `exclude*` flags are reserved options every sort accepts |
+| `preloaded` | Embed the first page in the record. The first preloaded entry is the client's default sort |
+| `privateOptions` | Option names withheld from the public `community.pageSorts`. Reserved options cannot be private |
+
+Quote the flags in the shell: the `[0]` brackets are glob characters, and zsh refuses to run the command when they match no file.
+
+## Removing
+
+```bash
+bitsocial page-sort remove @pkcprotocol/active-page-sort
+```
+
+Before deleting, `remove` connects to the daemon at `--pkcRpcUrl` and checks every community's `settings.pages`. If any entry names the package, it refuses and lists them (oclif prefixes each output line with ` ›   `):
+
+```
+Error: Page sort "@pkcprotocol/active-page-sort" is still used by:
+  - mycommunity.bso settings.pages.posts[0]
+Change or remove those entries first, e.g. bitsocial community edit <address> --jsonFile <file with the new settings.pages>,
+or pass --force to remove it anyway (the communities keep publishing without this sort).
+```
+
+The reason: a community whose `settings.pages` names a missing sort keeps publishing but skips that entry and logs an error on every start, so the board silently loses the sort.
+
+- **The daemon must be running** for the check. If it cannot be reached, `remove` refuses and says so.
+- **`--force`** skips the check, both for in-use packages and for an unreachable daemon.
+
+## Troubleshooting
+
+- **`list` shows `failed to load`:** the package's entry point throws on import, or its export fails `validatePageSortFileFactory`. Reinstall a working version.
+- **Daemon log `Failed to load page sort "<name>"`:** same cause, seen at start or reload. A previously loaded version keeps serving.
+- **Pages missing under the expected key:** compare the `sortName` in `page-sort list` with what the client requests. See [Upgrades that change sortName](#upgrades-that-change-sortname).
+- **`remove` says it cannot check usage:** start the daemon (or point `--pkcRpcUrl` at it), or pass `--force`.
+
+## Writing a page sort package
+
+The factory contract (`sortName`, `score`, options, the client-side re-sort with `publicOptions`) is defined by pkc-js; see [page-sorts.md in pkc-js](https://github.com/pkcprotocol/pkc-js/blob/master/docs/protocol/page-sorts.md). The package's `main` (default `index.js`) must default-export the factory. Test it locally with `bitsocial page-sort install ./path/to/package` and check the `sortName` in `bitsocial page-sort list`.

src/cli/commands/community/edit.ts

@@ -47,8 +47,8 @@ For modifying complex settings like challenges, consider using a web UI instead:
         },
         {
             description:
-                "Add two challenges to the community. The first challenge will be a question and answer, and the second will be an image captcha",
-            command: `bitsocial community edit mycommunity.bso --settings.challenges[0].name question --settings.challenges[0].options.question "what is the password?" --settings.challenges[0].options.answer thepassword --settings.challenges[1].name captcha-canvas-v3`
+                "Add two challenges to the community. The first challenge will be a question and answer, and the second will be a math question",
+            command: `bitsocial community edit mycommunity.bso '--settings.challenges[0].name' question '--settings.challenges[0].options.question' "what is the password?" '--settings.challenges[0].options.answer' thepassword '--settings.challenges[1].name' text-math`
         },
         {
             description:
@@ -61,7 +61,7 @@ For modifying complex settings like challenges, consider using a web UI instead:
         },
         {
             description: "Remove a role from a moderator/admin/owner",
-            command: "bitsocial community edit bitsocial.bso --roles['rinse12.bso'] null"
+            command: `bitsocial community edit bitsocial.bso '--roles["rinse12.bso"]' null`
         },
         {
             description: "Enable settings.fetchThumbnailUrls to fetch the thumbnail of url submitted by authors",