- Python 3.11+ — for AppHub backend
- Node.js 22+ — for Console frontend
- Docker & Docker Compose — for local runtime
- Go 1.24+ — for Portainer init helper (optional)
websoft9/
├── apphub/ # Backend API (Python/FastAPI)
│ ├── src/ # Application source
│ └── tests/ # Test suite
├── console/ # Frontend UI (React/TypeScript/Vite)
│ └── src/ # Application source
├── docker/ # Container build & runtime
│ ├── Dockerfile # Single-container image build
│ └── docker-compose.yml
├── install/ # Installation scripts
├── apphub/src/cli/ # CLI entrypoints (websoft9 command)
├── scripts/ # Utility scripts
└── docs/ # Documentation
The runtime sync consumer supports root protocols 1/2, catalog contract 1,
and library contracts 1/2. These are independent of the platform release
version and the generated browsing manifest, which remains schema 1.
- Root protocol
1reads each component's contract from its component manifest. The local browsing schema is not a remote compatibility requirement. - Root protocol
2validates declared component contracts before any no-change shortcut. Root, component and index identities must agree. Root/component/index checksums are required for new-contract publications, including on cold image builds. - Library contract changes or an unknown local contract require a verified full
package, not index reconciliation or app-level deltas. A publisher's
supportsPartialUpdate: falseis also authoritative. Full contract-2 libraries must contain matchinglibrary.jsonand both shared metadata dictionaries. - Unchanged verified root-2 snapshots reuse their cached component manifests and index. Root-1 publications without component schema declarations still require component requests. Dictionary-only updates and repairs remain independent of application changes.
- Catalog packages may contain the four JSON files directly below the archive's
wrapper directory (the current publisher layout), or under
json/. The consumer normalizes both layouts before validating the catalog fingerprint. - A complete unchanged dataset does not back up or rewrite assets or toggle availability. Full packages, app deltas and changed dictionaries are prepared and verified before activation while a trusted active dataset remains ready. App-artifact fallback also prepares a verified full library before activation. Official installs are still blocked during activation and rollback; an untrusted or missing baseline is never opened while downloads are in progress.
- The successful state caches publication manifests and component contract versions.
A failed attempt preserves active dataset metadata and reports
syncStatus: failedpluslastSyncError; unsupported contracts reportincompatible. Unknown future contracts do not imply that upgrading to the latest release fixes them. - A missing root never triggers legacy ZIP fallback. This change does not update, delete or certify legacy ZIPs or modify historical Websoft9 clients.
Run the focused Python regressions from the repository root:
python3 -m pytest -q --no-cov -p no:cacheprovider \
apphub/tests/test_appstore_manifest_builder.py \
apphub/tests/test_platform_sync_dictionary.py \
apphub/tests/test_install_secrets.py \
apphub/tests/test_install_secret_rules.py \
apphub/tests/test_appstore_sync_manager_versions.py \
apphub/tests/test_appstore_sync_router.py \
apphub/tests/test_platform_readiness.py \
apphub/tests/test_appstore_recovery.py \
apphub/tests/test_local_app_store.py
cd console
npm run typecheck
npm run test:unit -- tests/unit/app-store-sync-feedback.spec.ts tests/unit/app-store-fetch.spec.tsRuntime snapshots now default to $WEBSOFT9_DATA_ROOT/appstore (normally
/opt/websoft9/data/appstore), not the container layer. Image builds separately
save /websoft9/appstore/image-state.json, outside the host data mount.
The successful state records complete Media/Library tree digests, verified
publication manifests, component contracts and the app index.
Before starting services, the entrypoint runs the sync script with
--recover-only, without network requests. If the active trees and all six
persistent snapshot directories already match the verified state, it marks the
container ready without rewriting files. Otherwise it tries the saved release
snapshot, then fully matching active assets, then the image baseline. Each
candidate must validate as a complete pair: both languages' generated manifests
and products, catalog fingerprint, app fingerprints, index, dictionaries and
saved tree digests. Top-level UUID brand uploads written by apphub-media under
/websoft9/media are preserved across sync, reload and recovery and excluded
from App Store snapshot digests. It stages and checks both components before
replacing either live tree. An old state without this verification record is not
trusted; use a complete verified image baseline or a successful online
synchronization instead.
Availability is container-local at /run/websoft9/appstore-assets.json
(WEBSOFT9_APPSTORE_RUNTIME_STATE_FILE overrides it). It is deliberately separate
from the latest network synchronization result:
- A recovered dataset remains usable if a later remote update is offline or incompatible; the previous validated pair is retained.
- Missing, corrupt or incomplete recovery blocks official browsing and installation with HTTP 503. Active updates also block new official installs. Console must not bypass this explicit response by falling back to static files; ordinary API or network failures retain the previous static fallback behavior.
- Startup continues when recovery fails so the platform can be repaired through online synchronization. A successful sync clears the block. Explicit local libraries, custom Compose installs and existing app operations are not gated.
- File rollback handles caught activation failures, including state and destination snapshots. It is not a crash-safe transaction or a multi-directory atomic switch. Recovery validates the entire pair again after every container restart.
Official catalog refresh shares the online sync lock and performs no downloads. It requires a ready, validated active dataset, builds browsing manifests in a temporary tree, and only enters the activation gate if the generated content changes. Changed projections update Media snapshots and saved digests together; activation failures restore files and state. A busy sync, unavailable assets or modified official source payloads produce an explicit official-refresh error, without preventing the independent custom catalog refresh. Modified official payloads must be repaired by synchronization, not blessed by refresh; use the custom catalog for locally maintained applications. Runtime sync refuses to write without its lock, rather than silently running concurrently.
The recovery tests simulate container Library resets with persistent Media, publisher outages and unsupported contracts, image fallback, damaged snapshots, failed activation, missing runtime markers and subsequent online repair. They do not replace an actual upgrade/recreation test of the frozen release image with its production data mount.
This consumer patch is not release approval. Credentials/setup capability coverage,
real-image upgrade/recreation acceptance, release validation helpers and actual
published-channel artifacts still need separate release qualification. The existing
validate_appstore_* shell helpers exercise historical publication layouts; do not
use their equal-dataset or legacy-path checks as proof of contract-2 compatibility.
No live CDN or object-store state can be certified from a local publisher checkout.
The existing rollback restores files after caught failures; it is not an atomic
multi-directory switch or a crash-safe transaction.
The repository Makefile wraps the commands below — it only orchestrates them, so nothing is
duplicated between it, package.json, pytest.ini and the CI workflows. Run make to list the
targets.
make # list targets
make doctor # check node / python / docker versions before anything else
make setup # npm ci + apphub/.venv with dev and runtime dependencies
make dev-up # build and start the local container (console on :9000)
make lint # console ESLint + apphub ruff (the checks CI runs)
make typecheck # tsc -b + mypy
make test # apphub contract suites + console unit tests
make build # console production build
make ci # everything a pull request has to pass locally| Target | Runs |
|---|---|
make setup-console |
cd console && npm ci |
make setup-apphub |
creates apphub/.venv, then installs ruff/mypy/pytest plus requirements.txt and requirements-dev.txt |
make dev-up / dev-down / dev-logs |
docker compose -f docker/docker-compose.dev.yml up -d --build / down / logs -f |
make lint-console |
npm run lint |
make lint-apphub |
ruff check src/ (merge gate; config in apphub/pyproject.toml) |
make format-check-apphub |
ruff format --check src/ tests/ (blocking in CI) |
make format-apphub |
ruff format src/ — one-shot whitespace-only baseline |
make i18n |
npm run i18n:check (stricter than CI) |
make typecheck-console |
npm run typecheck |
make typecheck-apphub |
mypy src/ (blocking; the tree is clean) |
make test-apphub |
the whole AppHub suite in one pytest process, with coverage reports and the branch-coverage floor (COVERAGE_MIN, default 69) |
make test-contracts |
a fast subset of the AppHub contract tests (previously the only CI test step) |
make test-fast |
the whole suite without coverage, for a quick local loop |
make test-unit |
npm run test:unit |
make test-e2e |
npm run test:e2e against a running platform (W9_BASE_URL, default http://127.0.0.1:9000) |
make build-console |
npm run build |
make image |
docker build -f docker/Dockerfile -t $IMAGE_REPO:$IMAGE_TAG (single architecture) |
make version-check |
mirrors the CI version-consistency job against version.json, apphub/setup.py and console/package.json |
make ci |
lint typecheck test version-check build |
make clean |
removes console/dist, htmlcov and coverage.xml |
PYTHON and NPM are overridable from the command line:
make test-apphub PYTHON=apphub/.venv/bin/python.
cd apphub
# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Run tests
pytest --cov=src
# Run development server
uvicorn src.main:app --reload --port 8000cd console
# Install dependencies
npm ci
# Start development server
npm run devcd docker
# Start with dev compose file
docker compose -f docker-compose.dev.yml up -d --buildThe SFTP action beside an application's volume browser opens the Terminal file
view using an independent, restricted file session. It does not start a shell or
overwrite the ordinary terminal workspace. A usable local connection opens files
directly. At the application SFTP entry, missing connections show a confirmation
only, not an embedded credential form. Its manual action opens /terminal?new_connection=1, which
opens Terminal's existing new-connection form. The automatic action first shows
a second confirmation with three short bulleted statements covering connection
reuse, account permissions, and server-side key storage. The confirmation states
that no operating-system account is created: installation prepares an SSH
public key for the selected existing host account. Its private key is stored on
the server; deleting an automatic local connection revokes its host public key
and removes the prepared private key and metadata files.
The entry briefly explains
why host SSH/SFTP access is needed. Both steps retain the original white dialog
style, with a responsive 680px width and smaller explanatory text, without extra
tables or notice panels. The second step requires an acknowledgement checkbox;
returning to the first step resets it. Only explicit confirmation verifies
and activates or creates an ordinary local SSH connection before opening the
restricted file view. Authorization, authentication, network, and directory
permission failures have distinct messages. Directory permission errors do not
offer another connection as a remedy.
Invalid file sessions remain in the original file panel with its standard directory
error display; they do not redirect
through the application page or automatically open a connection confirmation.
Removing a saved host closes its terminal channels and invalidates its application
file sessions for the current operator. The corresponding tabs remain selectable
but show an ended state without recreating sessions or changing position. File
panels immediately stop loading, display their connection error, and disable file
operations, including creation and upload. Browser reload preserves ended terminal
and file tabs and terminal output; deleted profiles are never automatically
reconnected. Browser transport disconnects do not terminate the server PTY:
refresh reattaches the same terminal ID and profile, synchronizes its server
snapshot, and resumes output without duplicating history. Temporary transport
loss can be restored on refresh; an explicitly exited or deleted session is not
automatically restarted. Detached PTYs expire after the configured idle timeout;
API/process restarts do not preserve in-memory PTYs. Disconnected terminals clear
the current shell prompt, hide the input cursor, and disable input. The original
bottom notice and right-aligned reconnect button remain; the notice is not
duplicated inside the terminal. Reconnect validates unavailable profile records:
an existing profile reattaches the same session ID, while a deleted profile shows
an explanation rather than opening a new connection form. Deleting
a local connection records an operator-scoped revocation in SQLite: installer
credentials cannot implicitly recreate it. Explicitly confirming the automatic
connection action may prepare a new authorization when credentials are absent and
create a new profile with a new ID; successful verification clears the revocation
record. Missing authorization or authentication failures leave the revocation
record intact. Manually saving a new local connection also restores access.
Deleting an automatic local profile removes the matching public
key even if other connections use it; subsequent authentication with that key
fails. Unrelated public-key entries are preserved. Cleanup failures are reported
instead of claiming that the connection was deleted. Manual connections do not
remove user-supplied host public keys.
The platform never enables SSH, changes SSH login policy, or changes application
volume ownership to make a connection succeed.
The production and development Compose definitions do not enable privileged
mode for the product container. Automatic SSH uses
ordinary TCP authentication and platform credentials in the existing data mount,
not host namespaces or extra Linux capabilities. It also works with
privileged: false, cap_drop: [ALL], and no-new-privileges, provided the
container's service user can read the owner-only credentials and the host's SSH
port is reachable. Preparing missing authorization requires the existing Docker
management socket and a short-lived privileged helper with host PID access.
The helper reuses the installed product image and enters the host namespaces to
run the fixed Python preparation program shipped with AppHub; no image is downloaded.
The program is passed directly to the helper command, so the host does not need
an updated installer or any newly installed preparation script. Cloud marketplace
images and their boot-time installation scripts remain unchanged; updating the
product Docker image delivers this behavior through the existing AppHub copy.
The main product container need not be privileged, but the Docker daemon must
permit this explicitly confirmed preparation operation. The helper is removed
on completion or failure. This verifies automatic SSH only, not all product
services. No host SSH directory mount or permanent helper is required. Changing
the running container's privilege setting requires recreation and ends existing
terminal/file sessions. The Docker management socket still permits host-level
operations; removing product-container privileged mode does not sandbox platform
operators. The temporary preparation helper remains privileged.
Authenticated POST /api/host-access/profile/local verifies and activates a local
profile belonging to the current operator. This explicit action permits recreation
after deletion, but never bypasses actual SSH authentication. If platform
credentials are absent, it runs the host preparation step through the existing
Docker management channel, then validates and saves the new SSH connection.
Deletion retains only the previous account and port in an owner-only
local-ssh-account.json record so re-preparation uses the same existing account;
no deleted key is retained. It first tries the runtime connection,
including credentials that were not remembered, then the saved local profiles.
Remote profiles are not attempted, stored credentials are not overwritten, and
temporary credentials remain temporary. Authentication or connection failures
retain their original error details and direct the user to Terminal's existing
connection form. When no existing connection works, the backend reads the
previously prepared local-ssh.json and local-ssh.key in the existing
data/config/host-access directory, verifies SSH, and saves a normal connection
for the current operator. Files must be regular, single-link, owner-only files
in an owner-only directory; the key is never returned to the browser. An existing
manual connection for the same account is not silently overwritten.
Installation and upgrades do not prepare SSH authorization. When the user explicitly
confirms automatic connection and usable credentials are absent, AppHub runs its
own preparation program through the temporary helper. It selects a previously
recorded account, the non-root SUDO_USER, or a single detected non-root SSH
account, in that order. Only when no non-root account is detected does it consider
root, and the existing SSH policy must allow root key login. Ambiguous accounts
require a manual connection instead of guessing an account.
Preparation appends a dedicated public key to the account's standard
.ssh/authorized_keys, preserves existing entries and key restrictions, verifies
login, and writes owner-only credentials through the existing data mount. It is
idempotent and removes a newly appended authorization if verification or
metadata storage fails. SSH is not installed, enabled, or reconfigured.
Preparation failure is reported by the connection action and does not affect
product installation. Existing credentials remain usable across product updates.
The host
needs Python 3, ssh, ssh-keygen, and an already running SSH service. Custom
authorization paths require a manual connection. If authorization is absent
or unsafe, automatic setup must succeed before a connection is saved; otherwise
the action reports an error and the user can connect manually. Privileged mode
alone does not authorize SSH. Platform operators must be trusted:
the prepared key grants the selected host account's access. Deleting an automatic
local connection authenticates to the host, locks and validates its authorization
file, removes only the matching public-key entry, and deletes both prepared
credential files. Metadata records the authorization path; older metadata resolves
the account's standard authorization path. Re-preparation is required to restore
installer authorization after its key has been deleted; the explicitly confirmed
automatic action now performs that re-preparation through Docker.
AWS and Azure images may use a non-root account, such as ubuntu, ec2-user,
or a user selected during provisioning. These accounts follow the same ordinary
connection workflow; no root account is assumed and no cloud login policy is
changed. For non-root accounts, host preparation configures passwordless sudo
solely for the host's sftp-server binary, and application SFTP opens an
elevated sftp-server channel strictly confined to the target Docker volume
while keeping ordinary terminal and host file sessions under the non-root
account's permissions.
Ordinary terminal and application file sessions share a session tab strip while
retaining independent state and permissions. Switching tabs keeps terminal
connections and rendering alive. Reopening the same application volume validates
and reuses its file session; expired sessions are rebuilt. Each volume remembers
its own directory, and closing a file tab only releases that file session.
File tabs use the normal connected status without an SFTP title suffix or a
special close confirmation. Opening the normal host list keeps the files open.
File tab titles retain the complete Docker volume name, including its application prefix. Long
titles truncate when needed; hovering shows the complete application and volume.
The selected tab scrolls into view. The strip retains its
original layout, hidden scrollbars, and inline new-connection button; it does
not show a connection counter or a separate right-hand button area. The browser
+ button opens the same saved-connection menu in ordinary and application file
workspaces, refreshing the host list on open. Opening or cancelling the menu or
its new-connection form does not navigate away or deselect the current file tab.
Successful connection from this menu opens a separate ordinary terminal session,
even when an application file tab is selected. It does not reopen, replace, or
close that application's restricted file session, and existing terminal sessions
are retained. New terminal and application file tabs append to the end, without
grouping terminals ahead of file sessions. Tab order and the current selection are
stored in session storage; tabs keep their natural width (220px maximum for inactive
tabs) and shrink only when space is insufficient. The add button remains visible at
the ten-tab workspace limit. File tabs use application and volume identifiers so
renewing a server session does not change their logical identity. Closing the selected tab automatically selects a remaining neighbor
across both workspace types; only closing the last tab shows the empty workspace.
Hidden terminals do not fit or send resize messages; visible terminals fit before
buffer playback to avoid narrow-column wrapping. Disconnected terminals keep their
historical output, show a persistent reconnect notice below the terminal canvas,
and disable input and cursor blinking. New sessions use the verified response's host profile ID rather
than an unsaved form ID. Login requests time out after 30 seconds, clear the busy
state, and show a localized error without treating a timeout as a force-save request.
The browser
workspace permits ten terminal and application file tabs in total, including
in-flight file session creation; existing file sessions can still be reused or
renewed at the limit. This is a console workspace limit, not a server-wide SSH
quota.
SFTP v3 does not expose a reliable creation time, so missing creation times are
displayed as a dash rather than substituted with modification times. Identity
labels reuse Terminal's name-and-ID format, such as root(0), and fall back to
plain numeric IDs when names cannot be resolved. Internal metadata lookup only
reads the remote host's fixed account tables and uses an isolated name cache;
file APIs still reject reads or writes outside the application volume. Host
account names need not match the names inside a container.
- Only the local Docker host and local volumes without driver options are supported.
- The backend verifies the Compose application label and obtains the volume root from Docker; frontend paths are not trusted.
- File sessions belong to the authenticated operator and expire after 30 minutes or a backend restart. Connection identity and volume ownership are rechecked.
- All file operations validate their source and destination paths. Symbolic links and special files are excluded; the volume root cannot be renamed or deleted.
- Copy and permission changes use SFTP rather than arbitrary remote shell commands. Owner/group changes are not available in this restricted view.
- SSH account permissions still apply. This API boundary is not an OS-level chroot: existing host administrators retain their rights. For hostile processes that can race path checks or create hard links, use a dedicated jailed SFTP account rather than treating a privileged shared connection as sandboxed.
Run pytest -q --no-cov tests/test_host_access.py from apphub. The real SFTP
integration test uses a temporary loopback-only SSH daemon and temporary keys;
it is skipped without root, sshd, or ssh-keygen.
Run node --test scripts/app-file-sessions.test.mjs from console for the
session reuse, expiry, concurrent entry, and isolated close regression checks.
- Formatter: ruff format (line length 127)
- Linter: ruff check (
E4,E7,E9,F,I,Bsubset,PIE,SIMsubset); flake8/pylint/black/isort were retired. FastAPI'sDepends()/Body()defaults are allowed throughflake8-bugbear.extend-immutable-calls - Import sorting: ruff (
Irules) - Type hints required for public APIs;
mypy src/is a blocking gate with no error codes muted (the pydantic models use v2@field_validator/json_schema_extra, andpydantic.mypyis enabled) - Test coverage:
--cov-branchoverapphub/src, currently ~70.8% (line coverage ~75%); the blocking floor isCOVERAGE_MIN(69) in theMakefile, also passed as--cov-fail-underby the PR workflow. Note thatpytest-covignores[coverage:report] fail_under, which is why the floor is an explicit flag
- Formatter: ESLint + Prettier
- Strict TypeScript mode enabled
- Component structure: feature-based folders under
src/features/
All pull requests must pass:
| Check | Tool |
|---|---|
| Linting | ruff check + ruff format --check, ESLint, ShellCheck (--severity=warning) |
| Tests | Pytest (single process, branch coverage floor COVERAGE_MIN = 69) + Console unit + install/tests/*.sh |
| E2E | Playwright against a real platform, required when console/** changes |
| Build | Docker image build |
| Security | Container vulnerability scan |
See CI/CD Guide for details.
- Fork the
mainbranch - Create a feature branch
- Make changes with tests
- Ensure all CI checks pass locally
- Submit a pull request to
main
See CONTRIBUTING.md for the full contribution workflow.