Skip to content

feat(dawarich): run managed Dawarich 1.19.0 on Redis 8 - #681

Merged
Medformatik merged 2 commits into
mainfrom
chore/dawarich-1.19
Oct 30, 2026
Merged

Medformatik merged 2 commits into
mainfrom
chore/dawarich-1.19

Conversation

@Medformatik

@Medformatik Medformatik commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

What changed and why

The managed Dawarich bundle moves from Dawarich 1.15.3 to 1.19.0, and its dedicated Redis from 7.4 to 8.10.

Dawarich 1.16.0 fixed two security issues:

  • a stale two-factor challenge that could switch accounts after sign-in;
  • shared-photo authorization that survived edits to a trip's shared window.

Later releases add geofencing and webhooks, archived places, and performance and data-quality fixes. App and worker images move together (freikin/dawarich:1.19.0, digest sha256:d9385242…, upstream commit 752641768f).

The worker stays separate. Dawarich 1.17 can run background jobs inside the web process, but only when SIDEKIQ_EMBEDDED=true. The managed app doesn't set it, so jobs never run twice, and the stock sidekiq-entrypoint.sh still ships in 1.19.0. Keeping dawarich-sidekiq avoids removing a service from existing deployments in this PR.

Redis 8.10 loads an RDB written by the currently pinned 7.4 image without migration. Values, the job list and the scheduled set survive, and the health command works. Valkey can't be used instead: Valkey 9.1.3 refuses Redis 7.4's RDB format 12 and exits. The licence field gains the AGPL-3.0 option Redis 8 offers, and the Renovate rule now follows the 8.x line.

Privacy subject export

Each version's export is tied to an exact, reviewed database schema, so I re-audited both exporters against a live 1.19.0 database:

  • the Rails collector that runs in the managed app;
  • the Python exporter that answers requests from backups.

Compared with 1.15.3, 1.19.0 adds seven tables, seven foreign keys and five columns, and removes nothing.

New export entries, in both exporters:

Entry Contents Withheld
geofence-events Arrival and departure events with position, device and place name (portable) Secret-named metadata keys, as elsewhere
webhooks Name, event types, places, state Signing secret; the URL is reduced to scheme, host and non-default port, because a webhook path or query is often the receiver's credential
webhook-deliveries Delivery status, attempts, HTTP status Response body and error message, which come from the receiving endpoint
legacy-area-place-mappings 1.18's area-to-place bookkeeping —
point-revisions, track-revisions Per-month revision counters —

Other changes to the export:

  • New columns: places gain visit_radius, geofence and archived_at, visits gain location_label, and trip sources gain skip_ssl_verification.
  • New preferences: the seven harmless preference keys 1.19 added are allowlisted in the settings export.
  • Enums: geofence and delivery enums export as the same labels from SQL as from Rails.

A gap in the fail-closed check, now closed. The revision counters carry user_id without a foreign-key constraint. The existing check compares only the foreign-key graph and the reviewed models, so it would not have noticed them. Both exporters now also compare the complete table inventory, and the exact-image test proves an added unconstrained table makes the collector refuse with unsupported_schema.

Fingerprint. The new schema fingerprint d7289062… was computed from the live 1.19.0 database. The same runner reproduces the committed 1.15.3 fingerprint 4ae830ea… exactly.

Backups. They now record the dawarich-1.19.0 contract. As agreed, there is no legacy profile, so the backup exporter refuses a backup taken on 1.15.3.

Timeline API compatibility

I seeded identical data into live 1.15.3 and 1.19.0 instances. The data included a visit with a place, one without, a track and points. On both versions, /api/v1/users/me, /settings, /timeline and /tracks validate against OpenMapX's response schemas, including the paging headers, and /health answers ok. The only difference is an extra visit_radius on a visit's place. The API is still authorized only by the user's API key.

Geofencing webhooks and network reach

Dawarich 1.18 lets each user send geofence events to a webhook URL. The managed app runs with SELF_HOSTED=true, so Dawarich accepts private addresses and Docker service names; it blocks only cloud-metadata, multicast and reserved ranges. Dawarich containers share the openmapx Docker network. A Dawarich user can therefore make the worker send a signed POST with a fixed JSON body to another OpenMapX service, and see its status code and error message. Dawarich stores the response body but does not show it in the UI or API.

This is the same class of reach 1.15.3 already allowed for the Immich and PhotoPrism integration URLs. The operator runbook and the release note now describe it. A natural follow-up is to put managed Dawarich on its own Docker network, so it can reach only its database, Redis and the internet; I left that architectural change out of this PR.

The privacy policy describes managed Dawarich generically and is unchanged. Whether it should mention user-configured webhook transfers is a legal-text decision for you.

Docs and release note

  • The operator runbook covers the 1.15.3 → 1.19.0 background jobs (SKIP_VISITS_FLEET_REDETECT), the separate worker, Redis compatibility, the refused 1.15.3 backups and the webhook reach.
  • Version references in the guides are updated.
  • release-notes/managed-dawarich-1-19.md declares migration, security and operator-action impact.

Verification

  • Exact-image subject export: OPENMAPX_RUN_DAWARICH_EXPORT_TESTS=1 with 1.19.0 passes. It now seeds subject and foreign rows in every new table and checks:
    • subject isolation;
    • the reduced webhook origin and enum labels;
    • that no webhook secret, URL path or query, delivery response or error, foreign revision, or secret-named metadata appears;
    • the refusal when an unconstrained owner table is added.
  • Retained-backup export: test-fixture.mjs passes through the Python exporter on a real restore. Its new rows include a webhook URL with credentials in it and a non-default port, and the fixture checks the projected origin, labels, coordinates and absence of secrets. It also checks the 45 reviewed relations and the fingerprint.
  • Webhook URL reduction: the Rails and SQL reductions agree on 14 inputs, including credentials in the URL, default and explicit ports, IPv6, non-HTTP schemes and malformed URLs. An automated security review caught that the first SQL version leaked part of a password from a URL with two @ (https://user:pa@ss@host became https://ss@host). The host pattern now excludes @ and whitespace, and both gates include that URL with a sentinel; it now exports as null.
  • Database groups: pnpm test:database api-privacy (24 files, 141 tests) and export-restore (2 files, 7 tests) pass against a disposable migrated PostGIS.
  • Repository checks: pnpm lint, pnpm check-types (32/32), pnpm check:policy, pnpm check-openapi and pnpm check-service-files pass. The full Vitest suite passes: 1,950 files, 24,055 tests.
  • Runtime smoke: the 1.19.0 web app is healthy without embedding Sidekiq. The separate worker boots on Redis 8.10, lists the new webhooks queue and passes the manifest's health command.
  • Not run: an upgrade of a real 1.15.3 deployment with data, OIDC sign-in against a live OpenMapX, the browser day viewer, CI image scans and any deployment. The release-acceptance checklist in the runbook still applies before enabling this in production.

Release and upgrade impact

Product impact: migration, security, operator-action

No package release: the built @openmapx/extension-cli and @openmapx/extension-contracts bundles are byte-identical to origin/main (built with tsup on both); the changed core privacy sources are not part of either package's delivered output.

Back up the managed Dawarich state before updating. The first start runs irreversible migrations and long background jobs. Keep the worker running until they finish, or set SKIP_VISITS_FLEET_REDETECT=1 on small servers. Review the geofencing-webhook reach before onboarding users you would not give access to the OpenMapX Docker network.

Move the managed bundle from Dawarich 1.15.3 to 1.19.0, which includes the
1.16 fixes for stale two-factor challenges and shared-photo authorization,
and its dedicated Redis from 7.4 to 8.10, which reads the data 7.4 saved.
The separate Sidekiq worker stays: 1.19 embeds jobs in the web process only
with SIDEKIQ_EMBEDDED=true, which the managed app does not set.

Re-audit the privacy subject projection against the live 1.19.0 schema in
both the Rails collector and the backup exporter. Export geofence events,
webhooks and their delivery state, legacy area-to-place mappings and the
point/track revision counters. Omit webhook signing secrets and delivery
response bodies and error messages, and reduce webhook URLs to their
origin, since a path or query is often the receiver's credential. Add the
1.19 preference keys to the settings allowlist.

Compare the complete table inventory as well as the FK graph before
collecting: the revision counters carry user_id without a constraint, so
the graph alone would have missed them. The new schema fingerprint comes
from the live 1.19.0 database, and the runner reproduces the committed
1.15.3 fingerprint.

Backups now record the dawarich-1.19.0 contract. Document the upgrade's
background jobs and the self-hosted reach of geofencing webhooks.
The backup exporter's host pattern accepted "@", so a URL with two "@"
signs, such as https://user:pa@ss@host/x, exported part of its password
as https://ss@host. Exclude "@" and whitespace from the host, accept only
hexadecimal IPv6 literals, and assert the malformed URL exports as null in
both the exact-image and retained-backup gates, as the Rails collector
already does.
@Medformatik Medformatik added documentation Documentation content, tooling or workspace dependencies dependencies Dependency updates and dependency maintenance docker Container images and Docker build or deployment tooling python Pull requests that update python code labels Oct 30, 2026
@Medformatik Medformatik self-assigned this Oct 30, 2026
@Medformatik
Medformatik merged commit b908d34 into main Oct 30, 2026
29 checks passed
@Medformatik
Medformatik deleted the chore/dawarich-1.19 branch October 30, 2026 06:08
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 30, 2026
@Medformatik Medformatik changed the title feat(dawarich): run managed Dawarich 1.19.0 on Redis 8 feat(dawarich): run managed Dawarich 1.19.0 on Redis 8 in isolated networks Oct 30, 2026
@Medformatik Medformatik changed the title feat(dawarich): run managed Dawarich 1.19.0 on Redis 8 in isolated networks feat(dawarich): run managed Dawarich 1.19.0 on Redis 8 Oct 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

dependencies Dependency updates and dependency maintenance docker Container images and Docker build or deployment tooling documentation Documentation content, tooling or workspace dependencies python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant