Skip to content

Roadmap: production-quality pure Kotlin torrent engine v2 #162

Open

Description

@linroid

Track a production-quality, pure Kotlin torrent downloader and seeder for Ketch: BitTorrent v1,
v2 (BEP 52) and hybrid torrents, on the SDK,
desktop, Android, iOS, CLI and daemon (browsers go through the daemon with RemoteKetch).

Status: in progress — refreshed 2026-10-11. Formats, peer wire, trackers, encryption, uTP,
port mapping, local discovery, web seeds and the Fast extension have landed. Streaming, storage
management, seed goals, creation, network policy and release qualification remain open.

Rules

  • Protocols, hashing, scheduling, congestion control and storage coordination are Kotlin, mostly
    in commonMain. Platform code is limited to sockets, files, TLS, secure random and lifecycle.
  • No native torrent engine, libutp, JNI binding or external process in product artifacts.
    Independent torrent clients are test-only peers.
  • Every area below is a release requirement; a setting may turn a capability off, but it must
    still be implemented.

Status

Area Status Remaining
Formats: v1, v2, hybrid; padding and empty files ✅ Done —
Inputs: local/HTTP(S)/SDK metainfo, btih/btmh/dual-topic magnets ✅ Done —
Metadata: fetch, serve, v2 hash exchange ✅ Done —
Peer wire: TCP both ways, v2/hybrid negotiation, Fast extension ✅ Done —
Trackers: HTTP(S)/UDP, tiers, IPv6, scrape, private policy ✅ Done —
Transport: uTP, TCP fallback, MSE/PE encryption ✅ Done —
Discovery: DHT, PEX, local discovery, explicit peers ✅ Done IPv6 local discovery dials unique-local peers only (#513)
Web seeds 🟡 Mostly BEP 19 done for v1/v2/hybrid; BEP 17
Reachability 🟡 Partial NAT-PMP/UPnP done (scripted gateways only); PCP, hole punching (BEP 55), a real-router check
Scheduling 🟡 Partial Per-file priorities (rarest-first, endgame and live selection are done)
Seeding 🟡 Partial Ratio/time goals and a seed queue (choking, control and restore are done)
Storage 🟡 Partial Import existing data, relocation, rename (recheck is done)
Product controls 🟡 Partial uTP/encryption capabilities advertised (#513); recheck, relocate, rename, create, streaming and proxy commands in SDK, daemon, CLI, apps and remote
Operations 🟡 Partial Shaped-network, physical-device and 72-hour soak evidence (diagnostics and the peer survey are done)
Streaming: sequential/deadline, seek, verified reads ❌ Open —
Network control: proxy, interface binding, IP filters ❌ Open —
Mobile: document-tree destinations, metered/battery policy ❌ Open —
Creation: v1/v2/hybrid metainfo and magnets ❌ Open —

Next

  1. Verified streaming.
  2. Storage management (import, relocate, rename), then mobile destinations and policy.
  3. Seed goals (ratio/time) and a seed queue.
  4. Torrent creation and export.
  5. Network policy (proxy, interface binding, IP filters), PCP and hole punching.
  6. Matching controls everywhere, then release qualification.

Out of scope

Browser-local WebTorrent/WebRTC, I2P/Tor, DHT feeds and mutable items, sync, search/indexing,
RSS automation, running a tracker, legacy BEP 30 Merkle torrents, FTP web seeds, super-seeding
and creating symlinks from metainfo. These are listed as unsupported in the docs; unknown
extensions stay ignorable.

Release gates

  • Download and upload interoperability with two independent implementations across v1/v2/hybrid,
    TCP/uTP, plaintext/encrypted, IPv4/IPv6 and every discovery source; skipped tests prove nothing.
  • Fuzzing of parsers and protocol state, fault injection (disconnects, corrupt data, crashes,
    storage loss), and bounded memory and connections.
  • Physical Android and iOS devices, migration of existing tasks, packaged CLI/daemon smoke tests
    and a 72-hour soak.

A missing required capability or a failed integrity, durability or privacy gate blocks release.

References

Support and limits ·
Verification ·
Implementation progress

The original design notes, per-PR history and earlier status records are in this issue's edit
history and comments.

Activity

  1. self-assigned this
    on Sep 8, 2026
  2. linroid commented on Sep 12, 2026

    @linroid
    OwnerAuthor

    Implementation progress archive through PR #191

    Active implementation stack

    Foundation review and baseline follow-up

    • Torrent v2 01a: require executed conformance with pinned clients #163–Torrent v2 01e: admit session state before storage allocation #167 passed their latest full CI runs after the report-isolation fix. Superseded canceled runs remain in GitHub history.
    • New review on Torrent v2 01e: admit session state before storage allocation #167: failed deletion must retain session admission. Fixed in 1c7f7b25, with retry/keep-data regression tests and 304 passing local torrent tests. The updated head passes all CI checks, including iOS and result publication.
    • Merge authorization for the foundation stack has been requested separately; no PR has been merged by this goal.
    • PR #168, ready for review and stacked on Torrent v2 01e: admit session state before storage allocation #167: streamed deterministic fixtures, alternating downloader order, raw timing/memory/first-verified-progress samples, and fixed child-process deadlines. The native measurement path now uses the pinned Transmission executable directly, measuring its daemon PID and excluding the Java RPC controller. The revised smoke test passes both engines with payload verification.
    • The 1/10 GiB baseline is incomplete: an initial native seeder JVM crashed on the Finalizer thread after one Kotlin 1 GiB sample; isolating the seeder as a pinned Transmission process kept the harness alive, but the isolated libtorrent4j downloader also exited with SIGSEGV on its first 1 GiB run. Both failed reports are preserved locally. Neither partial run is an accepted performance baseline.
    • Benchmark execution has finished locally. Publication of the prepared benchmark evidence awaits explicit export approval; no production release gate is marked complete.
    • Next: complete and assess the large baseline, then continue production profiles, deterministic harness, runtime and format capabilities. No production performance target has been relaxed.

    Resource admission and v2 integrity follow-up

    • PR #169, ready for review and stacked on test(torrent): isolate and stream the large-fixture TCP benchmark #168: aggregate payload-handle admission before blocking storage dispatch, with cancellation/provider-failure coverage. 308 local torrent tests and both pinned conformance scenarios pass; CI is running.

    • PR #170, ready for review and stacked on feat(torrent): bound aggregate payload file operations #169: incremental common Kotlin SHA-256, streaming v2 file roots, and bounded block-proof verification. All 11 focused JVM tests and the native-runtime guard pass; CI is running.

    • The independent integrity primitives advance step 05 while the remaining foundation/runtime work continues. They do not yet enable v2 downloads: identity/metainfo, wire acquisition, layer storage, hybrid verification and verified commits remain incomplete.

    • PR #171, ready for review and stacked on feat(torrent): add common SHA-256 and v2 Merkle integrity #170: full v2/hybrid magnet identities, strict multihash decoding, duplicate/conflict handling, and all-topic authentication. All 324 local torrent tests, both pinned conformance scenarios, and the native-runtime guard pass; CI is running. The v1 runtime still rejects v2 topics until format/wire integration lands.

    • PR #172, ready for review and stacked on feat(torrent): retain v2 and hybrid magnet identities #171: bounded v2 raw-info trees and external piece-layer authentication, including binary paths, exact topics, missing/corrupt layers and shared-root reuse. All 333 local torrent tests, both pinned conformance scenarios, and the native-runtime guard pass; CI is running. Enclosing document loading, hybrid consistency, output mapping and runtime integration remain incomplete.

    • PR #173, ready for review and stacked on feat(torrent): validate v2 info trees and external piece layers #172: bounded document imports require validated external layers; hybrid layouts check filenames/order/lengths/alignment and preserve v1 indices with virtual padding. All 342 local torrent tests, both pinned conformance scenarios, and the native-runtime guard pass; CI is running. Large streamed imports, output mapping and dual payload/runtime integration remain incomplete. The feat(torrent): validate v2 info trees and external piece layers #172 empty-directory review is fixed in 6818eff5 and inherited here.

    Layout and payload verification slices

    • feat(torrent): validate v2 info trees and external piece layers #172 and feat(torrent): validate v2 documents and hybrid layouts #173 passed all CI checks at their current heads. The feat(torrent): validate v2 documents and hybrid layouts #173 request to omit the piece-layer dictionary was evaluated against current BEP 52 and answered with the explicit mandatory-field rule; strict document validation is retained.

    • PR #174, ready for review and stacked on feat(torrent): validate v2 documents and hybrid layouts #173: file-aligned v2/hybrid addressing, virtual padding, original hybrid file IDs, and 10 TiB arithmetic without per-piece index arrays. Four focused layout tests and the native-runtime guard pass; CI is running.

    • PR #175, ready for review and stacked on Add v2 and hybrid torrent content layout #174: streamed payload authentication against imported v2 roots/layers and both hashes for hybrids, including virtual padding and short piece-layer tails. All 350 local torrent tests, both existing pinned-client v1 conformance scenarios, and the native-runtime guard pass; CI is running.

    • These slices advance step 06. Staging the same verified bytes, durable storage commits, generation/cancellation barriers, filesystem mapping and complete v2 runtime/interoperability remain pending. No full roadmap step or release gate is marked complete by these primitives.

    • PR #176, ready for review and stacked on Verify streamed v2 and hybrid payload pieces #175: fixes an existing runtime race where canceled blocking writes published piece availability/progress. Publication now occurs after the I/O cancellation boundary under the store mutex. The blocked-write regression failed before the fix and passes after it, including recheck recovery of bytes written during cancellation. All 351 local torrent tests, both existing pinned-client v1 conformance scenarios, and the native-runtime guard pass. V2 storage adapters and generation-tagged lifecycle work remain pending.

    • Review follow-up on Verify streamed v2 and hybrid payload pieces #175: fixed unbounded hybrid virtual-padding hashing in 8b365cd1. Verifier construction now enforces the existing 16 MiB runtime piece ceiling before layout allocation or hashing; metadata parsing can still describe larger protocol-valid dimensions. The regression rejects the reported one-byte payload/2^62-byte piece immediately and tests the ceiling. All five focused verifier tests and the native-runtime guard pass. Publish piece availability after storage cancellation boundary #176 is restacked onto the fix; combined verifier/storage regressions pass. Add v2 and hybrid torrent content layout #174 now passes all CI checks.

    • PR #177, ready for review and stacked on Publish piece availability after storage cancellation boundary #176: versioned output components preserve ordinary Unicode names, encode unsafe raw names, bound long components and separate sibling case/normalization collisions across files and directories. Hybrid IDs remain original v1 indices and padding has no output. All 356 local torrent tests, both existing pinned-client v1 conformance scenarios, and the native-runtime guard pass. Provider/root/ownership validation, persisted mapping migration and v2 storage integration remain pending.

    • CI follow-up: Verify streamed v2 and hybrid payload pieces #175 run 34699215911 failed its Windows job 103567988029 while the native reference seeder waited to enter seeding state (IndependentSeederTest line 57, 20-second fixture-startup timeout), before the Kotlin downloader connected. Publish piece availability after storage cancellation boundary #176 passed its Windows job with the same verifier change. The failed workflow is still running; GitHub declined an individual-job rerun. This failure remains unresolved pending a permitted rerun and, if repeated, further native fixture diagnosis. Captured report evidence is retained locally; no test or production acceptance limit has been relaxed.

    • CI follow-up resolved for the current Verify streamed v2 and hybrid payload pieces #175 head: Windows retry job 103570382993 on run 34699215911 attempt 2 passed, and all required checks are now green. The first attempt's native fixture-startup timeout remains in the run history; no acceptance limit was relaxed.

    • Map v2 output names with versioned collision-safe components #177 review fix a1a0fc56 handles sharp-S aliases symmetrically (ß/ẞ/ss) before assigning output paths. All four mapping tests pass, and the fix is inherited by the next storage slice.

    • PR #178, ready for review and stacked on Map v2 output names with versioned collision-safe components #177: fresh-directory v2/hybrid filesystem storage integrates output mapping, dual verification, file-relative writes, selected progress, private leased buffers, shared handle admission and cancellation-safe flush/publication. Existing destinations are rejected and live cleanup checks OS ownership identities. All 362 local torrent tests, both existing pinned-client v1 conformance scenarios and the native-runtime guard pass at 4f7301df. Engine/state admission, persistent ownership/checkpoint v2, resume/import/recheck, committed reads/proof serving, generation-tagged sessions and alternate providers remain pending; v2 interoperability is not yet verified.

    • PR #179, ready for review and stacked on Add verified fresh-directory v2 and hybrid piece storage #178: committed reads authenticate owned-file snapshots and retain buffer leases until consumer close; corrupt/truncated/replaced files revoke affected availability without exposing bytes. Recheck discards old progress, scans selected owned files with at most 64 KiB scratch, flushes matching data and publishes through the cancellation boundary. All 366 local torrent tests, both existing pinned-client v1 conformance scenarios and the native-runtime guard pass at 6810f3d4; blocked-recheck cancellation/retry and replacement preservation are covered. Restart/import ownership, checkpoint v2, session admission/integration and wire proof serving remain pending.

    • Add verified fresh-directory v2 and hybrid piece storage #178 now passes all CI checks. Review fix e69f7615 on Authenticate v2 stored reads and rebuild availability with bounded rechecks #179 rechecks through one handle per file and flushes once for matching pieces. Tentative bits are rolled back if flush or cancellation prevents publication; the regression asserts one flush for four pieces and no publication on flush failure. All five focused recheck tests pass.

    • PR #180, ready for review and stacked on Authenticate v2 stored reads and rebuild availability with bounded rechecks #179: bounded authenticated content catalog keyed by full v2 identity, preserving exact info/layers while excluding caller context. Loads revalidate all requested hybrid topics and layers; corrupt/oversized entries are rejected without overwrite. Publication flushes a unique temporary object before atomic rename and cleans up owned temporaries on failure. All 372 local torrent tests, both existing pinned-client v1 conformance scenarios and the native-runtime guard pass at d6b88c6d. Runtime admission, full-profile streaming/spilling, reference accounting/GC, directory-fsync power-loss guarantees, task ownership and checkpoint v2 references remain pending.

    • Persist authenticated v2 content in a bounded catalog #180 review fix 5d61004a replaces catalog publication with atomic no-replace hard-link creation on JVM/Android and iOS, followed by authentication of the winning target. A race regression inserts valid and invalid competing entries at publication and proves neither is overwritten. All six catalog tests and iOS simulator compilation pass; unsupported hard-link providers fail explicitly rather than use a replacing fallback.

    • PR #181, ready for review and stacked on Persist authenticated v2 content in a bounded catalog #180: bounded version 2 checkpoint codec with full catalog identity, hybrid topics, task/output/mapping binding, selections, generations, counters and bitfield hints. Prior-parent ownership rows compact repeated prefixes; validation rejects malformed/foreign claims and preserves v1 decoder routing. All 378 local torrent tests, both existing pinned-client v1 conformance scenarios and the native-runtime guard pass at b9aaa3bd. OS ownership recovery, expected task/root matching, atomic TaskStore publication, journals, v1 migration, streaming large state and resume integration remain pending. Hint bits are not accepted as committed progress.

    • PR #182, ready for review and stacked on Encode v2 checkpoints with catalog binding and compact ownership references #181: task-bound store snapshots reference the authenticated catalog and restore only recorded paths whose OS identities and kinds still match. Restore validates the expected task, canonical output root and selection before adopting ownership; checkpoint hint bits never publish availability, and recheck authenticates bytes again. Owned completed files are trimmed to their declared length. All 383 local torrent tests, both existing pinned-client v1 conformance scenarios and the native-runtime guard pass at bbcdeb143c6dff114d50c4970e7a96b99fedcb17. Atomic TaskStore publication, durable creation journals, missing-file repair/import, v1 migration, state admission and engine resume integration remain pending.

    • Encode v2 checkpoints with catalog binding and compact ownership references #181 CI follow-up: the original iOS job failed before compilation because the runner could not resolve services.gradle.org. Individual-job retry on run 34702397460 attempt 2 is running; its setup passed and simulator tests have started. The other platform jobs passed. This retry remains pending until required checks are green.

    • PR #183, ready for review and stacked on Recover v2 store ownership from task-bound catalog checkpoints #182: atomic checkpoint-file persistence publishes catalog content before task references, flushes a unique temporary snapshot before replacement, and preserves previous state on flush/replacement failure. Loads enforce task binding and byte bounds and authenticate referenced catalog content. Four new regressions cover recovery, failure/retry, missing catalog and malformed/foreign/oversized state. All 387 local JVM torrent tests, both pinned-client v1 conformance scenarios and the native-runtime guard passed before the subsequent import-only CI fixes. Single-writer task ownership is required; TaskStore/engine wiring, durable creation journals, state admission, migration and directory-fsync power-loss guarantees remain pending.

    • iOS CI diagnosis for Encode v2 checkpoints with catalog binding and compact ownership references #181/Recover v2 store ownership from task-bound catalog checkpoints #182: after the Encode v2 checkpoints with catalog binding and compact ownership references #181 DNS retry passed setup, both runs exposed missing okio.use imports in common tests (recheck, catalog and recovery). Fixed the imports in their originating PRs, propagated them through Authenticate v2 stored reads and rebuild availability with bounded rechecks #179–Persist authenticated v2 task checkpoints with atomic replacement #183, and added the same import to the new checkpoint-file tests. Full torrent iOS simulator test-source compilation and all 18 affected JVM tests pass locally. Fresh CI at the restacked heads remains pending; the earlier failed runs are not treated as resolved checks.

    • PR #184, ready for review and stacked on Persist authenticated v2 task checkpoints with atomic replacement #183: flushed, checksummed creation records allow v2 store ownership recovery before the first checkpoint, including partial initialization. The log binds full content/task/output/selection/mapping identity, uses compact parent references and bounded replay, discards torn final appends and rejects complete corruption. All claims must match authenticated output paths and current OS identities before adoption; payload progress requires recheck. A failed append can retry in the same instance, while restart preserves paths created before successful ownership recording. All 392 JVM torrent tests, both existing pinned-client v1 conformance scenarios, the native-runtime guard and full torrent iOS simulator test-source compilation pass at 31fa11f528e3f21c3282301a9ed46ea3f82f7ce0. Runtime wiring, state admission, log lifecycle/compaction, process-kill/mobile lifecycle coverage and directory-fsync power-loss guarantees remain pending.

    • Fresh Persist authenticated v2 task checkpoints with atomic replacement #183 CI run 34703978581 is queued on GitHub at the corrected head; no additional rerun was launched. Its superseded run was canceled.

    • PR #185, ready for review and stacked on Recover v2 payload ownership before the first checkpoint #184: four independent JVM children exit without cleanup during partial payload write, after a flushed piece without a checkpoint, before checkpoint replacement and after replacement. Recovery rechecks bytes, loads the expected complete old/new snapshot, completes the payload and preserves an unrelated neighbor during cleanup. All four process-crash tests pass at 9f23c0db; controlled process exits do not establish power-loss ordering, physical mobile lifecycle behavior or v2 runtime/network integration.

    • Executed iOS evidence for the Recover v2 payload ownership before the first checkpoint #184 base (31fa11f5): all 375 torrent simulator tests passed with zero failures/errors/skips using :library:torrent:iosSimulatorArm64Test -PenableIosSimulatorTests=true. An initial invocation omitted the flag and skipped execution; that successful build is not counted as test evidence. The new Verify v2 recovery across abrupt process exits #185 tests are JVM-only and leave those common/iOS sources unchanged.

    • PR #186, ready for review and stacked on Verify v2 recovery across abrupt process exits #185: bounded BEP 52 hash request/response/reject codec preserves full roots and unsigned indices, validates aligned power-of-two ranges and exact response lengths, and accounts for omitted proof layers. Three focused JVM tests with exact wire vectors and malformed-input cases pass; full torrent iOS simulator test-source compilation passes at c238102b. This advances step 08 without enabling v2 semantics in the v1 runtime. Tree-bound validation, request correlation, resource admission, proof authentication/serving, negotiation and executed v2 interoperability remain pending.

    • PR #187, ready for review and stacked on Add bounded BEP 52 hash exchange wire messages #186: authenticates exactly correlated hash responses against trusted file roots, validates file-tree bounds, reduces aligned base hashes and uncle proofs, and rejects noncanonical padding. Four focused JVM tests cover valid leaf/piece-layer ranges, whole layers, per-byte corruption, selector mismatch, malformed/partial proofs, padding and maximum signed 64-bit file-length arithmetic. All four tests and iOS test-source compilation pass at 1403f581. Outstanding-request ownership, byte admission/caching, intermediate anchors, proof serving and live v2 integration remain pending; hash verification alone does not authorize payload progress.

    • Encode v2 checkpoints with catalog binding and compact ownership references #181 now passes every required CI check at its corrected head, including iOS and result publication. Recover v2 payload ownership before the first checkpoint #184–Add bounded BEP 52 hash exchange wire messages #186 still had active checks at the latest inspection; their pending checks are not treated as complete.

    • PR #188, ready for review and stacked on Authenticate peer hash proofs against trusted file roots #187: connection-owned hash-request tracking reserves shared credit before issuance, validates trusted tree bounds, correlates and authenticates replies within deadlines, and retains delivered-result credit until consumer release. Expiry/rejection/failure/cancellation/close release pending credit; identity-based local tickets prevent stale cancellation callbacks from releasing replacement requests. All 407 JVM torrent tests, both pinned-client v1 conformance scenarios, the native-runtime guard and iOS test-source compilation pass at e18b978ca24c7e15ba0185cb0699fba92a3e3857. Live connection routing/timers, unsolicited-frame reader admission, cache ownership, serving and executed v2 interoperability remain pending.

    • Recover v2 payload ownership before the first checkpoint #184, Verify v2 recovery across abrupt process exits #185 and Add bounded BEP 52 hash exchange wire messages #186 now pass all required CI checks. The Add bounded BEP 52 hash exchange wire messages #186 minimum-length and proof-count review suggestions were checked against current BEP 52; both conflict with its explicit rules, so the existing behavior is retained. Source-backed replies: Add bounded BEP 52 hash exchange wire messages #186 (comment) and Add bounded BEP 52 hash exchange wire messages #186 (comment) . Authenticate peer hash proofs against trusted file roots #187 still had iOS CI running at the latest inspection.

    • PR #189, ready for review and stacked on Track and admit authenticated peer hash exchanges #188: connects hash-request ownership to frame issuance, body admission before reads and actor response dispatch. Consumers retain frame credit through dispatch and verified-result credit independently. Partial-write failures and failed/timed-out/unadmitted reads close the stream. Four focused JVM tests pass, including loopback TCP exchange; all 390 iOS simulator torrent tests execute and pass with zero failures/errors/skips at 880eca8d. Handshake negotiation, the complete actor/timers, reply serving, payload integration and independent-client v2 interoperability remain pending. The inherited 64 KiB frame ceiling must become profile-aware to carry the required one-million-piece desktop bitfield (125,000 bytes plus message ID); no production target is relaxed.

    • Authenticate peer hash proofs against trusted file roots #187 now passes all required CI checks. Track and admit authenticated peer hash exchanges #188 still had iOS CI running at the latest inspection.

    • PR #190, ready for review and stacked on Connect admitted hash exchange to bounded peer transport #189: trusted piece-count bounds permit the one-million-piece desktop bitfield while unrelated/default messages retain the 64 KiB limit. Readers validate message type and exact bitfield length before body reads; padding, indices and availability use the same count bound. The transport retains credit for the larger frame through dispatch. All 415 JVM torrent tests, 394 executed iOS simulator torrent tests, both pinned-client v1 conformance scenarios and the native-runtime guard pass at 3b1f9cfe99e5de79b13e63d10ee9ee01b2937163. This removes the previously recorded bitfield wire-size obstacle; aggregate resource/RSS gates and full v2 runtime integration remain pending.

    • Connect admitted hash exchange to bounded peer transport #189 review fix bd9c0a87 bounds request writes by the outstanding exchange deadline and rechecks it before returning a ticket. The regression reproduced the old expired-ticket return and now covers suspended backpressure and a nonsuspending write finishing at its deadline. Nine focused exchange/transport tests and iOS compilation passed, and the fix is inherited by Bound desktop bitfields by authenticated piece counts #190's full suites. Review reply: Connect admitted hash exchange to bounded peer transport #189 (comment) . Track and admit authenticated peer hash exchanges #188 passes all required CI; fresh Connect admitted hash exchange to bounded peer transport #189/Bound desktop bitfields by authenticated piece counts #190 CI remains pending.

    • PR #191, ready for review and stacked on Bound desktop bitfields by authenticated piece counts #190: direct v1/v2 and explicitly offered hybrid-upgrade handshakes retain full expected identities separately from wire tags. Negotiation rejects unexpected mode changes, ambiguous local aliases, malformed framing, self-connections and expected-peer-ID mismatches; scratch admission and a ten-second deadline bound I/O, with failure closing the connection. Five focused JVM tests pass, including a validated-document v2 handshake followed by authenticated hash exchange over loopback TCP. All 399 iOS simulator torrent tests execute and pass at c6f4ddf9. Wire tags remain routing hints: full metainfo/hash/payload authentication is still required. Multi-torrent listener routing, the full payload actor, magnet integration and independent-client v2 interoperability remain pending.

  3. linroid commented on Sep 20, 2026

    @linroid
    OwnerAuthor

    Runtime capability progress (2026-09-20)

    Recovered the unpublished torrent-v2-runtime-capabilities branch after temporary worktree cleanup. Local commit b55c4816 builds on 86f59382 (global/task request rate controls). No additional PR was opened; these changes remain grouped for the larger runtime capability PR.

    The scoped v2 engine now accepts a decoded checkpoint, admits retained recovery records, and validates storage ownership before exposing the session. Resume rehashes actual payloads rather than trusting saved hints. Tests persist/reload the checkpoint and authenticated catalog, cover modified payloads, insufficient recovery admission, and invalid task binding without leaking engine claims.

    Validation: 619 JVM tests and 599 executed iOS simulator tests, zero failures/errors/skips; native torrent runtime guard passed. PR #228 required CI remains green at 8b6b1c1f. Independent engine conformance was not rerun in this checkout.

    This is partial integration, not roadmap completion. Checkpoint decoding/catalog transient admission, automatic checkpoint saving, TaskStore/source wiring, discovery, full v2 interop, and the remaining production gates are still pending. No merge or release performed.

  4. linroid commented on Sep 20, 2026

    @linroid
    OwnerAuthor

    Consolidated review stack

    Consolidation is complete: all 66 original PRs #163–#228 are closed as superseded, without merging. Each has a comment linking to its replacement. All original branch tips and review discussions are retained. The eight replacements below are open, ready for review, and passing all required CI checks. Review and merge bottom-up; merging still requires authorization.

    Order Replacement Supersedes
    1 #231 — Contracts, verification infrastructure and resource limits #163–#169
    2 #232 — V2 identity, metainfo and Merkle integrity #170–#173
    3 #233 — V2 layout and verified storage #174–#179
    4 #234 — V2 catalog, checkpoints and crash recovery #180–#185
    5 #235 — V2 handshake, hash exchange and bounded transport #186–#194
    6 #236 — V2 scheduling and download execution #195–#209
    7 #237 — Tracker lifecycle, editing and discovery privacy #210–#226
    8 #238 — V2 engine lifecycle, rate controls and restart recovery #227–#228

    Preservation checks verified every original tip and exact boundary tree. The runtime group additionally includes the tested rate-control and restart-recovery commits. Fresh consolidation CI exposed a listener shutdown race; its cancellation fix and regression test are the only additional file changes relative to those boundaries and are documented on all eight replacements. All 28 original inline review discussions and their responses are linked from the replacements; review approval is not transferred.

    Validation: all eight required CI gates passed. The final consolidated stack also passed 620 JVM tests, 600 executed iOS simulator tests, and the native-runtime guard locally. The retirement audit verified 66 closed PRs, 66 replacement links, 66 unchanged original remote branch tips, and eight ready replacement PRs with the expected bases and heads.

    This consolidates existing implementation work. The production roadmap and release gates remain unfinished; nothing was merged or released.

  5. linroid commented on Sep 20, 2026

    @linroid
    OwnerAuthor

    Public v2/hybrid download-and-restart workflow is now implemented in #240, ready for review and stacked on #238. The original eight consolidated groups remain unchanged.

    The public Ketch/source path now resolves v2/hybrid metainfo and full-identity magnets, authenticates external piece layers, downloads selected files, reports verified progress, applies shared/live limits, pauses/resumes, restores serialized task records with rechecking and creation-journal recovery, and removes only owned files. Full SHA-256 identity and saved discovery privacy survive restart.

    Local validation passed: 635 JVM, 613 iOS simulator, and 612 Android host tests, plus the no-native-runtime guard. Pure v2 and hybrid magnet downloads also passed against the independent test-only libtorrent seeder. All four automated review findings are fixed and resolved. All final-head CI checks pass on de5623c5, including JVM, iOS, Android host/device, JavaScript, macOS/Windows filesystem coverage, and the aggregate Required Checks gate.

    This completes the public download/restart integration slice, not the full production roadmap. V2 incoming routing, uploading/seeding, PEX, hybrid v1-only peer participation, second-client v2 interoperability, and the remaining production gates remain open. The documented supported workflow and these boundaries are included in the PR.

  6. linroid commented on Nov 14, 2026

    @linroid
    OwnerAuthor

    Progress since the last refresh (2026-11-14), by row of the status table:

    Area Change PR
    Product controls Export a task's magnet link and .torrent file (API, REST, apps, ketch export) #666 (merged)
    Formats Torrents within the file limit but over 100,000 bencode values were refused; one limit for every reader #667 (merged)
    Product controls A torrent's own trackers and how each answers, with a Trackers tab #669 (merged)
    Product controls, Storage Force recheck: check a finished torrent's files again and repair what is missing or replaced, v1/v2/hybrid #670
    Creation Create v1, v2 and hybrid torrents (SDK and ketch create), verified against libtorrent #671
    Storage Use files already in place (import existing data), so created torrents can be seeded #672
    Scheduling Per-file priorities: download chosen files first #673
    Seeding Seed queue: waiting torrents start as soon as a seeder stops or reaches its goal #674
    Network control IP filter: block peers by address, CIDR block or range #675

    #670 to #675 are stacked and merge in order as CI passes.

    Still open after these: storage relocation and rename, proxy and interface binding for torrents, verified streaming, mobile document-tree destinations and battery policy, BEP 17 web seeds, an app screen for creating torrents, editing a task's trackers, and the release gates that need hardware or time (physical devices, shaped networks, the 72-hour soak).

  7. linroid commented on Nov 14, 2026

    @linroid
    OwnerAuthor

    Progress update. Everything below is merged to main.

    Area PR What it adds
    Create torrents #671 v1, v2 and hybrid torrents from the SDK and ketch create, checked against libtorrent
    Use files already on disk #672 A torrent added with overwrite takes the files in place and checks them
    File priorities #673 Download chosen files before the others, peers and web seeds
    Seed queue #674 A torrent waiting to seed takes the slot as soon as a seeder stops
    IP filter #675 Block peers by address, CIDR block or range
    Edit trackers #677 Add and remove a task's own trackers, in the inspector and over REST
    Move files #678, #679 Move a torrent's files to another folder; a "Move files…" row action on desktop
    Proxy #680 Peers, HTTP trackers and web seeds through a SOCKS5 or HTTP proxy, with DHT, uTP and UDP trackers off
    Tests #681 A race in the adopt-files test

    Earlier in this series: export (#666), large v1 metainfo (#667), tracker status (#669), check files again (#670).

    Known limits of what landed:

    • Moving files has no control for remote devices or on Android, shows no progress for a copy between volumes, and resets a torrent's upload and download totals.
    • The torrent proxy is read when the engine starts, and an HTTP proxy may leave tracker host names to the system's DNS.
    • A task that has never run keeps edited trackers and file priorities only until the app restarts.

    Roadmap rows not started:

    • Binding torrents to a network interface (it has to cover HTTP trackers as well as peers to be worth having)
    • Verified streaming
    • Mobile document-tree destinations and battery policy
    • BEP 17 web seeds
    • An app screen for creating torrents
    • A move-files control for remote devices

    Release gates that cannot be met from an agent session and still need a person: physical devices, the 72-hour soak, and shaped-network runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationenhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions