Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1362,6 +1362,31 @@ docker/ # Docker image of `ketch server` (Dockerfile, entrypoint, compose
metainfo's usable trackers, else a magnet's. Trackers Ketch adds to public torrents are never
listed. The inspector's Trackers tab (`TrackersTab`, `trackerView`), offered for torrents on
devices with the capability, asks again every 3 s while it shows
- Moving files: `TorrentController.relocate(taskId, directory)` (REST
`PUT /api/torrents/{id}/location` with a `TorrentRelocateRequest`, answering the snapshot;
capability `relocate`) moves what a task saved into another folder under the name it has. The
controller pauses a downloading or queued task and stops a seeder, then `SourceHost.relocate`
(core, under the task's control lock, with no execution active) reserves `<directory>/<name>`,
refusing a relative folder, the folder the task is in, one inside its files, a name that is
taken or another unfinished task's output, runs the source's move and saves the new
`outputPath` and source state in one record write, also in a `Completed` state.
`TorrentDownloadSource.relocate` moves the tree (`moveTorrentTree`: a rename, else between
volumes a copy that is deleted again when it fails, then the original deleted), deletes the
bookkeeping beside the old place (the v1 `.ketch-<hash>-<task>` folder, the v2 creation log)
and answers the state with the new `savePath`, no checkpoint and `relocated`, so the next run
checks the files where they are and takes them as the task's own: v2 and hybrid stores through
`adoptExisting`, and v1 stores through `TorrentPieceStore(adoptExisting)`, which records the
selected files in place and their folders as owned as it initializes (also for a task added
to use files already there). The task then goes on: resumed, or reopened to be checked and
seed again (`prepareRecheck`, `SourceHost.reopen`); a paused one is checked when it next
starts. Payload counters start again with the new checkpoint, and pieces kept for skipped
files in the v1 sidecar are downloaded again when their files are chosen. The server lets a
caller move only files it may delete, into a folder and to a path (`<directory>/<name>`) it
may save at (`DestinationGuard.checkMove`, 403 `path_rejected`). A copy between volumes whose
original cannot be deleted afterwards still counts as moved, with a warning, and a moved task
removed with its files before it runs again takes them as its own first (`v1Store` and
`v2Store` with `adopt`). The
apps have no control for it yet
- Editing trackers: `TorrentController.editTrackers(taskId, TorrentTrackerEdit(add, remove))`
(REST `PUT /api/torrents/{id}/trackers`, answering the list; capability `tracker-edit`) removes
the trackers at the endpoints `remove` names and appends each announce URL of `add` as a tier
Expand Down
1 change: 1 addition & 0 deletions docs/developers.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ These exist on servers listing `torrent.control`:
| `GET /api/torrents/{id}/priority`, `PUT` | The files it downloads before its others, as `{"fileIds": [...]}`; `PUT` sets them, an empty list for the usual order (`file-priority` capability) |
| `POST /api/torrents/{id}/recheck` | Check its files against the torrent's hashes again and download what is missing; answers its snapshot (`recheck` capability) |
| `GET /api/torrents/{id}/trackers` | Its own trackers in tier order: endpoint (never the path), state (`idle`, `announcing`, `working`, `failed`, `timed-out`), peers named, seeders, leechers and what the tracker said (`trackers` capability) |
| `PUT /api/torrents/{id}/location` | Moves its files into `{"directory": absolute folder}` under the name they have and answers the snapshot; `path_rejected` for a folder callers may not save to, `invalid_input` for a folder the files cannot move into, `storage_failure` when the move fails (`relocate` capability) |
| `PUT /api/torrents/{id}/trackers` | Adds and removes its own trackers: `{"add": [announce URLs], "remove": [endpoints]}`, answering the new list; `invalid_input` for a URL that is not `http`, `https` or `udp` (`tracker-edit` capability) |
| `GET /api/torrents/{id}/export?metainfo=true` | Its info hashes and magnet link (none for a private torrent), with its `.torrent` file as Base64 when `metainfo` is true (`export` capability) |
| `GET /api/torrents/{id}/events` | Deprecated: SSE `snapshot` events of it; the event stream carries them |
Expand Down
17 changes: 17 additions & 0 deletions docs/torrent.md
Original file line number Diff line number Diff line change
Expand Up @@ -547,6 +547,23 @@ trackers, and a magnet link's trackers can be edited only once its metadata is f
`{"add": ["udp://tracker.example:6969/announce"], "remove": ["udp://old.example:1337"]}`, which
answers the new list; SDK: `TorrentController.editTrackers(taskId, TorrentTrackerEdit(...))`.

### Moving a torrent's files

A torrent's files can be moved to another folder without adding it again:
`PUT /api/torrents/{id}/location` with `{"directory": "/absolute/folder"}`, on devices listing
the `relocate` capability (SDK: `TorrentController.relocate(taskId, directory)`); the apps have
no button for it yet. The files keep their name inside the new folder. A downloading torrent
stops for the move and goes on after it; a finished one is checked at its new place and seeds
again if it was seeding; a paused one is checked when it next starts. On the same volume the move
is a rename; between volumes the files are copied and the originals deleted once the copy is
complete, which takes as long as copying them does, and a copy that fails is removed again,
leaving the files where they were. The move is refused for a folder that is not an absolute
path, the folder the torrent is in already, a folder inside its own files, and a folder that
already has something of the torrent's name. A server lets callers move files only into folders
they may save to (see [the daemon's folders](developers.md)). After a move the torrent's upload
and download totals start again, and the parts of skipped files it had kept are downloaded
again if you choose those files later.

### How download settings apply to torrents

Torrent tasks follow the same download settings as HTTP and FTP tasks, with these differences:
Expand Down
60 changes: 60 additions & 0 deletions library/core/src/commonMain/kotlin/com/linroid/ketch/core/Ketch.kt
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ import com.linroid.ketch.core.engine.decodeState
import com.linroid.ketch.core.engine.unknownFilesMessage
import com.linroid.ketch.core.file.DefaultFileNameResolver
import com.linroid.ketch.core.file.FileNameResolver
import com.linroid.ketch.core.file.OutputPathReservations
import com.linroid.ketch.core.file.requireDownloadDirectory
import com.linroid.ketch.core.geo.CountryLookup
import com.linroid.ketch.core.speed.SpeedModeKeeper
Expand Down Expand Up @@ -106,6 +107,7 @@ import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeoutOrNull
import okio.Path.Companion.toPath
import kotlin.concurrent.Volatile
import kotlin.concurrent.atomics.AtomicBoolean
import kotlin.concurrent.atomics.ExperimentalAtomicApi
Expand Down Expand Up @@ -679,6 +681,12 @@ class Ketch(
}
}

override suspend fun relocate(
task: DownloadTask,
directory: String,
move: suspend (from: String, to: String, state: SourceState?) -> SourceState?,
): String = relocateTask(requireHandle(task), directory, move)

override suspend fun reopen(task: DownloadTask): Boolean {
val handle = handleOf(task) ?: return false
return reopenCompleted(handle)
Expand Down Expand Up @@ -712,6 +720,58 @@ class Ketch(
}
}

/** What [SourceHost.relocate] does for [handle]. */
private suspend fun relocateTask(
handle: RealDownloadTask,
directory: String,
move: suspend (from: String, to: String, state: SourceState?) -> SourceState?,
): String {
val taskId = handle.taskId
coordinator.awaitCompletion(taskId)
return handle.controlLock.withLock {
val record = handle.record.value
val from = record.outputPath
check(from != null && record.state != TaskState.CANCELED && !coordinator.isActive(taskId)) {
"The download has nothing to move, or is running"
}
check("://" !in from && "://" !in directory) { "Document folders cannot be moved between" }
val source = from.toPath().normalized()
val folder = directory.toPath().normalized()
require(folder.isAbsolute) { "The folder to move to must be an absolute path" }
val target = folder / source.name
require(target != source) { "The download is in that folder already" }
require(generateSequence(folder) { it.parent }.none { it == source }) {
"A download cannot be moved into itself"
}
val to = target.toString()
val taken = unfinishedOutputs(taskId).any { other ->
"://" !in other && other.toPath().normalized().let { owned ->
owned == target || generateSequence(owned.parent) { it.parent }.any { it == target } ||
generateSequence(target.parent) { it.parent }.any { it == owned }
}
}
require(!taken && OutputPathReservations.reserveUnique(to).also { reserved ->
if (reserved != to) OutputPathReservations.release(reserved)
} == to) { "The folder already has something of that name" }
try {
// Saved even when the caller is cancelled once the files are at their new place.
withContext(NonCancellable) {
val state = move(from, to, record.sourceState)
handle.record.update {
it.copy(outputPath = to, sourceState = state, updatedAt = Clock.System.now())
}
(handle.mutableState.value as? DownloadState.Completed)?.let { completed ->
handle.mutableState.value = completed.copy(outputPath = to)
}
}
} finally {
OutputPathReservations.release(to)
}
log.i { "Moved the files of taskId=$taskId" }
to
}
}

/** [task] when it is a task of this instance. */
private fun handleOf(task: DownloadTask): RealDownloadTask? =
(task as? RealDownloadTask)?.takeIf { real -> _tasks.value.any { it === real } }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -141,4 +141,22 @@ interface SourceHost {
* task is the caller's to stop first.
*/
suspend fun reopen(task: DownloadTask): Boolean = false

/**
* Moves what [task] saved into the folder [directory], under the name it has: [move] gets the
* path it saves at, the path beside [directory] reserved for it and its saved source state,
* moves the files and answers the source state to save with the new path. Runs under the
* task's control lock, so nothing selects, shares or reopens it meanwhile. Returns the new
* path.
*
* @throws IllegalStateException for a task that runs, was canceled or saved nowhere yet, or
* that saves to a document URI
* @throws IllegalArgumentException when [directory] is where the task is already, lies inside
* what it saved, or the name is taken there, by a file or by another download
*/
suspend fun relocate(
task: DownloadTask,
directory: String,
move: suspend (from: String, to: String, state: SourceState?) -> SourceState?,
): String = throw IllegalStateException("This download cannot be moved")
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package com.linroid.ketch.endpoints.torrent

import kotlinx.serialization.Serializable

/**
* Body of `PUT /api/torrents/{id}/location`.
*
* @property directory the folder, an absolute path on the server, the task's files move into
*/
@Serializable
data class TorrentRelocateRequest(val directory: String)
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import kotlinx.serialization.Serializable
* its `.torrent` file)
* - `GET /api/torrents/{id}/priority`, `PUT` — the files it downloads before its others
* - `POST /api/torrents/{id}/recheck` — check its files again and download what is missing
* - `PUT /api/torrents/{id}/location` — moves its files into another folder
* - `GET /api/torrents/{id}/trackers` — its own trackers and how each answers
* - `PUT /api/torrents/{id}/trackers` — adds and removes trackers (`TorrentTrackerEdit`)
* - `GET /api/torrents/{id}/events` — SSE stream of its snapshots
Expand Down Expand Up @@ -96,6 +97,11 @@ data class Torrents(val parent: Api = Api()) {
@Resource("recheck")
data class Recheck(val parent: ById)

/** Moves the task's files into another folder: `PUT` with a `TorrentRelocateRequest`. */
@Serializable
@Resource("location")
data class Location(val parent: ById)

/**
* The task's own trackers in tier order, a list of `TorrentTracker`; `PUT` with a
* `TorrentTrackerEdit` changes them and answers the list.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import com.linroid.ketch.api.Interest
import com.linroid.ketch.api.log.KetchLogger
import com.linroid.ketch.endpoints.model.ErrorResponse
import com.linroid.ketch.endpoints.torrent.TorrentPriorityRequest
import com.linroid.ketch.endpoints.torrent.TorrentRelocateRequest
import com.linroid.ketch.endpoints.torrent.TorrentSeedingRequest
import com.linroid.ketch.endpoints.torrent.TorrentSequentialRequest
import com.linroid.ketch.endpoints.torrent.TorrentSelectionRequest
Expand Down Expand Up @@ -371,6 +372,16 @@ internal class RemoteTorrentController(
}
}

override suspend fun relocate(taskId: String, directory: String) {
requireCapability(TorrentCapability.RELOCATE)
call<TorrentSnapshot>(notFoundIsNull = false) {
httpClient.put(Torrents.ById.Location(parent = Torrents.ById(id = taskId))) {
contentType(ContentType.Application.Json)
setBody(TorrentRelocateRequest(directory))
}
}
}

override suspend fun trackers(taskId: String): List<TorrentTracker>? {
requireCapability(TorrentCapability.TRACKERS)
return call(notFoundIsNull = true) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,29 @@ internal class DestinationGuard(
}
}

/**
* Checks that [task]'s files may be moved into [directory] (only the folder for a task this
* server does not have, which no move then happens for): that they may be deleted where
* they are ([checkDeletion]), as a move takes them away from there, and that both the folder
* and the path they get in it, the folder and the name they have, are where a download may be
* saved, so no move makes a hidden or protected file out of a download's name.
*
* @throws PathRejectedException if the files, the folder or the new path lie outside the
* folders
*/
suspend fun checkMove(task: DownloadTask?, directory: String) {
task?.let { checkDeletion(it) }
val policy = policy(ketch.status().system.downloadDirectory)
if (passesAsUri(directory) || !policy.contains(directory)) {
throw PathRejectedException(directory)
}
val from = task?.outputPath ?: (task?.state?.value as? DownloadState.Completed)?.outputPath
val name = from?.trimEnd('/', '\\')?.substringAfterLast('/')?.substringAfterLast('\\')
if (name.isNullOrEmpty()) return
val target = directory.trimEnd('/', '\\') + "/" + name
if (!policy.contains(target)) throw PathRejectedException(target)
}

/**
* Checks that removing [task] with its files deletes nothing outside the folders.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -606,7 +606,7 @@ private fun Route.apiRoutes(
) {
serverRoutes(ketch, destinations, onConfigChanged)
downloadRoutes(ketch, destinations)
ketch.controller(TorrentController)?.let { torrentRoutes(ketch, it) }
ketch.controller(TorrentController)?.let { torrentRoutes(ketch, it, destinations) }
ketch.controller(CountryDatabase)?.let { countryRoutes(it) }
ketch.controller(SpeedModes)?.let { speedRoutes(it) }
connectionRoutes(ketch, connectionStreams)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import com.linroid.ketch.api.KetchApi
import com.linroid.ketch.api.log.KetchLogger
import com.linroid.ketch.endpoints.model.ErrorResponse
import com.linroid.ketch.endpoints.torrent.TorrentPriorityRequest
import com.linroid.ketch.endpoints.torrent.TorrentRelocateRequest
import com.linroid.ketch.endpoints.torrent.TorrentSeedingRequest
import com.linroid.ketch.endpoints.torrent.TorrentSequentialRequest
import com.linroid.ketch.endpoints.torrent.TorrentSelectionRequest
Expand All @@ -16,6 +17,7 @@ import com.linroid.ketch.torrent.api.TorrentInterest
import com.linroid.ketch.torrent.api.TorrentPageRequest
import com.linroid.ketch.torrent.api.TorrentSettingsChange
import com.linroid.ketch.torrent.api.TorrentTrackerEdit
import com.linroid.ketch.server.DestinationGuard
import io.ktor.http.HttpStatusCode
import io.ktor.server.resources.get
import io.ktor.server.resources.post
Expand Down Expand Up @@ -45,7 +47,11 @@ private const val MAX_CURSOR_LENGTH = 4096
* The deprecated per-torrent event stream follows [TorrentInterest] on [ketch]; clients of the
* event stream follow it there.
*/
internal fun Route.torrentRoutes(ketch: KetchApi, controller: TorrentController) {
internal fun Route.torrentRoutes(
ketch: KetchApi,
controller: TorrentController,
destinations: DestinationGuard,
) {
get<Torrents.Capabilities> {
call.respond(controller.capabilities())
}
Expand Down Expand Up @@ -152,6 +158,15 @@ internal fun Route.torrentRoutes(ketch: KetchApi, controller: TorrentController)
call.respond(controller.snapshot(taskId) ?: throw notFound(taskId))
}

put<Torrents.ById.Location> { resource ->
val taskId = resource.parent.id
val body = call.receiveJson<TorrentRelocateRequest>()
log.d { "PUT /api/torrents/$taskId/location" }
destinations.checkMove(ketch.tasks.value.firstOrNull { it.taskId == taskId }, body.directory)
controller.relocate(taskId, body.directory)
call.respond(controller.snapshot(taskId) ?: throw notFound(taskId))
}

get<Torrents.ById.Trackers> { resource ->
val taskId = resource.parent.id
log.d { "GET /api/torrents/$taskId/trackers" }
Expand Down
Loading
Loading