From e708cbbdaa95366eacc39d063ee991d38063bf98 Mon Sep 17 00:00:00 2001 From: "opencode.siil.arti.ee" Date: Tue, 17 Nov 2026 15:55:27 +0000 Subject: [PATCH] docs: resolve grants, resume and partial-file questions in backup design --- .../2026-11-17-usb-image-backup-design.md | 168 +++++++++++++----- 1 file changed, 122 insertions(+), 46 deletions(-) diff --git a/docs/superpowers/specs/2026-11-17-usb-image-backup-design.md b/docs/superpowers/specs/2026-11-17-usb-image-backup-design.md index 15a9aaf..00e2008 100644 --- a/docs/superpowers/specs/2026-11-17-usb-image-backup-design.md +++ b/docs/superpowers/specs/2026-11-17-usb-image-backup-design.md @@ -18,10 +18,12 @@ byte-exact image that can be restored later with the existing write flow. into that file. - Reuse the existing progress screen, notifications, foreground service, wakelock and error handling. -- Compute SHA-256 while reading and show it on the success screen. +- Compute SHA-256 over the finished image and show it on the success screen. - The produced file is a plain raw image; restoring it needs no changes to the write flow. -- Cancellation and retry behave like the write flow. +- A USB hiccup mid-job can resume after reconnecting, instead of starting over. +- Interrupted jobs leave an obviously incomplete file, so it cannot be mistaken + for a good backup. ## Non-goals @@ -31,8 +33,9 @@ byte-exact image that can be restored later with the existing write flow. partial recovery. A SCSI read error aborts the job. - Compression, split files, sparse/trimmed images, or incremental backups. - Direct card-to-card cloning, USB hubs, or a second USB device as destination. -- Resuming an interrupted backup. Retry starts over and truncates the output. -- Changes to the write/verify flow. +- A user-facing cancel button. Like the write flow, a job ends by completing, + failing, or the service being destroyed. +- Changes to the write/verify flow beyond a warning for `.part` files. ## Constraints @@ -48,6 +51,9 @@ byte-exact image that can be restored later with the existing write flow. - USB 2.0 OTG reads are roughly 20–40 MB/s, so a 64 GB backup takes tens of minutes. The existing foreground service, progress notification and wakelock are required. +- SAF append mode (`"wa"`) is the only universally supported way to extend an + existing document; random-access writes are not guaranteed. The design never + seeks the output. - minSdk 23, targetSdk 36, Compose UI, FOSS and GPlay flavors. ## Approaches considered @@ -80,10 +86,13 @@ one-off card; the app feature is for doing it phone-only and repeatably. read-only for this flow: selecting a device does not touch it yet. 3. After a device is selected, launch `ActivityResultContracts.CreateDocument("application/octet-stream")` with a - suggested name such as `backup-card-reader-20261117-1530.img` (device name - sanitized; `usb-drive` when empty). Cancelling returns to the start screen. -4. Launch the confirmation screen in backup mode with the source device and - the destination URI. + suggested name such as + `backup-card-reader-20261117-1530.img.part` (device name sanitized; + `usb-drive` when empty). The `.part` suffix marks the file as incomplete + until the job succeeds. Cancelling returns to the start screen. +4. On picker result, take a persistable write grant for the destination (see + "URI permissions"), then open the confirmation screen in backup mode with + the source device and the destination URI. 5. On confirm, the confirmation screen takes the USB permission, shows the existing lay-flat sheet (the reader must stay connected for the whole job), then starts the backup job and opens the progress screen. @@ -100,12 +109,29 @@ Reuse `ConfirmOperationActivity` with a new `"operation"` string extra - Confirm button: "Start backup". - USB permission, device-detach handling, and theme setup are unchanged. +### URI permissions + +`MainActivity` takes persistable read and write grants on the CreateDocument +result (read is needed for the resume-time prefix hash and the success card's +file size), mirroring the existing read-grant handling in `openUri`, and never +releases them (the system evicts the oldest grants past its cap, as the +existing comment notes). This keeps the destination usable after the +confirmation activity finishes, when the task is swiped away, and when the +progress screen is reopened from the result notification or a retry. + +Some providers do not support persistable grants; that is caught and ignored. +The temporary grant plus `FLAG_GRANT_WRITE_URI_PERMISSION` on the intents still +covers the running job. A retry that can no longer open the URI fails with the +same "could not write the file" error. + ### Intents and state - New `Intents.START_BACKUP` action and `getStartBackupJobIntent(destinationUri, sourceDevice, jobId)`: - `data` is the destination URI, with `FLAG_GRANT_WRITE_URI_PERMISSION` when - the activity holds it; extras: `destDevice`, `jobId`. + `data` is the destination URI, with + `FLAG_GRANT_READ_URI_PERMISSION or FLAG_GRANT_WRITE_URI_PERMISSION` when the + activity holds them; extras: `destDevice`, `jobId`. There is no offset + extra: the resume point is derived from the destination file (see "Resume"). - `getConfirmOperationActivityIntent` gains the `operation` extra. - `JobStatusInfo` gains `isBackup: Boolean = false` and `sha256: String? = null`. `getFinishedIntent` gains `isBackup` and `sha256` parameters, defaulted so @@ -126,52 +152,94 @@ Backup body: 1. Build and initialize the mass storage device; pick LUN 0, or the only LUN with media (same rule as the write flow today). 2. `devSize = blocks * blockSize`; this is the progress total. -3. Best-effort free-space check (API 26+, local storage volume only). If it can - prove `available < devSize`, fail before writing with a clear - "not enough space" error. Otherwise continue and rely on the write failure - being mapped. -4. Open the destination with `contentResolver.openFileDescriptor(uri, "wt")`, - wrap it in a `FileOutputStream`/`BufferedOutputStream`. On failure, report - an "could not open the destination" error. -5. Read with `BlockDeviceInputStream` using the same buffer/prefetch settings - as `verifyImage`, in chunks (target 1 MiB). For each chunk: wakelock, feed - the SHA-256 `MessageDigest`, write to the output, notify progress. The - existing `timeoutWatchdog(IO_TIMEOUT)` guards USB stalls. -6. Flush, `fd.sync()`, close the file descriptor, then broadcast `FINISHED` - with `isBackup = true` and the hex digest. +3. Best-effort free-space check (API 26+, local storage volume only). Check + against `devSize - existingFileSize` when resuming. If it can prove space is + insufficient, fail before writing with a clear "not enough space" error. + Otherwise continue and rely on the write failure being mapped. +4. Open the destination for append (`"wa"`), and query the current document + size. That size is the resume offset; a fresh document is 0 bytes. +5. Recompute the running SHA-256 for the existing prefix by reading the + destination from 0 to the resume offset. This is a local read and is fast + relative to the USB transfer; it only happens on a resumed job. +6. Read the device with `BlockDeviceInputStream`, seeked to the resume offset, + using the same buffer/prefetch settings as `verifyImage`, in chunks (target + 1 MiB). For each chunk: wakelock, update the digest, append to the output, + flush, notify progress. The existing `timeoutWatchdog(IO_TIMEOUT)` guards + USB stalls. +7. Flush, `fd.sync()`, close the file descriptor. +8. Rename the document to drop the `.part` suffix (best-effort; use the URI + returned by `DocumentsContract.renameDocument` when non-null). Then + broadcast `FINISHED` with `isBackup = true` and the hex digest. -Failure behaviour: +### Resume -- Leaving the partial file in place is fine (the picker never overwrites an - existing document, and retry truncates). -- USB errors map through the existing `toJobException`. +A restarted backup resumes instead of restarting from zero. Because writes +always append in file order, the destination file is always a valid prefix of +the device, at every length. The service therefore trusts the file size rather +than any offset carried in an intent: + +- Fresh job: the new document is empty, so it starts at 0. +- After a USB error and reconnect: `AutoJobRestarter` starts the same backup + intent for the same URI and device; the service sees a non-empty document, + seeks the device there, re-hashes the prefix, and continues. +- After a process death or force-stop, the same applies when the user restarts + the job for the same file. Starting a new backup via the picker creates a new + document (SAF does not overwrite), so no stale bytes are mixed in. +- If the document has grown past what a previous run meant to write, the extra + bytes are still device bytes in order, so continuing from the actual end is + correct. + +`AutoJobRestarter` gains an `isBackup` mode that calls +`getStartBackupJobIntent`; its `resumeOffset` is ignored (the service derives +it), and `isVerifying` is always false for backup. + +### Output file safety + +- The suggested name ends in `.part`. On success it is renamed to drop the + suffix. If the provider refuses the rename, the success screen shows the + actual current name, so nothing is hidden from the user. +- A failed or interrupted job leaves the `.part` file in place; it is the + resume target, so it is not deleted. +- The restore flow gains a guard: if the user opens a file whose name ends in + `.part`, show a warning dialog ("this looks like an incomplete backup; + restoring it may write only part of the card") with continue/cancel, modeled + on the existing Windows-image warning. Nothing else in the write flow + changes. + +### Failure behaviour + +- USB errors map through the existing `toJobException`; recoverable ones drive + the reconnect dialog and the resume path above. - `ENOSPC` maps to a "not enough space" error; a lost URI permission or a failed open maps to a clear "could not write the file" error. -- `SKIP_VERIFY` is ignored; there is no verify pass. Hashing already happens - inline while reading. - -No verification pass runs: the read is the operation, and the digest covers -exactly the bytes written. +- `SKIP_VERIFY` is ignored; there is no verify pass. The digest already covers + the whole file. +- The partial `.part` file is kept, as described above. ### Telemetry Same scope as the write job, with a `job.type = backup` tag, the output -filename, and the device name/vidpid. No file contents or paths beyond the -existing conventions. +filename, the device name/vidpid, and the resume offset when non-zero. No file +contents or paths beyond the existing conventions. ### Restore -Unchanged. The backup file is a normal raw image; the write flow checks the -device is at least as large as the image (equal is accepted), writes it, and -verifies as usual. A restore of a full-card image takes about twice as long as -the backup because of the verify pass; that is existing behaviour. +Unchanged apart from the `.part` warning. The backup file is a normal raw +image; the write flow checks the device is at least as large as the image +(equal is accepted), writes it, and verifies as usual. A restore of a +full-card image takes about twice as long as the backup because of the verify +pass; that is existing behaviour. ## Testing - Unit (`WorkerServiceFlowImpl`): back up a `MemoryBufferBlockDeviceDriver` filled with a deterministic pattern into a `ByteArrayOutputStream`; assert exact bytes, SHA-256, progress totals, and that the stream closes. Test error - propagation when the driver throws, and cancellation. + propagation when the driver throws (partial output present and complete + prefix), and resuming from a non-empty output stream with the digest still + covering the whole file. +- Unit: cancellation (coroutine cancellation) leaves a valid prefix and the + stream closed. - Robolectric: intent builders set the right action, data, extras and flags; `JobStatusInfo.percent` and defaults remain compatible. - Existing write/verify, stream and service tests must keep passing. @@ -185,37 +253,45 @@ the backup because of the verify pass; that is existing behaviour. - `app/src/main/java/eu/depau/etchdroid/ui/ViewModel.kt` - `app/src/main/java/eu/depau/etchdroid/ui/ConfirmOperationActivity.kt` - `app/src/main/java/eu/depau/etchdroid/ui/ProgressActivity.kt` +- `app/src/main/java/eu/depau/etchdroid/ui/Notifications*` / notification helpers - `app/src/main/res/values/strings.xml` - new unit test file(s) ## UX sketch ``` -Start screen Device picker Save dialog Confirm -[ Write an image ] Select USB device backup-card-…img Back up USB drive -[ Back up a drive ] (card reader) [Save] [Cancel] card reader → file - [Cancel] [Start backup] +Start screen Device picker Save dialog Confirm +[ Write an image ] Select USB device backup-card-…img.part Back up USB drive +[ Back up a drive ] (card reader) [Save] [Cancel] card reader → file + [Cancel] [Start backup] Progress: Backing up… 12.4 GB of 59.6 GB • 18.2 MB/s Done: Backup complete + backup-card-reader-20261117-1530.img SHA-256: 9f2c… (selectable) ``` ## Left to the builder - Exact chunk size and buffer tuning (measure on the 64 GB card). -- Hashing/formatting details and filename sanitization rules. +- Hashing/formatting details and device-name sanitization rules. - Whether the free-space check is worth its complexity on the providers the user actually picks. - Test file organization. +- Whether the read-back-to-hash prefix should report progress through the + normal progress channel or stay silent. ## Risks - A long job is battery- and thermally heavy; the wakelock is renewed while progress updates flow, as in the write job. - Some cheap card readers are flaky (existing reconnect/timeout handling - applies); a disconnect mid-backup leaves a truncated image and reports an - error. + applies); a disconnect mid-backup is resumable, but the reader must be + reconnected for the job to continue. - If SAF writes are slower than USB reads on some devices, the transfer is limited by storage; buffers may need tuning. Internal storage is normally much faster than USB 2.0. +- A provider that does not support append or rename degrades: no resume + (append is still supported per SAF, but a provider that ignores it can + corrupt), and the `.part` name stays. `"wa"` is part of the SAF contract, so + this is a theoretical concern.