Designing Reliable Offline Attachment Queues for Photos, PDFs, and Field Documents

When a Field Attachment Is Captured but Not Sent

Inside a mechanical plant room, a technician completes an inspection, photographs the equipment, attaches a signed PDF, and taps submit. Coverage disappears before the first file reaches the server. The screen has already returned to the work list.

That sequence contains three separate commitments: the inspection row is saved locally, each attachment is secured in managed device storage, and the server acknowledges durable receipt. The interface may finish submission after the first two commitments, but it must continue marking the record as pending evidence until the third.

Small form rows often synchronize as short, atomic requests. JPEGs and PDFs behave differently. Their transfers last longer, consume more storage and memory, and remain exposed to process termination, power loss, credential expiry, and network changes. Reusing record-synchronization assumptions for these payloads creates a dangerous gap between successful capture and confirmed delivery.

A Repeatable Interruption Fixture

I use a plant-room fixture with three JPEG files ranging from 2 MiB to 5 MiB and one 620 KiB signed PDF. Connectivity is interrupted after the form row commits and before the first attachment acknowledgement. Shutdowns occur from 1 to 15 seconds after submit, including termination while the manifest is being written and while the first payload is in flight.

The expected result is straightforward: after restart, every secured file remains discoverable, and no unacknowledged attachment appears as delivered.

Model Each Attachment as a Durable Queue Job

Recovery boundaries should determine the data model. Keep inspection status and attachment references in the business record. Put file transfer state in a separate durable manifest that references an immutable managed file.

A useful manifest contains the client-generated attachment ID, stable local parent ID, nullable server parent ID, managed-file path, original filename, media type, byte length, capture timestamp, and optional hash algorithm and value. Transport fields include state, attempt count, next-attempt time, upload-session ID, acknowledged offset, and last error code.

Journal the State Transitions

Record transitions from captured to ready, ready to uploading, uploading to uploaded, and uploaded to reconciled. Commit a transition before transfer starts and advance it only after parsing a server response associated with the same attachment ID. Store recent error text separately so a temporary network message cannot replace durable state.

On startup, return an interrupted uploading item to a recoverable condition. A practical stale-checkpoint interval falls between 2 and 10 minutes, selected according to expected payload size and link speed. Business completion remains independent: a finished inspection can still own photos awaiting transport or parent reconciliation.

Capture, Bind, and Compress Files Before Enqueueing

A crash-safe intake sequence begins with a uniquely named temporary file. Close it, reopen it for validation, record its byte length, and move it on the same storage volume into managed attachment storage. Commit the queue manifest only after that move succeeds.

1. Assign the attachment ID and create a temporary name ending in .part.
2. Write and close the payload before validating its readable structure.
3. Move the validated file into managed storage under the attachment ID.
4. Bind the manifest to the parent’s stable local ID and commit it.
5. Reconcile the nullable server parent ID later without renaming the payload.

This ordering prevents a committed manifest from pointing at a half-written camera file. A startup scan can then compare temporary files, managed payloads, and manifests. Classify a temporary artifact as abandoned only when no intake operation references it and its modification time is at least 30 to 120 minutes old. The grace period protects recoverable work when the device clock has shifted.

Transform Without Damaging the Source

Process images as streams or decoder-sized blocks, with a configured working-memory ceiling such as 4 to 8 MiB on constrained hardware. Write the derivative to a new file rather than recompressing the managed original in place.

Record the processing policy version, output dimensions, orientation handling, and source hash. A matching marker allows the queue to reuse an existing derivative instead of applying JPEG compression again. For PDFs, validate the header, final trailer, and readable page structure before choosing a transformed copy; otherwise enqueue the original bytes.

Resume Partial Uploads Without Creating Duplicates

Upload recovery begins with stable identity. Send the attachment ID as an idempotency key when creating the server upload, then persist the returned session ID. Repeated creation requests should resolve to that same attachment or session.

Older mobile hardware can use fixed chunks in the 128 to 512 KiB range. Persist only the cumulative offset confirmed by the server, within 1 to 5 seconds of acknowledgement and before sending the next chunk. If a 256 KiB chunk is transmitted twice after a timeout, the server compares the session and expected offset instead of appending duplicate bytes.

Keep Completion in Three Steps

• Transfer: accept bytes into the upload session.
• Verification: compare server-reported length and, when available, a SHA-256 digest.
• Registration: bind the verified attachment identity to the parent record.

The local payload remains protected until registration returns both the attachment ID and parent binding. This separation prevents an incomplete binary from appearing as a finished field document.

Timeouts, resets, and temporary server unavailability can follow a bounded retry schedule beginning at 15 to 45 seconds, expanding through 1-to-3-minute and 5-to-15-minute windows with jitter, and capped at 30 to 60 minutes. Media-type rejection, authorization requiring fresh credentials, parent-policy conflicts, and declared size violations should move directly to user attention.

Whole-file endpoints provide no byte-level continuation. Idempotency still prevents duplicate registration, though each retry must restart from byte zero.

Protect Device Storage While Records Reconcile

Evaluate storage before capture, before generating a derivative, and after reconciliation. Cleanup may remove abandoned temporary artifacts and regenerable previews. It must preserve pending originals, paused jobs, and uploaded files still awaiting parent reconciliation.

For a 256 MiB attachment allocation, a defensible reference policy warns below 64 MiB free and blocks new capture below a 32 MiB reserve. Those thresholds require calibration against the deployment’s camera files and transform workload. A 7 MiB source can briefly need more than 14 MiB while the source, derivative, and queue metadata coexist.

Resolve Parent Changes in Metadata

When the parent receives a server ID, update the manifest mapping while retaining the attachment ID and managed path. Superseded photos should remain governed by explicit retention state. If a parent is deleted locally with pending evidence, move its jobs into review for 24 to 72 hours. A permanent server rejection still requires a recorded retention or export decision before deleting the files.

Check Wi-Fi and battery policy between chunks. A transfer may pause when power or connectivity changes, provided its session identifier and acknowledged offset are already durable.

Make Delayed Attachment Delivery Visible and Testable

Drive the interface from durable queue state. Show captured, waiting, uploading, paused, uploaded-awaiting-reconciliation, delivered, and needs-attention as distinct conditions. At record level, aggregate the number and byte size of unreconciled attachments. At item level, expose local-copy status, retry timing, the last confirmed transfer point, and an actionable failure reason.

Exact byte progress belongs only on protocols that confirm offsets. Whole-file requests should use an indeterminate transfer indicator rather than optimistic percentages.

Release Against Forced Failure

Terminate the process after temporary-file close, managed-file move, manifest commit, each tested chunk boundary, final-byte acceptance, and before parent registration. Repeat with storage below warning and stop thresholds, credentials expiring mid-transfer, the clock shifted by 10 to 30 minutes, and parent synchronization arriving in both orders.

Run two workers concurrently with the same attachment ID and verify that the server returns one attachment identity or one resumable session. The release criterion is recovery from termination or power interruption without losing the only local payload, exposing an incomplete server object, or silently promoting an unconfirmed upload to delivered.

Build the attachment path as a durable evidence chain: secure immutable bytes first, advance only to server-acknowledged boundaries, and retain the local payload until parent-bound registration succeeds.