The most trustworthy audit trail in a field application is written on the disconnected device, minutes or days before any server sees it. That claim inverts how most enterprise logging is designed, and it holds up under the only test that matters: reconstructing what a technician actually did during a long offline interval.
Authorship Belongs to the Disconnected Installation
A server-side log records the moment a payload arrived. It cannot testify to the inspection completed in a basement plant room at 06:12, the correction entered forty minutes later, or the second attempt after the tablet rebooted. Only the installation running on that device was present.
So the architecture assigns authorship to the device and gives the server four different jobs: validating identity context, detecting replay or chain discontinuity, reconciling conflicts, and retaining accepted evidence for its full life. Server receipt time becomes an additional timestamp on the record, never a substitute occurrence time. Server decisions (rejection, quarantine, conflict resolution) are stored separately from the submitted evidence, so no reconciliation outcome can rewrite what the device originally claimed.
Design the threat review around six concrete cases rather than an abstract notion of tampering: manual clock adjustment, duplicate transmission, local row modification, authorization expiry, upload interruption, and concurrent edits from two devices touching the same record. Each one produces a different failure signature, and each one has a specific countermeasure in the sections that follow.
One honest boundary belongs here, in the design phase rather than in a footnote at the end: this approach improves provenance and makes inconsistency visible, but a fully compromised device with access to the application process and its signing material can still manufacture a sequence that looks internally coherent.
The Event Envelope That Survives Every Retry
Start from the business transaction, not the synchronization job. When a user finishes an inspection, the domain update and its audit event are inserted inside one local database transaction. The transaction commits only when both writes succeed. A completed field action that survives without its event is a defect, not an edge case.
A workable .NET envelope carries EventId, AggregateId, ActionType, ActorId, DeviceEnrollmentId, InstallationId, SessionId, SchemaVersion, OccurredAt as DateTimeOffset, UtcOffsetMinutes, LocalSequence as Int64, PreviousEventHash, PayloadHash, SynchronizationState, and an immutable payload.
Generate the EventId on the device as a 128-bit UUID and reuse it unchanged through every retry; the server enforces uniqueness on that value. Allocate LocalSequence inside the same write transaction, scoped to the InstallationId, so sequence 417 issued after a reinstall can never be confused with sequence 417 from the installation it replaced.
Keep synchronization state outside the bytes covered by the event digest. Moving a record from Pending to Acknowledged is bookkeeping about delivery and must leave the historical envelope byte-identical. Use explicit states — Pending, InFlight, Acknowledged, Rejected, Quarantined — because a single uploaded Boolean collapses five different situations into one.
Envelope Sign-Off
- Stable identity: EventId is generated before the first upload attempt and reused unchanged for every retry.
- Separated provenance: actor, enrolled device, installation, and session each occupy their own field.
- Atomic write: the domain change and its event share one local transaction.
- Digest isolation: delivery state lives outside the hashed bytes.
Who Acted, Which Device, and Whose Clock
Identity splits into four references because reviewers ask four different questions. ActorId names the person last verified. DeviceEnrollmentId names the managed device relationship. InstallationId defines the local event and sequence namespace. SessionId points at the cached authentication context that was in force. Reconciliation evaluates each one independently instead of treating a username string as proof of all four.
Persist the authentication issue time, expiry time, the authorization snapshot or role identifiers, and the last successful online validation with the session context. When offline authorization expires, apply an explicit domain result: block creation, permit draft-only work, or accept the event into pending review. Silently extending a credential converts an access-control question into an invisible one.
Chronology needs several independent signals. Store OccurredAt as DateTimeOffset, the original UTC offset in minutes, the LocalSequence, and ServerReceivedAt. The type choice matters more than it looks; Microsoft guidance on choosing .NET date and time types explains why an offset-aware value preserves information a bare DateTime discards.
Flag a clock regression whenever event N reports an earlier wall-clock value than event N-1 in the same installation sequence. Classify a large receipt-time gap separately, because a four-day offline interval in a rural service territory is not evidence of manipulation. Reconciliation labels such as ClockConsistent, ClockRegressed, OffsetChanged, ReceiptSkewLarge, and ClockUnknown annotate the record; the raw timestamps stay exactly as submitted.
Hash Chains That Claim Only What They Prove
Calculate the chain from a canonical event representation, never from the database row or a runtime object serializer. Construct the digest input in a documented field order: SchemaVersion, EventId, AggregateId, ActionType, ActorId, DeviceEnrollmentId, InstallationId, LocalSequence, OccurredAt, UtcOffsetMinutes, PayloadHash, PreviousEventHash.
Encoding rules carry the weight. Declare one integer byte order, write identifiers in their 16-byte binary form, normalize text to UTF-8 with a stated normalization form, and distinguish null from empty using explicit length markers. Hash a canonical payload byte sequence rather than serializer output, whose property ordering, number formatting, or null handling can shift after an application upgrade and silently break every chain built before it.
A SHA-256 digest is 32 bytes. Store it as fixed-length binary locally; hexadecimal and Base64 belong only at interchange boundaries. Where the platform offers protected key storage, authenticate each entry with a device-bound key and record a KeyId in the envelope. Rotation opens a declared chain segment linked to the final digest signed by the outgoing key, and device replacement is handled the same explicit way.
Two Different Guarantees
Encryption at rest protects payload confidentiality if storage is copied. Predecessor chaining and authenticated digests answer a separate question: whether accepted history was altered after a known checkpoint. Shipping one and claiming the other is the most common overstatement in offline audit design.
Uploading History Without Writing It Twice
The synchronization worker reads a bounded page of Pending events in local sequence order, marks that page InFlight in a short transaction, transmits the immutable envelopes, and processes a per-EventId result. Acknowledgements commit in a second transaction. Any event without a durable per-item result returns to Pending after an interruption.
Apply two batch limits at once — for example, no more than 100 envelopes and no more than 512 KiB of encoded request data, so a single oversized payload cannot defeat an event-count cap. Retry network and service failures on a bounded progression such as 5 seconds, 15 seconds, 45 seconds, 2 minutes, 5 minutes, with jitter, resetting after confirmed progress.
Transport success means a complete response came back. Acceptance means the server durably stored the event and its reconciliation result. Persist Acknowledged only for EventIds explicitly confirmed as durable. A partial response can acknowledge events 801 through 820 while leaving 821 through 840 pending, and the client must honor that boundary rather than marking the batch complete.
On a duplicate EventId, the server returns the previously stored status and result identifier. If the duplicate arrives with different immutable bytes, quarantine it as an identity collision or mutation attempt rather than overwriting the first copy. Reconciliation outcomes are stored as Accepted, Rejected, Conflicted, Superseded, or PendingReview; a later compensating event may change business state without erasing either the submitted event or its original result.
Two Projections for the Person Reading the Record
Build the review model as two projections over the same retained records. The evidence projection shows the submitted envelope, digest relationships, server receipt, and reconciliation decisions. The business projection interprets schema versions, corrections, compensating actions, and conflict outcomes into a readable account of where the record stands now.
For every event, expose ActorId, DeviceEnrollmentId, InstallationId, SessionId, device occurrence time, original UTC offset, local sequence, server receipt time, schema version, chain status, clock status, and reconciliation status. Render a correction beside the event it references and preserve both payloads; evidence exports never replace a corrected payload.
Keep a schema-version-specific reader for every event version still inside the retention window. When an old payload can no longer be interpreted, export the original bytes and mark interpretation unavailable rather than coercing it into the current schema. Conflict resolution records should name the reviewer, the action, the decision time, the affected EventIds, and a reason code, and the resolution itself appends as another event.
An export package can bundle a line-delimited canonical event file, payload files, chain-head checkpoints, reconciliation results, schema descriptors, and a manifest carrying a digest for each file. Separate access to raw payloads, signing keys, conflict decisions, and export generation by role, since technical immutability says nothing about appropriate disclosure or key custody. Retention periods and required export fields belong to organizational policy, contract terms, and jurisdiction, and should be confirmed before deployment rather than after the first request arrives.
Failure Cases to Run Before the First Truck Rolls
Organize release testing around durable boundaries instead of happy-path screens. Interrupt execution at three points in every case: immediately before transmission, after server persistence but before acknowledgement, and during the local acknowledgement commit.
Run separate cases for airplane mode, process termination, device reboot, duplicate upload, partial acknowledgement, low-storage write failure, expired authorization, actor change, reinstall, storage restoration, and signing-key rotation. Fill local storage until the audit insert fails, then confirm the associated domain change rolls back with it. Inject an unknown SchemaVersion, a payload digest mismatch, a missing predecessor, and an invalid authenticated digest; each should land in a visible quarantine record while unrelated valid streams keep flowing.
The split-acknowledgement break deserves its own rehearsal. Kill the sync worker after the server commits a batch but before the client stores acknowledgements. On restart, the repeated EventIds must resolve to their already stored outcomes without producing a single additional evidence record. After a reinstall, verify that the new InstallationId begins a distinct sequence and chain while the server retains the prior installation's final accepted chain head.
Ship-Or-Hold Signal
If any failure case produces a completed field action with no corresponding event, or an event that vanishes instead of entering quarantine, the trail is not ready for deployment.
Then run the clock test that summarizes the whole design: move the device wall clock backward by 6 hours, create two events, move it forward by 30 hours, and create a third. Every reported time stays visible, every reconciliation label is added rather than substituted, and the installation-scoped Int64 sequence still records exactly what happened next.
Join the Conversation
Share your thoughts.
Your Comment