04
Provenance and Audit Trails
How does a system prove what it did?
Building for the stranger who arrives fourteen months later with adversarial intent.
6 lessons · 9 checks · 54 min
Why it sits here
Fourth because determinism is what makes replay possible and replay is what makes provenance meaningful. You cannot prove what a system did until its decisions are reproducible, and you cannot reproduce them until the model is out of the hot path. This module is where module 03 pays off.
Lessons
- Nine terms that are not synonymsMost arguments about audit systems are vocabulary collisions. Fixing the words fixes half the disagreement.8m
- Event sourcing, and the two things that breakNeither of them is the fold. Determinism is lost somewhere else entirely.10m
- Accounting got here firstThe framing is claimed, canonically, since 2015. What survives is the part nobody implements.10m
- BitemporalityThe question audits actually ask, and the reason one timestamp column cannot answer it.9m
- Lineage has a ceiling, and hash chains have limitsTwo things commonly sold as answers that answer narrower questions than advertised.9m
- The anchor caseEvery failure in this module, in one system, with people imprisoned on the output.8m
Pre-reading
13 sources · ~8h if you read all of itRanked, not exhaustive. The lessons stand on their own, so treat the first tier as the genuinely load-bearing sources and the rest as depth when a lesson makes you want it.
Read properly
- Immutability Changes EverythingPat Helland, CIDR 2015The canonical source for the accounting framing, and for "the truth is the log, the database is a cache". Six pages. Read it before claiming any of this.40 min
- Bitemporal HistoryMartin FowlerThe clearest treatment of actual time versus record time, with a salary-correction example that maps onto revenue restatement without translation.45 min
- Temporal features in SQL:2011Kulkarni & Michels, SIGMOD Record 41(3)Nine pages that turn bitemporality from a concept into a DDL statement, including the query that is literally "what we believed then about the world earlier".60 min
- RFC 6962: Certificate TransparencyLaurie, Langley, Kasper, IETFSections 2 and 5 only. The Merkle mechanics, and the only honest account anywhere of who tamper-evidence protects you from.30 min
- Accounting PatternsMartin Fowler, eaaDevAccount, Entry, Transaction, and the three adjustment strategies. Gives you shared vocabulary with the finance side of a correction argument.30 min
- Bates & Ors v Post Office Ltd (No 6: Horizon Issues)[2019] EWHC 3408 (QB), Fraser JThe Audit Store and transaction-correction-tool passages. Not a technical text, and the most persuasive document in this module.20 min
Skim one section
- An Empirical Characterization of Event Sourced SystemsOvereem, Spoor, Jansen, Brinkkemper · arXiv:2104.01146Nineteen real systems, twenty-five engineers. The only evidence-based item here; everything else is expert opinion.60 min
- Resources, Events, Agents (REA)McCarthy, The Accounting Review, 1982The accounting profession’s own events-first model, and the bridge between your domain and the provenance standard. Start with the overview; the paper is paywalled.20 min
- PROV PrimerW3C Working Group NoteEntity, Activity, Agent plus qualified relations. Steal Role, Plan and actedOnBehalfOf; skip the RDF serialisation.40 min
- Versioning in an Event Sourced SystemGreg YoungRead the versioning chapters. The rule that you may add to an event but never change its meaning is the load-bearing one.90 min
- Event SourcingMartin Fowler, eaaDevSlow down on external systems and external queries. The exchange-rate example is the whole determinism problem in two sentences.30 min