05
Building a Workflow Engine
What decisions does building one actually force?
The architecture, and two opposite answers to one question: who assembles the workflow, and what is the durable artifact.
6 lessons · 7 checks · 53 min
Why it sits here
Fifth because it is where the previous four collide in one product. Module 02 told you what an engine does; this is what it costs to build, read from the source of a real one rather than its marketing. It also contains the sharpest live question in the space, which is what verifies a generated artifact.
Lessons
- Four communities coined this separatelyPick which altitude you mean before writing a line, because the choice decides your unit of retry.6m
- The execution model, and the scheduler lessonOne decision here is worth more than the rest of the module combined.9m
- The data model between stepsThe decision that shapes everything downstream, and the one place a major engine is quietly leaky.11m
- Persistence, history and the retention squeezeFour schema decisions to copy, and one knob you cannot avoid turning.8m
- Is a workflow code or data?The deepest question in the module, and it has an answer.8m
- Who assembles the workflow?Two products answer one question opposite ways. The third position already won, which changes what the live question is.11m
Pre-reading
10 sources · ~5h if you read all of itRanked, not exhaustive. The lessons stand on their own, so treat the first tier as the genuinely load-bearing sources and the rest as depth when a lesson makes you want it.
Read properly
- workflow-execute.tsn8n sourceA graph interpreter whose entire state is a serialisable value. Where partial re-execution, retry semantics and branch termination become concrete. Do not read linearly.90 min
- partial-execution-utils/n8n sourceThe eight-step re-run-from-the-failed-step algorithm decomposed into named files, including the dirtiness TODOs.40 min
- execution-entity.ts and execution-data.tsn8n sourceThe persistence schema after seven years of production pressure. Ten minutes here saves a migration later.10 min
- Durable execution and sandboxingActivepieces docsThe run-log checkpoint model, simple enough to implement in a week. MIT, so you can read the implementation too.25 min
- Graph and code views, and How Cody worksCodeWords docsThe clearest statement of the generate-then-deploy position, including the decisive "Can I modify the code? Not directly."20 min
- Rearchitecting the Workflows control plane for the agentic eraCloudflareA single account-level coordinator collapsing under agent-created load. The failure mode that arrives when workflows are created at machine speed.20 min
Skim one section
- The item-linking documentation clustern8n docsA vendor documenting the failure modes of its own most opinionated decision. Read as a specification of what you sign up for with an array-of-items model.20 min
- ai-workflow-builder tools/n8n sourceThe third position shipped. The tool surface partitions into graph mutations, grounding and static validation. Superseded by their AI Assistant, but the argument holds.30 min
- interfaces.ts, four definitionsn8n sourceFive execution entry points, design-time methods, the item envelope, the typed AI ports, the versioning class. Jump to the four; the file is 135KB.25 min