Skip to content
← 05 Building a Workflow Engine

Persistence, history and the retention squeeze

Four schema decisions to copy, and one knob you cannot avoid turning.

Lesson 04 of 06 · 8 min
  1. Split the metadata row from the payload blob. Every list view, status poll and pruning query touches the narrow table. Put the blob in the same row and you will never index your way out.
  2. Snapshot the workflow definition into the execution. This is how you render a two-month-old failed run whose graph has since been rewritten. Without it, execution history is a lie.
  3. Instrument payload size as columns. You cannot manage retention pressure you cannot query.
  4. Make the storage location a column. Offloading old payloads to object storage should be a per-row fact, not a global mode.
Predict

Your execution-data table hits 30 GB. The recommended fix is to stop saving execution data on success. What have you just deleted?

The rest of this lesson unlocks when you commit above.