Storage Split
MySQL holds what a screen filters, sorts or joins. Blob holds what is replayed or read whole by key. Four data types must never become tables, and that one rule is what keeps 32 GB sufficient for decades.
The rule, and the prices behind it
MySQL storage on Azure Flexible Server costs about $0.115 per GB per month and is grow-only: provisioned storage can be scaled up but never back down, so a table that bloats once raises the bill floor permanently. Blob costs about $0.023 per GB hot, $0.01 cool and $0.002 archive, shrinks freely, and takes lifecycle policies. MySQL is therefore five to ten times blob hot and about fifty times blob archive per GB, with mistakes that do not undo.
The rubric applied to every data type: if the admin UI or the daemon filters it, sorts it or joins it, it is a row. If the access pattern is "give me the whole thing for id X" or "stream the range", it is a blob. Anything projected at more than a million rows a year, or more than 1 KB per row of payload nobody queries, defaults to blob with a thin index row pointing at it. Journal rows carry a summary inline and a BlobRef for the rest.
One hygiene rule with real gigabytes behind it: EF Core's default decimal(65,30) costs about 30 bytes per value. Money and quantity columns are decimal(18,6), nine bytes, throughout.
The four things that must never be tables
The budget is safe by an order of magnitude only because four specific data types stay out of MySQL. Each is a decision, with the cost of deciding the other way.
| Data | In blob | If put in MySQL instead |
|---|---|---|
| Intraday 1-minute bars | ~1 GB/yr gz | ~6.5 GB/yr, plus a five-year backfill of ~33 GB. The budget is gone in year one or two on this alone |
| Backtest daily state | ~4 GB/yr gz | A monthly tournament is 40 accounts × 6,300 days, about 3M rows a month, ~7 GB/yr; dead by year four, for write-once data no query joins |
| Claude prompt and response payloads | ~1 GB/yr | +1 GB/yr of LONGTEXT nobody filters; the parsed answer stays in the row |
| Full reason chains and raw news bodies | ~300 MB/yr gz | +1.2 GB/yr of uncompressed inline JSON |
Sum of doing all four wrong: about 15 GB a year, overrun during year two. With the split: about 350 MB a year. Where MySQL is deliberately spent, the spend is reversible: DailyBars costs about 0.7 GB and buys indexed access for the daemon's derived signals, but the blob copy is canonical, so the table is evictable and rebuildable at zero data loss.
The number to design against is 32 GB, not the 35 that gets quoted in conversation; Pulumi provisions 32 GiB and the extra three is not there. At the planned scale of about forty accounts the database reaches 80 percent of that around year 25. At the full catalog, one isolation account per strategy, around year nine. What actually binds first is not capacity: IOPS scale with provisioned storage and sit in the low hundreds at 32 GiB, which is why the engine batches its writes, and a server at 100 percent goes read-only, which under the resumability rule means the daemon cannot safely trade. The mitigation is alerts at 60 and 80 percent plus autogrow, treated as a smoke alarm rather than a plan. The standing rule: never solve a storage problem by journaling less. The journal is the product.
The path is the index
Blob has no index. Finding an object means either a MySQL column holds the exact path or the path is derivable from values you already have; if neither, the dataset is unfindable and that is a design bug. So the path carries the query. One container, plutus-artifacts. Domain first, then partition, then leaf. Date partitions are yyyy-MM-dd or yyyy-MM, sortable as strings so prefix listing works. Symbol partitions use the symbol, not the key, because a path should be readable by a human debugging at 07:15 without a database. Extensions state the encoding. Nothing mutable is ever in a path: no latest, no current.
Why one file per symbol for daily bars but one file per month for intraday: access pattern. A backtest reads one symbol's entire daily history at once, so one file per symbol is one request. Intraday is read as date ranges, so monthly partitions let you fetch three months without touching twenty years.
The news archive is one file per feed per day rather than one per day, because the two feeds have different fetch mechanics and each feed-day must be writable once, at its own boundary, and never rewritten. The measured result of the backfill, against an estimate of about 150 MB a year:
Blob first, then the row
Blob and MySQL are not in the same transaction, and never will be. So the order is fixed: write the blob, then commit the MySQL row carrying its BlobRef. Never the reverse. Crash between the two and you get an orphan blob, which is harmless, costs a fraction of a cent and is cleaned up by a sweep. Reverse the order and a crash gives you a dangling reference: a row promising a document that does not exist, which is corruption that surfaces months later in a drill-down. Orphans are litter; dangling refs are lies. The rule is repeated in the design standards because it is a correctness issue, not a blob issue.
Update: don't. Blobs in Plutus are immutable, so there is no update pattern. Bars re-adjusted after a split are a new full file and the old version is retained by versioning. A backtest re-run is a new runId and a new prefix; runs are never overwritten, because the trial registry depends on it. A corrected news body is a new object and the row's RawBlobRef moves. Versioning is on so an accidental overwrite is recoverable, and soft delete at seven days so an accidental delete is too.
- 32 GiB provisioned, not 35
- ~350 MB/yr of growth with the split
- ~15 GB/yr without it
- $0.115 per GB-month in MySQL, one-way
- $0.023 per GB-month in blob, hot
- +$3.70/mo to grow 32 → 64 GiB, forever
Compute scales both ways. Storage only goes up. Pull the levers in that order.