JSON Documents

A column kept its JSON for one of two reasons: its shape genuinely depends on a type code, so a dollar-mag7 config is not a spy-hold config and no fixed set of columns fits both; or it is a model's structured answer. Nothing in any of them is queried by the engine. If a field turns out to be filtered on, it is promoted to a column and the document is amended; that is the whole reason the review happened.

Field names are camelCase in JSON and PascalCase in the C# record.

No document carries a version inside it. The version is the sibling SchemaVersion column, so the database can index and migrate by version without parsing JSON. A version field inside the document would be two sources of truth.

Absent is not null. A field a variant does not use is omitted, never null. A reader treats "not present" and "present with a reason" differently. This is the same Maybe<T> discipline as MarketState. Why absence carries a reason.

Money is a decimal string, "1250.00", not a JSON number, because JSON numbers are doubles. Percentages are on the 0 to 100 scale unless the field is named Fraction. Timestamps are ISO-8601 UTC with a Z.

Variants for codes that are seeded but not yet built are marked not yet defined rather than invented. Each gets its subsection before its class is written; that is the order, not the reverse.

Every strategy config shares an envelope: a no-trade band on weight drift, a turnover cap per cycle, and the symbol of the instrument that holds cash. The control strategy adds two fields and nothing else, because a control with a tunable is not a control.

// StrategyConfigs.ConfigJson · ModuleCode = spy-hold
{
  "rebalanceBandFraction": 0.02,
  "maxTurnoverPerCycleFraction": 0.25,
  "cashInstrumentSymbol": "CASH",
  "symbol": "SPY",
  "targetWeight": 1.0
}

The first real strategy reads the broad dollar index over a lookback and tilts toward a domestic basket while the dollar is strong. Everything a backtest needs to reproduce the run is in this document, and its SHA-256 is the ConfigHash the manifest names.

// StrategyConfigs.ConfigJson · ModuleCode = dollar-mag7
{
  "rebalanceBandFraction": 0.02,
  "maxTurnoverPerCycleFraction": 0.25,
  "cashInstrumentSymbol": "CASH",
  "dollarSeriesId": "DTWEXBGS",          // FRED broad dollar, via the GMI ingest
  "lookbackDays": 63,
  "strongDollarThresholdPct": 3.0,       // 63-day change above this is a headwind
  "weakDollarThresholdPct": -3.0,
  "longBasket":  { "AAPL": 0.15, "MSFT": 0.15, "GOOGL": 0.14, "AMZN": 0.14,
                   "META": 0.14, "NVDA": 0.14, "TSLA": 0.14 },
  "hedgeBasket": { "IWM": 0.5, "XLU": 0.5 },
  "tiltFraction": 0.30
}

A stimulus trigger is the values that crossed, not the world. The world is MarketStateBlobRef on the same row. The document exists so the reason chain can say what crossed without opening the market-state blob, and it is small by design; if it grows past a few hundred bytes the values belong in the blob. An S4 regime transition carries both dates, because a backtest acts on detectedUtc and never on effectiveDate.

// Stimuli.TriggerJson · StimulusTypes.Code = S4 (regime transition)
{
  "sourceCode": "gmi.fred-regime",
  "observedUtc": "2026-08-29T11:04:00Z",
  "from": "Expansion",
  "to": "Slowdown",
  "effectiveDate": "2026-08-28",         // when it became true
  "detectedUtc": "2026-08-29T11:04:00Z"  // when anyone could have known
}

A document that lives only in a design file drifts. Three things stop that.

The SchemaVersion column. A reader that sees a version it does not know refuses the row loudly rather than parsing what it half-understands.

One C# record per shape, with JsonUnmappedMemberHandling.Disallow, so an unknown field is a deserialization error, not a silent drop. The strategy config records exist today in Plutus.Strategies; each is a record that is the shape, with defaults that match the document. The other shapes are written down and get their records as the phase that writes them arrives.

Serialization through one place. DocumentSerializer.Read and Write are the only calls that touch a document, and Hash is what produces the config hash the schema stores beside it. Canonical on write, no indentation, camelCase, so the hash is stable across machines and runs.

public static class DocumentSerializer
{
    public static readonly JsonSerializerOptions Options = new(JsonSerializerDefaults.Web)
    {
        WriteIndented = false,
        UnmappedMemberHandling = JsonUnmappedMemberHandling.Disallow,
        NumberHandling = JsonNumberHandling.Strict,
        DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
    };

    public static T Read<T>(string json) => ...   // throws InvalidDataException, named for T
    public static string Write<T>(T document) => JsonSerializer.Serialize(document, Options);
    // SHA-256 of the canonical serialization — the ConfigHash the schema stores beside the document
    public static string Hash<T>(T document) => ...
}

When a field in one of these turns out to be queried, the change is: add the column, backfill from the document, amend the shape, bump the version. Not: add a JSON path index and hope.

The news archiver does the opposite of the DocumentSerializer on purpose. Its FMP models carry a [JsonExtensionData] dictionary, so a field the vendor adds tomorrow is kept in the archive rather than dropped. The archive's job is to record what was sent, byte for byte if possible; the four random days re-fetched during Gate 1 verified exactly that. Rejecting an unknown field there would mean losing data the system did not know it needed yet.

The documents here are the other kind of thing. A strategy config is a promise between the admin UI that writes it, the record that reads it, and the manifest that hashes it. An unknown field in a config is either a typo or a schema change nobody wrote down, and both should fail at the boundary rather than run. An archive keeps unknown fields because it does not interpret them; a contract rejects them because it does.

The third is the real enforcement. A documented schema nobody's code references is documentation; a record type every writer must use is a contract.

• Validate a strategy config at save, at load, or both; which is the hard failure
• Two data integrations, FMP and FRED via GMI, share a kind code but not a shape
• Whether a Claude answer keeps the fields that were promoted to columns, as a self-contained copy