v4 Cutover & Promotion Runbook¶

How a strategy goes live and gets promoted on the v4 deployment plane: immutable Release rows + a single ACTIVE pointer in the deployments table are the sole truth of what trades. Registration composes a Release; promotion is an epoch-guarded pointer swap; the runtime resolves config from the ACTIVE Release (fail-closed). MLflow stage aliases (champion/staging/Production) and strategy-as-model are retired — they no longer drive what trades.

Mental model: a Release is an immutable, fully-pinned composition (image digest + content-addressed resolved config + per-slot model bindings + a gate snapshot). The ACTIVE pointer (ENV#ACTIVE, with an epoch) names the one Release that trades for a (strategy, environment). Nothing trades except the ACTIVE Release.

Control-plane CLI¶

All commands live under tradai strategy and take --env/-e (default dev) and --json.

Command	What it does
`tradai strategy assert-cutover-ready <strategies...> -e <env>`	Verify every `(strategy, env)` ACTIVE pointer names a cutover-ready Release. Non-zero exit if any ML Release is legacy/empty-slot/dangling. A non-ML (pure-indicator) strategy is cutover-ready with an empty-slot Release (`is_ml=False`), so it passes — running this over all strategies (ML + non-ML) is correct. The gate before the runtime flip. Rollout: the ACTIVE pointer is first-write-wins, so re-registering a corrected Release does not move it — `repoint-active` the affected `(strategy, env)`.
`tradai strategy repoint-active <name> <ENV#REL#...> -e <env>`	Audited manual repoint of ACTIVE onto a cutover-ready Release (admin op). Refuses missing/legacy releases.
`tradai strategy promote <name> <ENV#REL#...> -e <env>`	Run the promotion gate (floors + regression vs current ACTIVE + cooldown); on pass, epoch-guarded swap with an atomic audit. Exit 2 + rejection audit on gate failure.
`tradai strategy rollback <name> <ENV#REL#...> -e <env>`	Repoint ACTIVE to a prior Release, bypassing gate + cooldown (incident response). Still epoch-guarded + audited.
`tradai strategy promote-cross-env <name> <measurement.json> --source-env <e> --target-env <e> [--expedited]`	Copy a source-env Release into the target env (re-measured), then gate + swap. `--expedited` carries the source snapshot unvalidated (incident).

`promote-cross-env` measurement JSON¶

You produce measurement.json from the target-env OOS backtest of the source artifact, then hand it to the command. Provenance refuses the run unless it pins the same artifact as the source: same image_digest and, per slot, the same (model_name, model_version). Both slot_versions and slot_models are required and must describe the same slot set — a wrong model at a matching version integer is rejected.

{
  "backtest_run_id": "<target-env backtest run id>",
  "image_digest": "sha256:<same digest as the source Release>",
  "environment": "PROD",
  "is_out_of_sample": true,
  "slot_versions": {"return_pred": 7},
  "slot_models": {"return_pred": "BtcReturnLgbm"},
  "metrics": {"sharpe": 1.4, "max_drawdown_pct": 12.0, "profit_factor": 1.6, "total_trades": 80, "is_out_of_sample": true},
  "resolved_config_uri": "s3://<bucket>/<env>/resolved_config/<hash>.json",
  "resolved_config_hash": "<sha256 of the target-env resolved config>",
  "window": ["2026-04-01", "2026-06-01"]
}

(is_out_of_sample is taken from the top-level field — provenance enforces it — so it is gated even if omitted from metrics. For --expedited, supply the source snapshot's metrics and pass --approved-by + --reason.)

Environment flags (gates)¶

These are the v4 activation switches. Defaults are chosen so merging is safe; an operator flips them during cutover. (Full reference: config-and-versioning.md.)

Flag	Default	Effect
`LOCAL_DEV_OVERRIDE`	unset	When `true`, the runtime uses the legacy S3-by-name config load (dev only). When unset/false, the runtime resolves config from the ACTIVE Release (authoritative, fail-closed). The cutover flip = deploy without it.
`RELEASE_STRICT_GATE`	`true`	Registration fail-closes Release composition: requires an `mlflow_run_id`, a contract-bearing model version, and a resolvable immutable image digest. Scope: registration only — not the runtime resolver or the promotion gate.
`REQUIRE_IMAGE_DIGEST`	`false`	When `true`, the runtime refuses to trade if `RUNNING_IMAGE_DIGEST` is absent. A mismatch always fails closed regardless. Flip on only after every launcher injects the digest.
`RELEASE_REQUIRE_LINEAGE`	`false`	When `true`, registration rejects a Release whose contract-bearing slots lack training-data `data_hash`. Independent of `RELEASE_STRICT_GATE`. Flip on only after producers emit lineage.
`TRADAI_GATE_*`	signed-off launch values	Promotion-gate thresholds (`MIN_SHARPE`, `MAX_DD`, `MIN_TRADES`, `REQUIRE_OOS`, `PF_EPSILON`, `MAX_DD_DELTA`, `COOLDOWN_HOURS`). See `config-and-versioning.md` for the `REQUIRE_OOS` footgun.

Transition stages (code-complete → fully live)¶

Advance one (strategy, env) at a time; every flip is reversible.

Ship + deploy infra. Merge the v4 stack; just infra-up-persistent <env> then -compute to deploy the deployments table + item_type-strategy_name-index GSI, the live-predictions table, the resolved-config/contracts S3 buckets, and the catalog-rebuild Lambda + schedule.
Produce real Releases (cross-repo). In ../tradai-strategies/, just release <strategy> <ver> registers a contract-bearing Release and first-write-sets the ACTIVE pointer. Legacy-backfilled rows are never cutover-ready, so this is the only path.
Pre-flip shadow validation. On a build where the legacy branch is still active, run the strategies and confirm release_parity status=match … image_match=true in the logs — this log only emits in the legacy branch, so observe it before the flip. Then tradai strategy assert-cutover-ready <all strategies> -e <env> → ready for every (strategy, env); repoint stragglers with repoint-active.
Flip the runtime. Deploy trading containers without LOCAL_DEV_OVERRIDE. The runtime now resolves the ACTIVE Release, verifies the resolved-config hash, asserts the image digest, and runs the per-slot two-assert. Add post-flip resolver smoke tests (parity logs are gone). Rollback: redeploy with LOCAL_DEV_OVERRIDE=true.
Flip the enforcements. REQUIRE_IMAGE_DIGEST=true after all launchers inject the digest; RELEASE_REQUIRE_LINEAGE=true after producers emit lineage. Both default off, so reverting is a flag change.
Promote in anger. Register a challenger, then tradai strategy promote … — verify the gate decision, epoch increment, ACTIVE move, and audit row. Rollback: tradai strategy rollback ….

Verifying state¶

# Is the ACTIVE pointer on a cutover-ready Release?
tradai strategy assert-cutover-ready PascalStrategy --env dev --json

# Promote a challenger (gate + epoch-guarded swap)
tradai strategy promote PascalStrategy "DEV#REL#01ARZ..." --env dev

# Incident rollback (no gate, no cooldown)
tradai strategy rollback PascalStrategy "DEV#REL#01PRIOR..." --env dev

Every promotion/rollback/repoint writes an immutable PromotionAuditRecord (ENV#PROMO#<challenger>#<ulid>) with the verdict, metrics, thresholds, and violations.

Granting platform-admin for PROD promote/rollback¶

Promoting or rolling back the PROD environment via the REST API (POST /api/v1/strategies/{name}/promote|rollback) is gated: the caller's JWT must carry a Cognito group in PLATFORM_ADMIN_GROUPS — platform-admin or admin (tradai.backend.api.dependencies). dev/staging are unrestricted for any authenticated caller.

The platform-admin and admin user-pool groups are provisioned by the persistent stack (infra/persistent/modules/cognito.py) but ship with no members by design — no privileged principal is seeded from IaC. Until you add one, every PROD promote/rollback returns 403. Granting membership is a deliberate, audited manual step:

export AWS_PROFILE=tradai AWS_REGION=eu-central-1

# 1. Find the user pool id (persistent stack output; or list pools):
aws cognito-idp list-user-pools --max-results 20 \
  --query "UserPools[?starts_with(Name, 'tradai')].{Name:Name,Id:Id}" --output table
POOL_ID=<pool-id-from-above>

# 2. Resolve the operator's username / sub (the JWT 'sub' claim):
aws cognito-idp list-users --user-pool-id "$POOL_ID" \
  --query "Users[].{User:Username,Email:Attributes[?Name=='email']|[0].Value}" --output table
USERNAME=<username-from-above>

# 3. Grant membership (repeat with --group-name admin only if you deliberately want the
#    legacy group; platform-admin is the canonical one):
aws cognito-idp admin-add-user-to-group \
  --user-pool-id "$POOL_ID" --group-name platform-admin --username "$USERNAME"

# 4. Verify:
aws cognito-idp admin-list-groups-for-user \
  --user-pool-id "$POOL_ID" --username "$USERNAME" \
  --query "Groups[].GroupName" --output text

The new group only takes effect on the operator's next token issuance — they must re-authenticate (a cached id/access token won't carry the claim). To revoke, swap admin-add-user-to-group for admin-remove-user-from-group.