Skip to content

EFS Trader Persistence (Infrastructure)

Operator reference for the EFS filesystem that backs the durable Freqtrade trade ledger (#97 M1). This is the infrastructure view (filesystem, access points, network path, single-writer deploy policy); the application-side TRADAI_PERSIST_DIR contract is delivered by the db_url-pinning work.

What gets provisioned

  • EFS filesystem (foundation stack) — encrypted, durable, with a daily AWS Backup of the (KB-scale) ledger. The Pulumi object is protected in prod.
  • Per-strategy access points (compute stack) — one per strategy slug, root /strategies/{slug}, POSIX user 1000:1000. Each per-strategy task def mounts its own access point at /data/persist, so strategies are isolated by AP root.
  • TRADAI_PERSIST_DIR=/data/persist is set on each per-strategy task def; the generic backtest task def is unchanged (no volume).

Network path

EFS uses NFS (port 2049). Access is gated in two layers:

  • Security group (hard gate): the EFS SG only allows NFS inbound from the ECS task SG.
  • NACL (defense-in-depth): the private NACL has NFS rules (2 ingress + 2 egress on 2049) for the EFS mount-target subnets.

The filesystem policy enforces TLS-in-transit and access-point-only mounts, but it grants ClientMount/ClientWrite to any AP on the filesystem — true per-strategy IAM isolation is tracked in #534. Today isolation rests on the SG gate + per-task-def AP pinning.

Single-writer deploy policy

SQLite on EFS tolerates one writer. Strategy services therefore deploy with maximumPercent=100 / minimumHealthyPercent=0 (stop the old task before the new one starts) instead of the AWS default 200/100 that would briefly run two writers. The runtime helper (ECSDeployHelper) also hard-caps desiredCount at 1 and uses the same deploy config, so neither a redeploy nor a manual scale can start a second writer.

Decommission / cleanup

  • The legacy tradai/live-trading & tradai/dry-run-trading ECR repos (removed from Pulumi in #529) have a one-time cleanup path in infra/persistent/delete-existing.sh.
  • The EFS filesystem is protected in prod — removing it requires the documented protected-resource removal path, not a plain pulumi destroy.

Troubleshooting

Symptom Cause Action
Task fails to mount /data/persist SG/NACL NFS path or AP misconfig Check the EFS SG allows 2049 from the task SG; verify the AP exists for the slug
pulumi up plans to delete the EFS in prod and fails protect=True on the filesystem Use the protected-resource removal path; never force-delete a ledger
Two tasks briefly running for one strategy during deploy Deploy policy not min-0 (older revision) Confirm max 100 / min 0 on the service; redeploy