TradAI Dev Environment — Full Component Map¶

Generated: 2026-04-27 Purpose: Complete audit of all AWS components in dev environment

Stack Architecture (deploy order)¶

PERSISTENT → FOUNDATION → COMPUTE → EDGE

Region: eu-central-1, AWS Profile: tradai

1. PERSISTENT Stack (permanent data resources)¶

S3 Buckets (5)¶

DynamoDB Tables (12)¶

ECR Repositories (24)¶

Services (6): tradai/backend, tradai/data-collection, tradai/strategy-service, tradai/mlflow, tradai/live-trading, tradai/dry-run-trading

Lambdas (18): tradai/lambda-base, lambda-backtest-consumer, lambda-sqs-consumer, lambda-orphan-scanner, lambda-health-check, lambda-trading-heartbeat-check, lambda-drift-monitor, lambda-retraining-scheduler, lambda-validate-strategy, lambda-data-collection-proxy, lambda-notify-completion, lambda-check-retraining-needed, lambda-compare-models, lambda-promote-model, lambda-model-rollback, lambda-cleanup-resources, lambda-update-status, lambda-pulumi-drift-detector

Cognito¶

• User Pool: tradai-users-dev
• MFA: Required (TOTP)
• Password: Min 12 chars, upper/lower/numbers/symbols
• Clients: Public (PKCE) + M2M (Client Credentials)

CodeArtifact¶

• Domain: tradai-dev
• Repository: tradai-dev

CloudTrail¶

• Audit log to tradai-logs-dev bucket

2. FOUNDATION Stack (networking & data infrastructure)¶

VPC & Networking¶

Security Groups¶

ALB, ECS, Lambda, RDS, NAT, VPC Link, VPC Endpoints, EC2 Consolidated

RDS PostgreSQL¶

SQS¶

SNS Topics¶

3. COMPUTE Stack (services & compute)¶

ECS Cluster¶

• Name: tradai-dev
• Container Insights: Enabled
• Log Group: /aws/ecs/tradai-dev (30d retention)

ECS Services¶

Dev mode: Consolidated EC2 t3.small runs backend-api, data-collection, mlflow, strategy-service (~$15/mo vs ~$37/mo Fargate)

Service Discovery¶

Namespace: tradai-dev.local - backend-api.tradai-dev.local:8000 - data-collection.tradai-dev.local:8002 - strategy-service.tradai-dev.local:8003 - mlflow.tradai-dev.local:5000

ALB (Application Load Balancer)¶

• Type: Internet-facing (public subnets)
• Routing:
• /mlflow/* → mlflow:5000
• /api/v1/live/* → live-trading:8004
• /api/v1/dry-run/* → dry-run-trading:8005
• /api/v1/* → backend-api:8000 (catch-all)

Lambda Functions¶

Required (always running)¶

Optional (Step Functions)¶

Step Functions Workflows¶

Backtest Workflow (tradai-backtest-workflow-dev, 2h timeout)¶

ValidateStrategy → EnsureData → UpdateStatus(RUNNING) → RunBacktest(ECS)
                                                              ↓ (error)
                                                        CleanupResources → NotifyFailure
    → HandleSuccess → NotifyCompletion

Retraining Workflow (tradai-retraining-workflow-dev, 3h timeout)¶

CheckRetrainingNeeded → RunRetraining(ECS) → CompareModels
    → PromoteModel → NotifyCompletion
    → ModelRollback → NotifyFailure

IAM Roles¶

• ECS Execution Role: ECR pull, CloudWatch Logs, Secrets Manager
• ECS Task Role: DynamoDB, S3, Secrets Manager, CloudWatch, SNS, CodeArtifact, ECS control
• Lambda Role: ECS control, DynamoDB, SNS, SQS, Secrets Manager, S3, Step Functions
• Consolidated EC2 Role: ECR, CloudWatch, DynamoDB, S3, Secrets Manager (dev/staging only)

4. EDGE Stack (API Gateway, monitoring)¶

API Gateway (HTTP API)¶

• VPC Link to ALB
• JWT auth (Cognito)
• CORS: * (dev)
• Throttling: 100 req/s default, 10 req/s on POST /backtests

Routes¶

WAF¶

• WebACL created but NOT associated (bug: $default stage ARN parsing)

CloudWatch¶

• Alarms: RDS CPU/disk, API 4xx/5xx, Lambda errors, ECS tasks, Step Functions
• Composite alarm → SNS alerts
• Dashboard with all metrics

End-to-End Data Flow¶

User → API Gateway (JWT) → POST /api/v1/data/sync
    → backend:8000 → data-collection:8002 → CCXT(Binance) → ArcticDB(S3)

User → API Gateway → POST /api/v1/backtests → SQS FIFO
    → Lambda(backtest-consumer) → Step Functions
    → ValidateStrategy → EnsureData → RunBacktest(ECS strategy-container)
    → Freqtrade backtest → S3 results + MLflow experiments
    → DynamoDB workflow_state (PENDING→RUNNING→COMPLETED)

User → API Gateway → GET /api/v1/backtests/{job_id} → status + results

Monitoring:
    Lambda health-check (2min) → ping services → CloudWatch + DynamoDB
    Lambda orphan-scanner (5min) → find orphaned ECS tasks
    Lambda drift-monitor (12h) → PSI analysis → SNS alerts
    Lambda retraining-scheduler (6h) → evaluate + launch retraining

Expected Artifacts¶

Service Endpoints (Backend API)¶

Backtest Management¶

• POST /api/v1/backtests — submit backtest job
• GET /api/v1/backtests — list backtests (paginated)
• GET /api/v1/backtests/{job_id} — get status
• POST /api/v1/backtests/{job_id}/cancel — cancel
• GET /api/v1/backtests/{job_id}/equity — equity curve
• GET /api/v1/backtests/{job_id}/report-data — full report

Data Operations¶

• POST /api/v1/sync — sync OHLCV data
• POST /api/v1/sync/incremental — incremental sync
• GET /api/v1/freshness — data freshness
• GET /api/v1/symbols — available symbols
• GET /api/v1/ohlcv — export OHLCV
• POST /api/v1/export — export with POST

Strategy Catalog¶

• GET /api/v1/catalog/strategies — list strategies
• GET /api/v1/catalog/strategies/{name} — details
• GET /api/v1/catalog/leaderboard — leaderboard

Model Management¶

• POST /api/v1/promote-model — promote
• POST /api/v1/demote-model — demote
• GET /api/v1/model-promotion-history — history
• POST /api/v1/compare-models — compare

Strategy Service Endpoints (internal, :8003)¶

• GET/POST /api/v1/configs — strategy configs
• POST /api/v1/strategies/register — register strategy
• POST /api/v1/hyperopt/run — hyperopt
• POST /api/v1/challenges/create — A/B testing

Data Collection Endpoints (internal, :8002)¶

• POST /api/v1/sync — sync OHLCV
• POST /api/v1/sync/incremental — incremental sync
• GET /api/v1/freshness — data freshness
• GET /api/v1/symbols — available symbols
• GET /api/v1/ohlcv — export data