TradAI Deployment Pipeline¶

Version: 1.0.0 | Date: 2026-03-28 | Status: CURRENT Source: justfile, infra/, scripts/, .github/workflows/

1. TL;DR¶

4-stage deployment: Lambda images → Pulumi infrastructure (4 stacks in order) → Service containers → ECS force-deploy. Full bootstrap via just infra-bootstrap. Pulumi CI via infra/pulumi-ci.sh.

2. Deployment Architecture¶

flowchart TD
    CA[CodeArtifact Login] --> BW[Build Library Wheels]
    BW --> BLB[Build Lambda Base Image]
    BLB --> BLA[Build All Lambdas<br/><i>auto-discovers lambdas/*/Dockerfile</i>]
    BLA --> ECR_L[Push Lambda Images to ECR]

    ECR_L --> P1[Pulumi: persistent<br/><i>S3, DynamoDB, ECR, Cognito, CodeArtifact</i>]
    P1 --> P2[Pulumi: foundation<br/><i>VPC, RDS, SQS, SNS, Security Groups</i>]
    P2 --> P3[Pulumi: compute<br/><i>ALB, ECS, Lambda, Step Functions</i>]
    P3 --> P4[Pulumi: edge<br/><i>API Gateway, WAF, CloudWatch</i>]

    P4 --> DBS[Docker Build Services<br/><i>backend, data-collection,<br/>strategy-service, mlflow</i>]
    DBS --> ECR_S[Push Service Images to ECR]
    ECR_S --> ECS[ECS Force Deploy<br/><i>Rolling update</i>]

    style P1 fill:#f9f,stroke:#333
    style P2 fill:#f9f,stroke:#333
    style P3 fill:#f9f,stroke:#333
    style P4 fill:#f9f,stroke:#333
    style ECR_L fill:#bbf,stroke:#333
    style ECR_S fill:#bbf,stroke:#333
    style ECS fill:#bfb,stroke:#333

3. First-Time Bootstrap¶

Step-by-step for a new environment:

What just infra-bootstrap Runs¶

The infra-bootstrap recipe executes the following sequence:

1. just infra-up-persistent dev -- S3 buckets, DynamoDB, ECR repos, Cognito, CodeArtifact
2. just infra-up-foundation dev -- VPC, subnets, NAT, RDS, SQS/SNS, security groups
3. just lambda-bootstrap -- Build wheel + base image + all lambdas + push to ECR
4. just service-push-all -- Build all 4 service Docker images + push to ECR
5. just infra-up-compute dev -- ALB, ECS, Lambda functions, Step Functions
6. just infra-up-edge dev -- API Gateway, WAF, CloudWatch dashboards and alarms

4. Lambda Deployment Pipeline¶

4.1 Build Chain¶

sequenceDiagram
    participant W as Wheel Build
    participant B as Base Image
    participant L as Lambda Images
    participant E as ECR
    participant A as AWS Lambda

    W->>W: uv build libs/tradai-common --out-dir dist
    W->>B: tradai_common*.whl
    B->>B: docker build -f lambdas/base/Dockerfile
    B->>L: tradai/lambda-base:latest
    L->>L: docker build -f lambdas/<name>/Dockerfile<br/>(17 lambdas in parallel)
    L->>E: docker push tradai/lambda-<name>:tag
    E->>A: aws lambda update-function-code --image-uri

The base image (lambdas/base/Dockerfile) uses public.ecr.aws/lambda/python:3.11, installs gcc, g++, make, tar, gzip, and then installs the tradai-common wheel plus boto3, pydantic, pydantic-settings, and httpx. Each individual Lambda extends this base image with its own handler.py.

4.2 Commands¶

4.3 Auto-Discovery¶

Lambda build discovers all directories in lambdas/ automatically. Any directory containing a Dockerfile (excluding base/) is treated as a Lambda function.

Current Lambda functions (17):

5. Infrastructure Deployment (Pulumi)¶

5.1 Stack Order¶

The 4 Pulumi stacks must be deployed in strict dependency order:

flowchart LR
    P[persistent] --> F[foundation] --> C[compute] --> E[edge]

    P:::persistent
    F:::foundation
    C:::compute
    E:::edge

    classDef persistent fill:#e1bee7,stroke:#333
    classDef foundation fill:#bbdefb,stroke:#333
    classDef compute fill:#c8e6c9,stroke:#333
    classDef edge fill:#ffe0b2,stroke:#333

5.2 Commands¶

5.3 Pulumi CI Script¶

infra/pulumi-ci.sh is used by GitHub Actions for automated deployments.

Usage:

./infra/pulumi-ci.sh <layer> <stack> <command> [s3-backend-url]

Arguments:

Required environment variables: - PULUMI_CONFIG_PASSPHRASE -- Stack encryption passphrase - S3_PULUMI_BACKEND_URL -- S3 backend for Pulumi state - AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY -- AWS credentials - AWS_REGION -- (optional, defaults to eu-central-1)

When layer=all, the script runs all stacks in order: persistent → foundation → compute → edge.

5.4 Preview Before Deploy¶

# Preview all stacks for dev environment
just infra-preview dev

# Preview individual stacks
just infra-preview-persistent dev
just infra-preview-foundation dev
just infra-preview-compute dev
just infra-preview-edge dev

On pull requests that touch infra/**, the deploy-infra.yml workflow automatically runs pulumi preview across all stacks for dev, staging, and prod, posting a summary table as a PR comment.

6. Service Container Deployment¶

6.1 Docker Build¶

Four services are containerized for ECS deployment:

# Build all 4 service images (linux/amd64)
just docker-build

# Build a single service
just docker-build-service backend

6.2 ECR Push¶

# Login to ECR + build + push all services
just service-push-all

# Push a single service
just service-push backend

# Verify images exist in ECR
just service-check-images

6.3 ECS Redeployment¶

# Force rolling update on all ECS services
just ecs-force-deploy-all

# Force rolling update on a specific service
just ecs-force-deploy strategy-service

# Check ECS service status
just ecs-status

# View recent ECS events
just ecs-events strategy-service

ECS services targeted by ecs-force-deploy-all: strategy-service, dry-run-trading, live-trading (trading services only).

The docker-build.yml GitHub Actions workflow triggers on version tags (after CI passes), builds all 4 service images in parallel, pushes to ECR with both the version tag and latest, then forces a new ECS deployment for backend-api, data-collection, strategy-service, and mlflow.

7. Library Publishing¶

Libraries are published to AWS CodeArtifact for consumption by the separate tradai-strategies repository.

flowchart LR
    L[CodeArtifact Login<br/><code>just codeartifact-login dev</code>] --> B[Build Wheels<br/><code>just build-libs</code>]
    B --> P[Publish to CodeArtifact<br/><code>just publish-all-libs dev</code>]

    style L fill:#fff3e0
    style P fill:#c8e6c9

The publish-libs.yml GitHub Actions workflow triggers on version tags (after CI passes) and publishes tradai-strategy to CodeArtifact automatically, with --skip-existing to handle re-runs.

8. Rollback Procedures¶

Service Rollback¶

Redeploy the previous working image by forcing a new ECS deployment. ECS uses the :latest tag, so push a known-good image and redeploy:

# Rebuild from a known-good commit
git checkout <good-commit>
just docker-build-service backend
just service-push backend
just ecs-force-deploy backend

Lambda Rollback¶

Rebuild Lambda images from the previous commit and push:

git checkout <good-commit>
just lambda-bootstrap

The deploy-lambdas.yml workflow updates each Lambda function code to the new image URI. AWS Lambda also supports version aliases for instant rollback without rebuilding.

Infrastructure Rollback¶

# Recover from a failed deployment (cancel pending ops, refresh state)
just infra-recover <layer> <stack>

# Re-deploy with current code (Pulumi converges to desired state)
just infra-up-<layer> <stack>

Pulumi stores state in S3. Each pulumi up converges the actual cloud state to match the declared code. Rolling back infrastructure means running pulumi up with the previous code version.

9. Environment Configuration¶

Local Development¶

The infra/.env file is sourced automatically by all just infra-* recipes via:

set -a && source ../.env && set +a

Required variables in infra/.env:

CI/CD (GitHub Actions)¶

GitHub Actions workflows use repository secrets:

CI/CD Workflow Triggers¶

10. Changelog¶

Dependencies¶

This document references: