TradAI Architecture Guide¶

Developer-friendly navigation guide to the TradAI codebase. This document bridges the detailed architecture reports with actual code locations.

Quick Navigation¶

System Overview¶

┌─────────────────────────────────────────────────────────────────────────┐
│                              TradAI Platform                            │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   ┌─────────────┐    ┌─────────────┐    ┌──────────────────────┐       │
│   │ tradai CLI  │───▶│   Backend   │───▶│  Strategy Service    │       │
│   │   (cli/)    │    │  (backend/) │    │ (strategy-service/)  │       │
│   └─────────────┘    └─────────────┘    └──────────────────────┘       │
│          │                  │                      │                    │
│          │                  │                      │                    │
│          ▼                  ▼                      ▼                    │
│   ┌─────────────────────────────────────────────────────────────┐      │
│   │                    Shared Libraries                          │      │
│   ├─────────────────┬─────────────────┬─────────────────────────┤      │
│   │  tradai-common  │   tradai-data   │    tradai-strategy      │      │
│   │   (entities,    │  (CCXT, Arctic) │  (TradAIStrategy,       │      │
│   │    aws, utils)  │                 │   metadata, enums)      │      │
│   └─────────────────┴─────────────────┴─────────────────────────┘      │
│                                │                                        │
│                                ▼                                        │
│   ┌─────────────────────────────────────────────────────────────┐      │
│   │                    External Services                         │      │
│   │   Freqtrade │ ArcticDB │ CCXT/Binance │ MLflow │ AWS        │      │
│   └─────────────────────────────────────────────────────────────┘      │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

3-Layer Architecture Pattern¶

All packages follow the same structure for consistency:

src/tradai/<package>/
├── api/              # Presentation Layer (FastAPI)
│   ├── routes.py     # API endpoints
│   ├── schemas.py    # Request/response models
│   └── dependencies.py
├── core/             # Business Logic Layer
│   ├── service.py    # Main service class
│   ├── entities.py   # Domain models (Pydantic)
│   └── settings.py   # Configuration
└── infrastructure/   # External Adapters
    └── repositories.py  # Database/API clients

Layer Rules¶

Why This Pattern?¶

• Testability: Core logic is isolated, easy to unit test
• Flexibility: Swap infrastructure without changing business logic
• Clarity: Clear boundaries prevent spaghetti code

Shared Libraries¶

tradai-common¶

Purpose: Base classes, entities, AWS utilities, exceptions

Location: libs/tradai-common/src/tradai/common/

Key Exports:

from tradai.common import (
    # Base classes
    BaseService,          # Service with Hydra config
    BaseSettings,         # Pydantic settings with S3 support
    LoggerMixin,          # Logging mixin

    # Entities
    DateRange,            # Immutable date range
    Timeframe,            # Timeframe parsing
    SymbolList,           # Symbol list with validation

    # Exceptions
    TradAIError,          # Base exception
    ValidationError,      # Input validation
    NotFoundError,        # Resource not found

    # AWS utilities
    get_secret,           # Secrets Manager
    S3Path,               # S3 path parsing
)

Design Document: tradai-common DESIGN.md

tradai-data¶

Purpose: Market data fetching and storage

Location: libs/tradai-data/src/tradai/data/

Key Exports:

from tradai.data import (
    # Core entities
    DateRange,            # Immutable date range
    OHLCVData,           # OHLCV result entity
    SymbolList,          # Symbol list with validation
    Timeframe,           # Timeframe parsing

    # Repository protocols
    DataRepository,       # Abstract base
    AsyncDataRepository,  # Async variant
    DataAdapter,         # Storage adapter protocol
)

from tradai.data.infrastructure.repositories import (
    CCXTRepository,       # Any CCXT-supported exchange
    AsyncCCXTRepository,  # Async concurrent fetching
    CCXTProRepository,    # WebSocket streaming
)

Example:

from tradai.common import ExchangeConfig, TradingMode
from tradai.data.infrastructure.repositories import CCXTRepository
from tradai.data.core.entities import SymbolList, DateRange, Timeframe

config = ExchangeConfig(name="binance", trading_mode=TradingMode.FUTURES)
repo = CCXTRepository(config)
data = repo.fetch_ohlcv(
    symbols=SymbolList.from_input(["BTC/USDT:USDT"]),
    date_range=DateRange.from_strings("2024-01-01", "2024-01-31"),
    timeframe=Timeframe.parse("1h"),
)
print(f"Fetched {data.row_count} rows")

Design Document: tradai-data DESIGN.md

tradai-strategy¶

Purpose: Strategy framework extending Freqtrade

Location: libs/tradai-strategy/src/tradai/strategy/

Key Exports:

from tradai.strategy import (
    # Base class
    TradAIStrategy,       # Extends Freqtrade IStrategy

    # Metadata
    StrategyMetadata,     # Required strategy info

    # Enums (type-safe)
    StrategyCategory,     # TREND_FOLLOWING, MEAN_REVERSION, etc.
    StrategyStatus,       # ACTIVE, TESTING, DEPRECATED
    SignalDirection,      # LONG, SHORT

    # Validation
    StrategyValidator,    # Server-side validation
)

Minimal Strategy:

from tradai.strategy import TradAIStrategy, StrategyMetadata
from tradai.strategy.enums import StrategyCategory

class MinimalStrategy(TradAIStrategy):
    timeframe = '1h'
    stoploss = -0.08

    def get_metadata(self) -> StrategyMetadata:
        return StrategyMetadata(
            name="MinimalStrategy",
            version="1.0.0",
            description="Simple RSI strategy",
            timeframe=self.timeframe,
            category=StrategyCategory.MEAN_REVERSION,
        )

    def populate_indicators(self, dataframe, metadata):
        import talib
        dataframe['rsi'] = talib.RSI(dataframe['close'])
        return dataframe

    def populate_entry_trend(self, dataframe, metadata):
        dataframe.loc[dataframe['rsi'] < 30, 'enter_long'] = 1
        return dataframe

    def populate_exit_trend(self, dataframe, metadata):
        return dataframe

Design Document: tradai-strategy DESIGN.md

Production Strategies: Located in separate private repo tradai-strategies/

Services¶

Backend (API Gateway)¶

Purpose: Main API gateway and orchestration

Port: 8000

Location: services/backend/src/tradai/backend/

Key Endpoints:

README: Backend

Strategy Service¶

Purpose: Strategy config management and MLflow integration

Port: 8003

Location: services/strategy-service/src/tradai/strategy_service/

Key Endpoints:

README: Strategy Service

Data Collection¶

Purpose: Market data fetching and storage

Port: 8002

Location: services/data-collection/src/tradai/data_collection/

Key Endpoints:

README: Data Collection

Strategy Development¶

tradai-uv vs tradai-strategies¶

Creating a New Strategy¶

1. Use the wizard:

   tradai strategy wizard
   

2. Or direct command:

   tradai strategy new MyStrategy --category momentum --timeframe 1h
   

3. Implement required methods:

4. get_metadata() - Return StrategyMetadata
5. populate_indicators() - Calculate indicators
6. populate_entry_trend() - Define entry signals

7. populate_exit_trend() - Define exit signals

8. Run backtest:

   tradai strategy-service backtest MyStrategy --timerange 20240101-20240131
   

Strategy File Structure¶

strategies/my-strategy/
├── __init__.py
├── strategy.py          # TradAIStrategy implementation
├── config.yaml          # Default configuration
└── tests/
    └── test_strategy.py

Detailed Architecture Reports¶

For AWS infrastructure, security, and deployment details, see:

Development Workflow¶

Daily Development¶

# Start services
tradai up

# Make changes...

# Check quality
just check      # lint + typecheck + test

# Format code
just fmt

Testing¶

# Run all tests
just test

# Run specific package tests
just test-package libs/tradai-data

# Run with coverage
just test-cov

Adding Dependencies¶

# Add to specific package
just add-dep libs/tradai-common pandas

# Sync all packages
uv sync --all-packages

Key Design Patterns¶

Architecture Diagrams¶

Lambda Infrastructure Overview¶

flowchart TB
    subgraph triggers["Triggers"]
        SQS[("SQS Queues")]
        EB[("EventBridge")]
        SF[("Step Functions")]
        CW[("CloudWatch Alarms")]
    end

    subgraph orchestration["Workflow Orchestration"]
        BC[backtest-consumer]
        SC[sqs-consumer]
        CR[cleanup-resources]
    end

    subgraph lifecycle["Model Lifecycle"]
        CRN[check-retraining-needed]
        CM[compare-models]
        PM[promote-model]
        MR[model-rollback]
        RS[retraining-scheduler]
    end

    subgraph monitoring["Monitoring & Health"]
        HC[health-check]
        THC[trading-heartbeat-check]
        DM[drift-monitor]
        OS[orphan-scanner]
    end

    subgraph deployment["Deployment & Validation"]
        VS[validate-strategy]
        NC[notify-completion]
    end

    subgraph data["Data Services"]
        DCP[data-collection-proxy]
    end

    subgraph compute["Compute"]
        ECS[("ECS Cluster")]
        MLF[("MLflow")]
        DDB[("DynamoDB")]
        SNS[("SNS Topics")]
    end

    SQS --> BC & SC
    EB --> RS & HC & THC & DM & OS
    SF --> CRN & CM & PM & VS & NC & DCP & CR
    CW --> MR

    BC & SC --> ECS
    RS --> ECS
    CRN & CM & PM & MR --> MLF
    DM --> DDB
    HC & THC --> DDB
    NC --> SNS
    OS --> ECS

Backtest Workflow (Step Functions)¶

stateDiagram-v2
    [*] --> ParallelValidation

    state ParallelValidation {
        [*] --> ValidateStrategy
        [*] --> CheckDataFreshness
        ValidateStrategy --> [*]
        CheckDataFreshness --> [*]
    }

    ParallelValidation --> EvaluateValidation

    state EvaluateValidation <<choice>>
    EvaluateValidation --> PrepareConfig: Valid
    EvaluateValidation --> FailValidation: Invalid
    EvaluateValidation --> FailDataStale: Data Stale

    FailValidation --> [*]
    FailDataStale --> [*]

    PrepareConfig --> RunBacktest
    RunBacktest --> TransformResults: Success
    RunBacktest --> CleanupAfterFailure: Error

    TransformResults --> UpdateStateCompleted
    UpdateStateCompleted --> CleanupResources
    CleanupResources --> NotifyCompletion

    CleanupAfterFailure --> UpdateStateFailed
    UpdateStateFailed --> NotifyFailure

    NotifyCompletion --> [*]
    NotifyFailure --> [*]

Streaming Data Flow¶

flowchart LR
    subgraph exchanges["Exchanges"]
        BIN[Binance WebSocket]
        OKX[OKX WebSocket]
    end

    subgraph collection["Data Collection Service"]
        WS[WebSocket Handler]
        BUF[Ring Buffer]
        FLUSH[Flush Timer]
    end

    subgraph storage["Storage Layer"]
        ARC[("ArcticDB")]
        S3[("S3 Bucket")]
    end

    subgraph consumers["Consumers"]
        BT[Backtesting]
        LT[Live Trading]
        AN[Analysis]
    end

    BIN -->|OHLCV| WS
    OKX -->|OHLCV| WS
    WS --> BUF
    BUF -->|buffer_size reached| FLUSH
    BUF -->|flush_interval| FLUSH
    FLUSH -->|write_dataframe| ARC
    ARC -->|versioned storage| S3

    ARC --> BT & LT & AN

Drift Monitoring Flow¶

flowchart TB
    subgraph trigger["Trigger"]
        EB[EventBridge<br/>rate: 6 hours]
    end

    subgraph lambda["drift-monitor Lambda"]
        FETCH[Fetch MLflow Runs]
        CALC[Calculate PSI]
        EVAL[Evaluate Severity]
    end

    subgraph sources["Data Sources"]
        MLF[("MLflow<br/>Experiments")]
        DDB[("DynamoDB<br/>Drift State")]
    end

    subgraph actions["Actions"]
        CW[CloudWatch<br/>Metrics]
        SNS[SNS Alert]
        RT[Trigger<br/>Retraining]
    end

    EB --> FETCH
    MLF --> FETCH
    FETCH --> CALC
    CALC --> EVAL
    DDB --> EVAL

    EVAL -->|PSI < 0.1| CW
    EVAL -->|0.1 ≤ PSI < 0.25| CW
    EVAL -->|0.1 ≤ PSI < 0.25| SNS
    EVAL -->|PSI ≥ 0.25| CW
    EVAL -->|PSI ≥ 0.25| SNS
    EVAL -->|PSI ≥ 0.25| RT

    EVAL -->|Update state| DDB

A/B Testing Architecture (Champion vs Challenger)¶

flowchart TB
    subgraph models["Model Registry"]
        CHAMP[Champion<br/>v2 - Production]
        CHALL[Challenger<br/>v3 - Staging]
    end

    subgraph testing["A/B Test Execution"]
        CREATE[Create Challenge]
        TRADES[Collect Trade Results]
        EVAL[Evaluate Performance]
    end

    subgraph metrics["Evaluation Metrics"]
        PROFIT[Total Profit]
        WIN[Win Rate]
        SHARPE[Sharpe Ratio]
        STAT[Statistical<br/>Significance]
    end

    subgraph outcomes["Outcomes"]
        PROMOTE[Promote Challenger<br/>→ Production]
        KEEP[Keep Champion]
        INCONC[Inconclusive<br/>Need More Data]
    end

    subgraph storage["State Storage"]
        DDB[("DynamoDB<br/>Challenge State")]
    end

    CHAMP --> CREATE
    CHALL --> CREATE
    CREATE --> DDB
    DDB --> TRADES
    TRADES --> DDB

    TRADES -->|min_trades reached| EVAL
    EVAL --> PROFIT & WIN & SHARPE
    PROFIT & WIN & SHARPE --> STAT

    STAT -->|p < 0.05,<br/>challenger better| PROMOTE
    STAT -->|p < 0.05,<br/>champion better| KEEP
    STAT -->|p ≥ 0.05| INCONC

    PROMOTE --> CHALL
    KEEP --> CHAMP

Retraining Pipeline¶

flowchart TB
    subgraph triggers["Triggers"]
        SCHED[Scheduled<br/>EventBridge]
        DRIFT[Drift Detected<br/>drift-monitor]
        MANUAL[Manual<br/>CLI/API]
    end

    subgraph scheduler["retraining-scheduler Lambda"]
        CHECK[Check Retraining<br/>Conditions]
        LAUNCH[Launch ECS Task]
    end

    subgraph training["Training ECS Task"]
        FETCH[Fetch Training Data]
        TRAIN[FreqAI Training<br/>Walk-Forward]
        UPLOAD[Upload to MLflow]
    end

    subgraph validation["Post-Training"]
        BACKTEST[Validation Backtest]
        COMPARE[compare-models Lambda]
        PROMOTE[promote-model Lambda]
    end

    subgraph registry["MLflow Registry"]
        STAGING[Staging]
        PROD[Production]
        ARCHIVE[Archived]
    end

    SCHED & DRIFT & MANUAL --> CHECK
    CHECK --> LAUNCH
    LAUNCH --> FETCH
    FETCH --> TRAIN
    TRAIN --> UPLOAD
    UPLOAD --> STAGING

    STAGING --> BACKTEST
    BACKTEST --> COMPARE
    COMPARE -->|Challenger wins| PROMOTE
    PROMOTE --> PROD
    PROMOTE -->|Archive old| ARCHIVE
    COMPARE -->|Champion wins| STAGING

Getting Help¶

• Development guidelines: See CLAUDE.md in repository root
• CLI usage: Run tradai --help
• Task tracking: python reports/implementation-tasks/tracker.py next
• Runbooks: Runbooks