TradAI ML Strategy Lifecycle Architecture¶

Version: 2.0.0 | Date: 2026-03-28 | Status: IMPLEMENTED -- E2E Validation Pending

Depends On: 11-LIVE-TRADING.md, 02-ARCHITECTURE-OVERVIEW.md

TL;DR: TradAI implements a complete ML strategy lifecycle: FreqAI walk-forward training with LightGBM/CatBoost/XGBoost, automated MLflow model registration with reproducibility manifests, and a full MLOps pipeline (drift detection via PSI/KS statistics, scheduled retraining, champion-vs-challenger comparison, and auto-rollback). All components (ML001-ML005, MO001-MO004) are built and unit-tested. End-to-end validation across the full pipeline remains pending.

1. Lifecycle Overview¶

flowchart LR
    A["Feature<br/>Engineering"] --> B["Model<br/>Training"]
    B --> C["MLflow<br/>Registry"]
    C --> D["Backtest<br/>Validation"]
    D --> E["Live/Paper<br/>Trading"]
    E --> F["Drift<br/>Detection"]
    F -->|"retrain<br/>trigger"| B
    F -->|"rollback<br/>trigger"| C

    style A fill:#1565c0,color:#fff
    style B fill:#1565c0,color:#fff
    style C fill:#2e7d32,color:#fff
    style D fill:#2e7d32,color:#fff
    style E fill:#f57f17,color:#000
    style F fill:#d32f2f,color:#fff

The ML lifecycle has six stages:

2. Training Pipeline¶

Architecture¶

TrainHandler delegates to four collaborators following single-responsibility:

Training Flow¶

1. TrainHandler.run() sets global random seeds (if configured) for reproducibility
2. Builds TrainingConfig from environment settings (strategy, FreqAI model, pairs, timeframe)
3. Executes Freqtrade backtesting as a subprocess with the chosen FreqAI model (e.g., LightGBMRegressor)
4. TrainingResultParser extracts validation metrics (Sharpe, profit, drawdown, trade count)
5. MLflowReporter.log() creates an MLflow run with all metrics and tags
6. ModelRegistrar.register() uploads model artifacts and creates a new model version
7. Updates DynamoDB job status to COMPLETED

See libs/tradai-common/src/tradai/common/entrypoint/training/handler.py

Training Configuration¶

TrainingConfig is a frozen Pydantic model that validates all training parameters:

• strategy -- FreqAI-enabled strategy class name
• freqai_model -- ML model type, validated against FreqAIModelRegistry (LightGBM, CatBoost, XGBoost)
• train_period_days -- Training window (7-365 days)
• backtest_period_days -- Walk-forward validation period (1-30 days)
• n_estimators, learning_rate, max_depth -- Model hyperparameters

See libs/tradai-common/src/tradai/common/entrypoint/training/config.py

3. Walk-Forward Training Model (FreqAI)¶

TradAI provides two custom FreqAI models:

TrainingPredictionModel implements the fit() method called by FreqAI for each training window. It extracts FitParams (train/test splits, sample weights), delegates to the configured ML framework, and logs per-window metrics.

TradAIPredictionModel uses a PredictionLoader protocol with two implementations: CSVLoader (local file) and MLflowLoader (model registry). The PredictionSource enum controls which loader is used.

Key files:

• libs/tradai-strategy/src/tradai/strategy/freqai/training_model.py
• libs/tradai-strategy/src/tradai/strategy/freqai/prediction_model.py
• libs/tradai-strategy/src/tradai/strategy/freqai/config_builder.py

4. Model Registry (MLflow)¶

Registry Lifecycle¶

Training Complete
    --> ModelRegistrar uploads artifacts (model files, feature importance, manifests)
    --> Creates new model version in MLflow (stage: None)
    --> Optionally promotes to Staging if metrics exceed thresholds
    --> compare-models Lambda compares champion vs challenger
    --> promote-model Lambda transitions winner to Production

Stage Transitions¶

Key Operations (RegistryMixin)¶

• get_model_version(name, version) -- Fetch version details
• create_model_version(name, source, run_id, tags) -- Register new version
• transition_model_version_stage(name, version, stage) -- Stage promotion/demotion
• get_registered_model(name) -- Get model with all versions
• search_registered_models(filter_string) -- Query models

See libs/tradai-common/src/tradai/common/mlflow/adapter.py and libs/tradai-common/src/tradai/common/mlflow/registry.py

5. Drift Detection¶

DriftDetector¶

The DriftDetector class provides drift analysis with configurable thresholds:

The analysis produces a DriftResult containing:

• Overall PSI and severity classification (DriftSeverity enum)
• Per-feature drift metrics (FeatureDrift with PSI, KS stat, mean/std shifts)
• Actionable recommendations (auto-generated)
• Alert message (when requires_attention=True)
• CloudWatch-compatible metric output (to_cloudwatch_metrics())

Key files:

• libs/tradai-common/src/tradai/common/drift/detector.py
• libs/tradai-common/src/tradai/common/drift/entities.py

drift-monitor Lambda¶

The Lambda runs on a schedule and:

1. Fetches backtest metrics from MLflow experiments for each monitored model
2. Compares current period against reference (training) period
3. Calculates PSI and KS statistics for key metrics (profit_total, win_rate, sharpe_ratio, max_drawdown, trades_count)
4. Publishes CloudWatch metrics under the DriftMonitoring namespace
5. Sends SNS alerts when drift exceeds thresholds
6. Persists drift state to DynamoDB via DynamoDBStateRepository

See lambdas/drift-monitor/handler.py

6. Retraining Pipeline¶

retraining-scheduler Lambda¶

The Lambda supports three trigger modes:

Each retraining request launches an ECS Fargate task with TRADING_MODE=train and model-specific configuration (strategy, FreqAI model, pairs, training period).

See lambdas/retraining-scheduler/handler.py

7. Model Comparison and Promotion¶

ModelComparator¶

The ModelComparator class fetches backtest metrics for both champion and challenger from MLflow, compares key performance indicators, and returns a PromotionDecision:

See libs/tradai-common/src/tradai/common/model_comparison/comparator.py

compare-models Lambda¶

Invoked by Step Functions after training completes. Receives model_name and challenger_run_id, returns a decision with confidence score and profit improvement metrics.

See lambdas/compare-models/handler.py

promote-model Lambda¶

Executes the promotion: transitions the challenger to Production, archives the previous champion. Requires a minimum confidence threshold.

See lambdas/promote-model/handler.py

model-rollback Lambda¶

Rolls back to a previous model version when:

• CloudWatch alarms trigger (drift or performance degradation)
• Manual rollback is requested

Includes a configurable cooldown period (ROLLBACK_COOLDOWN_HOURS, default 24h) to prevent rapid rollback oscillation. Persists rollback state to DynamoDB.

See lambdas/model-rollback/handler.py

8. Key Source Files¶

9. Known Limitations¶

10. Changelog¶

Dependencies¶