Backtesting Strategies¶

Complete guide to running, analyzing, and comparing backtests.

Local vs Service Backtesting¶

Local mode runs Freqtrade as a subprocess — fast and requires no running services. Service mode sends the request to Backend → Strategy Service, providing comprehensive tracking and result storage.

flowchart TD
    User["User runs<br/><code>tradai backtest quick MyStrategy</code>"]
    Decision{{"--local flag<br/>or --strategy-dir?"}}

    User --> Decision

    subgraph local ["Local Execution Path"]
        FreqtradeCLI["FreqtradeCLIBuilder<br/>constructs command"]
        FreqtradeProc["FreqtradeBacktester<br/>subprocess"]
        LocalResult["Results printed<br/>to terminal"]
        FreqtradeCLI --> FreqtradeProc --> LocalResult
    end

    subgraph service ["Service Execution Path"]
        BackendAPI["Backend API<br/>POST /backtests"]
        SQS["SQS Queue<br/>BacktestJobMessage"]
        StepFn["Step Functions<br/>orchestration"]
        ECS["ECS Task<br/>runs Freqtrade"]
        DynamoDB["DynamoDB<br/>BacktestJobStatus"]
        MLflow["MLflow<br/>metrics + artifacts"]
        ECS --> DynamoDB
        ECS --> MLflow
    end

    Decision -- "Yes" --> FreqtradeCLI
    Decision -- "No" --> BackendAPI
    BackendAPI --> SQS --> StepFn --> ECS

Quick Backtest¶

For rapid iteration during development:

tradai backtest quick MyStrategy

This runs a focused 30-day backtest on BTC/USDT:USDT with minimal output:

Quick backtest: MyStrategy on BTC/USDT:USDT (30d)
✓ Completed in 12.3s

Profit: +5.23% | Trades: 15 | Sharpe: 1.45

Quick Backtest Options¶

# Custom period
tradai backtest quick MyStrategy --period 7d

# Different symbol
tradai backtest quick MyStrategy --symbol ETH/USDT:USDT

# With version
tradai backtest quick MyStrategy@1.0

# Non-blocking (returns job ID immediately)
tradai backtest quick MyStrategy --no-wait

# Strategy in external repo (forces local mode)
tradai backtest quick StochRsiStrategy --strategy-dir ../tradai-strategies

# Or set env var once
export TRADAI_STRATEGY_DIR=../tradai-strategies
tradai backtest quick StochRsiStrategy --period 30d

Full Backtest¶

Run a comprehensive backtest through the strategy service:

# Quick backtest (recommended for development)
tradai backtest quick MyStrategy --timerange 20240101-20240331 --symbol BTC/USDT:USDT

# Full backtest via strategy service
just run-backtest MyStrategy 1.0

View Results¶

Show Backtest with Visualizations¶

tradai backtest show <job_id>

Output includes: - Performance summary (trades, profit, win rate) - Risk metrics (Sharpe, Sortino, Calmar ratios) - ASCII equity curve - Monthly returns bar chart

Example:

═══ BACKTEST RESULTS ═══

┌─────────────────────────────────────────────────┐
│              Performance Summary                 │
├─────────────────────────────────────────────────┤
│ Trades: 45 | Win Rate: 71.1%                    │
│ Profit: $1,234.56 (+12.35%)                     │
│                                                 │
│ Risk Metrics                                    │
│ Sharpe: 1.85 | Sortino: 2.41 | Calmar: 0.95    │
│ Max Drawdown: -8.23%                            │
│ Profit Factor: 1.85                             │
└─────────────────────────────────────────────────┘

Equity Curve
$11200 ┤                                     ██
$10800 ┤                               ██████
$10400 ┤                         ██████
$10000 ┤ ████████████████████████
       └─────────────────────────────────────────
         Jan      Feb      Mar      Apr      May

Monthly Returns
Jan: ████████████  +5.2%
Feb: ██████        +2.1%
Mar: ████          -1.5%
Apr: ██████████    +4.8%
May: ████          +1.7%

View in MLflow: http://localhost:5001
Generate report: tradai backtest report abc123

Disable Charts¶

For CI/scripting:

tradai backtest show <job_id> --no-chart

Compare Backtests¶

Compare two backtest runs side by side:

tradai backtest compare run1-abc123 run2-def456

Output:

                    Backtest Comparison
┌──────────────────┬─────────────┬─────────────┬──────────┐
│ Metric           │ run1-abc123 │ run2-def456 │ Diff     │
├──────────────────┼─────────────┼─────────────┼──────────┤
│ sharpe_ratio     │ 1.45        │ 1.82        │ +0.37    │
│ sortino_ratio    │ 2.10        │ 2.45        │ +0.35    │
│ total_profit_pct │ 12.35%      │ 15.20%      │ +2.85%   │
│ max_drawdown_pct │ -8.23%      │ -6.50%      │ +1.73%   │
│ win_rate         │ 71.1%       │ 68.0%       │ -3.1%    │
│ profit_factor    │ 1.85        │ 2.10        │ +0.25    │
└──────────────────┴─────────────┴─────────────┴──────────┘

Winner by Sharpe: run2-def456
Winner by Profit: run2-def456

Custom Metrics¶

Compare specific metrics:

tradai backtest compare run1 run2 --metrics sharpe_ratio,profit_factor

Generate HTML Report¶

Create an interactive HTML report with Plotly charts:

tradai backtest report <job_id>

This generates backtest_report.html with: - Interactive equity curve (zoom, pan, hover) - Drawdown chart - Monthly returns heatmap - Trade distribution analysis - Win/loss breakdown

Report Options¶

# Custom output path
tradai backtest report <job_id> --output my_analysis.html

# Don't open browser
tradai backtest report <job_id> --no-open

Export Trades¶

Get detailed trade log (no 100-trade limit):

# Table view
tradai backtest trades <job_id>

# Export to CSV
tradai backtest trades <job_id> --format csv --output trades.csv

# Export to JSON
tradai backtest trades <job_id> --format json --output trades.json

# Export to Excel
tradai backtest trades <job_id> --format excel --output trades.xlsx

Trade data includes: - Open/close date and price - Pair and side (long/short) - Profit/loss - Duration - Exit reason

List Backtests¶

View recent backtest jobs:

tradai backtest list

Output:

                      Backtest Jobs
┌──────────────┬───────────┬──────────┬────────┬────────┬─────────────────────┐
│ Job ID       │  Status   │ Profit % │ Sharpe │ Trades │ Created             │
├──────────────┼───────────┼──────────┼────────┼────────┼─────────────────────┤
│ abc123def456 │ completed │ +12.35%  │   1.45 │     45 │ 2024-12-22T10:30:00 │
│ xyz789uvw012 │  running  │        - │      - │      - │ 2024-12-22T10:45:00 │
│ mno345pqr678 │   failed  │        - │      - │      - │ 2024-12-22T09:15:00 │
└──────────────┴───────────┴──────────┴────────┴────────┴─────────────────────┘

Filter by Status¶

tradai backtest list --status completed
tradai backtest list --status running
tradai backtest list --limit 50

MLflow Integration¶

Backtests are automatically tracked in MLflow:

# View MLflow UI
open http://localhost:5001

MLflow stores: - Run metrics (Sharpe, Sortino, profit, etc.) - Parameters (stoploss, stake amount) - Artifacts (trade logs, equity curves)

Workflow Example¶

Complete development → backtest → analyze workflow:

# 1. Create strategy
tradai strategy new MomentumV2 --category momentum --timeframe 1h

# 2. Edit strategy logic (in the tradai-strategies repo)
code strategies/momentum_v2/src/momentum_v2/strategy.py

# 3. Test indicators
tradai indicator test "ta.RSI(df['close'], 14)" --symbol BTC/USDT:USDT

# 4. Lint strategy
tradai strategy lint ./momentum_v2/

# 5. Quick backtest
tradai backtest quick MomentumV2 --period 7d

# 6. Extended backtest (longer period)
tradai backtest quick MomentumV2 --period 90d

# 7. View results
tradai backtest show <job_id>

# 8. Compare with previous version
tradai backtest compare <new_job_id> <old_job_id>

# 9. Generate report
tradai backtest report <job_id>

Quick Reference¶

Quality Thresholds¶

Use these benchmarks to decide whether results are ready for validation:

Metrics Reference¶

Environment Variables¶