Strategy CI/CD Pipeline¶

Complete guide for the strategy development CI/CD pipeline.

Pipeline Overview¶

┌─────────────────────────────────────────────────────────────────────┐
│                        Pull Request                                 │
├─────────────────────────────────────────────────────────────────────┤
│  lint  →  typecheck  →  unit-tests  →  smoke-backtest              │
└─────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────┐
│                     Main Branch Merge                               │
├─────────────────────────────────────────────────────────────────────┤
│  lint  →  typecheck  →  unit-tests  →  smoke-backtest              │
└─────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────┐
│                     Tag Release (*-v*)                              │
├─────────────────────────────────────────────────────────────────────┤
│  lint → typecheck → tests → full-backtest → docker → ECR → MLflow  │
└─────────────────────────────────────────────────────────────────────┘

Pipeline Stages¶

Stage 1: Lint¶

lint:
  script:
    - just lint $STRATEGY_NAME

Checks: - Ruff formatting and linting - Import sorting - Code style compliance

Failure Actions:

# Auto-fix issues
just fmt $STRATEGY_NAME

# Check specific issues
uv run ruff check --show-fixes $STRATEGY_NAME

Stage 2: Type Check¶

typecheck:
  script:
    - just typecheck $STRATEGY_NAME

Checks: - mypy strict type checking - Protocol compliance - Missing annotations

Failure Actions:

# Run mypy locally
uv run mypy $STRATEGY_NAME --show-error-codes

# Check specific file
uv run mypy strategies/$STRATEGY_NAME/src/*/strategy.py

Stage 3: Unit Tests¶

test:
  script:
    - just test $STRATEGY_NAME

Requirements: - All tests pass - Coverage >= 80% - No warnings treated as errors

Failure Actions:

# Run tests with verbose output
just test $STRATEGY_NAME -v

# Run specific test
uv run pytest strategies/$STRATEGY_NAME/tests/test_strategy.py::test_metadata -v

Stage 4: Smoke Backtest¶

smoke-backtest:
  script:
    - just backtest-smoke $STRATEGY_NAME

Configuration: - 30-day timerange - Single pair (BTC/USDT:USDT) - Basic validation only

Purpose: - Verify strategy can run - Catch major errors - Quick feedback (~5 minutes)

Stage 5: Full Backtest (Release Only)¶

full-backtest:
  script:
    - just backtest-full $STRATEGY_NAME --timerange 20240101-20241201

Configuration: - 12-month timerange - All configured pairs - Full metrics validation

Quality Gates: | Metric | Threshold | |--------|-----------| | Sharpe Ratio | >= 1.0 | | Profit Factor | >= 1.2 | | Max Drawdown | <= 20% | | Total Trades | >= 50 |

Stage 6: Docker Build¶

docker-build:
  script:
    - docker build -t $ECR_REPO:$TAG strategies/$STRATEGY_NAME

Dockerfile Template:

FROM freqtradeorg/freqtrade:stable

# Install strategy package
COPY . /app
RUN pip install /app

# Copy configs
COPY configs /freqtrade/user_data/configs

# Entry point
ENTRYPOINT ["freqtrade", "trade"]

Stage 7: ECR Push¶

ecr-push:
  script:
    - aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_REGISTRY
    - docker push $ECR_REPO:$TAG
    - docker push $ECR_REPO:latest

Tags: - v1.0.0 - Specific version - latest - Most recent

Stage 8: MLflow Registration¶

mlflow-register:
  script:
    - just mlflow-register $STRATEGY_NAME $TAG

Registers: - Model version with backtest metrics - Docker image URI - Strategy metadata - Initial stage: None

GitHub Actions Configuration¶

Workflow File¶

# .github/workflows/strategy-ci.yml
name: Strategy CI/CD

on:
  pull_request:
    paths:
      - 'strategies/**'
  push:
    branches: [main]
    tags:
      - '*-v*'

env:
  AWS_REGION: us-east-1

jobs:
  detect-strategies:
    runs-on: ubuntu-latest
    outputs:
      strategies: ${{ steps.detect.outputs.strategies }}
    steps:
      - uses: actions/checkout@v4
      - id: detect
        run: |
          # Detect changed strategies
          STRATEGIES=$(git diff --name-only ${{ github.event.before }} ${{ github.sha }} | grep '^strategies/' | cut -d'/' -f2 | sort -u | jq -R -s -c 'split("\n")[:-1]')
          echo "strategies=$STRATEGIES" >> $GITHUB_OUTPUT

  lint-test:
    needs: detect-strategies
    runs-on: ubuntu-latest
    strategy:
      matrix:
        strategy: ${{ fromJson(needs.detect-strategies.outputs.strategies) }}
    steps:
      - uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install UV
        run: pip install uv

      - name: Setup
        run: |
          cd strategies/${{ matrix.strategy }}
          uv sync

      - name: Lint
        run: just lint ${{ matrix.strategy }}

      - name: Type Check
        run: just typecheck ${{ matrix.strategy }}

      - name: Test
        run: just test ${{ matrix.strategy }}

  smoke-backtest:
    needs: lint-test
    runs-on: ubuntu-latest
    strategy:
      matrix:
        strategy: ${{ fromJson(needs.detect-strategies.outputs.strategies) }}
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: CodeArtifact Login
        run: |
          export CODEARTIFACT_TOKEN=$(aws codeartifact get-authorization-token \
            --domain tradai \
            --query authorizationToken \
            --output text)
          pip config set global.extra-index-url "https://aws:${CODEARTIFACT_TOKEN}@tradai-${AWS_REGION}.d.codeartifact.${AWS_REGION}.amazonaws.com/pypi/tradai-python/simple/"

      - name: Smoke Backtest
        run: just backtest-smoke ${{ matrix.strategy }}

  release:
    if: startsWith(github.ref, 'refs/tags/')
    needs: smoke-backtest
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Extract strategy from tag
        id: extract
        run: |
          TAG=${GITHUB_REF#refs/tags/}
          STRATEGY=$(echo $TAG | sed 's/-v[0-9].*//')
          VERSION=$(echo $TAG | sed 's/.*-v//')
          echo "strategy=$STRATEGY" >> $GITHUB_OUTPUT
          echo "version=$VERSION" >> $GITHUB_OUTPUT
          echo "tag=$TAG" >> $GITHUB_OUTPUT

      - name: Configure AWS
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Full Backtest
        run: just backtest-full ${{ steps.extract.outputs.strategy }}

      - name: Build Docker
        run: |
          docker build \
            -t ${{ secrets.ECR_REGISTRY }}/${{ steps.extract.outputs.strategy }}:${{ steps.extract.outputs.tag }} \
            -t ${{ secrets.ECR_REGISTRY }}/${{ steps.extract.outputs.strategy }}:latest \
            strategies/${{ steps.extract.outputs.strategy }}

      - name: Push to ECR
        run: |
          aws ecr get-login-password | docker login --username AWS --password-stdin ${{ secrets.ECR_REGISTRY }}
          docker push ${{ secrets.ECR_REGISTRY }}/${{ steps.extract.outputs.strategy }}:${{ steps.extract.outputs.tag }}
          docker push ${{ secrets.ECR_REGISTRY }}/${{ steps.extract.outputs.strategy }}:latest

      - name: Register to MLflow
        run: just mlflow-register ${{ steps.extract.outputs.strategy }} ${{ steps.extract.outputs.tag }}

Bitbucket Pipelines Configuration¶

# bitbucket-pipelines.yml
image: python:3.11

definitions:
  caches:
    uv: ~/.cache/uv

pipelines:
  pull-requests:
    '**':
      - step:
          name: Lint & Test
          caches:
            - uv
          script:
            - pip install uv
            - STRATEGIES=$(git diff --name-only origin/main | grep '^strategies/' | cut -d'/' -f2 | sort -u)
            - for STRATEGY in $STRATEGIES; do just lint $STRATEGY && just test $STRATEGY; done

      - step:
          name: Smoke Backtest
          caches:
            - uv
          script:
            - pip install uv awscli
            - aws codeartifact login --tool pip --domain tradai --repository tradai-python
            - STRATEGIES=$(git diff --name-only origin/main | grep '^strategies/' | cut -d'/' -f2 | sort -u)
            - for STRATEGY in $STRATEGIES; do just backtest-smoke $STRATEGY; done

  branches:
    main:
      - step:
          name: Lint & Test
          script:
            - pip install uv
            - for STRATEGY in strategies/*/; do just lint $(basename $STRATEGY) && just test $(basename $STRATEGY); done

  tags:
    '*-v*':
      - step:
          name: Full Release
          services:
            - docker
          script:
            - pip install uv awscli
            - TAG=$BITBUCKET_TAG
            - STRATEGY=$(echo $TAG | sed 's/-v[0-9].*//')
            - VERSION=$(echo $TAG | sed 's/.*-v//')

            # Full backtest
            - just backtest-full $STRATEGY

            # Docker build and push
            - aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_REGISTRY
            - docker build -t $ECR_REGISTRY/$STRATEGY:$TAG -t $ECR_REGISTRY/$STRATEGY:latest strategies/$STRATEGY
            - docker push $ECR_REGISTRY/$STRATEGY:$TAG
            - docker push $ECR_REGISTRY/$STRATEGY:latest

            # MLflow registration
            - just mlflow-register $STRATEGY $TAG

Required Secrets¶

GitHub Secrets¶

Secret	Description
`AWS_ACCESS_KEY_ID`	ECR/CodeArtifact access
`AWS_SECRET_ACCESS_KEY`	ECR/CodeArtifact access
`ECR_REGISTRY`	ECR registry URL
`MLFLOW_TRACKING_URI`	MLflow server URL

Bitbucket Variables¶

Variable	Description	Secured
`AWS_ACCESS_KEY_ID`	AWS access key	Yes
`AWS_SECRET_ACCESS_KEY`	AWS secret key	Yes
`ECR_REGISTRY`	ECR registry URL	No
`MLFLOW_TRACKING_URI`	MLflow server	No

Versioning Convention¶

Tag Format¶

<strategy-slug>-v<semver>

Examples: - momentum-v1.0.0 - trend-following-v2.1.3 - ml-rsi-v1.0.0-rc1

Version Incrementing¶

Change Type	Example	Version Bump
Bug fix	Fix signal calculation	1.0.0 → 1.0.1
New indicator	Add MACD	1.0.1 → 1.1.0
Breaking change	New config format	1.1.0 → 2.0.0

CodeArtifact Integration¶

Setup¶

# Login (12-hour token)
just codeartifact-login

# Or manually
export CODEARTIFACT_TOKEN=$(aws codeartifact get-authorization-token \
  --domain tradai \
  --query authorizationToken \
  --output text)

pip config set global.extra-index-url \
  "https://aws:${CODEARTIFACT_TOKEN}@tradai-us-east-1.d.codeartifact.us-east-1.amazonaws.com/pypi/tradai-python/simple/"

Publishing tradai-strategy¶

When tradai-strategy package is updated in tradai-uv:

# In tradai-uv
cd libs/tradai-strategy
uv build
twine upload --repository codeartifact dist/*

Consuming in Strategies¶

# strategies/my-strategy/pyproject.toml
[project]
dependencies = [
    "tradai-strategy>=1.0.0",  # From CodeArtifact
    "freqtrade>=2025.6",
]

Manual Commands¶

Local Development¶

# Full CI locally
just ci momentum-strategy

# Individual stages
just lint momentum-strategy
just typecheck momentum-strategy
just test momentum-strategy
just backtest-smoke momentum-strategy

Release Process¶

# 1. Ensure main is up to date
git checkout main
git pull

# 2. Update version in pyproject.toml
# version = "1.0.0" → "1.1.0"

# 3. Commit
git add .
git commit -m "chore(momentum-strategy): bump version to 1.1.0"
git push

# 4. Create and push tag
git tag momentum-strategy-v1.1.0
git push --tags

Troubleshooting¶

Pipeline Failures¶

Stage	Common Issue	Resolution
Lint	Formatting issues	`just fmt $STRATEGY`
Typecheck	Missing types	Add type annotations
Test	Fixture missing	Check conftest.py
Backtest	Data missing	Sync data first
Docker	Build fails	Check Dockerfile
ECR	Push denied	Check AWS credentials
MLflow	Register fails	Check MLflow connection

CodeArtifact Token Expired¶

# Re-login
just codeartifact-login

# CI: Token auto-refreshed in pipeline

ECR Push Permission Denied¶

# Check AWS credentials
aws sts get-caller-identity

# Verify ECR permissions
aws ecr describe-repositories --repository-names tradai/$STRATEGY

Strategy Lifecycle - Full development workflow
Strategy Repo Guide - Repository setup
Pulumi Deployment - Infrastructure (ECR, CodeArtifact)

Strategy CI/CD Pipeline¶

Pipeline Overview¶

Pipeline Stages¶

Stage 1: Lint¶

Stage 2: Type Check¶

Stage 3: Unit Tests¶

Stage 4: Smoke Backtest¶

Stage 5: Full Backtest (Release Only)¶

Stage 6: Docker Build¶

Stage 7: ECR Push¶

Stage 8: MLflow Registration¶

GitHub Actions Configuration¶

Workflow File¶

Bitbucket Pipelines Configuration¶

Required Secrets¶

GitHub Secrets¶

Bitbucket Variables¶

Versioning Convention¶

Tag Format¶

Version Incrementing¶

CodeArtifact Integration¶

Setup¶

Publishing tradai-strategy¶

Consuming in Strategies¶

Manual Commands¶

Local Development¶

Release Process¶

Troubleshooting¶

Pipeline Failures¶

CodeArtifact Token Expired¶

ECR Push Permission Denied¶

Related Documentation¶