Config Versioning — Verification & Reproduction Guide¶

How to verify that config versioning works end-to-end, both locally and on AWS.

Epic: #308 | PRs: #317, #319, #320, #323, #326, #328, #330

What to Verify¶

Config versioning tracks config_version_id through 5 propagation points:

API submit -> Step Functions input -> ECS container env -> DynamoDB job -> S3 result.json -> MLflow run tag

Plus: - Config CRUD (create/list/show/activate/deprecate) - Content-addressable dedup (same data = same config_id) - Config data merge (active config defaults merge into backtest params) - Backward compatibility (runs without config_version_id still work)

Option A: Local Verification (Docker Compose + LocalStack)¶

Prerequisites¶

cd /path/to/tradai
docker compose --profile localstack up --build -d

Wait for LocalStack to initialize. Verify the config-versions table exists:

docker exec tradai-localstack awslocal dynamodb list-tables --region eu-central-1
# Should include "tradai-config-versions-dev"

Step 1: Create a Config Version¶

# Via CLI
tradai config create StochRsiStrategy --data '{"timeframe":"1h","stoploss":-0.02,"stake_amount":150}'

# Or via API
curl -s -X POST http://localhost:8000/api/v1/configs \
  -H "Content-Type: application/json" \
  -d '{
    "strategy_name": "StochRsiStrategy",
    "config_data": {"timeframe":"1h","stoploss":-0.02,"stake_amount":150},
    "description": "test config"
  }' | python -m json.tool

Save the config_id from the response (e.g., v1-a3b2c1d4).

Step 2: Activate the Config Version¶

tradai config activate StochRsiStrategy <CONFIG_ID>

# Or via API
curl -s -X POST http://localhost:8000/api/v1/configs/StochRsiStrategy/<CONFIG_ID>/activate

Step 3: Verify Config is Active¶

tradai config list StochRsiStrategy
# Should show status=ACTIVE for your config_id

tradai config show StochRsiStrategy <CONFIG_ID>
# Should show full config_data and status=active

Step 4: Submit Backtest with Config Version¶

# With explicit config_version_id
tradai backtest quick StochRsiStrategy --config-version <CONFIG_ID>

# Or with "ACTIVE" to auto-resolve
tradai backtest quick StochRsiStrategy --config-version ACTIVE

# Or via API
curl -s -X POST http://localhost:8000/api/v1/backtests \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "strategy": "StochRsiStrategy",
      "symbols": ["BTC/USDT:USDT"],
      "timeframe": "1h",
      "start_date": "2025-01-01",
      "end_date": "2025-02-01",
      "stake_amount": 1000.0,
      "exchange": "binance_futures",
      "config_version_id": "ACTIVE"
    },
    "experiment_name": "config-version-test"
  }'

Expected: Backend log shows Resolved 'ACTIVE' to config version: <CONFIG_ID> and Merged config_data defaults from <CONFIG_ID>: ['stoploss'] (stake_amount is NOT merged because the explicit value 1000.0 takes precedence).

Step 5: Verify Config Data Merge¶

The stoploss: -0.02 from config_data should appear in the backtest params (merged as default). The stake_amount: 150 from config_data should NOT override the explicit 1000.0.

Step 6: Run Automated Tests¶

# Integration tests (13 repo + 7 API tests)
uv run pytest tests/integration/test_config_version_repository.py tests/integration/test_config_api.py -v

# E2E test (full local pipeline)
uv run pytest tests/e2e/test_config_version_e2e.py -v

Option B: AWS Verification (Dev Environment)¶

Prerequisites¶

aws sso login --profile tradai

ALB_URL="http://tradai-dev-1942285475.eu-central-1.elb.amazonaws.com"

Step 1: Create and Activate a Config Version¶

# Create
curl -s -X POST $ALB_URL/api/v1/configs \
  -H "Content-Type: application/json" \
  -d '{
    "strategy_name": "StochRsiStrategy",
    "config_data": {"timeframe":"1h","stoploss":-0.03},
    "description": "AWS verification test"
  }'
# Save config_id from response

# Activate
curl -s -X POST $ALB_URL/api/v1/configs/StochRsiStrategy/<CONFIG_ID>/activate

Step 2: Submit Backtest with Config Version¶

Important: you MUST include task_definition in the request. Due to #332, task_definition is in _PROTECTED_FIELDS and cannot be merged from config_data, so omitting it causes "TaskDefinition not found".

curl -s -X POST $ALB_URL/api/v1/backtests \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "strategy": "StochRsiStrategy",
      "symbols": ["BTC/USDT:USDT"],
      "timeframe": "1h",
      "start_date": "2025-01-01",
      "end_date": "2025-02-01",
      "stake_amount": 1000.0,
      "stake_currency": "USDT",
      "exchange": "binance_futures",
      "task_definition": "tradai-strategy-stochrsi-dev",
      "config_version_id": "<CONFIG_ID>"
    },
    "experiment_name": "config-version-aws-test",
    "priority": 5
  }'

Save the job_id.

Step 3: Wait for Completion¶

# Poll status
curl -s $ALB_URL/api/v1/backtests/<JOB_ID> | python -m json.tool

Or use the e2e script: ./scripts/e2e-backtest.sh

Step 4: Verify 5 Propagation Points¶

Point 1: Step Functions input¶

# Find execution ARN
aws stepfunctions list-executions \
  --state-machine-arn "arn:aws:states:eu-central-1:600802701449:stateMachine:tradai-backtest-workflow-dev" \
  --max-results 5 --profile tradai --region eu-central-1 \
  --query 'executions[?contains(name, `<JOB_ID>`)].executionArn' --output text

# Check input
aws stepfunctions describe-execution \
  --execution-arn "<EXECUTION_ARN>" \
  --profile tradai --region eu-central-1 \
  --query 'input' --output text | python -m json.tool

Expected: "config_version_id": "<CONFIG_ID>" in the input JSON.

Point 2: ECS container environment¶

Visible in Step Functions execution history (RunBacktest state input) or ECS task details. The container receives CONFIG_VERSION_ID=<CONFIG_ID> and S3_RESULTS_BUCKET=tradai-results-dev as environment variables.

# Check ECS container logs for confirmation
MSYS_NO_PATHCONV=1 aws logs get-log-events \
  --log-group-name "/aws/ecs/tradai-dev" \
  --log-stream-name "strategy/strategy/<ECS_TASK_ID>" \
  --profile tradai --region eu-central-1 \
  --query 'events[*].message' --output json | head -50

Point 3: DynamoDB job record¶

aws dynamodb get-item \
  --table-name tradai-workflow-state-dev \
  --key '{"run_id":{"S":"<JOB_ID>"}}' \
  --profile tradai --region eu-central-1 \
  --query 'Item.config_version_id'

Expected: {"S": "<CONFIG_ID>"}

Point 4: S3 result JSON¶

The result is uploaded to s3://tradai-results-dev/backtests/<STRATEGY>/<DATE>/<JOB_ID>.json:

# Find the result file
aws s3 ls s3://tradai-results-dev/backtests/StochRsiStrategy/ --recursive \
  --profile tradai --region eu-central-1 | grep <JOB_ID>

# Download and check
aws s3 cp s3://tradai-results-dev/backtests/StochRsiStrategy/<DATE>/<JOB_ID>.json - \
  --profile tradai --region eu-central-1 | python -m json.tool | grep config_version_id

Expected: "config_version_id": "<CONFIG_ID>"

If you don't know the date, search recursively:

aws s3 ls s3://tradai-results-dev/ --recursive \
  --profile tradai --region eu-central-1 | grep <JOB_ID>

Point 5: MLflow run tag¶

MLflow is internal-only (Cloud Map mlflow.tradai-dev.local:5000), not exposed through ALB. Query via SSM on EC2:

# Get MLflow run ID from DynamoDB
MLFLOW_RUN_ID=$(aws dynamodb get-item \
  --table-name tradai-workflow-state-dev \
  --key '{"run_id":{"S":"<JOB_ID>"}}' \
  --profile tradai --region eu-central-1 \
  --query 'Item.mlflow_run_id.S' --output text)

# Query MLflow tag via SSM (MLflow runs on EC2 at localhost:5000)
aws ssm send-command \
  --instance-ids i-09ff8e414fe16f7e7 \
  --document-name "AWS-RunShellScript" \
  --parameters "commands=[\"curl -s http://localhost:5000/api/2.0/mlflow/runs/get?run_id=$MLFLOW_RUN_ID | python3 -m json.tool | grep -A1 config_version_id\"]" \
  --profile tradai --region eu-central-1

# Get SSM command output (replace COMMAND_ID from previous response)
aws ssm get-command-invocation \
  --command-id <COMMAND_ID> \
  --instance-id i-09ff8e414fe16f7e7 \
  --profile tradai --region eu-central-1 \
  --query StandardOutputContent --output text

Expected: tag config_version_id = <CONFIG_ID>

Step 5: Backward Compatibility Check¶

Submit a backtest without config_version_id:

curl -s -X POST $ALB_URL/api/v1/backtests \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "strategy": "StochRsiStrategy",
      "symbols": ["BTC/USDT:USDT"],
      "timeframe": "1h",
      "start_date": "2025-01-01",
      "end_date": "2025-02-01",
      "stake_amount": 1000.0,
      "exchange": "binance_futures",
      "task_definition": "tradai-strategy-stochrsi-dev"
    },
    "experiment_name": "backward-compat-test"
  }'

Expected: Backtest completes normally. DynamoDB config_version_id is empty/absent. MLflow tag config_version_id is absent.

Config CRUD API Reference¶

Method	Endpoint	Description
POST	`/api/v1/configs`	Create config version
GET	`/api/v1/configs/{strategy}`	List versions for strategy
GET	`/api/v1/configs/{strategy}/{config_id}`	Get specific version
POST	`/api/v1/configs/{strategy}/{config_id}/activate`	Activate version
POST	`/api/v1/configs/{strategy}/{config_id}/deprecate`	Deprecate version

CLI Reference¶

tradai config create <STRATEGY> --data '<JSON>'     # Create from JSON string
tradai config create <STRATEGY> --file config.json   # Create from file
tradai config list <STRATEGY>                        # List all versions
tradai config list <STRATEGY> --status active        # Filter by status
tradai config show <STRATEGY> <CONFIG_ID>            # Show version details
tradai config activate <STRATEGY> <CONFIG_ID>        # Activate version
tradai config deprecate <STRATEGY> <CONFIG_ID>       # Deprecate version

tradai backtest quick <STRATEGY> --config-version <CONFIG_ID>   # Backtest with specific version
tradai backtest quick <STRATEGY> --config-version ACTIVE        # Backtest with active version

Verification Checklist¶

#	Check	Local	AWS
1	Config CRUD works (create/list/show/activate/deprecate)	`curl localhost:8000`	`curl $ALB_URL`
2	Content-addressable dedup (same data = same config_id)	POST same data twice	POST same data twice
3	Config data merge (defaults from active version)	Check backend logs	Check backend logs
4	CLI `tradai config` commands work	`tradai config list`	N/A (CLI needs local backend)
5	CLI `--config-version` flag works	`tradai backtest quick -cv ACTIVE`	N/A
6	SF input contains config_version_id	N/A (no SF locally)	`describe-execution`
7	ECS env has CONFIG_VERSION_ID	Check Docker container env	CloudWatch logs
8	DynamoDB has config_version_id	LocalStack `get-item`	`get-item`
9	S3 result JSON has config_version_id	LocalStack `s3 ls/cp`	`s3 ls/cp` (path: `backtests/<strategy>/<date>/<job_id>.json`)
10	MLflow tag has config_version_id	`localhost:5001` API	SSM or direct MLflow API
11	Backward compat (no config_version_id)	Submit without field	Submit without field
12	Integration tests pass (21 tests)	`uv run pytest tests/integration/test_config_*.py`	N/A
13	E2E test passes	`uv run pytest tests/e2e/test_config_version_e2e.py`	N/A

Known Issues¶

Issue	Impact	Workaround
#332 `task_definition` in `_PROTECTED_FIELDS`	Submitting with `config_version_id` but without `task_definition` fails with "TaskDefinition not found"	Always include `task_definition` explicitly in the request
#329 B1 Pulumi drift	SF definition was hot-patched on 2026-04-10 (added `S3_RESULTS_BUCKET`), not deployed via `pulumi up`. Next `pulumi up` may overwrite	Run `just infra-up-compute dev` to reconcile
#329 C3 MLflow intermittent 503	MLflow sometimes falls back to SQLite after restart	Restart MLflow container on EC2: `docker restart mlflow`

Reference¶

Execution report (2026-04-10) — AWS E2E validation results
Config & Model Versioning guide — architecture and lifecycle docs
E2E Backtest Pipeline (AWS) — general AWS backtest operations guide
Epic #308 — parent issue with full scope
Follow-ups #329, #331, #332 — open items