Summary¶

Infrastructure audit comparing issue acceptance criteria against real AWS dev environment (account 600802701449, eu-central-1), not code or reports.

Audit date: 2026-04-27

#89 - Deploy dev environment to AWS - PARTIALLY RESOLVED¶

What the issue requires vs reality¶

Issues¶

1. Architecture changed: Issue describes Fargate deployment, actual uses consolidated EC2 for dev (cost optimization). Not a bug, but the issue verification commands won't find the 4 main services via aws ecs list-services.

2. tradai-models-dev DynamoDB table still does not exist - mentioned as "1 missing" in the issue body, never created. Required by #92 (promotion/rollback).

#90 - E2E test backtest flow in AWS - PARTIALLY RESOLVED¶

Acceptance criteria verification¶

Critical finding¶

The full E2E pipeline was NEVER tested through the production path:

API Gateway -> SQS FIFO -> backtest-consumer Lambda -> Step Functions

Evidence: - tradai-backtest-consumer-dev Lambda: 0 bytes in CloudWatch logs (never invoked) - SQS queue: 0 messages currently, no messages-not-visible - DynamoDB: 0 items with SQS-originated run IDs - Step Functions executions were triggered via direct StartExecution API calls (names like backtest-6a89061f-...), not via the Lambda consumer

The backtest workflow itself works (Step Functions -> ECS -> S3 -> DynamoDB), but the entry point (API -> SQS -> Lambda) remains untested.

#91 - E2E test model training flow in AWS - RESOLVED¶

Evidence¶

Notes¶

• Very thorough testing was done: multiple verification runs with different scenarios (happy path, skip, invalid input, allowlist, config version)
• Both E2ETestStrategy and StochRsiStrategy strategies have been validated

#92 - E2E test promotion and rollback flow - NOT RESOLVED (correctly OPEN)¶

Evidence¶

Blockers¶

1. Missing tradai-models-dev table - required for model stage tracking
2. No model has ever been promoted - E2ETestStrategy has 34 versions, all at stage "None"
3. promote-model and model-rollback Lambdas never executed - the promotion/rollback lifecycle is completely untested
4. Note: MLflow 3.x uses aliases instead of stages, so current_stage=None might be expected, but no aliases are set either (aliases=null)

#308-313 - Config Versioning - ALL RESOLVED¶

#308 (E2E epic) - Evidence¶

#309 (Backend API + DI) - Evidence¶

• GET /api/v1/configs/{strategy} returns valid JSON response
• Config versions exist in DynamoDB -> DI + service wiring works

#310 (CLI commands) - Evidence¶

• DynamoDB has 2 config versions for StochRsiStrategy with real lifecycle (v1=deprecated, v2=active)
• This data was created via CLI/API -> commands exist

#311 (Pipeline data flow) - Evidence¶

• CONFIG_VERSION_ID in backtest workflow ASL: 1 occurrence
• MLflow experiments with config version test prefix exist

#312 (Training pipeline) - Evidence¶

• CONFIG_VERSION_ID in retraining workflow ASL: 1 occurrence
• Retraining execution verify-cv-verify-91-* (config version verification) succeeded

#313 (LocalStack + tests) - Evidence¶

• LocalStack init-aws.sh: 7 references to config-versions
• DynamoDB table exists with correct GSIs (config_hash-index, status-index)
• E2E experiments in MLflow confirm tests were executed

Action Items¶

Must fix (from closed issues with gaps)¶

1. Create tradai-models-dev DynamoDB table - Missing since #89, blocks #92
2. Add to Pulumi persistent stack

3. Required for model promotion/rollback lifecycle

4. Test API Gateway -> SQS -> backtest-consumer -> Step Functions path (#90)

5. Submit a backtest through POST https://z9uaqcerrd.execute-api.eu-central-1.amazonaws.com/api/v1/backtests
6. Verify backtest-consumer Lambda picks it up
7. The current backtest flow bypasses SQS entirely

Should fix¶

1. Complete #92 - Test full promotion/rollback lifecycle
2. Promote E2ETestStrategy to Staging/Production
3. Test model-rollback Lambda
4. Verify DynamoDB state tracking

Consider¶

1. Update #89 to reflect consolidated EC2 architecture change
2. Issue describes Fargate, reality uses EC2 for dev cost savings
3. Not a regression, but documentation should match

Related¶

• 394 - Dev environment health audit (broader infrastructure issues)¶

• Audit documents: docs/verification/dev-environment-audit.md, docs/verification/dev-environment-health-audit-20260427.md