Documentation Validation Report¶

Date: 2026-04-01 Validator: Claude QA (quant trading + devcontainer context) Method: Two-pass validation: (1) static analysis via parallel agents, (2) runtime execution inside devcontainer. Devcontainer: Built via devcontainer up, commands run with devcontainer exec.

Summary¶

Issue Legend¶

• BUG: Command fails or produces wrong output
• GAP: Missing information needed to complete a step
• STALE: Outdated reference (renamed file, changed API, etc.)
• WRONG-PATH: File path or URL doesn't exist
• WRONG-CMD: Command syntax is incorrect
• MISLEADING: Content is technically correct but misleading

Detailed Findings¶

quickstarts/onboarding.md¶

No issues found. All cross-reference links verified.

quickstarts/01-your-first-backtest.md¶

Runtime confirmed (devcontainer): tradai strategy backtest --help returns No such command 'backtest'. Did you mean 'test', 'ab-test'?

• Lines 28-33: [WRONG-CMD] tradai strategy backtest does not exist. No backtest subcommand on strategy_app. The correct command is tradai strategy-service backtest or tradai backtest quick.
• Line 54: [MISLEADING] tradai data sync requires tradai-data-collection package to be installed (lazy-loaded from tradai.data_collection.cli). New devs without it will silently not see the data subcommand. No prerequisite note.

quickstarts/02-create-new-strategy.md¶

Runtime confirmed (devcontainer): tradai strategy new MyMomentumStrategy --category momentum --timeframe 1h --can-short -y -o /tmp/strat-test succeeds. Generated 17 files. Must run from repo root or set TRADAI_ROOT (not documented).

• Line 19: [MISLEADING] Directory shown as strategies/mymomentstrategy/ but tradai strategy new outputs to CWD, not a strategies/ subdirectory (unless --output-dir is used).
• Lines 19-33: [GAP] Generated directory tree is incomplete -- actual output has 17 files. Missing from docs: Dockerfile, README.md, MyMomentumStrategy.py (root-level), .pre-commit-config.yaml, configs/backtest.json, configs/freqai_training.json, tradai.yaml, tests/__init__.py, src/.../exceptions.py.
• Line 9: [GAP] tradai strategy new requires running from repo root or TRADAI_ROOT env var. Error: "Could not find cookiecutter-tradai-strategy". Not documented.
• Line 71: [MISLEADING] Test path strategies/mymomentstrategy/tests/ uses incorrect strategies/ prefix (same root cause).

quickstarts/03-collect-market-data.md¶

Runtime confirmed (devcontainer): tradai data sync --help, tradai data list-symbols --help, tradai data check-freshness --help all work. Flags --timeframe, --days, --incremental all verified.

• Lines 19-25: [WRONG-CMD] Exchange JSON uses "type": "futures" but the actual ExchangeConfig field is "trading_mode". Pydantic v2 silently ignores unknown fields, so this works only because the default is already FUTURES. Would silently fail for spot mode.
• Line 30: [STALE] DATA_COLLECTION_ARCTIC_LIBRARY="futures" but actual default (per ArcticSettingsMixin and .env.example) is ohlcv.
• Line 164: [STALE] Storage diagram shows Library: "futures" but actual default library name is ohlcv.
• Line 181: [STALE] Configuration table states DATA_COLLECTION_ARCTIC_LIBRARY default is futures; actual default is ohlcv.

developer-guides/01-setup.md¶

Runtime validated: just setup executed. Steps 1-2 pass. Step 3 fails if core.hooksPath is set (pre-commit refuses). Step 4 never reached. Doc does not mention this common failure mode.

• Line 22: [GAP] just setup can fail at pre-commit install if core.hooksPath is set globally. No troubleshooting guidance for this.
• Line 33: [MISLEADING] git clone https://github.com/tradai-bot/tradai.git then cd tradai -- The remote is technically tradai.git, but the local directory used throughout the project is tradai-uv. Developers who already have the repo as tradai-uv will be confused.
• Line 42: [STALE] Setup output says TradAI Setup (Step 1/4): Installing Python 3.11... but actual justfile outputs Step 1/4: Installing Python 3.11... (no "TradAI Setup" prefix, different emoji).
• Line 43: [STALE] Setup output says TradAI Setup (Step 2/4): Syncing dependencies... but actual says Step 2/4: Syncing all workspace packages + dev tools....
• Lines 44-45: [STALE] Same prefix/wording mismatch pattern on remaining setup steps.
• Lines 59-78: [STALE] just doctor expected output shows custom formatted block with "Checking required tools..." etc. Actual implementation uses a Rich Table with "TradAI Doctor" title and "Check / Status / Message" columns. Output format completely different.
• Lines 143-147: [BUG] .env described as containing BINANCE_API_KEY and BINANCE_SECRET_KEY. These env vars do NOT exist in .env.example or anywhere in the codebase. Exchange config uses DATA_COLLECTION_EXCHANGES (JSON) in services/data-collection/.env.example.
• Line 149: [MISLEADING] Claims BACKEND_URL and TRADAI_UI_URL in .env are for Docker container-to-container communication. Actually .env.example sets BACKEND_URL=http://localhost:8000 (a host URL for CLI). Docker internal URLs like BACKEND_DATA_COLLECTION_URL are in docker-compose.yaml, not .env.

developer-guides/docker-setup.md¶

Runtime confirmed: just ps works. All services healthy. Postgres mapped to host port 5433 not 5432. MLflow /health returns OK.

• Line 19: [MISLEADING] cd tradai after cloning tradai.git -- same clone URL issue as 01-setup.md.
• Lines 24-26: [BUG] Instructs to copy per-service .env.example files but omits .env.docker. docker-compose.yaml references .env.docker in env_file: blocks. Without it, Docker services will fail to start.
• Line 59: [MISLEADING] Lists postgres at port 5432. Base docker-compose.yaml has NO host port mapping. Override maps to host port 5433: ${POSTGRES_PORT:-5433}:5432. Developers will fail connecting to 5432.
• Lines 79-84: [MISLEADING] Shows MLFLOW_TRACKING_URI=http://mlflow:5000 as internal URL. Actual docker-compose uses BACKEND_MLFLOW_TRACKING_URI and STRATEGY_SERVICE_MLFLOW_TRACKING_URI, not MLFLOW_TRACKING_URI.
• Lines 107-120: [STALE] Multi-stage Dockerfile example is significantly different from actual: doc uses pip install uv && uv sync, actual uses COPY --from=ghcr.io/astral-sh/uv:latest and wheel builds. Doc shows 2 stages, actual has 4. Doc CMD is python -m tradai.backend, actual is backend serve.
• Lines 141-144: [MISLEADING] Dev volume mounts ./services/backend/src:/app/src:cached and ./libs:/app/libs:cached do NOT exist in docker-compose. Actual dev mounts are for user_data, strategies, AWS creds. Override file says "use uv run locally instead of Docker".
• Lines 149-155: [GAP] Volume list includes mlflow-data, localstack-data, redis-data but these only exist in override (optional profiles). Base compose only defines postgres-data and user-data.
• Lines 183-189: [STALE] LocalStack env config shows vars in services/data-collection/.env but they're actually set in docker-compose.override.yaml environment blocks.
• Line 299: [WRONG-CMD] BACKEND_PORT=8080 docker compose up backend -- Actual env var is HOST_BACKEND_PORT (per docker-compose.yaml: ${HOST_BACKEND_PORT:-8000}:8000). BACKEND_PORT would change the internal container port.

developer-guides/documentation.md¶

Runtime confirmed (devcontainer): just docs-build succeeds (7.22s). Warning: templates/dryrun-signoff-checklist.md referenced in nav but excluded from built site.

• Line nav: [BUG] templates/dryrun-signoff-checklist.md in mkdocs.yml nav but excluded from build. Dead nav link.
• Line 12: [GAP] just docs-deploy uses mkdocs gh-deploy (GitHub Pages) but CI deploys to Cloudflare Pages. Inconsistency between manual and CI deployment targets not explained.
• Lines 37-42: [GAP] "Symlinked Content" table lists libs/*/DESIGN.md but omits that prepare-docs.sh also symlinks services/*/DESIGN.md. Service DESIGN docs not mentioned.
• Line 154: [MISLEADING] from tradai.common import BaseService -- technically valid but presented without context.
• Lines 220-227: [GAP] Trigger paths for auto-deploy omit .github/workflows/docs.yml which IS included in the actual workflow file.
• Lines 229-233: [MISLEADING] "Manual Deployment" shows just docs-deploy without explaining it deploys to GitHub Pages while CI deploys to Cloudflare Pages -- different hosting destinations.

developer-guides/02-create-strategy.md¶

Runtime confirmed (devcontainer): tradai strategy lint takes PATH not NAME. tradai strategy lint MyMomentumStrategy would fail -- needs tradai strategy lint ./path/to/strategy/.

• Lines 99-121: [BUG] Generated structure tree shows configs/tradai.yaml but actual template places tradai.yaml at project root, not inside configs/. Also missing: freqai_training.json in configs, .pre-commit-config.yaml at root, {{cookiecutter.strategy_name}}.py at root.
• Lines 188-198: [MISLEADING] Uses import pandas_ta as ta with ta.ema(), ta.rsi() but framework's own examples and base class docs use import talib with talib.RSI(), talib.EMA(). Inconsistent TA library usage.
• Line 286: [BUG] tradai strategy lint MyMomentumStrategy -- lint command takes a PATH, not a strategy name. Should be tradai strategy lint strategies/my_momentum_strategy/.
• Line 335: [BUG] Test assertion assert metadata.category.value == "momentum" -- with use_enum_values=True, metadata.category is already the string "momentum", not an enum. .value would fail. Should be assert metadata.category == "momentum".
• Line 360: [MISLEADING] just test-package strategies/my_momentum_strategy -- path depends on where tradai strategy new was run.
• Line 372: [MISLEADING] Indicator CLI example uses ta.RSI() (TA-Lib syntax) but strategy code examples use pandas_ta. No clarification of the difference.
• Line 405: [BUG] Quick reference table shows tradai strategy lint NAME -- should be PATH.
• Cross-doc: [MISLEADING] Inconsistent TA library usage: strategy examples use pandas_ta, indicator CLI uses talib.

developer-guides/03-develop.md¶

• Lines 12, 82-109, 184: [MISLEADING] Indicator tables and dev workflow use TA-Lib syntax (ta.SMA, ta.RSI) which is correct for the CLI, but inconsistent with 02-create-strategy.md which uses pandas_ta. Never explained.
• Line 200: [MISLEADING] just test-package strategies/my_strategy -- path assumes strategy is under strategies/ in workspace.
• Line 244: [MISLEADING] tradai strategy check PATH -- correct but table format is confusing.

developer-guides/04-backtest.md¶

• Lines 93-98: [WRONG-CMD] tradai strategy backtest --strategy MyStrategy --start ... --end ... -- there is NO tradai strategy backtest command. Should use tradai backtest quick or just run-backtest.
• Line 297: [STALE] code strategies/momentum_v2/src/momentum_v2/strategy.py -- assumes strategies live under strategies/ in the workspace (separate repo convention).

developer-guides/05-validate.md¶

• Lines 342-358: [GAP] Imports CheckSeverity from tradai.strategy.validation_entities but doesn't mention it's also available from tradai.strategy.preflight.entities.
• Lines 367-369: [MISLEADING] SanityCheckService.check() takes BacktestMetrics (Pydantic model), not a raw dict. Doc shows backtest_metrics without specifying type.
• Lines 371-382: [BUG] Doc shows result.has_errors, result.error_count etc. from ValidationResultMixin, but SanityCheckResult has field warnings not issues. The mixin's _get_issues() looks for self.issues which would return []. Properties would all return empty/false.
• Lines 384-392: [MISLEADING] gate = CIBacktestGate(thresholds) -- thresholds must be CIBacktestThresholds, not a plain dict. Doc should show constructing the proper type first.

developer-guides/06-optimize.md¶

• Lines 401-405: [MISLEADING] Troubleshooting suggests uv pip install freqtrade but project uses UV workspaces. Should recommend just setup or uv sync.

developer-guides/05-e2e-backtest-aws.md¶

No issues found.

developer-guides/06-e2e-backtest-scripts.md¶

No issues found.

developer-guides/strategy-lifecycle.md¶

Runtime confirmed (devcontainer): - tradai strategy promote → No such command 'promote'. - tradai strategy rollback → No such command 'rollback'. - tradai strategy register takes PATH + --env/--count, NOT name + --backtest-run-id/--docker-image-uri - just --list | grep backtest-smoke → no match. backtest-smoke and backtest-full do not exist.

• Lines 143-151: [WRONG-CMD] just test momentum-strategy -- just test takes verbose and workers args, not a strategy name. "momentum-strategy" would be interpreted as the verbose parameter. Same issue on line 150 with -k flag.
• Lines 190-191: [WRONG-CMD] just lint momentum-strategy -- just lint takes NO arguments (lints entire project). No per-strategy lint recipe exists.
• Line 205: [WRONG-CMD] just backtest-smoke momentum-strategy -- recipe does NOT exist in justfile.
• Lines 211-213: [WRONG-CMD] just backtest-full momentum-strategy, --timerange, --pairs -- recipes do NOT exist in justfile.
• Line 280: [BUG] tradai strategy register MomentumStrategy --backtest-run-id ... --docker-image-uri ... -- actual register command takes a PATH (to strategy dir), not a name. Options are --env and --count, not --backtest-run-id or --docker-image-uri. Command signature is completely wrong.
• Line 308: [MISLEADING] tradai strategy stage -- correct command but doc omits --config option.
• Line 360: [WRONG-CMD] tradai strategy promote MomentumStrategy --version 2 -- NO promote command exists. Closest is tradai strategy set-version MomentumStrategy 2 --stage Production.
• Line 389: [MISLEADING] tradai deploy strategy arguments differ from actual CLI signature.
• Line 460: [WRONG-CMD] tradai strategy rollback MomentumStrategy --version 1 -- NO rollback command exists on strategy. Correct is tradai deploy rollback MomentumStrategy --env dev.

strategy-templates.md¶

• Line 315: [MISLEADING] just run-backtest mystrategy 1.0.0 -- syntactically correct but requires strategy service to be running.
• Line 328: [MISLEADING] just test-package strategies/NAME -- assumes strategy is under strategies/ in workspace.
• Line 334: [GAP] Tips mention "Use simple structure first" but tradai strategy new has no --structure option. Only exposed through tradai strategy wizard or direct cookiecutter.

developer-guides/07-deploy.md¶

Runtime confirmed (devcontainer): tradai deploy strategy --help shows --mode, --exchange-secret, --skip-live-checks, --skip-tests, --skip-checks, --skip-build, --dry-run, --force. Doc only covers a subset.

• Line 31: [GAP] tradai deploy strategy docs are incomplete. Actual CLI has --mode / -m option (live vs dry-run) and --exchange-secret option that are not documented.
• Line 31: [GAP] --skip-live-checks, --skip-tests, --skip-checks, --skip-build, --dry-run, --force CLI options also not documented.

developer-guides/08-pulumi-deployment.md¶

• Line 142: [STALE] Claims dynamodb.py has "8 DynamoDB tables" but actual DYNAMODB_TABLES in config.py defines 12 tables. (The overview table on line 15 correctly says 12 -- internal inconsistency.)
• Line 143: [STALE] Claims ecr.py has "12 ECR repositories" but actual ALL_ECR_REPOS has 24 repos (6 service + 18 lambda). (Overview table on line 15 correctly says 24.)
• Line 114: [STALE] Claims staging uses "t4g.small RDS" but actual config.py has db.t4g.micro for all non-prod. Only prod uses db.t4g.small.
• Line 209: [MISLEADING] SERVICES code snippet is oversimplified. Actual code uses conditional expressions (e.g., desired_count: 1 if ENVIRONMENT != "prod" else 2).

developer-guides/pulumi-modules.md¶

• Line 47: [STALE] Claims compute stack has "18 container-image Lambdas" but compute defines only 17. The 18^th (update-nat-routes) is an inline runtime Lambda in the foundation stack.
• Line 297: [MISLEADING] Lists update-nat-routes as a container-image Lambda in compute stack. Actually: (a) it's an inline runtime Lambda using runtime="python3.11", and (b) it's in the foundation stack (infra/foundation/modules/nat_instance.py).
• Line 364: [STALE] Claims staging RDS uses db.t4g.small but config.py sets db.t4g.micro for all non-prod.

developer-guides/09-pulumi-operations.md¶

No issues found.

developer-guides/strategy-cicd.md¶

• Line 196: [GAP] docs-freshness.yml workflow exists in .github/workflows/ but is not listed in the "Other Workflows" table.

developer-guides/production-checklist.md¶

• Line 147: [WRONG-CMD] cd infra && pulumi preview -- No Pulumi.yaml at infra/ root. Pulumi projects are in subdirs (infra/persistent/, infra/foundation/, infra/compute/, infra/edge/). Should use just infra-preview dev or cd into a specific stack. Same issue on lines 150 (pulumi up) and 153 (pulumi stack output).
• Line 189: [WRONG-CMD] pulumi stack export --stack prod > backup.json and pulumi up --target-replace -- Bare Pulumi commands need to run from a specific stack directory after sourcing infra/.env. The justfile recipes handle this automatically.
• Line 77: [MISLEADING] "No secrets in environment variables" -- Lines 230-241 show MLFLOW_TRACKING_USERNAME and MLFLOW_TRACKING_PASSWORD marked as <from-secrets-manager>. Should clarify secrets are injected from Secrets Manager into env vars at runtime.

developer-guides/performance-tuning.md¶

• Line 18: [GAP] tradai data sync --symbols BTC/USDT:USDT --timeframe 1h --days 365 -- The exact flags (--timeframe, --days) could not be verified against the actual CLI function signature.
• Line 68: [GAP] tradai backtest --strategy {} -- The tradai backtest subcommand exists but a direct --strategy flag is not evidenced. CLI shows tradai backtest show, tradai backtest compare, etc.
• Line 87-88: [MISLEADING] from tradai.data.infrastructure.repositories import AsyncCCXTRepository -- Import is correct but the code snippet's method call doesn't match the actual AsyncFetchRepository protocol signature.
• Line 108: [WRONG] adapter.append(new_data, symbol) -- ArcticAdapter has no append method. It has save() and save_batch().
• Line 114: [WRONG] adapter.save(data, symbols, datetime.now(UTC), dedup=True) -- ArcticAdapter.save() has no dedup parameter. Actual signature: save(self, data: OHLCVData, symbols: SymbolList, latest_query_date: datetime, timeframe: str | None = None).
• Lines 125-131: [STALE] from fastapi_cache.decorator import cache -- fastapi-cache is not a dependency in any pyproject.toml. Aspirational pattern not implemented.
• Lines 139-143: [STALE] SQLALCHEMY_POOL_SIZE, SQLALCHEMY_MAX_OVERFLOW, SQLALCHEMY_POOL_TIMEOUT -- These env var names are not used anywhere in the codebase.
• Lines 370-378: [WRONG-CMD] tradai benchmark --strategy MyStrategy --days 30 -- No tradai benchmark CLI command exists. Runtime: No such command 'benchmark'.
• Line 385: [MISLEADING] Link to 04-backtest.md works but could be confused since the file is in a different context.

developer-guides/strategy-repo.md¶

• Line 98: [MISLEADING] Tag trigger *-v* vs CLAUDE.md's <strategy>-v* -- Slightly inconsistent glob patterns (though functionally equivalent). All strategy-repo commands are for a separate repo and unverifiable here.

contributing.md¶

• Lines 19-21: [BUG] git clone https://github.com/tradai-bot/tradai.git then cd tradai-uv -- Cloning tradai.git creates directory tradai, not tradai-uv. The cd command will fail.
• Line 13: [MISLEADING] "Python 3.11+ (required)" -- pyproject.toml specifies >=3.11,<3.12 (strictly 3.11.x). "3.11+" implies 3.12+ is acceptable.
• Line 62: [MISLEADING] URL https://github.com/tradai-bot/tradai/blob/main/CLAUDE.md -- repo name tradai vs local dir tradai-uv could confuse readers.
• Lines 188-189: [MISLEADING] uv run pytest -m unit and uv run pytest -m integration -- Project standard says to always use just commands, not raw uv run pytest.
• Lines 195-199: [GAP] Test markers table lists only 3 markers (unit, integration, slow) but the project has 14 registered markers in pyproject.toml.
• Lines 304-314: [WRONG] from tradai.common.entities import DateRange, ExchangeConfig -- DateRange is NOT in tradai.common; it's in tradai.data.core.entities.DateRange. ExchangeConfig is correct.
• Line 321: [GAP] GitHub Issues URL consistent with remote but may not match how users navigate.
• Lines 333-454: [GAP] "OpenCode Development" section has duplicated "Session Management" subsections (lines 397-407 and 419-431). Should be deduplicated.
• Lines 379-393: [MISLEADING] "Background Task Management" code block uses background_task(), background_output() etc. as Python -- these are OpenCode-specific pseudo-code, not runnable Python.

Home + Architecture Docs Findings¶

index.md (Home)¶

• Line 125: [MISLEADING] just setup description says "Install Python 3.11, UV, deps, pre-commit hooks" but UV must already be installed. Should say "Install Python 3.11, deps, pre-commit hooks".

architecture/index.md¶

No issues found.

architecture/overview.md¶

• Lines 12-15: [MISLEADING] Mermaid diagram shows ALB --> Strategy Service and ALB --> Data Collection. Both have empty ALB_PATH_PATTERNS -- they are internal-only services via Service Discovery. Arrows should go from Backend, not ALB.

architecture/02-ARCHITECTURE-OVERVIEW.md¶

No issues found. Service specs, Lambda counts, subnet CIDRs all verified.

architecture/03-VPC-NETWORKING.md¶

• Line 5: [STALE] TL;DR says "9 VPC endpoints" but actual is 11 (2 gateway + 9 interface). Body of doc correctly lists all 11.

architecture/04-SECURITY.md¶

No issues found. Cognito, IAM, S3 encryption, RDS, WAF, CloudTrail all verified.

architecture/05-SERVICES.md¶

No issues found. Service specs, ECR mappings, Lambda table, Docker Compose all verified.

architecture/06-STEP-FUNCTIONS.md¶

• Line 12: [BUG] Cross-reference says "05-SERVICES.md Section 6" but backtest execution modes are in Section 4. Section 6 is "Docker Compose".

architecture/07-COST-ANALYSIS.md¶

• Lines 10, 157-158: [MISLEADING] TL;DR and cost table say "add $28 for 4 interface endpoints" but body (Optimization 2) correctly lists 9 interface endpoints at ~$65.70/month. TL;DR contradicts body.

architecture/09-PULUMI-CODE.md¶

• Lines 428-429: [STALE] Says infra/.env should contain AWS_PROFILE=tradai. The .env.example says AWS profile is set via pulumi config set aws:profile tradai, not env var.

architecture/10-CANONICAL-CONFIG.md¶

• Lines 125-131: [GAP] Service Discovery table lists only 3 services but config.py defines 7. At minimum strategy-service is missing.
• Line 379: [BUG] update-nat-routes Lambda timeout listed as 30s but actual code uses timeout=120.

architecture/11-LIVE-TRADING.md¶

• Line 755: [MISLEADING] Example uses curl for health check but 05-SERVICES.md documents ECS uses python -c "import httpx..." (Alpine lacks curl). Section is marked aspirational.

architecture/12-ML-LIFECYCLE.md¶

No issues found. All 21 source file paths, Lambda handlers, and class references verified.

architecture/adr/index.md¶

• Line 8: [GAP] ADR-002 through ADR-007 listed in table but no .md files exist. Only 001-nat-instance.md exists.

architecture/adr/001-nat-instance.md¶

• Line 34: [MISLEADING] Claims "saving ~$64/month" but code docstring says "~$32/month savings". Different baselines.

design/index.md¶

• Lines 6, 11-13: [WRONG] Claims DESIGN docs are "10k-15k lines each". Actual: tradai-common=756, tradai-data=809, tradai-strategy=660 lines. Off by 10x.

design/tradai-common-design.md¶

• Line 258: [WRONG-PATH] Lists settings.py but actual file is base_settings.py.
• Lines 254-392: [GAP] Module tree omits many files: _optional.py, base_service.py, cli.py, events.py, factories.py, runtime.py, and several aws/ and clients/ files.
• Line 334: [MISLEADING] Says config/catalog.py is "StrategyConfig schema" but StrategyConfig is in config/loader.py.

design/tradai-data-design.md¶

• Lines 17, 127, 141, 503, 601, 725: [STALE] Says "ABC" throughout but actual code uses Protocol. 6 occurrences.
• Lines 466-475: [STALE] Claims "ArcticAdapter (Future)" but it's fully implemented (233+ lines).
• Line 678: [BUG] Constructor example ArcticAdapter(bucket_name="my-bucket", library="futures") -- actual params are bucket and library_name.

design/tradai-strategy-design.md¶

• Lines 96-141: [GAP] Module tree omits checks.py. All other references verified correct.

design/backend-design.md¶

• Line 24: [WRONG] Says BacktestService but actual class is BackendService.
• Line 98: [WRONG] Env var BACKEND_SQS_QUEUE_URL -- actual is BACKEND_BACKTEST_QUEUE_URL.
• Line 99: [WRONG] Env var BACKEND_DYNAMODB_TABLE -- actual is BACKEND_DYNAMODB_TABLE_NAME.

design/data-collection-design.md¶

• Line 35: [STALE] Claims FreshnessCheck entity -- doesn't exist.
• Line 36: [STALE] Claims FreshnessStatus entity -- doesn't exist.
• Line 32: [GAP] Omits /export endpoint.

design/strategy-service-design.md¶

• Line 25: [WRONG] Says StrategyService but actual class is StrategyServiceService.
• Line 86: [WRONG] MLflow URI default listed as http://localhost:5000 but actual is http://localhost:5001.
• Line 87: [WRONG] Env var STRATEGY_SERVICE_STRATEGIES_DIR with default strategies/ -- actual is STRATEGY_SERVICE_STRATEGY_PATH with default "".

design/mlflow-design.md¶

No issues found.

Architecture App-Architecture Docs Findings¶

architecture/13-DATA-FLOW-ARCHITECTURE.md¶

• Line 66: [MISLEADING] Conflates BacktestJobSubmitter Protocol with ExecutorStrategy pattern -- these are two separate abstractions in the codebase.
• Line 167: [WRONG] References LibraryDataCollectionService and LibraryAsyncDataCollectionService -- actual class names are DataCollectionService and AsyncDataCollectionService (no Library prefix).

architecture/14-STATE-MACHINES.md¶

No issues found. All enum states, values, transitions verified.

architecture/15-ERROR-HANDLING.md¶

No issues found. Exception hierarchy and resilience presets verified.

architecture/16-OBSERVABILITY.md¶

No issues found. All class names, file paths, logger format verified.

architecture/17-CONFIGURATION-MANAGEMENT.md¶

No issues found. All paths, classes, entity fields verified.

architecture/18-CI-CD-PIPELINE.md¶

• Line 18: [WRONG] CI schedule shown as "Weekly / Sunday" but actual cron is 0 6 * * 1 = Monday 06:00 UTC.
• Line 86: [STALE] Docs Freshness trigger listed as "scheduled, dispatch" but also triggers on push to main for docs paths.
• Workflow files: [STALE] docker-build.yml and publish-libs.yml contain Bitbucket references in comments (deprecated).

architecture/19-TESTING-STRATEGY.md¶

• Line 107: [WRONG] just test-strict comment says "fails if < 80%" but actual pyproject.toml enforces fail_under = 60.
• Lines 196-199: [WRONG] Claims "80% minimum enforced via --cov-fail-under" but 80% is a roadmap target. Actual enforcement is 60%.

architecture/20-DEPLOYMENT-PIPELINE.md¶

No issues found. All commands, Lambda counts, cross-references verified.

Reference: Libraries + Services Findings¶

reference/index.md¶

• Line 31: [BUG] from tradai.data import OHLCVRepository -- doesn't exist. Actual: DataRepository (Protocol), CCXTRepository.

libs/index.md¶

• Lines 17-26: [MISLEADING] Directory tree implies all libs follow 3-layer pattern (api/core/infrastructure). Only services do. Libs have varied structures.

libs/tradai-common.md¶

No issues found.

libs/tradai-data.md¶

• Line 33: [BUG] ArcticAdapter(bucket_name="my-bucket", library="futures") -- actual params: bucket, library_name.

libs/tradai-strategy.md¶

No issues found.

libs/DESIGN.md¶

No issues found.

services/index.md¶

No issues found.

services/backend.md¶

• Line 195: [BUG] BACKEND_EXECUTOR_MODE default documented as local but actual default is sqs.
• Lines 240-241: [WRONG-PATH] infrastructure/dynamodb_repository.py, infrastructure/sqs_executor.py don't exist. Actual: executor_factory.py, local.py, memory.py, sqs.py.
• Lines 229-245: [GAP] Architecture listing incomplete -- missing many api/ and core/ files.

services/data-collection.md¶

No issues found.

services/strategy-service.md¶

• Lines 24-27: [WRONG-CMD] CLI shown as strategy-service serve (hyphen). Actual entry point: strategy_service (underscore).
• Line 249: [BUG] --cov=strategy_service should be --cov=tradai.strategy_service.
• Line 252: [WRONG-PATH] tests/unit/test_service.py doesn't exist. Actual: test_strategy_service.py or test_core_service.py.
• Lines 260-272: [GAP] Architecture listing severely incomplete -- missing many routes, services, schemas.
• Lines 277-295: [STALE] BacktestHandler described as in strategy-service but it's in tradai.common.entrypoint.handlers. TradingMode.BACKTEST should be ExecutionMode.BACKTEST.

services/mlflow.md¶

• Line 73: [WRONG-PATH] from tradai.common.mlflow_adapter import MLflowAdapter -- actual: from tradai.common.mlflow.adapter import MLflowAdapter.
• Line 91: [MISLEADING] Health URL localhost:5001/mlflow/health -- static prefix only applies behind ALB. From host: localhost:5001/health.

Reference: Lambda Docs Findings¶

reference/lambdas/index.md¶

• Line 10: [BUG] @lambda_handler shown without settings arg. Actual usage: @lambda_handler(SettingsClass).
• Line 38: [BUG] model-rollback trigger listed as "CloudWatch Alarm" but IaC configures it as Step Functions/Direct invocation.
• Line 127: [WRONG-PATH] infra/modules/lambda_funcs.py -> actual: infra/compute/modules/lambda_funcs.py
• Line 148: [WRONG-PATH] infra/modules/iam.py -> actual: IAM policy is in infra/compute/modules/lambda_funcs.py

Lambda timeout/memory mismatches (13 of 17 docs wrong)¶

Lambda schedule mismatches (3 docs wrong)¶

Lambda env var name mismatches (2 docs)¶

• model-rollback.md, notify-completion.md: ALERT_SNS_TOPIC_ARN -> actual: SNS_ALERTS_TOPIC_ARN

Clean Lambda docs (2 of 18)¶

• pulumi-drift-detector.md, update-status.md: No issues.

Reference: SDK + API + Misc Findings¶

reference/sdk/tradai-common.md¶

• Line 270: [BUG] MLflowAdapter(tracking_uri=...) -- actual param is base_url, not tracking_uri.

reference/sdk/tradai-data.md¶

• Lines 229, 303, 434: [STALE] library_name="futures" should be "ohlcv" (3 occurrences).

reference/sdk/tradai-strategy.md¶

• Line 436: [WRONG-CMD] tradai strategy gate doesn't exist. Correct: tradai validate gate.

reference/cli.md¶

• Lines 578-658: [GAP] Entire tradai data command group documented but these commands are in the data-collection service CLI, not aggregated into root tradai CLI. The tradai data subcommand only appears when tradai-data-collection package is installed (lazy-loaded).

reference/environment.md¶

• Line 237: [MISLEADING] Exchange JSON uses "market_type": "futures" but ExchangeConfig field is trading_mode. market_type is silently ignored by Pydantic.

Clean docs (8 of 13)¶

• reference/sdk/index.md, api-docs/index.md, api-docs/common-entities.md, api-docs/data-entities.md, exchange-connectivity.md, arcticdb-data-model.md, api.md, glossary.md: No issues.

Operations: Runbooks Findings¶

runbooks/index.md¶

• Line 34: [WRONG-CMD] SNS topic names wrong: doc says tradai-{env}-alerts but actual is tradai-alerts-{env}. Second topic is tradai-registration-{env} not tradai-{env}-notifications.

runbooks/service-recovery.md¶

• Line 39: [WRONG-PATH] Log group /ecs/tradai-${SERVICE_NAME}-${ENVIRONMENT} wrong. Actual: /ecs/tradai/${ENVIRONMENT}/services with stream prefix.
• Line 230: [MISLEADING] rollback_state table key is model_name not service_name.

runbooks/model-drift-response.md¶

• Lines 128, 136: [MISLEADING] MLflow port 5001 is local dev only, not clarified.

runbooks/trading-issues.md¶

No issues found.

runbooks/infrastructure-issues.md¶

No issues found.

runbooks/database-operations.md¶

• Line 35: [MISLEADING] MultiAZStatus should be MultiAZ for single-instance RDS.
• Line 318: [BUG] get-item missing strategy_name partition key (table has composite key).
• Line 359: [BUG] status-index GSI partition key is strategy_name not status.

runbooks/log-investigation.md¶

• Line 35: [WRONG-PATH] VPC flow logs: /aws/vpc/flowlogs/tradai-{env} -> actual: /aws/vpc/tradai-flow-logs.
• Line 36: [WRONG-PATH] CloudTrail: /aws/cloudtrail/tradai-{env} -> actual: /aws/cloudtrail/tradai.
• Line 33: [GAP] API Gateway log group may not exist (no access logging in infra code).

runbooks/performance-degradation.md¶

• Lines 294, 302: [WRONG-PATH] Strategy log group: /ecs/tradai-strategy-${ENV} -> actual: /ecs/tradai/${ENV}.
• Lines 327, 340: [WRONG-PATH] Data collection log group format wrong. Actual: /ecs/tradai/${ENV}/services with stream prefix.

runbooks/security-incidents.md¶

• Lines 199, 251, 258: [WRONG-PATH] VPC flow logs path wrong (same as log-investigation).
• Line 234: [WRONG-PATH] ECS service log group format wrong.

runbooks/debug-workflows.md¶

• Line 27: [BUG] DynamoDB key job_id should be run_id (workflow-state table partition key).
• Line 65: [BUG] State machine name tradai-${ENV}-backtest -> actual: tradai-backtest-workflow-${ENV}.

Operations: Testing + Troubleshooting Findings¶

testing-in-aws.md¶

No critical issues found.

ai/worktree-testing.md¶

• Line 34/44: [GAP] PYTHONPATH missing $WT/infra/persistent (it's a Python package with pyproject.toml).

troubleshooting/index.md¶

No critical issues found.

troubleshooting/common-issues.md¶

• Line 213: [WRONG-PATH] configs/strategies/ directory doesn't exist.
• Lines 217, 220: [WRONG-CMD] S3 bucket tradai-${ENV}-configs -> actual: tradai-configs-${ENV}.
• Line 235: [WRONG-CMD] tradai strategy validate configs/my-strategy.json -- takes directory/file path, not JSON.
• Line 264: [WRONG-CMD] pip list -> should be uv pip list.
• Line 314: [WRONG-CMD] tradai data sync --symbol BTC/USDT:USDT -> symbols are positional: tradai data sync BTC/USDT:USDT.
• Line 319: [WRONG-CMD] tradai data check doesn't exist -> tradai data check-freshness.
• Line 336: [WRONG-CMD] tradai data symbols -> tradai data list-symbols.
• Line 339: [WRONG-CMD] tradai data info doesn't exist.
• Line 348: [WRONG-CMD] ArcticDB bucket tradai-arcticdb-local -> actual: tradai-arcticdb-dev.
• Line 490: [WRONG-CMD] tradai strategy new --ml freqai MyMLStrategy -> name is positional first: tradai strategy new MyMLStrategy --ml freqai.
• Lines 518-519: [WRONG-CMD] tradai backtest --local / tradai backtest don't exist -> tradai backtest quick <strategy> [--local].

troubleshooting/debug-workflows.md¶

• Lines 108, 111: [WRONG-CMD] S3 bucket tradai-${ENV}-results -> actual: tradai-results-${ENV}.
• Line 205: [MISLEADING] Lambda name should be backtest-consumer not sqs-consumer in backtest context.
• Line 386: [WRONG-CMD] State machine tradai-${ENV}-backtest -> actual: tradai-backtest-workflow-${ENV}.

troubleshooting/service-interactions.md¶

• Lines 126, 173-178: [WRONG-CMD] ArcticDB shown as LMDB backend (lmdb://./user_data/arcticdb) but service uses S3 storage.
• Line 187: [WRONG-CMD] S3 bucket tradai-${ENV}-arcticdb -> actual: tradai-arcticdb-${ENV}.
• Line 249: [WRONG-CMD] Lambda heartbeat-check -> actual: trading-heartbeat-check.
• Lines 311, 320: [WRONG-CMD] S3 bucket name pattern wrong (same as common-issues).
• Line 386: [WRONG-CMD] State machine name wrong (same as debug-workflows).

templates/dryrun-signoff-checklist.md¶

No issues found.

Cross-cutting: S3 bucket naming pattern¶

8 occurrences across 3 docs use tradai-${ENV}-<type> but actual pattern is tradai-<type>-${ENV}.

Overall Summary¶

By Severity¶

Total issues: ~80 across 25 guides (3 guides clean)

Top 5 Most Problematic Docs¶

Cross-Cutting Issues¶

1. Git clone URL: Multiple docs use git clone .../tradai.git then cd tradai -- but local directory is often tradai-uv. Affects: 01-setup.md, docker-setup.md, contributing.md
2. tradai strategy backtest doesn't exist: Referenced in 01-your-first-backtest.md and 04-backtest.md. Should be tradai backtest quick or tradai strategy-service backtest.
3. TA library inconsistency: Strategy examples use pandas_ta, indicator CLI uses talib. Never explained. Affects: 02-create-strategy.md, 03-develop.md, 04-backtest.md
4. strategies/ path assumption: Multiple docs assume strategies live under strategies/ but tradai strategy new outputs to CWD. Affects: 02-create-strategy.md, 03-develop.md, strategy-lifecycle.md, strategy-templates.md
5. ArcticDB library name: Default is ohlcv but docs say futures. Affects: 03-collect-market-data.md (3 occurrences)
6. Pulumi bare commands: Several docs show pulumi preview/pulumi up without specifying which stack directory. Affects: production-checklist.md
7. Non-existent strategy subcommands: promote, rollback referenced in strategy-lifecycle.md but don't exist at all
8. tradai benchmark referenced in performance-tuning.md but doesn't exist
9. tradai strategy new prerequisite: Must run from repo root or set TRADAI_ROOT -- never documented

Runtime Validation Log (Devcontainer)¶

Commands executed inside devcontainer (devcontainer exec --workspace-folder .):