Skip to content

check-retraining-needed

Evaluates whether a model requires retraining based on drift detection, scheduled intervals, or manual triggers.

Overview

Property Value
Trigger Step Functions state machine
Runtime Python 3.11
Timeout 30 seconds
Memory 256 MB

Input Schema

{
    "model_name": "PascalStrategy",  # Required
    "force": false,                   # Optional, forces retraining
    "manual_trigger": false           # Optional, marks as manual request
}

Output Schema

{
    "model_name": "PascalStrategy",
    "decision": "needs_retraining",  # needs_retraining | recently_trained | no_retraining | invalid_model
    "trigger": "drift_detected",     # manual | drift_detected | scheduled
    "reason": "Significant drift detected (PSI=0.350)",
    "hours_since_retrain": 168.5,
    "days_since_retrain": 7,
    "drift_severity": "significant",
    "drift_psi": 0.35
}

Environment Variables

Variable Required Default Description
DYNAMODB_TABLE_NAME Yes - State repository table
DRIFT_STATE_TABLE No "tradai-drift-state" Drift state table
RETRAINING_STATE_TABLE No "tradai-retraining-state" Retraining state table
RETRAINING_INTERVAL_DAYS No 7 Days between scheduled retraining
MIN_HOURS_BETWEEN_RETRAINING No 24 Minimum hours between attempts
MODEL_STRATEGY_MAP No "" (guard inactive) JSON {model_name: strategy_class} — the ONE executable Freqtrade class allowed to train each model (#538 RD-1). Its keys are the allowed models; an unknown model, a missing/malformed strategy, or a strategy that isn't the mapped class returns invalid_model on the would-launch paths. Empty string leaves the guard inactive (permits any format-valid model). Supersedes the old flat ALLOWED_MODELS.

Decision Logic

flowchart TD
    A[Start] --> V{model_name format valid?}
    V -->|No| X[INVALID_MODEL]
    V -->|Yes| AL{model in MODEL_STRATEGY_MAP? (or map empty)}
    AL -->|Not in a non-empty map| X
    AL -->|Yes / map empty| B{Force or manual_trigger?}
    B -->|Yes| C[NEEDS_RETRAINING: MANUAL]
    B -->|No| MH{Hours since last retrain?}
    MH -->|< min_hours| I[RECENTLY_TRAINED]
    MH -->|>= min_hours| D{Check drift state}
    D -->|Significant drift| E[NEEDS_RETRAINING: DRIFT_DETECTED]
    D -->|No drift| F{Days since last?}
    F -->|>= interval| H[NEEDS_RETRAINING: SCHEDULED]
    F -->|< interval| J[NO_RETRAINING]

Key Features

  • Input validation: rejects model_name not matching ^[A-Z][A-Za-z0-9_]{1,49}\Z (PascalCase identifier, max 50 chars, \Z blocks trailing newlines)
  • Pairing gate (#538 RD-1): when MODEL_STRATEGY_MAP is set, rejects any model not in the map, and on the would-launch paths any strategy that isn't the mapped executable class — prevents bogus names, and a legit model paired with the wrong class, from wasting Fargate compute
  • Checks drift severity against thresholds
  • Validates minimum interval between retraining attempts
  • Supports forced retraining and manual triggers
  • Returns decision rationale for audit trail

Step Functions Integration

The Lambda feeds the EvaluateRetrainingNeed Choice state, which branches on $.check_result.decision:

Decision Route
needs_retraining RunRetraining (ECS Fargate)
invalid_model HandleInvalidModel → NotifyFailure
recently_trained / no_retraining SkipRetraining

See infra/compute/asl_templates/retraining_workflow.json.j2 for the full ASL definition.