Dependency Management¶

How dependency updates and CVE suppressions work in this repo, and what you must do by hand. Read this before adding anything to .pip-audit-ignore.txt.

The three moving parts¶

Why the CVE floors need a custom manager¶

constraint-dependencies forces a minimum version on a transitive dependency without adding it as a real dependency. It is the mechanism that keeps Security Scan green when some package deep in the tree ships a CVE fix.

No dependency bot reads that field. Renovate's pep621 manager covers project.dependencies, optional-dependencies, dependency-groups, tool.uv.sources and tool.uv.dev-dependencies — and nothing else. Dependabot has no equivalent mechanism at all. So renovate.json carries a customManagers regex entry that reads the array directly.

It uses matchStringsStrategy: "recursive" — stage 1 anchors to the constraint-dependencies array, stage 2 matches each entry inside it. This matters: a flat file-wide regex matches 56 entries instead of 11, because it also captures everything in [dependency-groups] and double-manages it.

What actually catches a CVE¶

Three independent sources, deliberately overlapping:

1. lockFileMaintenance (at any time, so effectively the hourly workflow cron) — re-resolves uv.lock. This is the one that catches transitive drift, which is what usually breaks Security Scan. A fixed transitive lands before pip-audit ever trips on it. It is exempt from both PR throttles (prConcurrentLimit/prHourlyLimit are 0 for this lane only) — sharing the human-review pool is what deadlocked it once already. It is no longer the only lock regenerator: since #1365 a CVE-floor bump re-locks itself via postUpgradeTasks.
2. GitHub Dependabot alerts → Renovate's vulnerabilityAlerts. Vulnerability PRs skip the queue — Renovate raises them ahead of routine updates and exempt from the concurrency and hourly limits (a built-in behavior; you do not configure the limits for them). But Renovate is not a daemon: it only acts when the workflow cron fires, so the practical latency to detect and open a CVE PR is one cron interval — ~1h with the hourly cron (0 * * * *), not instantaneous. Requires Dependabot alerts to be enabled on the repo and the App to hold Dependabot alerts: Read — without either it silently does nothing.
3. pip-audit in the Security Scan CI job — the merge gate. This is the authoritative one: a PR does not merge if it fails.

When they disagree, investigate. Do not suppress to make them agree.

Adding a .pip-audit-ignore.txt entry¶

Only when there is genuinely no fix available. Every entry must carry:

• Which package and version.
• Why the fix is not reachable (upstream cap, no release yet, disputed advisory).
• Why the vulnerable code path is not exercised here, if that is the argument.
• A re-check date.

Suppressions rot, and a stale one is worse than no suppression — it permanently masks that CVE class, so a real regression goes unnoticed. An August 2026 audit found five entries already fixed in the locked versions plus one duplicate, and two of them had open issues tracking work that was already done.

Before trusting an entry's inline comment, check the lock. The comments record what was true when written, and a later floor bump can silently supersede them. Verify with the locked version:

# What is actually locked?
grep -A2 '^name = "<package>"' uv.lock

# Is that version still affected?
curl -s https://api.osv.dev/v1/query \
  -d '{"package":{"name":"<package>","ecosystem":"PyPI"},"version":"<locked>"}'

If the locked version is at or above the fix, delete the entry.

Renovate¶

Self-hosted, in .github/workflows/renovate.yml. No automerge — it opens PRs, humans merge.

just renovate-validate     # same check CI runs

Running it manually¶

Actions → Renovate → Run workflow. dryRun is a choice, not a boolean. The options are live / extract / full: live is the sentinel for a real run (the workflow maps it to an empty RENOVATE_DRY_RUN, so it really opens PRs), extract stops after dependency extraction, and full does a complete run but writes nothing. (Renovate itself also accepts a lookup value; it is deliberately not offered here.) Set logLevel: debug when diagnosing why something was or wasn't picked up.

Ownership¶

Weekly lockFileMaintenance and CVE-floor PRs are reviewed by Alexander. There is no CODEOWNERS file, so nothing assigns them automatically — do not let the queue go unowned.

Things that fail silently¶

Everything below fails by doing nothing, with a green job. None of them turn anything red, so verify each positively rather than assuming success:

GitHub App permissions¶

The self-hosted run mints an App token (see renovate.yml), so Renovate acts with the App's permissions, not GITHUB_TOKEN's. An under-scoped App fails by doing nothing on the missing surface, with no error. The full matrix Renovate needs for what this repo configures:

The App creation, install scope, and the two secrets (RENOVATE_APP_ID, RENOVATE_APP_PRIVATE_KEY) are the manual owner steps tracked in #1101.

What Renovate does not track¶

The 23 lambda Dockerfiles use ARG BASE_IMAGE=tradai-lambda-base:latest with FROM ${REGISTRY:+${REGISTRY}/}${BASE_IMAGE}. That is intentional and should stay that way:

• It is a first-party artifact built by deploy-lambdas.yml from this repo's own source — there is no upstream to poll, and bumping it would be circular.
• The tag has no version ordering (dev-<sha7>, manual-<timestamp>, or a branch name), so "newer" is undefined.
• The FROM line is not statically resolvable — the full image reference is assembled by BuildKit at build time and does not exist in the file.

All 23 inherit from lambdas/base/Dockerfile, whose FROM public.ecr.aws/lambda/python:3.11 is tracked — so the whole fleet is covered through that one chokepoint.

ghcr.io/mlflow/mlflow is in ignoreDeps for now: the server image (v3.11.1) and the Python lib (3.13.0) have already drifted, and letting Renovate move the server image independently would change the thing under investigation. Remove it once that is resolved.

Dependency bumps and the version-bump guard¶

Since #1224, version-bump-guard treats a publishable package's [project] table — dependencies included — as artifact-affecting: change it and the same PR must increment that package's version. It is a required check, so this is not advisory.

This does not bite the CVE floors, and the distinction matters because the two look alike. A floor lives in the root [tool.uv] constraint-dependencies (see the table at the top of this page), and the root is not a publishable package — so the guard never looks at it. What it bites is the pep621 manager's PRs, the ones editing project.dependencies in libs/*, services/* or cli/.

Renovate cannot decide a release version on its own — it knows how to move a dependency constraint, nothing more. Since #1365 it does not have to: a postUpgradeTasks command runs scripts/dev/bump-changed-packages.py (which delegates the "did the artifact change?" decision to the guard itself, so the two cannot disagree) and then uv lock, in that order — the bump invalidates the lock, so re-locking must come second. A pep621 PR against a publishable package now arrives with its version already moved.

Human PRs still need the bump made: run just bump-changed when version-bump-guard fails you, rather than hand-editing each pyproject.

Read the guard's own measurement carefully. Replaying a year of history, its docstring records that 58 of 62 artifact-affecting pyproject.toml commits would newly fail — but that is a one-off backlog, not the steady state: 22 of the 58 were edits adding a test tool to a dev extra, and the migration to [dependency-groups] (which PEP 735 keeps out of the wheel, and which the guard ignores) has since removed that source from five of the seven packages.

Two things that are not affected, so you do not go looking for a bump that is not required:

• Root [tool.uv].constraint-dependencies — out of scope, because the root is not a publishable package. This is where the CVE floors live, which is exactly why the floor PRs are not the ones this blocks.
• Lockfile-only changes — uv.lock is not an artifact input.

The Renovate half of #1354 shipped in #1365 (its recommended option D), and just bump-changed is option C. Options A (release-please) and B (CI auto-bump on push) were considered and not recommended in that issue — they are not outstanding work.

Automerge¶

Low-risk updates auto-merge once the full CI suite passes; everything else waits for a human. This is a positive allowlist in renovate.json — never a denylist (Renovate's addLabels is additive, so a denylist would leak the automerge label onto excluded deps).

Auto-merged (minor/patch/pin/digest only): github-actions (non-major, SHA-pinned), dev/CI/docs tooling by explicit package name (ruff, mypy, pytest*, pre-commit, mkdocs*, …), npm devDependencies, and lockFileMaintenance.

Always manual: all runtime/data/ML/exchange libraries (numpy, pandas, ccxt, freqtrade, pydantic, sqlalchemy, …), the CVE floors, mlflow, docker base images, and every major. A ~60% coverage suite cannot vouch for numeric/behavioral drift in the runtime libs, so they never auto-merge. Widen the allowlist by name, deliberately — never invert it to a denylist.

How Renovate is allowed to merge without a human — the ruleset bypass¶

dev requires 1 approving review, and Renovate cannot approve its own PRs. Rather than a repo-wide "Actions can approve PRs" loosening or a fake auto-approval, the approval requirement lives in a repository ruleset that the tradai-renovate App is exempt from, while the required status checks stay in classic branch protection with no bypass. GitHub evaluates the two layers as a union, and a ruleset bypass exempts the actor only from that ruleset's rules — so Renovate skips the approval but still cannot merge until CI is green.

The ruleset is the committed source of truth: .github/rulesets/dev-require-approval.json.

Rollout runbook (order matters)¶

⚠️ Known risk: GitHub's computed mergeable_state/reviewDecision does not account for ruleset-bypass eligibility — it stays BLOCKED for a bypass-listed actor even though that actor's raw merge call succeeds. Renovate gates on that field, so it may see BLOCKED and skip. Prove it works before weakening classic protection.

1. Apply the ruleset (classic still requires 1 approval — nothing changes for humans yet):

   gh api repos/tradai-bot/tradai/rulesets --method POST \
     --input .github/rulesets/dev-require-approval.json
   

2. Isolation proof. On the first green, allowlisted Renovate PR, confirm Renovate merges it with no approval. Equivalent raw check with an app installation token: PUT /repos/tradai-bot/tradai/pulls/{n}/merge returns 200 even though gh pr view {n} --json mergeStateStatus shows BLOCKED.
3. Kill-switch: if a green allowlisted PR has not merged within 2 Renovate cron cycles (~2h), treat the bypass as non-functional — do not proceed to step 3; use the fallback below.
4. Only on success — zero the classic approval count (keep the require-PR block so the app still cannot push directly to dev):

   gh api repos/tradai-bot/tradai/branches/dev/protection/required_pull_request_reviews \
     --method PATCH -F required_approving_review_count=0
   

   Humans now satisfy the review via the ruleset (no app bypass); Renovate bypasses it; both are still gated by the classic required checks.

Fallback (if the isolation proof fails): a narrowly-scoped auto-approve — a separate minimal approver identity (second GitHub App or bot PAT with only pull-requests: write) approves labeled Renovate PRs via a guarded workflow (trigger on labeled/opened/synchronize, guard on user.login == 'tradai-renovate[bot]' and the automerge label). This records a real approval, so mergeable_state clears and Renovate merges normally. Classic keeps its 1 approval; no repo-wide loosening. Do this instead of step 3.

Rollback (never drop human review)¶

Restore classic protection first, then remove the ruleset:

gh api repos/tradai-bot/tradai/branches/dev/protection/required_pull_request_reviews \
  --method PATCH -F required_approving_review_count=1     # restore human gate FIRST
gh api repos/tradai-bot/tradai/rulesets/<id> --method DELETE   # then drop the ruleset