Your build passed. The scanner found issues.
Most teams add a security scanner to CI and call it done. Then a critical vulnerability ships to production because the scanner ran in report-only mode with exit code 0. The build passed. The finding was a warning. Nobody noticed. This guide covers how to close that gap — from exit-code gating to posture-score thresholds.
Monitoring vs enforcement: the two-step mistake
When you first add a scanner to a CI pipeline, the temptation is to run it in observation mode — collect findings, publish a report, then decide later which ones to enforce. This is the right first step. The mistake is never moving to the second step.
A scanner running with exit-code: '0' (or its equivalent) always returns success, regardless of what it found. The build turns green. The PR merges. The findings land in a dashboard that nobody checks until something breaks in production.
Enforcement means the scanner returns a non-zero exit code when findings at or above a specified severity are present. The CI runner interprets that as a failed step. The build stops. The PR cannot merge until the issue is resolved or explicitly accepted.
- name: Scan image
uses: aquasecurity/trivy-action@master
with:
scan-type: 'image'
image-ref: 'myapp:latest'
severity: 'CRITICAL,HIGH'
exit-code: '0' # ← never fails- name: Scan image
uses: aquasecurity/trivy-action@master
with:
scan-type: 'image'
image-ref: 'myapp:latest'
severity: 'CRITICAL,HIGH'
exit-code: '1' # ← fails on findingsThe difference is one character. Setting exit-code: '1' in the Trivy GitHub Action causes the step to fail — and therefore block the workflow — any time a finding at the configured severity is present. The exit-code: '0' default is intentional: Trivy ships conservative defaults so it never breaks a pipeline before a team has had time to triage existing findings. But leaving it at 0 permanently defeats the purpose.
Choosing a severity threshold that teams actually respect
The goal is a threshold that catches real risk without generating enough noise to train engineers to ignore it. A policy that blocks every medium-severity CVE in a typical Node.js app will produce dozens of daily failures — most of them unfixed upstream. Teams respond by disabling the gate or adding blanket suppressions, which is worse than no gate at all.
A practical starting point: gate on CRITICAL only first. Most teams have a backlog of HIGH-severity findings in their existing images that have no fix available yet — blocking on HIGH on day one means an immediate CI failure storm and the first team bypass within a week. Start with CRITICAL, clear the backlog, then expand the gate to HIGH. This is an incremental ratchet, not a cliff edge.
| Severity | Action | Rationale |
|---|---|---|
| CRITICAL | Block merge immediately | Exploitable, public PoC exists or EPSS > 0.5; ship = compromise |
| HIGH | Block merge (once baseline is clean) | Significant exploitability; block after resolving existing inventory |
| MEDIUM | Log + ticket; do not block | Often theoretical; blocking creates noise that trains bypass habits |
| LOW / UNKNOWN | Log only | Track for awareness; no pipeline impact |
One additional filter: set ignore-unfixed: true in tools that support it. This skips findings where no patched version exists yet — findings you cannot resolve regardless of effort. They should be tracked separately and re-evaluated when a fix ships, but blocking a deployment on an unfixed vulnerability achieves nothing except friction.
Where CVE gates fall short
A CVE-only gate covers the image layer — known vulnerabilities in OS packages and language dependencies that have been assigned a CVE number and a severity score. It does not cover:
- Host and kernel configuration — a container running as root with
--privileged, a Docker daemon API exposed on port 2375, or a host with automatic updates disabled does not have a CVE. It has a misconfiguration. A misconfiguration that gives an attacker the same access as a CRITICAL CVE. - Internet exposure — a container with a CVE-free image bound to
0.0.0.0is reachable from the internet. A container with a critical CVE bound to127.0.0.1is not. Exposure context is not in the CVE database. - AI agent and MCP server configuration — an MCP server running with auto-approval enabled, a plaintext API key in
.mcp.json, or an unvetted MCP server that has not been audited before being installed will not appear in any vulnerability feed. These are posture findings, not CVEs.
A CVE gate is necessary. It is not sufficient. A server that passes Trivy with zero CRITICAL findings can still be one misconfigured port binding away from full compromise.
The posture-score gate: a threshold the whole box must pass
Keelix adds a second gate layer built around the whole-box posture score — a 0–100 number that aggregates host configuration, container security settings, internet exposure, and AI/MCP agent posture alongside CVE counts. The gate is an exit-code check: if the score falls below a configured threshold, the process exits non-zero and the CI step fails.
This is the pattern described as the monitoring-vs-enforcement two-step: run Keelix in audit mode first to establish a baseline score for your server, then set the threshold just below that baseline and gate on it. The threshold rises incrementally as the team resolves findings — continuous improvement, not a cliff edge on day one.
- name: Keelix security gate
uses: jakelamon/keelix@v0.1.0
with:
fail-on: critical # block on any CRITICAL finding
min-score: 70 # block if posture score < 70
# score covers host config, containers, exposure + AI/MCPThe fail-on: critical flag mirrors the CVE-gate pattern — any CRITICAL finding fails the step. The min-score: 70 adds the posture dimension: a server with no critical CVEs but a collapsed score (exposed Docker API, privileged containers, auto-approved MCP servers) will still fail the gate. Raise the threshold as your team ships fixes.
One class of finding image-only gates will never catch
An MCP server configured for auto-approval, a plaintext secret in the agent config file, or a container running as root are not vulnerabilities in the CVE sense — they carry no CVE ID, no CVSS score, and no NVD entry. They will not appear in Trivy, Grype, or Docker Scout results. They will still compromise your server.
The posture gate catches them because it operates at the level of running configuration, not image content. The Keelix scan runs on the box itself, inspects the live state — which MCP servers are running, what permissions they have, what secrets are stored unencrypted — and rolls that into the posture score. When the gate fires on a score below threshold, the reason might be a CVE. It might also be a misconfigured AI agent that no image scanner would ever find.
A recommended two-gate pipeline
Image CVE gate (PR / pre-merge)
Run a CVE scanner (Trivy, Grype) with exit-code: 1 on CRITICAL severity. Fast feedback to the developer — catches known vulnerabilities in packages before they land on main. Add ignore-unfixed: true to avoid noise from unresolvable findings.
Posture gate (pre-deploy / pre-push to registry)
Run keelix scan with a minimum-score threshold on the target environment or a staging replica. This gate catches the configuration and exposure findings that image scanners miss: exposed ports, privileged containers, MCP auto-approval, plaintext secrets in agent configs.
Set the threshold incrementally
Run keelix scan on your server today and record the baseline score. Set min-score 5-10 points below that baseline as the initial gate. Fix a batch of findings, raise the threshold, repeat. A cliff-edge gate on day one causes bypasses; a ratchet builds genuine improvement.
Track medium/low separately
Pipe medium and low findings to a ticket tracker or Keelix Cloud logbook. Do not block the build on them. The goal is visibility without friction — medium findings you cannot fix today should be tracked, not silently dropped and not used as a blocking condition that trains engineers to suppress alerts.
Quick reference: exit codes
Every security scanner that supports CI gating uses the same Unix convention: exit code 0 = success (no findings at threshold); exit code 1 = findings found (fail the step); exit code 2 = scan error (configuration, network, or auth failure — also fails the step, for different reasons). Your CI runner — GitHub Actions, GitLab CI, CircleCI, or any POSIX shell — interprets any non-zero exit code as a failed step and halts the pipeline at that point.
If your scanner is returning exit code 0 every time, either there are genuinely no findings at the configured severity, or you have explicitly configured it to always return 0 (the Trivy default). Check the exit-code / exit-on-severity /fail-build parameter for your specific tool. The parameter name varies; the behaviour is universal.
Related
- Keelix GitHub Action — YAML reference, inputs, and gate patterns
- MCP Security — grade the AI agents and MCP servers on your box
- Keelix vs Trivy — CVE scanning vs whole-box posture
- Harden Docker — the configuration checklist CI catches with Keelix
- Compliance gate in CI — CIS/SOC 2/ISO checks as a GitHub Action
Gate on the whole box, not just the image.
Free, local-first, Apache-2.0. Run keelix scan today to get your baseline score — then set the threshold and let CI enforce it.