Field note / release integrity
Fail-closed release gates: audit, merge and deploy
A release gate is only meaningful when malformed evidence and failed prerequisites stop later high-impact operations.
The difference between 'clean' and 'unknown'
Security audits can return structured findings, exit nonzero, time out or fail to produce a usable report. Those outcomes are not interchangeable. A gate that catches any JSON parse failure and substitutes zero high-severity advisories may turn 'could not inspect' into 'passed'.
To fail closed, require a well-formed supported audit result, explicit vulnerability counts, and a documented policy for nonzero exits. Return a separately labeled unavailable or invalid state when the report cannot be trusted.
A minimal acceptance matrix
- Valid no high/critical findings: pass only when the output schema is complete and the tool execution is understood.
- High or critical findings: block or follow an explicit controlled exception policy.
- Malformed JSON, unexpected schema or failed audit invocation: block and label the result 'unavailable' rather than clean.
- Missing required lockfile or unavailable dependency graph: block, or use a separately justified equivalent control without claiming the original audit passed.
Publishing is a dependency chain
A successful test step alone does not establish that the intended commit was merged, that the expected branch protection checks ran, or that the artifact being published matches that reviewed commit.
A release orchestrator should verify each git command result, the intended PR/base/commit, the actual merge result, and the immutable identity of the publish or deploy artifact. A skipped or failed merge must not be treated as an optional note if publishing depends on it.
Avoid privileged administrative merge flags in automated paths unless a narrowly documented policy genuinely requires them. Prefer standard checks, reviews and protected-branch rules.
Public LOLM example
At a pinned LOLM source revision, the Python release orchestrator catches an npm audit JSON parsing exception and sets the high/critical count to zero. Its merge helper requests --admin, while a later publishing path can be entered without checking merged.ok.
These are manual Python control-flow findings, not outputs of Qira Proof's workflow-pattern scanner. No actual unsafe publish, bypass of branch protection or production exploit was demonstrated. We documented the separate remediation as public GitHub issue #72.
Read the source-backed example · Review public issue #72Verification tests that matter
- Test invalid JSON and missing metadata independently of genuinely zero results.
- Inject an audit-process failure and assert that publish is not invoked.
- Inject failed git checkout, push or PR merge operations and assert that every later publish/deploy call remains blocked.
- Test a successful run against the exact merged commit, with an auditable receipt recording what actually happened.