Base-vs-head CI differ: which PR failures are introduced, pre-existing, external, or fixed.
Which of these CI failures are actually yours?
A red PR with 12 failing checks usually has one real problem. The rest were already failing on the base branch, belong to a deploy preview service, or are waiting on a maintainer to approve the workflow run. Today you find that out by clicking through every check twice, once on the PR and once on the base, and comparing by eye.
cidiff does that comparison for you. It reads the check runs and commit statuses on the PR head commit and on its base commit, and classifies every check:
| category | meaning |
|---|---|
introduced | Passing on base, failing on head. This one is the PR's to fix |
pre-existing | Failing on both head and base with the same check name |
external | A hosted or deploy surface (Vercel, Netlify, Cloudflare, and friends, matched by check name), or a check waiting on maintainer approval (action_required) |
fixed | Failing on base, passing on head, or gone while other head checks ran |
passing | Passing on head |
indeterminate | Failing, but there is no usable base evidence to prove it either way |
pending | The head run has not finished |
Exit code 1 when introduced failures exist, 0 otherwise, so it drops straight into a CI gate or a maintainer's triage script.
cidiff was extracted from the CI triage inside Greenlight, where it classified 89 failing checks across 46 open PRs in one sweep and found zero introduced. Getting that number right depends on refusing to guess:
action_required, waiting for a maintainer to approve the workflow, so the head commit had zero check runs. A naive differ reported every base failure as fixed and every base pass as passing. cidiff detects the empty head, reads the check suites, and says in plain warnings: CI has not run on this head, these failures cannot be classified yet.# Classify a PR's checks against its base
npx @reneturcios/cidiff pr usebruno/bruno 9569
# Machine readable output
npx @reneturcios/cidiff pr owner/repo 123 --json
# Compare two commits directly
npx @reneturcios/cidiff runs owner/repo --head <sha> --base <sha>
Example output (the fork PR case above, real output):
cidiff usebruno/bruno PR #9569: 0 introduced, 0 pre-existing, 0 external, 0 fixed, 6 indeterminate, 0 pending, 2 passing
[WARNING] No check runs on head commit 885c6fafed1e (only 2 commit status(es)). The workflow system has produced no evidence to diff against; base check failures below are indeterminate, not fixed.
[WARNING] 3 check suite(s) on the head commit are action_required: CI is waiting on maintainer approval to run (typical for fork PRs). The failures cannot be classified until CI actually runs.
[INDETERMINATE] Unit Tests (Linux): Failing on base, and the head commit has no check runs, so there is nothing to compare. CI may not have run on the head yet
[PASSING] CodeRabbit, security/snyk (Bruno)
Verdict: no introduced failures. Any failing checks are pre-existing, external, or unproven either way (indeterminate).
Known limit, stated plainly: the base comparison uses the check evidence recorded for the PR's base commit (the base branch tip stored in the PR object), and checks are matched by exact name. A renamed check looks like a removal plus an addition. Both limits are visible in the output rather than hidden.
Set GITHUB_TOKEN for the higher authenticated GitHub API rate limit. No other credentials are needed for public repos.
Add to your MCP client config:
{
"mcpServers": {
"cidiff": {
"command": "npx",
"args": ["-y", "@reneturcios/cidiff", "mcp"]
}
}
}
Tools:
diff_pr_ci: repo plus PR number, returns the full classified report as JSON.diff_ci_runs: repo plus head and base SHAs, same classification without a PR.docker run -i ghcr.io/cloudcorprecords/cidiff mcp
docker run --rm ghcr.io/cloudcorprecords/cidiff pr owner/repo 123
cidiff is Wave 1, package 2 of a planned family of small, single-purpose developer tools by the same author, all built the same way: one bin that serves a CLI and an MCP server, deterministic checks with evidence, no invented answers. Its sibling shipcheck verifies finished releases, and it verifies cidiff's own releases in return.
npm install
npm run build
npm test # 19 tests, classification logic runs on fixtures, no network
MIT
Source-derived launch command. Check the maintainer’s required arguments and credentials before running:
npx -y @reneturcios/cidiffMerge this template into ~/Library/Application Support/Claude/claude_desktop_config.json. Keep existing servers. Add any arguments, credentials, and permissions required by the maintainer; this template has not been install-tested.
{
"mcpServers": {
"io-github-cloudcorprecords-cidiff": {
"command": "npx",
"args": [
"-y",
"@reneturcios/cidiff"
]
}
}
}Restart Claude Desktop completely for changes to take effect. Confirm the server appears connected in the client’s tool list, then try a read-only example from its documentation.
Claude Desktop setup referenceio.github.CloudCorpRecords/cidiff works with any MCP-compatible client. Copy the config snippet from the Configuration section above and add it to the file shown for your client, then restart the application.
~/Library/Application Support/Claude/claude_desktop_config.jsonRestart Claude Desktop completely for changes to take effect.~/.cursor/mcp.jsonRestart Cursor for changes to take effect..vscode/mcp.jsonReload VS Code window for changes to take effect.~/.codeium/windsurf/mcp_config.jsonRestart Windsurf for changes to take effect..mcp.jsonSave at the project root, then start Claude Code in that project and review the MCP server approval prompt. Keep real credentials out of shared files.