fix(ci): gate changed-line coverage with diff-cover#6706
fix(ci): gate changed-line coverage with diff-cover#6706bladehan1 wants to merge 1 commit intotronprotocol:developfrom
Conversation
- Use diff-cover changed-line coverage as the changed-code gate; keep the overall coverage delta gate at -0.1% and JaCoCo reports as summary-only snapshots. - Handle non-Java PRs: when diff-cover reports no changed Java lines (workflow-only, docs-only, proto-only), emit NA and treat it as SKIPPED in the enforcement step so the gate does not wrongly fail. - Harden the diff-cover step: guard against empty SRC_ROOTS before invoking diff-cover, verify diff-cover.json was written, and surface exit codes in failure messages. - Clarify metric semantics in the step summary: Changed-line is LINE coverage (diff-cover) while Overall / Delta are INSTRUCTION coverage (jacoco-report).
|
The changed-line LINE direction is solid — much closer to industry-standard patch coverage than the old whole-file INSTRUCTION measure. Small framing nit on the design notes: "Gate thresholds preserved ... One follow-up worth flagging alongside the planned |
|
Thanks for the careful read. On the framing nit — agreed, calling On the method-level diff suggestion — added to Follow up. Aggregate delta is opaque about which methods/classes drove a regression. Comparing per-method |
What does this PR do?
Replace the
coverage-changed-filesgate (whole-file INSTRUCTION coverage) with a true changed-line LINE coverage gate powered bydiff-cover, and make the gate robust against non-Java PRs and tool failures.Why are these changes required?
madrapps/jacoco-report'scoverage-changed-filesoutput, which is the whole-file INSTRUCTION coverage of each changed file — not the coverage of the lines the PR actually changed.diff-coverprovides the industry-standard "changed-line" measurement.jacoco-report's default PR comment needspull-requests: write, which fork PRs do not get. Job logs kept filling up with 403 errors.This PR has been tested by:
pr-build.ymlis exercised end-to-end on every PR's own Coverage Gate run rather than through a shell unit-test framework.Changed-line Coverage: NA,Changed-line Gate: SKIPPED (no changed Java source lines),Overall Delta Gate: PASS. Gate passes end-to-end.Changed-line Coverage: 100%,Changed-line Gate: PASS (> 60%).Changed-line Coverage: 0%,Changed-line Gate: FAIL (<= 60%). The Coverage Gate job exited 1 as expected (the uncovered helper also dragged Overall Delta below the-0.1%threshold, so both gates were exercised in this run).Follow up
Overallto LINE oncemadrapps/jacoco-report1.8 shipscoverage-type, or by self-parsing the JaCoCo XML. Unifies the metrics panel on one counter.bats) for the enforce step, so the three-state logic (NA/ numeric / error) is covered without needing a real CI run.comment-type: summaryif reviewers prefer on-PR comment visibility; restoring comments requires grantingpull-requests: writeon thecoverage-gatejob.Extra details
coverage-basejob runs independently ofdocker-build-debian11(they share noneeds:).docker-build-debian11is already in the ~16-minute critical-path band (alongsidebuild-ubuntu24andbuild-rockylinux, all of which includetestWithRocksDb); adding a parallelcoverage-baseof similar length does not extend the critical path. The gate job waits for both, so total PR wall-clock is unchanged.Changed-line Coverageuses LINE coverage (diff-cover), whilePR Overall Coverage/Base Overall Coverage/Deltause INSTRUCTION coverage.madrapps/jacoco-reportv1.7.2 is the latest release and hard-codes thecoverage-overalloutput to INSTRUCTION; there is nocoverage-typeinput yet — see the openmadrapps/jacoco-report#82on milestone1.8. The METRICS panel and step summary now print an explicit note so readers do not accidentally compare the two counters.diff-cover --src-rootsis limited to*/src/main/java. PRs touching onlysrc/test/java,build/generated(proto output), docs, or CI will be reported asSKIPPED. In practice proto-only PRs with non-trivial generated-code coverage shifts are rare in this repository; we accept the gap.MIN_CHANGED=60is kept despite the metric switch. The threshold is reused for the new LINE-based changed-line gate without re-tuning. The Overall Delta gate is not modified by this PR; its-0.1%threshold is retained simply because that path is structurally untouched.exit 1branches only fire in genuine error states:SRC_ROOTSis empty (unexpected workspace shape) ordiff-cover.jsonis missing (tool crash). Neither can be triggered by ordinary PR content.