[tests] Misc improvements to the app size tests#25522
Conversation
- Make assertion failure messages context-aware: in CI, mention the build artifact; locally, just suggest WRITE_KNOWN_FAILURES=1. - Gate the PublishPipelineArtifact step on the directory actually existing to avoid noisy 'succeeded with issues' when tests pass. - Fix misleading assertion messages for preserved-API checks: 'No added APIs.' → 'Unexpected APIs were added to the preserved set.' - Fix SKILL.md documentation to show actual artifact naming pattern (including uploadPrefix) and realistic file name examples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR improves the developer experience around the app size unit tests by making failure output more actionable, reducing CI noise from artifact publishing, and aligning documentation with the actual artifact naming/file patterns used by the pipeline and tests.
Changes:
- Gate publishing the “updated expected sizes” pipeline artifact on the output directory existing to avoid noisy “succeeded with issues” when there’s nothing to publish.
- Make AppSizeTest assertion failure hints context-aware (CI vs local) and correct misleading preserved-API assertion messages.
- Update the SKILL.md instructions to match the real artifact naming pattern (including
uploadPrefix) and the current{Platform}-{Variant}-*.txtfile naming.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tools/devops/automation/templates/tests/run-tests.yml | Adds a pre-check step and conditions the artifact publish on the updated-expected-sizes directory existing. |
| tests/dotnet/UnitTests/AppSizeTest.cs | Refines failure messaging for app size + preserved API diffs, and centralizes the “how to update” hint logic. |
| .github/skills/update-expected-app-size/SKILL.md | Updates documentation to reflect actual artifact name pattern and realistic expected file naming examples. |
✅ [PR Build #d424e44] Build passed (Detect API changes) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
✅ [PR Build #d424e44] Build passed (Build packages) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
✅ [PR Build #d424e44] Build passed (Build macOS tests) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
🚀 [CI Build #d424e44] Test results 🚀Test results✅ All tests passed on VSTS: test results. 🎉 All 183 tests passed 🎉 Tests counts✅ cecil: All 1 tests passed. Html Report (VSDrops) Download macOS tests✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
artifact; locally, just suggest WRITE_KNOWN_FAILURES=1.
to avoid noisy 'succeeded with issues' when tests pass.
APIs.' → 'Unexpected APIs were added to the preserved set.'
(including uploadPrefix) and realistic file name examples.