Improve test case failure/skip states and their output (#4116)#4129
Merged
gcampbell-msft merged 4 commits intomicrosoft:mainfrom Nov 11, 2024
Merged
Improve test case failure/skip states and their output (#4116)#4129gcampbell-msft merged 4 commits intomicrosoft:mainfrom
gcampbell-msft merged 4 commits intomicrosoft:mainfrom
Conversation
8b2e432 to
da93bcf
Compare
Contributor
Author
|
@microsoft-github-policy-service agree company="KDAB" |
gcampbell-msft
requested changes
Oct 16, 2024
Collaborator
gcampbell-msft
left a comment
There was a problem hiding this comment.
@redstrate This looks great!
If you could please make a CHANGELOG entry, then we can approve and accept this PR! Thanks.
da93bcf to
a97b4f0
Compare
Contributor
Author
Done! |
Currently if a test fails, it's not immediately clear why that is because we give the exit code - and not the output from ctest. That is now fixed and the ctest output is shown which greatly improves the usefulness of the Test Results pane and other places where this output is shown like Code Lens. Skipped test cases were also not actually marked as skipped, and that is now fixed as well.
a97b4f0 to
f1da5cf
Compare
gcampbell-msft
approved these changes
Oct 17, 2024
gcampbell-msft
requested changes
Oct 21, 2024
Collaborator
gcampbell-msft
left a comment
There was a problem hiding this comment.
See other comments.
gcampbell-msft
approved these changes
Nov 8, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change addresses item #4116
This changes visible test output
The following changes are proposed:
The purpose of this change
Currently if a test fails, it's not immediately clear why that is because we give the exit code - and not the output from ctest. That is now fixed and the ctest output is shown which greatly improves the usefulness of the Test Results pane and other places where this output is shown like Code Lens.
Skipped test cases were also not actually marked as skipped, and that is now fixed as well.
Other Notes/Information
Comparison screenshots to show the improvement:
Failed Tests
Skipped Tests
Unfortunately this google test example I used doesn't set test source location, but if it did then code lens would be affected as well like in this outdated screenshot from the issue: