Skip to content

fix(test_helpers): allow optional labels on BUG markers in CompilationTestHelper - #6191

Open
reginaldalfret wants to merge 1 commit into
google:masterfrom
reginaldalfret:oss-fix/error-prone-multi-case-diagnostic
Open

reginaldalfret wants to merge 1 commit into
google:masterfrom
reginaldalfret:oss-fix/error-prone-multi-case-diagnostic

Conversation

@reginaldalfret

Copy link
Copy Markdown

Description

Fixes #6144.

When test cases in \CompilationTestHelper\ have identical test lines (or repeated assertions across multiple test cases in the same input source), test failure output identifies the failure only by line number, e.g.:
\Did not see an error on line 6 matching Exception created but not thrown.\

When diagnosing failures in CI logs, the reader has to open the test file and count lines to determine which specific scenario or method failed.

Changes

  1. Supports an optional label syntax on bug markers:
    • // BUG[my-case-label]: Diagnostic contains: ...\
    • // BUG[my-case-label]: Diagnostic matches: ...\
  2. Formats failing line messages with the label when present:
    • \Did not see an error on line 6 [my-case-label] matching ...\
    • \Did not see an error on line 6 [my-case-label] containing [...]\
    • \No expected error message with key [X] as expected from line 6 [my-case-label] ...\
  3. Backwards compatible: unlabeled markers (// BUG: Diagnostic contains:) format as \line 6, identical to existing behavior.
  4. Added unit tests in \CompilationTestHelperTest.

Signed-off-by: reginaldalfret [email protected]

…nTestHelper

In test files using CompilationTestHelper, multiple test cases with identical assertions (such as identical marked lines or repeated markers across test cases) currently report only the line number on failure, requiring the reader to manually open the file and count lines to determine which test case broke.

Support an optional label in bug markers, e.g. \// BUG[label]: Diagnostic contains: ...\ or \// BUG[label]: Diagnostic matches: ...\, and include the label alongside the line number in failure reports (\line X [label]\), while preserving full backwards compatibility for unlabeled markers.

Fixes google#6144

Signed-off-by: reginaldalfret <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CompilationTestHelper: a failure cannot say which of several identical cases lost its diagnostic

1 participant