Remove autoscan ITs and add withoutSemantic tests

[romainbrenguier]

Summary

• Remove the its/autoscan/ integration test module (test classes, pom.xml, 261 diff JSON files, CI job)
• Add test_without_semantic() unit tests to ~575 check test files to replace the autoscan coverage

The autoscan ITs validated that the Java analyzer works without bytecode by comparing results with/without compiled binaries using SonarQube Orchestrator. This is slow and heavyweight. The same coverage is achieved by adding withoutSemantic() unit tests to each rule's check test, which is faster, more granular, and easier to maintain.

Current state (WIP)

Done

• Removed its/autoscan/ directory entirely
• Removed autoscan module from its/pom.xml
• Removed autoscan CI job from .github/workflows/build.yml and from promote job's needs
• Removed Autoscan Test documentation section from README.md
• Added test_without_semantic() to ~575 check test files
• Compilation passes (mvn test-compile -pl java-checks)

Remaining work

• Run full test suite (mvn test -pl java-checks) and fix remaining failures
  • Some tests may need verifyIssues() changed to verifyNoIssues() (or vice versa)
  • A few tests may need their check expression or file path adjusted
• Manually add test_without_semantic() to skipped complex tests:
  • MissingPackageInfoCheckTest (uses onFiles() plural + caching)
  • MockitoAnnotatedObjectsShouldBeInitializedCheckTest (uses testCodeSourcesPathInModule + classpath)
  • AbstractRegexCheckTest (uses local inner class as check)
• Verify no regressions in other modules

Test plan

• mvn test -pl java-checks passes
• mvn validate -pl its/autoscan fails (module removed)
• CI workflow YAML is valid
• No other modules affected

🤖 Generated with Claude Code

[gitar-bot]

⚠️ Edge Case: verifyIssues() + withoutSemantic() reuses semantic sample files

Many new test_without_semantic() methods call verifyIssues() while pointing at the same sample file whose // Noncompliant comments were authored for a full semantic analysis (e.g. AccessibilityChangeCheckTest, MathClampMethodsCheckTest, InstanceOfPatternMatchingCheckTest). withoutSemantic() disables type resolution, so any check that relies on semantics will raise a different (usually smaller) issue set and verifyIssues() will fail because the raised issues no longer match the expected comments. This is the bulk of the acknowledged remaining work; for semantic-dependent rules the intent (verify the check doesn't crash without bytecode) is better expressed with verifyNoIssues(), and only syntax-only rules should keep verifyIssues(). Each affected test needs to be triaged individually.

_{Was this helpful? React with 👍 / 👎}

[gitar-bot]

💡 Quality: Stray double blank line before test_without_semantic in many files

A large number of the modified test files introduce two consecutive blank lines between the previous method's closing brace and the new @test method (e.g. AccessibilityChangeCheckTest lines 53-54, BufferedReaderBoilerplateCheckTest, CommentsMustStartWithCorrectNumberOfSlashesCheckTest, ControlCharacterInLiteralCheckTest, DefaultEncodingUsageCheckTest, ImportDeclarationOrderCheckTest 254-255, ReadlnWithPromptCheckTest, StringIndexOfRangesCheckTest, etc.). This is a formatting inconsistency across ~575 files; collapse to a single blank line for consistency.

Use a single blank line between methods.:

    .verifyIssues();
}

@Test
void test_without_semantic() {

• Apply fix

_{Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎}

[gitar-bot]

💡 Quality: Import of org.junit.jupiter.api.Test added out of order

In DefaultEncodingUsageCheckTest the new import org.junit.jupiter.api.Test; is appended after the org.sonar.java... imports instead of being grouped with the other org.junit.jupiter imports, breaking alphabetical import ordering (a convention this very project enforces via ImportDeclarationOrderCheck). Move the import up next to the other junit imports.

Group the junit Test import with the other junit imports in alphabetical order.:

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.sonar.java.checks.verifier.CheckVerifier;

• Apply fix

_{Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎}

[gitar-bot]

Code Review ⚠️ Changes requested 0 resolved / 3 findings

Removes the autoscan integration test module and adds test_without_semantic() across check test files, but verifyIssues() incorrectly reuses semantic sample files in many tests, and contains minor formatting inconsistencies.

⚠️ Edge Case: verifyIssues() + withoutSemantic() reuses semantic sample files

📄 java-checks/src/test/java/org/sonar/java/checks/AccessibilityChangeCheckTest.java:55-63

Many new test_without_semantic() methods call verifyIssues() while pointing at the same sample file whose // Noncompliant comments were authored for a full semantic analysis (e.g. AccessibilityChangeCheckTest, MathClampMethodsCheckTest, InstanceOfPatternMatchingCheckTest). withoutSemantic() disables type resolution, so any check that relies on semantics will raise a different (usually smaller) issue set and verifyIssues() will fail because the raised issues no longer match the expected comments. This is the bulk of the acknowledged remaining work; for semantic-dependent rules the intent (verify the check doesn't crash without bytecode) is better expressed with verifyNoIssues(), and only syntax-only rules should keep verifyIssues(). Each affected test needs to be triaged individually.

💡 Quality: Stray double blank line before test_without_semantic in many files

📄 java-checks/src/test/java/org/sonar/java/checks/AccessibilityChangeCheckTest.java:53-54 📄 java-checks/src/test/java/org/sonar/java/checks/ImportDeclarationOrderCheckTest.java:254-255

A large number of the modified test files introduce two consecutive blank lines between the previous method's closing brace and the new @Test method (e.g. AccessibilityChangeCheckTest lines 53-54, BufferedReaderBoilerplateCheckTest, CommentsMustStartWithCorrectNumberOfSlashesCheckTest, ControlCharacterInLiteralCheckTest, DefaultEncodingUsageCheckTest, ImportDeclarationOrderCheckTest 254-255, ReadlnWithPromptCheckTest, StringIndexOfRangesCheckTest, etc.). This is a formatting inconsistency across ~575 files; collapse to a single blank line for consistency.

Use a single blank line between methods.

    .verifyIssues();
}

@Test
void test_without_semantic() {

💡 Quality: Import of org.junit.jupiter.api.Test added out of order

📄 java-checks/src/test/java/org/sonar/java/checks/DefaultEncodingUsageCheckTest.java:19-22

In DefaultEncodingUsageCheckTest the new import org.junit.jupiter.api.Test; is appended after the org.sonar.java... imports instead of being grouped with the other org.junit.jupiter imports, breaking alphabetical import ordering (a convention this very project enforces via ImportDeclarationOrderCheck). Move the import up next to the other junit imports.

Group the junit Test import with the other junit imports in alphabetical order.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.sonar.java.checks.verifier.CheckVerifier;

🤖 Prompt for agents

Code Review: Removes the autoscan integration test module and adds `test_without_semantic()` across check test files, but `verifyIssues()` incorrectly reuses semantic sample files in many tests, and contains minor formatting inconsistencies.

1. ⚠️ Edge Case: verifyIssues() + withoutSemantic() reuses semantic sample files
   Files: java-checks/src/test/java/org/sonar/java/checks/AccessibilityChangeCheckTest.java:55-63

   Many new test_without_semantic() methods call verifyIssues() while pointing at the same sample file whose // Noncompliant comments were authored for a full semantic analysis (e.g. AccessibilityChangeCheckTest, MathClampMethodsCheckTest, InstanceOfPatternMatchingCheckTest). withoutSemantic() disables type resolution, so any check that relies on semantics will raise a different (usually smaller) issue set and verifyIssues() will fail because the raised issues no longer match the expected comments. This is the bulk of the acknowledged remaining work; for semantic-dependent rules the intent (verify the check doesn't crash without bytecode) is better expressed with verifyNoIssues(), and only syntax-only rules should keep verifyIssues(). Each affected test needs to be triaged individually.

2. 💡 Quality: Stray double blank line before test_without_semantic in many files
   Files: java-checks/src/test/java/org/sonar/java/checks/AccessibilityChangeCheckTest.java:53-54, java-checks/src/test/java/org/sonar/java/checks/ImportDeclarationOrderCheckTest.java:254-255

   A large number of the modified test files introduce two consecutive blank lines between the previous method's closing brace and the new @Test method (e.g. AccessibilityChangeCheckTest lines 53-54, BufferedReaderBoilerplateCheckTest, CommentsMustStartWithCorrectNumberOfSlashesCheckTest, ControlCharacterInLiteralCheckTest, DefaultEncodingUsageCheckTest, ImportDeclarationOrderCheckTest 254-255, ReadlnWithPromptCheckTest, StringIndexOfRangesCheckTest, etc.). This is a formatting inconsistency across ~575 files; collapse to a single blank line for consistency.

   Fix (Use a single blank line between methods.):
       .verifyIssues();
   }
   
   @Test
   void test_without_semantic() {

3. 💡 Quality: Import of org.junit.jupiter.api.Test added out of order
   Files: java-checks/src/test/java/org/sonar/java/checks/DefaultEncodingUsageCheckTest.java:19-22

   In DefaultEncodingUsageCheckTest the new `import org.junit.jupiter.api.Test;` is appended after the `org.sonar.java...` imports instead of being grouped with the other `org.junit.jupiter` imports, breaking alphabetical import ordering (a convention this very project enforces via ImportDeclarationOrderCheck). Move the import up next to the other junit imports.

   Fix (Group the junit Test import with the other junit imports in alphabetical order.):
   import org.junit.jupiter.api.Test;
   import org.junit.jupiter.params.ParameterizedTest;
   import org.junit.jupiter.params.provider.ValueSource;
   import org.sonar.java.checks.verifier.CheckVerifier;

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.
Unblock → Override a blocking verdict and allow merging.

Comment with these commands to change the behavior for this request:

Auto-apply	Compact	Unblock
gitar auto-apply:on	gitar display:verbose	gitar unblock

_{Was this helpful? React with 👍 / 👎 | Gitar}