Sep'23 Regression testing - Part 1

[tyuldashev]

JDK <-> Framework

Project	JDK	Framework	Status	Comment	Sample
Gradle	1.8	JUnit 5	x		strings
Maven	1.8	TestNG	x		structures
Gradle	11	JUnit 5	x		lambda
Maven	11	TestNG	x		objects
Maven	17	JUnit 5	x		stdlib
Gradle	17	TestNG	x		algorithms/CorrectBracketSequences

JDK <-> Framework <-> Parametrization

IntelliJ build system

JDK	Framework	Parametrization	Status	Comment	Sample
1.8	JUnit 4	On		Not applicable
1.8	JUnit 5	On	x		arrays/ArrayOfObjects
1.8	TestNG	On		Not applicable
11	JUnit 4	On		Not applicable
11	JUnit 5	On	x		arrays/FinalStaticFieldArray
11	TestNG	On	x		enums/ClassWithEnum
17	JUnit 4	On		Not applicable
17	JUnit 5	On	x		collections/QueueUsages
17	TestNG	On	x		collections/Optionals

JDK <-> Fuzzing/Symbolic

JDK	Fuzzing/Symbolic	Symbolic 100%	Fuzzing 100%
1.8	exceptions/JvmCrashExamples	nested	invokes
11	controlflow/CycleDependedCondition	make.symbolic/ClassWithComplicatedMethods	reflection
17	types/Generics	natives	stream

Fuzzing/Symbolic <-> Timeout

Test Generation Timeout	Fuzzing/Symbolic	Symbolic 100%	Fuzzing 100%
60	arrays	arrays	arrays
20	primitives	primitives	primitives
10	math	math	math

• timeout limits only test generation, building and reading classes phases are not included.

Mocks

• No mocks

• No mocks in settings, but forced mock for Random

• Mock outside class

• Mock outside package

• Mock static methods

• strings

• structures

• lambda

• objects

• stdlib

• algorithms/CorrectBracketSequences

• arrays/ArrayOfObjects

• arrays/FinalStaticFieldArray

• enums/ClassWithEnum

• exceptions/JvmCrashExamples

• collections/Optionals

• collections/QueueUsages

• controlflow/CycleDependedCondition - to check summaries

• controlflow/Switch

• types/Generics

• nested

• invokes

• make.symbolic/ClassWithComplicatedMethods

• mixed/LoggerExample - to check force mocking

• natives

• reflection

• stream

• thirdparty/numbers/ArithmeticUtils

• threads

• unsafe

• wrappers

• annotations/NotNullAnnotation

• math

• primitives

• mocks

[tyuldashev]

Regression defects found:

• Array parameters shown as "[I" in region method summary #2585