[SYSTEMDS-3971] Improve Scuro test suite - #2614
Open
shieru1214 wants to merge 3 commits into
Open
shieru1214 wants to merge 3 commits into
shieru1214 wants to merge 3 commits into
Conversation
The five optimizer tests built one modality each and then called the same helper, which holds all the assertions. They become a table of modality sets and a small factory, driven by one subTest per set, so the number of combinations stays the same. The keyword arguments keep the inputs as the separate tests had them, including the text case that used one sentence instead of ten. test_audio_representations built its audio modality inline although _create_audio_modality already exists and the other two audio tests use it. It now calls the helper as well. Assisted-by: AI
The three per-modality window aggregation tests ran the same helper over data of the same shape and dtype, because window_aggregation dispatches on the data layout and not on the modality type. They become one test with the modality and the aggregation as subTest dimensions, which also splits the inner aggregation loop into separately reported cases. The 3d and 2d shape tests differed only in the input dimensions and in the hard coded expected shape. Window aggregation only changes the first axis, so one expression covers both. Assisted-by: AI
The four tests ran the same eleven line sequence and differed only in the operator and in the expected booleans. The booleans are the actual content, so they move into a table with one subTest per operator. The commutativity check now compares the measured result with the operator's own "commutative" attribute instead of a second hand written copy of it. The concatenation test also compared the wrong operand, so its chain against n-ary property was never covered. Assisted-by: AI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Purpose
This PR only changes test files and does not change the Scuro source code. The main purpose is to identify redundant or overly large component tests and refactor them into smaller tests, while also reducing the runtime if possible.
2. Overview
This PR focuses on the representation operator, window operation, and fusion parts.
test_unimodal_optimizer.pytest_unimodal_representations.pytest_window_operations.pytest_fusion_orders.py3. Changes
3.1 Representation Operator
In
test_unimodal_optimizer.py, the text, image, audio, video, and text-image tests previously created their modalities separately before calling the same optimizer helper. I moved the shared data and loader setup into_create_modality, while the input combinations are now stored inMODALITY_SETS.This keeps the differences between the five cases visible in one place. For example, the video case still uses 10 frames, and the two text cases keep their original sentence counts. Each case is executed as a labeled
subTest, so a failure directly shows which modality combination caused it. The same five cases still run the full optimizer.In
test_unimodal_representations.py, I reused the existing_create_audio_modalityhelper intest_audio_representations. This removes a repeated block that created the audio data, loader, and modality. The test input and assertions remain unchanged, but the audio setup now only needs to be maintained in one place.3.2 Window Operation
In
test_window_operations.py, the audio, video, and text cases all applied the same four window aggregations to 1D data. These inputs have the same data layout (single level), and the aggregation path is selected byDataLayoutrather thanModalityType.I combined the three methods into one test that runs three modalities and four aggregation methods, keeping all 12 combinations. A failure now reports both the modality and aggregation method, instead of only the name of the original test method.
I also combined the separate 2D and 3D shape tests. Both tests used the same three window operators, and their expected output shapes follow the same rule. The new test keeps all six combinations of two dimensions and three operators. Each case includes
dimsandoperatorlabels, which makes it easier to find the failing combination.3.3 Fusion
In
test_fusion_orders.py, the separate tests forAverage,Concatenation,RowMax, andHadamardfollowed the same fusion steps. I combined them intoFUSION_PROPERTIESand one shared test method with foursubTests.The expected differences for chain order and multi-input fusion are stored in
FUSION_PROPERTIES. Commutativity is checked by comparing the actual result with the operator'scommutativeattribute. This checks whether the declared property agrees with the implementation. Another compatible fusion operator can be covered by adding one entry to the table.This PR also changes the previous
Concatenationcomparison. The old test compared a two-input result with a three-input result, which were already different in shape. The new test compares chained three-input fusion with direct three-input fusion, so all four operators now use the same comparison.Finally, the fusion input was reduced. The test checks input order and fusion calling style rather than data size, so the smaller input follows the same test path while processing less data.