Skip to content

[SYSTEMDS-3971] Improve Scuro test suite - #2614

Open
shieru1214 wants to merge 3 commits into
apache:mainfrom
shieru1214:scuro-v3
Open

shieru1214 wants to merge 3 commits into
apache:mainfrom
shieru1214:scuro-v3

Conversation

@shieru1214

Copy link
Copy Markdown

1. Purpose

This PR only changes test files and does not change the Scuro source code. The main purpose is to identify redundant or overly large component tests and refactor them into smaller tests, while also reducing the runtime if possible.

2. Overview

This PR focuses on the representation operator, window operation, and fusion parts.

Stage File Main change
Representation Operator test_unimodal_optimizer.py Combines five modality optimizer tests into one test
Representation Operator test_unimodal_representations.py Reuses the existing audio modality helper
Window Operation test_window_operations.py Combines repeated 1D modality tests and the separate 2D and 3D shape tests
Fusion test_fusion_orders.py Combines four fusion tests

3. Changes

3.1 Representation Operator

In test_unimodal_optimizer.py, the text, image, audio, video, and text-image tests previously created their modalities separately before calling the same optimizer helper. I moved the shared data and loader setup into _create_modality, while the input combinations are now stored in MODALITY_SETS.

This keeps the differences between the five cases visible in one place. For example, the video case still uses 10 frames, and the two text cases keep their original sentence counts. Each case is executed as a labeled subTest, so a failure directly shows which modality combination caused it. The same five cases still run the full optimizer.

In test_unimodal_representations.py, I reused the existing _create_audio_modality helper in test_audio_representations. This removes a repeated block that created the audio data, loader, and modality. The test input and assertions remain unchanged, but the audio setup now only needs to be maintained in one place.

3.2 Window Operation

In test_window_operations.py, the audio, video, and text cases all applied the same four window aggregations to 1D data. These inputs have the same data layout (single level), and the aggregation path is selected by DataLayout rather than ModalityType.

I combined the three methods into one test that runs three modalities and four aggregation methods, keeping all 12 combinations. A failure now reports both the modality and aggregation method, instead of only the name of the original test method.

I also combined the separate 2D and 3D shape tests. Both tests used the same three window operators, and their expected output shapes follow the same rule. The new test keeps all six combinations of two dimensions and three operators. Each case includes dims and operator labels, which makes it easier to find the failing combination.

3.3 Fusion

In test_fusion_orders.py, the separate tests for Average, Concatenation, RowMax, and Hadamard followed the same fusion steps. I combined them into FUSION_PROPERTIES and one shared test method with four subTests.

The expected differences for chain order and multi-input fusion are stored in FUSION_PROPERTIES. Commutativity is checked by comparing the actual result with the operator's commutative attribute. This checks whether the declared property agrees with the implementation. Another compatible fusion operator can be covered by adding one entry to the table.

This PR also changes the previous Concatenation comparison. The old test compared a two-input result with a three-input result, which were already different in shape. The new test compares chained three-input fusion with direct three-input fusion, so all four operators now use the same comparison.

Finally, the fusion input was reduced. The test checks input order and fusion calling style rather than data size, so the smaller input follows the same test path while processing less data.

The five optimizer tests built one modality each and then called the same
helper, which holds all the assertions. They become a table of modality
sets and a small factory, driven by one subTest per set, so the number of
combinations stays the same. The keyword arguments keep the inputs as the
separate tests had them, including the text case that used one sentence
instead of ten.

test_audio_representations built its audio modality inline although
_create_audio_modality already exists and the other two audio tests use
it. It now calls the helper as well.

Assisted-by: AI
The three per-modality window aggregation tests ran the same helper over
data of the same shape and dtype, because window_aggregation dispatches on
the data layout and not on the modality type. They become one test with
the modality and the aggregation as subTest dimensions, which also splits
the inner aggregation loop into separately reported cases.

The 3d and 2d shape tests differed only in the input dimensions and in the
hard coded expected shape. Window aggregation only changes the first axis,
so one expression covers both.

Assisted-by: AI
The four tests ran the same eleven line sequence and differed only in the
operator and in the expected booleans. The booleans are the actual
content, so they move into a table with one subTest per operator.

The commutativity check now compares the measured result with the
operator's own "commutative" attribute instead of a second hand written
copy of it. The concatenation test also compared the wrong operand, so its
chain against n-ary property was never covered.

Assisted-by: AI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant