[SYSTEMDS-3332] Support for matrix-vector operations in ternary ifelse(X,B,v)

[mike0609king]

This PR introduces broadcasting ifelse as well as optimizations for certain cases. The broadcasting is done such that the resulting values are the maximum rows and columns of the inputs.

The following optimizations have been implemented

• the previous optimization for scalar values and certain non zero entries have been extended to contain broadcasting behavior
• if the first input matrix block is a column vector, then the entire row of either second or third input matrix block can be copied to the result matrix.
• if the first input matrix block is a row vector, then the entire column of either second or third input matrix block can be copied to the resulting matrix. The benchmarks also for larger matrices have shown, that this optimization is effective although we are iterating column first. Note that this seems to be effective although it is not good for caching.

Measurements. In the following benchmarks a row size of 2111 and column size of 300 have been used. Note that multiple tests are run for each optimization (around 30 for each); the median value is displayed to be more robust to outliers.

• Scalar optimization. In this case we are testing all test cases involving a scalar as first input and comparing them. Meaning we only test components **.functions.ternary.FullIfElseTest#testScalar*
  • Without optimization: 0.024s
  • With optimization: 0.000s (mean was 0.003s)
• Row optimization. In this case we test the row vector optimization, meaning we only run the tests **.functions.ternary.FullIfElseTest#testRow*
  • Without optimization: 0.074s
  • With optimization: 0.035s
• Column optimization. In this case we test the column vector optimization, meaning we only run the tests **.functions.ternary.FullIfElseTest#testCol*
  • Without optimization: 0.068s
  • With optimization: 0.038s
• Overall. This executes all tests in **.functions.ternary.FullIfElseTest with and without optimizations
  • Without optimization: 0.069s
  • With optimization: 0.027s

Column optimization case. Running larger matrices for all tests took extremely long. This is why I only did this for the column case, which deserved to be scrutinized more. The testColMatrixMatrixDenseCP test with a row size of 4111 and column size of 3000 was executed out with and without optimization the optimized version ran in 0.838s and the one without optimization required 1.547s. Other seeds for the matrix yielded similar results.

Additional comments

• The eps value in the tests were set to 1e-10 otherwise there are errors with the matrix comparisons. This issue seems to be present in other tests such as the FullTransposeTest as well. Maybe some exact values get lost while matrices are stored in files due to compression.
• Tests have been implemented for every matrix, row vector, column vector and scalar value combination. Those tests were of cause not generated by hand by instead by a python script to avoid human error. Same for the aggregation of the measurements above.