bug: datafusion-spark string literals don't interpret escape sequences like Spark

[andygrove]

Describe the bug

DataFusion-spark treats \t and \n in SQL string literals as literal backslash characters, while Apache Spark interprets them as escape sequences (tab and newline). This affects any function that receives string arguments containing these sequences.

To Reproduce

PySpark (Spark behavior):

SELECT soundex('\thello');  -- returns tab + "hello" (soundex passes through non-alpha input)
SELECT soundex('\nhello');  -- returns newline + "hello"
SELECT length('\thello');   -- 6 (tab is one character)
SELECT length('\nhello');   -- 6 (newline is one character)

DataFusion-spark (current behavior):

SELECT soundex('\thello');  -- returns literal "\thello" (backslash-t-hello)
SELECT soundex('\nhello');  -- returns literal "\nhello" (backslash-n-hello)
SELECT length('\thello');   -- 7 (\t is two characters: backslash and t)
SELECT length('\nhello');   -- 7 (\n is two characters: backslash and n)

Expected behavior

DataFusion-spark should interpret \t, \n, and other escape sequences in string literals the same way Spark does, for Spark compatibility.

Additional context

This is a string literal parsing issue, not specific to soundex. It affects all string functions. The .slt tests at string/soundex.slt lines 83 and 193 have expected values that match DataFusion's literal interpretation rather than Spark's escape interpretation.

This was discovered by running a PySpark validation script against the .slt test files (see #17045, #21508).

[SamAya21]

Hello, I would like to try working on this bug. I haven't worked on an issue for this project before do you think this bug is fine as a first issue or would you recommend any of the other 4 bug reports you've posted recently.

[Abaydoun54]

take (working with SamAya21)

[SamAya21]

take

[SamAya21]

Is there a suggested way to rewrite test cases related to this change for example there are some test cases in sqlllogictest/test_files/spark/string/soundex.stl. There are two test cases with \nhello and \thello, but editing them have a new line or tab never matches the expected or just breaks the test file. There is also a simliar issue in spark/array/array.stl where a new lines gets added in a middle an array.