Skip to content

[BUGFIX] Preserve raw strings in database snapshots - #756

Open
wouter90 wants to merge 3 commits into
TYPO3:mainfrom
wouter90:fix/json-snapshot-restoration
Open

wouter90 wants to merge 3 commits into
TYPO3:mainfrom
wouter90:fix/json-snapshot-restoration

Conversation

@wouter90

@wouter90 wouter90 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Snapshot rows already contain database values. Exporting a Doctrine JSON type causes the raw JSON string to be encoded again during restoration, changing objects and arrays into strings. Other types can likewise receive values already converted by the database.

Use the string type for columns whose binding type is STRING so restoration preserves their stored representation. Keep the existing handling for other binding types.

Cover JSON, datetime and text conversion in the unit suite and verify an export/import round trip, including JSON scalars and SQL NULL. Document running the functional regressions through a Core checkout.

Snapshot rows already contain database values. Exporting a Doctrine
JSON type causes the raw JSON string to be encoded again during
restoration, changing objects and arrays into strings. Other types
can likewise receive values already converted by the database.

Use the string type for columns whose binding type is STRING so
restoration preserves their stored representation. Keep the existing
handling for other binding types.

Cover JSON, datetime and text conversion in the unit suite and verify
an export/import round trip, including JSON scalars and SQL NULL.
Document running the functional regressions through a Core checkout.

Releases: main
Avoid constructing default query restrictions, which require TCA schema
services on TYPO3 14.3. The snapshot regression only needs a restriction
container that accepts removing all restrictions.

@CybotTM CybotTM left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the fix on TYPO3 Core main. The change to DatabaseAccessor does what it should. The new functional test fails without the fix and passes with it on SQLite, PostgreSQL, MariaDB and MySQL. Without the fix, only the JSON column differs: the restore encodes every value a second time, so 42 becomes "42".

I approve. These points are suggestions:

  1. Unit test: keep only the JSON case. The plain text case passes with and without the fix. The database datetime case fails without the fix only because a unit test uses the Doctrine DateTimeType. In a TYPO3 run, ConnectionPool replaces datetime with the TYPO3 DateTimeType, which keeps strings unchanged. The functional test shows the same result: the created_at column comes back unchanged on main too.
  2. Description: the sentence "Other types can likewise receive values already converted" is not true for datetime in TYPO3. CONTRIBUTING.rst also asks for an issue link, steps to reproduce and the results of the Core test runs.
  3. Backport to branch 9: branch 9 has the same lookupName() and bulkInsert() code. TYPO3 13.4 requires ^9.2.1 and TYPO3 14.3 requires ^9.7.0. Recent bugfixes, for example 480a74d, went to main, 9.
  4. CI: the CI of this repository does not run the functional test, because ci.yml runs only -s unit. A manual step is acceptable, but please state it in the description.

Comment thread CONTRIBUTING.rst Resolved

@CybotTM CybotTM left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my review above. If you prefer to keep all three unit test cases, use the inline suggestion below instead of my earlier suggestion to delete two of them.

Assert the exported column types and raw values directly instead of
performing Doctrine conversions in the unit test. Cover JSON, datetime
and text as string-bound types and retain an integer control case.

Document that DatabaseAccessor regression tests also run on SQLite;
they do not require DatabaseSnapshot's server-database code path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants