Skip to content

Test hardening#22799

Draft
NickSdot wants to merge 4 commits into
php:PHP-8.5from
NickSdot:test-hardening
Draft

Test hardening#22799
NickSdot wants to merge 4 commits into
php:PHP-8.5from
NickSdot:test-hardening

Conversation

@NickSdot

@NickSdot NickSdot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Test files sweep to make exception assertions more robust, and slightly unify style.

Specifically:

  • adds thrown class to all exception assertions to protected against regressions
  • further established the already recommended single exception formatting
  • adds missing last line whitespaces were missing

Target format:

echo $e::class, ': ', $e->getMessage(), \PHP_EOL; // + variations like line(), code(), etc.

Approach:
Rewrites are handled by this deterministic helper for conservative and safe, automated replacements. All exception assertion "flavours" from php-src were extracted, and the tests where they were found promoted to source fixtures. Rules were then ran against the fixtures, the results manually reviewed, and promoted to target fixtures. All replacement rules are tested against the source and target fixtures. The actual "fixing" works as follows:

  1. Scan PHPT files for exception-message output inside catch blocks.
  2. Run the original PHPT through run-tests.php; skipped or already failing tests are not rewritten.
  3. Rewrite the PHPT to the normalised exception-output shape.
  4. Run the rewritten PHPT through run-tests.php.
  5. If output changed as expected, update the expected section from the actual run output.
  6. Run the updated PHPT once more.
  7. Keep only rewrites that pass verification.

Modules & Commits

Extensions

Module Commit
ext/random 9dc4e19
ext/uri 179f176
ext/intl 6afaaf8
ext/phar 003dbba

Zend

Module Commit
Zend tbd

Changelog:

  • 2026-07-20: added ext/intl and ext/phar

PoC, marked as draft for now, and in coordination with Tim for now only applied to ext/random and ext/uri.

@NickSdot

Copy link
Copy Markdown
Contributor Author

@LamentXU123 I noticed your like and since you own ext/intl and ext/phar I applied the fixers to it and pushed two new commits. Would love your feedback!

Aside: since I didn't built with all extensions ext/intl serves as a good example for non-provable rewrites. In my run listformatter_with_parameters_error.phpt was skipped.

@thg2k

thg2k commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Is the \PHP_EOF really necessary? I think it just is silly in the tests. Just PHP_EOF should be fine.

@NickSdot

Copy link
Copy Markdown
Contributor Author

Is the \PHP_EOF really necessary? I think it just is silly in the tests. Just PHP_EOF should be fine.

Nah. Picked it out of habit. No strong opinion; can adjust to whatever we decide.

@LamentXU123

Copy link
Copy Markdown
Member

I actually think this change is trivial. I don't have strong opinions on this (I am on the fence of this huge code churn) so might need to wait for other's opinion.

@LamentXU123

Copy link
Copy Markdown
Member

nit: any reasons for targeting 8.5?

@NickSdot

Copy link
Copy Markdown
Contributor Author

I actually think this change is trivial. I don't have strong opinions on this (I am on the fence of this huge code churn) so might need to wait for other's opinion.

Sometimes not avoidable to move things forward. :) Personally, I think it adds enough value to be justified. Flipping \PHP_EOL to PHP_EOL would decrease churn by a bit. Do you also think it's silly? If yes could pull the trigger on making it PHP_EOL.

nit: any reasons for targeting 8.5?

Nope. Could be anything. Which one you want?

@LamentXU123

Copy link
Copy Markdown
Member

Nope. Could be anything. Which one you want?

Should be targeting master. (hint: convert this to draft when you are changing the base to avoid accidentally request a review from everyone)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants