ext/standard: applied fixers to improve test robustness by NickSdot · Pull Request #23136 · php/php-src

NickSdot · 2026-08-08T06:50:53Z

Asserts error class; extracted from: #22799

GitHub is lagging this, easier to review with: https://diffshub.com/php/php-src/pull/23136

NickSdot · 2026-08-10T06:40:26Z

@bukka following your comment yesterday I reviewed this one once again, and wanted to double check.

Out of 4,210 total lines, 13 (in 4 files) are affected by the same style-only thematic. Are you here okay with unifying those 13 lines with the rest or do you want me to make a manual follow up?
And while we are on it... Currently line endings are untouched; some said output always should be \n, others don't wanted them to be touched. Would you prefer to convert to \n or nah?

I would make a manual follow up to get these addressed.

Files affected by the style-only thematic:
ext/standard/tests/array/gh22678.phpt
ext/standard/tests/filters/gh20370.phpt
ext/standard/tests/strings/gh20906_1.phpt
ext/standard/tests/strings/setlocale_null_byte.phpt

Girgias

There are loads of ZPP tests that should be removed, would make the diff smaller and reduce the test suite aswell.

Girgias · 2026-08-22T12:54:02Z

This is a ZPP test and should be removed.

Girgias · 2026-08-22T12:54:23Z

Ditto ZPP test

Girgias · 2026-08-22T12:54:51Z

Girgias · 2026-08-22T12:55:08Z

Girgias · 2026-08-22T12:55:24Z

Girgias · 2026-08-22T12:55:51Z

Girgias · 2026-08-22T12:56:03Z

Girgias · 2026-08-22T12:56:25Z

Girgias · 2026-08-22T12:58:48Z

Girgias · 2026-08-22T12:59:03Z

NickSdot · 2026-08-22T16:21:30Z

There are loads of ZPP tests that should be removed, would make the diff smaller and reduce the test suite aswell.

@Girgias that's done in #23413 -- when merged I will rebase and resolve here.

NickSdot · 2026-09-16T13:47:34Z

Rebased.

Girgias

So there are still a few ZPP tests, but everything looks good to me. (I did ignore most of the printf variation tests as those are kinda broken already)

Girgias · 2026-09-19T16:09:19Z

  [2]=>
  string(6) "Line 3"
 }
-file(): Argument #2 ($flags) must be a valid flag value


Note for future, it might be a good idea to improve the error message.

Girgias · 2026-09-19T16:10:25Z

-flock(): Argument #2 ($operation) must be of type int, array given
+TypeError: flock(): Argument #2 ($operation) must be of type int, array given
 --- Iteration 4 ---
-flock(): Argument #2 ($operation) must be of type int, array given
+TypeError: flock(): Argument #2 ($operation) must be of type int, array given
 --- Iteration 5 ---
-flock(): Argument #2 ($operation) must be of type int, string given
+TypeError: flock(): Argument #2 ($operation) must be of type int, string given
 --- Iteration 6 ---
-flock(): Argument #2 ($operation) must be of type int, string given
+TypeError: flock(): Argument #2 ($operation) must be of type int, string given
 --- Iteration 7 ---
-flock(): Argument #2 ($operation) must be of type int, string given


Seems like we have some ZPP tests here (keep the stream resource one as that's not fully ZPP)

Girgias · 2026-09-19T16:14:29Z

-tempnam(): Argument #2 ($prefix) must not contain any null bytes
+ValueError: tempnam(): Argument #2 ($prefix) must not contain any null bytes
 -- Iteration 6 --
-tempnam(): Argument #2 ($prefix) must be of type string, array given


Girgias · 2026-09-19T16:14:41Z

-tempnam(): Argument #1 ($directory) must not contain any null bytes
+ValueError: tempnam(): Argument #1 ($directory) must not contain any null bytes
 -- Iteration 6 --
-tempnam(): Argument #1 ($directory) must be of type string, array given


Girgias · 2026-09-19T16:16:29Z

 Warning: Undefined variable $obj in %s on line %d
-register_shutdown_function(): Argument #1 ($callback) must be a valid callback, first array member is not a valid class name or object
+TypeError: register_shutdown_function(): Argument #1 ($callback) must be a valid callback, first array member is not a valid class name or object

 Warning: Undefined variable $obj in %s on line %d
-register_shutdown_function(): Argument #1 ($callback) must be a valid callback, first array member is not a valid class name or object
-register_shutdown_function(): Argument #1 ($callback) must be a valid callback, first array member is not a valid class name or object
-register_shutdown_function(): Argument #1 ($callback) must be a valid callback, class bar does not have a method "foo"
-register_shutdown_function(): Argument #1 ($callback) must be a valid callback, first array member is not a valid class name or object
-register_shutdown_function(): Argument #1 ($callback) must be a valid callback, function "bar" not found or invalid function name


A lot of these are ZPP tests.

Do we really remove bug tests?

Hmmmm not usually.

Girgias · 2026-09-19T16:32:35Z

@@ -125,5 +125,5 @@ string(10) "helloworld"
 -- Iteration 20 --
 string(10) "helloworld"
 -- Iteration 21 --
-strip_tags(): Argument #2 ($allowed_tags) must be of type array|string|null, resource given


ZPP test and seems the try catch block is only for this case?

Girgias · 2026-09-19T16:32:58Z

-stristr(): Argument #2 ($needle) must be of type string, array given
+TypeError: stristr(): Argument #2 ($needle) must be of type string, array given
 -- Iteration 9 --
-stristr(): Argument #2 ($needle) must be of type string, array given
+TypeError: stristr(): Argument #2 ($needle) must be of type string, array given
 -- Iteration 10 --
-stristr(): Argument #2 ($needle) must be of type string, array given


Girgias · 2026-09-19T16:33:33Z

 Position of 'Hello' with offset '0' is => int(0)
 Position of 'Hello' with offset '1' is => int(74)
 Position of 'Hello' with offset '' is => int(0)
 Position of 'Hello' with offset 'string12' is => 
-strpos(): Argument #3 ($offset) must be of type int, string given


Girgias · 2026-09-19T16:34:04Z

@@ -89,11 +89,11 @@ string(6) "m1tatm"
 -- Iteration 6 --
 string(6) "tm0atm"
 -- Iteration 7 --
-strtr(): Argument #2 ($from) must be of type string, array given


Girgias · 2026-09-19T16:34:22Z

Loads of ZPP tests

github-actions Bot added the Extension: standard label Aug 8, 2026

NickSdot marked this pull request as ready for review August 8, 2026 08:19

NickSdot requested a review from bukka as a code owner August 8, 2026 08:19

NickSdot mentioned this pull request Aug 16, 2026

Tests: Test exception type in error tests #22799

Draft

NickSdot force-pushed the test/hardening/ext-standard branch 4 times, most recently from ebced5a to f9206bf Compare August 21, 2026 18:25

Girgias reviewed Aug 22, 2026

View reviewed changes

NickSdot mentioned this pull request Aug 22, 2026

ext/standard: Removes ZPP style tests #23413

Merged

NickSdot added 3 commits September 16, 2026 21:30

ext/standard: applied fixers to improve test robustness

1144382

fix: kept sibling catch types

4cd430a

fix: removed extra line

713ec9a

NickSdot force-pushed the test/hardening/ext-standard branch from f9206bf to 713ec9a Compare September 16, 2026 13:40

chore: removed leading slashes

b4ec613

NickSdot requested a review from Girgias September 16, 2026 13:47

Girgias reviewed Sep 19, 2026

View reviewed changes

Girgias merged commit f978c5a into php:master Sep 20, 2026
18 checks passed

Conversation

NickSdot commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NickSdot commented Aug 10, 2026

Uh oh!

Girgias left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NickSdot commented Aug 22, 2026

Uh oh!

NickSdot commented Sep 16, 2026

Uh oh!

Girgias left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

NickSdot commented Aug 8, 2026 •

edited

Loading