Skip to content

Add std-out/simple-data-objects to benchmark comparison - #116

Merged
dereuromark merged 4 commits into
masterfrom
feature/benchmark-simple-data-objects
Jul 27, 2026
Merged

Add std-out/simple-data-objects to benchmark comparison#116
dereuromark merged 4 commits into
masterfrom
feature/benchmark-simple-data-objects

Conversation

@dereuromark

Copy link
Copy Markdown
Contributor

Adds std-out/simple-data-objects (compiled-closure DTO hydration) to the external benchmark comparison. Its author asked for this in the r/PHP thread so the numbers are reproducible on both sides rather than traded back and forth.

What's in it

  • run-external.php - simple and nested creation, auto-detected exactly like the other libraries. If the package is not installed the section is skipped, same as jms/serializer today.
  • run-sdo.php - a dedicated head-to-head. run-external.php only measures hydration (where simple-data-objects leads), so this script rounds out the picture: hydration, serialization, field transforms (both their Cast attribute and their Pipe middleware), and per-instance memory. All three transform variants (ours, their Cast, their Pipe) are verified to produce identical output for a whitespace-padded payload, so the scenarios compare equal work.
  • generate.php - regenerates the benchmark DTOs with strictTypes enabled. The bin/dto CLI does not expose that flag and transform inlining requires it.
  • src/SimpleDataObjects/* - mirror DTOs matching the existing benchmark shapes.

PHP version constraint

std-out/simple-data-objects requires PHP ^8.4. Putting it in require-dev would break composer install on the PHP 8.2/8.3 the benchmark still supports, so it lives under suggest and is opt-in:

cd benchmark
composer require --dev std-out/simple-data-objects

The scripts guard every call with class_exists(), so nothing breaks when it is absent.

Notes

  • Benchmark-only change; no library src/ is touched.
  • createdAt is left out of the nested serialization payload on purpose: our toArray() keeps a DateTimeImmutable there while their DateTimeImmutableCast formats it back to a string, which would compare non-equivalent work.
  • Both libraries trade blows - simple-data-objects wins hydration and memory, php-collective/dto wins serialization and (with strictTypes) compiled transforms. Absolute numbers are hardware-dependent and left out on purpose; run it locally.

Adds the compiled-closure DTO library std-out/simple-data-objects to the
external comparison, requested by its author so the numbers are
reproducible on both sides.

- run-external.php: simple and nested creation, auto-detected like the
  other libraries and skipped when absent.
- run-sdo.php: a dedicated head-to-head covering hydration, serialization,
  field transforms (their Cast and Pipe paths) and per-instance memory,
  since run-external only measures hydration.
- generate.php: regenerates the benchmark DTOs with strictTypes enabled,
  which the bin/dto CLI does not expose and which transform inlining needs.

The dependency requires PHP 8.4+, so it stays out of require-dev (which
would break composer install on 8.2/8.3) and lives under suggest; install
it with composer require --dev std-out/simple-data-objects.
Copilot AI review requested due to automatic review settings July 27, 2026 14:27
@dereuromark dereuromark added the enhancement New feature or request label Jul 27, 2026
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.20%. Comparing base (a81ce61) to head (a60b0d1).

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #116   +/-   ##
=========================================
  Coverage     83.20%   83.20%           
  Complexity     1569     1569           
=========================================
  Files            46       46           
  Lines          3859     3859           
=========================================
  Hits           3211     3211           
  Misses          648      648           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds std-out/simple-data-objects as an opt-in benchmark target and introduces a dedicated head-to-head script to compare hydration/serialization/transforms/memory against php-collective/dto, plus regeneration of generated benchmark DTOs with strictTypes enabled for transform inlining.

Changes:

  • Extend run-external.php to auto-detect and benchmark simple-data-objects (flat + nested hydration), including configuring its on-disk metadata cache.
  • Add run-sdo.php for a direct comparison across scenarios (hydration, serialization, transforms, per-instance memory) and add mirror DTO shapes under benchmark/src/SimpleDataObjects/.
  • Add benchmark/generate.php and update benchmark/config/dto.php to generate benchmark DTOs with strictTypes, plus README/composer updates for opt-in installation via suggest.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
benchmark/src/Support/UcfirstTrimPipe.php Adds a simple-data-objects Pipe implementation used in transform benchmarks.
benchmark/src/Support/UcfirstCast.php Adds a simple-data-objects Cast implementation used in transform benchmarks.
benchmark/src/Support/Transform.php Centralizes transform helpers used by generated benchmark DTO config.
benchmark/src/SimpleDataObjects/UserData.php Adds flat mirror DTO for simple-data-objects hydration/serialization scenarios.
benchmark/src/SimpleDataObjects/TransformUserData.php Adds Pipe-based transform mirror DTO for simple-data-objects.
benchmark/src/SimpleDataObjects/TransformUserCastData.php Adds Cast-based transform mirror DTO for simple-data-objects.
benchmark/src/SimpleDataObjects/OrderItemData.php Adds nested mirror DTO component for simple-data-objects.
benchmark/src/SimpleDataObjects/OrderData.php Adds nested order mirror DTO (incl. collection + DateTime cast) for simple-data-objects.
benchmark/src/SimpleDataObjects/MiniData.php Adds minimal mirror DTO for small-scenario comparisons.
benchmark/src/SimpleDataObjects/AddressData.php Adds address mirror DTO for nested scenarios.
benchmark/run-sdo.php New head-to-head benchmark script covering hydration, serialization, transforms, and memory.
benchmark/run-external.php Adds simple-data-objects to external comparison + configures its cache directory.
benchmark/README.md Documents opt-in install (PHP 8.4+) and new head-to-head workflow.
benchmark/generate.php Adds generator wrapper to regenerate benchmark DTOs with strictTypes enabled.
benchmark/config/dto.php Adds TransformUser + Mini definitions used by the head-to-head script and strictTypes generation.
benchmark/composer.json Adds suggest entry for opt-in simple-data-objects dependency (PHP 8.4+).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread benchmark/run-sdo.php
Comment thread benchmark/run-sdo.php
- Exit with a clear message when std-out/simple-data-objects or the
  generated DTOs are absent, instead of a fatal.
- Verify each scenario produces identical output on both sides before
  timing, so the ratios always compare equal work.
Copilot AI review requested due to automatic review settings July 27, 2026 14:39
@dereuromark

Copy link
Copy Markdown
Contributor Author

Results: php-collective/dto vs std-out/simple-data-objects

Ran run-sdo.php on this branch (PHP 8.5, opcache on, JIT off, both sides warmed, identical DTO shapes, every scenario passes the new output-equivalence gate before timing). This box is noisy, so absolute ops/s swing 2-4x between runs from background load. What is stable is the direction of each axis and the rough ratio range, so that is what I am reporting rather than fake-precise numbers. Run it on your own hardware for absolutes.

Who wins what

Axis Winner Typical magnitude
Hydration, flat simple-data-objects ~1.2-1.45x
Hydration, small (3 scalars) simple-data-objects ~1.4-1.8x
Hydration, nested + collection simple-data-objects ~1.4-2.6x, grows with size
Serialization, flat php-collective/dto ~1.4-3x
Serialization, nested + collection php-collective/dto ~1.3-1.9x, all sizes
Serialization, tiny (3 scalars) roughly a tie 0.95-1.05x
Transforms, vs their Cast attribute php-collective/dto ~1.4-1.9x
Transforms, vs their Pipe middleware php-collective/dto ~3-6x
Memory per instance (flat) simple-data-objects 3.9x (187 B vs 736 B)

Reading of it

  • Hydration and per-instance memory go to simple-data-objects. The memory gap is structural, not timing noise, so it is the same every run. On our side it is dominated by the incrementally-built _touchedFields hashtable (measured in isolation at 607 vs 219 bytes for the touched-tracking alone). A future change to a bitmask would close most of it.
  • Serialization goes to php-collective/dto everywhere except tiny objects, and by more than the 5-10% margin floated in the thread. toArrayFast() emits a flat array literal, which is hard to beat.
  • Field transforms now go to php-collective/dto. The ~1M vs ~420K figure from the thread was measured against our old transformValue() runtime dispatch. Inline schema transforms into generated fast-path code #115 compiles transformFrom / transformTo into a direct call in the generated code (when strictTypes is on), which flips that scenario. Their compiled Cast is the fair comparison and we lead it; their Pipe middleware runs per value at runtime so the gap there is larger and less meaningful.

Net: the two libraries trade blows along a clean line. simple-data-objects is the faster hydrator with a much smaller footprint; php-collective/dto is the faster serializer and (post-#115) the faster transformer. Neither is uniformly ahead, which is the honest headline.

Reproduce

cd benchmark
composer require --dev std-out/simple-data-objects   # opt-in, needs PHP 8.4+
cd ..
php benchmark/generate.php                            # regenerate DTOs with strictTypes
php -d opcache.enable_cli=1 benchmark/run-sdo.php 20000

run-external.php also lists simple-data-objects in its comparison table, but it only measures hydration, so on its own it makes the picture look one-sided. run-sdo.php is the fuller story.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comment thread benchmark/run-sdo.php
opcache_get_status() returns false when OPcache is disabled, so indexing
it emitted warnings; capture once and check is_array() first.
Copilot AI review requested due to automatic review settings July 27, 2026 14:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Comment thread benchmark/run-sdo.php
Comment thread benchmark/run-sdo.php
Comment thread benchmark/run-sdo.php
Comment thread benchmark/run-external.php
- Validate iterations is positive before use (avoids division by zero).
- Abort if the cache dir cannot be created, in both run-sdo and run-external.
- Make the equivalence gate order-insensitive: recursively sort associative
  keys and encode with JSON_THROW_ON_ERROR so mismatches are meaningful.
Copilot AI review requested due to automatic review settings July 27, 2026 15:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

@dereuromark
dereuromark merged commit a984625 into master Jul 27, 2026
13 checks passed
@dereuromark
dereuromark deleted the feature/benchmark-simple-data-objects branch July 27, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants