Add embedding similarity calculations

[dkotter]

What?

Partially closes #962

Adds a Vector_Math class with various public methods that can be used to run similarity calculations against two vector embeddings. Also adds a Vector_Ranker class to score vectors against one another.

Why?

#892 and #975 brought in embedding code from the PHP AI Client. #976 brought in the data layer to store embeddings. This PR is one of the next steps in the overall embedding goal, allowing us to run comparisons from one vector to another.

How?

• Adds a Vector_Math class with various public methods that can be used to run similarity calculations against two vector embeddings
• Adds a Vector_Ranker class to score vectors against one another
• Adds another command to our temporary embeddings WP-CLI command to test things

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Fable 5.1, Opus 5
Used for: Planning approach and executing on that. Review of approach and final review and testing by me

Testing Instructions

Easiest way to test is the custom WP-CLI command added as part of this PR:

1. Open a terminal window
2. Ensure you have embeddings generated for at least two posts

wp ai embeddings generate --post-id=1 --dry-run=false --chunk --provider=ollama --model=nomic-embed-text:latest
wp ai embeddings generate --post-id=2 --dry-run=false --chunk --provider=ollama --model=nomic-embed-text:latest

3. Compare the same post against itself:

wp ai embeddings compare 1 1

4. You should see a cosine similarity of 1 and euclidean distance of 0
5. Compare the posts against each other:

wp ai embeddings compare 1 2

6. You should see similarity numbers for each chunk as well as a single number for the overall comparison

Changelog Entry

Added - New Vector_Math and Vector_Ranker classes, allowing users to run similarity and ranking queries against vectors.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: andreilupu <euthelup@git.wordpress.org>
Co-authored-by: ColinM-sys <colinmcdonough@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[codecov]

Codecov Report

❌ Patch coverage is 32.17391% with 234 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.07%. Comparing base (eca57d2) to head (863fe51).

Files with missing lines	Patch %	Lines
includes/CLI/Embeddings_Command.php	0.00%	230 Missing ⚠️
includes/Embeddings/Vector_Ranker.php	90.32%	3 Missing ⚠️
includes/Embeddings/Vector_Math.php	98.79%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #993      +/-   ##
=============================================
- Coverage      81.50%   80.07%   -1.43%     
- Complexity      2923     3013      +90     
=============================================
  Files            122      124       +2     
  Lines          11653    11992     +339     
=============================================
+ Hits            9498     9603     +105     
- Misses          2155     2389     +234     

Flag	Coverage Δ
unit	80.07% <32.17%> (-1.43%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[andreilupu]

Great work on this 👏 To me, the math, classes, and tests are really solid. One thing I hit while reading through the compare command:

When two posts share more than one provider/model pair, there's currently no way to compare them.

In resolve_stored_model() (includes/CLI/Embeddings_Command.php#L708-L714), the multiple-shared-models branch prints the shared pairs formatted as flags:

Posts 42 and 99 share more than one model:
  --provider=openai --model=text-embedding-3-small
  --provider=ollama --model=nomic-embed-text:latest
Error: Posts have more than one model in common. Embed both with the same model, then compare.

Two issues with this:

1. The --provider=... --model=... lines read as flags the user should pass to compare, but compare doesn't accept those flags — its synopsis only has --metric and --pairs.
2. The suggested fix ("Embed both with the same model, then compare") doesn't resolve the situation: the posts already share a model — two of them — and since save() upserts on (object_type, object_id, provider, model, chunk_index), re-embedding with either model won't remove the other shared pair. The only way out today is deleting rows by hand (SQL or Embedding_Repository::delete_for_object() via wp eval), since no delete command is exposed.

This is easy to hit in practice: embed a couple of posts with Ollama while testing locally, later re-embed them with OpenAI, and compare becomes unusable for those posts.

Suggestion: add optional --provider=<provider> and --model=<model> flags to compare (mirroring generate) and use them to disambiguate in resolve_stored_model(). To be clear, this is still a same-model comparison on both sides; the flags only select which shared model's vectors to use, not a way to compare across models. Auto-detection can stay as the default for the common single-shared-model case, and the current error output already prints exactly the flag values the user would need to pass, so the UX would compose nicely.

[dkotter]

When two posts share more than one provider/model pair, there's currently no way to compare them

Thanks for the review. I did initially have model and provider as arguments you could pass in (thus the discrepancy in that error message) but removed that to keep things as simple as possible.

This WP-CLI command is only there for test purposes and the plan is to remove all that code prior to the next release. So I'm happy to make changes there as needed but not overly concerned with making it perfect as it's just a test tool.

Curious if you're seeing actual issues in the embedding code or if this is just an issue with that WP-CLI command? Also worth noting the work is not finished here (as you mention, there isn't a real way to delete embeddings yet). All of the actual sync work will be coming in a new PR.

[andreilupu]

Oh, just the WP-CLI command, the embedding code itself looks solid to me. Vector_Math and Vector_Ranker both check out (I specifically verified the cosine clamp, zero-vector handling, sort directions, and key preservation in rank()).

That explains the error message, now it makes sense, and given the command is throwaway, I wouldn't polish it further.

I'll admit I was hoping these CLI commands would actually ship, though. As a plugin developer, I'd probably reach for wp ai embeddings compare more often than the classes themselves. It's the fastest way to check "did my content get embedded, do these two posts actually look related to the model" without writing SQL against a table of binary blobs.

That said, I get that freezing a debug tool into a public contract before the sync layer exists would be premature. Totally fine waiting for a proper wp ai embeddings suite later.

[dkotter]

I'll admit I was hoping these CLI commands would actually ship, though. As a plugin developer, I'd probably reach for wp ai embeddings compare more often than the classes themselves. It's the fastest way to check "did my content get embedded, do these two posts actually look related to the model" without writing SQL against a table of binary blobs.

That said, I get that freezing a debug tool into a public contract before the sync layer exists would be premature. Totally fine waiting for a proper wp ai embeddings suite later.

So I guess no reason we couldn't leave these types of commands in, just wasn't sure if they'd provide enough value to justify having in place (and having to maintain going forward).

I do think once we have all the pieces in place here, most of the functionality that's in the command will be moved out into various other embedding classes (and the command can then be updated to use as much of that as possible). So your note here around the same post being embedded twice with different models is something we'll want to account for in the larger scope here.