Repository navigation
Reject non-finite values in embedding vectors - #263
henryperkins wants to merge 5 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## trunk #263 +/- ##
============================================
+ Coverage 87.12% 87.15% +0.02%
- Complexity 1463 1470 +7
============================================
Files 70 70
Lines 4615 4624 +9
============================================
+ Hits 4021 4030 +9
Misses 594 594
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. Props Bot commandsYou can trigger Props Bot actions by commenting on this PR:
|
|
One downstream coordination note: WordPress/wordpress-develop#12530 currently imports PHP AI Client 1.4.0 for the WordPress 7.1 embedding API, so its vendored If this fix looks right, would a 1.4.1 patch release followed by updating #12530 to that tag be the cleanest path? I’m not suggesting this should block the Core wrapper review—just flagging the dependency while the import is still open. |
There was a problem hiding this comment.
Fix looks solid 👍
Verification script
<?php
declare(strict_types=1);
use WordPress\AiClient\Common\Exception\InvalidArgumentException;
use WordPress\AiClient\Results\DTO\Embedding;
require_once __DIR__ . '/vendor/autoload.php';
$failures = 0;
function assertRejected(string $label, float $value): void
{
global $failures;
try {
new Embedding([0.1, $value], 2);
echo "[FAIL] {$label}: expected InvalidArgumentException, none thrown\n";
$failures++;
} catch (InvalidArgumentException $e) {
echo "[PASS] {$label}: rejected (\"{$e->getMessage()}\")\n";
}
}
function assertArrayRejected(string $label, array $values): void
{
global $failures;
try {
new Embedding($values, count($values));
echo "[FAIL] {$label}: expected InvalidArgumentException, none thrown\n";
$failures++;
} catch (InvalidArgumentException $e) {
echo "[PASS] {$label}: rejected (\"{$e->getMessage()}\")\n";
}
}
function assertAccepted(string $label, array $values): void
{
global $failures;
try {
$embedding = new Embedding($values, count($values));
} catch (InvalidArgumentException $e) {
echo "[FAIL] {$label}: expected success, got exception (\"{$e->getMessage()}\")\n";
$failures++;
return;
}
$json = json_encode($embedding);
if ($json === false) {
echo "[FAIL] {$label}: constructed but json_encode() failed (\"" . json_last_error_msg() . "\")\n";
$failures++;
return;
}
echo "[PASS] {$label}: accepted, encodes to {$json}\n";
}
echo "-- Non-finite values (must be rejected) --\n";
assertRejected('NAN', NAN);
assertRejected('INF', INF);
assertRejected('-INF', -INF);
echo "\n-- Non-finite values in longer/other positions (must still be rejected) --\n";
assertArrayRejected('trailing INF', [0.1, 0.2, INF]);
assertArrayRejected('trailing NAN', [0.1, 0.2, NAN]);
assertArrayRejected('trailing -INF', [0.1, 0.2, -INF]);
assertArrayRejected('leading INF', [INF, 0.1, 0.2]);
echo "\n-- Finite values (must still be accepted) --\n";
assertAccepted('floats', [0.1, 0.2]);
assertAccepted('integers', [1, 2]);
assertAccepted('mixed int/float', [1, 0.5]);
echo "\n" . ($failures === 0 ? "All checks passed.\n" : "{$failures} check(s) failed.\n");
exit($failures === 0 ? 0 : 1);Straightforward, correct fix. is_finite() seems right guard, it catches NAN/INF/-INF, so every ordinary float and every int passes through untouched.
I was thinking would it be simpler to handle the type and finiteness checks directly in the existing validation loop and remove isEmbeddingList()? Something like:
if (!is_array($values) || !array_is_list($values)) {
throw new InvalidArgumentException('Embedding values must be a list array.');
}
foreach ($values as $value) {
if (!is_int($value) && !is_float($value)) {
throw new InvalidArgumentException('Embedding values must be integers or floats.');
}
if (is_float($value) && !is_finite($value)) {
throw new InvalidArgumentException('Embedding values must be finite numbers.');
}
}|
@dkotter this worth getting into 1.5.0 before continuing with that release process? |
|
|
||
| foreach ($values as $value) { | ||
| if (is_float($value) && !is_finite($value)) { | ||
| throw new InvalidArgumentException('Embedding values must be finite numbers.'); |
There was a problem hiding this comment.
I'm wondering if this should be RuntimeException instead of InvalidArgumentException? The realistic trigger here is someone generates an embedding result and it gets passed in here where it errors out. So it's not necessarily an argument the user is passing in, it's an invalid return from the LLM. We use RuntimeException elsewhere for those cases but likely splitting hairs here.
May be nice to return the index and value so a user can track down where things went wrong though
There was a problem hiding this comment.
Updated in 3640f3b. I kept the SDK’s custom InvalidArgumentException and added the component index, with NAN, INF, and -INF labels. Other invalid values are reported by type, so arrays and objects aren’t converted to strings.
Keeping that exception also preserves EmbeddingResult::isArrayShape(), which catches it and returns false.
I agree with your point about bad provider responses. Would keeping the DTO exception work for you, with provider-side translation handled separately?
There was a problem hiding this comment.
We could likely have better test coverage here. For instance, no new tests for EmbeddingResult are here, which is the most likely path someone will use in production. May also want tests to verify things like PHP_FLOAT_MAX, -0.0, 0.0, and plain ints still pass.
There was a problem hiding this comment.
Added the boundary tests: positive and negative PHP_FLOAT_MAX, integers, signed zero, and successful JSON encoding. These check value and type preservation, including the zero’s sign.
I also added EmbeddingResult tests for NAN, INF, and -INF in a later vector through fromArray(), plus direct-constructor normalization and isArrayShape() returning false.
The tests pass on PHP 7.4, 8.0, 8.4, and 8.5.
| throw new InvalidArgumentException('Embedding values must be integers or floats.'); | ||
| } | ||
|
|
||
| foreach ($values as $value) { |
There was a problem hiding this comment.
As is mentioned elsewhere, I'd move this into isEmbeddingList so we aren't iterating over everything twice. Those iterations add up when dealing with 3072 dimension vector, for instance
There was a problem hiding this comment.
Folded the finite check into isEmbeddingList(). Valid vectors now get one element-validation pass. The second scan only runs after validation fails, to find the index and value for the error.
List and dimension checks are unchanged. I also updated the message to say “integers or finite floats.” Lint passes, including PHPStan and PHP compatibility checks. Could you take another look?
Summary
NAN,INF, and-INFat theEmbeddingconstructor boundary.Fixes #262.
Why
Non-finite values satisfy PHP's
is_float()check but cannot be represented in JSON. Rejecting them when constructing anEmbeddingprevents an invalid result object from failing later during serialization, logging, caching, persistence, or custom provider handling.Testing
A test-only commit first demonstrated the gap: all three new cases failed because no
InvalidArgumentExceptionwas thrown.After the implementation commit:
AI assistance