Repository navigation
feat: Support Vercel AI Gateway as an alternative transport - #7
Conversation
Jev is available through the Vercel AI Gateway (evaluation modality), which makes the SDK usable while direct TypeSafe API access is waitlisted. Enable it with JevClient.builder().vercelGateway(). - GatewayProtocol translates primitives to the gateway wire format (spec v4): Noul -> boolean, Choice options -> criteria map, Score legend -> ordered level array, and maps answers back onto JevResponse so existing helpers like isTrue() keep working. - JevClient gains vercelGateway() and gatewayModel() builder options; the shared request construction removes the duplication between evaluate() and evaluateAsync(). Native API behavior is unchanged. - Score with min/max ranges has no gateway equivalent and now fails fast with a JevValidationException in gateway mode.
c46decd to
caa2df3
Compare
| } | ||
| } else if (primitive instanceof Score score) { | ||
| question.put("type", "score"); | ||
| if (score.legend() == null || score.legend().size() < 2) { |
There was a problem hiding this comment.
There is a little bit of a distinction when sending Score natively and through the Gateway:
- Native: Requires either continuous numeric bounds (
min/max) or a discrete rubric mapping (legend). They are mutually exclusive by API design. - Gateway: Strictly requires discrete rubric levels (at least 2 entries in legend/criteria). Pure min/max bounds cannot be sent over the gateway payload.
This check breaks logic; a Score's legend can be null in the case that a min and max range is set, so this will throw an exception for a valid Score object.
During this I noticed that it would be good to add a mutual exclusivity check for range and legend inside the Score model class, do you think you can do that, and then you can fix the check here to check for both the range and the legend, although a little bit out of scope, it's not a problem.
There was a problem hiding this comment.
@Olti1947 Thanks for the review! I checked the official docs and min/max doesn't exist in the Score schema. It takes criteria: an ordered array of 2–10 level descriptions, where the array position is the level number.
Score is now (name, instructions, criteria), min/max and legend are gone, and the gateway mapping became a straight passthrough since it takes the same shape. Tests updated + verified against the live gateway.
Address review feedback on Olti1947#7: a Score with a min/max range (and a null legend) is a valid object, but the gateway check rejected it as if the missing legend were the problem, and a Score with neither a range nor a legend was constructible at all. - Score's compact constructor now enforces the model invariant: exactly one of {min/max range, non-empty legend}, and a range needs both bounds. - GatewayProtocol now reasons about the range explicitly: a range-based Score gives a precise 'cannot be sent over the gateway' error, and the legend branch trusts the model invariant. - Add ScoreTest covering range-only, legend-only, both, neither, empty legend, and partial range.
Follow-up to review feedback on Olti1947#7: the min/max range does not exist in the actual Score schema. Per the official docs, a score question takes 'criteria' — an ordered array of 2 to 10 level descriptions, low end of the scale first, where the position in the array is the level number. - Score is now (name, instructions, criteria) with a varargs convenience constructor; the compact constructor validates 2..10 non-blank levels. min/max and legend are removed. - GatewayProtocol maps criteria straight through — the gateway's score question takes the same ordered array, so the range/legend special cases are gone. - ScoreTest covers the new invariants (level bounds, blank levels, varargs); gateway tests updated to the new shape. Verified end-to-end against the live gateway (interpolated score and per-level probabilities come back as documented).
266582a to
9d11d89
Compare
|
One more thing that came out of the schema check for this PR: the mismatch isn't limited to Score -- the native request/response shapes also differ from the documented API. Filed as #10 with the details. |
Description
Jev is now generally available through the Vercel AI Gateway (evaluation modality, model
typesafe-ai/jev). This PR letsjev-javatalk to Jev via the gateway, which makes the SDK usable today for developers who are still on the TypeSafe API waitlist — a Vercel AI Gateway key (vck_...) is all that's needed. The gateway speaks a different wire protocol (POST /v4/ai/evaluation-model, AI SDK evaluation spec v4) than the native/v1/systemone, so overridingbaseUrlalone doesn't work.Details of Changes
GatewayProtocolclass isolating the gateway wire format. It maps native primitives to gateway questions (Noul→boolean,Choiceoptions → criteria map,Scorelegend → ordered level array) and maps gateway answers back onto the existingJevResponse, so helpers likeisTrue()keep working unchanged.JevClient.Buildergains.vercelGateway()and.gatewayModel(String)(defaulttypesafe-ai/jev). The native API remains the default and is fully unchanged when these are not used.evaluate()andevaluateAsync()into a sharedbuildHttpRequest().Scorewithmin/maxranges has no gateway equivalent, so it now fails fast with a clearJevValidationExceptionin gateway mode.GatewayProtocolTest, no network required) covering primitive→question mapping and response parsing; README section documenting gateway usage.Related Issues
Closes #4
Type of Change
Proposed API Usage
Verified end-to-end against the live gateway: choice, boolean, and score primitives all return correct structured answers (~640ms warm latency).