diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index bbf98ad..eeb9580 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -12,7 +12,7 @@ "name": "use-crystallize", "source": "./use-crystallize", "description": "Everything you need to use Crystallize with your agent, skills, agents, commands, hooks and MCP servers.", - "version": "3.5.0", + "version": "3.6.0", "category": "commerce", "tags": [ "commerce", diff --git a/README.md b/README.md index 5b89d96..7912df7 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ This repo contains three sub-projects: | Project | Path | Description | | -------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------- | -| **Skills** | `use-crystallize/skills/` | Markdown-based skill modules for AI agents (query, mutation, content-model, pricing, permissions, etc.) | +| **Skills** | `use-crystallize/skills/` | Markdown-based skill modules for AI agents (query, mutation, content-model, pricing, vector-ranking, etc.) | | **MCP Server** | `use-crystallize/mcp-servers/crystallize/` | Cloudflare Workers MCP server providing authenticated access to Crystallize APIs | | **Docs** | `docs/` | Astro Starlight documentation site deployed to [crystallizeapi.github.io/ai](https://crystallizeapi.github.io/ai) | @@ -57,7 +57,7 @@ bun type-check # TypeScript type checking Skills are plain markdown files in `use-crystallize/skills/` — no build step required. Each skill has a `SKILL.md` with YAML frontmatter and an optional `references/` directory with supporting docs. -Available skills: `content-model`, `data-creation`, `information-architecture`, `js-api-client`, `mass-operations`, `mutation`, `permissions`, `plugins`, `pricing`, `query`, `taxonomy` — the directory itself is the authoritative list. +Available skills: `content-model`, `data-creation`, `information-architecture`, `js-api-client`, `mass-operations`, `mutation`, `permissions`, `plugins`, `pricing`, `query`, `taxonomy`, `vector-ranking` — the directory itself is the authoritative list. ## Using the Claude Plugin diff --git a/use-crystallize/.claude-plugin/plugin.json b/use-crystallize/.claude-plugin/plugin.json index 61d537c..be4f747 100644 --- a/use-crystallize/.claude-plugin/plugin.json +++ b/use-crystallize/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "use-crystallize", - "version": "3.5.0", + "version": "3.6.0", "description": "Everything you need to use Crystallize with your agent, skills, agents, commands, hooks and MCP servers.", "author": { "name": "Crystallize", diff --git a/use-crystallize/skills/mutation/SKILL.md b/use-crystallize/skills/mutation/SKILL.md index 0a8762a..443b4c4 100644 --- a/use-crystallize/skills/mutation/SKILL.md +++ b/use-crystallize/skills/mutation/SKILL.md @@ -191,3 +191,9 @@ If the user is working in a JS/TS project, prefer generating code using `@crysta - [Core API Mutations](references/core-api.md) - Item CRUD, variants, components, customers, publish/unpublish, delete, media uploads - [Shop API Cart Mutations](references/shop-api-mutations.md) - Cart hydration, item management, checkout flow, cart lifecycle - [Shop API Order Mutations](references/shop-api-order-mutations.md) - Order creation (from cart or direct), payments, pipelines, metadata + +## Related skills + +[[query]] covers reads across the same APIs. For bulk writes that would otherwise hit rate limits, use +[[mass-operations]]. Vector ranking has its own Core API mutations — `upsertVocabulary`, `setItemTaste` +and `igniteDiscoApi` — documented in [[vector-ranking]]. diff --git a/use-crystallize/skills/mutation/references/core-api.md b/use-crystallize/skills/mutation/references/core-api.md index 1f9f3e3..a3ca47e 100644 --- a/use-crystallize/skills/mutation/references/core-api.md +++ b/use-crystallize/skills/mutation/references/core-api.md @@ -13,6 +13,7 @@ See [SKILL.md](../SKILL.md) for endpoint URLs and authentication headers. - [Order Mutations](#order-mutations) - Update order metadata - [Media & Images](#media--images) - Upload images for items and variants - [Flow Mutations](#flow-mutations) - Manage item workflows +- [Vector Ranking Mutations](#vector-ranking-mutations) - Vocabularies, item taste, re-indexing - [Error Handling](#error-handling) --- @@ -555,6 +556,70 @@ mutation SetFlowStage { --- +## Vector Ranking Mutations + +Discovery's vector ranking is authored entirely on the Core API. Four calls, in this order: + +| Mutation | Notes | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `upsertVocabulary(input: UpsertVocabularyInput!)` | **Full replace**, not a patch — omitted dimensions are dropped | +| `setItemTaste(input: SetItemTasteInput!)` | One item, one language, one vocabulary. Writes the **draft** | +| `publishItems(ids: [ID!]!, language: String!)` | The indexer reads the published version — skipping this fails silently | +| `igniteDiscoApi(stacks: opensearch)` | Async; poll `bulkTask(id:)` until `complete`, then allow propagation. `stacks: opensearch` is required for vectors to be built | + +```graphql +mutation UpsertVocabulary($input: UpsertVocabularyInput!) { + upsertVocabulary(input: $input) { + name + dimensions { + id + weight + } + lastUpdated + } +} + +mutation SetItemTaste($input: SetItemTasteInput!) { + setItemTaste(input: $input) { + __typename + ... on Product { + id + } + ... on BasicError { + errorName + message + } + } +} + +mutation Index { + igniteDiscoApi(stacks: opensearch) { + __typename + ... on BulkTaskIgnition { + id + type + status + createdAt + } + ... on BasicError { + errorName + message + } + } +} +``` + +All three results are unions whose error members implement `BasicError`, so a single fragment covers +every failure and `errorName` identifies it. `setItemTaste` and `igniteDiscoApi` can both return +`ExperimentalFeaturesNotAvailableError`, which means vectors are not enabled for the tenant. + +Read back with `vocabulary(name:)` and `item(id:, language:) { taste { vocabulary entries { key weight } } }`. + +**Re-run `igniteDiscoApi` after every change to vocabularies or taste entries** — an unindexed change +has no effect and raises no error. Omitting `stacks: opensearch` likewise fails silently: the index +rebuilds, but without vectors. Full guidance, including vocabulary design, positional weights and +key validation, is in the [[vector-ranking]] skill. + ## Error Handling The Core API uses union return types. Always handle potential errors: diff --git a/use-crystallize/skills/query/SKILL.md b/use-crystallize/skills/query/SKILL.md index 7dc6010..5e52d49 100644 --- a/use-crystallize/skills/query/SKILL.md +++ b/use-crystallize/skills/query/SKILL.md @@ -26,6 +26,8 @@ Before writing queries, understand the context. Ask clarifying questions: - Know the exact path? Need strong consistency? → **Catalogue API** - Admin interface? Orders, customers, shapes? → **Core API** - Cart/checkout operations? → **Shop API** +- Need results _ordered_ by relevance rules, personalization, or similarity? → **Discovery API** with + `rankBy` / `context` / `nearestTo` — see [[vector-ranking]] ## How It Works @@ -114,7 +116,9 @@ The Discovery API is the primary API for frontend development. It supports: - Faceted navigation - Sorting and cursor-based pagination -The Discovery API has two entry points: `search` for full-text queries with facets, and `browse` for shape-typed access where each shape becomes its own query type. +The Discovery API has three entry points: `search` for full-text queries across all shapes, `browse` for shape-typed access where each shape becomes its own query type, and `autocomplete` for type-ahead on `name`. A fourth query, `topics`, walks the topic map. + +**The Discovery schema is generated per tenant** from its shapes and index settings — filter, facet and sort fields differ between tenants, and ranking arguments exist only on tenants served for ranking. Introspect rather than assume. > **Note**: The Discovery API uses lowercase type names in inline fragments (`... on product`, `... on category`) because types are derived from your shape identifiers. You can still use it for interface (`... on Product`, `... on Folder`). @@ -226,11 +230,18 @@ query { 5. **Handle async updates** - Discovery API may have sub-second delay for recently published content 6. **Protect APIs in production** - Configure authentication for sensitive data 7. **Use Core API for complex filters** - Only Core API supports filtering orders by customer, SKU, payment provider +8. **Detect the Discovery schema, don't hardcode it** - Filter/sort/facet fields and the ranking arguments are tenant-generated; introspect before building a query ## References - [Core API Queries Reference](references/core-api.md) - Items, customers, orders, shapes with advanced filtering -- [Discovery API Reference](references/discovery-api.md) - Detailed search, filter, and faceting documentation +- [Discovery API Reference](references/discovery-api.md) - Search, browse, autocomplete, filters, facets, sorting, fuzzy matching, pagination and profiling - [Catalogue API Reference](references/catalogue-api.md) - Path-based query documentation - [Shop API Queries Reference](references/shop-api-queries.md) - Cart and checkout query documentation (`/cart` endpoint) - [Shop API Order Queries Reference](references/shop-api-order-queries.md) - Order queries by ID or customer (`/order` endpoint) + +## Related skills + +Reading is only half of it — [[mutation]] covers writes across the same APIs, and [[js-api-client]] +wraps all of them for JS/TS. For ranking Discovery results by relevance rules, a shopper's taste, or +similarity to another item, use [[vector-ranking]]. diff --git a/use-crystallize/skills/query/references/discovery-api.md b/use-crystallize/skills/query/references/discovery-api.md index 758ab3a..1bbb925 100644 --- a/use-crystallize/skills/query/references/discovery-api.md +++ b/use-crystallize/skills/query/references/discovery-api.md @@ -1,6 +1,7 @@ # Discovery API Reference -The Discovery API is the primary API for powering storefronts with product information and marketing content. It is a read-only API optimized for high performance. +The Discovery API is the primary API for powering storefronts with product information and marketing +content. It is a read-only API optimized for high performance. ## Base URL @@ -8,32 +9,57 @@ The Discovery API is the primary API for powering storefronts with product infor https://api.crystallize.com/{tenant-identifier}/discovery ``` -Replace `{tenant-identifier}` with your tenant name. +Replace `{tenant-identifier}` with your tenant name — the **bare** identifier, no `@` (that prefix +belongs to the Core API). ## Authentication -By default, the Discovery API is open. If you configure restricted access for your Catalogue API, you need to provide authentication: +By default, the Discovery API is open. If you configure restricted access for your Catalogue API, you +need to provide authentication: - Static token via header - Access tokens for programmatic access > **Important**: Always secure your API with authentication in production environments. -## Key Features +## The schema is generated per tenant -### Semantic Schema +This is the single most important thing to know before writing a query. The Discovery schema is +**derived from the tenant's shapes and index settings**, so it differs between tenants and changes when +the tenant is re-indexed: -The Discovery API follows the structure of your shapes and components. Field names in queries match your shape definitions, making the API intuitive to use. +- Every shape becomes a type and a `browse` field — `product`, `category`, `brand`, … +- Filter, facet and sort inputs (`TenantFilter`, `ProductFacet`, `TenantSort`, …) are generated from + indexed component fields — `price_default`, `stock_oslo`, `specs_label`, `variants_topics`, … +- `TenantLanguage` is an enum of the tenant's languages +- Ranking inputs and their enums appear **only** on tenants served for ranking (see below) -### Combined Browse and Search +**Introspect, do not assume.** An un-ignited tenant answers +`{"success": false, "message": "There is no ignited Tenant for ."}` rather than serving a +schema at all. -One query model for both browsing categories and searching products. This simplifies frontend development. +> **Note**: The Discovery API uses **lowercase** type names in inline fragments (`... on product`, +> `... on category`) because types are derived from your shape identifiers. Interface fragments keep +> their capital (`... on Product`, `... on Folder`, `... on Document`). -### Filtering and Faceting +## Queries -Filter by any attribute, component, or price range. Get facet counts for building filter UIs. +| Query | Use for | +| -------------- | ----------------------------------------------------------------------------------- | +| `search` | Full-text search across **all** shapes; polymorphic hits | +| `browse` | Shape-typed access — each shape becomes its own query with all its component fields | +| `autocomplete` | Type-ahead; hardcoded on `name` | +| `topics` | Children of a topic in the topic map | -## Query Structure +`search`, `autocomplete` and every field under `browse` take the same argument set: + +```text +language, publicationState, path, pathResolutionMethod, term, +pagination, options, rankBy, context, nearestTo, filters, facets, sorting +``` + +`Folder.children` and `Topic.items` take that same set, which is what makes nested category listings +filterable and rankable in one round trip. ### Basic Search @@ -42,18 +68,23 @@ Filter by any attribute, component, or price range. Get facet counts for buildin search(language: en, filters: { type_in: [product] }, pagination: { limit: 20, after: "XXXX" }) { summary { totalHits - facets + hasMoreHits endCursor: endToken + facets } hits { id name path + shape + score } } } ``` +Hits are polymorphic — use one inline fragment per shape, and read `shape` to tell them apart. + ### Filtering by Shape ```graphql @@ -71,7 +102,7 @@ Filter by any attribute, component, or price range. Get facet counts for buildin ```graphql { - search(language: en, filters: { price_sales: { range: { gte: 20, lte: 20 } } }) { + search(language: en, filters: { price_sales: { range: { gte: 20, lte: 100 } } }) { hits { name path @@ -93,9 +124,64 @@ Filter by any attribute, component, or price range. Get facet counts for buildin } ``` +## Filter operators + +Filters compose with `AND` and `OR`, each taking a list of nested filters. + +| Input | Operators | +| ------------------------------ | ------------------------------------------------------------------------------------------------------------ | +| `StringFilter` | `exists`, `equals`, `not_equals`, `in`, `not_in`, `contains`, `not_contains`, `phrase`, `regex`, `not_regex` | +| `StringFilterWithAutocomplete` | the above plus `autocomplete: { term, options }` | +| `NumberFilter` | `exists`, `equals`, `not_equals`, `in`, `not_in`, `range: { gt, gte, lt, lte }` | +| `DateFilter` | `exists`, `equals`, `not_equals`, `in`, `not_in`, `range: { gt, gte, lt, lte }` | +| `BooleanFilter` | `exists`, `equals`, `not_equals` | + +`type_in: [ItemType]` (`product`, `document`, `folder`) is the common way to narrow a `search`. + +## Typo tolerance (fuzzy search) + +Search is **exact by default**. Opt into typo tolerance through `options.fuzzy`: + +```graphql +{ + search(language: en, term: "gren", options: { fuzzy: { fuzziness: SINGLE, prefixLength: 1 } }) { + hits { + name + path + } + } +} +``` + +| Option | Default | Meaning | +| --------------- | ------- | ------------------------------------------------------------- | +| `fuzziness` | `NONE` | Max single-character edits: `NONE`, `SINGLE`, `DOUBLE` | +| `prefixLength` | `0` | Leading characters that must match exactly before edits apply | +| `maxExpensions` | `50` | Max term variations generated | + +Raising `fuzziness` widens the candidate set and costs latency — prefer `SINGLE` before `DOUBLE`, and +use `prefixLength` to keep short, common terms precise. + +## Autocomplete + +```graphql +{ + autocomplete(language: en, term: "espr", pagination: { limit: 8 }) { + hits { + name + path + } + } +} +``` + +`autocomplete` matches on `name` and otherwise takes the same arguments as `search` — including filters +and ranking. + ## Browse Queries -The `browse` API provides **shape-typed access** — each shape becomes its own query type with all component fields available directly. This is the recommended approach for storefronts. +The `browse` API provides **shape-typed access** — each shape becomes its own query type with all +component fields available directly. This is the recommended approach for storefronts. ### Browse by Shape @@ -103,6 +189,11 @@ The `browse` API provides **shape-typed access** — each shape becomes its own { browse { product(language: en, pagination: { limit: 25 }) { + summary { + totalHits + hasMoreHits + endCursor + } hits { name path @@ -148,6 +239,7 @@ The `browse` API provides **shape-typed access** — each shape becomes its own - Use `path: "/shop/*"` for direct children (wildcard) - Use `path: "/shop/exact-item"` for a specific item +- `pathResolutionMethod` (`canonical`, `alias`, `history`, `shortcut`) controls how a path is resolved - Use aliases to combine multiple browse queries in one request ### Combined Query with Aliases @@ -177,11 +269,31 @@ The `browse` API provides **shape-typed access** — each shape becomes its own } ``` +## Sorting + +`sorting` takes one or more generated fields plus `score`, each `asc` or `desc`: + +```graphql +{ + browse { + product(language: en, sorting: { price_default: asc, itemId: asc }) { + hits { + name + path + } + } + } +} +``` + +**Always add a deterministic secondary field** (such as `itemId`) so pagination stays stable across +pages. Note that `sorting` does **not** compose predictably with ranking — see below. + ## Pagination ### Cursor-Based Pagination (Recommended) -Use `paginationToken` (returned as `endCursor` in summary) for efficient, consistent pagination: +Use `paginationToken` (returned as `endToken` in summary) for efficient, consistent pagination: ```graphql { @@ -190,7 +302,7 @@ Use `paginationToken` (returned as `endCursor` in summary) for efficient, consis summary { totalHits hasMoreHits - endCursor + endCursor: endToken } hits { name @@ -204,20 +316,25 @@ Use `paginationToken` (returned as `endCursor` in summary) for efficient, consis **Flow:** 1. First request: omit `after` (or set to `null`) -2. Use `summary.endCursor` as the `after` value for the next page +2. Use `summary.endToken` as the `after` value for the next page 3. Stop when `summary.hasMoreHits` is `false` -> **Note**: `skip`-based pagination is deprecated. Use cursor-based pagination (`after`) for all new implementations. `skip` becomes increasingly expensive on large result sets. +`pagination` accepts `limit`, `after`, `before` and `skip`. + +> **Note**: `skip`-based pagination is deprecated for ordinary queries. Use cursor-based pagination +> (`after`). `skip` becomes increasingly expensive on large result sets. **The exception is ranked +> queries** — see below. + +## Faceting -The same cursor pattern works with `search`: +Get counts for filter values. `StringFacet` takes `key` and `limit`; `NumberFacet` and `DateFacet` also +require `boundaries`. ```graphql { - search(language: en, pagination: { limit: 20, after: "CURSOR" }) { + search(language: en, term: "blue", facets: { shape: { limit: 5 } }) { summary { - totalHits - hasMoreHits - endCursor + facets } hits { name @@ -227,34 +344,62 @@ The same cursor pattern works with `search`: } ``` -## Faceting +`summary.facets` is a `Hash`, and accepts an optional `key` argument to pull a single facet out. + +`summary.priceRange(priceIdentifier: "default", quantity: 1) { min max }` gives the price bounds of the +current result set — useful for a range slider that matches the active filters. + +## Ranking, personalization and similarity + +Ranking-enabled tenants additionally accept `rankBy`, `context` and `nearestTo`, and expose `rankScore` +and `rankExplain` on hits. That surface — vocabularies, `setItemTaste`, `igniteDiscoApi`, the five +`rankBy` signals, `context.userTaste`, `nearestTo` and the rerank window — is covered by the +[[vector-ranking]] skill. + +Two things to know from here: -Get counts for filter values: +1. **The arguments are absent from the schema until the tenant is served for ranking.** Referencing them + on an ordinary tenant is a GraphQL validation error, not an unranked result. Detect the capability: + `{ __type(name: "RankByInput") { name } }`. +2. **Ranked queries page differently.** When a rerank runs, cursor tokens fall back to offset + pagination — use `skip` + `limit`, and remember that `skip` offsets into a bounded rerank window + (`options.rerankWindow`, default 500, cap 2000). + +## Profiling + +Every query can report how it was served: ```graphql { - search(language: en, term: "blue", facets: { shape: { limit: 5 } }) { + search(language: en, term: "chair") { summary { - facets - } - hits { - name - path + profiling { + executionTime + queryEngine + collection + webNode + lastIndexCompletedAt + } } } } ``` +`lastIndexCompletedAt` is the reliable way to confirm a re-index actually landed. + ## Async Updates -The Discovery API is asynchronously updated from your published data and therefore eventually consistent: +The Discovery API is asynchronously updated from your published data and therefore eventually +consistent: - Typical delay: under 1 second - Large imports may take longer to surface +- A full re-index (`igniteDiscoApi`) takes minutes, not seconds, to propagate For cases requiring exact current state, use the Catalogue API instead. ## Related Links +- [[vector-ranking]] — ranking, personalization and similarity - [Crystallize Discovery API Documentation](https://crystallize.com/docs/developer/apis/discovery-api) - [Demo tenant: Furnitut](https://www.furnitut.com/) diff --git a/use-crystallize/skills/vector-ranking/SKILL.md b/use-crystallize/skills/vector-ranking/SKILL.md new file mode 100644 index 0000000..8b41c4b --- /dev/null +++ b/use-crystallize/skills/vector-ranking/SKILL.md @@ -0,0 +1,209 @@ +--- +name: vector-ranking +description: > + Rank and personalize the Crystallize Discovery catalogue with vectors — author vocabularies and item + taste on the Core API, then rank with rankBy, context.userTaste and nearestTo on Discovery. Use when + the user wants personalized search or category ordering, "more like this" / similar-product / + pairing recommendations, cold-start onboarding from a few picks, boosting by margin, stock, sales + velocity, newness, campaign priority or review score, penalizing by return rate, explaining why a + product ranks where it does, or ranking a catalogue for one shopper, buyer or learner. Trigger on + "vector", "vector ranking", "vector search", "personalization", "taste", "vocabulary", "rankBy", + "rankScore", "rankExplain", "tasteCosine", "nearestTo", "userTaste", "upsertVocabulary", + "setItemTaste", "igniteDiscoApi", "rerankWindow", "cosine similarity", "recommendations", + "more like this", "boost by margin", "relevance tuning" — and on any request to reorder Discovery + results by something other than a plain sort field. +metadata: + author: Crystallize + version: "1.0" +--- + +# Crystallize Vector Ranking + +Discovery ranks the whole catalogue for the person in front of it, inside the same query you already use +for search and browse. You describe products in **a vocabulary of your own** — roast, flavour, fit, +finish, use case, margin band. A shopper is described in that same vocabulary. The index scores every +item by cosine similarity and sorts. Then you put your own weighted rules on top, and every position +can come back with an explanation. + +**These are sparse, author-supplied, named vectors — not dense ML embeddings, and not a trained model.** +Nothing is learned; the vocabulary is the model. There is nothing to train and nothing to sync. + +> Verified on 2026-09-15 against the live Core API (`upsertVocabulary`, `vocabulary`, `setItemTaste`, +> `igniteDiscoApi`) and against a ranking-enabled Discovery tenant by introspection, and reconciled with +> the Discovery vector-search documentation as of the same day. Where the public docs and the served +> schema disagree, this skill follows the schema and says so. Ranking is a capability that is enabled +> per tenant — how it is served is an implementation detail and is deliberately not documented here. + +## Detect the capability before you write a query + +**This is the failure mode that catches agents.** Ranking is enabled **per tenant**. Until it is, +`rankBy`, `context` and `nearestTo` are **absent from that tenant's Discovery schema** — referencing them +is a GraphQL validation error, not a query that quietly returns unranked results. The surface can also +disappear again if ranking is disabled for a tenant. + +```graphql +# Probe once, cache the answer. Non-null => this tenant is served for ranking. +{ + __type(name: "RankByInput") { + name + } +} +``` + +An un-ignited tenant does not even serve the schema: + +```json +{ "success": false, "message": "There is no ignited Tenant for ." } +``` + +A tenant that has ranking enabled but still refuses a vector argument answers with an error stating +that ranking is not available for this tenant. Do not match on the message text — treat the presence of +the argument in the schema as the capability check. + +Three enums are generated **per tenant**, so they are never hardcodable: + +| Enum | Built from | Gotcha | +| ---------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------- | +| `TenantVocabularyIdentifier` | your vocabularies | a new vocabulary is only a valid value **after the next index run** | +| `TenantRankByField` | NUMBER and DATE **filterable** attributes, **facet fields excluded** | it is _not_ "any numeric field" — introspect it | +| `TenantRankByTieBreaker` | sortable fields (token, number, date) | `tieBreaker` is **required** on every `rankBy` | + +Introspect all three rather than guessing: + +```graphql +{ + v: __type(name: "TenantVocabularyIdentifier") { + enumValues { + name + } + } + f: __type(name: "TenantRankByField") { + enumValues { + name + } + } + tb: __type(name: "TenantRankByTieBreaker") { + enumValues { + name + } + } +} +``` + +## Two APIs + +| API | Endpoint | Auth | Used for | +| ------------- | ------------------------------------------------ | ----------------- | ------------------------------------------------- | +| **Core** | `https://api.crystallize.com/@/core` | access token pair | vocabularies, taste entries, publishing, indexing | +| **Discovery** | `https://api.crystallize.com//discovery` | none | querying | + +Core takes `@tenant` with an at-sign; Discovery takes the bare `tenant`. Vocabulary and index mutations +live on **Core**, not on the PIM API. Everything before Discovery is authenticated and server-side; the +queries themselves need no credentials, so a browser can build the shopper vector client-side and call +Discovery directly. + +## Five concepts + +| Concept | What it is | +| ------------------ | ------------------------------------------------------------------------------------------------------------- | +| **Vocabulary** | A named set of dimensions, each carrying a weight list. A tenant can have several; each is scored on its own. | +| **Dimension** | One axis inside a vocabulary — `roast`, `flavor`, `region`. Carries a weight list. | +| **Entry** | A `dimensionId:value` key attached to one item, **in a meaningful order**. | +| **Item vector** | Built at index time. Every entry becomes a key with the weight its dimension and position give it. | +| **Shopper vector** | The same shape, sent with the query as `context.userTaste`. | + +Two properties fall out of this and drive every design decision: + +- **Shoppers and products share the keys.** A shopper who likes `flavor:chocolate` at `1.0` is compared + to products carrying `flavor:chocolate`. Nothing has to be learned. +- **Vocabularies are summed.** One shopper vector per vocabulary; the index adds the per-vocabulary + cosines. A product matching two vocabularies outranks a product matching one. + +## The pipeline + +```text +Core upsertVocabulary once per vocabulary — FULL REPLACE, not a patch +Core setItemTaste once per item, per vocabulary — writes the DRAFT +Core publishItems the step that is easy to miss +Core igniteDiscoApi(stacks: opensearch) poll bulkTask until "complete", then let it propagate +Discovery search(rankBy:) your rules: margin, stock, velocity, recency +Discovery search(context:) ranked for this shopper +Discovery search(rankBy: + context:) your rules, plus this shopper +Discovery search(nearestTo:) ranked by a reference item +``` + +**Re-run the index after any change to vocabularies or taste entries** — not only the first time. An +unindexed change has no effect and raises no error. `stacks: opensearch` is required for vectors to be +built; see step 5 in the authoring reference. + +Authoring detail — vocabulary design, positional weights, key validation, publishing and indexing — is +in [references/vocabulary-authoring.md](references/vocabulary-authoring.md). + +## Choosing the query path + +| You want | Use | Notes | +| ------------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------ | +| A house order for everyone (margin, stock, velocity, newness) | `rankBy` alone | The baseline every personalized result builds on | +| Results ordered for this shopper | `context.userTaste` alone | Reranks toward taste automatically; no `terms`, so `rankExplain` is null | +| Your rules **and** this shopper, weighted independently | `rankBy` + `context`, with a `tasteCosine` term | The only way to weight vocabularies separately | +| "Similar products", basket pairings | `nearestTo` | `k` replaces `pagination.limit`; anchor is excluded | +| Neighbours that also respect stock/margin | `nearestTo` + `rankBy` with `tasteCosine`, `from: nearestTo` | `nearestTo` alone ignores `context.userTaste` | + +Signals, normalization, `rankScore`/`rankExplain` and the rerank window are in +[references/ranking-signals.md](references/ranking-signals.md). Shopper vectors, `magnitude`, where the +vector comes from and `nearestTo` are in [references/personalization.md](references/personalization.md). + +## Where the vector arguments are accepted + +Verified by introspection on a ranking-enabled tenant — `rankBy`, `context` and `nearestTo` are on: + +- `search` and `autocomplete` +- **every** shape field under `browse` (`browse { product(...) }`, `browse { category(...) }`, …) +- `Folder.children` and `Topic.items` + +They are **not** on `Topic.children`, which takes `language` only. Note the naming: the public docs say +"the children queries on topics and folders", but the field on `Topic` is `items`. + +## Failure modes + +Four of these produce **no error at all** — they are the reason this skill exists. + +| Symptom | Cause | Fix | +| --------------------------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| Order unrelated to taste, no errors | Taste is on the draft only | `publishItems`, index again, retest | +| Order unchanged after editing taste | No index run since the change | `igniteDiscoApi(stacks: opensearch)`, wait for `complete` | +| Index rebuilt, still no vectors | `igniteDiscoApi` run without `stacks: opensearch` | Re-run with `stacks: opensearch` | +| Rankings subtly wrong, no errors | `magnitude` miscomputed client-side | Assert `sqrt(Σ w²)` against a known case | +| `rankScore` comes back `null` | No rerank ran — no `rankBy`, or every term had `weight: 0` | Give at least one term a non-zero weight | +| Vocabulary "does not exist in enum" | No index run since it was created | `igniteDiscoApi(stacks: opensearch)`, wait for `complete` | +| `context` / `rankBy` unknown in schema | Ranking not enabled for the tenant, or not indexed | Confirm ranking is enabled, then index and wait for `complete` plus propagation. Detect the capability, don't assume it | +| `Malformed taste entry key` | Key has no colon | Use `dimensionId:value` | +| `Unknown dimension` | Prefix is not a declared dimension | Add it to the vocabulary, or fix the key | +| `tieBreaker` validation error | Required field missing | Add a `tieBreaker` | +| `ExperimentalFeaturesNotAvailableError` | Vectors not enabled for the tenant | Enable the feature before authoring taste | +| Page 2 empty under ranking | `skip` ran past the rerank window | Raise `options.rerankWindow` (default 500, cap 2000) | + +## How ranking composes with the rest of the query + +- **`filters`, `facets` and `term` apply as usual.** Ranking decides the order **inside** the filtered + set; it does not change which items match. +- **Items without a vector are still returned.** They score 0 on taste and sort after the ones that + match, rather than disappearing. `tasteCosine` contributes 0 for them, so unmapped products fall back + to your other rules. +- **Ranking happens inside a bounded window** of the top matches — see `rerankWindow` in + [references/ranking-signals.md](references/ranking-signals.md). +- **Cursor pagination degrades under ranking.** When a rerank runs, `after`/`before` fall back to offset + pagination — page with `skip` and `limit`, and size the window for the depth you intend to page. +- **Do not combine `sorting` with ranking.** The public docs contradict themselves here (one page says + sorting selects which candidates enter the window, another says it is applied after the ranking + score). When `rankBy` or `context` is present the reranked order is what you get back — treat any + `sorting` you pass as unspecified behaviour. + +## Related skills + +Use [[query]] for the rest of the Discovery API — filters, facets, pagination, browse vs search. +Use [[mutation]] for the Core API mutations these sit next to, and [[js-api-client]] to call either +from JS/TS. Vocabularies overlap conceptually with topic maps: [[taxonomy]] designs the classification, +and a taste entry can be **derived** from a topic assignment — see "Keep one source of truth" in +[references/vocabulary-authoring.md](references/vocabulary-authoring.md). [[content-model]] covers the +shapes whose numeric components become `TenantRankByField` values. diff --git a/use-crystallize/skills/vector-ranking/references/personalization.md b/use-crystallize/skills/vector-ranking/references/personalization.md new file mode 100644 index 0000000..fc45791 --- /dev/null +++ b/use-crystallize/skills/vector-ranking/references/personalization.md @@ -0,0 +1,216 @@ +# Personalization & Similarity (`context.userTaste`, `nearestTo`) + +Discovery API, `https://api.crystallize.com//discovery`, no auth. **Nothing about the shopper is +stored in Crystallize** — you send the vector with each query. + +## `context.userTaste` + +```text +UserTasteInput { vocabulary: TenantVocabularyIdentifier! # enum, generated per tenant + weights: JSON! # { "dimensionId:value": number } + magnitude: Float! } # L2 norm of weights + +ContextInput { userTaste: [UserTasteInput!]! } # one entry per vocabulary +``` + +`userTaste` must be **non-empty**, and a vocabulary may appear **at most once**. `weights` maps +`dimensionId:value` to a number — **positive means likes, negative means dislikes**. + +Weight keys contain a colon, which is not a valid GraphQL name, so **pass the context as a variable**, +never inline in the query document. + +```graphql +query SearchWithTaste($context: ContextInput!) { + search(context: $context, pagination: { limit: 24 }) { + summary { + totalHits + } + hits { + ... on product { + itemId + name + rankScore + topicPaths(leafOnly: true) + defaultVariant { + sku + defaultPrice + firstImage { + url + } + } + } + } + } +} +``` + +```json +{ + "context": { + "userTaste": [ + { + "vocabulary": "taste", + "weights": { "flavor:berry": 1.0, "roast:light": 1.0 }, + "magnitude": 1.4142135623730951 + }, + { + "vocabulary": "origin", + "weights": { "region:kenya": 1.0 }, + "magnitude": 1.0 + } + ] + } +} +``` + +The two entries are scored separately and **added**, so Kenyan light-roast berry coffees rise above +coffees that satisfy only one of them. + +Supplied **without** `rankBy`, results are reranked toward that taste automatically. There are no +`terms` on this path, so `rankExplain` is `null`. + +## You supply the magnitude + +`magnitude` is the length of the shopper vector — one number that lets the index compare **direction** +(what they like) without being fooled by **size** (how much they like it). It is `sqrt(Σ w²)`: square +every weight, add them up, take the square root. For `{ "flavor:berry": 1.0, "roast:light": 1.0 }` that +is `sqrt(1 + 1) = 1.414`. + +**A miscomputed magnitude raises no error.** It quietly distorts every cosine, so the results look +plausible and are wrong. Compute it in exactly one place and unit test it against a known case. + +```ts +type SparseVector = Record; + +const magnitude = (w: SparseVector) => Math.sqrt(Object.values(w).reduce((sum, x) => sum + x * x, 0)); + +const toUserTaste = (vocabulary: string, weights: SparseVector) => ({ + vocabulary, + weights, + magnitude: magnitude(weights), +}); +``` + +**Prune near-zero weights before sending.** They cost payload and contribute nothing beyond rounding. + +## Where the shopper vector comes from + +The vector is UI state or session state, whichever you have. All of these are one query each: + +- **Sliders and chips.** A slider from light to dark spread over `roast:light`, `roast:medium`, + `roast:dark`; a chip that sets `flavor:chocolate` to 1. +- **Cold start from picks.** Let a new visitor pick two or three products, read what they are made of, + rebuild the vectors with the same positional rule, and sum them. No history needed. Item taste is + **not** exposed on Discovery hits — read `topicPaths` from the hit if your taste keys are derived from + topics, or read `item { taste { ... } }` from Core server-side. +- **Decayed session behaviour.** Add a little weight for every product a shopper opens or adds, decay it + over time, and send the result. The profile lives in the browser or your session store. +- **Agents.** An AI agent holds a structured taste vector for a customer and sends it with every search, + so the results it reads are already ranked for that customer. + +## Blending taste with your rules + +`context` alone ranks purely by taste. To put the shopper **on top of** the catalogue rules, add one +`tasteCosine` term to a `rankBy` and **pass both** — taste terms read the shopper vector from `context`. + +```graphql +query Blended($context: ContextInput!, $rankBy: RankByInput!) { + search(term: "espresso", context: $context, rankBy: $rankBy, pagination: { limit: 24 }) { + hits { + ... on product { + name + rankScore + rankExplain { + signal + index + contribution + } + } + } + } +} +``` + +```json +{ + "rankBy": { + "terms": [ + { "signal": "relevance", "weight": 1.0 }, + { "signal": "fieldBoost", "weight": 0.6, "field": "margin", "normalize": true }, + { "signal": "inStockBoost", "weight": 0.4, "field": "stock_default" }, + { "signal": "recency", "weight": 0.3, "field": "publishedAt", "halfLifeDays": 60 }, + { "signal": "tasteCosine", "weight": 1.2, "vocabulary": "taste", "from": "userTaste" } + ], + "tieBreaker": "sku", + "explain": true + } +} +``` + +```text +rankScore = 1.0·relevance + 0.6·margin + 0.4·inStock + 0.3·recency + 1.2·cos(taste) +``` + +The split is explicit: the first four terms are **your rules** and apply to everyone, the last is **this +shopper**. Raise the taste weight and the shelf becomes more personal; lower it and your rules take +over. + +**One `tasteCosine` term per vocabulary** lets you weight vocabularies independently — taste at 1.2, +origin at 0.3 — which plain `context` cannot do. This is the main reason to reach for the blended path. + +`tasteCosine` contributes **0** for items with no vector in that vocabulary, so unmapped products fall +back to your other rules rather than disappearing. + +## `nearestTo` — "more like this" + +Ranks by similarity to a **reference item's own stored vector** instead of a hand-built one. Use it for +"similar products" on a product page, or pairing suggestions from a basket. + +```text +NearestToInput { vocabulary: TenantVocabularyIdentifier!, like: NearestToLikeInput!, k: Int! } +NearestToLikeInput { sku: String, itemId: String } # pass exactly one +``` + +```graphql +{ + search(nearestTo: { vocabulary: taste, like: { sku: "ET-006" }, k: 12 }) { + hits { + ... on product { + itemId + name + rankScore + } + } + } +} +``` + +Four things to know: + +- **The anchor item is excluded** from the results. +- **An anchor with no vector for that vocabulary yields an empty result** — not a fallback to relevance. + Check the anchor before rendering an empty shelf. +- **`k` replaces `pagination.limit`** when `nearestTo` is set, and is itself capped by the rerank window. +- **`nearestTo` alone does not combine `context.userTaste`.** It scores from the anchor vector only. + +To blend neighbours with stock, margin or the shopper's own taste, add a `tasteCosine` term with +`from: nearestTo` to a `rankBy`: + +```json +{ + "rankBy": { + "terms": [ + { "signal": "tasteCosine", "weight": 1.0, "vocabulary": "taste", "from": "nearestTo" }, + { "signal": "inStockBoost", "weight": 0.5, "field": "stock_default" }, + { "signal": "fieldBoost", "weight": 0.3, "field": "margin", "normalize": true } + ], + "tieBreaker": "sku" + } +} +``` + +## Verifying personalization actually works + +Run the same query **with and without** `context` and compare the order. If the two match, no vectors +reached the index — the items were never published after `setItemTaste`, or the index was never re-run. +See steps 4–6 in [vocabulary-authoring.md](vocabulary-authoring.md). diff --git a/use-crystallize/skills/vector-ranking/references/ranking-signals.md b/use-crystallize/skills/vector-ranking/references/ranking-signals.md new file mode 100644 index 0000000..40cb3f7 --- /dev/null +++ b/use-crystallize/skills/vector-ranking/references/ranking-signals.md @@ -0,0 +1,209 @@ +# Ranking Signals (`rankBy`) + +Discovery API, `https://api.crystallize.com//discovery`, no auth. + +By default Search orders hits by full-text relevance — a BM25-based score. `rankBy` blends several +weighted signals into one score instead: + +```text +rankScore = Σ (weight × signal) +``` + +Positive weights boost, negative weights penalize — a high return rate can legitimately push a product +down. Different surfaces carry different rules: a category page may lean on margin, a search page on +relevance, a clearance page on stock. + +## Schema + +```text +RankByInput { terms: [RankByTermInput!]!, tieBreaker: TenantRankByTieBreaker!, explain: Boolean } + +RankByTermInput { signal: RankBySignal! # required + weight: Float! # required + field: TenantRankByField # fieldBoost | inStockBoost | recency + halfLifeDays: Float # recency + vocabulary: TenantVocabularyIdentifier # tasteCosine + from: TasteCosineFrom # tasteCosine + normalize: Boolean } + +RankBySignal = relevance | fieldBoost | inStockBoost | recency | tasteCosine +TasteCosineFrom = userTaste | nearestTo +``` + +`terms` must hold at least one term. `tieBreaker` is **required** — it settles ties deterministically so +the order stays stable across pages. + +## The five signals + +| Signal | Required | Optional | What it scores | +| -------------- | ----------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| `relevance` | — | — | Text match against `term`. **Contributes 0 when there is no `term`.** | +| `fieldBoost` | `field` | `normalize` | A rankable number on the item, used directly as a boost. | +| `inStockBoost` | `field` | — | Binary: contributes **1** when the named numeric field is `> 0`. Ignores quantity. | +| `recency` | `field`, `halfLifeDays` | — | Exponential time-decay on the age of a date field. | +| `tasteCosine` | `vocabulary`, `from` | — | Cosine to the shopper vector (`from: userTaste`) or to an anchor item (`from: nearestTo`). See [personalization.md](personalization.md). | + +## The field enums are per tenant + +`field` and `tieBreaker` are **not free-form strings** — they are enums generated from the tenant's +index settings. Introspect them; do not guess. + +| Enum | Built from | +| ---------------------------- | ------------------------------------------------------------------------- | +| `TenantRankByField` | NUMBER and DATE **filterable** attributes, **with facet fields excluded** | +| `TenantRankByTieBreaker` | sortable fields (token, number and date) | +| `TenantVocabularyIdentifier` | the tenant's vocabularies, **only after the next index run** | + +`TenantRankByField` being facet-excluded is the trap: a number you can facet on is not necessarily a +number you can boost on. + +```graphql +{ + f: __type(name: "TenantRankByField") { + enumValues { + name + } + } + tb: __type(name: "TenantRankByTieBreaker") { + enumValues { + name + } + } +} +``` + +Typical values on a commerce tenant: `price_default`, `price_sales`, `stock_default`, `stock_oslo`, +`publishedAt`, `createdAt`, `updatedAt`, `position`, `depth`, and any numeric component +(`dimensions_weight_number`, `sold_30d_number`, `margin_number`, `rating_number`, …). + +## Example + +```graphql +query Catalogue($rankBy: RankByInput!) { + search(term: "espresso", rankBy: $rankBy, pagination: { limit: 24 }) { + hits { + ... on product { + name + score + rankScore + rankExplain { + signal + index + contribution + } + } + } + } +} +``` + +```json +{ + "rankBy": { + "terms": [ + { "signal": "relevance", "weight": 1.0 }, + { "signal": "fieldBoost", "weight": 0.6, "field": "margin", "normalize": true }, + { "signal": "fieldBoost", "weight": 0.5, "field": "sold_30d", "normalize": true }, + { "signal": "inStockBoost", "weight": 0.4, "field": "stock_default" }, + { "signal": "recency", "weight": 0.3, "field": "publishedAt", "halfLifeDays": 60 } + ], + "tieBreaker": "sku", + "explain": true + } +} +``` + +```text +rankScore = 1.0·relevance + 0.6·margin + 0.5·sold_30d + 0.4·inStock + 0.3·recency +``` + +## Zero weights are dropped + +A term with `weight: 0` is removed **before** scoring. If _every_ term is 0, the query falls back to +plain relevance and **skips the rerank window entirely** — no rerank runs, and `rankScore` comes back +`null`. Turning a signal off by zeroing it is fine; zeroing all of them silently disables ranking. + +## Normalization + +`normalize` applies **min–max normalization across the rerank window**, so signals on wildly different +scales can share a weight. The default depends on the signal — set `normalize` explicitly to override. + +| Signal | `normalize` default | Why | +| -------------- | ------------------- | ------------------- | +| `relevance` | **on** | unbounded magnitude | +| `fieldBoost` | **on** | unbounded magnitude | +| `tasteCosine` | off | already roughly 0–1 | +| `recency` | off | already roughly 0–1 | +| `inStockBoost` | off | already roughly 0–1 | + +Because normalization is computed over the window, the same item can score differently under different +`rerankWindow` sizes. That is expected, not a bug. + +## Multi-valued fields are collapsed + +Anything with several values per item — fields under `variants` (such as `price_*` / `stock_*`), fields +under `shortcuts`, or a repeatable component — is collapsed to one value before scoring. **The direction +is fixed and cannot be overridden:** + +| Signal | Collapses to | Rationale | +| -------------- | ------------ | ----------------------------------------- | +| `fieldBoost` | **lowest** | the "from" price a shopper is shown | +| `inStockBoost` | **highest** | buyable in any variant counts as in stock | +| `recency` | **highest** | the most recent date wins | + +## `rankScore` and `rankExplain` + +`rankScore` on each hit is the **raw, un-normalized value the hits were ordered by** — the actual sort +key. It is distinct from `score`, which stays text relevance. `rankScore` is `null` whenever no rerank +ran: no `rankBy`, or every term dropped. + +With `explain: true` each hit carries one `rankExplain` entry per term, **in `rankBy.terms` order**: + +```text +RankExplainEntry { signal: RankBySignal, index: Int, contribution: Float } +``` + +`index` disambiguates a repeated signal — two `fieldBoost` terms, say — and the contributions **sum to +`rankScore` by construction**, which makes it a usable self-check. + +```json +{ + "name": "Cobán Dark", + "rankScore": 1.71, + "rankExplain": [ + { "signal": "relevance", "index": 0, "contribution": 0.42 }, + { "signal": "fieldBoost", "index": 1, "contribution": 0.51 }, + { "signal": "fieldBoost", "index": 2, "contribution": 0.3 }, + { "signal": "inStockBoost", "index": 3, "contribution": 0.4 }, + { "signal": "recency", "index": 4, "contribution": 0.08 } + ] +} +``` + +`rankExplain` is populated **only** with `explain: true`, and is `null` on the `userTaste`-only and +`nearestTo`-only paths, which have no `terms`. + +Ship `explain: true` behind a merchandiser flag, not in the hot storefront path — it is a tuning tool. + +## The rerank window + +`options.rerankWindow` is the **single knob shared by all three rerank paths** (`rankBy`, `context` and +`nearestTo`). The top-N candidates are scored and re-sorted; results beyond N keep plain relevance +order. + +- default **500** +- hard server cap **2000** +- rerank cost is per candidate, so a larger window costs more + +```graphql +search( + term: "espresso", + rankBy: $rankBy, + options: { rerankWindow: 1000 } +) { hits { ... on product { name rankScore } } } +``` + +**Paging happens inside the window.** Under ranking, `skip` offsets into the reranked window, and a +`skip` past the window returns nothing. Size the window for the depth you intend to page, not just for +the first page. Cursor tokens (`after`/`before`) fall back to offset pagination when a rerank is active, +so use `skip` + `limit` on ranked queries. diff --git a/use-crystallize/skills/vector-ranking/references/vocabulary-authoring.md b/use-crystallize/skills/vector-ranking/references/vocabulary-authoring.md new file mode 100644 index 0000000..45cf01f --- /dev/null +++ b/use-crystallize/skills/vector-ranking/references/vocabulary-authoring.md @@ -0,0 +1,313 @@ +# Vocabulary & Taste Authoring (Core API) + +Everything on this page runs against the **Core API** at `https://api.crystallize.com/@/core` +with an access token pair. Vocabulary and index mutations are **not** on the PIM API. + +## 1. Design the vocabulary + +Decide what a person could plausibly have a preference _about_. That is your dimension list. Two tests: + +- **Could a shopper move a slider for it?** Roast level, yes. SKU prefix, no. Dimensions nobody has + taste about add noise to every score. +- **Does assignment order mean anything?** If the first flavour listed is the signature note, give the + dimension a positional weight list. If not, one weight is enough. + +Keep genuinely different concerns in **separate vocabularies**. What a coffee tastes like and where it +comes from are questions a shopper may weigh differently, so they are two vocabularies rather than five +dimensions in one. Vocabularies are scored independently and summed, so splitting a concern out is what +lets a shopper — and you, via a per-vocabulary `tasteCosine` term — weight it on its own. + +```text +taste roast [1.0] one roast level, order meaningless + flavor [1.0, 0.6, 0.3] ordered: the signature note dominates + body [0.8] matters, but less than roast + +origin region [1.0] + process [1.0, 0.5] +``` + +**Values become the second half of every key, so keep them slug-shaped and stable:** +`flavor:blackcurrant`, not `flavor:Black Currant`. Renaming a value later means rewriting every item +that used it and re-indexing. + +### Business dimensions are welcome + +A vocabulary does not have to be about taste in the narrow sense. Margin band, stock depth, seasonality, +audience or tier all work as dimensions **when you want them inside a similarity score**. Use `rankBy` +instead when they should stay separate and be weighted on their own — that is usually the better +default for anything you already hold as a number on the product. + +### Weights are positional + +A dimension's `weight` is a **list**. One element means every entry in that dimension weighs the same. +More than one means entry _i_ takes `weight[min(i, length - 1)]`, clamping to the last element once you +run past the end. + +```text +weight: [1.0, 0.6, 0.3] 1st entry 1.0 + 2nd entry 0.6 + 3rd entry 0.3 + 4th entry 0.3 (clamped to last) +``` + +## 2. Create the vocabulary + +`upsertVocabulary` creates or replaces a vocabulary **in full**. It is a full replace, not a patch — +send the whole dimension list every time. **Any dimension you omit is gone.** + +```graphql +mutation UpsertVocabulary($input: UpsertVocabularyInput!) { + upsertVocabulary(input: $input) { + name + dimensions { + id + weight + } + lastUpdated + } +} +``` + +```json +{ + "input": { + "name": "taste", + "dimensions": [ + { "id": "roast", "weight": [1.0] }, + { "id": "flavor", "weight": [1.0, 0.6, 0.3] }, + { "id": "body", "weight": [0.8] } + ] + } +} +``` + +Schema: + +```text +UpsertVocabularyInput { name: String!, dimensions: [GraphqlInputVocabularyDimension!]! } +GraphqlInputVocabularyDimension { id: String!, weight: [Float!]! } +GraphqlVocabulary { name: String!, dimensions: [GraphqlVocabularyDimension!]!, lastUpdated: Date } +``` + +Read it back with the `vocabulary` query — useful as an idempotency check before a bulk re-author: + +```graphql +query { + vocabulary(name: "taste") { + name + lastUpdated + dimensions { + id + weight + } + } +} +``` + +## 3. Attach taste to items + +`setItemTaste` writes the ordered entries for **one item, one language, one vocabulary**. Two +vocabularies means two calls per item. + +```graphql +mutation SetItemTaste($input: SetItemTasteInput!) { + setItemTaste(input: $input) { + __typename + ... on Product { + id + } + ... on Folder { + id + } + ... on Document { + id + } + ... on BasicError { + errorName + message + } + } +} +``` + +Every error member of the union implements the `BasicError` interface, so one fragment covers all of +them — `errorName` tells you which one you got. Spelling out individual error types is only worth it +when you branch on a field the interface does not carry. + +```json +{ + "input": { + "itemId": "", + "language": "en", + "vocabulary": "taste", + "entries": [ + { "key": "roast:light" }, + { "key": "flavor:berry" }, + { "key": "flavor:blackcurrant" }, + { "key": "flavor:caramel" }, + { "key": "body:light", "weight": 1.2 } + ] + } +} +``` + +Schema: + +```text +SetItemTasteInput { itemId: String!, language: String!, vocabulary: String!, + entries: [GraphqlInputItemTasteEntry!]! } +GraphqlInputItemTasteEntry { key: String!, weight: Float } +SetItemTasteResult (union) Product | Folder | Document + | ItemNotFoundError | ExperimentalFeaturesNotAvailableError + | UnauthorizedError | UnknownError +``` + +`ExperimentalFeaturesNotAvailableError` is in the result union and is **not** in the public docs. It +means vectors are not enabled for the tenant at all — read `errorName` off the `BasicError` fragment to +tell it apart from the others. + +### Order is the input to positional weights + +Entries are read **in array order, per dimension**. Above, `flavor:berry` takes 1.0, +`flavor:blackcurrant` 0.6 and `flavor:caramel` 0.3. The optional per-entry `weight` overrides the +vocabulary weight for that entry only — and **still consumes its positional slot**, so it does not shift +the entries that follow. + +### The key format is validated + +Every key is `:`. Everything before the first colon must be a **declared dimension +in that vocabulary**. Either mistake rejects the whole call, so there are **no partial writes** — fix +the key and resend the full set of entries. + +| Bad entry | Rejection message | +| ---------------------- | --------------------------------------------------------------------------------------- | +| `{ key: "chocolate" }` | `Malformed taste entry key "chocolate": expected ":"` | +| `{ key: "mood:cozy" }` | `Unknown dimension "mood" in vocabulary "taste". Known dimensions: roast, flavor, body` | + +### Read taste back + +Core exposes taste on the item, which is the reliable way to verify a bulk author before indexing. +It is **not** exposed on Discovery hits. + +```graphql +query { + item(id: "", language: "en") { + taste { + vocabulary + entries { + key + weight + } + } + } +} +``` + +```text +GraphqlItemTaste { vocabulary: String!, entries: [GraphqlItemTasteEntry!]! } +GraphqlItemTasteEntry { key: String!, weight: Float } +``` + +### Keep one source of truth + +If you also assign topics for the same concepts, **derive** the taste entries from the topic assignments +— for example by storing the key in the topic's `meta` — rather than maintaining two parallel lists. +See [[taxonomy]] for the topic side. + +## 4. Publish + +`setItemTaste` writes to the **draft** version. The indexer reads the **published** version. If your +items were published before you attached taste, publish them again or the served documents carry no +vectors. + +```graphql +mutation Publish($ids: [ID!]!, $language: String!) { + publishItems(ids: $ids, language: $language) { + __typename + } +} +``` + +**Skipping this step produces no error.** Queries still return results, and the order may even change — +it just has no relation to taste. The check in step 6 is the only thing that catches it. + +## 5. Index + +`igniteDiscoApi` rebuilds the tenant's Discovery index and materializes the vectors. Run it after **any** +change to vocabularies or taste entries, not only the first time. + +**Pass `stacks: opensearch`.** The argument is optional in the schema but it selects which index stack +is built, and vector ranking is only served from the `opensearch` stack — an ignition without it +produces a working Discovery index with no vectors in it, and no error. + +```graphql +mutation Index { + igniteDiscoApi(stacks: opensearch) { + __typename + ... on BulkTaskIgnition { + id + type + status + createdAt + } + ... on BasicError { + errorName + message + } + } +} +``` + +```text +igniteDiscoApi(stacks: DiscoIgnitionStacks): IgnitionBulkTaskResult! +DiscoIgnitionStacks = atlasSearch | both | opensearch +IgnitionBulkTaskResult (union) BulkTaskIgnition | ExperimentalFeaturesNotAvailableError + | InvalidIdError | UnauthorizedError | UnknownError +``` + +The mutation is asynchronous. Poll the returned task until it reports `complete`, then allow a few +minutes for Discovery to propagate the new index. + +```graphql +query Task($id: ID!) { + bulkTask(id: $id) { + ... on BulkTask { + id + status + } # pending, started, complete, error + } +} +``` + +From then on the tenant's Discovery schema includes `context`, `rankBy` and `nearestTo`, and every +vocabulary you created becomes a value of the `TenantVocabularyIdentifier` enum. **A vocabulary is only +a valid enum value after the next index run** — until then, queries referencing it fail schema +validation. + +## 6. Verify + +Two checks, in this order. Both are cheap and both catch a silent failure. + +```graphql +# a) Did the index actually rebuild? The timestamp must have moved. +{ + search { + summary { + profiling { + lastIndexCompletedAt + } + } + } +} +``` + +```text +b) Run the same query with and without `context` and compare the order. + If the two match, no vectors reached the index — go back to step 4 (publish), then step 5 (index). +``` + +## Authoring at scale + +`setItemTaste` is one call per item per vocabulary, so a full catalogue is a bulk job. Drive it from +[[mass-operations]] or a script over [[js-api-client]], and **index once at the end** rather than per +item. Order matters: vocabulary → taste → publish → index.