Skip to content

Let a provider live in its own gem - #21

Open
codenamev wants to merge 1 commit into
obie:mainfrom
codenamev:laya-provider
Open

codenamev wants to merge 1 commit into
obie:mainfrom
codenamev:laya-provider

Conversation

@codenamev

Copy link
Copy Markdown

Both providers here are HTTP clients with a key, and Providers::REGISTRY is frozen, so a provider cannot ship separately, and one that runs a model in this process cannot exist at all. This adds the seam for both, shaped after what RubyLLM does with Provider.register and provider gems.

Nothing about OpenRouter or Typesafe changes, and the gem still has no runtime dependencies.

What it adds

  • Providers.register(name, klass), so a provider defined anywhere can join the registry, plus registered?. build now forwards extra keyword arguments, so a provider can take options of its own.
  • Base#requires_api_key? (default true) and Base#transport (default nil, meaning "use Client's HTTP transport"). A provider that answers in-process returns a callable of the same shape Client already takes, so retries, error mapping and the typed answers are reused untouched.
  • Base#env_var defaults to nil instead of raising, for a provider with no credential to read.
  • A "Writing a provider" section in the README covering the contract, the local-provider case, and the ruby_decision_model-providers-<name> gem convention.

The diff against existing files is 27 lines:

lib/ruby_decision_model/client.rb         |  8 ++++--
lib/ruby_decision_model/providers.rb      | 46 ++++++++++++++++++++++++++++---
lib/ruby_decision_model/providers/base.rb | 20 ++++++++++++++--

Tests

test/extension_test.rb is new, and its fixture is a provider that answers from memory: registration, a provider with no key, answering without HTTP, an injected transport: still winning, and the retry policy still applying to a local provider. That covers the contract without adding a dependency. Full suite passes, 109 runs.

Why I wanted it

I wrote ruby-laya, which runs ONNX exports of the open-weights Laya checkpoints locally. With these hooks it becomes an ordinary provider in a separate gem, ruby_decision_model-providers-laya:

client = RubyDecisionModel::Client.new(provider: :laya)
response = client.ask(state: ticket, questions: questions)

response["team"].choice      # => "billing"
response.model               # => "english", the checkpoint that answered
response.usage.cost          # => nil, nothing was billed

That gem is unpublished on purpose: it depends on this hook, so it waits on whatever shape you settle on here.

Happy to change the shape

If you would rather keep the registry closed and have providers vendored in, or split this into "registration" and "non-HTTP providers" as two PRs, or name the hooks differently, say the word. The local-provider half is the part I actually need; the registration half is what keeps it out of your tree.

Both providers here are HTTP clients with a key, and the registry is
closed, so a provider that runs a model in-process, or one that simply
ships separately, cannot exist. This is the seam for both, modelled on
what RubyLLM does with Provider.register and its provider gems.

Providers.register(name, klass) adds a provider from anywhere, and
build forwards extra keyword arguments so a provider can take its own
options. Base gains requires_api_key? and transport, each defaulting to
what the hosted providers already did, and env_var defaults to nil for a
provider with no credential. Client honours both.

Nothing about OpenRouter or Typesafe changes, and the gem still has no
runtime dependencies: the new test fixture is a provider that answers
from memory, which is the whole contract a gem outside this repository
has to meet.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant