LLM Provider Gateway
/
A store that adopts more than one AI-assisted feature – product-description generation, catalog translation, image generation, semantic search embeddings – otherwise repeats the same provider plumbing in every module: a separate API-key field here, a slightly different one there, no shared record of what was spent, and no single place to swap a model. LLM Provider Gateway for Adobe Commerce and Magento Open Source replaces that duplication with one admin-side gateway: every provider key is configured once, each consuming module is routed to a concrete model from the admin, and every call writes one accounting row so token usage and estimated spend are visible in a single grid.
The gateway is rule-based and deterministic. Routing, pricing, accounting and logging are plain configuration and code – there is no model, no inference and no decision-making inside the extension. The only AI involved runs at the external provider you configure a key for; nothing executes locally except the HTTP call you set up, and no data leaves the store except to the provider you selected.
A consuming module asks for a capability – chat, image or text embedding – and tags the request with its own module name. The gateway resolves that module and its quality tier to a concrete model, calls the right provider in the right request shape, normalizes the response, records the usage, and returns a provider-agnostic result. Switching a module from Anthropic to OpenAI, or from a senior model to a cheaper one, is a configuration change in the admin – the consuming module's code does not change. Seven providers ship out of the box – Anthropic, OpenAI, OpenRouter, Google Gemini, Ollama (local, keyless), Stability AI and Voyage AI – behind one interface per capability, and each has a Test Connection button that verifies a saved key against a read-only endpoint rather than a billable request.
Which models are available is managed in the admin, not fixed by the release. The Model Catalog is a grid where a model can be added, corrected or withdrawn, and Sync from Provider reads a provider's own list and shows what it disagrees with – models newly offered, models no longer listed, differing limits or prices – before anything is written. This matters most for a local Ollama server, where the model set is a property of the merchant's own machine, and for models released after the extension: a model added this way reaches consuming modules without waiting for an update.
Every call is metered. The Usage Dashboard is an admin grid over the accounting table, filterable by date, module, provider, capability and status, with token counts, latency, terminal status and an estimated USD cost per call drawn from a per-model price table you can override in the admin. An optional Call Log keeps redacted request and response bodies for troubleshooting, and a daily cron purges them past a retention window you set. API keys are stored encrypted and masked.
The gateway also protects the budget and the integration contract. One master switch stops every consuming module's requests before validation, routing or any provider call – the setting to reach for during a provider incident or an unexpected bill, while the dashboards and connection tests keep working. Each consuming module can carry a daily spend cap in USD (with a global default): once its recorded spend for the day reaches the cap, further calls are rejected before any provider request. A request that needs something the configured model cannot do – structured JSON output, tool calls, image input, streaming – fails immediately with a message naming the exact configuration field to fix, instead of silently degrading after spend. And a consuming module can declare both what it needs from a model and which models it was tested with: the admin dropdowns offer those models, saving an unsupported one is rejected, and routing enforces the declaration on every call. A per-module setting lets a merchant use a newer model the module was never tested with, while the module's stated requirements stay enforced.
The extension is admin-only and ships no remote code: provider adapters and the model catalog live inside the package, nothing is fetched or executed from a remote source, and there is no storefront output – so it is compatible with both Luma and Hyvä themes, and with Magento's native checkout and any third-party checkout layer.
Key Features
- One encrypted, masked key store shared by every consuming module – Anthropic, OpenAI, OpenRouter, Google Gemini, Ollama, Stability AI and Voyage AI configured once
- Test Connection per provider – verifies a saved key against a read-only endpoint, never a billable inference request, and remembers the verdict
- Capability clients for chat (with streaming, vision and tool calls), image generation and text embeddings, identical across providers
- Per-module model routing chosen in the admin – switch provider or model with no code change in the consumer – with an optional fallback chain tried when a call fails with a retryable error
- Model Catalog admin grid – add, correct or withdraw a model without waiting for an extension release, with filters, search and mass actions
- Sync from Provider – reads a provider's own model list and shows what it disagrees with (newly offered, no longer listed, differing limits or prices) before anything is written; essential for a local Ollama server, whose model set differs on every machine
- Master switch – one setting stops every consuming module's requests at the gateway, while dashboards and connection tests keep working
- Daily spend caps per consuming module (plus a global default) – calls stop before any provider request once the day's budget is reached
- Fail-fast feature gating – a request needing structured output, tool calls, image input or streaming on a model that lacks it is rejected before any spend, with the configuration field to fix named in the message
- Structured JSON output across providers – schema-conforming answers behind one interface, including on Anthropic
- Compatibility declarations – a consuming module states what it needs and what it was tested with; admin dropdowns, save validation and routing all enforce it, and a per-module setting admits newer models while the requirements stay enforced
- Per-call accounting: token counts, latency, terminal status and an estimated USD cost for every request
- Pricing stated per billing unit – prompt caching is charged at its own read and write rates rather than as fresh input, and models billed per image, per minute or per request are costed correctly
- Usage Dashboard admin grid, filterable by date, module, provider, capability and status, with a per-period cost view
- Optional Call Log of redacted request/response bodies for troubleshooting, with a daily retention-purge cron
- Hardened by default – input size guards, SSRF-restricted image fetch, normalized error codes, no remote code, admin-only with no storefront footprint (Luma and Hyvä both supported)
Versions
Model catalog sync per provider, daily spend cap, JSON-schema enforcement, adapter URL dedup. Verified on Magento 2.4.7 - 2.4.9, PHP 8.1 - 8.5.
FAQ
Adobe Commerce and Magento Open Source have no shared notion of "the store's model provider", the way they have one for mail transport or search. So every extension that adds an AI feature brings its own API key field, its own vendor coupling and its own blind spot on cost.
LLM Provider is the missing layer. It holds the keys, the model registry, the price table and the call log in one place, and exposes PHP contracts that any module can call instead of talking to a vendor itself. It costs nothing, because its value is in what gets built on top of it – the source ships readable in the package, as every Magento module's does, under the same commercial licence as the rest of the range. Free of charge, not open source.
- You are installing an AI extension that requires it. The AI series – AI Field Validator, AI FAQ and the rest – depends on this gateway. Configure providers once and every one of them uses the same keys, models and budgets.
- You want to choose the vendor, and change your mind later. A consuming module asks for a tier, senior or junior; which model that resolves to is admin configuration. Moving a feature between Anthropic, OpenAI and a local model changes no code.
- Prompts must not leave your infrastructure. Ollama is a first-class provider, local and keyless, and the same consumer code runs against it.
- You need to see the spend. Every call is recorded with its module, model, token counts and computed cost, and each consuming module has its own daily spend cap.
- You are building your own module. The contract is three files and one
call – see the article, or
DEV.mdinside the module.
- You want a feature, not a foundation. Installed alone the gateway adds an admin section and a set of contracts. Nothing on your storefront changes; it is the layer the features stand on.
- You do not have provider accounts and do not want any. You bring your own keys. Nothing is resold here, and there is no bundled quota. The exception is Ollama, which needs no key but does need a server you run.
- You need Azure OpenAI or AWS Bedrock. Neither is included; their authentication is heavier and they were deferred.
- You need an asynchronous or batch API, or GraphQL access. The seams exist, the implementations do not.
- You are on Magento 2.4.6 or earlier. Supported versions are 2.4.7 – 2.4.9 on PHP 8.1 – 8.5.
- One encrypted key store. Anthropic, OpenAI, OpenRouter, Google Gemini, Ollama, Stability AI and Voyage AI, configured once and shared by every consuming module.
- Three capabilities behind one interface each – chat, with streaming, vision and tool calls; image generation; and text embeddings. The same consumer code runs against any provider that offers the capability.
- Structured JSON output across providers. A JSON Schema is enforced natively on OpenAI, and on Anthropic through a synthetic tool the adapter injects, so the consumer gets a decoded array either way and never learns which vendor answered.
- A compatibility contract per consuming module.
etc/llm_module.xmldeclares required features, a minimum context size and the models a module was tested with. Requirements are enforced before any request leaves the server; the tested list orders the admin dropdown and can be opened up per module. - Model registry with Sync from Provider. Reads a provider's own model list and shows what it disagrees with – newly offered models, models no longer listed, differing limits or prices – before anything is written.
- Per-model pricing and a call log. Cost is computed from a price table you can override, recorded per module and per call, and capped daily per module.
- Three new tables and no modified core tables. Admin-panel only, so it is theme-agnostic: Luma and Hyvä alike, with no storefront output of its own.
- It does not sell you model access. No quota, no account with us, no metering. Your keys, your provider contracts, your bills.
- It does not train, fine-tune or store a vector index. Embeddings are returned to the caller; what to do with them belongs to the consuming module.
- It does not guarantee a schema.
ResponseFormat::jsonSchema()is enforced by the provider, andgetStructuredContent()is a decode of what came back. A consumer that needs a hard guarantee validates the decoded array itself. - It does not read your provider's bill. Cost is computed locally from the price table, so the figures are an estimate of what a call cost, not an invoice. Keep the table current if you rely on the numbers.
- The daily cap is a soft ceiling, not a hard stop. A call's cost is known only once it has been answered, so the call that crosses the cap completes; the next one is refused. Size the cap with that one call in mind.