LLM Provider Gateway

A store that adopts more than one AI-assisted feature – product-description generation, catalog translation, image generation, semantic search embeddings – otherwise repeats the same provider plumbing in every module: a separate API-key field here, a slightly different one there, no shared record of what was spent, and no single place to swap a model. LLM Provider Gateway for Adobe Commerce and Magento Open Source replaces that duplication with one admin-side gateway: every provider key is configured once, each consuming module is routed to a concrete model from the admin, and every call writes one accounting row so token usage and estimated spend are visible in a single grid.

The gateway is rule-based and deterministic. Routing, pricing, accounting and logging are plain configuration and code – there is no model, no inference and no decision-making inside the extension. The only AI involved runs at the external provider you configure a key for; nothing executes locally except the HTTP call you set up, and no data leaves the store except to the provider you selected.

A consuming module asks for a capability – chat, image or text embedding – and tags the request with its own module name. The gateway resolves that module and its quality tier to a concrete model, calls the right provider in the right request shape, normalizes the response, records the usage, and returns a provider-agnostic result. Switching a module from Anthropic to OpenAI, or from a senior model to a cheaper one, is a configuration change in the admin – the consuming module's code does not change. Seven providers ship out of the box – Anthropic, OpenAI, OpenRouter, Google Gemini, Ollama (local, keyless), Stability AI and Voyage AI – behind one interface per capability, and each has a Test Connection button that verifies a saved key against a read-only endpoint rather than a billable request.

Which models are available is managed in the admin, not fixed by the release. The Model Catalog is a grid where a model can be added, corrected or withdrawn, and Sync from Provider reads a provider's own list and shows what it disagrees with – models newly offered, models no longer listed, differing limits or prices – before anything is written. This matters most for a local Ollama server, where the model set is a property of the merchant's own machine, and for models released after the extension: a model added this way reaches consuming modules without waiting for an update.

Every call is metered. The Usage Dashboard is an admin grid over the accounting table, filterable by date, module, provider, capability and status, with token counts, latency, terminal status and an estimated USD cost per call drawn from a per-model price table you can override in the admin. An optional Call Log keeps redacted request and response bodies for troubleshooting, and a daily cron purges them past a retention window you set. API keys are stored encrypted and masked.

The gateway also protects the budget and the integration contract. One master switch stops every consuming module's requests before validation, routing or any provider call – the setting to reach for during a provider incident or an unexpected bill, while the dashboards and connection tests keep working. Each consuming module can carry a daily spend cap in USD (with a global default): once its recorded spend for the day reaches the cap, further calls are rejected before any provider request. A request that needs something the configured model cannot do – structured JSON output, tool calls, image input, streaming – fails immediately with a message naming the exact configuration field to fix, instead of silently degrading after spend. And a consuming module can declare both what it needs from a model and which models it was tested with: the admin dropdowns offer those models, saving an unsupported one is rejected, and routing enforces the declaration on every call. A per-module setting lets a merchant use a newer model the module was never tested with, while the module's stated requirements stay enforced.

The extension is admin-only and ships no remote code: provider adapters and the model catalog live inside the package, nothing is fetched or executed from a remote source, and there is no storefront output – so it is compatible with both Luma and Hyvä themes, and with Magento's native checkout and any third-party checkout layer.