One AI gateway for every module on the store

Sooner or later a store wants something a language model is good at: drafting product copy, answering a shopper's question from your own content, checking a merchandiser's input before it goes live. The extensions that do those things each arrive with the same baggage. Each has its own API key field, buried in its own configuration section. Each is welded to one vendor, so the model you liked last year is the model you are stuck with. And none of them can tell you what any of it cost, because none of them can see what the others spent.

Adobe Commerce and Magento Open Source have no shared place for that. There is no notion of "the store's model provider", the way there is a notion of the store's mail transport or its search engine.

What it is for

LLM Provider is that shared place. It is a free, vendor-neutral gateway: one encrypted key store, one model registry, one price table, one call log – and a set of PHP contracts that any module can call instead of talking to a vendor itself.

Eight providers ship with it: Anthropic, OpenAI, OpenRouter, Google Gemini, Ollama, Stability AI, Voyage AI and TypeSafe. They are configured once, by you, with your own keys, and shared by every module that consumes the gateway. Ollama runs locally and needs no key at all, which is the answer when the content in the prompt is not allowed to leave the building.

Four capabilities are exposed behind one interface each – chat (with streaming, vision and tool calls), image generation, text embeddings, and decisions: yes/no, choice and score questions about one piece of content, answered with probabilities – so switching a feature from one vendor to another is a dropdown, not a redeployment. Every call is attributed to the module that made it, priced against a per-model table you can edit, and counted against that module's own daily spend cap.

How it can be used

Run the AI features you install. The gateway is the dependency the AI series shares. Install it once, configure the providers you actually use, and every consuming module picks up the same keys, the same models and the same budget discipline.

Change your mind about a vendor. A consuming module names a tier – senior or junior – rather than a vendor. Which model each tier resolves to is admin configuration. Moving a feature from Anthropic to OpenAI, or onto a local Ollama model, changes no consumer code at all.

Ask for a verdict, not prose. Some checks want a number: does a product description contain a phone number, which tone does a text take, how clear is it on a scale. A decision asks yes/no, choice and score questions like these about one piece of content and returns each answer with its probabilities, so the consuming module compares a number with a threshold instead of parsing text. The provider's model decides – TypeSafe Jev 1.13 ships, reached through your OpenRouter key or TypeSafe's own, and OpenAI GPT-6 Luna ships through OpenRouter. Answers are not always repeatable to the digit: with Jev the same question read 0.06 on one call and 0.08 on the next, so a threshold wants a margin.

Keep the data on your own hardware. Ollama is a first-class provider, not an afterthought. For stores where prompts contain customer data, the same consumer code runs against a model on your own server.

See what it costs before the invoice does. Usage is recorded per module, per model and per call, with cost computed from the price table. The daily cap is per consuming module, so one runaway feature cannot spend another's budget.

Building a consumer

This is the part the gateway exists for, and it is three files and one call.

1. Depend on it. In composer.json, and in etc/module.xml so that configuration and dependency injection load in the right order:

"require": { "webmaster-ramos/module-llm-provider": "^1.0" }

<sequence><module name="WebRamos_LlmProvider"/></sequence>

^1.0 serves chat, images and embeddings. A module that asks for decisions requires 1.1.0 or later – ^1.1 – the release that added them.

2. Declare what you were tested with. etc/llm_module.xml is the compatibility contract. required_features and min_context_tokens are enforced before any HTTP request happens; the <models> list means "tested with these" and orders the admin dropdown:

<module name="Acme_AiAssistant" capability="chat">
    <required_features>
        <feature>json_schema</feature>
    </required_features>
    <min_context_tokens>100000</min_context_tokens>
    <models>
        <model id="anthropic:claude-haiku-4-5" tier="junior" recommended="true"/>
        <model id="anthropic:claude-opus-4-6" tier="senior"/>
        <model id="openai:gpt-5-mini" tier="junior"/>
    </models>
</module>

The declaration is opt-in: a module without one is unrestricted. A merchant who wants a model released after your extension can switch on Allow Untested Models for that module, and the gateway admits any registry model that still meets your stated requirements, labelled (untested).

3. Ask for a model, not a vendor. The consumer ships a config group under webramos_llm/modules/<Vendor_Module>/ and uses the gateway's own backend model, so an admin cannot pick something your declaration rules out:

<field id="model_senior" translate="label" type="select" sortOrder="10" showInDefault="1">
    <label>Senior Model</label>
    <source_model>Acme\AiAssistant\Model\Config\Source\SeniorModels</source_model>
    <backend_model>WebRamos\LlmProvider\Model\Config\Backend\WhitelistedModel</backend_model>
</field>

model_senior answers for every capability the module calls at the senior tier. A module that also asks for decisions adds a decision field, decision_model_senior, with the same backend model over a source of capability decision. The gateway reads it before model_senior for decision calls, so the merchant picks the decision model without moving the chat one – and a value there is always used, never skipped for the general field.

4. Make the call. One interface, LlmChatClientInterface, and a request you build with setters. setCallerModule() is what attributes the cost and the log entry:

$request = $this->requestFactory->create()
    ->setCallerModule(Config::MODULE_NAME)
    ->setMessages([Message::user($prompt)])
    ->setResponseFormat(ResponseFormat::jsonSchema(self::PAIRS_SCHEMA))
    ->setMaxTokens(self::MAX_TOKENS);

try {
    $response = $this->chatClient->complete($request, $correlationId);
} catch (LlmException $e) {
    if (!$e->getErrorCode()->isRetryable()) {
        throw $e;
    }
    $response = $this->chatClient->complete($request, $correlationId);
}

$data = $response->getStructuredContent();

ResponseFormat::jsonSchema() is the part worth knowing about. On OpenAI it maps to native structured outputs; on Anthropic, which has no such mechanism, the adapter injects a synthetic tool whose input schema is your schema and forces it. getStructuredContent() returns a decoded array either way, so the consumer never learns which vendor answered. Schema validation is the provider's, by documented contract – if you need a hard guarantee, validate the decoded array yourself.

Decisions have their own client, LlmDecisionClientInterface::decide(), with questions built by Question::yesNo(), ::choice() and ::score(). The README inside the package carries the usage section – chat, streaming, structured output, and a complete decision example with the rules to know before relying on the numbers.

How it compares

Extensions that add an AI feature to Magento usually bundle the integration into the feature. That is the fastest way to ship one feature and the most expensive way to ship four: four key fields, four vendor couplings, four blind spots on cost, and no way to move any of them without touching code.

The other shape sells the model with the extension, metered by the vendor. That removes the setup and replaces it with someone else's price per call, someone else's retention policy for your prompts, and no local record of what was sent.

This one is neither. It is infrastructure: your keys, your provider choice, your price table, your log, on your own servers – and it is free, because the value is in what gets built on top of it. What it does not give you is a feature. On its own it renders nothing to a shopper; it is the layer the features stand on.

It is not the only layer of this shape. Plumrocket AI Connector is also a stand-alone AI layer for Magento, with a REST API and Hugging Face models. The side-by-side, including where Plumrocket is the better choice, is in LLM Provider Gateway vs Plumrocket AI Connector.

What it does not try to be

It is not an AI feature. Installed alone it adds an admin section and a set of contracts, and nothing on your storefront changes.

It does not resell model access. There is no bundled quota and no account with us – you bring keys from the providers you already have, or you run Ollama and bring none.

It does not decide anything about your catalogue. Prompts, schemas and what to do with an answer belong to the consuming module, and when a module asks for a decision, the provider's model decides – the gateway routes the questions, checks the answers against them and records the call.

And it does not hide the bill. Every call is logged with its module, model, token counts and computed cost, in your own database, where you can query it.