Structured Logging For Cloud Collectors
/
A merchant runs their store in containers and pays for a log platform, but when
an order fails at midnight the only trace is a line of free text inside
var/log/system.log on a container that has since been replaced. Nothing was
searchable, and nothing survived. This extension for Magento sends that same
record to the container's standard output as one structured line, in whichever
format the collector already understands, so the failure is a query rather than
an archaeology exercise.
It works by replacing the destination of the logs Adobe Commerce and Magento
Open Source already write, not by adding new ones. Disabled - which is how it
ships - it is invisible: system.log, the exception.log split, the debug
gate and the payment, shipping and cron channel files behave exactly as they do
without it. Enabled, each record goes instead to standard output, to standard
output and standard error split by severity, or to the same files as before,
in one of eleven formats. No message is rewritten and no logging call in any
extension has to change.
The formats are the ones log platforms parse without a translation layer:
compact JSON, Elastic Common Schema for Elasticsearch and OpenSearch, Logstash,
RFC 5424 syslog, Google Cloud Logging, Datadog with its reserved attributes,
GELF for Graylog, logfmt for Grafana Loki, Monolog's own JSON, Magento's line
format, and a template a merchant writes themselves. Every JSON line carries a
request id - taken from the X-Request-Id header a load balancer sets, or
generated once per process - so one customer's failing request can be followed
across every line it produced, and exceptions arrive as typed error objects
rather than as a string.
Proving coverage is part of the product. A console command lists every logger in the installation, shows where each one writes, and flags any log that would still write its own file or any record that would be written twice - so a merchant can see, before trusting it, that nothing is quietly missing.
WebRamos Structured Logging is a rule-based, deterministic, on-premise extension for Magento Open Source and Adobe Commerce. It uses no external services, sends no data off-site, and stores nothing of its own - it adds no database table and keeps no state. Admin only - no storefront output, theme-independent.
Key Features
- Eleven log formats: compact JSON, Elastic Common Schema, Logstash, syslog RFC 5424, Google Cloud Logging, Datadog, GELF, logfmt, Monolog JSON, the stock line format and a custom template
- Three destinations: standard output and standard error split by severity, standard output only, or the usual var/log files
- Disabled by default and invisible when disabled - every record lands where it did before
- Covers the main log and the payment, shipping, cron, Admin Adobe IMS, Service Proxy and Payment Services channels, each record written once – one line in every JSON format, size-bounded on stdout and stderr
- A request id on every line, from the X-Request-Id header or generated once per process
- Exceptions written as typed error objects with a bounded stack trace; an exception logged on its own keeps its first line as the message and lists the exceptions it wraps
- A maximum line size, 8000 bytes by default – what the official php-fpm image passes whole – that trims an oversized JSON record to fit and marks it instead of letting it arrive in pieces
- Service, environment and version fields, settable per environment in the deployment file
- Service fields that read an environment variable (`%env.NAME%`), so the locked configuration stays the same across releases
- Scanner-induced errors demoted to warnings so error-level alerting stays quiet
- A console command that reports every logger, its destination, and anything still writing its own file
- A minimum severity threshold, so debug detail can stay out of production
- No database tables, no external services, no data leaving the store
- Any extension can add its own format with one class and one line of configuration
Versions
Feature release: an error record that carries an exception reaches the log platform as one valid JSON line – the exception's first line as its message, and a line limit that trims a long record instead of letting it arrive in pieces.
Show details Hide details
Added
- "Maximum Line Size (bytes)" in the General group (
webramos_structured_logging/general/max_line_bytes). On standard output and standard error every line of the seven JSON formats – Compact JSON, Monolog JSON, Elastic Common Schema, Logstash, Google Cloud Logging, Datadog and GELF – stays within that many bytes, newline included. Default 8000; 0 turns the limit off; 1 to 1023 count as 1024. A longer record is trimmed – large context values and the bottom of the stack trace first – and marked as truncated, and stays one valid JSON line. The var/log files and the Magento line, custom template, syslog and logfmt formats are never cut. - When a logged exception wraps others, Compact JSON, Elastic Common Schema, Google Cloud Logging, Datadog and GELF list up to five of them – type, message and file – nearest first, always keeping the root cause.
Changed
- An exception logged on its own, Magento's usual way of reporting an error, carried its whole text as the message, every wrapped exception included. Those five formats now write the exception's first line as the message and its stack trace once, in the stack field. A message the calling code wrote is never changed; the other six formats keep the whole text.
- Measured on a 30 KB error record with two chained exceptions: the Compact JSON line falls from 43 KB to 11 KB with the new message and under 8 KB with the limit, Monolog JSON (35 KB) and Logstash (43 KB) under 8 KB too; behind the official php-fpm image it arrives as one line, not 2 to 6 pieces.
Upgrade notes
- Read an exception's stack trace from the format's stack field, not from the message; for Google Cloud Logging, from the lines after the message's first line.
- The line limit is on after the upgrade, at 8000 bytes, which fits the
log_limit = 8192of the official php-fpm Docker image. Set it to 0 to keep the unbounded lines of 1.1.0, or raise php-fpm'slog_limitfirst and the line limit after it, staying under the container runtime's 16 KB split. - In those five formats an exception logged on its own no longer carries the stack traces of the exceptions it wraps. The root cause keeps its type, message and file; for its full trace, log the root cause itself or use Monolog JSON or Logstash.
Feature release: the service fields can name an environment variable, so a deployment's locked configuration stays the same from one release to the next. Ships License Agreement 1.1.0.
Show details Hide details
Added
- The Service, Environment and Version fields accept
%env.NAME%, the placeholder the custom line template already uses. The extension replaces it with theNAMEenvironment variable when it writes a record, so one stored value serves every release. A value may mix text and placeholders, for examplebuild-%env.BUILD_ID%; a value without one is used exactly as before. - The coverage report (
bin/magento webramos:structured-logging:inspect) lists the three service fields: where each value comes from (locked in a configuration file, saved in the database, or the default), the stored value and, for a placeholder, the resolved value and any variable that is not set. A value locked as plain text is marked. The section never changes the command's exit code.
Changed
- The locked-configuration example in the documentation uses
%env.NAME%instead of a per-release value, next to a warning: Magento hashes the wholesystemsection ofapp/etc/env.phpandapp/etc/config.php, so a value there that changes from one release to the next makes every storefront and admin page answer "The configuration file has changed" untilbin/magento app:config:importruns. - The License Agreement is now version 1.1.0, effective 2026-10-08. Section 7.1 also lets the agencies or developers a customer engages access the extension on the customer's behalf, on the same condition as its employees and contractors: bound by confidentiality and use restrictions at least as protective as the Agreement's. No other term changed.
Upgrade notes
- A service field that already contained
%env.NAME%was written literally until now; from 1.1.0 it is resolved, to an empty value when the variable is not set. - To take a per-release value out of the locked configuration, replace it in
app/etc/env.phpwith its placeholder, for example'version' => '%env.APP_VERSION%', and runbin/magento app:config:importonce on that release.
Fix release for the Custom Format settings group.
Show details Hide details
Fixed
- The Custom Format group in Stores > Configuration no longer appears empty when Format is set to anything other than Custom line template. It now shows a note explaining when its settings apply.
First public release.
Show details Hide details
Added
- Structured, collector-ready logs. Every log record your store writes can
leave as one structured line on stdout or stderr instead of free text in
var/log/*.log, so the log collector you already run can index and search it. - Eleven formats. Compact JSON, Magento's own line format, Monolog JSON, Elastic Common Schema 8.x, Logstash, RFC 5424 syslog, Google Cloud Logging, Datadog, GELF 1.1 for Graylog, logfmt for Grafana Loki, and a custom line template you write yourself.
- Three destinations. stdout and stderr split by severity, stdout only, or
the usual
var/logfiles – keeping the format change without changing where the records land. - Off by default, and invisible when off. With the extension disabled every
record goes exactly where it went before:
system.log, theexception.logsplit, the debug gate, and the payment, shipping and cron channel files. - Full channel coverage. The main log plus the payment, shipping, cron, Admin Adobe IMS, Service Proxy and Payment Services channels. A record that several handlers would have written is emitted once, not twice.
- Request correlation. Every line carries a request id, taken from the
X-Request-Idheader your load balancer sets or generated once per process, so one customer's failing request can be followed across every line it wrote. - Service, environment and version fields on every JSON line, set in the configuration or locked per environment in the deployment file.
- Exceptions stay structured. An exception is written as a typed error object with class, code, message and a bounded stack trace, not flattened into the message string.
- Scanner noise stays out of your alerts. Vulnerability scanners probing for files that do not exist make Magento log errors; those records are written as warnings instead, so an alert bound to the error level does not fire on them.
- A coverage report you can run.
bin/magento webramos:structured-logging:inspectlists every logger in the installation, shows which destination each one writes to, and flags any log that would still write its own file or any record that would be written twice. - Extensible by other extensions. A format is one class and one line of configuration; a third-party module can add, replace or remove one without touching this extension.
Upgrade notes
- Requires Monolog 3, which Adobe Commerce and Magento Open Source ship from 2.4.8 onwards. On 2.4.7 and earlier, which ship Monolog 2, the package declines to install rather than failing later.
FAQ
Adobe Commerce and Magento Open Source write their logs as free text into a few
files under var/log. On one server that is fine. In a container it fails
twice over: the files vanish when the container is replaced, and the lines
carry no fields, so finding one customer's failed checkout means reading rather
than searching. This extension changes where those records go and what shape
they arrive in - one structured line per record, on standard output, in a
format the log platform you already pay for parses without a translation layer.
No logging call anywhere in the store has to change.
When the store runs in containers - ECS, Kubernetes, Cloud Run, plain Docker - and its logs currently die with them. When there is already a log platform beside the store (Elastic or OpenSearch, Grafana Loki, Graylog, Datadog, Google Cloud Logging, Splunk, CloudWatch) and Magento is the one source still feeding it prose. When an incident means correlating one request across several log lines and there is no request id to correlate on. And when staging and production have to be told apart in the same index, which needs a service, an environment and a version on every line, locked per environment rather than typed into a form.
When there is no log collector: on a single server where someone reads
var/log over SSH, structured records buy nothing, and a log viewer
extension - a grid in the admin panel over the same files - is the tool that
actually helps. When the problem is that var/log fills the disk: that is
rotation and archiving, not routing. When what is wanted is an audit trail of
which administrator changed which field: that is a different product entirely,
and this extension logs nothing of its own. And on Adobe Commerce or Magento
Open Source 2.4.7 or older: the extension is written against Monolog 3,
which the platform ships from 2.4.8 onwards, so on 2.4.7 the package declines
to install rather than failing later.
It replaces the destination of the logs the platform already writes. Disabled -
the shipped default - it is invisible, and every record lands exactly where it
did before, including the exception.log split and the per-channel files.
Enabled, each record goes to one of three destinations: standard output and
standard error split by severity, standard output only, or the same var/log
files as before - in one of eleven formats: compact JSON, Elastic Common
Schema 8.x, Logstash, RFC 5424 syslog, Google Cloud Logging, Datadog, GELF 1.1,
logfmt, Monolog JSON, Magento's line format, and a custom template.
Seven loggers are covered: the main application log plus the payment,
shipping, cron, Admin Adobe IMS, Service Proxy and Payment Services channels.
A record several handlers would have written is emitted once, not twice. Every
JSON line carries a request id from the X-Request-Id header or generated once
per process; exceptions arrive as typed error objects with a bounded stack
trace; service, environment and version are configuration values that can be
locked in app/etc/env.php, where they beat the database and are read before
it is even reachable, and each can name an environment variable
(%env.APP_VERSION%) so the locked file stays the same from release to
release. Scanner probes for files that do not exist are recorded
as warnings rather than errors, so error-level alerting stays quiet.
An error record arrives whole. PHP-FPM and the container runtime cut a long line into pieces that are not JSON, and an exception with a deep stack trace is exactly that line. An exception Magento logs on its own therefore keeps only its first line as the message, with the trace written once and the exceptions it wraps as a short list, and a JSON line on standard output longer than the Maximum Line Size setting – 8000 bytes by default, which the official PHP-FPM Docker image passes whole – is trimmed to fit and marked as truncated. A line marked truncated is still one record in one piece, and every field the trim did not reach is intact.
One console command, webramos:structured-logging:inspect, lists every logger
in the installation with its destination and flags any log still writing its
own file or any record written twice. It can measure var/log around a marker
record to prove that rather than infer it, and it refuses to run that probe
while the extension is disabled.
It does not show you the logs. There is no grid, no reader, no admin page
beyond its configuration section - if reading var/log from the admin panel is
the requirement, a log viewer extension is the right purchase and this is not.
It does not ship anything anywhere. There is no batching, no push exporter and no OTLP: it writes lines, and the collector already running beside the store does the shipping. A backend that only accepts pushed payloads needs an agent in front of it.
It does not rotate, archive or clean up var/log, and it does not cap file
sizes. It does not record who changed what in the admin panel. It does not
redact: it carries other extensions' records unchanged, so a module that logs
something sensitive still logs it. It adds no database table and stores nothing
of its own. And writers that bypass the platform's logger entirely -
error_log(), direct file writes, shell redirects - are outside its reach; the
inspect command's probe finds the ones that write files, and names them.