Structured Logging For Cloud Collectors
/
A merchant runs their store in containers and pays for a log platform, but when
an order fails at midnight the only trace is a line of free text inside
var/log/system.log on a container that has since been replaced. Nothing was
searchable, and nothing survived. This extension for Magento sends that same
record to the container's standard output as one structured line, in whichever
format the collector already understands, so the failure is a query rather than
an archaeology exercise.
It works by replacing the destination of the logs Adobe Commerce and Magento
Open Source already write, not by adding new ones. Disabled - which is how it
ships - it is invisible: system.log, the exception.log split, the debug
gate and the payment, shipping and cron channel files behave exactly as they do
without it. Enabled, each record goes instead to standard output, to standard
output and standard error split by severity, or to the same files as before,
in one of eleven formats. No message is rewritten and no logging call in any
extension has to change.
The formats are the ones log platforms parse without a translation layer:
compact JSON, Elastic Common Schema for Elasticsearch and OpenSearch, Logstash,
RFC 5424 syslog, Google Cloud Logging, Datadog with its reserved attributes,
GELF for Graylog, logfmt for Grafana Loki, Monolog's own JSON, Magento's line
format, and a template a merchant writes themselves. Every JSON line carries a
request id - taken from the X-Request-Id header a load balancer sets, or
generated once per process - so one customer's failing request can be followed
across every line it produced, and exceptions arrive as typed error objects
rather than as a string.
Proving coverage is part of the product. A console command lists every logger in the installation, shows where each one writes, and flags any log that would still write its own file or any record that would be written twice - so a merchant can see, before trusting it, that nothing is quietly missing.
WebRamos Structured Logging is a rule-based, deterministic, on-premise extension for Magento Open Source and Adobe Commerce. It uses no external services, sends no data off-site, and stores nothing of its own - it adds no database table and keeps no state. Admin only - no storefront output, theme-independent.
Key Features
- Eleven log formats: compact JSON, Elastic Common Schema, Logstash, syslog RFC 5424, Google Cloud Logging, Datadog, GELF, logfmt, Monolog JSON, the stock line format and a custom template
- Three destinations: standard output and standard error split by severity, standard output only, or the usual var/log files
- Disabled by default and invisible when disabled - every record lands where it did before
- Covers the main log and the payment, shipping, cron, Admin Adobe IMS, Service Proxy and Payment Services channels, with one line per record
- A request id on every line, from the X-Request-Id header or generated once per process
- Exceptions written as typed error objects with a bounded stack trace
- Service, environment and version fields, settable per environment in the deployment file
- Scanner-induced errors demoted to warnings so error-level alerting stays quiet
- A console command that reports every logger, its destination, and anything still writing its own file
- A minimum severity threshold, so debug detail can stay out of production
- No database tables, no external services, no data leaving the store
- Any extension can add its own format with one class and one line of configuration
Versions
See CHANGELOG.public.md. Summary: first public release - structured,
collector-ready logging for Adobe Commerce and Magento Open Source, with
eleven formats, three destinations, full channel coverage, request
correlation, and a console command that reports which logs are covered and
which are not.
FAQ
Adobe Commerce and Magento Open Source write their logs as free text into a few
files under var/log. On one server that is fine. In a container it fails
twice over: the files vanish when the container is replaced, and the lines
carry no fields, so finding one customer's failed checkout means reading rather
than searching. This extension changes where those records go and what shape
they arrive in - one structured line per record, on standard output, in a
format the log platform you already pay for parses without a translation layer.
No logging call anywhere in the store has to change.
When the store runs in containers - ECS, Kubernetes, Cloud Run, plain Docker - and its logs currently die with them. When there is already a log platform beside the store (Elastic or OpenSearch, Grafana Loki, Graylog, Datadog, Google Cloud Logging, Splunk, CloudWatch) and Magento is the one source still feeding it prose. When an incident means correlating one request across several log lines and there is no request id to correlate on. And when staging and production have to be told apart in the same index, which needs a service, an environment and a version on every line, locked per environment rather than typed into a form.
When there is no log collector: on a single server where someone reads
var/log over SSH, structured records buy nothing, and a log viewer
extension - a grid in the admin panel over the same files - is the tool that
actually helps. When the problem is that var/log fills the disk: that is
rotation and archiving, not routing. When what is wanted is an audit trail of
which administrator changed which field: that is a different product entirely,
and this extension logs nothing of its own. And on Adobe Commerce or Magento
Open Source 2.4.7 or older: the extension is written against Monolog 3,
which the platform ships from 2.4.8 onwards, so on 2.4.7 the package declines
to install rather than failing later.
It replaces the destination of the logs the platform already writes. Disabled -
the shipped default - it is invisible, and every record lands exactly where it
did before, including the exception.log split and the per-channel files.
Enabled, each record goes to one of three destinations: standard output and
standard error split by severity, standard output only, or the same var/log
files as before - in one of eleven formats: compact JSON, Elastic Common
Schema 8.x, Logstash, RFC 5424 syslog, Google Cloud Logging, Datadog, GELF 1.1,
logfmt, Monolog JSON, Magento's line format, and a custom template.
Seven loggers are covered: the main application log plus the payment,
shipping, cron, Admin Adobe IMS, Service Proxy and Payment Services channels.
A record several handlers would have written is emitted once, not twice. Every
JSON line carries a request id from the X-Request-Id header or generated once
per process; exceptions arrive as typed error objects with a bounded stack
trace; service, environment and version are configuration values that can be
locked in app/etc/env.php, where they beat the database and are read before
it is even reachable. Scanner probes for files that do not exist are recorded
as warnings rather than errors, so error-level alerting stays quiet.
One console command, webramos:structured-logging:inspect, lists every logger
in the installation with its destination and flags any log still writing its
own file or any record written twice. It can measure var/log around a marker
record to prove that rather than infer it, and it refuses to run that probe
while the extension is disabled.
It does not show you the logs. There is no grid, no reader, no admin page
beyond its configuration section - if reading var/log from the admin panel is
the requirement, a log viewer extension is the right purchase and this is not.
It does not ship anything anywhere. There is no batching, no push exporter and no OTLP: it writes lines, and the collector already running beside the store does the shipping. A backend that only accepts pushed payloads needs an agent in front of it.
It does not rotate, archive or clean up var/log, and it does not cap file
sizes. It does not record who changed what in the admin panel. It does not
redact: it carries other extensions' records unchanged, so a module that logs
something sensitive still logs it. It adds no database table and stores nothing
of its own. And writers that bypass the platform's logger entirely -
error_log(), direct file writes, shell redirects - are outside its reach; the
inspect command's probe finds the ones that write files, and names them.