Logging and Diagnostics on CCv2: From Ephemeral Nodes to Searchable Signal

On CCv2 you cannot SSH into a node to tail a log, and even if you could, the node's disk is ephemeral, so the log you want died with the pod that recycled. Diagnostics on Commerce Cloud is therefore a different discipline from on-premise: logs must stream off the node to a searchable store, log levels must be chosen so the signal is not buried in noise, and requests must carry correlation so a single customer's journey is traceable across the nodes it bounced between. Get this right and production incidents are minutes of searching; get it wrong and they are hours of guessing at a system you cannot see. This guide is the logging and diagnostics setup for CCv2, connecting the hAC power-user guide (runtime diagnostics), the Dynatrace observability guide (metrics), and the integration alerting guide (turning signal into alerts).

The Ephemeral-Node Reality#

The Cloud Portal ops guide's ephemeral-storage rule reaches logging directly: a node's local log file is not durable. So the platform streams logs to a centralized aggregation store (Kibana/OpenSearch on CCv2), and that store, not the node, is where you read logs. Consequences for how you work:

• Search, do not tail. Diagnostics is querying the aggregated store across all nodes for a time window and pattern, not watching one file. This is more powerful (you see the whole cluster at once) but requires the search literacy the on-premise tail-and-grep habit does not build.
• Local files are for the moment, not the record. Anything written to a node's disk is gone at the next recycle; the aggregated store is the system of record for what happened.
• Retention is finite and configured. The aggregation store holds a bounded window; know yours, and export anything you need beyond it before it ages out (the data maintenance guide's retention thinking applies to logs).

Log Levels: Signal Versus Noise#

The single most impactful logging discipline is level hygiene, because a log full of noise hides the one line that matters:

• Production runs at WARN or higher for the platform, with your own packages at INFO for the events operators actually need to see. DEBUG in production is a firehose that buries signal and costs performance and storage.
• Change levels at runtime, per node, temporarily (the hAC power-user guide): when investigating, raise your package to DEBUG in hAC, reproduce, and lower it again. No redeploy, no permanent noise. This is the correct answer to "we need more logging," not shipping a build with DEBUG left on.
• Log at the right level for the message: an expected, handled condition is not an ERROR (crying-wolf errors train operators to ignore the log); a genuine failure is not a DEBUG (buried where nobody looks). The integration alerting guide's severity discipline starts with logging the event at a level that matches its actual severity.
• Never log secrets or personal data (the data protection guide): passwords, tokens, full card numbers, and PII in logs are a compliance incident, and the aggregated, retained, searchable store makes a logged secret a durable exposure.

Correlation: Tracing One Request Across Nodes#

On a multi-node cluster with sticky-but-not-guaranteed routing, one customer's journey (and one background flow) touches multiple nodes, and a log line on its own tells you little. Correlation IDs are what make a request traceable:

• Assign a correlation/request ID at the edge and propagate it through the request (into the logging MDC so every log line carries it) and across service and integration calls (into outbound headers, the webhooks and integration guides), so a single ID follows the work everywhere it goes.
• Search by correlation ID in the aggregated store to reconstruct the whole request across nodes and services: the storefront request, the OCC calls, the integration hops, in order. This turns "something failed for this customer" from an archaeology dig into one query.
• Include it in error responses (safely) so a support ticket carrying the ID lets you find the exact request instantly, the PK-analyzer-for-requests equivalent of the hAC guide's PK trick.

Without correlation, distributed logs are a pile of disconnected lines; with it, they are a traceable narrative.

Structured Logging#

Logs that are searchable and analyzable, not just readable, pay off in the aggregated store:

• Log structured fields (key-value or JSON) rather than only free-text, so the store can filter and aggregate on them: correlation ID, user or order reference (non-PII), operation, outcome, duration. A structured log is queryable ("all failed checkouts in this hour with duration over 2s"); a prose log is only greppable.
• Consistent messages for the same event, so a pattern is findable. Ad hoc phrasing of the same failure across the codebase makes it un-searchable.
• Log the durations that matter: slow-operation logging (a service call, a query, an integration hop over a threshold) feeds directly into the performance investigation the persistence and FlexibleSearch guides describe.

Runtime Diagnostics When Logs Are Not Enough#

Logs tell you what happened; sometimes you need the live state, and CCv2 gives you that without shell access (the hAC power-user guide):

• hAC log-level changes and JDBC logging: raise levels live, and turn on JDBC statement logging to catch the slow query behind an "it is slow" report (then turn it off; it is verbose).
• Thread dumps: a hung-request investigation is three thread dumps ten seconds apart from the affected node, read for what is stuck. hAC exposes this, which is exactly what you have instead of jstack over SSH.
• Cache and cluster screens: the persistence and clustering guides' diagnostic screens for "stale data on some nodes" and cache-health questions.
• Dynatrace for the metric-shaped questions: rates, latencies, resource anomalies, and distributed traces (the observability guide), complementing the log store's event-shaped answers. Use logs for "what happened to this request," metrics for "what is the trend," and the two together for most real incidents.

From Logs to Alerts#

Logging is diagnostics; the integration alerting guide's point stands: a log nobody reads is a record, not a safeguard. Wire the log store's known-bad patterns into alerts (error-rate spikes, specific exception signatures, integration failure patterns) routed to owners, so the system tells you it is unhealthy rather than waiting for you to search. The observability guide covers the Dynatrace problem-detection side; the log store covers the pattern-match side; together they are the difference between finding an incident from an alert and finding it from a customer.

Checklist#

• Logs stream to the centralized store (Kibana/OpenSearch); diagnostics is search, not tail; retention window known
• Production at WARN+ for the platform, INFO for your packages; runtime level changes via hAC, not redeploys
• Message severity matches real severity; no crying-wolf errors, no buried failures
• No secrets or PII in logs, ever
• Correlation IDs assigned at the edge, propagated through the request and across service/integration calls, searchable
• Structured, consistent log fields (correlation, references, operation, outcome, duration) for querying and aggregation
• Slow-operation durations logged to feed performance investigation
• Runtime diagnostics fluency: hAC level changes, JDBC logging, thread dumps, cache/cluster screens
• Known-bad log patterns wired into alerts routed to owners

Diagnostics on CCv2 is the skill of seeing a system you cannot touch, and it rests on three moves: get every log off the ephemeral node into a searchable store, keep the signal clean with level and severity discipline, and thread a correlation ID through everything so distributed logs tell a story. Teams that build this find production incidents in minutes; teams that treat CCv2 logging like on-premise tail spend those incidents wishing they could SSH into a node that no longer exists.