Useful monitoring tells you which operation is affected, how many users are affected and what evidence to inspect next. Begin with a service boundary and a workload baseline. Then connect request measurements to JVM and dependency signals. The local lab below makes the meaning of a counter concrete before introducing a telemetry pipeline.
Run a local measurement exercise
The Java 25 download creates an HTTP server bound to loopback on an automatically chosen port, sends ten requests to it, reads its simple metrics endpoint and shuts down. Two requests deliberately return HTTP 503. No external service or monitoring account is needed.
Download the source, checks and README (ZIP)
javac MonitorLab.java
java MonitorLabExpect requests=10, errors=2, a variable handler_nanos total and an error fraction of 20.0%. The exercise excludes the metrics request from the work count. Its handler timer includes response handling, but excludes client-side DNS and connection setup. Stating those boundaries is part of defining a useful metric.
Change the loop so only the first request fails and update the corresponding checks: the expected fraction becomes 1/10, or 10%. Ten invented requests are a demonstration, not a statistically useful production baseline. The sample’s in-memory counters reset on restart, and concurrent snapshots can be slightly staggered.
Choose signals around a reader-visible operation
On narrow screens, scroll this table horizontally. Keyboard users can focus it and use the arrow keys.
| Signal | Question it answers | Interpretation trap |
|---|---|---|
| Request rate by route template | Has demand changed? | A rising lifetime counter is not a requests-per-second rate. |
| Errors divided by eligible requests | What fraction of work is failing? | Agree on which status codes and cancellations count. |
| Latency distribution | How long do typical and slow requests take? | A mean hides a long tail; averages of percentiles are misleading. |
| JVM memory, GC and CPU | Is runtime pressure associated with the symptom? | A correlation needs investigation before a tuning change. |
| Dependency spans | Where does one request spend time? | Sampling and instrumentation coverage affect what is visible. |
Give charts units and a common time window. For a monotonic request counter, calculate the increase over the interval and divide by elapsed seconds, with restart handling. Use route templates such as /orders/{id} rather than one label per order. High-cardinality labels increase telemetry cost and can expose sensitive identifiers.
For a fictional five-minute window with 1,200 eligible requests and 12 failures, the error fraction is 12/1,200 = 1%. Compare with the same operation and similar load in a baseline window; a deployment that changes request mix can change aggregate numbers even when individual routes behave similarly.
Add an observable telemetry pipeline
The OpenTelemetry Java agent can instrument supported libraries when attached at JVM startup. Choose a release, verify its runtime and library coverage, and record that release with your deployment configuration. Start in a test environment with a receiver you control.
java -javaagent:opentelemetry-javaagent.jar -Dotel.service.name=orders-lab -Dotel.exporter.otlp.protocol=http/protobuf -Dotel.exporter.otlp.endpoint=http://localhost:4318 -jar your-service.jarThis command requires the downloaded agent, your application JAR and an OTLP HTTP receiver listening on port 4318; it is a configuration example rather than a command for the single-file lab. Set remote receiver authentication and TLS according to your environment. The agent configuration reference explains properties and precedence. Verify an actual request appears under the intended service name, inspect a trace, and check exporter logs when data is absent.
Compare resource use and request behavior with and without instrumentation under comparable load. Preserve trace context across supported service boundaries and keep credentials, bodies and personal data out of attributes. The local exercise tests metric arithmetic; it does not establish agent compatibility with every application.
Attach each alert to a decision
Write the condition, evaluation window, minimum traffic, owner and first action together. An illustrative alert might require an elevated failure fraction sustained over several minutes with enough traffic to interpret it. Set actual thresholds from your service objective and observed workload. Exercise the alert path in a test environment: induce a known fault, check delivery, restore the dependency and confirm recovery clears the condition.
For Java on z/OS, keep the same user-facing service boundary while adding platform evidence relevant to the deployment, such as workload management and subsystem delays. Runtime, agent and operating-system support must be checked against that environment. Historical Wily/Introscope integrations illustrate the need to connect Java measurements with host workload; they do not establish compatibility for a present deployment.
When an alert fires, record the time, affected route, deployment and request identifiers available to you, then follow the Java incident troubleshooting guide. Monitoring should shorten the route to evidence rather than prescribe a memory or thread setting from a single chart.