Observability & Continuous Profiling in Dockerized Spring Boot

Implementing robust observability across AWS Fargate and Azure Container Apps

cloud container infrastructure observability

Highlights

Integration Details: Comprehensive list of tools for logging, metrics, tracing, and continuous profiling including directions (pull vs push) and authentication/authorization methods.
Multi-Tenant Centralized UI: Strategies using Grafana or similar platforms for cost-effective centralized observability across 30 app deployments.
Data Storage Requirements: Detailed architectural insights on storage types, capacities, and considerations for long-term retention (approximately 1 year) across various observability pillars.

Introduction

As applications grow in complexity—especially when deployed as Dockerized JVMs in containerized cloud environments like AWS Fargate and Azure Container Apps—the need for robust observability becomes paramount. The concept of observability encompasses not only logging, metrics, and traces (the three classical pillars) but also continuous profiling. This comprehensive approach ensures that the system remains efficient, issues are detected and mitigated early, and performance optimization becomes a continuous process.

In this guide, we will integrate the three pillars of observability combined with continuous profiling into a Spring Boot application. We also explore strategic integrations and implementation approaches that account for multi-tenant centralized UIs, diverse data storage requirements, and advanced deployment scenarios—considering sidecar containers or agent-like software when beneficial.

1. Three Pillars of Observability + Continuous Profiling

The observability strategy is built upon four key pillars:

A. Metrics Collection

Tool: Micrometer with Prometheus Integration.
Integration: Spring Boot Actuator exposes metrics in a Prometheus compatible format.
Direction: Traditionally, Prometheus works on a pull-based model where it scrapes metrics endpoints. However, a push gateway can be employed if a push model fits better with the dynamic container scaling.
Authentication/Authorization: Basic Authentication or OAuth can secure endpoints, and mutual TLS is an option for more secure interactions.

Configuration involves adding relevant dependencies:


# application.yml snippet for Prometheus metrics
management:
  endpoints:
    web:
      exposure:
        include: prometheus,health,info
  metrics:
    export:
      prometheus:
        enabled: true

Additionally, consider the use of Prometheus Pushgateway where direct scraping from dynamic app containers behind a load balancer is not feasible.

B. Logging

Tool: Logback integrated with Grafana Loki.
Integration: Configure your Spring Boot application to produce structured JSON logs using frameworks like Logstash Logback Encoder and then ship these logs to Loki.
Direction: Push - logs are actively pushed from the source.
Authentication/Authorization: API keys are typically employed, sometimes in conjunction with OAuth; tenant-based isolation via identifiers is also common.

For a typical Maven project, add the following dependency:


<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>6.6</version>
</dependency>

Logs can be collected via a sidecar container running a tool like Promtail, which efficiently pushes log data to a centralized Loki instance.

C. Tracing

Tool: Micrometer Tracing integrated with Zipkin (or Jaeger as an alternative).
Integration: Tracing data is collected using Spring Cloud Sleuth (or OpenTelemetry) and is integrated with the tracing backend.
Direction: Push - traces are sent from the application to the tracing server.
Authentication/Authorization: Typically, API keys or mutual TLS (mTLS) is implemented while pushing trace data.

Example configuration:


# application.yml snippet for tracing with Zipkin
management:
  tracing:
    sampling:
      probability: 1.0
    export:
      zipkin:
        endpoint: http://zipkin-server:9411/api/v2/spans

D. Continuous Profiling

Tools: Java Flight Recorder (JFR) complemented by Async Profiler or similar options.
Integration: Continuous profiling can be integrated with JFR using JVM arguments and then processed by tools such as JDK Mission Control or Async Profiler which might be bundled as sidecar containers.
Direction: Depending on design — can be a push model where profiling data is periodically sent to a profiling server or a pull model where data is periodically retrieved.
Authentication/Authorization: JMX authentication is typically employed when accessing profiling endpoints.

Adding continuous profiling involves configuring your JVM with options like:


# Example JVM options for enabling JFR
-XX:StartFlightRecording=duration=1h,filename=recording.jfr,settings=profile

These tools allow for ongoing performance analysis, critical for detecting memory leaks, CPU spikes, and identifying performance bottlenecks.

2. Integrations Overview and Data Flow Direction

The following table details a summary of the integrations, direction of data flow, and authentication/authorization strategies for each component:

Observability Component	Tool/Integration	Data Flow Direction	AuthN/AuthZ Method
Metrics	Micrometer with Prometheus	Pull (default) or Push via Pushgateway	Basic Auth, OAuth, or mTLS
Logging	Logback with Grafana Loki	Push (via Promtail sidecar)	API Keys, OAuth, Tenant ID-based filtering
Tracing	Micrometer Tracing with Zipkin/Jaeger	Push	API keys, mTLS
Continuous Profiling	Java Flight Recorder (JFR) & Async Profiler	Pull or Scheduled Push	JMX Authentication (optionally mTLS if using a proxy)

This consolidated approach ensures that each component of the observability stack is securely integrated and balances the architectural constraints imposed by containerized environments behind load balancers.

3. Optional: Multi-Tenant Centralized UI for 30 App Deployments

When managing observability for multiple applications (30 app deployments in this case), having a centralized UI is often more cost-effective than multiple, separate interfaces per application. It minimizes redundancy, optimizes resource usage, and simplifies authentication management.

Centralized UI Considerations

A centralized UI such as Grafana provides excellent support for multi-tenancy. This approach offers:

Tenant Isolation: Visual and data isolation via dashboards, labels, and organizational roles.
Scalability: Ability to integrate data from various backends like Prometheus for metrics, Loki for logs, and Zipkin for tracing.
Cost Efficiency: Reduced overhead compared to deploying and maintaining separate UIs for each of the 30 app deployments.

Grafana supports multi-tenancy through the configuration of organizations and data source provisioning. You can secure access by integrating authentication methods such as LDAP, Keycloak, or OAuth 2.0, ensuring that each tenant sees only the data relevant to their applications.

The centralized UI strategy is especially crucial when load balancer configurations might randomly direct data collection requests to one container among many. With a multi-tenant centralized UI, observability data from various instances is aggregated without dependency on a single randomly picked container.

4. Data Storage Locations and Requirements

Long-term storage (approximately 1 year) is a critical requirement. Each observability pillar has distinct storage methods and capacity requirements depending on the data volume, retention policy, and the specific operational characteristics.

A. Metrics Storage

Tool/Platform: Prometheus with potential integration with long-term storage solutions such as Thanos or Cortex.
Storage Type: Time-series database stored on persistent disk volumes.
Requirements: Disk-based storage is generally required. The capacity depends on:

The frequency of metric collection (could range from several to hundreds of data points per minute).
The cardinality of the metrics being stored (label dimensions on each metric).
Retention period – For 1 year of data, consider several GB up to TBs depending on scale.

B. Logging Storage

Tool/Platform: Grafana Loki collecting logs from applications.
Storage Type: Logs can be stored on disk or in object storage (e.g., AWS S3 or Azure Blob Storage).
Requirements:

Capacity planning should account for high log verbosity from many dynamically deployed containers.
Retention of 1 year requires robust storage solutions; effective compression strategies reduce required disk space.
Some deployments opt for a hybrid approach—local disk for short-term access and object storage for long-term archival of logs.

C. Tracing Storage

Tool/Platform: Zipkin, Jaeger or similar distributed tracing systems.
Storage Type: Traces are often stored on disk-based storage or in distributed databases such as Elasticsearch, Cassandra, or even cloud-native options.
Requirements:

Tracing data requires persistent storage for a 1-year retention scenario, with capacities potentially estimated from several hundred GB to a few TB per app instance based on trace volume.
High read/write throughput logical for real-time analysis is required as traces often undergo constant access for debugging and performance monitoring.

D. Continuous Profiling Data Storage

Tool/Platform: Java Flight Recorder (JFR) generates data which can be stored locally and later aggregated.
Storage Type: Disk or network-attached storage (NAS), sometimes even object storage if profiling files need to be archived.
Requirements:

Sizing considerations involve the frequency of profiling sessions; a few GBs up to tens of GB per application instance annually.
Efficient indexing might be necessary when summarizing profiling data aggregated over long periods.

Collectively, staying compliant with a 1-year data retention policy across dynamic and multi-container environments signifies using persistent storage devices rather than in-memory databases. Strategies such as using sidecar containers that ship logs, metrics, and traces external to the container may mitigate issues with load balancer randomness during data collection.

5. Deployment Strategies and Considerations

The deployment architecture should be designed to handle multiple containers behind load balancers, ensuring that observability endpoints are directly accessible or aggregated efficiently. Here are the main strategies:

Sidecar Containers

Deploying sidecar containers is a recommended strategy. Sidecars can be used for agents that assist in:

Collecting logs using Promtail to forward to Loki.
Maintaining lifecycle management for continuous profiling agents.
Enabling local scraping of metrics if containers are behind a load balancer, and then proxying the data to a centralized Prometheus instance.

This architecture allows the main Spring Boot application container to remain lightweight with its core business logic, while other specialized containers handle the observability workload.

Agent-like Software Integration

In scenarios where running sidecar containers is not feasible, embedding agent-like software into the application container via a process supervisor (e.g., Supervisord) is a practical alternative. This integration ensures that:

Profiling and logging agents are started as separate processes within the same container.
The separation of observability concerns is logically maintained while deploying a monolithic container.
However, this approach might introduce increased complexity in container management and monitoring, so it is essential to balance the workload and understand the impact on the JVM’s performance.

Each method has benefits and trade-offs. In cloud-native environments like AWS Fargate and Azure Container Apps, sidecar containers are generally easier to scale and maintain.

6. Practical Implementation Steps

Below is a summary of how you might implement the observability solution:

Step 1: Spring Boot Configuration

Include the necessary dependencies:

Add Spring Boot Actuator for managing all endpoints.
Add Micrometer for metrics collection.
Integrate Spring Cloud Sleuth or OpenTelemetry for distributed tracing.
Include the appropriate libraries (e.g., Logback with the Logstash encoder for structured logging).

Step 2: Docker and Container Orchestration

Create a Dockerfile that encapsulates your Spring Boot application. For example:


# Dockerfile example
FROM openjdk:17-jdk-slim
COPY target/app.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-XX:StartFlightRecording=duration=1h,filename=recording.jfr,settings=profile", "-jar", "app.jar"]

Use Docker Compose or Kubernetes YAML files to define the observability stack, including sidecar containers for Promtail, the Prometheus Pushgateway, and other supporting services.

Step 3: Multi-Tenant UI and Aggregation

Deploy Grafana for a centralized UI:

Configure data sources (Prometheus for metrics, Loki for logs, Zipkin for traces).
Establish organization and role-based accesses.
Integrate multi-tenant isolation using labels and additional metadata from each app deployment.

Step 4: Secure Data Flow and Storage Provisions

Ensure that all endpoints are secured using:

Basic Auth, OAuth, or mTLS for metrics scraping.
API Keys and tenant ID authentication for log shipping to Loki.
Mutual TLS and API key configurations for trace endpoints such as Zipkin/Jaeger.
JMX and secure JVM options for continuous profiling data collection.

Provision persistent storage for:

Prometheus: Disk-based volumes that might require scaling up to TBs based on cardinality and data frequency.
Loki: Disk or object storage, leveraging native compression and retention policies tailored for 1-year data retention.
Tracing: Elasticsearch, Cassandra, or similar distributed store depending on the trace volume.
JFR recordings: Local disks or NAS, maintained with appropriate archival policies.

Conclusion

Implementing a robust observability stack for a Dockerized Spring Boot application running on AWS Fargate and Azure Container Apps involves orchestrating the three pillars — metrics, logging, and tracing — alongside continuous profiling. Each component must be integrated with careful attention to its data flow direction: whether using a pull model (as with Prometheus) or a push model (as with Loki and Zipkin).

Additionally, the need for a centralized, multi-tenant UI is paramount, primarily due to the complexities associated with managing 30 app deployments. Grafana provides an efficient solution by aggregating data from multiple sources, offering tenant-based isolation and a cost-effective approach compared with maintaining separate UIs per deployment.

Data storage strategies play a crucial role. Each observability component has distinct storage requirements: metrics need disk-based time-series databases, logs require durable disk or object storage with compression capabilities, traces demand high throughput persistent storage, and continuous profiling data must be carefully managed due to its potential size. Adopting sidecar containers helps overcome limitations imposed by load balancers by ensuring that observability endpoints are efficiently reached and that data collection is not hindered by container instance randomness.

Careful engineering choices—ranging from the selection of appropriate tools (Micrometer, Logback, OpenTelemetry, and JFR) to secure deployments with proper authentication and authorization—ensure that the observability system is both scalable and reliable. This holistic approach not only improves application performance monitoring but also empowers you with actionable insights, helping maintain high levels of operational excellence even as application complexity grows.