Thiết kế website giá rẻ

Question

I have a RabbitMQ Spring Boot application, and I would like to use Spring Boot Actuator in order to export queue depth metrics related to the number of messages per consumer. I’m expecting the metrics to be read and exported on a regular interval. The purpose of these metrics is to scale out/in the application to increase the number of consumers if the messages pass a certain threshold.

To summarize the architecture, I have a Spring boot application using Spring boot actuator and micrometer to provide the metrics, and use Open Telemetry to export the metrics to an Open Telemetry collector. The collector would then send the metrics to AWS Cloudwatch.

The problem that I’m experiencing is that I’m continually seeing the message No metric data to export - skipping export and I’m not sure why. The application is idle when I start it up, but even when I post messages on the queues to keep the application active, I’m still seeing the same message.

I am expecting the meters that I’ve registered in io.micrometer.registry.otlp.OtlpMeterRegistry would be passed on or shared in the io.opentelemetry.sdk.internal.ComponentRegistry, but when I debug this, I can see that the meters are not populated in io.opentelemetry.sdk.internal.ComponentRegistry when getComponents() is called which is puzzling to me given that the logs show that the meters are being shared between registries.

Here is the part of the gradle file that contains the libraries that I”m using to gather metrics. I must emphasize that I cannot upgrade to Spring Boot 3.x.

implementation("org.springframework.boot:spring-boot-starter-actuator:2.7.0")
implementation 'io.micrometer:micrometer-core:1.9.0'
implementation 'io.micrometer:micrometer-registry-otlp:1.9.0'
implementation 'io.opentelemetry:opentelemetry-sdk:1.14.0'
implementation 'io.opentelemetry:opentelemetry-sdk-metrics:1.14.0'
implementation 'io.opentelemetry:opentelemetry-exporter-otlp-metrics:1.14.0'

Here is the main spring configuration to export the metrics to Open Telemetry.

@Configuration
public class OtelMetricsExporterConfiguration {

    private static final Logger LOG = LoggerFactory.getLogger(OtelMetricsExporterConfiguration.class);

    @Bean
    public PeriodicMetricReader periodicMetricReader() {
        MetricExporter metricExporter = OtlpGrpcMetricExporter.builder()
                .setEndpoint("http://localhost:4317")
                .build();

        return PeriodicMetricReader.builder(metricExporter)
                .setInterval(Duration.ofSeconds(10)) //10 seconds just for debugging
                .build();
    }

    @Bean
    public OpenTelemetry openTelemetry(PeriodicMetricReader periodicMetricReader) {
        SdkMeterProvider meterProvider = SdkMeterProvider.builder()
                .registerMetricReader(periodicMetricReader)
                .build();

        OpenTelemetrySdk openTelemetrySdk = OpenTelemetrySdk.builder()
                .setMeterProvider(meterProvider)
                .build();

        GlobalOpenTelemetry.set(openTelemetrySdk);
        return openTelemetrySdk;
    }

    @Bean
    public OtlpMeterRegistry otlpMeterRegistry(Clock clock) {
        return new OtlpMeterRegistry(OtlpConfig.DEFAULT, clock);
    }

    @Bean
    public MeterRegistryCustomizer<MeterRegistry> configureRabbitMetrics(RabbitQueueMetricGenerator rabbitQueueMetricGenerator) {
        return registry -> {
            Set<Metric> metrics = rabbitQueueMetricGenerator.generateMetrics();
            for (Metric metric : metrics) {
                LOG.debug("Registering metric: {} - Value: {}", metric.name(), metric.value());
                Gauge.builder(metric.name(), metric, Metric::value).tag("queue", metric.name())
                        .register(registry);
            }
        };
    }

    @Bean
    public MeterRegistryCustomizer<MeterRegistry> checkSharedMeters(OpenTelemetry openTelemetry) {
        return otlpMeterRegistry -> {
            MeterProvider otelMeterProvider = openTelemetry.getMeterProvider();
            otlpMeterRegistry.getMeters().forEach(meter -> {
                boolean isShared = otelMeterProvider.get(meter.getId().getName()) != null;
                if (isShared) {
                    LOG.debug("Meter {} is shared between Micrometer and OpenTelemetry.", meter.getId().getName());
                } else {
                    throw new IllegalStateException("Meter " + meter.getId().getName() + " is NOT shared between Micrometer and OpenTelemetry.");
                }
            });
        };
    }

    @Bean
    public MeterRegistryCustomizer<MeterRegistry> configureMetrics() {
        return registry -> {
            registry.config().meterFilter(MeterFilter.deny(id -> {
                LOG.debug("Denying meter: {}", id.getName());
                return false;
            }));
        };
    }

    @Bean
    public MeterRegistryCustomizer<MeterRegistry> verifyMeterData() {
        return registry -> {
            registry.forEachMeter(meter -> {
                meter.measure().forEach(measurement -> {
                    LOG.debug("Meter: {}, Measurement: {}, Value: {}", meter.getId().getName(), measurement.getStatistic(), measurement.getValue());
                });
            });
        };
    }

    @Bean
    public MeterRegistryCustomizer<MeterRegistry> verifyGaugeValues() {
        return registry -> {
            registry.getMeters().stream()
                    .filter(meter -> meter instanceof Gauge)
                    .map(meter -> (Gauge) meter)
                    .forEach(gauge -> {
                        LOG.debug("Gauge: {}, Value: {}", gauge.getId().getName(), gauge.value());
                    });
        };
    }
}

I’ve added debugging logs, and I am able to confirm the meters are getting registered in the spring managed micrometer registry (io.micrometer.registry.otlp.OtlpMeterRegistry) as shown in the logs below.

o.m.t.c.OtelMetricsExporterConfiguration : Registering metric: test-queue-messages-queued-per-consumer - Value: 0
o.m.t.c.OtelMetricsExporterConfiguration : Registering metric: test2-queued-per-consumer - Value: 0
o.m.t.c.OtelMetricsExporterConfiguration : Registering metric: test-queue-messages-total - Value: 0
o.m.t.c.OtelMetricsExporterConfiguration : Registering metric: test2-total - Value: 0
o.m.t.c.OtelMetricsExporterConfiguration : Registering metric: test-queue-consumers - Value: 0
o.m.t.c.OtelMetricsExporterConfiguration : Registering metric: test-consumers - Value: 0
o.m.t.c.OtelMetricsExporterConfiguration : Meter test-queue-messages-total is shared between Micrometer and OpenTelemetry.
o.m.t.c.OtelMetricsExporterConfiguration : Meter test-consumers is shared between Micrometer and OpenTelemetry.
o.m.t.c.OtelMetricsExporterConfiguration : Meter test-queue-consumers is shared between Micrometer and OpenTelemetry.
o.m.t.c.OtelMetricsExporterConfiguration : Meter test2-queued-per-consumer is shared between Micrometer and OpenTelemetry.
o.m.t.c.OtelMetricsExporterConfiguration : Meter test2-total is shared between Micrometer and OpenTelemetry.
o.m.t.c.OtelMetricsExporterConfiguration : Meter test-queue-messages-queued-per-consumer is shared between Micrometer and OpenTelemetry.
o.m.t.c.OtelMetricsExporterConfiguration : Meter: test-queue-messages-total, Measurement: VALUE, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Meter: test-consumers, Measurement: VALUE, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Meter: test-queue-consumers, Measurement: VALUE, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Meter: test2-queued-per-consumer, Measurement: VALUE, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Meter: test2-total, Measurement: VALUE, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Meter: test-queue-messages-queued-per-consumer, Measurement: VALUE, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Gauge: test-queue-messages-total, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Gauge: test-consumers, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Gauge: test-queue-consumers, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Gauge: test2-queued-per-consumer, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Gauge: test2-total, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Gauge: test-queue-messages-queued-per-consumer, Value: 0.0
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.gc.max.data.size
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.gc.live.data.size
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.gc.memory.allocated
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.gc.memory.promoted
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.usage.after.gc
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.gc.overhead
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.count
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.total.capacity
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.count
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.total.capacity
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.count
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.buffer.total.capacity
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.committed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.committed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.committed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.committed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.committed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.used
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.committed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.memory.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.peak
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.daemon
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.live
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.states
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.states
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.states
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.states
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.states
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.threads.states
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.classes.loaded
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.classes.unloaded
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: logback.events
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: logback.events
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: logback.events
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: logback.events
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: logback.events
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: process.uptime
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: process.start.time
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: system.cpu.count
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: system.load.average.1m
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: system.cpu.usage
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: process.cpu.usage
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: process.files.open
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: process.files.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: disk.free
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: disk.total
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.acquire
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.creation
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.usage
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.timeout
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.idle
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.active
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.pending
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: hikaricp.connections.min
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jdbc.connections.active
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jdbc.connections.idle
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jdbc.connections.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jdbc.connections.min
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.connections
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.channels
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.published
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.consumed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.acknowledged
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.rejected
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.failed_to_publish
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.acknowledged_published
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.not_acknowledged_published
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: rabbitmq.unrouted_published
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.completed
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.active
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.queued
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.queue.remaining
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.pool.size
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.pool.core
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: executor.pool.max
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: jvm.gc.pause
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: application.started.time
o.m.t.c.OtelMetricsExporterConfiguration : Denying meter: application.ready.time

Thiết kế website giá rẻ

Danh mục

Why are the micrometer registry’s metrics not copied/shared to the Open Telemetry registry for export?