I have requirement for vast majority of metrics(gauges) to be polled once per 1h or even once per 24h.

There is an otlp collector in k8s where we push our metrics in accordance to the next config:

  otlp:
    metrics:
      export:
        enabled: true
        url: http://host:4318/v1/metrics
        step: 60m

So here we started facing unexpected things – if I just push metrics once per hour then grafana will show this –

And if we look at these lines closer – there are number of points were added each 30secs for 5 minutes in a row.

So, I know that I could set ‘step’ parameter to 30s, then I would see non-interrupted line. But I do not really need to poll my application’s query each 30s, right? I created next supplier –

@Slf4j
public class ThrottlingSup<T> implements Supplier<T> {
    private final Supplier<T> peer;
    private final AtomicReference<T> value = new AtomicReference<>();


    public ThrottlingSup(Supplier<T> peer, Duration duration) {
        this.peer = peer;
        engage(new Timer(), duration);
    }

    private void engage(Timer timer, Duration duration) {
        timer.scheduleAtFixedRate(new TimerTask() {
            @Override
            public void run() {
                value.set(null);
            }
        }, 0, duration.toMillis());
    }

    @Override
    public T get() {
        return value.updateAndGet(prev -> {
            if (prev == null) {
                return peer.get();
            }
            return prev;
        });
    }
}

Idea is simple, – I prevent real consumption by returning back cached value for certain period of time, and only after that period I retrieve another value.
But in this case I will get “U-shaped” chart.

However, ideally, i wanted just timeline with one point per hour(or day) chained together without blank spaces.

How can I achieve this?