I have a multibranch pipeline that uses a Jenkins Template. After nightly restart, I get the following error and the old status of the jobs are not picked up again.
Aug 13, 2024 1:29:21 AM INFO com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider reconnectLater
Attempting to reconnect Kubernetes client in 5 mins
Aug 13, 2024 1:29:21 AM WARNING org.jenkinsci.plugins.workflow.cps.CpsFlowExecution onLoad
Error initializing storage and loading nodes, will try to create placeholders for: CpsFlowExecution[Owner[somePrj/Backend/develop/246:SomePrj/Backend/develop #246]]
Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 8124e099-cddb-40a4-a632-e0af04517d1c
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.saveActions(CpsFlowExecution.java:1286)
at org.jenkinsci.plugins.workflow.graph.FlowNode.save(FlowNode.java:486)
at org.jenkinsci.plugins.workflow.graph.FlowNode.persistSafe(FlowNode.java:492)
at org.jenkinsci.plugins.workflow.graph.FlowNode$1.add(FlowNode.java:442)
at org.jenkinsci.plugins.workflow.graph.FlowNode$1.add(FlowNode.java:432)
at java.base/java.util.AbstractList.add(AbstractList.java:111)
at hudson.model.Actionable.addAction(Actionable.java:155)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.createPlaceholderNodes(CpsFlowExecution.java:733)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.onLoad(CpsFlowExecution.java:798)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.getExecution(WorkflowRun.java:719)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.onLoad(WorkflowRun.java:577)
at hudson.model.RunMap.retrieve(RunMap.java:233)
at hudson.model.RunMap.retrieve(RunMap.java:61)
at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657)
at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:639)
at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:537)
at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:502)
at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:435)
at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:254)
at org.jenkinsci.plugins.workflow.job.WorkflowJob.getLastBuild(WorkflowJob.java:242)
at org.jenkinsci.plugins.workflow.job.WorkflowJob.getLastBuild(WorkflowJob.java:105)
at org.jenkinsci.plugins.prometheus.JobCollector.appendJobMetrics(JobCollector.java:219)
at org.jenkinsci.plugins.prometheus.JobCollector.lambda$collect$0(JobCollector.java:162)
at org.jenkinsci.plugins.prometheus.util.Jobs.forEachJob(Jobs.java:19)
at org.jenkinsci.plugins.prometheus.JobCollector.collect(JobCollector.java:155)
at io.prometheus.client.CollectorRegistry.collectorNames(CollectorRegistry.java:113)
at io.prometheus.client.CollectorRegistry.register(CollectorRegistry.java:50)
at org.jenkinsci.plugins.prometheus.service.DefaultPrometheusMetrics.registerCollector(DefaultPrometheusMetrics.java:63)
at org.jenkinsci.plugins.prometheus.service.DefaultPrometheusMetrics$1.doRun(DefaultPrometheusMetrics.java:82)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:92)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Setting Pipeline Speed / Durability to MAX_SURVIVABILITY in Jenkinsfile and also in Global Settings did not really help.
I initially thought that this exception comes from boozallen jenkins template plugin, but I cannot really find any supporting stacktrace here.
I cannot stop overnightly restarts, because the system is configured so and is out of my control.
Funny thing is, it does not happen to other projects which does not use Multibranch setup, or are using templates.