I have a microservice running Quarkus with Vert.x/Mutiny, Panache, and Hibernate.
Periodically (usually after a month or so), some operations will start to fail with a token recognition error at: '^@'
-error.
I guess some layer in the application stack is corrupting some query cache or something, but I don’t really know where to start looking. Do anyone recognise this?
Here is the stack-trace:
ERROR c.e.h.q.ExceptionHandler [vert.x-eventloop-thread-0] Unhandled error Internal Server Error: java.lang.IllegalArgumentException: org.hibernate.query.SyntaxExcept
ion: At 1:65, token recognition error at: '^@' [FROM MyEntity WHERE instanceId^@^@^@^@^@]
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:143)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:167)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:173)
at org.hibernate.internal.AbstractSharedSessionContract.createQuery(AbstractSharedSessionContract.java:848)
at org.hibernate.internal.AbstractSharedSessionContract.createQuery(AbstractSharedSessionContract.java:753)
at org.hibernate.internal.AbstractSharedSessionContract.createQuery(AbstractSharedSessionContract.java:136)
at io.quarkus.hibernate.orm.runtime.session.TransactionScopedSession.createQuery(TransactionScopedSession.java:367)
at org.hibernate.engine.spi.SessionLazyDelegator.createQuery(SessionLazyDelegator.java:558)
at org.hibernate.engine.spi.SessionLazyDelegator.createQuery(SessionLazyDelegator.java:67)
at org.hibernate.Session_OpdLahisOZ9nWRPXMsEFQmQU03A_Synthetic_ClientProxy.createQuery(Unknown Source)
at io.quarkus.hibernate.orm.panache.common.runtime.CommonPanacheQueryImpl.createBaseQuery(CommonPanacheQueryImpl.java:366)
at io.quarkus.hibernate.orm.panache.common.runtime.CommonPanacheQueryImpl.createQuery(CommonPanacheQueryImpl.java:344)
at io.quarkus.hibernate.orm.panache.common.runtime.CommonPanacheQueryImpl.singleResultOptional(CommonPanacheQueryImpl.java:315)
at io.quarkus.hibernate.orm.panache.runtime.PanacheQueryImpl.singleResultOptional(PanacheQueryImpl.java:174)
at org.example.MyStore.getInstance(MyStore.java:85)
at org.example.MyStore_Subclass.getInstance$$superforward(Unknown Source)
at org.example.MyStore_Subclass$$function$$1.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:136)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
at org.example.MyStore_Subclass.getInstance(Unknown Source)
at org.example.MyStore_ClientProxy.getInstance(Unknown Source)
at org.example.MyWebServiceImpl.lambda$files$1(MyWebServiceImpl.java:84)
at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
at io.smallrye.mutiny.operators.uni.builders.UniCreateFromItemSupplier.subscribe(UniCreateFromItemSupplier.java:28)
at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
The getInstance()
-method is just calling PanacheRepositoryBase::find("instanceId", instanceId)
. I am not compiling to Native code and the service is running in an AWS instance (which has been completely restarted without the error going away, so I would assume the underlying metal has been changed as well).