I am getting System table verification failed during server start up. Here is the stack trace.
gw.pl.exception.GWLifecycleException: An exception was thrown while starting a component. Setting runlevel to SHUTDOWN at com.guidewire.pl.system.server.InitTab.increaseRunLevelTo(InitTab.java:680) ~[pl-10.101.1.jar:?] at com.guidewire.pl.system.server.InitTab.setRunLevel(InitTab.java:380) ~[pl-10.101.1.jar:?] at com.guidewire.pl.system.servlet.GuidewireStartupServlet.init(GuidewireStartupServlet.java:88) [pl-10.101.1.jar:?] at javax.servlet.GenericServlet.init(GenericServlet.java:158) [servlet-api.jar:4.0.FR] at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1134) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1089) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:983) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4902) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5211) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [catalina.jar:9.0.40] at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717) [catalina.jar:9.0.40] at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:706) [catalina.jar:9.0.40] at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:978) [catalina.jar:9.0.40] at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1848) [catalina.jar:9.0.40] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) [tomcat-util.jar:9.0.40] at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118) [?:?] at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:773) [catalina.jar:9.0.40] at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:427) [catalina.jar:9.0.40] at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1576) [catalina.jar:9.0.40] at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:309) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366) [catalina.jar:9.0.40] at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:936) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:843) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [catalina.jar:9.0.40] at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384) [catalina.jar:9.0.40] at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374) [catalina.jar:9.0.40] at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) [tomcat-util.jar:9.0.40] at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) [?:?] at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardService.startInternal(StandardService.java:421) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [catalina.jar:9.0.40] at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930) [catalina.jar:9.0.40] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [catalina.jar:9.0.40] at org.apache.catalina.startup.Catalina.start(Catalina.java:772) [catalina.jar:9.0.40] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:342) [bootstrap.jar:9.0.40] at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:473) [bootstrap.jar:9.0.40] Caused by: gw.api.webservice.exception.ServerStateException: System table verification failed. at com.guidewire.pc.domain.productmodel.impl.ProductModelImpl.verifySystemTables(ProductModelImpl.java:1739) ~[pc-10.1.1.jar:?] at com.guidewire.pc.domain.productmodel.impl.ProductModelImpl.synchronizeSystemTablesWithFilesNoLocking(ProductModelImpl.java:860) ~[pc-10.1.1.jar:?] at com.guidewire.pc.domain.productmodel.impl.ProductModelImpl.synchronizeProductModelIfNeeded(ProductModelImpl.java:301) ~[pc-10.1.1.jar:?] at com.guidewire.pc.domain.productmodel.impl.ProductModelImpl.start(ProductModelImpl.java:247) ~[pc-10.1.1.jar:?] at com.guidewire.pl.system.server.InitTab.startDependency(InitTab.java:465) - (com.guidewire.pc.domain.productmodel.impl.ProductModelImpl@1ab1b155) ~[pl-10.101.1.jar:?] at com.guidewire.pc.system.server.PCInitTab.applicationEnterNoDaemons(PCInitTab.java:60) ~[pc-10.1.1.jar:?] at com.guidewire.pl.system.server.InitTab.enterNoDaemons(InitTab.java:881) ~[pl-10.101.1.jar:?] at com.guidewire.pl.system.server.InitTab.increaseRunLevelTo(InitTab.java:650) ~[pl-10.101.1.jar:?] ... 47 more
Code base is same as PRD and prod is up and happy. Test env is having the issue. Tried to compare the prd branch vs test branch and there is no difference.