com.tc.exception.TCNotRunningException: Terracotta is not running.

Terracotta | namratag | 5 years ago
  1. 0

    Terracotta 3.5.2 : com.tc.exception.TCNotRunningException

    Terracotta | 5 years ago | namratag
    com.tc.exception.TCNotRunningException: Terracotta is not running.
  2. 0

    When using a nonstop configuration on a distributed cache with bigmemory max 4.0.0, an initialization race condition exists. If you try to put to a nonstop cache too quickly after initializing the CacheManager, the put fails to affect L1 or L2, and an exception is silently logged, but not thrown to client code: {code} Exception in thread "init Store asynchronously testCache" java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running. at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:183) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.doInit(NonStopStoreWrapper.java:188) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.access$100(NonStopStoreWrapper.java:51) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper$1.run(NonStopStoreWrapper.java:121) at java.lang.Thread.run(Thread.java:662) Caused by: com.tc.exception.TCNotRunningException: Terracotta is not running. at com.tc.object.locks.ClientLockManagerImpl.checkState(ClientLockManagerImpl.java:724) at com.tc.object.locks.ClientLockManagerImpl.unlock(ClientLockManagerImpl.java:223) at com.tc.object.bytecode.ManagerImpl.unlock(ManagerImpl.java:804) at com.tc.platform.PlatformServiceImpl.commitLock(PlatformServiceImpl.java:105) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.doCommitLock(ToolkitLockingApi.java:175) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.unlock(ToolkitLockingApi.java:162) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.unlock(AggregateIsolatedToolkitTypeRoot.java:118) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:76) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:31) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:30) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:17) at com.terracotta.toolkit.TerracottaToolkit.getSet(TerracottaToolkit.java:260) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:226) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:223) at com.terracotta.toolkit.nonstop.AbstractToolkitObjectLookup.getInitializedObject(AbstractToolkitObjectLookup.java:30) at com.terracotta.toolkit.nonstop.NonStopInvocationHandler.invoke(NonStopInvocationHandler.java:35) at $Proxy17.getReadWriteLock(Unknown Source) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadEnabledNodesSet.<init>(BulkLoadEnabledNodesSet.java:42) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadToolkitCache.<init>(BulkLoadToolkitCache.java:49) at org.terracotta.modules.ehcache.store.ClusteredStoreBackend.<init>(ClusteredStoreBackend.java:37) at org.terracotta.modules.ehcache.store.ClusteredStore.<init>(ClusteredStore.java:141) at org.terracotta.modules.ehcache.store.EnterpriseClusteredStore.<init>(EnterpriseClusteredStore.java:105) at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.newStore(EnterpriseTerracottaClusteredInstanceFactory.java:25) at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.createStore(TerracottaClusteredInstanceFactory.java:92) at net.sf.ehcache.terracotta.ClusteredInstanceFactoryWrapper.createStore(ClusteredInstanceFactoryWrapper.java:93) at net.sf.ehcache.CacheManager.createTerracottaStore(CacheManager.java:623) at net.sf.ehcache.Cache$1.call(Cache.java:1106) at net.sf.ehcache.Cache$1.call(Cache.java:1103) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:176) ... 4 more {code} The exception logged varies between the "Terracotta is not running" message above and something about the store already being shutdown. I've even seen cases where no exception is logged at all, but the put still fails. I've attached a reproduction test case as a maven project (bigmemory-init.zip). To reproduce, install BigMemory Max 4.0.0 and start the server. Extract the zip and run mvn test in the project root. If you comment out the nonstop configuration in src/main/resources/ehcache.xml, the failures no longer occur, which leads me to believe that this is an initialization race condition specific to nonstop caches.

    Terracotta Project Issue Tracker | 4 years ago | Greg Adams
    java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running.
  3. 0

    When using a nonstop configuration on a distributed cache with bigmemory max 4.0.0, an initialization race condition exists. If you try to put to a nonstop cache too quickly after initializing the CacheManager, the put fails to affect L1 or L2, and an exception is silently logged, but not thrown to client code: {code} Exception in thread "init Store asynchronously testCache" java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running. at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:183) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.doInit(NonStopStoreWrapper.java:188) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.access$100(NonStopStoreWrapper.java:51) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper$1.run(NonStopStoreWrapper.java:121) at java.lang.Thread.run(Thread.java:662) Caused by: com.tc.exception.TCNotRunningException: Terracotta is not running. at com.tc.object.locks.ClientLockManagerImpl.checkState(ClientLockManagerImpl.java:724) at com.tc.object.locks.ClientLockManagerImpl.unlock(ClientLockManagerImpl.java:223) at com.tc.object.bytecode.ManagerImpl.unlock(ManagerImpl.java:804) at com.tc.platform.PlatformServiceImpl.commitLock(PlatformServiceImpl.java:105) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.doCommitLock(ToolkitLockingApi.java:175) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.unlock(ToolkitLockingApi.java:162) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.unlock(AggregateIsolatedToolkitTypeRoot.java:118) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:76) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:31) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:30) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:17) at com.terracotta.toolkit.TerracottaToolkit.getSet(TerracottaToolkit.java:260) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:226) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:223) at com.terracotta.toolkit.nonstop.AbstractToolkitObjectLookup.getInitializedObject(AbstractToolkitObjectLookup.java:30) at com.terracotta.toolkit.nonstop.NonStopInvocationHandler.invoke(NonStopInvocationHandler.java:35) at $Proxy17.getReadWriteLock(Unknown Source) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadEnabledNodesSet.<init>(BulkLoadEnabledNodesSet.java:42) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadToolkitCache.<init>(BulkLoadToolkitCache.java:49) at org.terracotta.modules.ehcache.store.ClusteredStoreBackend.<init>(ClusteredStoreBackend.java:37) at org.terracotta.modules.ehcache.store.ClusteredStore.<init>(ClusteredStore.java:141) at org.terracotta.modules.ehcache.store.EnterpriseClusteredStore.<init>(EnterpriseClusteredStore.java:105) at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.newStore(EnterpriseTerracottaClusteredInstanceFactory.java:25) at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.createStore(TerracottaClusteredInstanceFactory.java:92) at net.sf.ehcache.terracotta.ClusteredInstanceFactoryWrapper.createStore(ClusteredInstanceFactoryWrapper.java:93) at net.sf.ehcache.CacheManager.createTerracottaStore(CacheManager.java:623) at net.sf.ehcache.Cache$1.call(Cache.java:1106) at net.sf.ehcache.Cache$1.call(Cache.java:1103) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:176) ... 4 more {code} The exception logged varies between the "Terracotta is not running" message above and something about the store already being shutdown. I've even seen cases where no exception is logged at all, but the put still fails. I've attached a reproduction test case as a maven project (bigmemory-init.zip). To reproduce, install BigMemory Max 4.0.0 and start the server. Extract the zip and run mvn test in the project root. If you comment out the nonstop configuration in src/main/resources/ehcache.xml, the failures no longer occur, which leads me to believe that this is an initialization race condition specific to nonstop caches.

    Terracotta Project Issue Tracker | 4 years ago | Greg Adams
    java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    why Terracotta is not running.

    Terracotta | 4 years ago | dancer82
    com.tc.exception.TCNotRunningException: Terracotta is not running.
  6. 0

    why Terracotta is not running.

    Terracotta | 4 years ago | dancer82
    com.tc.exception.TCNotRunningException: Terracotta is not running.

    Root Cause Analysis

    1. com.tc.exception.TCNotRunningException

      Terracotta is not running.

      at com.tc.object.locks.ClientLockManagerImpl.waitUntilRunning()
    2. com.tc.object
      ManagerUtilInternal.beginLock
      1. com.tc.object.locks.ClientLockManagerImpl.waitUntilRunning(ClientLockManagerImpl.java:618)
      2. com.tc.object.locks.ClientLockManagerImpl.lock(ClientLockManagerImpl.java:92)
      3. com.tc.object.bytecode.ManagerImpl.lock(ManagerImpl.java:795)
      4. com.tc.object.bytecode.ManagerUtilInternal.beginLock(ManagerUtilInternal.java:56)
      4 frames
    3. org.terracotta.locking
      LongLockStrategy.beginLock
      1. org.terracotta.locking.strategy.LongLockStrategy.beginLock(LongLockStrategy.java:16)
      2. org.terracotta.locking.strategy.LongLockStrategy.beginLock(LongLockStrategy.java:7)
      2 frames
    4. com.terracotta.toolkit
      ConcurrentDistributedMapDsoArray.get
      1. com.terracotta.toolkit.collections.ConcurrentDistributedServerMapDso.beginLock(ConcurrentDistributedServerMapDso.java:168)
      2. com.terracotta.toolkit.collections.ConcurrentDistributedServerMapDso.get(ConcurrentDistributedServerMapDso.java:551)
      3. com.terracotta.toolkit.collections.ConcurrentDistributedMapDsoArray.get(ConcurrentDistributedMapDsoArray.java:217)
      3 frames
    5. org.terracotta.collections
      ConcurrentDistributedMap.get
      1. org.terracotta.collections.ConcurrentDistributedMap.get(ConcurrentDistributedMap.java:212)
      1 frame
    6. org.terracotta.cache
      TerracottaDistributedCache.containsKey
      1. org.terracotta.cache.TerracottaDistributedCache.getNonExpiredEntry(TerracottaDistributedCache.java:223)
      2. org.terracotta.cache.TerracottaDistributedCache.getNonExpiredEntryCoherent(TerracottaDistributedCache.java:152)
      3. org.terracotta.cache.TerracottaDistributedCache.containsKey(TerracottaDistributedCache.java:147)
      3 frames
    7. org.terracotta.modules
      ClusteredStore.containsKeyInMemory
      1. org.terracotta.modules.ehcache.store.ClusteredStoreBackendImpl.containsKey(ClusteredStoreBackendImpl.java:62)
      2. org.terracotta.modules.ehcache.store.backend.StrictBackend.containsKey(StrictBackend.java:68)
      3. org.terracotta.modules.ehcache.store.ClusteredStore.internalContainsKey(ClusteredStore.java:525)
      4. org.terracotta.modules.ehcache.store.ClusteredStore.containsKey(ClusteredStore.java:510)
      5. org.terracotta.modules.ehcache.store.ClusteredStore.containsKeyInMemory(ClusteredStore.java:517)
      5 frames
    8. Ehcache
      Cache.get
      1. net.sf.ehcache.Cache.searchInStoreWithStats(Cache.java:1884)
      2. net.sf.ehcache.Cache.get(Cache.java:1549)
      3. net.sf.ehcache.Cache.get(Cache.java:1522)
      3 frames
    9. com.googlecode.ehcache
      EhCacheInterceptor.invoke
      1. com.googlecode.ehcache.annotations.interceptor.EhCacheInterceptor.invokeCacheable(EhCacheInterceptor.java:119)
      2. com.googlecode.ehcache.annotations.interceptor.EhCacheInterceptor.invoke(EhCacheInterceptor.java:77)
      2 frames
    10. Spring AOP
      JdkDynamicAopProxy.invoke
      1. org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
      2. org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
      3. org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
      4. org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
      4 frames
    11. Unknown
      $Proxy68.lookup
      1. $Proxy68.lookup(Unknown Source)
      1 frame
    12. com.echovox.cash
      CashController.lookup
      1. com.echovox.cash.service.PaymentEntrypointSetGeneratorServiceImpl.generatePaymentEntrypointSet(PaymentEntrypointSetGeneratorServiceImpl.java:131)
      2. com.echovox.cash.web.controller.CashController.lookup(CashController.java:99)
      2 frames
    13. Java RT
      Method.invoke
      1. sun.reflect.GeneratedMethodAccessor295.invoke(Unknown Source)
      2. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      3. java.lang.reflect.Method.invoke(Method.java:597)
      3 frames
    14. Spring MVC
      FrameworkServlet.doPost
      1. org.springframework.web.servlet.mvc.multiaction.MultiActionController.invokeNamedMethod(MultiActionController.java:473)
      2. org.springframework.web.servlet.mvc.multiaction.MultiActionController.handleRequestInternal(MultiActionController.java:410)
      3. org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
      4. org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
      5. org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
      6. org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:807)
      7. org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
      8. org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
      8 frames
    15. JavaServlet
      HttpServlet.service
      1. javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
      2. javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      2 frames
    16. Glassfish Core
      ApplicationFilterChain.doFilter
      1. org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      2. org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      2 frames
    17. com.echovox.cash
      SessionTimeoutFilter.doFilter
      1. com.echovox.cash.web.filter.SessionTimeoutFilter.doFilter(SessionTimeoutFilter.java:71)
      1 frame
    18. Glassfish Core
      ApplicationFilterChain.doFilter
      1. org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      2. org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      2 frames
    19. com.echovox.cash
      CacheFilter.doFilter
      1. com.echovox.cash.web.filter.CacheFilter.doFilter(CacheFilter.java:60)
      1 frame
    20. Glassfish Core
      ApplicationFilterChain.internalDoFilter
      1. org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      1 frame