java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running.

Terracotta Project Issue Tracker | Greg Adams | 4 years ago
  1. 0

    When using a nonstop configuration on a distributed cache with bigmemory max 4.0.0, an initialization race condition exists. If you try to put to a nonstop cache too quickly after initializing the CacheManager, the put fails to affect L1 or L2, and an exception is silently logged, but not thrown to client code: {code} Exception in thread "init Store asynchronously testCache" java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running. at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:183) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.doInit(NonStopStoreWrapper.java:188) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.access$100(NonStopStoreWrapper.java:51) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper$1.run(NonStopStoreWrapper.java:121) at java.lang.Thread.run(Thread.java:662) Caused by: com.tc.exception.TCNotRunningException: Terracotta is not running. at com.tc.object.locks.ClientLockManagerImpl.checkState(ClientLockManagerImpl.java:724) at com.tc.object.locks.ClientLockManagerImpl.unlock(ClientLockManagerImpl.java:223) at com.tc.object.bytecode.ManagerImpl.unlock(ManagerImpl.java:804) at com.tc.platform.PlatformServiceImpl.commitLock(PlatformServiceImpl.java:105) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.doCommitLock(ToolkitLockingApi.java:175) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.unlock(ToolkitLockingApi.java:162) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.unlock(AggregateIsolatedToolkitTypeRoot.java:118) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:76) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:31) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:30) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:17) at com.terracotta.toolkit.TerracottaToolkit.getSet(TerracottaToolkit.java:260) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:226) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:223) at com.terracotta.toolkit.nonstop.AbstractToolkitObjectLookup.getInitializedObject(AbstractToolkitObjectLookup.java:30) at com.terracotta.toolkit.nonstop.NonStopInvocationHandler.invoke(NonStopInvocationHandler.java:35) at $Proxy17.getReadWriteLock(Unknown Source) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadEnabledNodesSet.<init>(BulkLoadEnabledNodesSet.java:42) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadToolkitCache.<init>(BulkLoadToolkitCache.java:49) at org.terracotta.modules.ehcache.store.ClusteredStoreBackend.<init>(ClusteredStoreBackend.java:37) at org.terracotta.modules.ehcache.store.ClusteredStore.<init>(ClusteredStore.java:141) at org.terracotta.modules.ehcache.store.EnterpriseClusteredStore.<init>(EnterpriseClusteredStore.java:105) at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.newStore(EnterpriseTerracottaClusteredInstanceFactory.java:25) at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.createStore(TerracottaClusteredInstanceFactory.java:92) at net.sf.ehcache.terracotta.ClusteredInstanceFactoryWrapper.createStore(ClusteredInstanceFactoryWrapper.java:93) at net.sf.ehcache.CacheManager.createTerracottaStore(CacheManager.java:623) at net.sf.ehcache.Cache$1.call(Cache.java:1106) at net.sf.ehcache.Cache$1.call(Cache.java:1103) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:176) ... 4 more {code} The exception logged varies between the "Terracotta is not running" message above and something about the store already being shutdown. I've even seen cases where no exception is logged at all, but the put still fails. I've attached a reproduction test case as a maven project (bigmemory-init.zip). To reproduce, install BigMemory Max 4.0.0 and start the server. Extract the zip and run mvn test in the project root. If you comment out the nonstop configuration in src/main/resources/ehcache.xml, the failures no longer occur, which leads me to believe that this is an initialization race condition specific to nonstop caches.

    Terracotta Project Issue Tracker | 4 years ago | Greg Adams
    java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running.
  2. 0

    When using a nonstop configuration on a distributed cache with bigmemory max 4.0.0, an initialization race condition exists. If you try to put to a nonstop cache too quickly after initializing the CacheManager, the put fails to affect L1 or L2, and an exception is silently logged, but not thrown to client code: {code} Exception in thread "init Store asynchronously testCache" java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running. at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:183) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.doInit(NonStopStoreWrapper.java:188) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.access$100(NonStopStoreWrapper.java:51) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper$1.run(NonStopStoreWrapper.java:121) at java.lang.Thread.run(Thread.java:662) Caused by: com.tc.exception.TCNotRunningException: Terracotta is not running. at com.tc.object.locks.ClientLockManagerImpl.checkState(ClientLockManagerImpl.java:724) at com.tc.object.locks.ClientLockManagerImpl.unlock(ClientLockManagerImpl.java:223) at com.tc.object.bytecode.ManagerImpl.unlock(ManagerImpl.java:804) at com.tc.platform.PlatformServiceImpl.commitLock(PlatformServiceImpl.java:105) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.doCommitLock(ToolkitLockingApi.java:175) at com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.unlock(ToolkitLockingApi.java:162) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.unlock(AggregateIsolatedToolkitTypeRoot.java:118) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:76) at com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:31) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:30) at com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:17) at com.terracotta.toolkit.TerracottaToolkit.getSet(TerracottaToolkit.java:260) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:226) at com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:223) at com.terracotta.toolkit.nonstop.AbstractToolkitObjectLookup.getInitializedObject(AbstractToolkitObjectLookup.java:30) at com.terracotta.toolkit.nonstop.NonStopInvocationHandler.invoke(NonStopInvocationHandler.java:35) at $Proxy17.getReadWriteLock(Unknown Source) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadEnabledNodesSet.<init>(BulkLoadEnabledNodesSet.java:42) at org.terracotta.modules.ehcache.store.bulkload.BulkLoadToolkitCache.<init>(BulkLoadToolkitCache.java:49) at org.terracotta.modules.ehcache.store.ClusteredStoreBackend.<init>(ClusteredStoreBackend.java:37) at org.terracotta.modules.ehcache.store.ClusteredStore.<init>(ClusteredStore.java:141) at org.terracotta.modules.ehcache.store.EnterpriseClusteredStore.<init>(EnterpriseClusteredStore.java:105) at org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.newStore(EnterpriseTerracottaClusteredInstanceFactory.java:25) at org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.createStore(TerracottaClusteredInstanceFactory.java:92) at net.sf.ehcache.terracotta.ClusteredInstanceFactoryWrapper.createStore(ClusteredInstanceFactoryWrapper.java:93) at net.sf.ehcache.CacheManager.createTerracottaStore(CacheManager.java:623) at net.sf.ehcache.Cache$1.call(Cache.java:1106) at net.sf.ehcache.Cache$1.call(Cache.java:1103) at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:176) ... 4 more {code} The exception logged varies between the "Terracotta is not running" message above and something about the store already being shutdown. I've even seen cases where no exception is logged at all, but the put still fails. I've attached a reproduction test case as a maven project (bigmemory-init.zip). To reproduce, install BigMemory Max 4.0.0 and start the server. Extract the zip and run mvn test in the project root. If you comment out the nonstop configuration in src/main/resources/ehcache.xml, the failures no longer occur, which leads me to believe that this is an initialization race condition specific to nonstop caches.

    Terracotta Project Issue Tracker | 4 years ago | Greg Adams
    java.lang.RuntimeException: com.tc.exception.TCNotRunningException: Terracotta is not running.
  3. 0

    why Terracotta is not running.

    Terracotta | 4 years ago | dancer82
    com.tc.exception.TCNotRunningException: Terracotta is not running.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    why Terracotta is not running.

    Terracotta | 4 years ago | dancer82
    com.tc.exception.TCNotRunningException: Terracotta is not running.
  6. 0

    Terracotta 3.5.2 : com.tc.exception.TCNotRunningException

    Terracotta | 5 years ago | namratag
    com.tc.exception.TCNotRunningException: Terracotta is not running.

    Root Cause Analysis

    1. com.tc.exception.TCNotRunningException

      Terracotta is not running.

      at com.tc.object.locks.ClientLockManagerImpl.checkState()
    2. com.tc.object
      ManagerImpl.unlock
      1. com.tc.object.locks.ClientLockManagerImpl.checkState(ClientLockManagerImpl.java:724)
      2. com.tc.object.locks.ClientLockManagerImpl.unlock(ClientLockManagerImpl.java:223)
      3. com.tc.object.bytecode.ManagerImpl.unlock(ManagerImpl.java:804)
      3 frames
    3. com.tc.platform
      PlatformServiceImpl.commitLock
      1. com.tc.platform.PlatformServiceImpl.commitLock(PlatformServiceImpl.java:105)
      1 frame
    4. com.terracotta.toolkit
      NonStopInvocationHandler.invoke
      1. com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.doCommitLock(ToolkitLockingApi.java:175)
      2. com.terracotta.toolkit.concurrent.locks.ToolkitLockingApi.unlock(ToolkitLockingApi.java:162)
      3. com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.unlock(AggregateIsolatedToolkitTypeRoot.java:118)
      4. com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:76)
      5. com.terracotta.toolkit.roots.impl.AggregateIsolatedToolkitTypeRoot.getOrCreateToolkitType(AggregateIsolatedToolkitTypeRoot.java:31)
      6. com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:30)
      7. com.terracotta.toolkit.factory.impl.AbstractPrimaryToolkitObjectFactory.getOrCreate(AbstractPrimaryToolkitObjectFactory.java:17)
      8. com.terracotta.toolkit.TerracottaToolkit.getSet(TerracottaToolkit.java:260)
      9. com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:226)
      10. com.terracotta.toolkit.NonStopToolkitImpl$9.lookupObject(NonStopToolkitImpl.java:223)
      11. com.terracotta.toolkit.nonstop.AbstractToolkitObjectLookup.getInitializedObject(AbstractToolkitObjectLookup.java:30)
      12. com.terracotta.toolkit.nonstop.NonStopInvocationHandler.invoke(NonStopInvocationHandler.java:35)
      12 frames
    5. Unknown
      $Proxy17.getReadWriteLock
      1. $Proxy17.getReadWriteLock(Unknown Source)
      1 frame
    6. org.terracotta.modules
      TerracottaClusteredInstanceFactory.createStore
      1. org.terracotta.modules.ehcache.store.bulkload.BulkLoadEnabledNodesSet.<init>(BulkLoadEnabledNodesSet.java:42)
      2. org.terracotta.modules.ehcache.store.bulkload.BulkLoadToolkitCache.<init>(BulkLoadToolkitCache.java:49)
      3. org.terracotta.modules.ehcache.store.ClusteredStoreBackend.<init>(ClusteredStoreBackend.java:37)
      4. org.terracotta.modules.ehcache.store.ClusteredStore.<init>(ClusteredStore.java:141)
      5. org.terracotta.modules.ehcache.store.EnterpriseClusteredStore.<init>(EnterpriseClusteredStore.java:105)
      6. org.terracotta.modules.ehcache.store.EnterpriseTerracottaClusteredInstanceFactory.newStore(EnterpriseTerracottaClusteredInstanceFactory.java:25)
      7. org.terracotta.modules.ehcache.store.TerracottaClusteredInstanceFactory.createStore(TerracottaClusteredInstanceFactory.java:92)
      7 frames
    7. Ehcache
      Cache$1.call
      1. net.sf.ehcache.terracotta.ClusteredInstanceFactoryWrapper.createStore(ClusteredInstanceFactoryWrapper.java:93)
      2. net.sf.ehcache.CacheManager.createTerracottaStore(CacheManager.java:623)
      3. net.sf.ehcache.Cache$1.call(Cache.java:1106)
      4. net.sf.ehcache.Cache$1.call(Cache.java:1103)
      4 frames
    8. org.terracotta.modules
      NonStopStoreWrapper$1.run
      1. org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.createStore(NonStopStoreWrapper.java:176)
      2. org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.doInit(NonStopStoreWrapper.java:188)
      3. org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.access$100(NonStopStoreWrapper.java:51)
      4. org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper$1.run(NonStopStoreWrapper.java:121)
      4 frames
    9. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:662)
      1 frame