net.sf.ehcache.distribution.RemoteCacheException: Error bootstrapping from remote peer. Message was: error unmarshalling return; nested exception is: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: com.atlassian.jira.i18n.BackingI18n

Atlassian JIRA | Trevor Campbell [Atlassian] | 2 years ago
  1. 0

    This generates stack trace such as: {noformat} 2014-08-01 07:15:53,459 main WARN [net.sf.ehcache.CacheManager] Cache com.atlassian.jira.task.TaskManagerImpl.taskMaprequested bootstrap but a CacheException occured. Error bootstrapping from remote peer. Message was: error unmarshalling return; nested exception is: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: com.atlassian.jira.i18n.BackingI18n net.sf.ehcache.distribution.RemoteCacheException: Error bootstrapping from remote peer. Message was: error unmarshalling return; nested exception is: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: com.atlassian.jira.i18n.BackingI18n at net.sf.ehcache.distribution.RMIBootstrapCacheLoader.doLoad(RMIBootstrapCacheLoader.java:176) at net.sf.ehcache.distribution.RMIBootstrapCacheLoader.load(RMIBootstrapCacheLoader.java:77) at net.sf.ehcache.Cache.bootstrap(Cache.java:1364) at net.sf.ehcache.CacheManager.initializeEhcache(CacheManager.java:1388) at net.sf.ehcache.CacheManager.addCacheNoCheck(CacheManager.java:1437) at net.sf.ehcache.CacheManager.addConfiguredCaches(CacheManager.java:839) at net.sf.ehcache.CacheManager.doInit(CacheManager.java:498) at net.sf.ehcache.CacheManager.init(CacheManager.java:392) at net.sf.ehcache.CacheManager.<init>(CacheManager.java:265) at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1123) at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1099) at com.atlassian.jira.CacheManagerRegistrar.getCacheManagerDelegate(CacheManagerRegistrar.java:130) {noformat}

    Atlassian JIRA | 2 years ago | Trevor Campbell [Atlassian]
    net.sf.ehcache.distribution.RemoteCacheException: Error bootstrapping from remote peer. Message was: error unmarshalling return; nested exception is: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: com.atlassian.jira.i18n.BackingI18n
  2. 0

    When CAS sends the PGT back to a clustered client, that cluster must perform synchronous replication, as the client will retrieve the PGT from the storage within milliseconds. However, when one of the nodes (configured for manual peer discovery because multicast does not work at that site) goes down, Ehcache throws the following exception: SEVERE: Servlet.service() for servlet default threw exception net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification(RMISynchronousCacheReplicator.java:136) at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.notifyElementPut(RMISynchronousCacheReplicator.java:119) at net.sf.ehcache.event.RegisteredEventListeners.notifyElementPut(RegisteredEventListeners.java:102) at net.sf.ehcache.Cache.put(Cache.java:699) at net.sf.ehcache.Cache.put(Cache.java:632) at org.jasig.cas.client.proxy.EhcacheBackedProxyGrantingTicketStorageImpl.save(EhcacheBackedProxyGrantingTicketStorageImpl.java:34) at org.jasig.cas.client.util.CommonUtils.readAndRespondToProxyReceptorRequest(CommonUtils.java:184) at org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter.preFilter(Cas20ProxyReceivingTicketValidationFilter.java:160) at org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:150) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) This exception propagates all the way to Tomcat (it's re-thrown in 3.1.12), which causes the application server to report HTTP result code 500. When CAS receives that code, it does not include the PGTIOU in the /serviceValidate response even though all the live client nodes have received the PGT. I think it should be OK for EhcacheBackedProxyGrantingTicketStorageImpl to catch the RemoteCacheException when replicating synchronously, log it, and NOT propagate it to allow CAS to operate normally. After all, one of the reasons to have a cluster is to be able to take down a node and continue operating the cluster.

    Apereo Issues | 6 years ago | Adam Rybicki
    net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    When CAS sends the PGT back to a clustered client, that cluster must perform synchronous replication, as the client will retrieve the PGT from the storage within milliseconds. However, when one of the nodes (configured for manual peer discovery because multicast does not work at that site) goes down, Ehcache throws the following exception: SEVERE: Servlet.service() for servlet default threw exception net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification(RMISynchronousCacheReplicator.java:136) at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.notifyElementPut(RMISynchronousCacheReplicator.java:119) at net.sf.ehcache.event.RegisteredEventListeners.notifyElementPut(RegisteredEventListeners.java:102) at net.sf.ehcache.Cache.put(Cache.java:699) at net.sf.ehcache.Cache.put(Cache.java:632) at org.jasig.cas.client.proxy.EhcacheBackedProxyGrantingTicketStorageImpl.save(EhcacheBackedProxyGrantingTicketStorageImpl.java:34) at org.jasig.cas.client.util.CommonUtils.readAndRespondToProxyReceptorRequest(CommonUtils.java:184) at org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter.preFilter(Cas20ProxyReceivingTicketValidationFilter.java:160) at org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:150) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) This exception propagates all the way to Tomcat (it's re-thrown in 3.1.12), which causes the application server to report HTTP result code 500. When CAS receives that code, it does not include the PGTIOU in the /serviceValidate response even though all the live client nodes have received the PGT. I think it should be OK for EhcacheBackedProxyGrantingTicketStorageImpl to catch the RemoteCacheException when replicating synchronously, log it, and NOT propagate it to allow CAS to operate normally. After all, one of the reasons to have a cluster is to be able to take down a node and continue operating the cluster.

    Apereo Issues | 6 years ago | Adam Rybicki
    net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused

    Root Cause Analysis

    1. net.sf.ehcache.distribution.RemoteCacheException

      Error bootstrapping from remote peer. Message was: error unmarshalling return; nested exception is: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: com.atlassian.jira.i18n.BackingI18n

      at net.sf.ehcache.distribution.RMIBootstrapCacheLoader.doLoad()
    2. Ehcache
      CacheManager.newInstance
      1. net.sf.ehcache.distribution.RMIBootstrapCacheLoader.doLoad(RMIBootstrapCacheLoader.java:176)
      2. net.sf.ehcache.distribution.RMIBootstrapCacheLoader.load(RMIBootstrapCacheLoader.java:77)
      3. net.sf.ehcache.Cache.bootstrap(Cache.java:1364)
      4. net.sf.ehcache.CacheManager.initializeEhcache(CacheManager.java:1388)
      5. net.sf.ehcache.CacheManager.addCacheNoCheck(CacheManager.java:1437)
      6. net.sf.ehcache.CacheManager.addConfiguredCaches(CacheManager.java:839)
      7. net.sf.ehcache.CacheManager.doInit(CacheManager.java:498)
      8. net.sf.ehcache.CacheManager.init(CacheManager.java:392)
      9. net.sf.ehcache.CacheManager.<init>(CacheManager.java:265)
      10. net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1123)
      11. net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1099)
      11 frames
    3. com.atlassian.jira
      CacheManagerRegistrar.getCacheManagerDelegate
      1. com.atlassian.jira.CacheManagerRegistrar.getCacheManagerDelegate(CacheManagerRegistrar.java:130)
      1 frame