net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused

Apereo Issues | Adam Rybicki | 6 years ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    When CAS sends the PGT back to a clustered client, that cluster must perform synchronous replication, as the client will retrieve the PGT from the storage within milliseconds. However, when one of the nodes (configured for manual peer discovery because multicast does not work at that site) goes down, Ehcache throws the following exception: SEVERE: Servlet.service() for servlet default threw exception net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification(RMISynchronousCacheReplicator.java:136) at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.notifyElementPut(RMISynchronousCacheReplicator.java:119) at net.sf.ehcache.event.RegisteredEventListeners.notifyElementPut(RegisteredEventListeners.java:102) at net.sf.ehcache.Cache.put(Cache.java:699) at net.sf.ehcache.Cache.put(Cache.java:632) at org.jasig.cas.client.proxy.EhcacheBackedProxyGrantingTicketStorageImpl.save(EhcacheBackedProxyGrantingTicketStorageImpl.java:34) at org.jasig.cas.client.util.CommonUtils.readAndRespondToProxyReceptorRequest(CommonUtils.java:184) at org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter.preFilter(Cas20ProxyReceivingTicketValidationFilter.java:160) at org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:150) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) This exception propagates all the way to Tomcat (it's re-thrown in 3.1.12), which causes the application server to report HTTP result code 500. When CAS receives that code, it does not include the PGTIOU in the /serviceValidate response even though all the live client nodes have received the PGT. I think it should be OK for EhcacheBackedProxyGrantingTicketStorageImpl to catch the RemoteCacheException when replicating synchronously, log it, and NOT propagate it to allow CAS to operate normally. After all, one of the reasons to have a cluster is to be able to take down a node and continue operating the cluster.

    Apereo Issues | 6 years ago | Adam Rybicki
    net.sf.ehcache.distribution.RemoteCacheException: Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused

    Root Cause Analysis

    1. net.sf.ehcache.distribution.RemoteCacheException

      Error doing put to remote peer. Message was: Connection refused to host: AAA.BBB.CCC.DDD; nested exception is: java.net.ConnectException: Connection refused

      at net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification()
    2. Ehcache
      Cache.put
      1. net.sf.ehcache.distribution.RMISynchronousCacheReplicator.replicatePutNotification(RMISynchronousCacheReplicator.java:136)
      2. net.sf.ehcache.distribution.RMISynchronousCacheReplicator.notifyElementPut(RMISynchronousCacheReplicator.java:119)
      3. net.sf.ehcache.event.RegisteredEventListeners.notifyElementPut(RegisteredEventListeners.java:102)
      4. net.sf.ehcache.Cache.put(Cache.java:699)
      5. net.sf.ehcache.Cache.put(Cache.java:632)
      5 frames
    3. Jasig CAS Client for Java - Core
      AbstractTicketValidationFilter.doFilter
      1. org.jasig.cas.client.proxy.EhcacheBackedProxyGrantingTicketStorageImpl.save(EhcacheBackedProxyGrantingTicketStorageImpl.java:34)
      2. org.jasig.cas.client.util.CommonUtils.readAndRespondToProxyReceptorRequest(CommonUtils.java:184)
      3. org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter.preFilter(Cas20ProxyReceivingTicketValidationFilter.java:160)
      4. org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:150)
      4 frames
    4. Glassfish Core
      CoyoteAdapter.service
      1. org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      2. org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      3. org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      4. org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      5. org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
      6. org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      7. org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
      8. org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      9. org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
      9 frames
    5. Grizzly HTTP
      JIoEndpoint$Worker.run
      1. org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
      2. org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
      3. org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
      3 frames
    6. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:619)
      1 frame