java.util.concurrent.TimeoutException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • My only Jenkins slave dies every day almost at the same time giving below error : {code:java} Jul 29, 2016 3:02:42 AM hudson.slaves.ChannelPinger$1 onDead INFO: Ping failed. Terminating the channel channel. java.util.concurrent.TimeoutException: Ping started at 1469775521795 hasn't completed by 1469775762018 at hudson.remoting.PingThread.ping(PingThread.java:126) at hudson.remoting.PingThread.run(PingThread.java:85) Jul 29, 2016 3:02:46 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run SEVERE: I/O error in channel channel java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:203) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) Jul 29, 2016 3:02:47 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Jul 29, 2016 3:03:00 AM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect INFO: Restarting agent via jenkins.slaves.restarter.UnixSlaveRestarter@e374354 Jul 29, 2016 3:03:39 AM hudson.remoting.jnlp.Main createEngine INFO: Trying protocol: JNLP2-connect Jul 29, 2016 3:03:41 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Server didn't accept the handshake: ICI-Internal_1 is already connected to this master. Rejecting this connection. {code} AT the same time, Jenkins Server (master) exhibits following logs: {code:java} Jul 29, 2016 3:02:37 AM hudson.slaves.ChannelPinger$1 onDead INFO: Ping failed. Terminating the channel ICI-Internal_1. java.util.concurrent.TimeoutException: Ping started at 1469775517149 hasn't completed by 1469775757149 at hudson.remoting.PingThread.ping(PingThread.java:126) at hudson.remoting.PingThread.run(PingThread.java:85) Jul 29, 2016 3:03:40 AM hudson.TcpSlaveAgentListener$ConnectionHandler run INFO: Accepted connection #2 from /12.170.11.58:46412 Jul 29, 2016 3:03:41 AM org.jenkinsci.remoting.engine.JnlpServerHandshake error WARNING: TCP agent connection handler #2 with /12.170.11.58:46412 is aborted: ICI-Internal_1 is already connected to this master. Rejecting this connection. Jul 29, 2016 3:03:41 AM hudson.TcpSlaveAgentListener$ConnectionHandler run INFO: Accepted connection #3 from /12.170.11.58:46413 Jul 29, 2016 3:03:41 AM org.jenkinsci.remoting.engine.JnlpServerHandshake error WARNING: TCP agent connection handler #3 with /12.170.11.58:46413 is aborted: ICI-Internal_1 is already connected to this master. Rejecting this connection. {code} I only understood that Client/Server both are not able to connect to each other due to some reason and failed. However, it works for rest of the day when it is started manually after above error. There is no firewall/selinux running on any of the server. -Amit
    via by Amit Naudiyal,
  • Almost on daily basis my Jenkins is shutting down is taking ALL slaves offline. The reasons for this is unknown to me and looks like a severe bug. Can you please help to check this? Based on my observation I notice that connecting new slaves seems to fail with an SSL exception. ---- Sep 22, 2015 8:08:42 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source) at sun.nio.ch.SocketChannelImpl.write(Unknown Source) at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:293) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:402) at org.eclipse.jetty.io.nio.SslConnection.process(SslConnection.java:337) at org.eclipse.jetty.io.nio.SslConnection.access$900(SslConnection.java:48) at org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.flush(SslConnection.java:738) at org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.shutdownOutput(SslConnection.java:641) at org.eclipse.jetty.io.nio.SslConnection.onIdleExpired(SslConnection.java:260) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.onIdleExpired(SelectChannelEndPoint.java:349) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:326) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Sep 22, 2015 8:08:48 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: handle failed java.lang.IllegalStateException: Internal error at sun.security.ssl.SSLEngineImpl.initHandshaker(Unknown Source) at sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source) at sun.security.ssl.SSLEngineImpl.readNetRecord(Unknown Source) at sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source) at javax.net.ssl.SSLEngine.unwrap(Unknown Source) at org.eclipse.jetty.io.nio.SslConnection.unwrap(SslConnection.java:536) at org.eclipse.jetty.io.nio.SslConnection.process(SslConnection.java:401) at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:193) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ---- Shortly afterwards I can see that Jenkins is taking ALL slaves offline ---- Sep 22, 2015 8:20:54 AM hudson.slaves.ChannelPinger$1 onDead INFO: Ping failed. Terminating the channel SLAVE-101051. java.util.concurrent.TimeoutException: Ping started at 1442902614156 hasn't completed by 1442902854206 at hudson.remoting.PingThread.ping(PingThread.java:126) at hudson.remoting.PingThread.run(PingThread.java:85) ---- Afterwards ALL slaves want to register back to Jenkins but Jenkins is rejecting it with ---- INFO: Accepted connection #288 from /10.0.209.109:64213 Sep 22, 2015 8:47:00 AM jenkins.slaves.JnlpSlaveHandshake error WARNING: TCP slave agent connection handler #288 with /10.0.209.109:64213 is aborted: SLAVE-719161 is already connected to this master. Rejecting this connection. Sep 22, 2015 8:47:00 AM hudson.TcpSlaveAgentListener$ConnectionHandler run ---- If Jenkins kicks out all slaves, I would expect Jenkins to allow it get automatically accepted again instead of referring to already existing connection. But that all slaves are being taken offline at once due to PING FAIL is rather a bug. Please find full logs attached as well!
    via by Hans Baer,
  • Almost on daily basis my Jenkins is shutting down is taking ALL slaves offline. The reasons for this is unknown to me and looks like a severe bug. Can you please help to check this? Based on my observation I notice that connecting new slaves seems to fail with an SSL exception. ---- Sep 22, 2015 8:08:42 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(Unknown Source) at sun.nio.ch.SocketChannelImpl.write(Unknown Source) at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:293) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:402) at org.eclipse.jetty.io.nio.SslConnection.process(SslConnection.java:337) at org.eclipse.jetty.io.nio.SslConnection.access$900(SslConnection.java:48) at org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.flush(SslConnection.java:738) at org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.shutdownOutput(SslConnection.java:641) at org.eclipse.jetty.io.nio.SslConnection.onIdleExpired(SslConnection.java:260) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.onIdleExpired(SelectChannelEndPoint.java:349) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:326) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Sep 22, 2015 8:08:48 AM org.eclipse.jetty.util.log.JavaUtilLog warn WARNING: handle failed java.lang.IllegalStateException: Internal error at sun.security.ssl.SSLEngineImpl.initHandshaker(Unknown Source) at sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source) at sun.security.ssl.SSLEngineImpl.readNetRecord(Unknown Source) at sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source) at javax.net.ssl.SSLEngine.unwrap(Unknown Source) at org.eclipse.jetty.io.nio.SslConnection.unwrap(SslConnection.java:536) at org.eclipse.jetty.io.nio.SslConnection.process(SslConnection.java:401) at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:193) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) ---- Shortly afterwards I can see that Jenkins is taking ALL slaves offline ---- Sep 22, 2015 8:20:54 AM hudson.slaves.ChannelPinger$1 onDead INFO: Ping failed. Terminating the channel SLAVE-101051. java.util.concurrent.TimeoutException: Ping started at 1442902614156 hasn't completed by 1442902854206 at hudson.remoting.PingThread.ping(PingThread.java:126) at hudson.remoting.PingThread.run(PingThread.java:85) ---- Afterwards ALL slaves want to register back to Jenkins but Jenkins is rejecting it with ---- INFO: Accepted connection #288 from /10.0.209.109:64213 Sep 22, 2015 8:47:00 AM jenkins.slaves.JnlpSlaveHandshake error WARNING: TCP slave agent connection handler #288 with /10.0.209.109:64213 is aborted: SLAVE-719161 is already connected to this master. Rejecting this connection. Sep 22, 2015 8:47:00 AM hudson.TcpSlaveAgentListener$ConnectionHandler run ---- If Jenkins kicks out all slaves, I would expect Jenkins to allow it get automatically accepted again instead of referring to already existing connection. But that all slaves are being taken offline at once due to PING FAIL is rather a bug. Please find full logs attached as well!
    via by Hans Baer,
  • I have set up Jenkins slave Windows service so that it is restarted by system on failure. It works OK if slave fails for whatever reasons. Unfortunately recently master keeps failing due to out of memory problems and after master reboot some Windows slave do not reconnect. Service has to be restarted in order to connect slave again. {panel:title=jenkins-slave.err.log} Oct 02, 2012 3:59:10 PM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Hudson agent is running in headless mode. Oct 02, 2012 3:59:10 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [http://server:8080/] Oct 02, 2012 3:59:10 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to server:21234 Oct 02, 2012 3:59:10 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Oct 02, 2012 3:59:11 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Oct 03, 2012 8:11:21 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Oct 03, 2012 8:12:22 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [http://server:8080/] Oct 03, 2012 8:12:22 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to server:21234 Oct 03, 2012 8:12:22 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Oct 03, 2012 8:12:22 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Oct 03, 2012 8:15:19 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Oct 03, 2012 8:15:59 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [http://server:8080/] Oct 03, 2012 8:15:59 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to server:21234 Oct 03, 2012 8:15:59 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Oct 03, 2012 8:15:59 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Oct 03, 2012 9:44:59 PM hudson.slaves.ChannelPinger$1 onDead INFO: Ping failed. Terminating the channel. java.util.concurrent.TimeoutException: Ping started on 1349293259718 hasn't completed at 1349293499718 at hudson.remoting.PingThread.ping(PingThread.java:120) at hudson.remoting.PingThread.run(PingThread.java:81) Oct 03, 2012 9:44:59 PM hudson.remoting.SynchronousCommandTransport$ReaderThread run SEVERE: I/O error in channel channel java.net.SocketException: socket closed at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at hudson.remoting.Command.readFrom(Command.java:90) at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48) Oct 03, 2012 9:44:59 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Oct 04, 2012 4:20:07 PM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Hudson agent is running in headless mode. Oct 04, 2012 4:20:07 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [http://server:8080/] Oct 04, 2012 4:20:07 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to server:21234 Oct 04, 2012 4:20:07 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Oct 04, 2012 4:20:07 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected {panel} {panel:title=jenkins-slave.wrapper.log} 2012-10-02 15:59:10 - Starting C:\Program Files\Java\jre7\bin\java.exe -Xrs -jar "C:\Jenkins\slave.jar" -jnlpUrl http://server:8080/computer/slave-1/slave-agent.jnlp 2012-10-02 15:59:10 - Started 5680 2012-10-04 16:20:05 - Stopping jenkinsslave-C__Jenkins 2012-10-04 16:20:05 - ProcessKill 5680 2012-10-04 16:20:05 - Finished jenkinsslave-C__Jenkins 2012-10-04 16:20:06 - Starting C:\Program Files\Java\jre7\bin\java.exe -Xrs -jar "C:\Jenkins\slave.jar" -jnlpUrl http://server:8080/computer/slave-1/slave-agent.jnlp 2012-10-04 16:20:06 - Started 6464 {panel} Note that before exception slave itself reconnected many times to master, but after exception it did not recover until service was restarted.
    via by Krzysztof Malinowski,
  • Slave stops responding for whatever reason. First ping thread notices slave does not respond: Nov 26, 2014 2:00:59 PM INFO hudson.slaves.ChannelPinger$1 onDead Ping failed. Terminating the channel. java.util.concurrent.TimeoutException: Ping started on 1417006619758 hasn't completed at 1417006859758 at hudson.remoting.PingThread.ping(PingThread.java:120) at hudson.remoting.PingThread.run(PingThread.java:81) (would be nice if we could see which channel/slave) A bit later: Nov 26, 2014 2:14:34 PM WARNING hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor monitor Failed to monitor SV-ARG-DEV-D23 for Clock Difference hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:541) at hudson.remoting.Request.callAsync(Request.java:208) at hudson.remoting.Channel.callAsync(Channel.java:766) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:76) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:280) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:1027) at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110) at hudson.remoting.PingThread.ping(PingThread.java:120) at hudson.remoting.PingThread.run(PingThread.java:81) Caused by: java.util.concurrent.TimeoutException: Ping started on 1417006619758 hasn't completed at 1417006859758 ... 2 more Yet node is not shown as offline, and still shows as executor busy. Threaddump shows executor thread on master: Executor #0 for DWI01164 : executing OKA.R201501/Team D/_publics/Public - 4 #103 / waiting for hudson.remoting.Channel@3b91e986:DWI01164 "Executor #0 for DWI01164 : executing OKA.R201501/Team D/_publics/Public - 4 #103 / waiting for hudson.remoting.Channel@3b91e986:DWI01164" Id=16260 Group=main TIMED_WAITING on hudson.remoting.UserRequest@e4ca0c4 at java.lang.Object.wait(Native Method) - waiting on hudson.remoting.UserRequest@e4ca0c4 at hudson.remoting.Request.call(Request.java:146) at hudson.remoting.Channel.call(Channel.java:739) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:168) at com.sun.proxy.$Proxy56.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:956) at hudson.Launcher$ProcStarter.join(Launcher.java:367) at hudson.tasks.Maven.perform(Maven.java:328) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756) at hudson.model.Build$BuildExecution.build(Build.java:198) at hudson.model.Build$BuildExecution.doRun(Build.java:159) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1706) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:232) It looks like ChannelPinger calls channel.close() which sets channel.outClosed , yet Request is stuck in a loop checking for channel.isInClosed(), which was not set. Should Request check for channel.isClosingOrClosed()?
    via by Wannes Sels,
    • java.util.concurrent.TimeoutException: Ping started at 1469775521795 hasn't completed by 1469775762018 at hudson.remoting.PingThread.ping(PingThread.java:126) at hudson.remoting.PingThread.run(PingThread.java:85)
    No Bugmate found.