hudson.remoting.ChannelClosedException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • When JNLP slave detect ping timeout, it tries to reconnect. But if master have not noticed the timeout yet, it rejects the new connection from slave. JNLP slave agent process aborts once the connection is rejected in such a way. STDOUT of JNLP process: {noformat} INFO: Ping failed. Terminating the channel. java.util.concurrent.TimeoutException: Ping started on 1456918109582 hasn't completed at 1456918349582 at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) Mar 02, 2016 6:32:29 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run SEVERE: I/O error in channel channel java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:190) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82) at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:72) at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103) at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48) Mar 02, 2016 6:32:29 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Mar 02, 2016 6:32:39 AM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect INFO: Restarting slave via jenkins.slaves.restarter.UnixSlaveRestarter@6523ff4a Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: dev127-virt2 Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Jenkins agent is running in headless mode. Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [http://jenkins.acme.com/hudson/, http://hudson.acme.com/hudson/] Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to jenkins.acme.com:37003 Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener error SEVERE: The server rejected the connection: dev127-virt2 is already connected to this master. Rejecting this connection. java.lang.Exception: The server rejected the connection: dev127-virt2 is already connected to this master. Rejecting this connection. at hudson.remoting.Engine.onConnectionRejected(Engine.java:306) at hudson.remoting.Engine.run(Engine.java:276) {noformat} Slave log on master: {noformat} JNLP agent connected from /10.16.180.145 <===[JENKINS REMOTING CAPACITY]===>ERROR: Connection terminated Connection terminated ha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@131dbee3[name=dev127-virt2] at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:211) at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:631) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcher.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69) at sun.nio.ch.IOUtil.write(IOUtil.java:40) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:336) at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.send(FifoBuffer.java:130) at org.jenkinsci.remoting.nio.FifoBuffer.send(FifoBuffer.java:254) at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:622) ... 7 more Slave.jar version: 2.47 This is a Unix slave Slave successfully connected and online Connection terminated {noformat} Master log: {noformat} 2016-03-02 06:32:38,352 WARNING [hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor] (Monitoring thread for Clock Difference started on Wed Mar 02 06:32:08 EST 2016) Failed to monitor dev127-virt2 for Clock Difference java.util.concurrent.TimeoutException at hudson.remoting.Request$1.get(Request.java:271) at hudson.remoting.Request$1.get(Request.java:206) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:97) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:280) ... (All monitors times out) 2016-03-02 06:32:42,860 INFO [hudson.TcpSlaveAgentListener] (TCP slave agent connection handler #41773 with /10.16.180.145:58248) Accepted connection #41773 from /10.16.180.145:58248 2016-03-02 06:32:42,865 WARNING [jenkins.slaves.JnlpSlaveHandshake] (TCP slave agent connection handler #41773 with /10.16.180.145:58248) TCP slave agent connection handler #41773 with /10.16.180.145:58248 is aborted: dev127-virt2 is already connected to this master. Rejecting this connection. 2016-03-02 06:32:42,866 WARNING [jenkins.slaves.JnlpSlaveHandshake] (TCP slave agent connection handler #41773 with /10.16.180.145:58248) TCP slave agent connection handler #41773 with /10.16.180.145:58248 is aborted: Unrecognized name: dev127-virt2 ... 2016-03-02 06:33:43,630 INFO [hudson.slaves.ChannelPinger] (Ping thread for channel hudson.remoting.Channel@5caca20e:dev127-virt2) Ping failed. Terminating the channel. java.util.concurrent.TimeoutException: Ping started on 1456918183629 hasn't completed at 1456918423630 at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) ... 2016-03-02 06:38:26,902 WARNING [hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor] (Monitoring thread for Free Temp Space started on Wed Mar 02 06:38:26 EST 2016) Failed to monitor dev127-virt2 for Free Temp Space hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:549) at hudson.remoting.Request.callAsync(Request.java:204) at hudson.remoting.Channel.callAsync(Channel.java:778) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:76) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:280) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:1105) at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110) at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) Caused by: java.util.concurrent.TimeoutException: Ping started on 1456918183629 hasn't completed at 1456918423630 ... 2 more ... (All monitors fails with "channel is already closed") {noformat}
    via by Oliver Gondža,
  • When JNLP slave detect ping timeout, it tries to reconnect. But if master have not noticed the timeout yet, it rejects the new connection from slave. JNLP slave agent process aborts once the connection is rejected in such a way. STDOUT of JNLP process: {noformat} INFO: Ping failed. Terminating the channel. java.util.concurrent.TimeoutException: Ping started on 1456918109582 hasn't completed at 1456918349582 at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) Mar 02, 2016 6:32:29 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run SEVERE: I/O error in channel channel java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:190) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read(BufferedInputStream.java:254) at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82) at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:72) at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103) at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48) Mar 02, 2016 6:32:29 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Mar 02, 2016 6:32:39 AM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$2$1 onReconnect INFO: Restarting slave via jenkins.slaves.restarter.UnixSlaveRestarter@6523ff4a Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main createEngine INFO: Setting up slave: dev127-virt2 Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener <init> INFO: Jenkins agent is running in headless mode. Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among [http://jenkins.acme.com/hudson/, http://hudson.acme.com/hudson/] Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to jenkins.acme.com:37003 Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Mar 02, 2016 6:32:42 AM hudson.remoting.jnlp.Main$CuiListener error SEVERE: The server rejected the connection: dev127-virt2 is already connected to this master. Rejecting this connection. java.lang.Exception: The server rejected the connection: dev127-virt2 is already connected to this master. Rejecting this connection. at hudson.remoting.Engine.onConnectionRejected(Engine.java:306) at hudson.remoting.Engine.run(Engine.java:276) {noformat} Slave log on master: {noformat} JNLP agent connected from /10.16.180.145 <===[JENKINS REMOTING CAPACITY]===>ERROR: Connection terminated Connection terminated ha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@131dbee3[name=dev127-virt2] at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:211) at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:631) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcher.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69) at sun.nio.ch.IOUtil.write(IOUtil.java:40) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:336) at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.send(FifoBuffer.java:130) at org.jenkinsci.remoting.nio.FifoBuffer.send(FifoBuffer.java:254) at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:622) ... 7 more Slave.jar version: 2.47 This is a Unix slave Slave successfully connected and online Connection terminated {noformat} Master log: {noformat} 2016-03-02 06:32:38,352 WARNING [hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor] (Monitoring thread for Clock Difference started on Wed Mar 02 06:32:08 EST 2016) Failed to monitor dev127-virt2 for Clock Difference java.util.concurrent.TimeoutException at hudson.remoting.Request$1.get(Request.java:271) at hudson.remoting.Request$1.get(Request.java:206) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:97) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:280) ... (All monitors times out) 2016-03-02 06:32:42,860 INFO [hudson.TcpSlaveAgentListener] (TCP slave agent connection handler #41773 with /10.16.180.145:58248) Accepted connection #41773 from /10.16.180.145:58248 2016-03-02 06:32:42,865 WARNING [jenkins.slaves.JnlpSlaveHandshake] (TCP slave agent connection handler #41773 with /10.16.180.145:58248) TCP slave agent connection handler #41773 with /10.16.180.145:58248 is aborted: dev127-virt2 is already connected to this master. Rejecting this connection. 2016-03-02 06:32:42,866 WARNING [jenkins.slaves.JnlpSlaveHandshake] (TCP slave agent connection handler #41773 with /10.16.180.145:58248) TCP slave agent connection handler #41773 with /10.16.180.145:58248 is aborted: Unrecognized name: dev127-virt2 ... 2016-03-02 06:33:43,630 INFO [hudson.slaves.ChannelPinger] (Ping thread for channel hudson.remoting.Channel@5caca20e:dev127-virt2) Ping failed. Terminating the channel. java.util.concurrent.TimeoutException: Ping started on 1456918183629 hasn't completed at 1456918423630 at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) ... 2016-03-02 06:38:26,902 WARNING [hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor] (Monitoring thread for Free Temp Space started on Wed Mar 02 06:38:26 EST 2016) Failed to monitor dev127-virt2 for Free Temp Space hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:549) at hudson.remoting.Request.callAsync(Request.java:204) at hudson.remoting.Channel.callAsync(Channel.java:778) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:76) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:280) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:1105) at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110) at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) Caused by: java.util.concurrent.TimeoutException: Ping started on 1456918183629 hasn't completed at 1456918423630 ... 2 more ... (All monitors fails with "channel is already closed") {noformat}
    via by Oliver Gondža,
  • Calling threadDump can result into empty page due to Channel is already closed exception. {noformat} 2012-12-20 07:25:26,698 WARNING [hudson.ExpressionFactory2$JexlExpression] (Handling GET /hudson/threadDump : ajp-127.0.0.1-8009-110) Caught exception evaluating: it.getAllThreadDumps().entrySet(). Reason: hudson.remoting.ChannelClosedException: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:499) at hudson.remoting.Request.callAsync(Request.java:182) at hudson.remoting.Channel.callAsync(Channel.java:708) at hudson.util.RemotingDiagnostics.getThreadDumpAsync(RemotingDiagnostics.java:86) at jenkins.model.Jenkins.getAllThreadDumps(Jenkins.java:2647) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258) at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104) at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83) at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57) at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51) at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80) at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:72) at org.apache.commons.jelly.expression.ExpressionSupport.evaluateRecurse(ExpressionSupport.java:61) at org.apache.commons.jelly.expression.ExpressionSupport.evaluateAsIterator(ExpressionSupport.java:94) at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:89) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:270) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:571) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:656) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:485) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.jenkinsci.plugins.reverse_proxy_auth.ReverseProxySecurityRealm$1.doFilter(ReverseProxySecurityRealm.java:142) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:183) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:442) at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:95) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.internalProcess(ActiveRequestResponseCacheValve.java:74) at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.invoke(ActiveRequestResponseCacheValve.java:47) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436) at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:385) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:451) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:890) at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:111) at hudson.remoting.PingThread.ping(PingThread.java:114) at hudson.remoting.PingThread.run(PingThread.java:81) Caused by: java.util.concurrent.TimeoutException: Ping started on 1355977288862 hasn't completed at 1355977528899 ... 2 more Caused by: java.util.concurrent.TimeoutException at hudson.remoting.Request$1.get(Request.java:249) at hudson.remoting.Request$1.get(Request.java:184) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) at hudson.remoting.PingThread.ping(PingThread.java:107) ... 1 more {noformat}
    via by vjuranek,
  • I observe that jobs starting at the same time the slave is performing some sort of cleanup action hang: Build Log: {noformat} 17:04:21 Started by upstream project "pipeline-test-2" build number 422 17:04:21 originally caused by: 17:04:21 Started by upstream project "master-test" build number 867 17:04:21 originally caused by: 17:04:21 Started by upstream project "master-build" build number 5131 17:04:21 originally caused by: 17:04:21 Started by an SCM change 17:04:21 [EnvInject] - Loading node environment variables. ... hang for 14h ... {noformat} Slave Log: {noformat} ... lots of output ... INFO: Deleting /tmp/1408400946520-0/jboss51x_centos5-x64-bld-slave-09.corp.XXX.com_10002/lib/jboss-j2se.jar (atime=1408400947, diff=-5424) Aug 25, 2014 5:01:12 PM hudson.plugins.tmpcleaner.TmpCleanTask visit INFO: Deleting /tmp/1408400946520-0/jboss51x_centos5-x64-bld-slave-09.corp.XXX.com_10002/lib/javassist.jar (atime=1408400947, diff=-5424) Aug 25, 2014 5:01:12 PM hudson.plugins.tmpcleaner.TmpCleanTask visit INFO: Deleting /tmp/1408400946520-0/jboss51x_centos5-x64-bld-slave-09.corp.XXX.com_10002/lib/jboss-system-jmx.jar (atime=1408400947, diff=-5424) Aug 25, 2014 5:01:12 PM hudson.plugins.tmpcleaner.TmpCleanTask visit INFO: Deleting /tmp/1408400946520-0/jboss51x_centos5-x64-bld-slave-09.corp.XXX.com_10002/lib/jboss-system.jar (atime=1408400947, diff=-5424) Aug 25, 2014 5:01:12 PM hudson.plugins.tmpcleaner.TmpCleanTask visit INFO: Deleting /tmp/1408400946520-0/jboss51x_centos5-x64-bld-slave-09.corp.XXX.com_10002/lib/jboss-mdr.jar (atime=1408400947, diff=-5424) Aug 25, 2014 5:01:12 PM hudson.plugins.tmpcleaner.TmpCleanTask visit INFO: Deleting /tmp/1408400946520-0/jboss51x_centos5-x64-bld-slave-09.corp.XXX.com_10002/lib/jboss-logging-spi.jar (atime=1408400947, diff=-5424) Aug 25, 2014 5:01:12 PM hudson.plugins.tmpcleaner.TmpCleanTask visit {noformat} After interrupting the job, I get this: {noformat} 0:30:26 ERROR: SEVERE ERROR occurs 10:30:26 org.jenkinsci.lib.envinject.EnvInjectException: java.lang.InterruptedException 10:30:26 at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvironmentVariablesNode(EnvironmentVariablesNodeLoader.java:77) 10:30:26 at org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:81) 10:30:26 at org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:39) 10:30:26 at hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:575) 10:30:26 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:481) 10:30:26 at hudson.model.Run.execute(Run.java:1689) 10:30:26 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 10:30:26 at hudson.model.ResourceController.execute(ResourceController.java:88) 10:30:26 at hudson.model.Executor.run(Executor.java:231) 10:30:26 Caused by: java.lang.InterruptedException 10:30:26 at java.lang.Object.wait(Native Method) 10:30:26 at hudson.remoting.Request.call(Request.java:146) 10:30:26 at hudson.remoting.Channel.call(Channel.java:722) 10:30:26 at hudson.FilePath.act(FilePath.java:1003) 10:30:26 at org.jenkinsci.plugins.envinject.service.EnvironmentVariablesNodeLoader.gatherEnvironmentVariablesNode(EnvironmentVariablesNodeLoader.java:44) 10:30:26 ... 8 more 10:30:26 Archiving artifacts 10:30:26 ERROR: Publisher hudson.tasks.Mailer aborted due to exception 10:30:26 hudson.remoting.ChannelClosedException: channel is already closed 10:30:26 at hudson.remoting.Channel.send(Channel.java:524) 10:30:26 at hudson.remoting.Request.call(Request.java:129) 10:30:26 at hudson.remoting.Channel.call(Channel.java:722) 10:30:26 at hudson.EnvVars.getRemote(EnvVars.java:404) 10:30:26 at hudson.model.Computer.getEnvironment(Computer.java:911) 10:30:26 at jenkins.model.CoreEnvironmentContributor.buildEnvironmentFor(CoreEnvironmentContributor.java:29) 10:30:26 at hudson.model.Run.getEnvironment(Run.java:2202) 10:30:26 at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:873) 10:30:26 at hudson.tasks.Mailer.perform(Mailer.java:134) 10:30:26 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) 10:30:26 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) 10:30:26 at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:714) 10:30:26 at hudson.model.Build$BuildExecution.post2(Build.java:182) 10:30:26 at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:663) 10:30:26 at hudson.model.Run.execute(Run.java:1714) 10:30:26 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) 10:30:26 at hudson.model.ResourceController.execute(ResourceController.java:88) 10:30:26 at hudson.model.Executor.run(Executor.java:231) 10:30:26 Caused by: java.io.IOException 10:30:26 at hudson.remoting.Channel.close(Channel.java:1007) 10:30:26 at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110) 10:30:26 at hudson.remoting.PingThread.ping(PingThread.java:120) 10:30:26 at hudson.remoting.PingThread.run(PingThread.java:81) 10:30:26 Caused by: java.util.concurrent.TimeoutException: Ping started on 1409011309670 hasn't completed at 1409011549671 10:30:26 ... 2 more 10:30:26 [BFA] Scanning build for known causes... 10:30:26 10:30:26 [BFA] Done. 0s 10:30:26 [EnvInject] - [ERROR] - SEVERE ERROR occurs: channel is already closed {noformat}
    via by Christian Goetze,
  • The Jenkins server version is 1.480.1 The Jenkins master doesn't belong to the same timezone as the slave (The slave is 8 Hours behind) Slave Launch method: Launch slave agents on Unix machines via SSH A job run on this slave immediately failed. The corresponding output is: Started by user Dominique Ledit Building remotely on vm-hou-bldrh53cFATAL: channel is already closed hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:493) at hudson.remoting.Request.call(Request.java:129) at hudson.remoting.Channel.call(Channel.java:664) at hudson.EnvVars.getRemote(EnvVars.java:202) at hudson.model.Computer.getEnvironment(Computer.java:844) at jenkins.model.CoreEnvironmentContributor.buildEnvironmentFor(CoreEnvironmentContributor.java:28) at hudson.model.Run.getEnvironment(Run.java:1952) at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:843) at hudson.model.AbstractBuild$AbstractBuildExecution.decideWorkspace(AbstractBuild.java:462) at hudson.model.AbstractBuild$AbstractBuildExecution.run (AbstractBuild.java:482) at hudson.model.Run.execute(Run.java:1502) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:902) at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110) at hudson.remoting.PingThread.ping(PingThread.java:120) at hudson.remoting.PingThread.run(PingThread.java:81) Caused by: java.util.concurrent.TimeoutException: Ping started on 1361148569659 hasn't completed at 1361148809659 ... 2 more
    via by Dominique Ledit,
    • hudson.remoting.ChannelClosedException: channel is already closed at hudson.remoting.Channel.send(Channel.java:549) at hudson.remoting.Request.callAsync(Request.java:204) at hudson.remoting.Channel.callAsync(Channel.java:778) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:76) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:280) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:1105) at hudson.slaves.ChannelPinger$1.onDead(ChannelPinger.java:110) at hudson.remoting.PingThread.ping(PingThread.java:125) at hudson.remoting.PingThread.run(PingThread.java:86) Caused by: java.util.concurrent.TimeoutException: Ping started on 1456918183629 hasn't completed at 1456918423630 ... 2 more

    Users with the same issue

    Unknown visitor1 times, last one,