hudson.util.IOException2

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • We have a sort of special CI environment where after projects build we execute them remotely and use hudson to monitor their progress. The remote execution of these programs take a while and at certain points no output is sent back to the master for long periods of time. During these long intervals where no output is sent back (just over 2 hours) I am occasionally seeing the job fail with the following: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:269) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:84) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request$1.get(Request.java:188) at hudson.remoting.Request$1.get(Request.java:157) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:261) ... 9 more Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request.abort(Request.java:223) at hudson.remoting.Channel.terminate(Channel.java:528) at hudson.remoting.Channel$ReaderThread.run(Channel.java:684) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:665) FATAL: Unable to delete script file /tmp/hudson24564.sh hudson.util.IOException2: remote file operation failed at hudson.FilePath.act(FilePath.java:544) at hudson.FilePath.delete(FilePath.java:741) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:94) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.FilePath.act(FilePath.java:541) ... 10 more FATAL: already closed java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:466) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:277) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) However, this is not predictable or reproducible which makes me think it corresponds to an external event such as GC, or even an network or OS event (eg TCP Error or Socket timeout). Anyway I thought I would put it up here and see if anyone else is getting this too. I am using Hudson ver. 1.293, The master and slave are both RHEL 4 An interesting development occurred when I upgraded recently and then set hudson.util.ProcessTreeKiller.disable=true. The jobs were still failing but the underlying process was eventually completing its job successfully (copying a large MySQL DB if you must know). This is the reason I reported this. This hints at a bug in hudson's remoting code. --Chad
    via by chad_lyon,
  • We have a sort of special CI environment where after projects build we execute them remotely and use hudson to monitor their progress. The remote execution of these programs take a while and at certain points no output is sent back to the master for long periods of time. During these long intervals where no output is sent back (just over 2 hours) I am occasionally seeing the job fail with the following: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:269) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:84) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request$1.get(Request.java:188) at hudson.remoting.Request$1.get(Request.java:157) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:261) ... 9 more Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request.abort(Request.java:223) at hudson.remoting.Channel.terminate(Channel.java:528) at hudson.remoting.Channel$ReaderThread.run(Channel.java:684) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:665) FATAL: Unable to delete script file /tmp/hudson24564.sh hudson.util.IOException2: remote file operation failed at hudson.FilePath.act(FilePath.java:544) at hudson.FilePath.delete(FilePath.java:741) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:94) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.FilePath.act(FilePath.java:541) ... 10 more FATAL: already closed java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:466) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:277) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) However, this is not predictable or reproducible which makes me think it corresponds to an external event such as GC, or even an network or OS event (eg TCP Error or Socket timeout). Anyway I thought I would put it up here and see if anyone else is getting this too. I am using Hudson ver. 1.293, The master and slave are both RHEL 4 An interesting development occurred when I upgraded recently and then set hudson.util.ProcessTreeKiller.disable=true. The jobs were still failing but the underlying process was eventually completing its job successfully (copying a large MySQL DB if you must know). This is the reason I reported this. This hints at a bug in hudson's remoting code. --Chad
    via by chad_lyon,
  • Unexpected termination of the channel
    via by Lynn Lin,
  • General Debug Questions?
    via by David Beck,
  • Jenkins users - Unexpected termination of the channel
    via by Unknown author,
  • Out hudson master installation runs successfully on a CentOS vmware instance and performs its very good jobs also on almost our configured slaves (AIX, SunOS, Win32, ..) We are facing a problem with a single slave running on a AMD64 machine running Linux OpenSuse 10.0. Almost every build the slave agent dies. Here follows the last failure: [exec] cd /home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/curl; CC="gcc" CFLAGS=" -DDSTK_64BIT -DLINUX -DL_ENDIAN -DUSE_SBUF -fPIC -DPTHREADS -DSMIME_ENV='"dstk.ini"' -DCRYPTOKI_ENV='"dstk.ini"' -DCSM_ENV='"dstk.ini"' -DMQPPS_ENV='"IDS_CONFIG_FILE"' -I/home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/output/Linux/64/gcc/include -DNDEBUG -DNO_ERR -DNO_BIO -DNO_LHASH -DNO_BUFFER -DUSE_SBUF -DVDADER_RULES -fPIC" ./configure --prefix=/home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/output/Linux/64/gcc --with-ssl=/home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/output/Linux/64/gcc --without-libidn FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:220) at hudson.tasks.Ant.perform(Ant.java:171) at hudson.model.Build$RunnerImpl.build(Build.java:156) at hudson.model.Build$RunnerImpl.doRun(Build.java:124) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:192) at hudson.model.Run.run(Run.java:656) at hudson.model.Build.run(Build.java:102) at hudson.model.ResourceController.execute(ResourceController.java:70) at hudson.model.Executor.run(Executor.java:65) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request$1.get(Request.java:152) at hudson.remoting.Request$1.get(Request.java:121) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:32) at hudson.Proc$RemoteProc.join(Proc.java:212) ... 8 more Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request.abort(Request.java:187) at hudson.remoting.Channel.terminate(Channel.java:437) at hudson.remoting.Channel$ReaderThread.run(Channel.java:593) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at hudson.remoting.Channel$ReaderThread.run(Channel.java:574) FATAL: null java.lang.NullPointerException at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:70) at hudson.model.Build$RunnerImpl.performAllBuildStep(Build.java:150) at hudson.model.Build$RunnerImpl.post2(Build.java:137) at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:254) at hudson.model.Run.run(Run.java:674) at hudson.model.Build.run(Build.java:102) at hudson.model.ResourceController.execute(ResourceController.java:70) at hudson.model.Executor.run(Executor.java:65) Having a look to the different build failures, we always see a regular pattern: Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at hudson.remoting.Channel$ReaderThread.run(Channel.java:574) Thanks in advance, Fulvio
    via by f_cavarretta,
  • Occasionally something happens (network failure, someone shuts down the machine, etc.) which makes a slave build fail. In this situation, currently Hudson gives an exception like this: hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:231) at hudson.tasks.Ant.perform(Ant.java:180) at hudson.model.Build$RunnerImpl.build(Build.java:138) at hudson.model.Build$RunnerImpl.doRun(Build.java:113) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:244) at hudson.model.Run.run(Run.java:842) at hudson.model.Build.run(Build.java:88) at hudson.model.ResourceController.execute(ResourceController.java:70) at hudson.model.Executor.run(Executor.java:90) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset at hudson.remoting.Request$1.get(Request.java:165) at hudson.remoting.Request$1.get(Request.java:134) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:32) at hudson.Proc$RemoteProc.join(Proc.java:223) ... 8 more This in itself is fine, but it then in turn marks the build as failed. This results in Hudson thinking the changes which occurred for this build were the cause for the build. This causes two problems: 1. When the build is re-run and then works, Hudson won't list the changes which occurred before the broken build as being changes which belong to the new build. 2. If you are running the CI Game plugin, the unlucky soul who committed the changes immediately before Hudson broke gets -10.0 points. IMO in this situation the points should be docked against Hudson itself, or some other equivalent placeholder player... On a larger scale, *any* build which fails due to Hudson itself should not be marked as failed for the same reason; this just happens to be the most common situation which we encounter in practice.
    via by trejkaz,
    • hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:269) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:84) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119)
    No Bugmate found.