hudson.remoting.RequestAbortedException: java.io.EOFException

Jenkins JIRA | chad_lyon | 8 years ago
  1. 0

    We have a sort of special CI environment where after projects build we execute them remotely and use hudson to monitor their progress. The remote execution of these programs take a while and at certain points no output is sent back to the master for long periods of time. During these long intervals where no output is sent back (just over 2 hours) I am occasionally seeing the job fail with the following: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:269) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:84) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request$1.get(Request.java:188) at hudson.remoting.Request$1.get(Request.java:157) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:261) ... 9 more Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request.abort(Request.java:223) at hudson.remoting.Channel.terminate(Channel.java:528) at hudson.remoting.Channel$ReaderThread.run(Channel.java:684) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:665) FATAL: Unable to delete script file /tmp/hudson24564.sh hudson.util.IOException2: remote file operation failed at hudson.FilePath.act(FilePath.java:544) at hudson.FilePath.delete(FilePath.java:741) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:94) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.FilePath.act(FilePath.java:541) ... 10 more FATAL: already closed java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:466) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:277) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) However, this is not predictable or reproducible which makes me think it corresponds to an external event such as GC, or even an network or OS event (eg TCP Error or Socket timeout). Anyway I thought I would put it up here and see if anyone else is getting this too. I am using Hudson ver. 1.293, The master and slave are both RHEL 4 An interesting development occurred when I upgraded recently and then set hudson.util.ProcessTreeKiller.disable=true. The jobs were still failing but the underlying process was eventually completing its job successfully (copying a large MySQL DB if you must know). This is the reason I reported this. This hints at a bug in hudson's remoting code. --Chad

    Jenkins JIRA | 8 years ago | chad_lyon
    hudson.remoting.RequestAbortedException: java.io.EOFException
  2. 0

    We have a sort of special CI environment where after projects build we execute them remotely and use hudson to monitor their progress. The remote execution of these programs take a while and at certain points no output is sent back to the master for long periods of time. During these long intervals where no output is sent back (just over 2 hours) I am occasionally seeing the job fail with the following: FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:269) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:84) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request$1.get(Request.java:188) at hudson.remoting.Request$1.get(Request.java:157) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55) at hudson.Proc$RemoteProc.join(Proc.java:261) ... 9 more Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request.abort(Request.java:223) at hudson.remoting.Channel.terminate(Channel.java:528) at hudson.remoting.Channel$ReaderThread.run(Channel.java:684) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:665) FATAL: Unable to delete script file /tmp/hudson24564.sh hudson.util.IOException2: remote file operation failed at hudson.FilePath.act(FilePath.java:544) at hudson.FilePath.delete(FilePath.java:741) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:94) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58) at hudson.model.Build$RunnerImpl.build(Build.java:195) at hudson.model.Build$RunnerImpl.doRun(Build.java:151) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:272) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) Caused by: java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.FilePath.act(FilePath.java:541) ... 10 more FATAL: already closed java.io.IOException: already closed at hudson.remoting.Channel.send(Channel.java:342) at hudson.remoting.Request.call(Request.java:104) at hudson.remoting.Channel.call(Channel.java:481) at hudson.Launcher$RemoteLauncher.kill(Launcher.java:466) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:277) at hudson.model.Run.run(Run.java:895) at hudson.model.Build.run(Build.java:112) at hudson.model.ResourceController.execute(ResourceController.java:93) at hudson.model.Executor.run(Executor.java:119) However, this is not predictable or reproducible which makes me think it corresponds to an external event such as GC, or even an network or OS event (eg TCP Error or Socket timeout). Anyway I thought I would put it up here and see if anyone else is getting this too. I am using Hudson ver. 1.293, The master and slave are both RHEL 4 An interesting development occurred when I upgraded recently and then set hudson.util.ProcessTreeKiller.disable=true. The jobs were still failing but the underlying process was eventually completing its job successfully (copying a large MySQL DB if you must know). This is the reason I reported this. This hints at a bug in hudson's remoting code. --Chad

    Jenkins JIRA | 8 years ago | chad_lyon
    hudson.remoting.RequestAbortedException: java.io.EOFException
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    Failed to join process v1.327/1.328 - crbeng - net.java.dev.hudson.users - MarkMail

    markmail.org | 1 year ago
    hudson.remoting.RequestAbortedException: java.io.EOFException
  5. 0

    Out hudson master installation runs successfully on a CentOS vmware instance and performs its very good jobs also on almost our configured slaves (AIX, SunOS, Win32, ..) We are facing a problem with a single slave running on a AMD64 machine running Linux OpenSuse 10.0. Almost every build the slave agent dies. Here follows the last failure: [exec] cd /home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/curl; CC="gcc" CFLAGS=" -DDSTK_64BIT -DLINUX -DL_ENDIAN -DUSE_SBUF -fPIC -DPTHREADS -DSMIME_ENV='"dstk.ini"' -DCRYPTOKI_ENV='"dstk.ini"' -DCSM_ENV='"dstk.ini"' -DMQPPS_ENV='"IDS_CONFIG_FILE"' -I/home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/output/Linux/64/gcc/include -DNDEBUG -DNO_ERR -DNO_BIO -DNO_LHASH -DNO_BUFFER -DUSE_SBUF -DVDADER_RULES -fPIC" ./configure --prefix=/home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/output/Linux/64/gcc --with-ssl=/home/dstk110/build/hudson/workspace/DSTK-Linux64/dstk/AddOn/output/Linux/64/gcc --without-libidn FATAL: command execution failed hudson.util.IOException2: Failed to join the process at hudson.Proc$RemoteProc.join(Proc.java:220) at hudson.tasks.Ant.perform(Ant.java:171) at hudson.model.Build$RunnerImpl.build(Build.java:156) at hudson.model.Build$RunnerImpl.doRun(Build.java:124) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:192) at hudson.model.Run.run(Run.java:656) at hudson.model.Build.run(Build.java:102) at hudson.model.ResourceController.execute(ResourceController.java:70) at hudson.model.Executor.run(Executor.java:65) Caused by: java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request$1.get(Request.java:152) at hudson.remoting.Request$1.get(Request.java:121) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:32) at hudson.Proc$RemoteProc.join(Proc.java:212) ... 8 more Caused by: hudson.remoting.RequestAbortedException: java.io.EOFException at hudson.remoting.Request.abort(Request.java:187) at hudson.remoting.Channel.terminate(Channel.java:437) at hudson.remoting.Channel$ReaderThread.run(Channel.java:593) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at hudson.remoting.Channel$ReaderThread.run(Channel.java:574) FATAL: null java.lang.NullPointerException at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:70) at hudson.model.Build$RunnerImpl.performAllBuildStep(Build.java:150) at hudson.model.Build$RunnerImpl.post2(Build.java:137) at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:254) at hudson.model.Run.run(Run.java:674) at hudson.model.Build.run(Build.java:102) at hudson.model.ResourceController.execute(ResourceController.java:70) at hudson.model.Executor.run(Executor.java:65) Having a look to the different build failures, we always see a regular pattern: Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2498) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at hudson.remoting.Channel$ReaderThread.run(Channel.java:574) Thanks in advance, Fulvio

    Jenkins JIRA | 9 years ago | f_cavarretta
    hudson.remoting.RequestAbortedException: java.io.EOFException

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. hudson.remoting.RequestAbortedException

      java.io.EOFException

      at hudson.remoting.Request.abort()
    2. Hudson :: Remoting Layer
      Channel$ReaderThread.run
      1. hudson.remoting.Request.abort(Request.java:223)
      2. hudson.remoting.Channel.terminate(Channel.java:528)
      3. hudson.remoting.Channel$ReaderThread.run(Channel.java:684)
      3 frames