java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown

jenkins-ci.org | 4 months ago
  1. 0

    [JIRA] (JENKINS-37293) ECS Agents are not deleted because Jenkins is unable to delete logs due to NFS locks

    Google Groups | 4 months ago | ev...@borgstrom.ca (JIRA)
    java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
  2. 0

    [JENKINS-37293] ECS Agents are not deleted because Jenkins is unable to delete logs due to NFS locks - Jenkins JIRA

    jenkins-ci.org | 4 months ago
    java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
  3. 0

    We have our Jenkins master running on a NFS 4.1 mount (AWS EFS). When a job completes we see the following in the logs: {noformat} Aug 09, 2016 4:21:09 PM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService deleteTask Delete ECS Slave task: arn:aws:ecs:us-east-1:XXXXXXXXXXXX:task/80b3bc38-774c-4bbe-9e79-f9db126a2b9b Aug 09, 2016 4:21:09 PM WARNING hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor monitor Failed to monitor jenkins-ecs-d6d51f89f024 for Free Disk Space java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown at hudson.remoting.Request$1.get(Request.java:282) at hudson.remoting.Request$1.get(Request.java:207) at hudson.remoting.FutureAdapter.get(FutureAdapter.java:59) at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:96) at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:305) Caused by: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown at hudson.remoting.Request.abort(Request.java:303) at hudson.remoting.Channel.terminate(Channel.java:847) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080) at hudson.remoting.Channel$1.handle(Channel.java:501) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83) Caused by: hudson.remoting.Channel$OrderlyShutdown ... 3 more Caused by: Command close created at at hudson.remoting.Command.<init>(Command.java:62) at hudson.remoting.Command.<init>(Command.java:47) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:822) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:822) at hudson.remoting.Channel.close(Channel.java:867) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:825) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1106) Aug 09, 2016 4:21:09 PM WARNING hudson.slaves.SlaveComputer kill Unable to delete agent logs java.io.IOException: Unable to delete '/var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts. at hudson.Util.deleteRecursive(Util.java:355) at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:670) at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:86) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:205) at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1338) at jenkins.model.Nodes$4.run(Nodes.java:219) at hudson.model.Queue._withLock(Queue.java:1320) at hudson.model.Queue.withLock(Queue.java:1197) at jenkins.model.Nodes.removeNode(Nodes.java:210) at jenkins.model.Jenkins.removeNode(Jenkins.java:1860) at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:70) at com.cloudbees.jenkins.plugins.amazonecs.ECSComputer.terminate(ECSComputer.java:62) at com.cloudbees.jenkins.plugins.amazonecs.ECSComputer.taskCompleted(ECSComputer.java:47) at hudson.model.queue.WorkUnitContext.synchronizeEnd(WorkUnitContext.java:140) at hudson.model.Executor.finish1(Executor.java:457) at hudson.model.Executor.run(Executor.java:430) Caused by: java.nio.file.FileSystemException: /var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024/.nfs9e9ba4fd7c349fa200000003: Device or resource busy at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108) at java.nio.file.Files.deleteIfExists(Files.java:1165) at hudson.Util.tryOnceDeleteFile(Util.java:296) at hudson.Util.tryOnceDeleteRecursive(Util.java:373) at hudson.Util.tryOnceDeleteContentsRecursive(Util.java:392) at hudson.Util.tryOnceDeleteRecursive(Util.java:372) at hudson.Util.deleteRecursive(Util.java:350) ... 15 more {noformat} Looking at the Jenkins master I do indeed see the log file still open: {noformat} eborgstrom@devops-jenkins-7d98bfe3:/var/lib/jenkins$ sudo lsof +D /var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 26119 jenkins 486w REG 0,25 22 11428909888000270242 /var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024/.nfs9e9ba4fd7c349fa200000003 eborgstrom@devops-jenkins-7d98bfe3:/var/lib/jenkins$ find /var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024 -ls 414054371676357616 4 drwxr-xr-x 2 jenkins jenkins 4096 Aug 9 23:21 /var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024 11428909888000270242 4 -rw-r--r-- 1 jenkins jenkins 22 Aug 9 23:21 /var/lib/jenkins/logs/slaves/jenkins-ecs-d6d51f89f024/.nfs9e9ba4fd7c349fa200000003 {noformat} This means agents remain around and need to be cleaned up manually. The logs should be fully closed before the slave is deleted.

    Jenkins JIRA | 4 months ago | Evan Borgstrom
    java.util.concurrent.ExecutionException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    [JENKINS-6817] FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel - Jenkins JIRA

    jenkins-ci.org | 11 months ago
    hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
  6. 0

    My Jenkins job is failing continuously with this error, earlier it used to pass. I am getting the error on the Jenkins job execution page, but the job is still running on windows node. I don't know if it is Jenkins problem or windows slave's issue. Here I am pasting the logs: {noformat} c:\jenkins\workspace\NSClient-Automation-1>FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41) at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34) at hudson.remoting.Request.call(Request.java:174) at hudson.remoting.Channel.call(Channel.java:739) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:168) at com.sun.proxy.$Proxy56.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:956) at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:137) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:97) at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:756) at hudson.model.Build$BuildExecution.build(Build.java:198) at hudson.model.Build$BuildExecution.doRun(Build.java:159) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) at hudson.model.Run.execute(Run.java:1706) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:232) Caused by: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown at hudson.remoting.Request.abort(Request.java:299) at hudson.remoting.Channel.terminate(Channel.java:802) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:951) at hudson.remoting.Channel$2.handle(Channel.java:475) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: hudson.remoting.Channel$OrderlyShutdown ... 3 more Caused by: Command close created at at hudson.remoting.Command.<init>(Command.java:56) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:921) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:919) at hudson.remoting.Channel.close(Channel.java:1002) at hudson.remoting.Channel.close(Channel.java:985) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:926) at hudson.remoting.Channel$2.handle(Channel.java:461) ... 1 more {noformat} Jenkins ver. 1.565.2

    Jenkins JIRA | 2 years ago | Sushant Gupta
    hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. hudson.remoting.Channel$OrderlyShutdown

      No message provided

      at hudson.remoting.Channel$CloseCommand.execute()
    2. Hudson :: Remoting Layer
      SynchronousCommandTransport$ReaderThread.run
      1. hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
      2. hudson.remoting.Channel$1.handle(Channel.java:501)
      3. hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:83)
      3 frames