hudson.model.Queue

Jenkins JIRA | Jesse Glick | 3 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    Jenkins had stopped responding to browser requests for Jenkins pages and I think it may be caused by the recent upgrade to Throttle Concurrent Builds 1.8.1 Requests were getting blocked waiting on 0x00000004181e4520 {code} "Handling GET /jenkins/ : RequestHandlerThread[#171]" daemon prio=10 tid=0x00000000168ee800 nid=0x193b waiting for monitor entry [0x000000004335b000] java.lang.Thread.State: BLOCKED (on object monitor) at hudson.model.Queue.getItems(Queue.java:687) - waiting to lock <0x00000004181e4520> (a hudson.model.Queue) at hudson.model.Queue$CachedItemList.get(Queue.java:216) at hudson.model.Queue.getApproximateItemsQuickly(Queue.java:717) at hudson.model.View.getApproximateQueueItemsQuickly(View.java:483) at sun.reflect.GeneratedMethodAccessor355.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) {code} This seems to be waiting on Throttle Concurrent Builds code. Further dumps after 10 minutes, 20 minutes and 30 minutes showed this same stack trace. {code} "Thread-126" daemon prio=10 tid=0x00002aaae0529800 nid=0x1785 runnable [0x0000000046590000] java.lang.Thread.State: RUNNABLE at java.util.WeakHashMap$HashIterator.hasNext(WeakHashMap.java:875) at java.util.AbstractCollection.toArray(AbstractCollection.java:139) at java.util.ArrayList.<init>(ArrayList.java:164) at hudson.plugins.throttleconcurrents.ThrottleJobProperty.getCategoryProjects(ThrottleJobProperty.java:141) - locked <0x000000041a79b778> (a java.util.HashMap) at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:118) at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:90) at hudson.model.Queue.isBuildBlocked(Queue.java:937) at hudson.model.Queue.maintain(Queue.java:1006) - locked <0x00000004181e4520> (a hudson.model.Queue) at hudson.model.Queue$1.call(Queue.java:303) at hudson.model.Queue$1.call(Queue.java:300) at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:69) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:104) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None {code} CPU usage was at ~100% for this thread for the 30 minutes that I was watching it before I restarted Jenkins. (6021 = 0x1785) {code} PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 6021 rcbuild_ 35 10 18.0g 6.2g 32m R 99.7 19.7 30:45.97 java {code} I have rolled back to Throttle Concurrent Builds 1.8 for now. Still learning how to investigate thread dumps but please let me know if there is anything I can do to help.

    Jenkins JIRA | 3 years ago | Geoff Cummings
    hudson.model.Queue
  2. Speed up your debug routine!

    Automated exception search integrated into your IDE

  3. 0

    Manual removing of Docker build node with OnceRetentionStrategy causing deadlock with next stacktrace: {code} Found one Java-level deadlock: ============================= "Handling GET / from 192.168.10.18 : RequestHandlerThread[#255] View/index.jelly View/sidepanel.jelly": waiting to lock monitor 0x00007f4b9ae22bc8 (object 0x000000078713b8f0, a hudson.model.Queue), which is held by "jenkins.util.Timer [#3]" "jenkins.util.Timer [#3]": waiting to lock monitor 0x00007f4b9ae219e8 (object 0x0000000787158e00, a hudson.model.Hudson), which is held by "Executor #0 for bfe04df8701e@vm : executing project-release #125" "Executor #0 for bfe04df8701e@vm : executing project-release #125": waiting to lock monitor 0x00007f4b997e99b8 (object 0x0000000792523c40, a org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy), which is held by "jenkins.util.Timer [#3]" Java stack information for the threads listed above: =================================================== "Handling GET / from 192.168.10.18 : RequestHandlerThread[#255] View/index.jelly View/sidepanel.jelly": at hudson.model.Queue.getItems(Queue.java:693) - waiting to lock <0x000000078713b8f0> (a hudson.model.Queue) at hudson.model.Queue$CachedItemList.get(Queue.java:229) at hudson.model.Queue.getApproximateItemsQuickly(Queue.java:723) at hudson.model.View.getApproximateQueueItemsQuickly(View.java:484) at sun.reflect.GeneratedMethodAccessor3580.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) at groovy.lang.MetaClassImpl$GetBeanMethodMetaProperty.getProperty(MetaClassImpl.java:3500) at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:61) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:227) at jenkins.widgets.BuildQueueWidget.index.run(index.groovy:5) at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:74) at org.kohsuke.stapler.jelly.groovy.GroovierJellyScript.run(GroovierJellyScript.java:62) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:147) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:150) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:147) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120) at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:95) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyClassTearOff.serveIndexJelly(JellyClassTearOff.java:112) at org.kohsuke.stapler.jelly.JellyFacet.handleIndexRequest(JellyFacet.java:127) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:735) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:813) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:88) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) "jenkins.util.Timer [#3]": at jenkins.model.Jenkins.removeNode(Jenkins.java:1703) - waiting to lock <0x0000000787158e00> (a hudson.model.Hudson) at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:65) at hudson.slaves.CloudRetentionStrategy$1.run(CloudRetentionStrategy.java:67) at hudson.model.Queue._withLock(Queue.java:1035) - locked <0x000000078713b8f0> (a hudson.model.Queue) at hudson.model.Queue.withLock(Queue.java:983) at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:58) - locked <0x0000000792523c40> (a org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy) at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:45) at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:66) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) "Executor #0 for bfe04df8701e@vm : executing project-release #125": at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:54) - waiting to lock <0x0000000792523c40> (a org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy) at hudson.slaves.CloudRetentionStrategy.check(CloudRetentionStrategy.java:45) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:661) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) - locked <0x0000000787158f18> (a java.lang.Object) at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1218) at jenkins.model.Jenkins.setNodes(Jenkins.java:1714) at jenkins.model.Jenkins.removeNode(Jenkins.java:1709) - locked <0x0000000787158e00> (a hudson.model.Hudson) at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:65) at org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy.done(OnceRetentionStrategy.java:81) at org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy.taskCompleted(OnceRetentionStrategy.java:63) at hudson.slaves.SlaveComputer.taskCompleted(SlaveComputer.java:296) at com.nirima.jenkins.plugins.docker.DockerComputer.taskCompleted(DockerComputer.java:34) at hudson.model.queue.WorkUnitContext.synchronizeEnd(WorkUnitContext.java:140) at hudson.model.Executor.run(Executor.java:253) Found 1 deadlock. {code}

    Jenkins JIRA | 2 years ago | Victor Antonovich
    hudson.model.Queue

    Root Cause Analysis

    1. hudson.model.Queue

      No message provided

      at hudson.model.Queue$CachedItemList.get()
    2. Hudson
      View.getApproximateQueueItemsQuickly
      1. hudson.model.Queue$CachedItemList.get(Queue.java:217)
      2. hudson.model.Queue.getApproximateItemsQuickly(Queue.java:715)
      3. hudson.model.View.getApproximateQueueItemsQuickly(View.java:483)
      3 frames