org.archive.crawler.admin.CrawlJob$MBeanCrawlController

JIRA | Gordon Mohr | 10 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    Productions crawl (2BC) found in 'pausing' state, 7 threads still waiting on frontier lock to complete pause for days. Can't unpause. SIGQUIT thread dump reveals deadlock: Found one Java-level deadlock: ============================= "RMI TCP Connection(128)-208.70.29.245": waiting to lock monitor 0x080a7120 (object 0x9bed2a28, a org.archive.crawler.admin.CrawlJob$MBeanCrawlController), which is held by "PoolThread-1" "PoolThread-1": waiting to lock monitor 0x080a70bc (object 0x9bed3270, a org.archive.crawler.frontier.BdbFrontier), which is held by "ToeThread #127: " "ToeThread #127: ": waiting to lock monitor 0x080a7120 (object 0x9bed2a28, a org.archive.crawler.admin.CrawlJob$MBeanCrawlController), which is held by "PoolThread-1" Java stack information for the threads listed above: =================================================== "RMI TCP Connection(128)-208.70.29.245": at org.archive.crawler.framework.CrawlController.requestCrawlPause(CrawlController.java:1441) - waiting to lock <0x9bed2a28> (a org.archive.crawler.admin.CrawlJob$MBeanCrawlController) at org.archive.crawler.admin.CrawlJob.pause(CrawlJob.java:1087) at org.archive.crawler.admin.CrawlJob.invoke(CrawlJob.java:1866) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at com.sun.jmx.remote.security.MBeanServerAccessController.invoke(MBeanServerAccessController.java:431) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264) at java.security.AccessController.doPrivileged(Native Method) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1366) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) "PoolThread-1": at org.archive.crawler.frontier.AbstractFrontier.unpause(AbstractFrontier.java:351) - waiting to lock <0x9bed3270> (a org.archive.crawler.frontier.BdbFrontier) at org.archive.crawler.framework.CrawlController.requestCrawlResume(CrawlController.java:1481) - locked <0x9bed2a28> (a org.archive.crawler.admin.CrawlJob$MBeanCrawlController) at org.archive.crawler.admin.CrawlJob.resume(CrawlJob.java:1093) at org.archive.crawler.admin.CrawlJobHandler.resumeJob(CrawlJobHandler.java:646) at org.archive.crawler.jspc.admin.console.action_jsp._jspService(Unknown Source) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:137) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:358) at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:342) at org.archive.crawler.admin.ui.RootFilter.doFilter(RootFilter.java:67) at org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApplicationHandler.java:334) at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:286) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567) at org.mortbay.http.HttpContext.handle(HttpContext.java:1807) at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:525) at org.mortbay.http.HttpContext.handle(HttpContext.java:1757) at org.mortbay.http.HttpServer.service(HttpServer.java:879) at org.mortbay.http.HttpConnection.service(HttpConnection.java:789) at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:960) at org.mortbay.http.HttpConnection.handle(HttpConnection.java:806) at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:218) at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:300) at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:511) "ToeThread #127: ": at org.archive.crawler.framework.CrawlController.toePaused(CrawlController.java:1786) - waiting to lock <0x9bed2a28> (a org.archive.crawler.admin.CrawlJob$MBeanCrawlController) at org.archive.crawler.frontier.AbstractFrontier.preNext(AbstractFrontier.java:623) - locked <0x9bed3270> (a org.archive.crawler.frontier.BdbFrontier) at org.archive.crawler.frontier.WorkQueueFrontier.next(WorkQueueFrontier.java:631) at org.archive.crawler.framework.ToeThread.run(ToeThread.java:144) Found 1 deadlock.

    JIRA | 10 years ago | Gordon Mohr
    org.archive.crawler.admin.CrawlJob$MBeanCrawlController

    Root Cause Analysis

    1. org.archive.crawler.admin.CrawlJob$MBeanCrawlController

      No message provided

      at org.archive.crawler.admin.CrawlJob.pause()
    2. org.archive.crawler
      CrawlJob.invoke
      1. org.archive.crawler.admin.CrawlJob.pause(CrawlJob.java:1087)
      2. org.archive.crawler.admin.CrawlJob.invoke(CrawlJob.java:1866)
      2 frames
    3. Java RT
      Thread.run
      1. com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
      2. com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
      3. com.sun.jmx.remote.security.MBeanServerAccessController.invoke(MBeanServerAccessController.java:431)
      4. javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
      5. javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
      6. javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
      7. java.security.AccessController.doPrivileged(Native Method)
      8. javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1366)
      9. javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
      10. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      11. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      12. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      13. java.lang.reflect.Method.invoke(Method.java:597)
      14. sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
      15. sun.rmi.transport.Transport$1.run(Transport.java:159)
      16. java.security.AccessController.doPrivileged(Native Method)
      17. sun.rmi.transport.Transport.serviceCall(Transport.java:155)
      18. sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
      19. sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
      20. sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
      21. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
      22. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
      23. java.lang.Thread.run(Thread.java:619)
      23 frames