java.lang.OutOfMemoryError

JIRA | Gordon Mohr | 1 decade ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    In a courtesy crawl for University of Michigan, encountered the following OutOfMemory error: <<< java.lang.OutOfMemoryError #131 http://www-mhp.physics.lsa.umich.edu/~keithr/S2DQ/S2H2v05_seginfo.html (0 attempts) XLXLLLLLLLLLL Current processor: Postselector ACTIVE for 28s601ms Where: ABOUT_TO_BEGIN_PROCESSOR for 17753ms java.lang.OutOfMemoryError at java.nio.Bits.reserveMemory(Bits.java:618) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285) at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54) at sun.nio.ch.IOUtil.write(IOUtil.java:69) at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:615) at com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:773) at com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:167 ) at com.sleepycat.je.log.LogBufferPool.getWriteBuffer(LogBufferPool.java:123) at com.sleepycat.je.log.LogManager.logInternal(LogManager.java:302) at com.sleepycat.je.log.SyncedLogManager.logItem(SyncedLogManager.java:50) at com.sleepycat.je.log.LogManager.log(LogManager.java:178) at com.sleepycat.je.log.LogManager.log(LogManager.java:130) at com.sleepycat.je.tree.LN.log(LN.java:384) at com.sleepycat.je.tree.LN.log(LN.java:332) at com.sleepycat.je.tree.Tree.insert(Tree.java:1674) at com.sleepycat.je.dbi.CursorImpl.putLN(CursorImpl.java:654) at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:678) at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:713) at com.sleepycat.je.Cursor.putInternal(Cursor.java:661) at com.sleepycat.je.Database.putInternal(Database.java:499) at com.sleepycat.je.Database.put(Database.java:443) at org.archive.crawler.frontier.BdbFrontier$BdbMultiQueue.put(BdbFrontier.java :830) at org.archive.crawler.frontier.BdbFrontier$BdbWorkQueue.enqueue(BdbFrontier.j ava:1058) at org.archive.crawler.frontier.BdbFrontier.sendToQueue(BdbFrontier.java:247) at org.archive.crawler.frontier.BdbFrontier.receive(BdbFrontier.java:234) at org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:245) at org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:204) at org.archive.crawler.frontier.BdbFrontier.schedule(BdbFrontier.java:216) at org.archive.crawler.postprocessor.Postselector.schedule(Postselector.java:2 71) at org.archive.crawler.postprocessor.Postselector.handleLinkCollection(Postsel ector.java:358) at org.archive.crawler.postprocessor.Postselector.innerProcess(Postselector.ja va:184) at org.archive.crawler.framework.Processor.process(Processor.java:102) at org.archive.crawler.framework.ToeThread.processCrawlUri(ToeThread.java:251) at org.archive.crawler.framework.ToeThread.run(ToeThread.java:127) >>> The Sun 142_02 JVM had been started with -Xmx1500m on a 2G machine, and was only showing a heap size of ~700MB at the time of the OOM -- so allocations even of giant contiguous ranges of memory should have succeeded. Suspect instead some other exhaustion is coming through as an OOM. Restarted crawl with -Xmx1000m, it's run much longer with no problems, even with heap growing to max.

    JIRA | 1 decade ago | Gordon Mohr
    java.lang.OutOfMemoryError

    1 unregistered visitors

    Root Cause Analysis

    1. java.lang.OutOfMemoryError

      No message provided

      at java.nio.Bits.reserveMemory()
    2. Java RT
      FileChannelImpl.write
      1. java.nio.Bits.reserveMemory(Bits.java:618)
      2. java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
      3. java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
      4. sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54)
      5. sun.nio.ch.IOUtil.write(IOUtil.java:69)
      6. sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:615)
      6 frames
    3. Berkeley DB Java Edition
      Database.put
      1. com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:773)
      2. com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:167)
      3. com.sleepycat.je.log.LogBufferPool.getWriteBuffer(LogBufferPool.java:123)
      4. com.sleepycat.je.log.LogManager.logInternal(LogManager.java:302)
      5. com.sleepycat.je.log.SyncedLogManager.logItem(SyncedLogManager.java:50)
      6. com.sleepycat.je.log.LogManager.log(LogManager.java:178)
      7. com.sleepycat.je.log.LogManager.log(LogManager.java:130)
      8. com.sleepycat.je.tree.LN.log(LN.java:384)
      9. com.sleepycat.je.tree.LN.log(LN.java:332)
      10. com.sleepycat.je.tree.Tree.insert(Tree.java:1674)
      11. com.sleepycat.je.dbi.CursorImpl.putLN(CursorImpl.java:654)
      12. com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:678)
      13. com.sleepycat.je.Cursor.putNoNotify(Cursor.java:713)
      14. com.sleepycat.je.Cursor.putInternal(Cursor.java:661)
      15. com.sleepycat.je.Database.putInternal(Database.java:499)
      16. com.sleepycat.je.Database.put(Database.java:443)
      16 frames
    4. org.archive.crawler
      ToeThread.run
      1. org.archive.crawler.frontier.BdbFrontier$BdbMultiQueue.put(BdbFrontier.java:830)
      2. org.archive.crawler.frontier.BdbFrontier$BdbWorkQueue.enqueue(BdbFrontier.java:1058)
      3. org.archive.crawler.frontier.BdbFrontier.sendToQueue(BdbFrontier.java:247)
      4. org.archive.crawler.frontier.BdbFrontier.receive(BdbFrontier.java:234)
      5. org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:245)
      6. org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:204)
      7. org.archive.crawler.frontier.BdbFrontier.schedule(BdbFrontier.java:216)
      8. org.archive.crawler.postprocessor.Postselector.schedule(Postselector.java:271)
      9. org.archive.crawler.postprocessor.Postselector.handleLinkCollection(Postselector.java:358)
      10. org.archive.crawler.postprocessor.Postselector.innerProcess(Postselector.java:184)
      11. org.archive.crawler.framework.Processor.process(Processor.java:102)
      12. org.archive.crawler.framework.ToeThread.processCrawlUri(ToeThread.java:251)
      13. org.archive.crawler.framework.ToeThread.run(ToeThread.java:127)
      13 frames