java.lang.OutOfMemoryError

JIRA | Gordon Mohr | 1 decade ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    In a courtesy crawl for University of Michigan, encountered the following OutOfMemory error: <<< java.lang.OutOfMemoryError #131 http://www-mhp.physics.lsa.umich.edu/~keithr/S2DQ/S2H2v05_seginfo.html (0 attempts) XLXLLLLLLLLLL Current processor: Postselector ACTIVE for 28s601ms Where: ABOUT_TO_BEGIN_PROCESSOR for 17753ms java.lang.OutOfMemoryError at java.nio.Bits.reserveMemory(Bits.java:618) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285) at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54) at sun.nio.ch.IOUtil.write(IOUtil.java:69) at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:615) at com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:773) at com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:167 ) at com.sleepycat.je.log.LogBufferPool.getWriteBuffer(LogBufferPool.java:123) at com.sleepycat.je.log.LogManager.logInternal(LogManager.java:302) at com.sleepycat.je.log.SyncedLogManager.logItem(SyncedLogManager.java:50) at com.sleepycat.je.log.LogManager.log(LogManager.java:178) at com.sleepycat.je.log.LogManager.log(LogManager.java:130) at com.sleepycat.je.tree.LN.log(LN.java:384) at com.sleepycat.je.tree.LN.log(LN.java:332) at com.sleepycat.je.tree.Tree.insert(Tree.java:1674) at com.sleepycat.je.dbi.CursorImpl.putLN(CursorImpl.java:654) at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:678) at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:713) at com.sleepycat.je.Cursor.putInternal(Cursor.java:661) at com.sleepycat.je.Database.putInternal(Database.java:499) at com.sleepycat.je.Database.put(Database.java:443) at org.archive.crawler.frontier.BdbFrontier$BdbMultiQueue.put(BdbFrontier.java :830) at org.archive.crawler.frontier.BdbFrontier$BdbWorkQueue.enqueue(BdbFrontier.j ava:1058) at org.archive.crawler.frontier.BdbFrontier.sendToQueue(BdbFrontier.java:247) at org.archive.crawler.frontier.BdbFrontier.receive(BdbFrontier.java:234) at org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:245) at org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:204) at org.archive.crawler.frontier.BdbFrontier.schedule(BdbFrontier.java:216) at org.archive.crawler.postprocessor.Postselector.schedule(Postselector.java:2 71) at org.archive.crawler.postprocessor.Postselector.handleLinkCollection(Postsel ector.java:358) at org.archive.crawler.postprocessor.Postselector.innerProcess(Postselector.ja va:184) at org.archive.crawler.framework.Processor.process(Processor.java:102) at org.archive.crawler.framework.ToeThread.processCrawlUri(ToeThread.java:251) at org.archive.crawler.framework.ToeThread.run(ToeThread.java:127) >>> The Sun 142_02 JVM had been started with -Xmx1500m on a 2G machine, and was only showing a heap size of ~700MB at the time of the OOM -- so allocations even of giant contiguous ranges of memory should have succeeded. Suspect instead some other exhaustion is coming through as an OOM. Restarted crawl with -Xmx1000m, it's run much longer with no problems, even with heap growing to max.

    JIRA | 1 decade ago | Gordon Mohr
    java.lang.OutOfMemoryError

    1 unregistered visitors

    Root Cause Analysis

    1. java.lang.OutOfMemoryError

      No message provided

      at java.nio.Bits.reserveMemory()
    2. Java RT
      FileChannelImpl.write
      1. java.nio.Bits.reserveMemory(Bits.java:618)
      2. java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
      3. java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:285)
      4. sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:54)
      5. sun.nio.ch.IOUtil.write(IOUtil.java:69)
      6. sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:615)
      6 frames
    3. Berkeley DB Java Edition
      Database.put
      1. com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:773)
      2. com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:167)
      3. com.sleepycat.je.log.LogBufferPool.getWriteBuffer(LogBufferPool.java:123)
      4. com.sleepycat.je.log.LogManager.logInternal(LogManager.java:302)
      5. com.sleepycat.je.log.SyncedLogManager.logItem(SyncedLogManager.java:50)
      6. com.sleepycat.je.log.LogManager.log(LogManager.java:178)
      7. com.sleepycat.je.log.LogManager.log(LogManager.java:130)
      8. com.sleepycat.je.tree.LN.log(LN.java:384)
      9. com.sleepycat.je.tree.LN.log(LN.java:332)
      10. com.sleepycat.je.tree.Tree.insert(Tree.java:1674)
      11. com.sleepycat.je.dbi.CursorImpl.putLN(CursorImpl.java:654)
      12. com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:678)
      13. com.sleepycat.je.Cursor.putNoNotify(Cursor.java:713)
      14. com.sleepycat.je.Cursor.putInternal(Cursor.java:661)
      15. com.sleepycat.je.Database.putInternal(Database.java:499)
      16. com.sleepycat.je.Database.put(Database.java:443)
      16 frames
    4. org.archive.crawler
      ToeThread.run
      1. org.archive.crawler.frontier.BdbFrontier$BdbMultiQueue.put(BdbFrontier.java:830)
      2. org.archive.crawler.frontier.BdbFrontier$BdbWorkQueue.enqueue(BdbFrontier.java:1058)
      3. org.archive.crawler.frontier.BdbFrontier.sendToQueue(BdbFrontier.java:247)
      4. org.archive.crawler.frontier.BdbFrontier.receive(BdbFrontier.java:234)
      5. org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:245)
      6. org.archive.crawler.util.BdbUriUniqFilter.add(BdbUriUniqFilter.java:204)
      7. org.archive.crawler.frontier.BdbFrontier.schedule(BdbFrontier.java:216)
      8. org.archive.crawler.postprocessor.Postselector.schedule(Postselector.java:271)
      9. org.archive.crawler.postprocessor.Postselector.handleLinkCollection(Postselector.java:358)
      10. org.archive.crawler.postprocessor.Postselector.innerProcess(Postselector.java:184)
      11. org.archive.crawler.framework.Processor.process(Processor.java:102)
      12. org.archive.crawler.framework.ToeThread.processCrawlUri(ToeThread.java:251)
      13. org.archive.crawler.framework.ToeThread.run(ToeThread.java:127)
      13 frames