java.lang.OutOfMemoryError: Direct buffer memory

Couchbase JIRA | Sergey Avseyev | 1 year ago
  1. 0

    when the connector is running, memory keeps decreasing, even when there is no document to produce, it stills eating up the memory till it crashed {code} [cb-io-1-2] WARN com.couchbase.client.core.endpoint.AbstractGenericHandler - [172.32.17.123:11210][KeyValueEndpoint]: Caught unknown exception: Direct buffer memory java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:658) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) at com.couchbase.client.deps.io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:437) at com.couchbase.client.deps.io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179) at com.couchbase.client.deps.io.netty.buffer.PoolArena.allocate(PoolArena.java:168) at com.couchbase.client.deps.io.netty.buffer.PoolArena.allocate(PoolArena.java:98) at com.couchbase.client.deps.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:250) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107) at com.couchbase.client.deps.io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104) at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) at java.lang.Thread.run(Thread.java:745) {code}

    Couchbase JIRA | 1 year ago | Sergey Avseyev
    java.lang.OutOfMemoryError: Direct buffer memory
  2. 0

    when the connector is running, memory keeps decreasing, even when there is no document to produce, it stills eating up the memory till it crashed {code} [cb-io-1-2] WARN com.couchbase.client.core.endpoint.AbstractGenericHandler - [172.32.17.123:11210][KeyValueEndpoint]: Caught unknown exception: Direct buffer memory java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:658) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) at com.couchbase.client.deps.io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:437) at com.couchbase.client.deps.io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179) at com.couchbase.client.deps.io.netty.buffer.PoolArena.allocate(PoolArena.java:168) at com.couchbase.client.deps.io.netty.buffer.PoolArena.allocate(PoolArena.java:98) at com.couchbase.client.deps.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:250) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107) at com.couchbase.client.deps.io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104) at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) at java.lang.Thread.run(Thread.java:745) {code}

    Couchbase JIRA | 1 year ago | Sergey Avseyev
    java.lang.OutOfMemoryError: Direct buffer memory
  3. 0

    java.lang.OutOfMemoryError: Direct buffer memory

    GitHub | 2 years ago | android-programmer
    java.lang.OutOfMemoryError: Direct buffer memory
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    It is a VPS instance with 1024MB memory, it runs a Jenkins. The Jenkins starts integration tests, the issue appears exactly after the 16th connect-disconnect in same JVM (possible connection leak?), the 16th connect to the cluster always throw the exception above. I've tested the all version of the Datastax driver (successful test cycles / test cycles): 2.1.3: ok (4/4) 2.1.4: ok (4/4) 2.1.5: ok (4/4) 2.1.6: failed (0/4) 2.1.7: failed (0/4) 2.1.8: failed (0/4) 2.2.0-rc1: failed (0/4) 2.2.0-rc2: failed (0/4) 2.2.0-rc3: failed (0/4) 3.0.0-alpha5: failed (0/4) The stack trace in the getErrors: {code} com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Error writing at com.datastax.driver.core.Connection$10.operationComplete(Connection.java:554) at com.datastax.driver.core.Connection$10.operationComplete(Connection.java:538) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) at io.netty.util.concurrent.DefaultPromise.notifyLateListener(DefaultPromise.java:621) at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:138) at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93) at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28) at com.datastax.driver.core.Connection$Flusher.run(Connection.java:875) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) at java.lang.Thread.run(Thread.java:745) Caused by: io.netty.handler.codec.EncoderException: java.lang.OutOfMemoryError: Direct buffer memory at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:107) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:643) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:636) at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:284) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:643) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:636) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:622) at io.netty.channel.DefaultChannelPipeline.write(DefaultChannelPipeline.java:939) at io.netty.channel.AbstractChannel.write(AbstractChannel.java:234) ... 5 more Caused by: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:658) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:645) at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:229) at io.netty.buffer.PoolArena.allocate(PoolArena.java:205) at io.netty.buffer.PoolArena.allocate(PoolArena.java:133) at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:271) at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146) at io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:83) at com.datastax.driver.core.Message$ProtocolEncoder.encode(Message.java:288) at com.datastax.driver.core.Message$ProtocolEncoder.encode(Message.java:257) at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:89) ... 15 more {code} I've tried to set log level, and found a possible root cause of the issue, it is thrown after every disconnect from Cassandra cluster: {code} 2015-11-22 21:08:59,954 WARN [io.netty.util.ThreadDeathWatcher] (threadDeathWatcher-2-1) Thread death watcher task raised an exception:: java.lang.NoClassDefFoundError: io/netty/util/Recycler $WeakOrderQueue at io.netty.util.Recycler$DefaultHandle.recycle(Recycler.java:150) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.Recycler.recycle(Recycler.java:111) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache$Entry.recycle(PoolThreadCache.java:459) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache.freeEntry(PoolThreadCache.java:442) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:414) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:406) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:263) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:254) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.free0(PoolThreadCache.java:235) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.access$000(PoolThreadCache.java:38) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$1.run(PoolThreadCache.java:64) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.ThreadDeathWatcher$Watcher.notifyWatchees(ThreadDeathWatcher.java:195) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:130) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) [netty-common-4.0.33.Final.jar:4.0.33.Final] at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0] Caused by: java.lang.ClassNotFoundException: io.netty.util.Recycler$WeakOrderQueue from [Module "deployment.gacivs-backend-dao-services-0.0.27-SNAPSHOT.war:main" from Service Module Loader] at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:213) [jboss-modules-1.3.3.Final.jar:1.3.3.Final] at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459) [jboss-modules-1.3.3.Final.jar:1.3.3.Final] at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408) [jboss-modules-1.3.3.Final.jar:1.3.3.Final] at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389) [jboss-modules-1.3.3.Final.jar:1.3.3.Final] at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134) [jboss-modules-1.3.3.Final.jar:1.3.3.Final] ... 15 more 2015-11-22 21:08:59,961 WARN [io.netty.util.ThreadDeathWatcher] (threadDeathWatcher-2-1) Thread death watcher task raised an exception:: java.lang.NoClassDefFoundError: io/netty/util/Recycler $WeakOrderQueue at io.netty.util.Recycler$DefaultHandle.recycle(Recycler.java:150) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.Recycler.recycle(Recycler.java:111) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache$Entry.recycle(PoolThreadCache.java:459) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache.freeEntry(PoolThreadCache.java:442) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:414) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$MemoryRegionCache.free(PoolThreadCache.java:406) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:263) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:254) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.free0(PoolThreadCache.java:235) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache.access$000(PoolThreadCache.java:38) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.buffer.PoolThreadCache$1.run(PoolThreadCache.java:64) [netty-buffer-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.ThreadDeathWatcher$Watcher.notifyWatchees(ThreadDeathWatcher.java:195) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:130) [netty-common-4.0.33.Final.jar:4.0.33.Final] at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) [netty-common-4.0.33.Final.jar:4.0.33.Final] at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0] {code} P.S.: the "Netty upgrade to 4.x" (https://datastax-oss.atlassian.net/browse/JAVA-622) issue resolved in the 2.1.6 version and the OutOfMemoryError: Direct buffer memory" appeared in the 2.1.6 version...

    DataStax JIRA | 1 year ago | Gábor AUTH
    com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Error writing
  6. 0

    OutOfMemory: Direct buffer memory

    GitHub | 3 years ago | defabricated
    io.netty.handler.codec.DecoderException: java.lang.OutOfMemoryError: Direct buffer memory

    7 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.OutOfMemoryError

      Direct buffer memory

      at java.nio.Bits.reserveMemory()
    2. Java RT
      ByteBuffer.allocateDirect
      1. java.nio.Bits.reserveMemory(Bits.java:658)
      2. java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
      3. java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
      3 frames
    3. com.couchbase.client
      DefaultThreadFactory$DefaultRunnableDecorator.run
      1. com.couchbase.client.deps.io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:437)
      2. com.couchbase.client.deps.io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
      3. com.couchbase.client.deps.io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
      4. com.couchbase.client.deps.io.netty.buffer.PoolArena.allocate(PoolArena.java:98)
      5. com.couchbase.client.deps.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:250)
      6. com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
      7. com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
      8. com.couchbase.client.deps.io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:107)
      9. com.couchbase.client.deps.io.netty.channel.AdaptiveRecvByteBufAllocator$HandleImpl.allocate(AdaptiveRecvByteBufAllocator.java:104)
      10. com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)
      11. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
      12. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
      13. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
      14. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
      15. com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
      16. com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
      16 frames
    4. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:745)
      1 frame