com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket

Couchbase JIRA | Michael Blow | 7 months ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    CouchbaseConnectorStressTest.testConnectDisconnectCycle performs the following in a loop: {noformat} Client client = Client.configure() .eventLoopGroup(ioPool()) .bucket(bucket) .password(cbPassword) .hostnames(cbNodes) .connectionNameGenerator(() -> threadName) .controlParam(DcpControl.Names.CONNECTION_BUFFER_SIZE, 16384) .bufferAckWatermark(80) .build(); client.controlEventHandler(ReferenceCountUtil::release); client.dataEventHandler(event -> { try { client.acknowledgeBuffer(event); } finally { ReferenceCountUtil.release(event); } }); timedAwait("client.connect()", client.connect()); timedAwait("client.disconnect()", client.disconnect()); {noformat} Between 10-20 seconds of iterations, I observe many exceptions like the following on the console: {noformat} ## testConnectDisconnectCycle START ### Starting 4 threads with run time of 60 seconds. #### [thread 0] Starting thread (will trace every 10 seconds) #### [thread 1] Starting thread (will trace every 10 seconds) #### [thread 2] Starting thread (will trace every 10 seconds) #### [thread 3] Starting thread (will trace every 10 seconds) #### [thread 1] ...iteration 1338... #### [thread 0] ...iteration 1001... #### [thread 3] ...iteration 213... #### [thread 2] ...iteration 450... [cb-io-1-6] 19:58:26 WARN DefaultChannelPipeline:299 - An exception 'java.lang.IllegalStateException: complete already: DefaultChannelPromise@752e0392(failure: java.io.IOException: Connection reset by peer)' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at com.couchbase.client.deps.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899) at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241) at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:572) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:513) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:427) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:399) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) at java.lang.Thread.run(Thread.java:745) {noformat} The error does not get thrown to the client, and iterations continue. Eventually after 30-40 seconds (and 1000s of iterations), I do get an error out of all threads: {noformat} #### [thread 3] ...iteration 1801... #### [thread 2] Failing fast due to 1 failure(s): [com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket] #### [thread 2] Exiting thread after 2320 iterations. #### [thread 1] Failing fast due to 1 failure(s): [com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket] #### [thread 1] Exiting thread after 2872 iterations. #### [thread 3] Failing fast due to 1 failure(s): [com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket] #### [thread 3] Exiting thread after 1808 iterations. ### Completed 8792 iterations with 4 failure(s). ## testConnectDisconnectCycle FAILED (com.couchbase.client.dcp.error.BootstrapException) com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket at com.couchbase.client.dcp.Client$4.call(Client.java:290) at com.couchbase.client.dcp.Client$4.call(Client.java:287) at rx.Completable$24$1.onError(Completable.java:1735) at rx.internal.operators.CompletableOnSubscribeConcatArray$ConcatInnerSubscriber.onError(CompletableOnSubscribeConcatArray.java:63) at com.couchbase.client.dcp.conductor.HttpStreamingConfigProvider$4$1.operationComplete(HttpStreamingConfigProvider.java:194) at com.couchbase.client.dcp.conductor.HttpStreamingConfigProvider$4$1.operationComplete(HttpStreamingConfigProvider.java:177) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:507) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:120) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPromise.setFailure(DefaultChannelPromise.java:87) at com.couchbase.client.dcp.transport.netty.ConnectInterceptingHandler.exceptionCaught(ConnectInterceptingHandler.java:108) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:290) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:269) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:261) at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:423) at com.couchbase.client.deps.io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:79) at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$1.fireExceptionCaught(CombinedChannelDuplexHandler.java:144) at com.couchbase.client.deps.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131) at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.exceptionCaught(CombinedChannelDuplexHandler.java:230) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:290) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:269) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:261) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1261) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:290) at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:269) at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:899) at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:87) at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:572) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:513) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:427) at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:399) at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140) at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at com.couchbase.client.deps.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221) at com.couchbase.client.deps.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899) at com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241) at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) ... 7 more {noformat}

    Couchbase JIRA | 7 months ago | Michael Blow
    com.couchbase.client.dcp.error.BootstrapException: Could not connect to Cluster/Bucket
  2. 0

    TcpServer: LEAK: ByteBuf.release() was not called in before it's garbage-collected

    GitHub | 2 years ago | riteshmodi
    java.io.IOException: Connection reset by peer
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    LEAK: ByteBuf.release() was not called in before it's garbage-collected. Spring Reactor TcpServer

    Stack Overflow | 2 years ago
    java.io.IOException: Connection reset by peer
  5. 0

    Closed Channel Exception with valid p12

    GitHub | 2 years ago | tonicebrian
    java.io.IOException: Connection reset by peer

  1. tzrlk 2 times, last 1 month ago
  2. iferca 1 times, last 2 months ago
  3. Hronom 4 times, last 3 months ago
  4. SteVio1989 6005 times, last 16 hours ago
  5. Alireza Mohamadi 11 times, last 3 months ago
14 more registered users
81 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.io.IOException

    Connection reset by peer

    at sun.nio.ch.FileDispatcherImpl.read0()
  2. Java RT
    SocketChannelImpl.read
    1. sun.nio.ch.FileDispatcherImpl.read0(Native Method)
    2. sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
    3. sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
    4. sun.nio.ch.IOUtil.read(IOUtil.java:192)
    5. sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
    5 frames
  3. com.couchbase.client
    DefaultThreadFactory$DefaultRunnableDecorator.run
    1. com.couchbase.client.deps.io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:221)
    2. com.couchbase.client.deps.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:899)
    3. com.couchbase.client.deps.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:241)
    4. com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
    5. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:572)
    6. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:513)
    7. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:427)
    8. com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:399)
    9. com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
    10. com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
    10 frames
  4. Java RT
    Thread.run
    1. java.lang.Thread.run(Thread.java:745)
    1 frame