java.net.SocketException: Connection reset by peer: socket write error

Couchbase JIRA | Andrei Baranouski | 2 years ago
  1. 0

    call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf ig.ini -I sdkd.args --variants HYBRID -d all:trace [0.32 INFO] (BRun run:436) Initializing S3 [1.19 INFO] (BRun run:441) Initializing history database [1.82 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools ============================================================ Running ReAdd2-HYBRID: 2014-08-26 22:52:11 workload=HybridWorkloadGroup failover/next_action=FO_READD_REBALANCE failover/count=2 failover/next_delay=45 testcase=FailoverScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0211 31\68eb149dd819e0fcf974500c9b7554ef To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ \ --workload HybridWorkloadGroup --failover-next_action \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45 \ --testcase FailoverScenario -C share\rexec --rexec_path \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 \ --cluster_ssh-username=root --cluster_ssh-password=couchbase --cluster_useSSL=True ---------------------------------------->8 [57.94 INFO] (Workload setupDesign:63) Creating design test_design [57.95 TRACE] (CouchbaseAdmin getResponseJson:167) PUT http://10.3.121.134:8092/default/_design/test_design [58.19 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8092/default/_design/test_design/_view/test_view?limit= 1 [58.23 INFO] (Workload setupDesign:80) Design creation done [58.23 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.24 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=true, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [58.24 TRACE] (Handle sendMessageAsync:185) > {"Command":"NEWHANDLE","Handle":101,"ReqID":1,"CommandData":{"Port":8091,"Bucket":"d efault","Options":{"SSL":true,"Username":"default","Password":"","OtherNodes":[["10.3.121.135",8091],["10.3.121.136",8091],["10.3. 3.206",8091]]},"Hostname":"10.3.121.134"}} [58.25 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101 [58.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net [58.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToke n=af08829b84f0328e [58.30 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net [58.30 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d42611 5821 [58.58 INFO] (SDKD log:137) [58.59 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an obje ct. [198.47 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [198.48 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.2 [198.48 INFO] (SDKD log:137) at Couchbase.IO.Strategies.SslConnection.Send[T](IOperation`1 operation) in c:\jenkins\workspace\. net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\SslConnection.cs:line 53 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation, IConnection connection) in c:\jenkins\workspace \.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 35 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-and rei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 45 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String p assword) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\Carr ierPublication\CarrierPublicationProvider.cs:line 41 at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-a ndrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 159 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei \couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs :line 134 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo lean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre serveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [198.49 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":101,"ReqID":2,"CommandData":{}} [198.50 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] [199.41 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re Add2-HYBRID/08-26-14/021131/68eb149dd819e0fcf974500c9b7554ef.zip [199.42 ERROR] (BRun main:513) Caught exception com.couchbase.sdkdclient.context.HarnessException: java.net.SocketException: Connection reset at com.couchbase.sdkdclient.context.HarnessException.create(HarnessException.java:56) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.scenario.PhasedScenario.run(PhasedScenario.java:80) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:169) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$4.call(RunContext.java:165) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamDecoder.implRead(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamDecoder.read(Unknown Source) ~[na:1.6.0_27] seems like it's a regression for ssl

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    java.net.SocketException: Connection reset by peer: socket write error
  2. 0

    very often in different tests I see the following exception http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/38/consoleFull Running SvcRestart-HYBRID: 2014-07-25 05:20:15 .............. [365.86 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object. [366.20 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [366.20 INFO] (SDKD log:137)  at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 51 at Couchbase.Configuration.ConfigContextBase..ctor(IBucketConfig bucketConfig, ClientConfiguration clientConfig, Func`2 ioStrategyFactory, Func`3 connectionPoolFactory, Func`5 saslFactory, IByteConverter converter, ITypeSerializer serializer) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 51 at Couchbase.Configuration.CouchbaseConfigContext..ctor(IBucketConfig bucketConfig, ClientConfiguration clientConfig, Func`2 ioStrategyFactory, Func`3 connectionPoolFactory, Func`5 saslFactory, IByteConverter converter, ITypeSerializer serializer) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 22 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 63 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 44 at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 155 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 82 at Sdkd.Couchbase.ClientFactory.FindExistingClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 55 at Sdkd.Couchbase.ClientFactory.CreateClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 262 at SdkdConsole.Daemon.Handle.CreateHandle(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 103 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [366.20 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [366.21 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@203.259 [366.21 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] client logs http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/38/artifact/log.txt Jeff, believe that this is a problem on client side?

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    java.net.SocketException: Connection reset by peer: socket write error
  3. 0

    git log commit 86219ee99ccafdd00c98d577c9bc2b97efef35bb Author: jeffrymorris <jeffrymorris@gmail.com> Date: Wed Aug 20 17:44:45 2014 -0700 NCBC-592: AuthenticationException in CouchbaseBucket.SendWithRetry Running Rb2Out-KV: 2014-08-20 23:33:57 ..... [25.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.33 [25.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@121.43 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=10, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [25.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET  [25.06 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds [25.06 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [25.06 INFO] (SDKD log:137)  "DSType": "DSTYPE_SEEDED", [25.06 INFO] (SDKD log:137)  "DS": { [25.07 INFO] (SDKD log:137)  "KSeed": "SimpleKey", [25.07 INFO] (SDKD log:137)  "VSeed": "SimpleValue", [25.07 INFO] (SDKD log:137)  "Count": 10, [25.07 INFO] (SDKD log:137)  "Continuous": true, [25.07 INFO] (SDKD log:137)  "VSize": 128, [25.07 INFO] (SDKD log:137)  "Repeat": "REP", [25.07 INFO] (SDKD log:137)  "KSize": 32 [25.07 INFO] (SDKD log:137)  }, [25.07 INFO] (SDKD log:137)  "Options": { [25.07 INFO] (SDKD log:137)  "DelayMax": 10, [25.07 INFO] (SDKD log:137)  "DelayMin": 1, [25.07 INFO] (SDKD log:137)  "IterWait": 1, [25.07 INFO] (SDKD log:137)  "TimeRes": 1 [25.09 INFO] (SDKD log:137)  } [25.09 INFO] (SDKD log:137) }  [55.06 INFO] (Scenario run:86) Starting CHANGE phase [55.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [56.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [57.12 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [58.13 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4% [59.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% ................................. [99.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 94% [100.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96% [101.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 98% [102.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [103.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [104.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [105.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [106.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [107.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [108.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [109.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [110.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [111.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [112.75 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409  [112.78 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [112.79 INFO] (Scenario run:97) CHANGE phase done [112.79 INFO] (Scenario run:103) Starting REBOUND for 90 seconds [113.23 INFO] (SDKD log:137)  [113.26 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavailableException' was thrown. [125.87 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [125.88 INFO] (SDKD log:137)  at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 212 at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 179 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [125.89 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [125.89 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.44 [125.90 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    java.net.SocketException: Connection reset by peer: socket write error
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    after some time the tests begin to fall with 2014-07-28 00:59:20,864 [27] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Could not authenticate default using Couchbase.Authentication.SASL.CramMd5Mechanism. Running Rb2Out-HYBRID: 2014-07-28 00:05:18 rebalance/mode=out workload=HybridWorkloadGroup rebalance/count=2 testcase=RebalanceScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-973\Rb2Out-HYBRID\07-28-14\025518\7d8cf558a26ca99fe4dc3028bc6b2ec0 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ --rebalance-mode out --workload HybridWorkloadGroup \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port \ 8675 --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 \ --cluster_ssh-username=root --cluster_ssh-password=couchbase ---------------------------------------->8 [1.47 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution [1.87 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds. [1.88 INFO] (RunContext run:124) Starting cluster and driver [1.89 INFO] (CBCluster startCluster:360) Node http://10.3.121.134:8091 is master now [1.89 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'  [1.89 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations.. [2.26 INFO] (SDKD log:137) --- Logging Self-Test --- [2.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message  [2.27 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675  [2.42 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0 [2.43 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [2.76 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}} [4.02 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.135:8091,ns_1@10.3.121.135> [4.06 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.136:8091,ns_1@10.3.121.136> [4.11 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.3.206:8091,ns_1@10.3.3.206> [4.18 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.18 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.70 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.134:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.3.206:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.135:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.136:8091 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.134 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.3.206 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.136 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.135 [4.94 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@641a034d [4.94 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@19c6866d [4.95 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@51c293f8 [4.95 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@5a9b8ff9 [4.95 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@6e0f0ad [18.52 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.135:8091 [26.45 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.136:8091 [30.42 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.3.206:8091 [35.78 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance [35.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [36.84 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [36.85 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups [36.85 INFO] (CBCluster setupMainBucket:209) Creating bucket default [36.87 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted [42.18 INFO] (CBCluster waitForBucketReady:203) Bucket creation done [42.23 INFO] (RunContext run:143) Driver and cluster initialized [42.26 INFO] (RunContext call:167) Running scenario.. [42.26 INFO] (Scenario run:72) Starting RAMP phase [42.26 INFO] (Workload setupDesign:63) Creating design test_design [42.51 INFO] (Workload setupDesign:80) Design creation done [42.51 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [42.52 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [42.53 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101  [42.54 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [42.56 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net  [42.56 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e  [42.59 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net  [42.59 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821  [42.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1 [42.91 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [43.02 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409  [43.39 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: { [43.39 INFO] (SDKD log:137)  "0": 44, [43.40 INFO] (SDKD log:137)  "264": 956 [43.40 INFO] (SDKD log:137) }  [43.40 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@101.2 => {[OK]: 44, [MEMD:GENERIC]: 956} [43.40 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3 [43.40 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102  [43.41 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.42 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4 [43.42 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [43.82 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: { [43.82 INFO] (SDKD log:137)  "0": 39, [43.83 INFO] (SDKD log:137)  "264": 961 [43.83 INFO] (SDKD log:137) }  [43.83 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@102.5 => {[OK]: 39, [MEMD:GENERIC]: 961} [43.83 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6 [43.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103  [43.83 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7 [43.85 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.85 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.85 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104  [43.86 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.87 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9 [43.88 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.88 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105  [43.88 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.90 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11 [43.90 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.90 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.90 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.91 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106  [43.91 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.92 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13 [43.93 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.93 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107  [43.94 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.97 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15 [43.97 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.97 INFO] (Workload call:112) Preloading items into cluster before workload [43.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.98 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108  [43.98 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.99 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17 [43.99 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}} [51.93 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 364, [MEMD:GENERIC]: 14636} [51.93 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19 [51.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.95 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22 [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@111.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111  [52.00 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114  [52.01 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112  [52.02 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113  [52.03 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.03 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21 [52.03 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119  [52.03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.03 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@110.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.04 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.04 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.05 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120  [52.07 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.07 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26 [52.07 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@115.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.08 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121  [52.10 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.10 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20 [52.10 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@109.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.11 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.11 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.12 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122  [52.12 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25 [52.13 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.13 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.13 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.14 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123  [52.16 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.16 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27 [52.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@116.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.17 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.17 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124  [52.24 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.25 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28 [52.25 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@117.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.26 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.26 INFO] (SDKD log:137)  [52.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125  [52.27 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.30 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.30 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29 [52.31 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1 [52.31 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@118.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.31 INFO] (SDKD log:137)  at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 75 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 40 at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 82 at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 121 at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 191 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 123 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [52.31 INFO] (SDKD log:137)  [52.32 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.32 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.33 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126  [52.34 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.35 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23 [52.35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@112.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.37 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127  [52.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24 [52.40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.40 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.41 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.42 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128  [52.45 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31 [52.46 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.53 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33 [52.53 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.55 INFO] (SDKD log:137) +++ Received exception of ID 0 [52.55 INFO] (SDKD log:137)  [52.56 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.63 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35 [52.63 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.67 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.68 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37 [52.68 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39 [52.77 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.05 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [53.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41 [53.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.15 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [53.15 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43 [53.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.17 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 3 [53.18 INFO] (SDKD log:137)  at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 75 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 40 at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 82 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 282 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 191 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 123 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [53.18 INFO] (SDKD log:137)  [53.19 INFO] (SDKD log:137) Unhandled Exception: Couchbase.Configuration.ConfigException: Could not bootstrap default. See log for details. [65.28 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [65.29 INFO] (SDKD log:137)  at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 210 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 82 at Sdkd.Couchbase.ClientFactory.FindExistingClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 55 at Sdkd.Couchbase.ClientFactory.CreateClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 262 at SdkdConsole.Daemon.Handle.CreateHandle(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 103 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [65.30 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [65.31 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.57 [65.31 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    java.net.SocketException: Connection reset by peer: socket write error
  6. 0

    call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf ig.ini -I sdkd.args --variants HYBRID -d all:trace [0.30 INFO] (BRun run:436) Initializing S3 [1.12 INFO] (BRun run:441) Initializing history database [1.75 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools ============================================================ Running ReAdd2-HYBRID: 2014-08-26 22:31:28 workload=HybridWorkloadGroup failover/next_action=FO_READD_REBALANCE failover/count=2 failover/next_delay=45 testcase=FailoverScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0198 88\0dcd5a5bc61e625f06a757ab9dfa7db5 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ \ --workload HybridWorkloadGroup --failover-next_action \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45 \ --testcase FailoverScenario -C share\rexec --rexec_path \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 --cluster_ssh-username=root --cluster_ssh-password=couchbase ..... [123.71 INFO] (Scenario run:86) Starting CHANGE phase [123.71 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135> [123.72 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver [124.10 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 [124.13 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136> [124.14 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver [124.40 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds [125.61 INFO] (SDKD log:137) [125.63 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavail ableException' was thrown. [169.41 INFO] (FailoverAction doNextAction:68) Readding nodes after failover [169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode [169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode [169.42 INFO] (FailoverAction doNextAction:85) Rebalancing nodes after failover [169.42 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/nodes [169.43 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/rebalance [169.45 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [169.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [170.46 TRACE] (CouchbaseAdmin getResponseJson:167) GET .... [196.67 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [196.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [197.59 INFO] (SDKD log:137) [Sdkd.ViewQuery|Error] ... System.NullReferenceException Object reference not set to an instance of a n object. at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Views\ViewComman d.cs:line 268 [197.61 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [197.61 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace\.net-sdkdclient-n g2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 210 at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace\.net -sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, ISer ver server) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs: line 179 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2- andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net \src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin e 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs :line 175 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo lean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre serveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [197.64 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [197.66 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73 [197.66 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":103,"ReqID":73,"CommandData":{}} [197.66 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] [197.68 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [197.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39% [198.80 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re Add2-HYBRID/08-26-14/019888/0dcd5a5bc61e625f06a757ab9dfa7db5.zip [198.83 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    java.net.SocketException: Connection reset by peer: socket write error

    6 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.net.SocketException

      Connection reset by peer: socket write error

      at java.net.SocketOutputStream.socketWrite0()
    2. Java RT
      BufferedWriter.flush
      1. java.net.SocketOutputStream.socketWrite0(Native Method)[na:1.6.0_16]
      2. java.net.SocketOutputStream.socketWrite(Unknown Source)[na:1.6.0_16]
      3. java.net.SocketOutputStream.write(Unknown Source)[na:1.6.0_16]
      4. sun.nio.cs.StreamEncoder.writeBytes(Unknown Source)[na:1.6.0_16]
      5. sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source)[na:1.6.0_16]
      6. sun.nio.cs.StreamEncoder.implFlush(Unknown Source)[na:1.6.0_16]
      7. sun.nio.cs.StreamEncoder.flush(Unknown Source)[na:1.6.0_16]
      8. java.io.OutputStreamWriter.flush(Unknown Source)[na:1.6.0_16]
      9. java.io.BufferedWriter.flush(Unknown Source)[na:1.6.0_16]
      9 frames
    3. com.couchbase.sdkdclient
      BRun.main
      1. com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188)[sdkdclient-1.0-SNAPSHOT.jar:na]
      2. com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254)[sdkdclient-1.0-SNAPSHOT.jar:na]
      3. com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26)[sdkdclient-1.0-SNAPSHOT.jar:na]
      4. com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118)[sdkdclient-1.0-SNAPSHOT.jar:na]
      5. com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117)[sdkdclient-1.0-SNAPSHOT.jar:na]
      6. com.couchbase.sdkdclient.stester.STester.close(STester.java:167)[sdkdclient-1.0-SNAPSHOT.jar:na]
      7. com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287)[sdkdclient-1.0-SNAPSHOT.jar:na]
      8. com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394)[sdkdclient-1.0-SNAPSHOT.jar:na]
      9. com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478)[sdkdclient-1.0-SNAPSHOT.jar:na]
      10. com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510)[sdkdclient-1.0-SNAPSHOT.jar:na]
      10 frames