com.couchbase.sdkdclient.context.HarnessException: Driver crashed

Couchbase JIRA | Andrei Baranouski | 2 years ago
tip
Your exception is missing from the Samebug knowledge base.
Here are the best solutions we found on the Internet.
Click on the to mark the helpful solution and get rewards for you help.
  1. 0

    very often in different tests I see the following exception http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/38/consoleFull Running SvcRestart-HYBRID: 2014-07-25 05:20:15 .............. [365.86 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object. [366.20 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [366.20 INFO] (SDKD log:137)  at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 51 at Couchbase.Configuration.ConfigContextBase..ctor(IBucketConfig bucketConfig, ClientConfiguration clientConfig, Func`2 ioStrategyFactory, Func`3 connectionPoolFactory, Func`5 saslFactory, IByteConverter converter, ITypeSerializer serializer) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 51 at Couchbase.Configuration.CouchbaseConfigContext..ctor(IBucketConfig bucketConfig, ClientConfiguration clientConfig, Func`2 ioStrategyFactory, Func`3 connectionPoolFactory, Func`5 saslFactory, IByteConverter converter, ITypeSerializer serializer) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 22 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 63 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 44 at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 155 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 82 at Sdkd.Couchbase.ClientFactory.FindExistingClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 55 at Sdkd.Couchbase.ClientFactory.CreateClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 262 at SdkdConsole.Daemon.Handle.CreateHandle(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 103 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [366.20 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [366.21 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@203.259 [366.21 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] client logs http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/38/artifact/log.txt Jeff, believe that this is a problem on client side?

    Couchbase JIRA | 3 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  2. 0

    git log commit 86219ee99ccafdd00c98d577c9bc2b97efef35bb Author: jeffrymorris <jeffrymorris@gmail.com> Date: Wed Aug 20 17:44:45 2014 -0700 NCBC-592: AuthenticationException in CouchbaseBucket.SendWithRetry Running Rb2Out-KV: 2014-08-20 23:33:57 ..... [25.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.33 [25.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@121.43 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=10, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [25.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET  [25.06 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds [25.06 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [25.06 INFO] (SDKD log:137)  "DSType": "DSTYPE_SEEDED", [25.06 INFO] (SDKD log:137)  "DS": { [25.07 INFO] (SDKD log:137)  "KSeed": "SimpleKey", [25.07 INFO] (SDKD log:137)  "VSeed": "SimpleValue", [25.07 INFO] (SDKD log:137)  "Count": 10, [25.07 INFO] (SDKD log:137)  "Continuous": true, [25.07 INFO] (SDKD log:137)  "VSize": 128, [25.07 INFO] (SDKD log:137)  "Repeat": "REP", [25.07 INFO] (SDKD log:137)  "KSize": 32 [25.07 INFO] (SDKD log:137)  }, [25.07 INFO] (SDKD log:137)  "Options": { [25.07 INFO] (SDKD log:137)  "DelayMax": 10, [25.07 INFO] (SDKD log:137)  "DelayMin": 1, [25.07 INFO] (SDKD log:137)  "IterWait": 1, [25.07 INFO] (SDKD log:137)  "TimeRes": 1 [25.09 INFO] (SDKD log:137)  } [25.09 INFO] (SDKD log:137) }  [55.06 INFO] (Scenario run:86) Starting CHANGE phase [55.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [56.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [57.12 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [58.13 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4% [59.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% ................................. [99.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 94% [100.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96% [101.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 98% [102.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [103.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [104.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [105.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [106.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [107.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [108.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [109.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [110.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [111.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [112.75 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409  [112.78 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [112.79 INFO] (Scenario run:97) CHANGE phase done [112.79 INFO] (Scenario run:103) Starting REBOUND for 90 seconds [113.23 INFO] (SDKD log:137)  [113.26 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavailableException' was thrown. [125.87 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [125.88 INFO] (SDKD log:137)  at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 212 at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 179 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [125.89 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [125.89 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.44 [125.90 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]

    Couchbase JIRA | 3 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  3. 0

    I took a few commits ahead from gerrit commit 2c00a9d3a397f5c1eddc56238cf4a19e6e6618dd Author: jeffrymorris <jeffrymorris@gmail.com> Date: Mon Aug 11 18:41:08 2014 -0700 Add config files to tests Change-Id: Ie093eac1da4ffc742796ffd4df8a9c63cc1c9c2f c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload [0.32 INFO] (BRun run:435) Initializing history database ============================================================ Running Rb2Out-HYBRID rebalance/mode=out workload=HybridWorkloadGroup rebalance/count=2 testcase=RebalanceScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-16-14\056660\742db0c42511883a3b8960d57d363be0 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ \ --rebalance-mode out --workload HybridWorkloadGroup \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147 --cluster_ssh-username=root --cluster_ssh-password=couchbase ---------------------------------------->8 [1.52 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution [1.93 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds. [1.95 INFO] (RunContext run:124) Starting cluster and driver [1.96 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now [1.96 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe' [1.96 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations.. [2.30 INFO] (SDKD log:137) --- Logging Self-Test --- [2.31 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message [2.31 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message [2.32 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message [2.32 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message [2.32 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675 [2.48 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0 [2.49 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [2.78 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}} [4.34 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147> [4.43 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa [4.43 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.43 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa [4.43 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.94 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091 [4.95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091 [4.95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091 [4.97 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146 [4.97 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144 [4.97 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091 [4.98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145 [4.98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147 [5.18 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@7b0bbd42 [5.19 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@10d4f27 [5.19 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@b22c9df [5.20 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@74bfd10a [5.20 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@6ae2d0b2 [17.28 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091 [24.00 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091 [36.81 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091 [43.42 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance [43.46 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [44.47 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [44.47 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups [44.48 INFO] (CBCluster setupMainBucket:209) Creating bucket default [44.49 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted [54.57 INFO] (CBCluster waitForBucketReady:203) Bucket creation done [54.62 INFO] (RunContext run:143) Driver and cluster initialized [54.65 INFO] (RunContext call:167) Running scenario.. [54.66 INFO] (Scenario run:72) Starting RAMP phase [54.66 INFO] (Workload setupDesign:63) Creating design test_design [54.93 INFO] (Workload setupDesign:80) Design creation done [54.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [54.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [54.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101 [54.96 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [54.97 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net [54.97 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e [55.00 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net [55.01 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821 [55.42 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1 [55.42 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12 }, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [55.43 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD [55.43 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [55.43 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [55.43 INFO] (SDKD log:137) "DS": { [55.43 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed", [55.44 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed", [55.44 INFO] (SDKD log:137) "Count": 1000, [55.44 INFO] (SDKD log:137) "Continuous": false, [55.44 INFO] (SDKD log:137) "VSize": 512, [55.44 INFO] (SDKD log:137) "Repeat": "rep", [55.44 INFO] (SDKD log:137) "KSize": 12 [55.45 INFO] (SDKD log:137) }, [55.45 INFO] (SDKD log:137) "Schema": { [55.45 INFO] (SDKD log:137) "InflateContent": "meh", [55.45 INFO] (SDKD log:137) "InflateLevel": 40 [55.45 INFO] (SDKD log:137) }, [55.46 INFO] (SDKD log:137) "Options": { [55.46 INFO] (SDKD log:137) "DelayMax": 0, [55.46 INFO] (SDKD log:137) "DelayMin": 0, [55.46 INFO] (SDKD log:137) "IterWait": 1, [55.46 INFO] (SDKD log:137) "TimeRes": 0 [55.46 INFO] (SDKD log:137) } [55.47 INFO] (SDKD log:137) } [56.75 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {} [56.76 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2 [56.76 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3 [56.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [56.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [56.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [56.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [56.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102 [56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [56.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [56.79 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4 [56.79 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12 }, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [56.79 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD [56.80 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [56.80 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [56.80 INFO] (SDKD log:137) "DS": { [56.81 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed", [56.81 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed", [56.81 INFO] (SDKD log:137) "Count": 1000, [56.81 INFO] (SDKD log:137) "Continuous": false, [56.81 INFO] (SDKD log:137) "VSize": 512, [56.82 INFO] (SDKD log:137) "Repeat": "rep", [56.82 INFO] (SDKD log:137) "KSize": 12 [56.82 INFO] (SDKD log:137) }, [56.82 INFO] (SDKD log:137) "Schema": { [56.82 INFO] (SDKD log:137) "InflateContent": "meh", [56.83 INFO] (SDKD log:137) "InflateLevel": 40 [56.83 INFO] (SDKD log:137) }, [56.83 INFO] (SDKD log:137) "Options": { [56.83 INFO] (SDKD log:137) "DelayMax": 0, [56.83 INFO] (SDKD log:137) "DelayMin": 0, [56.84 INFO] (SDKD log:137) "IterWait": 1, [56.84 INFO] (SDKD log:137) "TimeRes": 0 [56.84 INFO] (SDKD log:137) } [56.84 INFO] (SDKD log:137) } [58.30 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {} [58.31 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@102.5 [58.31 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6 [58.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [58.31 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [58.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [58.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [58.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103 [58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7 [58.34 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCou nt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [58.35 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.35 INFO] (SDKD log:137) "DS": {}, [58.36 INFO] (SDKD log:137) "ViewParameters": {}, [58.36 INFO] (SDKD log:137) "DesignName": "test_design", [58.36 INFO] (SDKD log:137) "Options": { [58.36 INFO] (SDKD log:137) "DelayMax": 10, [58.36 INFO] (SDKD log:137) "DelayMin": 1, [58.37 INFO] (SDKD log:137) "IterWait": 1, [58.37 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.37 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.37 INFO] (SDKD log:137) "TimeRes": 1 [58.38 INFO] (SDKD log:137) }, [58.38 INFO] (SDKD log:137) "ViewName": "test_view" [58.38 INFO] (SDKD log:137) } [58.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104 [58.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9 [58.40 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.41 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.41 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.41 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.41 INFO] (SDKD log:137) "DS": {}, [58.42 INFO] (SDKD log:137) "ViewParameters": {}, [58.42 INFO] (SDKD log:137) "DesignName": "test_design", [58.42 INFO] (SDKD log:137) "Options": { [58.42 INFO] (SDKD log:137) "DelayMax": 10, [58.42 INFO] (SDKD log:137) "DelayMin": 1, [58.42 INFO] (SDKD log:137) "IterWait": 1, [58.43 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.43 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.43 INFO] (SDKD log:137) "TimeRes": 1 [58.43 INFO] (SDKD log:137) }, [58.43 INFO] (SDKD log:137) "ViewName": "test_view" [58.44 INFO] (SDKD log:137) } [58.44 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.44 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105 [58.44 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.57 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11 [58.58 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.58 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.58 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.59 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.59 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.59 INFO] (SDKD log:137) "DS": {}, [58.59 INFO] (SDKD log:137) "ViewParameters": {}, [58.59 INFO] (SDKD log:137) "DesignName": "test_design", [58.60 INFO] (SDKD log:137) "Options": { [58.60 INFO] (SDKD log:137) "DelayMax": 10, [58.60 INFO] (SDKD log:137) "DelayMin": 1, [58.60 INFO] (SDKD log:137) "IterWait": 1, [58.60 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.60 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.60 INFO] (SDKD log:137) "TimeRes": 1 [58.61 INFO] (SDKD log:137) }, [58.61 INFO] (SDKD log:137) "ViewName": "test_view" [58.61 INFO] (SDKD log:137) } [58.61 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106 [58.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13 [58.73 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.74 INFO] (SDKD log:137) "DS": {}, [58.74 INFO] (SDKD log:137) "ViewParameters": {}, [58.74 INFO] (SDKD log:137) "DesignName": "test_design", [58.74 INFO] (SDKD log:137) "Options": { [58.75 INFO] (SDKD log:137) "DelayMax": 10, [58.75 INFO] (SDKD log:137) "DelayMin": 1, [58.75 INFO] (SDKD log:137) "IterWait": 1, [58.75 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.75 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.76 INFO] (SDKD log:137) "TimeRes": 1 [58.76 INFO] (SDKD log:137) }, [58.76 INFO] (SDKD log:137) "ViewName": "test_view" [58.76 INFO] (SDKD log:137) } [58.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107 [58.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15 [58.77 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.78 INFO] (Workload call:112) Preloading items into cluster before workload [58.78 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.78 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.79 INFO] (SDKD log:137) "DS": {}, [58.79 INFO] (SDKD log:137) "ViewParameters": {}, [58.79 INFO] (SDKD log:137) "DesignName": "test_design", [58.79 INFO] (SDKD log:137) "Options": { [58.79 INFO] (SDKD log:137) "DelayMax": 10, [58.80 INFO] (SDKD log:137) "DelayMin": 1, [58.80 INFO] (SDKD log:137) "IterWait": 1, [58.80 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.80 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.80 INFO] (SDKD log:137) "TimeRes": 1 [58.81 INFO] (SDKD log:137) }, [58.81 INFO] (SDKD log:137) "ViewName": "test_view" [58.81 INFO] (SDKD log:137) } [58.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108 [58.81 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.82 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17 [58.82 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}} [58.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [58.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.83 INFO] (SDKD log:137) "DS": { [58.83 INFO] (SDKD log:137) "KSeed": "SimpleKey", [58.84 INFO] (SDKD log:137) "VSeed": "SimpleValue", [58.84 INFO] (SDKD log:137) "Count": 15000, [58.84 INFO] (SDKD log:137) "Continuous": false, [58.84 INFO] (SDKD log:137) "VSize": 128, [58.84 INFO] (SDKD log:137) "Repeat": "REP", [58.85 INFO] (SDKD log:137) "KSize": 32 [58.85 INFO] (SDKD log:137) }, [58.85 INFO] (SDKD log:137) "Options": { [58.85 INFO] (SDKD log:137) "DelayMax": 0, [58.85 INFO] (SDKD log:137) "DelayMin": 0, [58.85 INFO] (SDKD log:137) "IterWait": 100, [58.86 INFO] (SDKD log:137) "TimeRes": 0 [58.86 INFO] (SDKD log:137) } [58.86 INFO] (SDKD log:137) } [249.74 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 4003, [MEMD:MEMD_EVBUCKET]: 10997} [249.74 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19 [249.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [249.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [249.74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [249.74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [249.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.77 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109 [249.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114 [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116 [249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115 [249.80 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20 [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.81 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@109.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110 [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113 [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111 [249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117 [249.82 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112 [249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118 [249.82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.82 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.83 INFO] (SDKD log:137) "DS": { [249.83 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.83 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.83 INFO] (SDKD log:137) "Count": 15000, [249.84 INFO] (SDKD log:137) "Continuous": true, [249.84 INFO] (SDKD log:137) "VSize": 128, [249.84 INFO] (SDKD log:137) "Repeat": "REP", [249.84 INFO] (SDKD log:137) "KSize": 32 [249.84 INFO] (SDKD log:137) }, [249.84 INFO] (SDKD log:137) "Options": { [249.84 INFO] (SDKD log:137) "DelayMax": 10, [249.85 INFO] (SDKD log:137) "DelayMin": 1, [249.85 INFO] (SDKD log:137) "IterWait": 1, [249.85 INFO] (SDKD log:137) "TimeRes": 1 [249.85 INFO] (SDKD log:137) } [249.86 INFO] (SDKD log:137) } [249.86 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25 [249.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.87 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.87 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119 [249.87 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.87 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.88 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.88 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.88 INFO] (SDKD log:137) "DS": { [249.89 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.89 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.89 INFO] (SDKD log:137) "Count": 15000, [249.89 INFO] (SDKD log:137) "Continuous": true, [249.89 INFO] (SDKD log:137) "VSize": 128, [249.90 INFO] (SDKD log:137) "Repeat": "REP", [249.90 INFO] (SDKD log:137) "KSize": 32 [249.90 INFO] (SDKD log:137) }, [249.90 INFO] (SDKD log:137) "Options": { [249.90 INFO] (SDKD log:137) "DelayMax": 10, [249.90 INFO] (SDKD log:137) "DelayMin": 1, [249.90 INFO] (SDKD log:137) "IterWait": 1, [249.91 INFO] (SDKD log:137) "TimeRes": 1 [249.91 INFO] (SDKD log:137) } [249.91 INFO] (SDKD log:137) } [249.91 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27 [249.92 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120 [249.92 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@116.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.92 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.93 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.93 INFO] (SDKD log:137) "DS": { [249.94 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.94 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.94 INFO] (SDKD log:137) "Count": 15000, [249.94 INFO] (SDKD log:137) "Continuous": true, [249.94 INFO] (SDKD log:137) "VSize": 128, [249.94 INFO] (SDKD log:137) "Repeat": "REP", [249.95 INFO] (SDKD log:137) "KSize": 32 [249.95 INFO] (SDKD log:137) }, [249.95 INFO] (SDKD log:137) "Options": { [249.95 INFO] (SDKD log:137) "DelayMax": 10, [249.95 INFO] (SDKD log:137) "DelayMin": 1, [249.96 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26 [249.96 INFO] (SDKD log:137) "IterWait": 1, [249.96 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@115.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.96 INFO] (SDKD log:137) "TimeRes": 1 [249.96 INFO] (SDKD log:137) } [249.96 INFO] (SDKD log:137) } [249.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121 [249.98 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.98 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.98 INFO] (SDKD log:137) "DS": { [249.98 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.99 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.99 INFO] (SDKD log:137) "Count": 15000, [249.99 INFO] (SDKD log:137) "Continuous": true, [249.99 INFO] (SDKD log:137) "VSize": 128, [249.99 INFO] (SDKD log:137) "Repeat": "REP", [249.99 INFO] (SDKD log:137) "KSize": 32 [250.00 INFO] (SDKD log:137) }, [250.00 INFO] (SDKD log:137) "Options": { [250.00 INFO] (SDKD log:137) "DelayMax": 10, [250.01 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24 [250.01 INFO] (SDKD log:137) "DelayMin": 1, [250.01 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.01 INFO] (SDKD log:137) "IterWait": 1, [250.02 INFO] (SDKD log:137) "TimeRes": 1 [250.02 INFO] (SDKD log:137) } [250.02 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.02 INFO] (SDKD log:137) } [250.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122 [250.03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.03 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.03 INFO] (SDKD log:137) "DS": { [250.04 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.04 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.04 INFO] (SDKD log:137) "Count": 15000, [250.04 INFO] (SDKD log:137) "Continuous": true, [250.04 INFO] (SDKD log:137) "VSize": 128, [250.04 INFO] (SDKD log:137) "Repeat": "REP", [250.05 INFO] (SDKD log:137) "KSize": 32 [250.05 INFO] (SDKD log:137) }, [250.05 INFO] (SDKD log:137) "Options": { [250.05 INFO] (SDKD log:137) "DelayMax": 10, [250.05 INFO] (SDKD log:137) "DelayMin": 1, [250.05 INFO] (SDKD log:137) "IterWait": 1, [250.06 INFO] (SDKD log:137) "TimeRes": 1 [250.07 INFO] (SDKD log:137) } [250.07 INFO] (SDKD log:137) } [250.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.08 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21 [250.08 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@110.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123 [250.09 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.09 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.10 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.11 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124 [250.12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.12 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.13 INFO] (SDKD log:137) "DS": { [250.13 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [250.14 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.15 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.15 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [250.15 INFO] (SDKD log:137) "Count": 15000, [250.15 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [250.15 INFO] (SDKD log:137) "Continuous": true, [250.15 INFO] (SDKD log:137) "VSize": 128, [250.16 INFO] (SDKD log:137) "Repeat": "REP", [250.16 INFO] (SDKD log:137) "KSize": 32 [250.16 INFO] (SDKD log:137) }, [250.16 INFO] (SDKD log:137) "Options": { [250.16 INFO] (SDKD log:137) "DelayMax": 10, [250.17 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1 [250.17 INFO] (SDKD log:137) "DelayMin": 1, [250.17 INFO] (SDKD log:137) "IterWait": 1, [250.18 INFO] (SDKD log:137) at System.Net.Sockets.Socket.get_RemoteEndPoint() at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line 38 at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString() at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer) at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer) [250.18 INFO] (SDKD log:137) "TimeRes": 1 [250.18 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [250.19 INFO] (SDKD log:137) } [250.19 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [250.19 INFO] (SDKD log:137) } [250.19 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 [250.20 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.21 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [250.21 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22 [250.21 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@111.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.22 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.23 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.23 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.23 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.23 INFO] (SDKD log:137) "DS": { [250.24 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.24 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.24 INFO] (SDKD log:137) "Count": 15000, [250.24 INFO] (SDKD log:137) "Continuous": true, [250.25 INFO] (SDKD log:137) "VSize": 128, [250.25 INFO] (SDKD log:137) "Repeat": "REP", [250.25 INFO] (SDKD log:137) "KSize": 32 [250.25 INFO] (SDKD log:137) }, [250.25 INFO] (SDKD log:137) "Options": { [250.27 INFO] (SDKD log:137) "DelayMax": 10, [250.28 INFO] (SDKD log:137) "DelayMin": 1, [250.28 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28 [250.28 INFO] (SDKD log:137) "IterWait": 1, [250.29 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@117.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.29 INFO] (SDKD log:137) "TimeRes": 1 [250.29 INFO] (SDKD log:137) } [250.29 INFO] (SDKD log:137) } [250.30 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.30 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.31 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125 [250.31 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.32 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.33 INFO] (SDKD log:137) "DS": { [250.33 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.33 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.33 INFO] (SDKD log:137) "Count": 15000, [250.34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23 [250.34 INFO] (SDKD log:137) "Continuous": true, [250.35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@112.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.35 INFO] (SDKD log:137) "VSize": 128, [250.35 INFO] (SDKD log:137) "Repeat": "REP", [250.35 INFO] (SDKD log:137) "KSize": 32 [250.35 INFO] (SDKD log:137) }, [250.36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.36 INFO] (SDKD log:137) "Options": { [250.36 INFO] (SDKD log:137) "DelayMax": 10, [250.36 INFO] (SDKD log:137) "DelayMin": 1, [250.37 INFO] (SDKD log:137) "IterWait": 1, [250.37 INFO] (SDKD log:137) "TimeRes": 1 [250.37 INFO] (SDKD log:137) } [250.37 INFO] (SDKD log:137) } [250.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126 [250.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.38 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.39 INFO] (SDKD log:137) "DS": { [250.39 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.40 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29 [250.40 INFO] (SDKD log:137) "Count": 15000, [250.40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@118.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.41 INFO] (SDKD log:137) "Continuous": true, [250.42 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.42 INFO] (SDKD log:137) "VSize": 128, [250.43 INFO] (SDKD log:137) "Repeat": "REP", [250.43 INFO] (SDKD log:137) "KSize": 32 [250.43 INFO] (SDKD log:137) }, [250.43 INFO] (SDKD log:137) "Options": { [250.44 INFO] (SDKD log:137) "DelayMax": 10, [250.44 INFO] (SDKD log:137) "DelayMin": 1, [250.44 INFO] (SDKD log:137) "IterWait": 1, [250.44 INFO] (SDKD log:137) "TimeRes": 1 [250.44 INFO] (SDKD log:137) } [250.45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31 [250.45 INFO] (SDKD log:137) } [250.45 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127 [250.45 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.46 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.46 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.46 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.47 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.47 INFO] (SDKD log:137) "DS": { [250.47 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.48 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.48 INFO] (SDKD log:137) "Count": 15000, [250.48 INFO] (SDKD log:137) "Continuous": true, [250.48 INFO] (SDKD log:137) "VSize": 128, [250.49 INFO] (SDKD log:137) "Repeat": "REP", [250.49 INFO] (SDKD log:137) "KSize": 32 [250.49 INFO] (SDKD log:137) }, [250.49 INFO] (SDKD log:137) "Options": { [250.50 INFO] (SDKD log:137) "DelayMax": 10, [250.50 INFO] (SDKD log:137) "DelayMin": 1, [250.50 INFO] (SDKD log:137) "IterWait": 1, [250.50 INFO] (SDKD log:137) "TimeRes": 1 [250.51 INFO] (SDKD log:137) } [250.51 INFO] (SDKD log:137) } [250.51 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128 [250.51 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33 [250.52 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35 [250.56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.56 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.56 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.56 INFO] (SDKD log:137) "DS": { [250.57 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.57 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.57 INFO] (SDKD log:137) "Count": 15000, [250.58 INFO] (SDKD log:137) "Continuous": true, [250.58 INFO] (SDKD log:137) "VSize": 128, [250.58 INFO] (SDKD log:137) "Repeat": "REP", [250.58 INFO] (SDKD log:137) "KSize": 32 [250.59 INFO] (SDKD log:137) }, [250.59 INFO] (SDKD log:137) "Options": { [250.59 INFO] (SDKD log:137) "DelayMax": 10, [250.59 INFO] (SDKD log:137) +++ Received exception of ID 0 [250.60 INFO] (SDKD log:137) "DelayMin": 1, [250.60 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [250.60 INFO] (SDKD log:137) "IterWait": 1, [250.60 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [250.60 INFO] (SDKD log:137) "TimeRes": 1 [250.60 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [250.60 INFO] (SDKD log:137) } [250.61 INFO] (SDKD log:137) } [250.61 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.61 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.62 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.62 INFO] (SDKD log:137) "DS": { [250.63 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.63 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.63 INFO] (SDKD log:137) "Count": 15000, [250.63 INFO] (SDKD log:137) "Continuous": true, [250.63 INFO] (SDKD log:137) "VSize": 128, [250.63 INFO] (SDKD log:137) "Repeat": "REP", [250.63 INFO] (SDKD log:137) "KSize": 32 [250.63 INFO] (SDKD log:137) }, [250.64 INFO] (SDKD log:137) "Options": { [250.64 INFO] (SDKD log:137) "DelayMax": 10, [250.64 INFO] (SDKD log:137) "DelayMin": 1, [250.64 INFO] (SDKD log:137) "IterWait": 1, [250.64 INFO] (SDKD log:137) "TimeRes": 1 [250.65 INFO] (SDKD log:137) } [250.65 INFO] (SDKD log:137) } [250.65 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.65 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.66 INFO] (SDKD log:137) "DS": { [250.66 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.66 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.66 INFO] (SDKD log:137) "Count": 15000, [250.66 INFO] (SDKD log:137) "Continuous": true, [250.66 INFO] (SDKD log:137) "VSize": 128, [250.67 INFO] (SDKD log:137) "Repeat": "REP", [250.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37 [250.67 INFO] (SDKD log:137) "KSize": 32 [250.68 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.68 INFO] (SDKD log:137) }, [250.68 INFO] (SDKD log:137) "Options": { [250.68 INFO] (SDKD log:137) "DelayMax": 10, [250.68 INFO] (SDKD log:137) "DelayMin": 1, [250.69 INFO] (SDKD log:137) "IterWait": 1, [250.69 INFO] (SDKD log:137) "TimeRes": 1 [250.69 INFO] (SDKD log:137) } [250.70 INFO] (SDKD log:137) } [250.70 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.70 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.71 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.71 INFO] (SDKD log:137) "DS": { [250.71 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39 [250.71 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.72 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.72 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.73 INFO] (SDKD log:137) "Count": 15000, [250.73 INFO] (SDKD log:137) "Continuous": true, [250.73 INFO] (SDKD log:137) "VSize": 128, [250.73 INFO] (SDKD log:137) "Repeat": "REP", [250.73 INFO] (SDKD log:137) "KSize": 32 [250.73 INFO] (SDKD log:137) }, [250.74 INFO] (SDKD log:137) "Options": { [250.74 INFO] (SDKD log:137) "DelayMax": 10, [250.74 INFO] (SDKD log:137) "DelayMin": 1, [250.75 INFO] (SDKD log:137) "IterWait": 1, [250.75 INFO] (SDKD log:137) "TimeRes": 1 [250.75 INFO] (SDKD log:137) } [250.76 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41 [250.76 INFO] (SDKD log:137) } [250.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.76 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.77 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.77 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.77 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.78 INFO] (SDKD log:137) "DS": { [250.78 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.78 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.79 INFO] (SDKD log:137) "Count": 15000, [250.79 INFO] (SDKD log:137) "Continuous": true, [250.79 INFO] (SDKD log:137) "VSize": 128, [250.79 INFO] (SDKD log:137) "Repeat": "REP", [250.79 INFO] (SDKD log:137) "KSize": 32 [250.79 INFO] (SDKD log:137) }, [250.79 INFO] (SDKD log:137) "Options": { [250.80 INFO] (SDKD log:137) "DelayMax": 10, [250.80 INFO] (SDKD log:137) "DelayMin": 1, [250.80 INFO] (SDKD log:137) "IterWait": 1, [250.81 INFO] (SDKD log:137) "TimeRes": 1 [250.81 INFO] (SDKD log:137) } [250.81 INFO] (SDKD log:137) } [250.81 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43 [250.82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.83 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.84 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.84 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@126.45 [250.85 INFO] (SDKD log:137) "DS": { [250.85 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.85 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@126.57 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.86 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.86 INFO] (SDKD log:137) "Count": 15000, [250.87 INFO] (SDKD log:137) "Continuous": true, [250.87 INFO] (SDKD log:137) "VSize": 128, [250.87 INFO] (SDKD log:137) "Repeat": "REP", [250.87 INFO] (SDKD log:137) "KSize": 32 [250.87 INFO] (SDKD log:137) }, [250.87 INFO] (SDKD log:137) "Options": { [250.88 INFO] (SDKD log:137) "DelayMax": 10, [250.88 INFO] (SDKD log:137) "DelayMin": 1, [250.88 INFO] (SDKD log:137) "IterWait": 1, [250.88 INFO] (SDKD log:137) "TimeRes": 1 [250.88 INFO] (SDKD log:137) } [250.88 INFO] (SDKD log:137) } [250.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.89 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.89 INFO] (SDKD log:137) "DS": { [250.90 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.90 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.90 INFO] (SDKD log:137) "Count": 15000, [250.90 INFO] (SDKD log:137) "Continuous": true, [250.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@127.47 [250.91 INFO] (SDKD log:137) "VSize": 128, [250.91 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@127.58 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.91 INFO] (SDKD log:137) "Repeat": "REP", [250.91 INFO] (SDKD log:137) "KSize": 32 [250.92 INFO] (SDKD log:137) }, [250.92 INFO] (SDKD log:137) "Options": { [250.92 INFO] (SDKD log:137) "DelayMax": 10, [250.93 INFO] (SDKD log:137) "DelayMin": 1, [250.93 INFO] (SDKD log:137) "IterWait": 1, [250.93 INFO] (SDKD log:137) "TimeRes": 1 [250.93 INFO] (SDKD log:137) } [250.93 INFO] (SDKD log:137) } [250.93 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.94 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.94 INFO] (SDKD log:137) "DS": { [250.94 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.95 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.95 INFO] (SDKD log:137) "Count": 15000, [250.96 INFO] (SDKD log:137) "Continuous": true, [250.96 INFO] (SDKD log:137) "VSize": 128, [250.96 INFO] (SDKD log:137) "Repeat": "REP", [250.96 INFO] (SDKD log:137) "KSize": 32 [250.96 INFO] (SDKD log:137) }, [250.96 INFO] (SDKD log:137) "Options": { [250.96 INFO] (SDKD log:137) "DelayMax": 10, [250.96 INFO] (SDKD log:137) "DelayMin": 1, [250.96 INFO] (SDKD log:137) "IterWait": 1, [250.96 INFO] (SDKD log:137) "TimeRes": 1 [250.96 INFO] (SDKD log:137) } [250.97 INFO] (SDKD log:137) } [250.97 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.97 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.97 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.98 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.98 INFO] (SDKD log:137) "DS": { [250.99 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.99 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.99 INFO] (SDKD log:137) "Count": 15000, [250.99 INFO] (SDKD log:137) "Continuous": true, [250.99 INFO] (SDKD log:137) "VSize": 128, [250.99 INFO] (SDKD log:137) "Repeat": "REP", [250.99 INFO] (SDKD log:137) "KSize": 32 [250.99 INFO] (SDKD log:137) }, [251.00 INFO] (SDKD log:137) "Options": { [251.00 INFO] (SDKD log:137) "DelayMax": 10, [251.00 INFO] (SDKD log:137) "DelayMin": 1, [251.00 INFO] (SDKD log:137) "IterWait": 1, [251.01 INFO] (SDKD log:137) "TimeRes": 1 [251.01 INFO] (SDKD log:137) } [251.02 INFO] (SDKD log:137) } [251.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@128.49 [251.06 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@128.59 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [251.07 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@129.60 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [251.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [251.09 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [251.09 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [251.09 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [251.09 INFO] (SDKD log:137) +++ Received exception of ID 0 [251.09 INFO] (SDKD log:137) "DS": { [251.09 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [251.09 INFO] (SDKD log:137) "KSeed": "SimpleKey", [251.09 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [251.10 INFO] (SDKD log:137) "VSeed": "SimpleValue", [251.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [251.10 INFO] (SDKD log:137) "Count": 15000, [251.10 INFO] (SDKD log:137) +++ Received exception of ID 0 [251.10 INFO] (SDKD log:137) "Continuous": true, [251.11 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [251.11 INFO] (SDKD log:137) "VSize": 128, [251.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [251.11 INFO] (SDKD log:137) "Repeat": "REP", [251.12 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [251.13 INFO] (SDKD log:137) "KSize": 32 [251.13 INFO] (SDKD log:137) }, [251.13 INFO] (SDKD log:137) "Options": { [251.13 INFO] (SDKD log:137) "DelayMax": 10, [251.14 INFO] (SDKD log:137) "DelayMin": 1, [251.14 INFO] (SDKD log:137) "IterWait": 1, [251.14 INFO] (SDKD log:137) "TimeRes": 1 [251.15 INFO] (SDKD log:137) } [251.15 INFO] (SDKD log:137) } [251.15 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 129 [251.15 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [251.17 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@129.60 [251.18 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@129.61 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize= 12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [251.19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD [251.20 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [251.21 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [251.21 INFO] (SDKD log:137) "DS": { [251.21 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed", [251.21 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed", [251.21 INFO] (SDKD log:137) "Count": 1000, [251.22 INFO] (SDKD log:137) "Continuous": false, [251.22 INFO] (SDKD log:137) "VSize": 512, [251.23 INFO] (SDKD log:137) "Repeat": "rep", [251.23 INFO] (SDKD log:137) "KSize": 12 [251.23 INFO] (SDKD log:137) }, [251.23 INFO] (SDKD log:137) "Schema": { [251.24 INFO] (SDKD log:137) "InflateContent": "meh", [251.24 INFO] (SDKD log:137) "InflateLevel": 40 [251.24 INFO] (SDKD log:137) }, [251.25 INFO] (SDKD log:137) "Options": { [251.25 INFO] (SDKD log:137) "DelayMax": 0, [251.25 INFO] (SDKD log:137) "DelayMin": 0, [251.26 INFO] (SDKD log:137) "IterWait": 1, [251.26 INFO] (SDKD log:137) "TimeRes": 0 [251.27 INFO] (SDKD log:137) } [251.27 INFO] (SDKD log:137) } [544.61 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {} [544.61 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@129.61 [544.61 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@129.62 [544.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [544.62 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@130.63 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [544.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [544.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [544.63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [544.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 130 [544.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@130.63 [544.67 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@130.64 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [544.67 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [544.68 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@131.65 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.68 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [544.68 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [544.68 INFO] (SDKD log:137) "DS": {}, [544.68 INFO] (SDKD log:137) "ViewParameters": { [544.69 INFO] (SDKD log:137) "limit": 0 [544.69 INFO] (SDKD log:137) }, [544.69 INFO] (SDKD log:137) "DesignName": "test_design", [544.69 INFO] (SDKD log:137) "Options": { [544.69 INFO] (SDKD log:137) "DelayMax": 10, [544.70 INFO] (SDKD log:137) "DelayMin": 1, [544.70 INFO] (SDKD log:137) "IterWait": 1, [544.70 INFO] (SDKD log:137) "ViewQueryCount": -1, [544.70 INFO] (SDKD log:137) "ViewQueryDelay": 0, [544.70 INFO] (SDKD log:137) "TimeRes": 1 [544.70 INFO] (SDKD log:137) }, [544.70 INFO] (SDKD log:137) "ViewName": "test_view" [544.71 INFO] (SDKD log:137) } [544.71 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.71 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 131 [544.71 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@131.65 [544.73 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@131.66 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [544.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [544.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@132.67 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [544.74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [544.74 INFO] (SDKD log:137) "DS": {}, [544.74 INFO] (SDKD log:137) "ViewParameters": { [544.74 INFO] (SDKD log:137) "limit": 0 [544.75 INFO] (SDKD log:137) }, [544.75 INFO] (SDKD log:137) "DesignName": "test_design", [544.75 INFO] (SDKD log:137) "Options": { [544.75 INFO] (SDKD log:137) "DelayMax": 10, [544.75 INFO] (SDKD log:137) "DelayMin": 1, [544.76 INFO] (SDKD log:137) "IterWait": 1, [544.76 INFO] (SDKD log:137) "ViewQueryCount": -1, [544.76 INFO] (SDKD log:137) "ViewQueryDelay": 0, [544.76 INFO] (SDKD log:137) "TimeRes": 1 [544.76 INFO] (SDKD log:137) }, [544.77 INFO] (SDKD log:137) "ViewName": "test_view" [544.77 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.77 INFO] (SDKD log:137) } [544.78 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.78 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 132 [544.78 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.81 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@132.67 [544.82 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@132.68 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [544.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [544.83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@133.69 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [544.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [544.84 INFO] (SDKD log:137) "DS": {}, [544.84 INFO] (SDKD log:137) "ViewParameters": { [544.85 INFO] (SDKD log:137) "limit": 0 [544.85 INFO] (SDKD log:137) }, [544.85 INFO] (SDKD log:137) "DesignName": "test_design", [544.85 INFO] (SDKD log:137) "Options": { [544.86 INFO] (SDKD log:137) "DelayMax": 10, [544.86 INFO] (SDKD log:137) "DelayMin": 1, [544.86 INFO] (SDKD log:137) "IterWait": 1, [544.86 INFO] (SDKD log:137) "ViewQueryCount": -1, [544.86 INFO] (SDKD log:137) "ViewQueryDelay": 0, [544.86 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.87 INFO] (SDKD log:137) "TimeRes": 1 [544.87 INFO] (SDKD log:137) }, [544.88 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.88 INFO] (SDKD log:137) "ViewName": "test_view" [544.88 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.88 INFO] (SDKD log:137) } [544.88 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 133 [544.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.90 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.91 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.92 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.92 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.93 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.94 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.94 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.94 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.96 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.97 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.97 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.97 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.97 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.00 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.00 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.02 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@133.69 [545.02 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.02 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@133.70 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [545.02 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [545.03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@134.71 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [545.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [545.03 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [545.03 INFO] (SDKD log:137) "DS": {}, [545.03 INFO] (SDKD log:137) "ViewParameters": { [545.03 INFO] (SDKD log:137) "limit": 0 [545.03 INFO] (SDKD log:137) }, [545.04 INFO] (SDKD log:137) "DesignName": "test_design", [545.04 INFO] (SDKD log:137) "Options": { [545.04 INFO] (SDKD log:137) "DelayMax": 10, [545.04 INFO] (SDKD log:137) "DelayMin": 1, [545.04 INFO] (SDKD log:137) "IterWait": 1, [545.04 INFO] (SDKD log:137) "ViewQueryCount": -1, [545.05 INFO] (SDKD log:137) "ViewQueryDelay": 0, [545.05 INFO] (SDKD log:137) "TimeRes": 1 [545.05 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.05 INFO] (SDKD log:137) }, [545.05 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.06 INFO] (SDKD log:137) "ViewName": "test_view" [545.06 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.06 INFO] (SDKD log:137) } [545.06 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.06 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [545.06 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.07 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 134 [545.07 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.07 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [545.07 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.08 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.08 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.08 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.08 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.09 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.09 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.10 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.10 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.11 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@134.71 [545.13 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@134.72 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [545.14 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds [545.14 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [545.15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [545.15 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [545.15 INFO] (SDKD log:137) "DS": {}, [545.15 INFO] (SDKD log:137) "ViewParameters": { [545.15 INFO] (SDKD log:137) "limit": 0 [545.15 INFO] (SDKD log:137) }, [545.16 INFO] (SDKD log:137) "DesignName": "test_design", [545.16 INFO] (SDKD log:137) "Options": { [545.16 INFO] (SDKD log:137) "DelayMax": 10, [545.16 INFO] (SDKD log:137) "DelayMin": 1, [545.16 INFO] (SDKD log:137) "IterWait": 1, [545.16 INFO] (SDKD log:137) "ViewQueryCount": -1, [545.17 INFO] (SDKD log:137) "ViewQueryDelay": 0, [545.17 INFO] (SDKD log:137) "TimeRes": 1 [545.17 INFO] (SDKD log:137) }, [545.17 INFO] (SDKD log:137) "ViewName": "test_view" [545.18 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.18 INFO] (SDKD log:137) } [545.18 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.18 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.18 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [575.14 INFO] (Scenario run:86) Starting CHANGE phase [575.20 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [576.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [577.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [579.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [581.33 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [583.27 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 1% [586.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [587.51 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [588.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3% [589.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3% [590.57 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3% [591.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4% [592.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5% [593.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5% [594.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 6% [595.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% [596.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% [597.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 8% [598.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9% [599.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9% [600.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10% [601.98 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10% [603.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11% [604.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11% [605.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12% [606.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12% [607.16 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13% [608.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13% [609.28 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14% [610.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14% [611.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15% [612.39 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15% [613.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16% [614.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16% [615.43 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17% [616.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18% [617.49 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18% [618.50 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19% [619.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19% [620.53 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20% [621.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20% [622.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21% [623.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21% [624.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22% [625.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22% [626.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 23% [627.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 24% [628.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 24% [629.79 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25% [630.80 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25% [631.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26% [632.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26% [633.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26% [634.87 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 27% [635.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 27% [636.90 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28% [637.91 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28% [638.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29% [639.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29% [640.95 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 30% [641.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31% [643.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31% [643.63 INFO] (SDKD log:137) +++ Received exception of ID 0 [643.63 INFO] (SDKD log:137) [643.66 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Collection was modified; enumeration operation may not execute. [644.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32% [645.04 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32% [646.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32% [647.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33% [648.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33% [649.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 34% [650.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35% [651.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35% [652.18 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36% [653.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36% [654.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [655.23 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [656.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 38% [657.25 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39% [658.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40% [659.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40% [660.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40% [661.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 41% [662.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42% [663.37 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42% [664.38 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43% [665.40 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43% [666.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 44% [667.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45% [668.43 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45% [669.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 46% [670.47 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 47% [671.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 47% [672.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48% [673.54 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48% [674.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49% [675.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49% [676.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50% [677.59 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50% [678.60 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51% [679.60 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51% [680.61 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51% [681.62 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 52% [682.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53% [683.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53% [684.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 54% [685.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 54% [686.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55% [687.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55% [688.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56% [689.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56% [690.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56% [691.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57% [692.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57% [693.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58% [694.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58% [695.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59% [696.70 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [696.71 INFO] (SDKD log:137) at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource) at System.Collections.Generic.List`1.Enumerator.MoveNextRare() at System.Collections.Generic.List`1.Enumerator.MoveNext() at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source) at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 47 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.UpdateConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Co nfiguration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 91 at Couchbase.Core.ClusterManager.NotifyConfigPublished(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 137 at Couchbase.Core.Buckets.CouchbaseBucket.CheckForConfigUpdates[T](IOperationResult`1 operationResult) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.c s:line 234 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couc hbase\Core\Buckets\CouchbaseBucket.cs:line 166 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [696.74 ERROR] (BRun maybePropagate:237) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [696.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59% [696.76 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73 [696.76 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na] c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>

    Couchbase JIRA | 3 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    after some time the tests begin to fall with 2014-07-28 00:59:20,864 [27] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Could not authenticate default using Couchbase.Authentication.SASL.CramMd5Mechanism. Running Rb2Out-HYBRID: 2014-07-28 00:05:18 rebalance/mode=out workload=HybridWorkloadGroup rebalance/count=2 testcase=RebalanceScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-973\Rb2Out-HYBRID\07-28-14\025518\7d8cf558a26ca99fe4dc3028bc6b2ec0 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ --rebalance-mode out --workload HybridWorkloadGroup \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port \ 8675 --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 \ --cluster_ssh-username=root --cluster_ssh-password=couchbase ---------------------------------------->8 [1.47 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution [1.87 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds. [1.88 INFO] (RunContext run:124) Starting cluster and driver [1.89 INFO] (CBCluster startCluster:360) Node http://10.3.121.134:8091 is master now [1.89 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'  [1.89 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations.. [2.26 INFO] (SDKD log:137) --- Logging Self-Test --- [2.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message  [2.27 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675  [2.42 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0 [2.43 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [2.76 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}} [4.02 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.135:8091,ns_1@10.3.121.135> [4.06 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.136:8091,ns_1@10.3.121.136> [4.11 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.3.206:8091,ns_1@10.3.3.206> [4.18 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.18 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.70 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.134:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.3.206:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.135:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.136:8091 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.134 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.3.206 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.136 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.135 [4.94 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@641a034d [4.94 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@19c6866d [4.95 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@51c293f8 [4.95 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@5a9b8ff9 [4.95 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@6e0f0ad [18.52 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.135:8091 [26.45 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.136:8091 [30.42 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.3.206:8091 [35.78 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance [35.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [36.84 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [36.85 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups [36.85 INFO] (CBCluster setupMainBucket:209) Creating bucket default [36.87 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted [42.18 INFO] (CBCluster waitForBucketReady:203) Bucket creation done [42.23 INFO] (RunContext run:143) Driver and cluster initialized [42.26 INFO] (RunContext call:167) Running scenario.. [42.26 INFO] (Scenario run:72) Starting RAMP phase [42.26 INFO] (Workload setupDesign:63) Creating design test_design [42.51 INFO] (Workload setupDesign:80) Design creation done [42.51 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [42.52 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [42.53 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101  [42.54 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [42.56 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net  [42.56 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e  [42.59 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net  [42.59 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821  [42.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1 [42.91 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [43.02 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409  [43.39 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: { [43.39 INFO] (SDKD log:137)  "0": 44, [43.40 INFO] (SDKD log:137)  "264": 956 [43.40 INFO] (SDKD log:137) }  [43.40 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@101.2 => {[OK]: 44, [MEMD:GENERIC]: 956} [43.40 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3 [43.40 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102  [43.41 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.42 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4 [43.42 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [43.82 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: { [43.82 INFO] (SDKD log:137)  "0": 39, [43.83 INFO] (SDKD log:137)  "264": 961 [43.83 INFO] (SDKD log:137) }  [43.83 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@102.5 => {[OK]: 39, [MEMD:GENERIC]: 961} [43.83 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6 [43.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103  [43.83 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7 [43.85 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.85 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.85 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104  [43.86 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.87 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9 [43.88 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.88 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105  [43.88 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.90 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11 [43.90 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.90 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.90 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.91 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106  [43.91 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.92 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13 [43.93 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.93 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107  [43.94 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.97 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15 [43.97 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.97 INFO] (Workload call:112) Preloading items into cluster before workload [43.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.98 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108  [43.98 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.99 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17 [43.99 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}} [51.93 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 364, [MEMD:GENERIC]: 14636} [51.93 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19 [51.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.95 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22 [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@111.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111  [52.00 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114  [52.01 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112  [52.02 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113  [52.03 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.03 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21 [52.03 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119  [52.03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.03 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@110.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.04 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.04 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.05 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120  [52.07 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.07 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26 [52.07 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@115.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.08 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121  [52.10 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.10 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20 [52.10 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@109.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.11 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.11 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.12 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122  [52.12 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25 [52.13 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.13 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.13 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.14 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123  [52.16 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.16 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27 [52.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@116.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.17 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.17 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124  [52.24 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.25 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28 [52.25 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@117.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.26 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.26 INFO] (SDKD log:137)  [52.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125  [52.27 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.30 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.30 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29 [52.31 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1 [52.31 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@118.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.31 INFO] (SDKD log:137)  at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 75 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 40 at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 82 at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 121 at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 191 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 123 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [52.31 INFO] (SDKD log:137)  [52.32 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.32 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.33 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126  [52.34 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.35 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23 [52.35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@112.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.37 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127  [52.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24 [52.40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.40 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.41 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.42 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128  [52.45 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31 [52.46 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.53 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33 [52.53 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.55 INFO] (SDKD log:137) +++ Received exception of ID 0 [52.55 INFO] (SDKD log:137)  [52.56 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.63 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35 [52.63 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.67 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.68 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37 [52.68 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39 [52.77 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.05 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [53.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41 [53.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.15 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [53.15 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43 [53.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.17 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 3 [53.18 INFO] (SDKD log:137)  at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 75 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 40 at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 82 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 282 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 191 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 123 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [53.18 INFO] (SDKD log:137)  [53.19 INFO] (SDKD log:137) Unhandled Exception: Couchbase.Configuration.ConfigException: Could not bootstrap default. See log for details. [65.28 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [65.29 INFO] (SDKD log:137)  at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 210 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 82 at Sdkd.Couchbase.ClientFactory.FindExistingClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 55 at Sdkd.Couchbase.ClientFactory.CreateClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 262 at SdkdConsole.Daemon.Handle.CreateHandle(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 103 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [65.30 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [65.31 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.57 [65.31 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]

    Couchbase JIRA | 3 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  6. 0

    call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf ig.ini -I sdkd.args --variants HYBRID -d all:trace [0.30 INFO] (BRun run:436) Initializing S3 [1.12 INFO] (BRun run:441) Initializing history database [1.75 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools ============================================================ Running ReAdd2-HYBRID: 2014-08-26 22:31:28 workload=HybridWorkloadGroup failover/next_action=FO_READD_REBALANCE failover/count=2 failover/next_delay=45 testcase=FailoverScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0198 88\0dcd5a5bc61e625f06a757ab9dfa7db5 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ \ --workload HybridWorkloadGroup --failover-next_action \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45 \ --testcase FailoverScenario -C share\rexec --rexec_path \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 --cluster_ssh-username=root --cluster_ssh-password=couchbase ..... [123.71 INFO] (Scenario run:86) Starting CHANGE phase [123.71 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135> [123.72 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver [124.10 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 [124.13 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136> [124.14 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver [124.40 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds [125.61 INFO] (SDKD log:137) [125.63 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavail ableException' was thrown. [169.41 INFO] (FailoverAction doNextAction:68) Readding nodes after failover [169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode [169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode [169.42 INFO] (FailoverAction doNextAction:85) Rebalancing nodes after failover [169.42 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/nodes [169.43 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/rebalance [169.45 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [169.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [170.46 TRACE] (CouchbaseAdmin getResponseJson:167) GET .... [196.67 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [196.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [197.59 INFO] (SDKD log:137) [Sdkd.ViewQuery|Error] ... System.NullReferenceException Object reference not set to an instance of a n object. at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Views\ViewComman d.cs:line 268 [197.61 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [197.61 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace\.net-sdkdclient-n g2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 210 at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace\.net -sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, ISer ver server) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs: line 179 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2- andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net \src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin e 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs :line 175 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo lean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre serveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [197.64 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [197.66 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73 [197.66 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":103,"ReqID":73,"CommandData":{}} [197.66 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] [197.68 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [197.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39% [198.80 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re Add2-HYBRID/08-26-14/019888/0dcd5a5bc61e625f06a757ab9dfa7db5.zip [198.83 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress

    Couchbase JIRA | 3 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. com.couchbase.sdkdclient.context.HarnessException

      Driver crashed

      at com.couchbase.sdkdclient.context.RunContext$5.call()
    2. com.couchbase.sdkdclient
      RunContext$5.call
      1. com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192)[sdkdclient-1.0-SNAPSHOT.jar:na]
      2. com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183)[sdkdclient-1.0-SNAPSHOT.jar:na]
      2 frames
    3. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)[na:1.6.0_27]
      2. java.util.concurrent.FutureTask.run(Unknown Source)[na:1.6.0_27]
      3. java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)[na:1.6.0_27]
      4. java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)[na:1.6.0_27]
      5. java.util.concurrent.FutureTask.run(Unknown Source)[na:1.6.0_27]
      6. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)[na:1.6.0_27]
      7. java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)[na:1.6.0_27]
      8. java.lang.Thread.run(Unknown Source)[na:1.6.0_27]
      8 frames