com.couchbase.sdkdclient.context.HarnessException: Driver crashed

Couchbase JIRA | Andrei Baranouski | 2 years ago
  1. 0

    very often in different tests I see the following exception http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/38/consoleFull Running SvcRestart-HYBRID: 2014-07-25 05:20:15 .............. [365.86 INFO] (SDKD log:137) Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object. [366.20 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [366.20 INFO] (SDKD log:137)  at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 51 at Couchbase.Configuration.ConfigContextBase..ctor(IBucketConfig bucketConfig, ClientConfiguration clientConfig, Func`2 ioStrategyFactory, Func`3 connectionPoolFactory, Func`5 saslFactory, IByteConverter converter, ITypeSerializer serializer) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 51 at Couchbase.Configuration.CouchbaseConfigContext..ctor(IBucketConfig bucketConfig, ClientConfiguration clientConfig, Func`2 ioStrategyFactory, Func`3 connectionPoolFactory, Func`5 saslFactory, IByteConverter converter, ITypeSerializer serializer) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 22 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 63 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Configuration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 44 at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 155 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 82 at Sdkd.Couchbase.ClientFactory.FindExistingClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 55 at Sdkd.Couchbase.ClientFactory.CreateClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 262 at SdkdConsole.Daemon.Handle.CreateHandle(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 103 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [366.20 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [366.21 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@203.259 [366.21 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] client logs http://sdkbuilds.couchbase.com/job/.net-sdkdclient-ng2-andrei/38/artifact/log.txt Jeff, believe that this is a problem on client side?

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  2. 0

    git log commit 86219ee99ccafdd00c98d577c9bc2b97efef35bb Author: jeffrymorris <jeffrymorris@gmail.com> Date: Wed Aug 20 17:44:45 2014 -0700 NCBC-592: AuthenticationException in CouchbaseBucket.SendWithRetry Running Rb2Out-KV: 2014-08-20 23:33:57 ..... [25.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.33 [25.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@121.43 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=10, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [25.05 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET  [25.06 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds [25.06 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [25.06 INFO] (SDKD log:137)  "DSType": "DSTYPE_SEEDED", [25.06 INFO] (SDKD log:137)  "DS": { [25.07 INFO] (SDKD log:137)  "KSeed": "SimpleKey", [25.07 INFO] (SDKD log:137)  "VSeed": "SimpleValue", [25.07 INFO] (SDKD log:137)  "Count": 10, [25.07 INFO] (SDKD log:137)  "Continuous": true, [25.07 INFO] (SDKD log:137)  "VSize": 128, [25.07 INFO] (SDKD log:137)  "Repeat": "REP", [25.07 INFO] (SDKD log:137)  "KSize": 32 [25.07 INFO] (SDKD log:137)  }, [25.07 INFO] (SDKD log:137)  "Options": { [25.07 INFO] (SDKD log:137)  "DelayMax": 10, [25.07 INFO] (SDKD log:137)  "DelayMin": 1, [25.07 INFO] (SDKD log:137)  "IterWait": 1, [25.07 INFO] (SDKD log:137)  "TimeRes": 1 [25.09 INFO] (SDKD log:137)  } [25.09 INFO] (SDKD log:137) }  [55.06 INFO] (Scenario run:86) Starting CHANGE phase [55.08 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [56.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [57.12 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [58.13 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4% [59.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% ................................. [99.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 94% [100.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 96% [101.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 98% [102.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [103.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [104.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [105.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [106.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [107.74 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [108.75 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [109.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [110.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [111.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 100% [112.75 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409  [112.78 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [112.79 INFO] (Scenario run:97) CHANGE phase done [112.79 INFO] (Scenario run:103) Starting REBOUND for 90 seconds [113.23 INFO] (SDKD log:137)  [113.26 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavailableException' was thrown. [125.87 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [125.88 INFO] (SDKD log:137)  at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 212 at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 179 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 235 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 156 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [125.89 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [125.89 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.44 [125.90 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  3. 0

    I took a few commits ahead from gerrit commit 2c00a9d3a397f5c1eddc56238cf4a19e6e6618dd Author: jeffrymorris <jeffrymorris@gmail.com> Date: Mon Aug 11 18:41:08 2014 -0700 Add config files to tests Change-Id: Ie093eac1da4ffc742796ffd4df8a9c63cc1c9c2f c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>call packages\sdkdclient\bin\brun.bat -I cluster_config.ini -I sdkd.args --variants HYBRID -d all:debug --no-upload [0.32 INFO] (BRun run:435) Initializing history database ============================================================ Running Rb2Out-HYBRID rebalance/mode=out workload=HybridWorkloadGroup rebalance/count=2 testcase=RebalanceScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1105\Rb2Out-HYBRID\08-16-14\056660\742db0c42511883a3b8960d57d363be0 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ \ --rebalance-mode out --workload HybridWorkloadGroup \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port \ 8675 --cluster_node=10.3.4.144 --cluster_node=10.3.4.145 \ --cluster_node=10.3.4.146 --cluster_node=10.3.4.147 --cluster_ssh-username=root --cluster_ssh-password=couchbase ---------------------------------------->8 [1.52 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution [1.93 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds. [1.95 INFO] (RunContext run:124) Starting cluster and driver [1.96 INFO] (CBCluster startCluster:360) Node http://10.3.4.144:8091 is master now [1.96 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe' [1.96 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations.. [2.30 INFO] (SDKD log:137) --- Logging Self-Test --- [2.31 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message [2.31 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message [2.32 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message [2.32 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message [2.32 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675 [2.48 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0 [2.49 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [2.78 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}} [4.34 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.4.147:8091,ns_1@10.3.4.147> [4.43 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa [4.43 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.43 DEBUG] (SSHConnection initUserKeys:79) Loaded indentity C:\Users\Administrator\.ssh\id_rsa [4.43 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.94 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.144:8091 [4.95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.145:8091 [4.95 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.146:8091 [4.97 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.146 [4.97 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.144 [4.97 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.4.147:8091 [4.98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.145 [4.98 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.4.147 [5.18 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@7b0bbd42 [5.19 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@10d4f27 [5.19 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@b22c9df [5.20 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@74bfd10a [5.20 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@6ae2d0b2 [17.28 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.145:8091 [24.00 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.146:8091 [36.81 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.4.147:8091 [43.42 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance [43.46 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [44.47 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [44.47 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups [44.48 INFO] (CBCluster setupMainBucket:209) Creating bucket default [44.49 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted [54.57 INFO] (CBCluster waitForBucketReady:203) Bucket creation done [54.62 INFO] (RunContext run:143) Driver and cluster initialized [54.65 INFO] (RunContext call:167) Running scenario.. [54.66 INFO] (Scenario run:72) Starting RAMP phase [54.66 INFO] (Workload setupDesign:63) Creating design test_design [54.93 INFO] (Workload setupDesign:80) Design creation done [54.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [54.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [54.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101 [54.96 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [54.97 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net [54.97 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e [55.00 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net [55.01 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821 [55.42 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1 [55.42 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12 }, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [55.43 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD [55.43 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [55.43 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [55.43 INFO] (SDKD log:137) "DS": { [55.43 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed", [55.44 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed", [55.44 INFO] (SDKD log:137) "Count": 1000, [55.44 INFO] (SDKD log:137) "Continuous": false, [55.44 INFO] (SDKD log:137) "VSize": 512, [55.44 INFO] (SDKD log:137) "Repeat": "rep", [55.44 INFO] (SDKD log:137) "KSize": 12 [55.45 INFO] (SDKD log:137) }, [55.45 INFO] (SDKD log:137) "Schema": { [55.45 INFO] (SDKD log:137) "InflateContent": "meh", [55.45 INFO] (SDKD log:137) "InflateLevel": 40 [55.45 INFO] (SDKD log:137) }, [55.46 INFO] (SDKD log:137) "Options": { [55.46 INFO] (SDKD log:137) "DelayMax": 0, [55.46 INFO] (SDKD log:137) "DelayMin": 0, [55.46 INFO] (SDKD log:137) "IterWait": 1, [55.46 INFO] (SDKD log:137) "TimeRes": 0 [55.46 INFO] (SDKD log:137) } [55.47 INFO] (SDKD log:137) } [56.75 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {} [56.76 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@101.2 [56.76 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3 [56.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [56.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [56.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [56.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [56.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102 [56.77 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [56.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [56.79 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4 [56.79 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12 }, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [56.79 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD [56.80 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [56.80 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [56.80 INFO] (SDKD log:137) "DS": { [56.81 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed", [56.81 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed", [56.81 INFO] (SDKD log:137) "Count": 1000, [56.81 INFO] (SDKD log:137) "Continuous": false, [56.81 INFO] (SDKD log:137) "VSize": 512, [56.82 INFO] (SDKD log:137) "Repeat": "rep", [56.82 INFO] (SDKD log:137) "KSize": 12 [56.82 INFO] (SDKD log:137) }, [56.82 INFO] (SDKD log:137) "Schema": { [56.82 INFO] (SDKD log:137) "InflateContent": "meh", [56.83 INFO] (SDKD log:137) "InflateLevel": 40 [56.83 INFO] (SDKD log:137) }, [56.83 INFO] (SDKD log:137) "Options": { [56.83 INFO] (SDKD log:137) "DelayMax": 0, [56.83 INFO] (SDKD log:137) "DelayMin": 0, [56.84 INFO] (SDKD log:137) "IterWait": 1, [56.84 INFO] (SDKD log:137) "TimeRes": 0 [56.84 INFO] (SDKD log:137) } [56.84 INFO] (SDKD log:137) } [58.30 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {} [58.31 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@102.5 [58.31 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6 [58.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [58.31 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [58.31 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [58.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [58.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103 [58.32 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7 [58.34 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCou nt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.35 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.35 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10.3 .4.147",8091]]}, Hostname=10.3.4.144} [58.35 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.35 INFO] (SDKD log:137) "DS": {}, [58.36 INFO] (SDKD log:137) "ViewParameters": {}, [58.36 INFO] (SDKD log:137) "DesignName": "test_design", [58.36 INFO] (SDKD log:137) "Options": { [58.36 INFO] (SDKD log:137) "DelayMax": 10, [58.36 INFO] (SDKD log:137) "DelayMin": 1, [58.37 INFO] (SDKD log:137) "IterWait": 1, [58.37 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.37 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.37 INFO] (SDKD log:137) "TimeRes": 1 [58.38 INFO] (SDKD log:137) }, [58.38 INFO] (SDKD log:137) "ViewName": "test_view" [58.38 INFO] (SDKD log:137) } [58.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104 [58.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9 [58.40 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.41 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.41 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.41 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.41 INFO] (SDKD log:137) "DS": {}, [58.42 INFO] (SDKD log:137) "ViewParameters": {}, [58.42 INFO] (SDKD log:137) "DesignName": "test_design", [58.42 INFO] (SDKD log:137) "Options": { [58.42 INFO] (SDKD log:137) "DelayMax": 10, [58.42 INFO] (SDKD log:137) "DelayMin": 1, [58.42 INFO] (SDKD log:137) "IterWait": 1, [58.43 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.43 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.43 INFO] (SDKD log:137) "TimeRes": 1 [58.43 INFO] (SDKD log:137) }, [58.43 INFO] (SDKD log:137) "ViewName": "test_view" [58.44 INFO] (SDKD log:137) } [58.44 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.44 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105 [58.44 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.57 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11 [58.58 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.58 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.58 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.59 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.59 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.59 INFO] (SDKD log:137) "DS": {}, [58.59 INFO] (SDKD log:137) "ViewParameters": {}, [58.59 INFO] (SDKD log:137) "DesignName": "test_design", [58.60 INFO] (SDKD log:137) "Options": { [58.60 INFO] (SDKD log:137) "DelayMax": 10, [58.60 INFO] (SDKD log:137) "DelayMin": 1, [58.60 INFO] (SDKD log:137) "IterWait": 1, [58.60 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.60 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.60 INFO] (SDKD log:137) "TimeRes": 1 [58.61 INFO] (SDKD log:137) }, [58.61 INFO] (SDKD log:137) "ViewName": "test_view" [58.61 INFO] (SDKD log:137) } [58.61 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.61 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106 [58.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13 [58.73 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.74 INFO] (SDKD log:137) "DS": {}, [58.74 INFO] (SDKD log:137) "ViewParameters": {}, [58.74 INFO] (SDKD log:137) "DesignName": "test_design", [58.74 INFO] (SDKD log:137) "Options": { [58.75 INFO] (SDKD log:137) "DelayMax": 10, [58.75 INFO] (SDKD log:137) "DelayMin": 1, [58.75 INFO] (SDKD log:137) "IterWait": 1, [58.75 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.75 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.76 INFO] (SDKD log:137) "TimeRes": 1 [58.76 INFO] (SDKD log:137) }, [58.76 INFO] (SDKD log:137) "ViewName": "test_view" [58.76 INFO] (SDKD log:137) } [58.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.76 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107 [58.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15 [58.77 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCo unt=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [58.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [58.78 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.78 INFO] (Workload call:112) Preloading items into cluster before workload [58.78 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.78 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10. 3.4.147",8091]]}, Hostname=10.3.4.144} [58.79 INFO] (SDKD log:137) "DS": {}, [58.79 INFO] (SDKD log:137) "ViewParameters": {}, [58.79 INFO] (SDKD log:137) "DesignName": "test_design", [58.79 INFO] (SDKD log:137) "Options": { [58.79 INFO] (SDKD log:137) "DelayMax": 10, [58.80 INFO] (SDKD log:137) "DelayMin": 1, [58.80 INFO] (SDKD log:137) "IterWait": 1, [58.80 INFO] (SDKD log:137) "ViewQueryCount": -1, [58.80 INFO] (SDKD log:137) "ViewQueryDelay": 0, [58.80 INFO] (SDKD log:137) "TimeRes": 1 [58.81 INFO] (SDKD log:137) }, [58.81 INFO] (SDKD log:137) "ViewName": "test_view" [58.81 INFO] (SDKD log:137) } [58.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [58.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108 [58.81 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [58.82 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17 [58.82 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}} [58.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [58.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [58.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [58.83 INFO] (SDKD log:137) "DS": { [58.83 INFO] (SDKD log:137) "KSeed": "SimpleKey", [58.84 INFO] (SDKD log:137) "VSeed": "SimpleValue", [58.84 INFO] (SDKD log:137) "Count": 15000, [58.84 INFO] (SDKD log:137) "Continuous": false, [58.84 INFO] (SDKD log:137) "VSize": 128, [58.84 INFO] (SDKD log:137) "Repeat": "REP", [58.85 INFO] (SDKD log:137) "KSize": 32 [58.85 INFO] (SDKD log:137) }, [58.85 INFO] (SDKD log:137) "Options": { [58.85 INFO] (SDKD log:137) "DelayMax": 0, [58.85 INFO] (SDKD log:137) "DelayMin": 0, [58.85 INFO] (SDKD log:137) "IterWait": 100, [58.86 INFO] (SDKD log:137) "TimeRes": 0 [58.86 INFO] (SDKD log:137) } [58.86 INFO] (SDKD log:137) } [249.74 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 4003, [MEMD:MEMD_EVBUCKET]: 10997} [249.74 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19 [249.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [249.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [249.74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [249.74 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [249.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.75 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.76 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.77 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.77 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109 [249.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114 [249.79 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116 [249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.80 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115 [249.80 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20 [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.81 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@109.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110 [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113 [249.81 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111 [249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117 [249.82 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112 [249.82 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118 [249.82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.82 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.83 INFO] (SDKD log:137) "DS": { [249.83 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.83 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.83 INFO] (SDKD log:137) "Count": 15000, [249.84 INFO] (SDKD log:137) "Continuous": true, [249.84 INFO] (SDKD log:137) "VSize": 128, [249.84 INFO] (SDKD log:137) "Repeat": "REP", [249.84 INFO] (SDKD log:137) "KSize": 32 [249.84 INFO] (SDKD log:137) }, [249.84 INFO] (SDKD log:137) "Options": { [249.84 INFO] (SDKD log:137) "DelayMax": 10, [249.85 INFO] (SDKD log:137) "DelayMin": 1, [249.85 INFO] (SDKD log:137) "IterWait": 1, [249.85 INFO] (SDKD log:137) "TimeRes": 1 [249.85 INFO] (SDKD log:137) } [249.86 INFO] (SDKD log:137) } [249.86 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25 [249.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.87 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.87 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119 [249.87 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.87 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.88 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.88 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.88 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.88 INFO] (SDKD log:137) "DS": { [249.89 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.89 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.89 INFO] (SDKD log:137) "Count": 15000, [249.89 INFO] (SDKD log:137) "Continuous": true, [249.89 INFO] (SDKD log:137) "VSize": 128, [249.90 INFO] (SDKD log:137) "Repeat": "REP", [249.90 INFO] (SDKD log:137) "KSize": 32 [249.90 INFO] (SDKD log:137) }, [249.90 INFO] (SDKD log:137) "Options": { [249.90 INFO] (SDKD log:137) "DelayMax": 10, [249.90 INFO] (SDKD log:137) "DelayMin": 1, [249.90 INFO] (SDKD log:137) "IterWait": 1, [249.91 INFO] (SDKD log:137) "TimeRes": 1 [249.91 INFO] (SDKD log:137) } [249.91 INFO] (SDKD log:137) } [249.91 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27 [249.92 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120 [249.92 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@116.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.92 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.93 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.93 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.93 INFO] (SDKD log:137) "DS": { [249.94 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.94 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.94 INFO] (SDKD log:137) "Count": 15000, [249.94 INFO] (SDKD log:137) "Continuous": true, [249.94 INFO] (SDKD log:137) "VSize": 128, [249.94 INFO] (SDKD log:137) "Repeat": "REP", [249.95 INFO] (SDKD log:137) "KSize": 32 [249.95 INFO] (SDKD log:137) }, [249.95 INFO] (SDKD log:137) "Options": { [249.95 INFO] (SDKD log:137) "DelayMax": 10, [249.95 INFO] (SDKD log:137) "DelayMin": 1, [249.96 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26 [249.96 INFO] (SDKD log:137) "IterWait": 1, [249.96 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@115.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [249.96 INFO] (SDKD log:137) "TimeRes": 1 [249.96 INFO] (SDKD log:137) } [249.96 INFO] (SDKD log:137) } [249.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [249.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [249.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121 [249.98 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [249.98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [249.98 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [249.98 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [249.98 INFO] (SDKD log:137) "DS": { [249.98 INFO] (SDKD log:137) "KSeed": "SimpleKey", [249.99 INFO] (SDKD log:137) "VSeed": "SimpleValue", [249.99 INFO] (SDKD log:137) "Count": 15000, [249.99 INFO] (SDKD log:137) "Continuous": true, [249.99 INFO] (SDKD log:137) "VSize": 128, [249.99 INFO] (SDKD log:137) "Repeat": "REP", [249.99 INFO] (SDKD log:137) "KSize": 32 [250.00 INFO] (SDKD log:137) }, [250.00 INFO] (SDKD log:137) "Options": { [250.00 INFO] (SDKD log:137) "DelayMax": 10, [250.01 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24 [250.01 INFO] (SDKD log:137) "DelayMin": 1, [250.01 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.01 INFO] (SDKD log:137) "IterWait": 1, [250.02 INFO] (SDKD log:137) "TimeRes": 1 [250.02 INFO] (SDKD log:137) } [250.02 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.02 INFO] (SDKD log:137) } [250.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122 [250.03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.03 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.03 INFO] (SDKD log:137) "DS": { [250.04 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.04 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.04 INFO] (SDKD log:137) "Count": 15000, [250.04 INFO] (SDKD log:137) "Continuous": true, [250.04 INFO] (SDKD log:137) "VSize": 128, [250.04 INFO] (SDKD log:137) "Repeat": "REP", [250.05 INFO] (SDKD log:137) "KSize": 32 [250.05 INFO] (SDKD log:137) }, [250.05 INFO] (SDKD log:137) "Options": { [250.05 INFO] (SDKD log:137) "DelayMax": 10, [250.05 INFO] (SDKD log:137) "DelayMin": 1, [250.05 INFO] (SDKD log:137) "IterWait": 1, [250.06 INFO] (SDKD log:137) "TimeRes": 1 [250.07 INFO] (SDKD log:137) } [250.07 INFO] (SDKD log:137) } [250.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.08 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21 [250.08 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@110.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123 [250.09 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.09 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.10 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.11 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124 [250.12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.12 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.12 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.13 INFO] (SDKD log:137) "DS": { [250.13 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [250.14 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.15 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.15 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [250.15 INFO] (SDKD log:137) "Count": 15000, [250.15 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [250.15 INFO] (SDKD log:137) "Continuous": true, [250.15 INFO] (SDKD log:137) "VSize": 128, [250.16 INFO] (SDKD log:137) "Repeat": "REP", [250.16 INFO] (SDKD log:137) "KSize": 32 [250.16 INFO] (SDKD log:137) }, [250.16 INFO] (SDKD log:137) "Options": { [250.16 INFO] (SDKD log:137) "DelayMax": 10, [250.17 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1 [250.17 INFO] (SDKD log:137) "DelayMin": 1, [250.17 INFO] (SDKD log:137) "IterWait": 1, [250.18 INFO] (SDKD log:137) at System.Net.Sockets.Socket.get_RemoteEndPoint() at Couchbase.IO.Strategies.EapConnection.<>c__DisplayClass3`1.<Send>b__0(FormatMessageHandler m) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\IO\Strategies\EapConnection.cs:line 38 at Common.Logging.Factory.AbstractLogger.FormatMessageCallbackFormattedMessage.ToString() at log4net.ObjectRenderer.DefaultRenderer.RenderObject(RendererMap rendererMap, Object obj, TextWriter writer) at log4net.ObjectRenderer.RendererMap.FindAndRender(Object obj, TextWriter writer) [250.18 INFO] (SDKD log:137) "TimeRes": 1 [250.18 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [250.19 INFO] (SDKD log:137) } [250.19 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [250.19 INFO] (SDKD log:137) } [250.19 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 [250.20 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.21 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [250.21 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22 [250.21 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@111.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.22 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.23 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.23 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.23 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.23 INFO] (SDKD log:137) "DS": { [250.24 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.24 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.24 INFO] (SDKD log:137) "Count": 15000, [250.24 INFO] (SDKD log:137) "Continuous": true, [250.25 INFO] (SDKD log:137) "VSize": 128, [250.25 INFO] (SDKD log:137) "Repeat": "REP", [250.25 INFO] (SDKD log:137) "KSize": 32 [250.25 INFO] (SDKD log:137) }, [250.25 INFO] (SDKD log:137) "Options": { [250.27 INFO] (SDKD log:137) "DelayMax": 10, [250.28 INFO] (SDKD log:137) "DelayMin": 1, [250.28 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28 [250.28 INFO] (SDKD log:137) "IterWait": 1, [250.29 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@117.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.29 INFO] (SDKD log:137) "TimeRes": 1 [250.29 INFO] (SDKD log:137) } [250.29 INFO] (SDKD log:137) } [250.30 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.30 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.31 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125 [250.31 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.32 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.32 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.33 INFO] (SDKD log:137) "DS": { [250.33 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.33 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.33 INFO] (SDKD log:137) "Count": 15000, [250.34 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23 [250.34 INFO] (SDKD log:137) "Continuous": true, [250.35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@112.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.35 INFO] (SDKD log:137) "VSize": 128, [250.35 INFO] (SDKD log:137) "Repeat": "REP", [250.35 INFO] (SDKD log:137) "KSize": 32 [250.35 INFO] (SDKD log:137) }, [250.36 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.36 INFO] (SDKD log:137) "Options": { [250.36 INFO] (SDKD log:137) "DelayMax": 10, [250.36 INFO] (SDKD log:137) "DelayMin": 1, [250.37 INFO] (SDKD log:137) "IterWait": 1, [250.37 INFO] (SDKD log:137) "TimeRes": 1 [250.37 INFO] (SDKD log:137) } [250.37 INFO] (SDKD log:137) } [250.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126 [250.38 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.38 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.38 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.38 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.39 INFO] (SDKD log:137) "DS": { [250.39 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.40 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29 [250.40 INFO] (SDKD log:137) "Count": 15000, [250.40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@118.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.41 INFO] (SDKD log:137) "Continuous": true, [250.42 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [250.42 INFO] (SDKD log:137) "VSize": 128, [250.43 INFO] (SDKD log:137) "Repeat": "REP", [250.43 INFO] (SDKD log:137) "KSize": 32 [250.43 INFO] (SDKD log:137) }, [250.43 INFO] (SDKD log:137) "Options": { [250.44 INFO] (SDKD log:137) "DelayMax": 10, [250.44 INFO] (SDKD log:137) "DelayMin": 1, [250.44 INFO] (SDKD log:137) "IterWait": 1, [250.44 INFO] (SDKD log:137) "TimeRes": 1 [250.44 INFO] (SDKD log:137) } [250.45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31 [250.45 INFO] (SDKD log:137) } [250.45 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127 [250.45 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.46 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.46 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [250.46 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.47 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.47 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.47 INFO] (SDKD log:137) "DS": { [250.47 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.48 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.48 INFO] (SDKD log:137) "Count": 15000, [250.48 INFO] (SDKD log:137) "Continuous": true, [250.48 INFO] (SDKD log:137) "VSize": 128, [250.49 INFO] (SDKD log:137) "Repeat": "REP", [250.49 INFO] (SDKD log:137) "KSize": 32 [250.49 INFO] (SDKD log:137) }, [250.49 INFO] (SDKD log:137) "Options": { [250.50 INFO] (SDKD log:137) "DelayMax": 10, [250.50 INFO] (SDKD log:137) "DelayMin": 1, [250.50 INFO] (SDKD log:137) "IterWait": 1, [250.50 INFO] (SDKD log:137) "TimeRes": 1 [250.51 INFO] (SDKD log:137) } [250.51 INFO] (SDKD log:137) } [250.51 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128 [250.51 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33 [250.52 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.55 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.55 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35 [250.56 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.56 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.56 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.56 INFO] (SDKD log:137) "DS": { [250.57 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.57 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.57 INFO] (SDKD log:137) "Count": 15000, [250.58 INFO] (SDKD log:137) "Continuous": true, [250.58 INFO] (SDKD log:137) "VSize": 128, [250.58 INFO] (SDKD log:137) "Repeat": "REP", [250.58 INFO] (SDKD log:137) "KSize": 32 [250.59 INFO] (SDKD log:137) }, [250.59 INFO] (SDKD log:137) "Options": { [250.59 INFO] (SDKD log:137) "DelayMax": 10, [250.59 INFO] (SDKD log:137) +++ Received exception of ID 0 [250.60 INFO] (SDKD log:137) "DelayMin": 1, [250.60 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [250.60 INFO] (SDKD log:137) "IterWait": 1, [250.60 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [250.60 INFO] (SDKD log:137) "TimeRes": 1 [250.60 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [250.60 INFO] (SDKD log:137) } [250.61 INFO] (SDKD log:137) } [250.61 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.61 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.62 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.62 INFO] (SDKD log:137) "DS": { [250.63 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.63 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.63 INFO] (SDKD log:137) "Count": 15000, [250.63 INFO] (SDKD log:137) "Continuous": true, [250.63 INFO] (SDKD log:137) "VSize": 128, [250.63 INFO] (SDKD log:137) "Repeat": "REP", [250.63 INFO] (SDKD log:137) "KSize": 32 [250.63 INFO] (SDKD log:137) }, [250.64 INFO] (SDKD log:137) "Options": { [250.64 INFO] (SDKD log:137) "DelayMax": 10, [250.64 INFO] (SDKD log:137) "DelayMin": 1, [250.64 INFO] (SDKD log:137) "IterWait": 1, [250.64 INFO] (SDKD log:137) "TimeRes": 1 [250.65 INFO] (SDKD log:137) } [250.65 INFO] (SDKD log:137) } [250.65 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.65 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.65 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.66 INFO] (SDKD log:137) "DS": { [250.66 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.66 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.66 INFO] (SDKD log:137) "Count": 15000, [250.66 INFO] (SDKD log:137) "Continuous": true, [250.66 INFO] (SDKD log:137) "VSize": 128, [250.67 INFO] (SDKD log:137) "Repeat": "REP", [250.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37 [250.67 INFO] (SDKD log:137) "KSize": 32 [250.68 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.68 INFO] (SDKD log:137) }, [250.68 INFO] (SDKD log:137) "Options": { [250.68 INFO] (SDKD log:137) "DelayMax": 10, [250.68 INFO] (SDKD log:137) "DelayMin": 1, [250.69 INFO] (SDKD log:137) "IterWait": 1, [250.69 INFO] (SDKD log:137) "TimeRes": 1 [250.69 INFO] (SDKD log:137) } [250.70 INFO] (SDKD log:137) } [250.70 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.70 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.71 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.71 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.71 INFO] (SDKD log:137) "DS": { [250.71 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39 [250.71 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.72 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.72 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.73 INFO] (SDKD log:137) "Count": 15000, [250.73 INFO] (SDKD log:137) "Continuous": true, [250.73 INFO] (SDKD log:137) "VSize": 128, [250.73 INFO] (SDKD log:137) "Repeat": "REP", [250.73 INFO] (SDKD log:137) "KSize": 32 [250.73 INFO] (SDKD log:137) }, [250.74 INFO] (SDKD log:137) "Options": { [250.74 INFO] (SDKD log:137) "DelayMax": 10, [250.74 INFO] (SDKD log:137) "DelayMin": 1, [250.75 INFO] (SDKD log:137) "IterWait": 1, [250.75 INFO] (SDKD log:137) "TimeRes": 1 [250.75 INFO] (SDKD log:137) } [250.76 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41 [250.76 INFO] (SDKD log:137) } [250.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.76 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.77 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.77 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.77 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.78 INFO] (SDKD log:137) "DS": { [250.78 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.78 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.79 INFO] (SDKD log:137) "Count": 15000, [250.79 INFO] (SDKD log:137) "Continuous": true, [250.79 INFO] (SDKD log:137) "VSize": 128, [250.79 INFO] (SDKD log:137) "Repeat": "REP", [250.79 INFO] (SDKD log:137) "KSize": 32 [250.79 INFO] (SDKD log:137) }, [250.79 INFO] (SDKD log:137) "Options": { [250.80 INFO] (SDKD log:137) "DelayMax": 10, [250.80 INFO] (SDKD log:137) "DelayMin": 1, [250.80 INFO] (SDKD log:137) "IterWait": 1, [250.81 INFO] (SDKD log:137) "TimeRes": 1 [250.81 INFO] (SDKD log:137) } [250.81 INFO] (SDKD log:137) } [250.81 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43 [250.82 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.83 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.84 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.84 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@126.45 [250.85 INFO] (SDKD log:137) "DS": { [250.85 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.85 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@126.57 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.86 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.86 INFO] (SDKD log:137) "Count": 15000, [250.87 INFO] (SDKD log:137) "Continuous": true, [250.87 INFO] (SDKD log:137) "VSize": 128, [250.87 INFO] (SDKD log:137) "Repeat": "REP", [250.87 INFO] (SDKD log:137) "KSize": 32 [250.87 INFO] (SDKD log:137) }, [250.87 INFO] (SDKD log:137) "Options": { [250.88 INFO] (SDKD log:137) "DelayMax": 10, [250.88 INFO] (SDKD log:137) "DelayMin": 1, [250.88 INFO] (SDKD log:137) "IterWait": 1, [250.88 INFO] (SDKD log:137) "TimeRes": 1 [250.88 INFO] (SDKD log:137) } [250.88 INFO] (SDKD log:137) } [250.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.89 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.89 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.89 INFO] (SDKD log:137) "DS": { [250.90 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.90 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.90 INFO] (SDKD log:137) "Count": 15000, [250.90 INFO] (SDKD log:137) "Continuous": true, [250.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@127.47 [250.91 INFO] (SDKD log:137) "VSize": 128, [250.91 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@127.58 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [250.91 INFO] (SDKD log:137) "Repeat": "REP", [250.91 INFO] (SDKD log:137) "KSize": 32 [250.92 INFO] (SDKD log:137) }, [250.92 INFO] (SDKD log:137) "Options": { [250.92 INFO] (SDKD log:137) "DelayMax": 10, [250.93 INFO] (SDKD log:137) "DelayMin": 1, [250.93 INFO] (SDKD log:137) "IterWait": 1, [250.93 INFO] (SDKD log:137) "TimeRes": 1 [250.93 INFO] (SDKD log:137) } [250.93 INFO] (SDKD log:137) } [250.93 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [250.94 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.94 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.94 INFO] (SDKD log:137) "DS": { [250.94 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.95 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.95 INFO] (SDKD log:137) "Count": 15000, [250.96 INFO] (SDKD log:137) "Continuous": true, [250.96 INFO] (SDKD log:137) "VSize": 128, [250.96 INFO] (SDKD log:137) "Repeat": "REP", [250.96 INFO] (SDKD log:137) "KSize": 32 [250.96 INFO] (SDKD log:137) }, [250.96 INFO] (SDKD log:137) "Options": { [250.96 INFO] (SDKD log:137) "DelayMax": 10, [250.96 INFO] (SDKD log:137) "DelayMin": 1, [250.96 INFO] (SDKD log:137) "IterWait": 1, [250.96 INFO] (SDKD log:137) "TimeRes": 1 [250.96 INFO] (SDKD log:137) } [250.97 INFO] (SDKD log:137) } [250.97 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [250.97 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_MUTATE_SET [250.97 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [250.98 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [250.98 INFO] (SDKD log:137) "DS": { [250.99 INFO] (SDKD log:137) "KSeed": "SimpleKey", [250.99 INFO] (SDKD log:137) "VSeed": "SimpleValue", [250.99 INFO] (SDKD log:137) "Count": 15000, [250.99 INFO] (SDKD log:137) "Continuous": true, [250.99 INFO] (SDKD log:137) "VSize": 128, [250.99 INFO] (SDKD log:137) "Repeat": "REP", [250.99 INFO] (SDKD log:137) "KSize": 32 [250.99 INFO] (SDKD log:137) }, [251.00 INFO] (SDKD log:137) "Options": { [251.00 INFO] (SDKD log:137) "DelayMax": 10, [251.00 INFO] (SDKD log:137) "DelayMin": 1, [251.00 INFO] (SDKD log:137) "IterWait": 1, [251.01 INFO] (SDKD log:137) "TimeRes": 1 [251.01 INFO] (SDKD log:137) } [251.02 INFO] (SDKD log:137) } [251.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@128.49 [251.06 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@128.59 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Option s={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [251.07 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@129.60 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [251.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [251.09 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command MC_DS_GET [251.09 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [251.09 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [251.09 INFO] (SDKD log:137) +++ Received exception of ID 0 [251.09 INFO] (SDKD log:137) "DS": { [251.09 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [251.09 INFO] (SDKD log:137) "KSeed": "SimpleKey", [251.09 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [251.10 INFO] (SDKD log:137) "VSeed": "SimpleValue", [251.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [251.10 INFO] (SDKD log:137) "Count": 15000, [251.10 INFO] (SDKD log:137) +++ Received exception of ID 0 [251.10 INFO] (SDKD log:137) "Continuous": true, [251.11 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [251.11 INFO] (SDKD log:137) "VSize": 128, [251.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [251.11 INFO] (SDKD log:137) "Repeat": "REP", [251.12 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [251.13 INFO] (SDKD log:137) "KSize": 32 [251.13 INFO] (SDKD log:137) }, [251.13 INFO] (SDKD log:137) "Options": { [251.13 INFO] (SDKD log:137) "DelayMax": 10, [251.14 INFO] (SDKD log:137) "DelayMin": 1, [251.14 INFO] (SDKD log:137) "IterWait": 1, [251.14 INFO] (SDKD log:137) "TimeRes": 1 [251.15 INFO] (SDKD log:137) } [251.15 INFO] (SDKD log:137) } [251.15 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 129 [251.15 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [251.17 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@129.60 [251.18 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@129.61 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize= 12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [251.19 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_LOAD [251.20 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [251.21 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [251.21 INFO] (SDKD log:137) "DS": { [251.21 INFO] (SDKD log:137) "KSeed": "ViewFillerSeed", [251.21 INFO] (SDKD log:137) "VSeed": "ViewFillerSeed", [251.21 INFO] (SDKD log:137) "Count": 1000, [251.22 INFO] (SDKD log:137) "Continuous": false, [251.22 INFO] (SDKD log:137) "VSize": 512, [251.23 INFO] (SDKD log:137) "Repeat": "rep", [251.23 INFO] (SDKD log:137) "KSize": 12 [251.23 INFO] (SDKD log:137) }, [251.23 INFO] (SDKD log:137) "Schema": { [251.24 INFO] (SDKD log:137) "InflateContent": "meh", [251.24 INFO] (SDKD log:137) "InflateLevel": 40 [251.24 INFO] (SDKD log:137) }, [251.25 INFO] (SDKD log:137) "Options": { [251.25 INFO] (SDKD log:137) "DelayMax": 0, [251.25 INFO] (SDKD log:137) "DelayMin": 0, [251.26 INFO] (SDKD log:137) "IterWait": 1, [251.26 INFO] (SDKD log:137) "TimeRes": 0 [251.27 INFO] (SDKD log:137) } [251.27 INFO] (SDKD log:137) } [544.61 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: {} [544.61 DEBUG] (Handle receiveMessage:158) < [SDKD:SDKD_EINVAL="Need at least one kv pair. None requested"] CB_VIEW_LOAD@129.61 [544.61 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@129.62 [544.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CLOSEHANDLE [544.62 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@130.63 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.62 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: {} [544.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket 1303659411 [544.62 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Release CouchbaseBucket client.Dispose()1303659411 [544.63 INFO] (SDKD log:137) [Sdkd.Handle|Warn] DispatchCommand(req, SdkHandle) == false [544.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.63 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 130 [544.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.67 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@130.63 [544.67 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@130.64 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [544.67 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [544.68 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@131.65 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.68 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [544.68 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [544.68 INFO] (SDKD log:137) "DS": {}, [544.68 INFO] (SDKD log:137) "ViewParameters": { [544.69 INFO] (SDKD log:137) "limit": 0 [544.69 INFO] (SDKD log:137) }, [544.69 INFO] (SDKD log:137) "DesignName": "test_design", [544.69 INFO] (SDKD log:137) "Options": { [544.69 INFO] (SDKD log:137) "DelayMax": 10, [544.70 INFO] (SDKD log:137) "DelayMin": 1, [544.70 INFO] (SDKD log:137) "IterWait": 1, [544.70 INFO] (SDKD log:137) "ViewQueryCount": -1, [544.70 INFO] (SDKD log:137) "ViewQueryDelay": 0, [544.70 INFO] (SDKD log:137) "TimeRes": 1 [544.70 INFO] (SDKD log:137) }, [544.70 INFO] (SDKD log:137) "ViewName": "test_view" [544.71 INFO] (SDKD log:137) } [544.71 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.71 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 131 [544.71 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.73 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@131.65 [544.73 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@131.66 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [544.73 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [544.74 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@132.67 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.74 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [544.74 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [544.74 INFO] (SDKD log:137) "DS": {}, [544.74 INFO] (SDKD log:137) "ViewParameters": { [544.74 INFO] (SDKD log:137) "limit": 0 [544.75 INFO] (SDKD log:137) }, [544.75 INFO] (SDKD log:137) "DesignName": "test_design", [544.75 INFO] (SDKD log:137) "Options": { [544.75 INFO] (SDKD log:137) "DelayMax": 10, [544.75 INFO] (SDKD log:137) "DelayMin": 1, [544.76 INFO] (SDKD log:137) "IterWait": 1, [544.76 INFO] (SDKD log:137) "ViewQueryCount": -1, [544.76 INFO] (SDKD log:137) "ViewQueryDelay": 0, [544.76 INFO] (SDKD log:137) "TimeRes": 1 [544.76 INFO] (SDKD log:137) }, [544.77 INFO] (SDKD log:137) "ViewName": "test_view" [544.77 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.77 INFO] (SDKD log:137) } [544.78 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.78 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.78 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 132 [544.78 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.79 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.81 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@132.67 [544.82 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@132.68 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [544.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [544.83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@133.69 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [544.83 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [544.83 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [544.84 INFO] (SDKD log:137) "DS": {}, [544.84 INFO] (SDKD log:137) "ViewParameters": { [544.85 INFO] (SDKD log:137) "limit": 0 [544.85 INFO] (SDKD log:137) }, [544.85 INFO] (SDKD log:137) "DesignName": "test_design", [544.85 INFO] (SDKD log:137) "Options": { [544.86 INFO] (SDKD log:137) "DelayMax": 10, [544.86 INFO] (SDKD log:137) "DelayMin": 1, [544.86 INFO] (SDKD log:137) "IterWait": 1, [544.86 INFO] (SDKD log:137) "ViewQueryCount": -1, [544.86 INFO] (SDKD log:137) "ViewQueryDelay": 0, [544.86 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.87 INFO] (SDKD log:137) "TimeRes": 1 [544.87 INFO] (SDKD log:137) }, [544.88 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.88 INFO] (SDKD log:137) "ViewName": "test_view" [544.88 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.88 INFO] (SDKD log:137) } [544.88 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [544.89 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 133 [544.89 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [544.90 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.91 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.92 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.92 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.93 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.94 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.94 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.94 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.96 INFO] (SDKD log:137) +++ Received exception of ID 0 [544.97 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [544.97 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [544.97 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [544.97 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.00 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.00 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.02 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@133.69 [545.02 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.02 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@133.70 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [545.02 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [545.03 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@134.71 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.4.145",8091],["10.3.4.146",8091],["10 .3.4.147",8091]]}, Hostname=10.3.4.144} [545.03 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [545.03 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [545.03 INFO] (SDKD log:137) "DS": {}, [545.03 INFO] (SDKD log:137) "ViewParameters": { [545.03 INFO] (SDKD log:137) "limit": 0 [545.03 INFO] (SDKD log:137) }, [545.04 INFO] (SDKD log:137) "DesignName": "test_design", [545.04 INFO] (SDKD log:137) "Options": { [545.04 INFO] (SDKD log:137) "DelayMax": 10, [545.04 INFO] (SDKD log:137) "DelayMin": 1, [545.04 INFO] (SDKD log:137) "IterWait": 1, [545.04 INFO] (SDKD log:137) "ViewQueryCount": -1, [545.05 INFO] (SDKD log:137) "ViewQueryDelay": 0, [545.05 INFO] (SDKD log:137) "TimeRes": 1 [545.05 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.05 INFO] (SDKD log:137) }, [545.05 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.06 INFO] (SDKD log:137) "ViewName": "test_view" [545.06 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.06 INFO] (SDKD log:137) } [545.06 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.06 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle [545.06 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.07 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 134 [545.07 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.07 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew [545.07 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.08 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.08 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.08 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.08 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.09 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.09 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.10 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.10 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.10 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.10 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.11 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.11 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [545.13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@134.71 [545.13 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@134.72 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={limit=0}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, Vie wQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [545.14 INFO] (Scenario run:83) RAMP phase started. Waiting for 30 seconds [545.14 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Got command CB_VIEW_QUERY [545.15 INFO] (SDKD log:137) [Sdkd.Handle|Warn] Command data: { [545.15 INFO] (SDKD log:137) "DSType": "DSTYPE_SEEDED", [545.15 INFO] (SDKD log:137) "DS": {}, [545.15 INFO] (SDKD log:137) "ViewParameters": { [545.15 INFO] (SDKD log:137) "limit": 0 [545.15 INFO] (SDKD log:137) }, [545.16 INFO] (SDKD log:137) "DesignName": "test_design", [545.16 INFO] (SDKD log:137) "Options": { [545.16 INFO] (SDKD log:137) "DelayMax": 10, [545.16 INFO] (SDKD log:137) "DelayMin": 1, [545.16 INFO] (SDKD log:137) "IterWait": 1, [545.16 INFO] (SDKD log:137) "ViewQueryCount": -1, [545.17 INFO] (SDKD log:137) "ViewQueryDelay": 0, [545.17 INFO] (SDKD log:137) "TimeRes": 1 [545.17 INFO] (SDKD log:137) }, [545.17 INFO] (SDKD log:137) "ViewName": "test_view" [545.18 INFO] (SDKD log:137) +++ Received exception of ID 0 [545.18 INFO] (SDKD log:137) } [545.18 INFO] (SDKD log:137) log4net:ERROR RendererMap: Exception while rendering object of type [Common.Logging.Factory.AbstractLogger+FormatMessageCallbackFormattedMessage] [545.18 INFO] (SDKD log:137) System.ObjectDisposedException: Cannot access a disposed object. [545.18 INFO] (SDKD log:137) Object name: 'System.Net.Sockets.Socket'. [575.14 INFO] (Scenario run:86) Starting CHANGE phase [575.20 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [576.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [577.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [579.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [581.33 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [583.27 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 1% [586.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [587.51 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 2% [588.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3% [589.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3% [590.57 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 3% [591.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 4% [592.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5% [593.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 5% [594.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 6% [595.85 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% [596.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 7% [597.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 8% [598.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9% [599.93 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 9% [600.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10% [601.98 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 10% [603.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11% [604.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 11% [605.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12% [606.14 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 12% [607.16 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13% [608.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 13% [609.28 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14% [610.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 14% [611.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15% [612.39 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 15% [613.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16% [614.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 16% [615.43 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 17% [616.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18% [617.49 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 18% [618.50 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19% [619.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 19% [620.53 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20% [621.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 20% [622.56 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21% [623.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 21% [624.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22% [625.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 22% [626.71 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 23% [627.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 24% [628.77 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 24% [629.79 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25% [630.80 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 25% [631.81 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26% [632.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26% [633.86 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 26% [634.87 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 27% [635.89 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 27% [636.90 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28% [637.91 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 28% [638.92 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29% [639.94 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 29% [640.95 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 30% [641.99 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31% [643.02 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 31% [643.63 INFO] (SDKD log:137) +++ Received exception of ID 0 [643.63 INFO] (SDKD log:137) [643.66 INFO] (SDKD log:137) Unhandled Exception: System.InvalidOperationException: Collection was modified; enumeration operation may not execute. [644.03 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32% [645.04 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32% [646.05 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 32% [647.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33% [648.06 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 33% [649.07 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 34% [650.10 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35% [651.17 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 35% [652.18 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36% [653.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 36% [654.22 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [655.23 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [656.24 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 38% [657.25 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39% [658.30 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40% [659.34 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40% [660.35 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 40% [661.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 41% [662.36 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42% [663.37 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 42% [664.38 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43% [665.40 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 43% [666.41 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 44% [667.42 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45% [668.43 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 45% [669.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 46% [670.47 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 47% [671.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 47% [672.52 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48% [673.54 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 48% [674.55 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49% [675.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 49% [676.58 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50% [677.59 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 50% [678.60 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51% [679.60 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51% [680.61 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 51% [681.62 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 52% [682.63 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53% [683.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 53% [684.64 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 54% [685.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 54% [686.65 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55% [687.66 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 55% [688.67 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56% [689.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56% [690.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 56% [691.69 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57% [692.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 57% [693.70 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58% [694.72 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 58% [695.73 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59% [696.70 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [696.71 INFO] (SDKD log:137) at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource) at System.Collections.Generic.List`1.Enumerator.MoveNextRare() at System.Collections.Generic.List`1.Enumerator.MoveNext() at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source) at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Configuration\CouchbaseConfigContext.cs:line 47 at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.UpdateConfig(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Co nfiguration\Server\Providers\CarrierPublication\CarrierPublicationProvider.cs:line 91 at Couchbase.Core.ClusterManager.NotifyConfigPublished(IBucketConfig bucketConfig) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 137 at Couchbase.Core.Buckets.CouchbaseBucket.CheckForConfigUpdates[T](IOperationResult`1 operationResult) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.c s:line 234 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, IServer server) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couc hbase\Core\Buckets\CouchbaseBucket.cs:line 166 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace-net2\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace-net2\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 231 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace-net2\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 153 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [696.74 ERROR] (BRun maybePropagate:237) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [696.76 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 59% [696.76 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73 [696.76 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:281) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:388) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:472) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:504) [sdkdclient-1.0-SNAPSHOT.jar:na] c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng>

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    after some time the tests begin to fall with 2014-07-28 00:59:20,864 [27] DEBUG Couchbase.IO.Strategies.DefaultIOStrategy - Could not authenticate default using Couchbase.Authentication.SASL.CramMd5Mechanism. Running Rb2Out-HYBRID: 2014-07-28 00:05:18 rebalance/mode=out workload=HybridWorkloadGroup rebalance/count=2 testcase=RebalanceScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-973\Rb2Out-HYBRID\07-28-14\025518\7d8cf558a26ca99fe4dc3028bc6b2ec0 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ --rebalance-mode out --workload HybridWorkloadGroup \ --rebalance-count 2 --testcase RebalanceScenario -C share\rexec \ --rexec_path C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port \ 8675 --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 \ --cluster_ssh-username=root --cluster_ssh-password=couchbase ---------------------------------------->8 [1.47 WARN] (Drivers getDriver:76) 'rexec' is now mapped to *local* execution only. Use RemoteExecutingDriver for remote execution [1.87 INFO] (RunContext run:102) Ramp for 30 seconds. Cluster modification: remove 2 nodes and rebalance. Rebound for 90 seconds. [1.88 INFO] (RunContext run:124) Starting cluster and driver [1.89 INFO] (CBCluster startCluster:360) Node http://10.3.121.134:8091 is master now [1.89 INFO] (HostPortDriver launch:32) Invoking SDKD as 'C:\temp\sdkd-out-debug\SdkdConsole.exe'  [1.89 DEBUG] (CBCluster startCluster:366) Stopping any existing rebalance operations.. [2.26 INFO] (SDKD log:137) --- Logging Self-Test --- [2.27 INFO] (SDKD log:137) [Sdkd.Main|Info] Info Message  [2.27 INFO] (SDKD log:137) [Sdkd.Main|Warn] Warn Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Error] Error Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Fatal] Fatal Message  [2.28 INFO] (SDKD log:137) [Sdkd.Main|Info] SDKD Listening on port 8675  [2.42 DEBUG] (Handle sendMessageAsync:183) > INFO@0.0 [2.43 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [2.76 DEBUG] (Handle receiveMessage:158) < INFO@0.0 => {"CAPS":{"DS_SHARED":true,"PREAMBLE":false,"CANCEL":true,"CONTINUOUS":true},"COMPONENTS":{"SDK":"1.0.0.0","CLR":"4.0.30319.17929"}} [4.02 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.135:8091,ns_1@10.3.121.135> [4.06 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.121.136:8091,ns_1@10.3.121.136> [4.11 DEBUG] (CBCluster clearSingleCluster:140) Failing over existing node <URI:10.3.3.206:8091,ns_1@10.3.3.206> [4.18 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.18 DEBUG] (SSHConnection connect:99) Connecting with User[root] Pass[HASH=3b8cf05627127baaf3808913209aac84] [4.70 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.134:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.3.206:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.135:8091 [4.71 INFO] (NodeHost createSSH:147) SSH Initialized for http://10.3.121.136:8091 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.134 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.3.206 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.136 [4.73 DEBUG] (SSHCommand execute:75) Running /etc/init.d/couchbase-server start && pkill -CONT -f memcached && pkill -CONT -f beam.smp && iptables -F && iptables -t nat -F on 10.3.121.135 [4.94 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@641a034d [4.94 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@19c6866d [4.95 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@51c293f8 [4.95 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@5a9b8ff9 [4.95 DEBUG] (CBCluster setupNewCluster:271) Provisioning initial node com.couchbase.cbadmin.client.CouchbaseAdmin@6e0f0ad [18.52 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.135:8091 [26.45 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.121.136:8091 [30.42 DEBUG] (CBCluster tryOnce:286) Adding node http://10.3.3.206:8091 [35.78 INFO] (CBCluster setupNewCluster:293) All nodes added. Will rebalance [35.84 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [36.84 INFO] (RebalanceWaiter sweepOnce:33) Rebalance complete [36.85 DEBUG] (CBCluster setupServerGroups:222) Not creating any groups [36.85 INFO] (CBCluster setupMainBucket:209) Creating bucket default [36.87 INFO] (CBCluster setupMainBucket:211) Bucket creation submitted [42.18 INFO] (CBCluster waitForBucketReady:203) Bucket creation done [42.23 INFO] (RunContext run:143) Driver and cluster initialized [42.26 INFO] (RunContext call:167) Running scenario.. [42.26 INFO] (Scenario run:72) Starting RAMP phase [42.26 INFO] (Workload setupDesign:63) Creating design test_design [42.51 INFO] (Workload setupDesign:80) Design creation done [42.51 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@101.1 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [42.52 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [42.53 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 101  [42.54 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [42.56 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving Common.Logging.Log4Net  [42.56 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly Common.Logging.Log4Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=af08829b84f0328e  [42.59 INFO] (SDKD log:137) [Sdkd.Main|Info] Resolving log4net  [42.59 INFO] (SDKD log:137) [Sdkd.Main|Info] Have assembly log4net, Version=1.2.10.0, Culture=neutral, PublicKeyToken=1b44e1d426115821  [42.91 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@101.1 [42.91 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@101.2 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [43.02 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409  [43.39 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: { [43.39 INFO] (SDKD log:137)  "0": 44, [43.40 INFO] (SDKD log:137)  "264": 956 [43.40 INFO] (SDKD log:137) }  [43.40 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@101.2 => {[OK]: 44, [MEMD:GENERIC]: 956} [43.40 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@101.3 [43.40 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@102.4 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.41 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 102  [43.41 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.42 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@102.4 [43.42 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_LOAD@102.5 => {DSType=DSTYPE_SEEDED, DS={KSeed=ViewFillerSeed, VSeed=ViewFillerSeed, Count=1000, Continuous=false, VSize=512, Repeat=rep, KSize=12}, Schema={InflateContent=meh, InflateLevel=40}, Options={DelayMax=0, DelayMin=0, IterWait=1, TimeRes=0}} [43.82 INFO] (SDKD log:137) [Sdkd.ViewLoader|Info] Summary details: { [43.82 INFO] (SDKD log:137)  "0": 39, [43.83 INFO] (SDKD log:137)  "264": 961 [43.83 INFO] (SDKD log:137) }  [43.83 DEBUG] (Handle receiveMessage:158) < CB_VIEW_LOAD@102.5 => {[OK]: 39, [MEMD:GENERIC]: 961} [43.83 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@102.6 [43.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.83 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@103.7 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.83 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 103  [43.83 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.85 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@103.7 [43.85 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@103.8 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.85 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@104.9 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.85 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.86 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 104  [43.86 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.87 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@104.9 [43.88 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@104.10 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.88 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@105.11 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.88 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 105  [43.88 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.90 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@105.11 [43.90 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@105.12 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.90 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.90 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@106.13 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.91 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 106  [43.91 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.92 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@106.13 [43.93 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@106.14 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.93 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.93 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@107.15 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 107  [43.94 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.97 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@107.15 [43.97 DEBUG] (Handle sendMessageAsync:183) > CB_VIEW_QUERY@107.16 => {DSType=DSTYPE_SEEDED, DS={}, ViewParameters={}, DesignName=test_design, Options={DelayMax=10, DelayMin=1, IterWait=1, ViewQueryCount=-1, ViewQueryDelay=0, TimeRes=1}, ViewName=test_view} [43.97 INFO] (Workload call:112) Preloading items into cluster before workload [43.97 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [43.97 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@108.17 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [43.98 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 108  [43.98 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [43.99 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@108.17 [43.99 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@108.18 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=false, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=0, DelayMin=0, IterWait=100, TimeRes=0}} [51.93 DEBUG] (Handle receiveMessage:158) < MC_DS_MUTATE_SET@108.18 => {[OK]: 364, [MEMD:GENERIC]: 14636} [51.93 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@108.19 [51.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@111.22 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@112.23 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@115.26 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@116.27 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@110.21 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@109.20 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@114.25 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@113.24 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.94 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@117.28 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.95 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@118.29 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [51.96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.96 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [51.99 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@111.22 [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.00 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@111.30 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.00 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 111  [52.00 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 110  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 115  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 109  [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 114  [52.01 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@119.31 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.01 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 116  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 117  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 118  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 112  [52.02 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.02 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 113  [52.03 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.03 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@110.21 [52.03 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 119  [52.03 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.03 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@110.32 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.04 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@120.33 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.04 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.05 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 120  [52.07 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.07 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@115.26 [52.07 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@115.34 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.08 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.08 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@121.35 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.09 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 121  [52.10 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.10 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@109.20 [52.10 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@109.36 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.11 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.11 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@122.37 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.12 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 122  [52.12 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.13 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@114.25 [52.13 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@114.38 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.13 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@123.39 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.13 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.14 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 123  [52.16 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.16 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@116.27 [52.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@116.40 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.17 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@124.41 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.17 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.18 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 124  [52.24 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.25 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@117.28 [52.25 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@117.42 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.26 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@125.43 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.26 INFO] (SDKD log:137)  [52.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.26 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 125  [52.27 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.30 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.30 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@118.29 [52.31 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 1 [52.31 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@118.44 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.31 INFO] (SDKD log:137)  at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 75 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 40 at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 82 at Couchbase.Core.Buckets.CouchbaseBucket.Upsert[T](String key, T value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 121 at Sdkd.Commands.SetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\SetCommand.cs:line 77 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 191 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 123 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [52.31 INFO] (SDKD log:137)  [52.32 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.32 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@126.45 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.32 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.33 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 126  [52.34 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.35 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@112.23 [52.35 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@112.46 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.37 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@127.47 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.37 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 127  [52.40 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@113.24 [52.40 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@113.48 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.40 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.41 DEBUG] (Handle sendMessageAsync:183) > NEWHANDLE@128.49 => {Port=8091, Bucket=default, Options={SSL=false, Username=default, Password=, OtherNodes=[["10.3.121.135",8091],["10.3.121.136",8091],["10.3.3.206",8091]]}, Hostname=10.3.121.134} [52.41 INFO] (SDKD log:137) [Sdkd.Control|Info] Got a new connection. Creating child handle  [52.42 INFO] (SDKD log:137) [Sdkd.Control|Info] Registering handle 128  [52.45 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.45 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@119.31 [52.46 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@119.50 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.52 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.53 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@120.33 [52.53 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@120.51 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.55 INFO] (SDKD log:137) +++ Received exception of ID 0 [52.55 INFO] (SDKD log:137)  [52.56 INFO] (SDKD log:137) Unhandled Exception: System.Security.Authentication.AuthenticationException: default [52.63 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.63 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@121.35 [52.63 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@121.52 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.67 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.68 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@122.37 [52.68 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@122.53 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [52.76 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [52.77 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@123.39 [52.77 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@123.54 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.05 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [53.05 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@124.41 [53.05 DEBUG] (Handle sendMessageAsync:183) > MC_DS_GET@124.55 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.15 INFO] (SDKD log:137) [Sdkd.ClientFactory|Info] Shared clients disabled. Creating anew  [53.15 DEBUG] (Handle receiveMessage:158) < NEWHANDLE@125.43 [53.16 DEBUG] (Handle sendMessageAsync:183) > MC_DS_MUTATE_SET@125.56 => {DSType=DSTYPE_SEEDED, DS={KSeed=SimpleKey, VSeed=SimpleValue, Count=15000, Continuous=true, VSize=128, Repeat=REP, KSize=32}, Options={DelayMax=10, DelayMin=1, IterWait=1, TimeRes=1}} [53.17 INFO] (LineGobbler doFilter:115) +++ Following exception has internal ID: 3 [53.18 INFO] (SDKD log:137)  at Couchbase.IO.Strategies.DefaultIOStrategy.Authenticate(IConnection connection) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 75 at Couchbase.IO.Strategies.DefaultIOStrategy.Execute[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\IO\Strategies\DefaultIOStrategy.cs:line 40 at Couchbase.Core.Server.Send[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Server.cs:line 82 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 282 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:line 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 191 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 123 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [53.18 INFO] (SDKD log:137)  [53.19 INFO] (SDKD log:137) Unhandled Exception: Couchbase.Configuration.ConfigException: Could not bootstrap default. See log for details. [65.28 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [65.29 INFO] (SDKD log:137)  at Couchbase.Core.ClusterManager.CreateBucket(String bucketName, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\ClusterManager.cs:line 210 at Couchbase.CouchbaseCluster.OpenBucket(String bucketname, String password) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\CouchbaseCluster.cs:line 82 at Sdkd.Couchbase.ClientFactory.FindExistingClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 55 at Sdkd.Couchbase.ClientFactory.CreateClient(ClientConfiguration config) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Couchbase\ClientFactory.cs:line 262 at SdkdConsole.Daemon.Handle.CreateHandle(Request req) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 84 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 103 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [65.30 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [65.31 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.57 [65.31 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na]

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed
  6. 0

    call packages\sdkdclient\bin\brun.bat -A S3Creds_tmp -I cluster_conf ig.ini -I sdkd.args --variants HYBRID -d all:trace [0.30 INFO] (BRun run:436) Initializing S3 [1.12 INFO] (BRun run:441) Initializing history database [1.75 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools ============================================================ Running ReAdd2-HYBRID: 2014-08-26 22:31:28 workload=HybridWorkloadGroup failover/next_action=FO_READD_REBALANCE failover/count=2 failover/next_delay=45 testcase=FailoverScenario Logging to c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkdclient-ng\log\SDK-SDK\CB-3.0.0-1174\ReAdd2-HYBRID\08-26-14\0198 88\0dcd5a5bc61e625f06a757ab9dfa7db5 To re-run the test, copy/paste the following into the shell. You may also copy/paste (except the first line) into an argfile 8<---------------------------------------- ./stester \ \ --workload HybridWorkloadGroup --failover-next_action \ FO_READD_REBALANCE --failover-count 2 --failover-next_delay 45 \ --testcase FailoverScenario -C share\rexec --rexec_path \ C:\temp\sdkd-out-debug\SdkdConsole.exe --rexec_port 8675 \ --cluster_node=10.3.121.134 --cluster_node=10.3.121.135 \ --cluster_node=10.3.121.136 --cluster_node=10.3.3.206 --cluster_ssh-username=root --cluster_ssh-password=couchbase ..... [123.71 INFO] (Scenario run:86) Starting CHANGE phase [123.71 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.135:8091,ns_1@10.3.121.135> [123.72 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver [124.10 INFO] (SDKD log:137) [Sdkd.Result|Warn] Unrecognized return code 409 [124.13 INFO] (CBCluster failoverNodes:588) Failing over <URI:10.3.121.136:8091,ns_1@10.3.121.136> [124.14 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/failOver [124.40 INFO] (FailoverAction change:104) Nodes failed over. Requested sleep for 45 seconds [125.61 INFO] (SDKD log:137) [125.63 INFO] (SDKD log:137) Unhandled Exception: Couchbase.ServerUnavailableException: Exception of type 'Couchbase.ServerUnavail ableException' was thrown. [169.41 INFO] (FailoverAction doNextAction:68) Readding nodes after failover [169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode [169.41 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/reAddNode [169.42 INFO] (FailoverAction doNextAction:85) Rebalancing nodes after failover [169.42 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/nodes [169.43 TRACE] (CouchbaseAdmin getResponseJson:167) POST http://10.3.121.134:8091/controller/rebalance [169.45 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [169.45 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 0% [170.46 TRACE] (CouchbaseAdmin getResponseJson:167) GET .... [196.67 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [196.68 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 37% [197.59 INFO] (SDKD log:137) [Sdkd.ViewQuery|Error] ... System.NullReferenceException Object reference not set to an instance of a n object. at Sdkd.Views.ViewCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Views\ViewComman d.cs:line 268 [197.61 INFO] (SDKD log:137) +++ Flushing remaining exception trace buffer [197.61 INFO] (SDKD log:137) at Couchbase.Configuration.ConfigContextBase.GetServer() in c:\jenkins\workspace\.net-sdkdclient-n g2-andrei\couchbase-net-client\Src\Couchbase\Configuration\ConfigContextBase.cs:line 210 at Couchbase.Core.Buckets.CouchbaseBucket.HandleIOError[T](IOperation`1 operation, IServer server) in c:\jenkins\workspace\.net -sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 194 at Couchbase.Core.Buckets.CouchbaseBucket.CanRetryOperation[T](IOperationResult`1 operationResult, IOperation`1 operation, ISer ver server) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs: line 179 at Couchbase.Core.Buckets.CouchbaseBucket.SendWithRetry[T](IOperation`1 operation) in c:\jenkins\workspace\.net-sdkdclient-ng2- andrei\couchbase-net-client\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 104 at Couchbase.Core.Buckets.CouchbaseBucket.Get[T](String key) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\couchbase-net-c lient\Src\Couchbase\Core\Buckets\CouchbaseBucket.cs:line 888 at Sdkd.Commands.GetCommand.ExecuteSingle(String key, String value) in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net \src\Sdkd\Commands\GetCommand.cs:line 18 at Sdkd.Commands.KVCommand.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\Sdkd\Commands\KVCommand.cs:lin e 25 at SdkdConsole.Daemon.Handle.DispatchCommand(Request req, CouchbaseBucket cli) in c:\jenkins\workspace\.net-sdkdclient-ng2-andr ei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs:line 295 at SdkdConsole.Daemon.Handle.Run() in c:\jenkins\workspace\.net-sdkdclient-ng2-andrei\sdkd-net\src\SdkdConsole\Daemon\Handle.cs :line 175 at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boo lean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean pre serveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() [197.64 ERROR] (BRun maybePropagate:243) Suppressing driver-related error com.couchbase.sdkdclient.context.HarnessException: Driver crashed at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) ~[na:1.6.0_27] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.6.0_27] at java.lang.Thread.run(Unknown Source) ~[na:1.6.0_27] [197.66 DEBUG] (Handle sendMessageAsync:183) > CLOSEHANDLE@103.73 [197.66 TRACE] (Handle sendMessageAsync:185) > {"Command":"CLOSEHANDLE","Handle":103,"ReqID":73,"CommandData":{}} [197.66 WARN] (STester close:170) While stopping driver java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.6.0_27] at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[na:1.6.0_27] at java.net.SocketOutputStream.write(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.implFlush(Unknown Source) ~[na:1.6.0_27] at sun.nio.cs.StreamEncoder.flush(Unknown Source) ~[na:1.6.0_27] at java.io.OutputStreamWriter.flush(Unknown Source) ~[na:1.6.0_27] at java.io.BufferedWriter.flush(Unknown Source) ~[na:1.6.0_27] at com.couchbase.sdkdclient.handle.Handle.sendMessageAsync(Handle.java:188) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.handle.Handle.close(Handle.java:254) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HandleManager.stopAll(HandleManager.java:26) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.HostPortDriver.stop(HostPortDriver.java:118) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.driver.ExecutingDriver.stop(ExecutingDriver.java:117) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.stester.STester.close(STester.java:167) ~[sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runHarness(BRun.java:287) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.runSingleTest(BRun.java:394) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.run(BRun.java:478) [sdkdclient-1.0-SNAPSHOT.jar:na] at com.couchbase.sdkdclient.batch.BRun.main(BRun.java:510) [sdkdclient-1.0-SNAPSHOT.jar:na] [197.68 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress [197.83 DEBUG] (RebalanceWaiter sweepOnce:38) Rebalance Progress: 39% [198.80 INFO] (BRun uploadFile:141) Uploaded RAW => http://sdk-testresults.couchbase.com.s3.amazonaws.com/SDK-SDK/CB-3.0.0-1174/Re Add2-HYBRID/08-26-14/019888/0dcd5a5bc61e625f06a757ab9dfa7db5.zip [198.83 TRACE] (CouchbaseAdmin getResponseJson:167) GET http://10.3.121.134:8091/pools/default/rebalanceProgress

    Couchbase JIRA | 2 years ago | Andrei Baranouski
    com.couchbase.sdkdclient.context.HarnessException: Driver crashed

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. com.couchbase.sdkdclient.context.HarnessException

      Driver crashed

      at com.couchbase.sdkdclient.context.RunContext$5.call()
    2. com.couchbase.sdkdclient
      RunContext$5.call
      1. com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:192)[sdkdclient-1.0-SNAPSHOT.jar:na]
      2. com.couchbase.sdkdclient.context.RunContext$5.call(RunContext.java:183)[sdkdclient-1.0-SNAPSHOT.jar:na]
      2 frames
    3. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)[na:1.6.0_16]
      2. java.util.concurrent.FutureTask.run(Unknown Source)[na:1.6.0_16]
      3. java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)[na:1.6.0_16]
      4. java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)[na:1.6.0_16]
      5. java.util.concurrent.FutureTask.run(Unknown Source)[na:1.6.0_16]
      6. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)[na:1.6.0_16]
      7. java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)[na:1.6.0_16]
      8. java.lang.Thread.run(Unknown Source)[na:1.6.0_16]
      8 frames