com.mongodb.MongoException$Network: Operation on server localhost:27017 failed

JIRA | Chung-yen Chang | 1 year ago
  1. 0

    While running the load phase of a YCSB test, wiredTiger encountered an error and panic. This has happened 3 out of 4 times so it seems fairly reproducible. The workload is doing a large number of inserts of documents that are 1K each. The mongod log reports some slow inserts (>100 msec) and then showed the following line about wiredTiger. This was seen on an EC2 instance running Enterprise version of mongod (3.1.7-pre, git version: 1b5b55cb48ff9daca74177b86d336613d207c5d1). {noformat} 015-08-11T08:53:47.184+0000 E STORAGE [thread1] WiredTiger (-31802) [1439283227:184443][18776:0x7fdf70a65700], file:WiredTiger.wt, WT_SESSION.checkpoint: WiredTiger.wt read error: failed to read 4096 bytes at offset 32768: WT_ERROR: non-specific WiredTiger error 2015-08-11T08:53:47.184+0000 E STORAGE [thread1] WiredTiger (-31802) [1439283227:184629][18776:0x7fdf70a65700], checkpoint-server: checkpoint server error: WT_ERROR: non-specific WiredTiger error 2015-08-11T08:53:47.184+0000 E STORAGE [thread1] WiredTiger (-31804) [1439283227:184661][18776:0x7fdf70a65700], checkpoint-server: the process must exit and restart: WT_PANIC: WiredTiger library panic 2015-08-11T08:53:47.184+0000 I - [conn9] Fatal Assertion 28559 2015-08-11T08:53:47.184+0000 I - [conn16] Fatal Assertion 28559 {noformat} One the console where the YCSB test were run, these error repeated many times around the problem and are mostly related. Part of this error message that is interesting is where it says "Caused by: java.io.EOFException), which could be related to the error in mongod log. {noformat} com.mongodb.MongoException$Network: Operation on server localhost:27017 failed at com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:215) at com.mongodb.DBCollectionImpl.writeWithCommandProtocol(DBCollectionImpl.java:461) at com.mongodb.DBCollectionImpl.insertWithCommandProtocol(DBCollectionImpl.java:426) at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:188) at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:167) at com.mongodb.DBCollection.insert(DBCollection.java:93) at com.mongodb.DBCollection.insert(DBCollection.java:78) at com.mongodb.DBCollection.insert(DBCollection.java:120) at com.yahoo.ycsb.db.MongoDbClient.insert(MongoDbClient.java:246) at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:148) at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461) at com.yahoo.ycsb.ClientThread.run(Client.java:269) Caused by: java.io.EOFException at org.bson.io.Bits.readFully(Bits.java:75) at org.bson.io.Bits.readFully(Bits.java:50) at org.bson.io.Bits.readFully(Bits.java:37) at com.mongodb.Response.<init>(Response.java:42) at com.mongodb.DBCollectionImpl.receiveWriteCommandMessage(DBCollectionImpl.java:532) at com.mongodb.DBCollectionImpl.access$300(DBCollectionImpl.java:48) at com.mongodb.DBCollectionImpl$2.execute(DBCollectionImpl.java:474) at com.mongodb.DBCollectionImpl$2.execute(DBCollectionImpl.java:461) at com.mongodb.DBPort.doOperation(DBPort.java:187) at com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:208) ... 11 more {noformat}

    JIRA | 1 year ago | Chung-yen Chang
    com.mongodb.MongoException$Network: Operation on server localhost:27017 failed
  2. 0

    While running the load phase of a YCSB test, wiredTiger encountered an error and panic. This has happened 3 out of 4 times so it seems fairly reproducible. The workload is doing a large number of inserts of documents that are 1K each. The mongod log reports some slow inserts (>100 msec) and then showed the following line about wiredTiger. This was seen on an EC2 instance running Enterprise version of mongod (3.1.7-pre, git version: 1b5b55cb48ff9daca74177b86d336613d207c5d1). {noformat} 015-08-11T08:53:47.184+0000 E STORAGE [thread1] WiredTiger (-31802) [1439283227:184443][18776:0x7fdf70a65700], file:WiredTiger.wt, WT_SESSION.checkpoint: WiredTiger.wt read error: failed to read 4096 bytes at offset 32768: WT_ERROR: non-specific WiredTiger error 2015-08-11T08:53:47.184+0000 E STORAGE [thread1] WiredTiger (-31802) [1439283227:184629][18776:0x7fdf70a65700], checkpoint-server: checkpoint server error: WT_ERROR: non-specific WiredTiger error 2015-08-11T08:53:47.184+0000 E STORAGE [thread1] WiredTiger (-31804) [1439283227:184661][18776:0x7fdf70a65700], checkpoint-server: the process must exit and restart: WT_PANIC: WiredTiger library panic 2015-08-11T08:53:47.184+0000 I - [conn9] Fatal Assertion 28559 2015-08-11T08:53:47.184+0000 I - [conn16] Fatal Assertion 28559 {noformat} One the console where the YCSB test were run, these error repeated many times around the problem and are mostly related. Part of this error message that is interesting is where it says "Caused by: java.io.EOFException), which could be related to the error in mongod log. {noformat} com.mongodb.MongoException$Network: Operation on server localhost:27017 failed at com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:215) at com.mongodb.DBCollectionImpl.writeWithCommandProtocol(DBCollectionImpl.java:461) at com.mongodb.DBCollectionImpl.insertWithCommandProtocol(DBCollectionImpl.java:426) at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:188) at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:167) at com.mongodb.DBCollection.insert(DBCollection.java:93) at com.mongodb.DBCollection.insert(DBCollection.java:78) at com.mongodb.DBCollection.insert(DBCollection.java:120) at com.yahoo.ycsb.db.MongoDbClient.insert(MongoDbClient.java:246) at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:148) at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461) at com.yahoo.ycsb.ClientThread.run(Client.java:269) Caused by: java.io.EOFException at org.bson.io.Bits.readFully(Bits.java:75) at org.bson.io.Bits.readFully(Bits.java:50) at org.bson.io.Bits.readFully(Bits.java:37) at com.mongodb.Response.<init>(Response.java:42) at com.mongodb.DBCollectionImpl.receiveWriteCommandMessage(DBCollectionImpl.java:532) at com.mongodb.DBCollectionImpl.access$300(DBCollectionImpl.java:48) at com.mongodb.DBCollectionImpl$2.execute(DBCollectionImpl.java:474) at com.mongodb.DBCollectionImpl$2.execute(DBCollectionImpl.java:461) at com.mongodb.DBPort.doOperation(DBPort.java:187) at com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:208) ... 11 more {noformat}

    JIRA | 1 year ago | Chung-yen Chang
    com.mongodb.MongoException$Network: Operation on server localhost:27017 failed
  3. 0

    InMemoryMongoServer doesn't work with multiple test classes in a module

    GitHub | 2 years ago | jewzaam
    com.mongodb.MongoException$Network: Operation on server localhost:27777 failed
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    MongoDB SocketTimeoutException occurs even though value is set to as high as 20s with write concern set to acknowledged?

    Stack Overflow | 1 year ago | gravetii
    com.mongodb.MongoException$Network: Operation on server 127.0.0.1:27013 failed
  6. 0

    MongoDB connectivity issue

    Stack Overflow | 11 months ago | Mounika Nannaka
    com.mongodb.MongoException$Network: can't say something

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.EOFException

      No message provided

      at org.bson.io.Bits.readFully()
    2. MongoDB Java Driver
      DBCollection.insert
      1. org.bson.io.Bits.readFully(Bits.java:75)
      2. org.bson.io.Bits.readFully(Bits.java:50)
      3. org.bson.io.Bits.readFully(Bits.java:37)
      4. com.mongodb.Response.<init>(Response.java:42)
      5. com.mongodb.DBCollectionImpl.receiveWriteCommandMessage(DBCollectionImpl.java:532)
      6. com.mongodb.DBCollectionImpl.access$300(DBCollectionImpl.java:48)
      7. com.mongodb.DBCollectionImpl$2.execute(DBCollectionImpl.java:474)
      8. com.mongodb.DBCollectionImpl$2.execute(DBCollectionImpl.java:461)
      9. com.mongodb.DBPort.doOperation(DBPort.java:187)
      10. com.mongodb.DBTCPConnector.doOperation(DBTCPConnector.java:208)
      11. com.mongodb.DBCollectionImpl.writeWithCommandProtocol(DBCollectionImpl.java:461)
      12. com.mongodb.DBCollectionImpl.insertWithCommandProtocol(DBCollectionImpl.java:426)
      13. com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:188)
      14. com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:167)
      15. com.mongodb.DBCollection.insert(DBCollection.java:93)
      16. com.mongodb.DBCollection.insert(DBCollection.java:78)
      17. com.mongodb.DBCollection.insert(DBCollection.java:120)
      17 frames
    3. com.yahoo.ycsb
      ClientThread.run
      1. com.yahoo.ycsb.db.MongoDbClient.insert(MongoDbClient.java:246)
      2. com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:148)
      3. com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461)
      4. com.yahoo.ycsb.ClientThread.run(Client.java:269)
      4 frames