com.datastax.driver.core.ClusterNameMismatchException: [/ip:9042] Host /ip:9042 reports cluster name 'xyz cluster' that doesn't match our cluster name 'dev3 dev3 cluster'. This host will be ignored.

DataStax JIRA | Vishy Kasar | 2 years ago
  1. 0

    We had few production clusters which over time ended up with incorrect entries in system.peers table. The entries pointed to nodes in another cluster. java driver connects to them, checks the cluster name and rejects such connections. However: 1. It does not close those connections thus leaking them 2. It keeps retrying the connections thus increasing the leaks Over time, the client runs out of file descriptors and we have to restart the process. You can recreate the issue by creating 3 node local cluster, injecting a reachable node from another cluster in system.peers of one node and throwing some heavy load. A test program that reproduces the issue is attached. Error creating pool to /ip:9042 com.datastax.driver.core.ClusterNameMismatchException: [/ip:9042] Host /ip:9042 reports cluster name 'xyz cluster' that doesn't match our cluster name 'dev3 dev3 cluster'. This host will be ignored. at com.datastax.driver.core.Connection.checkClusterName(Connection.java:240) at com.datastax.driver.core.Connection.initializeTransport(Connection.java:156) at com.datastax.driver.core.Connection.<init>(Connection.java:112) at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:35) at com.datastax.driver.core.Connection$Factory.open(Connection.java:522) at com.datastax.driver.core.HostConnectionPool.<init>(HostConnectionPool.java:86) at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:269) at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:39) at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:301) at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:293) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:695)

    DataStax JIRA | 2 years ago | Vishy Kasar
    com.datastax.driver.core.ClusterNameMismatchException: [/ip:9042] Host /ip:9042 reports cluster name 'xyz cluster' that doesn't match our cluster name 'dev3 dev3 cluster'. This host will be ignored.
  2. 0

    We had few production clusters which over time ended up with incorrect entries in system.peers table. The entries pointed to nodes in another cluster. java driver connects to them, checks the cluster name and rejects such connections. However: 1. It does not close those connections thus leaking them 2. It keeps retrying the connections thus increasing the leaks Over time, the client runs out of file descriptors and we have to restart the process. You can recreate the issue by creating 3 node local cluster, injecting a reachable node from another cluster in system.peers of one node and throwing some heavy load. A test program that reproduces the issue is attached. Error creating pool to /ip:9042 com.datastax.driver.core.ClusterNameMismatchException: [/ip:9042] Host /ip:9042 reports cluster name 'xyz cluster' that doesn't match our cluster name 'dev3 dev3 cluster'. This host will be ignored. at com.datastax.driver.core.Connection.checkClusterName(Connection.java:240) at com.datastax.driver.core.Connection.initializeTransport(Connection.java:156) at com.datastax.driver.core.Connection.<init>(Connection.java:112) at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:35) at com.datastax.driver.core.Connection$Factory.open(Connection.java:522) at com.datastax.driver.core.HostConnectionPool.<init>(HostConnectionPool.java:86) at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:269) at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:39) at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:301) at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:293) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:695)

    DataStax JIRA | 2 years ago | Vishy Kasar
    com.datastax.driver.core.ClusterNameMismatchException: [/ip:9042] Host /ip:9042 reports cluster name 'xyz cluster' that doesn't match our cluster name 'dev3 dev3 cluster'. This host will be ignored.
  3. 0

    system.local cluster_name is not according to origin defaults

    GitHub | 1 year ago | slivne
    com.datastax.driver.core.ClusterNameMismatchException: [localhost/127.0.0.1:9042] Host localhost/127.0.0.1:9042 reports cluster name 'urchin' that doesn't match our cluster name 'Test Cluster'. This host will be ignored.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. com.datastax.driver.core.ClusterNameMismatchException

      [/ip:9042] Host /ip:9042 reports cluster name 'xyz cluster' that doesn't match our cluster name 'dev3 dev3 cluster'. This host will be ignored.

      at com.datastax.driver.core.Connection.checkClusterName()
    2. DataStax Java Driver for Apache Cassandra - Core
      SessionManager$3.call
      1. com.datastax.driver.core.Connection.checkClusterName(Connection.java:240)
      2. com.datastax.driver.core.Connection.initializeTransport(Connection.java:156)
      3. com.datastax.driver.core.Connection.<init>(Connection.java:112)
      4. com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:35)
      5. com.datastax.driver.core.Connection$Factory.open(Connection.java:522)
      6. com.datastax.driver.core.HostConnectionPool.<init>(HostConnectionPool.java:86)
      7. com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:269)
      8. com.datastax.driver.core.SessionManager.access$400(SessionManager.java:39)
      9. com.datastax.driver.core.SessionManager$3.call(SessionManager.java:301)
      10. com.datastax.driver.core.SessionManager$3.call(SessionManager.java:293)
      10 frames
    3. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      2. java.util.concurrent.FutureTask.run(FutureTask.java:138)
      3. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      4. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      5. java.lang.Thread.run(Thread.java:695)
      5 frames