java.net.NoRouteToHostException: No route to host

JIRA | Ti Leggett | 2 years ago
  1. 0

    First off this is for v1.4.2 but that's not listed in the version dropdown. I have two ES nodes that are 2 different ES clusters that logstash sends indexes to - one for all indexes and one just for web indexes: output { elasticsearch { cluster => "all" node_name => "logstash" host => "all-indexes" port => 9300 protocol => 'node' } if ("apache" in [tags]) { elasticsearch { cluster => "web" node_name => "logstash" host => "web-indexes" port => 9300 protocol => 'node' } } } Both are ES 1.1.1. Yesterday I had a kernel update to push out on the web-indexes machine, so I rebooted. When I did, logstash eventually crashed because it could no longer talk to it: log4j, [2014-08-05T13:53:34.699] WARN: org.elasticsearch.discovery.zen: [web-indexes] master_left and no other node elected to become master, current nodes: {[web-indexes][IWa9aAqyTAiR2jhXHMesDA][all-indexes][inet[/192.168.0.2:9302]]{client=true, data=false},} log4j, [2014-08-05T15:18:47.745] WARN: org.elasticsearch.discovery.zen: [web-indexes] master_left and no other node elected to become master, current nodes: {[web-indexes][IWa9aAqyTAiR2jhXHMesDA][all-indexes][inet[/192.168.0.2:9302]]{client=true, data=false},} log4j, [2014-08-05T15:19:38.860] WARN: org.elasticsearch.transport.netty: [web-indexes] exception caught on transport layer [[id: 0xb3379c8b]], closing connection java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) That traceback is repeated several more times before logstash finally crashed for good. Restarting logstash when both ES servers were online again worked

    JIRA | 2 years ago | Ti Leggett
    java.net.NoRouteToHostException: No route to host
  2. 0

    First off this is for v1.4.2 but that's not listed in the version dropdown. I have two ES nodes that are 2 different ES clusters that logstash sends indexes to - one for all indexes and one just for web indexes: output { elasticsearch { cluster => "all" node_name => "logstash" host => "all-indexes" port => 9300 protocol => 'node' } if ("apache" in [tags]) { elasticsearch { cluster => "web" node_name => "logstash" host => "web-indexes" port => 9300 protocol => 'node' } } } Both are ES 1.1.1. Yesterday I had a kernel update to push out on the web-indexes machine, so I rebooted. When I did, logstash eventually crashed because it could no longer talk to it: log4j, [2014-08-05T13:53:34.699] WARN: org.elasticsearch.discovery.zen: [web-indexes] master_left and no other node elected to become master, current nodes: {[web-indexes][IWa9aAqyTAiR2jhXHMesDA][all-indexes][inet[/192.168.0.2:9302]]{client=true, data=false},} log4j, [2014-08-05T15:18:47.745] WARN: org.elasticsearch.discovery.zen: [web-indexes] master_left and no other node elected to become master, current nodes: {[web-indexes][IWa9aAqyTAiR2jhXHMesDA][all-indexes][inet[/192.168.0.2:9302]]{client=true, data=false},} log4j, [2014-08-05T15:19:38.860] WARN: org.elasticsearch.transport.netty: [web-indexes] exception caught on transport layer [[id: 0xb3379c8b]], closing connection java.net.NoRouteToHostException: No route to host at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) That traceback is repeated several more times before logstash finally crashed for good. Restarting logstash when both ES servers were online again worked

    JIRA | 2 years ago | Ti Leggett
    java.net.NoRouteToHostException: No route to host
  3. Speed up your debug routine!

    Automated exception search integrated into your IDE

  4. 0

    shard allocated for local recovery (post api), should exists, but doesn't

    GitHub | 5 years ago | bbgordonn
    java.net.NoRouteToHostException: No route to host
  5. 0

    No route to self

    Google Groups | 2 years ago | Matt Hughes
    java.net.NoRouteToHostException: No route to host

  1. poroszd 2 times, last 2 months ago
13 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.net.NoRouteToHostException

    No route to host

    at sun.nio.ch.SocketChannelImpl.checkConnect()
  2. Java RT
    SocketChannelImpl.finishConnect
    1. sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    2. sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    2 frames
  3. ElasticSearch
    DeadLockProofWorker$1.run
    1. org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150)
    2. org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
    3. org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
    4. org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    5. org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    6. org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    7. org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    7 frames
  4. Java RT
    Thread.run
    1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    3. java.lang.Thread.run(Thread.java:745)
    3 frames