java.net.SocketException: Too many open files

Terracotta Project Issue Tracker | Eric Dahl | 4 years ago
  1. 0

    Originally described on the terracotta forums [here|http://forums.terracotta.org/forums/posts/list/0/7572.page] 1. Start up terracotta server {code} $ tar xzf terracotta-3.7.0.tar.gz $ terracotta-3.7.0/bin/start-tc-server.sh {code} 2. Make some requests to pull the config at port 9530 {code} $ for i in $(seq 1000); do curl -s "http://localhost:9530/config" > /dev/null; done {code} 3. Check terracotta's open sockets {code} $ lsof -i -P | grep $(pgrep java) | grep CLOSE_WAIT | wc -l 781 {code} It seems that Terracotta's HTTP server isn't properly closing its TCP connections. The file descriptor limit can be reached easily, causing Terracotta to die: {code} 2012-10-08 15:17:02,245 [WorkerThread(dso-http-bridge, 0)] ERROR com.tc.net.protocol.HttpConnectionContext - Exception thrown java.net.SocketException: Too many open files at java.net.Socket.createImpl(Socket.java:414) at java.net.Socket.getImpl(Socket.java:477) at java.net.Socket.getSoTimeout(Socket.java:1050) at com.tc.server.TerracottaConnector$SocketWrapper.getSoTimeout(TerracottaConnector.java:156) at org.mortbay.jetty.bio.SocketConnector$Connection.<init>(SocketConnector.java:183) at com.tc.server.TerracottaConnector.handleSocketFromDSO(TerracottaConnector.java:39) at com.tc.server.HttpConnectionHandler.handleEvent(HttpConnectionHandler.java:35) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145) {code} Sometimes the terracotta server becomes unresponsive and the process crashes: {code} java.lang.RuntimeException: java.lang.AssertionError: Protocol is 2 at com.tc.net.core.CoreNIOServices$CommThread.run(CoreNIOServices.java:293) Caused by: java.lang.AssertionError: Protocol is 2 at com.tc.net.protocol.ProtocolSwitch.getReadBuffers(ProtocolSwitch.java:139) at com.tc.net.core.TCConnectionImpl.getReadBuffers(TCConnectionImpl.java:794) at com.tc.net.core.TCConnectionImpl.doReadFromBufferInternal(TCConnectionImpl.java:433) at com.tc.net.core.TCConnectionImpl.doReadFromBuffer(TCConnectionImpl.java:306) at com.tc.net.core.TCConnectionImpl.doReadInternal(TCConnectionImpl.java:290) at com.tc.net.core.TCConnectionImpl.doRead(TCConnectionImpl.java:266) at com.tc.net.core.CoreNIOServices$CommThread.selectLoop(CoreNIOServices.java:624) at com.tc.net.core.CoreNIOServices$CommThread.run(CoreNIOServices.java:290) {code} The CLOSE_WAIT sockets will never die without the application (terracotta) closing them.

    Terracotta Project Issue Tracker | 4 years ago | Eric Dahl
    java.net.SocketException: Too many open files
  2. 0

    Originally described on the terracotta forums [here|http://forums.terracotta.org/forums/posts/list/0/7572.page] 1. Start up terracotta server {code} $ tar xzf terracotta-3.7.0.tar.gz $ terracotta-3.7.0/bin/start-tc-server.sh {code} 2. Make some requests to pull the config at port 9530 {code} $ for i in $(seq 1000); do curl -s "http://localhost:9530/config" > /dev/null; done {code} 3. Check terracotta's open sockets {code} $ lsof -i -P | grep $(pgrep java) | grep CLOSE_WAIT | wc -l 781 {code} It seems that Terracotta's HTTP server isn't properly closing its TCP connections. The file descriptor limit can be reached easily, causing Terracotta to die: {code} 2012-10-08 15:17:02,245 [WorkerThread(dso-http-bridge, 0)] ERROR com.tc.net.protocol.HttpConnectionContext - Exception thrown java.net.SocketException: Too many open files at java.net.Socket.createImpl(Socket.java:414) at java.net.Socket.getImpl(Socket.java:477) at java.net.Socket.getSoTimeout(Socket.java:1050) at com.tc.server.TerracottaConnector$SocketWrapper.getSoTimeout(TerracottaConnector.java:156) at org.mortbay.jetty.bio.SocketConnector$Connection.<init>(SocketConnector.java:183) at com.tc.server.TerracottaConnector.handleSocketFromDSO(TerracottaConnector.java:39) at com.tc.server.HttpConnectionHandler.handleEvent(HttpConnectionHandler.java:35) at com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145) {code} Sometimes the terracotta server becomes unresponsive and the process crashes: {code} java.lang.RuntimeException: java.lang.AssertionError: Protocol is 2 at com.tc.net.core.CoreNIOServices$CommThread.run(CoreNIOServices.java:293) Caused by: java.lang.AssertionError: Protocol is 2 at com.tc.net.protocol.ProtocolSwitch.getReadBuffers(ProtocolSwitch.java:139) at com.tc.net.core.TCConnectionImpl.getReadBuffers(TCConnectionImpl.java:794) at com.tc.net.core.TCConnectionImpl.doReadFromBufferInternal(TCConnectionImpl.java:433) at com.tc.net.core.TCConnectionImpl.doReadFromBuffer(TCConnectionImpl.java:306) at com.tc.net.core.TCConnectionImpl.doReadInternal(TCConnectionImpl.java:290) at com.tc.net.core.TCConnectionImpl.doRead(TCConnectionImpl.java:266) at com.tc.net.core.CoreNIOServices$CommThread.selectLoop(CoreNIOServices.java:624) at com.tc.net.core.CoreNIOServices$CommThread.run(CoreNIOServices.java:290) {code} The CLOSE_WAIT sockets will never die without the application (terracotta) closing them.

    Terracotta Project Issue Tracker | 4 years ago | Eric Dahl
    java.net.SocketException: Too many open files
  3. 0

    java.net.SocketException: socket closed, What's going on?

    experts-exchange.com | 1 year ago
    java.net.SocketException:
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Problen in starting WAS4 Application Server

    Coderanch | 9 years ago | Prabu Bala
    java.net.SocketException: Socket closed
  6. 0

    Error in tomcat "too many open files"

    Stack Overflow | 3 years ago | alicia
    java.net.SocketException: Too many open files

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.net.SocketException

      Too many open files

      at java.net.Socket.createImpl()
    2. Java RT
      Socket.getSoTimeout
      1. java.net.Socket.createImpl(Socket.java:414)
      2. java.net.Socket.getImpl(Socket.java:477)
      3. java.net.Socket.getSoTimeout(Socket.java:1050)
      3 frames
    3. com.tc.server
      TerracottaConnector$SocketWrapper.getSoTimeout
      1. com.tc.server.TerracottaConnector$SocketWrapper.getSoTimeout(TerracottaConnector.java:156)
      1 frame
    4. Jetty Server
      SocketConnector$Connection.<init>
      1. org.mortbay.jetty.bio.SocketConnector$Connection.<init>(SocketConnector.java:183)
      1 frame
    5. com.tc.server
      HttpConnectionHandler.handleEvent
      1. com.tc.server.TerracottaConnector.handleSocketFromDSO(TerracottaConnector.java:39)
      2. com.tc.server.HttpConnectionHandler.handleEvent(HttpConnectionHandler.java:35)
      2 frames
    6. com.tc.async
      StageImpl$WorkerThread.run
      1. com.tc.async.impl.StageImpl$WorkerThread.run(StageImpl.java:145)
      1 frame