org.acegisecurity.ldap.LdapDataAccessException: Failed to obtain InitialDirContext due to unexpected exception; nested exception is javax.naming.ServiceUnavailableException: ru20waddc.internal.synopsys.com:389; socket closed

Jenkins JIRA | Oleg Nenashev | 3 years ago
  1. 0

    We have experienced a strange issue with Jenkins 1.509.4. Web interface has gone offline after several days. * "RMI TCP Connection" thread is not present in the thread dump * "RMI Scheduler" and "RMI accept" exist in the thread * There is no info in jenkins.log * There's no BLOCKED threads * Jenkins internal jobs work well, only the web interface is not available After analysis of logs, seems that issue has been caused by "Too many open files" error: {quote} Oct 14, 2013 5:27:48 PM winstone.Logger logInternal SEVERE: Error during HTTP listener init or shutdown java.net.SocketException: Too many open files at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at winstone.HttpListener.run(HttpListener.java:139) at java.lang.Thread.run(Thread.java:724) {quote} According to logs, there is a huge number of LDAP errors {quote} WARNING: Failed to search LDAP for username=USER_NAME org.acegisecurity.ldap.LdapDataAccessException: Failed to obtain InitialDirContext due to unexpected exception; nested exception is javax.naming.ServiceUnavailableException: ru20waddc.internal.synopsys.com:389; socket closed at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:193) at org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:261) at org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:241) at org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:123) at org.acegisecurity.ldap.LdapTemplate.searchForSingleEntry(LdapTemplate.java:246) at org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:119) at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:591) at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:553) at org.acegisecurity.ui.rememberme.TokenBasedRememberMeServices.loadUserDetails(TokenBasedRememberMeServices.java:308) at org.acegisecurity.ui.rememberme.TokenBasedRememberMeServices.autoLogin(TokenBasedRememberMeServices.java:218) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) ... at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at winstone.RequestDispatcher.forward(RequestDispatcher.java:331) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:227) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Caused by: javax.naming.ServiceUnavailableException: LDAP_ADDRESS; socket closed at com.sun.jndi.ldap.Connection.readReply(Connection.java:454) at com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:364) at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:213) at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740) at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316) at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193) at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211) at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154) at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84) at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307) at javax.naming.InitialContext.init(InitialContext.java:242) at javax.naming.InitialContext.<init>(InitialContext.java:216) at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101) at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:180) ... 39 more {quote}

    Jenkins JIRA | 3 years ago | Oleg Nenashev
    org.acegisecurity.ldap.LdapDataAccessException: Failed to obtain InitialDirContext due to unexpected exception; nested exception is javax.naming.ServiceUnavailableException: ru20waddc.internal.synopsys.com:389; socket closed
  2. 0

    We have experienced a strange issue with Jenkins 1.509.4. Web interface has gone offline after several days. * "RMI TCP Connection" thread is not present in the thread dump * "RMI Scheduler" and "RMI accept" exist in the thread * There is no info in jenkins.log * There's no BLOCKED threads * Jenkins internal jobs work well, only the web interface is not available After analysis of logs, seems that issue has been caused by "Too many open files" error: {quote} Oct 14, 2013 5:27:48 PM winstone.Logger logInternal SEVERE: Error during HTTP listener init or shutdown java.net.SocketException: Too many open files at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) at java.net.ServerSocket.implAccept(ServerSocket.java:530) at java.net.ServerSocket.accept(ServerSocket.java:498) at winstone.HttpListener.run(HttpListener.java:139) at java.lang.Thread.run(Thread.java:724) {quote} According to logs, there is a huge number of LDAP errors {quote} WARNING: Failed to search LDAP for username=USER_NAME org.acegisecurity.ldap.LdapDataAccessException: Failed to obtain InitialDirContext due to unexpected exception; nested exception is javax.naming.ServiceUnavailableException: ru20waddc.internal.synopsys.com:389; socket closed at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:193) at org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:261) at org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:241) at org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:123) at org.acegisecurity.ldap.LdapTemplate.searchForSingleEntry(LdapTemplate.java:246) at org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:119) at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:591) at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:553) at org.acegisecurity.ui.rememberme.TokenBasedRememberMeServices.loadUserDetails(TokenBasedRememberMeServices.java:308) at org.acegisecurity.ui.rememberme.TokenBasedRememberMeServices.autoLogin(TokenBasedRememberMeServices.java:218) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) ... at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at winstone.RequestDispatcher.forward(RequestDispatcher.java:331) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:227) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Caused by: javax.naming.ServiceUnavailableException: LDAP_ADDRESS; socket closed at com.sun.jndi.ldap.Connection.readReply(Connection.java:454) at com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:364) at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:213) at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2740) at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316) at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193) at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211) at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154) at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84) at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307) at javax.naming.InitialContext.init(InitialContext.java:242) at javax.naming.InitialContext.<init>(InitialContext.java:216) at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101) at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:180) ... 39 more {quote}

    Jenkins JIRA | 3 years ago | Oleg Nenashev
    org.acegisecurity.ldap.LdapDataAccessException: Failed to obtain InitialDirContext due to unexpected exception; nested exception is javax.naming.ServiceUnavailableException: ru20waddc.internal.synopsys.com:389; socket closed
  3. 0

    LDAP Read Timeouts

    Google Groups | 10 months ago | Ashish Yadav
    org.acegisecurity.ldap.LdapDataAccessException: LdapCallback;null; nested exception is javax.naming.PartialResultException [Root exception is javax.naming.NamingException: LDAP response read timed out, timeout used:60000ms. [Root exception is com.sun.jndi.ldap.LdapReferralException: Continuation Reference; remaining name '']; remaining name '']
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    [JENKINS-8569] DomainDnsZones UnknownHostException - Jenkins JIRA

    jenkins-ci.org | 8 months ago
    org.acegisecurity.AuthenticationServiceException: LdapCallback;null; nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: DomainDnsZones.internal.synopsys.com:389 [Root exception is java.net.ConnectException: Connection timed out]]; nested exception is org.acegisecurity.ldap.LdapDataAccessException: LdapCallback;null; nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: DomainDnsZones.internal.synopsys.com:389 [Root exception is java.net.ConnectException: Connection timed out]]
  6. 0

    ldaps connectivity for enabliling authenitication via corporate ldap

    Google Groups | 5 years ago | Gandhi, Pawan
    org.acegisecurity.AuthenticationServiceException: LdapCallback;null; nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: simple bind failed: DomainDnsZones.test.net:636 [Root exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target]]; nested exception is org.acegisecurity.ldap.LdapDataAccessException: LdapCallback;null; nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: simple bind failed: DomainDnsZones.test.net:636 [Root exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target]]

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.acegisecurity.ldap.LdapDataAccessException

      Failed to obtain InitialDirContext due to unexpected exception; nested exception is javax.naming.ServiceUnavailableException: ru20waddc.internal.synopsys.com:389; socket closed

      at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect()
    2. Acegi Security Core
      FilterBasedLdapUserSearch.searchForUser
      1. org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:193)
      2. org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:261)
      3. org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:241)
      4. org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:123)
      5. org.acegisecurity.ldap.LdapTemplate.searchForSingleEntry(LdapTemplate.java:246)
      6. org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:119)
      6 frames
    3. Hudson
      LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername
      1. hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:591)
      2. hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:553)
      2 frames
    4. Acegi Security Core
      RememberMeProcessingFilter.doFilter
      1. org.acegisecurity.ui.rememberme.TokenBasedRememberMeServices.loadUserDetails(TokenBasedRememberMeServices.java:308)
      2. org.acegisecurity.ui.rememberme.TokenBasedRememberMeServices.autoLogin(TokenBasedRememberMeServices.java:218)
      3. org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104)
      3 frames