com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out]

Atlassian JIRA | Mark Lassau [Atlassian] | 3 years ago
  1. 0

    [JRA-34820] LDAP Synchronisation can fail unexpectedly due to mistiming in the "LDAP response read time out" - Atlassian JIRA

    atlassian.com | 11 months ago
    com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out]
  2. 0

    h3. Description In this bug, a read timeout exception is thrown before the timeout has passed, as can be seen in the logs below: {noformat} 2014-01-29 07:36:47,601 QuartzScheduler_Worker-2 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] synchronisation for directory [ 10000 ] starting 2014-01-29 07:36:57,975 QuartzScheduler_Worker-2 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 10000 ] in [ 10374ms ] 2014-01-29 07:36:58,084 QuartzScheduler_Worker-2 ERROR ServiceRunner [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 10000 ]. com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out] at com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults(SpringLDAPConnector.java:400) at com.atlassian.crowd.directory.SpringLDAPConnector.searchEntitiesWithRequestControls(SpringLDAPConnector.java:435) at com.atlassian.crowd.directory.MicrosoftActiveDirectory.findTombstonesSince(MicrosoftActiveDirectory.java:882) at com.atlassian.crowd.directory.MicrosoftActiveDirectory.findUserTombstonesSince(MicrosoftActiveDirectory.java:824) {noformat} h3. Expected Behaviour The LDAP connection does not timeout before the *Read Timeout (seconds)* is reached in the User Directory Configuration, for example [Connecting to an LDAP Directory|https://confluence.atlassian.com/display/JIRA/Connecting+to+an+LDAP+Directory#ConnectingtoanLDAPDirectory-ServerSettings]. h3. Actual Behaviour The connection times out before the *Read Timeout (seconds)* interval is met. h3. Verification Review when the synchronisation started in the logs, for example in the below snippet it's {{07:36:47,601}}: {noformat} 2014-01-29 07:36:47,601 QuartzScheduler_Worker-2 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] synchronisation for directory [ 10000 ] starting {noformat} And compare that to when the timeout occurs: {noformat} 2014-01-29 07:36:58,084 QuartzScheduler_Worker-2 ERROR ServiceRunner [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 10000 ]. com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out] {noformat} Here we can see the timeout occurred at {{07:36:58,084}} which is around 11 seconds, and the *Read Timeout (seconds)* was set to 60. It's highly likely this is a result of the Java bug. If the timeout occurs after the interval, this is normal and expected behaviour. For example if the timeout occurred at {{07:37:47,684}} this would be expected behaviour. h3. Cause This bug is triggered randomly in some environments, when the Read Timeout field in LDAP directory properties has been set to non-zero value. This bug is caused by a known [bug (#6968459)|http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6968459] affecting Java SE. h3. Workaround Normally these exceptions can be safely ignored as the synchronisation is self-correcting. That is, problems encountered in one synchronisation round will get fixed in the following synchronisation round. If the synchronisation fails to complete successfully repeatedly, a known workaround is to disable read timeout by setting the Read Timeout field in LDAP directory properties to 0. A side-effect of this change is that Crowd will not be able to recover automatically from LDAP requests that take too long to run, which might cause Crowd to stop communicating with LDAP directories until it is restarted. h4. For JIRA 6.3.x customers If you have confirmed that you are experiencing the aforementioned Java bug, we have developed a temporary fix until Oracle fixes the issue in the JDK itself. This is in the form of 2 JARs, a [new version of Crowd for JIRA|https://maven.atlassian.com/service/local/repositories/atlassian-public/content/com/atlassian/crowd/crowd-ldap/2.8.0-OD-6-JIRA-06/crowd-ldap-2.8.0-OD-6-JIRA-06.jar] and a [patched+shaded version of the JDK's LDAP JNDI provider|https://maven.atlassian.com/3rdparty/shaded/atlassian/com/sun/jndi/ldap/atlassian-ldap-jndi-provider/1.8.1-atlassian/atlassian-ldap-jndi-provider-1.8.1-atlassian.jar]. Source for the patched+shaded version of the JDK's LDAP JNDI provider is available at https://bitbucket.org/atlassian/atlassian-ldap-jndi-provider To install these files and apply them as a workaround: - Stop yourJIRA instance - Navigate to JIRA's {{webapp/WEB-INF/lib}} directory. - Replace the existing `crowd-ldap-???.jar` with [crowd-ldap-2.8.0-OD-6-JIRA-06.jar|https://maven.atlassian.com/service/local/repositories/atlassian-public/content/com/atlassian/crowd/crowd-ldap/2.8.0-OD-6-JIRA-06/crowd-ldap-2.8.0-OD-6-JIRA-06.jar] - Add the [atlassian-ldap-jndi-provider-1.8.1-atlassian.jar|https://maven.atlassian.com/3rdparty/shaded/atlassian/com/sun/jndi/ldap/atlassian-ldap-jndi-provider/1.8.1-atlassian/atlassian-ldap-jndi-provider-1.8.1-atlassian.jar] file to this directory as well. - Start your JIRA instance. *IMPORTANT*: This will only work if the you are running Java 8 and a version of JIRA that uses crowd-ldap-2.8.0-OD-6 or compatible (JIRA 6.3.x)

    Atlassian JIRA | 3 years ago | Mark Lassau [Atlassian]
    com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out]
  3. 0

    h3. Description In this bug, a read timeout exception is thrown before the timeout has passed, as can be seen in the logs below: {noformat} 2014-01-29 07:36:47,601 QuartzScheduler_Worker-2 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] synchronisation for directory [ 10000 ] starting 2014-01-29 07:36:57,975 QuartzScheduler_Worker-2 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] failed synchronisation complete for directory [ 10000 ] in [ 10374ms ] 2014-01-29 07:36:58,084 QuartzScheduler_Worker-2 ERROR ServiceRunner [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 10000 ]. com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out] at com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults(SpringLDAPConnector.java:400) at com.atlassian.crowd.directory.SpringLDAPConnector.searchEntitiesWithRequestControls(SpringLDAPConnector.java:435) at com.atlassian.crowd.directory.MicrosoftActiveDirectory.findTombstonesSince(MicrosoftActiveDirectory.java:882) at com.atlassian.crowd.directory.MicrosoftActiveDirectory.findUserTombstonesSince(MicrosoftActiveDirectory.java:824) {noformat} h3. Expected Behaviour The LDAP connection does not timeout before the *Read Timeout (seconds)* is reached in the User Directory Configuration, for example [Connecting to an LDAP Directory|https://confluence.atlassian.com/display/JIRA/Connecting+to+an+LDAP+Directory#ConnectingtoanLDAPDirectory-ServerSettings]. h3. Actual Behaviour The connection times out before the *Read Timeout (seconds)* interval is met. h3. Verification Review when the synchronisation started in the logs, for example in the below snippet it's {{07:36:47,601}}: {noformat} 2014-01-29 07:36:47,601 QuartzScheduler_Worker-2 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] synchronisation for directory [ 10000 ] starting {noformat} And compare that to when the timeout occurs: {noformat} 2014-01-29 07:36:58,084 QuartzScheduler_Worker-2 ERROR ServiceRunner [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 10000 ]. com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out] {noformat} Here we can see the timeout occurred at {{07:36:58,084}} which is around 11 seconds, and the *Read Timeout (seconds)* was set to 60. It's highly likely this is a result of the Java bug. If the timeout occurs after the interval, this is normal and expected behaviour. For example if the timeout occurred at {{07:37:47,684}} this would be expected behaviour. h3. Cause This bug is triggered randomly in some environments, when the Read Timeout field in LDAP directory properties has been set to non-zero value. This bug is caused by a known [bug (#6968459)|http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6968459] affecting Java SE. h3. Workaround Normally these exceptions can be safely ignored as the synchronisation is self-correcting. That is, problems encountered in one synchronisation round will get fixed in the following synchronisation round. If the synchronisation fails to complete successfully repeatedly, a known workaround is to disable read timeout by setting the Read Timeout field in LDAP directory properties to 0. A side-effect of this change is that Crowd will not be able to recover automatically from LDAP requests that take too long to run, which might cause Crowd to stop communicating with LDAP directories until it is restarted. h4. For JIRA 6.3.x customers If you have confirmed that you are experiencing the aforementioned Java bug, we have developed a temporary fix until Oracle fixes the issue in the JDK itself. This is in the form of 2 JARs, a [new version of Crowd for JIRA|https://maven.atlassian.com/service/local/repositories/atlassian-public/content/com/atlassian/crowd/crowd-ldap/2.8.0-OD-6-JIRA-06/crowd-ldap-2.8.0-OD-6-JIRA-06.jar] and a [patched+shaded version of the JDK's LDAP JNDI provider|https://maven.atlassian.com/3rdparty/shaded/atlassian/com/sun/jndi/ldap/atlassian-ldap-jndi-provider/1.8.1-atlassian/atlassian-ldap-jndi-provider-1.8.1-atlassian.jar]. Source for the patched+shaded version of the JDK's LDAP JNDI provider is available at https://bitbucket.org/atlassian/atlassian-ldap-jndi-provider To install these files and apply them as a workaround: - Stop yourJIRA instance - Navigate to JIRA's {{webapp/WEB-INF/lib}} directory. - Replace the existing `crowd-ldap-???.jar` with [crowd-ldap-2.8.0-OD-6-JIRA-06.jar|https://maven.atlassian.com/service/local/repositories/atlassian-public/content/com/atlassian/crowd/crowd-ldap/2.8.0-OD-6-JIRA-06/crowd-ldap-2.8.0-OD-6-JIRA-06.jar] - Add the [atlassian-ldap-jndi-provider-1.8.1-atlassian.jar|https://maven.atlassian.com/3rdparty/shaded/atlassian/com/sun/jndi/ldap/atlassian-ldap-jndi-provider/1.8.1-atlassian/atlassian-ldap-jndi-provider-1.8.1-atlassian.jar] file to this directory as well. - Start your JIRA instance. *IMPORTANT*: This will only work if the you are running Java 8 and a version of JIRA that uses crowd-ldap-2.8.0-OD-6 or compatible (JIRA 6.3.x)

    Atlassian JIRA | 3 years ago | Mark Lassau [Atlassian]
    com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out]
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    When setting up a directory in JIRA 4.3, if the BaseDN does not contain an Organizational Unit (OU), users cannot log in JIRA. However, the Test Connection is successful. The message "Sorry, an error occurred trying to log you in - please try again." is displayed to the user on the log in screen. Example: *Works* BaseDN: ou=office,dc=mycompany,dc=com *Does not Work* BaseDN:dc=mycompany,dc=com Message on the log files when it does not work: {code} 2011-03-16 14:59:06,599 QuartzWorker-0 INFO ServiceRunner [atlassian.crowd.directory.DbCachingRemoteDirectory] synchronisation complete in [ 26ms ] 2011-03-16 14:59:06,648 QuartzWorker-0 ERROR ServiceRunner [atlassian.crowd.directory.DbCachingDirectoryPoller] Error occurred while refreshing the cache for directory [ 10000 ]. com.atlassian.crowd.exception.OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]] at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAllUsers(UsnChangedCacheRefresher.java:255) at com.atlassian.crowd.directory.ldap.cache.AbstractCacheRefresher.synchroniseAll(AbstractCacheRefresher.java:34) ... Caused by: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]] at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at com.atlassian.crowd.directory.ldap.cache.UsnChangedCacheRefresher.synchroniseAllUsers(UsnChangedCacheRefresher.java:237) ... 10 more Caused by: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]] at com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults(SpringLDAPConnector.java:352) at com.atlassian.crowd.directory.SpringLDAPConnector.searchEntitiesWithRequestControls(SpringLDAPConnector.java:386) ... Caused by: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]] at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:203) at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:315) ... Caused by: javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]] at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:224) at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(LdapNamingEnumeration.java:171) at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:295) ... 13 more Caused by: javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com] at com.sun.jndi.ldap.LdapReferralContext.<init>(LdapReferralContext.java:74) at com.sun.jndi.ldap.LdapReferralException.getReferralContext(LdapReferralException.java:132) at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreReferrals(LdapNamingEnumeration.java:339) at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:208) ... 15 more Caused by: java.net.UnknownHostException: mycompany.com at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:195) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432) {code}

    Atlassian JIRA | 6 years ago | Clarissa Gauterio [Atlassian]
    com.atlassian.crowd.exception.OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]]
  6. 0

    [JRA-23955] BaseDN without an OU does not work when setting up a directory in JIRA - Atlassian JIRA

    atlassian.com | 9 months ago
    com.atlassian.crowd.exception.OperationFailedException: java.util.concurrent.ExecutionException: com.atlassian.crowd.exception.OperationFailedException: org.springframework.ldap.PartialResultException: nested exception is javax.naming.PartialResultException [Root exception is javax.naming.CommunicationException: mycompany.com:389 [Root exception is java.net.UnknownHostException: mycompany.com]]

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. com.atlassian.crowd.exception.OperationFailedException

      org.springframework.ldap.CommunicationException: avengers-the-initiative.net:389; nested exception is javax.naming.CommunicationException: avengers-the-initiative.net:389 [Root exception is java.net.SocketTimeoutException: connect timed out]

      at com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults()
    2. com.atlassian.crowd
      MicrosoftActiveDirectory.findUserTombstonesSince
      1. com.atlassian.crowd.directory.SpringLDAPConnector.pageSearchResults(SpringLDAPConnector.java:400)
      2. com.atlassian.crowd.directory.SpringLDAPConnector.searchEntitiesWithRequestControls(SpringLDAPConnector.java:435)
      3. com.atlassian.crowd.directory.MicrosoftActiveDirectory.findTombstonesSince(MicrosoftActiveDirectory.java:882)
      4. com.atlassian.crowd.directory.MicrosoftActiveDirectory.findUserTombstonesSince(MicrosoftActiveDirectory.java:824)
      4 frames