com.atlassian.crowd.manager.directory.monitor.DirectoryMonitorRegistrationException: org.quartz.ObjectAlreadyExistsException: Unable to store Job with name: '262145' and group: 'DirectoryPoller', because one already exists with this identification.

Atlassian JIRA | Michael Heemskerk | 5 years ago
  1. 0

    We sometimes see the following in our logs after a restart: 2012-01-26 21:07:31,347 ERROR [http-8080-4] c.a.c.d.l.DbCachingRemoteDirectoryInstanceLoaderImpl Could not add a monitor for the directory with id: 262145 com.atlassian.crowd.manager.directory.monitor.DirectoryMonitorRegistrationException: org.quartz.ObjectAlreadyExistsException: Unable to store Job with name: '262145' and group: 'DirectoryPoller', because one already exists with this identification. at com.atlassian.crowd.manager.directory.monitor.poller.AbstractQuartzDirectoryPollerManager.addPoller(AbstractQuartzDirectoryPollerManager.java:29) ~[crowd-core-2.5.0-m0.jar:na] at sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_27] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_27] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at $Proxy72.authenticate(Unknown Source) [na:na] at com.atlassian..user.service.impl.UserServiceImpl.authenticate(UserServiceImpl.java:77) [] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_27] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_27] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_27] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_27] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at com.atlassian..profiling.ProfilingAspect.profileMethod(ProfilingAspect.java:34) [-core-1.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_27] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_27] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110) [spring-tx-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at $Proxy74.authenticate(Unknown Source) [na:na] This happens when multiple users try to login at the same time just after a restart. From looking at the sources, there appears to be a race condition in DirectoryMonitorManagerImpl.addMonitor where the check on whether or not the monitor already exists is done without sufficient locking.

    Atlassian JIRA | 5 years ago | Michael Heemskerk
    com.atlassian.crowd.manager.directory.monitor.DirectoryMonitorRegistrationException: org.quartz.ObjectAlreadyExistsException: Unable to store Job with name: '262145' and group: 'DirectoryPoller', because one already exists with this identification.
  2. 0

    We sometimes see the following in our logs after a restart: 2012-01-26 21:07:31,347 ERROR [http-8080-4] c.a.c.d.l.DbCachingRemoteDirectoryInstanceLoaderImpl Could not add a monitor for the directory with id: 262145 com.atlassian.crowd.manager.directory.monitor.DirectoryMonitorRegistrationException: org.quartz.ObjectAlreadyExistsException: Unable to store Job with name: '262145' and group: 'DirectoryPoller', because one already exists with this identification. at com.atlassian.crowd.manager.directory.monitor.poller.AbstractQuartzDirectoryPollerManager.addPoller(AbstractQuartzDirectoryPollerManager.java:29) ~[crowd-core-2.5.0-m0.jar:na] at sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_27] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_27] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at $Proxy72.authenticate(Unknown Source) [na:na] at com.atlassian..user.service.impl.UserServiceImpl.authenticate(UserServiceImpl.java:77) [] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_27] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_27] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_27] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_27] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at com.atlassian..profiling.ProfilingAspect.profileMethod(ProfilingAspect.java:34) [-core-1.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_27] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_27] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110) [spring-tx-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) [spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE] at $Proxy74.authenticate(Unknown Source) [na:na] This happens when multiple users try to login at the same time just after a restart. From looking at the sources, there appears to be a race condition in DirectoryMonitorManagerImpl.addMonitor where the check on whether or not the monitor already exists is done without sufficient locking.

    Atlassian JIRA | 5 years ago | Michael Heemskerk
    com.atlassian.crowd.manager.directory.monitor.DirectoryMonitorRegistrationException: org.quartz.ObjectAlreadyExistsException: Unable to store Job with name: '262145' and group: 'DirectoryPoller', because one already exists with this identification.

    Root Cause Analysis

    1. com.atlassian.crowd.manager.directory.monitor.DirectoryMonitorRegistrationException

      org.quartz.ObjectAlreadyExistsException: Unable to store Job with name: '262145' and group: 'DirectoryPoller', because one already exists with this identification.

      at com.atlassian.crowd.manager.directory.monitor.poller.AbstractQuartzDirectoryPollerManager.addPoller()
    2. com.atlassian.crowd
      AbstractQuartzDirectoryPollerManager.addPoller
      1. com.atlassian.crowd.manager.directory.monitor.poller.AbstractQuartzDirectoryPollerManager.addPoller(AbstractQuartzDirectoryPollerManager.java:29)[crowd-core-2.5.0-m0.jar:na]
      1 frame
    3. Java RT
      Method.invoke
      1. sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source)[na:na]
      2. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)[na:1.6.0_27]
      3. java.lang.reflect.Method.invoke(Method.java:597)[na:1.6.0_27]
      3 frames
    4. Spring AOP
      JdkDynamicAopProxy.invoke
      1. org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      2. org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      3. org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      4. org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      5. org.springframework.aop.framework.adapter.ThrowsAdviceInterceptor.invoke(ThrowsAdviceInterceptor.java:124)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      6. org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      7. org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      8. org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      9. org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)[spring-aop-3.1.0.RELEASE.jar:3.1.0.RELEASE]
      9 frames
    5. Unknown
      $Proxy72.authenticate
      1. $Proxy72.authenticate(Unknown Source)[na:na]
      1 frame