org.springframework.remoting.RemoteAccessException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • A build failed with an rather unusal timeout exception. It looks like it was trying to load some configuration from the Bamboo server which didn't respond within 5 minutes: {noformat} 09-Jan-2015 00:48:27 Error occurred while running Task 'Build DevStatus(9)' of type com.atlassian.bamboo.plugins.maven:task.builder.mvn3. 09-Jan-2015 00:48:27 com.atlassian.bamboo.task.TaskException: Failed to execute task 09-Jan-2015 00:48:27 at com.atlassian.bamboo.plugins.maven.task.Maven3BuildTask.execute(Maven3BuildTask.java:114) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl$3.call(TaskExecutorImpl.java:281) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl$3.call(TaskExecutorImpl.java:278) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl.executeTaskWithPrePostActions(TaskExecutorImpl.java:198) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:278) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:105) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:75) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:188) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:112) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:110) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) 09-Jan-2015 00:48:27 at java.lang.Thread.run(Thread.java:745) 09-Jan-2015 00:48:27 Caused by: org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:151) 09-Jan-2015 00:48:27 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 09-Jan-2015 00:48:27 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) 09-Jan-2015 00:48:27 at com.sun.proxy.$Proxy8.getAdministrationConfiguration(Unknown Source) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.utils.BambooUrl.withBaseUrlFromConfiguration(BambooUrl.java:78) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.addVariables(CustomVariableContextImpl.java:526) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:481) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:349) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.getVariables(CustomVariableContextImpl.java:262) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.process.EnvironmentVariableAccessorImpl.getEnvironment(EnvironmentVariableAccessorImpl.java:85) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.process.ProcessServiceImpl.createExternalProcess(ProcessServiceImpl.java:167) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.process.ProcessServiceImpl.executeExternalProcess(ProcessServiceImpl.java:209) 09-Jan-2015 00:48:27 at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) 09-Jan-2015 00:48:27 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 09-Jan-2015 00:48:27 at java.lang.reflect.Method.invoke(Method.java:606) 09-Jan-2015 00:48:27 at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) 09-Jan-2015 00:48:27 at com.sun.proxy.$Proxy58.executeExternalProcess(Unknown Source) 09-Jan-2015 00:48:27 at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) 09-Jan-2015 00:48:27 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 09-Jan-2015 00:48:27 at java.lang.reflect.Method.invoke(Method.java:606) 09-Jan-2015 00:48:27 at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) 09-Jan-2015 00:48:27 at com.sun.proxy.$Proxy58.executeExternalProcess(Unknown Source) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.plugins.maven.task.Maven3BuildTask.execute(Maven3BuildTask.java:76) 09-Jan-2015 00:48:27 ... 14 more 09-Jan-2015 00:48:27 Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) 09-Jan-2015 00:48:27 ... 36 more {noformat} Nothing obvious on the server {noformat} 2015-01-09 00:49:27,519 ERROR [bambooAgentMessageListenerConnector-1] [DefaultBuildExecutionManager] Plan FUZ-MASTER79-HAL-290 timer being stopped but plan not detected to have begun. 2015-01-09 00:49:27,519 INFO [bambooAgentMessageListenerConnector-1] [DefaultErrorHandler] Recording an error: Exception occurred while executing custom pre-build action 'com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction@2eb314e '. Building will abort... org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:151) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) at com.sun.proxy.$Proxy8.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.v2.build.agent.remote.NonSavingAdminConfigManager.getAdministrationConfiguration(NonSavingAdminConfigManager.java:33) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) at com.sun.proxy.$Proxy26.getAdministrationConfiguration(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) at com.sun.proxy.$Proxy26.getAdministrationConfiguration(Unknown Source) at com.atlassian.sandbox.bamboo.actions.SandboxCommandDecorationProvider.decorate(SandboxCommandDecorationProvider.java:30) at com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction.called(SandboxPreBuildAction.java:75) at com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction.call(SandboxPreBuildAction.java:53) at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.runPreBuildActions(PrepareBuildTask.java:134) at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:102) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:188) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:112) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:110) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:745) Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) ... 29 more {noformat}
    via by Martin Meinhold,
  • A build failed with an rather unusal timeout exception. It looks like it was trying to load some configuration from the Bamboo server which didn't respond within 5 minutes: {noformat} 09-Jan-2015 00:48:27 Error occurred while running Task 'Build DevStatus(9)' of type com.atlassian.bamboo.plugins.maven:task.builder.mvn3. 09-Jan-2015 00:48:27 com.atlassian.bamboo.task.TaskException: Failed to execute task 09-Jan-2015 00:48:27 at com.atlassian.bamboo.plugins.maven.task.Maven3BuildTask.execute(Maven3BuildTask.java:114) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl$3.call(TaskExecutorImpl.java:281) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl$3.call(TaskExecutorImpl.java:278) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl.executeTaskWithPrePostActions(TaskExecutorImpl.java:198) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:278) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:105) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:75) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:188) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:112) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:110) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) 09-Jan-2015 00:48:27 at java.lang.Thread.run(Thread.java:745) 09-Jan-2015 00:48:27 Caused by: org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:151) 09-Jan-2015 00:48:27 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) 09-Jan-2015 00:48:27 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) 09-Jan-2015 00:48:27 at com.sun.proxy.$Proxy8.getAdministrationConfiguration(Unknown Source) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.utils.BambooUrl.withBaseUrlFromConfiguration(BambooUrl.java:78) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.addVariables(CustomVariableContextImpl.java:526) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:481) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:349) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.variable.CustomVariableContextImpl.getVariables(CustomVariableContextImpl.java:262) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.process.EnvironmentVariableAccessorImpl.getEnvironment(EnvironmentVariableAccessorImpl.java:85) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.process.ProcessServiceImpl.createExternalProcess(ProcessServiceImpl.java:167) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.process.ProcessServiceImpl.executeExternalProcess(ProcessServiceImpl.java:209) 09-Jan-2015 00:48:27 at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) 09-Jan-2015 00:48:27 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 09-Jan-2015 00:48:27 at java.lang.reflect.Method.invoke(Method.java:606) 09-Jan-2015 00:48:27 at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) 09-Jan-2015 00:48:27 at com.sun.proxy.$Proxy58.executeExternalProcess(Unknown Source) 09-Jan-2015 00:48:27 at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) 09-Jan-2015 00:48:27 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 09-Jan-2015 00:48:27 at java.lang.reflect.Method.invoke(Method.java:606) 09-Jan-2015 00:48:27 at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) 09-Jan-2015 00:48:27 at com.sun.proxy.$Proxy58.executeExternalProcess(Unknown Source) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.plugins.maven.task.Maven3BuildTask.execute(Maven3BuildTask.java:76) 09-Jan-2015 00:48:27 ... 14 more 09-Jan-2015 00:48:27 Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) 09-Jan-2015 00:48:27 at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) 09-Jan-2015 00:48:27 at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) 09-Jan-2015 00:48:27 ... 36 more {noformat} Nothing obvious on the server {noformat} 2015-01-09 00:49:27,519 ERROR [bambooAgentMessageListenerConnector-1] [DefaultBuildExecutionManager] Plan FUZ-MASTER79-HAL-290 timer being stopped but plan not detected to have begun. 2015-01-09 00:49:27,519 INFO [bambooAgentMessageListenerConnector-1] [DefaultErrorHandler] Recording an error: Exception occurred while executing custom pre-build action 'com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction@2eb314e '. Building will abort... org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:151) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) at com.sun.proxy.$Proxy8.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.v2.build.agent.remote.NonSavingAdminConfigManager.getAdministrationConfiguration(NonSavingAdminConfigManager.java:33) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) at com.sun.proxy.$Proxy26.getAdministrationConfiguration(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) at com.sun.proxy.$Proxy26.getAdministrationConfiguration(Unknown Source) at com.atlassian.sandbox.bamboo.actions.SandboxCommandDecorationProvider.decorate(SandboxCommandDecorationProvider.java:30) at com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction.called(SandboxPreBuildAction.java:75) at com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction.call(SandboxPreBuildAction.java:53) at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.runPreBuildActions(PrepareBuildTask.java:134) at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:102) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:188) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:112) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:110) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:745) Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) ... 29 more {noformat}
    via by Martin Meinhold,
  • Today we experienced an issue where our Bamboo server ran out of disk space. It came to our attention due to to the fact that all of the remote agents went offline at the same time. We had set up the agent wrapper.conf to continue to try and reconnect to the server for 6 hours after an initial connection failure. When the server ran out of disk space we saw the following in the agent logs: {noformat} INFO | jvm 1 | 2011/08/25 03:25:05 | org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 1 | 2011/08/25 03:25:05 | Caused by: javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQConnection.doSyncSendPacket(ActiveMQConnection.java:1329) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1314) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1758) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:74) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:59) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:536) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:513) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate$2.doInJms(JmsTemplate.java:479) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:430) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:477) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:588) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:576) INFO | jvm 1 | 2011/08/25 03:25:05 | at com.atlassian.bamboo.v2.build.agent.remote.sender.JmsTemplateMessageSender.send(JmsTemplateMessageSender.java:33) INFO | jvm 1 | 2011/08/25 03:25:05 | at com.atlassian.bamboo.v2.build.agent.remote.heartbeat.AgentHeartBeatJob.execute(AgentHeartBeatJob.java:51) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.quartz.core.JobRunShell.run(JobRunShell.java:202) INFO | jvm 1 | 2011/08/25 03:25:05 | at com.atlassian.bamboo.quartz.SystemAuthorizedThreadPool$1.run(SystemAuthorizedThreadPool.java:38) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) INFO | jvm 1 | 2011/08/25 03:25:05 | Caused by: java.lang.Throwable: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device {noformat} Even after we freed 50GB on the server disk, we saw the following in the agent logs: {noformat} INFO | jvm 86 | 2011/08/25 10:31:31 | 2011-08-25 10:31:31,663 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239' was unsuccessful. Run 1 / 15. Exception thrown when running task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239', with message: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 86 | 2011/08/25 10:31:31 | 2011-08-25 10:31:31,664 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Waiting 1000ms before retrying... INFO | jvm 86 | 2011/08/25 10:31:32 | 2011-08-25 10:31:32,674 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239' was unsuccessful. Run 2 / 15. Exception thrown when running task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239', with message: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 86 | 2011/08/25 10:31:32 | 2011-08-25 10:31:32,674 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Waiting 2000ms before retrying... INFO | jvm 86 | 2011/08/25 10:31:34 | 2011-08-25 10:31:34,683 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239' finished successfully during retry number 2 INFO | jvm 86 | 2011/08/25 11:07:16 | 2011-08-25 11:07:16,646 INFO [QuartzScheduler_Worker-10] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... INFO | jvm 86 | 2011/08/25 11:10:46 | 2011-08-25 11:10:46,650 INFO [QuartzScheduler_Worker-7] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... INFO | jvm 86 | 2011/08/25 11:10:51 | 2011-08-25 11:10:51,647 INFO [QuartzScheduler_Worker-1] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... INFO | jvm 86 | 2011/08/25 11:12:01 | 2011-08-25 11:12:01,651 INFO [QuartzScheduler_Worker-6] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... {noformat} Attempting to restart one of the agents we saw: {noformat} INFO | jvm 2 | 2011/08/25 04:29:09 | 2011-08-25 04:29:09,370 WARN [Thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.buildqueue.PipelineDefinition com.atlassian.bamboo.buildqueue.manager.RemoteAgentManager.registerAgent(com.atlassian.bamboo.buildqueue.PipelineDefinition) throws com.atlassian.bamboo.license.BambooLicenseException; target is null INFO | jvm 2 | 2011/08/25 04:29:09 | javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 2 | 2011/08/25 04:29:09 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:170) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:52) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:21) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.Thread.run(Thread.java:662) INFO | jvm 2 | 2011/08/25 04:29:09 | Exiting due to fatal exception. INFO | jvm 2 | 2011/08/25 04:29:09 | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean' defined in class path resource [applicationContextRemoteAgentStartup.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 2 | 2011/08/25 04:29:09 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:170) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:52) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:21) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.Thread.run(Thread.java:662) INFO | jvm 2 | 2011/08/25 04:29:09 | 2011-08-25 04:29:09,371 FATAL [Thread-1] [AgentBootstrap] Exiting due to fatal exception. INFO | jvm 2 | 2011/08/25 04:29:09 | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean' defined in class path resource [applicationContextRemoteAgentStartup.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 2 | 2011/08/25 04:29:09 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:170) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:52) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:21) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.Thread.run(Thread.java:662) ERROR | wrapper | 2011/08/25 04:29:09 | JVM exited unexpectedly. {noformat} It was only when we restarted the server that the agents reconnected successfully (we didn't restart any of the agents). It seems that when the disk space error occurred and the agents all took themselves offline, Bamboo made it impossible for them to come back online even though they were trying continuously to do so. This seems like a bug relating to Bamboo's management of connection state for the remote agents.
    via by David Corley,
  • Seen the following exception on the agent and I believe they indicate a serious issue (an unhandled exception) in the agent as it 'went offline': {noformat} 2014-04-03 03:42:22,089 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [RemoteExecutionPhaseServiceImpl] A2. JIRA Master Tier 2 - Acceptance Tests and Suite Level Acceptance Tests - Deploy to Unicorn DEV instance # 492 (MASTERTWO-ATSLAT-JOB1-492) execution started 2014-04-03 03:42:22,119 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [TaskExecutorImpl] Skipping execution of task 'Task for destroying instance. Use only when want a clean run. Otherwise keep this task disabled. ' as it is disabled 2014-04-03 03:42:22,168 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [TaskExecutorImpl] Starting task 'Initial setup' of type 'com.atlassian.bamboo.plugins.bamboonicorn:setupunicorn' 2014-04-03 03:58:15,717 WARN [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.configuration.AdministrationConfig uration com.atlassian.bamboo.configuration.AdministrationConfigurationAccessor.getAdministrationConfiguration(); target is null javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy2.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.utils.BambooUrl.withBaseUrlFromConfiguration(BambooUrl.java:55) at com.atlassian.bamboo.variable.CustomVariableContextImpl.addVariables(CustomVariableContextImpl.java:505) at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:464) at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:334) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteBambooVariables(CustomVariableContextImpl.java:197) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteString(CustomVariableContextImpl.java:153) at com.atlassian.bamboo.task.TaskExecutorImpl.substituteConfigurationVariables(TaskExecutorImpl.java:281) at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:182) at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:98) at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:75) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:181) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:101) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:108) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:662) 2014-04-03 03:58:15,719 FATAL [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [ExecuteBuildTask] Failed to execute the build 'MASTERTWO-ATSLAT-JOB1-492' org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy2.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.utils.BambooUrl.withBaseUrlFromConfiguration(BambooUrl.java:55) at com.atlassian.bamboo.variable.CustomVariableContextImpl.addVariables(CustomVariableContextImpl.java:505) at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:464) at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:334) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteBambooVariables(CustomVariableContextImpl.java:197) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteString(CustomVariableContextImpl.java:153) at com.atlassian.bamboo.task.TaskExecutorImpl.substituteConfigurationVariables(TaskExecutorImpl.java:281) at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:182) at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:98) at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:75) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:181) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:101) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:108) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:662) 2014-04-03 03:58:15,796 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [RemoteExecutionPhaseServiceImpl] A2. JIRA Master Tier 2 - Acceptance Tests and Suite Level Acceptance Tests - Deploy to Unicorn DEV instance #492 (MASTERTWO-ATSLAT-JOB1-492) execution finished 2014-04-03 03:58:15,818 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [RemoteResultProcessor] Build A2. JIRA Master Tier 2 - Acceptance Tests and Suite Level Acceptance Tests - Deploy to Unicorn DEV instance #492 (MASTERTWO-ATSLAT-JOB1-492) completed on Bamboo agent. Sending results to server. 2014-04-03 03:58:15,867 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [DefaultBuildAgent] Setting context to null on agent-02-05.buildeng.atlassian.com/869204611 2014-04-03 03:58:15,867 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [BuildAgentControllerImpl] Agent 'agent-02-05.buildeng.atlassian.com' ready to take build from queue... {noformat} The build log contained the following output: {noformat} ... busy doing work ... simple 03-Apr-2014 03:53:15 Finished task 'Initial setup' error 03-Apr-2014 03:58:15 Failed to execute the build 'MASTERTWO-ATSLAT-JOB1-492': Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException simple 03-Apr-2014 03:58:15 Finalising the build... simple 03-Apr-2014 03:58:15 Stopping timer. simple 03-Apr-2014 03:58:15 Build MASTERTWO-ATSLAT-JOB1-492 completed. {noformat} The _Initial setup_ task was not the last task for that job, only number 2 of 4. It would be nice if the Bamboo agent would do a retry and fail after 3 or 5 failed tries. Also, it would be nice it there would a meaningful message in the build log (on error level) that the communication with the server failed.
    via by Martin Meinhold,
  • My company recently purchased most the Atlassian suite, including Bamboo with a few remote agents. I started a remote agent last week and it was working. A few days later and I cannot get it to start up due to exception: INFO | jvm 41 | 2011/04/04 11:40:15 | 2011-04-04 11:40:15,208 WARN [Thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.buildqueue.PipelineDefinition com.atlassian.bamboo.buildqueue.manager.RemoteAgentManager.registerAgent(com.atlassian.bamboo.buildqueue.PipelineDefinition) throws com.atlassian.bamboo.license.BambooLicenseException; target is null INFO | jvm 41 | 2011/04/04 11:40:15 | javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 41 | 2011/04/04 11:40:15 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 41 | 2011/04/04 11:40:15 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 41 | 2011/04/04 11:40:15 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 41 | 2011/04/04 11:40:15 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 41 | 2011/04/04 11:40:15 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 41 | 2011/04/04 11:40:15 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 41 | 2011/04/04 11:40:15 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 41 | 2011/04/04 11:40:15 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) Please let me know what further info would help investigation.
    via by Kallin nagelberg,
  • Today we experienced an issue where our Bamboo server ran out of disk space. It came to our attention due to to the fact that all of the remote agents went offline at the same time. We had set up the agent wrapper.conf to continue to try and reconnect to the server for 6 hours after an initial connection failure. When the server ran out of disk space we saw the following in the agent logs: {noformat} INFO | jvm 1 | 2011/08/25 03:25:05 | org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 1 | 2011/08/25 03:25:05 | Caused by: javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQConnection.doSyncSendPacket(ActiveMQConnection.java:1329) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1314) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1758) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:74) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.apache.activemq.pool.PooledProducer.send(PooledProducer.java:59) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:536) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:513) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate$2.doInJms(JmsTemplate.java:479) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:430) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:477) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:588) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:576) INFO | jvm 1 | 2011/08/25 03:25:05 | at com.atlassian.bamboo.v2.build.agent.remote.sender.JmsTemplateMessageSender.send(JmsTemplateMessageSender.java:33) INFO | jvm 1 | 2011/08/25 03:25:05 | at com.atlassian.bamboo.v2.build.agent.remote.heartbeat.AgentHeartBeatJob.execute(AgentHeartBeatJob.java:51) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.quartz.core.JobRunShell.run(JobRunShell.java:202) INFO | jvm 1 | 2011/08/25 03:25:05 | at com.atlassian.bamboo.quartz.SystemAuthorizedThreadPool$1.run(SystemAuthorizedThreadPool.java:38) INFO | jvm 1 | 2011/08/25 03:25:05 | at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) INFO | jvm 1 | 2011/08/25 03:25:05 | Caused by: java.lang.Throwable: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device {noformat} Even after we freed 50GB on the server disk, we saw the following in the agent logs: {noformat} INFO | jvm 86 | 2011/08/25 10:31:31 | 2011-08-25 10:31:31,663 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239' was unsuccessful. Run 1 / 15. Exception thrown when running task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239', with message: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 86 | 2011/08/25 10:31:31 | 2011-08-25 10:31:31,664 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Waiting 1000ms before retrying... INFO | jvm 86 | 2011/08/25 10:31:32 | 2011-08-25 10:31:32,674 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239' was unsuccessful. Run 2 / 15. Exception thrown when running task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239', with message: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: java.util.concurrent.ExecutionException: java.io.IOException: No space left on device INFO | jvm 86 | 2011/08/25 10:31:32 | 2011-08-25 10:31:32,674 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Waiting 2000ms before retrying... INFO | jvm 86 | 2011/08/25 10:31:34 | 2011-08-25 10:31:34,683 INFO [QuartzScheduler_Worker-2] [RetryingTaskExecutor] Task 'Sending message com.atlassian.bamboo.v2.build.agent.messages.UpdateHeartbeatMessage@2d20d239' finished successfully during retry number 2 INFO | jvm 86 | 2011/08/25 11:07:16 | 2011-08-25 11:07:16,646 INFO [QuartzScheduler_Worker-10] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... INFO | jvm 86 | 2011/08/25 11:10:46 | 2011-08-25 11:10:46,650 INFO [QuartzScheduler_Worker-7] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... INFO | jvm 86 | 2011/08/25 11:10:51 | 2011-08-25 11:10:51,647 INFO [QuartzScheduler_Worker-1] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... INFO | jvm 86 | 2011/08/25 11:12:01 | 2011-08-25 11:12:01,651 INFO [QuartzScheduler_Worker-6] [AgentHeartBeatJob] Not sending a new heartbeat since an old one is still being sent, dropping the current heartbeat... {noformat} Attempting to restart one of the agents we saw: {noformat} INFO | jvm 2 | 2011/08/25 04:29:09 | 2011-08-25 04:29:09,370 WARN [Thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.buildqueue.PipelineDefinition com.atlassian.bamboo.buildqueue.manager.RemoteAgentManager.registerAgent(com.atlassian.bamboo.buildqueue.PipelineDefinition) throws com.atlassian.bamboo.license.BambooLicenseException; target is null INFO | jvm 2 | 2011/08/25 04:29:09 | javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 2 | 2011/08/25 04:29:09 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:170) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:52) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:21) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.Thread.run(Thread.java:662) INFO | jvm 2 | 2011/08/25 04:29:09 | Exiting due to fatal exception. INFO | jvm 2 | 2011/08/25 04:29:09 | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean' defined in class path resource [applicationContextRemoteAgentStartup.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 2 | 2011/08/25 04:29:09 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:170) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:52) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:21) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.Thread.run(Thread.java:662) INFO | jvm 2 | 2011/08/25 04:29:09 | 2011-08-25 04:29:09,371 FATAL [Thread-1] [AgentBootstrap] Exiting due to fatal exception. INFO | jvm 2 | 2011/08/25 04:29:09 | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentRegistrationBean' defined in class path resource [applicationContextRemoteAgentStartup.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 2 | 2011/08/25 04:29:09 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 2 | 2011/08/25 04:29:09 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1203) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1167) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:427) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:122) INFO | jvm 2 | 2011/08/25 04:29:09 | at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:88) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:170) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:52) INFO | jvm 2 | 2011/08/25 04:29:09 | at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:21) INFO | jvm 2 | 2011/08/25 04:29:09 | at java.lang.Thread.run(Thread.java:662) ERROR | wrapper | 2011/08/25 04:29:09 | JVM exited unexpectedly. {noformat} It was only when we restarted the server that the agents reconnected successfully (we didn't restart any of the agents). It seems that when the disk space error occurred and the agents all took themselves offline, Bamboo made it impossible for them to come back online even though they were trying continuously to do so. This seems like a bug relating to Bamboo's management of connection state for the remote agents.
    via by David Corley,
  • Seen the following exception on the agent and I believe they indicate a serious issue (an unhandled exception) in the agent as it 'went offline': {noformat} 2014-04-03 03:42:22,089 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [RemoteExecutionPhaseServiceImpl] A2. JIRA Master Tier 2 - Acceptance Tests and Suite Level Acceptance Tests - Deploy to Unicorn DEV instance # 492 (MASTERTWO-ATSLAT-JOB1-492) execution started 2014-04-03 03:42:22,119 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [TaskExecutorImpl] Skipping execution of task 'Task for destroying instance. Use only when want a clean run. Otherwise keep this task disabled. ' as it is disabled 2014-04-03 03:42:22,168 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [TaskExecutorImpl] Starting task 'Initial setup' of type 'com.atlassian.bamboo.plugins.bamboonicorn:setupunicorn' 2014-04-03 03:58:15,717 WARN [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.configuration.AdministrationConfig uration com.atlassian.bamboo.configuration.AdministrationConfigurationAccessor.getAdministrationConfiguration(); target is null javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy2.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.utils.BambooUrl.withBaseUrlFromConfiguration(BambooUrl.java:55) at com.atlassian.bamboo.variable.CustomVariableContextImpl.addVariables(CustomVariableContextImpl.java:505) at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:464) at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:334) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteBambooVariables(CustomVariableContextImpl.java:197) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteString(CustomVariableContextImpl.java:153) at com.atlassian.bamboo.task.TaskExecutorImpl.substituteConfigurationVariables(TaskExecutorImpl.java:281) at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:182) at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:98) at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:75) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:181) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:101) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:108) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:662) 2014-04-03 03:58:15,719 FATAL [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [ExecuteBuildTask] Failed to execute the build 'MASTERTWO-ATSLAT-JOB1-492' org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy2.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.utils.BambooUrl.withBaseUrlFromConfiguration(BambooUrl.java:55) at com.atlassian.bamboo.variable.CustomVariableContextImpl.addVariables(CustomVariableContextImpl.java:505) at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:464) at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:334) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteBambooVariables(CustomVariableContextImpl.java:197) at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteString(CustomVariableContextImpl.java:153) at com.atlassian.bamboo.task.TaskExecutorImpl.substituteConfigurationVariables(TaskExecutorImpl.java:281) at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:182) at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:98) at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:75) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:181) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:101) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:108) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:662) 2014-04-03 03:58:15,796 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [RemoteExecutionPhaseServiceImpl] A2. JIRA Master Tier 2 - Acceptance Tests and Suite Level Acceptance Tests - Deploy to Unicorn DEV instance #492 (MASTERTWO-ATSLAT-JOB1-492) execution finished 2014-04-03 03:58:15,818 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [RemoteResultProcessor] Build A2. JIRA Master Tier 2 - Acceptance Tests and Suite Level Acceptance Tests - Deploy to Unicorn DEV instance #492 (MASTERTWO-ATSLAT-JOB1-492) completed on Bamboo agent. Sending results to server. 2014-04-03 03:58:15,867 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [DefaultBuildAgent] Setting context to null on agent-02-05.buildeng.atlassian.com/869204611 2014-04-03 03:58:15,867 INFO [3-BAM::agent-02-05.buildeng.atlassian.com::Agent:pool-6-thread-1] [BuildAgentControllerImpl] Agent 'agent-02-05.buildeng.atlassian.com' ready to take build from queue... {noformat} The build log contained the following output: {noformat} ... busy doing work ... simple 03-Apr-2014 03:53:15 Finished task 'Initial setup' error 03-Apr-2014 03:58:15 Failed to execute the build 'MASTERTWO-ATSLAT-JOB1-492': Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException simple 03-Apr-2014 03:58:15 Finalising the build... simple 03-Apr-2014 03:58:15 Stopping timer. simple 03-Apr-2014 03:58:15 Build MASTERTWO-ATSLAT-JOB1-492 completed. {noformat} The _Initial setup_ task was not the last task for that job, only number 2 of 4. It would be nice if the Bamboo agent would do a retry and fail after 3 or 5 failed tries. Also, it would be nice it there would a meaningful message in the build log (on error level) that the communication with the server failed.
    via by Martin Meinhold,
  • My company recently purchased most the Atlassian suite, including Bamboo with a few remote agents. I started a remote agent last week and it was working. A few days later and I cannot get it to start up due to exception: INFO | jvm 41 | 2011/04/04 11:40:15 | 2011-04-04 11:40:15,208 WARN [Thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.buildqueue.PipelineDefinition com.atlassian.bamboo.buildqueue.manager.RemoteAgentManager.registerAgent(com.atlassian.bamboo.buildqueue.PipelineDefinition) throws com.atlassian.bamboo.license.BambooLicenseException; target is null INFO | jvm 41 | 2011/04/04 11:40:15 | javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 41 | 2011/04/04 11:40:15 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 41 | 2011/04/04 11:40:15 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 41 | 2011/04/04 11:40:15 | at $Proxy10.registerAgent(Unknown Source) INFO | jvm 41 | 2011/04/04 11:40:15 | at com.atlassian.bamboo.v2.build.agent.remote.AgentRegistrationBean.registerAgent(AgentRegistrationBean.java:41) INFO | jvm 41 | 2011/04/04 11:40:15 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) INFO | jvm 41 | 2011/04/04 11:40:15 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) INFO | jvm 41 | 2011/04/04 11:40:15 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) INFO | jvm 41 | 2011/04/04 11:40:15 | at java.lang.reflect.Method.invoke(Method.java:597) INFO | jvm 41 | 2011/04/04 11:40:15 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1237) Please let me know what further info would help investigation.
    via by Kallin nagelberg,
  • Hi, we get random JMS timeout exceptions when the remote agent is running on Windows Server: INFO | jvm 1 | 2013/02/15 08:53:19 | 2013-02-15 08:53:19,796 WARN [1-BAM::CMD-CI-WIN-32BI.intl.<some>.net::Agent:pool-3-thread-1] [JmsClientInterceptor] Remote access error: ReflectiveMethodInvocation: public abstract com.atlassian.bamboo.configuration.AdministrationConfiguration com.atlassian.bamboo.configuration.AdministrationConfigurationManager.getAdministrationConfiguration(); target is null INFO | jvm 1 | 2013/02/15 08:53:19 | javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 1 | 2013/02/15 08:53:19 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 1 | 2013/02/15 08:53:19 | at $Proxy2.getAdministrationConfiguration(Unknown Source) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:366) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:299) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteBambooVariables(CustomVariableContextImpl.java:172) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteString(CustomVariableContextImpl.java:137) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.task.TaskExecutorImpl.substituteConfigurationVariables(TaskExecutorImpl.java:228) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:176) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:93) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:87) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:206) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:103) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:111) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:52) INFO | jvm 1 | 2013/02/15 08:53:19 | at java.lang.Thread.run(Unknown Source) INFO | jvm 1 | 2013/02/15 08:53:19 | 2013-02-15 08:53:19,812 FATAL [1-BAM::CMD-CI-WIN-32BI.intl.<some>.net::Agent:pool-3-thread-1] [ExecuteBuildTask] Failed to execute the build 'PROCESSING-CI131-NCRYPTWIN-518' INFO | jvm 1 | 2013/02/15 08:53:19 | org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 1 | 2013/02/15 08:53:19 | Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException INFO | jvm 1 | 2013/02/15 08:53:19 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) INFO | jvm 1 | 2013/02/15 08:53:19 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) INFO | jvm 1 | 2013/02/15 08:53:19 | at $Proxy2.getAdministrationConfiguration(Unknown Source) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.getCommonBuildVariables(CustomVariableContextImpl.java:366) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.filterVariableContextByName(CustomVariableContextImpl.java:299) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteBambooVariables(CustomVariableContextImpl.java:172) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.variable.CustomVariableContextImpl.substituteString(CustomVariableContextImpl.java:137) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.task.TaskExecutorImpl.substituteConfigurationVariables(TaskExecutorImpl.java:228) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:176) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:93) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask.call(ExecuteBuildTask.java:87) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:206) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:103) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:111) INFO | jvm 1 | 2013/02/15 08:53:19 | at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:52) INFO | jvm 1 | 2013/02/15 08:53:19 | at java.lang.Thread.run(Unknown Source) All builds are processes without issues for 1-2 weeks and then we get one of these exceptions and we have to rerun the jobs. Some more info: INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [AgentHeartBeatJobScheduler] Scheduled AgentHeartBeatJobScheduler to run every 60s. Next run at Fri Feb 15 08:12:54 EET 2013 INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] ********************************************************************************************************************************************** INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] * * INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] * Bamboo agent 'CMD-CI-WIN-32BI.intl.<some>.net' ready to receive builds. INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] * Remote Agent Home: C:\bamboo INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] * Broker URL: failover:(tcp://<some ip>:54663?wireFormat.maxInactivityDuration=300000)?maxReconnectAttempts=10&initialReconnectDelay=15000 INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] * * INFO | jvm 1 | 2013/02/15 08:12:54 | 2013-02-15 08:12:54,203 INFO [Thread-1] [RemoteAgent] **********************************************************************************************************************************************
    via by Julius Hutuleac,
    • org.springframework.remoting.RemoteAccessException: Cannot access JMS invoker remote service at [null]; nested exception is javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:151) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) at com.sun.proxy.$Proxy8.getAdministrationConfiguration(Unknown Source) at com.atlassian.bamboo.v2.build.agent.remote.NonSavingAdminConfigManager.getAdministrationConfiguration(NonSavingAdminConfigManager.java:33) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) at com.sun.proxy.$Proxy26.getAdministrationConfiguration(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) at com.sun.proxy.$Proxy26.getAdministrationConfiguration(Unknown Source) at com.atlassian.sandbox.bamboo.actions.SandboxCommandDecorationProvider.decorate(SandboxCommandDecorationProvider.java:30) at com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction.called(SandboxPreBuildAction.java:75) at com.atlassian.sandbox.bamboo.actions.SandboxPreBuildAction.call(SandboxPreBuildAction.java:53) at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.runPreBuildActions(PrepareBuildTask.java:134) at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:102) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:188) at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:112) at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:110) at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49) at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31) at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20) at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52) at java.lang.Thread.run(Thread.java:745) Caused by: javax.jms.JMSException: edu.emory.mathcs.backport.java.util.concurrent.TimeoutException at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.createJMSException(MultiplexingRequestor.java:205) at org.logicblaze.lingo.jms.impl.MultiplexingRequestor.request(MultiplexingRequestor.java:133) at com.atlassian.bamboo.lingo.BambooMultiplexingRequestor.request(BambooMultiplexingRequestor.java:66) at org.logicblaze.lingo.jms.JmsClientInterceptor.invoke(JmsClientInterceptor.java:138) ... 29 more
    No Bugmate found.