com.amazonaws.AmazonServiceException: Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77)

Jenkins JIRA | Ximon Eighteen | 2 years ago
  1. 0

    [JENKINS-26854] EC2 slave launch stops working after a while with AmazonServiceException "Request has expired" - Jenkins JIRA

    jenkins-ci.org | 1 year ago
    com.amazonaws.AmazonServiceException: Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77)
  2. 0

    After Jenkins first starts it is able to launch EC2 slaves, both manually and when jobs indicate they need to use the slave label. A few hours later (not sure how long, maybe 24 hours?) slaves no longer start, manually or automatically. In "Manage Jenkins -> System Log -> All Jenkins Logs" the following error occurs repeatedly. Restarting Jenkins solves the problem. {code}Started EC2 alive slaves monitor Feb 09, 2015 5:14:47 AM INFO hudson.model.AsyncPeriodicWork$1 run Finished EC2 alive slaves monitor. 0 ms Feb 09, 2015 5:15:51 AM INFO hudson.plugins.ec2.EC2Cloud provision Excess workload after pending Spot instances: 1 Feb 09, 2015 5:15:53 AM WARNING hudson.plugins.ec2.EC2Cloud provision Failed to count the # of live instances on EC2 com.amazonaws.AmazonServiceException: Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:8087) at hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:228) at hudson.plugins.ec2.EC2Cloud.addProvisionedSlave(EC2Cloud.java:299) at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:389) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745){code} I also don't understand the log statement {{Excess workload after pending Spot instances: 1}} as I have not ticked the "Use Spot instance" tick box. In my cloud settings I have ticked the "Use EC2 instance profile to obtain credentials" and have set both the access key and secret key values to "THIS VALUE IS NOT USED - THE INSTANCE PROFILE IS USED INSTEAD".

    Jenkins JIRA | 2 years ago | Ximon Eighteen
    com.amazonaws.AmazonServiceException: Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77)
  3. 0

    After Jenkins first starts it is able to launch EC2 slaves, both manually and when jobs indicate they need to use the slave label. A few hours later (not sure how long, maybe 24 hours?) slaves no longer start, manually or automatically. In "Manage Jenkins -> System Log -> All Jenkins Logs" the following error occurs repeatedly. Restarting Jenkins solves the problem. {code}Started EC2 alive slaves monitor Feb 09, 2015 5:14:47 AM INFO hudson.model.AsyncPeriodicWork$1 run Finished EC2 alive slaves monitor. 0 ms Feb 09, 2015 5:15:51 AM INFO hudson.plugins.ec2.EC2Cloud provision Excess workload after pending Spot instances: 1 Feb 09, 2015 5:15:53 AM WARNING hudson.plugins.ec2.EC2Cloud provision Failed to count the # of live instances on EC2 com.amazonaws.AmazonServiceException: Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:8087) at hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:228) at hudson.plugins.ec2.EC2Cloud.addProvisionedSlave(EC2Cloud.java:299) at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:389) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745){code} I also don't understand the log statement {{Excess workload after pending Spot instances: 1}} as I have not ticked the "Use Spot instance" tick box. In my cloud settings I have ticked the "Use EC2 instance profile to obtain credentials" and have set both the access key and secret key values to "THIS VALUE IS NOT USED - THE INSTANCE PROFILE IS USED INSTEAD".

    Jenkins JIRA | 2 years ago | Ximon Eighteen
    com.amazonaws.AmazonServiceException: Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    [JENKINS-25029] Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcdef) - Jenkins JIRA

    jenkins-ci.org | 7 months ago
    com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: 12c06f12-abcd-4c56-bae5-51ff7fa83423)
  6. 0

    {code} Oct 07, 2014 11:41:27 AM hudson.plugins.ec2.EC2Cloud addProvisionedSlave INFO: Provisioning for AMI ami-abcda06b; Estimated number of total slaves: 241; Estimated number of slaves for ami ami-abcda06b: 30 Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:27 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: Launching ami-abcda06b for template aws builder Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-abcda06b]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-1a2bce59]}, {Name: instance.group-id,Values: [sg-e9bb378c, sg-1237d80c]}, {Name: key-name,Values: [jenkins-builder]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:Name,Values: [jenkins-builder.elastic]}, {Name: tag:created_by,Values: [ci.corp.mydomain.com]}, {Name: tag:Service,Values: [jenkins]}, {Name: tag:Team,Values: [releng]}, {Name: tag:Stage,Values: [prod]}, {Name: instance-state-name,Values: [stopped, stopping]}],} Oct 07, 2014 11:41:28 AM hudson.plugins.ec2.EC2Cloud addProvisionedSlave INFO: Provisioning for AMI ami-abcda06b; Estimated number of total slaves: 242; Estimated number of slaves for ami ami-abcda06b: 31 Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:14.0 Oct 07, 2014 11:41:28 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:13.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:12.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:11.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:10.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:9.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:8.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:7.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:6.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:5.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:4.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:3.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:2.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:1.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:0.0 Oct 07, 2014 11:41:28 AM hudson.plugins.sauce_ondemand.SauceEnvironmentUtil getBuildWrapper INFO: Could not find SauceOnDemandBuildWrapper on project hudson.model.FreeStyleProject@222a3030[gerrit_cdc] Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave aws builder (ami-abcda06b) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-7705eb7d' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcd14da-de6a-40fa-967b-945290c35287) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.createTags(AmazonEC2Client.java:4990) at hudson.plugins.ec2.SlaveTemplate.updateRemoteTags(SlaveTemplate.java:667) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:410) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:284) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:370) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:366) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave aws builder (ami-abcda06b) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-9407e99e' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcddcc1-f486-4598-b140-fb1656d24aef) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:385) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:284) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:370) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:366) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-abcda06b]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-1a2bce59]}, {Name: instance.group-id,Values: [sg-e9bb378c, sg-1237d80c]}, {Name: key-name,Values: [jenkins-builder]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:Name,Values: [jenkins-builder.elastic]}, {Name: tag:created_by,Values: [ci.corp.mydomain.com]}, {Name: tag:Service,Values: [jenkins]}, {Name: tag:Team,Values: [releng]}, {Name: tag:Stage,Values: [prod]}, {Name: instance-state-name,Values: [stopped, stopping]}],} No existing instance found - created: {InstanceId: i-1abceb45,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.152,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 58ad84d4-2934-4647-b5e5-d7084db6c33a,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d4fbeeb1,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a7b54391,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-1abceb45,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.152,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 58ad84d4-2934-4647-b5e5-d7084db6c33a,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d4fbeeb1,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a7b54391,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.EC2RetentionStrategy start INFO: Start requested for aws builder (i-1abceb45) No existing instance found - created: {InstanceId: i-2bcdeb44,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.243,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 2988c7e3-b1ae-4d35-b294-9c1fc36b9d62,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d6fbeeb3,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a0b54396,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-2bcdeb44,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.243,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 2988c7e3-b1ae-4d35-b294-9c1fc36b9d62,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d6fbeeb3,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a0b54396,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.EC2RetentionStrategy start INFO: Start requested for aws builder (i-2bcdeb44) No existing instance found - created: {InstanceId: i-a709e7ad,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:30 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.22,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 171b131a-7067-4ac8-b106-89e1400304b1,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d1fbeeb4,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-3ab3450c,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:30 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:30 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-a709e7ad,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:30 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.22,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 171b131a-7067-4ac8-b106-89e1400304b1,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d1fbeeb4,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-3ab3450c,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:30 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} O {code} Not sure this is related or not too: {code} Oct 06, 2014 4:28:24 PM hudson.plugins.ec2.EC2Cloud provision WARNING: Failed to count the # of live instances on EC2 com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: 12c06f12-abcd-4c56-bae5-51ff7fa83423) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:8087) at hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:212) at hudson.plugins.ec2.EC2Cloud.addProvisionedSlave(EC2Cloud.java:272) at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:361) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) {code}

    Jenkins JIRA | 2 years ago | Sam Xiao
    com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: 12c06f12-abcd-4c56-bae5-51ff7fa83423)

  1. aldrinleal 1 times, last 4 months ago
  2. tyson925 3 times, last 5 months ago
41 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. com.amazonaws.AmazonServiceException

    Request has expired. (Service: AmazonEC2; Status Code: 400; Error Code: RequestExpired; Request ID: 59f7935f-15f0-455c-a6f1-f6057f5ffc77)

    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse()
  2. AWS SDK for Java - Core
    AmazonHttpClient.execute
    1. com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886)
    2. com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484)
    3. com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256)
    3 frames
  3. AWS Java SDK for Amazon EC2
    AmazonEC2Client.describeInstances
    1. com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798)
    2. com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137)
    3. com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:8087)
    3 frames
  4. hudson.plugins.ec2
    EC2Cloud.provision
    1. hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:228)
    2. hudson.plugins.ec2.EC2Cloud.addProvisionedSlave(EC2Cloud.java:299)
    3. hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:389)
    3 frames
  5. Hudson
    SafeTimerTask.run
    1. hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281)
    2. hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51)
    3. hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368)
    4. hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
    4 frames
  6. Java RT
    ThreadPoolExecutor$Worker.run
    1. java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    2. java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    3. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    4. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    5. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    6. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    6 frames