com.amazonaws.AmazonServiceException: The instance ID 'i-ebe58f04' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: 2d4058ea-e5ca-44c8-aeb5-08d4fe637d41)

Jenkins JIRA | Paul Sokolovsky | 2 years ago
  1. 0

    Background: "Zombie slaves" (build slaves started by EC2 plugin, but not used for builds, and not shutdown properly) has been a problem with EC2 plugin since we started to use it. With older EC2 plugin versions, such slaves were visible in Jenkins, marked as "offline". We upgraded to newer plugin version (from 1.18 to 1.24, then 1.25/1.26) and rejoiced, as zombie slave issues appeared to have been gone. Until we checked our AWS web console and saw bunch of zombie slaves :-(. So, the only change is that they're no longer visible in Jenkins, and thus only harder to notice. So, I went to investigate what's up with it now. Here's typical exception regarding a zombie slave in Jenkins log: {code} Mar 02, 2015 6:36:57 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave Kernel cloud (ami-cb2509a2) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-ebe58f04' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: 2d4058ea-e5ca-44c8-aeb5-08d4fe637d41) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.createTags(AmazonEC2Client.java:4990) at hudson.plugins.ec2.SlaveTemplate.updateRemoteTags(SlaveTemplate.java:732) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:426) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:287) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:398) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:394) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code} Looking at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:426) (https://github.com/jenkinsci/ec2-plugin/blob/master/src/main/java/hudson/plugins/ec2/SlaveTemplate.java#L426), there's already a loop to try to perform operation with sleep between attempts. But it checks for "InvalidInstanceRequestID" error. Note that there was already https://github.com/jenkinsci/ec2-plugin/commit/9a596f0577b29a3e1835143f5d51520babdd7c1f to correct "typo in error code". However, looking at http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html , there's no "InvalidInstanceRequestID" error selector (as currently in the code), but there's "InvalidInstanceID" (as in the exception above). So the proper fix appears to be to replace error code in https://github.com/jenkinsci/ec2-plugin/blob/master/src/main/java/hudson/plugins/ec2/SlaveTemplate.java#L429 to "InvalidInstanceID.NotFound".

    Jenkins JIRA | 2 years ago | Paul Sokolovsky
    com.amazonaws.AmazonServiceException: The instance ID 'i-ebe58f04' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: 2d4058ea-e5ca-44c8-aeb5-08d4fe637d41)
  2. 0

    Background: "Zombie slaves" (build slaves started by EC2 plugin, but not used for builds, and not shutdown properly) has been a problem with EC2 plugin since we started to use it. With older EC2 plugin versions, such slaves were visible in Jenkins, marked as "offline". We upgraded to newer plugin version (from 1.18 to 1.24, then 1.25/1.26) and rejoiced, as zombie slave issues appeared to have been gone. Until we checked our AWS web console and saw bunch of zombie slaves :-(. So, the only change is that they're no longer visible in Jenkins, and thus only harder to notice. So, I went to investigate what's up with it now. Here's typical exception regarding a zombie slave in Jenkins log: {code} Mar 02, 2015 6:36:57 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave Kernel cloud (ami-cb2509a2) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-ebe58f04' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: 2d4058ea-e5ca-44c8-aeb5-08d4fe637d41) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.createTags(AmazonEC2Client.java:4990) at hudson.plugins.ec2.SlaveTemplate.updateRemoteTags(SlaveTemplate.java:732) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:426) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:287) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:398) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:394) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) {code} Looking at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:426) (https://github.com/jenkinsci/ec2-plugin/blob/master/src/main/java/hudson/plugins/ec2/SlaveTemplate.java#L426), there's already a loop to try to perform operation with sleep between attempts. But it checks for "InvalidInstanceRequestID" error. Note that there was already https://github.com/jenkinsci/ec2-plugin/commit/9a596f0577b29a3e1835143f5d51520babdd7c1f to correct "typo in error code". However, looking at http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html , there's no "InvalidInstanceRequestID" error selector (as currently in the code), but there's "InvalidInstanceID" (as in the exception above). So the proper fix appears to be to replace error code in https://github.com/jenkinsci/ec2-plugin/blob/master/src/main/java/hudson/plugins/ec2/SlaveTemplate.java#L429 to "InvalidInstanceID.NotFound".

    Jenkins JIRA | 2 years ago | Paul Sokolovsky
    com.amazonaws.AmazonServiceException: The instance ID 'i-ebe58f04' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: 2d4058ea-e5ca-44c8-aeb5-08d4fe637d41)
  3. 0

    {code} Oct 07, 2014 11:41:27 AM hudson.plugins.ec2.EC2Cloud addProvisionedSlave INFO: Provisioning for AMI ami-abcda06b; Estimated number of total slaves: 241; Estimated number of slaves for ami ami-abcda06b: 30 Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:27 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: Launching ami-abcda06b for template aws builder Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-abcda06b]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-1a2bce59]}, {Name: instance.group-id,Values: [sg-e9bb378c, sg-1237d80c]}, {Name: key-name,Values: [jenkins-builder]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:Name,Values: [jenkins-builder.elastic]}, {Name: tag:created_by,Values: [ci.corp.mydomain.com]}, {Name: tag:Service,Values: [jenkins]}, {Name: tag:Team,Values: [releng]}, {Name: tag:Stage,Values: [prod]}, {Name: instance-state-name,Values: [stopped, stopping]}],} Oct 07, 2014 11:41:28 AM hudson.plugins.ec2.EC2Cloud addProvisionedSlave INFO: Provisioning for AMI ami-abcda06b; Estimated number of total slaves: 242; Estimated number of slaves for ami ami-abcda06b: 31 Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:14.0 Oct 07, 2014 11:41:28 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:13.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:12.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:11.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:10.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:9.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:8.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:7.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:6.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:5.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:4.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:3.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:2.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:1.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:0.0 Oct 07, 2014 11:41:28 AM hudson.plugins.sauce_ondemand.SauceEnvironmentUtil getBuildWrapper INFO: Could not find SauceOnDemandBuildWrapper on project hudson.model.FreeStyleProject@222a3030[gerrit_cdc] Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave aws builder (ami-abcda06b) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-7705eb7d' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcd14da-de6a-40fa-967b-945290c35287) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.createTags(AmazonEC2Client.java:4990) at hudson.plugins.ec2.SlaveTemplate.updateRemoteTags(SlaveTemplate.java:667) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:410) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:284) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:370) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:366) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave aws builder (ami-abcda06b) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-9407e99e' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcddcc1-f486-4598-b140-fb1656d24aef) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:385) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:284) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:370) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:366) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-abcda06b]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-1a2bce59]}, {Name: instance.group-id,Values: [sg-e9bb378c, sg-1237d80c]}, {Name: key-name,Values: [jenkins-builder]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:Name,Values: [jenkins-builder.elastic]}, {Name: tag:created_by,Values: [ci.corp.mydomain.com]}, {Name: tag:Service,Values: [jenkins]}, {Name: tag:Team,Values: [releng]}, {Name: tag:Stage,Values: [prod]}, {Name: instance-state-name,Values: [stopped, stopping]}],} No existing instance found - created: {InstanceId: i-1abceb45,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.152,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 58ad84d4-2934-4647-b5e5-d7084db6c33a,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d4fbeeb1,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a7b54391,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-1abceb45,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.152,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 58ad84d4-2934-4647-b5e5-d7084db6c33a,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d4fbeeb1,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a7b54391,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.EC2RetentionStrategy start INFO: Start requested for aws builder (i-1abceb45) No existing instance found - created: {InstanceId: i-2bcdeb44,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.243,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 2988c7e3-b1ae-4d35-b294-9c1fc36b9d62,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d6fbeeb3,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a0b54396,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-2bcdeb44,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.243,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 2988c7e3-b1ae-4d35-b294-9c1fc36b9d62,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d6fbeeb3,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a0b54396,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.EC2RetentionStrategy start INFO: Start requested for aws builder (i-2bcdeb44) No existing instance found - created: {InstanceId: i-a709e7ad,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:30 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.22,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 171b131a-7067-4ac8-b106-89e1400304b1,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d1fbeeb4,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-3ab3450c,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:30 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:30 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-a709e7ad,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:30 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.22,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 171b131a-7067-4ac8-b106-89e1400304b1,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d1fbeeb4,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-3ab3450c,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:30 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} O {code} Not sure this is related or not too: {code} Oct 06, 2014 4:28:24 PM hudson.plugins.ec2.EC2Cloud provision WARNING: Failed to count the # of live instances on EC2 com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: 12c06f12-abcd-4c56-bae5-51ff7fa83423) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:8087) at hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:212) at hudson.plugins.ec2.EC2Cloud.addProvisionedSlave(EC2Cloud.java:272) at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:361) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) {code}

    Jenkins JIRA | 2 years ago | Sam Xiao
    com.amazonaws.AmazonServiceException: The instance ID 'i-7705eb7d' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcd14da-de6a-40fa-967b-945290c35287)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    {code} Oct 07, 2014 11:41:27 AM hudson.plugins.ec2.EC2Cloud addProvisionedSlave INFO: Provisioning for AMI ami-abcda06b; Estimated number of total slaves: 241; Estimated number of slaves for ami ami-abcda06b: 30 Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:27 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: Launching ami-abcda06b for template aws builder Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-abcda06b]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-1a2bce59]}, {Name: instance.group-id,Values: [sg-e9bb378c, sg-1237d80c]}, {Name: key-name,Values: [jenkins-builder]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:Name,Values: [jenkins-builder.elastic]}, {Name: tag:created_by,Values: [ci.corp.mydomain.com]}, {Name: tag:Service,Values: [jenkins]}, {Name: tag:Team,Values: [releng]}, {Name: tag:Stage,Values: [prod]}, {Name: instance-state-name,Values: [stopped, stopping]}],} Oct 07, 2014 11:41:28 AM hudson.plugins.ec2.EC2Cloud addProvisionedSlave INFO: Provisioning for AMI ami-abcda06b; Estimated number of total slaves: 242; Estimated number of slaves for ami ami-abcda06b: 31 Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:14.0 Oct 07, 2014 11:41:28 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: Launching ami-abcda06b for template aws builder Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:13.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:12.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:11.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:10.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:9.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:8.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:7.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:6.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:5.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:4.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:3.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:2.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:1.0 Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update INFO: Started provisioning aws builder (ami-abcda06b) from ec2-us-east-1 with 1 executors. Remaining excess workload:0.0 Oct 07, 2014 11:41:28 AM hudson.plugins.sauce_ondemand.SauceEnvironmentUtil getBuildWrapper INFO: Could not find SauceOnDemandBuildWrapper on project hudson.model.FreeStyleProject@222a3030[gerrit_cdc] Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave aws builder (ami-abcda06b) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-7705eb7d' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcd14da-de6a-40fa-967b-945290c35287) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.createTags(AmazonEC2Client.java:4990) at hudson.plugins.ec2.SlaveTemplate.updateRemoteTags(SlaveTemplate.java:667) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:410) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:284) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:370) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:366) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Oct 07, 2014 11:41:28 AM hudson.slaves.NodeProvisioner update WARNING: Provisioned slave aws builder (ami-abcda06b) failed to launch com.amazonaws.AmazonServiceException: The instance ID 'i-9407e99e' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcddcc1-f486-4598-b140-fb1656d24aef) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:385) at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:284) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:370) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:366) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Looking for existing instances with describe-instance: {InstanceIds: [],Filters: [{Name: image-id,Values: [ami-abcda06b]}, {Name: availability-zone,Values: [us-east-1a]}, {Name: subnet-id,Values: [subnet-1a2bce59]}, {Name: instance.group-id,Values: [sg-e9bb378c, sg-1237d80c]}, {Name: key-name,Values: [jenkins-builder]}, {Name: instance-type,Values: [m3.large]}, {Name: tag:Name,Values: [jenkins-builder.elastic]}, {Name: tag:created_by,Values: [ci.corp.mydomain.com]}, {Name: tag:Service,Values: [jenkins]}, {Name: tag:Team,Values: [releng]}, {Name: tag:Stage,Values: [prod]}, {Name: instance-state-name,Values: [stopped, stopping]}],} No existing instance found - created: {InstanceId: i-1abceb45,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.152,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 58ad84d4-2934-4647-b5e5-d7084db6c33a,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d4fbeeb1,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a7b54391,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-1abceb45,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.152,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 58ad84d4-2934-4647-b5e5-d7084db6c33a,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d4fbeeb1,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a7b54391,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.152,PrivateDnsName: ip-10-32-1-152.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.EC2RetentionStrategy start INFO: Start requested for aws builder (i-1abceb45) No existing instance found - created: {InstanceId: i-2bcdeb44,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.243,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 2988c7e3-b1ae-4d35-b294-9c1fc36b9d62,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d6fbeeb3,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a0b54396,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-2bcdeb44,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:29 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.243,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 2988c7e3-b1ae-4d35-b294-9c1fc36b9d62,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d6fbeeb3,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-a0b54396,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:29 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.243,PrivateDnsName: ip-10-32-1-243.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:29 AM hudson.plugins.ec2.EC2RetentionStrategy start INFO: Start requested for aws builder (i-2bcdeb44) No existing instance found - created: {InstanceId: i-a709e7ad,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:30 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.22,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 171b131a-7067-4ac8-b106-89e1400304b1,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d1fbeeb4,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-3ab3450c,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:30 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} Oct 07, 2014 11:41:30 AM hudson.plugins.ec2.SlaveTemplate provisionOndemand INFO: No existing instance found - created: {InstanceId: i-a709e7ad,ImageId: ami-abcda06b,State: {Code: 0,Name: pending},PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,PublicDnsName: ,StateTransitionReason: ,KeyName: jenkins-builder,AmiLaunchIndex: 0,ProductCodes: [],InstanceType: m3.large,LaunchTime: Tue Oct 07 11:41:30 PDT 2014,Placement: {AvailabilityZone: us-east-1a,GroupName: ,Tenancy: default},KernelId: aki-98e26fa8,Monitoring: {State: disabled},SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,PrivateIpAddress: 10.32.1.22,StateReason: {Code: pending,Message: pending},Architecture: x86_64,RootDeviceType: ebs,RootDeviceName: /dev/sda1,BlockDeviceMappings: [],VirtualizationType: paravirtual,ClientToken: 171b131a-7067-4ac8-b106-89e1400304b1,Tags: [{Key: Service,Value: jenkins}, {Key: Name,Value: jenkins-builder.elastic}, {Key: Stage,Value: prod}, {Key: Team,Value: releng}, {Key: created_by,Value: ci.corp.mydomain.com}],SecurityGroups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],SourceDestCheck: true,Hypervisor: xen,NetworkInterfaces: [{NetworkInterfaceId: eni-d1fbeeb4,SubnetId: subnet-1a2bce59,VpcId: vpc-26a44a43,Description: ,OwnerId: 742630462602,Status: in-use,PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,SourceDestCheck: true,Groups: [{GroupName: corp,GroupId: sg-e9bb378c}, {GroupName: jenkins,GroupId: sg-1237d80c}],Attachment: {AttachmentId: eni-attach-3ab3450c,DeviceIndex: 0,Status: attaching,AttachTime: Tue Oct 07 11:41:30 PDT 2014,DeleteOnTermination: true},PrivateIpAddresses: [{PrivateIpAddress: 10.32.1.22,PrivateDnsName: ip-10-32-1-22.us-east-1.compute.internal,Primary: true,}]}],EbsOptimized: false,} O {code} Not sure this is related or not too: {code} Oct 06, 2014 4:28:24 PM hudson.plugins.ec2.EC2Cloud provision WARNING: Failed to count the # of live instances on EC2 com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: 12c06f12-abcd-4c56-bae5-51ff7fa83423) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4137) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:8087) at hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:212) at hudson.plugins.ec2.EC2Cloud.addProvisionedSlave(EC2Cloud.java:272) at hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:361) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) {code}

    Jenkins JIRA | 2 years ago | Sam Xiao
    com.amazonaws.AmazonServiceException: The instance ID 'i-7705eb7d' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcd14da-de6a-40fa-967b-945290c35287)
  6. 0

    Jenkins issues - [JIRA] [ec2] (JENKINS-25029) Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcdef)

    nabble.com | 1 year ago
    com.amazonaws.AmazonServiceException: The instance ID 'i-7705eb7d' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcd14da-de6a-40fa-967b-945290c35287)
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. com.amazonaws.AmazonServiceException

    The instance ID 'i-ebe58f04' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: 2d4058ea-e5ca-44c8-aeb5-08d4fe637d41)

    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse()
  2. AWS SDK for Java - Core
    AmazonHttpClient.execute
    1. com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:886)
    2. com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:484)
    3. com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:256)
    3 frames
  3. AWS Java SDK for Amazon EC2
    AmazonEC2Client.createTags
    1. com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8798)
    2. com.amazonaws.services.ec2.AmazonEC2Client.createTags(AmazonEC2Client.java:4990)
    2 frames
  4. hudson.plugins.ec2
    EC2Cloud$1.call
    1. hudson.plugins.ec2.SlaveTemplate.updateRemoteTags(SlaveTemplate.java:732)
    2. hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:426)
    3. hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:287)
    4. hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:398)
    5. hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:394)
    5 frames
  5. jenkins.util
    ContextResettingExecutorService$2.call
    1. jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    1 frame
  6. Java RT
    Thread.run
    1. java.util.concurrent.FutureTask.run(FutureTask.java:262)
    2. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    3. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    4. java.lang.Thread.run(Thread.java:745)
    4 frames