com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID:xxxxx)

Jenkins JIRA | Yoann Dubreuil | 2 years ago
  1. 0

    EC2 Jenkins slaves sometimes fails to launch because AWS throttle the API calls. While this would normally not be a problem, this results in a bunch of Jenkins offline slaves which don't go away and can clutter up the UI. When too many requests hit the EC2 API, Amazon responds with a 503 error like this one: {code} ERROR: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID:xxxxx) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:934) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:506) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:273) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8567) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4143) at hudson.plugins.ec2.EC2Computer._describeInstanceOnce(EC2Computer.java:158) at hudson.plugins.ec2.EC2Computer._describeInstance(EC2Computer.java:141) at hudson.plugins.ec2.EC2Computer.getState(EC2Computer.java:117) at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:60) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:232) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) {code} The EC2 Jenkins plugin could better handle this kind of AWS exceptions. If this error is thrown, maybe it could attempt again after a few minutes. Enhancing this behavior requires to wrap all API call to handle this error. Any advice on a way to do it properly ?

    Jenkins JIRA | 2 years ago | Yoann Dubreuil
    com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID:xxxxx)
  2. 0

    EC2 Jenkins slaves sometimes fails to launch because AWS throttle the API calls. While this would normally not be a problem, this results in a bunch of Jenkins offline slaves which don't go away and can clutter up the UI. When too many requests hit the EC2 API, Amazon responds with a 503 error like this one: {code} ERROR: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX) com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID:xxxxx) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:934) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:506) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:273) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8567) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4143) at hudson.plugins.ec2.EC2Computer._describeInstanceOnce(EC2Computer.java:158) at hudson.plugins.ec2.EC2Computer._describeInstance(EC2Computer.java:141) at hudson.plugins.ec2.EC2Computer.getState(EC2Computer.java:117) at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:60) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:232) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) {code} The EC2 Jenkins plugin could better handle this kind of AWS exceptions. If this error is thrown, maybe it could attempt again after a few minutes. Enhancing this behavior requires to wrap all API call to handle this error. Any advice on a way to do it properly ?

    Jenkins JIRA | 2 years ago | Yoann Dubreuil
    com.amazonaws.AmazonServiceException: Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID:xxxxx)
  3. 0

    [JIRA] [ec2-plugin] (JENKINS-24663) No such template for muiltipul eucalyptus cloud

    Google Groups | 10 months ago | fran...@gmail.com (JIRA)
    com.amazonaws.AmazonServiceException: The instance ID 'i-6b799381' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: a170e6f9-6759-4c10-935d-145f08e0942f)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    If multiple Eucalyptus clouds are defined in the plugin, the following error shows when the EC2 build image is launched: {noformat} ERROR: The instance ID 'i-85a826d1' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: c0492dd8-2548-4637-9a03-161c37654b72) com.amazonaws.AmazonServiceException: The instance ID 'i-85a826d1' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: c0492dd8-2548-4637-9a03-161c37654b72) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1307) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:894) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:597) at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:363) at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:329) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:308) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:13032) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:9037) at hudson.plugins.ec2.EC2Computer._describeInstanceOnce(EC2Computer.java:166) at hudson.plugins.ec2.EC2Computer._describeInstance(EC2Computer.java:160) at hudson.plugins.ec2.EC2Computer.getState(EC2Computer.java:125) at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:80) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:253) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {noformat} Logging into Eucalyptus shows in fact that the instance has launched correctly with all the correct settings (keys, security groups, etc.) as defined in the cloud configuration. It appears as though it is searching the first cloud for the instance, when it was launched in the second, similar to the issue in [JENKINS-12547], which resolved the issue for Amazon EC2 (but it appears not Eucalyptus) Also of note, when you attempt to launch the node manually using the *Provision via eucalyptus* option, you get the following: {noformat} java.lang.Exception: No such template: TEMPLATE_NAME {noformat} This has been reported in and appears related to [JENKINS-24663]

    Jenkins JIRA | 4 months ago | John Gallucci
    com.amazonaws.AmazonServiceException: The instance ID 'i-85a826d1' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: c0492dd8-2548-4637-9a03-161c37654b72)
  6. 0

    GitHub comment 219#128115636

    GitHub | 1 year ago | rangagopalan
    com.amazonaws.AmazonServiceException: AWS was not able to validate the provided access credentials (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: 0c07c8e9-a558-4a09-a743-485369c9622d)
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. com.amazonaws.AmazonServiceException

    Request limit exceeded. (Service: AmazonEC2; Status Code: 503; Error Code: RequestLimitExceeded; Request ID:xxxxx)

    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse()
  2. AWS SDK for Java - Core
    AmazonHttpClient.execute
    1. com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:934)
    2. com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:506)
    3. com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:273)
    3 frames
  3. AWS Java SDK for Amazon EC2
    AmazonEC2Client.describeInstances
    1. com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8567)
    2. com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:4143)
    2 frames
  4. hudson.plugins.ec2
    EC2ComputerLauncher.launch
    1. hudson.plugins.ec2.EC2Computer._describeInstanceOnce(EC2Computer.java:158)
    2. hudson.plugins.ec2.EC2Computer._describeInstance(EC2Computer.java:141)
    3. hudson.plugins.ec2.EC2Computer.getState(EC2Computer.java:117)
    4. hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:60)
    4 frames
  5. Hudson
    SlaveComputer$1.call
    1. hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:232)
    1 frame
  6. jenkins.util
    ContextResettingExecutorService$2.call
    1. jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    1 frame
  7. Java RT
    Thread.run
    1. java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    2. java.util.concurrent.FutureTask.run(Unknown Source)
    3. java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    4. java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    5. java.lang.Thread.run(Unknown Source)
    5 frames