com.amazonaws.AmazonServiceException: The service is unavailable. Please try again shortly. (Service: AmazonEC2; Status Code: 503; Error Code: Unavailable; Re-rw-r---quest ID: ff9e539b-f87f-4143-80b6-e11fbf61378e)

Jenkins JIRA | Jeroen Smolenaers | 1 year ago
  1. 0

    We use ec2-plugin to start on-demand slaves and terminate them after 30m of idle time. This was working fine, until last week when we noticed a full build queue and an empty list of build executors. On EC2, there actually was a slave running (i-1e42a2a7). Below is the relevant fragment from the log: {code} WARNING: Failed to fetch EC2 instance: i-1e42a2a7 com.amazonaws.AmazonServiceException: The service is unavailable. Please try again shortly. (Service: AmazonEC2; Status Code: 503; Error Code: Unavailable; Re-rw-r---quest ID: ff9e539b-f87f-4143-80b6-e11fbf61378e) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:11768) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:5899) at hudson.plugins.ec2.EC2AbstractSlave.getInstance(EC2AbstractSlave.java:232) at hudson.plugins.ec2.EC2AbstractSlave.fetchLiveInstanceData(EC2AbstractSlave.java:378) at hudson.plugins.ec2.EC2AbstractSlave.isAlive(EC2AbstractSlave.java:350) at hudson.plugins.ec2.EC2OndemandSlave.terminate(EC2OndemandSlave.java:63) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:297) at hudson.plugins.ec2.EC2RetentionStrategy._check(EC2RetentionStrategy.java:99) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:71) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:40) at hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:70) at hudson.model.Queue._withLock(Queue.java:1286) at hudson.model.Queue.withLock(Queue.java:1169) at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:61) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Oct 30, 2015 12:50:32 PM hudson.plugins.ec2.EC2OndemandSlave terminate INFO: EC2 instance already terminated: i-1e42a2a7 Oct 30, 2015 12:50:32 PM hudson.plugins.ec2.EC2OndemandSlave terminate INFO: Removed EC2 instance from jenkins master: i-1e42a2a7 {code}

    Jenkins JIRA | 1 year ago | Jeroen Smolenaers
    com.amazonaws.AmazonServiceException: The service is unavailable. Please try again shortly. (Service: AmazonEC2; Status Code: 503; Error Code: Unavailable; Re-rw-r---quest ID: ff9e539b-f87f-4143-80b6-e11fbf61378e)
  2. 0

    We use ec2-plugin to start on-demand slaves and terminate them after 30m of idle time. This was working fine, until last week when we noticed a full build queue and an empty list of build executors. On EC2, there actually was a slave running (i-1e42a2a7). Below is the relevant fragment from the log: {code} WARNING: Failed to fetch EC2 instance: i-1e42a2a7 com.amazonaws.AmazonServiceException: The service is unavailable. Please try again shortly. (Service: AmazonEC2; Status Code: 503; Error Code: Unavailable; Re-rw-r---quest ID: ff9e539b-f87f-4143-80b6-e11fbf61378e) at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:11768) at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:5899) at hudson.plugins.ec2.EC2AbstractSlave.getInstance(EC2AbstractSlave.java:232) at hudson.plugins.ec2.EC2AbstractSlave.fetchLiveInstanceData(EC2AbstractSlave.java:378) at hudson.plugins.ec2.EC2AbstractSlave.isAlive(EC2AbstractSlave.java:350) at hudson.plugins.ec2.EC2OndemandSlave.terminate(EC2OndemandSlave.java:63) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:297) at hudson.plugins.ec2.EC2RetentionStrategy._check(EC2RetentionStrategy.java:99) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:71) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:40) at hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:70) at hudson.model.Queue._withLock(Queue.java:1286) at hudson.model.Queue.withLock(Queue.java:1169) at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:61) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Oct 30, 2015 12:50:32 PM hudson.plugins.ec2.EC2OndemandSlave terminate INFO: EC2 instance already terminated: i-1e42a2a7 Oct 30, 2015 12:50:32 PM hudson.plugins.ec2.EC2OndemandSlave terminate INFO: Removed EC2 instance from jenkins master: i-1e42a2a7 {code}

    Jenkins JIRA | 1 year ago | Jeroen Smolenaers
    com.amazonaws.AmazonServiceException: The service is unavailable. Please try again shortly. (Service: AmazonEC2; Status Code: 503; Error Code: Unavailable; Re-rw-r---quest ID: ff9e539b-f87f-4143-80b6-e11fbf61378e)
  3. 0

    GitHub comment 219#128115636

    GitHub | 1 year ago | rangagopalan
    com.amazonaws.AmazonServiceException: AWS was not able to validate the provided access credentials (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: 0c07c8e9-a558-4a09-a743-485369c9622d)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Jenkins issues - [JIRA] [ec2] (JENKINS-25029) Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcdef)

    nabble.com | 1 year ago
    com.amazonaws.AmazonServiceException: The instance ID 'i-9407e99e' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidInstanceID.NotFound; Request ID: abcddcc1-f486-4598-b140-fb1656d24aef)
  6. 0

    AWS Developer Forums: Why I am not authorized to perform this ...

    amazon.com | 1 year ago
    com.amazonaws.AmazonServiceException: You are not authorized to perform operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: 93db2696-b48a-496e-ab19-e0b64c987c6c)

  1. aldrinleal 1 times, last 4 months ago
  2. tyson925 3 times, last 5 months ago
41 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. com.amazonaws.AmazonServiceException

    The service is unavailable. Please try again shortly. (Service: AmazonEC2; Status Code: 503; Error Code: Unavailable; Re-rw-r---quest ID: ff9e539b-f87f-4143-80b6-e11fbf61378e)

    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse()
  2. AWS SDK for Java - Core
    AmazonHttpClient.execute
    1. com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160)
    2. com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748)
    3. com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467)
    4. com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302)
    4 frames
  3. AWS Java SDK for Amazon EC2
    AmazonEC2Client.describeInstances
    1. com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:11768)
    2. com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:5899)
    2 frames
  4. hudson.plugins.ec2
    EC2RetentionStrategy.check
    1. hudson.plugins.ec2.EC2AbstractSlave.getInstance(EC2AbstractSlave.java:232)
    2. hudson.plugins.ec2.EC2AbstractSlave.fetchLiveInstanceData(EC2AbstractSlave.java:378)
    3. hudson.plugins.ec2.EC2AbstractSlave.isAlive(EC2AbstractSlave.java:350)
    4. hudson.plugins.ec2.EC2OndemandSlave.terminate(EC2OndemandSlave.java:63)
    5. hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:297)
    6. hudson.plugins.ec2.EC2RetentionStrategy._check(EC2RetentionStrategy.java:99)
    7. hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:71)
    8. hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:40)
    8 frames
  5. Hudson
    SafeTimerTask.run
    1. hudson.slaves.ComputerRetentionWork$1.run(ComputerRetentionWork.java:70)
    2. hudson.model.Queue._withLock(Queue.java:1286)
    3. hudson.model.Queue.withLock(Queue.java:1169)
    4. hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:61)
    5. hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)
    5 frames
  6. Java RT
    Thread.run
    1. java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    2. java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    3. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    4. java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    5. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    6. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    7. java.lang.Thread.run(Thread.java:745)
    7 frames