java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.ListenableFutureTask@44d138e5 rejected from java.util.concurrent.ThreadPoolExecutor@3ee65af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]

DataStax JIRA | Andy Tolbert | 1 year ago
  1. 0

    I noticed the following exception rarely being raised when closing a Cluster instance at times: {noformat} java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.ListenableFutureTask@44d138e5 rejected from java.util.concurrent.ThreadPoolExecutor@3ee65af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) at com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:480) at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:49) at com.datastax.driver.core.Cluster$Manager.triggerOnDown(Cluster.java:1726) at com.datastax.driver.core.Cluster$Manager.signalConnectionFailure(Cluster.java:2037) at com.datastax.driver.core.Connection.closeAsync(Connection.java:619) at com.datastax.driver.core.HostConnectionPool.discardAvailableConnections(HostConnectionPool.java:628) at com.datastax.driver.core.HostConnectionPool.closeAsync(HostConnectionPool.java:606) at com.datastax.driver.core.SessionManager.closeAsync(SessionManager.java:160) at com.datastax.driver.core.Cluster$Manager.close(Cluster.java:1576) at com.datastax.driver.core.Cluster$Manager.access$200(Cluster.java:1261) at com.datastax.driver.core.Cluster.closeAsync(Cluster.java:552) at com.datastax.driver.core.Cluster.close(Cluster.java:563) at com.datastax.driver.core.policies.DCAwareRoundRobinPolicyTest.should_not_use_remote_hosts_if_some_nodes_are_up_in_local_dc(DCAwareRoundRobinPolicyTest.java:149) {noformat} I think this is a small side effect of calling signalConnectionFailure being moved from {{Connection#defunct}} to {{Connection#closeAsync}} in [JAVA-966]. We should check in the triggerOn* methods like we do in the on* methods to see if the cluster is closed/closing before submitting to the executor. In theory there still might be a very thin window where the executor is shutdown between a condition check and submitting the Task.

    DataStax JIRA | 1 year ago | Andy Tolbert
    java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.ListenableFutureTask@44d138e5 rejected from java.util.concurrent.ThreadPoolExecutor@3ee65af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]
  2. 0

    I noticed the following exception rarely being raised when closing a Cluster instance at times: {noformat} java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.ListenableFutureTask@44d138e5 rejected from java.util.concurrent.ThreadPoolExecutor@3ee65af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) at com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:480) at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:49) at com.datastax.driver.core.Cluster$Manager.triggerOnDown(Cluster.java:1726) at com.datastax.driver.core.Cluster$Manager.signalConnectionFailure(Cluster.java:2037) at com.datastax.driver.core.Connection.closeAsync(Connection.java:619) at com.datastax.driver.core.HostConnectionPool.discardAvailableConnections(HostConnectionPool.java:628) at com.datastax.driver.core.HostConnectionPool.closeAsync(HostConnectionPool.java:606) at com.datastax.driver.core.SessionManager.closeAsync(SessionManager.java:160) at com.datastax.driver.core.Cluster$Manager.close(Cluster.java:1576) at com.datastax.driver.core.Cluster$Manager.access$200(Cluster.java:1261) at com.datastax.driver.core.Cluster.closeAsync(Cluster.java:552) at com.datastax.driver.core.Cluster.close(Cluster.java:563) at com.datastax.driver.core.policies.DCAwareRoundRobinPolicyTest.should_not_use_remote_hosts_if_some_nodes_are_up_in_local_dc(DCAwareRoundRobinPolicyTest.java:149) {noformat} I think this is a small side effect of calling signalConnectionFailure being moved from {{Connection#defunct}} to {{Connection#closeAsync}} in [JAVA-966]. We should check in the triggerOn* methods like we do in the on* methods to see if the cluster is closed/closing before submitting to the executor. In theory there still might be a very thin window where the executor is shutdown between a condition check and submitting the Task.

    DataStax JIRA | 1 year ago | Andy Tolbert
    java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.ListenableFutureTask@44d138e5 rejected from java.util.concurrent.ThreadPoolExecutor@3ee65af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]
  3. 0

    notebook throws errors that the shell doesn't

    Google Groups | 1 year ago | Marcin Jurek
    java.util.concurrent.RejectedExecutionException: Task com.google.common.util.concurrent.ListenableFutureTask@258259ab rejected from java.util.concurrent.ThreadPoolExecutor@7e05fb0b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 20]
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Test fails with SparkContext has been shutdown

    GitHub | 2 years ago | jamborta
    java.util.concurrent.RejectedExecutionException: Task org.apache.spark.executor.Executor$TaskRunner@656a0389 rejected from java.util.concurrent.ThreadPoolExecutor@130e4b63[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
  6. 0

    SparkContext shut down if runJob() returns a stream runtime object

    GitHub | 2 years ago | adrianfr
    java.util.concurrent.RejectedExecutionException: Task org.apache.spark.scheduler.TaskResultGetter$$anon$2@458eb02b rejected from java.util.concurrent.ThreadPoolExecutor@1acffaba[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4]

  1. tyson925 16 times, last 1 week ago
  2. Nikolay Rybak 1 times, last 1 month ago
  3. rlovtangen 109 times, last 2 months ago
  4. rp 2 times, last 2 months ago
  5. rp 1 times, last 5 months ago
4 more registered users
32 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.util.concurrent.RejectedExecutionException

    Task com.google.common.util.concurrent.ListenableFutureTask@44d138e5 rejected from java.util.concurrent.ThreadPoolExecutor@3ee65af[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]

    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution()
  2. Java RT
    ThreadPoolExecutor.execute
    1. java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
    2. java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
    3. java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
    3 frames
  3. Guava
    AbstractListeningExecutorService.submit
    1. com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:480)
    2. com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:49)
    2 frames
  4. DataStax Java Driver for Apache Cassandra - Core
    DCAwareRoundRobinPolicyTest.should_not_use_remote_hosts_if_some_nodes_are_up_in_local_dc
    1. com.datastax.driver.core.Cluster$Manager.triggerOnDown(Cluster.java:1726)
    2. com.datastax.driver.core.Cluster$Manager.signalConnectionFailure(Cluster.java:2037)
    3. com.datastax.driver.core.Connection.closeAsync(Connection.java:619)
    4. com.datastax.driver.core.HostConnectionPool.discardAvailableConnections(HostConnectionPool.java:628)
    5. com.datastax.driver.core.HostConnectionPool.closeAsync(HostConnectionPool.java:606)
    6. com.datastax.driver.core.SessionManager.closeAsync(SessionManager.java:160)
    7. com.datastax.driver.core.Cluster$Manager.close(Cluster.java:1576)
    8. com.datastax.driver.core.Cluster$Manager.access$200(Cluster.java:1261)
    9. com.datastax.driver.core.Cluster.closeAsync(Cluster.java:552)
    10. com.datastax.driver.core.Cluster.close(Cluster.java:563)
    11. com.datastax.driver.core.policies.DCAwareRoundRobinPolicyTest.should_not_use_remote_hosts_if_some_nodes_are_up_in_local_dc(DCAwareRoundRobinPolicyTest.java:149)
    11 frames