java.util.concurrent.RejectedExecutionException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • In one of our tests, we've seen this exception during server shutdown: {code} 2015-06-05 10:34:44,387 ERROR [org.xnio.listener] (XNIO-1 I/O-2) XNIO001007: A channel event listener threw an exception: java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@5e7925db rejected from org.xnio.XnioWorker$TaskPool@3d00aa37[Shutting down, pool size = 7, active threads = 0, queued tasks = 0, completed tasks = 52] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at org.xnio.XnioWorker.execute(XnioWorker.java:741) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54) ... {code} This is a very basic test that only starts the server and then shuts it down using {{:shutdown}}. After looking into this for a while, I believe that this is caused by the core management server ({{UndertowHttpManagementService}}) shutting down an XNIO worker (= thread pool) while the {{:shutdown}} management operation is still running (or, in fact, finishing, trying to close the network connection). I have a Byteman-based reproducer that inserts artifical pauses to certain well-defined places. I'm not sure if this has some connection to the graceful shutdown system, but I believe that even if it does, something like this shouldn't happen. Steps to reproduce: # {{./bin/standalone.sh -c standalone-full-ha.xml}} and wait until it starts completely # {{jps -v | grep "\-D\\[Standalone\\]"}} to figure out the PID of the newly started server # {{bminstall.sh -b -Dorg.jboss.byteman.transform.all $PID}} # {{bmsubmit.sh reproducer.btm}}, where {{reproducer.btm}} is a Byteman script reproduced below # {{./bin/jboss-cli.sh -c}} # {{:read-resource}} repeat few times # {{:shutdown(timeout=1)}} (or plain {{:shutdown}}) The Byteman script: {code} RULE XnioWorker.TaskPool/ThreadPoolExecutor shutdown CLASS java.util.concurrent.ThreadPoolExecutor METHOD shutdown() AFTER INVOKE advanceRunState IF TRUE DO Thread.sleep(10000) ENDRULE RULE Remoting onClose handler CLASS org.jboss.remoting3.remote.RemoteReadListener$1 METHOD handleEvent(java.nio.channels.Channel) AT ENTRY IF TRUE DO Thread.sleep(5000) ENDRULE {code}
    via by Ladislav Thon,
  • GitHub comment 7985#137879121
    via GitHub by wildfly-ci
    ,
  • JsseSslStreamSourceConduit returns -1 before underlying socket returns -1 Resulting in {code} java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@13e2ee8 rejected from org.xnio.XnioWorker$TaskPool@1f9d2ff[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 482] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at org.xnio.XnioWorker.execute(XnioWorker.java:584) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:813) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:123) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:813) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:123) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.nio.AbstractNioStreamChannel.invokeCloseHandler(AbstractNioStreamChannel.java:384) at org.xnio.nio.NioTcpChannel.close(NioTcpChannel.java:163) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.nio.WorkerThread.run(WorkerThread.java:154) {code}
    via by Brad Maxwell,
  • Exception {code} ERROR [org.xnio.listener] (XNIO-2 I/O-1) XNIO001007: A channel event listener threw an exception: java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@2d696007 rejected from org.xnio.XnioWorker$TaskPool@74a153ee[Shutting down, pool size = 9, active threads = 0, queued tasks = 0, completed tasks = 50] {code}, details at [1] could be hit when stopping server. This happens intermediately. I run transaction integration tests on WildFly/EAP7 DR builds. My tests works in way of doing some settings via CLI, deploy application, do the test, undeploy application, stop the server. During the stopping the server this exception is shown in log. Adding server log from my test for possible more details. [1] {code} INFO [org.jboss.as.server] (Thread-3) WFLYSRV0220: Server shutdown has been requested. INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-8) WFLYJCA0019: Stopped Driver service with driver-name = module_ojdbc7.jar INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-7) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS] INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:jboss/DefaultJMSConnectionFactory INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA] INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 77) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) WFLYJCA0019: Stopped Driver service with driver-name = h2 INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0019: Host default-host stopping ERROR [org.xnio.listener] (XNIO-2 I/O-1) XNIO001007: A channel event listener threw an exception: java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@2d696007 rejected from org.xnio.XnioWorker$TaskPool@74a153ee[Shutting down, pool size = 9, active threads = 0, queued tasks = 0, completed tasks = 50] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364) at org.xnio.XnioWorker.execute(XnioWorker.java:741) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:906) at org.xnio.IoUtils.safeClose(IoUtils.java:134) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:133) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.xnio.ChannelListeners$DelegatingChannelListener.handleEvent(ChannelListeners.java:1092) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.xnio.StreamConnection.invokeCloseListener(StreamConnection.java:80) at org.xnio.Connection.writeClosed(Connection.java:117) at org.xnio.nio.AbstractNioStreamConnection.writeClosed(AbstractNioStreamConnection.java:47) at org.xnio.nio.NioSocketConduit.terminateWrites(NioSocketConduit.java:181) at org.xnio.nio.NioSocketConduit.truncateWrites(NioSocketConduit.java:191) at org.xnio.conduits.ConduitStreamSinkChannel.close(ConduitStreamSinkChannel.java:186) at org.xnio.IoUtils.safeClose(IoUtils.java:134) at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.forceTermination(WriteReadyHandler.java:57) at org.xnio.nio.NioSocketConduit.forceTermination(NioSocketConduit.java:107) at org.xnio.nio.WorkerThread.run(WorkerThread.java:490) INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 71) AMQ151003: resource adaptor stopped INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 71) AMQ221002: Apache ActiveMQ Artemis Message Broker version 1.0.0 [8e39e2e6-1fc3-11e5-b5ab-adb038d9447d] stopped INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0008: Undertow HTTP listener default suspending INFO [com.arjuna.ats.jbossatx] (MSC service thread 1-3) ARJUNA032018: Destroying TransactionManagerService INFO [com.arjuna.ats.jbossatx] (MSC service thread 1-3) ARJUNA032014: Stopping transaction recovery manager DEBUG [com.arjuna.ats.arjuna] (Listener:4712) Recovery listener existing com.arjuna.ats.internal.arjuna.recovery.WorkerService DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-3) PeriodicRecovery: Mode <== TERMINATED DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-3) PeriodicRecovery: shutdown scan wait complete DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread exiting INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0007: Undertow HTTP listener default stopped, was bound to localhost/127.0.0.1:8080 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0004: Undertow 1.3.0.Beta2 stopping INFO [org.jboss.as] (MSC service thread 1-7) WFLYSRV0050: EAP 7.0.0.Alpha1 (WildFly Core 2.0.0.Alpha5) stopped in 237ms {code}
    via by Ondřej Chaloupka,
  • Exception {code} ERROR [org.xnio.listener] (XNIO-2 I/O-1) XNIO001007: A channel event listener threw an exception: java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@2d696007 rejected from org.xnio.XnioWorker$TaskPool@74a153ee[Shutting down, pool size = 9, active threads = 0, queued tasks = 0, completed tasks = 50] {code}, details at [1] could be hit when stopping server. This happens intermediately. I run transaction integration tests on WildFly/EAP7 DR builds. My tests works in way of doing some settings via CLI, deploy application, do the test, undeploy application, stop the server. During the stopping the server this exception is shown in log. Adding server log from my test for possible more details. [1] {code} INFO [org.jboss.as.server] (Thread-3) WFLYSRV0220: Server shutdown has been requested. INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-8) WFLYJCA0019: Stopped Driver service with driver-name = module_ojdbc7.jar INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-7) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS] INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:jboss/DefaultJMSConnectionFactory INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/JmsXA] INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 77) WFLYMSGAMQ0006: Unbound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) WFLYJCA0019: Stopped Driver service with driver-name = h2 INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0019: Host default-host stopping ERROR [org.xnio.listener] (XNIO-2 I/O-1) XNIO001007: A channel event listener threw an exception: java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@2d696007 rejected from org.xnio.XnioWorker$TaskPool@74a153ee[Shutting down, pool size = 9, active threads = 0, queued tasks = 0, completed tasks = 50] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364) at org.xnio.XnioWorker.execute(XnioWorker.java:741) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:906) at org.xnio.IoUtils.safeClose(IoUtils.java:134) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:133) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.xnio.ChannelListeners$DelegatingChannelListener.handleEvent(ChannelListeners.java:1092) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.xnio.StreamConnection.invokeCloseListener(StreamConnection.java:80) at org.xnio.Connection.writeClosed(Connection.java:117) at org.xnio.nio.AbstractNioStreamConnection.writeClosed(AbstractNioStreamConnection.java:47) at org.xnio.nio.NioSocketConduit.terminateWrites(NioSocketConduit.java:181) at org.xnio.nio.NioSocketConduit.truncateWrites(NioSocketConduit.java:191) at org.xnio.conduits.ConduitStreamSinkChannel.close(ConduitStreamSinkChannel.java:186) at org.xnio.IoUtils.safeClose(IoUtils.java:134) at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.forceTermination(WriteReadyHandler.java:57) at org.xnio.nio.NioSocketConduit.forceTermination(NioSocketConduit.java:107) at org.xnio.nio.WorkerThread.run(WorkerThread.java:490) INFO [org.apache.activemq.artemis.ra] (ServerService Thread Pool -- 71) AMQ151003: resource adaptor stopped INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 71) AMQ221002: Apache ActiveMQ Artemis Message Broker version 1.0.0 [8e39e2e6-1fc3-11e5-b5ab-adb038d9447d] stopped INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0008: Undertow HTTP listener default suspending INFO [com.arjuna.ats.jbossatx] (MSC service thread 1-3) ARJUNA032018: Destroying TransactionManagerService INFO [com.arjuna.ats.jbossatx] (MSC service thread 1-3) ARJUNA032014: Stopping transaction recovery manager DEBUG [com.arjuna.ats.arjuna] (Listener:4712) Recovery listener existing com.arjuna.ats.internal.arjuna.recovery.WorkerService DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-3) PeriodicRecovery: Mode <== TERMINATED DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-3) PeriodicRecovery: shutdown scan wait complete DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread exiting INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0007: Undertow HTTP listener default stopped, was bound to localhost/127.0.0.1:8080 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0004: Undertow 1.3.0.Beta2 stopping INFO [org.jboss.as] (MSC service thread 1-7) WFLYSRV0050: EAP 7.0.0.Alpha1 (WildFly Core 2.0.0.Alpha5) stopped in 237ms {code}
    via by Ondra Chaloupka,
  • JsseSslStreamSourceConduit returns -1 before underlying socket returns -1 Resulting in {code} java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@13e2ee8 rejected from org.xnio.XnioWorker$TaskPool@1f9d2ff[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 482] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at org.xnio.XnioWorker.execute(XnioWorker.java:584) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:813) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:123) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:813) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:123) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.nio.AbstractNioStreamChannel.invokeCloseHandler(AbstractNioStreamChannel.java:384) at org.xnio.nio.NioTcpChannel.close(NioTcpChannel.java:163) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.nio.WorkerThread.run(WorkerThread.java:154) {code}
    via by Brad Maxwell,
  • JsseSslStreamSourceConduit returns -1 before underlying socket returns -1 Resulting in {code} java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@13e2ee8 rejected from org.xnio.XnioWorker$TaskPool@1f9d2ff[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 482] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at org.xnio.XnioWorker.execute(XnioWorker.java:584) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:813) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:123) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.channels.TranslatingSuspendableChannel.close(TranslatingSuspendableChannel.java:813) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.channels.TranslatingSuspendableChannel$3.handleEvent(TranslatingSuspendableChannel.java:123) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72) at org.xnio.nio.AbstractNioStreamChannel.invokeCloseHandler(AbstractNioStreamChannel.java:384) at org.xnio.nio.NioTcpChannel.close(NioTcpChannel.java:163) at org.xnio.IoUtils.safeClose(IoUtils.java:137) at org.xnio.nio.WorkerThread.run(WorkerThread.java:154) {code}
    via by Brad Maxwell,
    • java.util.concurrent.RejectedExecutionException: Task org.jboss.remoting3.remote.RemoteReadListener$1$1@5e7925db rejected from org.xnio.XnioWorker$TaskPool@3d00aa37[Shutting down, pool size = 7, active threads = 0, queued tasks = 0, completed tasks = 52] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at org.xnio.XnioWorker.execute(XnioWorker.java:741) at org.jboss.remoting3.remote.RemoteReadListener$1.handleEvent(RemoteReadListener.java:54)

    Users with the same issue

    Unknown visitor1 times, last one,
    rp
    1 times, last one,
    bandoca
    3 times, last one,
    johnxfly
    22 times, last one,
    tyson925
    44 times, last one,
    92 more bugmates