org.mule.module.launcher.MuleApplicationClassLoader

MuleSoft JIRA | Rodrigo Merino | 5 months ago
  1. 0

    On high concurrency scenarios, when a situation happens that causes all worker threads to thrown an exception. all those threads become unnecessary blocked and each blocks and waits to load the same class. See for instance, these thread dump fragments: {code} java.lang.Thread.State: BLOCKED (on object monitor) at org.mule.module.launcher.FineGrainedControlClassLoader.loadClass(FineGrainedControlClassLoader.java:76) - waiting to lock <0x00000000b3978eb8> (a org.mule.module.launcher.MuleApplicationClassLoader) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.mule.module.launcher.application.CompositeApplicationClassLoader.loadClass(CompositeApplicationClassLoader.java:60) at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640) at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361) at java.util.ResourceBundle.getBundle(ResourceBundle.java:1082) at org.mule.config.i18n.MessageFactory.getBundle(MessageFactory.java:229) at org.mule.config.i18n.MessageFactory.getString(MessageFactory.java:198) at org.mule.config.i18n.MessageFactory.createMessage(MessageFactory.java:113) at org.mule.config.i18n.MessageFactory.createMessage(MessageFactory.java:64) at org.mule.config.i18n.CoreMessages.messageIsOfType(CoreMessages.java:111) at org.mule.api.MessagingException.generateMessage(MessagingException.java:149) at org.mule.api.MessagingException.<init>(MessagingException.java:71) {code} {code} java.lang.Thread.State: RUNNABLE at java.util.zip.ZipCoder.getBytes(ZipCoder.java:80) at java.util.zip.ZipFile.getEntry(ZipFile.java:311) - locked <0x00000000b278d8c8> (a java.util.jar.JarFile) at java.util.jar.JarFile.getEntry(JarFile.java:240) at java.util.jar.JarFile.getJarEntry(JarFile.java:223) at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1005) at sun.misc.URLClassPath.getResource(URLClassPath.java:212) at java.net.URLClassLoader$1.run(URLClassLoader.java:365) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) - locked <0x00000000b255dd30> (a org.mule.module.reboot.MuleContainerSystemClassLoader) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.mule.module.launcher.FineGrainedControlClassLoader.findParentClass(FineGrainedControlClassLoader.java:164) at org.mule.module.launcher.FineGrainedControlClassLoader.loadClass(FineGrainedControlClassLoader.java:115) - locked <0x00000000b34dad48> (a org.mule.module.launcher.MuleSharedDomainClassLoader) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.mule.module.launcher.FineGrainedControlClassLoader.findParentClass(FineGrainedControlClassLoader.java:164) at org.mule.module.launcher.FineGrainedControlClassLoader.loadClass(FineGrainedControlClassLoader.java:115) - locked <0x00000000b3978eb8> (a org.mule.module.launcher.MuleApplicationClassLoader) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.mule.module.launcher.application.CompositeApplicationClassLoader.loadClass(CompositeApplicationClassLoader.java:60) at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640) at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361) at java.util.ResourceBundle.getBundle(ResourceBundle.java:1082) at org.mule.config.i18n.MessageFactory.getBundle(MessageFactory.java:229) at org.mule.config.i18n.MessageFactory.getString(MessageFactory.java:198) at org.mule.config.i18n.MessageFactory.createMessage(MessageFactory.java:113) at org.mule.config.i18n.MessageFactory.createMessage(MessageFactory.java:64) at org.mule.config.i18n.CoreMessages.messageIsOfType(CoreMessages.java:111) at org.mule.api.MessagingException.generateMessage(MessagingException.java:149) at org.mule.api.MessagingException.<init>(MessagingException.java:71) {code} The first fragments shows the state of the threads waiting for the second to finish loading the bundle class. There are 2 problems with this: * A resource bundle as a java class is being loaded, when only the .properties file is provided. The java class load will fail and fallback to the properties. * Since the lock is at the *loadClass* method, the next thread will attempt to load the class again.

    MuleSoft JIRA | 5 months ago | Rodrigo Merino
    org.mule.module.launcher.MuleApplicationClassLoader

    Root Cause Analysis

    1. org.mule.module.launcher.MuleApplicationClassLoader

      No message provided

      at java.lang.ClassLoader.loadClass()
    2. Java RT
      ClassLoader.loadClass
      1. java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      1 frame
    3. Mule 3 Launcher
      CompositeApplicationClassLoader.loadClass
      1. org.mule.module.launcher.application.CompositeApplicationClassLoader.loadClass(CompositeApplicationClassLoader.java:60)
      1 frame
    4. Java RT
      ResourceBundle.getBundle
      1. java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640)
      2. java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501)
      3. java.util.ResourceBundle.findBundle(ResourceBundle.java:1465)
      4. java.util.ResourceBundle.findBundle(ResourceBundle.java:1419)
      5. java.util.ResourceBundle.findBundle(ResourceBundle.java:1419)
      6. java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361)
      7. java.util.ResourceBundle.getBundle(ResourceBundle.java:1082)
      7 frames
    5. Mule Core
      MessagingException.<init>
      1. org.mule.config.i18n.MessageFactory.getBundle(MessageFactory.java:229)
      2. org.mule.config.i18n.MessageFactory.getString(MessageFactory.java:198)
      3. org.mule.config.i18n.MessageFactory.createMessage(MessageFactory.java:113)
      4. org.mule.config.i18n.MessageFactory.createMessage(MessageFactory.java:64)
      5. org.mule.config.i18n.CoreMessages.messageIsOfType(CoreMessages.java:111)
      6. org.mule.api.MessagingException.generateMessage(MessagingException.java:149)
      7. org.mule.api.MessagingException.<init>(MessagingException.java:71)
      7 frames