play.api.http.HttpErrorHandlerExceptions$$anon$1

tip

This might be an issue with the file location in the Spark submit command. Try it with "spark-submit --master spark://master:7077 hello_world_from_pyspark.py {file location}"

tip

Check if you've set a name in Application -> Run. If you didn't, the generated XML is gonna have missing information and then this exception will be thrown.

tip

Needs to add netty-transport-native-epoll with a OS classifier to the classpath, it's not included in netty-all

You have a different solution? A short tip here would help you and many other users who saw this issue last week.

  • Error in spark-submit but not in pyspark
    via Stack Overflow by nanounanue
    ,
    • play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[JobExecutionException: Cannot initialize task 'DataSink (collect())': Deserializing the OutputFormat (org.apache.flink.api.java.Utils$CollectHelper@1f158fd6) failed: Could not read the user code wrapper: io.fabrick.aperture.doctor.stream.WordCount$$anon$2$$anon$1]] at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:265)[play_2.11-2.4.2.jar:2.4.2] at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:191)[play_2.11-2.4.2.jar:2.4.2] at play.api.GlobalSettings$class.onError(GlobalSettings.scala:179)[play_2.11-2.4.2.jar:2.4.2] at play.api.DefaultGlobal$.onError(GlobalSettings.scala:212)[play_2.11-2.4.2.jar:2.4.2] at play.api.http.GlobalSettingsHttpErrorHandler.onServerError(HttpErrorHandler.scala:94)[play_2.11-2.4.2.jar:2.4.2] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$9$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:158)[play-netty-server_2.11-2.4.2.jar:2.4.2] at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$9$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:155)[play-netty-server_2.11-2.4.2.jar:2.4.2] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)[scala-library-2.11.7.jar:na] at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:216)[scala-library-2.11.7.jar:na] at scala.util.Try$.apply(Try.scala:192)[scala-library-2.11.7.jar:na] Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot initialize task 'DataSink (collect())': Deserializing the OutputFormat (org.apache.flink.api.java.Utils$CollectHelper@1f158fd6) failed: Could not read the user code wrapper: io.fabrick.aperture.doctor.stream.WordCount$$anon$2$$anon$1 at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$org$apache$flink$runtime$jobmanager$JobManager$$submitJob$7.apply(JobManager.scala:1012)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$org$apache$flink$runtime$jobmanager$JobManager$$submitJob$7.apply(JobManager.scala:996)[flink-runtime_2.11-1.0.0.jar:1.0.0] at scala.collection.Iterator$class.foreach(Iterator.scala:742)[scala-library-2.11.7.jar:na] at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)[scala-library-2.11.7.jar:na] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)[scala-library-2.11.7.jar:na] at scala.collection.AbstractIterable.foreach(Iterable.scala:54)[scala-library-2.11.7.jar:na] at org.apache.flink.runtime.jobmanager.JobManager.org$apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:996)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:380)[flink-runtime_2.11-1.0.0.jar:1.0.0] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)[scala-library-2.11.7.jar:na] at org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:36)[flink-runtime_2.11-1.0.0.jar:1.0.0] Caused by: java.lang.Exception: Deserializing the OutputFormat (org.apache.flink.api.java.Utils$CollectHelper@1f158fd6) failed: Could not read the user code wrapper: io.fabrick.aperture.doctor.stream.WordCount$$anon$2$$anon$1 at org.apache.flink.runtime.jobgraph.OutputFormatVertex.initializeOnMaster(OutputFormatVertex.java:63)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$org$apache$flink$runtime$jobmanager$JobManager$$submitJob$7.apply(JobManager.scala:1008)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$org$apache$flink$runtime$jobmanager$JobManager$$submitJob$7.apply(JobManager.scala:996)[flink-runtime_2.11-1.0.0.jar:1.0.0] at scala.collection.Iterator$class.foreach(Iterator.scala:742)[scala-library-2.11.7.jar:na] at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)[scala-library-2.11.7.jar:na] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)[scala-library-2.11.7.jar:na] at scala.collection.AbstractIterable.foreach(Iterable.scala:54)[scala-library-2.11.7.jar:na] at org.apache.flink.runtime.jobmanager.JobManager.org$apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:996)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:380)[flink-runtime_2.11-1.0.0.jar:1.0.0] at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)[scala-library-2.11.7.jar:na] Caused by: org.apache.flink.runtime.operators.util.CorruptConfigurationException: Could not read the user code wrapper: io.fabrick.aperture.doctor.stream.WordCount$$anon$2$$anon$1 at org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:284)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobgraph.OutputFormatVertex.initializeOnMaster(OutputFormatVertex.java:60)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$org$apache$flink$runtime$jobmanager$JobManager$$submitJob$7.apply(JobManager.scala:1008)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$org$apache$flink$runtime$jobmanager$JobManager$$submitJob$7.apply(JobManager.scala:996)[flink-runtime_2.11-1.0.0.jar:1.0.0] at scala.collection.Iterator$class.foreach(Iterator.scala:742)[scala-library-2.11.7.jar:na] at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)[scala-library-2.11.7.jar:na] at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)[scala-library-2.11.7.jar:na] at scala.collection.AbstractIterable.foreach(Iterable.scala:54)[scala-library-2.11.7.jar:na] at org.apache.flink.runtime.jobmanager.JobManager.org$apache$flink$runtime$jobmanager$JobManager$$submitJob(JobManager.scala:996)[flink-runtime_2.11-1.0.0.jar:1.0.0] at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:380)[flink-runtime_2.11-1.0.0.jar:1.0.0] Caused by: java.lang.ClassNotFoundException: io.fabrick.aperture.doctor.stream.WordCount$$anon$2$$anon$1 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)[na:1.8.0_60] at java.lang.ClassLoader.loadClass(ClassLoader.java:424)[na:1.8.0_60] at java.lang.ClassLoader.loadClass(ClassLoader.java:357)[na:1.8.0_60] at java.lang.Class.forName0(Native Method)[na:1.8.0_60] at java.lang.Class.forName(Class.java:348)[na:1.8.0_60] at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:64)[flink-core-1.0.0.jar:1.0.0] at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613)[na:1.8.0_60] at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)[na:1.8.0_60] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)[na:1.8.0_60] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)[na:1.8.0_60]

    Users with the same issue

    jokesterjokester
    1 times, last one,
    olle.hallinolle.hallin
    1 times, last one,
    poroszdporoszd
    1 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,
    1580 more bugmates