org.apache.flink.client.program.ProgramInvocationException

tip

This might be an issue with the file location in the Spark submit command. Try it with "spark-submit --master spark://master:7077 hello_world_from_pyspark.py {file location}"

tip

Check if you've set a name in Application -> Run. If you didn't, the generated XML is gonna have missing information and then this exception will be thrown.

tip

Needs to add netty-transport-native-epoll with a OS classifier to the classpath, it's not included in netty-all

You have a different solution? A short tip here would help you and many other users who saw this issue last week.

  • Re: Output from Beam (on Flink) to Kafka
    via by William McCarthy,
  • Re: Output from Beam (on Flink) to Kafka
    via by Aljoscha Krettek,
  • Re: Output from Beam (on Flink) to Kafka
    via by Maximilian Michels,
  • Re: Output from Beam (on Flink) to Kafka
    via by William McCarthy,
  • Re: Output from Beam (on Flink) to Kafka
    via by William McCarthy,
  • Re: Output from Beam (on Flink) to Kafka
    via by Aljoscha Krettek,
  • Re: Output from Beam (on Flink) to Kafka
    via by Maximilian Michels,
  • Re: Output from Beam (on Flink) to Kafka
    via by Maximilian Michels,
  • Re: Output from Beam (on Flink) to Kafka
    via by Maximilian Michels,
  • Re: Output from Beam (on Flink) to Kafka
    via by William McCarthy,
    • org.apache.flink.client.program.ProgramInvocationException: The main method caused an error. at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:520) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:403) at org.apache.flink.client.program.Client.runBlocking(Client.java:248) at org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:866) at org.apache.flink.client.CliFrontend.run(CliFrontend.java:333) at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1189) at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1239) Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified 'FlinkPipelineRunner', supported pipeline runners [BlockingDataflowPipelineRunner, DataflowPipelineRunner, DirectPipelineRunner] at com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1486) at com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:101) at com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:286) at com.google.cloud.dataflow.contrib.kafka.examples.TopHashtagsExample.main(TopHashtagsExample.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:505) ... 6 more Caused by: java.lang.ClassNotFoundException: FlinkPipelineRunner at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1473) ... 14 more

    Users with the same issue

    Unknown visitor
    Unknown visitor1 times, last one,
    jokesterjokester
    2 times, last one,
    zoknizokni
    3 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,
    Unknown visitor
    Unknown visitor1 times, last one,
    1580 more bugmates