Searched on Google with the first line of a JAVA stack trace?

We can recommend more relevant solutions and speed up debugging when you paste your entire stack trace with the exception message. Try a sample exception.

Recommended solutions based on your search

Solutions on the web

via spark-user by Andrew Or, 1 year ago
self.serializer.dump_stream(self._batched(iterator), stream) File "/tmp/hadoop/yarn/local/usercache/root/filecache/23/spark-assembly-1.0.1.2.1.3.0-563-hadoop2.4.0.2.1.3.0-563.jar/pyspark/serializers.py", line 123, in dump_stream for obj in iterator
N2.AGT: org.apache.spark.api.python.PythonException: Traceback
 (most recent call last):
    File
 "/tmp/hadoop/yarn/local/usercache/root/filecache/23/spark-assembly-1.0.1.2.1.3.0-563-hadoop2.4.0.2.1.3.0-563.jar/pyspark/worker.py",
 line 77, in main
     serializer.dump_stream(func(split_index, iterator), outfile)
   File
 "/tmp/hadoop/yarn/local/usercache/root/filecache/23/spark-assembly-1.0.1.2.1.3.0-563-hadoop2.4.0.2.1.3.0-563.jar/pyspark/serializers.py",
 line 191, in dump_stream
     self.serializer.dump_stream(self._batched(iterator), stream)
   File
 "/tmp/hadoop/yarn/local/usercache/root/filecache/23/spark-assembly-1.0.1.2.1.3.0-563-hadoop2.4.0.2.1.3.0-563.jar/pyspark/serializers.py",
 line 123, in dump_stream
     for obj in iterator:
   File
 "/tmp/hadoop/yarn/local/usercache/root/filecache/23/spark-assembly-1.0.1.2.1.3.0-563-hadoop2.4.0.2.1.3.0-563.jar/pyspark/serializers.py",
 line 180, in _batched
     for item in iterator:
   File
 "/root/spark-1.0.1.2.1.3.0-563-bin-2.4.0.2.1.3.0-563/python/pyspark/rdd.py",
 line 612, in func
   File
 "/root/spark-1.0.1.2.1.3.0-563-bin-2.4.0.2.1.3.0-563/examples/src/main/python/pi.py",
 line 36, in f
 SystemError: unknown opcode
 org.apache.spark.api.python.PythonRDD$$anon$1.read(PythonRDD.scala:115)
 org.apache.spark.api.python.PythonRDD$$anon$1.<init>(PythonRDD.scala:145)
         org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:78)
         org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
         org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
 org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
         org.apache.spark.scheduler.Task.run(Task.scala:51)
 org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
         java.lang.Thread.run(Thread.java:744)
 Driver stacktrace:
 at org.apache.spark.scheduler.DAGScheduler.org
 $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1044)	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1028)	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1026)	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1026)	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:634)	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:634)	at scala.Option.foreach(Option.scala:236)	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:634)	at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1229)	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)	at akka.actor.ActorCell.invoke(ActorCell.scala:456)	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)	at akka.dispatch.Mailbox.run(Mailbox.scala:219)	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)