org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 174, in main process() File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 169, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 2407, in pipeline_func return func(split, prev_func(split, iterator)) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 2407, in pipeline_func return func(split, prev_func(split, iterator)) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 2407, in pipeline_func return func(split, prev_func(split, iterator)) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 346, in func return f(iterator) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 1041, in <lambda> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "/home/ubuntu/spark/python/pyspark/rdd.py", line 1041, in <genexpr> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "<stdin>", line 9, in <lambda> TypeError: unorderable types: NoneType() < str()

Searched on Google with the first line of a JAVA stack trace?

We can recommend more relevant solutions and speed up debugging when you paste your entire stack trace with the exception message. Try a sample exception.

Recommended solutions based on your search

Samebug tips

Do you know how to solve this issue? Write a tip to help other users and build your expert profile.

Solutions on the web

via GitHub by md6nguyen
, 9 months ago
func return f(iterator) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 1041, in <lambda> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "/home/ubuntu/spark/python/pyspark/rdd.py", line 1041, in <genexpr
via GitHub by jokereactive
, 8 months ago
Traceback (most recent call last): File "/home/spark/spark/python/lib/pyspark.zip/pyspark/worker.py", line 172, in main process() File "/home/spark/spark/python/lib/pyspark.zip/pyspark/worker.py", line 167, in process
via GitHub by ssallys
, 1 year ago
167, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/usr/local/spark-2.0.0-bin-hadoop2.7/python/pyspark/rdd.py", line 2371, in pipeline_func return func(split, prev_func(split, iterator)) File "/usr/local
via Stack Overflow by Joshua Holbrook
, 5 months ago
/worker.py", line 169, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/Users/myuser/dev/mycompany/myproject/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2408, in pipeline_func File "/Users/myuser/dev/mycompany
via Stack Overflow by thestackexchangeguy
, 3 months ago
Traceback (most recent call last): File "D:\Spark\python\lib\pyspark.zip\pyspark\worker.py", line 177, in main File "D:\Spark\python\lib\pyspark.zip\pyspark\worker.py", line 172, in process File "C:\Program Files\Anaconda3\lib\site-packages
via Stack Overflow by rajman
, 11 months ago
Traceback (most recent call last): File "/usr/share/dse/spark/python/lib/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/usr/share/dse/spark/python/lib/pyspark.zip/pyspark/worker.py", line 106, in process
org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 174, in main process() File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 169, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 2407, in pipeline_func return func(split, prev_func(split, iterator)) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 2407, in pipeline_func return func(split, prev_func(split, iterator)) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 2407, in pipeline_func return func(split, prev_func(split, iterator)) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 346, in func return f(iterator) File "/home/ubuntu/spark/python/pyspark/rdd.py", line 1041, in <lambda> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "/home/ubuntu/spark/python/pyspark/rdd.py", line 1041, in <genexpr> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "<stdin>", line 9, in <lambda> TypeError: unorderable types: NoneType() < str()
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRDD.scala:193)
at org.apache.spark.api.python.PythonRunner$$anon$1.(PythonRDD.scala:234)
at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:152)
at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Users with the same issue

Samebug visitor profile picture
Unknown user
Once, 1 week ago

Write tip

Know the solutions? Share your knowledge to help other developers to debug faster.