Pattern selector

Most relevant patterns first. Most helpful ones displayed. Click here to show all.

  1. Thread.run() has thrown a SparkException
    Java Runtime
    25
    15
    1
  2. Executor$TaskRunner.run() has thrown a SparkException
    Spark Project Core
    26
    15
    1

Your stack trace compared to the selected pattern

Jump to solutionsExpand frames
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 25.0 failed 1 times, most recent failure: Lost task 0.0 in stage 25.0 (TID 30, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/databricks/spark/python/pyspark/worker.py", line 111, in main process() File "/databricks/spark/python/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/databricks/spark/python/pyspark/serializers.py", line 263, in dump_stream vs = list(itertools.islice(iterator, batch)) File "/databricks/spark/python/pyspark/rdd.py", line 1295, in takeUpToNumLeft yield next(iterator) File "<ipython-input-46-4a4c467a0b3d>", line 13, in <lambda> IndexError: invalid index to scalar variable.
10 matching frames hidden
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

External results for this pattern (10)

  1. wcy405100via GitHub3 months ago
    Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 20, GPU1, executor 2): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/worker.py", line 174, in main process() File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/worker.py", line 169, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/rdd.py", line 2406, in pipeline_func return func(split, prev_func(split, iterator)) File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/rdd.py", line 2406, in pipeline_func return func(split, prev_func(split, iterator)) File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/rdd.py", line 2406, in pipeline_func return func(split, prev_func(split, iterator)) File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/rdd.py", line 345, in func return f(iterator) File "/opt/cloudera/parcels/SPARK2/lib/spark2/python/pyspark/rdd.py", line 793, in func r = f(it) File "/usr/lib/python2.7/site-packages/tensorflowonspark/TFSparkNode.py", line 433, in _train queue = mgr.get_queue(qname) File "/usr/lib64/python2.7/multiprocessing/managers.py", line 667, in temp token, exp = self._create(typeid, *args, **kwds) File "/usr/lib64/python2.7/multiprocessing/managers.py", line 567, in _create id, exposed = dispatch(conn, None, 'create', (typeid,)+args, kwds) File "/usr/lib64/python2.7/multiprocessing/managers.py", line 105, in dispatch raise convert_to_error(kind, result) RemoteError: --------------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.7/multiprocessing/managers.py", line 207, in handle_request result = func(c, *args, **kwds) File "/usr/lib64/python2.7/multiprocessing/managers.py", line 386, in create obj = callable(*args, **kwds) File "./tfspark.zip/tensorflowonspark/TFManager.py", line 34, in <lambda> TFManager.register('get_queue', callable=lambda qname: qdict[qname]) KeyError: 'input' ---------------------------------------------------------------------------
    Show stack trace
  2. sky-ghvia GitHub3 months ago
    Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 8, orange): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000005/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000005/pyspark.zip/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000001/pyspark.zip/pyspark/rdd.py", line 317, in func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522678317407_0003/container_1522678317407_0003_01_000001/pyspark.zip/pyspark/rdd.py", line 759, in func File "/home/parallel/sky/python2.7/lib/python2.7/site-packages/tensorflowonspark/TFSparkNode.py", line 372, in _train mgr = _get_manager(cluster_info, util.get_ip_address(), util.read_executor_id()) File "/home/parallel/sky/python2.7/lib/python2.7/site-packages/tensorflowonspark/util.py", line 37, in read_executor_id **with open("executor_id", "r") as f: IOError: [Errno 2] No such file or directory: 'executor_id'**
    Show stack trace
  3. deepak-2017via GitHub3 months ago
    Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 13, impetus-i0053.impetus.co.in, executor 1): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000002/pyspark.zip/pyspark/worker.py", line 177, in main process() File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000002/pyspark.zip/pyspark/worker.py", line 172, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 346, in func File "/hadoop/yarn/local/usercache/mapr/appcache/application_1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 794, in func File "/usr/lib/python2.7/site-packages/tensorflowonspark/TFSparkNode.py", line 394, in _train queue.put(item, block=True) AttributeError: 'AutoProxy[get_queue]' object has no attribute 'put'
    Show stack trace
  4. duhanminvia GitHub3 months ago
    Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "F:\tool\spark-1.6.1-bin-2.5.0-cdh5.3.6\python\lib\pyspark.zip\pyspark\worker.py", line 111, in main File "F:\tool\spark-1.6.1-bin-2.5.0-cdh5.3.6\python\lib\pyspark.zip\pyspark\worker.py", line 106, in process File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 317, in func return f(iterator) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 759, in func r = f(it) File "F:\tool\python35\lib\site-packages\tensorflowonspark\TFSparkNode.py", line 143, in _mapfn TFSparkNode.mgr = TFManager.start(authkey, ['control'], 'remote') File "F:\tool\Python35\lib\site-packages\tensorflowonspark\TFManager.py", line 52, in start mgr.start() File "F:\tool\Python35\lib\multiprocessing\managers.py", line 479, in start self._process.start() File "F:\tool\Python35\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "F:\tool\Python35\lib\multiprocessing\context.py", line 313, in _Popen return Popen(process_obj) File "F:\tool\Python35\lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__ reduction.dump(process_obj, to_child) File "F:\tool\Python35\lib\multiprocessing\reduction.py", line 59, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'start.<locals>.<lambda>'
    Show stack trace
  5. Job aborted due to stage failure: Task 0 in stage 105.0 failed 1 times, most recent failure: Lost task 0.0 in stage 105.0 (TID 1063, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/home/aziz/spark/python/lib/pyspark.zip/pyspark/worker.py", line 172, in main process() File "/home/aziz/spark/python/lib/pyspark.zip/pyspark/worker.py", line 167, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/home/aziz/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 263, in dump_stream vs = list(itertools.islice(iterator, batch)) File "/home/aziz/spark/python/pyspark/rdd.py", line 1306, in takeUpToNumLeft yield next(iterator) File "<ipython-input-123-32e41e450739>", line 3, in <lambda> File "/home/aziz/spark/python/lib/pyspark.zip/pyspark/mllib/linalg/distributed.py", line 313, in __init__ self.index = long(index) ValueError: invalid literal for int() with base 10: '006CCBB6-2304-4A52-8DAD-A88729FCC79F'
    Show stack trace
  6. Job aborted due to stage failure: Task 0 in stage 2.0 failed 4 times, most recent failure: Lost task 0.3 in stage 2.0 (TID 22, spark-w-0.c.clean-feat-131014.internal): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/usr/lib/spark/python/pyspark/worker.py", line 98, in main command = pickleSer._read_with_length(infile) File "/usr/lib/spark/python/pyspark/serializers.py", line 164, in _read_with_length return self.loads(obj) File "/usr/lib/spark/python/pyspark/serializers.py", line 422, in loads return pickle.loads(obj) ImportError: No module named nltk.tokenize
    Show stack trace
  7. Dongvia Stack Overflow4 months ago
    Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/home/worker/software/spark/python/lib/pyspark.zip/pyspark/worker.py", line 98, in main command = pickleSer._read_with_length(infile) File "/home/worker/software/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 164, in _read_with_length return self.loads(obj) File "/home/worker/software/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 422, in loads return pickle.loads(obj) TypeError: Required argument 'fileno' (pos 1) not found
    Show stack trace
  8. Job aborted due to stage failure: Task 1 in stage 63.0 failed 1 times, most recent failure: Lost task 1.0 in stage 63.0 (TID 745, localhost): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/usr/local/spark/python/lib/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/usr/local/spark/python/lib/pyspark.zip/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/usr/local/spark/python/lib/pyspark.zip/pyspark/serializers.py", line 263, in dump_stream vs = list(itertools.islice(iterator, batch)) File "<stdin>", line 1, in <lambda> File "/usr/local/spark/python/lib/pyspark.zip/pyspark/sql/types.py", line 1272, in __getattr__ raise AttributeError(item) AttributeError: lower
    Show stack trace
  9. msharkyvia Stack Overflow4 months ago
    Job aborted due to stage failure: Task 0 in stage 134.0 failed 4 times, most recent failure: Lost task 0.3 in stage 134.0 (TID 557, 10.3.1.31, executor 1): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "~/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/worker.py", line 174, in main process() File "~/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/worker.py", line 169, in process serializer.dump_stream(func(split_index, iterator), outfile) File "~/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/serializers.py", line 268, in dump_stream vs = list(itertools.islice(iterator, batch)) File "~/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/rdd.py", line 1339, in takeUpToNumLeft yield next(iterator) File "<ipython-input-53-37fce322868d>", line 6, in <lambda> File "~/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/ml/linalg/__init__.py", line 790, in dense return DenseVector(elements) File "~/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/ml/linalg/__init__.py", line 275, in __init__ ar = np.array(ar, dtype=np.float64) ValueError: could not convert string to float: Latitude
    Show stack trace
  10. jezdezvia GitHub4 months ago
    Job aborted due to stage failure: Task 336 in stage 2.0 failed 4 times, most recent failure: Lost task 336.3 in stage 2.0 (TID 561, ip-172-31-0-58.us-west-2.compute.internal): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/mnt/yarn/usercache/hadoop/appcache/application_1494526327812_0001/container_1494526327812_0001_01_000004/pyspark.zip/pyspark/worker.py", line 172, in main process() File "/mnt/yarn/usercache/hadoop/appcache/application_1494526327812_0001/container_1494526327812_0001_01_000004/pyspark.zip/pyspark/worker.py", line 167, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/usr/lib/spark/python/pyspark/rdd.py", line 2371, in pipeline_func File "/usr/lib/spark/python/pyspark/rdd.py", line 2371, in pipeline_func File "/usr/lib/spark/python/pyspark/rdd.py", line 2371, in pipeline_func File "/usr/lib/spark/python/pyspark/rdd.py", line 2371, in pipeline_func File "/usr/lib/spark/python/pyspark/rdd.py", line 317, in func File "/usr/lib/spark/python/pyspark/rdd.py", line 748, in processPartition File "<ipython-input-9-0a90aa3474d5>", line 3, in calculate_total_sessions_length File "/mnt/yarn/usercache/hadoop/appcache/application_1494526327812_0001/container_1494526327812_0001_01_000004/pyspark.zip/pyspark/accumulators.py", line 162, in add self._value = self.accum_param.addInPlace(self._value, term) File "/mnt/yarn/usercache/hadoop/appcache/application_1494526327812_0001/container_1494526327812_0001_01_000004/pyspark.zip/pyspark/accumulators.py", line 212, in addInPlace value1 += value2 TypeError: unsupported operand type(s) for +=: 'int' and 'NoneType'
    Show stack trace