Pattern selector

Most relevant patterns first. Most helpful ones displayed. Click here to show all.

  1. Thread.run() has thrown a PythonException
    Java Runtime
    15
    21
    0
  2. ThreadPoolExecutor$Worker.run() has thrown a PythonException
    Java Runtime
    17
    21
    0

Your stack trace compared to the selected pattern

Jump to solutionsExpand frames
org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin- hadoop2.7\python\lib\pyspark.zip\pyspark\worker.py", line 177, in main File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin- hadoop2.7\python\lib\pyspark.zip\pyspark\worker.py", line 172, in process File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin- hadoop2.7\python\pyspark\rdd.py", line 2423, in pipeline_func return func(split, prev_func(split, iterator)) File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin-hadoop2.7\python\pyspark\rdd.py", line 2423, in pipeline_func return func(split, prev_func(split, iterator)) File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin-hadoop2.7\python\pyspark\rdd.py", line 2423, in pipeline_func return func(split, prev_func(split, iterator)) File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin-hadoop2.7\python\pyspark\rdd.py", line 346, in func return f(iterator) File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin-hadoop2.7\python\pyspark\rdd.py", line 1041, in <lambda> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin-hadoop2.7\python\pyspark\rdd.py", line 1041, in <genexpr> return self.mapPartitions(lambda i: [sum(1 for _ in i)]).sum() File "D:\spark-2.2.0-bin-hadoop2.7\spark-2.2.0-bin-hadoop2.7\python\pyspark\rdd.py", line 2053, in <lambda> return self.map(lambda x: (f(x), x)) File "D:<filePath>", line 15, in <lambda> map(lambda x: x[1]).sortBy(lambda x:x.request_tm).map(lambda x: x.sku_id) AttributeError: 'ResultIterable' object has no attribute 'request_tm'
10 matching frames hidden
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

External results for this pattern (10)

  1. leewyangvia GitHub3 weeks ago
    Traceback (most recent call last): File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000002/ pyspark.zip/pyspark/worker.py", line 177, in main process() File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000002/ pyspark.zip/pyspark/worker.py", line 172, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 346, in func File "/hadoop/yarn/local/usercache/mapr/appcache/application_ 1525710710928_0173/container_1525710710928_0173_01_000001/pyspark.zip/pyspark/rdd.py", line 794, in func File "/usr/lib/python2.7/site-packages/tensorflowonspark/TFSparkNode.py", line 394, in _train queue.put(item, block=True) AttributeError: 'AutoProxy[get_queue]' object has no attribute 'put'
    Show stack trace
  2. leewyangvia GitHub3 weeks ago
    Traceback (most recent call last): File "/data4/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000009/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/data4/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000009/pyspark.zip/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000001/pyspark.zip/pyspark/rdd.py", line 317, in func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000001/pyspark.zip/pyspark/rdd.py", line 759, in func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000001/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 310, in _train File "/data4/hadoop/yarn/local/usercache/amantrach/appcache/ application_1515444508016_3200860/container_e82_1515444508016_3200860_01_ 000009/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 100, in _get_manager logging.info("Connected to TFSparkNode.mgr on {0}, ppid={1}, state={2}".format(host, ppid, str(TFSparkNode.mgr.get('state')))) AttributeError: 'NoneType' object has no attribute 'get'
    Show stack trace
  3. sky-ghvia GitHub3 weeks ago
    Traceback (most recent call last): File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000003/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000003/pyspark.zip/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000001/pyspark.zip/pyspark/rdd.py", line 317, in func File "/home/parallel/sky/hadoop/tmp/nm-local-dir/usercache/parallel/appcache/application_1522720244417_0004/container_1522720244417_0004_01_000001/pyspark.zip/pyspark/rdd.py", line 759, in func File "/home/parallel/sky/python2.7/lib/python2.7/site-packages/tensorflowonspark/TFSparkNode.py", line 372, in _train mgr = _get_manager(cluster_info, util.get_ip_address(), util.read_executor_id()) File "/home/parallel/sky/python2.7/lib/python2.7/site-packages/tensorflowonspark/TFSparkNode.py", line 114, in _get_manager raise Exception(msg) Exception: No TFManager found on this node, please ensure that: 1. Spark num_executors matches TensorFlow cluster_size 2. Spark cores/tasks per executor is 1. 3. Spark dynamic allocation is disabled.
    Show stack trace
  4. zfh01234via GitHub3 months ago
    Traceback (most recent call last): File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/worker.py", line 172, in main process() File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/worker.py", line 167, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/rdd.py", line 2371, in pipeline_func File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/rdd.py", line 2371, in pipeline_func File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/rdd.py", line 2371, in pipeline_func File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/rdd.py", line 317, in func File "/app/spark-2.0.0/python/lib/pyspark.zip/pyspark/rdd.py", line 762, in func File "/app/run/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 433, in _train AttributeError: 'NoneType' object has no attribute 'get_queue'
    Show stack trace
  5. li7huivia GitHub3 months ago
    Traceback (most recent call last): File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/worker.py", line 177, in main process() File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/worker.py", line 172, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/rdd.py", line 2423, in pipeline_func File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/rdd.py", line 346, in func File "/root/spark-2.2.0-bin-hadoop2.6/python/lib/pyspark.zip/pyspark/rdd.py", line 794, in func File "/home/data2/nm-local-dirs/usercache/root/appcache/application_1527165162039_0004/container_1527165162039_0004_01_000001/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 136, in _mapfn File "/usr/lib/python2.7/site-packages/tensorflow/__init__.py", line 24, in <module> from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import File "/usr/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 63, in <module> from tensorflow.python.framework.framework_lib import * # pylint: disable=redefined-builtin File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/framework_lib.py", line 25, in <module> from tensorflow.python.framework.ops import Graph File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 55, in <module> from tensorflow.python.platform import app File "/usr/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 24, in <module> from tensorflow.python.platform import flags File "/usr/lib/python2.7/site-packages/tensorflow/python/platform/flags.py", line 25, in <module> from absl.flags import * # pylint: disable=wildcard-import ImportError: No module named absl.flags
    Show stack trace
  6. amantracvia GitHub3 months ago
    Traceback (most recent call last): File "/data4/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000009/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/data4/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000009/pyspark.zip/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000001/pyspark.zip/pyspark/rdd.py", line 2346, in pipeline_func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000001/pyspark.zip/pyspark/rdd.py", line 317, in func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000001/pyspark.zip/pyspark/rdd.py", line 759, in func File "/data3/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000001/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 310, in _train File "/data4/hadoop/yarn/local/usercache/amantrach/appcache/application_1515444508016_3200860/container_e82_1515444508016_3200860_01_000009/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 100, in _get_manager logging.info("Connected to TFSparkNode.mgr on {0}, ppid={1}, state={2}".format(host, ppid, str(TFSparkNode.mgr.get('state')))) AttributeError: 'NoneType' object has no attribute 'get'
    Show stack trace
  7. duhanminvia GitHub3 months ago
    Traceback (most recent call last): File "F:\tool\spark-1.6.1-bin-2.5.0-cdh5.3.6\python\lib\pyspark.zip\pyspark\worker.py", line 111, in main File "F:\tool\spark-1.6.1-bin-2.5.0-cdh5.3.6\python\lib\pyspark.zip\pyspark\worker.py", line 106, in process File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 2346, in pipeline_func return func(split, prev_func(split, iterator)) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 317, in func return f(iterator) File "F:\tool\python35\lib\site-packages\pyspark\rdd.py", line 759, in func r = f(it) File "F:\tool\python35\lib\site-packages\tensorflowonspark\TFSparkNode.py", line 143, in _mapfn TFSparkNode.mgr = TFManager.start(authkey, ['control'], 'remote') File "F:\tool\Python35\lib\site-packages\tensorflowonspark\TFManager.py", line 52, in start mgr.start() File "F:\tool\Python35\lib\multiprocessing\managers.py", line 479, in start self._process.start() File "F:\tool\Python35\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "F:\tool\Python35\lib\multiprocessing\context.py", line 313, in _Popen return Popen(process_obj) File "F:\tool\Python35\lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__ reduction.dump(process_obj, to_child) File "F:\tool\Python35\lib\multiprocessing\reduction.py", line 59, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'start.<locals>.<lambda>'
    Show stack trace
  8. Traceback (most recent call last): File "/usr/lib/spark/python/pyspark/worker.py", line 98, in main command = pickleSer._read_with_length(infile) File "/usr/lib/spark/python/pyspark/serializers.py", line 164, in _read_with_length return self.loads(obj) File "/usr/lib/spark/python/pyspark/serializers.py", line 422, in loads return pickle.loads(obj) ImportError: No module named nltk.tokenize
    Show stack trace
  9. mooperdvia GitHub4 months ago
    Traceback (most recent call last): File "/hadoop/yarn/local/usercache/centos/appcache/application_1480271222291_0048/container_1480271222291_0048_01_000020/pyspark.zip/pyspark/worker.py", line 172, in main process() File "/hadoop/yarn/local/usercache/centos/appcache/application_1480271222291_0048/container_1480271222291_0048_01_000020/pyspark.zip/pyspark/worker.py", line 167, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/hadoop/yarn/local/usercache/centos/appcache/application_1480271222291_0048/container_1480271222291_0048_01_000020/pyspark.zip/pyspark/serializers.py", line 263, in dump_stream vs = list(itertools.islice(iterator, batch)) File "/usr/hdp/2.5.0.0-1245/spark2/python/lib/pyspark.zip/pyspark/rdd.py", line 1306, in takeUpToNumLeft File "/home/centos/fun-functions/spark-parrallel-read-from-s3/tick.py", line 38, in distributedJsonRead File "/usr/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action response = action(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__ response = getattr(parent.meta.client, operation_name)(**params) File "/usr/lib/python2.7/site-packages/botocore/client.py", line 251, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/lib/python2.7/site-packages/botocore/client.py", line 526, in _make_api_call operation_model, request_dict) File "/usr/lib/python2.7/site-packages/botocore/endpoint.py", line 141, in make_request return self._send_request(request_dict, operation_model) File "/usr/lib/python2.7/site-packages/botocore/endpoint.py", line 166, in _send_request request = self.create_request(request_dict, operation_model) File "/usr/lib/python2.7/site-packages/botocore/endpoint.py", line 150, in create_request operation_name=operation_model.name) File "/usr/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit return self._emit(event_name, kwargs) File "/usr/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit response = handler(**kwargs) File "/usr/lib/python2.7/site-packages/botocore/signers.py", line 90, in handler return self.sign(operation_name, request) File "/usr/lib/python2.7/site-packages/botocore/signers.py", line 147, in sign auth.add_auth(request) File "/usr/lib/python2.7/site-packages/botocore/auth.py", line 678, in add_auth raise NoCredentialsError NoCredentialsError: Unable to locate credentials
    Show stack trace
  10. Traceback (most recent call last): File "D:\Spark\python\lib\pyspark.zip\pyspark\worker.py", line 177, in main File "D:\Spark\python\lib\pyspark.zip\pyspark\worker.py", line 172, in process File "C:\Program Files\Anaconda3\lib\site-packages\pyspark\rdd.py", line 2423, in pipeline_func return func(split, prev_func(split, iterator)) File "C:\Program Files\Anaconda3\lib\site-packages\pyspark\rdd.py", line 2423, in pipeline_func return func(split, prev_func(split, iterator)) File "C:\Program Files\Anaconda3\lib\site-packages\pyspark\rdd.py", line 2423, in pipeline_func return func(split, prev_func(split, iterator)) File "C:\Program Files\Anaconda3\lib\site-packages\pyspark\rdd.py", line 346, in func return f(iterator) File "C:\Program Files\Anaconda3\lib\site-packages\pyspark\rdd.py", line 794, in func r = f(it) File "C:\Program Files\Anaconda3\lib\site-packages\tensorflowonspark\TFSparkNode.py", line 290, in _mapfn TFSparkNode.mgr = TFManager.start(authkey, ['control'], 'remote') File "C:\Program Files\Anaconda3\lib\site-packages\tensorflowonspark\TFManager.py", line 41, in start mgr.start() File "C:\Program Files\Anaconda3\lib\multiprocessing\managers.py", line 513, in start self._process.start() File "C:\Program Files\Anaconda3\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\Program Files\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Program Files\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__ reduction.dump(process_obj, to_child) File "C:\Program Files\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'start.<locals>.<lambda>'
    Show stack trace