org.apache.spark.SparkException

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • GitHub comment 2#74824749
    via GitHub by frensjan
    ,
    • org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, ip-172-31-6-203.ec2.internal): org.apache.spark.api.python.PythonException: Traceback (most recent call last): File "/mnt1/yarn/usercache/hadoop/appcache/application_1445358635170_0009/container_1445358635170_0009_01_000003/pyspark.zip/pyspark/worker.py", line 111, in main process() File "/mnt1/yarn/usercache/hadoop/appcache/application_1445358635170_0009/container_1445358635170_0009_01_000003/pyspark.zip/pyspark/worker.py", line 106, in process serializer.dump_stream(func(split_index, iterator), outfile) File "/mnt1/yarn/usercache/hadoop/appcache/application_1445358635170_0009/container_1445358635170_0009_01_000003/pyspark.zip/pyspark/serializers.py", line 263, in dump_stream vs = list(itertools.islice(iterator, batch)) File "chunk.py", line 406, in uri_set_copy copy_to_workspace(uri_set.source_uri, uri_set.workspace_target) File "chunk.py", line 178, in copy_to_workspace with rasterio.open(source_uri, "r") as src: File "/usr/local/lib64/python2.7/site-packages/rasterio/__init__.py", line 118, in open s.start() File "rasterio/_base.pyx", line 67, in rasterio._base.DatasetReader.start (rasterio/_base.c:2460) File "rasterio/_err.pyx", line 67, in rasterio._err.GDALErrCtxManager.__exit__ (rasterio/_err.c:948) IOError: `/vsicurl/http://raster-foundry-kdeloach.s3.amazonaws.com/1-a583a814-bd2b-4003-888a-4f84b484d274.tif' does not exist in the file system, and is not recognised as a supported dataset name. at org.apache.spark.api.python.PythonRDD$$anon$1.read(PythonRDD.scala:138) at org.apache.spark.api.python.PythonRDD$$anon$1.<init>(PythonRDD.scala:179) at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:97) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69) at org.apache.spark.rdd.RDD.iterator(RDD.scala:242) at org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$3.apply(PythonRDD.scala:248) at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1772) at org.apache.spark.api.python.PythonRDD$WriterThread.run(PythonRDD.scala:208)
    No Bugmate found.