org.apache.spark.SparkException: Failed to execute user defined function($anonfun$1: (vector) => double)

Stack Overflow | Igor Kustov | 3 months ago
tip
Do you find the tips below useful? Click on the to mark them and say thanks to poroszd . Or join the community to write better ones.
  1. 0
    samebug tip
    You should use java.sql.Timestamp or Date to map BsonDateTime from mongodb.
  2. 0

    Issue in collect () in sparklyr Kmeans

    Stack Overflow | 2 months ago | Praveen
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 98.0 failed 1 times, most recent failure: Lost task 0.0 in stage 98.0 (TID 338, localhost): org.apache.spark.SparkException: Failed to execute user defined function($anonfun$3: (struct) => vector)
  3. 0

    MultilayerPerceptronClassifier in Spark. Layers and strange errors

    Stack Overflow | 3 months ago | Igor Kustov
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 68.0 failed 1 times, most recent failure: Lost task 0.0 in stage 68.0 (TID 68, localhost): org.apache.spark.SparkException: Failed to execute user defined function($anonfun$1: (vector) => double)
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Support reading DECIMAL(18,2) columns from Parquet

    GitHub | 4 months ago | ash211
    java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.IllegalArgumentException

    requirement failed: A & B Dimension mismatch!

    at scala.Predef$.require()
  2. Scala
    Predef$.require
    1. scala.Predef$.require(Predef.scala:224)
    1 frame
  3. org.apache.spark
    FeedForwardModel.predict
    1. org.apache.spark.ml.ann.BreezeUtil$.dgemm(BreezeUtil.scala:41)
    2. org.apache.spark.ml.ann.AffineLayerModel.eval(Layer.scala:164)
    3. org.apache.spark.ml.ann.FeedForwardModel.forward(Layer.scala:483)
    4. org.apache.spark.ml.ann.FeedForwardModel.predict(Layer.scala:530)
    4 frames
  4. Spark Project ML Library
    PredictionModel$$anonfun$1.apply
    1. org.apache.spark.ml.classification.MultilayerPerceptronClassificationModel.predict(MultilayerPerceptronClassifier.scala:322)
    2. org.apache.spark.ml.classification.MultilayerPerceptronClassificationModel.predict(MultilayerPerceptronClassifier.scala:296)
    3. org.apache.spark.ml.PredictionModel$$anonfun$1.apply(Predictor.scala:187)
    4. org.apache.spark.ml.PredictionModel$$anonfun$1.apply(Predictor.scala:186)
    4 frames
  5. Spark Project Catalyst
    GeneratedClass$GeneratedIterator.processNext
    1. org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)
    1 frame
  6. Spark Project SQL
    SparkPlan$$anonfun$4.apply
    1. org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
    2. org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:370)
    3. org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:246)
    4. org.apache.spark.sql.execution.SparkPlan$$anonfun$4.apply(SparkPlan.scala:240)
    4 frames
  7. Spark
    Executor$TaskRunner.run
    1. org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
    2. org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:803)
    3. org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
    4. org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
    5. org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
    6. org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
    7. org.apache.spark.scheduler.Task.run(Task.scala:86)
    8. org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
    8 frames
  8. Java RT
    Thread.run
    1. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    2. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    3. java.lang.Thread.run(Thread.java:745)
    3 frames