org.apache.spark.SparkException: Checkpoint RDD ReliableCheckpointRDD[232] at aggregate at ALS.scala:1182(0) has different number of partitions from original RDD itemFactors-10 MapPartitionsRDD[230] at mapValues at ALS.scala:1131(18)

Stack Overflow | Hsiang | 7 months ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    Spark MLlib error on trainimplicit

    Stack Overflow | 7 months ago | Hsiang
    org.apache.spark.SparkException: Checkpoint RDD ReliableCheckpointRDD[232] at aggregate at ALS.scala:1182(0) has different number of partitions from original RDD itemFactors-10 MapPartitionsRDD[230] at mapValues at ALS.scala:1131(18)

    Root Cause Analysis

    1. org.apache.spark.SparkException

      Checkpoint RDD ReliableCheckpointRDD[232] at aggregate at ALS.scala:1182(0) has different number of partitions from original RDD itemFactors-10 MapPartitionsRDD[230] at mapValues at ALS.scala:1131(18)

      at org.apache.spark.rdd.ReliableRDDCheckpointData.doCheckpoint()
    2. Spark
      RDD$$anonfun$doCheckpoint$1.apply
      1. org.apache.spark.rdd.ReliableRDDCheckpointData.doCheckpoint(ReliableRDDCheckpointData.scala:73)
      2. org.apache.spark.rdd.RDDCheckpointData.checkpoint(RDDCheckpointData.scala:74)
      3. org.apache.spark.rdd.RDD$$anonfun$doCheckpoint$1.apply$mcV$sp(RDD.scala:1655)
      4. org.apache.spark.rdd.RDD$$anonfun$doCheckpoint$1.apply(RDD.scala:1652)
      4 frames