java.io.NotSerializableException: com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1

DataStax JIRA | DOAN DuyHai | 1 year ago
  1. 0

    {code:java} val Keywords = Seq("love","hate",":-)",":)",":-(",":(") val sc = new SparkContext(conf) val batchDuration: Duration = Seconds(5) val ssc = new StreamingContext(sc, batchDuration) val stream: ReceiverInputDStream[Status] = TwitterUtils.createStream(ssc, Some(authorization), Nil, StorageLevel.MEMORY_ONLY_SER_2) stream.flatMap(_.getText.toLowerCase.split( """\s+""")) .filter(Keywords.contains(_)) .countByValueAndWindow(batchDuration, batchDuration) .transform((rdd, time) => rdd.map { case (keyword, count) => (keyword, count, now(time)) }) .repartitionByCassandraReplica(KEYSPACE,"stream_read") .transform(rdd => rdd.joinWithCassandraTable(KEYSPACE, "stream_read", AllColumns, SomeColumns("key"))) .map{case((keyword,count,date),row) => (row.getString("value"),count,date)} .saveToCassandra(KEYSPACE, TABLE, SomeColumns("keyword", "count", "interval")) {code} Exception: {code} 2015-06-19 11:18:40,442 [sparkDriver-akka.actor.default-dispatcher-4] ERROR akka.actor.OneForOneStrategy - com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1 java.io.NotSerializableException: com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1 at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183) {code} I had a similar NotSerializableException when using *joinWithCassandraTable()* so I switched to *transform(rdd => rdd.joinWithCassandraTable(...))* For the record, I'm using the connector version *1.3.0-M1*, I had the same issue with *1.2.1* and I guess the issue is still there for *1.4.0-M1*

    DataStax JIRA | 1 year ago | DOAN DuyHai
    java.io.NotSerializableException: com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1
  2. 0

    {code:java} val Keywords = Seq("love","hate",":-)",":)",":-(",":(") val sc = new SparkContext(conf) val batchDuration: Duration = Seconds(5) val ssc = new StreamingContext(sc, batchDuration) val stream: ReceiverInputDStream[Status] = TwitterUtils.createStream(ssc, Some(authorization), Nil, StorageLevel.MEMORY_ONLY_SER_2) stream.flatMap(_.getText.toLowerCase.split( """\s+""")) .filter(Keywords.contains(_)) .countByValueAndWindow(batchDuration, batchDuration) .transform((rdd, time) => rdd.map { case (keyword, count) => (keyword, count, now(time)) }) .repartitionByCassandraReplica(KEYSPACE,"stream_read") .transform(rdd => rdd.joinWithCassandraTable(KEYSPACE, "stream_read", AllColumns, SomeColumns("key"))) .map{case((keyword,count,date),row) => (row.getString("value"),count,date)} .saveToCassandra(KEYSPACE, TABLE, SomeColumns("keyword", "count", "interval")) {code} Exception: {code} 2015-06-19 11:18:40,442 [sparkDriver-akka.actor.default-dispatcher-4] ERROR akka.actor.OneForOneStrategy - com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1 java.io.NotSerializableException: com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1 at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183) {code} I had a similar NotSerializableException when using *joinWithCassandraTable()* so I switched to *transform(rdd => rdd.joinWithCassandraTable(...))* For the record, I'm using the connector version *1.3.0-M1*, I had the same issue with *1.2.1* and I guess the issue is still there for *1.4.0-M1*

    DataStax JIRA | 1 year ago | DOAN DuyHai
    java.io.NotSerializableException: com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1
  3. 0

    ADF11g app cluster runtime Not Serializable Exception

    Oracle Community | 6 years ago | 807463
    java.io.NotSerializableException: oracle.adf.view.rich.component.rich.input.RichInputText
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Not Serializable Exception - Group Layout

    Stack Overflow | 2 years ago | bSky
    java.io.NotSerializableException: javax.swing.GroupLayout
  6. 0

    java.io.NotSerializableException when implementing UserDetailsService

    Stack Overflow | 4 years ago | chaldaean
    java.io.NotSerializableException: com.samsung.diary.security.UserDetailsServiceImpl

    5 unregistered visitors
    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.io.NotSerializableException

      com.datastax.spark.connector.writer.DefaultRowWriter$$anon$1

      at java.io.ObjectOutputStream.writeObject0()
    2. Java RT
      ObjectOutputStream.writeObject0
      1. java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
      1 frame