com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types

DataStax JIRA | Rafi Shamim | 6 months ago
  1. 0

    I turned this into a ticket after being suggested to do so in this [java-driver-user list post|https://groups.google.com/a/lists.datastax.com/forum/#!topic/java-driver-user/GIz1IWJDVNQ]. I ran into a problem where occasionally, under somewhat heavy load, most or all instances of my app would lose connections to nodes in the Cassandra cluster. After turning on debug logs, I found that it was because connections were getting defuncted: {noformat} c.d.driver.core.Host.STATES : Defuncting Connection[/172.23.31.8:9042-19, inFlight=0, closed=false] because: [/172.23.31.8:9042] Unexpected exception triggered (com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types) {noformat} Earlier in the logs, the InvalidTypeException comes from the query logger. It failed when trying to format a CustomType into a String. {noformat} c.d.driver.core.Connection : Connection[/172.23.31.8:9042-19, inFlight=0, closed=false] connection error com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types at com.datastax.driver.core.DataType$Custom.format(DataType.java:910) at com.datastax.driver.core.QueryLogger.parameterValueAsString(QueryLogger.java:753) at com.datastax.driver.core.QueryLogger.appendParameters(QueryLogger.java:723) at com.datastax.driver.core.QueryLogger.logQuery(QueryLogger.java:650) at com.datastax.driver.core.QueryLogger$ConstantThresholdQueryLogger.maybeLogSlowQuery(QueryLogger.java:269) at com.datastax.driver.core.QueryLogger$ConstantThresholdQueryLogger.maybeLogNormalOrSlowQuery(QueryLogger.java:260) at com.datastax.driver.core.QueryLogger.update(QueryLogger.java:623) at com.datastax.driver.core.Cluster$Manager.reportLatency(Cluster.java:1597) at ... {noformat} It turns out that what was happening was: - Under moderate/heavy load, the driver would try to log slow queries (I had slow query logging on, and normal query logging off). - The logging would fail with that exception, which would defunct the connection, and cause the request to fail. - The request would then get retried on another host and that would likely fail too, causing a feedback loop until the load went down enough for slow queries to stop getting logged. - At some point, during some of these incidents, all connections to Cassandra might have been closed simultaneously, meaning my app could not talk to the database at all until reconnection happened. My table schema is rather old, and it still uses the LexicalUUIDType, which is a CustomType. I can't easily switch to UUIDType since it is the partition key. For now, as a workaround, I turned off all query logging. Can this be fixed so that query logging can be used with CustomTypes? It seems like a bug that an issue with formatting a string for a log statement would cause connections to get dropped. Note that version 3.0 of the driver doesn't seem to have this issue, since it can format CustomTypes into a String without error. I think in version 2.1 the exception could get handled better. Or the {{DataType$Custom.format}} method could use {{Bytes.toHexString}} (as version 3.0 of the driver does) instead of throwing InvalidTypeException.

    DataStax JIRA | 6 months ago | Rafi Shamim
    com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types
  2. 0

    I turned this into a ticket after being suggested to do so in this [java-driver-user list post|https://groups.google.com/a/lists.datastax.com/forum/#!topic/java-driver-user/GIz1IWJDVNQ]. I ran into a problem where occasionally, under somewhat heavy load, most or all instances of my app would lose connections to nodes in the Cassandra cluster. After turning on debug logs, I found that it was because connections were getting defuncted: {noformat} c.d.driver.core.Host.STATES : Defuncting Connection[/172.23.31.8:9042-19, inFlight=0, closed=false] because: [/172.23.31.8:9042] Unexpected exception triggered (com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types) {noformat} Earlier in the logs, the InvalidTypeException comes from the query logger. It failed when trying to format a CustomType into a String. {noformat} c.d.driver.core.Connection : Connection[/172.23.31.8:9042-19, inFlight=0, closed=false] connection error com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types at com.datastax.driver.core.DataType$Custom.format(DataType.java:910) at com.datastax.driver.core.QueryLogger.parameterValueAsString(QueryLogger.java:753) at com.datastax.driver.core.QueryLogger.appendParameters(QueryLogger.java:723) at com.datastax.driver.core.QueryLogger.logQuery(QueryLogger.java:650) at com.datastax.driver.core.QueryLogger$ConstantThresholdQueryLogger.maybeLogSlowQuery(QueryLogger.java:269) at com.datastax.driver.core.QueryLogger$ConstantThresholdQueryLogger.maybeLogNormalOrSlowQuery(QueryLogger.java:260) at com.datastax.driver.core.QueryLogger.update(QueryLogger.java:623) at com.datastax.driver.core.Cluster$Manager.reportLatency(Cluster.java:1597) at ... {noformat} It turns out that what was happening was: - Under moderate/heavy load, the driver would try to log slow queries (I had slow query logging on, and normal query logging off). - The logging would fail with that exception, which would defunct the connection, and cause the request to fail. - The request would then get retried on another host and that would likely fail too, causing a feedback loop until the load went down enough for slow queries to stop getting logged. - At some point, during some of these incidents, all connections to Cassandra might have been closed simultaneously, meaning my app could not talk to the database at all until reconnection happened. My table schema is rather old, and it still uses the LexicalUUIDType, which is a CustomType. I can't easily switch to UUIDType since it is the partition key. For now, as a workaround, I turned off all query logging. Can this be fixed so that query logging can be used with CustomTypes? It seems like a bug that an issue with formatting a string for a log statement would cause connections to get dropped. Note that version 3.0 of the driver doesn't seem to have this issue, since it can format CustomTypes into a String without error. I think in version 2.1 the exception could get handled better. Or the {{DataType$Custom.format}} method could use {{Bytes.toHexString}} (as version 3.0 of the driver does) instead of throwing InvalidTypeException.

    DataStax JIRA | 6 months ago | Rafi Shamim
    com.datastax.driver.core.exceptions.InvalidTypeException: Cannot format values of custom types
  3. 0

    I am receiving an InvalidTypeException when attempting to set a collection on a UDT. I am copying the example from the Datastax site: [http://www.datastax.com/documentation/developer/java-driver/2.1/java-driver/reference/udtApi.html] Setup: {code:java} session.execute("CREATE TYPE complex.phone (" + " alias text," + " number text);" ); session.execute("CREATE TYPE complex.address (" + " street text," + " zip_code int," + " phones list<frozen<phone>>);" // NOTE: In the example this is 'phones list<phone>' - but I receive the following error if I don't set this to frozen: InvalidQueryException: Non-frozen collections are not allowed inside collections: list<phone> ); session.execute("CREATE TABLE complex.users (" + " id uuid PRIMARY KEY," + " name text," + " addresses frozen <address>);"); {code} And then running the following as a test: {code:java} PreparedStatement insertUserPreparedStatement = session.prepare("INSERT INTO complex.users (id, name, addresses) VALUES (?, ?, ?);"); PreparedStatement selectUserPreparedStatement = session.prepare("SELECT * FROM complex.users WHERE id = ?;"); UserType addressUDT = session.getCluster().getMetadata().getKeyspace("complex").getUserType("address"); UserType phoneUDT = session.getCluster().getMetadata().getKeyspace("complex").getUserType("phone"); UDTValue phone1 = phoneUDT.newValue() .setString("alias", "home") .setString("number", "1-707-555-1234"); UDTValue phone2 = phoneUDT.newValue() .setString("alias", "work") .setString("number", "1-800-555-9876"); UDTValue address = addressUDT.newValue() .setString("street", "123 Arnold Drive") .setInt("zip_code", 95476) .setList("phones", ImmutableList.of(phone1, phone2)); // ERROR THROWN HERE Map<String, UDTValue> addresses = new HashMap<String, UDTValue>(); addresses.put("Work", address); UUID userId = UUID.fromString("fbdf82ed-0063-4796-9c7c-a3d4f47b4b25"); session.execute(insertUserPreparedStatement.bind(userId, "G. Binary", addresses)); {code} This throws the error: {code:java} Exception in thread "main" com.datastax.driver.core.exceptions.InvalidTypeException: Column phones is of type 'org.apache.cassandra.db.marshal.FrozenType(org.apache.cassandra.db.marshal.ListType(org.apache.cassandra.db.marshal.UserType(complex,70686f6e65,616c696173:org.apache.cassandra.db.marshal.UTF8Type,6e756d626572:org.apache.cassandra.db.marshal.UTF8Type)))', cannot set to a list at com.datastax.driver.core.AbstractData.setList(AbstractData.java:277) at com.datastax.driver.core.AbstractData.setList(AbstractData.java:297) at DatastaxUDTExample.loadUsers(DatastaxUDTExample.java:92) at DatastaxUDTExample.main(DatastaxUDTExample.java:57) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) {code}

    DataStax JIRA | 2 years ago | Nathan Scully
    com.datastax.driver.core.exceptions.InvalidTypeException: Column phones is of type 'org.apache.cassandra.db.marshal.FrozenType(org.apache.cassandra.db.marshal.ListType(org.apache.cassandra.db.marshal.UserType(complex,70686f6e65,616c696173:org.apache.cassandra.db.marshal.UTF8Type,6e756d626572:org.apache.cassandra.db.marshal.UTF8Type)))', cannot set to a list
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    Where clause does not work for Date type

    GitHub | 2 years ago | vsubrama001
    com.datastax.driver.core.exceptions.InvalidTypeException: Invalid type for value 0 of CQL type timestamp, expecting class java.util.Date but class java.lang.String provided

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. com.datastax.driver.core.exceptions.InvalidTypeException

      Cannot format values of custom types

      at com.datastax.driver.core.DataType$Custom.format()
    2. DataStax Java Driver for Apache Cassandra - Core
      Cluster$Manager.reportLatency
      1. com.datastax.driver.core.DataType$Custom.format(DataType.java:910)
      2. com.datastax.driver.core.QueryLogger.parameterValueAsString(QueryLogger.java:753)
      3. com.datastax.driver.core.QueryLogger.appendParameters(QueryLogger.java:723)
      4. com.datastax.driver.core.QueryLogger.logQuery(QueryLogger.java:650)
      5. com.datastax.driver.core.QueryLogger$ConstantThresholdQueryLogger.maybeLogSlowQuery(QueryLogger.java:269)
      6. com.datastax.driver.core.QueryLogger$ConstantThresholdQueryLogger.maybeLogNormalOrSlowQuery(QueryLogger.java:260)
      7. com.datastax.driver.core.QueryLogger.update(QueryLogger.java:623)
      8. com.datastax.driver.core.Cluster$Manager.reportLatency(Cluster.java:1597)
      8 frames