org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server

Apache's JIRA Issue Tracker | Saisai Shao | 1 year ago
  1. 0

    [SPARK-9877] StandaloneRestServer NPE when submitting application - ASF JIRA

    apache.org | 1 year ago
    org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server
  2. 0

    [SPARK-9877] StandaloneRestServer NPE when submitting application - ASF JIRA

    apache.org | 11 months ago
    org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server
  3. 0

    When submitting a simple SparkPi application using standalone cluster mode: {code} ./bin/spark-submit --verbose --master spark://hw12100.local:6066 --deploy-mode cluster --class org.apache.spark.examples.SparkPi examples/target/scala-2.10/spark-examples-1.5.0-SNAPSHOT-hadoop2.6.0.jar {code} Client side throw such exceptions: {noformat} 15/08/12 17:14:16 INFO rest.RestSubmissionClient: Submitting a request to launch an application in spark://hw12100.local:6066. Exception in thread "main" org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server at org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:258) at org.apache.spark.deploy.rest.RestSubmissionClient.org$apache$spark$deploy$rest$RestSubmissionClient$$postJson(RestSubmissionClient.scala:219) at org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:84) at org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:80) at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108) at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771) at org.apache.spark.deploy.rest.RestSubmissionClient.createSubmission(RestSubmissionClient.scala:80) at org.apache.spark.deploy.rest.RestSubmissionClient$.run(RestSubmissionClient.scala:404) at org.apache.spark.deploy.rest.RestSubmissionClient$.main(RestSubmissionClient.scala:416) at org.apache.spark.deploy.rest.RestSubmissionClient.main(RestSubmissionClient.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:194) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null') at [Source: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 Server Error</title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /v1/submissions/create. Reason: <pre> Server Error</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> ; line: 1, column: 2] at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419) at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508) at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:437) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1462) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:683) at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3105) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3051) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2161) at org.json4s.jackson.JsonMethods$class.parse(JsonMethods.scala:19) at org.json4s.jackson.JsonMethods$.parse(JsonMethods.scala:44) at org.apache.spark.deploy.rest.SubmitRestProtocolMessage$.parseAction(SubmitRestProtocolMessage.scala:112) at org.apache.spark.deploy.rest.SubmitRestProtocolMessage$.fromJson(SubmitRestProtocolMessage.scala:130) at org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:241) ... 20 more {noformat} also following exception in the master side: {noformat} java.lang.NullPointerException at org.apache.spark.deploy.rest.StandaloneSubmitRequestServlet.handleSubmit(StandaloneRestServer.scala:177) at org.apache.spark.deploy.rest.SubmitRequestServlet.doPost(RestSubmissionServer.scala:258) at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) {noformat} Through investigating the code, {{masterEndpoint}} is null when creating {{StandaloneRestServer}} object, this object should be created after {{masterEndpoint}} is not null.

    Apache's JIRA Issue Tracker | 1 year ago | Saisai Shao
    org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    When submitting a simple SparkPi application using standalone cluster mode: {code} ./bin/spark-submit --verbose --master spark://hw12100.local:6066 --deploy-mode cluster --class org.apache.spark.examples.SparkPi examples/target/scala-2.10/spark-examples-1.5.0-SNAPSHOT-hadoop2.6.0.jar {code} Client side throw such exceptions: {noformat} 15/08/12 17:14:16 INFO rest.RestSubmissionClient: Submitting a request to launch an application in spark://hw12100.local:6066. Exception in thread "main" org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server at org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:258) at org.apache.spark.deploy.rest.RestSubmissionClient.org$apache$spark$deploy$rest$RestSubmissionClient$$postJson(RestSubmissionClient.scala:219) at org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:84) at org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:80) at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108) at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771) at org.apache.spark.deploy.rest.RestSubmissionClient.createSubmission(RestSubmissionClient.scala:80) at org.apache.spark.deploy.rest.RestSubmissionClient$.run(RestSubmissionClient.scala:404) at org.apache.spark.deploy.rest.RestSubmissionClient$.main(RestSubmissionClient.scala:416) at org.apache.spark.deploy.rest.RestSubmissionClient.main(RestSubmissionClient.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:194) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null') at [Source: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 Server Error</title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /v1/submissions/create. Reason: <pre> Server Error</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> ; line: 1, column: 2] at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419) at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508) at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:437) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1462) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:683) at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3105) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3051) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2161) at org.json4s.jackson.JsonMethods$class.parse(JsonMethods.scala:19) at org.json4s.jackson.JsonMethods$.parse(JsonMethods.scala:44) at org.apache.spark.deploy.rest.SubmitRestProtocolMessage$.parseAction(SubmitRestProtocolMessage.scala:112) at org.apache.spark.deploy.rest.SubmitRestProtocolMessage$.fromJson(SubmitRestProtocolMessage.scala:130) at org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:241) ... 20 more {noformat} also following exception in the master side: {noformat} java.lang.NullPointerException at org.apache.spark.deploy.rest.StandaloneSubmitRequestServlet.handleSubmit(StandaloneRestServer.scala:177) at org.apache.spark.deploy.rest.SubmitRequestServlet.doPost(RestSubmissionServer.scala:258) at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) {noformat} Through investigating the code, {{masterEndpoint}} is null when creating {{StandaloneRestServer}} object, this object should be created after {{masterEndpoint}} is not null.

    Apache's JIRA Issue Tracker | 1 year ago | Saisai Shao
    org.apache.spark.deploy.rest.SubmitRestProtocolException: Malformed response received from server
  6. 0

    Cannot create a Pact from a file that is generated by ConsumerPactBuilder

    GitHub | 2 years ago | pparth
    com.fasterxml.jackson.core.JsonParseException: Unexpected character ('/' (code 47)): maybe a (non-standard) comment? (not recognized as one since Feature 'ALLOW_COMMENTS' not enabled for parser) at [Source: /pathto/file.json; line: 1, column: 2]

  1. tyson925 1 times, last 4 months ago
  2. pradhul 3 times, last 9 months ago
  3. kid 1 times, last 1 month ago
  4. regisso 1 times, last 6 months ago
37 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. com.fasterxml.jackson.core.JsonParseException

    Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null') at [Source: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 Server Error</title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /v1/submissions/create. Reason: <pre> Server Error</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> ; line: 1, column: 2]

    at com.fasterxml.jackson.core.JsonParser._constructError()
  2. Jackson-core
    ReaderBasedJsonParser.nextToken
    1. com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419)
    2. com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
    3. com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:437)
    4. com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1462)
    5. com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:683)
    5 frames
  3. Jackson
    ObjectMapper.readValue
    1. com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3105)
    2. com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3051)
    3. com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2161)
    3 frames
  4. json4s-jackson
    JsonMethods$.parse
    1. org.json4s.jackson.JsonMethods$class.parse(JsonMethods.scala:19)
    2. org.json4s.jackson.JsonMethods$.parse(JsonMethods.scala:44)
    2 frames
  5. org.apache.spark
    RestSubmissionClient$$anonfun$createSubmission$3.apply
    1. org.apache.spark.deploy.rest.SubmitRestProtocolMessage$.parseAction(SubmitRestProtocolMessage.scala:112)
    2. org.apache.spark.deploy.rest.SubmitRestProtocolMessage$.fromJson(SubmitRestProtocolMessage.scala:130)
    3. org.apache.spark.deploy.rest.RestSubmissionClient.readResponse(RestSubmissionClient.scala:241)
    4. org.apache.spark.deploy.rest.RestSubmissionClient.org$apache$spark$deploy$rest$RestSubmissionClient$$postJson(RestSubmissionClient.scala:219)
    5. org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:84)
    6. org.apache.spark.deploy.rest.RestSubmissionClient$$anonfun$createSubmission$3.apply(RestSubmissionClient.scala:80)
    6 frames
  6. Scala
    TraversableLike$WithFilter.foreach
    1. scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
    2. scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    3. scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
    4. scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
    4 frames
  7. org.apache.spark
    RestSubmissionClient.main
    1. org.apache.spark.deploy.rest.RestSubmissionClient.createSubmission(RestSubmissionClient.scala:80)
    2. org.apache.spark.deploy.rest.RestSubmissionClient$.run(RestSubmissionClient.scala:404)
    3. org.apache.spark.deploy.rest.RestSubmissionClient$.main(RestSubmissionClient.scala:416)
    4. org.apache.spark.deploy.rest.RestSubmissionClient.main(RestSubmissionClient.scala)
    4 frames
  8. Java RT
    Method.invoke
    1. sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    2. sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    3. sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    4. java.lang.reflect.Method.invoke(Method.java:497)
    4 frames
  9. Spark
    SparkSubmit.main
    1. org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
    2. org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
    3. org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:194)
    4. org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
    5. org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    5 frames