java.security.PrivilegedActionException

java.io.FileNotFoundException: File does not exist: /spark-history/application_1463473041784_0002

Solutions on the web204

  • via GitHub by henryon
    , 11 months ago
    java.io.FileNotFoundException: File does not exist: /spark-history/application_1463473041784_0002
  • via GitHub by ljank
    , 1 year ago
    java.io.FileNotFoundException: File /user/spark/applicationHistory/application_1464847406968_512780 does not exist
  • java.io.FileNotFoundException: File does not exist: /user/spark/jobhistory/application_1460147926973_0091_1
  • Stack trace

    • java.security.PrivilegedActionException: java.io.FileNotFoundException: File does not exist: /spark-history/application_1463473041784_0002 at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:356) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637) at com.linkedin.drelephant.security.HadoopSecurity.doAs(HadoopSecurity.java:99) at org.apache.spark.deploy.history.SparkFSFetcher.fetchData(SparkFSFetcher.scala:189) at org.apache.spark.deploy.history.SparkFSFetcher.fetchData(SparkFSFetcher.scala:55) at com.linkedin.drelephant.analysis.AnalyticJob.getAnalysis(AnalyticJob.java:231) at com.linkedin.drelephant.ElephantRunner$ExecutorJob.run(ElephantRunner.java:180) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: File does not exist: /spark-history/application_1463473041784_0002 at sun.reflect.GeneratedConstructorAccessor15.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toIOException(WebHdfsFileSystem.java:424) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$600(WebHdfsFileSystem.java:91) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.shouldRetry(WebHdfsFileSystem.java:695) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:661) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:497) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:526) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:522) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHdfsFileStatus(WebHdfsFileSystem.java:877) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getFileStatus(WebHdfsFileSystem.java:892) at org.apache.spark.deploy.history.SparkFSFetcher.org$apache$spark$deploy$history$SparkFSFetcher$$shouldThrottle(SparkFSFetcher.scala:324) at org.apache.spark.deploy.history.SparkFSFetcher$$anon$1.run(SparkFSFetcher.scala:242) at org.apache.spark.deploy.history.SparkFSFetcher$$anon$1.run(SparkFSFetcher.scala:189) ... 13 more

    Write tip

    You have a different solution? A short tip here would help you and many other users who saw this issue last week.

    Users with the same issue

    10 times, 1 year ago