java.lang.RuntimeException: Failed to apply upload policy

GitHub | HenryCaiHaiying | 6 months ago
  1. 0

    GitHub comment 232#235795657

    GitHub | 6 months ago | llamahunter
    java.lang.RuntimeException: Failed to apply upload policy
  2. 0

    GitHub comment 232#235802943

    GitHub | 6 months ago | HenryCaiHaiying
    java.lang.RuntimeException: Failed to apply upload policy
  3. 0

    GitHub comment 160#197607132

    GitHub | 10 months ago | skanda83
    java.lang.RuntimeException: Failed to apply upload policy
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    While trying to do a distcp from our native HDFS cluster to S3, we get the following error/stacktrace : We are using hadoop-aws.jar version 2.7.1. We are using aws-java-sdk.jar version 1.10.69 (we bumped this up from 2.7.4 because we were getting errors seen in HADOOP-12420 + other authentication errors). Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:287) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at org.apache.hadoop.tools.DistCp.configureOutputFormat(DistCp.java:333) at org.apache.hadoop.tools.DistCp.createJob(DistCp.java:237) at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:174) at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153) I dug into both classes com.amazonaws.services.s3.transfer.TransferManager & org.apache.hadoop.fs.s3a.S3AFileSystem. The only difference is S3AFileSystem created a ThreadPoolExecutor (which implements AbstractExecutorService which implements ExecutorService). I also checked on the classpath to make sure the version of the jars being picked up is what I expected. Help would be much appreciated.

    Apache's JIRA Issue Tracker | 9 months ago | Jenny Dong
    java.lang.NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V
  6. 0

    spark - redshift - s3: class path conflict

    Stack Overflow | 3 days ago | Karol Sudol
    java.lang.NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManager <init(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.NoSuchMethodError

    com.amazonaws.services.s3.transfer.TransferManager.<init>(Lcom/amazonaws/services/s3/AmazonS3;Ljava/util/concurrent/ThreadPoolExecutor;)V

    at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize()
  2. Apache Hadoop Amazon Web Services support
    S3AFileSystem.initialize
    1. org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:287)
    1 frame
  3. Hadoop
    FileSystem.get
    1. org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
    2. org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
    3. org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
    4. org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
    5. org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
    5 frames
  4. com.pinterest.secor
    HadoopS3UploadManager$1.run
    1. com.pinterest.secor.util.FileUtil.getFileSystem(FileUtil.java:87)
    2. com.pinterest.secor.util.FileUtil.moveToCloud(FileUtil.java:206)
    3. com.pinterest.secor.uploader.HadoopS3UploadManager$1.run(HadoopS3UploadManager.java:66)
    3 frames
  5. Java RT
    Thread.run
    1. java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    2. java.util.concurrent.FutureTask.run(FutureTask.java:266)
    3. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    4. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    5. java.lang.Thread.run(Thread.java:745)
    5 frames