org.apache.hadoop.fs.FileAlreadyExistsException: Failed to rename hdfs://10.0.1.227:8020/gobblin/task-output/productA/2016 to /gobblinlandingzone/productA/2016: dst already exists

Google Groups | Unknown author | 1 year ago
tip
Click on the to mark the solution that helps you, Samebug will learn from it.
As a community member, you’ll be rewarded for you help.
  1. 0

    TimeBasedWriterPartitioner

    Google Groups | 1 year ago | Unknown author
    org.apache.hadoop.fs.FileAlreadyExistsException: Failed to rename hdfs://10.0.1.227:8020/gobblin/task-output/productA/2016 to /gobblinlandingzone/productA/2016: dst already exists
  2. 0

    Reading from Kafka only successful first run

    Google Groups | 1 month ago | sborny
    org.apache.hadoop.fs.FileAlreadyExistsException: Failed to rename hdfs://172.28.0.50:8020/user/gobblin/work/task-output/clicks.test6/2017 to /user/ubuntu/gobblin/gobblin-kafka/clicks.test6/2017: dst already exists

    Root Cause Analysis

    1. org.apache.hadoop.fs.FileAlreadyExistsException

      Failed to rename hdfs://10.0.1.227:8020/gobblin/task-output/productA/2016 to /gobblinlandingzone/productA/2016: dst already exists

      at gobblin.util.HadoopUtils.renamePath()
    2. gobblin.util
      ParallelRunner$6.call
      1. gobblin.util.HadoopUtils.renamePath(HadoopUtils.java:169)
      2. gobblin.util.HadoopUtils.renamePath(HadoopUtils.java:152)
      3. gobblin.util.HadoopUtils.movePath(HadoopUtils.java:213)
      4. gobblin.util.ParallelRunner$6.call(ParallelRunner.java:312)
      5. gobblin.util.ParallelRunner$6.call(ParallelRunner.java:305)
      5 frames
    3. Java RT
      Thread.run
      1. java.util.concurrent.FutureTask.run(FutureTask.java:262)
      2. java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      3. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      4. java.lang.Thread.run(Thread.java:745)
      4 frames