org.apache.hadoop.fs.FileAlreadyExistsException: Failed to rename hdfs://10.0.1.227:8020/gobblin/task-output/productA/2016 to /gobblinlandingzone/productA/2016: dst already exists


Solutions on the web

Solution icon of googlegroups
via Google Groups by Unknown author, 1 year ago
Failed to rename hdfs://10.0.1.227:8020/gobblin/task-output/productA/2016 to /gobblinlandingzone/productA/2016: dst already exists

Solution icon of googlegroups
via Google Groups by sborny, 6 months ago
Failed to rename hdfs://172.28.0.50:8020/user/gobblin/work/task-output/clicks.test6/2017 to /user/ubuntu/gobblin/gobblin-kafka/clicks.test6/2017: dst already exists

Solution icon of googlegroups
Failed to rename file:/home/lihui/gobblin-dist/data/work/task-output/testforhdfs/age=2 to /home/lihui/gobblin-dist/data/work/job-output/testforhdfs/age=2: dst already exists

Stack trace

org.apache.hadoop.fs.FileAlreadyExistsException: Failed to rename hdfs://10.0.1.227:8020/gobblin/task-output/productA/2016 
to /gobblinlandingzone/productA/2016: dst already exists
	at gobblin.util.HadoopUtils.renamePath(HadoopUtils.java:169)
	at gobblin.util.HadoopUtils.renamePath(HadoopUtils.java:152)
	at gobblin.util.HadoopUtils.movePath(HadoopUtils.java:213)
	at gobblin.util.ParallelRunner$6.call(ParallelRunner.java:312)
	at gobblin.util.ParallelRunner$6.call(ParallelRunner.java:305)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

Write tip

You have a different solution? A short tip here would help you and many other users who saw this issue last week.

Users with the same issue

You are the first who have seen this exception. Write a tip to help other users and build your expert profile.