java.lang.AssertionError: execution path does not include the hadoop lib location null/lib/native

Apache's JIRA Issue Tracker | Mostafa Elhemali | 4 years ago
  1. 0

    In order to set the path for loading native libraries, MR relies on the default value of the mapreduce.admin.user.env configuration setting the LD_LIBRARY_PATH environment entry. There are two problems with this setting in Windows: a) LD_LIBRARY_PATH doesn't mean anything in Windows. b) It sets it using $HADOOP_COMMON_HOME, instead of %HADOOP_COMMON_HOME%. The default value here should be platform-dependent (use the PATH variable in Windows instead of LD_LIBRARY_PATH), or we should rely on another mechanism. The net effect is that in Windows unless this configuration is over-ridden MR jobs fail with this error: {code} 2013-05-29 13:51:41,049 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:393) at org.apache.hadoop.fs.FileUtil.canRead(FileUtil.java:928) at org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskChecker.java:177) at org.apache.hadoop.util.DiskChecker.checkDirAccess(DiskChecker.java:164) at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:98) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:288) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:431) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:164) at org.apache.hadoop.mapred.YarnChild.configureLocalDirs(YarnChild.java:235) at org.apache.hadoop.mapred.YarnChild.configureTask(YarnChild.java:294) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143) {code}

    Apache's JIRA Issue Tracker | 4 years ago | Mostafa Elhemali
    java.lang.AssertionError: execution path does not include the hadoop lib location null/lib/native
  2. 0

    In order to set the path for loading native libraries, MR relies on the default value of the mapreduce.admin.user.env configuration setting the LD_LIBRARY_PATH environment entry. There are two problems with this setting in Windows: a) LD_LIBRARY_PATH doesn't mean anything in Windows. b) It sets it using $HADOOP_COMMON_HOME, instead of %HADOOP_COMMON_HOME%. The default value here should be platform-dependent (use the PATH variable in Windows instead of LD_LIBRARY_PATH), or we should rely on another mechanism. The net effect is that in Windows unless this configuration is over-ridden MR jobs fail with this error: {code} 2013-05-29 13:51:41,049 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:393) at org.apache.hadoop.fs.FileUtil.canRead(FileUtil.java:928) at org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskChecker.java:177) at org.apache.hadoop.util.DiskChecker.checkDirAccess(DiskChecker.java:164) at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:98) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:288) at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:431) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:164) at org.apache.hadoop.mapred.YarnChild.configureLocalDirs(YarnChild.java:235) at org.apache.hadoop.mapred.YarnChild.configureTask(YarnChild.java:294) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143) {code}

    Apache's JIRA Issue Tracker | 4 years ago | Mostafa Elhemali
    java.lang.AssertionError: execution path does not include the hadoop lib location null/lib/native
  3. 0

    Hadoop-Mapreduce-trunk - Build # 2202 - Failure

    apache.org | 1 year ago
    java.lang.AssertionError: null
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    cannot build latest on master branch

    GitHub | 2 years ago | Ed42
    java.lang.AssertionError: latch was counted down
  6. 0

    race condition building reactor from source?

    GitHub | 2 years ago | Ed42
    java.lang.AssertionError: latch was counted down

  1. michallos 34 times, last 6 days ago
  2. andyglick 2 times, last 3 months ago
  3. WoodenDoors 2 times, last 9 months ago
10 unregistered visitors
Not finding the right solution?
Take a tour to get the most out of Samebug.

Tired of useless tips?

Automated exception search integrated into your IDE

Root Cause Analysis

  1. java.lang.AssertionError

    execution path does not include the hadoop lib location null/lib/native

    at org.junit.Assert.fail()
  2. JUnit
    Assert.assertTrue
    1. org.junit.Assert.fail(Assert.java:93)
    2. org.junit.Assert.assertTrue(Assert.java:43)
    2 frames
  3. Hadoop
    TestMiniMRChildTask.testMapRedExecutionEnv
    1. org.apache.hadoop.mapred.TestMiniMRChildTask.testMapRedExecutionEnv(TestMiniMRChildTask.java:396)
    1 frame