java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.163:54331; java.lang.AssertionError: null while mapping key $0000000080ed00000000$nfs://home/tomk/h2o-1.7.0.1066/../airlines_all.hex

JIRA | Tom Kraljevic | 3 years ago
tip
Do you know that we can give you better hits? Get more relevant results from Samebug’s stack trace search.
  1. 0

    What I did: running on 163. 16 nodes (on one host) with -Xmx10g import airlines_all.csv go to glm2 autoframe the source key 08:12:55.219 main INFO WATER: ----- H2O started ----- 08:12:55.221 main INFO WATER: Build git branch: master 08:12:55.221 main INFO WATER: Build git hash: 85e743cca1bb124399e18dc0359f6d8f221d2711 08:12:55.222 main INFO WATER: Build git describe: 85e743c 08:12:55.222 main INFO WATER: Build project version: 1.7.0.1066 08:12:55.222 main INFO WATER: Built by: 'jenkins' 08:12:55.222 main INFO WATER: Built on: 'Wed Oct 9 22:36:27 PDT 2013' 08:12:55.222 main INFO WATER: Java availableProcessors: 32 08:12:55.228 main INFO WATER: Java heap totalMemory: 1.92 gb 08:12:55.229 main INFO WATER: Java heap maxMemory: 8.89 gb 08:12:55.229 main INFO WATER: ICE root: '/home/tomk/ice_root' 08:12:55.262 main WARN WATER: Multiple local IPs detected: + /10.0.12.1 /10.0.1.1 /10.0.4.1 /192.168.1.163 + Attempting to determine correct address... + Using /192.168.1.163 08:12:55.298 main INFO WATER: Internal communication uses port: 54322 + Listening for HTTP and REST traffic on http://192.168.1.163:54321/ 08:12:55.336 main INFO WATER: H2O cloud name: 'tomtest' 08:12:55.336 main INFO WATER: (v1.7.0.1066) 'tomtest' on /192.168.1.163:54321, discovery address /239.243.12.228:61427 08:12:55.339 main INFO WATER: Cloud of size 1 formed [/192.168.1.163:54321] 08:12:55.591 main INFO WATER: Log dir: '/home/tomk/ice_root/h2ologs' 08:12:59.056 FJ-10-1 INFO WATER: Cloud of size 16 formed [/192.168.1.163:54321, /192.168.1.163:54323, /192.168.1.163:54325, /192.168.1.163:54327, /192.168.1.163:54329, /192.168.1.163:54331, /192.168.1.163:54333, /192.168.1.163:54335, /192.168.1.163:54337, /192.168.1.163:54339, /192.168.1.163:54341, /192.168.1.163:54343, /192.168.1.163:54345, /192.168.1.163:54347, /192.168.1.163:54349, /192.168.1.163:54351] 08:13:37.146 #:54325-0 INFO WATER: Start remote task#14 class water.parser.DParseTask from /192.168.1.163:54325 08:13:56.159 FJ-9-23 INFO WATER: Done remote task#14 class water.parser.DParseTask to /192.168.1.163:54325 08:14:36.887 # Session INFO WATER: Converting ValueArray to Frame: node(/192.168.1.163:54321) convNum(0) key(nfs://home/tomk/h2o-1.7.0.1066/../airlines_all.hex.autoframe)... java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.163:54331; java.lang.AssertionError: null while mapping key $0000000080ed00000000$nfs://home/tomk/h2o-1.7.0.1066/../airlines_all.hex 08:15:07.060 # Session INFO WATER: at water.Request2.set(Request2.java:422) 08:15:07.061 # Session INFO WATER: at water.api.RequestArguments$Argument.check(RequestArguments.java:539) 08:15:07.062 # Session INFO WATER: at water.api.RequestQueries.buildQuery(RequestQueries.java:148) 08:15:07.062 # Session INFO WATER: at water.api.Request.serve(Request.java:95) 08:15:07.063 # Session INFO WATER: at water.api.RequestServer.serve(RequestServer.java:220) 08:15:07.063 # Session INFO WATER: at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:391) 08:15:07.064 # Session INFO WATER: at java.lang.Thread.run(Thread.java:662) 08:15:07.094 # Session INFO WATER: Caused by: water.DException$DistributedException: from /192.168.1.163:54331; java.lang.AssertionError: null while mapping key $0000000080ed00000000$nfs://home/tomk/h2o-1.7.0.1066/../airlines_all.hex 08:15:07.120 # Session INFO WATER: at water.nbhm.NonBlockingHashMap.putIfMatchUnlocked(NonBlockingHashMap.java:369) 08:15:07.120 # Session INFO WATER: at water.H2O.putIfMatch(H2O.java:439) 08:15:07.120 # Session INFO WATER: at water.DKV.DputIfMatch(DKV.java:66) 08:15:07.121 # Session INFO WATER: at water.DKV.put(DKV.java:22) 08:15:07.121 # Session INFO WATER: at water.DKV.put(DKV.java:16) 08:15:07.121 # Session INFO WATER: at water.UKV.put(UKV.java:27) 08:15:07.121 # Session INFO WATER: at water.UKV.put(UKV.java:23) 08:15:07.122 # Session INFO WATER: at water.UKV.put(UKV.java:84) 08:15:07.122 # Session INFO WATER: at water.fvec.AppendableVec.closeChunk(AppendableVec.java:55) 08:15:07.122 # Session INFO WATER: at water.fvec.NewChunk.close(NewChunk.java:103) 08:15:07.122 # Session INFO WATER: at water.fvec.Chunk.close(Chunk.java:145) 08:15:07.122 # Session INFO WATER: at water.ValueArray$Converter.map(ValueArray.java:627) 08:15:07.123 # Session INFO WATER: at water.MRTask.lcompute(MRTask.java:66) 08:15:07.123 # Session INFO WATER: at water.DRemoteTask.compute2(DRemoteTask.java:75) 08:15:07.123 # Session INFO WATER: at water.MRTask.lcompute(MRTask.java:62) 08:15:07.123 # Session INFO WATER: at water.DRemoteTask.compute2(DRemoteTask.java:75) 08:15:07.123 # Session INFO WATER: at water.MRTask.lcompute(MRTask.java:62) 08:15:07.124 # Session INFO WATER: at water.DRemoteTask.compute2(DRemoteTask.java:75) 08:15:07.124 # Session INFO WATER: at water.MRTask.lcompute(MRTask.java:62) 08:15:07.124 # Session INFO WATER: at water.DRemoteTask.compute2(DRemoteTask.java:75) 08:15:07.124 # Session INFO WATER: at water.H2O$H2OCountedCompleter.compute(H2O.java:599) 08:15:07.124 # Session INFO WATER: at jsr166y.CountedCompleter.exec(CountedCompleter.java:429) 08:15:07.125 # Session INFO WATER: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) 08:15:07.125 # Session INFO WATER: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) 08:15:07.129 # Session INFO WATER: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) 08:15:07.129 # Session INFO WATER: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    JIRA | 3 years ago | Tom Kraljevic
    java.lang.RuntimeException: water.DException$DistributedException: from /192.168.1.163:54331; java.lang.AssertionError: null while mapping key $0000000080ed00000000$nfs://home/tomk/h2o-1.7.0.1066/../airlines_all.hex

    Root Cause Analysis

    1. water.DException$DistributedException

      from /192.168.1.163:54331; java.lang.AssertionError: null while mapping key $0000000080ed00000000$nfs://home/tomk/h2o-1.7.0.1066/../airlines_all.hex

      at water.nbhm.NonBlockingHashMap.putIfMatchUnlocked()
    2. water.nbhm
      NonBlockingHashMap.putIfMatchUnlocked
      1. water.nbhm.NonBlockingHashMap.putIfMatchUnlocked(NonBlockingHashMap.java:369)
      1 frame
    3. water
      UKV.put
      1. water.H2O.putIfMatch(H2O.java:439)
      2. water.DKV.DputIfMatch(DKV.java:66)
      3. water.DKV.put(DKV.java:22)
      4. water.DKV.put(DKV.java:16)
      5. water.UKV.put(UKV.java:27)
      6. water.UKV.put(UKV.java:23)
      7. water.UKV.put(UKV.java:84)
      7 frames
    4. water.fvec
      Chunk.close
      1. water.fvec.AppendableVec.closeChunk(AppendableVec.java:55)
      2. water.fvec.NewChunk.close(NewChunk.java:103)
      3. water.fvec.Chunk.close(Chunk.java:145)
      3 frames
    5. water
      H2O$H2OCountedCompleter.compute
      1. water.ValueArray$Converter.map(ValueArray.java:627)
      2. water.MRTask.lcompute(MRTask.java:66)
      3. water.DRemoteTask.compute2(DRemoteTask.java:75)
      4. water.MRTask.lcompute(MRTask.java:62)
      5. water.DRemoteTask.compute2(DRemoteTask.java:75)
      6. water.MRTask.lcompute(MRTask.java:62)
      7. water.DRemoteTask.compute2(DRemoteTask.java:75)
      8. water.MRTask.lcompute(MRTask.java:62)
      9. water.DRemoteTask.compute2(DRemoteTask.java:75)
      10. water.H2O$H2OCountedCompleter.compute(H2O.java:599)
      10 frames
    6. jsr166y
      ForkJoinWorkerThread.run
      1. jsr166y.CountedCompleter.exec(CountedCompleter.java:429)
      2. jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
      3. jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
      4. jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
      5. jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
      5 frames