java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name.

JIRA | Earl Hathaway | 3 years ago
  1. 0

    say you load a dataset. it didn't load correctly because of a bad column name. I edited the file then reran the same R code to load the data. d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv') Now a number of operations emit this (exception below) edit: it gets worse. even if I reload the data and specify a different key, eg d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv', 'fml') the key is set properly, but operations such as > min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. create the below exception as well. To get exec2 to work again you have to bounce the whole cluster. + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:29:39.990 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex 04:29:40.230 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex max_column_display=2147483647 04:29:40.350 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.38 = bad_2k2.hex[,2000] 04:29:40.366 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) ================ new exception 04:33:21.503 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name.
  2. 0

    say you load a dataset. it didn't load correctly because of a bad column name. I edited the file then reran the same R code to load the data. d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv') Now a number of operations emit this (exception below) edit: it gets worse. even if I reload the data and specify a different key, eg d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv', 'fml') the key is set properly, but operations such as > min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. create the below exception as well. To get exec2 to work again you have to bounce the whole cluster. + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:29:39.990 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex 04:29:40.230 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex max_column_display=2147483647 04:29:40.350 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.38 = bad_2k2.hex[,2000] 04:29:40.366 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) ================ new exception 04:33:21.503 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name.
  3. 0

    it's not necessarily a good thing to do, but we should support it [load a dataset; call it d] d$col2000 <- min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: 2 better yet… once you've done this, stuff breaks everywhere > d$col2000 <- min(d$col2000) + d$col2000 Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. > head(d) Error in data.frame(col1 = 0.542521227151155, col2 = 0.267292166827247, : arguments imply differing number of rows: 1, 0 > 04:23:10.668 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.8 = bad_2k.hex[,2000] = c(0.000574313336983323) 04:23:10.747 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:10.748 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:51.514 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex 04:23:51.633 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex max_column_display=2147483647 04:23:51.753 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.9 = bad_2k.hex[,2000] 04:23:51.768 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    it's not necessarily a good thing to do, but we should support it [load a dataset; call it d] d$col2000 <- min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: 2 better yet… once you've done this, stuff breaks everywhere > d$col2000 <- min(d$col2000) + d$col2000 Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. > head(d) Error in data.frame(col1 = 0.542521227151155, col2 = 0.267292166827247, : arguments imply differing number of rows: 1, 0 > 04:23:10.668 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.8 = bad_2k.hex[,2000] = c(0.000574313336983323) 04:23:10.747 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:10.748 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:51.514 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex 04:23:51.633 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex max_column_display=2147483647 04:23:51.753 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.9 = bad_2k.hex[,2000] 04:23:51.768 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name.
  6. 0

    There is an error in unlocking the dataset after the previous model fails. The error in the previous model is a javaOutOfMemory and the following model is successfully created, despite the locked job error. The only issue is that that R unit test would fail overall. The test is too long to run, so the script for repo is attached. Output from logs: 05:22:21.748 # Session INFO HTTPD: POST /2/Exec2.json str=arcene.train.full = arcene.train.full 05:22:21.879 # Session ERRR WATER: env.remove_and_unlock() failed + java.lang.IllegalArgumentException: Dataset arcene.train.full is already in use. Unable to use it now. Consider using a different destination name. + at water.Lockable$PriorWriteLock.atomic(Lockable.java:85) + at water.Lockable$PriorWriteLock.atomic(Lockable.java:74) + at water.TAtomic.atomic(TAtomic.java:19) + at water.Atomic.compute2(Atomic.java:58) + at water.Atomic.fork(Atomic.java:42) + at water.Atomic.invoke(Atomic.java:34) + at water.Lockable.write_lock(Lockable.java:60) + at water.exec.Env.remove_and_unlock(Env.java:349) + at water.api.Exec2.serve(Exec2.java:71) + at water.api.Request.serveGrid(Request.java:165) + at water.Request2.superServeGrid(Request2.java:482) + at water.api.Exec2.serveGrid(Exec2.java:78) + at water.api.Request.serve(Request.java:142) + at water.api.RequestServer.serve(RequestServer.java:484) + at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:424) + at java.lang.Thread.run(Thread.java:745) Output from test: [2014-08-14 17:37:37] [ERROR] : Error: Test failed: 'Testing memory performance of Strong Rules' Not expected: http://127.0.0.1:54321/2/Exec2.json returned the following error: Frame is already locked by job null. 1: withWarnings(test(conn)) 2: withCallingHandlers(expr, warning = wHandler) 3: test(conn) 4: h2o.glm(x = c(1:7000), y = "arcene.train.label", data = arcene.train.full, family = "binomial", lambda_search = T, alpha = 1, nfolds = 0, use_all_factor_levels = T) 5: .h2o.get.glm(data@h2o, as.character(res$destination_key), return_all_lambda) 6: h2o.getFrame(h2o, pre$json$glm_model$dataKey) 7: .h2o.exec2(expr = key, h2o = h2o, dest_key = key) 8: .h2o.__exec2_dest_key(h2o, expr, dest_key) 9: .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) 10: stop(paste(myURL, " returned the following error:\n", .h2o.__formatError(res$error))) 11: .handleSimpleError(function (e) { e$calls <- head(sys.calls()[-seq_len(frame + 7)], -2) signalCondition(e) }, "http://127.0.0.1:54321/2/Exec2.json returned the following error:\n Frame is already locked by job null.\n", quote(.h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr))).

    JIRA | 2 years ago | Ariel Rao
    java.lang.IllegalArgumentException: Dataset arcene.train.full is already in use. Unable to use it now. Consider using a different destination name.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.IllegalArgumentException

      Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name.

      at water.Lockable$PriorWriteLock.atomic()
    2. water
      Lockable.write_lock
      1. water.Lockable$PriorWriteLock.atomic(Lockable.java:84)
      2. water.Lockable$PriorWriteLock.atomic(Lockable.java:73)
      3. water.TAtomic.atomic(TAtomic.java:19)
      4. water.Atomic.compute2(Atomic.java:57)
      5. water.Atomic.fork(Atomic.java:42)
      6. water.Atomic.invoke(Atomic.java:34)
      7. water.Lockable.write_lock(Lockable.java:59)
      7 frames
    3. water.exec
      Env.remove_and_unlock
      1. water.exec.Env.remove_and_unlock(Env.java:326)
      1 frame
    4. water.api
      Request.serveGrid
      1. water.api.Exec2.serve(Exec2.java:62)
      2. water.api.Request.serveGrid(Request.java:129)
      2 frames
    5. water
      Request2.superServeGrid
      1. water.Request2.superServeGrid(Request2.java:472)
      1 frame
    6. water.api
      RequestServer.serve
      1. water.api.Exec2.serveGrid(Exec2.java:71)
      2. water.api.Request.serve(Request.java:108)
      3. water.api.RequestServer.serve(RequestServer.java:315)
      3 frames
    7. water
      NanoHTTPD$HTTPSession.run
      1. water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421)
      1 frame
    8. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:724)
      1 frame