java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name.

JIRA | Earl Hathaway | 3 years ago
  1. 0

    say you load a dataset. it didn't load correctly because of a bad column name. I edited the file then reran the same R code to load the data. d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv') Now a number of operations emit this (exception below) edit: it gets worse. even if I reload the data and specify a different key, eg d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv', 'fml') the key is set properly, but operations such as > min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. create the below exception as well. To get exec2 to work again you have to bounce the whole cluster. + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:29:39.990 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex 04:29:40.230 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex max_column_display=2147483647 04:29:40.350 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.38 = bad_2k2.hex[,2000] 04:29:40.366 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) ================ new exception 04:33:21.503 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name.
  2. 0

    say you load a dataset. it didn't load correctly because of a bad column name. I edited the file then reran the same R code to load the data. d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv') Now a number of operations emit this (exception below) edit: it gets worse. even if I reload the data and specify a different key, eg d <- h2o.importFile(l, '/Users/earl/work/bad.2k.csv', 'fml') the key is set properly, but operations such as > min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. create the below exception as well. To get exec2 to work again you have to bounce the whole cluster. + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:29:39.990 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex 04:29:40.230 # Session INFO HTTPD: POST /Inspect.json key=bad_2k2.hex max_column_display=2147483647 04:29:40.350 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.38 = bad_2k2.hex[,2000] 04:29:40.366 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) ================ new exception 04:33:21.503 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.38 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name.
  3. 0

    it's not necessarily a good thing to do, but we should support it [load a dataset; call it d] d$col2000 <- min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: 2 better yet… once you've done this, stuff breaks everywhere > d$col2000 <- min(d$col2000) + d$col2000 Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. > head(d) Error in data.frame(col1 = 0.542521227151155, col2 = 0.267292166827247, : arguments imply differing number of rows: 1, 0 > 04:23:10.668 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.8 = bad_2k.hex[,2000] = c(0.000574313336983323) 04:23:10.747 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:10.748 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:51.514 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex 04:23:51.633 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex max_column_display=2147483647 04:23:51.753 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.9 = bad_2k.hex[,2000] 04:23:51.768 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name.
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    it's not necessarily a good thing to do, but we should support it [load a dataset; call it d] d$col2000 <- min(d$col2000) Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: 2 better yet… once you've done this, stuff breaks everywhere > d$col2000 <- min(d$col2000) + d$col2000 Error in .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) : http://127.0.0.1:54321/2/Exec2.json returned the following error: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. > head(d) Error in data.frame(col1 = 0.542521227151155, col2 = 0.267292166827247, : arguments imply differing number of rows: 1, 0 > 04:23:10.668 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.8 = bad_2k.hex[,2000] = c(0.000574313336983323) 04:23:10.747 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:10.748 # Session ERRR WATER:▫ + java.lang.ArrayIndexOutOfBoundsException: 2 + ▸ at water.fvec.Vec.elem2ChunkIdx(Vec.java:407) + ▸ at water.fvec.Vec.chunk(Vec.java:505) + ▸ at water.fvec.Vec.at(Vec.java:513) + ▸ at water.fvec.Frame.toString(Frame.java:475) + ▸ at water.api.Exec2.serve(Exec2.java:50) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724) 04:23:51.514 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex 04:23:51.633 # Session INFO HTTPD: POST /Inspect.json key=bad_2k.hex max_column_display=2147483647 04:23:51.753 # Session INFO HTTPD: POST /2/Exec2.json str=Last.value.9 = bad_2k.hex[,2000] 04:23:51.768 # Session ERRR WATER:▫ + java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name. + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:84) + ▸ at water.Lockable$PriorWriteLock.atomic(Lockable.java:73) + ▸ at water.TAtomic.atomic(TAtomic.java:19) + ▸ at water.Atomic.compute2(Atomic.java:57) + ▸ at water.Atomic.fork(Atomic.java:42) + ▸ at water.Atomic.invoke(Atomic.java:34) + ▸ at water.Lockable.write_lock(Lockable.java:59) + ▸ at water.exec.Env.remove_and_unlock(Env.java:326) + ▸ at water.api.Exec2.serve(Exec2.java:62) + ▸ at water.api.Request.serveGrid(Request.java:129) + ▸ at water.Request2.superServeGrid(Request2.java:472) + ▸ at water.api.Exec2.serveGrid(Exec2.java:71) + ▸ at water.api.Request.serve(Request.java:108) + ▸ at water.api.RequestServer.serve(RequestServer.java:315) + ▸ at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421) + ▸ at java.lang.Thread.run(Thread.java:724)

    JIRA | 3 years ago | Earl Hathaway
    java.lang.IllegalArgumentException: Dataset Last.value.4 is already in use. Unable to use it now. Consider using a different destination name.
  6. 0

    There is an error in unlocking the dataset after the previous model fails. The error in the previous model is a javaOutOfMemory and the following model is successfully created, despite the locked job error. The only issue is that that R unit test would fail overall. The test is too long to run, so the script for repo is attached. Output from logs: 05:22:21.748 # Session INFO HTTPD: POST /2/Exec2.json str=arcene.train.full = arcene.train.full 05:22:21.879 # Session ERRR WATER: env.remove_and_unlock() failed + java.lang.IllegalArgumentException: Dataset arcene.train.full is already in use. Unable to use it now. Consider using a different destination name. + at water.Lockable$PriorWriteLock.atomic(Lockable.java:85) + at water.Lockable$PriorWriteLock.atomic(Lockable.java:74) + at water.TAtomic.atomic(TAtomic.java:19) + at water.Atomic.compute2(Atomic.java:58) + at water.Atomic.fork(Atomic.java:42) + at water.Atomic.invoke(Atomic.java:34) + at water.Lockable.write_lock(Lockable.java:60) + at water.exec.Env.remove_and_unlock(Env.java:349) + at water.api.Exec2.serve(Exec2.java:71) + at water.api.Request.serveGrid(Request.java:165) + at water.Request2.superServeGrid(Request2.java:482) + at water.api.Exec2.serveGrid(Exec2.java:78) + at water.api.Request.serve(Request.java:142) + at water.api.RequestServer.serve(RequestServer.java:484) + at water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:424) + at java.lang.Thread.run(Thread.java:745) Output from test: [2014-08-14 17:37:37] [ERROR] : Error: Test failed: 'Testing memory performance of Strong Rules' Not expected: http://127.0.0.1:54321/2/Exec2.json returned the following error: Frame is already locked by job null. 1: withWarnings(test(conn)) 2: withCallingHandlers(expr, warning = wHandler) 3: test(conn) 4: h2o.glm(x = c(1:7000), y = "arcene.train.label", data = arcene.train.full, family = "binomial", lambda_search = T, alpha = 1, nfolds = 0, use_all_factor_levels = T) 5: .h2o.get.glm(data@h2o, as.character(res$destination_key), return_all_lambda) 6: h2o.getFrame(h2o, pre$json$glm_model$dataKey) 7: .h2o.exec2(expr = key, h2o = h2o, dest_key = key) 8: .h2o.__exec2_dest_key(h2o, expr, dest_key) 9: .h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr) 10: stop(paste(myURL, " returned the following error:\n", .h2o.__formatError(res$error))) 11: .handleSimpleError(function (e) { e$calls <- head(sys.calls()[-seq_len(frame + 7)], -2) signalCondition(e) }, "http://127.0.0.1:54321/2/Exec2.json returned the following error:\n Frame is already locked by job null.\n", quote(.h2o.__remoteSend(client, .h2o.__PAGE_EXEC2, str = expr))).

    JIRA | 2 years ago | Ariel Rao
    java.lang.IllegalArgumentException: Dataset arcene.train.full is already in use. Unable to use it now. Consider using a different destination name.

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. java.lang.IllegalArgumentException

      Dataset bad_2k.hex is already in use. Unable to use it now. Consider using a different destination name.

      at water.Lockable$PriorWriteLock.atomic()
    2. water
      Lockable.write_lock
      1. water.Lockable$PriorWriteLock.atomic(Lockable.java:84)
      2. water.Lockable$PriorWriteLock.atomic(Lockable.java:73)
      3. water.TAtomic.atomic(TAtomic.java:19)
      4. water.Atomic.compute2(Atomic.java:57)
      5. water.Atomic.fork(Atomic.java:42)
      6. water.Atomic.invoke(Atomic.java:34)
      7. water.Lockable.write_lock(Lockable.java:59)
      7 frames
    3. water.exec
      Env.remove_and_unlock
      1. water.exec.Env.remove_and_unlock(Env.java:326)
      1 frame
    4. water.api
      Request.serveGrid
      1. water.api.Exec2.serve(Exec2.java:62)
      2. water.api.Request.serveGrid(Request.java:129)
      2 frames
    5. water
      Request2.superServeGrid
      1. water.Request2.superServeGrid(Request2.java:472)
      1 frame
    6. water.api
      RequestServer.serve
      1. water.api.Exec2.serveGrid(Exec2.java:71)
      2. water.api.Request.serve(Request.java:108)
      3. water.api.RequestServer.serve(RequestServer.java:315)
      3 frames
    7. water
      NanoHTTPD$HTTPSession.run
      1. water.NanoHTTPD$HTTPSession.run(NanoHTTPD.java:421)
      1 frame
    8. Java RT
      Thread.run
      1. java.lang.Thread.run(Thread.java:724)
      1 frame