java.lang.IllegalStateException: MemcpyAsync H2H failed: [839371888] -> [38700713984]

There are no available Samebug tips for this exception. Do you have an idea how to solve this issue? A short tip would help users who saw this issue last week.

  • GitHub comment 1822#232371459
    via GitHub by Jydago
    ,
  • Error 74 when running DL4J using GPU
    via GitHub by Jydago
    ,
  • Cannot use gpu with aws ami
    via GitHub by davireis
    ,
    • java.lang.IllegalStateException: MemcpyAsync H2H failed: [839371888] -> [38700713984] at org.nd4j.jita.handler.impl.CudaZeroHandler.memcpyAsync(CudaZeroHandler.java:510) at org.nd4j.jita.allocator.impl.AtomicAllocator.memcpyAsync(AtomicAllocator.java:774) at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.set(BaseCudaDataBuffer.java:296) at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.setData(BaseCudaDataBuffer.java:366) at org.nd4j.linalg.jcublas.buffer.CudaIntDataBuffer.<init>(CudaIntDataBuffer.java:74) at org.nd4j.linalg.jcublas.buffer.factory.CudaDataBufferFactory.createInt(CudaDataBufferFactory.java:266) at org.nd4j.linalg.factory.Nd4j.createBuffer(Nd4j.java:1164) at org.nd4j.linalg.api.shape.Shape.createShapeInformation(Shape.java:1742) at org.nd4j.linalg.api.ndarray.BaseShapeInfoProvider.createShapeInformation(BaseShapeInfoProvider.java:12) at org.nd4j.jita.constant.ProtectedCudaShapeInfoProvider.createShapeInformation(ProtectedCudaShapeInfoProvider.java:52) at org.nd4j.linalg.jcublas.CachedShapeInfoProvider.createShapeInformation(CachedShapeInfoProvider.java:41) at org.nd4j.linalg.api.ndarray.BaseNDArray.<init>(BaseNDArray.java:149) at org.nd4j.linalg.jcublas.JCublasNDArray.<init>(JCublasNDArray.java:341) at org.nd4j.linalg.jcublas.JCublasNDArrayFactory.create(JCublasNDArrayFactory.java:233) at org.nd4j.linalg.factory.Nd4j.create(Nd4j.java:3612) at org.nd4j.linalg.api.ndarray.BaseNDArray.create(BaseNDArray.java:1732) at org.nd4j.linalg.api.ndarray.BaseNDArray.permute(BaseNDArray.java:4213) at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.preOutput(ConvolutionLayer.java:224) at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.activate(ConvolutionLayer.java:252) at org.deeplearning4j.nn.layers.BaseLayer.activate(BaseLayer.java:390) at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.activationFromPrevLayer(MultiLayerNetwork.java:553) at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.feedForwardToLayer(MultiLayerNetwork.java:676) at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:1793) at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:152) at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:56) at org.deeplearning4j.optimize.Solver.optimize(Solver.java:51) at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1431) at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1468) at NeuralNetworkBuilder.Model.ModelTrainer$$anonfun$trainModelLocally$1.apply(ModelTrainer.scala:91) at NeuralNetworkBuilder.Model.ModelTrainer$$anonfun$trainModelLocally$1.apply(ModelTrainer.scala:82) at scala.collection.immutable.Range.foreach(Range.scala:166) at NeuralNetworkBuilder.Model.ModelTrainer.trainModelLocally(ModelTrainer.scala:82) at CNNs.Laptop.testHotspots$.main(testHotspots.scala:77) at CNNs.Laptop.testHotspots.main(testHotspots.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
    No Bugmate found.