You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
RuntimeError: CUDA out of memory. Tried to allocate 84.00 MiB (GPU 0; 3.82 GiB total capacity; 2.37 GiB already allocated; 76.44 MiB free; 2.52 GiB reserved in total by PyTorch)
#32
Open
husamhamu opened this issue
Mar 22, 2022
· 1 comment
Hi,
can you please share with us a way to solve this error:
RuntimeError: CUDA out of memory. Tried to allocate 84.00 MiB (GPU 0; 3.82 GiB total capacity; 2.37 GiB already allocated; 76.44 MiB free; 2.52 GiB reserved in total by PyTorch)
First, I though it might be a compatibility issue, even though the message is quite clear that this is not the case so nothing really worked for me.
Now I am having a hard time figuring out how to solve it, I would appreciate some help. Thanks
The text was updated successfully, but these errors were encountered:
Traceback (most recent call last):
File "./tools/train.py", line 109, in
main()
File "./tools/train.py", line 105, in main
logger=logger)
File "/home/husam/qd-3dt/tools/../qd3dt/apis/train.py", line 66, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "/home/husam/qd-3dt/tools/../qd3dt/apis/train.py", line 270, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/home/husam/qd-3dt/mmcv/mmcv/runner/runner.py", line 361, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/husam/qd-3dt/mmcv/mmcv/runner/runner.py", line 264, in train
self.model, data_batch, train_mode=True, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/apis/train.py", line 44, in batch_processor
losses = model(**data)
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/core/fp16/decorators.py", line 49, in new_func
return old_func(*args, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/models/detectrackers/base.py", line 86, in forward
return self.forward_train(img, img_meta, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/models/detectrackers/quasi_dense_3d_sep_uncertainty.py", line 147, in forward_train
x = self.extract_feat(img)
File "/home/husam/qd-3dt/tools/../qd3dt/models/detectrackers/quasi_dense_3d_sep_uncertainty.py", line 118, in extract_feat
x = self.neck(x)
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/models/necks/dlaup.py", line 158, in forward
x, y = ida(layers[-i - 2:]) # y : aggregation nodes
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/models/necks/dlaup.py", line 107, in forward
x = node(torch.cat([x, layers[i]], 1))
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/husam/qd-3dt/tools/../qd3dt/models/utils/conv_module.py", line 151, in forward
x = self.conv(x)
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 345, in forward
return self.conv2d_forward(input, self.weight)
File "/home/husam/.pyenv/versions/qd_3dt/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 342, in conv2d_forward
self.padding, self.dilation, self.groups)
RuntimeError: CUDA out of memory. Tried to allocate 84.00 MiB (GPU 0; 3.82 GiB total capacity; 2.37 GiB already allocated; 76.44 MiB free; 2.52 GiB reserved in total by PyTorch)
Hi,
can you please share with us a way to solve this error:
RuntimeError: CUDA out of memory. Tried to allocate 84.00 MiB (GPU 0; 3.82 GiB total capacity; 2.37 GiB already allocated; 76.44 MiB free; 2.52 GiB reserved in total by PyTorch)
First, I though it might be a compatibility issue, even though the message is quite clear that this is not the case so nothing really worked for me.
Now I am having a hard time figuring out how to solve it, I would appreciate some help. Thanks
The text was updated successfully, but these errors were encountered: