question on training on large gpu

[johndpope]

(torch) ➜ gitWorkspace nvidia-htop.py
Fri Nov 12 08:32:56 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:21:00.0 On | N/A |
| 41% 67C P2 213W / 370W | 5398MiB / 24234MiB | 36% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| GPU PID USER GPU MEM %CPU %MEM TIME COMMAND |
| 0 2637 root 725MiB 2.5 0.3 04:35:08 /usr/lib/xorg/Xorg v |
| 0 3700 jp 133MiB 5.7 2.3 04:35:05 /usr/bin/gnome-shell |
| 0 5077 jp 198MiB 7.9 0.9 04:34:57 /opt/google/chrome/c |
| 0 11864 jp 64MiB 1.1 0.5 04:16:50 /snap/code/80/usr/sh |
| 0 26251 jp 287MiB 0.1 4.6 01:34:40 python generate.py |
| 0 58358 jp 3985MiB 99.8 15.7 47:52 python scripts/train |
+-----------------------------------------------------------------------------+

Currently - I'm seeing usage at 16% - 5GB memory of 24GB card. Is there some low hanging fruit to get code to use more resources?

I did take a look here
https://towardsdatascience.com/7-tips-for-squeezing-maximum-performance-from-pytorch-ca4a40951259
(only thing that stood out is tensor(init) not calling cuda() directly.)

were there efforts to reduce ram requirements (that I could remove)?

[ciaua]

A simple thing to try is to increase the number of workers in dataloader.