Adding an assert or a flag between Double and Float type

[ghost]

For people who won't use docker but want to do it locally the crucial line is: line 51 in SSD/train.py
from ploc, plabel = ploc.float(), plabel.float()
to ploc, plabel = ploc.double(), plabel.double()

Related to local deployment with stable version of pytorch (S1 loss expects double, no float)

[skierat]

What do you mean by "S1 loss expects double"? Do you mean SmoothL1Loss and CrossEntropyLoss expects double? What kind of error are you getting without the change?

[ghost]

(the CrossEntropyLoss) The error without the change:

epoch: 0	iteraion: 0	loss: 27.687554715789148
Traceback (most recent call last):
  File "main.py", line 250, in <module>
    train(train_loop_func, logger, args)
  File "main.py", line 201, in train
    logger, args, mean, std)
  File "/home/ionelia/DeepLearningExamples/PyTorch/Detection/SSD/src/train.py", line 71, in train_loop
    loss.backward()
  File "/home/ionelia/.local/lib/python3.7/site-packages/torch/tensor.py", line 198, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/ionelia/.local/lib/python3.7/site-packages/torch/autograd/__init__.py", line 100, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: expected dtype Double but got dtype Float (validate_dtype at /pytorch/aten/src/ATen/native/TensorIterator.cpp:143)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x46 (0x7fd5d8d35536 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: at::TensorIterator::compute_types() + 0xce3 (0x7fd616456183 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #2: at::TensorIterator::build() + 0x44 (0x7fd616458b64 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #3: at::native::smooth_l1_loss_backward_out(at::Tensor&, at::Tensor const&, at::Tensor const&, at::Tensor const&, long) + 0x193 (0x7fd6162a6963 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0xfa5bb7 (0x7fd5da136bb7 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cuda.so)
frame #5: at::native::smooth_l1_loss_backward(at::Tensor const&, at::Tensor const&, at::Tensor const&, long) + 0x16e (0x7fd6162af08e in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x117b29f (0x7fd61679629f in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0x10c3c76 (0x7fd6166dec76 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #8: <unknown function> + 0x2ca82ab (0x7fd6182c32ab in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #9: <unknown function> + 0x10c3c76 (0x7fd6166dec76 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #10: torch::autograd::generated::SmoothL1LossBackward::apply(std::vector<at::Tensor, std::allocator<at::Tensor> >&&) + 0x1f7 (0x7fd617ec1af7 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #11: <unknown function> + 0x2d89c05 (0x7fd6183a4c05 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #12: torch::autograd::Engine::evaluate_function(std::shared_ptr<torch::autograd::GraphTask>&, torch::autograd::Node*, torch::autograd::InputBuffer&) + 0x16f3 (0x7fd6183a1f03 in /home/ionelia/.local/lib/python3.7/site-epoch: 0	iteraion: 0	loss: 27.687554715789148
packages/torch/lib/libtorch_cpu.so)
frame #13: torch::autograd::Engine::thread_main(std::shared_ptr<torch::autograd::GraphTask> const&, bool) + 0x3d2 (0x7fd6183a2ce2 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #14: torch::autograd::Engine::threpoch: 0	iteraion: 0	loss: 27.687554715789148
ead_init(int) + 0x39 (0x7fd61839b359 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #15: torch::autograd::python::PythonEngine::thread_init(int) + 0x38 (0x7fd624ada998 in /home/ionelia/.local/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #16: <unknown function> + 0xbd6df (0x7fd625bac6df in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
frame #17: <unknown function> + 0x76db (0x7fd6282ca6db in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #18: clone + 0x3f (0x7fd62860388f in /lib/x86_64-linux-gnu/libc.so.6)