[ASR/Kaldi] multi-gpu problem

[Slyne]

Related to ASR/Kaldi
(e.g. GNMT/PyTorch or FasterTransformer/All)

Describe the bug
When deployed on multi-gpus based on this example, the RTF decreased a lot.
There is a known issue under this example release notes section. Not sure if this is the same issue.

To Reproduce
Steps to reproduce the behavior:
Run this example on multiple gpus.

Expected behavior
Expect the RTF will increase as the number of gpus increases.

Environment
Please provide at least:

• Container version: nvcr.io/nvidia/tritonserver:20.03-py3
• GPUs in the system: 2x Tesla V100-SXM2-16GB):
• CUDA driver version: 418.67