High (100%) CPU Utilization for Transformer Container. ThreadPoolExecutor not using max_workers defined.

[pchunduru10]

Hi I am trying to run the transformers in kubernetes pod on my instance and its using up all the CPU cores available and throttling other pods in the cluster eventually crashing the sytem.

I tried to create a sed patch in in the dokcer-entrypoint file for the transformers to override ThreadPoolExecutor max workers which is set to. None (by default) to a value that user defines (with an ENV variable)

This doesnt seem to work as well, When I /bin/bash into the container the fix seems to work as below, but its not perofrming as expected and still using all the cores on the instance it was deployed.

class Vectorizer:
    executor: ThreadPoolExecutor

    def __init__(self, model_path: str, cuda_support: bool, cuda_core: str, cuda_per_process_memory_fraction: float,
                 model_type: str, architecture: str, direct_tokenize: bool, onnx_runtime: bool,
                 use_sentence_transformer_vectorizer: bool, model_name: str):
        self.executor = ThreadPoolExecutor(max_workers=2)

Appreciate any help regarding this issue.