Converted msmarco-distilbert-cos-v5 model with convert_model.py fails when loaded with Tch tensor error: cannot find the tensor named distilbert.transformer.layer.0.attention.v_lin.bias

[owi1972]

I converted the model
https://huggingface.co/sentence-transformers/msmarco-distilbert-cos-v5
with the current branch and the ./util/convert_model.py script

In the the example/sentence_embeddings_local.rs
it says that you should use --prefix distilbert and --suffix for Distil-based models.

I tried with or without --prefix or --suffix but always got the same error
when using the model in the sentence_embeddings_local.rs example

Tch tensor error: cannot find the tensor named distilbert.transformer.layer.0.attention.v_lin.bias

Is there an example on how to convert a model or model classes? What am I doing wrong?

[linkedlist771]

I have also met the same error as you, this rustber only supports some models. This error raises because this model's layer has not been manually implemented yet. It seems that this project is not maintaining. I suggest you to use the onnx type to make inference. Here is a project about it https://github.com/nbigaouette/onnxruntime-rs.

[guillaume-be]

Hello,

The conversion script outputs the weights that are being converted. Can you please check if a prefix or suffix is mismatching the expected weights (i.e. distilbert.transformer.layer.0.attention.v_lin.bias)? Is the error always referring to distilbert.transformer.layer.0.attention.v_lin.bias or a different set of weights at each tryout?