Newly Created Agent Speaks Chinese

[ghost]

LlamaSharp config from appsettings.json

  "LlamaSharp": {
    "Interactive": true,
    "ModelDir": "C:\\Models\\TheBloke\\CodeLlama-7B-GGUF",
    "DefaultModel": "codellama-7b.Q5_K_M.gguf",
    "MaxContextLength": 1024,
    "NumberOfGpuLayer": 20
  },

LLM Provider for LlamaSharp:

    {
      "Provider": "llama-sharp",
      "Models": [
        {
          "Name": "codellama-7b.Q5_K_M.gguf",
          "Type": "chat"
        }
      ]
    }

Agent configuration:

{
  "id": "ddf46fa4-5686-408a-a574-b3da43f3ed99",
  "name": "test-agent",
  "description": "generic agent to test chat functionality using locally installed LLM",
  "instruction": "",
  "templates": [],
  "functions": [],
  "responses": [],
  "samples": [],
  "is_public": false,
  "is_router": false,
  "allow_routing": false,
  "disabled": false,
  "icon_url": null,
  "profiles": [],
  "routing_rules": [],
  "llm_config": {
    "is_inherit": true,
    "provider": "llama-sharp",
    "model": "codellama-7b.Q5_K_M.gguf"
  },
  "plugin": {
    "id": "00000000-0000-0000-0000-000000000000",
    "name": "BotSharp",
    "description": null,
    "assembly": "BotSharp.Core",
    "icon_url": null,
    "agent_ids": [
      "ddf46fa4-5686-408a-a574-b3da43f3ed99"
    ],
    "enabled": true,
    "menus": null
  },
  "created_datetime": "2024-01-15T17:19:45.2410911Z",
  "updated_datetime": "2024-01-15T17:19:45.2410912Z"
}

Message Sent:

{
  "text": "hello world"
}

To allow CPU initialization, 'LlamaSharp.Backend.Cpu' is referenced.

Response:
你好，我是小芭。您可以对我说“小芭，帮我做什么”，我会尽力帮助您。

Rough translation:

Hello, I am Xiaoba. You can say to me "Xiaoba, what can I do for you?" and I will try my best to help you.

---

This response comes back for everything. The model have been tested in LM studio and doesn't appear to have any affinity for Chinese, What am I doing wrong?

[Oceania2018]

This issue should be related to LLamaSharp, maybe you can raise issue to that repo.
I haven't encountered this situation.

[ghost]

OK, thanks. I was worried it was some kind of default. The bot shouldn't be responding in Chinese. You're suggesting this is an error in the LlamaSharp library?

This is the console output from the call

...................................................................................................
llama_new_context_with_model: n_ctx      = 1024
llama_new_context_with_model: freq_base  = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: KV self size  =  512.00 MiB, K (f16):  256.00 MiB, V (f16):  256.00 MiB
llama_build_graph: non-view tensors processed: 676/676
llama_new_context_with_model: compute buffer total size = 93.32 MiB
llama_new_context_with_model: n_ctx      = 1024
llama_new_context_with_model: freq_base  = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: KV self size  =  512.00 MiB, K (f16):  256.00 MiB, V (f16):  256.00 MiB
llama_build_graph: non-view tensors processed: 676/676
llama_new_context_with_model: compute buffer total size = 93.32 MiB
??,???????????"??,?????",????????
user:Loaded LlmProvider azure-openai settings with 2 models.
Loaded LlmProvider llama-sharp settings with 1 models.
Sending [test-agent] assistant: ??,???????????"??,?????",????????
Token Usage: 0 prompt + 0 completion = 0 total tokens (146.46s). One-Way cost: £0.0000, accumulated cost: £0.0000. [codellama-7b.Q5_K_M.gguf]

??,???????????"??,?????",???????? is what comes back in the response as

你好，我是小芭。您可以对我说“小芭，帮我做什么”，我会尽力帮助您。

Because of the following line

Token Usage: 0 prompt + 0 completion = 0 total tokens (146.46s). One-Way cost: £0.0000, accumulated cost: £0.0000. 

I was making the assumption that the response was not being generated by the model but a returnable from something else. The channel invoked by the agent is "openapi" (testing via the api so this is correct), and for some reason the llm provider for azure-openai is also being loaded. Am I correct in assuming this should be restricted to Llama only?

The model has also been tested using the LlamaSharp examples and responds, as intended, in English. It seems to me that the solution has some kind of overriding agent or functionality active. The bot refers to itself as Xiaoba, and when asking "what can you do for me" in response to the initial reply, it retunrs a list of actions such as "weather" and "news".

[Oceania2018]

If you're able to, can you run it in debug mode and go deeper to figure out what is happing?

[ghost]

Stepping through each line, there is nothing I can see to indicate any error, and nothing immediate which seems to be configured that would specify Chinese for output language.

Do you have an example agent, configured to use Llama I could use to test and compare?

Using the default AI Assistant agent in the solution also returns Chinese only. This time, however, it calls itself Little Mango

你好，我是小芒果。您可以问我一些关于医疗保健的问题。

Hello, I am Little Mango. You can ask me questions about health care.

[Oceania2018]

@kiebor Any clue you've found?