Multi-language instructions

[andypotato]

I tried to find instructions on how to use localGPT with other document languages than English, however there is no documentation on this so far.

Going through the backlog of issues I found a couple of starting points:

• Replace the default instructor model (hkunlp/instructor-large) with a model supporting multiple languages, eg "intfloat/multilingual-e5-base"
• Using a LLM with support for other languages than English

To make things easier to find, I suggest we collect all information related to using localGPT in other languages in this discussion thread.

[devor-64]

I'm interested by this subject as well. foreign languages are not well handled but maybe another model would do the trick.

[PromtEngineer]

I agree, probably need to have a list of models that are tested and add which languages are supported. Will see, how can I document this somewhere.

[MatKollar]

Hello, any updates on this topic ? Do we have some documentation of supported languages for embeddings and LLMs?

[omaer0]

I have just suceeded in getting localGPT running on my Linux laptop (16G RAM, Nvidia with 8GB VRAM), and also have a partial sucess with German.
constants.py:
EMBEDDING_MODEL_NAME = "intfloat/multilingual-e5-large" # Uses 2.5 GB of VRAM
MODEL_ID = "TheBloke/Llama-2-7b-Chat-GGUF"
MODEL_BASENAME = "llama-2-7b-chat.Q4_K_M.gguf"

prompt_template_utils.py:
system_prompt = """You are a helpful assistant, you will use the provided context to answer user questions in German.
Read the given context before answering questions and think step by step. If you can not answer a user question based on the provided context, inform the user. Do not use any other information for answering user. Provide a detailed answer to the question in German. Übersetzte alle deine Antworten ins Deutsche."""

Even with the modified system_prompt, the answers are initially in English, but the LLM can be coaxed to answer in German. In the example below, I used a .txt file with "Kritik der reinen Vernunft" by Immanuel Kant from Project Gutenberg (with the English prefix and annex removed) as source document:

Enter a query: Was versteht man unter reiner Vernunft?
> Question: 
Was versteht man unter reiner Vernunft? 

> Answer: 
 Understood! Based on the provided context, "reine Vernunft" refers to the discipline of pure reason or pure intellect. In this 
context, it seems to be a philosophical concept that is being discussed. 
To answer your question, "Was versteht man unter reiner Vernunft?", one could translate it as "What does one understand by
pure reason?" or "What is meant by pure intellect?" 
In the given context, the author appears to be discussing the limitations of empirical reasoning and the potential of pure 
reason to transcend those limits. The author suggests that pure reason offers the promise of extending our knowledge 
beyond the boundaries of experience, but only if it is properly understood and applied. However, if one misinterprets or 
misunderstands pure reason, it can lead to endless conflicts and disputes. 
Therefore, in order to answer your question directly, "reine Vernunft" refers to the discipline of reasoning that seeks to extend 
our knowledge beyond the limits of experience through systematic unity and coherence, rather than merely accumulating 
empirical data. 

Enter a query: übersetzte deine Antwort ins deutsche

Question: 
übersetzte deine Antwort ins deutsche 

> Answer: 
 Gerne! Ich bin hier, um Ihre Fragen in Deutsch zu beantworten, basierend auf dem gegebenen Kontext. Bitte stellen Sie Ihre 
Frage und ich werde mein Bestes tun, um eine ausführliche Antwort
zu liefern. 

Enter a query: Was versteht man unter reiner Vernunft?
> Question: 
Was versteht man unter reiner Vernunft? 

> Answer: 
 Unter "reiner Vernunft" versteht man in diesem Kontext die unbeeinflussten, aus den Erfahrungen und dem Verstand
entspringenden Überlegungen und Schlussfolgerungen, die sich auf die Welt und ihre Ordnung beziehen. Es handelt sich um
eine unabhängige und objektive Urteilsbildung, die sich nicht von besonderen Interessen oder Vorurteilen beeinflussen lässt. 
Der Ausdruck "reine Vernunft" soll hencein verstanden werden als die Fähigkeit, auf die Objekte der Erfahrung und ihre
Beziehungen zu urteilen, ohne dabei jemals das Ziel haben zu müssen, etwas zu bewerkstelligen oder ein bestimmtes 
Ergebnis zu erreichen. Stattdessen ist es das Ziel der reinen Vernunft, die Wahrheit zu erkennen und zu verstehen, 
unabhängig von den persönlichen Vorschriften oder Interessen des Betrachtende.

However, trying the same dialog with unmodified system_prompt results still in English-only answers

[GabrielGuilherme88]

EMBEDDING_MODEL_NAME = "intfloat/multilingual-e5-large" # Uses 2.5 GB of VRAM

Just by making this change, were you able to generate translations?

[omaer0]

Hi, I'm unsure about what you mean by "generate translations", can you explain?

My aim was not to get a text translation, but to have a local document in German (in my case Immanuel Kant's 'Critique of pure reason'), ingest it using the multilingual-e5-large embedding, and then get a summary or explanation of concepts presented in the document in German using the Llama-2-7b pre-trainded LLM.

The only changes I did are the ones described in my previous post.

I got the idea for this experiment by the example in chapter 4 of the book 'Transformers for Natural
Language Processing' by Denis Rothman, ISBN 978-1-80324-733-5, where a pretrained ROBERTa transformer model is built. The model first loads a compilation of an English translation of Immanuel Kant’s books, trains its own tokenizer for use during the pretraining process, and finally uses the trained model to perform an experimental downstream language modeling task and fills a mask using Immanuel Kant’s logic. The response to the query
fill_mask("Human thinking involves human .")
was in this example

[{'score': 0.022831793874502182,
 'sequence': '<s> Human thinking involves human reason.</s>',
 'token': 393},
 {'score': 0.011635891161859035,
 'sequence': '<s> Human thinking involves human object.</s>',
 'token': 394},
 {'score': 0.010641072876751423,
 'sequence': '<s> Human thinking involves human priori.</s>',
 'token': 575},
 {'score': 0.009517930448055267,
 'sequence': '<s> Human thinking involves human conception.</s>',
 'token': 418},
 {'score': 0.00923212617635727,
 'sequence': '<s> Human thinking involves human experience.</s>',
 'token': 531}]

Disclaimer: I'm still in the early part of my learning curve about LLM and generative pre-trained transformers, and I just wanted to see how much sense or nonsense could be obtained from a local, offline LLM when it is fed a document in German.