How easy is it to use another model like mpt7b #238 & GPT4ALL support or open source models #49

[arunsoman]

As suggested creating a discussion thread so that we could discuss and come up with a solution to support any LLM in this wonderful gpt-engineer

[arunsoman]

If we could do something like LangChain does, then it will be just a matter of installing the models separately and just using the llm

from langchain import PromptTemplate, LLMChain

from langchain.llms import GPT4All
llm = GPT4All(model=local_path, callbacks=callbacks, verbose=True)

or
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
from langchain.llms import OpenAI
llm = OpenAI()

template = """Question: {question}
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = LLMChain(prompt=prompt, llm=llm)
llm_chain.run(question)

langchain supports a large number of models today, link below
https://python.langchain.com/docs/modules/model_io/models/llms

How easy will it be for us to use langchain as a project dependency and use
from langchain import LLMChain as the entry point to access the model behind

[BobbtheBuilder]

this repo seams to make this much easier to implement. it provides an API interface as a backend for any local LLM

https://github.com/bentoml/OpenLLM

https://www.youtube.com/watch?v=8nZZ2oQhx4E

[brainshower]

With the newer coding fine-tuned models coming out, this is going to become more important very soon.

[arunsoman]

I can code once we have finalized the approach