Integrating Internet access for local LLMs

[chaoticsoap]

Seems quite possible using llm-axe
Please include this!

simple script example:

from llm_axe import OnlineAgent, OllamaChat

llm = OllamaChat(model="llama3:instruct")
online_agent = OnlineAgent(llm)

prompt = "Can you tell me what the latest update for Unreal Engine 5 is?"
resp = online_agent.search(prompt)
print(resp)

#OUTPUT:
#Based on information from the internet, 
#the latest version of Unreal Engine 5 is 5.3.

[jakobdylanc]

You can already achieve online capability by using ex. Perplexity's online models: https://docs.perplexity.ai/docs/model-cards (this obviously isn't local though)

I feel like online capability should exist at the LLM level like above, not my app's level. Maybe I'll change my mind on that someday.

If it were in my app, why is llm-axe the best option? I'm sure there are a million other options which is another reason I hesitate :)

[chaoticsoap]

Ah, I'm looking for a way to use an abliterated, uncensored model + including private/own server online search for correct info
The best model I've found is llama3-alpha-centauri-uncensored

I only mentioned llm-axe as I stumbled upon it and the integration seems really easy

[jakobdylanc]

Someone should definitely make an "online adapter" for local LLMs. Maybe ollama will add this feature someday.