Remote deployment

[ItzCrazyKns]

Get help from the community regarding deployment on a remote server or exposing it locally to your network.

[spammenotinoz]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free.
Secondly the backend and search API's are exposed to the client.
So really intended for a trusted audience, otherwise Architectual changes are required.

[spammenotinoz]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free. Secondly the backend and search API's are exposed to the client. So really intended for a trusted audience, otherwise Architectual changes are required.

I have not YET reviewed the code, but I see this project uses langchain, so in theory if you used something like LiteLLM Proxy in the same docker network, just set the baseUrl to LiteLLM Proxy (don't expose LiteLLM), then simply reference lite LiteLLM by the internal container name, so you don't need to define an OpenAI key at all, just use a dummy key.
Remove the settings \ section from the GUI.
Add some authentication.
You do not need to expose SearchNG, just reference it by the internal docker name.
ie: in the config.toml
[API_ENDPOINTS]
SEARXNG = "http://searxng:8080" #use the internal port not the external port
And in docker compose rem out the ports;
In docker args for Perplexica revise to - SEARXNG_API_URL=http://searxng:8080

So that leave the issue of having to expose the perplexica-backend (which makes all the API calls exposed).
Wouldn't be so much of a problem if a secret key is being used, but I didn't pickup on one.

I haven't tried the project, might take a closer look, not quite sure the benefits of this over the Free version of Perplexity.

[delfireinoso]

I haven't tried the project, might take a closer look, not quite sure the benefits of this over the Free version of Perplexity.

For my experience Perplexity is ahead the free version of Perpelxity, and approaching the paid version if you use a good model. It lacks the image parts, but the project is on first steps.

You are right the project has some limitations on the network settings and would benefit by some changes to make it useful out of the LAN. For the moment it's restricted to one user, so it's not safe to expose a strangers. But I think that someday this changes will be introduced, and a private Perplexity clone is better for me than the real

[ishaan-jaff]

hi @spammenotinoz, thanks for using / mentioning LiteLLM. Can we hop on a quick call sometime this week? I’d love to learn how we can improve litellm for you

My cal for your convenience: https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version?month=2023-10
My linkedin if you prefer DMs: https://www.linkedin.com/in/reffajnaahsi/

[ItzCrazyKns]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free. Secondly the backend and search API's are exposed to the client. So really intended for a trusted audience, otherwise Architectual changes are required.

I have not YET reviewed the code, but I see this project uses langchain, so in theory if you used something like LiteLLM Proxy in the same docker network, just set the baseUrl to LiteLLM Proxy (don't expose LiteLLM), then simply reference lite LiteLLM by the internal container name, so you don't need to define an OpenAI key at all, just use a dummy key. Remove the settings \ section from the GUI. Add some authentication. You do not need to expose SearchNG, just reference it by the internal docker name. ie: in the config.toml [API_ENDPOINTS] SEARXNG = "http://searxng:8080" #use the internal port not the external port And in docker compose rem out the ports; In docker args for Perplexica revise to - SEARXNG_API_URL=http://searxng:8080

So that leave the issue of having to expose the perplexica-backend (which makes all the API calls exposed). Wouldn't be so much of a problem if a secret key is being used, but I didn't pickup on one.

I haven't tried the project, might take a closer look, not quite sure the benefits of this over the Free version of Perplexity.

While building the images we update the URL in the config and use the internal port automatically (you can see more in the Docker Images). It is left there for the non Docker users so they do not need to update anything.

[ItzCrazyKns]

From my quick scan, this project appears to expose the API keys in the browser code. So even if you modified the GUI to hide settings, the API keys can still be obtained and exposed. Not an issue I suppose if you are using Groq while it's free. Secondly the backend and search API's are exposed to the client. So really intended for a trusted audience, otherwise Architectual changes are required.

The keys were exposed in the browser to allow different users use different models (with different API keys) are the same time. The project is still meant to be hosted locally. If we had to change how keys are stored, we can store them in the config file and update them from the frontend but this would limit each user to use the same API key for each provider. Still we need another way (besides this) to use custom OpenAI because users might want to have a different URL for them (I believe that is not big of an issue, we can expose that because it generally doesn't involves an API key or even if it does it might be local).