Instructions to run KG_RAG on mac #34
Conversation
config.yaml
Outdated
There was a problem hiding this comment.
Removed hardcoded paths and added /workspaces to standardize the local dev across platforms.
| @@ -11,7 +11,7 @@ asttokens==2.4.0 | |||
| async-lru==2.0.4 | |||
| async-timeout==4.0.3 | |||
| attrs==23.1.0 | |||
| auto-gptq==0.4.2 | |||
| # auto-gptq==0.4.2 | |||
There was a problem hiding this comment.
I couldn't install this and find any usage in the codebase. Can we remove this from the requirements?
There was a problem hiding this comment.
I would probably keep this, because, some users utilize quantized model to run KG-RAG. And I presume this was added by them.
| # nvidia-cublas-cu11==11.10.3.66 | ||
| # nvidia-cuda-cupti-cu11==11.7.101 | ||
| # nvidia-cuda-nvrtc-cu11==11.7.99 | ||
| # nvidia-cuda-runtime-cu11==11.7.99 | ||
| # nvidia-cudnn-cu11==8.5.0.96 | ||
| # nvidia-cufft-cu11==10.9.0.58 | ||
| # nvidia-curand-cu11==10.2.10.91 | ||
| # nvidia-cusolver-cu11==11.4.0.1 | ||
| # nvidia-cusparse-cu11==11.7.4.91 | ||
| # nvidia-nccl-cu11==2.14.3 | ||
| # nvidia-nvtx-cu11==11.7.91 |
There was a problem hiding this comment.
I couldn't install the specific versions. Is this required?
There was a problem hiding this comment.
i know local models such as llama and sentence transformers make use of nvidia gpu to run the operations (which I tried in the linux server). so this maybe useful for that. but I haven't checked it otherwise.
| @@ -185,7 +185,7 @@ tornado==6.3.3 | |||
| tqdm==4.66.1 | |||
| traitlets==5.10.0 | |||
| transformers==4.33.2 | |||
| triton==2.0.0 | |||
| # triton==2.0.0 | |||
There was a problem hiding this comment.
Is this required? Couldn't install this either.
There was a problem hiding this comment.
i presume this is also related to nvidia gpu. so same explanation as above
| @@ -392,5 +392,6 @@ def interactive(question, vectorstore, node_context_df, embedding_function_for_c | |||
| output = llm_chain.run(context=node_context_extracted, question=question) | |||
| elif "gpt" in llm_type: | |||
| enriched_prompt = "Context: "+ node_context_extracted + "\n" + "Question: " + question | |||
| output = get_GPT_response(enriched_prompt, system_prompt, llm_type, llm_type, temperature=config_data["LLM_TEMPERATURE"]) | |||
| chat_model_id, chat_deployment_id = get_gpt35() | |||
There was a problem hiding this comment.
Noticed that chat_deployment_id was same as chat_model_id if not using this step when using GPT_API_TYPE : 'open_ai'.
There was a problem hiding this comment.
if api type is open_ai, I think chat_deployment_id is None. Please see my response given below.
There was a problem hiding this comment.
can you try changing the absolute path to relative path?
for example:
instead of:
VECTOR_DB_DISEASE_ENTITY_PATH : '/workspaces/KG_RAG/data/disease_with_relation_to_genes.pickle'
try:
VECTOR_DB_DISEASE_ENTITY_PATH : 'KG_RAG/data/disease_with_relation_to_genes.pickle'
since the code is run as a module from the KG_RAG directory, I think this should be fine and the users do not need to change the path. Can you please change it and test it? If it works fine, then please change it to the relative path format.
| - [Step 4: Update config.yaml](https://github.com/BaranziniLab/KG_RAG#step-4-update-configyaml) | ||
| - [Step 5: Run the setup script](https://github.com/BaranziniLab/KG_RAG#step-5-run-the-setup-script) | ||
| - [Step 6: Run KG-RAG from your terminal](https://github.com/BaranziniLab/KG_RAG#step-6-run-kg-rag-from-your-terminal) | ||
| - [Step 2: Setup using Dev Containers](https://github.com/BaranziniLab/KG_RAG#step-2-setup-using-dev-containers) |
There was a problem hiding this comment.
I really liked the container-based setup. However, I am aware that some users have managed to run KG-RAG without it, skipping straight from Step 1 to the 'Create a virtual environment' step. Therefore, if the container-based setup is only necessary for macOS users, could you please mark this step as 'Optional' and note that it's specifically for macOS installation?
| @@ -392,5 +392,6 @@ def interactive(question, vectorstore, node_context_df, embedding_function_for_c | |||
| output = llm_chain.run(context=node_context_extracted, question=question) | |||
| elif "gpt" in llm_type: | |||
| enriched_prompt = "Context: "+ node_context_extracted + "\n" + "Question: " + question | |||
| output = get_GPT_response(enriched_prompt, system_prompt, llm_type, llm_type, temperature=config_data["LLM_TEMPERATURE"]) | |||
| chat_model_id, chat_deployment_id = get_gpt35() | |||
There was a problem hiding this comment.
One problem with the new line 395 is that it will always call gpt-3.5, regardless of whether the user specified other gpt models, such as gpt-4.
I think the better option here is to change line 395:
from:
chat_model_id, chat_deployment_id = get_gpt35()
to:
chat_deployment_id = chat_model_id if openai.api_type == "azure" else None
Do you agree?
What
Created a PR to set up KG on macOS successfully. Related to issue #35.
Changes
Added a
.devcontainerdirectory to the root path. This contains aDockerfileandpostCreateCommand.shshell script to set up the environment. Note: This requires installing docker on the local machine first.Added a sample
.gpt_config.envfile for quick modification and setup by users.Commented certain libraries in the
requirements.txtfile which showed errors during installation, same as described in issue Issues setting up KGRAG on macOS #35. Since I couldn't find any direct usage of these libraries, I commented them out to successfully install the dependencies and test KG_RAG using the instructions provided in the README file.Changes to
kg_rag/run_setup.py-- Create
LLM_CACHE_DIRdoes not exist-- Raise value error if there's an exception encountered while downloading llama model.
Changes to
kg_rag/utitlities.py-- Modified the parameters getting passed to
get_GPT_responsewhen using interactive modeUpdated the
.gitignorefile to not check theLLM_CACHE_DIRand__pycache__files