The OpenROAD chat assistant aims to provide easy and quick access to information regarding tools, responses to questions and commonly occurring problems in OpenROAD and its native flow OpenROAD-flow-scripts.
The current architecture uses certain retrieval techniques on OpenROAD documentation and other online data sources. We aim to continuously improve the architecture and the associated the dataset to improve accuracy, coverage and robustness.
- Installation and Troubleshooting Assistance: The chatbot will provide users with quick and accurate solutions to common installation issues and troubleshooting steps.
- Easy Access to Existing Resources: The chatbot will be able to summarize relevant information from OpenROAD documentation, user guides, and online resources to provide concise and actionable answers to user queries.
We have divided our app into three components, each of which can be hosted on a separate machine for scalability.
- Backend: Generates the necessary chat endpoints for users to communicate with.
- Frontend: We use Streamlit to communicate with a chat endpoint, providing a user-friendly chat interface.
- Evaluation: Besides the vanilla chat interface, we also have a human evaluation interface for research and development.
This setup involves the setting of both the frontend and backend components. We shall begin with backend:
Ensure you have docker and docker-compose installed in your system.
Step 1: Clone the repository:
git clone https://github.com/The-OpenROAD-Project/ORAssistant.gitStep 2: Copy the .env.example file, and update your .env file with the appropriate API keys.
cd backend
cp .env.example .envStep 3: Start the docker container by running the following command:
docker compose upPrerequisites:
Python 3.12, recommended using a virtual environment likeconda.wgetpandocgit
Step 1: Install the required dependencies.
pip install -r backend/requirements.txtStep 2: Copy the .env.example file, and update your .env file with the appropriate API keys.
cd backend
cp .env.example .envStep 3: For populating the data folder with OR/ORFS docs, OpenSTA docs and Yosys docs, run:
python build_docs.py
# Alternatively, pull the latest docs
mkdir data
huggingface-cli download The-OpenROAD-Project/ORAssistant_RAG_Dataset --repo-type dataset --local-dir data/Step 4: To run the server, run:
python main.pyOptionally: To interact with the chatbot in your terminal, run:
python chatbot.pyThe backend will then be hosted at http://0.0.0.0:8000.
Open http://0.0.0.0:8000/docs for the API docs.
Note: Please refer to the frontend README for more detailed instructions.
- Step 1: Set up the
.envas per the instructions in the frontend README. Get the Google Sheet API Key
cd frontend
cp .env.example .env- Step 2: Install the necessary requirements. You are encouraged to use a virtual environment for this.
pip install -r requirements.txt- Step 3: Run streamlit application
streamlit run streamlit_app.pyOpenROAD documentation, OpenROAD-flow-scripts documentation, manpages and OpenSTA documentation is chunked and embedded into FAISS Vector Databases.
Documents are first retrieved from the vectorstore using a hybrid retriever, combining vector and semantic search methods. These retrieved documents undergo re-ranking using a cross-encoder re-ranker model.
flowchart LR
id0([Query]) --> id1
id1([Vectorstore]) --- id2([Semantic Retriever])
id1([Vectorstore]) --- id3([MMR Retriever])
id1([Vectorstore]) --- id4([BM25 Retriever])
id2([Semantic Retriever]) -- Retrieved Docs ---> id5([Reranking])
id3([MMR Retriever]) -- Retrieved Docs ---> id5([Reranking])
id4([BM25 Retriever]) -- Retrieved Docs ---> id5([Reranking])
id5([Reranking]) ---> id6(top-n docs)
Depending on the input query, each query can be forwarded to any one of the following retrievers,
- General OR/ORFS information
- OR tools and commands
- OR/ORFS installation
- OR Error Messages
- OpenSTA docs
- Yosys docs
- Klayout docs
The retrievers act as separate tools and can be accessed by the LLM's tool-calling capabilities.
The langgraph framework has been used to make effective use of the multiple retriever tools. Upon receiving a query, a routing LLM call classifies the query and forwards it to the corresponding retriever tool. Relevant documents are the queried from the vectorstore by the tool and sent to the LLM for response generation.
graph TD
__start__ --> router_agent
router_agent -.-> retrieve_general
router_agent -.-> retrieve_cmds
router_agent -.-> retrieve_install
router_agent -.-> retrieve_errinfo
router_agent -.-> retrieve_opensta
router_agent -.-> retrieve_yosys
router_agent -.-> retrieve_klayout
retrieve_general --> generate
retrieve_cmds --> generate
retrieve_install --> generate
retrieve_errinfo --> generate
retrieve_opensta --> generate
retrieve_yosys --> generate
retrieve_klayout --> generate
generate --> __end__
The RAG dataset used in this project can be found here.
To modify the dataset, please refer to build_docs.py
make format
make check
ORAssistant is currently hosted at this link. Feel free to use the application and provide feedback!
This work is completed as part of the Google Summer of Code 2024 project under the UCSC Open-Source Program Office. Please see their contributions at this link.
2024-10-27: Our work has been accepted to Workshop on Open-Source EDA Technology 2024. Kudos to our talented contributors Palaniappan R and Aviral Kaintura (contributed equally)!
If you use this software in any published work, we would appreciate a citation! Please use the following reference:
@article{kaintura2024orassistant,
title={ORAssistant: A Custom RAG-based Conversational Assistant for OpenROAD},
author={Kaintura, Aviral and R, Palaniappan and Luar, Shui Song and Almeida, Indira Iyer},
journal={arXiv preprint arXiv:2410.03845},
year={2024}
}
@misc{orassistant2024,
author = {Kaintura, Aviral and R, Palaniappan and Luar, Shui Song and Iyer, Indira},
title = {ORAssistant Repository},
year = {2024},
url = {https://github.com/The-OpenROAD-Project/ORAssistant},
note = {Accessed: 20xx-xx-xx}
}