Imagine a world where you can understand the world around you, simply by looking at it. With Google Glass Open Source Assistant, you can harness the power of open-source AI-powered vision to unlock new possibilities. By combining the capabilities of LLava Vision from Ollama and OpenAI's Transcription capabilities, you can turn your Google Glass into a powerful tool for understanding and interacting with the world.
To make Google Glass run Open Source Models, follow these simple steps:
- Install the Essentials: Download and install Tailscale on your hosted Ollama machine and Google Glass from F-Droid. Create an account, sign in on both Google Glass and the hosted machine. This allows you to remotely connect to your local Ollama. Copy the hosted machine's Tailscale IP Address
- Configure Your Ollama: Follow steps here to configure your Ollama host to the Tailscale IP Address: https://github.com/ollama/ollama/blob/main/docs/faq.md
- Run LLAMA VISION: Download LLAVA from Ollama on your host machine with the following args:
OLLAMA_HOST=<TAILSCALE_IP_ADDRESS>:11434 ollama pull llama3.2-vision:11b. - Start Ollama: Start Ollama on your host machine using the following args:
OLLAMA_HOST=<TAILSCALE_IP_ADDRESS>:11434 ollama serve. - Get Your OpenAI API Key: Generate an OpenAI API Key from the OpenAI platform.
- Turn Your API Key into a QR Code: Convert your API Key into a QR Code.
- Turn Your Tailscale IP Address into a QR Code: Convert your <TAILSCALE_IP_ADDRESS> into a QR Code.
- Install and Launch the GlassAssistant App: Install and launch the GlassAssistant app on your Google Glass.
- Scan the QR Code: Scan the generated QR Code to connect your GlassAssistant app to your OpenAI API Key.
- Scan the QR Code: Scan the generated QR Code to connect your GlassAssistant app to your Tailscale IP Address.
- Start Exploring: Hold the camera button and speak for a custom prompt, or tap Glass to use a default prompt. The results are vertically scrollable, and you can tap Glass or click the camera button to return to the Camera view.
