-
Notifications
You must be signed in to change notification settings - Fork 471
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
28 changed files
with
527 additions
and
379 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,23 +1,22 @@ | ||
| # Friend: Open-Source AI Wearable with 24h+ on single charge | ||
| <div align="center"> | ||
|
|
||
| # **Friend** | ||
|
|
||
|  | ||
| Open Source AI Wearable device that records everything you say, gives you proactive feedback and advice. 24h+ on single charge. | ||
|
|
||
| [](https://discord.gg/kEXXsnb5b3)   | ||
| [](https://opensource.org/licenses/MIT) | ||
|  | ||
|
|
||
| [](https://discord.gg/kEXXsnb5b3)     | ||
| [](https://opensource.org/licenses/MIT)    | ||
| [](https://github.com/BasedHardware/Friend) | ||
|
|
||
| <div align="center"> | ||
| <h3> | ||
|
|
||
| [Homepage](https://basedhardware.com/) | [Documentation](https://basedhardware.github.io/Friend/) | [Buy Assembled Device](https://buy.stripe.com/6oE04101u6yx8Wk7ss) | ||
|
|
||
| </h3> | ||
| </div> | ||
|
|
||
|
|
||
| Friend is an AI wearable device that records everything you say, gives you proactive feedback and advice. | ||
|
|
||
|
|
||
| </div> | ||
|
|
||
| ## Features | ||
|
|
||
|
|
@@ -26,186 +25,24 @@ Friend is an AI wearable device that records everything you say, gives you proac | |
| - **Open-Source Software**: Access and contribute to the pin's software stack, designed with openness and community collaboration in mind. | ||
| - **Wearable Design**: Experience unparalleled convenience with ergonomic and lightweight design, perfect for everyday wear. | ||
|
|
||
| ## How it works | ||
|
|
||
| ```mermaid | ||
| graph TD; | ||
| A[Device] -- Streams Audio --> B[Phone App]; | ||
| B -- Transmits --> C[Deepgram]; | ||
| C -- Returns Transcript --> D[Phone App]; | ||
| D -- Saves Transcript --> E[Phone Storage]; | ||
| classDef lightMode fill:#FFFFFF, stroke:#333333, color:#333333; | ||
| classDef darkMode fill:#333333, stroke:#FFFFFF, color:#FFFFFF; | ||
| classDef lightModeLinks stroke:#333333; | ||
| classDef darkModeLinks stroke:#FFFFFF; | ||
| class A,B,C,D,E lightMode | ||
| class A,B,C,D,E darkMode | ||
| linkStyle 0 stroke:#FF4136, stroke-width:2px | ||
| linkStyle 1 stroke:#1ABC9C, stroke-width:2px | ||
| linkStyle 2 stroke:#FFCC00, stroke-width:2px | ||
| linkStyle 3 stroke:#2ECC40, stroke-width:2px | ||
| ``` | ||
|
|
||
| ## Structure | ||
|
|
||
| There are 3 different apps in these repositories located in different branches and folders. Our goal is to merge them into one big project. | ||
|
|
||
| Folder "AppStandalone": Standalone version of the app that doesn't require any hardware to use it. | ||
|
|
||
| Folder "AppWithWearable": Wearable-connected version of the app that requires the "Friend necklace" to use it. | ||
|
|
||
| Folder "AppWithWearableReactNative": Wearable-connected version of the app that is built in React native | ||
|
|
||
| ## Hardware Buying Guide (<$20 total) | ||
|
|
||
| - Board: Seeed Studio XIAO nRF52840 Sense | $15 on [Seedstudio](https://www.seeedstudio.com/Seeed-XIAO-BLE-Sense-nRF52840-p-5253.html) and $24 on [Amazon](https://amzn.to/3TZD1pO) and also [link for europe ](https://amzn.eu/d/3eG6gaA) and [India](https://robu.in/product/seeed-studio-xiao-nrf52840-sense-tinyml-tensorflow-lite-imu-microphone-bluetooth5-0/) | <$15/unit | ||
|
|
||
| - Rechargeable Battery on [Amazon](https://amzn.to/3TXlE9f) $7 | ||
| - Slider switch | $7 for 30+ units [on Amazon](https://www.amazon.com/Tnuocke-Vertical-Position-Latching-SS12F15-G5/dp/B099N3HFPG/ref=sr_1_2?dib=eyJ2IjoiMSJ9.vWYaZHNA7Z38_YnK7oxLKHvVPX-koqNn5CYGqZXKURCQso_zrAwckot4_h8c77Kgy2-m8FcrQymrZff0rlZIGdECJFA5Rwc5EQObrZ5wDb9zjnwVjonhSZfHlyM-KkJKO4_E6mKcC_I0vZg2vS1PBPkNSAXj9H9pTVK3D2iVtuoMNsxvAEwrYUPx3gYOiIjMOzJYoj8aHPmy2W1R4bWcPWp8IvhbO1GO29TT0jVE97U.ZavNMHkM9KYGMLSB_3DREpSJjhQ16_cjqOpo7aPAfHY&dib_tag=se&keywords=slider%2Bswitch&qid=1711332053&sr=8-2&th=1) and also [link for europe ](https://amzn.eu/d/9U0gjPB) but you only need one | <$1/unit | ||
|
|
||
| - Wires. I didn't try [these ones ](https://www.amazon.com/dp/B09X4629C1)but they should work | ||
|
|
||
| - 3D-print design of the case is located [here](https://github.com/BasedHardware/Friend/tree/main/assets/3d_printing_designs) | ||
|
|
||
| ## Getting Started | ||
|
|
||
| Follow these steps to get started with your Friend. | ||
|
|
||
| ### Install the app | ||
|
|
||
| 1. Clone the repo `git clone https://github.com/BasedHardware/friend.git` | ||
| 2. Choose which version of the app you want to install (see Structure). | ||
| - Don't have the device? run `cd apps/AppStandalone` in terminal | ||
| - Have the device/NRF Board? run `cd apps/AppWithWearable` in terminal | ||
| 3. Install [Flutter](https://docs.flutter.dev/get-started/install/macos/mobile-ios?tab=download) and [CocoaPods](https://guides.cocoapods.org/using/getting-started.html) | ||
| 4. Install your environment variables | ||
|
|
||
| - For AppWithWearable, open file ble_receive_w_a_v.dart located in `apps/AppWithWearable/lib/custom_code/actions/` Find "DEEPGRAM_API_KEY" and provide your own api-key for Deepgram for transcriptions to work | ||
|
|
||
| <img src="https://github.com/BasedHardware/Friend/assets/43514161/d0fb89d2-07fd-44e3-8563-68f938bb2319" alt="CleanShot 2024-03-25 at 21 58 42" width="400"> | ||
|
|
||
| then, go to apps/AppWithWearable/lib/custom_code/actions and in the "stream_api_response" file, add your openai key instead of "<key>" | ||
|  | ||
|
|
||
| - For AppStandalone, update variables in in .env.template file | ||
|
|
||
| 5. iOS: [Install XCode](https://apps.apple.com/us/app/xcode/id497799835?mt=12) then navigate to the iOS folder. You might need to launch Xcode to select a team and specify a bundle identifier. | ||
| Android: Download/install [android Studio ](https://developer.android.com/studio) then navigate to the Android folder | ||
| Don't run in web/simulator: Bluetooth will not work | ||
| 6. Run `flutter clean ` then `flutter pub get` then `pod install` | ||
| 7. When everything is installed, run `flutter run `, this should run your app on a selected device | ||
|
|
||
| **No-Code Alternative**: | ||
|
|
||
| - Don't have the device? [Clone this Flutterflow Project ](https://app.flutterflow.io/project/friend-0x9u40) | ||
| - Have the wearable device? [Copy this Flutterflow Project](https://app.flutterflow.io/project/friend-share-19bk3d) | ||
|
|
||
| # Install Firmware | ||
|
|
||
| Follow these steps to install the firmware: | ||
|
|
||
| We have uploaded a build of the firmware in ``firmware/Build``, that will always contain the most up to date build of the firmware. To use this firmware please download it and follow step 6 below. If you would like to build the firmware yourself please follow all the steps below. | ||
|
|
||
| 1. Set up nRF Connect by following the tutorial in this video: [https://youtu.be/EAJdOqsL9m8](https://youtu.be/EAJdOqsL9m8?feature=shared) | ||
|
|
||
| 2. In the nRF Connect Extension inside your VS Code, click "Open an existing application" and open the `firmware` folder from the root of this repo. | ||
|
|
||
| <img src="docs/images/vscode_extension.png" alt="VS Code Extension" width="200"> | ||
|
|
||
| 3. In the application panel of the extension, click the "Add Build Configuration" icon. | ||
|
|
||
| <img src="docs/images/addbuild.png" alt="Add Build Configuration" width="200"> | ||
|
|
||
| 4. Choose the board as "xiao_ble_sense" and select the configuration as "prj.conf". Then, click "Build Configuration". | ||
|
|
||
| <img src="docs/images/build_settings.png" alt="Build Settings" width="400"> | ||
|
|
||
| 5. Once the build succeeds, you will find the `zephyr.uf2` file in the `firmware/build/zephyr` directory. | ||
|
|
||
| 6. Double-click on the reset button of the device. The device will appear on your computer as a disk. Drag and drop the `zephyr.uf2` file into it. | ||
|
|
||
| > **Note:** On a Mac, you might see an error message after dropping the file, indicating that the process did not complete. This is just a Mac-specific error; the firmware is successfully uploaded. | ||
| <img src="docs/images/pinout.jpg" alt="Pinout" width="300"> | ||
|
|
||
| That's it! You have successfully installed the firmware on your device. | ||
|
|
||
| ### Testing Audio Recording on Your Computer | ||
|
|
||
| Follow these steps to test audio recording on your computer using a Python script: | ||
|
|
||
| 1. Open your terminal and navigate to the project's root directory. | ||
|
|
||
| 2. Change to the "test" folder: | ||
|
|
||
| ``` | ||
| cd test | ||
| ``` | ||
|
|
||
| 3. Install the required Python modules: | ||
|
|
||
| ``` | ||
| pip install -r requirements.txt | ||
| ``` | ||
|
|
||
| 4. Run the Python script: | ||
|
|
||
| ``` | ||
| python local_laptop_client.py | ||
| ``` | ||
|
|
||
| The recorded audio files will be stored periodically in the `test/recordings` directory. | ||
|
|
||
| That's it! You have now set up and tested audio recording on your computer. | ||
|
|
||
| ### Assemble the device | ||
|
|
||
| **Step 0:** Make sure you have bought everything from the buying guide above | ||
|
|
||
| <img src="https://github.com/BasedHardware/Friend/assets/43514161/fdc7f8bd-6205-49a8-aa31-ea4ef6655ba4" alt="Components" width="300"> | ||
|
|
||
| **Step 1:** You need to design the case using 3D printer. Find .stl file [here](https://github.com/BasedHardware/Friend/blob/main/3d-printing%20designs/Cover%20%2B%20Case.stl). If you don't know how to do it, send this file to someone who has a 3d printer | ||
|
|
||
| **Step 2:** | ||
| Solder everything together like on the picture below. using a soldering kit. Don't have it? buy [this one for $9](https://a.co/d/0XdthUV) | ||
|
|
||
| <img src="https://github.com/BasedHardware/Friend/assets/43514161/c254668c-1662-412f-8b2c-05a97fb68419" alt="Soldering" width="400"> | ||
|
|
||
| <img src="https://github.com/BasedHardware/Friend/assets/43514161/5fe4cb81-eb64-41c6-b24c-e2da104b465e" alt="Soldered" width="300"> | ||
|
|
||
| **Step 3:** | ||
| Fit everything in the case. Biggest hole is for the usb port. In my example, I put the battery first, then the board and then the switch, however it's not an ideal design. If you will figure out a better solution, please contribute! | ||
|
|
||
| <img src="https://github.com/BasedHardware/Friend/assets/43514161/4abae04c-2477-4b9a-a74c-077a463f4c29" alt="Assembled" width="300"> | ||
|
|
||
| **Step 4:** Use hot glue to attach the lid to the case. You can also use a scotch tape first for testing purposes. Last, on the USB-port side, you'll find 2 small round holes. This is where the thread should go through. | ||
|
|
||
| <img src="https://github.com/BasedHardware/Friend/assets/43514161/2ffcfbf4-6637-4bb6-89e5-bd75cf78eebd" alt="Lid" width="200"> | ||
|
|
||
| Congratulations! you now have a fully working and assembled device! | ||
|
|
||
| ## Contributing | ||
|
|
||
| [Join our Discord!](https://discord.gg/kEXXsnb5b3) | ||
| We welcome contributions from the community! If you're interested in improving Friend, our current biggest goal is to combine both apps together (AppStandalone with AppWithWearable). | ||
|
|
||
| - Standalone App brings great prompts and rich structure | ||
| - AppWithWearable brings simple bluetooth connecting functionality | ||
|
|
||
| ## Support | ||
|
|
||
| For open-source support, please open an issue on GitHub and/or ask in our [Discord Community](https://discord.gg/kEXXsnb5b3). For commercial support, license inquiries, or any other questions, please contact us directly at [[email protected]](mailto:[email protected]). | ||
| ## Get Started with our Documentation: | ||
| - [Introduction](https://basedhardware.github.io/Friend/) | ||
| - [App setup](https://basedhardware.github.io/Friend/get_started/Setup/) | ||
| - [Buying Guide](https://basedhardware.github.io/Friend/assembly/Buying_Guide/) | ||
| - [Build the device](https://basedhardware.github.io/Friend/assembly/Build_the_device/) | ||
| - [Install firmware](https://basedhardware.github.io/Friend/assembly/Install_firmware/) | ||
|
|
||
| ## Disclaimer | ||
| ## More links: | ||
| - [Contributing](https://basedhardware.github.io/Friend/info/Contribution/) | ||
| - [Support](https://basedhardware.github.io/Friend/info/Support/) | ||
|
|
||
| Please note that the Friend is a prototype project and is provided "as is", without warranty of any kind. Use of the device should comply with all local laws and regulations concerning privacy and data protection. | ||
| ## Made by the Community, with -❤️-: | ||
|
|
||
| Thank you for your interest in Friend, the open-source AI wearable. We're excited to see what you'll build with it! | ||
| <a href="https://github.com/BasedHardware/Friend/graphs/contributors"> | ||
| <img src="https://contrib.rocks/image?repo=BasedHardware/Friend" /> | ||
| </a> | ||
|
|
||
| ## Licensing | ||
|
|
||
| Friend is available under MIT License | ||
| Friend is available under MIT License | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| layout: default | ||
| title: Build the device | ||
| nav_order: 2 | ||
| --- | ||
|
|
||
| # Build the device | ||
|
|
||
| If you purchased an unassembled Friend device or built it yourself using our hardware guide, follow the steps below to flash the firmware: | ||
|
|
||
| {: .note } | ||
| Important: If you purchased an assembled device please skip this step | ||
|
|
||
|
|
||
| **Step 0:** Make sure you have bought everything from the buying guide above | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/build_device_snapshot_1.jpeg" alt="Components" width="300"> | ||
|
|
||
| **Step 1:** You need to design the case using 3D printer. Find .stl file [here](https://github.com/BasedHardware/Friend/blob/main/3d-printing%20designs/Cover%20%2B%20Case.stl). If you don't know how to do it, send this file to someone who has a 3d printer | ||
|
|
||
| **Step 2:** | ||
| Solder everything together like on the picture below. using a soldering kit. Don't have it? buy [this one for $9](https://a.co/d/0XdthUV) | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/build_device_snapshot_2.png" alt="Soldering" width="400"> | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/build_device_snapshot_3.jpeg" alt="Soldered" width="300"> | ||
|
|
||
| **Step 3:** | ||
| Fit everything in the case. Biggest hole is for the usb port. In my example, I put the battery first, then the board and then the switch, however it's not an ideal design. If you will figure out a better solution, please contribute! | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/build_device_snapshot_4.jpeg" alt="Assembled" width="300"> | ||
|
|
||
| **Step 4:** Use hot glue to attach the lid to the case. You can also use a scotch tape first for testing purposes. Last, on the USB-port side, you'll find 2 small round holes. This is where the thread should go through. | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/build_device_snapshot_5.jpeg" alt="Lid" width="200"> | ||
|
|
||
| Congratulations! you now have a fully working and assembled device! | ||
|
|
||
|
|
||
| [Next Step: Install Firmware →](https://basedhardware.github.io/Friend/assembly/Install_firmware/){: .btn .btn-purple } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| --- | ||
| layout: default | ||
| title: Buying Guide | ||
| nav_order: 1 | ||
| --- | ||
|
|
||
| ### Buying Guide | ||
|
|
||
| You have two options: | ||
|
|
||
| 1. Buying an assembled device: | ||
| [Buy Assembled Device](https://buy.stripe.com/6oE04101u6yx8Wk7ss){: .btn .btn-green .mr-4 } | ||
|
|
||
| 2. Assemble the Device yourself with off-the-shelf components: | ||
|
|
||
| - Board: Seeed Studio XIAO nRF52840 Sense | $15 on [Seedstudio](https://www.seeedstudio.com/Seeed-XIAO-BLE-Sense-nRF52840-p-5253.html) and $24 on [Amazon](https://amzn.to/3TZD1pO) and also [link for europe ](https://amzn.eu/d/3eG6gaA) and [India](https://robu.in/product/seeed-studio-xiao-nrf52840-sense-tinyml-tensorflow-lite-imu-microphone-bluetooth5-0/) | <$15/unit | ||
|
|
||
| - Rechargeable Battery on [Amazon](https://amzn.to/3TXlE9f) $7 | ||
| - Slider switch | $7 for 30+ units [on Amazon](https://www.amazon.com/Tnuocke-Vertical-Position-Latching-SS12F15-G5/dp/B099N3HFPG/ref=sr_1_2?dib=eyJ2IjoiMSJ9.vWYaZHNA7Z38_YnK7oxLKHvVPX-koqNn5CYGqZXKURCQso_zrAwckot4_h8c77Kgy2-m8FcrQymrZff0rlZIGdECJFA5Rwc5EQObrZ5wDb9zjnwVjonhSZfHlyM-KkJKO4_E6mKcC_I0vZg2vS1PBPkNSAXj9H9pTVK3D2iVtuoMNsxvAEwrYUPx3gYOiIjMOzJYoj8aHPmy2W1R4bWcPWp8IvhbO1GO29TT0jVE97U.ZavNMHkM9KYGMLSB_3DREpSJjhQ16_cjqOpo7aPAfHY&dib_tag=se&keywords=slider%2Bswitch&qid=1711332053&sr=8-2&th=1) and also [link for europe ](https://amzn.eu/d/9U0gjPB) but you only need one | <$1/unit | ||
|
|
||
| - Wires. I didn't try [these ones ](https://www.amazon.com/dp/B09X4629C1)but they should work | ||
|
|
||
| - 3D-print design of the case is located [here](https://github.com/BasedHardware/Friend/tree/main/assets/3d_printing_designs) | ||
|
|
||
| [Next Step: Build the device →](https://basedhardware.github.io/Friend/assembly/Build_the_device/){: .btn .btn-purple } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| --- | ||
| layout: default | ||
| title: Install firmware | ||
| nav_order: 3 | ||
| --- | ||
|
|
||
| # Install firmware | ||
|
|
||
| If you purchased an unassembled Friend device or built it yourself using our hardware guide, follow the steps below to flash the firmware: | ||
|
|
||
| {: .note } | ||
| Important: If you purchased an assembled device please skip this step | ||
|
|
||
|
|
||
| We have uploaded a build of the firmware in `firmware/Build`, that will always contain the most up to date build of the firmware. To use this firmware please download it and follow step 6 below. If you would like to build the firmware yourself please follow all the steps below. | ||
|
|
||
| 1. Set up nRF Connect by following the tutorial in this video: [https://youtu.be/EAJdOqsL9m8](https://youtu.be/EAJdOqsL9m8) | ||
|
|
||
| 2. In the nRF Connect Extension inside your VS Code, click "Open an existing application" and open the `firmware` folder from the root of this repo. | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/install_firmware_screenshot_1.png" alt="install_firmware_screenshot_1" width="100%"> | ||
|
|
||
|
|
||
| 3. In the application panel of the extension, click the "Add Build Configuration" icon. | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/addbuild.png" alt="Add Build Configuration" width="200"> | ||
|
|
||
| 4. Choose the board as "xiao_ble_sense" and select the configuration as "prj.conf". Then, click "Build Configuration". | ||
|
|
||
| <img src="https://basedhardware.github.io/Friend/images/build_settings.png" alt="Build Settings" width="400"> | ||
|
|
||
| 5. Once the build succeeds, you will find the `zephyr.uf2` file in the `firmware/build/zephyr` directory. | ||
|
|
||
| 6. Double-click on the reset button of the device. The device will appear on your computer as a disk. Drag and drop the `zephyr.uf2` file into it. | ||
|
|
||
| > **Note:** On a Mac, you might see an error message after dropping the file, indicating that the process did not complete. This is just a Mac-specific error; the firmware is successfully uploaded. | ||
| <img src="https://basedhardware.github.io/Friend/images/pinout.jpg" alt="Pinout" width="300"> | ||
|
|
||
| That's it! You have successfully installed the firmware on your device. | ||
|
|
||
| [Next Step: Audio Test →](https://basedhardware.github.io/Friend/assembly/audio_test/){: .btn .btn-purple } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| --- | ||
| layout: default | ||
| title: Audio Test | ||
| nav_order: 4 | ||
| --- | ||
|
|
||
| # Audio Test | ||
|
|
||
| Follow these steps to test audio recording on your computer using a Python script: | ||
|
|
||
| 1. Open your terminal and navigate to the project's root directory. | ||
|
|
||
| 2. Change to the "test" folder: | ||
|
|
||
| ``` | ||
| cd test | ||
| ``` | ||
|
|
||
| 3. Install the required Python modules: | ||
|
|
||
| ``` | ||
| pip install -r requirements.txt | ||
| ``` | ||
|
|
||
| 4. Run the Python script: | ||
|
|
||
| ``` | ||
| python local_laptop_client.py | ||
| ``` | ||
|
|
||
| The recorded audio files will be stored periodically in the `test/recordings` directory. | ||
|
|
||
| That's it! You have now set up and tested audio recording on your computer. | ||
|
|
Oops, something went wrong.