Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
laviRZ committed Jan 16, 2024
1 parent 9042312 commit 9c87aa1
Showing 1 changed file with 15 additions and 3 deletions.
18 changes: 15 additions & 3 deletions rknn-readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,18 @@ This is a fork of PhotonVision. It uses the NPU on the RK3588 processor to run o
The expected performance is 60 FPS with a 640x640 model.

## Installation
1. Install Ubuntu 22.04 on the board. For an Orange Pi 5, I recommend [this unofficial project](https://github.com/Joshua-Riek/ubuntu-rockchip/releases/latest). The default username and password for that image is ubuntu:ubuntu, and for the [official images](https://drive.google.com/drive/folders/1i5zQOg1GIA4_VNGikFl2nPM0Y2MBw2M0), it's orangepi:orangepi. Make sure to use an Ubuntu 22.04 (Jammy) "server" image.
### Option 1: Flash the board with a pre-installed image (For Orange Pi 5 only):
1. Download the latest Orange Pi 5 image from the [releases page](https://github.com/lavirz/photonvision/releases/latest).
2. Flash the image to an SD card using [Balena Etcher](https://www.balena.io/etcher/).
3. Insert the SD card into the board, and turn it on. The default username and password for that image is ubuntu:ubuntu.
4. That's it! PhotonVision with the RKNN pipeline should be installed and running on the board.
### Option 2: Install PhotonVision on an fresh Ubuntu 22.04 installation:
1. Install Ubuntu 22.04 on the board. For an Orange Pi 5, I recommend [this unofficial project](https://github.com/Joshua-Riek/ubuntu-rockchip/releases/latest). The default username and password for that image is ubuntu:ubuntu, and for the [official images](https://drive.google.com/drive/folders/1i5zQOg1GIA4_VNGikFl2nPM0Y2MBw2M0), it's orangepi:orangepi. Make sure to use an Ubuntu 22.04 (Jammy) server image.
2. Install PhotonVision using the [official instructions](https://docs.photonvision.org/en/latest/getting-started/installation.html). This requires you to ssh into the board, which requires you to have the board's IP address. You can do this by connecting the board to a monitor and keyboard and run the `ifconfig` command, or by using a tool like [Angry IP Scanner](https://angryip.org/) and be connected to the same network as the board, such as the robot's network.
3. This gets you to a "stock" PhotonVision installation. In order to get the object detection working, download the jar, and upload it using the "offline update" button in the setings page:
3. This gets you to a "stock" PhotonVision installation. In order to get the object detection working, download the latest linuxarm64 jar from the [releases page](https://github.com/lavirz/photonvision/releases/latest) and upload it using the "offline update" button in the setings page:
<img src="https://i.postimg.cc/50sHKZbk/pvss-offline-update.png" width="800"/>
4. That's it! after uploading the jar, PhotonVision will restart automatically.
1. That's it! after uploading the jar, PhotonVision will restart automatically, and the RKNN pipeline should be available.


## Usage
1. Go to the "Vision" tab, and click "Add Pipeline". Select "RKNN" as the pipeline type. Alternatively, you can edit an existing pipeline and change the type to "RKNN".
Expand All @@ -19,3 +26,8 @@ The expected performance is 60 FPS with a 640x640 model.
- Make sure to use a low enough exposure so that 1000/exposure > the camera fps. For example, if the camera is running at 60 FPS, the exposure should be 16.6ms or lower. If the exposure is too high, the camera will output frames slower than the chosen FPS, and the latency will be higher.
- When the "Processed" tab is hidden, the code doesn't draw the boxes on the image, which saves up to 1ms of latency, thus recommended when used in a competition.
- When the selected stream resolution is not the resolution of the camera input, the code will resize the image, which adds latency. If the camera is used as a driver camera, it is recommended to change it to a lower resolution so the stream would run smoother, but if not, it's best to keep the stream resolution the same as the camera resolution, so the code doesn't resize the image, and save latency.

## Custom models
If you want to use your own model YOLOv5/8, you can train one and export it to the RKNN format using these Kaggle notebooks. Kaggle is a free service that allows you to train models on the cloud, and it's very easy to use. You can also use your own computer to train the model, but it's recommended to use a GPU, and it's much easier to use Kaggle. Kaggle gives you 30 hours of free GPU time per week, which is more than enough to train a model. You can also use Google Colab, but it's a bit more complicated to use, and offers slower hardware.
- YOLOv5 <a href="https://www.kaggle.com/lavirz/yolov5-to-rknn"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
- YOLOv8 <a href="https://www.kaggle.com/lavirz/yolov8-to-rknn"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>

0 comments on commit 9c87aa1

Please sign in to comment.