From 79590926cf7e2bc65c706fda9d50bac8089f5fda Mon Sep 17 00:00:00 2001 From: lxowalle Date: Mon, 5 Feb 2024 14:18:29 +0800 Subject: [PATCH] Add mmf guide for rv nano --- .../lichee/RV_Nano/8_mmf_development_guide.md | 82 +++++++++++++++++ docs/hardware/en/sidebar.yaml | 2 + .../lichee/RV_Nano/8_mmf_development_guide.md | 90 +++++++++++++++++++ docs/hardware/zh/sidebar.yaml | 2 + 4 files changed, 176 insertions(+) create mode 100644 docs/hardware/en/lichee/RV_Nano/8_mmf_development_guide.md create mode 100644 docs/hardware/zh/lichee/RV_Nano/8_mmf_development_guide.md diff --git a/docs/hardware/en/lichee/RV_Nano/8_mmf_development_guide.md b/docs/hardware/en/lichee/RV_Nano/8_mmf_development_guide.md new file mode 100644 index 0000000000..357ea2c125 --- /dev/null +++ b/docs/hardware/en/lichee/RV_Nano/8_mmf_development_guide.md @@ -0,0 +1,82 @@ +--- +title: Multimedia Framework (MMF) Development Guide +keywords: riscv, licheerv, nano, mmf +update: + - date: 2024-02-04 + version: v0.1 + author: lxowalle +--- + +## Overview +This document is used to introduce the method of using SDK to develop MMF, to provide a development idea for developers who want to start developing MMF but can't get started. +MMF full name is Multimedia Framework, an framework that offers a unified API format for video input/output, audio input/output, image signal processing, and hardware encoding/decoding, allowing users to quickly implement multimedia-related functions by calling these APIs. + +## Build MMF development environment +Please refer to the method introduced in `LicheeRV Nano->System Development->cvi_mmf_sdk` to build the MMF development environment. + +Or follow the instructions below: + +```shell +Copy code +# Download dependencies +sudo apt install pkg-config build-essential ninja-build automake autoconf libtool wget curl git gcc libssl-dev bc slib squashfs-tools android-sdk-libsparse-utils android-sdk-ext4-utils jq cmake python3-distutils tclsh scons parallel ssh-client tree python3-dev python3-pip device-tree-compiler libssl-dev ssh cpio squashfs-tools fakeroot libncurses5 flex bison + +# Download SDK and toolchain +git clone https://github.com/sipeed/LicheeRV-Nano-Build.git +wget https://sophon-file.sophon.cn/sophon-prod-s3/drive/23/03/07/16/host-tools.tar.gz +tar xvf host-tools.tar.gz + +# Compile all examples with the build_middleware command +cd LicheeRV-Nano-Build +git checkout v4.1.0-licheervnano +ln -s ../host-tools ./ +source build/cvisetup.sh +defconfig cv1812cp_licheerv_nano_sd +build_middleware +``` + +The above instructions describe how to install the MMF-related compilation environment and how to compile the MMF examples provided by the SDK. + +- Note: You may fail to compile sample_cvg. If you do not need this example, try again after deleting the `LicheeRV-Nano-Build/middleware/v2/sample/cvg` folder. If you need this demo, try compiling with `build_all`, which requires compiling the entire SDK, so the compilation time will be longer. +## Development Document +Please refer to the `LicheeRV Nano->Board Introduction` to find most of the document. + +For MMF applications, pay attention to the following document: + +- [Media Processing Software Development Reference](https://doc.sophgo.com/cvitek-develop-docs/master/docs_latest_release/CV180x_CV181x/en/01.software/MPI/Media_Processing_Software_Development_Reference/build/html/index.html) +- [SDK LicheeRV-Nano-Build](https://github.com/sipeed/LicheeRV-Nano-Build) + +## Connecting to the Development Board via Network +The purpose of connecting to the development board via network is to upload the firmware we compiled to the board. + +Please refer to the methods in `LicheeRV Nano->peripheral` to get the IP address of the development board. Any one of the three methods can be implemented: Ethernet connection, WIFI connection, or USB RNDIS connection. + +## Compiling and Running an Example +MMF will use the hardware directly, so incorrect operation can lead to system crash, and you must be careful to pay attention to details when developing MMF. The suggestion is to modify your own program by using the examples. + +Compile and run sample_vio example: + +```shell +# Ensure that the basic MMF compilation environment has been set up and that build_middleware has been compiled once +# Compile sample_vio +cd middleware/v2/sample/vio +make + +# Upload to the development board (account root, password cvitek) +scp sample_vio root@xxx.xxx.xxx.xxx:/root # xxx.xxx.xxx.xxx is the board's IP address + +# Log into the development board +ssh root@xxx.xxx.xxx.xxx + +# If you need to use the display, you need to run fb_load.sh to make sure the driver is loaded (it only needs to be executed once) +/opt/fb_load.sh + +# Run the example +cd ~ +./sample_vio +``` +The instructions above explain how to compile a specific MMF example, and how to upload and run the example on the development board. Developers can modify the example based on their applications to eventually develop the functionality they desire. + +## Have an unsolvable problem? +1. Please remain patient and carefully review the development document to see if anything was overlooked. For example, check if the input parameters are correct, whether resources have been properly released, etc. +2. Post your problem on [maixhub](https://maixhub.com/discussion) or GitHub. Please organize the functionality you want to achieve, the problems you encountered, the solutions you tried, and the ways to reproduce them, by the way, many times can be solved in the process of organizing your thoughts. \ No newline at end of file diff --git a/docs/hardware/en/sidebar.yaml b/docs/hardware/en/sidebar.yaml index f0c5e182e0..c76c84d05a 100644 --- a/docs/hardware/en/sidebar.yaml +++ b/docs/hardware/en/sidebar.yaml @@ -197,6 +197,8 @@ items: file: lichee/RV_Nano/6_develop_mainline.md - label: test file: lichee/RV_Nano/7_test_report.md + - label: MMF Development Guide + file: lichee/RV_Nano/8_mmf_development_guide.md - label: Lichee Zero file: lichee/Zero/Zero.md - label: Lichee Nano diff --git a/docs/hardware/zh/lichee/RV_Nano/8_mmf_development_guide.md b/docs/hardware/zh/lichee/RV_Nano/8_mmf_development_guide.md new file mode 100644 index 0000000000..f96f35dfd3 --- /dev/null +++ b/docs/hardware/zh/lichee/RV_Nano/8_mmf_development_guide.md @@ -0,0 +1,90 @@ +--- +title: 多媒体框架（MMF）开发指南 +keywords: riscv, licheerv, nano, mmf +update: + - date: 2024-02-04 + version: v0.1 + author: lxowalle +--- + +## 简述 + +本文用来介绍使用算能SDK开发MMF的方法，为想要上手开发MMF却无从下手的开发者提供一个开发思路。 +MMF全称是多媒体软件架构（Multimedia Framework），这个架构将视频输入输出、音频输入输出、图像信号处理、硬件编解码等功能提供了统一格式的API，用户可以通过调用封装好的API来快速实现多媒体相关的功能。 + +## 搭建MMF的开发环境 + +请参考`LicheeRV Nano->系统开发->cvi_mmf_sdk`中介绍的方法来搭建MMF的开发环境 + +或者参考下面的指令 + +```shell +# 下载依赖 +sudo apt install pkg-config build-essential ninja-build automake autoconf libtool wget curl git gcc libssl-dev bc slib squashfs-tools android-sdk-libsparse-utils android-sdk-ext4-utils jq cmake python3-distutils tclsh scons parallel ssh-client tree python3-dev python3-pip device-tree-compiler libssl-dev ssh cpio squashfs-tools fakeroot libncurses5 flex bison + +# 下载sdk和工具链 +git clone https://github.com/sipeed/LicheeRV-Nano-Build.git +wget https://sophon-file.sophon.cn/sophon-prod-s3/drive/23/03/07/16/host-tools.tar.gz +tar xvf host-tools.tar.gz + +# 用build_middleware命令编译所有示例 +cd LicheeRV-Nano-Build +git checkout v4.1.0-licheervnano +ln -s ../host-tools ./ +source build/cvisetup.sh +defconfig cv1812cp_licheerv_nano_sd +build_middleware +``` + +上述指令介绍了如何安装MMF相关的编译环境，以及如何编译SDK提供的MMF示例。 + +- 注意：你可能在编译sample_cvg时会失败，如果不需要使用这个示例，则删掉`LicheeRV-Nano-Build/middleware/v2/sample/cvg`文件夹后再重试。如果你需要这个demo，则使用build_all编译后再尝试，build_all需要编译整个sdk，因此编译时间较长 + +## 开发资料 + +请参考`LicheeRV Nano->板卡介绍`中来找到大部分资料 + +对于MMF应用可以着重看下面的资料 + +- [MMF媒体软件开发指南](https://doc.sophgo.com/cvitek-develop-docs/master/docs_latest_release/CV180x_CV181x/zh/01.software/MPI/Media_Processing_Software_Development_Reference/build/html/index.html)或者[Media Processing Software Development Reference](https://doc.sophgo.com/cvitek-develop-docs/master/docs_latest_release/CV180x_CV181x/en/01.software/MPI/Media_Processing_Software_Development_Reference/build/html/index.html) +- [SDK LicheeRV-Nano-Build](https://github.com/sipeed/LicheeRV-Nano-Build) + +## 通过网络连接开发板 + +通过网络连接开发板的目的是为了让我们编译的固件可以被上传到板子中。 + +请参考`LicheeRV Nano->外设使用`中的方法来获取开发板的IP地址，三种方法任意实现其中一种即可：`网线连接`、`WIFI连接`、`USB RNDIS连接` + +## 编译和运行一个示例 + +MMF内部封装了直接操作硬件的方法，因此稍有疏忽就有导致系统崩溃的可能，开发MMF一定要注意保持耐心和注重细节。建议是通过示例来修改出自己的程序。 + +以sample_vio为例 + +```shell +# 确保已经搭建好了MMF的基本编译环境，并且已经使用build_middleware编译过一次 +# 编译sample_vio +cd middleware/v2/sample/vio +make + +# 上传到开发板(账户root，密码cvitek) +scp sample_vio root@xxx.xxx.xxx.xxx:/root # xxx.xxx.xxx.xxx是板子的IP地址 + +# 登入开发板 +ssh root@xxx.xxx.xxx.xxx + +# 如果需要使用显示屏，则需要运行fb_load.sh以确保加载了驱动（只需要执行一次） +/opt/fb_load.sh + +# 运行示例 +cd ~ +./sample_vio +``` + +上述指令介绍了如何编译某一个MMF示例，以及上传示例到开发板和运行这个示例。开发者可以根据自己的应用来修改示例，最后开发出自己想要实现的功能。 + +## 出现无法解决的问题怎么办 + +1. 请保持耐心、并仔细的参阅开发资料，尝试找到是否存在遗漏的地方。比如输入参数是否正确、资源是否真正的释放等。 +2. 在[maixhub](https://maixhub.com/discussion)或github上发布你的问题。请整理好你想要实现的功能、遇到的问题、尝试过的解决方案以及复现方法，顺便一提，很多时候在整理思路的过程中就能找到解决的方法了。 + diff --git a/docs/hardware/zh/sidebar.yaml b/docs/hardware/zh/sidebar.yaml index bea58f1779..c73c5b707b 100644 --- a/docs/hardware/zh/sidebar.yaml +++ b/docs/hardware/zh/sidebar.yaml @@ -215,6 +215,8 @@ items: file: lichee/RV_Nano/6_develop_mainline.md - label: 板卡测试 file: lichee/RV_Nano/7_test_report.md + - label: MMF开发指南 + file: lichee/RV_Nano/8_mmf_development_guide.md - label: LicheePI Zero file: lichee/Zero/Zero.md - label: LicheePI Nano