Skip to content

A pipeline framework for developing video and image processing application. Supports multiple GPUs and Machine Learning tooklits

License

Notifications You must be signed in to change notification settings

Apra-Labs/ApraPipes

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Header Image

ApraPipes

A pipeline framework for developing video and image processing applications. Supports multiple GPUs and Machine Learning toolkits.
Learn more about ApraPipes here https://apra-labs.github.io/ApraPipes.

Build status

Aprapipes is automatically built and tested on Ubuntu (18.04 and 20.04), Jetson Boards (Jetpack 4.4) and Windows (11) x64 Visual Studio 2019 Community.

OS Version With Cuda Tests Status
Windows 2019 No Test Results CI-Win-NoCUDA
Windows 2019 Yes Test Results CI-Win-CUDA
Ubuntu x64_86 20.04 No Test Results CI-Linux-NoCUDA
Ubuntu x64_86 18.04 Yes Test Results CI-Linux-CUDA
Ubuntu ARM64 (Jetsons) 18.04 Yes Test Results CI-Linux-ARM64
Ubuntu x64_86-WSL 20.04 Yes Test Results CI-Linux-CUDA-wsl
Ubuntu x64_86-docker 18.04 Yes No CI-Linux-CUDA-Docker

Getting Started with ApraPipes

Please select your Operating System
  • Note : Make sure to clone using recursive flag
    git clone --recursive https://github.com/Apra-Labs/ApraPipes.git
    

Windows (Version ≥ 10)

Windows Logo

Requirements

Prerequisites

Visual Studio

  • Install Visual Studio 2019 Community
    • Install Desktop development C++
    • .NET Desktop development
    • Universal Windows Development Platform

Cuda

  • Create an account on developer.nvidia.com if you're not already a member. Note : Otherwise the next step will show HTTP 404/403 error.
  • Windows 10/11 : Cuda Toolkit 10.2 or CUDA Toolkit 11.8.

Cudnn

  • Download Cudnn and extract files where cuda is installed. Note: Please be aware that this process requires some effort. Here are the necessary steps:

    • Download the correct zip file matching your cuda version. Do not download the exe/installer/deb package.
    • Windows:
  • Clone with submodules and LFS.

    git clone --recursive https://github.com/Apra-Labs/ApraPipes.git
    
Build

Open PowerShell as an administrator and execute the following commands

Build Without Cuda

If your windows system does not have an NVIDIA GPU use this script

build_windows_no_cuda.bat

Build With Cuda

build_windows_cuda.bat

To Build With Documentation

build_windows_cuda.bat --build-doc
Test

Run Tests

  • list all tests
    _build/BUILD_TYPE/aprapipesut.exe --list_content
    
  • run all tests
    _build/BUILD_TYPE/aprapipesut.exe
    
  • run all tests disabling memory leak dumps and better progress logging
    _build/BUILD_TYPE/aprapipesut.exe -p -l all --detect_memory_leaks=0
    
  • run one test
    _build/BUILD_TYPE/aprapipesut.exe --run_test=filenamestrategy_tests/boostdirectorystrategy
    
  • run one test with arguments
    _build/BUILD_TYPE/aprapipesut.exe --run_test=unit_tests/params_test -- -ip 10.102.10.121 -data ArgusCamera
    
    • Look at the unit_tests/params_test to check for sample usage of parameters in test code.

Linux (Ubuntu ≥ 18.04)

Linux Logo

Requirements

Prerequisites

Cuda

  • Create an account on developer.nvidia.com if you're not already a member. Note : Otherwise the next step will show HTTP 404/403 error.
  • Ubuntu 18.04/20.04:
    18.04 - CUDA Toolkit 10.2
    20.04 - CUDA Toolkit 11.8

Cudnn

  • Download Cudnn

  • Linux:

  • Clone with submodules and LFS.

    git clone --recursive https://github.com/Apra-Labs/ApraPipes.git
    
Build
  • Run this command to make the script file executable.
chmod +x build_linux_*.sh

Build Without Cuda

If your windows system does not have an NVIDIA GPU use this script

./build_linux_no_cuda.sh

Build With Cuda

./build_linux_cuda.sh

To Build With Documentation

./build_linux_cuda.sh --build-doc

Build can take ~2 hours depending on the machine configuration.

Test

Run Tests

  • list all tests
    ./_build/aprapipesut --list_content
    
  • run all tests
    ./_build/aprapipesut
    
  • run all tests disabling memory leak dumps and better progress logging
    ./_build/aprapipesut -p -l all --detect_memory_leaks=0
    
  • run one test
    ./_build/aprapipesut --run_test=filenamestrategy_tests/boostdirectorystrategy
    
  • run one test with arguments
    ./_buildaprapipesut --run_test=unit_tests/params_test -- -ip 10.102.10.121 -data ArgusCamera
    
    • Look at the unit_tests/params_test to check for sample usage of parameters in test code.

Jetson Boards - Nano, TX2, NX, AGX (Jetpack ≥ 4.4)

Nvidia Logo

Requirements

Prerequisites

  • Setup the board with Jetpack 4.4 or higher as supported.

  • Clone with submodules and LFS.

    git clone --recursive https://github.com/Apra-Labs/ApraPipes.git
    
Build

Build for Jetson (Only Cuda Build)

  • Run this command to make the script file executable.
chmod +x build_jetson.sh
  • ApraPipes builds CUDA version on Jetson Boards.
./build_jetson.sh
  • To Build With Documentation
./build_jetson.sh --build-doc

Build can take ~12 hours on Jetson Nano. Note: Jetson build can also be done using Ubuntu 18.04 x86_64 Laptop via cross compilation.

Cross Compilation using qemu

Cross compilation using qemu

Conceptual steps adapted from here:

  • On any Intel Ubuntu 18.04 computer (physical or virtual including wsl ) mount a Jetson SD Card Image as described above
  • Copy relevant files from mounted image to created a rootfs
  • Install qemu on ubuntu host
  • chroot into emulated aarm64 environment using script provided in the github link above
  • install extra tools and build aprapipes and aprapipesut
  • the built aprapipesut can be copied to a Jetson board and run.

This approach can use all 12-16 cores of a laptop and hence builds faster.

Test

Run Tests

  • list all tests ./_build/aprapipesut --list_content
  • run all tests ./_build/aprapipesut
  • run one test ./_build/aprapipesut --run_test=filenamestrategy_tests/boostdirectorystrategy
  • run one test with arguments ./_build/aprapipesut --run_test=unit_tests/params_test -- -ip 10.102.10.121 -data ArgusCamera
  • Look at the unit_tests/params_test to check for sample usage of parameters in test code

Docker

Nvidia Logo

Requirements

Prerequisites

  • Ensure virtualization is enabled in both the BIOS settings of your computer and the Windows virtualization feature -Refer this article to enable them
  • Install WSL 2 on your system:
    wsl --install
    
  • Set WSL 2 as the default version using the command line:
    wsl --set-default-version 2
    
  • Install Ubuntu-18.04 from Microsoft store , Refer this article for any issues regarding installation
  • Install Docker Desktop on Windows -from here
  • Enable Docker integration with WSL 2 (in Docker Desktop settings -> Resources -> WSL integration -> Enable Ubuntu-18.04 -> Apply&restart)
  • Install nvida-container-toolkit using (WSL Ubuntu-18.04) for docker to access Host-system GPU -Follow this document to install nvidia-container-toolkit
  • Note:"Follow the exact instructions outlined in the document to ensure the correct and successful installation of the NVIDIA Container Toolkit"
Build

Build for Docker

docker pull ghcr.io/kumaakh/aprapipes-build-x86-ubutu18.04-cuda:last-good
  • Mount an external volume as a build area, and then use the Windows command line to create a Docker container using the above image with the following command:
    docker run -dit --gpus all -v "</path/to/external_volume>":"/mnt/b/" --name <give-container-name> a799cc26f4b7
    
    ..your command should look like this [where D:\ws\docker-pipes->local_folder_path , pipes->container_name ]
    docker run -dit --gpus all -v "D:\ws\docker-pipes":"/mnt/b/" --name pipes a799cc26f4b7
    
  • After creating the container, execute the following command to access its command line interface
    docker exec -it <container-name> /bin/bash
    
  • Note:"When inside the container, build all contents within the mounted external folder"
  • clone the repository with submodules and LFS as described above
  • build using build_linux_*.sh scripts as described above

This build will be fairly fast (~10 mins) as entire vcpkg cache comes down with the docker image

Update Submodules

git submodule update --init --recursive

Update Documentation

To update documentation, refer to Documentation Guidelines in the Contribution-Guidelines.

To regenerate documentation

Run,

./build_documentation.sh