This repo contains the implementation of ICG-Net. For the Benchmark and Checkpoints, please refer to the Benchmark Repository.
To get a local copy up and running follow these steps:
Pytorch 2.2, Cuda 12.1
To install MinkowskiEngine with Pytorch >= 2.0 and Cuda >= 12.0, you will need to install our patched version of MinkowskiEngine.sudo apt-get install libopenblas-dev
export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"
# Install conda environment
conda env create -f conda_cu121.yml
conda activate icg_cuda121
# Install Pytorch and Dependencies
pip install torch torchvision torchaudio torch_geometric
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.2.0+cu121.html
# Install Patched MinkowskiEngine
git clone [email protected]:renezurbruegg/MinkowskiEngine.git
cd MinkowskiEngine
# Link conda cuda version for minkowski compilation
export CUDA_HOME=${CONDA_PREFIX}/envs/icg_cuda121
# Link cuda libraries.
export LD_LIBRARY_PATH=${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH}
# Link cuda headers. This might not be necessary for all systems.
sudo ln -s ${CONDA_PREFIX}/lib/libcudart.so.12 /usr/lib/libcudart.so
python setup.py install --force_cuda --blas=openblas --blas_include_dirs=${CONDA_PREFIX}/include --cuda_home=$CUDA_HOME
cd -
# Install third party requirements
cd icg_net/third_party/pointnet2
python setup.py install
cd -
# Install icg_net as pip package
pip install -e .Pytorch 1.12 Cuda 11.3
sudo apt-get install libopenblas-dev
# Install conda environment
export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"
conda env create -f conda_cu113.yml
conda activate icg_cuda113
# Install Pytorch and Dependencies
pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
pip install torch_geometric
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-1.12.0+cu113.html
# Install MinkowskiEngine
git clone --recursive "https://github.com/NVIDIA/MinkowskiEngine"
cd MinkowskiEngine
git checkout 02fc608bea4c0549b0a7b00ca1bf15dee4a0b228
python setup.py install --force_cuda --blas=openblas
# Install third party requirements
cd icg_net/third_party/pointnet2
python setup.py install
cd -
# Install icg_net as pip package
pip install -e .from icg_net import ICGNetModule, get_model
from icg_net.typing import ModelPredOut
# Load model
model: ICGNetModule = get_model(
"icg_benchmark/data/51--0.656/config.yaml",
device="cuda" if torch.cuda.is_available() else "cpu",
)
pc, normals = # ... load pc
out: ModelPredOut = model(
torch.from_numpy(np.asarray(o3dc.points)).float(),
normals=torch.from_numpy(np.asarray(o3dc.normals)).float(),
grasp_pts=grasp_pts,
grasp_normals=grasp_normals,
n_grasps=512,
each_object=True,
return_meshes=True,
return_scene_grasps=True,
)
print(out)Distributed under the BSD-2 License. See LICENSE.txt for more information.
If you use this code in your research, please cite the following paper:
@article{zurbrugg2024icgnet,
title={ICGNet: A Unified Approach for Instance-Centric Grasping},
author={Zurbr{\"u}gg, Ren{\'e} and Liu, Yifan and Engelmann, Francis and Kumar, Suryansh and Hutter, Marco and Patil, Vaishakh and Yu, Fisher},
journal={arXiv preprint arXiv:2401.09939},
year={2024}
}