Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RGBD_Inertial mode fails on Normal.bag file #8

Open
muskie82 opened this issue Jan 11, 2021 · 3 comments
Open

RGBD_Inertial mode fails on Normal.bag file #8

muskie82 opened this issue Jan 11, 2021 · 3 comments

Comments

@muskie82
Copy link

Thanks for sharing your work.

I tried to run RGB-D inertial mode on Normal.bag, but tracking fails immediately. Do you have any idea why?

Commands:

  • rosbag play Normal.bag /camera/color/image_raw:=/rgb/image_raw /camera/aligned_depth_to_color/image_raw:=/depth_to_rgb/image_raw /camera/imu:=/imu
  • rosrun ORB_SLAM3 RGBD_inertial Vocabulary/ORBvoc.bin Examples/ROS/ORB_SLAM3/Myd435i.yaml

Message:

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGBD-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- fx: 611.738
- fy: 610.111
- cx: 313.172
- cy: 252.256
- bf: 40
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- fps: 30
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 2.6155

Left camera to Imu Transform (Tbc): 
[0.00087480003, -0.0047406, 0.99998838, 1.30441e-05;
 -0.99996811, -0.0079460004, 0.00083709997, 0.0149225;
 0.0079418998, -0.9999572, -0.0047474001, 0.00031683501;
 0, 0, 0, 1]

IMU frequency: 500 Hz
IMU gyro noise: 0.01 rad/s/sqrt(Hz)
IMU gyro walk: 2e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.1 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0002 m/s^3/sqrt(Hz)
last KF is empty!
not IMU meas
last KF is empty!
First KF:0; Map init KF:0
New Map created with 988 points
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 1
mnInitialFrameId = 0
15 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:6; Map init KF:0
New Map created with 982 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 17
mnInitialFrameId = 15
24 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:10; Map init KF:6
New Map created with 1003 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 27
mnInitialFrameId = 25
28 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:13; Map init KF:10
New Map created with 998 points
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
@dzqiu
Copy link

dzqiu commented Mar 17, 2021

@muskie82 did you test it using vins-rgbd dataset？

@feng899
Copy link

feng899 commented Jul 20, 2021

Thanks for sharing your work.

I tried to run RGB-D inertial mode on Normal.bag, but tracking fails immediately. Do you have any idea why?

Commands:

  • rosbag play Normal.bag /camera/color/image_raw:=/rgb/image_raw /camera/aligned_depth_to_color/image_raw:=/depth_to_rgb/image_raw /camera/imu:=/imu
  • rosrun ORB_SLAM3 RGBD_inertial Vocabulary/ORBvoc.bin Examples/ROS/ORB_SLAM3/Myd435i.yaml

Message:

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGBD-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- fx: 611.738
- fy: 610.111
- cx: 313.172
- cy: 252.256
- bf: 40
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- fps: 30
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 2.6155

Left camera to Imu Transform (Tbc): 
[0.00087480003, -0.0047406, 0.99998838, 1.30441e-05;
 -0.99996811, -0.0079460004, 0.00083709997, 0.0149225;
 0.0079418998, -0.9999572, -0.0047474001, 0.00031683501;
 0, 0, 0, 1]

IMU frequency: 500 Hz
IMU gyro noise: 0.01 rad/s/sqrt(Hz)
IMU gyro walk: 2e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.1 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0002 m/s^3/sqrt(Hz)
last KF is empty!
not IMU meas
last KF is empty!
First KF:0; Map init KF:0
New Map created with 988 points
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 1
mnInitialFrameId = 0
15 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:6; Map init KF:0
New Map created with 982 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 17
mnInitialFrameId = 15
24 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:10; Map init KF:6
New Map created with 1003 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 27
mnInitialFrameId = 25
28 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:13; Map init KF:10
New Map created with 998 points
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
Fail to track local map!
IMU is not or recently initialized. Reseting active map...

请问您解决了吗？

@calmelo
Copy link

calmelo commented Jul 30, 2021

Thanks for sharing your work.

I tried to run RGB-D inertial mode on Normal.bag, but tracking fails immediately. Do you have any idea why?

Commands:

* ` rosbag play Normal.bag /camera/color/image_raw:=/rgb/image_raw /camera/aligned_depth_to_color/image_raw:=/depth_to_rgb/image_raw /camera/imu:=/imu`

* `rosrun ORB_SLAM3 RGBD_inertial Vocabulary/ORBvoc.bin Examples/ROS/ORB_SLAM3/Myd435i.yaml`

Message:

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGBD-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- fx: 611.738
- fy: 610.111
- cx: 313.172
- cy: 252.256
- bf: 40
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- fps: 30
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 2.6155

Left camera to Imu Transform (Tbc): 
[0.00087480003, -0.0047406, 0.99998838, 1.30441e-05;
 -0.99996811, -0.0079460004, 0.00083709997, 0.0149225;
 0.0079418998, -0.9999572, -0.0047474001, 0.00031683501;
 0, 0, 0, 1]

IMU frequency: 500 Hz
IMU gyro noise: 0.01 rad/s/sqrt(Hz)
IMU gyro walk: 2e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.1 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0002 m/s^3/sqrt(Hz)
last KF is empty!
not IMU meas
last KF is empty!
First KF:0; Map init KF:0
New Map created with 988 points
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 1
mnInitialFrameId = 0
15 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:6; Map init KF:0
New Map created with 982 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 17
mnInitialFrameId = 15
24 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:10; Map init KF:6
New Map created with 1003 points
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 27
mnInitialFrameId = 25
28 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:13; Map init KF:10
New Map created with 998 points
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
virtual int g2o::SparseOptimizer::optimize(int, bool): 0 vertices to optimize, maybe forgot to call initializeOptimization()
Fail to track local map!
IMU is not or recently initialized. Reseting active map...

I think that maybe you can try to change the camera-intrinsics as VINS-RGBD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@muskie82 @dzqiu @calmelo @feng899