Quick experimentations about tracking drift detection

[s-trinh]

Experimentation

In this discussion, I will just summarize some tests about tracking drift detection and MBT.
These are just some experimentations I have done for curiosity, nothing concrete, nothing valid.

---

getProjectionError()

First option I know is to leverage MBT getProjectionError() and something like Hinkley statistical test.

Distance error between contour and model?

Another option could be?

• compute Canny edges detection
• for each line in the model:
  • search along the line normal in both direction
  • report the distance between the first detected edge
• normally if the object is correctly tracked, the projected model and the detected edges should match

Caveats:

• should probably not work with partial occlusion
• in theory, should be more or less equivalent to compute the gradient at each sampled location method?

Idea:

• try detect directly lines instead of Canny edges detection and try to match the lines instead?

Forward / backward tracking?

Something similar to Forward-Backward Error: Automatic Detection of Tracking Failures, Z. Kalal, ICPR 2010?

Forward:

• track image (n+1) with knowledge about (n) in the MBT (the usual tracking)

Backward:

• initialize another MBT instance
• use initFromPose() and track() on image (n+1)
• track on image (n)
• compute the relative pose at time (n)

Test on mbt/cube/ sequence

• use this cube.xml config file for tracking drift with MBT edges with old <edge_threshold>5000</edge_threshold> parameter
• MBT edges + KLT is used to provide the "ground truth"

"Contour" error

res_contours_error_graph.mp4

• top-left:
  • Canny edges detection
  • for each line in the model
    • sample
    • compute normal to the line
    • search in both direction if we found an edge
  • dashed horizontal lines are the median
• bottom:
  • in red the projection error
  • in blue the "distance error" between the model and the edges in the image
  • "pose error" is the relative pose between the edge MBT with <edge_threshold>5000</edge_threshold> and edge + KLT tracker

"Forward / Backward" tracking

Very quick test gives:

Idea:

• here F/B is done between the previous and the current image. Maybe try on a slightly bigger timeframe?

[s-trinh]

Forward / Backward tracking

Forward / backward tracking for different time window sizes:

• 2, 5, 10 and 30 images
• i.e. for 30 images, the compared poses are done for image at t - 30

Visualizations

• top left is the tracking result for the backward sequence:
  • green is the projected model with the pose estimated as usual
  • red is the projected model with the pose estimated from the backward sequence
• top right is the tracking result for the current image
• bottom are the graphs with projection error plot, norm of the translation/theta.u error (backward pose) and pose error using the "ground truth" pose

Win size=2

res_forward_backward_error_graph_win_2.mp4

Win size=5

res_forward_backward_error_graph_win_5.mp4

Win size=10

res_forward_backward_error_graph_win_10.mp4

Win size=30

res_forward_backward_error_graph_win_30.mp4