Scores of Tuning and mots_eval

[andreapereza2]

Hi,

I am trying to achieve the same results as your document ( sMOTSA_car 76.2, sMOTSA_ped 46.8, MOTSA_car 87.8, MOTSA_ped 65.1, MOTSP_car 87.2 and MOTSP_ped 75.7 )

I ran the code (forwarding and tracking) with your pre-trained model and then I obtained the next Score with mots tools:

sMOTSA_car 76.4
sMOTSA_ped 45.9
MOTSA_car 88.1
MOTSA_ped 64.2
MOTSP_car 87.2
MOTSP_ped 75.6

¿Why is it not the same results?

I also performed the Tuning and it gave me the following results:

---

Best CAR sMOTSA train 76.9 Settings: {'tracker': 'hungarian', 'reid_comp': 'euclidean', 'detection_confidence_threshold_car': 0.8467975034901326, 'reid_weight_car': 1.0, 'mask_iou_weight_car': 0.0, 'bbox_center_weight_car': 0.0, 'bbox_iou_weight_car': 0.0, 'association_threshold_car': 0.7461357158141632, 'keep_alive_car': 3, 'reid_euclidean_offset_car': 9.086529154055489, 'reid_euclidean_scale_car': 1.4463662918504006, 'new_reid_threshold_car': 2.0, 'box_offset': 50.0, 'box_scale': 0.02, 'new_reid': False}
[('0002', 233), ('0006', 270), ('0007', 800), ('0008', 390), ('0010', 294), ('0013', 340), ('0014', 106), ('0016', 209), ('0018', 339)] False
Scores(sMOTSA_car=76.4, sMOTSA_ped=42.0, MOTSA_car=88.1, MOTSA_ped=61.1, MOTSP_car=87.2, MOTSP_ped=75.3, IDS_car=90, IDS_ped=99)
Val scores sMOTSA 76.4 MOTSA 88.1 MOTSP 87.2 IDS 90

Best PED sMOTSA train 57.4 Settings: {'tracker': 'hungarian', 'reid_comp': 'euclidean', 'detection_confidence_threshold_pedestrian': 0.9467424694204876, 'reid_weight_pedestrian': 1.0, 'mask_iou_weight_pedestrian': 0.0, 'bbox_center_weight_pedestrian': 0.0, 'bbox_iou_weight_pedestrian': 0.0, 'association_threshold_pedestrian': 0.47942774751307976, 'keep_alive_pedestrian': 7, 'reid_euclidean_offset_pedestrian': 9.456790784885905, 'reid_euclidean_scale_pedestrian': 1.3434556538327107, 'new_reid_threshold_pedestrian': 2.0, 'box_offset': 50.0, 'box_scale': 0.02, 'new_reid': False}
[('0002', 233), ('0006', 270), ('0007', 800), ('0008', 390), ('0010', 294), ('0013', 340), ('0014', 106), ('0016', 209), ('0018', 339)] False
Scores(sMOTSA_car=74.1, sMOTSA_ped=45.9, MOTSA_car=86.4, MOTSA_ped=64.0, MOTSP_car=86.8, MOTSP_ped=75.7, IDS_car=106, IDS_ped=82)
Val scores sMOTSA 45.9 MOTSA 64.0 MOTSP 75.7 IDS 82

I looked in the resulting .txt file for the model configuration values that had given a Score of sMOTSA_car 76.9 and redid the forwarding and tracking with those values.

{'tracker': 'hungarian', 'reid_comp': 'euclidean', 'detection_confidence_threshold_car': 0.8444034601995319, 'detection_confidence_threshold_pedestrian': 0.9437229009151353, 'reid_weight_car': 1.0, 'reid_weight_pedestrian': 1.0, 'mask_iou_weight_car': 0.0, 'mask_iou_weight_pedestrian': 0.0, 'bbox_center_weight_car': 0.0, 'bbox_center_weight_pedestrian': 0.0, 'bbox_iou_weight_car': 0.0, 'bbox_iou_weight_pedestrian': 0.0, 'association_threshold_car': 0.14643277295045268, 'association_threshold_pedestrian': 0.3216206102237347, 'keep_alive_car': 3, 'keep_alive_pedestrian': 6, 'reid_euclidean_offset_car': 9.296872422879806, 'reid_euclidean_scale_car': 1.4421416272085135, 'reid_euclidean_offset_pedestrian': 9.296872422879806, 'reid_euclidean_scale_pedestrian': 1.4421416272085135, 'new_reid_threshold_car': 2.0, 'new_reid_threshold_pedestrian': 2.0, 'box_offset': 50.0, 'box_scale': 0.02, 'new_reid': False} Scores(sMOTSA_car=76.9, sMOTSA_ped=57.4, MOTSA_car=88.6, MOTSA_ped=74.9, MOTSP_car=87.4, MOTSP_ped=78.3, IDS_car=233, IDS_ped=139)

When I retrieve the Scores again with mots_eval I get this

sMOTSA_car 76.2
sMOTSA_ped 47.0
MOTSA_car 87.9
MOTSA_ped 65.3
MOTSP_car 87.2
MOTSP_ped 75.7

Why don't I get the same Score as in the tuning? Should I only have done the result tracking and used the detections obtained with the original model configuration?

It's a bit confusing for me the tuning. are the random parameters just to do the tracking keeping the detections with the default model parameters? is the value of 'Best PED sMOTSA train' testing in the training and then the 'Scores' that appears is the result of that configuration in the val? in the .txt file of all the iterations, the scores that come out are directly in the val, no?

I look forward to your comments

Thank you very much