"Soft-Landing Strategy for Alleviating the Task Discrepancy Problem in Temporal Action Localization Tasks"

paper

Training and assessing the model with the linear evaluation.

1. Prepare the data in the dataset folder. In this code, we will work with the HACS dataset as the official I3D feature is available here. Extract the downloaded feature into the HACS_DATA folder. The file structure should be as follows:

   HACS_DATA
   |--training
   |  |--__9PFRfSjE0.npy
   |  |--__9Ux_pexqs.npy
   |  |--...
   |  |--...
   |  ...
   |--validation
   |  |--_-lqeH_0xXU.npy
   |  |--_0EG7L9nCgc.npy
   |  |--...
   |  |--...
   |  ...
   |--training_duration.pkl
   |--training_subset_list.pkl
   |--validation_duration.pkl
   |--validation_subset_list.pkl
   
   

2. Make visualizing directory here and select any 6 .npy files from ./HACS_DATA/validation/. Place them in ./visualizing/. This is for visualizing the TSM (Temporal Self-similarity Matrix) during the training procedure.

3. Download evaluation.tar.gz and extract it into your working directory so that the file tree looks like this:

   evalutaion
   |--training_label
   |  |--__9PFRfSjE0.npy
   |  |--__9Ux_pexqs.npy
   |  |--...
   |  |--...
   |  ...
   |--validation_label
   |  |--_-lqeH_0xXU.npy
   |  |--_0EG7L9nCgc.npy
   |  |--...
   |  |--...
   
   
   

   This is for the linear evaluation of action/non-action snippet.

4. Run following command:

   python main_simsiam.py --yaml_path=yamls/hacs_canonical.yaml
   

   The code performs both training and linear evaluation on action/non-action snippet features at every saving_epoch, as defined in yamls/hacs_canonical.yaml. Expect to observe consistent improvements in the evaluation results throughout the training procedure.

5. After training, you can choose one of the saved model (which resulted in the best linear evaluation result) and run following

   python process_feature.py --yaml_path=yamls/hacs_canonicla.yaml --load_model=150
   

   to generate SoLa features. With SoLa features, you can get better result in G-TAD downstream head.