Skip to content
This repository has been archived by the owner on May 27, 2024. It is now read-only.

Latest commit

 

History

History
122 lines (105 loc) · 5.67 KB

TRAINING.md

File metadata and controls

122 lines (105 loc) · 5.67 KB
Raw

Step 1: Prepare the data and model

We experimented on the ActEv dataset and the ETH & UCY dataset. The original ActEv annotations can be downloaded from here. Please do obtain the data copyright and download the raw videos from their website. You can download our prepared features from the project page by running the script bash scripts/download_prepared_data.sh. This will download the following data, and will require about 31 GB of disk space:

  • next-data/final_annos/: This folder includes extracted features and annotations for both experiments. In ActEv experiments, it includes person appearance features, person keypoint features, scene semantic features and scene object features. In ETH/UCY, person keypoint features are not used. Data format notes are here.
  • next-data/actev_personboxfeat/: This folder includes person appearance features for ActEv experiments
  • next-data/ethucy_personboxfeat/: This folder includes person appearance features for ETH/UCY experiments

Then download the pretrained model following instructions from here.

Step 2: Preprocess - ActEv

Preprocess the data for training and testing. The following is for ActEv experiments.

python code/preprocess.py next-data/final_annos/actev_annos/virat_2.5fps_resized_allfeature/ \
  actev_preprocess --obs_len 8 --pred_len 12 --add_kp \
  --kp_path next-data/final_annos/actev_annos/anno_kp/ --add_scene \
  --scene_feat_path next-data/final_annos/actev_annos/ade20k_out_36_64/ \
  --scene_map_path next-data/final_annos/actev_annos/anno_scene/ \
  --scene_id2name next-data/final_annos/actev_annos/scene36_64_id2name_top10.json \
  --scene_h 36 --scene_w 64 --video_h 1080 --video_w 1920 --add_grid \
  --add_person_box --person_box_path next-data/final_annos/actev_annos/anno_person_box/ \
  --add_other_box --other_box_path next-data/final_annos/actev_annos/anno_other_box/ \
  --add_activity --activity_path next-data/final_annos/actev_annos/anno_activity/ \
  --person_boxkey2id_p next-data/final_annos/actev_annos/person_boxkey2id.p

Step 3: Train the models - ActEv

You can train your model by running:

python code/train.py actev_preprocess next-models/actev_single_model model \
  --runId 2 --is_actev --add_kp --add_activity \
  --person_feat_path next-data/actev_personboxfeat --multi_decoder

By default this will train a model for ActEv dataset, periodically saving model files to next-models/actev_single_model/model/02/save at the current working directory. The script will also periodically evaluate the model on the validation set and save the latest best model to next-models/actev_single_model/model/02/best. Detailed commands of the training script:

Training options

  • --batch_size: How many trajectories to use in each minibatch during training. Default is 32.
  • --num_epochs: Number of training epochs. Default is 100.
  • --init_lr: Initial Learning rate. Default is 0.2.
  • --keep_prob: 1 - dropout rate. Default is 0.7.
  • --optimizer: Optimizer to use. Default is AdaDelta.
  • --act_loss_weight: Weight for activity label classification loss. Default is 1.0.
  • --grid_loss_weight: Weight for activity location classification loss. Default is 0.1.

Basic model options

  • --emb_size: Embedding size. Default is 128.
  • --enc_hidden_size: Encoder hidden size. Default is 256.
  • --dec_hidden_size: Decoder hidden size. Default is 256.
  • --activation_func: Activation function. Default is tanh. You could choose from relu/lrelu/tanh.

Step 4: Test the model - ActEv

You can use following command to test the newly trained model:

python code/test.py actev_preprocess next-models/actev_single_model model \
  --runId 2 --load_best --is_actev --add_kp --add_activity \
  --person_feat_path next-data/actev_personboxfeat --multi_decoder

The best model on the validation set will be used.

Train/test the models - ETH/UCY

Preprocess the data for training and testing. The following is for ETH/UCY experiments. We conduct leave-one-scene-out experiment therefore we need to preprocess the data once for each scene.

for dataset in {eth,hotel,univ,zara1,zara2};
  do
    python code/preprocess.py next-data/final_annos/ucyeth_annos/original_trajs/${dataset}/ ethucy_exp/preprocess_${dataset} \
    --person_boxkey2id next-data/final_annos/ucyeth_annos/${dataset}_person_boxkey2id.p \
    --obs_len 8 --pred_len 12 --min_ped 1 --add_scene \
    --scene_feat_path next-data/final_annos/ucyeth_annos/ade20k_e10_51_64/ \
    --scene_map_path next-data/final_annos/ucyeth_annos/scene_feat/ \
    --scene_id2name next-data/final_annos/ucyeth_annos/scene51_64_id2name_top10.json \
    --scene_h 51 --scene_w 64 --video_h 576 --video_w 720 --add_grid --add_person_box \
    --person_box_path next-data/final_annos/ucyeth_annos/person_box/ --add_other_box \
    --other_box_path next-data/final_annos/ucyeth_annos/other_box/ \
    --feature_no_split --reverse_xy --traj_pixel_lst \
    next-data/final_annos/ucyeth_annos/traj_pixels.lst ;
  done

As an example, you can train your model on ZARA1 by running:

python code/train.py ethucy_exp/preprocess_zara1/ next-model/ethucy_single_model/zara1/ model --runId 2  \
--scene_h 51 --scene_w 64 --person_feat_path next-data/ethucy_personboxfeat/zara1/

Please refer to the code/paper for more training options.

Similar for testing:

python code/test.py ethucy_exp/preprocess_zara1/ next-model/ethucy_single_model/zara1/ model --runId 2  \
--scene_h 51 --scene_w 64 --person_feat_path next-data/ethucy_personboxfeat/zara1/