简体中文 | English
PaddleVideo包含视频分类和动作定位方向的多个主流领先模型，其中TSN, TSM和SlowFast是End-to-End的视频分类模型，Attention LSTM是比较流行的视频特征序列模型，BMN是视频动作定位模型。TSN是基于2D-CNN的经典解决方案，TSM是基于时序移位的简单高效视频时空建模方法，SlowFast是FAIR在ICCV2019提出的3D视频分类模型，特征序列模型Attention LSTM速度快精度高。BMN模型是百度自研模型，为2019年ActivityNet夺冠方案。基于百度飞桨产业实践，我们自研并开源了ppTSM，该模型基于TSM进行优化，在保持模型参数量和计算量不增加的前提下，精度得到大幅提升。同时，我们的通用优化策略可以广泛适用于各种视频模型，未来我们将进行更多的模型优化工作，比如TPN、SlowFast、X3D等，敬请期待。
| 领域 | 模型 | 配置 | 测试集 | 精度指标 | 精度% | 下载链接 |
|---|---|---|---|---|---|---|
| 行为识别 | PP-TSM | pptsm.yaml | Kinetics-400 | Top-1 | 76.16 | ppTSM.pdparams |
| 行为识别 | PP-TSN | pptsn.yaml | Kinetics-400 | Top-1 | 75.06 | ppTSN.pdparams |
| 行为识别 | AGCN | agcn.yaml | FSD-10 | Top-1 | 90.66 | AGCN.pdparams |
| 行为识别 | ST-GCN | stgcn.yaml | FSD-10 | Top-1 | 71.33 | STGCN.pdparams |
| 行为识别 | TimeSformer | timesformer.yaml | Kinetics-400 | Top-1 | 77.29 | TimeSformer.pdparams |
| 行为识别 | SlowFast | slowfast_multigrid.yaml | Kinetics-400 | Top-1 | 75.84 | SlowFast.pdparams |
| 行为识别 | TSM | tsm.yaml | Kinetics-400 | Top-1 | 70.86 | TSM.pdparams |
| 行为识别 | TSN | tsn.yaml | Kinetics-400 | Top-1 | 69.81 | TSN.pdparams |
| 行为识别 | AttentionLSTM | attention_lstm.yaml | Youtube-8M | Hit@1 | 89.0 | AttentionLstm.pdparams |
| 视频动作定位 | BMN | bmn.yaml | ActivityNet | AUC | 67.23 | BMN.pdparams |
-
Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification, Xiang Long, Chuang Gan, Gerard de Melo, Jiajun Wu, Xiao Liu, Shilei Wen
-
BMN: Boundary-Matching Network for Temporal Action Proposal Generation, Tianwei Lin, Xiao Liu, Xin Li, Errui Ding, Shilei Wen.
-
SlowFast Networks for Video Recognition, Feichtenhofer C, Fan H, Malik J, et al.
-
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition, Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, Luc Van Gool
-
Temporal Shift Module for Efficient Video Understanding, Ji Lin, Chuang Gan, Song Han
-
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius, Heng Wang, Lorenzo Torresani
-
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition, Sijie Yan, Yuanjun Xiong, Dahua Lin
-
Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition, Lei Shi, Yifan Zhang, Jian Cheng, Hanqing Lu
-
Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks, Lei Shi, Yifan Zhang, Jian Cheng, Hanqing Lu