how absolute position embedding is applied?

[zhouzhengbo2022]

Thanks author for this great project. I have a question about how absolute position embedding is applied as shown in the code

self.pos_embed = nn.Parameter(torch.zeros(1, channels, 14, 14))

https://github.com/huawei-noah/Efficient-AI-Backbones/blob/master/vig_pytorch/vig.py#L110

since it is initialized as zero and have not updated during the training.

[iamhankai]

It is used in

Efficient-AI-Backbones/vig_pytorch/vig.py

Line 140 in 9172762

x = self.stem(inputs) + self.pos_embed

It will be updated during training.