augmentations

[radekosmulski]

I researched modern NLP methods a bit, as a follow up to our yesterday's conversation. With regards to tangible outputs, I annotated 40 recordings using dl-annotator, should be enough to train on them.

Also, on the birdcall competition, one of the organizers shared a few insights with regards to augmentation. I considered what would make sense to do in the context of this competition and implemented a couple of augmentations after transforming audio to spectrogram (adding noise, frequency shift, time dilation, etc). I haven't had a chance to test how they work though - still experimenting with how one can train on this data. Hoping to run a training run or two over the weekend to see how the augmentations perform.

[aza]

I think @pcbermant has linked to this before, but this paper on specaugment (https://arxiv.org/pdf/1904.08779.pdf) has some more augmentation techniques.

Other potential data augmentation techniques:

• Gaussian blur the spectrogram (or, separately, vertical and horizontal blurs)
• Sharpen the spectrogram as image
• Modify gamma, contrast, etc of spectrogram as image
• Random dropout of pixels of spectrogram

And then there are a number of self-supervised tasks that might be useful:

• Split the spectrogram into tiles and have the neural net recover their original order
• Mask a section of the spectrogram and have the neural net reconstruct the masked portion
• Combine two recordings and have the neural net try to cover the original separated signals