Adding some audio transforms and augmentations to tonic #273

MinaKh · 2023-11-24T12:33:26Z

This branch includes following updates in tonic/develop:

Three transforms are added to audio_transforms.py:
- SwapAxes, AmplitudeScale and robustAmplitudeScale
A new script is added: audio_augmentations.py containing wrapper classes for following audio augmentations:
- RandomAmplitudeScale
- RandomPitchShift
- RandomTimeStreatch
- RIR: adding room impulse response (echo effect)
- AddWhiteNoise
corresponding tests added: test_audio_trnasform.py
A new script added for testing audio augmentations ---> test_audio_augmentations.py
A tutorial notebook is added in docs/tutorials/

fabrizio-ottati · 2023-11-24T15:09:53Z

@MinaKh I think you should add torchaudio to the test requirements.txt, 'cause it fails as dependency in the CI tests.

MinaKh · 2023-11-24T15:21:33Z

@MinaKh I think you should add torchaudio to the test requirements.txt, 'cause it fails as dependency in the CI tests.

Thanks, just did!

fabrizio-ottati · 2023-11-24T15:33:23Z

It seems that the tests now are running using CUDA. Maybe it is better to run them with everything on CPU? Or we could modify the requirements.txt to install the GPU version of PyTorch (both vision and audio).

MinaKh · 2023-11-24T15:44:44Z

It seems that the tests now are running using CUDA. Maybe it is better to run them with everything on CPU? Or we could modify the requirements.txt to install the GPU version of PyTorch (both vision and audio).

@fabrizio-ottati No, I can set them to run on cpu. Also there are other tests that I passed on my machine but fail on CI. I need to fix without need to install extra packages. Thanks!

biphasic

Most transforms are encapsulated enough so that I can add them, but some stuff that uses QUTNoise things I won't be able to merge like that, unless it is made a bit more general. For example, maybe Tonic has a AddNoise class, but then in your user code at SynSense you call it with AddNoise(QUTNoise), after the principle of dependency injection

biphasic · 2023-11-25T13:02:20Z

tonic/audio_augmentations.py

+import torch
+import torchaudio
+import torchaudio.functional as F
+from qut_noise import QUTNoise


this is a dependency that I won't be able to add to this library like that.

biphasic · 2023-11-25T13:03:21Z

tonic/audio_augmentations.py

+
+
+@dataclass
+class AddHomeNoise:


This is very specific, is there a way how you can abstract it to noise from different datasets?

@biphasic I have removed all noise related augmentations from this PR.

biphasic · 2023-11-25T13:04:07Z

tonic/audio_augmentations.py

+
+    def __call__(self, audio):
+        SAMPLE_RIR = download_asset(
+            "tutorial-assets/Lab41-SRI-VOiCES-rm1-impulse-mc01-stu-clo-8000hz.wav"


this sort of hardcoded things I cannot merge into a public library

this sort of hardcoded things I cannot merge into a public library

I removed this hard coded audio path. Instead the room impulse audio needs to be passed by user. The corresponding test is also updated.

biphasic · 2023-11-25T13:07:19Z

The Github Actions test suite doens't have a GPU installed, therefore there's no point in installing any CUDA dependencies. Tests should always run on CPU please. Thank you

MinaKh · 2023-11-27T10:52:35Z

The Github Actions test suite doens't have a GPU installed, therefore there's no point in installing any CUDA dependencies. Tests should always run on CPU please. Thank you

Thanks @biphasic, As far as I checked my tests are not running on GPU. The error might be caused by general imports of torch and torchaudio, or by the difference in my local version and the installed one on the server. So I included the versions in the requirements.
At this point I need to be able to run tests on GitHub to understand the issue better and fix it. Currently tests are not running automatically after my pushes (perhaps needs to be authorized every time by you and other admins?).

biphasic · 2023-11-28T17:51:22Z

requirements.txt

+torch==1.12.0+cu113
+torchaudio==0.12.0+cu113


cu113 means CUDA 11.3, so this is installing a fixed version of pytorch with CUDA backend

biphasic · 2023-11-28T17:52:17Z

@MinaKh are the tests passing on your local machine?

biphasic · 2023-11-28T18:06:06Z

Also I just relaxed the Github actions approval to the minimum level possible, I hope it now works without my manual approval!

fabrizio-ottati · 2023-12-05T10:59:41Z

@MinaKh I need to sit and do it properly. I will update you this afternoon

fabrizio-ottati · 2023-12-05T11:34:48Z

I don't know why but it keeps installing the CUDA version even if I have specificied to use the CPU wheel of PyTorch.

fabrizio-ottati · 2023-12-05T11:41:48Z

Okay, it seems I convinced it @MinaKh :)

codecov-commenter · 2023-12-05T11:59:36Z

Codecov Report

Attention: Patch coverage is 97.19626% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 77.72%. Comparing base (e5bd291) to head (0af124a).
Report is 22 commits behind head on develop.

Files	Patch %	Lines
tonic/audio_transforms.py	92.00%	2 Missing ⚠️
tonic/audio_augmentations.py	98.78%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #273      +/-   ##
===========================================
+ Coverage    76.84%   77.72%   +0.88%     
===========================================
  Files           53       55       +2     
  Lines         3001     3174     +173     
===========================================
+ Hits          2306     2467     +161     
- Misses         695      707      +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

MinaKh · 2023-12-05T12:34:18Z

Okay, it seems I convinced it @MinaKh :)

Thanks @fabrizio-ottati

fabrizio-ottati · 2023-12-05T12:59:51Z

@biphasic all the tests are passing now. I have created a separate test/torch_requirements.txt so that I can pull the CPU wheel of PyTorch. Moreover, following torchaudio documentation, specific combinations of torch and torchaudio versions need to be used to ensure safe installation.

Now the code is ready to be reviewed and CI should be safe.

MinaKh · 2023-12-05T13:17:37Z

Most transforms are encapsulated enough so that I can add them, but some stuff that uses QUTNoise things I won't be able to merge like that, unless it is made a bit more general. For example, maybe Tonic has a AddNoise class, but then in your user code at SynSense you call it with AddNoise(QUTNoise), after the principle of dependency injection

@biphasic I removed those classes (noise augmentations) from this branch. currently it is very specific and I will prepare another PR later for that.

fabrizio-ottati · 2024-01-21T22:01:50Z

What's the status on this PR? :)

MinaKh · 2024-01-22T09:20:02Z

What's the status on this PR? :)

It is ready for final review...

MinaKh added 12 commits November 23, 2023 14:23

new transform added: SwapAxes

b9c6618

audio_augmentation module is added: with RandomTimeStretch

b89f146

RandomPitchShift transfrom added

fbe2b61

RandomAmplitudeScale is added

2cce804

RIR transform (room impulse response) added

ebaadad

AmplitudeScale and RobustAmplitudeScale transforms added

df239fa

Noise augmentations added

60cd283

typos fixed in docstrings

a52c334

fixes in docstrings

9fc506a

tests for added transforms

6cb92f7

tests for audio augmentations

80e3da8

tests_passed

d2c0436

MinaKh marked this pull request as draft November 24, 2023 12:40

MinaKh marked this pull request as ready for review November 24, 2023 12:43

MinaKh marked this pull request as draft November 24, 2023 14:39

torchaudio added to requirements

af12bb4

biphasic requested changes Nov 25, 2023

View reviewed changes

MinaKh added 2 commits November 27, 2023 10:15

removing torchaudio dependency temporarily to run the tests

8465bb7

requirments updated with torch and torchaudio versions

10760db

MinaKh added 2 commits November 27, 2023 14:09

removing hard coded room impulse audio from RIR transform

78837f0

RIR test updated

b7f76b6

biphasic reviewed Nov 28, 2023

View reviewed changes

fabrizio-ottati added 6 commits December 5, 2023 12:26

Create torch_requirements.txt

5bd4a36

Update requirements.txt

ea025f0

Update torch_requirements.txt

89dc597

Update ci-pipeline.yml

21e241f

Update torch_requirements.txt

94f641f

Update ci-pipeline.yml

f54fd8c

Update torch_requirements.txt

0f8be68

fabrizio-ottati added 2 commits December 5, 2023 12:51

Update ci-pipeline.yml

23bdd42

Testing with python>=3.8 and python<=3.11

04aac21

noise related augmentations removed

628bb55

MinaKh requested a review from biphasic December 5, 2023 13:18

MinaKh added 3 commits December 6, 2023 18:25

sample_length was removed from some transforms (when not needed)

8dc340b

bug fixed in test

f57d4d5

tutorial added for audio transforms/augmentations

6db78de

biphasic added 2 commits May 15, 2024 10:28

Merge remote-tracking branch 'origin/main' into add_audio_transforms

c1e53dc

shorten GH actions pipeline to three Python versions

0af124a

biphasic marked this pull request as ready for review May 15, 2024 08:30

biphasic added 2 commits May 15, 2024 15:48

add torch requirements to documentation github action

b81f886

pin torchvision version to something compatible with torch 2.1

2fb1664

biphasic approved these changes May 15, 2024

View reviewed changes

biphasic merged commit 5a20a54 into neuromorphs:develop May 15, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding some audio transforms and augmentations to tonic #273

Adding some audio transforms and augmentations to tonic #273

MinaKh commented Nov 24, 2023 •

edited

Loading

fabrizio-ottati commented Nov 24, 2023

MinaKh commented Nov 24, 2023

fabrizio-ottati commented Nov 24, 2023

MinaKh commented Nov 24, 2023

biphasic left a comment

biphasic Nov 25, 2023

biphasic Nov 25, 2023

MinaKh Jan 22, 2024

biphasic Nov 25, 2023

MinaKh Nov 27, 2023

biphasic commented Nov 25, 2023

MinaKh commented Nov 27, 2023

biphasic Nov 28, 2023

biphasic commented Nov 28, 2023

biphasic commented Nov 28, 2023

fabrizio-ottati commented Dec 5, 2023

fabrizio-ottati commented Dec 5, 2023

fabrizio-ottati commented Dec 5, 2023

codecov-commenter commented Dec 5, 2023 •

edited

Loading

MinaKh commented Dec 5, 2023

fabrizio-ottati commented Dec 5, 2023

MinaKh commented Dec 5, 2023

fabrizio-ottati commented Jan 21, 2024

MinaKh commented Jan 22, 2024



		@dataclass
		class AddHomeNoise:

		torch==1.12.0+cu113
		torchaudio==0.12.0+cu113

Adding some audio transforms and augmentations to tonic #273

Adding some audio transforms and augmentations to tonic #273

Conversation

MinaKh commented Nov 24, 2023 • edited Loading

fabrizio-ottati commented Nov 24, 2023

MinaKh commented Nov 24, 2023

fabrizio-ottati commented Nov 24, 2023

MinaKh commented Nov 24, 2023

biphasic left a comment

Choose a reason for hiding this comment

biphasic Nov 25, 2023

Choose a reason for hiding this comment

biphasic Nov 25, 2023

Choose a reason for hiding this comment

MinaKh Jan 22, 2024

Choose a reason for hiding this comment

biphasic Nov 25, 2023

Choose a reason for hiding this comment

MinaKh Nov 27, 2023

Choose a reason for hiding this comment

biphasic commented Nov 25, 2023

MinaKh commented Nov 27, 2023

biphasic Nov 28, 2023

Choose a reason for hiding this comment

biphasic commented Nov 28, 2023

biphasic commented Nov 28, 2023

fabrizio-ottati commented Dec 5, 2023

fabrizio-ottati commented Dec 5, 2023

fabrizio-ottati commented Dec 5, 2023

codecov-commenter commented Dec 5, 2023 • edited Loading

Codecov Report

MinaKh commented Dec 5, 2023

fabrizio-ottati commented Dec 5, 2023

MinaKh commented Dec 5, 2023

fabrizio-ottati commented Jan 21, 2024

MinaKh commented Jan 22, 2024

MinaKh commented Nov 24, 2023 •

edited

Loading

codecov-commenter commented Dec 5, 2023 •

edited

Loading