I hate subtitling my music videos on TikTok, Reels, and YouTube. This allows me and others to auto-caption their music videos using the original video plus an audio recording of their voice speaking the words of the song
For the MVP I'm using an offline speech recognition model from Vosk and MoviePY for subtitles.
Vosk generates a JSON "transcript" containing objects with the words and their time bounds in the audio.
Fortunately, subtitles use text + time bounds— go figure. Pass those into MoviePY and voila
If you want to run this WIP MVP:
- Set up your Python venv and install the requirements
- Install/make sure FFMPEG is installed on your system
- Download the vosk-model-en-us-0.22 language model from the Vosk website and unzip the folder in the outermost repo directory
- Run mvp.py, giving 'h4t' as the two inputs when prompted to caption the default inputs in the folder
- Repeat with your own synced voice + video inputs