Tested on OS X Mavericks. An option is to install Homebrew and do the following:
brew install portaudio
brew install ffmpeg
sudo easy_install pyaudio
sudo easy_install pydub
sudo easy_install numpy
sudo easy_install scipy
sudo easy_install matplotlib
sudo easy_install pip
sudo pip install MySQL-python
sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib
However installing portaudio and/or ffmpeg from source is also doable.