Debian Multimedia :: Audacious / Audio Freezes For A Few Seconds

I have a very fresh and basic install of the weekly jessie build (this week). I've installed nvidia drivers, some non-free plugins, virtualbox, basic system tools, etc. The issue is while playing in audacious, the song will freeze for about 5 seconds. It's always around 5 seconds. Sometimes it's once or twice per song, sometimes I'll play 10 songs without it happening. BTW, all music is on a separate hard drive on an NTFS partition.

In audacious, I'm using the Winamp skin. I noticed when trying to play for the first time, it wouldn't play unless I changed the setting in audacious to use ALSA. That was the only way I could go to Settings and select my SPDIF digital output. But then doing that, other sounds no longer worked (in iceweasel). So I downloaded pavu, was able to select my onboard digital stereo output, set audacious back to pulse audio, and everything now works. Only issue is audacious is freezing every now and then... Is there a driver or module that I need to check for? Is there a way to get everything to use ALSA?

I once tried AV Linux for the low latency kernel but since it was based on Wheezy and I wasn't familiar with repos, everything was extremely outdated, so I migrated away. Should I need an LL kernel for 320kbps audio playback? I haven't sat and watched a full mkv yet, some of those have 1.5mpbs audio.