I've played around with this further, and I think I was able to get MP3 playback on the DSP while still running the main CPU at full speed.

I had to set the CPU frequency scaling strategy to null, though. (via "echo null > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor") It somewhat of a crude solution, but it's easier than compiling a kernel.