I installed the OGG support package (from one of the repos) and then I had installed Kagu.
I seem to be able to play my ogg files that I had burned previosly just fine. Copied them over from my desktop to the Nokia N810 and it plas in Kagu absolutely fine.

Did not need to rename the files or anything silly like that.