Unfortunately, it's not that simple for flite. I think the festival-based vox files have to be converted into some C structure-based format. It would require recompiling flite, at least. I don't have the expertise to try.