So it's just pocketsphinx running all the time - isn't Saera also using pocketsphinx already...? But indeed maybe looking at the code and configuration of Jasper could be of help (how to solve some stuff, maybe also some dictionaries or phonetic models could be used, maybe some interesting ideas could be borrowed and reimplemented...).

edit: I think it is also worth noting that Ubuntu since 13.04 or 13.10 ship with pocketspinx by default, as they want to implement voice control in their Unity interface - maybe they have some nice models and dictionaries of phonetics in different languages which can be reused here.