Overclocking solved the choppiness totally.

BTW, as you have mentioned the idle cpu usage is kinda high , like 30% @ 250MHz, did you find any solution for that?.

Asterisks idles like 2% @ 250Mhz IIRC,

asterisk 1.8 support GV support, but unfortunately only asterisk 1.6 available in repositories for N900