i do work with some VoIP, and usually when one party doesnt hear the other, it indicates packet loss or latency. VoIP uses UDP as its transport mechanism, and does not resend packets like TCP does.

VoIP that we work with is over a VPN tunnel and we have teleworker connections (ISP gives a larger upload and download pipe). i think this is more due to the VPN than the VoIP, but may be a place to start looking.

when talking to the ISP, see if your connection is dropping packets or spiking on round trip times (latency). simple things like using good connectors, and eliminating as many splits in the coax as possible makes an impovement(if you are using a cable connection). if you can get your cable modem on one of the first splits of the coax, you can clear up some issues.