tbh, I think there should be a hardware button for answering a call, it might be the only thing that can fix this delay.
A daemon can capture the camera key when a call's received, and release it when the call's answered/ended, that way you would retain whatever functionality you have on the camera key, and still answer calls with it.

Though I'd imagine camera-ui would open the camera after answering the call :/