Well it turns out that Recaller is the culprit. I have auto recording enabled and sometimes when I get a call, recaller crashes and all of my widgets disappear :/.