Hm, if what I was asking for is impossible maybe I'll try that out. I used to find it sometimes useful when one of my old phones did that, but also sometimes very annoying (buzzing periodically all night, just out of reach so it's a lot of effort to placate it...)

Perhaps it's functionality I missed reading about in Call Notify, which Matthew suggested above.