I personally hate the concept of entire UI fading. That just makes me feel like the entire OS crashed, or kernel froze, and I bet I'm not alone with this.

I suggest the following:

1) Minimize unresponsive app to the dashboard with a throbber

2) If the app will not respond within, say 30 secs, bring up above suggested "kill this?" dialogue.

3) If the app hogs all resources, terminate automatically in one minute

4) Add notification area to the desktop, which tells you what happened. Windowsy "Your app just crashed, click OK" popups are just silly and should never be used.

EDIT: I guess the tray would do that better