I found that the gpe summary from the testing repo was causing the desktop to reset. I uninstalled it and installed from the normal extras and the problem went away.