This is because the media backend shuts down after a while when not in use in order to save battery. The osso-media-server backend still has problems with properly resuming after that. The mplayer backend handles this better.