Contacts generally slow. I have a like 3500 contacts.
I had them on my HTC Tytn II - WM6.0 and it worked a lot faster.

any chance to speed up w/o deleting most of contacts....