Try downloading eVince, another document viewer from the Software section. I found the performance better than the packaged PDF reader and I know it features more document manipulation option.

Out of curiosity, PDFs are bloated to begin with. Has anyone with experience on other handhelds had a PDF program that was 'fast'? I loaded up a 280KB subway map and wasn't impressed with how the machine choked until I thought about it... 280KB. That's huge.