Run a traceroute to one of the slow pages (according to your description any should be fine). Most likely you'll see then where the problem is.