Ah, no, we're not talking about actual batteries here, I think. Although yeah, they can leak and catch fire. I think the worry is about fake batteries whose internals have been replaced by an explosive; since batteries are generally enclosed in a metallic sheath, and can't really be opened without being destroyed, they seem like a perfect place to hide a bomb.

But yeah, I may be wrong, and the postal services just paranoid about real batteries...

EDIT: Yeah, I guess I should read up a bit more before spouting my opinions. I guess the issue at hand really is about the dangers of regular batteries; here's a US government advisory on the subject...