Battery will get hotter not due to overclocking but due to how much current is being pulled off it. Obviously, with overclocking, CPU uses more power and hence it will pull more current than stock when in load. But this doesn't necessarily mean that the battery will be cooler with a non-overclocked N900 at load. So it's quite inaccurate to base your phone's stability or instability by that temperature.

Unfortunately, there is no CPU core temp sensor. So just forget about trying to measure the temperature as the value you see won't be that much useful to you to base anything with.