The trouble with an updated hardware model is compatibility. One thing to avoid this which I could imagine - and would recommend to Nokia - is a device which would electrically be identical, but just have a bigger screen (double the diagonal = 4 times the area. Same resolution, just bigger pixels). Same look. Possibly two batteries instead of one, but no other extras. Then sell it at exactly the same price as the 770.

Why that:
diversify for different user populations without getting into trouble with incompatibility issues.
Relatively easy to develop 9electrically identical). Bigger screens as lower tech, might even be cheaper in purchasing. Manufacturing would also not be that different (well, somewhat, for the mechanics...)

I assume that the 770 - and the whole maemo idea/family, commercially, has not reached break-even for Nokia, and will not, for quite some time to come. So there is money being invested without sufficient return. And a bigger (but not "better") device would be an easy option to test the market agains other devices: what I have seen from the windows world : they are biger than the Nokia 770.