It should be OK...
I say this based on the fact that N800 leather cover sold by Nokia is attached to the back of the unit via a magnet...

I adapted this cover for my newish N810 and have experienced no ill effects with on and off use for close to 30 days now.