Anyway, the only hardware testing program for Nokia 770 that I know about at this moment is memtester: http://pyropus.ca/software/memtester/

A binary for Nokia 770 is here: http://ufo2000.xcomufo.com/files/memtester.gz
It should be run as root and you need to specify the size of block you want to allocate and test from the command line:
# ./memtester 20

This way is allocates and tests 20MB of memory. That's only about 1/3 of all the memory used in the device but I don't know about anything better yet. If you do, please let me know.