I used http://www.gentoo.org/proj/en/base/embedded/handbook/ to setup
a cross compiler for the N810 and emerge system with slonopotamus' CFLAGS.

Unfortunately, I couldn't get it past GNU gdbm -- the build fails due to some library being in an "unknown format". I assume libtool is searching the amd64 library path instead of the arm one.

I think this is the way to go; selfhosting is a nice thing to have but even syncing the portage tree is a way too slow operation. Not to say I prefer building the image once (on my HDD) then writing it to the SD Card instead of building it on-the-fly to the SD Card (considering write cicles and such).