Regarding the rebuilding of libraries, Sailfish is based on Mer, which uses the Open Build Service to build everything.

OBS makes re/building stuff for multiple architectures very easy once you have all the targets, builders and related infrastructure setup.
You basically just upload a new source tarball & a spec file and the package and all packages depending on it are automatically rebuilt.

It also supports personal side subprojects and sending packages between project with a review step. Sou you can for example configure your own subproject against the Mer core repo, make sure your library builds fine on all the Mer architecture target and then send submit the package for review.

So while ARM indeed is currently more prone for exotic build failures, the rebuilding logistics itself should not be an issue for Sailfish due to OBS.

PS.: I kinda expect that once a Sailfish device is publicly released that either a public Sailfish OBS shows up or a Sailfish target will be added to the Mer OBS. Like the Harmattan target on the late COBS.