It seems after serious debugging I find the WiFi problem, it seems the stlc45xx driver is returning bogus signal levels for some APs to the Android WiFi service which crashes and restarts the whole environment.

I have reported this to the stlc45xx mailing list so it gets fixed there and in the meantime I'll code a fix for this bug in Android too.