The bleeding-edge wifi drivers contain some bugfixes to osso-wlan (wlancond), propably this is the reason that with or without the drivers loaded, the connection is better for SIP? See blog.