I was trying this just yesterday and my compass works just fine, correctly pointing north with TOHKBD attached. I even tried to move detached keyboard part around the Jolla but even that did not influence the readings.

Could be that it is also using the accelerometer/gyroscope data for additional filtering ? Like ignoring magnetometer signals indicating rotation if the gyroscope reports no rotation ?