you got it, so maybe the sleep timeout is still too short? you can boot system, as root run 'cat /proc/kmsg' and measure time between switching to host mode or attaching cable and final message with disk partitions detected

I'll try it this evening, usb card reader with my regular bootable card should be enough for such test.