OK, got it to boot. There is one more gotcha. The host mode is not enabled until any usb client (!!) module is inserted. So you need to add e.g. 'g_ether' to list of modules. Also for me the device scan took more than 5 seconds. Worked with sleep 7.

The bug/feature of host mode not working with no gadget driver loaded is known, I was already bitten by this in OS2007 days.