Thanks Johnx for the details.

I was lazy and didn't want to build xutils-dev so I created the makefile out of scratchbox. Then I modified the makefile to fit to armel and started the build with skipping xmkmf.