Needs some deeper tutorials, but an example from debian/rules goes like this (for the kernel zImage build)
kernel-stamp: configure-stamp
cd $(KSRC) && $(MAKE) ARCH=$(ARCH) $(NJOBS) $(EXTRAVERSION) zImage

cd ./kernel && make ARCH=arm -j6 zImage

Since $(KSRC) contains the build path and
$(MAKE) the path to 'make'
$(ARCH) is the architecture we are crosscompiling to
$(NJOBS) is number of concurrent threads (I believe default is 6)
$(EXTRAVERSION) is not defined, thus empty

So, you could compile using something like
cd kernel
make rm581_defconfig
make ARCH=arm -j6 zImage