I was going to mention them, but you beat me to it.

I played around with a gumstix for a project my employer was bidding on and it is a pretty nice little system. No Java, as far as I know, but they do have a nice Debian-like packaging system with lots of add-on packages. There's on on-board LCD controller, ethernet (although you need an add-on board for the PHY), a couple serial ports, and a bunch of I/O.

The main selling points are the physically small size and that the price is quite reasonable in small quantities. A couple hundred for a development system IIRC.

We were going to mount it as a daughter card on a bigger board with various peripherals on it. Unfortunately, we did not win the contract so I didn't go any further with my testing of it.