Stock kernel and NAT support
 

Hi all.
After two days of pajama hacking I'm now sure that stock 2009.42-11 kernel cannot support NAT modules in a proper way, because the netfilter core is compiled in without CONFIG_NF_CONNTRACK or CONFIG_NF_CONNTRACK_MODULE (see line 214 of net/netfilter/core.c).

So, no matter how hard we try to get a "debian-polite" way to add needed modules to this kernel, they will simply not work.

I'm posting this to see how other people involved in tethering projects would think about what we are in front of, specifically:

* Build a new, more modulable, kernel without getting too far from the rx51_defconfig and go with it.
* Go completely userland via proxies of any sort, even heavily tweaked for low resource usage.

Both ways have a lot of bad points, but I'd like to know which one people would choose.

(btw, I think I found an almost elegant way to package "internal" kernel modules, ask freely if interested)

Re: Stock kernel and NAT support
 

Just in case you hadn't seen it, there's some discussion of iptables and kernels starting around page 2 in this: http://talk.maemo.org/showthread.php?t=30916&page=2

I'd be very grateful to see an easily installable solution to this!

Re: Stock kernel and NAT support
 

Yes, I've seen the other discussion but I think the issue, although fundamental for tethering, is a bit more large.

What I'd like to start here is an effort to have ONE community kernel or a common toolbox with specially tuned proxies to serve most tethering needs.

Re: Stock kernel and NAT support
 

well, since the amount of feedback was so heavy (maybe I posted this in the wrong section, feel free to tell me), I'll work the two ways in parallel.

I don't know if it's legit to create a garage project for an unique community kernel, so I'll host on my own space waiting for community input.

For the generalized proxing toolkit I'll first create a garage project containing these things:

* a non caching userland http / https proxy
* a userland forwarding daemon
* a cli tool to easily configure both
* a gui tool to configure both

* maybe (maybe) python bindings for the cli stuff

please, keep in mind that I'll publish stuff only when I reach a "debian polite" and "maemo polite" way to distribute it.

stay tuned, i'll be back in a week.

Re: Stock kernel and NAT support
 

You might want to take a look at what Jebba has been up to in the NAT and kernel areas:

http://wiki.maemo.org/User:Jebba/NAT

http://wiki.maemo.org/User:Jebba/Kernel

I do think getting the various kernel options together under one roof would be a great step.

Re: Stock kernel and NAT support
 

I admit I've stumbled on Jebba's pages just a pair of day ago, and there's a lot of things down there... I'll study his stuff more deeply in next days.

Thank you fir the hint, even if I'm a fan of reinventing the wheel (for learning sake, poor me)!

Re: Stock kernel and NAT support
 

On my N810 I used a slighly different userland aproach.
I took the userland-network (slirp) code from qemu and
plumbed it to the interface the to be natted clients were behind
using pcap.
This gives you a completely userland-based NAT solution
with even a dhcp-server and dns-forwarding incorporated.
Of course it also has all non functional disadvantages of a
userland solution like additional delay and cpu-utilization but
apart from that from a functional perspective it behaves like
NAT. I even used to use VoIP and proprietary tunnel-clients
behind it.

Re: Stock kernel and NAT support
 

Although it's more work, I think both approaches (new community kernel, and userland stuff) are worth pursuing. The userland solution would be useful for people who just want to install a normal app and don't want to tinker too much with their device. A community kernel, on the other hand, would be of interest to those who don't mind hacking a bit. I would prefer to see only one or two community kernels, though, so that the community doesn't get too divided like the Zaurus community rather did.

Re: Stock kernel and NAT support
 

Very, very interesting! Looks like the best approach for the userland way, I'm looking into it.

Re: Stock kernel and NAT support
 

Hi mikalil,

I am an Acquisition Editor with a publishing house. We're working on a book on Nokia application programming and we need a technical reviewer to review the book. I went through your posts and your knowledge on the subject looks impressive. If you're interested, please get in touch with me at taruns@packtpub.com for details.

Re: Stock kernel and NAT support
 

Hi mikalil,

I am an Acquisition Editor with a publishing house. We're working on a book on Nokia application programming and we need a technical reviewer to review the book. I went through your posts and your knowledge on the subject looks impressive. If you're interested, please get in touch with me at taruns@packtpub.com for details.

Re: Stock kernel and NAT support
 

Do you happen to have any notes online anywhere with a bit more info on how to do it ? (see this thread about related NAT discussion)

Re: Stock kernel and NAT support
 

It is a bit sad that such a fascinating and tantalizing post was made by a one-post user. How do you extract the SLiRP code from qemu and "plumb it to" the wlan interface "using pcap"?

Re: Stock kernel and NAT support
 

Unfortunately my spare-time for topics like these was a little limited recently. Regarding your questions the slirp-code inside qemu is
concentrated in a directory called slirp. So the idea is to simply take
this code build something like the main-loop of qemu add some
glue-code to make references happy and instead of communicating
with virtual ethernet-interfaces of virtual machines make it use
pcap instead. Under http://article.gmane.org/gmane.comp....ors.qemu/39831 you
can find a patch which brings pcap-support to qemu (not in the
official qemu because of some drawbacks which don't hurt us
here). So all code is basically there what is missing is to remove
the actual qemu and add some glue, command-line handling and
so on...

Re: Stock kernel and NAT support
 

Obviously the task is even easier than I had thought. Yesterday I
finally found the time to port the patch mentioned to
qemu-0.12.2. Once I had fired up a qemu virtual machine with
one virtual ethernet interface as
userspace-networking (using slirp code) and one virtual ethernet
connected to eth0 using pcap both connected to vlan=0 (behaves
like a bridge) on my sles11-linux-box I had exactly the desired behaviour.
(user-space nat, dhcp-server, dns-forwarding)
What is running inside the virtual-machine is of course
quite irrelevant. In fact the whole virtual-machine-stuff shouldn't be there
making the approach quite bloated and unnecessarily complicated to
port to freemantle-arm-scratchbox.
The commandline for this looks something like "qemu -boot d
-cdrom /windows/D/iso/SUSE_SLES-11-0-0.001.ISO -m 512
-net nic,model=e1000,vlan=0 -net pcap,ifname=eth0,vlan=0
-net nic,model=e1000,vlan=0 -net user,vlan=0".
So if somebody is willing to help it would be helpful to compile the
qemu-tools out of the qemu-0.12.2 package for
freemantel-arm-scratchbox.
The configuration which makes this work should also be fit for
compiling a qemu stripped down to just the networking stuff. I'm
thinking of something like qemu-img just for networking instead
of blockdevices.

Re: Stock kernel and NAT support
 

Hi Wek,
unfortunately I'm living a bit busy right now so I've put on hold a lot of things, even this one, but I'm posting some of my doubts regarding the silrp solution. Maybe it cuold be useful for other readers.

What we are trying to do here is bridging a tap/tun to an interface with UNSPEC link type (gprs0), is it even possible? Even if the qemu code is about untagged vlan, somebody already tried to bridge that interface with anything resembles an ethernet?

Then, let's talk about layer 3: for what I can see we add an interface beetwen the gprs0 and an ipotethical pan0 (we're still talking about tethering, right?). But at IP level what it gives us?

Even if we bridge successfully everything how could the stuff route correctly packets out of gprs0 without senting out routing errors?

Imho the whole approach as flaws so big that any resolutuin wouldn't be any smaller than rewrite lot of ip routines in userland...

I will elaborate more only if requested, since I'm living on the run :)

Re: Stock kernel and NAT support
 

Shame on me not to think of that aproach earlier especially as I know
vde.sourceforge.net already.
This might be aproach where no codechanges at all might be needed.
VDE is the equivalent to an ethernet-switch realized as a userspace
application. There are several plugins available. Amongst these
there are a tap-plugin and a slirp plugin. Haven't checked enough
to see if there is a pcap plugin as well. If not it is probably not
complicated to write. But maybe we wouldn't even need it.
Haven't checked about bridge & tap - support in the official
freemantle-kernel. Does anybody know if these features are there?
If yes the solution would be to setup a vde-switch with a tap and
a slirp - end and to bridge the tap with the wlan0 or pan0 device.

As from what we have seen (kernel module piping data to user
space) regarding Joikuspot on N900 this guy obviously hasn't
found a way to handle the NAT in the kernel either (without building
a new kernel).
So I think my aproach will perform at least as well as this one
with the two advantages that it is free and that the userspace-nat-code
used is really well tested in the virtualization world (at least qemu, kvm
and virtualbox) and as it is open-source any application-support
missing can be added instead of being bound to the protocols supported
by a closed source product. Btw. does anybody know which protocols
are supported by the Joikuspot-NAT-code - probably no difference to
what works on the symbian version. I'm thinking of things like active FTP,
IPSec-Tunnels, telco-protocols... where e.g. the linux-kernel has
Nat-Helpers for.

Re: Stock kernel and NAT support
 

Well I have it working on my PC which should be enough proof
for the principle ;-) And - as already stated - I had it running
on my unfortunately stolen N810 between wlan0 and pan0 or
alternatively an usb-plugged ethernet (unfortunately not possible
with N900 due to missing OTG as it looks).
Actually it shouldn't make any difference how the internet
connection looks like as long as a working socket-interface is
provided to userspace-applications.

Re: Stock kernel and NAT support
 

Well if anyone knows how to actually use it, vde2 can be installed in the Easy Debian chroot on any of the Maemo devices. But I have no idea how to use anything in that package to build a userspace router.

Re: Stock kernel and NAT support
 

Setting up a virtual-switch with one pcap-leg to pan0 and/or
wlan0 (checked seems to be
there) and one slirp-leg should do the trick...
As an alternative to the pcap-leg you can try a tap-leg for vde and
bridge it with wlan0 and/or pan0 (bridge enabled in the kernel?
bridge-utils available?).
Note regarding slirp in vde: Once inherited from qemu but dates
March 2008 which would mean that latest refactoring in qemu
didn't go in it. Maybe not ideal but for our purpose probably
sufficient.

Re: Stock kernel and NAT support
 

wek: can you post the commands to set that up?

According to ifconfig, we have "wlan0" which is the wireless network, and "gprs0" which is the data connection we'd want to share.

Re: Stock kernel and NAT support
 

The binary-packet I just downloaded unfortunately doesn't have
the vde_pcapplug so I can't check if it works like that but you
can try something like this as a minimal setup (everything in an
extra console till it works afterwards all modules offer daemonization):

vde_switch -s /var/run/vde_sock
slirpplug -s /var/run/vde_sock -D
vde_pcapplug -s /var/run/vde_sock wlan0

We don't care for gprs0 as this is gonna be done by the
IP-stack.
I found a couple of hours time in the train so I'm quite far
with stripping down qemu to what we need here. So I will
try to go for that to have everything in a single binary and
to avoid the socket-communication.

Re: Stock kernel and NAT support
 

wek:

Ok, so the first command runs OK (I needed to be root to have permissions to /var/run/vde2/vde_sock)
The second command doesn't exist, but there's one that seems to do the same thing, "slirpvde", and it returns, "IP address of your DNS(s): 0.0.0.0"
The third command fails, with the error message, "vde_pcapplug: Open wlan0: wlan0: That device is not up"

I think the problem here is that I have to manually bring up the wlan0 connection in ad-hoc mode (Maemo only allows one network interface to be active at a time, so I have the gprs0 connection open).

How do I bring up wlan0 in ad-hoc mode without disconnecting gprs0?

Also, is slirpvde the same as slirpplug? "man slirpvde" says that it is "a slirp interface for a VDE network".

Re: Stock kernel and NAT support
 

i did not try vde yet as i don't have debian-chroot installed atm. Setting the wlan0 to ad-hoc however basically works like this:

iwconfig wlan0 mode ad-hoc
ifconfig wlan0 [ip adress]
iwconfig wlan0 essid [name]

for testing this should do, have not yet tried to connect with another machine though.

misses encryption and dhcp server, so connecting hosts need to have a fitting manual ip setup

Re: Stock kernel and NAT support
 

Good news! And bad news.

Combining greygoo's instructions with wek's instructions (with my "slirpvde" modification) sets up a joikuspot-style router with a stock kernel... for a second, until my N900 rebooted.

0. Install Easy Debian and apt-get install vde2

1. make sure the N900 is on the GPRS data network (not wifi)

2. run the three commands from greygoo as root (I don't have iwconfig in Maemo, so I also ran these in a root Debian terminal):

3. run the three vde2 commands in three Easy Debian root terminals (sudo debian):
4. Find the "N900" hotspot with your N800 (or laptop if you aren't cool enough to own an N800) and connect.

5. Get the IP address (yes, you get one!)

6. Try to load a web page... see the web browser show the title of the web page at the title bar... :D then watch the N900 reboot... :( ...or watch the web page load at a snail's pace, or stall, or not load at all...

Re: Stock kernel and NAT support
 

As you said good and bad news...

Your IP-Address should be 0.0.0.0 instead of 10.0.0.1 to keep
the IP-Stack out of the game.
But I'm afraid this is not our problem with stability.
I would expect that libpcap is not properly built against
freemantle but this is just a guess.
Something else which came to my mind is the following:
Per default, the device is probably opened promiscuous, which
might not work properly with wlan0 and as I remember from my gone n810 pan0.
Don't know if slirpvde (sorry for my mistake here
with the naming) has an option to do that.
I remember that I had changed that in my implementation on
the N810. Of course if the interface is not opened
promiscuous then slirp has to be forced to use the mac-address
of your wlan0 or whatever device you are connecting to.
Also remember to have built a mac-selection-option into
slirp. Can't remember if that went into the current slirp (the one
in current qemu) but it is for shure not in the old implementation
coming with vde2.
Looks as if I should try to dig out my old implementation (last
version is for shure gone but there might be something left
somewhere) and give it a try to see if the n900 behaves
differently in comparison to the n810...

Re: Stock kernel and NAT support
 

Tried a few more times. N900 didn't reboot, but the whole thing was terribly, terribly slow and not very stable-feeling.

Corrected my instructions; I had the order of 1 and 3 wrong. First you set up wlan0, then you set up the router.

wek: How do I run the VDE apps as daemons?

Re: Stock kernel and NAT support
 

In case of slirpvde use -d for a daemon (surprise ;-) ). The
others are probably similar.
With -p you can create a pidfile which you should use in
your up/down-script to clean up properly.
The reliability-problem could be due to various reasons.
Have you set the wlan0-ip to 0.0.0.0?
If you use wireshark (tcpdump or any other sniffer) on the
box connected via our hotspot which mac for 10.0.2.2 do
you see? The same you get when you do ifconfig wlan0
on the N900?
Try with pan as an alternative to see if it has to to with
ad-hoc on N900 in principle. v(Would probably anyway
more suitable because of power-consumption on the
long - aeh more short ;-) - run.)

Re: Stock kernel and NAT support
 

Reboot might be caused by a bug in the minstrel wifi driver, you can try recompiling the module with this patch.

Re: Stock kernel and NAT support
 

Finally found some time to invest into the stripped-down qemu approach.
Now I have a 200K (probably can be made smaller but for a test this
should suffice) binary running on my SLES machine basically doing
what qole achieved with the 3 vde daemons.
I observed a thruput-issue as well: Limitation to 570KBytes/s per
TCP-stream. As pcap doesn't have any knowledge of TCP-streams
and I don't think it is a delay/windowsize issue it probably
originates somewhere inside slirp. Though with a 100KBytes/s
shaping per TCP-stream of many servers and the bandwidth offered
by the providers I have been using till now this is kind of academic.
CPU-Load isn't dramatic as well.
Now the bad news: Last time I used scratchbox was chinook ;-)

Re: Stock kernel and NAT support
 

yep, that's it.
Writing this connected via PAN to my N900 which does GPRS (well
in the train even this is luck).
The vmware-image with scratchbox is really cool.

Re: Stock kernel and NAT support
 

Added a udev-rule which whenever a bnep[0-9]* comes up
starts an instance of my userspace-nat daemon and kills
it again when the bnep-device disappears.
I think I'm gonna stay with bluetooth connection for now.
It is simple, costs comparably little CPU-Power and when
doing a terminal session to my N900 it is much less choppy
as when I do that when Laptop and N900 see each other
via a WLAN-Access-Point at home.
And what might be interesting in addition PAN leaves
WLAN unused to e.g. connect to a public WLAN-Hotspot
and share this connection.
One drawback I see at the moment is that 2 PAN clients
connected to my N900 don't see each other at the moment
as they are living each in their private userspace network-stack.
Does anybody know if it is easily possible to get the linux-bridge
to work with stock kernel N900?
Other solution would be to have on the fly interface-adding/removal
to a single instance of the userspace-nat-daemon.
By the way I can even switch back an forth between WLAN-Hotspot
and UMTS as I'm just doing while writing this text.

Re: Stock kernel and NAT support
 

How to get IP address?

I need to implement the scheme: internet wifi NAP> N800 (Easy Debian)> usbnet> PC (Ubuntu)
It may be an option Bluetooth PAN, but there is less difference in the speed of usb.

I combined the instructions:
http://wiki.maemo.org/USB_networking (the role of the host performs the N800)
http://talk.maemo.org/showpost.php?p...4&postcount=25 (wlan0 changed to usb0)


My actions:

(/etc/network/interfaces is empty, not usb0, bnep0)

1. N800:
/etc/init.d/usbnet start
/etc/init.d/usbnet start
/etc/init.d/usbnet stop
/etc/init.d/usbnet start

2. PC: modprobe usbnet
(подсоединяю кабель)
3. N800: ifconfig usb0 10.0.0.1 up
4. PC: ifconfig usb0 10.0.0.2 up
root@usxakov:/home/temp.info/n800/man# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_req=1 ttl=64 time=3.09 ms
64 bytes from 10.0.0.1: icmp_req=2 ttl=64 time=0.491 ms
64 bytes from 10.0.0.1: icmp_req=3 ttl=64 time=0.491 ms
^C
--- 10.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.491/1.357/3.091/1.226 ms

5. N800:
vde_switch -s /var/run/vde_sock -d
slirpvde -s /var/run/vde_sock -D -d
vde_pcapplug -s /var/run/vde_sock usb0 -d

6. PC:
root@usxakov:/etc/network# dhclient -d -s 10.0.0.1
Internet Systems Consortium DHCP Client 4.1-ESV-R4
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/usb0/72:2c:ae:5a:c8:76
Sending on LPF/usb0/72:2c:ae:5a:c8:76
Sending on Socket/fallback
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 3
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 8
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 7
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 9
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 12
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 13
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 15
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 20
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 20
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 17
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 18
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 11
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 11
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 11
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 15
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 11
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 16
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 15
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 13
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 11
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 15
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 10
DHCPDISCOVER on usb0 to 10.0.0.1 port 67 interval 18
No DHCPOFFERS received.
No working leases in persistent database - sleeping.

P.S.
debian-n8x0-v3c.img.ext2 - works well, but does not support vde_pcapplug (vde 2.2.3)
debian-squeeze-img.tar.bz2 - browsers do not work
ubuntu-mer-java-img-v2.tar.bz2 - no image connected