maemo.org - Talk - ANNOUNCE: Diablo-Turbo first beta available

Announce: First beta available
18/03/2011 Update 2:
kernel-dt-sd: adds many bugfixes and optimized bluetooth UART FIFO
ramzswap: fixes startup when there is no swap file
mini-rc: adds workaround for shutdown issue
bluez-utils: new package in DT, adds optimized arm sbc encoder for people using a2dp
dbus: fix for boingomobile

What's this
An effort to improve the user experience in N8x0 running Diablo/Community SSU, by providing a new kernel (and some other related packages) with increased interactivity and new features.

What improvements can you expect
The device will be more responsive, since the cpu time and other resources will be used more fairly.
While using the DSP (playing music with the built-in media player and some others which use it as backend) you can get up to 40% more battery life (but lower speed), or 20% more speed (but less battery life). Advanced-systemui (ASUI) provides a nice UI to change this setting (op_dsp), or you can change it from xterm by typing "echo X >/sys/power/op_dsp". You can read what values can be used and their meaning in /sys/power/op_dsp_range and /sys/power/op_list_info. You can set a default value at boot time by editing /etc/kernel-dt-tunables (OP_DSP variable)
You can set the device to fully boot to desktop if it is off when connected to charger, by editing /etc/kernel-dt-tunables (BOOTREASON variable)
A new touchscreen driver which will produce less false events. It can be tuned by editing /etc/kernel-dt-tunables
Rotation support (you will need to have Community SSU installed)

Downloads (look at the bottom for install instructions):
https://garage.maemo.org/frs/?group_id=2036

The gory details: What you get with each package:

kernel-dt-sd:: The diablo turbo kernel and initfs modules. The installer backups the current running kernel and initfs modules. If you uninstall this package, everything will be restored to the previous state.
This kernel provides the following features and improvements over the stock diablo one:

Very much improved touchscreen driver: I've found that the touchscreen is prone to lose touchscreen events, may be for aging hardware or for whatever reason, producing penup events. I have improved the touchscreen driver to avoid that problem as much as possible, by adding polled checks when a penup event is about to be produced and when the pen is supposed not to be on the screen. Two new kernel tunables to adjust the touchscreen sensitivity: /sys/devices/platform/*/penup_time (time in ms) and /sys/devices/platform/*/min_pressure (from 1 to 4091). To check if this driver is improving your device, use the touchscreen for a while (i.e. make a drawing), then from a xterm type "dmesg|grep ^TSC"; if you see any messages, then it is working for you.
High speed mmc patch
mmcblockX ordering fix patch for internal and external mmc
usbnet for windows fix
Rotation support
From community SSU kernel: FM band selection support for Japan/Europe; DSP framebuffer sharing
OP_DSP: Under the Diablo/Community SSU kernel, the CPU/DSP frequency is locked at 330/220 MHZ when using the DSP (playing music). DT kernel supports all CPU/DSP frequencies when using the DSP, so you can select if you want the most performance or the most battery life. Provides a new kernel tunable: /sys/power/op_dsp
Writable /proc/bootreason: provides support for booting on charger feature.
SWAP_PREFETCH: When the virtual memory subsystem is idle for more than 5 seconds, it will start copying pages from swap into the swapcache while keeping the copy in swap. This means the following: actually, when you use a memory hog, much of the system is swapped out. If you close the memory hog and leave the tablet for a while, when you come back to use it again it will be sluggish because it has to move the pages to memory back from swap. With this feature, when you use the tablet again, it will be very snappy because the needed pages are already in ram, and if what you use is a memory hog, it will be snappy because the pages are already in swap.
SD Scheduler: The diablo kernel ships the O1 scheduler (official scheduler for 2.6.21 kernel) which fails to provide 100% interactivity all the time. Not only that but it is far far less forgiving of the many userspace applications that do not use extremely high performance userspace programming techniques, locking and thread manipulation (ie just about all games for example).
The SD scheduler was the favourite replacement scheduler of many people at the time, in particular gamers. It provides fairness at all costs determined by nice level, maintaining as good interactivity as can be allowed within the constraints of strict fairness.
What does that mean?:
-- No process will be stalled. For example, under O1 it is easy to stall the watchdog kicker process by swapping. Once it has been stalled for more than 20 s, the tablet will reboot. SD means no more reboots caused by load.
-- Low latency
-- Better interactivity
-- Fair CPU distribution. For example, under O1 mplayer will always skip when playing ogg if you connect to a wifi AP, because it won't get enough cpu to keep the buffer full. Under SD, you can make it to not skip, by adjusting two kernel tunables.
Provides two new kernel tunables: /proc/sys/kernel/rr_interval and /proc/sys/kernel/interactive
You can read more about the SD scheduler design here: http://ck.wikia.com/wiki/SD
Virtual memory kernel tasks run at the effective priority of the running task
Files too big to fit into pagecache are put at the tail of the inactive list to minimize harm to the pagecache. New kernel tunable /proc/sys/vm/tail_largefiles
Voluntary kernel preemption: in non-preempted kernels, when a task calls the kernel, it can't be interrupted untill the call returns. This means that, sometimes, a low priority task can block for a long time more important tasks. A voluntary preempted kernel adds some points where the task can be interrupted even when in a kernel call, allowing other tasks to do their job.
256 HZ timer: The diablo kernel has a slow 128 HZ timer, strongly related to the number of task switches the task scheduler can do. This kernel is configured with a 256 HZ timer, and additionally the SD scheduler allows a task switch within a much more shorter time (defined by the rr_interval tunable). This means that the device will be much more interactive.
Improved Bluetooth UART FIFO: Reduces cpu usage when using bluetooth devices.
Many bugfixes from kernel mainline

ramzswap-dt: The Diablo-Turbo kernel swap module. Provides swap space as compressed ram, greatly increasing interactivity for a small amount of ram. For those using it, replaces ramzez-init, and has several improvements over old ramzez-init (more efficient use of ram and cpu, returns memory to the OS when it is no longer used, allows use of backing swap file).
Swap size can be configured by editing /etc/default/ramzswap, or with ramzez gui application. Swap size is 96 MB by default. It is recommended to use a traditional swap file of about 32 MB, just in case the ramzswap partition fills (doesn't happen often).

dt-init-scripts: Provides a startup script to adjust kernel tunables and provides support for fully booting when pluggin in the charger. Configured by editing /etc/kernel-sd-tunables. Also, fixes initfs linuxrc so ext3 rootfs partitions are mounted as such at boot and not as ext2. If uninstalled will restore initfs linuxrc script to its previous state.

mini-rc: Support for dt-init-scripts at shutdown, so booting on charger configuration changes are properly written to initfs when shutting down or rebooting.

dbus, dbus-1-utils, libdbus: Bugfixes: provides proper support for eavesdropping. A must for developers.

gconf2, libgconf2: Bugfixes. DBus eavesdropping exposed a nasty bug in libgconf2, this package fixes it along several other bugs and optimizations.

sysvinit: killall5 freezes itself under the SD kernel, so shutdown/reboot scripts hang and the device reboots when the watchdog triggers. This package fixes that allowing proper shutdown/reboot to happen.

gpsdriver-guard: This one has been available in extras-devel since a while. It is a fix to prevent the N810 gps driver from growing indefinitely eating all available memory.

bluez-utils: This package has an optimized arm sbc encoder, reducing greatly cpu usage and thereby increasing battery life for people using a2dp bluetooth devices.

NOTES:

The backups are held in /usr/share/Diablo-Turbo/backups and take a bit of space. You should have at least 8 MB of free rootfs space (4 MB if running in mmc) before installing the kernel.
The osso-software-version-rx?? (??=34 for N800, 44 for N810, 48 for N810-WiMax) package (Diablo and Community SSUs) locks dbus, gconf2 and minirc versions. To avoid any problem with apt-get or the gui installer you should install osso-software-version-rx??-unlocked
For now, I have tested installation only with dpkg. Any issue with GUI installer will be supported later, but not at this stage.
kernel and kernel modules sources haven't been properly packaged for autobuilding and manual patching and building is required. While I manage to create simple deb source packages, I'm a rpm guy and I'm lost with the debian build system (and to be sincere, don't like it at all). I need help to build a proper kernel source package.

Known issues:

If you're running your system from mmc, the recommended filesystem is ext3. If you have ext2, it is recommended to convert to ext3; DT has a issue at shutdown where it sometimes hangs at the killall5 stage, so when it happens the filesystem is not unmounted before shutdown, and ext2 has issues if it is not unmounted before shutdown. Latest testing kernel seems to definitely fix the shutdown issue.
There is a race in some driver which occasionally crashes the kthread, causing the hardware keys to stop responding; for me it is about twice a month. I'm trying to locate the culprit.

How to install:
(Updated instructions thanks to ggoebel)
Pre-requisites:
rootsh: http://wiki.maemo.org/Root_access

Optional Pre-requisites: CSSU installed on top of latest Diablo firmware, Advanced systemui
Updating Firmware: http://wiki.maemo.org/Updating_the_firmware
CSSU: http://wiki.maemo.org/Diablo_Community_Project
ASUI:http://talk.maemo.org/showthread.php?t=63292

Download Diablo Turbo zImage and *.deb files from:
https://garage.maemo.org/frs/?group_id=2036
(don't download files under the sources or obsolete headers)

Create directory /media/mmc1/dt (your external SDHC card)
move all saved dt files to this location

[Verify you have at least 8 MB free]
Open filemanager, press the info button or file->details

[become root:]
start utilities, xterm
> sudo gainroot

[Repository extras-devel or CSSU repository must be enabled to install this]
(where ?? substitute 34 for N800, 44 for N810, 48 for N810-wimax)
> dpkg -r osso-software-version-rx??
> apt-get install osso-software-version-rx??-unlocked
> cd /media/mmc1/dt

[now to install all debs expect an error about sysvinit when you execute the following...]
> dpkg -i *.deb

[work around the aforementioned error by installing sysvinit again]
> dpkg -i sysvinit_2.85-22.osso15-dt1_armel.deb

[optionally, upgrade to the testing kernel]
> fiasco-flasher -f -k zImage

reboot

start utilities, xterm
> uname -a
[you should see the following...]
Linux noname 2.6.21-omap1 #2 PREEMPT [build date] armv6l unknown