maemo.org - Talk
Page 5 of 8 « First 34 5 67 Last »
Show 40 post(s) from this thread on one page

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Maemo 5 / Fremantle (https://talk.maemo.org/forumdisplay.php?f=40)
-   -   BFS for the power kernel (https://talk.maemo.org/showthread.php?t=58780)

Tigerite 2011-05-30 07:38
Re: BFS for the power kernel
 
Which changes were those? I think they're all present in the git repository - plus an update to BFS 404 and ck's latest patchsets (although there's one more I'll be posting today, an update to the most recent mm-lru_cache_add_lru_tail-1.patch). If by lxp's wl12xx patches you mean these ones I'm pretty sure they are there too, as of "kernel-power v47 -> kernel-bfs" (minus the meta-package patch). I need to recompile the bleeding-edge modules against the latest headers and modules, which I'll try to do at some point today as well, along with fcam-drivers.

Oh and I've had no distortion since running 404, even with the sched_reset_on_fork patch in place - this may have been because I raised CONFIG_HZ to 300 (from 100, although I'm not sure this is needed any more since 404 apparently "addresses a long-standing bug that affected all configurations, but was only demonstrable on lower Hz configurations (i.e. 100Hz) that caused fluctuating performance and latencies"), or because I'm using the pulseaudio package by Jyri Sarha (of Nokia) from here:

Code:
The old free list implementation used objects in FIFO style. This is
bad because it tries keep all the objects ever used alive and in
memory. This minimizes the changes that an allocated object is already
in cache. When there is shortage of physical memory this may also
increase change that newly allocated object is swapped out. LIFO
(e.g. stack) style free list should help these issues. Like the old
one the new implementation is also lock free. This version (v2.1) of
the patch has a potential weakness fixed. The previous version (2.0)
did segfault when popping from empty flist, this does not.
.. or because of the ck patchsets. I'm going to lower the CONFIG_HZ back to 100 today to eliminate that possibility at least.

PS I also compiled with CONFIG_WATCHDOG=n and haven't noticed any side effects from doing so.

epitaph 2011-05-30 08:11
Re: BFS for the power kernel
 
Quote:
Originally Posted by Tigerite (Post 1018360)
Which changes were those? I think they're all present in the git repository - plus an update to BFS 404 and ck's latest patchsets (although there's one more I'll be posting today, an update to the most recent mm-lru_cache_add_lru_tail-1.patch). If by lxp's wl12xx patches you mean these ones I'm pretty sure they are there too, as of "kernel-power v47 -> kernel-bfs" (minus the meta-package patch). I need to recompile the bleeding-edge modules against the latest headers and modules, which I'll try to do at some point today as well, along with fcam-drivers.

Oh and I've had no distortion since running 404, even with the sched_reset_on_fork patch in place - this may have been because I raised CONFIG_HZ to 300 (from 100, although I'm not sure this is needed any more since 404 apparently "addresses a long-standing bug that affected all configurations, but was only demonstrable on lower Hz configurations (i.e. 100Hz) that caused fluctuating performance and latencies"), or because I'm using the pulseaudio package by Jyri Sarha (of Nokia) from here:

Code:
The old free list implementation used objects in FIFO style. This is
bad because it tries keep all the objects ever used alive and in
memory. This minimizes the changes that an allocated object is already
in cache. When there is shortage of physical memory this may also
increase change that newly allocated object is swapped out. LIFO
(e.g. stack) style free list should help these issues. Like the old
one the new implementation is also lock free. This version (v2.1) of
the patch has a potential weakness fixed. The previous version (2.0)
did segfault when popping from empty flist, this does not.
.. or because of the ck patchsets. I'm going to lower the CONFIG_HZ back to 100 today to eliminate that possibility at least.

PS I also compiled with CONFIG_WATCHDOG=n and haven't noticed any side effects from doing so.
BFS-Kernel is really good especially when you need Flash. But with the wonderpatch and cgroups tuning BFS is useless. Where can I find BFS 404?

Hurrian 2011-05-30 09:15
Re: BFS for the power kernel
 
Quote:
Originally Posted by epitaph (Post 1018370)
BFS-Kernel is really good especially when you need Flash. But with the wonderpatch and cgroups tuning BFS is useless. Where can I find BFS 404?
http://cache.ohinternet.com/images/d...L_WUT_PEAR.jpg
BFS is a CPU scheduler, not a "renice cgroups" patch. Con Kolivas already explained the differences. Maemo is already doing a terrible job of trying to implement it's own renice'd cgroups (see syspart). This is where BFS comes in. BFS has simple prioritization AND it helps fix the "Linux is using 100% CPU... on I/O!" thing CFS has. Which helps on a device with 256MB RAM, constantly swapping, and accessing large amounts of data from the eMMC for no reason at all.

Tigerite 2011-05-30 09:31
Re: BFS for the power kernel
 
I couldn't have put it better myself - besides which the cgroups patch can also cause regressions, because it's heuristic. By the way I am running with swappiness 0 (!) with the -ck patches and BFS 404, which runs beautifully on my device. I might even try to put dirty_ratio to 1 (as recommended with 404 - at the moment I've got it set to 8)..

Also, I've included a 404 version of the patch mentioned here, but disabled (set to 0) both parameters by default. So, should anyone wish to have a similar effect to the "200 lines miracle patch", the following knobs are available:

Code:
Threads are kept at the same fork_depth as their parent process, and can
optionally have their CPU entitlement all managed as one process together
by enabling the group_thread_accounting feature. This feature is disabled
by default in this patch, as many desktop applications such as firefox,
amarok, etc are multithreaded. By disabling this feature and enabling the
fork_depth_penalty feature it favours CPU towards desktop
applications. This latter feature is also disabled in this patch by default.

Extensive testing is required to ensure this does not cause regressions in
common workloads.

There are two sysctls to enable/disable these features.

They are in /proc/sys/kernel/

group_thread_accounting - groups CPU accounting by threads
fork_depth_penalty - penalises according to depth of forking from init
Please take particular note of the part in bold. This patch does, in ck's own words, add a very small amount of overhead to the kernel (even when disabled) but I think it's worth it.

epitaph 2011-05-30 10:51
Re: BFS for the power kernel
 
Quote:
Originally Posted by Tigerite (Post 1018420)
I couldn't have put it better myself - besides which the cgroups patch can also cause regressions, because it's heuristic. By the way I am running with swappiness 0 (!) with the -ck patches and BFS 404, which runs beautifully on my device. I might even try to put dirty_ratio to 1 (as recommended with 404 - at the moment I've got it set to 8)..

Also, I've included a 404 version of the patch mentioned here, but disabled (set to 0) both parameters by default. So, should anyone wish to have a similar effect to the "200 lines miracle patch", the following knobs are available:

Code:
Threads are kept at the same fork_depth as their parent process, and can
optionally have their CPU entitlement all managed as one process together
by enabling the group_thread_accounting feature. This feature is disabled
by default in this patch, as many desktop applications such as firefox,
amarok, etc are multithreaded. By disabling this feature and enabling the
fork_depth_penalty feature it favours CPU towards desktop
applications. This latter feature is also disabled in this patch by default.

Extensive testing is required to ensure this does not cause regressions in
common workloads.

There are two sysctls to enable/disable these features.

They are in /proc/sys/kernel/

group_thread_accounting - groups CPU accounting by threads
fork_depth_penalty - penalises according to depth of forking from init
Please take particular note of the part in bold. This patch does, in ck's own words, add a very small amount of overhead to the kernel (even when disabled) but I think it's worth it.
I hate to repeat myself but with cgroups enabled and some additional tuning my device is really fast. I don't want to start a fight here but where can I find 404 and Con's patch didn't make it upstream. Anyway who wants Flash?

Tigerite 2011-05-30 12:09
Re: BFS for the power kernel
 
At the moment, you have to build the kernel from the git tree for BFS 404. My device is also very smooth and fast with BFS, and there's very little swap I/O (I've yet to see more than ~70000 bytes used in /proc/swaps). None of Con's work has ever made it upstream, if you mean into the mainline kernel, there are many historical reasons for this and I would have thought we all know them by now.. moreover he doesn't want BFS in mainline.

MastaG 2011-05-30 12:17
Re: BFS for the power kernel
 
So he doesn't want it in mainline?
But will it be merged in the power kernel then?
Or do we have to compile it ourselfs?

MONVMENTVM 2011-05-30 12:24
Re: BFS for the power kernel
 
A power kernel with the latest BFS patch packaged in a .deb would be really great.

Hurrian 2011-05-30 12:35
Re: BFS for the power kernel
 
Quote:
Originally Posted by MastaG (Post 1018545)
So he doesn't want it in mainline?
But will it be merged in the power kernel then?
Or do we have to compile it ourselfs?
CK doesn't want it in mainline because the Linux devs will just **** it up. Read bfs-faq.txt, lots of rage against the kernel.org devs and you'll know why BFS was made. Probably epitaph needs to read it too.

Tigerite 2011-05-30 13:02
Re: BFS for the power kernel
 
You can't just patch kernel-power with BFS. However, you can do the reverse; the BFS kernel has already got all the power47 patches, plus the -ck patchset which I backported to 2.6.28 (to the best of my ability). Talking of which, the git tree has now been updated with the latest to correlate with 404 as closely as possible.

MONVMENTVM 2011-05-30 13:04
Re: BFS for the power kernel
 
Quote:
Originally Posted by Tigerite (Post 1018584)
You can't just patch kernel-power with BFS. However, you can do the reverse; the BFS kernel has already got all the power47 patches, plus the -ck patchset which I backported to 2.6.28 (to the best of my ability). Talking of which, the git tree has now been updated with the latest to correlate with 404 as closely as possible.
That's what I actually meant sorry. Could you please make a .deb of that BFS kernel?

Tigerite 2011-05-30 13:26
Re: BFS for the power kernel
 
1 Attachment(s)
Ok, here you go:

Modules deb (required by all of the below)

Multiboot deb

U-boot uImage
Also requires "dummy" kernel-bfs package, see attached

Standard kernel and flasher deb

Headers deb for compiling fcam-drivers on scratchbox, etc
Also requires "dummy" kernel-bfs package, see attached

UPDATED LINKS with watchdog re-enabled - please let me know if there are any problems.

Thomashamka 2011-05-30 13:28
Re: BFS for the power kernel
 
http://gingerbreadn900.blogspot.com/

hawaii 2011-05-30 13:49
Re: BFS for the power kernel
 
1 Attachment(s)
Attached are fcam-drivers for BFS6.

The pulseaudio package didn't resolve distortion, neither did disabling the scheduler reset fork patch on my quick build.

Tigerite 2011-05-30 14:21
Re: BFS for the power kernel
 
Hmm, that's a real shame. I've still had no distortion so far (other than very slight during the "ringing" phase of a call on an odd occasion). Have you tried setting swappiness to 0? I've also got some schedtool settings:

Code:
schedtool -I `pidof hildon-desktop`
schedtool -I `pidof hildon-status-menu`
schedtool -D `pidof trackerd`
schedtool -D `pidof tracker-indexer`
schedtool -F -p 99 `pidof Xorg`
schedtool -I `pidof mafw-dbus-wrapper`
schedtool -R -p 5 `pidof pulseaudio`
.. and a few of the tunables from this thread, although I have nr_requests as 32, page-cluster as default of 5, and swappiness 0 as mentioned.

hawaii 2011-05-30 14:30
Re: BFS for the power kernel
 
I have similar scheduling on boot as well as I/O tuning settings. It may have been bumping CONFIG_HZ to 300, so I'll see what that does.

Tigerite 2011-05-30 14:45
Re: BFS for the power kernel
 
I've updated the links with watchdog re-enabled. Please let me know if it works.

Tigerite 2011-05-30 15:01
Re: BFS for the power kernel
 
.. and by popular request ;), here are the bleeding-edge wireless modules

hawaii 2011-05-30 15:16
Re: BFS for the power kernel
 
The newest kernel package without WATCHDOG=n works fine. My *just compiled* fcam-drivers don't work with your kernel,

Wanna toss those up?

EDIT: Still have distortion. Set swappiness to 0 and it seems (?) to have gone away, but it was always too random to predict.

MONVMENTVM 2011-05-30 16:18
Re: BFS for the power kernel
 
Thanks a lot Tigerite... will give this a shot right now.

Tigerite 2011-05-30 17:01
Re: BFS for the power kernel
 
Sorry for the delay - I've been out; here are the fcam-drivers (version 1.0.7.2), they should work with both the latest kernel-power and kernel-bfs.

Ayle 2011-05-30 18:08
Re: BFS for the power kernel
 
If I'm installing it over kernel-power which deb do I download?

MONVMENTVM 2011-05-30 19:52
Re: BFS for the power kernel
 
Quote:
Originally Posted by Ayle (Post 1018815)
If I'm installing it over kernel-power which deb do I download?
the kernel deb, the flasher and the modules did suffice for me.

pfontana77 2011-05-31 22:24
Re: BFS for the power kernel
 
I've installed the kernel deb, flasher and module plus the bootimg deb, so i have 3 choice in the multiboot.

I've just rebooted the phone with the bfs kernel, i wanna test for a few days before saying that is faster.

hawaii 2011-06-01 03:51
Re: BFS for the power kernel
 
-10bfs6 is highly stable and rock solid. I've been hammering on it and am extremely happy with performance.

pfontana77 2011-06-01 09:07
Re: BFS for the power kernel
 
First impression are very positive, i'm having much more responsiveness in launching and switching from an application to another. I see benefit also in web browsing with Opera, but i need more test on this.

Also, i've installed i applied the patch in Ubuntu Natty kernel, so also on desktop i see more responsiveness.

Seems a very good scheduler ... I think that this should be packaged in extras-devel.

MONVMENTVM 2011-06-01 14:10
Re: BFS for the power kernel
 
I'm having some problems lately and I don't know yet if they are related to the swap settings or the schedtool settings I've taken from here:

http://talk.maemo.org/showpost.php?p...&postcount=175

First I was really impressed by the performance of the BFS kernel. Switching apps, browsing the web while playing music and doing other stuff worked really smooth. It wasn't long though until tried the swap and schedtool settings mentioned in the post above and now I have some really strange performance issues:

When doing some I/O heavier stuff, like updating the package list in FAM, starting an application or downloading from the web, it is getting completely unresponsive for a couple of minutes. Interestingly enough at the same time conky shows me very low CPU usage.

Today I was browsing TMO on microb when suddenly the phone got completely unresponsive for a few minutes. Then it immediately got back to normal showing me a missed call. So it was so unresponsive that neither the call UI popped up nor the ringtone or the vibration kicked in.

Just now I wanted to check my Swappolube settings to post them here and it took minutes for the Swappolube GUI to load leaving the phone completely unresponsive. Now a few minutes later the app was up and running and responsiveness was back to normal.

Anyone have a clue what might be going on here?


EDIT: Checking a bit further with Conky and stuff it definitely seems to be an I/O problem. Right now I'm installing the latest qmltube update in FAM and it's taking already about 10 minutes while it's still in the download stage (which should already be complete though). Interestingly though CPU is at the lowest clock and running at a load of 4% to 12% which shows that the CPU has not much to do. The average system loads however are above 6 or even at 7 some times so there are clearly many processes waiting to be done, which leads me to believe they're waiting for I/O. Swap usage is also relatively high at almost 300MB but that doesn't indicate much. What could that be?

Tigerite 2011-06-01 18:51
Re: BFS for the power kernel
 
That's not so good. Could you please post your vm settings, in particular (mine are in brackets):

/proc/sys/vm/dirty_ratio (8)
/proc/sys/vm/dirty_background_ratio (4)
/proc/sys/vm/swappiness (0)
/proc/sys/vm/vfs_cache_pressure (1000)
/proc/sys/vm/overcommit_memory (2)
/proc/sys/vm/overcommit_ratio (100)
/proc/sys/vm/oom_kill_allocating_task (1)
/proc/sys/vm/min_free_kbytes (4096)

It's probably also worth checking the /sys/block/mmcblk0/queue settings, such as read_ahead_kb (512), nr_requests (32) and scheduler (cfq). I have tweaked the iosched folder settings underneath that to match the ones posted by freemangordon here (my page_cluster is the default of 5 however and I don't think nr_requests needs to be so high). Finally I have some schedtool settings too but I doubt they would help to fix your problems.

epitaph 2011-06-01 18:56
Re: BFS for the power kernel
 
Quote:
Originally Posted by Tigerite (Post 1020368)
That's not so good. Could you please post your vm settings, in particular (mine are in brackets):

/proc/sys/vm/dirty_ratio (8)
/proc/sys/vm/dirty_background_ratio (4)
/proc/sys/vm/swappiness (0)
/proc/sys/vm/vfs_cache_pressure (1000)
/proc/sys/vm/overcommit_memory (2)
/proc/sys/vm/overcommit_ratio (100)
/proc/sys/vm/oom_kill_allocating_task (1)
/proc/sys/vm/min_free_kbytes (4096)

It's probably also worth checking the /sys/block/mmcblk0/queue settings, such as read_ahead_kb (512), nr_requests (32) and scheduler (cfq). I have tweaked the iosched folder settings underneath that to match the ones posted by freemangordon here (my page_cluster is the default of 5 however and I don't think nr_requests needs to be so high). Finally I have some schedtool settings too but I doubt they would help to fix your problems.
Can you post benchmark result with dd please? Freemangordon's setting isn't working on my device the results are worst.

MONVMENTVM 2011-06-01 19:40
Re: BFS for the power kernel
 
OK I'm not 100% sure but I think I found the problem. I didn't run into these issues again until now but I'll have to test it tomorrow.

So after installing the BFS kernel I also decided to try schedtool and your swap settings so I also installed swappolube. (I tweaked the settings in /etc/init.d/rc$ before but I wanted to tinker around again and it seemed more comfortable that's why I went for it.)

Anyway in Swappolube laptop_mode was enabled, which I guess was the culprit. I don't think that it makes much sense to use with flash storage as it is basically related to the spin down of hard drives and makes writes bursty. That explains why downloading makes the device unresponsive for a couple of minutes while the cpu is actually idle and doing nothing.

Bad_Habit 2011-06-01 19:49
Re: BFS for the power kernel
 
Overall performance is great, but I confirm freezes (not more than 20 sec) on high mem load and hard I/O (rebuilding tracker db, browsing in microb, angrybirds, fapt-get update; browser and addressbook suffer, others perform rather well and I can use phone just fine)

I use your vm config and schedtool commands

Thanks a lot, this is awesome

MONVMENTVM 2011-06-01 20:07
Re: BFS for the power kernel
 
Well I'm a bit confused now. Just to check I enabled laptop_mode again, started FAM and did an update while I played some music through MediaBox and surfed on TMO with microb. I had no audio stuttering and no performance issues at all. It's still wonderful.

What the hell was going on before all day long?

freemangordon 2011-06-01 20:31
Re: BFS for the power kernel
 
Quote:
Originally Posted by epitaph (Post 1020373)
Can you post benchmark result with dd please? Freemangordon's setting isn't working on my device the results are worst.
Are you sure you have ONLY settings from the above mentioned post? Because I can assure you, with those settings (and original scheduler, i.e. kp46) my device feels and behaves exactly (if not better) as the one from n900 ad . But it is with default values in /proc/sys/vm/ (excluding swappiness)

@Tigerite what I saw while playing with io congestion problem is that lower page_cluster value is, better UI responsiveness is. And nr_requests requests have to be that high in order to have both fast io and responsive UI. Did you try uncompressing easydebian image (or other very big file) to check the difference between low (32) and relatively high (512) values.

epitaph 2011-06-01 20:45
Re: BFS for the power kernel
 
Quote:
Originally Posted by freemangordon (Post 1020432)
Are you sure you have ONLY settings from the above mentioned post? Because I can assure you, with those settings (and original scheduler, i.e. kp46) my device feels and behaves exactly (if not better) as the one from n900 ad . But it is with default values in /proc/sys/vm/ (excluding swappiness)
Did you make a benchmark with dd before and after? It's not my goal to copy your device or setting 100%, I'm just asking if you did a benchmark and if you can share it? I did one, and I can confirm that I modified your post to make the benchmark be faster with my device but I'm using other tweakings too.

Here a benchmark without my tune-up tools:

/dev/mmcblk0p1
900000+0 Datensätze ein
900000+0 Datensätze aus
460800000 Bytes (461 MB) kopiert, 25,3318 s, 18,2 MB/s

/dev/mmcblk1p1
900000+0 Datensätze ein
900000+0 Datensätze aus
460800000 Bytes (461 MB) kopiert, 28,351 s, 16,3 MB/s

Here with my tune-up tools:

/dev/mmcblk0p1
900000+0 Datensätze ein
900000+0 Datensätze aus
460800000 Bytes (461 MB) kopiert, 21,748 s, 21,2 MB/s

/dev/mmcblk1p1
900000+0 Datensätze ein
900000+0 Datensätze aus
460800000 Bytes (461 MB) kopiert, 27,0826 s, 17,0 MB/s

/dev/mmcblk1p1 is a 8 GB Class 6 SD-Card.

I use this script dd if=/dev/mmcblk0p1 of=/dev/zero bs=512 count=9000000

freemangordon 2011-06-01 20:58
Re: BFS for the power kernel
 
No, did not do it in that way, my goal was different. And having class2 sansisk my sdcard io speed will be far lower than yours anyway :D.

TBH I did not make any benchmarking, how am I supposed to benchmark something like "UI responsiveness"? I am not saying that with my settings you will achieve maximum possible io speed, but overall system speedup and responsiveness gain. Nor I am saying my settings are the best ;) .

Just out of curiosity I will ask you the same question I have asked :

How is your system behaving while uncompressing qole's easydebian archive or downloading torrent at high speed?

epitaph 2011-06-01 21:03
Re: BFS for the power kernel
 
Quote:
Originally Posted by freemangordon (Post 1020452)
No, did not do it in that way, my goal was different. And having class2 sansisk my sdcard io speed will be far lower than yours anyway :D.

TBH I did not make any benchmarking, how am I supposed to benchmark something like "UI responsiveness"? I am not saying that with my settings you will achieve maximum possible io speed, but overall system speedup and responsiveness gain. Nor I am saying my settings are the best ;) .

Just out of curiosity I will ask you the same question I have asked :

How is your system behaving while uncompressing qole's easydebian archive or downloading torrent at high speed?
Thank you for your fast reply, but I don't have interest in uncompressing easy-debian atm, and I think I don't use your tool to download torrent files anyway. Maybe I can tell you if you post a benchmark before and after with your setting and with your device.
It's not an excuse to me that you have only a Class 2 SD-Card. If you want you can calculate a relative in percent. Is that mathematical correct saying?

freemangordon 2011-06-01 21:18
Re: BFS for the power kernel
 
Quote:
Originally Posted by epitaph (Post 1020455)
Thank you for your fast reply, but I don't have interest in uncompressing easy-debian atm, and I think I don't use your tool to download torrent files anyway. Maybe I can tell you if you post a benchmark before and after with your setting and with your device.
It's not an excuse to me that you have only a Class 2 SD-Card. If you want you can calculate a relative in percent. Is that mathematical correct saying?
and which dd are you using?

epitaph 2011-06-01 21:22
Re: BFS for the power kernel
 
Quote:
Originally Posted by freemangordon (Post 1020462)
and which dd are you using?
d & d. Deaf & Dumb. I didn't know I have a choice? It's not the Busybox version nor the Busybox extended version.

freemangordon 2011-06-01 21:24
Re: BFS for the power kernel
 
Quote:
Originally Posted by epitaph (Post 1020466)
d & d. Develop and Duckish.
WTF? Are you high man? I am out.

epitaph 2011-06-01 21:43
Re: BFS for the power kernel
 
Quote:
Originally Posted by freemangordon (Post 1020467)
WTF? Are you high man? I am out.
No, and I want not do it again, but it's really stange your question as a developer you should verify your settings but I think it's this one here: /opt/maemo/usr/bin/gnu/dd.


All times are GMT. The time now is 13:41.
Page 5 of 8 « First 34 5 67 Last »
Show 40 post(s) from this thread on one page

vBulletin® Version 3.8.8