Re: [Under consideration] More efficient and flexible use of internal flash
 

For camera you don't need to modify ke-recv.
You need it for normal export partition for computer when connect by usb. additionally without modification you will have problem to access files which was copied from computer.

in my script for change vfat to ext3 when you connect to comuter in mass storage mode partition will be unmounted on n900. After diconnect it will be mounted back.

p.s. For ext2 you will need little bit modify ke-recv. in my patch i add only ext3.

and look like kamera and ke-recv will be fixed in next update because bugs for them get nokia's internal bug tracking number...

Re: More efficient and flexible use of internal flash
 

This seems like the best solution to me. Perhaps now that the camera is working with non FAT partitions the loopback file is not necessary. Although mass storage mode might not behave without it.
Also, would ext2 be preferable to ext3?

What would be useful would be a guide or a script to set this up, I wouldn't really trust myself to do this properly. The closest thing I can find is this.

One other thing somewhat related. How trustworthy are programs with regards to putting large files in /opt? is it likely that one will ever run out of space on the 256mb NAND flash? Might mounting /usr on the internal card be necessary?

Thanks.

Re: More efficient and flexible use of internal flash
 

correct, the loopback file is only useful for mass storage mode (or until recently, to keep the camera happy).
regarding ext2: I guess the Nokia engineers have carefully evaluated both fs
and decided that the journaling in ext3 is not really an issue, but rather an advantage
as there is no fsck before mounting (see bugs #7572, 7708)

you can find HOWTOs in this thread and the other one, e.g http://talk.maemo.org/showpost.php?p...1&postcount=66
I agree we should definitely compile the information on the wiki.

I'm really busy with other stuff ATM and I'd appreciate if someone
could collect the links to all relevant posts and summarize the possible solutions
(e.g. default, single /home with optional loopback, swapped vfat+ext3, MMC partitioning,
what should be exported per USB, $HOME layout and integration of MyDocs).
Then we could modify the ke-recv and upstart packages to support all our solutions
and hopefully propagate them to Nokia, so that it works out of the box.

It is quite likely that you will run out of space on NAND if you install additional applications. The optification is still work in progress and I think there
are several design flaws which need to be fixed first (e.g. /var/lib/dpkg or apt should
be moved, /tmp is too small).
Moving /usr completely to eMMC is AFAIK not an option as it may contain some essential components which are required before mounting /home (but I may be wrong).

Re: [Under consideration] More efficient and flexible use of internal flash
 

I'm interested to know what effect on performance it has, to repartition and format the eMMC.

My understanding is that the eMMC works like a memory card. A while ago I tried formatting one to ext3, it worked at approximately half the original speed. Then when I formatted it on a PC back to FAT32, it was still about half the original speed. It was only when I used the manufacturers flash formatting software did it go back to full speed.

The clusters of sectors in the filesystem should be aligned with the physical blocks of the flash memory. Otherwise it results in multiple accesses, slowing everything down.

The flash formatting software put the start of partition at an offset from the start of the card memory, and had a different size for the FAT data when compared with the Windows partitioning & formatting.

It is necessary to get both the address of the start of the partition, and also the address of the first cluster of data (which follows the metadata) correct, to get the full performance.

This is a link I found to an explanation of cluster alignment:

http://www.hjreggel.net/cardspeed/cs_calign.html

Re: [Under consideration] More efficient and flexible use of internal flash
 

thanks. that's a useful observation and the same problem also affects new harddisks
with blocksizes > 512bytes (AFAIK the flash erase block size is typicall 64K).
I think the the eMMC has a different wear leveling strategy than consumer flash media
but I may be wrong.
I've just uploaded a bonnie++ port to extras-devel so that you can
start performing benchmarks with differents fs and devices (NAND, eMMC, MMC).

Re: [Under consideration] More efficient and flexible use of internal flash
 

some results of my bonnie++ benchmarks
(after disabling tracker "tracker-processes -k"
and using "/usr/sbin/bonnie\+\+ -n0 -f -d." on different media)
I had to disable the file creation benchmarks as they were extremely slow on fat.

fatpart=eMMC partition, fatMMC=on class 6 SD card, xLOOPy=x filesystem in loop file mounted on y on eMMC partition
ext2LOOPext3=easydeb image in home

raw results:
please replicate the tests on your device. there is noticeably variance.

Re: [Under consideration] More efficient and flexible use of internal flash
 

I tried this test with a new N900 with the default partitions as shipped from the factory.

The first time I ran the tests I got poor results, so I rebooted my N900, logged in with putty via ssh, and run the tests while the display was off.

I ran /usr/sbin/bonnie\+\+ -u root -n0 -f -d
with the following directories:

ext3-emmc /home/tmp/
nand /tmp2/
fatpart-emmc /home/user/MyDocs/tmp/
fatmmc-sd /media/mmc1/tmp/
ext2LOOPfat /.debian/tmp2/

The results are here:

raw data:

The microSD card I was using is an old 2GB I had lying around, which is very slow as you can see. What was the model & make of your microSD card?

It looks like you've lost some performance on the fat partition of the eMMC.

Re: [Under consideration] More efficient and flexible use of internal flash
 

it is probably a good idea to setup a standardized procedure for the benchmarks. How about:

a fresh, reflashed device with 51 firmware, with only ssh and gainroot installed,
no applications started, only Wifi, tests run after reboot as user via ssh.
run "tracker-processes -k" and the for each fs use
in addition to the results also report SD card specs, partition scheme and disk usage
for each tested partition.

mine is a Transcend MicroSDHC 8GB Class 6 SD card.

it would be interesting to see a comparison of ext2 vs. ext3 partition
to check whether journaling has a large effect.

Re: [Under consideration] More efficient and flexible use of internal flash
 

Titan, I think what you suggest will give the most accurate results. When I run the test I also thought about reflashing it, but I didn't want to lose my installation. So I ran 'top' to see if there were any busy processes first.

I did run tracker-processes -k

I've remembered now from my previous tests on a flash card a while ago that formating FAT32 to a cluster size of 32KB significantly improved performance. I think most flash cards and USB sticks are formatted to this (though I'm not sure how to see the parameters on the N900). ext 2 & 3 only support a maximum of 4KB per cluster (8 sectors).

I think ext3 journalling has an effect on writes only.

When you did your write testing, what parameters did you use?

Re: [Under consideration] More efficient and flexible use of internal flash
 

I also did use a "fresh" installation, so my results may be biased as well (but all processes were sleeping).
IIRC you need to set -S15 for mkfs.vfat
I used standard vfat options with -F32 and ca. 520MB free on a 2GB FAT partition.
journalling may also affect parallel read+write operations.

BTW, I realized that the NAND performance may be the best-case scenario
as it is a compressed FS and bonnie only writes zeros.
We should also benchmark copying from /dev/random (worst case) and from large typical files (e.g. libqt) from RAMdisk.