maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Nokia N900 (https://talk.maemo.org/forumdisplay.php?f=44)
-   -   Root space - What is "optifying"? (https://talk.maemo.org/showthread.php?t=40131)

chicoelnino 2010-01-11 21:16

Root space - What is "optifying"?
 
Could someone break down what is meant by root space? And how do I check mine etc...

Much appreciated!

Rob1n 2010-01-11 21:34

Re: Root space
 
The N900 has two (or three if you have a memory card installed) separate flash memory areas. The first is a 256M high-speed flash which is attached directly to the processor. The second is a 32GB built-in flash.

The root (or rootfs) is so-called because it's at the root of the directory tree (the lowest level). This is on the 256M high-speed memory, because it contains the libraries and applications that are used most often. The /home and the /home/MyDocs directories are on the 32GB internal memory - /opt is actually just a link to /home/opt.

The reason the root memory is only 256M is because that was the largest available at the time the phone was designed.

You can see what free space you have where by running X Terminal and typing:
Code:

df -h

RevdKathy 2010-01-11 21:39

Re: Root space
 
Quote:

Originally Posted by Rob1n (Post 465181)
The N900 has two (or three if you have a memory card installed) separate flash memory areas. The first is a 256M high-speed flash which is attached directly to the processor. The second is a 32GB built-in flash.

The root (or rootfs) is so-called because it's at the root of the directory tree (the lowest level). This is on the 256M high-speed memory, because it contains the libraries and applications that are used most often. The /home and the /home/MyDocs directories are on the 32GB internal memory - /opt is actually just a link to /home/opt.

The reason the root memory is only 256M is because that was the largest available at the time the phone was designed.

You can see what free space you have where by running X Terminal and typing:
Code:

df -h

So which figure should I be looking at? The line marked rootfs says it has a total of 227.6M. The line with a total of 256M is tmpfs.

Which one should I be worried about?

Rob1n 2010-01-11 21:42

Re: Root space
 
There's some rounding errors I think (or possibly binary vs decimal) - the rootfs line is the one you want to be worried about anyway.

RevdKathy 2010-01-11 21:51

Re: Root space
 
Quote:

Originally Posted by Rob1n (Post 465196)
There's some rounding errors I think (or possibly binary vs decimal) - the rootfs line is the one you want to be worried about anyway.

Ahh. Then I should probably be worried. Which is slightly worrying as I have about 3 items from devel, all carefully checked.

Rob1n 2010-01-11 21:56

Re: Root space
 
How much free do you have? And have you rebooted after installing any apps?

jaysire 2010-01-11 22:12

Re: Root space
 
Quote:

Originally Posted by RevdKathy (Post 465191)
So which figure should I be looking at? The line marked rootfs says it has a total of 227.6M. The line with a total of 256M is tmpfs.
Which one should I be worried about?

Here's my df (diskfree) output. The parameter -h just means "human readable", which is Linux for "change bytes to megabytes and kilobytes and whatnot".

Code:

Filesystem                Size      Used Available Use% Mounted on
rootfs                  227.9M    189.1M    34.6M  85% /
ubi0:rootfs            227.9M    189.1M    34.6M  85% /
tmpfs                    1.0M    68.0k    956.0k  7% /tmp
tmpfs                  256.0k    80.0k    176.0k  31% /var/run
none                    10.0M    72.0k      9.9M  1% /dev
tmpfs                    64.0M      4.0k    64.0M  0% /dev/shm
/dev/mmcblk0p2            2.0G    188.2M      1.7G  10% /home
/dev/mmcblk0p1          27.0G      9.7G    17.3G  36% /home/user/MyDocs

Our things of "interest" in this output are the first, next to last and last line.

We all have 227.9 megabytes of root file system available, but what's interesting is the "available" -column that says I have 34.6 megabytes or 85% in use.

The last two lines indicate that /home/user/MyDocs has a total of 27 gigabytes (with 17.3 gigs free) and /home (our home directory) has a total of 2 gigabytes (of which 1.7 gigs are still free).

This is called partitioning: The internal memory/drive has been divided into several parts, the most interesting of which are 227.9 megabytes, 27 gigabytes and 2 gigabytes in size.

These partitions can then be "mapped" (or mounted) to certain directories. That is why Linux lacks drives like windows (ie. C: and D:). Instead, the physical disk or partition is mapped (connected, mounted) to a directory. So when Linux-guys insert a dvd, it doesn't become D: like on Windows. Instead it has to be manually or automatically mounted to a directory. That could be any directory, but usually ends up being something sensible like /mnt/dvd.

This is done to protect the system: We can fill up one partition and leave the other ones with plenty of space to function well. In the Maemo-case, the root mount is very small in size (228 megabytes) and should be left alone as much as possible. Instead, programs should be installed to the /opt -directory.

Interestingly, opt is part of the root filesystem! But if we look at /opt with ls, it says:

Code:

Nokia-N900-42-11:~# ls -la /opt
lrwxrwxrwx    1 root    root            9 Oct 30 13:36 /opt -> /home/opt

... which means /opt is actually a symbolic link to /home/opt. So /opt actually points to /home/opt, which has 1.7 gigs free, making it a good place to install apps.

Presumably the 18 megabyte download of today's new firmware release was significantly more when extracted. Upgrading stuff on Linux is often comparable to unzipping (extracting) packages on top of old versions (and maybe cleaning up the old versions before that). So the installer for today's firmware might have checked that it could fit completely on the rootfs. Let's say the unextracted firmware was 30 megabytes. The installer would then probably halt installation if less than 30 megabytes are available on the rootfs.

The installer is also probably not able to account for files that will be replaced (if we install a file that is 2 megabytes in size, we need zero megabytes of free space if this file replaces an older version that is also 2 megabytes. In fact, by installing the new 2 megabyte version of a file, we can INCREASE free space if we replace a more inefficiently compressed file that used to be 3 megabytes).

Sorry for writing so much, but you asked ;)

wmarone 2010-01-11 22:15

Re: Root space
 
Quote:

Originally Posted by jaysire (Post 465259)
We all have 227.9 megabytes of root file system available, but what's interesting is the "available" -column that says I have 34.6 megabytes or 85% available.

I think you mean 85% used (15% available.)

RevdKathy 2010-01-12 08:37

Re: Root space
 
I have about 80% used, with 45mb free. I guess I'm just one of those cautious people that likes to have oodles of free space 'just in case'. (My Girl guide training - always have food in the cupboard, a bit of money in the bank, a day or two of annual leave in hand and space on your drives. :p )

Since others seem to have the same or less, and aren't worried, I shall hereby stop worrying.

tk421 2010-01-12 08:54

Re: Root space
 
The Conky app brings up a list of system info. I had a look but it mut have been in one of the extra reps. Worth installing if you can.
http://i45.tinypic.com/id6r2p.jpg

zehjotkah 2010-01-12 09:09

Re: Root space
 
and how can we see, which programs take space from root?
because i've only 43,4mb free...

tpinhao 2010-01-12 09:18

Re: Root space
 
Quote:

Originally Posted by zehjotkah (Post 465869)
and how can we see, which programs take space from root?
because i've only 43,4mb free...

ONLY ?!?!?!? :p

It's more than enough, i have 30 Mb and it's great, i read somewhere that it is enogh even to have only 10Mb free, do not stress because of 43 Mb.

zehjotkah 2010-01-12 09:21

Re: Root space
 
after a reboot 44,1mb ;)

marks97 2010-01-12 11:40

Re: Root space
 
Quote:

Originally Posted by tpinhao (Post 465880)
ONLY ?!?!?!? :p

It's more than enough, i have 30 Mb and it's great, i read somewhere that it is enogh even to have only 10Mb free, do not stress because of 43 Mb.

We need 45mb free for the next update though.
http://konttoristhoughts.blogspot.co...ve-on-101.html

bald.eagle.one 2010-01-12 12:12

Re: Root space
 
i got 16.3M and not worrying
yet..

because this is stil testing - imo still beta firmware (upgraded to 1.0.1)
so when the big one comes it will land on clean device - will make sure of that - most likely through NSU but tempted to do it linux way ;) (livecd and so on)

slender 2010-01-12 12:24

Re: Root space
 
Quote:

Originally Posted by zehjotkah (Post 465869)
and how can we see, which programs take space from root?
because i've only 43,4mb free...

http://talk.maemo.org/showthread.php...401#post465401
http://talk.maemo.org/showpost.php?p...&postcount=592
http://talk.maemo.org/showpost.php?p...&postcount=620
http://talk.maemo.org/showpost.php?p...&postcount=621
http://talk.maemo.org/showpost.php?p...&postcount=637

There is no easy way to get more space after doing app managers uninstall/disable on extras & devel and reboots.

NvyUs 2010-01-12 12:47

Re: Root space
 
10mb will not be enpugh to except a big firmware update. ppl needed over 20mb free to get yesterdays update and that was quite small

felbutss 2010-01-12 14:23

Re: Root space
 
Quote:

Originally Posted by marks97 (Post 466081)
We need 45mb free for the next update though.
http://konttoristhoughts.blogspot.co...ve-on-101.html



what the hell. why cant the rootfs be larger. i dont mind if it was 500mb. how many people here have 45mb+ avaliable???????? what do we do now???????? i have 23.4mb available. how do we see whats taking all the room. and how do we clean it noob style???????????? once we find the apps taking all the space we will know which apps need optimization.

Rob1n 2010-01-12 14:50

Re: Root space
 
Unfortunately what's taking up all the space are libraries and built-in apps. The top rootfs space users on my N900 are:
Code:

posix-locales
microb-engine
libqt4-gui
python2.5
libc6
hildon-theme-alpha
nokia-maps-ui
adobe-flashplayer
hildon-theme-beta

The first -devel app is quite a way down the list - mc (and most of that looks to be locale files, which I'm not sure can be optified).

floffe 2010-01-12 14:51

Re: Root space
 
It can't be bigger because it's part of the OMAP3430 package, and 256M was the largest size available there.

fatalsaint 2010-01-12 15:21

Re: Root space
 
Quote:

Originally Posted by jaysire (Post 465259)
Here's my df (diskfree) output. The parameter -h just means "human readable", which is Linux for "change bytes to megabytes and kilobytes and whatnot".
--snip--

So using your df output and the assumption that they are *partitions* and not separate flash memory...

Could it not be possible to delete the /home/user/MyDocs partition of 27GB and resize the /home partition to fill that space.. thus giving /opt the full 30+GB of available space for applications? Ext2/3 support resizing - I imagine since we're using symlinks at least / is ext2/3, what about /home? A small FAT partition could be used for the Camera (as I read that it won't work if a certain folder is ext2/3).

Also, symlinking /usr, /lib, and /var in the same way would also cause the rootfs to be virtually unused correct? It would be mostly a glorified boot partition with configuration files (/etc)? (Or mounting over existing /usr, /lib, and /var with the bind option to achieve the same effect?)

And for anybody wondering why the numbers don't add up to exactly 256MB or 32GB, it's because every filesystem type has overhead of it's own. When you format a drive in NTFS, FAT/32, Ext2, Ext3, etc - it uses a part of the space to store it's own information to know what all is going on with that partition. This is why the rootfs shows "227.9M" for example instead of 256.

Rob1n 2010-01-12 15:24

Re: Root space
 
You could do all that, yes. Remember that the rootfs is far quicker to access than the eMMC though, so anything moved will have a performance penalty.

mankir 2010-01-12 15:30

Re: Root space
 
How much is the performance penalty, when using /usr on eMMC? Because it solved the problem completely for me and is still useable, since ruskie posted the instructions!

fatalsaint 2010-01-12 15:30

Re: Root space
 
Quote:

Originally Posted by Rob1n (Post 466482)
You could do all that, yes. Remember that the rootfs is far quicker to access than the eMMC though, so anything moved will have a performance penalty.

So I guess where I was confused was the term "flash memory"... what is the onboard 256MB "flash memory" if the internal 32GB is MMC "flash memory"?

I see.. the 256MB is connected directly. What do you mean by "hi-speed" though... same ideal as the Speed Class ratings for MMC?

SonicSpinner 2010-01-12 15:34

Re: Root space
 
i got 86 mb free. :). ready for the big fw.

Rob1n 2010-01-12 15:35

Re: Root space
 
It's all flash memory, just different types (and connected in different ways).

I've no idea what the actual performance penalty is - it's worth noting that the gtk-icon-cache removal in the latest firmware was done because it took up too much space on the rootfs but was too slow if moved to eMMC. It's probably not really an issue for individual files but is more likely to be noticed if lots of files need to be opened at the same time.

texaslabrat 2010-01-12 16:47

Re: Root space
 
Quote:

Originally Posted by fatalsaint (Post 466474)
So using your df output and the assumption that they are *partitions* and not separate flash memory...

Could it not be possible to delete the /home/user/MyDocs partition of 27GB and resize the /home partition to fill that space.. thus giving /opt the full 30+GB of available space for applications? Ext2/3 support resizing - I imagine since we're using symlinks at least / is ext2/3, what about /home? A small FAT partition could be used for the Camera (as I read that it won't work if a certain folder is ext2/3).

Yes, you could do that but then you would lose the ability to easily plug-n-pray with windows devices (though maybe you in particular don't care about that).

Quote:

Also, symlinking /usr, /lib, and /var in the same way would also cause the rootfs to be virtually unused correct? It would be mostly a glorified boot partition with configuration files (/etc)? (Or mounting over existing /usr, /lib, and /var with the bind option to achieve the same effect?)
Not really an option due to the way that the system boots up and the order of operations in mounting the various volumes. I tried to move some stuff via symlink similar to how you suggested and it made the device unbootable. There is a utility floating around that will symlink pseudo-optify things on a per-application basis (using the apt-cache to figure out the necessary files and such) which will only move those things that are GUI-based (and thus not dependencies for booting).

Quote:

And for anybody wondering why the numbers don't add up to exactly 256MB or 32GB, it's because every filesystem type has overhead of it's own. When you format a drive in NTFS, FAT/32, Ext2, Ext3, etc - it uses a part of the space to store it's own information to know what all is going on with that partition. This is why the rootfs shows "227.9M" for example instead of 256.
In addition to the formatting overhead, the difference between a "Megabyte" and a "Mebibyte" causes some of the discrepency ;) Storage is sold in the former, most computer systems/utilities (including df) use the latter.

RevdKathy 2010-01-12 19:08

Re: Root space
 
I have 45.3 free - will I squeeze the new update under the wire?

(and will I get any more if I temporarily diable extras? - no, testing and devel are never enabled unless I open them to raid something on someone else's instruction!)

texaslabrat 2010-01-12 20:19

Re: Root space
 
Quote:

Originally Posted by RevdKathy (Post 466901)
I have 45.3 free - will I squeeze the new update under the wire?

(and will I get any more if I temporarily diable extras? - no, testing and devel are never enabled unless I open them to raid something on someone else's instruction!)

Well, going strictly by the "heads up" bulletin thing you should be OK. I would imagine it will come in less than 45 MB since they have told people to have 45 free (you always build in a little padding).

I don't think you'd free up much just by disabling a repository...I can't imagine the little db files taking up that much space. However, as I said before I don't think you're in trouble space-wise so long as you don't make it worse than it currently is ;)

I wonder if there might be a simpler solution to all of this though. If the space is really just needed in order to download/unpack the files in preparation for installation...couldn't they just be downloaded into /opt by the update software? It's not like this is a performance-critical kind of thing we're dealing with. Just a random /ponder on my part.

jaysire 2010-01-12 20:55

Re: Root space
 
Quote:

Originally Posted by wmarone (Post 465268)
I think you mean 85% used (15% available.)

Darn it, thought I fixed all my careless mistakes already. :) Thanks for the heads up.

jaysire 2010-01-12 21:22

Re: Root space
 
Quote:

Originally Posted by fatalsaint (Post 466474)
So using your df output and the assumption that they are *partitions* and not separate flash memory...

I won't go into anymore details, since most of what I could say has already been said by others :)

But I'd like to address the matter of symlinking stuff in /usr from the (slower) ext3-partition that is mounted in /home. I would probably not just move everything, since most of the stuff doesn't take that much space.

There are some apps that take up a lot of space though, one of them being nokia-maps (maemo maps, I presume) in /usr/share/nokia-maps, which takes up about 10 megabytes. I could easily see myself symlinking from /home/share/nokia-maps, because if we only have 227.9 megabytes of "fast" memory, it stands to reason that MOST applications won't be on the fast memory partition and I haven't seen them taking any huge performance hit yet.

I guess Nokia elected to put their maemo maps on the rootfs to gain some (small?) performance increase, since it's quite sluggish as it is out of the box.

I found the command

du -d 2 /var /usr /lib | sort -n

to be quite effective when it comes to check which directories take up a lot of space. It sorts 2 levels of directories in /var /usr and /lib in ascending order according to disk space usage (in kilobytes).

/usr/share/nokia-maps and /usr/share/locale take about 10 megabytes each and /usr/lib/locale/ has a single file "locale-archive" that takes up about 16 megabytes and I don't even know what it does.

I'm not saying you should remove this stuff or even move it behind a symlink. I'm just saying it's good to know what takes up space on your device so you can do something about it if the need rises. If in doubt, research (google) it :) Like this hit for locale-archive:

http://linux.derkeiler.com/Mailing-L...5-08/1529.html

paai 2010-01-13 07:02

Re: Root space
 
I see people here talking about 'optifying', which I suppose to mean 'moving apps to /opt', thereby freeing space on the rootfs.

But how do I do that safely, and how do I know which apps can be optified?

Paai

ossipena 2010-01-13 07:33

Re: Root space
 
Quote:

Originally Posted by paai (Post 467764)
I see people here talking about 'optifying', which I suppose to mean 'moving apps to /opt', thereby freeing space on the rootfs.

But how do I do that safely, and how do I know which apps can be optified?

Paai

you can do it safely when compiling your application. stuff that already has been installed can't be optified (at least so easily)

ossipena 2010-01-13 07:44

Re: Root space
 
and keep in mind: one can always reflash the newest firmware.

sometimes it is a good solution to get a rid from unused software.

RevdKathy 2010-01-13 08:20

Re: Root space
 
Quote:

Originally Posted by paai (Post 467764)
I see people here talking about 'optifying', which I suppose to mean 'moving apps to /opt', thereby freeing space on the rootfs.

But how do I do that safely, and how do I know which apps can be optified?

Paai

'Optifying' is not something users do. It's something done to apps during the development process. If apps have not been through this, they will eat up your rootfs when you install them, and by then the only thing the user can do is uninstall.

The problem is that some of the basic libraries (including the ones even the ovi apps rely on) are not yet fully optified. so you [b[will[/b] use a fair bit of the space however careful you are.

But in terms of the apps themselves, all the ones in Ovi, and in maemo-extras are optified. They have to have been optified to get that far. Apps still in testing (less so) and devel are (highly) likely not to have been optified. No point in doing that for every build of something that's still in very early experimentation. It's a sort of 'tidying up before public release' process.

Which gives you a fair idea of what you might want to uninstall. ;)

If you still have 20% available (45mb) you should be ok for the Next big Thing.

jcompagner 2010-01-13 08:29

Re: Root space
 
hmm getting more space currenrlt is pretty hard.
i am at 33MiB

then i did

apt-get autoremove (that did remove quite a few packages mostly qt4)
apt-get upgrade (to look if the new compression did kick in, upgraded 2 small packages)
apt-get autoclean
apt-get clean
reboot

and after reboot: 33MiB
hmm

i guess when the new firmware arrives i need to seriously start uninstalling..

mankir 2010-01-13 09:03

Re: Root space
 
No RevdKathy, you are wrong again! There is a mentioned before script, to do the optifying after installation. It takes a bit of time, it moves the files from /usr to /opt on the mmc and symlink them them. It is the same procedure, only the symlinks are eating the rootfs like in optifyed packages as well...

RevdKathy 2010-01-13 09:05

Re: Root space
 
Quote:

Originally Posted by mankir (Post 467891)
No RevdKathy, you are wrong again! There is a mentioned before script, to do the optifying after installation. It takes a bit of time, it moves the files from /usr to /opt on the mmc and symlink them them. It is the same procedure...

Really? I wasn't aware this was something users can do. That's good to know. Please can you supply the link to the procedure so people can go about it? Presumably we can't do this to the basic libraries, though.

Will it not also slow things down if they're in a different place?

mankir 2010-01-13 09:10

Re: Root space
 
Sorry for writing you are wrong again! Ok you are right, the eMMC is slower than the NAND. But it will only make a difference, when you access many files at the same time. Normally you won't feel a difference, i guess!

Here is the script: http://talk.maemo.org/showthread.php?t=33429

mece 2010-01-13 09:12

Re: Root space
 
For those who have installed python apps from testing or devel. The python in extras is now optified, which gives you a nice 20 megs extra in rootfs.

I had to manually remove some stuff to get it updated.

Here's what I did:

This will most likely remove some programs. You will see a list of what will be removed and asked to verify. Please note that you might not have the same programs installed as I did. FM Radio, Quicknote and Witter are a couple of python apps that will be removed.


in xterm as root:

apt-get remove python
apt-get remove python2.5
apt-get remove python-osso
apt-get autoremove
apt-get clean

now you should have much more space.

Install whatever programs that were removed. The new optified python libraries should be installed in the process.

Check with df -h that you see the opt mounts for python added to the list.


All times are GMT. The time now is 01:08.

vBulletin® Version 3.8.8