Moving /usr to /opt/maemo/usr -> major road blocks
 

What is (in your opinion) the "best" way to merge '/usr' with '/opt/maemo/usr' and de-duplicate etc. before? This is not really helpful, because there are only some thoughts noted. Has anyone ever really tried it?

I thought about copying everything from '/usr' to '/opt/maemo/usr' and mounting '/usr' as 'opt/maemo'usr' by editing /usr/lib/genfstab.awk (/etc/fstab/)


...but:
Because '/usr' already contains links to /opt/*, i f*cked up and now /opt/maemo/usr contains unresolved symbolic links of files that now point to themself... of course those files are gone. :mad:

I know that

-p (preserve)
-R or -r (recursive)

were necessary options, but maybe i should have used also this?

-L (dereference)

Currently running a 'find /usr -type l -exec ls -l {} \; > /home/user/symlinks.txt' to try to resolve this without reflashing, only reinstalling some packages if doable.



EDIT:

Why is the system generally in such a messy state without my doings?

Alone within /usr/lib, there are at least those directories targeted by files using symlinks:

/opt/maemo/usr/lib (f.e. libavcodec etc.)
/opt//lib (f.e the Qt stuff)
/opt/usr/lib (f.e. libcups)

Why? Isn't there an agreed standard? This makes it almost impossible to choose /usr/lib as a target for relocation, and i will have to rely on sym-linking single files or directories only from /usr/bin /usr/lib /usr/share as before, which is what i wanted to avoid (creating even more of a mess).

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Additional question:

If i would create a separate partition for /usr on the EMMC and do the necessary edits in genfstab.awk, would this work? If yes, should it be the last partition on the EMMC (/dev/mmc0blk4)?

(I plan to reflash this evening, so if any could answer before it would be helpful)

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Wait, doesn't /usr contain data that is needed to mount the emmc. If you move it to the emmc how is it gonna mount the emmc?

Ya follow?

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

I feared something like this. So how can we free up space without having to rely on ugly hacks (like symlinking too many directories/files)?

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Also, could You explain why You're planning to move /usr/ (containing crucial elements) from super-fast NAND to "slow" eMMC? Seems like best way to slow system down, by not a small factor.

/Estel

// Edit

You were faster, so additional question - why You want to free up rootfs? It is in Your best interest, to have it used as much as possible, due to mentioned NAND speed.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

I think the speed difference between NAND and EMMC is minimal.

I see two ways of freeing up root space.

1. Deleting unused locals, can free up ~20-25MB.
2. Symlinking stuff that you definitely will not be needing for bootup.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

I have bookmarked this post some time ago

http://talk.maemo.org/showpost.php?p...&postcount=254

I think it's relevant

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Because i already freed up several (in the multiple tens) megabytes by moving and symlinking single files and whole directories, repeatedly over the course of 12 months, but said space grew smaller and smaller starting shortly after regaining something and during today's update not even some bytes were left. History repeating. You can have not enough free space; packages will keep eating your rootfs space with impunity for no particular reason.

Because /usr should be able to use a separate partition by mounting it from somewhere else (compile custom kernel with mmc driver enabled?). It is supported in Linux per se, so why is Maemo restricted?

Because there were users that moved the whole rootfs out of NAND.

Because Nokia went cheap on the same level that they went cheap on the RAM as they went cheap everywhere regarding HW on the Symbian phones.

Because optifying seems like being a "only-nice-to-have' extra during development of so many packages.

Because i will see if speed is really that lame.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

I call shennanigans.

While he HAS made a copy of /usr on a different partition, it is not in use...

Rationale:

1. Fstab does NOTHING, it is just there for 'completeness' and is re-created EVERY boot by genfstab.awk and some other convoluted magic.

2. You CANNOT boot the n900 with /usr on emmc. Consider Backupmenu, it is stored in /usr/share/backupmenu and is run without mounting emmc and Mydocs.

How would backup menu run if it were located on emmc before emmc was mounted?

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

I did both; but this helps only so-so (and not endlessly).

Another question:

Why is the eMMC stuff not in /lib or anywhere else, but in /usr/lib (/usr/lib/genfstab.awk)? Seriously, was this a good design decision? IMHO far way from it.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Well, all you need to mount a partition is a running kernel (plus any necessary modules) and /bin/mount. As long as /bin stays in / you should be OK (tm).

So even if /usr is used very early in Maemo, if you mount /usr *first* thing when booting (i.e. first lines of /sbin/preinit) you should be OK (tm).

So much for the theory though... :)

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

PS.:
http://talk.maemo.org/showthread.php?t=36011
http://talk.maemo.org/showthread.php...78#post1001236

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

As far as I remember, there were comparison tests for NAND and eMMC speed, and NAND is *much* faster. Can't link to results now, though. Also, keep in mind, that more crucial things on eMMC, more I/O conflicts between it and regular read/writes. We have already avoided it by using swap on microSD, so introducing it back again, by putting things from NAND into eMMC doesn't seems like good design decision at all.

By the way, I wonder, why You have problems with space on rootfs. Currently, I got 55 MB free, *without* deleting any locales, moving any things to eMMC etc. I think You got something eating Your rootfs space (.cap and .ivs files from aircrack? Not optified packages?), cause during normal circumstances, it should not result in "out of space" in any point of time.

Of course it's up to You, and experiments may be worth trying just for sake of knowledge, but it seems for me that You're cutting of leg, instead of curing it.

/Estel

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Non-optified packages, as i have much stuff installed. I travel alot and it's kinda safe feeling to have "everything" with me, just in case and while being offline.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

So, isn't it easier to just optify such packages, instead of moving everything, including some system "internals"?

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

How? With the perl/python script? This still doesn't resolve that critical system files don't belong in /usr...

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

By script, or manually symlinking, one package at a time. I suppose you doesn't have 364646 non optified packages installed?

As for system files not belonging to /usr/, even if we agree about that, what You gain by tinkering with it?... It feels like just aesthetic.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

@Estel,

I for one would be happy if there was a strict separation between /{bin,sbin,lib} and /usr/{bin,sbin,lib}.

In theory /bin and /sbin and /lib should contain what is required for booting. The rest can and should go to /usr.

Whenever I installed linux in the past I always made a separate partition for /usr and I kept / read-only (also /usr, but that was more flexible).

I just "feels right" to be able to say: this partition is all I need to boot, so I want it separate from the rest.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Just a thought (sorry for resurrecrting this thread), most problems with symlinks (except the ones that _need_ rootfs as in booting routine) is that at some point ../.. leads nowhere (my understanding of actual problem is limited, very), Tanner had similar problems with libs/includes... Has anyone tried just optifying to the same level of depth? so ../.. would result in exactly 2 level shallower call? If when building something symlinked to /opt/dir1/ calls ../ it ends up in /opt, proposed solution (most likely ********) link files to /opt/. Same depth wouldn't help those situations? It guarantees a mess in /opt/ (you can put /lib files and /bin etc), but should keep those nasty one-level down calls in check? Any thoughts???

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Has anyone tried bind mounting /opt/maemo/usr into /usr (if not via /etc/fstab, then via the boot scripts)? That should be closer to "the real thing" compared to a symlink.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Problem with this is (no idea, just shooting possible problems, correct me pls if anyone knows) all those hildon-*, maemo-* and osso-* files might be needed on bootup and as opt is mounted later on remounting whole /usr(/bin) could result in reboot loop. Just guessing. Linking non-essential for start files from there should be easier...

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

How about mounting both /opt and /usr from /sbin/preinit? That's about as early in the boot process as one can go, and shouldn't require anything from /usr.

Also, another option could be to leave a skeleton set of files in /usr on the NAND to bridge the gap before the mount is done. However, this would make things more than a bit complicated.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Too newb here to see anytning wrong with this approach, but seems like you could mount MyDocs at the same time and move whole bulk to eMMC, so there is probably a reason why it hasn't been done before. Wish someone more knowledgable in that aspect could chime in

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Okay, everybody, settle down.

/usr contains lots of binaries that are ran on startup on any other Linux system, if you want to actually do some useful stuff.

The problem here is, you moved it to a different partition. WAT DO?

You need to mount /usr, even before anything else has been mounted. genfstab is in usr, so that's a no-go.

You'd have to load modules for /home filesystem, mount /home from preinit, mount /home/opt/maemo/usr to /usr, then just let pymaemo-optify mount /home/opt to /opt.

Geez, genfstab is ******ed. Why can't we replace that POS with whatever GNOME's having nowadays to detect external media?

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Sorry, too newb to discern whether your last sentence was rhetorical, but in case it wasn't, would it be doable? GNOME should be GNuish, if we could load it somehow early on boot, could this possibly result in free rootfs space? Or would the scenario right above be feasible? Amount of rootfs is one of the few but painful unsolved problems on N900

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

To be honest, I *never* even encountered any problem with rootfs space. At current point, with many things installed, I've 50 MB free, and still I "fail" to see any reason, why I should like to move most often use files/binaries/whatever out of very fast NAND to slow eMMC.

/Estel

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Well, most libraries behave very badly when symlinked out of rootfs. So when you apt-get libboost1.42-all-dev you will be left with few mb out of your current 50 (yeah, full libboost is a cow). Then you start following all optifying instructions out there, get back to around 40mb free and another cow-dev comes around, you end up <10mb, optify some more... neverending story really.

EDIT: about speed: copy 10mb file for a test to your ultrafast NAND, you will not end up with 40mb left, more like 43-45, depending on how compressed it is. I assume that it is compression on the go for rootfs (if someone has better explanation, lets hear it), which makes microB binary for example start much faster from slower eMMC as it then is uncompressed and starts almost instantly. From compressed NAND it uses more CPU and takes longer to start (at least my experience).

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Those who "never" had this problem probably joined in 2011 and I wonder if they ever even read the long threads about this problem that were produced just before the N900 was released to the common people.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

We just need DeviceKit. As long as the kernel exposes mmcblk1 as removable, this thing'll pick it up.

(NOTE: GNOME used to do this with HAL scripts. Maemo uses HAL, and I don't think pulling out HAL is a good idea. P.S. HAL is deprecated everywhere!)

Anyways, we'll need to simply remove whatever part of a script runs genfstab.awk, and uses variables set by that thing in every damn bootup script.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Would implementing only udisks from the DeviceKit work (leaving network, audio, input, display etc to HAL)?

EDIT: first block - udisks requires gudev (at least configure says so), svn pulled quite empty sourceforge tree with gudev.mdp/mds and traylib/test.cs, nothing in ./src per se :/ Maybe ubuntu armel gudev package will work?

Nope. Requires libgudev which requires libudev0 (this ubuntu deb installs fine) AND requires libglib2.0-0 >= 2.22.0 (vs 2.20.0 on device): (http://maemo.org/community/maemo-dev...lib_on_maemo5/ bad things can happen?)

Forcing ubuntu libs worked, buuut udisks requires even newer libglib according to configure >=2.31.13... this is not gonna work :( Oneiric got only 2.30.
Would forcing Precise libglib be a 'very bad idea' or just a 'bad idea'??

Passing variables to see how many other things are missing:
libglib, libgio 2.31.13, libpolkit-gobject-1 0.92, libpolkit-agent-1 0.92.... no way :(

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Getting the HAL/mounting discussion back in here:
Posted by szopin in libglib thread:

udisks requires libglib 2.31.13, available on Precise (.18) in armel flavour. Going to backup and try forcing it later on today. If that is just 'not going to work' let me know guys, reverting backups is not always successfull and would hate to have to resort to flashing/reinstalling everything

Hurrian:

Wait, hold on there.
You'll need udisks, a very new udev and take out HAL.
That last one will probably break Maemo.
/endquote

So there is no way to keep HAL for everything except storage mounting? Was hoping udisks could be used for mounting whole fs from /opt and keep HAL for the rest as it seems a gargantuan task

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

No, no, wrong, just no.
udisks mounts the sdcard for us. We define the eMMC (home, opt, MyDocs and hopefully /usr mounts) in fstab.
Maemo being Maemo, it's rootfs is mounted by the kernel at boot-time.

And udev works together with udisks to expose the devices to the system. No going around that. Part of the package.

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Ok, so how could DeviceKit be possibly used to save rootfs space?

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Come on, it's pure sophism. I'm perfectly aware of "early adopters" problems, but we're far away from times, when optification was arcane magic.

Of course, it's is obstacle for developing on device (yet, instructions on how to use whole 32GB eMMC as root may apply here), but to be honest, szopin's point about compression is much more interesting, IMO. Compression is a fact AFAIK, but isn't it possible to measure NAND actual speed (even including compression) just like we measure eMMC speed? I.E random access and raw read/write, separately? That would be much more appropriate, than "feelings" about something starting faster or not.

I'm quite convinced, that huge speed loss due to transparent compression in NAND is a myth, but it's also pure assumption - based on the fact, that compressing "bandwidth" based on CPU power is still much wider than writing capabilities of NAND (same apply for encrypted swap via TrueCrypt - despite encryption eating CPU, it doesn't make speed to suffer in any way, as it's still encrypting faster, than eMMC is able to write) - at worst, it may result in minimal speed lost on 100% CPU load (and if compression doesn't have highest/high priority).

/Estel

Re: Moving /usr to /opt/maemo/usr -> major road blocks
 

Estel: Yeah I know this is just as reliable as xxxpatch users' testimonies, but do me and yourself a favour: close all microB windows, click on microB and watch the time, takes around 0.5 second for the bookmark window to open up. Move microb-engine to opt, repeat the above, starts instantly. As startup loss of half a second is not even worth arguing about random read/writes etc. doesn't matter as it is not random but ~5MB worth of read/decompression. It does visually for me take more time to start and for you most likely also will. Lets not argue about non-issues.