![]() |
How to upgrade the internal memory by extending the root filesystem to a memory card
I found out how you can upgrade the Nokia 770 Internet Tablet's internal memory by extending the root filesystem to a memory card. My step-by-step guide with detailed instructions has been posted on the Maemo Wiki. Following the steps, one can expand the unit's memory, make it persist even through flashing, and protect the internal flash memory's durability all at the same time. All power users are welcome to check it out and see if it helps them to get even more out of our favorite Linux device. Here's the link: http://maemo.org/maemowiki/ExtendedRootFilesystem
|
Wow! Nice Wiki page :)
Just a couple of (probably stupid) questions. What happens when you plug the USB cable into the 770 after you've done this hack? Does it affect how the device appears to the host PC? And I assume just like when running swap on a MMC plugging and unplugging the MMC while the device is on would be bad? That might be worth mentioning in the Wiki if so. Can you combine this with the swap space on MMC hack also? |
Great guide, now i'll just have to get myself a 1gb card and give it a try. :)
Simon, I think the first paragraph in the "First Step - Preparing the memory card:" section covers the usb connection issue. Not sure which method you used for creating the swap, but i used the one that has you create another partition and i can connect the usb without any problem. The device appears the same on my pc, just shows as being something like 48mb instead of 64. |
Oh yes, it does mention the first partition works the same as normal. I missed that bit sorry.
I haven't actually played with swap partitions yet. I was going to try that on the 64MB card first. I haven't had a chance to play with it on my Linux box to figure out how to partition it yet. |
What does this do to the device's internal storage? Does it just sit unused?
|
Hey there - thanks for the nice comments! :)
As has been pointed out, the normal functioning of the memory card won't be affected as we're using an independent partition, so there's no problem with unmounting. It shouldn't be removed from the device, of course, unless you revert the changes first. With the script I provided, this can be done painlessly. And yes, it can be combined with the swap. I recommend a swap file instead of a swap partion since we already created a new partition. While you could create a third one, it's probably better to simply create a swap file on the new partition, with a big memory card you'll have more than enough space. Then you can always change its size by recreating the swap file, that's easier than repartitioning and resizing a partition, so you get more flexibility. I'll add that information to the Wiki page soon... The internal memory is still used for everything not swapped out onto the memory card. While you could move pretty much everything onto the card, I only use it for home directories and configuration files. If everything were to be put on the memory card, you'd lose the ability to instantly restore the device to your customized working state after reflashing, since then you'd have to update the files on the memory card as well. It would be fun to investigate a three-partition-setup: The first partition as the normal memory card, the second partition for home directories and configuration files, and a third one for all of the remaining stuff. So you see, there's still a lot of room for further improvement! |
I feel foolish asking this question but i'd love to get a step by step guide to this - lol yes even more detailed than wolfram - i get confused at the first step.
|
Well, actually the first step is the most difficult one, steps two and three are handled by the script. But that's what the forums are there for, to ask for help and discuss solutions. If you can describe your situation in more detail, I or someone else can try to help you better.
|
How to create and use a swap file together with an Extended Root Filesystem
I just updated the Extended Root Filesystem HOWTO with information on how to create and use a swap file together with an Extended Root Filesystem.
Since it's a file on the new partition, it won't be unmounted when the device is connected to a computer by USB. And as a regular file, it can be resized easily. It's a compromise between creating a swapfile and using a new partition for swap - which combines the advantages of both (and much more). What's best about my way of (de)activating swap is that this works independently of the actual swap file or partition. The startup, reboot and shutdown scripts will work the same no matter if there's swap available or not. To add or remove a swap file or partition, simply modify /etc/fstab (after manually deactivating swap with swapoff -a), without having to touch the scripts again. Even if you decide against swap, it's enough to delete or comment out the line(s) in /etc/fstab, the scripts will keep working. I'd actually like Nokia to include something like that in their own system scripts with a future version of the 770's OS. I hope you find it useful and welcome further discussion here. |
How big?
OK -- I bought a 1 GB mmc card. I have root access. I'm about to boot up from a LiveCD and repartition and reformat the brandnew card.
What size should I make the second partition? I will be putting a swap file on there. I'd like to be able to put a lot of unnecessary applications on my 770 :D -- well, you know what I mean: lots of applications that I want to check out but won't use regularly, and apps I want as insurance (so that I can work on some file if I ever have to, for my job and such -- something like the gnumeric spreadsheet app). What size do you think should be big enough, without being too big and squandering space? Your guidance here would be most appreciated. Thanks, Roger |
Use free and top for guesses.
Quote:
Ok so this site says that the 770 has 64MB of RAM inuxdevices.com/articles/AT5858395674.html . Based on the Oracle guess then you'd want 128MB swap space. That would still leave you with 64mg of flash ram and 812MB on your MMC card. Now when you say you are going to install lot's of applications that is not the same as using memory or swap. The installation is what you'd put in your 812MB of space. To give you a perspective the largest app that I have installed so far is 2,408,576 while rsync is the smallest at 241,548. If you run xterm, then it will consume 2.4MB of memory. I think you'd have more than enough for both areas, if you use the 128/812 configuration. But remember Oracle's recommendation is for heavy hitting databases. 128 feels to big here. You are really on a single user system and not on a multi-user system. I've noticed that I cannot seem to run two copies of the same application. If I run marbles, then hit the launcher for the second copy, the 770 switches me back to the first running copy in memory. If I go to the xterm command and run free I see Mem total 61828; Mem used 58428; Mem free 3400; and Mem shared 0. It is interesting that this version of the free command does not have the cached count. Compare these number with two other linux systems [drkludge@bagheera ~]$ free total used free shared buffers cached Mem: 514744 498552 16192 0 18036 199612 -/+ buffers/cache: 280904 233840 Swap: 2064376 0 2064376 This machine has 512MB used as a workstation. It has a 2gig swap partition and note that under the used column zero is used. [drkludge@baloonew ~]$ free total used free shared buffers cached Mem: 2054724 2027660 27064 0 57600 1753672 -/+ buffers/cache: 216388 1838336 Swap: 2064376 696 2063680 [drkludge@baloonew ~]$ uptime 00:58:36 up 77 days, 10:50, 1 user, load average: 0.19, 0.06, 0.01 (I hope the formatting stays alined or it will be confusing to try and read these numbers.) Here's a file server that has been up for 77 days. It has 2gig of memory and a 2gig swap partition. Only 696MB of the swap partition is being used. That's a terrible waste of a 2300+MHZ athlon 64. A 500MHZ AMDK62 junker would serve NFS and SAMBA just as good. The only hope for this configuration is that when a request for user data is made, then it will come out of the used, shared, buffers, or cached memory pools. The buffers/cache line tells you how fast the linux kernel can rearrange memory from the buffers and cached pools. You really hope that you don't take from Swap because it is on disk in this case. Note that this machine has a swap partition the same size as the physical RAM and not much of it is used. :p Now the small memory foot print of the 770 is where a swap partition would really shine. I have 3400 free right now. If I fire up vim or gaim, then the O/S will really have to figure out where to put stuff. I think I saw where thoughtfix had success with just 16MB for a swap partition. An interesting project would be to try various sizes and record the results: 16MB, 32MB, 64MB, 96MB, and 128MB. The down side is that you'd have to reformat and completely reinstall your apps/data. You can allocate space for 128MB of swap in a partition but build it for only 16MB to start. The rest would go to your user data. If you find at some point, say, 32MB is the magic number, then you want to reformat the MMC card for 32MB swap/the rest user data. For some reason I was amazed to find the free command on the 770. I found it more amazing to see the top command. The top command allows you to see what is going on in your 770 almost realtime. Hit the fullscreen button on the 770 before you run top in the xterm window. The memory numbers at the very top of the screen will be cut off if you don't. It's impossible to scroll up fast enough to see 'em. Top shows Mem used 54860K; Mem free 6968K; Mem shared 0K; Mem buffers 160K; and Mem cached 20840K. If I alt tab with the keyboard to see marbles, then I find that the 770 has paused the game. I can only see small changes in the top numbers when I create another xterm _tab_ with Properties button>file>newtab while in xterm full screen mode. Finally, just as I have noticed that GUI apps are inactive when you switch to another application, I wonder what the impact to memory and swap is when the machine powers down the display? Hmmm...16MB, 32MB, 48MB, or 64MB may be good starter numbers. With 1Gig of MMC, I'd save most of that for music, videos, pdf books to read, applications, or application data files, etc. As you can see much of this is a guess. You'd have to take time to perform long term trend analysis. :eek: So go with 64MB and don't worry about it! As with all caching or swaping schemes you need memory to burn and you have it. :D The moment you shutdown the device, the caching and swaping have to build up again on the next boot. I hope this helps. Perhaps I need to clean this up and put it on the wiki too as "page two" of the swap file article--bad pun intended. :p Greg |
I'm still unsure as to whether it's really necessary to partition the card at all. Mount an ext2 partition using "mount -o loop" - requires the scratchbox mount command rather than the built in one. Use mkfs on a linux box to create the partition files (dd if=/dev/zero ... to make the empty files) then gzip and copy to mmc, unzip and mount.
It may be slightly less efficient, but makes changing the size of the (pseudo) partition much easier, or just add extra partition files as needed. |
RogerS: I'm using a 1 GB MMC memory card, too. I repartitioned it so there's a 64 MB VFAT partition (exactly the same size as the original MMC that ships with the Nokia 770) and allocated the rest of the card to an EXT3 partition (which also holds the swap file). With such a Linux partition, you'd hardly need the FAT partition, I only use it to hold the files needed to restore the Extended Root Filesystem after a reflash (xterm, my Root.sh script, etc.).
dr_kludge: Quote:
msaunby: That's an interesting idea worth a try. It's like using a swap file instead of a swap partion, you're using an ext3 loopback mount instead of a partition. The scratchbox mount command could be stored on the MMC, too, so it's always available after a reflash. |
Quote:
Quote:
Since I've seen it repeated so much, I guess I will start with a 24MB swap file until larger sizes are definitively shown to be more effective. I guess the only problem I can see with all but 64 mb of the mmc card as the second partition -- as Wolf does -- is that it won't make moving a large movie from my PC to the 770 easy -- from earlier reports it seems a full-length movie can be larger than 700 mb. I won't be able to use USB transfer to the second partition. Dr K, Wolf -- what would be the best/easiest way to transfer huge files to the 770 then? Instead of 64/960 (W) or 960/64 (K), would this make something like 824/200 more optimal? Or 700/324 for more app room? Or 900/124 for longer movies? I guess I don't want to be squeezed in the space I'm allocating for apps, nor do I want to waste any space. Not that I'm the only one who wants easy optimization.:) |
Large file/media transfer ideas for the Nokia 770
Quote:
Other options: 1.) If you have a wireless lan then use a PC or other device that can serve files via a web site. This method may be slower than using the read/writer. Plus additional complexity is required to set up a web server. I am presuming that you can use right click on the filename on the server and save it. I haven't played with the browser on the 770 yet. 2.) With a USB hub and other junk attached to the 770, then you could take a 1 GIG USB thumb drive and mount it on the 770 with your video. That will take some work to setup. I do have one mounted right now. I'll document it at some point. 3.) Set up ethernet networking on the 770. There again the complexity goes way up. First you'd have to download ssh/scp to your 770. Then you'd have to setup another PC with ssh or sshd. Finally, you'd have to configure the 770 to use ethernet networking. 4.) I am sure there's even more ways that will become available. For example, if someone ports an NFS or Samba client to the 770, then you would be able to mount these shares to the 770 and copy your desired files to the MMC on the device. At some point an ftp client could be created to access an ftp server with the files. Since python is now available on the 770, a bittorent client could bring files to the 700. All of these would require ethernet networking or a wireless LAN to be available to you. 5.) Unless someone beats me and compiles vblade for the 770, this is a very cool idea. Someone has already done this on the nslu2 device http://www.nslu2-linux.org/ . http://www.linuxjournal.com/article/8149 http://www.coraid.com/support/linux/ http://en.wikipedia.org/wiki/ATA_over_Ethernet http://sourceforge.net/projects/aoetools/ Greg |
Yes, I've put everything that's modifyable by the user onto the Extended Root Filesystem (i. e. memory card's newly created and sym-linked EXT3 partition). Movies and such I'd also store in the home directory which is on that partition. For file transfer, I installed SSH on the Nokia 770, so I can upload/download files with scp/sftp (Secure FTP - should work with Windows FTP clients, too).
KDE users should give the "fish" protocol a try - then you can browse the Nokia 770's filesystem just like your local harddisks. If you don't want to be restricted to fixed partition sizes, the method of creating files which are mounted as loopback would be a useful alternative, then you could recreate a bigger one more easily - but of course, with only a single partition, it would get unmounted when the USB connection is established. Every method has its advantages and disadvantages... |
Quote:
I do have a Compact Flash card reader and have used the approach Dr. K mentions with photos in my camera -- but it doesn't read mmc :( Maybe I'll be getting a new reader . . . My only question about that is, Will the PC see both partitions? Thanks, both, for your help. BTW, I'm posting this from Firefox running on the Mepis LiveCD. Just about to tackle the repartitioning and reformatting of the new 1GB mmc card. And, yes, I do always get into the water one toe at a time. How did you know? |
New Reader is order
Quote:
Yep. A new reader may be in order. I have a 9 IN 1 job. It looks like it will take the RS-MMC but did not work. There are some RS-MMC dongles that adapt the memory to the USB bus. There again I found out I have an older model that did not work. If you don't want to fool with ssh/scp just yet, then the reader will be the easiest method. Most modern linux distros will see the FAT partition automatically via dbus. But you may have to manually mount the ext3 partition!? I also played with a bluetooth dongle but did not have enough time to make PLAN work--it may not work. As I have more time I'll be writing up how-tos based on my initial experience. I just received my device last week. Played with the flasher; Mounted stuff; Have usb hub and four things hooked to it. I found one device crashed the 770 when I hook it to the hub. Still wondering about that one. Greg |
After following the instructions for extending the root filesystem, I started to enter the commands for creating a swap file.
When I entered the first command, dd .... 24576 the message I got was dd: /Root/swapfile: No such file or directory I had downloaded the Root.sh file to my PC and then copied it to the 64mb mmc partition and ran it, as you described. The only odd thing, of course, is that I must have done Step 2 twice -- once entering the commands manually and once in the Root.sh file. Any idea what's going on and how I can proceed? Thanks, Roger PS: Apologies for cross-posting -- I first entered this info at your website, but also would like to share the information here at itT forums. |
Quote:
I booted my PowerBook off a PowePC Ubuntu live CD and partitioned my 1GB rsmmc card as 800MB FAT16 and 200MB linux ext3. I ran your script, and it transferred everything perfectly, giving me much more space. I then set up the swap as you recommended. This made a huge difference in multitasking! Thanks to you, I am now chatting with a friend via gaim, have two large web pages loaded, a PDF document open, and xterm as well. AND, I can still mount my fat partition on my mac via usb to transfer some mp3s over without interrupting anything. Thanks! |
Quote:
#touch /Root/swapfile |
Hi - I have a question or two about partioning (sorry Im new with this Linux stuff ;) I've burned a cd with Ubuntu live and want to partion my 1GB mmc.
1.)When I reboot into linux how can i access parted? 2.) When I execute /dev/sda does this just partion the 770 drive (i havent run parted yet so maybe this is self-explanitory - i just dont want to format my c drive ;) 3.)How do i specify the size of the partions? I'd like to have 800mb fat for movies and 200mb for linux. 4.)Does the sudo mkfs.ext3 /dev/sda2 command format just the linux partion? Is this all i need to enter? THANKS for your help - Im just a little confused by the linux stuff. |
Quote:
--- Edited later to add: Going through this once has reassured me that the patch to the firmware image with root access works fine. Explaining how the mmc card is partitioned really solidified in my mind what I was doing. Seeing that the complex steps are reduced to (A) a shell script that's easy to run and (B) a handful of commands to enter for a swapfile, I don't find this all that daunting, the way I did last week. So, to eliminate the chance of problems from what I described here, tonight I'm going to start from scratch: reflash the 770, repartition the mmc card, run that script and make the swapfile. Having done it once, I can say with first-hand knowledge, "it's not that hard." |
Swap wearing out the card?
I'm very grateful for Wolfram's description of how to use the MMC card effectively. I was planning to work out a similar description myself, and now I don't have to.
I don't plan to use the MMC for swap, though. As far as I can tell, flash memory is only specified to be good for up to 100,000 erase-write cycles, and it seems to me that I could run through that in no time by using flash memory for swap. Isn't anyone else worried about this? Regards, Neil |
Cantona,
I am very much in the position you describe, except for using a different LiveCD, one with Mepis instead of Ubuntu. In my case, I worried very closely about whether the mmc card was automatically mounted (it wasn't). But seeing how the mount/unmount worked with the hard disks looked easy enough. Partition software In the K menu in Mepis -- the equivalent of Windows' Start menu -- I looked for utilities and found QTparted in a folder of system utilities. I'm guessing that it is parted built with QT widgets. Resize partition 1 QTparted is graphical, and highlighting the first item (partition) in the list in the righthand pane made a menu option available that I was able to use to specify a new size for it -- instead of 995 mb, 64mb (as Wolf suggests) or 128mb (as Dr K suggests -- or does he say 995-128?) or 800mb as you want. I think this is identified as FAT16. You're not doing anything here except specifying the size you want it, and that the rest of the mmc card is unused. Was this command called Resize partition? Something like that. Anyway now you have a smaller-than-1GB partition and the rest of the disk is identified as free space. Create partition 2 You select the line about the free space and choose the Create operation, where you set the size of this second partition (all available space in my case and yours) and specify ext3 as the partition type. The graphic now shows two partitions, but you have to choose Commit for the changes to be made. This means you can go back and adjust things if you didn't get them right in the two dialogs. Shutdown Then after all this, the 770 has to be rebooted. As Wolfram notes in his instructions, you should use the shutdown command. In my case, the shutdown command didn't work -- I wasn't root. In Xterm, I typed "sudo gainroot' and hit enter to gain root access, and then the shutdown command as Wolfram indicated. Make sure you have Xterm In order to become root, I had just reflashed my 770 with a new firmware image, the one with root enabled. So before I even started, I made sure I installed Xterm. I don't know if parted is a command line utility, but I would bet that the Ubuntu LiveCD has this kind of software with a graphical interface, even if it's not QTparted. Maybe some other, more experienced Linux users will correct my misstatements and identify the pertinent differences that show up in Ubuntu. Step 2 -- stop I then made a mistake I should warn you against. Wolfram describes the second step in great detail and lists five very long commands to enter in XTerm. Don't enter these. The shell script Farther down, after talking about step three in general terms (for instance, he suggests directories to move but doesn't explain how to do that for Linux innocents like me), he provides a shell script that does step 2 and 3 for you. In my case, I chose to follow his suggestions as to the directories to swap out, but you can edit those first before running the script. I also had difficulty getting this shell script onto the 770 mmc card so I could run it. I couldn't get Opera on the 770 to download it,. So I downloaded it using Firefox on the LiveCD and mailed it to myself at Gmail. Then I couldn't download it as an attachment. Ultimately I re-attached the 770 to the PC and transferred the file to the mmc card in the usual fashion, and then I was able to run it in Xterm. Roger PS: I found a good description of how QTparted works in PC magazine: pcmag.com/article2/0,1895,1902337,00.asp This was after I used it, of course. PPS: I looked at this post before clicking submit and realized it was so long it was hard to follow. So I inserted the bold phrases just to make it easier to read. I don't really think this way: heading, text, heading, text. |
WOW! Thank you Roger - that is a huge help!
|
Hello all!
Sorry that I can't help you more right now, I'm very busy at work, so I don't have as much time to post as I'd like. But I see Roger is doing a great job, both learning and teaching, that's nice. I've updated the HOWTO a bit to include more information and a link back here so others can find and join the discussion. I also noticed that "How to Partition" is a much bigger problem than anticipated, unfortunately it can't be automated like the others steps since it's done outside of the Nokia 770, depending on your actual computer setup which widely varies between different users. Anyway, that's why a forum is such a good place to talk about it and help each other. Last not least I really appreciate all the comments, both about what works for you and where you see a need for improvement. I'll try to update the Wiki page as often as necessary. Of course, being a Wiki page, it can be updated with your latest discoveries as well. Let's all work together to make the Nokia 770 Internet Tablet even better and greater. Happy hacking, |
I have not been able to get root extended to my 512 memory card. I thought that creating the ext3 partition would be trivial, but was a bit of a nightmare. Tried parted with fc3 (installed), Ubuntu 5.X live, QTparted in fc3. Problem with parted was, that no ext3 was supported, and I tried with an extended partition, no file system in it and formatting that to an ext3, input/output error. Finally I created a 300 MB fat16 partition, left Linux and created a 200 MB ext3 partition, using Partition magic, in windows.
Now, as I at last have an ext3 partiton, I can't get the root.sh work for me. may be the ext3/windows is not suitable or maybe the memory card is faulty. I'm going to try with the 64 MB card that came with 770. Wishes of some of us have been answered: http://gparted.sourceforge.net/download.php a really easy to use GPL Live CD-partitioning software (latest release 29.1) edited: used gparted/sourceforge yesterday, partitioning was done in a minute. Ran Root.sh, did not work, reflashed, restored my settings, installed Xterm, ran Root.sh again. Now it worked. Created a 24 MB swap, opened a digitv -> mediaconverter->280x160 007-avi.file from mounted slug hdd through wlan. Film went on smoothly as watched on TV. Thank's a lot for the instructions and great effort Wolfram! |
At least thanks to all your help Im making progress - lol now have a root enabled version on my 770. Current issue is with partitioning my 1GB card - qtparted wont let me reduce the exisiting partion below 7000mb ( i think i dont have it in front of me - but in any case it is nowhere near 64 or 128) Any ideas?
|
Quote:
|
It worked! THANK YOU ALL! I could not have done it without your help!
|
Since partitioning the memory card is the hardest step in extending the root filesystem, I'm trying to automate this process as well. Here's my first attempt. You'll require the "sfdisk" program which is part of the "util-linux" Ubuntu package (installed by default!) but probably available for other GNU/Linux distributions as well.
Connect the Nokia 770 to your Linux computer. Make sure the memory card's partition is auto-deteced but not mounted. If it's auto-mounted, unmount it. I'll use /dev/sda to refer to the memory card, but if it's a different device on your system, substitute it accordingly. Code:
sfdisk /dev/sda -uM << EOF Please give this a try and report back how it worked for you - then I'll add it to the HOWTO. |
Step 1: advices request
Hi!
Thank you very much Wolfram for this step-by-step howto :-) Yesterday I've received and configured my 1GB MMC-Mobile card: I think I've done it well as the system works but I'm not exactly sure whether what I've done at step 1 was right or not. See below. I've used fdisk (on FC4) to partition the RSMMC card. Out of the box, it was shown as FAT16 by fdisk (type 6). In your tutorial, you mentioned VFAT: what does this exactly mean as I was not able to find it in supported formats by fdisk? Someone told me that formats specified by fdisk were just here for information, but the real partition type depends on what you use to format each partition: right? Well, I've removed the existing partition and created 2 brand new partitions using the original format (ie. FAT16). After this, the 1st partition (768MB, to be mounted on host when USB connecting the N770) has been formated using mkfs.vfat, and the 2nd partition (256MB, to be used for swap and extension of root filesystem) using mkfs.ext3: was all of this right? When I display the partition table using fdisk, the 2 partitions are still displayed as FAT16: should I simply ignore this? TIA for your help! |
Quote:
Quote:
|
extrootfs_1.0_arm.deb
To make it even easier, I've created a Maemo installer package: extrootfs_1.0_arm.deb
It bundles the script which creates the Extended Root Filesystem with utilities for partitioning and formatting the memory card (cfdisk, mke2fs, mkfs.ext2, mkfs.ext3, sfdisk). It's never been easier to extend the Nokia 770's root filesystem onto the memory card. Now you can repartition and reformat from within the device. I've also updated the HOWTO since the old Root.sh script has been deprecated and replaced with this release. Simply install the package and you've got everything you need to set it up. Setup has become so simple: Code:
sudo gainroot By the way, "sfdisk" is included, too - so you can try that with the info I posed above instead of using "cfdisk" for automatic partitioning. |
Wow. This is really something.
Thank you, Wolfram (or should we say, Thank you, Stefan Daniel?) |
Hi,
I get an error message in cfdisk when running: /var/lib/install/sbin/cfdisk /dev/mmcblk0 FATAL ERROR: Bad primary partition 0: Partition ends in the final partial cylinder I have root access, and mmc card is unmounted. Any ideas? |
Never mind...I formatted it in ubuntu instead. :D
|
Is there a command that lets you see how much swap is being used? I'm trying to guage the effects now that I've set up swap with swapon.
|
the command free will show memory usage.
|
All times are GMT. The time now is 02:26. |
vBulletin® Version 3.8.8