Re: Maemo Mapper Mod - the new map file format
 

It doesn't matter if it writes to the same place or not, it's the number of writes that matters - the writes are wear-levelled over the card so there's no difference if you re-write the same VFAT block or not (not taking into account possible wear-levelling sectioning of the card, which is up to the vendor).

Maybe that's what you meant anyway, just thought I should mention it to avoid confusion.

Re: Maemo Mapper Mod - the new map file format
 

That's what I mean, the number of writing, but has to be to the same location.

The GDBM uses default allocation unit size as the hash table(bucket) size, normally it's 32k on a SDHC card.

Each data item uses about 12 bytes, so each bucket contains about 3000 entries. For every entry update it writes disk to sync it, so it writes about 3000 times to one block just to fill it. Sounds like a killing machine to SD card.

syzygy, I am glad you even make the repository smaller. Honestly I didn't dig deep into the detail implementation of the gdbm. It's a very old piece of junk. I can't find even a working version on Windows platform, none works actually. I ported it to Windows but didn't do any clean up. Time to move on to rid of it.

Re: Maemo Mapper Mod - the new map file format
 

What I dislike most is the GDBM header, it contains all the important information and locates in just one allocation unit. So each tile update it needs to write to the header once, you have 50k tiles, it write 50k times.

If the operating system doesn't have the dynamically changing allocation support for SD card(I don't know if maemo has or not), it pretty much ensure a kill of that block some time down the road.

Re: Maemo Mapper Mod - the new map file format
 

I still think there may be a misunderstanding here - if by dynamically changing allocation support you really mean what I called 'wear levelling'. That's a feature of the SD card itself, not of the operating system (unlike the built-in 256MB flash. That's why the internal flash is using JFFS2, as that's a file system with wear-levelling built in. That's not necessary for cards.)

Nevertheless, I'm still very interested in your experiments with alternative map storage formats.

Re: Maemo Mapper Mod - the new map file format
 

My understanding is that it needs operating system support, although I am not very sure. If it's up to the card controller support, then that's something even we can't know, since none of the manufacturers specify it.

Anyway, Traditional GPS doesn't have this problem since they rarely do real-time downloading maps, the maps are pre-installed. I don't want to bet on the chance.

Re: Maemo Mapper Mod - the new map file format
 

@nutter: I'm relatively sure that it happens on the SD card itself. If I understand correctly, even the SD controller on the N800 doesn't have to know anything about the actual mapping of physical blocks to logical blocks. AFAIK, the only thing really needs to be avoided are lots of small writes, since every time 1KB is written a whole block gets erased and written. Better to wait and do 1 64KB write, than 64 1KB writes. BTW, thanks for the work on tile based storage.

-John

Re: Maemo Mapper Mod - the new map file format
 

Nutter, I think you may be somewhat confused about how the system works. 30,000 writes to an SD card takes a substantial amount of time. I think those writes you're talking about take place in RAM, and then are written once to the card. You can change the contents of a file almost indefinitely in RAM, but you don't have to write the whole database to a file each time. It's trivial to manipulate files in memory indefinitely, and only write to storage once when finished. That's the way almost every program works.

Re: Maemo Mapper Mod - the new map file format
 

Nutter,

I'm in love again with MM1.4, on my N800 and OS2008. VERY snappy performance overall, and as good or better file sizes compared to MM2.2.

I've encountered a couple of minor problems, and was wondering if you had any clue as to what is going on.

First, I believe you were correct that MapsIn1 crashed when it encountered corrupt tiles during translation. These showed up as a black 4x2 tile area when viewed. I tried to correct this by checking the 'Overwrite' button, and downloading the area again. Same result--there was still a black area. I then manually deleted the offending .map files, navigated to the area, and new ones auto-downloaded properly. I also encountered a crash of MM when trying to download a larger area, but I don't think this had anything to do with the corrupt file issue. It's important to have the overwrite function work properly, especially for updating maps.

Second, something is strange with the hardware keys. With MM2.2 I had Zoom In and Zoom Out mapped to the up and down rocker buttons on the front panel. These will not work as zoom buttons, but are fine for other functions. I reset the buttons and reselected, but no go.

Ideas?

Re: Maemo Mapper Mod - the new map file format
 

hi nutter,

it seems that you made sth. very useful for all maemo mapper users.
Did you talk to the maintainer of maemo mapper, john costigan?
I think it would be the best idea to integrate your great improvements
directly in maemo mapper releases.
So people get the speed and storage improvements when they simply update
maemo mapper.

So - did you have contact to john - will he integrate your code in the main release of maemo mapper ?

Thanks for your mod again and again ....

Regards,
asys3

Re: Maemo Mapper Mod - the new map file format
 

I checked some resources, TA-t3 is right about the flash card features. Operating system can support wear levelling, also SD controller supports it as well. But not all SD card support that, many cheap ones don't. As for MM2.x writing to flash card, I checked the source code, it does write that many times. Since for every tile downloading, the code does a "sync", that's the flushing the file to the disk. It will write to header, hash table and data section.

syzygy, I guess your hardware key problem is due to that you use MM2.x before, and its key configuration method/value may be different from the MM1.x, and downgrade may confuse the MM1.x. BTW, I didn't change anything in both MM1.x and MM2.x except replacing their file system, so the problem may well be from the incompatibility of the two. I need to look into the source code to see how old MM1.4 and new MM2.2 handle the hardware key and replacing tile issue, to find some resolution for you.

BTW, since this is the engineering experiment to probe the way to improve MM, so I tried to limit the code change as small as possible.