Ok, I've got a project update. At first I had been all giddy about this, then trashed the idea after struggling fruitlessly with GDBM. But after Gnuite told me about a future DB change that'd once again make this script possible, I began dev work on it again. And since I believe in the principles of FOSS, I'm going to share the source code as I work on it to get some input, help and to improve it greatly.

To start with I came up with a simple engine for converting a decimal number to a grid address ID. The principle behind this is that you'll start with a zoom level, say level 16 (I'm counting zooms based on the way the grid addressing works with level 20 being a full 4 quadrant map of the earth) and going down to level 4 (you don't need any zooms higher than level 4). That's 12 levels. That's also one bloody heck of a lot of files. I did a test with 200 files and it came out to about 1mb average size for the entire lot.

It'll also pull 5460 tiles for the first six levels you go down. It takes close to 1 million tiles or about 5gb to do the entire state of Michigan. I could be off by a little bit, but that's a pretty fair estimate. And that's starting with just one grid tile on one zoom level as your focal point for starting the script. So the total file size and number of tiles is significantly smaller than I had first estimated. Mostly because the math works out a lot differently.

But anyways, here's the address engine that I have right now. Again, it's a bit rough, but this is a first effort and I figure that if we work together long enough this will evolve into a full fledged downloader script. Now, given everything I've learned so far, you're looking at having to run this script for close to 6 hours to do just a state level snapshot. But that certainly beats the 18-32 hours it would take with the Nokia tablet.

Also, given that maemo mapper will eventually be going to an sqllite database, I figure we may as well start building the downloader to import to that format of database. The only three areas I can really think we need to work on right now (I'm not saying they're the only areas, just the most immediate three) is the downloading module for grabbing the files, the sqllite database handler and importer, and an initial config section.

The config section would hold all the user editable configs for the script obviously, as most of these configs can't really be passed from the command line. Now it's also reasonable to think that these could be put out in a separate file and handled that way so as to avoid requiring end users to edit the actual parent script. So, anywho, without further adieu, here's the address engine. Now mind you there's some testing stuff in here still, so feel free to ignore that.

Ok, now with that out of the way, who's interested in helping me develop this. I'm sure gnuite would appreciate it if we could do the dev work instead and take that load off his hands.

edit: Ok, minor correction. It looks like I was estimating too small again. It looks like my original data and tile numbers are gonna be much closer to the mark.