Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Installing a whole lot of offline routing maps is something I'd like Marble to handle fine. When calculating routes, it determines the right country by checking the bounding boxes of all countries and then queries the monav backend. There are some heuristics (checking large countries first, trying the last used country before all others) to speed it up, though even iterating through all countries should be fast.

I think the culprit for increased memory usage is a change in the bounding boxes that happened in the last map update. I switched from cloudmade to geofabrik as the map data source. The latter provide detailed bounding box polygons which are unfortunately quite large for some countries. The largest one, Indonesia, for example has a size of 7 MB (!). In the current implementation these bounding boxes are loaded into memory. If you happen to have all ~300 maps installed, this sums up to around 75 MB which is a problem on the N900.

Fortunately there's a workaround I can implement: Detect 'large' bounding boxes at load time and replace them with a simple rectangular bounding box. This will decrease memory usage significantly, but implies that I need to change the backend query as well: Since the rectangular bounding box is a very rough approximation of the original shape, there will be some false positives when route requests lie inside the rectangular bbox, but not in the polygon. In that case other countries have to be evaluated as well. A better fix will be to replace the bounding boxes with a simplified polygon approximation. This should be doable with a variant of the Douglas Peucker algorithm that does not allow polygon shrinking.

To cut a long story short: I'll implement a workaround for the final 1.2 version. A later map update with simplified polygons will fix the underlying issue.

Note that in contrast to offline routing, installing a huge amount of offline search databases (which are included in the offline maps now) does lead to a slowdown of offline search queries. That's because there's currently no way to avoid searching a country database based on the search query.

Re England, Scotland: The great britain map does cover them and there are indeed files missing in the offline maps for england and scotland. You're also right that turkey and russia are split by continents, leading to a similar problem like the georgia name clash. I'll work on an update of the maps on the server in the next days to fix it. Other than those maps not working installing them shouldn't affect Marble negatively, however.

@niqbal: What kind of problems did you experience? Was offline routing not working for a certain area?

@Fellfrosch: Great to hear :) We don't have special Marble donations, but since KDE provides servers and covers our expenses (e.g. developer sprints) it makes sense to donate to KDE. Currently we're looking to get the T-shirts we'll provide as prizes for the Voice of Marble contest sponsored.

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

As for memory usage and out-of-memory, AFAIK Marble doesn't know if it's using real RAM or virtual memory (correct me if I'm wrong), so if Mentalist get oom, that would mean Marble is occuping around 1GB memory, so of course something is very wrong. If I'm correct, only one drawback from exceding physical RAM (256 MB) into virtual memory (768 MB or whatever ammount user format it to) should be huge slowdown.

If I'm wrong and Marble, by some flags, use only physical RAM (and trigger oom when 256 MB is exceed), is such a behavior necessary? V We got vmemory exactly to avoid oom, slow working is better than doesn't working at all.

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Being playing with Marble, and have the routing working with some voices converted from Tomtom. the routing is nice and clear, and apart from the odd hiccup (approaching 1 roundabout, voice - take first exit, map - 2nd exit (straight on, icon - turn right)is great.

Damn sight more stable than Sygic...
More convenient than Nokia Maps and OMVoiceserver.

I'm sold on it, but have a couple of requests..

Could it auto-pause and resume music playback via d-bus commands either side of a voice announcement (like Sygic). If the music is playing all you get is a nasty compressed noise, with the music paused or stopped it's fine.

Also I have noticed that quite often I can be approaching a junction, and the relevant part of the map is obscured by the guidance banner. Is there any way that could be improved in future versions. Possibly by moving the banner to the top, or orientating the map so that the direction of travel is "up the screen".

Thanks for a great app though.

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Done that. :)
Do you have enough voices for the German translation? I'm sure my voice isn't that nice, but if you really need a german voice let me know. A bad voice is better than no voice.

One request. Is there a possibility to route manually. Means, without an routing service, just setting waypoints somewhere in the map? That would be great, for using marble maps outdoor on tracks which the routing services don't know.
I've found out, that I can import KML files from google maps. But I haven't found a possibility to create KML files in marble maps without using a routing service.

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

I have a bug to report for marble. When editing a map source .dgml file the 'parameter' string is unable to accept special characters. i.e.:

To add virtual earth road maps the following url is used:
as a marble source string it is:
To get the returned tile as a UK OS map the string is:
In marble it should be:

However, with the inclusion of the ampersand the program crashes on launch (presumably when it is trying to parse all the map source .dgml files.)

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

@Mentalist Traceur: I implemented the workaround mentioned above (using a rectangular bbox instead of the large polygons), it will be available in RC 2. Please report if the behavior gets better. Independent of that, can you please check the memory consumption of Marble? Use e.g. htop. Best measure it first on startup and then when the routing config dialog is opened (which will trigger the loading of monav maps). The memory usage here should decrease significantly with RC 2, so it'd be good to have some numbers before installing that. I can't test it myself currently, had to send in my N900 for repair today (USB port is broken).

@Estel: We don't do any unusual/low level memory management. I think an out-of-memory situation will occur earlier than occupying all physically installed memory - the memory is shared with other application and the operating system.

@gazza_d: Can you send me a link to the problematic roundabout (preferably using openstreetmap.org). I'll look into the data to see what's going wrong there.

@Fellfrosch: We've got one German contribution so far, but the more, the better!
There's no manual routing implemented and kml editing is on the todo list, but so far nobody worked on it.

@vi_: As discussed on IRC the .dgml file syntax needs to be correct xml. I just fixed the crash in Marble though, so loading a syntactically incorrect .dgml file will now (RC 2) not result in a crash. The map will be ignored instead. Starting "/opt/marble/bin/marble.sh --debug-info" will give you a warning in the shell in that situation.

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Nice to here that kml editing is at least on the todo list. Hope that there will be soon somebody, who'll even work on it ;)
OK, if there is still only one contributor i'll try it. So I have a 50/50 chance to win. I think i'll do the recording on Wednesday :cool:

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Hi, these are stats using the latest version of Marble in the repos, with ALL maps installed (excluding Wales, Scotland and England, which should be covered by Great Britain.) Quick note: Cyprus displays the same "always update available" thing that Georgia had and Turkey and Russia currently still have. I'm guessing Cyprus registers as between two different continents too?

Anyway, this is a quick test using "top" and "free". No htop, sorry. I don't keep it installed on-device (though if you need something particular I can't get you otherwise, let me know and I'll oblige).

Upon launching, with every MoNav map installed, Marble takes a while to load... a couple of minutes I'd say, 3-5 or so. Watching in top as this happens demonstrates that Marble slowly rises in memory consumption, up to somewhere between 35 and 40 % RAM. (This is only RAM, not all virtual memory). However, when launched by itself it doesn't seem to impact the amount of data in swap much.

Launching the MoNav configuration menu actually didn't give me ANY issue in terms of delay and/or memory increase this time (1-2% at the most, although the amount in RAM fluctuates around that much over time anyway, as the N900 slowly swaps - well, slowly on my device, where swappiness is at 20. On a standard device swappiness is at 100, and I think as a result, it will hit this issue sooner). I wonder if this has to do with the fact that I've basically installed almost every map already? *Shrug*

Anyway, so then I "updated" Cyprus - during both the download and install, Marble's RAM memory consumption never rose above 42%. BUT, executing "free" every few minutes showed that it was putting more and more data into SWAP. After that finished I "updated" Russia - the RAM consumption never exceeded 45%, although the RAM being used pushed out all the buffers' space, to the point that only 2600~ bytes where free in memory, with only about 100-200~ more on top of that being buffers. The amount of space in SWAP continued to decrease, down to the point where by the end of the Russia map reinstall, almost half of SWAP space was used. Doesn't mean Marble took up all of it, but a decent amount, certainly.

Leaving MoNav's and then Marble's entire routing configuration menus didn't decrease either the RAM or SWAP space use, except by a few hundred bytes. At this point, I had approximately 8100+ bytes free in RAM (13200+ counting buffers) and still just barely under half of SWAP space free.

This suggests that the main cause of slow down is less the _immediate_ almost-total memory hogging as I had originally thought, but the immediate memory use followed by progressive memory use with each map install - the memory used with each map install doesn't seem to be freed until Marble close.

Which explains why I had continued slowness and occasionally device reboots, etc. The N900 would swap out a lot with each new map install - longer uptimes and especially many installs in a row would both contribute to SWAP space fragmentation, which means slower SWAP write (as flash media slows on writes with fragmentation) speeds on each consecutive write.

As for testing RC2, if that's not what's in -devel right now, then I would much appreciate it if you put it in -devel before I tested it. It's just far easier for me that way, on top of the argument I had in favor of -devel before. If you're not willing to do that though, I'll do my duty as a good tester and deal with downloading the RC2 from wherever not-in-maemo-repository it happens to reside.

(By the way, that female contribution to the voice contest I promised is still coming, I just haven't gotten around to getting her to record...)

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Thanks for the detailed analysis. It was pretty easy to find the problem now (a memory leak). It explains the behavior you describe: At startup the maps are loaded into memory. I measured it on the desktop, the loading leads to an additional memory consumption of 220 MB. Because of the memory leak, after the installation of a new map (causing another parsing), another 220 MB are allocated and so on.

RC 2 will have the memory leak fixed and contains the workaround for the too detailed polygons. This brings the memory consumption down to 66 MB with all monav maps installed (in comparison it's 52 MB without any monav map).

I'll test it on the N900 when I get the device back, but I'm pretty sure it will behave much better in this (rather extreme) setup. RC 2 is scheduled for 2011-07-06; I'm not sure yet if I'll do an intermediate release before.

Thanks for getting a female contribution to the voice of marble contest, much appreciated :)

Re: [ANNOUNCE] Marble - Find Your Way and Explore the World
 

Ooooo... that explains a lot.

Did I ever mention how awesome you (both you individually and you the entire set of Marble developers) are?

Honestly, I think the more 'intermediate' releases are made the better. I mean, you know my opinion on the extras-devel thing - it's not enabled by default and everyone enabling it should _hopefully_ know what the hell they're doing, plus it's an opportunity to catch some bugs slightly sooner and/or convenience some users sooner.

*Nod* Happy to help. I'll also do some male voices for a couple of languages personally if I have time. :)

- Edit -

I'm stupid, yesterday was the 20th, not the 30th. Okay, so it's 17 days before RC2. My bad.
So, today was an utter mind**** in terms of dates. I had my computer at work set to the wrong date... hence the above edit...