maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Applications (https://talk.maemo.org/forumdisplay.php?f=41)
-   -   [Announce] modRana: a flexible GPS navigation system (https://talk.maemo.org/showthread.php?t=58861)

skykooler 2011-07-08 21:22

Re: [Announce] modRana: a flexible GPS navigation system
 
How did you make it keep the display on? I am writing a photo frame app and can't figure out how to do that.

MartinK 2011-07-08 22:16

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by superpj101 (Post 1044838)
interested on this one. also any chance for a simple converter to convert it to sqlite format? for less space.

Well, it is certainly technically feasible to do - even both ways.
Maybe a CLI utility for starters. :)

Quote:

Originally Posted by skykooler (Post 1047390)
How did you make it keep the display on? I am writing a photo frame app and can't figure out how to do that.

Check out the n900 device module in modRana source:
http://modrana.org/trac/browser/trun...device_n900.py
Especially these lines:
http://modrana.org/trac/browser/trun...ce_n900.py#L49
http://modrana.org/trac/browser/trun...e_n900.py#L150

In a nutshell, you have to connect to the MCE through DBUS and pause blanking every <60s for another 60s. :)

Sin 2011-07-10 17:09

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by Sin (Post 1044445)
hello, I didn't read whole 740 post so, sorry if it is being asked twice,

How can I download tiles to my pc? I should find a way to do that because I chose around here, 80 km, +8 down, +5 up (or sth like that), modrana says ~355000 tiles to be downloaded. I left my N900 to download them last night, at the morning I saw it downloaded 30k of them, AND was downloading too slow if I compare to beginning. So, I need to download them to my pc then should move them to n900. Tiles are in /home/user/MyDocs/.maps/OpenStreetMap I , I guess ?

Quote:

Originally Posted by dwaradzyn (Post 1027429)
On your PC you can use Mobile Atlas Creator. Set atlas format to "OSMTracker tile storage". There is plenty of information on how to use this tool including video tutorials. Just google it.

Tryed gmapcatcher, but file structure seemed to be different. Now trying this, file structure seems to be same with modRana. Hopefully this will help, after waiting some 7-8 hours to be downloaded.

MartinK 2011-07-11 22:59

Re: [Announce] modRana: a flexible GPS navigation system
 
I have just released V0.25-5. :)
  • it contains an improved version of Gkuennings patch, that fixes overlapping announcements being skipped
  • modRana no longer uses the google provided triggering distances, but a minimal distance (100 m default, configurable in Options) + Gkuennings patch to assure sanely close turn announcements - this might also come in handy in the future when using other online/offline routing methods
  • there is now a bit more advanced distance rounding that should make the voice messages shorter without sacrificing any significant precision
  • modRana now automatically enlarges the point reached distance when it is smaller than current per second speed ( = modRana should no longer skip turns due to high speed)
  • T-b-T update is now synchronized with position update ( = it should better react to position changes)

petur 2011-07-12 00:09

Re: [Announce] modRana: a flexible GPS navigation system
 
Martin,

just an enhancement idea: right now, maps get downloaded when they need to be displayed (if configured likewise). It would be great to increase the rectangle that triggers a download.

Today I was driving around and modRana was downloading on-the-go. The link wasn't very good so downloads were not very fast, which made some tiles take some time before downloaded.

In this use-case, having the download trigger earlier would cause blue/green tiles never to become visible - ie making sure they are downloaded before they come into view.

I admit this does increase download amounts a bit. Maybe this can be optional or only done for the route ahead (if any).

woody14619 2011-07-12 16:10

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by petur (Post 1049223)
Martin,

just an enhancement idea: right now, maps get downloaded when they need to be displayed (if configured likewise). It would be great to increase the rectangle that triggers a download.

On a related note, an option to download all tiles along a freshly computed path would be nice. If I route to an address, good odds I'm going to drive there. And if there's a bad patch of non-coverage along the way, having the tiles pre-downloaded would really be nice. :)

All the parts needed are there... It's effectively a "download along route" for the currently selected route, without the need to save, find, re-load and hit the dozen button combos needed to do it. :)

slender 2011-07-12 16:32

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by petur (Post 1049223)
Martin,

just an enhancement idea: right now, maps get downloaded when they need to be displayed (if configured likewise). It would be great to increase the rectangle that triggers a download.

Today I was driving around and modRana was downloading on-the-go. The link wasn't very good so downloads were not very fast, which made some tiles take some time before downloaded.

In this use-case, having the download trigger earlier would cause blue/green tiles never to become visible - ie making sure they are downloaded before they come into view.

I admit this does increase download amounts a bit. Maybe this can be optional or only done for the route ahead (if any).

I had this problem especially when I was using in portrait mode and did ~90 turns.

When not routing maybe rectangle area or triangle where the forward area is further away and when routing then rectangle follows current route in advance.

petur 2011-07-12 23:19

Re: [Announce] modRana: a flexible GPS navigation system
 
Another enhancement idea:

Right now, the location of the current position on the display is selectable (middle or 75% IIRC). For straight roads this works fine, but elsewhere, this could do better.

How about a nifty way to locate the current position such that a maximum of the route ahead is shown on the screen? Maybe it could select between 5 positions (center + 75% towards each edge) depending on where the route is going...

skykooler 2011-07-13 11:51

Re: [Announce] modRana: a flexible GPS navigation system
 
I just had a chance to test it with the new features yesterday. I just got a Bluetooth earpiece, and was using that and ModRana's voive directions to navigate. I now have a new request: If there is going to be a long distance before the next direction, coult it say how far away it is? For example, sayin "Drive 5 km, then turn right on Cherry St."

jj0 2011-07-14 07:31

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by skykooler (Post 1050091)
If there is going to be a long distance before the next direction, coult it say how far away it is? For example, sayin "Drive 5 km, then turn right on Cherry St."

I second this, this would be a very useful add-in, often times I take the highway, where only upon inspecting modRana's map, I see that the next turn is in 100km or so, so having it pronounced would be great.

In fact most non-free routing apps do this - say: Take the highway and drive for 100km...

MartinK 2011-07-14 23:10

Re: [Announce] modRana: a flexible GPS navigation system
 
I'm leaving for State for state of the map Europe (anyone else going ? :) ) on Friday and I'll process all your feedback when I get back on Sunday/Monday. :)

7thd 2011-07-16 07:17

Re: [Announce] modRana: a flexible GPS navigation system
 
I use Modrana not in the least for its tracklogging options.
I cruised Israel by touring car and Modrana was perfectly usable to orientate and make tracklogs to keep a journal to match the photos I shot. In some films I shot I just aimed at my screenie now and then. I'm from the Nether lands so I was positively surprised that it did log the altitudes so well (Dead sea f.i. is 400 meters below sea level).

I like hiking and sailing.
While I use Modrana with Virtual Earth sat map for all navigation purposes I have a bulky Tomtom too. Sometimes in the car I use both:p --> Modrana for the photo-artistic side more or less zoomed out. On my Tomtom I have Offroad navigator installed and that has a function I would much like on Modrana as well: absolute directions for next waypoint. If it could show the angle to the next waypoint you would be able to make accurate decisions at sea (where you not only evaluate the absolute angle, but also stream, wind etc). Cumulus has this option but I much prefer Modrana UI.

lawabb 2011-07-18 04:17

Re: [Announce] modRana: a flexible GPS navigation system
 
Hi, Have just come across modrana while trying to find info on rana - looks good so far! Anyway this looks like the place to be for info on modrana. I'm interested on getting modrana working on a PC and was wondering about the state of the gpsd module. I have a working usb gps on my laptop (running kubuntu), but (as with rana) the gps side of things are not working. From a quick look at the gps code eg

def gpsStatus(self):
return(self.socket_cmd("M"))

seems to be using the M command. At http://gpsd.berlios.de/protocol-transition.html it says "The old M command has no equivalent. Mode is reported in the TPV response." So perhaps the gpsd code is out of date, or am I missing something?

MartinK 2011-07-19 20:46

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by petur (Post 1049223)
Martin,

just an enhancement idea: right now, maps get downloaded when they need to be displayed (if configured likewise). It would be great to increase the rectangle that triggers a download.

Today I was driving around and modRana was downloading on-the-go. The link wasn't very good so downloads were not very fast, which made some tiles take some time before downloaded.

In this use-case, having the download trigger earlier would cause blue/green tiles never to become visible - ie making sure they are downloaded before they come into view.

I admit this does increase download amounts a bit. Maybe this can be optional or only done for the route ahead (if any).

Quote:

Originally Posted by slender (Post 1049591)
I had this problem especially when I was using in portrait mode and did ~90 turns.

When not routing maybe rectangle area or triangle where the forward area is further away and when routing then rectangle follows current route in advance.

OK, I have summarized this to a feature request ticket. :)

Quote:

Originally Posted by skykooler (Post 1050091)
I just had a chance to test it with the new features yesterday. I just got a Bluetooth earpiece, and was using that and ModRana's voive directions to navigate. I now have a new request: If there is going to be a long distance before the next direction, coult it say how far away it is? For example, sayin "Drive 5 km, then turn right on Cherry St."

Quote:

Originally Posted by jj0 (Post 1050679)
I second this, this would be a very useful add-in, often times I take the highway, where only upon inspecting modRana's map, I see that the next turn is in 100km or so, so having it pronounced would be great.

In fact most non-free routing apps do this - say: Take the highway and drive for 100km...

This sound useful & quite easy to implement. What about this:
X - distance to next turn
If the next turn is more than 10 km away, announce at X - 2 km "next turn is in (X-2) km" - no turn description as no one would remember it anyway IMO. Any thoughts on the this & the trigger distances ?

Quote:

Originally Posted by petur (Post 1049842)
Another enhancement idea:

Right now, the location of the current position on the display is selectable (middle or 75% IIRC). For straight roads this works fine, but elsewhere, this could do better.

How about a nifty way to locate the current position such that a maximum of the route ahead is shown on the screen?

Maybe it could select between 5 positions (center + 75% towards each edge) depending on where the route is going...

Good point, ticket added. :)

Quote:

Originally Posted by woody14619 (Post 1049579)
On a related note, an option to download all tiles along a freshly computed path would be nice. If I route to an address, good odds I'm going to drive there. And if there's a bad patch of non-coverage along the way, having the tiles pre-downloaded would really be nice. :)

All the parts needed are there... It's effectively a "download along route" for the currently selected route, without the need to save, find, re-load and hit the dozen button combos needed to do it. :)

Well, yeah, you are right, the infrastructure is all there, should be easy to add. :)

MartinK 2011-07-19 21:07

Re: [Announce] modRana: a flexible GPS navigation system
 
SotM EU 2011 in Vienna was really nice. :)
I have found about the Kothic Python based offline map renderer that might just work with modRana and also gave an unplanned lightning talk about modRana. :D There was no visualizer, I had no N900 TV-out cable or slides so I just showed an old youtoube video and spoke about what features modRana has. :)

Most of the conference talks are online as slides or recordings, as are many photos on flicker or in my SotMEU2011 gallery. :)

Quote:

Originally Posted by 7thd (Post 1051957)
has a function I would much like on Modrana as well: absolute directions for next waypoint. If it could show the angle to the next waypoint you would be able to make accurate decisions at sea (where you not only evaluate the absolute angle, but also stream, wind etc). Cumulus has this option but I much prefer Modrana UI.

Well, I actually needed this when walking form the train station to the conference - I had the conference as a POI, the tiles preloaded, but no roaming so I could not get an online route. :D
I really need to add this soon. :)

@lawabb
I sent you an email regarding this. :)
Basically, support for new GPSD protocol is implemented in V0.26-5+ but I forgot to upgrade the non-Maemo packages on the modRana website.

skykooler 2011-07-20 05:00

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by MartinK (Post 1054353)
This sound useful & quite easy to implement. What about this:
X - distance to next turn
If the next turn is more than 10 km away, announce at X - 2 km "next turn is in (X-2) km" - no turn description as no one would remember it anyway IMO. Any thoughts on the this & the trigger distances ?

Ok - but I usually like to know at least the turn direction so I know (for example) which lane to get in, as there may not be time nearer the destination. Perhaps this could be a toggle? Also, I would probably use this a lot in the city, where the distance is a lot less than 10 km - perhaps this can be configurable too?

slender 2011-07-20 20:07

Re: [Announce] modRana: a flexible GPS navigation system
 
I promised to post some example from tomtom gui so here it goes
http://aijaa.com/v.php?i=000948347385.jpg

1. Animated compass needle showing your current direction in degrees.

2. Graphical sign for next turn & distance to turn

3. The number or name of the road your are turning

4. from top left. Calculated time for routed trip, current time, speed (sometimes in format xx/80 to show limit), total distance to end. On far right are the bars for satellite coverage.

5. Zoom buttons (Far away from each other to minimize occurrence of miss taps of wrong button)

Of course you can just turn on/ off most of these from settings. IMO e.g. white text on blue background really really jumps out from screen. On N900 maybe the best compromise would be to use right or left edge of screen with column of ~200px

gkuenning 2011-07-21 07:14

Re: [Announce] modRana: a flexible GPS navigation system
 
Martin, quick question (because I'm too lazy to dig through the code right now!):

How hard would it be to allow turn-by-turn routes to be imported from a file? I'd like to be able to use Modrana to navigate bike rides where I take a complicated loop starting and ending at my house. I think I could construct the loop using Google Maps, if there were an easy way to transfer it to the phone and load it into Modrana.

(Mappero has a well-hidden feature where you can ask it to start a route at the end of the previous route, which can be used for the same application, but it's pretty clumsy. I like the idea of doing it on my laptop with a big screen and mouse, and then sending the result off to the phone.)

kureyon 2011-07-22 02:27

Re: [Announce] modRana: a flexible GPS navigation system
 
3 Attachment(s)
There is a problem with displaying Chinese characters when using POI search:
  • In map view I centre the view over somewhere in Hong Kong
  • The using POI online search to search for something (in this case I searched for restaurants)
  • The list returned will show Chinese characters as a rectangular outline
  • However clicking on that POI to show details will display the Chinese characters properly
  • Unfortunately showing the POIs on the map view will again turn the Chinese characters into rectangular outlines

beermad 2011-07-23 12:14

Re: [Announce] modRana: a flexible GPS navigation system
 
Terrific application and amazingly full of features considering how new it is.

As an interim measure until you manage to include a feature to import tile files into the database, I'm writing a Perl script to do the job. I can see from the structure of the SQLite databases that it's designed to have multiple store.sqlite.n files (presumably because otherwise the store could get ridiculously large: it looks like my 200 000 tile files would make for a file of about 2G). What logic do you use to decide when to create a new one? Is it when the last one gets to a certain size threshold?

Once I've incorporated that into my script I'll post it here for anybody who might find it useful. Looks like it's a slow process if you've got a lot of tiles, though; my current test job has been running for about 90 minutes and has only imported 13 000 tiles.

droll 2011-07-23 14:15

Re: [Announce] modRana: a flexible GPS navigation system
 
try copying the tiles to a pc and running your script there.

MartinK 2011-07-23 17:18

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by slender (Post 1055032)
I promised to post some example from tomtom gui so here it goes
http://aijaa.com/v.php?i=000948347385.jpg

Thanks ! :)

Quote:

Originally Posted by slender (Post 1055032)
1. Animated compass needle showing your current direction in degrees.

So this is basically current bearing vs north ? How useful is it actually ?

Quote:

Originally Posted by slender (Post 1055032)
2. Graphical sign for next turn & distance to turn

3. The number or name of the road your are turning

This info should be hopefully available once modRana starts using Monav. BTW, I'm already starting to test it's Python bindings.

Quote:

Originally Posted by slender (Post 1055032)
5. Zoom buttons (Far away from each other to minimize occurrence of miss taps of wrong button)

The zoom buttons in modRana are IMO already far enough - but is shows nicely how elaborately is the TomTom GUI designed. :)

Quote:

Originally Posted by slender (Post 1055032)
On N900 maybe the best compromise would be to use right or left edge of screen with column of ~200px

And maybe automatically switch to the bottom once the device is rotated to portrait ?

Quote:

Originally Posted by gkuenning (Post 1055218)
Martin, quick question (because I'm too lazy to dig through the code right now!):

How hard would it be to allow turn-by-turn routes to be imported from a file?

Well not that difficult IMO. :)
Actually, thanks to your speed based voice message triggering patch modRana no longer needs the Google provided triggering distances. So a simple GPX file with some points having routing messages (using the <cmt>, <desc> or eveng <extension> tag ? GPX 1.1 schema) should be now quite easily usable for routing.

Some notes to this:
  • modRana already does GPX loading for the tracklogs
  • the Turn-by-Turn navigation module currently just processes the raw Google Directions result
    • a simple generic class for storing route/tracklog info might be needed
    • which would also help adding new online/offline routing providers, storing and loading routes, automatic route reloading at startup, etc.

Quote:

Originally Posted by kureyon (Post 1055817)
There is a problem with displaying Chinese characters when using POI search:

Thanks for reporting! I'll check it out - looks like a Unicode conversion error to me.

sajk 2011-07-23 18:21

Re: [Announce] modRana: a flexible GPS navigation system
 
each time I try routing I always get "no route found". can someone tell me where I am going wrong?

MartinK 2011-07-23 18:22

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by beermad (Post 1056807)
Terrific application and amazingly full of features considering how new it is.

As an interim measure until you manage to include a feature to import tile files into the database, I'm writing a Perl script to do the job.

There is some rough sqlite tile storage documentation on t the modRana project website.
Its currently quite basic so please let me know if you find any errors or omissions.

Quote:

Originally Posted by beermad (Post 1056807)
I can see from the structure of the SQLite databases that it's designed to have multiple store.sqlite.n files (presumably because otherwise the store could get ridiculously large: it looks like my 200 000 tile files would make for a file of about 2G). What logic do you use to decide when to create a new one? Is it when the last one gets to a certain size threshold?

It lists the folder for available store files, then it checks them sequentially until it finds one that is under the maximum size threshold. If no such store is found, new one is created. Maximum store size threshold is 3.7 GB.

The Python code that does this in modRana is here:
http://modrana.org/trac/browser/trun...eTiles.py#L200

Quote:

Originally Posted by beermad (Post 1056807)
Once I've incorporated that into my script I'll post it here for anybody who might find it useful.

Can including it in the modRana package once it's done ? - Only if you don't mind, of course. :)
This way it could be placed in /usr/bin and would be easily callable as it would be in $PATH. :)

Quote:

Originally Posted by beermad (Post 1056807)
Looks like it's a slow process if you've got a lot of tiles, though; my current test job has been running for about 90 minutes and has only imported 13 000 tiles.

I hit something similar - the solution was not to commit after every tile but just once every 50 or so tiles. This made the process quite faster IIRC.

beermad 2011-07-24 09:32

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by MartinK (Post 1056975)
It lists the folder for available store files, then it checks them sequentially until it finds one that is under the maximum size threshold. If no such store is found, new one is created. Maximum store size threshold is 3.7 GB.

Ah, good. Nice and big. I wondered if maybe the size was restricted for performance reasons

Quote:

Can including it in the modRana package once it's done ? - Only if you don't mind, of course. :)
This way it could be placed in /usr/bin and would be easily callable as it would be in $PATH. :)
Quite happy with that in principle. In practice it may be tricky as the script has dependencies on the Perl DBI and DBD-SQLite libraries, so my original design is for it to be run on a normal Linux system rather than on the N900 (probably rather faster there as well). But I'll see if I can port the necessary libraries and bundle up what's needed.


Quote:

I hit something similar - the solution was not to commit after every tile but just once every 50 or so tiles. This made the process quite faster IIRC.
Good point. I need to build in a restart facility as well; a long-running process like this needs to be interruptible.

jj0 2011-07-26 09:23

Re: [Announce] modRana: a flexible GPS navigation system
 
Is there a problem in the latest version with displaying composite tiles?

I haven't used it in a long time, just using OSM, but tried just now a couple of times with OSM and Virtual Earth Sat and it just gets stuck on Loading... for each tile. Individually both work, but composite doesn't...

jj0 2011-07-26 10:34

Re: [Announce] modRana: a flexible GPS navigation system
 
It seems the display fails for overlaid images, when one doesn't have the tiles to display, i.e. you zoom too much in for instance, so that no sat images are available.

Whereas before it overlaid also the not-available tile info, not it just shows loading...

But when I zoom out a level, it works.

beermad 2011-07-26 15:15

Re: [Announce] modRana: a flexible GPS navigation system
 
OK, I've got a working Perl script to import map tiles into the database.

At the moment, it won't run on the N900 itself, as I'm having a few problems getting the SQLite module to build on my scratchbox installation; I'll work on that over the next few days when I have spare time and see if I can get that working.

Dependencies: You'll need to install the Perl DBI and DBD-SQLite modules. These should be available from your disto's repositories (otherwise, get them from CPAN).

It took about 75 minutes to import my 200 000 tiles into the database on my dual-core Athlon machine; obviously this will vary depending on the speed of the machine it's done on. The script is interruptible - just hit Control-C and it'll shut down cleanly. Then restart at your leisure.

Getting it to work on an OS other than Linux is left as an exercise for the reader, as I'm happily Microsoft-free.

It demonstrates well the oft-mentioned clustering problem with FAT partitions; my tile files allocated 12.5G of space, whereas the store file is only 543M.

Instructions:

Download the script from http://www.beermad.org.uk/mapconvert

Either copy your ~/MyDocs/.maps directory to your Linux box or mount your MyDocs partition on it. Try to preserve the timestamps if copying (as they're recorded in the database, I assume they're used to work out if a new copy of any tile is needed if they've changed on the tile server) - for example, if using scp:
Code:

scp -pr ~/MyDocs/.maps mycomputer:/tmp
The script only works on tiles in the current working directory, so for each map-type, cd into the relevant directory:
Code:

cd /path/to/tile_directory
Then run the script (no parameters needed).

If necessary, copy the lookup.sqlite and store.sqlite.* files back to your N900.

If there are any tiles already in the database, it leaves them there and it doesn't try to over-write any if one it finds already exists (at some time I should change this so it does over-write if the tile file is newer).

If you already have tiles in a database, I recommend making a copy first, just in case anything goes wrong. I've tested the script a fair bit, but can't guarantee something won't screw up.

MartinK 2011-07-26 17:45

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by jj0 (Post 1058458)
It seems the display fails for overlaid images, when one doesn't have the tiles to display, i.e. you zoom too much in for instance, so that no sat images are available.

Whereas before it overlaid also the not-available tile info, not it just shows loading...

But when I zoom out a level, it works.

ModRana now overlays the two tiles only once and then caches the result & displays the result. Before the tiles are combined, only the loading tile is shown.

Like this, when tiles from one layer are missing, nothing is shown. :)
This can be IMO improved:
  • just showing the "good" layer
  • overlaying with the error tile to inform the user that one layer is broken

Quote:

Originally Posted by beermad (Post 1058668)
OK, I've got a working Perl script to import map tiles into the database.

Can I put your script & HOWTO on the modRana wiki ? :)

beermad 2011-07-26 18:10

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by MartinK (Post 1058766)
Can I put your script & HOWTO on the modRana wiki ? :)

Certainly. Be my guest. Hope it's useful to a few people.

beermad 2011-07-26 23:57

Re: [Announce] modRana: a flexible GPS navigation system
 
Hi Martin,

From looking through the database code and structure to produce the migration script, I think I've found a few points where the code and schema could be more efficient.
  • When you're retrieving tiles, you're pulling the epoch_timestamp from the databases. Except when you're checking if you need (or want) to download an updated tile, this is unnecessary, so just wastes CPU cycles.
  • After retrieving details from the lookup table, it seems inefficient to find the right tile in the store table using x,y,z when you've already done that on the lookup table. It would be more efficient to have a single primary index in the store table, referenced in the lookup table entry.
  • Moving on logically from the previous idea, it might be possible to make the lookup table more efficient. At the moment you need to work through three indexes (x,y,z) to find the right tile. It might be more processor efficient to have a single index on this table which is either a combination of the x,y,z parameters or a hash based on them (neither option is simple, but would probably speed things up a worthwhile amount).

白い熊 2011-07-30 16:13

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by MartinK (Post 1058766)
ModRana now overlays the two tiles only once and then caches the result & displays the result. Before the tiles are combined, only the loading tile is shown.

Like this, when tiles from one layer are missing, nothing is shown. :)
This can be IMO improved:
  • just showing the "good" layer
  • overlaying with the error tile to inform the user that one layer is broken

I'd vote for the 2nd option, that way you know one layer is missing...

白い熊 2011-07-30 16:13

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by MartinK (Post 1056939)
Thanks for reporting! I'll check it out - looks like a Unicode conversion error to me.

Same problem with Japanese POIs.

白い熊 2011-07-30 19:10

Re: [Announce] modRana: a flexible GPS navigation system
 
When downloading a large area, modRana segfaults. Thought it was only on the N900, but the PC version exhibits the same behavior.

When trying to download a 40km area, I get an estimated download of 90 thousand tiles, modRana segfaults after cca 10 thousand tiles with:

Code:

Batch tile dl working... (threads: 8) pending: 46496, done: 43730
Batch tile dl working... (threads: 8) pending: 46489, done: 43737
Batch tile dl working... (threads: 8) pending: 46482, done: 43744
*** glibc detected *** python: corrupted double-linked list: 0x0bbafbe8 ***
======= Backtrace: =========
/lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x6aac1)[0xb75acac1]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x6c3ec)[0xb75ae3ec]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xb75b13dd]
/usr/lib/libsqlite3.so.0(+0x15c80)[0xb623fc80]
/usr/lib/libsqlite3.so.0(sqlite3_free+0x81)[0xb6230a51]
/usr/lib/libsqlite3.so.0(+0x4c1c2)[0xb62761c2]
/usr/lib/libsqlite3.so.0(sqlite3_close+0x11f)[0xb627635f]
/usr/lib/python2.6/lib-dynload/_sqlite3.so(pysqlite_connection_close+0xfd)[0xb62d315d]
python(PyEval_EvalFrameEx+0x44d0)[0x80e08c0]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python(PyEval_EvalFrameEx+0x476f)[0x80e0b5f]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python(PyEval_EvalFrameEx+0x476f)[0x80e0b5f]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python[0x8174d87]
python(PyObject_Call+0x4a)[0x8066d7a]
python(PyEval_EvalFrameEx+0x3133)[0x80df523]
python(PyEval_EvalFrameEx+0x5437)[0x80e1827]
python(PyEval_EvalFrameEx+0x5437)[0x80e1827]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python[0x8174c9c]
python(PyObject_Call+0x4a)[0x8066d7a]
python[0x806f931]
python(PyObject_Call+0x4a)[0x8066d7a]
python(PyEval_CallObjectWithKeywords+0x42)[0x80dc2f2]
python[0x8113faf]
/lib/i386-linux-gnu/i686/cmov/libpthread.so.0(+0x5c39)[0xb78a7c39]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(clone+0x5e)[0xb760e93e]
======= Memory map: ========
08048000-08235000 r-xp 00000000 08:04 20017      /usr/bin/python2.6
08235000-08284000 rwxp 001ed000 08:04 20017      /usr/bin/python2.6
08284000-0828d000 rwxp 00000000 00:00 0
08b3c000-0bbe4000 rwxp 00000000 00:00 0          [heap]
af4fe000-af4ff000 ---p 00000000 00:00 0
af4ff000-afcff000 rwxp 00000000 00:00 0
afcff000-afd00000 ---p 00000000 00:00 0
afd00000-b0500000 rwxp 00000000 00:00 0
b0500000-b05ff000 rwxp 00000000 00:00 0
b05ff000-b0600000 ---p 00000000 00:00 0
b0b22000-b0d23000 rwxp 00000000 00:00 0
b0e24000-b0e25000 ---p 00000000 00:00 0
b0e25000-b1bfe000 rwxp 00000000 00:00 0
b1cb1000-b23ff000 rwxp 00000000 00:00 0
b23ff000-b2400000 ---p 00000000 00:00 0
b2400000-b2c00000 rwxp 00000000 00:00 0
b2c00000-b2d00000 rwxp 00000000 00:00 0
b2d7d000-b2d7e000 ---p 00000000 00:00 0
b2d7e000-b357e000 rwxp 00000000 00:00 0
b357e000-b3582000 r-xp 00000000 08:04 16174      /lib/i386-linux-gnu/i686/cmov/libnss_dns-2.13.so
b3582000-b3583000 r-xp 00004000 08:04 16174      /lib/i386-linux-gnu/i686/cmov/libnss_dns-2.13.so
b3583000-b3584000 rwxp 00005000 08:04 16174      /lib/i386-linux-gnu/i686/cmov/libnss_dns-2.13.so
b35b1000-b366c000 rwxp 00000000 00:00 0
b366c000-b382b000 rwxp 00000000 08:04 19603      /usr/share/file/magic.mgc
b382b000-b38e6000 rwxp 00000000 00:00 0
b38e6000-b397c000 r-xp 00000000 08:04 65546      /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Oblique.ttf
b397c000-b3a21000 r-xp 00000000 08:04 63682      /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Bold.ttf
b3a21000-b3b97000 rwxp 00000000 00:00 0
b3c00000-b3cb4000 rwxp 00000000 00:00 0
b3cb4000-b3d00000 ---p 00000000 00:00 0
b3d7b000-b3d97000 r-xp 00000000 08:04 14758      /lib/i386-linux-gnu/libgcc_s.so.1
b3d97000-b3d98000 rwxp 0001b000 08:04 14758      /lib/i386-linux-gnu/libgcc_s.so.1
b3d98000-b3d99000 ---p 00000000 00:00 0
b3d99000-b50bd000 rwxp 00000000 00:00 0
b50bd000-b50dc000 r-xp 00000000 08:04 25356      /usr/lib/libjpeg.so.62.0.0
b50dc000-b50dd000 rwxp 0001e000 08:04 25356      /usr/lib/libjpeg.so.62.0.0
b50f0000-b50f4000 r-xp 00000000 00:14 1386718    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-jpeg.so                                                                                                                                                                                           
b50f4000-b50f5000 rwxp 00004000 00:14 1386718    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-jpeg.so                                                                                                                                                                                           
b50f5000-b51a5000 r-xp 00000000 08:04 63683      /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf
b51a5000-b51a7000 r-xp 00000000 08:04 32692      /usr/lib/pango/1.6.0/modules/pango-basic-fc.so
b51a7000-b51a8000 rwxp 00001000 08:04 32692      /usr/lib/pango/1.6.0/modules/pango-basic-fc.so
b51a8000-b51a9000 r-xs 00000000 08:04 261540    /var/cache/fontconfig/c05880de57d1f5e948fdfacc138775d9-le32d4.cache-3
b51a9000-b51af000 r-xs 00000000 08:04 271570    /var/cache/fontconfig/945677eb7aeaf62f1d50efc3fb3ec7d8-le32d4.cache-3
b51af000-b51b1000 r-xs 00000000 08:04 261521    /var/cache/fontconfig/ea47318ec9849e1a71e80a5d69d13859-le32d4.cache-3
b51b1000-b51b2000 r-xs 00000000 08:04 261402    /var/cache/fontconfig/e3fa16a14183b06aa45b3e009278fd14-le32d4.cache-3
b51b2000-b51b4000 r-xs 00000000 08:04 261398    /var/cache/fontconfig/b5ea634b0fb353b8ea17632d1f9ef766-le32d4.cache-3
b51b4000-b51b8000 r-xs 00000000 08:04 261360    /var/cache/fontconfig/6eb3985aa4124903f6ff08ba781cd364-le32d4.cache-3
b51b8000-b51bc000 r-xs 00000000 08:04 261351    /var/cache/fontconfig/515ca1ebc4b18308bea979be5704f9db-le32d4.cache-3
b51bc000-b51c3000 r-xs 00000000 08:04 271552    /var/cache/fontconfig/6d41288fd70b0be22e8c3a91e032eec0-le32d4.cache-3
b51c3000-b51df000 r-xs 00000000 08:04 32186      /usr/share/mime/mime.cache
b51df000-b51e0000 ---p 00000000 00:00 0
b51e0000-b59e0000 rwxp 00000000 00:00 0
b59e0000-b59f9000 r-xp 00000000 08:04 19598      /usr/lib/libmagic.so.1.0.0
b59f9000-b59fa000 rwxp 00018000 08:04 19598      /usr/lib/libmagic.so.1.0.0
b59fa000-b59fc000 rwxp 00000000 00:00 0
b59fc000-b59fe000 r-xs 00000000 08:04 261325    /var/cache/fontconfig/ddd4086aec35a5275babba44bb759c3c-le32d4.cache-3
b59fe000-b5a0b000 r-xs 00000000 08:04 261341    /var/cache/fontconfig/d52a8644073d54c13679302ca1180695-le32d4.cache-3
b5a0b000-b5a0e000 r-xp 00000000 08:04 19969      /usr/lib/python2.6/lib-dynload/_json.so
b5a0e000-b5a0f000 rwxp 00002000 08:04 19969      /usr/lib/python2.6/lib-dynload/_json.so
b5a0f000-b5a26000 r-xp 00000000 08:04 19939      /usr/lib/python2.6/lib-dynload/_ctypes.so
b5a26000-b5a29000 rwxp 00016000 08:04 19939      /usr/lib/python2.6/lib-dynload/_ctypes.so
b5a29000-b5a2a000 ---p 00000000 00:00 0
b5a2a000-b622a000 rwxp 00000000 00:00 0
b622a000-b62b5000 r-xp 00000000 08:04 17124      /usr/lib/libsqlite3.so.0.8.6
b62b5000-b62b8000 rwxp 0008a000 08:04 17124      /usr/lib/libsqlite3.so.0.8.6
b62b9000-b62bf000 r-xs 00000000 08:04 271525    /var/cache/fontconfig/e13b20fdb08344e0e664864cc2ede53d-le32d4.cache-3
b62bf000-b62c1000 r-xs 00000000 08:04 261345    /var/cache/fontconfig/7ef2298fde41cc6eeb7af42e48b7d293-le32d4.cache-3
b62c1000-b62c5000 r-xp 00000000 00:14 1386700    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-png.so                                                                                                                                                                                           
b62c5000-b62c6000 rwxp 00003000 00:14 1386700    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-png.soAborted


白い熊 2011-07-30 19:58

Re: [Announce] modRana: a flexible GPS navigation system
 
Well, the crashes seem to be arbitrary... After a series of crashes at roughly 10 thou, I've had a download running now, which is 50 thou and going...

kureyon 2011-07-31 05:14

Re: [Announce] modRana: a flexible GPS navigation system
 
1) Track Logs: Any particular reason why they're stored under /opt? Also their file permissions are owned by root.

May I suggest that all user "generated" data and maybe config files are stored in the home directory or in MyDocs.

2) Another thing is modRana doesn't seem to reload the tracklogs directory. Eg I export a track from GPSJinni and copy it to the tracklogs directory whilst modRana is running. I need to restart modRana before it sees the copied file.

Thanks!

白い熊 2011-07-31 09:41

Re: [Announce] modRana: a flexible GPS navigation system
 
This downloading is a serious problem. I'm trying to download a 40km area just at the lowest level of zoom that OSM will go to, that's around 350k tiles, but it keeps crashing at 15k, which I already have.

How to get around it?

Tried switching it to just save the tiles, so that I'd later import them into the sql db, but the same thing, just bombs, WTF?

MartinK 2011-07-31 13:07

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by kureyon (Post 1061448)
1) Track Logs: Any particular reason why they're stored under /opt? Also their file permissions are owned by root.

This was for legacy reasons - Rana, the modRana predecessor was developed for the Neo FreeRunner, where everything run under root. :D So it didn't matter where they were stored.

Quote:

Originally Posted by kureyon (Post 1061448)
May I suggest that all user "generated" data and maybe config files are stored in the home directory or in MyDocs.

Well, I've just sent V0.27-2 to the autobuilder, that does just this :)
check out the updated data storage article for details about the new paths.
In short:
  • map data - nothing changes, still MyDocs/.maps
  • POI- nothing changes, still MyDocs/.maps
  • tracklogs are now stored in MyDocs/tracklogs, old tracklogs should be migrated there automatically
  • configuration files are now in ~/.modrana

Quote:

Originally Posted by kureyon (Post 1061448)
2) Another thing is modRana doesn't seem to reload the tracklogs directory. Eg I export a track from GPSJinni and copy it to the tracklogs directory whilst modRana is running. I need to restart modRana before it sees the copied file.
Thanks!

OK, I'll change it to reload the directory once it is listed from the GUI.

Quote:

Originally Posted by 白い熊 (Post 1061186)
Same problem with Japanese POIs.

I'm looking at this right now. :)

MartinK 2011-07-31 13:56

Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:

Originally Posted by beermad (Post 1058965)
Hi Martin,

From looking through the database code and structure to produce the migration script, I think I've found a few points where the code and schema could be more efficient.
  • When you're retrieving tiles, you're pulling the epoch_timestamp from the databases. Except when you're checking if you need (or want) to download an updated tile, this is unnecessary, so just wastes CPU cycles.

Good catch. :)
Quote:

Originally Posted by beermad (Post 1058965)
  • After retrieving details from the lookup table, it seems inefficient to find the right tile in the store table using x,y,z when you've already done that on the lookup table. It would be more efficient to have a single primary index in the store table, referenced in the lookup table entry.

Well, I'm not really a database architect - I basically just thought quite a long time about how to make an universal schema and asked a friend who works with databases a bit. :) Well, like this, the lookup and store databases are independent - so when the lookup one is corrupted, it should be possible to regenerate it just form the stores.
Quote:

Originally Posted by beermad (Post 1058965)
  • Moving on logically from the previous idea, it might be possible to make the lookup table more efficient. At the moment you need to work through three indexes (x,y,z) to find the right tile. It might be more processor efficient to have a single index on this table which is either a combination of the x,y,z parameters or a hash based on them (neither option is simple, but would probably speed things up a worthwhile amount).

Yeah - say we have x=1 y=2 and z=17 -> 1217, also x=1,y=21,z=7 -> 1217 ...
A hash with separators might work though: 1,2,17 vs 1,21,7 - would something like this be usable ?

Also, would it be possible to maintain backward compatibility by adding this new indexes and still storing the old info ? Converting all the existing database files users might already have would be quite a headache and also some developers might be already working on supporting the format in its current form (IIRC the CloudGPS developer, maybe also some others).

There is a version filed in the schema, so it would be possible to do something like this:
  • 1 = current version
  • 2 = old info + new indexes
  • 3 = just new indexes


Quote:

Originally Posted by 白い熊 (Post 1061533)
This downloading is a serious problem. I'm trying to download a 40km area just at the lowest level of zoom that OSM will go to, that's around 350k tiles, but it keeps crashing at 15k, which I already have.

I remember getting this some time ago. I'll run a few large downloads to check if I can still reproduce it.
Still, this seems more like a bug in Glib that is being triggered by modRana. There is even a post mentioning a similar behaviour (glib and long lists).

Quote:

Originally Posted by 白い熊 (Post 1061533)
How to get around it?

First, try the new version that was just released - some unrelated changes might have fixed it. :)
If this does not help, you can try some other batch download software, like Gmapcatcher and then importing the tiles with the SQLite import script.

Quote:

Originally Posted by 白い熊 (Post 1061533)
Tried switching it to just save the tiles, so that I'd later import them into the sql db, but the same thing, just bombs, WTF?

Interesting - looks like it is independent from the storage method. So the long lists + threaded access to them might really be the issue.

lsolano 2011-07-31 15:16

Re: [Announce] modRana: a flexible GPS navigation system
 
I think it would be better to store everything under MyDocs/Modrana.

So, we'd have:

MyDocs/Modrana/tracklogs
MyDocs/Modrana/pois
MyDocs/Modrana/config (not sure about this one, maybe it is better ~/.modrana)

Of course, maps should be kept at MyDocs/.maps to share it with other applications.


All times are GMT. The time now is 22:57.

vBulletin® Version 3.8.8