maemo.org - Talk
Page 80 of 207 « First 707879 80 818290 Last »
Show 40 post(s) from this thread on one page

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Applications (https://talk.maemo.org/forumdisplay.php?f=41)
-   -   [Announce] modRana: a flexible GPS navigation system (https://talk.maemo.org/showthread.php?t=58861)

beermad 2011-07-26 23:57
Re: [Announce] modRana: a flexible GPS navigation system
 
Hi Martin,

From looking through the database code and structure to produce the migration script, I think I've found a few points where the code and schema could be more efficient.
  • When you're retrieving tiles, you're pulling the epoch_timestamp from the databases. Except when you're checking if you need (or want) to download an updated tile, this is unnecessary, so just wastes CPU cycles.
  • After retrieving details from the lookup table, it seems inefficient to find the right tile in the store table using x,y,z when you've already done that on the lookup table. It would be more efficient to have a single primary index in the store table, referenced in the lookup table entry.
  • Moving on logically from the previous idea, it might be possible to make the lookup table more efficient. At the moment you need to work through three indexes (x,y,z) to find the right tile. It might be more processor efficient to have a single index on this table which is either a combination of the x,y,z parameters or a hash based on them (neither option is simple, but would probably speed things up a worthwhile amount).

白い熊 2011-07-30 16:13
Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:
Originally Posted by MartinK (Post 1058766)
ModRana now overlays the two tiles only once and then caches the result & displays the result. Before the tiles are combined, only the loading tile is shown.

Like this, when tiles from one layer are missing, nothing is shown. :)
This can be IMO improved:
  • just showing the "good" layer
  • overlaying with the error tile to inform the user that one layer is broken
I'd vote for the 2nd option, that way you know one layer is missing...

白い熊 2011-07-30 16:13
Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:
Originally Posted by MartinK (Post 1056939)
Thanks for reporting! I'll check it out - looks like a Unicode conversion error to me.
Same problem with Japanese POIs.

白い熊 2011-07-30 19:10
Re: [Announce] modRana: a flexible GPS navigation system
 
When downloading a large area, modRana segfaults. Thought it was only on the N900, but the PC version exhibits the same behavior.

When trying to download a 40km area, I get an estimated download of 90 thousand tiles, modRana segfaults after cca 10 thousand tiles with:

Code:
Batch tile dl working... (threads: 8) pending: 46496, done: 43730
Batch tile dl working... (threads: 8) pending: 46489, done: 43737
Batch tile dl working... (threads: 8) pending: 46482, done: 43744
*** glibc detected *** python: corrupted double-linked list: 0x0bbafbe8 ***
======= Backtrace: =========
/lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x6aac1)[0xb75acac1]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x6c3ec)[0xb75ae3ec]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xb75b13dd]
/usr/lib/libsqlite3.so.0(+0x15c80)[0xb623fc80]
/usr/lib/libsqlite3.so.0(sqlite3_free+0x81)[0xb6230a51]
/usr/lib/libsqlite3.so.0(+0x4c1c2)[0xb62761c2]
/usr/lib/libsqlite3.so.0(sqlite3_close+0x11f)[0xb627635f]
/usr/lib/python2.6/lib-dynload/_sqlite3.so(pysqlite_connection_close+0xfd)[0xb62d315d]
python(PyEval_EvalFrameEx+0x44d0)[0x80e08c0]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python(PyEval_EvalFrameEx+0x476f)[0x80e0b5f]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python(PyEval_EvalFrameEx+0x476f)[0x80e0b5f]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python[0x8174d87]
python(PyObject_Call+0x4a)[0x8066d7a]
python(PyEval_EvalFrameEx+0x3133)[0x80df523]
python(PyEval_EvalFrameEx+0x5437)[0x80e1827]
python(PyEval_EvalFrameEx+0x5437)[0x80e1827]
python(PyEval_EvalCodeEx+0x848)[0x80e2968]
python[0x8174c9c]
python(PyObject_Call+0x4a)[0x8066d7a]
python[0x806f931]
python(PyObject_Call+0x4a)[0x8066d7a]
python(PyEval_CallObjectWithKeywords+0x42)[0x80dc2f2]
python[0x8113faf]
/lib/i386-linux-gnu/i686/cmov/libpthread.so.0(+0x5c39)[0xb78a7c39]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(clone+0x5e)[0xb760e93e]
======= Memory map: ========
08048000-08235000 r-xp 00000000 08:04 20017      /usr/bin/python2.6
08235000-08284000 rwxp 001ed000 08:04 20017      /usr/bin/python2.6
08284000-0828d000 rwxp 00000000 00:00 0
08b3c000-0bbe4000 rwxp 00000000 00:00 0          [heap]
af4fe000-af4ff000 ---p 00000000 00:00 0
af4ff000-afcff000 rwxp 00000000 00:00 0
afcff000-afd00000 ---p 00000000 00:00 0
afd00000-b0500000 rwxp 00000000 00:00 0
b0500000-b05ff000 rwxp 00000000 00:00 0
b05ff000-b0600000 ---p 00000000 00:00 0
b0b22000-b0d23000 rwxp 00000000 00:00 0
b0e24000-b0e25000 ---p 00000000 00:00 0
b0e25000-b1bfe000 rwxp 00000000 00:00 0
b1cb1000-b23ff000 rwxp 00000000 00:00 0
b23ff000-b2400000 ---p 00000000 00:00 0
b2400000-b2c00000 rwxp 00000000 00:00 0
b2c00000-b2d00000 rwxp 00000000 00:00 0
b2d7d000-b2d7e000 ---p 00000000 00:00 0
b2d7e000-b357e000 rwxp 00000000 00:00 0
b357e000-b3582000 r-xp 00000000 08:04 16174      /lib/i386-linux-gnu/i686/cmov/libnss_dns-2.13.so
b3582000-b3583000 r-xp 00004000 08:04 16174      /lib/i386-linux-gnu/i686/cmov/libnss_dns-2.13.so
b3583000-b3584000 rwxp 00005000 08:04 16174      /lib/i386-linux-gnu/i686/cmov/libnss_dns-2.13.so
b35b1000-b366c000 rwxp 00000000 00:00 0
b366c000-b382b000 rwxp 00000000 08:04 19603      /usr/share/file/magic.mgc
b382b000-b38e6000 rwxp 00000000 00:00 0
b38e6000-b397c000 r-xp 00000000 08:04 65546      /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Oblique.ttf
b397c000-b3a21000 r-xp 00000000 08:04 63682      /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-Bold.ttf
b3a21000-b3b97000 rwxp 00000000 00:00 0
b3c00000-b3cb4000 rwxp 00000000 00:00 0
b3cb4000-b3d00000 ---p 00000000 00:00 0
b3d7b000-b3d97000 r-xp 00000000 08:04 14758      /lib/i386-linux-gnu/libgcc_s.so.1
b3d97000-b3d98000 rwxp 0001b000 08:04 14758      /lib/i386-linux-gnu/libgcc_s.so.1
b3d98000-b3d99000 ---p 00000000 00:00 0
b3d99000-b50bd000 rwxp 00000000 00:00 0
b50bd000-b50dc000 r-xp 00000000 08:04 25356      /usr/lib/libjpeg.so.62.0.0
b50dc000-b50dd000 rwxp 0001e000 08:04 25356      /usr/lib/libjpeg.so.62.0.0
b50f0000-b50f4000 r-xp 00000000 00:14 1386718    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-jpeg.so                                                                                                                                                                                           
b50f4000-b50f5000 rwxp 00004000 00:14 1386718    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-jpeg.so                                                                                                                                                                                           
b50f5000-b51a5000 r-xp 00000000 08:04 63683      /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf
b51a5000-b51a7000 r-xp 00000000 08:04 32692      /usr/lib/pango/1.6.0/modules/pango-basic-fc.so
b51a7000-b51a8000 rwxp 00001000 08:04 32692      /usr/lib/pango/1.6.0/modules/pango-basic-fc.so
b51a8000-b51a9000 r-xs 00000000 08:04 261540    /var/cache/fontconfig/c05880de57d1f5e948fdfacc138775d9-le32d4.cache-3
b51a9000-b51af000 r-xs 00000000 08:04 271570    /var/cache/fontconfig/945677eb7aeaf62f1d50efc3fb3ec7d8-le32d4.cache-3
b51af000-b51b1000 r-xs 00000000 08:04 261521    /var/cache/fontconfig/ea47318ec9849e1a71e80a5d69d13859-le32d4.cache-3
b51b1000-b51b2000 r-xs 00000000 08:04 261402    /var/cache/fontconfig/e3fa16a14183b06aa45b3e009278fd14-le32d4.cache-3
b51b2000-b51b4000 r-xs 00000000 08:04 261398    /var/cache/fontconfig/b5ea634b0fb353b8ea17632d1f9ef766-le32d4.cache-3
b51b4000-b51b8000 r-xs 00000000 08:04 261360    /var/cache/fontconfig/6eb3985aa4124903f6ff08ba781cd364-le32d4.cache-3
b51b8000-b51bc000 r-xs 00000000 08:04 261351    /var/cache/fontconfig/515ca1ebc4b18308bea979be5704f9db-le32d4.cache-3
b51bc000-b51c3000 r-xs 00000000 08:04 271552    /var/cache/fontconfig/6d41288fd70b0be22e8c3a91e032eec0-le32d4.cache-3
b51c3000-b51df000 r-xs 00000000 08:04 32186      /usr/share/mime/mime.cache
b51df000-b51e0000 ---p 00000000 00:00 0
b51e0000-b59e0000 rwxp 00000000 00:00 0
b59e0000-b59f9000 r-xp 00000000 08:04 19598      /usr/lib/libmagic.so.1.0.0
b59f9000-b59fa000 rwxp 00018000 08:04 19598      /usr/lib/libmagic.so.1.0.0
b59fa000-b59fc000 rwxp 00000000 00:00 0
b59fc000-b59fe000 r-xs 00000000 08:04 261325    /var/cache/fontconfig/ddd4086aec35a5275babba44bb759c3c-le32d4.cache-3
b59fe000-b5a0b000 r-xs 00000000 08:04 261341    /var/cache/fontconfig/d52a8644073d54c13679302ca1180695-le32d4.cache-3
b5a0b000-b5a0e000 r-xp 00000000 08:04 19969      /usr/lib/python2.6/lib-dynload/_json.so
b5a0e000-b5a0f000 rwxp 00002000 08:04 19969      /usr/lib/python2.6/lib-dynload/_json.so
b5a0f000-b5a26000 r-xp 00000000 08:04 19939      /usr/lib/python2.6/lib-dynload/_ctypes.so
b5a26000-b5a29000 rwxp 00016000 08:04 19939      /usr/lib/python2.6/lib-dynload/_ctypes.so
b5a29000-b5a2a000 ---p 00000000 00:00 0
b5a2a000-b622a000 rwxp 00000000 00:00 0
b622a000-b62b5000 r-xp 00000000 08:04 17124      /usr/lib/libsqlite3.so.0.8.6
b62b5000-b62b8000 rwxp 0008a000 08:04 17124      /usr/lib/libsqlite3.so.0.8.6
b62b9000-b62bf000 r-xs 00000000 08:04 271525    /var/cache/fontconfig/e13b20fdb08344e0e664864cc2ede53d-le32d4.cache-3
b62bf000-b62c1000 r-xs 00000000 08:04 261345    /var/cache/fontconfig/7ef2298fde41cc6eeb7af42e48b7d293-le32d4.cache-3
b62c1000-b62c5000 r-xp 00000000 00:14 1386700    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-png.so                                                                                                                                                                                           
b62c5000-b62c6000 rwxp 00003000 00:14 1386700    /media/0/kumatux.org-apps/gdk-pixbuf_2.23.3_jj_2011-04-27-002509_debian-unstable_2.6.38-1-686_gcc-4.6.0_i686_macbook-pro-5.2/lib/gdk-pixbuf-2.0/2.10.0/loaders/li
bpixbufloader-png.soAborted

白い熊 2011-07-30 19:58
Re: [Announce] modRana: a flexible GPS navigation system
 
Well, the crashes seem to be arbitrary... After a series of crashes at roughly 10 thou, I've had a download running now, which is 50 thou and going...

kureyon 2011-07-31 05:14
Re: [Announce] modRana: a flexible GPS navigation system
 
1) Track Logs: Any particular reason why they're stored under /opt? Also their file permissions are owned by root.

May I suggest that all user "generated" data and maybe config files are stored in the home directory or in MyDocs.

2) Another thing is modRana doesn't seem to reload the tracklogs directory. Eg I export a track from GPSJinni and copy it to the tracklogs directory whilst modRana is running. I need to restart modRana before it sees the copied file.

Thanks!

白い熊 2011-07-31 09:41
Re: [Announce] modRana: a flexible GPS navigation system
 
This downloading is a serious problem. I'm trying to download a 40km area just at the lowest level of zoom that OSM will go to, that's around 350k tiles, but it keeps crashing at 15k, which I already have.

How to get around it?

Tried switching it to just save the tiles, so that I'd later import them into the sql db, but the same thing, just bombs, WTF?

MartinK 2011-07-31 13:07
Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:
Originally Posted by kureyon (Post 1061448)
1) Track Logs: Any particular reason why they're stored under /opt? Also their file permissions are owned by root.
This was for legacy reasons - Rana, the modRana predecessor was developed for the Neo FreeRunner, where everything run under root. :D So it didn't matter where they were stored.

Quote:
Originally Posted by kureyon (Post 1061448)
May I suggest that all user "generated" data and maybe config files are stored in the home directory or in MyDocs.
Well, I've just sent V0.27-2 to the autobuilder, that does just this :)
check out the updated data storage article for details about the new paths.
In short:
  • map data - nothing changes, still MyDocs/.maps
  • POI- nothing changes, still MyDocs/.maps
  • tracklogs are now stored in MyDocs/tracklogs, old tracklogs should be migrated there automatically
  • configuration files are now in ~/.modrana

Quote:
Originally Posted by kureyon (Post 1061448)
2) Another thing is modRana doesn't seem to reload the tracklogs directory. Eg I export a track from GPSJinni and copy it to the tracklogs directory whilst modRana is running. I need to restart modRana before it sees the copied file.
Thanks!
OK, I'll change it to reload the directory once it is listed from the GUI.

Quote:
Originally Posted by 白い熊 (Post 1061186)
Same problem with Japanese POIs.
I'm looking at this right now. :)

MartinK 2011-07-31 13:56
Re: [Announce] modRana: a flexible GPS navigation system
 
Quote:
Originally Posted by beermad (Post 1058965)
Hi Martin,

From looking through the database code and structure to produce the migration script, I think I've found a few points where the code and schema could be more efficient.
  • When you're retrieving tiles, you're pulling the epoch_timestamp from the databases. Except when you're checking if you need (or want) to download an updated tile, this is unnecessary, so just wastes CPU cycles.
Good catch. :)
Quote:
Originally Posted by beermad (Post 1058965)
  • After retrieving details from the lookup table, it seems inefficient to find the right tile in the store table using x,y,z when you've already done that on the lookup table. It would be more efficient to have a single primary index in the store table, referenced in the lookup table entry.
Well, I'm not really a database architect - I basically just thought quite a long time about how to make an universal schema and asked a friend who works with databases a bit. :) Well, like this, the lookup and store databases are independent - so when the lookup one is corrupted, it should be possible to regenerate it just form the stores.
Quote:
Originally Posted by beermad (Post 1058965)
  • Moving on logically from the previous idea, it might be possible to make the lookup table more efficient. At the moment you need to work through three indexes (x,y,z) to find the right tile. It might be more processor efficient to have a single index on this table which is either a combination of the x,y,z parameters or a hash based on them (neither option is simple, but would probably speed things up a worthwhile amount).
Yeah - say we have x=1 y=2 and z=17 -> 1217, also x=1,y=21,z=7 -> 1217 ...
A hash with separators might work though: 1,2,17 vs 1,21,7 - would something like this be usable ?

Also, would it be possible to maintain backward compatibility by adding this new indexes and still storing the old info ? Converting all the existing database files users might already have would be quite a headache and also some developers might be already working on supporting the format in its current form (IIRC the CloudGPS developer, maybe also some others).

There is a version filed in the schema, so it would be possible to do something like this:
  • 1 = current version
  • 2 = old info + new indexes
  • 3 = just new indexes


Quote:
Originally Posted by 白い熊 (Post 1061533)
This downloading is a serious problem. I'm trying to download a 40km area just at the lowest level of zoom that OSM will go to, that's around 350k tiles, but it keeps crashing at 15k, which I already have.
I remember getting this some time ago. I'll run a few large downloads to check if I can still reproduce it.
Still, this seems more like a bug in Glib that is being triggered by modRana. There is even a post mentioning a similar behaviour (glib and long lists).

Quote:
Originally Posted by 白い熊 (Post 1061533)
How to get around it?
First, try the new version that was just released - some unrelated changes might have fixed it. :)
If this does not help, you can try some other batch download software, like Gmapcatcher and then importing the tiles with the SQLite import script.

Quote:
Originally Posted by 白い熊 (Post 1061533)
Tried switching it to just save the tiles, so that I'd later import them into the sql db, but the same thing, just bombs, WTF?
Interesting - looks like it is independent from the storage method. So the long lists + threaded access to them might really be the issue.

lsolano 2011-07-31 15:16
Re: [Announce] modRana: a flexible GPS navigation system
 
I think it would be better to store everything under MyDocs/Modrana.

So, we'd have:

MyDocs/Modrana/tracklogs
MyDocs/Modrana/pois
MyDocs/Modrana/config (not sure about this one, maybe it is better ~/.modrana)

Of course, maps should be kept at MyDocs/.maps to share it with other applications.


All times are GMT. The time now is 12:09.
Page 80 of 207 « First 707879 80 818290 Last »
Show 40 post(s) from this thread on one page

vBulletin® Version 3.8.8