Active Topics
-
Camera phone competition March 2023 "Endless" (35)
to General by Maemish - 18 hrs, 37 mins ago -
TLS 1.2 for N9 is ready (28)
to MeeGo / Harmattan by smartblu9 - 19 hrs, 51 mins ago -
Maemo repository (3)
to Community by torsen - 1 day, 11 hrs ago -
Error after installing Leste on sdcard (11)
to Maemo 7 / Leste by Maemish - 2 days, 8 hrs ago -
Introducing ubiboot N9 (multiboot OS loader) (1,393)
to Alternatives by smatkovi - 3 days, 15 hrs ago - more...
|
2011-06-06
, 18:47
|
|
Posts: 82 |
Thanked: 214 times |
Joined on Jan 2010
@ Cape town
|
#32
|
I've been awfully quiet lately - mostly due to exams - so here's an update.
IMAP support is almost nearly complete. I just need to finish a few things with regards to efficient flag handling, but otherwise it's quite solid. PUSH and all of that is there too
Behind the scenes, quite a few things have changed, but those are boring and I won't go into them.
The visible changes (what's new) are section headers (messages grouped by day) and context menus for sections, threads and messages. So if you hold tap on a date header, you'll have options like "Mark all messages below as read". For a thread, something like "Mark all messages as read" "Reply to thread" and for a single message, the usual "Reply" "Forward" etc.
The only thing stopping me from releasing an Alpha deb at this moment is what seems to be a lack of threading when actually run on the N900. It appears as if one of the threads is blocking the whole application... Doesn't happen on the desktop, so I need to debug that first.
Otherwise, here is a video of the latest
http://www.youtube.com/watch?v=tIySh8Wtu8U
Apologizes for the quality - you'd think a Core i7 would be enough to run at full speed with recording... Apparently not. The reason some areas are white, is that those are by default set by the theme - which on the N900 will be more suitable - and we avoid doing repaints whenever possible
Last edited by cb22; 2011-06-06 at 18:49.
IMAP support is almost nearly complete. I just need to finish a few things with regards to efficient flag handling, but otherwise it's quite solid. PUSH and all of that is there too
Behind the scenes, quite a few things have changed, but those are boring and I won't go into them.
The visible changes (what's new) are section headers (messages grouped by day) and context menus for sections, threads and messages. So if you hold tap on a date header, you'll have options like "Mark all messages below as read". For a thread, something like "Mark all messages as read" "Reply to thread" and for a single message, the usual "Reply" "Forward" etc.
The only thing stopping me from releasing an Alpha deb at this moment is what seems to be a lack of threading when actually run on the N900. It appears as if one of the threads is blocking the whole application... Doesn't happen on the desktop, so I need to debug that first.
Otherwise, here is a video of the latest
http://www.youtube.com/watch?v=tIySh8Wtu8U
Apologizes for the quality - you'd think a Core i7 would be enough to run at full speed with recording... Apparently not. The reason some areas are white, is that those are by default set by the theme - which on the N900 will be more suitable - and we avoid doing repaints whenever possible
Last edited by cb22; 2011-06-06 at 18:49.
| The Following 8 Users Say Thank You to cb22 For This Useful Post: | ||
 |
|
2011-06-07
, 10:04
|
|
Posts: 92 |
Thanked: 92 times |
Joined on May 2011
@ Stellenbosch, South Africa
|
#33
|
Hi,
Sounds like you're making some good progress, despite the scourge that is exams
What I understand with regards to threading (bear in mind this is coming off a Win32/C# background) is that the following happens: for each core in the device a single (not necessarily your application's thread) can be run at any one time. i.e. if you have a dual core processor then 2 threads can be run concurrently without blocking one another. What the OS/Kernel does is simply blocks threads as required to give the illusion that they are all running simultaneously.
So, on the N900 (with it's little single core processor) you can run at most 1 thread at any one time. The best way to allow a "multithreading" system to work then is to make sure that the operating system prioritises your UI threads and other such things above your worker threads, this will lead to your worker threads being blocked more often and a bit of a general slowdown in terms of processing speed BUT your UI should remain perfectly usable while that is happening. Another alternative, without dropping the priority of your worker threads, is to make use of Sleep() function calls within them whenever they are likely to have nothing to do for a bit. Those Sleep() calls will be intercepted by the kernel and it will block that thread's execution for as long as it needs to in order to allow other threads to get some work done.
Just another thing that you may not expect when coding a multithreaded app is that the user is likely to go "usable UI = can do stuff", which from a coding perspective isn't always the case. You may want to make sure that your app can handle multiple different operations at the same time, say the user opening one email, then deciding that actually that wasn't the one they wanted and clicking on a different one (while your background thread would be loading the email they clicked on).
Overall though, good luck and I really look forward to being able to test it out, it certainly sounds impressive
Best Regards,
SPARTAN563
Sounds like you're making some good progress, despite the scourge that is exams
What I understand with regards to threading (bear in mind this is coming off a Win32/C# background) is that the following happens: for each core in the device a single (not necessarily your application's thread) can be run at any one time. i.e. if you have a dual core processor then 2 threads can be run concurrently without blocking one another. What the OS/Kernel does is simply blocks threads as required to give the illusion that they are all running simultaneously.
So, on the N900 (with it's little single core processor) you can run at most 1 thread at any one time. The best way to allow a "multithreading" system to work then is to make sure that the operating system prioritises your UI threads and other such things above your worker threads, this will lead to your worker threads being blocked more often and a bit of a general slowdown in terms of processing speed BUT your UI should remain perfectly usable while that is happening. Another alternative, without dropping the priority of your worker threads, is to make use of Sleep() function calls within them whenever they are likely to have nothing to do for a bit. Those Sleep() calls will be intercepted by the kernel and it will block that thread's execution for as long as it needs to in order to allow other threads to get some work done.
Just another thing that you may not expect when coding a multithreaded app is that the user is likely to go "usable UI = can do stuff", which from a coding perspective isn't always the case. You may want to make sure that your app can handle multiple different operations at the same time, say the user opening one email, then deciding that actually that wasn't the one they wanted and clicking on a different one (while your background thread would be loading the email they clicked on).
Overall though, good luck and I really look forward to being able to test it out, it certainly sounds impressive
Best Regards,
SPARTAN563
|
|
2011-06-07
, 17:23
|
|
Posts: 82 |
Thanked: 214 times |
Joined on Jan 2010
@ Cape town
|
#34
|
Originally Posted by SPARTAN563
Hey SPARTAN563 - I take it you're from Stellenbosch, South Africa? Well - they aren't really many other Stellenbosch's around... Greetings from Cape Town in that case
Hi,
Sounds like you're making some good progress, despite the scourge that is exams
What I understand with regards to threading (bear in mind this is coming off a Win32/C# background) is that the following happens: for each core in the device a single (not necessarily your application's thread) can be run at any one time. i.e. if you have a dual core processor then 2 threads can be run concurrently without blocking one another. What the OS/Kernel does is simply blocks threads as required to give the illusion that they are all running simultaneously.
So, on the N900 (with it's little single core processor) you can run at most 1 thread at any one time. The best way to allow a "multithreading" system to work then is to make sure that the operating system prioritises your UI threads and other such things above your worker threads, this will lead to your worker threads being blocked more often and a bit of a general slowdown in terms of processing speed BUT your UI should remain perfectly usable while that is happening. Another alternative, without dropping the priority of your worker threads, is to make use of Sleep() function calls within them whenever they are likely to have nothing to do for a bit. Those Sleep() calls will be intercepted by the kernel and it will block that thread's execution for as long as it needs to in order to allow other threads to get some work done.
Just another thing that you may not expect when coding a multithreaded app is that the user is likely to go "usable UI = can do stuff", which from a coding perspective isn't always the case. You may want to make sure that your app can handle multiple different operations at the same time, say the user opening one email, then deciding that actually that wasn't the one they wanted and clicking on a different one (while your background thread would be loading the email they clicked on).
Overall though, good luck and I really look forward to being able to test it out, it certainly sounds impressive
Best Regards,
SPARTAN563
Indeed - and that's what makes this case so peculiar. My architecture is split between a backend (that handles retrieving mails and updating a database) and the frontend (that is the GUI). The GUI isn't very threaded - it's essentially not needed. The "expensive" operations that I have complete so quickly (and should scale nicely too), that adding in the complexity of threads, callbacks and the like was just not necessary in most cases. The only thing in the GUI which is threaded is the search feature. Which works like a charm.
The backend is a different case all together. 99.99% of the time, it is blocking on IO (The N900's connection will generally be too slow to saturate the processor with the work I'm doing) and thus it's highly threaded. Each account gets its own thread to run in, and it receives (and sends) info to the front end via DBUS, which is threaded again.
Now, the DBUS thread essentially does nothing besides sit around and wait for events to happen - it should certainly not be blocking anything by using 100% of the CPU. But it appears to stop my other threads from running. On my desktop, a specific function call was needed, which essentially tells Python (which has the horrible Global Interpreter Lock) to allow other threads to run in the background while the main event loop is chugging along.
If I remove the DBUS bit everything works, but DBUS is a pretty integral component... So I'm going to have to find a way around this... Probably a platform specific oddity...
PS) Don't suppose you're studying - Know anyone up at UCT?
|
|
2011-06-08
, 08:58
|
|
Posts: 92 |
Thanked: 92 times |
Joined on May 2011
@ Stellenbosch, South Africa
|
#35
|
Hi again, yeah I am (well, not from but studying there).
That seems very, very weird. I have noticed that the N900 does tend to block quite badly on IO ops but I didn't have that issue with my app (it was loading records from a CSV file into a database), then again that was all C++/Qt so it actually did what it said on the tin
What version of Python are you using to develop in? IIRC the N900 has 2.5.4 on it which may (or may not) have a bug in its thread handling (I have no idea, just a theory).
That being said, if you are using DBUS to do all the work, then why not run a background process for handling all your work and then another for the actual UI? Would also allow you to have push notifications when the app isn't running. That would also make the "multithreading" side a bit easier since your kernel could then fully take control of that for you and all you would need to worry about is making sure that there isn't any loop to keep that thread active. Speaking of which, are you calling a Sleep() function in your event checker loop? (I assume you are checking through a list to see if there are any events that need to be handled and then handling them, repeating until you need to exit sort of thing?)
If you'd like I would be more than happy to look at the code for you and help you debug it (I am a bit bored of some of my current projects and could do with a change of scenery), would also love to help out testing a beta version (or alpha even, provided it doesn't try to eat my sister... :P)
RE:PS) Studying? What is that :P (First year engineering, so really not all that hard). Yeah, have a friend studying a PPE up there right now, and a cousin who is doing engineering IIRC.
That seems very, very weird. I have noticed that the N900 does tend to block quite badly on IO ops but I didn't have that issue with my app (it was loading records from a CSV file into a database), then again that was all C++/Qt so it actually did what it said on the tin
What version of Python are you using to develop in? IIRC the N900 has 2.5.4 on it which may (or may not) have a bug in its thread handling (I have no idea, just a theory).
That being said, if you are using DBUS to do all the work, then why not run a background process for handling all your work and then another for the actual UI? Would also allow you to have push notifications when the app isn't running. That would also make the "multithreading" side a bit easier since your kernel could then fully take control of that for you and all you would need to worry about is making sure that there isn't any loop to keep that thread active. Speaking of which, are you calling a Sleep() function in your event checker loop? (I assume you are checking through a list to see if there are any events that need to be handled and then handling them, repeating until you need to exit sort of thing?)
If you'd like I would be more than happy to look at the code for you and help you debug it (I am a bit bored of some of my current projects and could do with a change of scenery), would also love to help out testing a beta version (or alpha even, provided it doesn't try to eat my sister... :P)
RE:PS) Studying? What is that :P (First year engineering, so really not all that hard). Yeah, have a friend studying a PPE up there right now, and a cousin who is doing engineering IIRC.
|
|
2011-06-08
, 15:09
|
|
Posts: 82 |
Thanked: 214 times |
Joined on Jan 2010
@ Cape town
|
#36
|
Hah, making me feel old. I'm doing second year engineering up at UCT - electrical and computer to be specific.
That's exactly what I have done - apologizes if it wasn't clear from my explanation. The daemon runs in the background and handles all the work (connecting to servers / etc) and it emits DBUS signals when appropriate / pop up notifications. The GUI, when running, receives these signals and updates its view as needed. It also calls a few DBUS methods every now and again, like if a user requests a sync and such.
As for the event loop, it's a standard gobject main loop, DBUS requires one. My actual work gets handled by the imaplib2 library, which is highly threaded in itself.
The code is up at https://github.com/cb22/immoral - feel free to take a look. Some parts of it (mostly the GUI bits) I'm not too proud of - but it's my first venture into using QML.
As for eating your sister... Well, it probably won't do that but it might kill any kittens you have floating around... Just kidding - the code doesn't even know how to delete mail off the server yet, which is a nice safety net
I write Signals I tomorrow... A most enjoyable (hear the sarcasm?) course consisting of the basics of Fourier transforms... After that it's party time, then overseas. So I'll probably get around to debugging in 2 - 3 days
EDIT: Also, the commit messages can be a bit, err, ******ed... My main use for github so far had been moving work between my laptop and desktop...
That's exactly what I have done - apologizes if it wasn't clear from my explanation. The daemon runs in the background and handles all the work (connecting to servers / etc) and it emits DBUS signals when appropriate / pop up notifications. The GUI, when running, receives these signals and updates its view as needed. It also calls a few DBUS methods every now and again, like if a user requests a sync and such.
As for the event loop, it's a standard gobject main loop, DBUS requires one. My actual work gets handled by the imaplib2 library, which is highly threaded in itself.
The code is up at https://github.com/cb22/immoral - feel free to take a look. Some parts of it (mostly the GUI bits) I'm not too proud of - but it's my first venture into using QML.
As for eating your sister... Well, it probably won't do that but it might kill any kittens you have floating around... Just kidding - the code doesn't even know how to delete mail off the server yet, which is a nice safety net
I write Signals I tomorrow... A most enjoyable (hear the sarcasm?) course consisting of the basics of Fourier transforms... After that it's party time, then overseas. So I'll probably get around to debugging in 2 - 3 days
EDIT: Also, the commit messages can be a bit, err, ******ed... My main use for github so far had been moving work between my laptop and desktop...
|
|
2011-06-08
, 15:28
|
|
Posts: 92 |
Thanked: 92 times |
Joined on May 2011
@ Stellenbosch, South Africa
|
#37
|
LOL, hardly. I'm doing E&E, but will branch into Computer Systems in 4th year. Unfortunately Stellies dropped their "E&E with Computer Science" course this year in favour of the other.
I see, I had a feeling that was the case but wasn't sure. Great minds think alike or something like that. As for the QML bit, I am really not going to look into that, QML and I are not friends. Even Qt and I have a grudging understanding between eachother. All of that is mainly my fault though, I spent the last 3 years writing up custom Window Forms controls in C# and I absolutely love them, so switching back to standard ones is more than a little painful
Hmm, good thing our res has a no pets policy then...or I'm certain the girls would promptly assassinate me :P
Uh, yeah that doesn't sound awfully fun...good luck with it though, I finished on Monday which was nice.
Anyway, will take a look at the code and see what I can find, with any luck I'll be able to fix something up but I'll keep you informed.
I see, I had a feeling that was the case but wasn't sure. Great minds think alike or something like that
. As for the QML bit, I am really not going to look into that, QML and I are not friends. Even Qt and I have a grudging understanding between eachother. All of that is mainly my fault though, I spent the last 3 years writing up custom Window Forms controls in C# and I absolutely love them, so switching back to standard ones is more than a little painful Hmm, good thing our res has a no pets policy then...or I'm certain the girls would promptly assassinate me :P
Uh, yeah that doesn't sound awfully fun...good luck with it though, I finished on Monday which was nice.
Anyway, will take a look at the code and see what I can find, with any luck I'll be able to fix something up but I'll keep you informed.
|
|
2011-06-12
, 00:05
|
|
Posts: 284 |
Thanked: 74 times |
Joined on Mar 2010
@ Wigan, UK
|
#38
|
Latest video looks awesome cb22.
Really looking forward to the alpha release
Really looking forward to the alpha release
|
|
2011-06-27
, 04:56
|
|
Posts: 82 |
Thanked: 214 times |
Joined on Jan 2010
@ Cape town
|
#39
|
Originally Posted by kaos_king
Thanks
Latest video looks awesome cb22.
Really looking forward to the alpha release
My main problem at the moment is that the damn thing doesn't run properly on the N900. It has everything to do with threads, but as anyone who works with them knows, debugging threads is the last thing you want to be doing.
Oh well, I've been cleaning up the code quite a bit, making everything prettier, and improved the search functionality in the mean time. All of that's just procrastinating however
|
|
2011-06-28
, 18:12
|
|
Posts: 6 |
Thanked: 8 times |
Joined on Oct 2010
|
#40
|
Just curious: How are you debugging? Are you aware that it's possible to remotely debug Python programs?
I'm using the Wing IDE debugger for this. Somewhat slow, but still more comfortable than print statements or similar...
Regards,
Dietmar
I'm using the Wing IDE debugger for this. Somewhat slow, but still more comfortable than print statements or similar...
Regards,
Dietmar
Not mentioning specifics or anything (*cough* Engineering faculty at the University of Cape Town *cough*) seem to get a sadistic pleasure out of having us write 3 tests and hand in 4 assignments in the same week.
/rant
So development has taken a bit of a back seat lately, but hopefully I'll carry on working on it soon enough, I'm pretty close to an alpha release
There are some little cool features too, like pull-to-sync, which I think is quite neat. The composer has a neat design too, I think.
At the moment, I'm busy implementing DBus support, for communication between the GUI and the daemon. Once that's done, I'll package up a deb and post it here.
At the moment only POP3 is supported (I've _just_ started on an IMAP plugin - OfflineIMAP is faaar to complex for my needs) and it currently doesn't have the ability to delete messages off the server. This is intentional, so you can test along without worrying about data loss. It does require a server that supports UIDs, but all modern ones should.