Page 1 of 2 1 2
Reply
Thread Tools
luis is offline luis
2011-02-28 , 15:18
Posts: 146 | Thanked: 15 times | Joined on Oct 2008
#1
Hi folks,

I know, not really the place to post this, but you've been quite helpful before, so.... sorry.

I want to do a task in python in a way that does not block the program. For example, downloading files from the internet.

Of course I could do a separate program and call a subprocess, or do somethink like "os.popen('wget .... &')". But I want to do this inside the main program, in pure python.

Which is the best way to do this? Can you point me to some reading?

Thanks!!

Cheers,
L.
 
juise-'s Avatar
juise- is offline juise-
2011-02-28 , 18:03
Posts: 186 | Thanked: 192 times | Joined on Jan 2010 @ Finland
#2
You use threads for that.

Here's something to get started, and to find more keywords to google for:
http://www.devshed.com/c/a/Python/Ba...g-in-Python/1/

But beware, multithreaded programming can get complicated. So if you just need to get something done, subprocess and wget might not be such a bad idea after all...
__________________
Trout have underwater weapons.
 
epage's Avatar
epage is offline epage
2011-02-28 , 18:13
Posts: 1,684 | Thanked: 1,562 times | Joined on Jun 2008 @ Austin, TX
#3
I generally recommend simplifying thread/lock management.

If you are doing a simple shell script, use the Queue class to retrieve results from the threads.

If you are using gobject or QtCore you deal with callbacks a lot. The problem with that is it spreads the logic of your code out. I've put together some constructs that simplify pushing things off into threads and then getting the results back. If so I can give you some pointers on them.
__________________
770, n810, n900, Ideapad S10-3t
TheOneRing, DialCentral, Gonvert, Quicknote, Multilist, ejpi, nQa, Waters of Shiloah
Programming Blog
 
luis is offline luis
2011-02-28 , 19:24
Posts: 146 | Thanked: 15 times | Joined on Oct 2008
#4
@ epage and juise:

Sorry, I should have been more clear. I don't want anything too complicated. I just want my app not to get blocked while it downloads some specific files from the internet, since those files are not used by the program itself.

Will take a look on threads.

Thanks!

L.
 
epage's Avatar
epage is offline epage
2011-02-28 , 20:10
Posts: 1,684 | Thanked: 1,562 times | Joined on Jun 2008 @ Austin, TX
#5
Originally Posted by luis View Post
@ epage and juise:

Sorry, I should have been more clear. I don't want anything too complicated. I just want my app not to get blocked while it downloads some specific files from the internet, since those files are not used by the program itself.

Will take a look on threads.

Thanks!

L.
You say app, which implies GUI (rather than script). This adds complication because you have to notify the UI thread that data is available. I have code to simplify this for both Qt and GTK. Interested?
__________________
770, n810, n900, Ideapad S10-3t
TheOneRing, DialCentral, Gonvert, Quicknote, Multilist, ejpi, nQa, Waters of Shiloah
Programming Blog
 
luis is offline luis
2011-02-28 , 21:28
Posts: 146 | Thanked: 15 times | Joined on Oct 2008
#6
Originally Posted by epage View Post
You say app, which implies GUI (rather than script). This adds complication because you have to notify the UI thread that data is available. I have code to simplify this for both Qt and GTK. Interested?
Yes, my app has a gtk GUI. I only want a dialog to open when the download is finished, but that the download does not block the program.
If you have a code for something like this, yes, I'm interested.

Thanks!
L.
 
jstokes is offline jstokes
2011-02-28 , 21:33
Posts: 235 | Thanked: 339 times | Joined on Nov 2010
#7
Originally Posted by luis View Post
Yes, my app has a gtk GUI
http://www.pygtk.org/pygtk2reference...t--spawn-async
 
v13 is offline v13
2011-02-28 , 21:34
Posts: 376 | Thanked: 511 times | Joined on Aug 2009 @ Greece
#8
Originally Posted by luis View Post
Hi folks,

I know, not really the place to post this, but you've been quite helpful before, so.... sorry.

I want to do a task in python in a way that does not block the program. For example, downloading files from the internet.
It depends on your program:

If you have a cpu-intensive program (let's say a password cracker - that runs all the time) that needs to communicate with the network you may need separate processes (but for the cpu-intensive part).

For most other cases you do this using select/poll or an API that is based on that (like asyncore or twisted).

On the other hand, if you're writing a GUI program you'll have to look at the toolkit's libraries for something that properly interacts with its main loop. Qt for example has a networking module that will fit your needs.
 
epage's Avatar
epage is offline epage
2011-02-28 , 22:03
Posts: 1,684 | Thanked: 1,562 times | Joined on Jun 2008 @ Austin, TX
#9
Take a look at the "login" and "_login" functions in the following file
https://github.com/epage/The-One-Rin...ice/session.py

"login" creates a AsyncLinearExecution which runs "_login" in the foreground thread but everything at a "yield" in a background thread. Looking at the "_login" function this takes what would be two or three callbacks with different kinds of state and simplifies it to a single function.

EDIT: And since I wrote that code, I've generalized it for both Qt and GTK. Makes callback code a lot simpler. Each piece of code after a "yield" is runs as a callback and you would have had to find a way to pass all that state to that callback.
__________________
770, n810, n900, Ideapad S10-3t
TheOneRing, DialCentral, Gonvert, Quicknote, Multilist, ejpi, nQa, Waters of Shiloah
Programming Blog
 
luis is offline luis
2011-03-01 , 02:16
Posts: 146 | Thanked: 15 times | Joined on Oct 2008
#10
Originally Posted by jstokes View Post
http://www.pygtk.org/pygtk2reference...t--spawn-async
Probably I didn't understand this, but isn't this something to call an external program? Kind of subprocess.Popen ? I want to run a method of the program itself async.

Thanks,
L.
 
Reply
Page 1 of 2 1 2

« Previous Thread | Next Thread »

 
Forum Jump


All times are GMT. The time now is 15:32.

Contact Us - Home - Archive - Top