maemo.org - Talk - View Single Post - Is there anyway to download a whole thread

ziggadebo	2011-10-11 , 19:50
Posts: 99 \| Thanked: 36 times \| Joined on Mar 2010	#9

Firstly marxian a big thank you for your help you certainly pointed me in a direction to get this working. The below is your code, I just changed it to get it working for me.

I couldn't get your python code to run try as I might (Probably more to do with my complete lack of understanding than your code) However It definitely won't run on the N900 as python2.5 doesn't support the io library.

Anyway I've taken what you've given me and with some help( well a lot of help) I've managed to get it working on the N900.

If anyone wants to use, the process/steps needed are as follows:
(I'm writing this as an absolute beginner so feel free to point out any errors)

I'm assuming that python2.5 has been installed on your N900. If not install it first.

Firstly using a text editor on your N900 (I use leafpad) copy the code below into it and save the file. Call it getathread.py - Save it to your MyDocs folder so that the output will be easily reachable when we need it.

Code:

import urllib
import sys
try:
    sys.argv[3] = int(sys.argv[3])
    sys.argv[4] = int(sys.argv[4])
except ValueError:
    print "Use numbers for the last two arguments or whatever"
    sys.exit(1)

def get_thread(fileName, link, start_page, end_page):
    for num in range(int(start_page), int(end_page) + 1):
        print "grabbing page " + str(num)
        page = urllib.urlopen("%s&page=%d" % (link, num)).read()
        file = open("%s-page%d.html" % (fileName, num) , 'w')
        file.write(page)
        print "downloaded: "+ str(num)
        file.close()





if __name__ == '__main__':
    sys.exit(get_thread(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]))

now go into your terminal

and change directory to the MyDocs directory

by typing

Code:

cd MyDocs

Now to run the script we need 4 variables (pieces of information)

1. File name you want to save the output as
2. link to the thread
3. First page of thread to download
4. Last page of thread to download

We will use this information to trigger the code.

So for this example, I will use the popular Kernal Power V49 thread

So our variables are
1. power49
2. http://talk.maemo.org/showthread.php?p=1105192
3. 1
4. 186

So to run our code we would type in terminal (note the variables are just separated by a single space)

Code:

python getathread.py power49 http://talk.maemo.org/showthread.php?p=1105192 1 186

you should then get output on the screen saying
grabbing page 1
downloaded: 1
grabbing page 2
downloaded: 2

etc......

When finished simply launch any of the files from filemanager or from your browser.

Quote & Reply |

The Following 2 Users Say Thank You to ziggadebo For This Useful Post:
handaxe, marxian