maemo.org - Talk - [ANNOUNCE] movgrab (latest) ...

Hi all,

Here is my first attempt to package the latest version of movgrab :

Edit the 30th April 2013 : Pushed version up to the latest (1.2.1) ...

Changelog :

Version 1.2.1

Fixes for many sites.
Added 'sshtunnel' proxy type.

movgrab :

movgrab is a downloader for all those pesky sites that insist you use a big fat browser that runs flash in order to see their content. It's a command-line app written in straight C, and so doesn't require you to install perl. Nor ruby. Nor python. Nor guile, scheme, glib, gtk, qt, gnome, kde, X-windows, m4, firefox or windows. No! Not any of that! It *should* compile on all posix unix systems, I think (although I'm a bit worried about the whole little-endian/big-endian thing).

Overview :

Written in straight C.
Command-line application.
No dependancies, everything needed is in the .tar.gz source file.
HTTP Proxy server support.
Write download to std-out to feed into another application: (eg movgrab -f - URL | mplayer -).
Progress display while downloading.
Fork into background with '-b' flag.

Homepage :

https://sites.google.com/site/columscode/home/movgrab

Download packages from extra-devel :

Activate all repositories following this tutorial : Repositories Activation

Then, as usual, as root, install with :

Code:

-bash-2.05b# apt-get install movgrab && movgrab --help

Reading package lists... Done

Building dependency tree       

Reading state information... Done

The following NEW packages will be installed:

  movgrab

0 upgraded, 1 newly installed, 0 to remove and 19 not upgraded.

Need to get 0B/77,5kB of archives.

After this operation, 188kB of additional disk space will be used.

Selecting previously deselected package movgrab.

(Reading database ... 47546 files and directories currently installed.)

Unpacking movgrab (from .../movgrab_1.1.14-1maemo0_armel.deb) ...

Setting up movgrab (1.1.14-1maemo0) ...



Movgrab: version 1.1.14

Author: Colum Paget

Email: colums.projects@gmail.com

Blogs: 

        tech: http://idratherhack.blogspot.com 

        rants: http://thesingularitysucks.blogspot.com 



Usage: movgrab [-t <type>] -a [<username>:<password>] [-p http://username:password@x.x.x.x:80 ] [-r] [-b] [-x] [-q] [-st <stream timeout>] [-f <format list>] [-v] [-P] [-Pp] [-o <output file>] [+o <extra output file>] url

        movgrab -test-sites



'-v'                increases verbosity/debug level

'-v -v'                prints out all webpages encountered

'-v -v -v'        maximum debugging

'-T'                Test mode, don't do final download

'-P <program>'                Player program (e.g. "mplayer")

'-Pp'                Percent download to launch player at (default 25%)

'-a'                Authentication info in Username:Password format.

'-q'                QUIET. No progress/informative output.

'-b'                Background. Fork into background and nohup

'-p'                address of HTTP proxy server in URL format.

'-w'                Wait for addresses to be entered on stdin.

'-st'                Connection inactivity timeout in seconds. Set high for sites that 'throttle'

'-t'                specifies website type.

'-r'                Resume download (only works when writing a single file, not with +o).

'-f'                specifies preferred video/audio formats for sites that offer more than one

                        example: flv:640x480,flv,mp4,mp3

                        Use -T to get a list of formats the site offers

                        Use * to mean 'any format' (the default)

                        example: flv:640x480,mp4:640x480,*

'-o'                specifies output file ( '-' for stdout)

'+o'                add an output file to list of output files ( '-' for stdout)

'-x'                try to avoid 'family filter' on some sites

'-n'                For pages with multiple movies (not movie formats, but movies) specifes item selection for download. Argument has the form:

                        -n all                Download all

                        -n 2                Download 2nd item

                        -n 2-5                Download 2nd to 5th item

                        -n 2-                Download from 2nd item onwards

                        -n -8                Download from 1st to 8th item

-test-sites                Test downloading from each supported website

The '-t' argument is optional, Movgrab will try to guess the download type from the provided url.



The url should be that of the page that the movie would normally appear on. Movgrab examines the page for embed tags and other information that would normally cause the movie to play on that page.



Download types are:

generic              Generic: Search in page for http://*.flv, http://*.mp3, http//*.mp4 etc, etc, etc

youtube              YouTube: http://www.youtube.com

metacafe             Metacafe: http://www.metacafe.com

dailymotion          Daily Motion: http://www.dailymotion.com

break                www.break.com

ehow                 www.ehow.com

vimeo                www.vimeo.com

5min                 www.5min.com

vbox7                www.vbox7.com

blip.tv              www.blip.tv

ted                  www.ted.com

myvideo              www.myvideo.de

clipshack            www.clipshack.com

mytopclip            www.mytopclip.com

redbalcony           www.redbalcony.com

mobango              www.mobango.com

yale                 Yale University: http://oyc.yale.edu

princeton            Princeton University: http://www.princeton.edu/WebMedia/

reuters              Reuters: http://www.reuters.com/

liveleak             Liveleak: http://www.liveleak.com

academicearth        Academic Earth: http://www.academicearth.org

photobucket          Photobucket: http://www.photobucket.com

aljazeera            Aljazeera: english.aljazeera.net

mefeedia             mefeedia.com

iviewtube            iViewTube: www.iviewtube.com

washingtonpost       Washington Post: www.washingtonpost.com

cbsnews              CBS News: www.cbsnews.com

france24             France24: www.france24.com

euronews             Euronews: www.euronews.net

metatube             www.metatube.com

motionfeeds          www.motionfeeds.com

escapist             www.escapistmagazine.com

guardian             www.guardian.co.uk

redorbit             www.redorbit.com

scivee               www.scivee.tv

izlese               www.izlese.org

uctv.tv              University of California Television: http://www.uctv.tv/

royalsociety.tv      http://royalsociety.org/

britishacademy       http://britac.studyserve.com

kitp                 KAVLI INSTITUTE: http://online.itp.ucsb.edu/plecture/

dotsub               dotsub.com

astronomy.com        astronomy.com

teachertube.com      teachertube.com

discovery            dsc.discovery.com

bloomberg            www.bloomberg.com (not bloomberg.tv)

nationalgeographic   National Geographic (http://video.nationalgeographic.com)

videobash            videobash

ibtimes              International Business Times

smh                  Sidney Morning Herald

presstv              Press TV (Iran)

videojug             www.videojug.com

animehere            www.animehere.com



If a website is not in the list, try 'movgrab -t generic <url>'



Feel free to email me and tell me if you've used this software!



Thanks for bug reports go to: Mark Gamar, Rich Kcsa, 'Rampant Badger', 'nibbles', 'deeice', Omair Eshkenazi, Matthias B, Ashish Disawal, Timo Juhani Lindfors, Abhisek Sanyal and others.



Special thanks to:

        'legatvs' for clive (http://clive.sourceforge.net) another downloader into whose code I had to look to figure out how to get youtube and daily motion working again.

        Robert Crowley (http://oldcoder.org/) For all sorts of bug reports and advice.

-bash-2.05b#

After successful installation, just use it !!!

Example :

- First we use '-T' option to show all formats available ...

Code:

-bash-2.05b$ movgrab -T http://www.youtube.com/watch?v=mL8puJMUDpk



Formats available for this Movie:webm:1920x1080 (233.7M), mp4:1920x1080 (224.5M), webm:1280x720 (118.9M), mp4:1280x720 (111.2M), webm:854x480 (50.1M), flv-h264:854x480 (47.3M), webm:640x360 (35.7M), flv-h264:640x360 (31.7M), mp4:480x360 (29.0M), flv:400x240 (14.9M), 3gp:400x240 (9.9M), 3gp:176x144 (3.6M), 



Selected format item:3gp:400x240

TEST MODE: would have downloaded 'AWESOME X: Beautiful Moments' url=http://r4---sn-nwj7kney.c.youtube.com/videoplayback?upn=Tzi_NDCvsb4&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cgcr%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&fexp=901803%2C923108%2C901444%2C922401%2C920704%2C912806%2C925703%2C925706%2C928001%2C922403%2C929901%2C913605%2C913546%2C913556%2C908493%2C920201%2C911116%2C901451&ms=au&algorithm=throttle-factor&burst=40&ip=62.147.196.247&itag=36&gcr=fr&sver=3&mt=1354832351&mv=m&source=youtube&key=yt1&ipbits=8&factor=1.25&cp=U0hUSVhQVF9JSkNONF9QSlhHOnFRdThmN3gxUjhF&expire=1354856640&id=98bf29b893140e99&newshard=yes&signature=AA3A9490D0CB7015E5B58FF4109D7FAD65CA5560.73511B67941FA7219579600F8ACC3778FDCF5082

-bash-2.05b$

- Then we select the one we want to download ...

Code:

-bash-2.05b$ movgrab -o "AWESOME X: Beautiful Moments.mp4" -f 3gp:176x144 http://www.youtube.com/watch?v=mL8puJMUDpk



Formats available for this Movie:webm:1920x1080 (233.7M), mp4:1920x1080 (224.5M), webm:1280x720 (118.9M), mp4:1280x720 (111.2M), webm:854x480 (50.1M), flv-h264:854x480 (47.3M), webm:640x360 (35.7M), flv-h264:640x360 (31.7M), mp4:480x360 (29.0M), flv:400x240 (14.9M), 3gp:400x240 (9.9M), 3gp:176x144 (3.6M), 



Selected format item:3gp:176x144



Getting: AWESOME X: Beautiful Moments...  Size: 3.6M  Format: 3gp:176x144

        Progress: 99.76%  3.6M of 3.6M  21.9k/s            

-bash-2.05b$

Hope you like it :) ...

A++