11

wget is a very useful tool to download stuff on the internet quickly, but can I use it to download from hosting sites, like FreakShare, IFile.it Depositfiles, Uploaded, Rapidshare? If so, how can I do that?

Zignd
  • 3,103
  • 5
    Don't most of those sites tend use javascript and other barriers to eliminate linking directly to the files? – Tim Jul 13 '12 at 17:08

3 Answers3

11

This kind of site tries to make it difficult not to use a graphical web browser, because if you use wget you'll be missing all these ads that pay for the bandwidth.

Some sites don't make advanced checks and can be tricked easily: tell wget to pretend that it's really Mozilla and that it's coming from the download site.

wget --user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1' \
     --referer=http://downloadsite.example.com/download-page-url
     http://downloadsite.example.com/download-page-url/filename.ext`

Most sites that check let you get away with --user-agent=Mozilla and --referer set to the URL of the file you're downloading.

With some sites, you might need to export the web browser cookies and pass --load-cookies to wget; at this point using wget starts to be more work than a manual download. The ultimate method, which can work all the time but requires some coding for each site, is to use a browser automation framework such as Selenium or Watir.

See also Automating web requests using curl?

Community
  • 1
Gilles 'SO- stop being evil'
  • 829,060
  • Awesome! Your answer helped me write this for Zippyshare Links: http://stackoverflow.com/a/21100640/1533054 – Sheharyar Jan 13 '14 at 20:23
7

For Zippyshare:

This method builds up on Gilles' answer. Like he said, the trick is to make the server think that the request is being made from a browser and to make that happen, you need a couple of things:

  • The Download Link (The actual link to the file)
  • Link Referrer (The webpage with the download button)
  • Zippyshare Session ID (Found in Cookies)

Here's a screenshot explaining where you can get each item:

Download From Zippyshare

Now open up your terminal, and use the following command (replacing necessary items):

wget <download_link> \
--referer='<referrer>' \
--cookies=off --header "Cookie: JSESSIONID=<session_id>" \
--user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1'

Example:

wget http://www16.zippyshare.com/d/29887835/8895183/hello.txt \
--referer='http://www16.zippyshare.com/v/29887835/file.html' \
--cookies=off --header "Cookie: JSESSIONID=26458C0893BF69F88EB5743D74FE0F8C" \
--user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1'

Note: In the command, it is in fact referer not referrer

Community
  • 1
Sheharyar
  • 453
  • 1
    I'd just like to add that Sheharyar uses Firebug in this screenshot. This is to everybody (including me) who tried to do this in Inspector (which is unfortunately not possible). – syntaxerror Jun 08 '15 at 16:11
3

I also would like to above perfect answers, I used following trick for downloading a child movie for my son from Turbobit:

Instead of SESSION_ID from above answer, you need to find "compid" from the cookie tab of FIREBUG:

enter image description here

After that, the command for my debian embedded system was soo easy:

wget --referer='http://http://turbobit.net/download/free/yilmacr4e351' --cookies=off --header "Cookie: compid=c9255510c34624435292d8a0e7e405be66aa01f9%7ED01343F87AB53A2C38D957A564DA1D55" --user-agent='Mozilla/5.0 (Windows NT 6.0) Gecko/20100101 Firefox/14.0.1' http://turbobit.net/download/redirect/B3EDACDA9B899937A149D5AAB6662327/yilmacr4e351/Ters.Yuz.2015.1080P.WEBDL.TR.EN.mkv &

I can shut down my computer now :)

Sertac TULLUK
  • 41