6

I am downloading a large file over the network. while downloading (say at 40%) the underlying machine get disconnected from network and before the curl quit, the network get connected. But in this situation, curl do not resume the process.

Eventually curls quits.

What are the switches for curl which will help me to resume this interrupted download ?

EDIT: More explanation to problem

18:43:20 PM    SHW@SHW:/tmp # curl http://ip-address.com/BigFile
18:43:40 PM                   <Downloading in progress>
18:43:50 PM                   <Downloading in progress>
18:44:10 PM                   <network get disconnected>
18:44:20 PM                   <Downloading get stuck>
18:44:30 PM                   <Network get connected>
18:44:40 PM                   <DOWNLOAD MUST RESUME NOW>      <==

As per above timestamp, I want curl to resume download before curl quits the started process. I do not want to re-excute the curl command

Braiam
  • 35,991
SHW
  • 14,786
  • 14
  • 66
  • 101

1 Answers1

14

you can use

curl -L -O --retry 999 --retry-max-time 0 -C - http://url
  • -C -: resume where the previous download left off
  • --retry 999 : retrying so many times
  • --retry-max-time 0 : prevent it from timing out on retrying

or

curl -L -o 'filename' -C - http://url

Update

export ec=18; while [ $ec -eq 18 ]; do /usr/bin/curl -O -C - "http://www.example.com/big-archive.zip"; export ec=$?; done

Explanation :

The exit code curl chucks when a download is interrupted is 18, and $? gives you the exit code of the last command in bash. So, while the exit code is 18, keep trying to download the file, maintaining the filename (-O).

My personal preference would be to use wget which has been built specifically for this use case. From the man page:

Wget has been designed for robustness over slow or unstable network connections;
if a download fails due to a network problem, it will keep retrying until the
whole file has been retrieved.  If the server supports regetting, it will
instruct the server to continue the download from where it left off.

wget is available for almost all Linux distributions - it probably is already installed on yours. Just use wget to download the file, it will re-establish the network connection until the file is completely transferred.

Avindra Goolcharan
  • 429
Rahul
  • 13,589