I started using GCP (Google Cloud Platform) recently. I created an Ubuntu VM in Google Cloud Engine, and wanted to download some files from a website via wget.
I am able to download it using a normal webbrowser on my windows computer and also using wget in a WSL Ubuntu kernel, which I installed in my local windows computer.
I'm using the following command:
wget -c --tries=0 -O sample.zip --user-agent="Mozilla" --referer="https://ida.loni.usc.edu/<.......>" --accept="text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" https://<.....url.....>.zip
And it gives me the following error:
HTTP request sent, awaiting response... 403 ERROR 403: (no description).
I've tried several other commands, such like trying different user-agent values, without user-agent and without referer. But it also does not work.
Lastly, I've also tried using curl and a python script, but with no success. Everything ends up in the same 403 (forbidden) error.
I already read the post why would curl and wget result in a 403 forbidden? and some similar ones, which talk about solutions for this 403 error problem. However, none of them worked for me.
My guess is that I am missing out something on my firewall configuration or in my request header.
UPDATE:
The files I want to download are large, they are over 500mb. Apparently, I am able to download a small metadata zip file from the https://ida.loni.usc.edu/ domain.
Any guesses on that?
Also, I already included a firwall rule to allow external IPs.
