Any way to encode the url in curl command?

Question

I have some url which has space in it's query param. I want to use this in curl, e.g.

curl -G "http://localhost:30001/data?zip=47401&utc_begin=2013-8-1 00:00:00&utc_end=2013-8-2 00:00:00&country_code=USA"

which gives out

Malformed Request-Line

As per my understanding o/p is due to the space present in query param.

Is there any away to encode the url automatically before providing it to curl command?

score 253 · Answer 1 · edited Apr 18 '18 at 12:28

253

curl supports url-encoding internally with --data-urlencode:

$ curl -G -v "http://localhost:30001/data" --data-urlencode "msg=hello world" --data-urlencode "msg2=hello world2"

-G is also necessary to append the data to the URL.

Trace headers

> GET /data?msg=hello%20world&msg2=hello%20world2 HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu)
> Host: localhost
> Accept: */*

edited Apr 18 '18 at 12:28

kevinarpe

answered Aug 14 '13 at 16:40

damphat

1

What if msg = '=' ? – Mmmh mmh Oct 15 '15 at 09:14
From curl doc: Note that the name part (msg in this case) is expected to be URL-encoded already. Also you can specify something like --request DELETE and it would indeed be a delete method instead of a GET. Not sure if order matters. – Federico Feb 03 '16 at 00:21
@damphat what happens when the request has two parameters like "msg1=Hello&msg2=World"? This will encode the & between the parameters which would mean wrong thing to send to the server – Ganesh Satpute Jul 28 '16 at 13:08
12

@GaneshSatpute: use multiple --data-urlencode parameters, one for each key-value pair. – Martijn Pieters Aug 09 '16 at 13:04
@MartijnPieters Yes. that works. Thanks for the answer – Ganesh Satpute Aug 10 '16 at 06:38
@DimitriKopriwa just repeat the --data-urlencode option, like in the example provided – Lucas Cimon May 02 '19 at 17:07
2

Is it possible to perform query params url-encoding for a POST request and provide a payload ? – Lucas Cimon May 02 '19 at 17:08
i wonder why this answer is not accepted. – Rakibul Haq Mar 17 '20 at 06:08
Is it possible to do urlencode on part of the url, instead of the data? I.e. to access localhost:123/hello%20world. – Dan M. Jul 11 '22 at 12:45

score 5 · Answer 2 · edited Oct 25 '15 at 13:49

5

 curl -G "$( echo "$URL" | sed 's/ /%20/g' )"

Where $URL is the url you want to do the translations on.

There are also more than one type of translation (encoding) you can have in a URL, so you may want to do:

curl -G "$(perl -MURI::Escape -e 'print uri_escape shift, , q{^A-Za-z0-9\-._~/:}' -- "$URL")"

instead.

edited Oct 25 '15 at 13:49

ctrl-alt-delor

answered Aug 14 '13 at 16:16

Drav Sloan

1

Note that echo "$URL" | sed 's/ /%20/' won't do the right thing if there are % characters in the URL. Also, spaces are normally encoded as + (and + as %2b). I recommend the Perl solution, which is reliable. – Gilles 'SO- stop being evil' Aug 14 '13 at 22:02
2

sed 's/ /%20/g' if you have more than one space to translate... – sebthebert Feb 06 '15 at 14:45
Note I had to install the Perl URI::Escape module. – buzz3791 Aug 21 '18 at 17:48

2 Answers2