44

Is it possible to use curl and post binary data without passing in a file name? For example, you can post a form using binary via --data-binary:

curl -X POST --data-binary @myfile.bin http://foo.com

However, this requires that a file exists. I was hoping to be able to log HTTP calls (such as to rest services) as the text of the curl command to reproduce the request. (this greatly assists debugging these services, for example)

However, logging curl commands that reference a file would not be useful, so I was hoping I could actually log the raw binary data, presumably base64 encoded, and yet still allow you to copy and paste the logged curl command and execute it.

So, is it possible to use curl and post binary data without referencing a file? If so, how would that work? What would an example look like?

Kirk Woll
  • 1,127
  • If you're not logging it to a file then where are you logging it to? – slm May 30 '13 at 00:07
  • @slm, I think you misunderstand. I am logging my client-side rest calls to a log file in plain text (as the verbatim equivalent curl command). However, I do not want to have to reference some random file for each binary curl command I log. I want the entire text of the curl command to be fully self-contained so that someone could copy that text into a terminal and run it to reproduce the call. – Kirk Woll May 30 '13 at 00:09

3 Answers3

61

You can pass data into curl via STDIN like so:

echo -e '...data...\n' | curl -X POST --data-binary @- http://foo.com

The @- tells curl to pull in from STDIN.

To pipe binary data to curl (for example):

echo -e '\x03\xF1' | curl -X POST --data-binary @- http://foo.com
slm
  • 369,824
  • 1
    But this still does not allow the binary data to be part of the text of the curl command. So is that not possible? – Kirk Woll May 30 '13 at 13:50
  • What you're asking for doesn't sound feasible. The data to the curl command can either be piped into curl or pulled into via the @ facility. To my knowledge there isn't any other method. – slm May 30 '13 at 14:58
  • 1
    Actually, your answer is perfect -- your example should just use binary instead. :) i.e. echo -e '\x03\xF1' | curl -X POST --data-binary @- http://foo.com Where \x03\xF1 is the raw binary data as hexadecimal, which is pretty much exactly what I want. – Kirk Woll May 30 '13 at 16:31
  • Great. I wasn't following you entirely. So you jut wanted to pipe binary data into curl, I thought you were asking for something else. Glad it worked for you. – slm May 30 '13 at 17:01
  • 3
    I tried this: cat 1.jpg | curl -X POST --data-binary @- http://foo.com. Worked like a charm. To validate, I also tried: cat 1.jpg | md5sum and md5sum 1.jpg. They both returned the same value. – dimitarvp May 30 '14 at 18:53
  • 1
    -X POST is not necessary, since the --data-binary @- part already tells curl to automatically use POST as its method (from man curl: "Sends the specified data in a POST request to the HTTP server"). It can therefore be omitted. – Yeti Oct 28 '21 at 11:28
2

Adding to this answer, the echo command appends a newline to its output by default. This adds a \n to the end of your binary data, therefore curl will receive that character too as input.

To avoid that, you can use the printf command or the -n switch like so:

printf '\x03\xF1' | curl -X POST --data-binary @- http://foo.com

or

echo -en '\x03\xF1' | curl -X POST --data-binary @- http://foo.com

This way no newline will be appended to the curl input and the bytes being fed to curl will be exactly those you pass to echo.

robg
  • 21
  • 1
-1

Not sure why, but the exact command line that slm suggested didn't work for me. With a slight modification, the following worked:

echo -e '...data...\n' | curl -s -T - sftp://user@10.10.10.10/~/test.txt
Richard Nienaber
  • 99
  • 2
    The reason why is that SFTP is an entirely different protocol from HTTP with very different capabilities, and both the man page and help message say --data-binary only applies to HTTP (and HTTPS). – dave_thompson_085 Oct 10 '17 at 02:05
  • That makes sense. I was combining several online sources to get that working so I missed the part of the question which made it HTTP specific. I'll leave my answer here in case someone else needs it. – Richard Nienaber Oct 10 '17 at 15:37