185

I'm trying to use the curl command to access a http url with a exclamation mark (!) in its path. e.g:

curl -v "http://example.org/!287s87asdjh2/somepath/someresource"

the console replies with bash: ... event not found.

What is going on here? and what would be the proper syntax to escape the exclamation mark?

netbrain
  • 2,205
  • 2
  • 16
  • 12
  • 5
    Bash 4.4 makes cases like "foo!" not invoke history expansion, but "foo!123" still does. – ilkkachu Apr 07 '21 at 13:32
  • 2
    The answers below all detail how to correctly use "!" in bash. But for this exact question, where the '!' is in a curl command in a URL, you could have simply URL encoded the problematic symbol instead where '!' is replaced with '%21' – Dan Ciborowski - MSFT Oct 15 '22 at 00:39

7 Answers7

195

The exclamation mark is part of history expansion in bash. To use it you need it enclosed in single quotes (eg: 'http://example.org/!132').

You might try to directly escape it with a backslash (\) before the character (eg: "http://example.org/\!132"). However, even though a backslash before the exclamation mark does prevent history expansion, the backslash is not removed on some shells in such a case. So it's better to use single quotes, so you're not passing a literal backslash to curl as part of the URL.

  • 13
    "http://example.org/\!132" actually expands without interpreting the backslash (POSIX compliance reasons, I believe). – Chris Down Mar 03 '12 at 19:14
  • @ChrisDown, I tried to clarify that was my second option in the text. Thanks for pointing out the potential for confusion. – Daniel Pittman Mar 03 '12 at 22:30
  • 8
    For the record: It's not portable to try escaping "!". The best-practices recommendation is to always quote (singe-quotes) "!". Related: "^" (caret), is a non-metacharacter that needs quoting for portability. Finally, "!" should not be used in an if statement; use it as an argument to test instead if possible (again because of Solaris /bin/sh). – Nicholas Wilson Oct 18 '12 at 10:31
  • 9
    Only single quotes worked for me. zsh was still interpreting \! and double quotes. – orkoden May 21 '14 at 16:29
  • @NicholasWilson, under what circumstances do you need to quote ^? I've never heard of such a thing. – Wildcard Jan 16 '18 at 04:58
  • 2
    On Solaris (rubbish old pre-XPG4 shell), '^' is an alias for | and is used to create a pipe. If you're sending scripts to customers and can't be sure what shell they'll run it in, you have to test with them all! – Nicholas Wilson Jan 16 '18 at 09:47
  • 1
    @Wildcard, ^ is pipe in thompson/Bourne shells and also a glob operator in zsh -o extendedglob. It's also special in rc, es, akanga, fish and probably other less common shells. ksh/POSIX shells are the odd ones out. – Stéphane Chazelas Apr 25 '23 at 07:14
  • This did not work for me, since the ! is part of a sed expression that I'm sending over ssh. – RonJohn Nov 22 '23 at 03:18
107

As well as the answer given by Daniel, you can also simply turn off history expansion altogether if you don't use it with:

set +H

or

set +o histexpand

(the latter also working in zsh, where histexpand is an alternative name for the banghist option for bash compatibility).

In (t)csh where history expansion originates, you'd disable it by assigning the empty string to the $histchars variable with:

set histchars = ''

Or:

set histchars

(not via unset histchars)

That also works in bash and zsh, the two shells that copied that feature, though the syntax is:

histchars=

There.

Beware that bash (contrary to (t)csh or zsh) does not ignore a $histchars variable found in the environment, so beware of the possible consequences if you export that variable.

Chris Down
  • 125,559
  • 25
  • 270
  • 266
  • 23
    Turning off history expansion altogether is the best advice I've heard all day! History expansion is dangerous and byzantine when there are much better alternatives (incremental history search with Ctrl-R) that let you preview & edit your command so you don't blindly fire away with command !-14 that you though was at !-12 that, oops, happened to be rm -rf *. Be safe. Disable history expansion! Eschew the !! – aculich Mar 06 '12 at 01:09
  • 6
    Biggest answer: history expansion is a huge security risk! It can be used to attack your Unix through a crafted URL. – dan Jun 15 '15 at 07:30
  • @aculich, or just use the POSIX-specified command fc -14 instead. But it's true that you can do that without history expansion being enabled also. Personally, I use !$ and !vi and sudo !! and even git add !vi:$ often enough to warrant leaving history expansion enabled. – Wildcard Jan 16 '18 at 05:02
  • I think I'll add this to my shell RC files. I've only ever used this as a neat "trick" – TonyH Aug 22 '19 at 14:02
  • alternate, IMO more descriptive syntax to turn off bash history expansion: set +o histexpand; see also https://superuser.com/a/133782 – ssc Jun 28 '20 at 10:15
  • For zsh users, the syntax is unsetopt banghist. – ndrplz Apr 30 '22 at 09:42
26

I would personally do single quotes, but for completeness, I will also note since it is a URL, you can encode the ! as %21, e.g. curl -v http://example.org/%21132 .

21

This also can do

curl -v "http://example.org/"'!'"287s87asdjh2/somepath/someresource"
or
curl -v "http://example.org/"\!"287s87asdjh2/somepath/someresource"

Which works because bash concatenates adjacent strings. This approach is particularly useful when you have other things that need shell expansion, so you can't use single quotes for the entire string:

curl -v 'http://example.org/!'"287s87asdjh2/${basepath}/someresource"

! character is used for history expansions in command line prompt.
so this can be a problem in prompt but not in shell script files.
as you can see history expansions work even in double quotes.

pavon
  • 238
mug896
  • 965
  • 9
  • 12
  • There are lots of ways of making Unix commands and English sentences use more characters than they need to, and be more confusing than they need to be.  How is this superior to the first / accepted / highest-voted answer, namely, putting the entire URL into single quotes? – G-Man Says 'Reinstate Monica' Jun 15 '15 at 07:10
  • 3
    @G-Man : It tells another way to construct bash arguments. I wasn't aware of this method. Nothing wrong in learning new stuff. – Sahil Singh Jul 06 '16 at 12:30
  • @SahilSingh How is this new? It concatenates three strings, two enclosed in double quotes and one enclosed in single quotes. There is no nesting here. – Raphael Mar 10 '17 at 06:44
  • @G-Man It is not obvious that when you put 2 strings next to each other they get concatenated. printf("hello""world") would work in c as well, but printf("hello"'w') won't work, so you see knowing that bash accommodates such expressions was new to me, but I do agree from utility point of view, this is not superior. I liked the answer, so did Mark Shust. – Sahil Singh Mar 10 '17 at 07:06
  • @SahilSingh: (1) I believe that you really meant to address that last comment to @Raphael; he’s the one who criticized what you said. (2) The analogy to C is flawed.  In C, "…" and '…' give you different data types.  printf('world') won’t work, so why would you expect printf("hello"'world') to? (3) I was hoping to refer you to the section in the bash man page that would make everything clear, and, to my surprise, I couldn’t find one. … (Cont’d) – G-Man Says 'Reinstate Monica' Mar 12 '17 at 06:07
  • (Cont’d) …  I guess that’s why I didn’t respond to your comment last summer — I recognized that you had a valid point, and I had nothing useful to add to it. (4) I guess the larger problem is that people misunderstand a lot of things about quotes in the shell. (4a) Quotes are needed only, well, only when they’re needed.  I’ve seen people say read "x" "y", which is a totally unnecessary use of quotes.  I’ve seen people say grep "orange" colors.  It’s a good idea to get into the habit of quoting regular expressions, … (Cont’d) – G-Man Says 'Reinstate Monica' Mar 12 '17 at 06:07
  • (Cont’d) …  but as long as they are simple strings with no special characters, it’s not necessary.  I presume that you know that echo hello is equivalent to echo "hello" or echo 'hello'. (4b) Quotes don’t quote strings, they quote characters, and sequential non-blank characters form words, so we can do things like echo h"e"l'l'o or echo "he"'llo'.  See Bash quotes unescaped on command substitution for more discussion and examples. – G-Man Says 'Reinstate Monica' Mar 12 '17 at 06:10
  • 3
    @G-Man It's also useful when there are other string expansions one does want to happen in the same string. This is an easy way of separating two types of quoting behavior. – WAF May 01 '17 at 15:09
  • @WAF: (1) When I post a four-part comment, and you respond with a comment that talks about “It”, your reference is unclear. *What* is also useful when there are other string expansions one does want to happen in the same string? (2) Do you mean the ability to use both quote characters in the same string, like "hello"'world'?  Yes, I know it can be useful; I never said that it couldn’t be.  I said that quotes were occasionally overused.  Specifically, I said that mug896’s answer was equivalent to Daniel Pittman’s answer, … (Cont’d) – G-Man Says 'Reinstate Monica' May 01 '17 at 23:05
  • (Cont’d) … which is (a) three years older and (b) accepted, with the only difference being that mug896’s answer used more quotes than were necessary for that answer. (3) Comments can be 600 characters long, and you can write several of them in a row. Your comment is 171 characters long (including seven for @ and my name). If you’re going to offer a broad, general observation as a counterpoint to a comment that has examples, you might want to put some examples into your comment. – G-Man Says 'Reinstate Monica' May 01 '17 at 23:05
14

I have come across the same problem, and my simple solution was to use a variable:

E=!  
curl -v "http://example.org/${E}287s87asdjh2/somepath/someresource"

Here the simplicity is that (1) It is portable across shells and commands (2) Does not require knowing escape syntax and ASCII codes.

Prem
  • 3,342
5

Ever since Bash 4.3, you can now use double quotes to quote the history expansion character (edit: as long as the the exclamation mark is at the end of the string or followed by a space)

$ $SHELL --version
GNU bash, version 4.3...
[...]
$ echo "Hello World!"
Hello World!
$ echo "Hello! World"
Hello! World
$ echo "Hello!World"
bash: !World: event not found
$ echo 'Hello!World'
Hello!World
Flimm
  • 4,218
  • this doesnt work outside of echo, echo seems to handle this differently on its own – phil294 Sep 30 '17 at 12:04
  • @Blauhirn This has nothing to do with echo, and everything to do with quoting and the version of bash that you're running. – Flimm Oct 01 '17 at 13:38
  • @don_crissti: more precisely, it does not trigger history expansion if "end of line" or followed by whitespace : echo "Hi! Hello." also works as intended. – MestreLion Nov 22 '21 at 04:33
  • @don_crissti: also, these exceptions to histexpand were indeed introduced in bash 4.3, so Flimm is correct in saying it does have to do with bash version: https://mywiki.wooledge.org/BashPitfalls#echo_.22Hello_World.21.22 – MestreLion Nov 22 '21 at 04:39
-1

To those who are using git bash in windows, the accepted answer from @DanielPittman works. However, you should replace the backslash (\) with a forward slash (/).

For example, in unix, it'd look something like this:

curl https://abc.com/services -H 'Authorization: Bearer 111A80BBZZCnS\!ZR412543s'

For windows, it'd be something like this (focus on the forward slash in the authorization header part)

curl https://abc.com/services -H 'Authorization: Bearer 111A80BBZZCnS/!ZR412543s'

  • This doesn't make much sense. You have the argument single quoted, so regardless of the slashes involved the exclam won't result in history expansion. – Wildcard Jan 16 '18 at 05:03
  • Ohh you're right. I only posted this answer because when I was using Daniel's answer (using backslash), an error pops up. – SamuelDev Jan 16 '18 at 05:12