Split command line into a list of arguments

Question

How do I take a string giving a shell command line such as:

"program arg1 arg2 \"long argument with spaces\" arg\\\"3"

and turn it into a list of unquoted arguments like:

("program" "arg1" "arg2" "long argument with spaces" "arg\"3")

I.e. almost the reverse of shell-quote-argument. shell-unquote-argument doesn't do the trick.

I realize this sounds like an XY problem. I'd like to offer a minibuffer prompt for the user to give command line arguments to an external program, then run that program via call-process. I could prompt for the command line arguments one at a time until the user gives a blank, but it seems like better UI to let them type all the arguments at once as they would at a real shell prompt. I could also error out on fancy characters and only let them type alphanumerics, dashes and spaces, but that seems limiting. As a third alternative, I could use the call-process variant that gives the command line to a shell instead using it directly as the argv, but I'd like to avoid giving the user the full power of the shell (pipes, redirects, etc.) as that would probably confuse more than help.

score 5 · Answer 1 · answered Nov 06 '18 at 21:19

5

Does this do what you want?

(split-string-and-unquote "program arg1 arg2 \"long argument with spaces\" \"arg\"3")

There seemed to be an error in your string with an unmatched \" at the end that caused this function to not work. I am not sure if it makes sense to have the single " in there or not. If it does, you might have to write your own parser.

answered Nov 06 '18 at 21:19

John Kitchin

11,555
1
19
41

You're right, I made a mistake with the nested quoting. Fixed. – Lassi Nov 06 '18 at 21:44
`split-string-and-unquote` is a great find, thanks for this. However, it only seems to handle double quotes, not single quotes `'foo bar'` and backslashes `foo\ bar`. – Lassi Nov 06 '18 at 21:46
@Lassi: Yes, `split-string-and-unquote` isn't parsing shell syntax. You give it an elisp string, and it returns a list of elisp strings. Which is fine, so long as the user understands the syntax to use. To pass the results to a shell command you would map `shell-quote-argument` over the list. – phils Nov 06 '18 at 22:18
n.b. If you want to guarantee that the string is interpreted exactly as the user's shell (whatever that may be!) interprets it, you would presumably need to talk to a shell process? – phils Nov 06 '18 at 22:29
Oh, noting that you said you would be using `call-process`, you obviously *wouldn't* need `shell-quote-argument`, as you're not going via a shell at all. – phils Nov 06 '18 at 22:32
`split-string-and-unquote` doesn't seem to work on your modified string either. It sounds like you need to write a shell parser for this if you need full shell syntax support. You might be able to escape ' and \_ split and unescape them. I don't know enough shell syntax to know if that will always work though. – John Kitchin Nov 07 '18 at 01:10
The full Unix shell syntax is extremely intricate, so I don't think it's a good idea to even try to support all of it (users won't even understand most of it and people are often surprised when they accidentally bump into the less-known features). I think it makes sense in many situations just to have the argument quoting without the other syntax (pipes, redirects, events, history, etc. etc.) – Lassi Nov 07 '18 at 09:30
split-string-shell-command is now available since version 28.1 :) – Adam Oudad Nov 13 '22 at 08:56

score 2 · Answer 2 · answered Nov 07 '18 at 09:21

Since there doesn't appear to be a function for the job, I wrote one:

(defun shell-command-line-to-argument-list (command-line)
  (let (args arg inquote)
    (with-temp-buffer
      (insert command-line)
      (goto-char (point-min))
      (while (not (eobp))
        (cond ((looking-at "\\s-+")
               (cond ((not inquote)
                      (when arg (push arg args))
                      (setq arg nil))
                     (t (setq arg (concat arg (match-string 0))))))
              ((looking-at "['\"]")
               (let ((ch (match-string 0)))
                 (cond ((not inquote)
                        (setq inquote ch)
                        (setq arg (or arg "")))
                       ((equal inquote ch)
                        (setq inquote nil))
                       (t (setq arg (concat arg ch))))))
              ((or (looking-at "\\\\\\(.?\\)") (looking-at "\\(.\\)"))
               (setq arg (concat (or arg "") (match-string 1)))))
        (goto-char (match-end 0)))
      (when inquote (error "Missing closing quote"))
      (when arg (push arg args))
      (nreverse args))))

(shell-command-line-to-argument-list "")
nil

(shell-command-line-to-argument-list "    ")
nil

(shell-command-line-to-argument-list "  'foo bar'baz  '' \"abcd efg\" \\ ghij\\ kl")
("foo barbaz" "" "abcd efg" " ghij kl")

(shell-command-line-to-argument-list " foo\\")
("foo")  ;; This is arguably a bug, but not a serious one.

(shell-command-line-to-argument-list " \"foo\\\" ")
;; Error: Missing closing quote

I'd like to contribute this to Emacs or to a relevant library in MELPA instead of keeping it around in my unrelated Emacs package. Any thoughts on what would be a good place for it?

split-string-shell-command is now available since version 28.1 :) — Adam Oudad, Nov 13 '22 at 08:55

score 1 · Answer 3 · answered Nov 13 '22 at 08:54

1

Since version 28.1, you can use split-string-shell-command which does just that.

(split-string-shell-command "program arg1 arg2 \"long argument with spaces\" arg\\\"3")
> ("program" "arg1" "arg2" "long argument with spaces" "arg\"3")

answered Nov 13 '22 at 08:54

Adam Oudad

121
3

Split command line into a list of arguments

3 Answers3