12

I use the command line heavily and over the years have migrated from bash to zsh as a daily driver shell. I usually use a slightly customized oh-my-zsh environment, but some systems are on prezto; the differences are not large.

The most productive plugins I've been using for zsh are zsh-syntax-highlighting and history-substring-search, and lately I've been using the very powerful fzf plugins for pulling up history.

Now, I'm finding one of the biggest pain points that remain for me in the command line is command argument reordering. Quite often I try to run a command

command very/long/filesystem/path/to/argumentA another/filesystem/or/network/path/argumentB

and realize I've got the order backwards.

Another even more common situation is when we do any "manual deployment" workflow: First you compare the new stuff with the real stuff, e.g.

diff /opt/app/static/www/a.html /home/user/docs/dev/src/a.html
cp /home/user/docs/dev/src/a.html /opt/app/static/www/a.html

Ok, ok, last example (this one has several steps), no more I promise. Perfect real world example right here. Let's get cracking with some file listing with sweet human sizes:

find /pool/backups -type f -print0 | xargs -0 ls -lh > filelisting

I want to size sort and pick some out interactively:

sort -rhk5 filelisting | fzf -m > /these/are/the_chosen

Nice, that works! Oh but I need just the paths now, but don't want to re-run find:

cut -d ' ' -f 10 /these/are/the_chosen

The output is garbage, because we encountered a setback with ls -lh getting frisky with spaces. But I've got a strategy: Join the contiguous spaces. Let's opt for tr -s to squeeze space chars, no need for a regex here. Though, tr requires stdin:

cat /these/are/the_chosen | tr -s ' ' | cut -d ' ' -f 9- > filelist

By this point, we're feeling the pain of cutting and pasting arguments around in commands. My choices here are always between awkward alternatives: I can move the long path or i can move the commands. With either move, I have to either reach over for the mouse to copy it, or i have to type it again in the new spot. I can't win. With a command line, even navigating around is cumbersome, and extremely so without word hopping hotkeys set up.

I can't even use my mouse to jump around rapidly on a shell prompt! (Hey, does anyone know of a shell that supports mouse events?) So, this is the inefficiency that I want to abolish. I want to eliminate the friction of grabbing a shell object (such as a valid path string) and move it freely left and right while I prototype out monster pipelines. If I had that feature I could spam that 5 times to shove the path to the left of the cut & flags, then construct the rest of the pipeline organically.

I believe the lack of line editing power is what the issue is here. In the very first example where I want to transpose the first 2 args, I can create a trivial shell script that perhaps I'd call cpto that inverts the arguments and delegates to the cp command. But I don't want to have to do that, and it would not help me in the general case, like in the third example.

I'd like to be able to reorder the arguments that I've entered using a simple key combination, like I can do for various types of lists if I'm in Vim with plugins like sideways.

Does such a plugin exist for zsh? Does such a plugin exist for any other shells?

If not, how difficult would it be to implement for zsh? I think that the zsh-syntax-highlighting plugin proves that it should be possible to tokenize arguments. Indeed the shell knows how to fetch individual arguments from history: https://stackoverflow.com/a/21439621/340947

The pain point is so severe and common that I'm liable to write a simple script to bind to a hotkey that grabs the last entry in history and swaps the last 2 args for me, and runs that. But that would not be as ideal as having a line editor operation so that the swap can be done interactively rather than committing to run the command.

Perhaps an improvement on that could be injecting !:0 !:2 !:1 (which zsh nicely auto expands for me) but there are plenty of problems with this also: (1) it won't work without already having attempted to run the wrong command. More than half the time I want to swap args after catching myself after having typed an incorrect command, and (2) often there are flags that were used which that snippet would fail to account for.

I've also seen the approach shown here which is fine but remains tremendously unsatisfying as the keystrokes need to be repeated a lot for long paths, and the Ctrl+Y behavior only recalls the most recent item that was cut, rather than hold a stack of them. It's good to know, but practically useless to me.

For completeness' sake, the tactic taken now is to use whatever suitable key combo to delete words to erase the shorter of the commands to reorder, reposition the cursor, use the mouse to copy the deleted argument from terminal output, and paste it back in. Ordinary folk don't bat an eye at this but it makes me die a little every time I do it because I cannot stop thinking about how easy it would be for the computer to do this task for me, and the injustice that I feel having to reach my hand over to the mouse.

Steven Lu
  • 2,282
  • 3
    I've found the default binding Alt+t to work both in bash and zsh to transpose the most recent two words.

    This is cool for simple args like flags (which are usually order independent! ha!), but when given a path, this does the (possibly useful, but largely not) transposition of the last 2 dirs in the path, not the entire paths themselves. And even if it could group by actual Word, it'd probably fail to properly abide by paths with spaces in them entered with escaped spaces.

    – Steven Lu Jan 09 '20 at 21:41
  • Note that it is actually the shell that's actually tokenizing the arguments to pass to the system API. The system APIs for starting processes only takes an executable path and list of strings; except for system(), where the system will actually just spawn the default shell (/bin/sh) to tokenize the string and call back to the proper process API. – Lie Ryan Jan 10 '20 at 06:35
  • 1
    You mention being adept in this reordering in vim itself. Have you put your shell/read line in vi mode already? You can edit the current command line in EDITOR by hitting v and reorder arguments the way you're used to. – kojiro Jan 10 '20 at 12:29
  • @LieRyan Yes. I understand this and did not state it in the question; your explanation is a very good one to supplement the question. This is also why I would prefer that the shell or a plugin for the shell perform the task of shuffling arguments, because the parsing of the command string into arguments is somewhat nontrivial and should be left to the shell to evaluate the tokenization. – Steven Lu Jan 10 '20 at 16:04
  • @kojiro I was also tempted to mention what you mentioned in my question, but I don't usually use this functionality because it's a pretty heavy context switch. I think that it is powerful to be able to edit the command line in vim, in which this operation as well as many others can be performed (sort of ... I think i'd need to add more intelligence to the vim config to recognize the buffer as zsh filetype), but I'm really looking for getting this done with the ZLE. – Steven Lu Jan 10 '20 at 16:07
  • the Ctrl+Y behavior doesn't even recall the full stack of things deleted with Ctrl+W Really? In bash it always works, control-w accumulates as long as you don't hit any other keys in between. So a long path with spaces in it might take multiple ctrl-w, but you can still yank it as one. Or get the cursor to the space between the args and ctrl-k to kill to end of line (or repeated alt-d to delete word if there are later args you want to keep), then ctrl-a, ctrl-right and you're in position to yank as the new first path arg. – Peter Cordes Jan 10 '20 at 20:02
  • @PeterCordes I confirmed that ctrl+w/ctrl+y works as described for me only with zsh -f meaning oh-my-zsh also appears to be breaking its intended behavior. These issues with omz are piling up and starting to be somewhat annoying. I will renew efforts to switch completely to prezto. And... I'm sure it all works fine with bash too. I just don't spend much time in bash at interactive shell. – Steven Lu Jan 10 '20 at 20:22
  • Ok, I wondered if you were just doing it wrong or if there was some difference from bash. I'd guess that ctrl+k would still work to move the last part of a line earlier. (I don't use ZSH myself. I used to be a sysadmin and valued the ability to be comfortable in default bash, and by now I'm very good with it. I'm sure I'm missing out on some things for some cases and maybe I'll try zsh one day :P) – Peter Cordes Jan 10 '20 at 20:27
  • @PeterCordes I recommend checking out the FZF tool. It can integrate with bash. Supremely powerful for recalling history and finding file paths. The more I use it the less I need zsh's enhancements over bash. – Steven Lu Jan 10 '20 at 20:41
  • 1
    Just an idea, since it sounds like you are already a heavy fzf user: would it be enough to have a function (also bound to a keystroke?) that takes all the arguments from the last command, and pipes them through fzf, and lets you multi-select the ones you want to keep, in the order you want to keep them? (In my experience, the output of fzf is in the order that the items were selected.) – iconoclast Jan 10 '20 at 21:04
  • 1
    @iconoclast I like your out-of-the-box thinking, that is a clever alternate way to approach it (similar to opening the command as a buffer in vim). It's really a bit more heavy-handed than the immediate behavior I'm looking for. But, it would be well suited to some other situations where heavy buffer editing is desired. – Steven Lu Jan 10 '20 at 21:20

6 Answers6

12

In zsh, by default all the widgets that operate on words including the transpose-words one bound by default to Alt+T in emacs mode work on words that are defined as sequences of alnum+$WORDCHARS characters.

The default value of $WORDCHARS has *?_-.[]~=/&;!#$%^(){}<>, so includes /, so should be fine for you to transpose paths as long as those paths don't include characters outside of that. That won't work for paths that contain things like :, @, ,... or are quoted though.

But you could use the select-word-style framework to change the definition of word on-demand.

If you add:

autoload -U select-word-style
zle -N select-word-style
bindkey '\ez' select-word-style

to you ~/.zshrc, then upon pressing Alt+Z, you'll get the choice:

Word styles (hit return for more detail):
(b)ash (n)ormal (s)hell (w)hitespace (d)efault (q)uit
(B), (N), (S), (W) as above with subword matching
?

After pressing "return for more detail":

(b)ash:       Word characters are alphanumerics only
(n)ormal:     Word characters are alphanumerics plus $WORDCHARS
(s)hell:      Words are command arguments using shell syntax
(w)hitespace: Words are whitespace-delimited
(d)efault:    Use default, no special handling (usually same as `n')
(q)uit:       Quit without setting a new style

so pressing S would allow you to transpose two shell words (so including those containing quoted spaces or command substitutions...) with Alt+T (or delete one with Ctrl+W, move back one with Alt+B, etc).

See

info zsh select-word-style

for details (assuming the zsh documentation has been installed on your system (zsh-doc package on Debian and derivatives)).

You'll find a section there that looks like it has been especially written for you which you can adapt to specify how you want transpose-words to behave whenever the cursor is on a filename or in-between words, etc:

Here are some examples of use of the word-context style to extend the context.

   zstyle ':zle:*' word-context \
          "*/*" filename "[[:space:]]" whitespace
   zstyle ':zle:transpose-words:whitespace' word-style shell
   zstyle ':zle:transpose-words:filename' word-style normal
   zstyle ':zle:transpose-words:filename' word-chars ''

This provides two different ways of using transpose-words depending on whether the cursor is on whitespace between words or on a filename, here any word containing a /. On whitespace, complete arguments as defined by standard shell rules will be transposed. In a filename, only alphanumerics will be transposed. Elsewhere, words will be transposed using the default style for :zle:transpose-words.

For instance, with:

autoload -U select-word-style
zle -N select-word-style
bindkey '\ez' select-word-style
select-word-style normal
zstyle :zle:transpose-words word-style shell

transpose-words would work with shell words always while all other word widgets would use the normal definition of word, and you could still use Alt+Z to change it (for widgets other than transpose-words).

  • Thank you so much for showing me this! It does seem based on this that I can configure what a word is for my purposes (and then probably can make a keybind that can do really basic magic on shell-words as I described in my question) but the concept of word is singular, meaning I'll just need to restore the word notion back to the way it was at the end of my routine, so my e.g. delete-word binding will continue to work as I'm used to (e.g. one path segment at a time). – Steven Lu Jan 09 '20 at 22:28
  • @StevenLu, as you can see in the quote of the manual I included, you can also select different word styles for different widgets and also different word styles depending on context. – Stéphane Chazelas Jan 09 '20 at 22:30
  • This documentation was pretty hard to find. Indeed I needed the zsh-doc package. Thanks again. – Steven Lu Jan 09 '20 at 23:04
  • I tried a bunch of things but was unable to make transpose-words do anything correctly for arguments that look like "multi word arg" or multi/word/arg. transpose word is always ignoring the quotes and slashes and transposing alphanumeric words, which is not what I'm trying to do. – Steven Lu Jan 09 '20 at 23:49
  • @StevenLu, see edit. Try it first with zsh -f. If it works with -f but not without, possibly you have a oh-my-zsh plugin that conflicts with select-word-style. – Stéphane Chazelas Jan 10 '20 at 07:21
  • You are right! WIth zsh -f shell words do work as I intend (even with escaped spaces being treated right) with transpose-words on the default Alt+T bind. So yes something in oh-my-zsh here is breaking this. Will be straightforward to address. I do want one more tweak and it would be perfect. I want the cursor to follow the thing that got transposed back. This should be a very simple function that moves the cursor that I can run afterward. Then I can make a corresponding bind that moves to the next shell-word and does the transpose, for going the other direction. – Steven Lu Jan 10 '20 at 16:12
  • zsh-syntax-highlighting will cause problem if it is not sourced at the end of .zshrc – RNA Apr 02 '20 at 14:57
  • One of these days i'm going to take a serious crack again at this. it will be time well spent... – Steven Lu Dec 02 '20 at 09:15
3

I did realize that zsh's vim mode is a bit more advanced than I realized before. There is an a text object which corresponds to argument. So I would be able to take advantage of some muscle memory by typing a sequence such as EscdaaBPiSpace. But you can see also that 8 keystrokes is far from ideal. Given that this operation requires plugins to achieve elegantly in Vim, the same would be the case here for the zsh line editor.

Furthermore putting zsh in vim mode does break the bindings that I already have, though they should be easy to replicate. Sometimes it would indeed be nice to have vim bindings in the shell. But I'm still leaning towards the stance that having to deal with the shell possibly being in insert mode is not worth the trouble.

Steven Lu
  • 2,282
  • Note to self: the approach I'm gonna take is (1) see about implementing zle plugin that moves arg under cursor leftward (2) check compatibility of https://github.com/softmoth/zsh-vim-mode with everything, hopefully everything will all work together – Steven Lu Jan 09 '20 at 23:25
  • 1
    To keep your existing bindings or emacs mode, you can always just bind a single key to vi-cmd-mode. That can be escape or something else. Zsh's vim/emacs modes is not a purely either/or choice, you can have a mix. – okapi Jan 10 '20 at 10:31
  • Nice. That means that no custom work should be needed to retain the customizations done that assumed emacs mode (which would be most of the mappings) – Steven Lu Jan 10 '20 at 15:59
  • 1
    For foo bar baz. your sequence results in foo ba zbar. I think you mean to use a instead of i, but that still results in 2 contiguous spaces after foo. It would be better to do daaBhP, so you don't need to add or remove spaces and rather use the one that comes with the paste. An alternative that doesn't use the zsh-specific daa could be BhD, so BhDBhP. – JoL Jan 10 '20 at 22:52
  • I admit I didn't test the sequence but it illustrates the main point which is 8 keystrokes compared to the desired single keystroke. – Steven Lu Jan 11 '20 at 07:13
2

I faced a similar issue a number of times, and don't believe there is a clean solution available. You should be able to do some hackery to get this done. The most hackerish approach is probably to just dump the command to history without executing it by prefixing it with echo and then discarding !:0, after which your above solution would work.

The better approach would be hooking into ZSH's version of readline (zle if I remember correctly). It's a lot more powerful than regular readline and you can probably manipulate it in place without ever writing the bad command to history. I would suggest looking at existing plugins that expand the line automatically in-place as a starting point. A good example is this plugin: https://github.com/wazum/zsh-directory-dot-expansion (which expands ... to ../.. as you type). In fact, looking at that repo, you'd probably be able to get a plugin for this logic by changing 1-2 lines from the author's original script.

Note that you wouldn't be auto-expanding the zle as the user types, but react to some user key sequence (similar to alt+T combo you mention). So your plugin would be more similar to tab-completion than simple auto-expansion.

  • Welcome to Unix! "prefixing it with echo and then discarding !:0" This is awesome because this is definitely also my go-to strategy when the command is a potentially destructive one such as filesystem operations. I appreciate the tip, but you know as well as I do that it is not a satisfactory answer. Having to do an additional 1-offset calculation in my head to get the history expansion indices right is not my idea of a solution. I do agree that a ZLE plugin is the way to go for this. I will look into writing this. But I'd really like to avoid that if somebody has already written one. – Steven Lu Jan 09 '20 at 22:18
  • Sorry if it's not clear, I wasn't advocating the first solution, merely mentioning it as the quickest option. The second solution would be superior imo. The tricky part is that you wouldn't be able to auto-expand like the author's original plugin does, but instead would need to listen for user to trigger a key-sequence similar to alt+T you use, so it would be more similar to tab completion plugin actually than auto-expansion. – Alexander Tsepkov Jan 09 '20 at 22:22
  • Yeah I gotta dig into it a bit to see how much manual parsing i'd need to do to get it functioning correctly. I was also considering checking if the zsh-syntax-highlighting plugin does something clever to do what it does. I should be good as long as I can call out to something to do the argument tokenizing for me. – Steven Lu Jan 09 '20 at 22:23
  • I should be able to combine the tricks from here https://stackoverflow.com/a/14099674/340947 with a trivial zle widget example to implement this. Will post back if I figure it out. – Steven Lu Jan 10 '20 at 00:06
  • 2
    @AlexanderTsepkov: would : be better than echo? It acts like # except that you don't lose things like access to the arguments through alt+. – iconoclast Jan 10 '20 at 20:57
0

In Bash, press Ctrl-X Ctrl-E to open the currently entered command line in your $EDITOR. Edit it as you like, then save and quit to execute it.

For instance, if your default editor is Vim, <C-x><C-e>f dE$p:wq will transform your initial command line into your desired command line and execute it. Of course, if you have a plugin like sideways.vim, you can use that, too.

This is a more general facility than just rapidly reordering arguments, so it’s useful for lots of transformations. Note that you can enter this mode while starting at a blank command line, too.

If you decide that you want to cancel this operation while in your editor, either exit your editor with non-zero status code (Vim: :cq) or just replace the command with an empty string and save and quit.

The technical name for this operation is edit-and-execute-command. You can re-bind the key combination in your .inputrc, or learn more in man bash.

You asked about Zsh; it looks like similar functionality is available, and may be enabled by default in certain Zsh configuration frameworks.

wchargin
  • 1,091
-1

To solve the specific case you mention about path, you wouldn’t need to use a complicated “switch order”, you’d need recent path history management like fzf to fuzzy search history. Zsh archwiki should mention about dirs, you should check it.

I find it very often I cd to a path and then need to do some actions on something inside that path which currently I’m out of the path. Path history management will push the effectiveness more like with fzf.

Tuyen Pham
  • 1,805
  • Well, there are a lot of different things I might want to do while I am entering a command on the shell... yes, if I am constructing paths, fzf is a very useful and helpful tool for navigating deep paths and I use it a lot. My pain point though isn't that path navigation is a problem (which it was, typing and tab completing and rinse and repeat; fzf helps make it faster, and it's pretty much near optimal efficiency) -- it's that there are common situations that I'll edit into the original question when I want to transpose entire path args. As powerful as fzf is, editing paths takes many steps. – Steven Lu Jan 12 '20 at 21:12
-1

Answer Still Under Construction

OK I took another crack at it today with a new helper (GPT4), and I got very far with it already. Take a gander at what we have already:

function move-current-arg-left {
  local -a args
  local buffer="${LBUFFER}${RBUFFER}"
  printf "buf: >>%q<<\n" "$buffer" >> ~/zsh_word_splitting_log.txt
  args=(${(z):-"$buffer"})

Finding the index of the current argument based on the cursor position

local idx local length=0 for i in {1..${#args[@]}}; do length=$(( $length + ${#args[i]} + 1 )) if (( length >= ${#LBUFFER} )); then idx=$i printf "idx is %d: >>%q<<\n" "$idx" "${args[idx]}" >> ~/zsh_word_splitting_log.txt break fi done

Swapping the current argument with the previous one if it's not the first argument

if (( idx > 1 )); then local temp=$args[$idx] args[$idx]=$args[$((idx-1))] args[$((idx-1))]=$temp

# Finding the new cursor position
local new_cursor_pos=0
for i in {1..$((idx-1))}; do
  new_cursor_pos=$(( $new_cursor_pos + ${#args[i]} + 1 ))
done
new_cursor_pos=$(( $new_cursor_pos + ${#LBUFFER} - $length + ${#args[idx]} + 1 ))

# Reconstructing LBUFFER and RBUFFER
LBUFFER=${(j: :)args[1,idx-1]}
RBUFFER=${(j: :)args[idx+1,-1]}
LBUFFER=&quot;$LBUFFER ${args[idx]} &quot;
CURSOR=$new_cursor_pos

fi }

zle -N move-current-arg-left bindkey "^X" move-current-arg-left

So far this proves the technique works.

https://imgur.com/a/DiASUwb

Backlog:

  • Set cursor position to follow moving arg
  • Implement other direction
Steven Lu
  • 2,282