1

I have few directories whose names looks like the following:

enter image description here

I want to remove the newline from the end recursively.

I checked Recursively rename directories

I also checked Remove newlines in file names

The solution it suggests is:

find -name $'*\n*' -exec rename  $'s|\n| |g' '{}' \;

But in my case find -name $'*\n*' returns nothing. If I remove the $ it can find the directories

% find . -name '*\n*'
./second?
% find . -name '*\r*'
./third?
./first?

However, when I run find . -name '*\n*' -exec rename $'s|\n| |g' '{}' \; it does not rename the directory. I also tried find . -name $'*\n*' -exec rename $'\n' ' ' {} \; from Recursively remove newline in file names. It is also not renaming the directories.

What can I do?

Ahmad Ismail
  • 2,678
  • 2
    On Linux, with GNU tools, find -name $'*\n*' works for me and finds filenames with hard newlines. -name '*\n*' is the same as -name '*n*' and finds names with an n... What shell are you actually running? You tagged this with both [tag:bash] and [tag:zsh], and they should both support $'...'. Are you sure it's the actual newline character you have there, and not something different? What do you get if you run ls --quoting-style=shell-escape (with a recent-ish GNU ls) or printf "%q\n" * (in Bash or Zsh) in the directory with those dirs? – ilkkachu Apr 11 '21 at 21:45
  • I'd remove the leading $ from both strings, i.e. find . -depth -name '*\n*' -exec rename 's|\n|_|g' {} \; – Chris Davies Apr 11 '21 at 21:46
  • 1
    Also, for the commands with rename, you need to know if you have the Perl one, or the util-linux one... What does rename --version say? (see also https://unix.stackexchange.com/a/510583/170373 and links therein) – ilkkachu Apr 11 '21 at 21:47
  • @ilkkachu printf "%q\n" * shows first$'\342\200\251' second$'\342\200\251' third$'\342\200\251' – Ahmad Ismail Apr 11 '21 at 22:56
  • @ilkkachu rename --version says /usr/bin/rename using File::Rename version 1.10 – Ahmad Ismail Apr 11 '21 at 22:58
  • @blueray, ah-ha. Well, that's not the regular newline then, but something completely different. Thanks for the clear clarifications. – ilkkachu Apr 11 '21 at 23:57
  • perl rename only renames a file if $_ is changed by the rename script, so you can just use find . -type d -exec rename 's/[\n\r]/ /g' {} +. directory names with newlines and/or carriage-returns in them will be renamed, directories without will be ignored. – cas Apr 12 '21 at 00:38
  • 1
    or find . -type d -exec rename 's/[\n\r]|\342\200\251/ /g' {} + – cas Apr 12 '21 at 00:42

2 Answers2

3

You said that

printf "%q\n" * shows first$'\342\200\251' second$'\342\200\251' third$'\342\200\251'

The $'\342\200\251' there isn't the regular newline, it's the Unicode U+2029 PARAGRAPH SEPARATOR, encoded in UTF-8. The output there has the bytes in octal; in hex, they would be e2 80 a9.

That's why find -name $'*\n*' doesn't match it. Without the dollar, -name '*\n*' would be the same as just -name '*n*' matching the letter n in second. The patterns -name takes are just shell glob patterns, where \ makes the next character non-special. E.g. \* matches just the single asterisk, while * matches anything. n isn't special, so \n is just the same as it.

Given we now know what it is, we can get rid of it in the same way we would of a newline. With the Perl rename (e.g. the one using File::Rename), this should just remove them:

find . -depth -name $'*\342\200\251' \
       -execdir rename -v $'s|\342\200\251||g' '{}' +

You need -depth for leaves to be renamed before branches they're on and -execdir (not standard but pretty common) for rename to be called from within the directories containing the files to rename with only the basenames of the files.

... rename -v 's|\342\200\251||g' should work too, since it's a Perl expression and Perl does interpret backslash escapes itself, too.

The command you had, has rename $'s|\n| |g' which would replace with a space, but since you have the characters at the end of the names, that would also be confusing.

Stéphane Chazelas
  • 544,893
ilkkachu
  • 138,973
3

With zsh:

autoload zmv # best in ~/.zshrc
zmv -v $'(**/)(*[\n\u2029]*)(#qD)' $'$1${2//[\u2029\n]}'

To remove both newlines (U+000A) or the paragraph separator character (U+2029).

Or:

zmv -v $'(**/)(*[[:cntrl:]]*)(#qD)' $'$1${2//[[:cntrl:]]}'

to remove all control characters.

Whether U+2029 will be classified as cntrl or not will depend on the system though. It is on Ubuntu 20.04, but not on FreeBSD 12.2 for instance. Run [[ $'\u2029' = [[:cntrl:]] ]] && echo yes to check on your system.

Stéphane Chazelas
  • 544,893
  • OP here. This is actually a better solution. But, as I have already accepted an answer, I do not want to change it. Moreover, @ilkkachu taught me to use printf "%q\n" * to find what the real control operator is so I think he deserves some appreciation. – Ahmad Ismail Apr 12 '21 at 12:58