How to change directory in shell script running in zsh shell

Question

I am working on a shell script (shell used, its version and OS are below). The script runs commands in a local git repository/directory with has several tags. The script outputs some string from greping files in a certain directory which is selected based on the repository tag name.

zsh --version
zsh 5.9 (x86_64-apple-darwin22.0)
OS: System Version: macOS 13.6.3
+
using oh-my-zsh, https://ohmyz.sh/

The script changes directory (cd command) to the git local repository. Then it gets the tags in an array by MY_ARRAY=($(git tag)). Then for each repository tag it does git checkout $my_tag --quiet to that tag. Then cd in one of the directories (directoryA) of the repository. Now it issues a pwd to check the working directory; the output is /git-repository/directoryA. At this moment the pwd output is as expected (/git-repository/directoryA). Now, while in /git-repository/directoryA, the script does LIST_OF_SUBDIRECTORIES=($(ls -d */)) to get the array of the sub-directories of directoryA; the output is LIST_OF_SUBDIRECTORIES is cw-events/ cw-metric-alarm/ eventbridge-scheduler/ msk/ secrets-manager/ ses/ sns-topic/ sqs/ ssm-params/ xlate/. Now for each of this subdirectories (a for loop is used), the script SHOULD cd to a subdirectory in the array LIST_OF_SUBDIRECTORIES like this:

if echo "$my_tag" | grep -q -E "$MY_SUBDIRECTORY"; then
          echo "MY_SUBDIRECTORY is $MY_SUBDIRECTORY"
          set -x
          cd $MY_SUBDIRECTORY 2>&1 | tee -a log_file.txt
          set +x
          echo "PWD Working Directory is $(pwd)"

The output of echo "MY_SUBDIRECTORY is $MY_SUBDIRECTORY" is MY_SUBDIRECTORY is cw-events/; the if condition is true so I would expect for the script to cd in cw-events/ but the above echo "PWD Working Directory is $(pwd)" is outputing /git-repository/directoryA.
The script starts with #!/bin/zsh. Trying in #!/bin/bash I receive some errors.
I found some other questions on https://blog.kubesimplify.com/how-to-change-directory-in-shell-scripts or Script to change current directory (cd, pwd) but actually I do not want the directory to be changed in the parent shell but in the child shell.
Why the script while in /git-repository/directoryA does not cd in /git-repository/directoryA/subdirectory-of-A?

Example of output: The example I can give fast is the output of running the script:

FIRST-SUB-Directory is /git-repository/directoryA
LIST_OF_SUBDIRECTORIES is cw-events/ secrets-manager/ sqs/ ssm-params/
MY_SUBDIRECTORY is cw-events/
+/path/to/script/script.sh:44> cd cw-events/
+/path/to/script/script.sh:44> tee -a log_file.txt
+/path/to/script/script.sh:45> set +x
PWD Working Directory is /git-repository/directoryA

Thank you.

Please give us a minimal example that reproduces the problem. We cannot really help debug code you don't show us, but you should be able to give a self-contained example that we can run to reproduce the issue. SO have a nice guide for this: https://stackoverflow.com/help/minimal-reproducible-example — terdon, Jan 19 '24 at 16:01

score 4 · Answer 1 · edited Jan 20 '24 at 08:53

This line is obviously wrong:

cd $MY_SUBDIRECTORY 2>&1 | tee -a log_file.txt

The left-hand side of a pipe runs in a subshell, meaning that internal changes (variables, current directory, etc.) don't affect the main shell. See CD with redirect to logger does not work for more explanations and advice on logging. Although (except for cd -/cd -2/cd +2... when navigating the directory stack) cd doesn't emit output except if it fails, so logging its output is pointless anyway.

If you just change the line to cd $MY_SUBDIRECTORY (or better cd -- $MY_SUBDIRECTORY), changing the directory does work, at least for the part of the script you posted. If you still end up in the original directory later in the script, you might be doing something else that creates a subshell (in some part of the code that you haven't posted).

Beyond that, I don't see anything obviously wrong, but there are several things that are more complicated than they need to be. I recommend doing things in a simpler way, so that there's less risk of mistakes.

You use echo and set -x and | tee … to log things. One is enough. set -x is fine in most circumstances. Piping to tee is tricky because it affects the behavior of the script (creating a subshell, changing file descriptors, hiding errors).
echo "$my_tag" | grep -q -E "$MY_SUBDIRECTORY" checks whether the value of my_tag¹ contains the value of MY_SUBDIRECTORY as a substring, assuming that the value of MY_SUBDIRECTORY doesn't contain extended regular expression operators (such as . or + common in git tags). There's a simpler way to check that:
```
if [[ $my_tag = *"$MY_SUBDIRECTORY"* ]]; then …
```
LIST_OF_SUBDIRECTORIES=($(ls -d */)) can be written just LIST_OF_SUBDIRECTORIES=( *(-/) ), using zsh's / glob qualifier, which allow filtering globs by file type and other characteristics. Here combined with - so the check of type is done after symlink resolution like */ does. This doesn't put a slash at the end of each element. You can use …=( *(-/M) ) if you do want a slash, or for this special case of listing just directories you can use …=( */ ), but you probably don't actually need the slashes anyway. Add the N glob qualifier if you want to get an empty list rather than a fatal error if there are no subdirectories:
```
LIST_OF_SUBDIRECTORIES=( *(N-/) )
```

^{¹ Strictly speaking, any line in the output of echo "$my_tag" as grep matches one line at a time and echo will also expand \n into a newline character (thankfully, git tags should be guaranteed not to contain backslashes)}

I did the change proposed, that is from "cd $MY_SUBDIRECTORY 2>&1 | tee -a log_file.txt" to "cd $MY_SUBDIRECTORY" and all works as expected. Thank you for the link and the other detailed explanations. — YAZ84, Jan 20 '24 at 05:01

Stéphane Chazelas · Accepted Answer · 2024-01-20T10:32:13.420

In cmd1 | cmd2, cmd1 and cmd2 run concurrently so they have to run in separate processes.

In many shells, both run in child processes even when they're builtin (such as cd). In zsh, like in ksh, cmd2 would run in the current shell process if it was builtin, but cmd1 always runs in a child process.

So in your cd ... | tee ..., cd runs in a child process and only changes the working directory of that short-lived child process.

zsh has tee's functionality builtin though, so you shouldn't need to do that. You can do:

cd -- $MY_SUBDIRECTORY >&1 >> file.log 2>&2 2>> file.log || exit

When a fd is redirected more than once for writing, zsh redirects it instead via a pipe to a process performing a tee-like function in the background.

Here, we're redirecting fd 1 to what it's already redirected to (>&1) which normally would be a no-op but here expresses that we want to keep stdout going there, then redirect it in append mode to file.log. We do the same for fd 2 (stderr). That's another improvement over your cd 2>&1 | tee where tee would bundle the normal and error messages of cd on stdout.

Another improvement is that the exit status of cd is preserved. I've added a || exit to make sure we don't carry on with the rest of the script if cd failed.

Note that cd only outputs something on stdout in things like cd -,cd -2, cd +2 when navigating the directory stack which shouldn't be the case here, so you can get away with just cd ... 2>&2 2>> file.log.

Also note that neither that nor your tee approach would redirect the xtrace output. It goes to stderr, but it's not output by cd itself, you'd need to redirect a command group inside which cd is run to capture that xtrace output. Something like:

set -x # or set -o xtrace
{
  cd -- $MY_SUBDIRECTORY
} 2>&1 2>> file.log 
set +x

Rather than doing set -x, set +x where you'd see a trace for set +x, you can also enable the xtrace option locally in a function or anonymous function:

() {
  set -o localoptions -o xtrace
  cd -- $MY_SUBDIRECTORY
} 2>&2 2>> file.log

Or using a helper function:

trace() {
  set -o localoptions -o xtrace
  "$@"
} >&1 >> file.log 2>&2 2>> file.log
trace cd -- $MY_SUBDIRECTORY || exit

Or using functions -t which enables tracing for a function like in ksh:

trace() {
  "$@"
} >&1 >> file.log 2>&2 2>> file.log
functions -t trace
trace cd -- $MY_SUBDIRECTORY || exit

Beware echo expands escape sequences such as \n, \uXXXX, \0123... by default and accepts options. For it to output something as-is followed by a newline, you need:

echo -E - $something

Using printf (compatible with sh) or print (compatible with ksh and more versatile) is generally preferred though:

printf '%s\n' "$something"
print -r -- "$something"

Also note that pwd is a builtin command that prints the contents of $PWD, so $(pwd) is a bit pointless.

print -r "PWD Working Directory is $PWD"

Would make more sense.

thank you for the detailed answer. It is useful for me and I am sure it will be useful for others. PS: I cannot cast a vote (upvote) as I do not have enough reputation. — YAZ84, Jan 22 '24 at 07:40

How to change directory in shell script running in zsh shell

2 Answers2