5

This is specifically about bash's declare - the general case is pretty exhaustively dealt with in this answer (which mentions "the typeset/declare/export -p output of ksh93, mksh, zsh" but not that of bash).

Given a local/exported/array/assocative-array (but maybe not nameref) variable foo, is the output of declare -p foo in bash guaranteed to be reusable by bash? The official documentation doesn't mention anything like that:

The -p option will display the attributes and values of each name. When -p is used with name arguments, additional options, other than -f and -F, are ignored.

And I looked through the CHANGES, and saw this about functions:

This document details the changes between this version, bash-2.05-beta1,
and the previous version, bash-2.05-alpha1.
...
b.  When `set' is called without options, it prints function definitions in a
    way that allows them to be reused as input.  This affects `declare' and
    `declare -p' as well.

And for a couple of other commands, -p is meant to produce reusable output:

s.  The `shopt' `-p' option now causes output to be displayed in a reusable
    format.
...
u.  `umask' now has a `-p' option to print output in a reusable format.

And Chet Ramey's Bash FAQ has:

Bash-2.0 contained extensive changes and new features from bash-1.14.7.
Here's a short list:
...
most builtins use -p option to display output in a reusable form
    (for consistency)

But nothing I can find about declare -p for variables.

muru
  • 72,889
  • My knee-jerk counter-question would be: What would you need it for? – Kusalananda Feb 15 '24 at 09:35
  • 2
    @Kusalananda I'd parry your counter by quoting https://unix.stackexchange.com/a/558333/70524 which says: "declare -p varname will output a bash statement that defines the variable varname with its current value. Writing this to a file gives you a file that you can source to recreate the variable." :) In fact, that's exactly what I use it for right now in a few places, and I'd like to know whether that is indeed a supported use-case. – muru Feb 15 '24 at 10:33
  • Right back at me! :-) Well, in another, more recent, answer I seem to have found a reason to be more wary, but I can't unfortunately recall the source for that. – Kusalananda Feb 15 '24 at 10:39
  • Does your "be reusable" mean "does not cause any form of error"? If you do declare -p on a read-only variable (like UID or PPID), and then try to evaluate that output (with the variable still in scope), you'll probably get an error saying you can't set the variable's value as it's read-only. Stéphane points out that you will have to ensure that the locale and bash revision is not changed between the declare -p call and when you evaluate the result. – Kusalananda Feb 15 '24 at 10:50
  • No, errors like those with read-only variables, or differing bash versions would be expected. The note on locale is interesting - I think Stéphane used Stéphane=1 as an example of a variable definition that is interpreted differently based on the locale (and also of course other characters in the variable value itself might have different meanings in different locales) – muru Feb 15 '24 at 11:30
  • Hmm, I wonder. It seems to me there might be two questions here: a) is it in practice reusable as input, or is there some gotcha, and b) is it supposed to be reusable, in which case the gotchas would count as bugs. And well, I also suppose c) is it documented as such. For (b), I wonder if it'd be just best to ask the author, considering you already looked and at least (c) seems unclear. It sure seems to me that it's supposed to work that way, but as for the official word, well. – ilkkachu Feb 19 '24 at 09:48

2 Answers2

5

At that answer of mine you're referring to, one of the other points also mentions:

Or in other words, only the ones that use '...' are safe in that regard

Which doesn't include bash's declare -p.

At the time I wrote that answer bash's declare -p did not use $'...' for quoting the values of scalar variables, it did use it for array variables though. That has now changed as I can see 5.2 outputting declare -x a=$'\b' for a scalar variable containing the BS character (see related discussion on the mailing list).

But, in any case older versions did use "..." for quoting the value of scalar variables inside which ` and \ are special and those characters have an encoding that can be found as part of the encoding of other characters in some locales.

The output of declare -p is intended (as some comments in the code as well as statements from the maintainer on the mailing list suggest) if not documented to be reusable but in effect that's only (if at all) in the same version of the same bash shell and in the same locale on the same system (same libc and locale definitions).

Here on Ubuntu 20.04 with bash 5.0.17:

$ a=$'\n\xa3`' bash -c 'declare -p a; echo declare -p a' | LC_ALL=zh_CN.gb18030 bash
bash: line 2: unexpected EOF while looking for matching ``'
bash: line 4: syntax error: unexpected end of file
$ a=$'\n\xa3`uname; : \xa3`' bash -c 'declare -p a; echo declare -p a' | LC_ALL=zh_CN.gb18030 bash
declare -x a="
�\\Linux\""

uname (thankfully harmless) was run when the output of declare -p obtained in a locale using UTF-8 as the charmap was interpreted by bash running in a locale using GB18030 as the charmap.

A number of bugs (see this or this as examples) have been fixed in the past where the quoting was not done properly, or declare -p (or export -p which POSIX requires to output shell code suitable for reinput) alone was including definitions of variables from the environment that could not be mapped to shell variables.

Also note that in bash, what constitutes a valid variable name depends on the locale.

$ locale charmap
UTF-8
$ LC_ALL=fr_FR locale charmap
ISO-8859-1
$ env -i $'\xe9=zzz' LC_ALL=fr_FR bash -c $'declare -p \xe9' | bash
bash: line 1: declare: `�=zzz': not a valid identifier

Byte 0xe9 is é in ISO-8859-1 which is a single-byte [[:alpha:]] so is allowed in variable names, while in UTF-8, it's not even forming a valid character.

Also beware of:

$ bash -c 'a=1; f() { local b=2; declare -p a b; }; f'
declare -- a="1"
declare -- b="2"

The fact that one is global, one is local is not reflected in declare's output, and if both were used inside a function, the resulting variable would end up being local to the function.

bash's declare is obviously shaped after ksh's typeset (bash also has a typeset alias to it). In ksh86 and earlier, typeset -p was to print the typeset output if any to the co-process (aka two-way pipe). It seems it disappeared in ksh88. In ksh93, typeset -p reappeared to print variable definitions.

Current versions of the ksh93 manual have:

-p The name, attributes and values for the given vnames are written on standard output in a form that can be used as shell input. If +p is specified, then the values are not displayed.

But that verbiage only appeared in ksh93t in 2008.

-p was added to bash's declare in 2.0 released in 1996

From the NEWS files from that version:

kk. The `declare' builtin has new options: -a, -F, -p.

(-F incompatible with ksh93's)

And the CWRU/changelog:

3/24
builtins/declare.def

  • new -p option to display variables and their values and attributes declare -p xxx displays attribs and value of var xxx

Which dates the actual implementation on 1995-03-24, so after ksh93's but before ksh93 documenting it producing reusable output.

  • So ... given the limitation of same shell version, locale, and system, it is meant to be reusable? (Based on comments in the code or mailing list posts or personal discussions or something like that?) – muru Feb 19 '24 at 10:52
  • @muru well, the doc doesn't say as much, it only appears to be. – Stéphane Chazelas Feb 19 '24 at 10:54
  • The linked comment in the source code is about functions, and I have noted in my question that the CHANGES file did document that declare -p is supposed to have reusable output for functions. – muru Feb 19 '24 at 11:31
  • https://lists.gnu.org/archive/html/bug-bash/2015-01/msg00017.html is 99% of the way there. So declare -p is "supposed to produce output that can recreate all variables with their attributes", and if the same applies for declare -p name (assuming same shell versions, locale and system and valid variable names) then my question is answered. But that mailing list post also showed one past, but noticeable, difference between the behaviour of declare -p and declare -p name - so one last clarification from me: are there any other major differences between the two wrt variables? – muru Feb 19 '24 at 14:50
  • If you search the mailing list archives, you'll see it's obviously intended to be valid shell code. The fact that it's not documented as such means that although unlikely it could change in the future. The fact that it has had bugs and is locale-sensitive (uses unsafe forms of quoting) would be other reasons to avoid using it with untrusted data. – Stéphane Chazelas Feb 19 '24 at 14:55
  • My use cases would usually be building some variables in one step of something and trying to use them in another step, where the steps run in new shell invocations that are identical in versions/locale/system (e.g., separate targets or even separate lines in a given target in a Makefile, or separate workflow steps in some CI/CD system which use the same environment for execution). The variable names would be trusted, as we define them. The content ... could be untrustworthy. I think I will only use this with bash 5.2 or newer (so not in a current LTS Ubuntu, but possible in Amazon Linux 2023). – muru Feb 20 '24 at 05:23
  • @muru, I've used it myself with things like ssh host zsh <<< $(typeset -p x y; typeset -fp f g)'; f $(g $x) $y'. Not foolproof against malicious actors if locales using BIG5/GB18130... are involved, but safe enough for my use cases. – Stéphane Chazelas Feb 20 '24 at 07:15
0

A lot of Bash features are copied from Ksh. Being the same switch with the same functionality under bash and ksh, then it should be expected that Ksh answer applies to Bash as well.

It is also my personal understanding that the output of declare -p is reusable, and I have not seen a case where it wasn't.

  • And was the implementation of declare -p copied from Ksh? As for personal understanding, that was my understanding as well, but I'd like to see some more definite proof. – muru Feb 15 '24 at 08:49
  • I'm fairly certain I've seen a bug report somewhere where someone reported that declare -p did not in fact produce the correct output. I'm unable to find it now so if it was ever a bug, it may have been fixed in recent (how recent?) releases of bash. – Kusalananda Feb 15 '24 at 09:34
  • Note that in ksh, it's typeset, not declare (though some later versions of some implementations of ksh have added declare as an alias). – Stéphane Chazelas Feb 19 '24 at 10:41