23

One of the most common things shell scripts need to do is to create and manipulate temporary files. Doing so safely is a pain, since you need to avoid name clashes, avoid race conditions, make sure the file has the correct permissions, etc. (See the GNU Coreutils manual and this Signs of Triviality blog post for a more detailed discussion of these issues.) Most Unix-like operating systems solve this problem by providing a mktemp command that takes care of all these gotchas. However, the syntax and semantics of these mktemp commands are not standardized. If you really want to create a temp file both safely and portably, you have to resort to ugly kludges such as the following:

tmpfile=$(
  echo 'mkstemp(template)' |
    m4 -D template="${TMPDIR:-/tmp}/baseXXXXXX"
) || exit

(This workaround exploits the fact that the macro processor m4 is part of POSIX, and m4 exposes the C standard library function mkstemp() which is also defined by POSIX.)

Given all this, why hasn't POSIX standardized a mktemp command, guaranteeing its presence and at least certain aspects of its behaviour? Is this a glaring oversight on the part of the POSIX committee, or has the idea of standardizing a mktemp actually been discussed by the committee and rejected for some technical or other reason?

  • 6
    @zevzek: There's nothing requiring the committee to name the standard command mktemp. They could call it something completely new and so allow it to coexist with the older (and perhaps soon-to-be-deprecated) variants. (Well, for certain values of "soon" measured in years or decades, but still.) – Psychonaut Oct 16 '20 at 09:41
  • 1
    You could at least have pointed to https://unix.stackexchange.com/q/555058/5132 on this WWW site. (-: – JdeBP Oct 16 '20 at 12:59
  • zevzek: there is a set of common functionality between BSD and Gnu mktemp. e.g.: mktemp, mktemp -d, mktemp "${TMPDIR:=/tmp}/tmp.XXXXXXXX", and mktemp -d "${TMPDIR:=/tmp}/tmp.XXXXXXXX" all seem to behave the same for me. – Hal Canary Apr 21 '21 at 17:26

2 Answers2

22

That comes up regularly on the Austin Group mailing list, and I'm not under the impression the Open Group would be opposed to specifying it. It just needs someone to propose something. See for instance this message from Eric Blake (Red Hat, sits on the weekly POSIX meeting) from 2011 (here copied from gmane):

Date: Tue, 10 May 2011 07:13:32 -0600
From: Eric Blake <eblake-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
To: Nico Schottelius <nico-posix-xuaVFQXs+5hIG4jRRZ66WA@public.gmane.org>
Cc: austin-group-l-7882/jkIBncuagvECLh61g@public.gmane.org
Newsgroups: gmane.comp.standards.posix.austin.general
Subject: Re: No mktemp in posix?
Organization: Red Hat
Message-ID: <4DC939FC.2040407@redhat.com>
References: <20110510105051.GA1767@ethz.ch>
Xref: news.gmane.org gmane.comp.standards.posix.austin.general:4151

On 05/10/2011 04:50 AM, Nico Schottelius wrote: > Good morning, > > digging through Issue 7, I haven't found any utility that gives > the ability to create a secure, temporary file that is usually > implemented in mktemp.

echo 'mkstemp(fileXXXXXX)' | m4

will output the name of a just-created temporary file. However, I agree that there does not seem to be any standardized utility that gives mkdtemp functionality, which is often more useful than mkstemp (after all, once you have a secure temporary directory, then you can create secure fifos within that directory, rather than having to wish for a counterpart 'mkftemp' function).

> Is there no mktemp utility by intent or can we add it in the > next issue?

I know both BSD and GNU have a mktemp(1) that wraps mktemp(), mkstemp(), and mkdtemp(). In my inbox, I have record of some off-list email in February of this year regarding some work between those teams to try and converge on some common functionality and to word that in a manner appropriate for the standard, although I can't find any publicly archived messages to that effect. But yes, I think adding mktemp(1) to the next revision of the standard would be worthwhile. I'll try to revive those efforts and actually post some proposed wording.

-- Eric Blake eblake-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org +1-801-349-2682 Libvirt virtualization library http://libvirt.org

In a more recent thread (worth reading), Geoff Clare (from the Open Group):

Date: Wed, 2 Nov 2016 15:13:46 +0000
From: Geoff Clare <gwc-7882/jkIBncuagvECLh61g@public.gmane.org>
To: austin-group-l-7882/jkIBncuagvECLh61g@public.gmane.org
Newsgroups: gmane.comp.standards.posix.austin.general
Subject: Re: [1003.1(2013)/Issue7+TC1 0001016]: race condition with set -C
Message-ID: <20161102151346.GB24416@lt.loopback>
Xref: news.gmane.org gmane.comp.standards.posix.austin.general:13408

Stephane Chazelas <stephane.chazelas-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote, on 02 Nov 2016: > > At the moment, there's no way (that I know) to create a temp file > reliably with POSIX utilities

Given an m4 utility that conforms to the 2008 standard, there is:

tmpfile=$(echo 'mkstemp(/tmp/fooXXXXXX)' | m4)

However, I don't know how widespread support for the new mkstemp() macro is.

-- Geoff Clare <g.clare-7882/jkIBncuagvECLh61g@public.gmane.org> The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

(which is where I learned that trick which you're referring to in your question).

8

The mktemp(1) utility has been introduced by OpenBSD-2.1, originally released on June 1, 1997.

It is meanwhile available on various platforms including Solaris and Linux.

If you like to have it in the POSIX standard, there is a big chance to get this done if you make a proposal for the related standard text.

Note that POSIX does not do "invention", but since this program already exists on at least one certified platform, this is not a problem.

Everybody can make a proposal at: https://austingroupbugs.net/ just apply for an account and make a "bug report"

If you like to make such a proposal, your homework would be to investigate the least common denominator of all available implementations (i.e. check what it common to *BSD, Solaris and Linux) and write a man page that describes the least common denominator. Do not forget to mention unspecified or undefined behavior that may exist due to incompatible behavior of different implementations.

If you make a good homework and your text proposal is available to the end of this month and does not need many corrections for an approval by the Austin core group teleconference, there is a good chance that this may be included in the next standard version (Issue-8) that is expected to be approved next year by the Austin group and later by IEEE. We just released draft 1.1 for the new standard version internally. If it does not make it into Issue-8, it will take some years to get a chance for Issue-9.

To give you an impression on what effort is needed for a non-trivial program that is going to be added to POSIX, I wrote a proposal for the POSIX standard text for gettext(1), xgettext(1), msgfmt(1) and the gettext(3) family together with a student from the Hasso Plattner Institute as part of a seminar from January to March and we did already spend another 3 months in the teleconferences (two per week) of the Austin core group.

schily
  • 19,173
  • 2
    mktemp did not originate in OpenBSD. See my comment in another Q&A here. mktemp originated in HP/UX with a different syntax. Todd C. Miller created a different one for OpenBSD in the mid-90s (copied by FreeBSD and NetBSD) and later made it also available as a standalone utility (www.mktemp.org). That's the one that was typically used on Linux until a (mostly compatible) mktemp utility was added to the GNU coreutils in 2007. – Stéphane Chazelas Oct 18 '20 at 05:47
  • Actually Ultrix 4.0 in 1990 had a mktemp command with the same API as HP/UX's, so it would need further research to pinpoint exactly where that one initially came from. – Stéphane Chazelas Oct 18 '20 at 07:44
  • The URL you mentioned does not exist and I cannot find an Ultrix man page in the net. But it seems that HP-UX is no longer relevant and FreeBSD could change it's implementation to use '-pinstead of-t`, so there is a good chance for a standardization. – schily Oct 18 '20 at 12:14
  • 1
    Sorry not sure what happened with that link. Should be there. See also there. A mktemp utility appeared in HP/UX 7.0 in 1989. What do you mean by it's no longer relevant? AFAICT, HP/UX is still being maintained. – Stéphane Chazelas Oct 18 '20 at 14:41
  • It has become silent about HP-UX in the past and IIRC next year, Intel will no longer sell Itanium chips and I did not hear about a port to a new architecture. – schily Oct 18 '20 at 18:05
  • I think I agree with you, HP-UX' future doesn't look great from today's standpoint. – Stéphane Chazelas Oct 18 '20 at 19:40
  • @schily, does the core group really care about the least common denominator between the systems you mentioned? Of them, only Solaris is POSIX-certified. *BSD and (GNU/)Linux are not. (By the way, thanks so much for posting this answer! Apart from the factual error about the origin of mktemp, this is the only answer that provides concrete steps on how to contribute to the standardization process.) – Psychonaut Oct 19 '20 at 07:57
  • @Psychonaut you would need to clearly distinguish between all common behavior and incompatible behavior and for the parts that are currently in conflict, it helps to ask the related mailtainers whether they are willing to implement a behavior that is compatible to the standard proposal. – schily Oct 22 '20 at 13:45
  • @schily I understand that; my question was rather whether the core group wants to hear about all those commonalities and differences in the behaviour of non-POSIX systems like *BSD and GNU/Linux. – Psychonaut Oct 22 '20 at 16:42
  • @Psychonaut When we are diskussing/preparing a new standard text, we usually run various test to find out differences and common behavior on as many platforms as possible. If this information is inlcuded in a "bug report" already, this helps a lot to reduce the time to deal with such a report. BTW: The tests we run include *BSD and Linux. – schily Oct 22 '20 at 20:49