21

I frequently come across the term "unix-like" on many web sites.

There is no standard; it is just in the way it behaves.

But if I were to develop a kernel from scratch, what would make it considered "unix-like"?

Basically, what are the things that make written code like Unix?

iconoclast
  • 9,198
  • 13
  • 57
  • 97

2 Answers2

19

There is no standard; it is just in the way it behaves.

I believe most "unix-like" OS's do make a very serious effort to adhere to the POSIX standard, which is overseen by the Open Group, who also control the Single UNIX Specification which defines a "true UNIX". The former is the core of the later.

So there is, in fact, a standard which defines the practicalities of unix-like operating systems. Have a look at the list of "fully" and "mostly" compliant OS's at the end of the wikipedia article on POSIX.

There are some obvious reasons linux in particular might not be considered fully compliant or certifiable by the Single Unix Specification (SUS), but these are not because any given linux system is necessarily incompatible with it. The wikipedia article summarizes the spec this way:

SUSv3 totals some 3700 pages, which are thematically divided into four main parts:

Base Definitions (XBD) - a list of definitions and conventions used in the specifications and a list of C header files which must be provided by compliant systems. 84 header files in total are provided.

Shell and Utilities (XCU) - a list of utilities and a description of the shell, sh. 160 utilities in total are specified.

System Interfaces (XSH) - contains the specification of various functions which are implemented as system calls or library functions. 1123 system interfaces in total are specified.

Rationale (XRAT) - the explanation behind the standard.

The standard user command line and scripting interface is the POSIX shell, an extension of the Bourne Shell based on an early version of the Korn Shell.

Other user-level programs, services and utilities include awk, echo, ed, vi, and hundreds of others. Required program-level services include basic I/O (file, terminal, and network) services.

A test suite accompanies the standard. It is called PCTS or the POSIX Certification Test Suite.

Additionally, SUS includes CURSES (XCURSES) specification, which specifies 372 functions and 3 header files. All in all, SUSv3 specifies 1742 interfaces.

This obviously refers specifically to a lot of userland components (such as the shell) that are simply not part of the linux kernel. So there is no way that linux.org et. al. can have the kernel alone certified -- in this sense, it is not an operating system at all. They could, of course, try to certify some particular system using the kernel, but this would be meaningless in light of the general distribution scheme: the kernel and the people who maintain it are independent of the people who maintain the userland core (GNU) who are independent of the people who maintain actual assembled OS distributions (Debian, Fedora, etc).

I suppose Debian or Fedora themselves could engage in the certification process (so, e.g., RedHat Enterprise could become a "certified unix"), but this begs the question that such is actually desirable. I would presume that the major reason for SUS systems is to run (commercial scale, non-consumer) software written for such, which is simply not the linux niche -- people doing this will pay thousands of dollars per license for the OS including lots of support, etc., because they are also paying tens or hundreds of thousands of dollars per license for whatever additional software it is they want to run on the system. Linux and the other outliers, on the other hand, have pursued design goals beyond simple compliance for commercial purposes, and there are various example of this, e.g. (from http://en.wikipedia.org/wiki/STREAMS ):

STREAMS was required for conformance with the Single UNIX Specification versions 1 (UNIX 95) and 2 (UNIX 98), but as a result of the refusal of the BSD and Linux developers to provide STREAMS,[citation needed] was marked as optional for POSIX compliance by the Austin Group in version 3 (UNIX 03).

An interesting accommodation that highlights the point that SUS and The Open Group != linux, != BSD, etc.

goldilocks
  • 87,661
  • 30
  • 204
  • 262
  • 3
    Note that being certified is different from complying. For example, it's impractical for the Linux Foundation to get each kernel version certified, because of cost and the rate of development. But that doesn't mean it (the kernel) doesn't fully or mostly comply. – strugee Aug 22 '13 at 05:02
  • 2
    @strugee The POSIX standard doesn't apply to or care about the kernel. What is standardized are commands (shell, ls, cat, ...) and APIs (what the libc provides, threads). What makes a Linux based distribution Unix like are mostly GNU components (commands and glibc). The kernel is outside the scope of certification/compliance. – jlliagre Aug 22 '13 at 06:28
  • @jlliagre I was talking about the Single UNIX Specification. – strugee Aug 22 '13 at 07:27
  • @jlliagre although I think I may have been slightly confused, see http://unix.stackexchange.com/questions/6171/why-isnt-gnu-linux-sus-v3-compliant – strugee Aug 22 '13 at 07:28
  • 3
    Not to refute but clarify (I'm repeating this from my comment on illuminÉ's answer): the standards are "what", the kernel is "how". I have only read bits and pieces of the standards, but I do not think they refer to a "kernel" at all (it's just the "system"). So WRT certification and compliance: it is "what" a kernel/OS does, not "how" it does it. – goldilocks Aug 22 '13 at 08:57
  • 3
    @strugee The Single Unix Specification apply to userland interfaces. The kernel is of course eventually involved but my comment was about your statement about the Linux Foundation getting kernels certified. A kernel cannot be certified, it is missing all the components the certification process interact with. What can be certified (or at least attempted to be made as compliant as possible) is an Operating System, i.e. what is commonly called a distribution in the Linux community. – jlliagre Aug 22 '13 at 09:12
  • @jlliagre ah. let me amend my comment: it's impractical for the combination of gnu/linux to be certified because of the quick release of kernels and other components. would that be accurate? – strugee Aug 22 '13 at 18:54
  • 2
    @strugee That would be possible reason however is not that much the quick rollover that matter but the fact there is no commitment to ensure no incompatible changes are added. For example Solaris 10 is compliant, guarantee this compatibility and there has been a dozen updates since it was Unix03 certified. Moreover, the Gnu/Linux combination try to be as compliant as it can/want but not more. It (almost) never attempted to be certified because it is both an expensive process and it wouldn't comply anyway because some requirements are missing (deliberately) and some extensions are incompatible – jlliagre Aug 23 '13 at 06:30
  • The quick rollover might be an issue but I think it would be incorrect to say that is the reason. @jlliagre Also since "GNU/Linux" is not one organization but two, and a certifiable (linux) system would require components from both, it is hard to see who would apply for cerification (or why they would). As you say, it would more likely to be the distributions, but I doubt that really gibes with their general business. I've added some stuff to the answer about this. – goldilocks Aug 23 '13 at 07:10
  • 1
    Gnu/Linux based OSes are not any more a niche market. There are certainly Red-Hat, Suse, Oracle Linux and other distribution customers that run Linux on thousands to million dollars hardware and software platforms. They might be interested for these distributions to be certified. The fact the Open Group took a step in that direction by marking optional a feature the kernel group decided never to implement is an interesting move. – jlliagre Aug 23 '13 at 08:41
  • 1
    @jilliagre : I did not mean that it was not worthwhile because linux is a "niche market". The growth of e.g. Redhat came to them as linux and not as a pure unix. It is not clear to me that their existing clientele would react positively to complete compliance with the Open Group -- if not, then this would definitely not be worthwhile. Today, it is probably more the pure unixes which occupy a niche market (and it may be a diminishing one). – goldilocks Aug 23 '13 at 14:04
  • I (apparently incorrectly) understood your last edit to mean linux is in a niche market which is not the one of commercial scale, non-consumer software and was only commenting that point. (btw my username is jlliagre, not jilliagre) – jlliagre Aug 25 '13 at 23:34
  • a few Linux distros did manage to get the certification Is there a Linux distro that's UNIX certified? – phuclv Oct 23 '18 at 04:06
15

To expand on the first answer about POSIX, to understand what "unix-like" means, well first one should try to understand what exactly is UNIX. Looking at the documentation from the Open Group, which owns the Unix trademark, you'll find details about the Single UNIX specification's evolution - here's UNIX03:

The UNIX 03 Product Standard is the mark for systems conforming to Version 3 of the Single UNIX Specification. It is a significantly enhanced version of the UNIX 98 Product Standard. The mandatory enhancements include alignment with ISO/IEC 9989:1999 C Programming Language, IEEE Std 1003.1-2001 and ISO/IEC 9945:2002. This Product Standard includes the following mandatory Product Standards: Internationalized System Calls and Libraries Extended V3,Commands and Utilities V4, C Language V2, and Internationalized Terminal Interfaces.

UNIX98:

The UNIX 98 Product Standard is a significantly enhanced version of the UNIX 95 Product Standard. The mandatory enhancements include (1) Threads interfaces, (2) Multibyte Support Extension (MSE), (3) Large File Support, (4) Dynamic Linking, (5) changes to remove hardware data-length dependencies or restrictions, and (6) Year 2000 changes. In addition the following optional enhancements are included: Software Administration facilities and a set of APIs for realtime support. This Product Standard includes the following mandatory Product Standards: Internationalized System Calls and Libraries Extended V2,Commands and Utilities V3, C Language, Transport Service (XTI) V2, Sockets V2 and Internationalized Terminal Interfaces. In addition, it may also conform to the Software Administration Product Standard.

UNIX95(my emphasis):

This Product Standard defines a consolidated platform for the support of a wide range of applications originally developed for one of the class of operating systems that were derived from the UNIX Operating System code and/or interfaces originally developed by AT&T, in addition to the facilities provided by the Base Product Standard. It has wider scope than Base. This Product Standard includes the following Product Standards: Internationalized System Calls and Libraries Extended, Commands and Utilities V2, C Language, Transport Service (XTI), Sockets and Internationalized Terminal Interfaces.

Server versions of the standard add Internet Server and IPv6 in some cases.

So of course we see the reference to AT&T Bell Laboratories and the C language is at the heart of what UNIX is: the C language, modular base tools, and the shell and how the kernel, filesystem and other key OS components were designed and implemented.

enter image description here

enter image description here

That's where the book The Design of the UNIX Operating System by Maurice J. Bach becomes invaluable reading because it's historical matters at this point. Of note of course is how this is related to other inventions like the C language indeed. C was developed by AT&T Bell to implement Unix with a language that could be as fast as assembly but portable across different hardware, and a lot of POSIX is an extension to standard C.

Insofar as the kernel itself is concerned, you'll often find a conceptual diagram such as this one to illustrate what a UNIX kernel traditionally was about:

enter image description here

Here are some excerpts from Mr Bach's classic book (1986) which discuss the foundations of the UNIX System V kernel:

However, they [application subsystems and programs] all use lower-level services ultimately provided by the kernel, and they avail themselves of those services via the set of system calls. There are about 64 systems calls in System V, of which fewer than 32 are used frequently. They have simple options that make them easy to use but provide the user with a lot of power. The set of system calls and the internal algorithms that implement them form the body of the kernel[...]

[...] its two major components are the file subsystem and the process subsystem.

Files are organized into file systems, which are treated as logical devices; a physical device such as a disk can contain several logical devices (file systems). Each file system has a superblock that describes the structure and contents of the file system, and each file in a file system is described by an inode that gives the attributes of the file. System calls that manipulate files do so via inodes. [and the buffer pool]

[...] There are two versions of the inode: the disk copy that stores the inode information when the file is not in use and the in-core copy that records information about active files.

The execution of user processes on UNIX systems is divided into two levels: user and kernel. When a process executes a system call, the execution mode of the process changes from user mode to kernel mode: the operating system executes and attempts to service the user request[...]

[...] the philosophy of the UNIX system is to provide operating system primitives that enable users to write small, modular programs that can be used as building blocks to build more complex programs. One such primitive visible to shell users is the capability to redirect I/O.

[...] In addition to servicing system calls, the kernel does general bookkeeping for the user community, controlling process scheduling, managing the storage and protection of processes in main memory, fielding interrupts, managing files and devices and taking care of system error conditions.

If you're interested with the different implementations of kernels in unix-like operating systems, you can also take a look the the FreeBSD implementation(4.4BSD) or at the Mach kernel or look at this comparison of their features.

The more you know about the design of UNIX, the more you understand what happened in the following diagram about the ancestry of UNIX and its history. Mr Bach is talking mostly about System V in his book but he discusses BSD too:

enter image description here

There's more to this than meets the eyes really. For instance, Mac OSX is UNIX03 certified but do you see it connected to any of the pure UNIXes (in red mostly)?

enter image description here

Above you can see how BSD, GNU, Microsoft and diverse individuals contributed to this universe. Even though GNU and ultimately linux have no direct lineage to UNIX, you see that GNU is an effort to re-engineer in the open source world the tools and software from commercial UNIX that had become closed. So looking at the GNU maintained software gives an idea for instance at the initial prototype apps and libraries.

Licensing wars played a role in the evolution(and stagnation sometimes) of UNIX. You can see immediately that UNIXes are lined up according to license type - closed vs. BSD (BSD allows for making the code closed source... see OSX) and GPL which allows Linux and GNU to complement themselves in the copyleft world. Here's the classic map of the linux kernel initially developed by Linus Torvalds, which also reveals what a kernel "can" be in a Unix-like operating system:

enter image description here

This hints at the idea that a "kernel" design type is not what makes the UNIX standard or what defines a unix-like OS. This is evidenced by the fact that many unix-like OSes may have either a monolithic kernel or a microkernel - monolithic was the classical design type for UNIX. In fact, even within pure UNIXes, HPUX has a monolithic kernel whereas AIX uses a microkernel. This debate about design is about performance and is not related to Unix ancestry or identity. On the other hand, there is a traditional conceptual approach to providing services to software, dealing with file systems etc. under UNIX/unix-like operating systems.

I believe such considerations will add context to the OS part of your question.

  • 4
    +1 Some good points here: 1) About the relationship between C and Unix (to add: C was developed by AT&T Bell to implement Unix with a language that could be as fast as assembly but portable across different hardware, and a lot of POSIX is an extension to standard C). 2) That kernel design is independent of standards. Standards are "what", kernels are "how". – goldilocks Aug 22 '13 at 08:50
  • 1
    @goldilocks Thank you, I've added your comment about C verbatim. I've tried to make it clear that kernel considerations are unrelated to the standard. The question assumes that there is something specific about the unix-like kernel yet there is not. On the other hand historically the first Unix kernels might have been in such and such a way. My understanding is limited, but I would assume the kernels have changed a lot because hardware changed a lot since the 70's. What's clear is that the kernel doesn't define Unix/unix-like, thought the linux kernel clearly defines GNU/Linux or Linux. –  Aug 22 '13 at 09:32