What are high memory and low memory on Linux?

Question

I'm interested in the difference between Highmem and Lowmem:

Why is there such a differentiation?
What do we gain by doing so?
What features does each have?

@hiro, you mean "HIGHMEM" is "kernel virtual address" as described by ldd3. I agree with you. it is confusing, ldd3 defined "LOWMEM" "HIGHMEM", also defined "kernel virtual address" "kernel logic address". they are the same thing, but has different name. that is the "beauty" of software, it is so description-language-depended. — steve, Dec 05 '15 at 00:06

score 84 · Accepted Answer · edited Jan 08 '15 at 17:42

84

On a 32-bit architecture, the address space range for addressing RAM is:

0x00000000 - 0xffffffff

or 4'294'967'295 (4 GB).

The linux kernel splits that up 3/1 (could also be 2/2, or 1/3 ¹) into user space (high memory) and kernel space (low memory) respectively.

The user space range:

0x00000000 - 0xbfffffff

Every newly spawned user process gets an address (range) inside this area. User processes are generally untrusted and therefore are forbidden to access the kernel space. Further, they are considered non-urgent, as a general rule, the kernel tries to defer the allocation of memory to those processes.

The kernel space range:

0xc0000000 - 0xffffffff

A kernel processes gets its address (range) here. The kernel can directly access this 1 GB of addresses (well, not the full 1 GB, there are 128 MB reserved for high memory access).

Processes spawned in kernel space are trusted, urgent and assumed error-free, the memory request gets processed instantaneously.

Every kernel process can also access the user space range if it wishes to. And to achieve this, the kernel maps an address from the user space (the high memory) to its kernel space (the low memory), the 128 MB mentioned above are especially reserved for this.

¹ Whether the split is 3/1, 2/2, or 1/3 is controlled by the CONFIG_VMSPLIT_... option; you can probably check under /boot/config* to see which option was selected for your kernel.

edited Jan 08 '15 at 17:42

Josh Kelley

217

answered Dec 26 '10 at 16:09

wag

35,944
12
67
51

This is old and I'm not sure you're around here. But I want to ask one thing: the reserved 128MB in kernel space (for high memory access), is it all the references of user-space memory area? So, a kernel process can access any user space by refer to this area, right? – Amumu Jul 02 '12 at 04:37
1

Why is it always in 1/4's? I.e. why couldn't it split it 5/1 or something? – mgalgs Aug 29 '12 at 23:48
What exactly does "can directly access" mean here? I mean, isn't the kernel itself accessed via the virtual memory mechanism ? – telenn Oct 08 '12 at 13:07
What is the use of that distinction? – taffer Jan 24 '13 at 11:57
1

I believe that what you say about high/low memory is wrong: I believe that in a pure 32bit system, the kernel can access the full 3GB of user space directly (kernel can access kernel space and user space). However when you have a PAE kernel things get more complex, now you have more than 3GB of RAM, each process can be 3GB, and you can not access the whole of user space directly. This is where high mem and that 128MB of memory in kernel space, comes in. With a 64bit kernel it gets simpler again, no high men, as all user space is accessible from the kernel. – ctrl-alt-delor Dec 02 '14 at 22:00
2

@mgalgs ¼, 2/4 and ¾ was just a set of default choices that were exposed. Since 2007, one can also select 5/16ths and 15/32ths. If you know to edit which #define line, you can pick an almost arbitrary split of your own. – jørgensen Jul 28 '15 at 15:13
@ctrl-alt-delor: highmem can be required even without PAE. The kernel can only access physical memory by mapping it into virtual memory. Ideally, it linearly maps it all, but it also wishes to share address space with user mappings to avoid invalidating the TLB when transferring to/from user mode. By default, each user mode process gets 3GB VM space for its mappings (to parts of physical memory, shared mem, etc.). The kernel then only has 1GB VM left to use, so <1GB of physical memory can be direct-mapped (low, "Normal" mem); the rest must be accessed via temporary mappings ("High" mem). – GreenReaper Mar 09 '19 at 03:24
@GreenReaper can you explain more? I don't understand the comment. Also I think on a 32 bit system, the VM limit per process was 3GB (for a ¾ partitioning), and for a non-PAE kernel the system RAM limit was around 3GB. – ctrl-alt-delor Mar 09 '19 at 10:33
My point was that the difference between low and high memory is whether the kernel has it directly mapped; it may become unable to do so as soon as 1GB, in large part because the kernel gives 3GB virtual addresses to user mode. PAE just makes high memory more likely, because you can address more RAM, and page tables become larger, further squeezing direct mappings. Physical address space is limited to 4GB, but devices e.g. video cards using mem-mapped I/O occupy physical address space too; so without PAE, which ups that to 64GB, you won't be able to access 4GB RAM, either as low or high mem. – GreenReaper Mar 10 '19 at 14:44
To give a real-world example: I just upped a VM to 2GB RAM, but found I could only use 1GB and a bit because I had no highmem features enabled, so was limited to direct mapped memory that was in turn limited by the kernel virtual address space. Instead of enabling highmem, which entails overhead for creating the mappings, I switched to having a 3GB kernel / 1GB user split. Now the kernel could directly map all RAM (still as "low" memory), at the cost of only allowing user programs 1GB virtual addresses each. A regular distro kernel would have had highmem in that situation, with or without PAE. – GreenReaper Mar 10 '19 at 14:50
it s not clear from the answer whether the talking is about virtual addresses or real phisycal addresses. As i understand any linux code works only via virtual addresses. Sometimes mapping is tricky so it happens so that virtual address is the same as physical address. I feel it should be described very clear every time we speak about any particular memory entities. – Alex May 15 '20 at 18:11

score 34 · Answer 2 · answered Dec 17 '10 at 20:55

The first reference to turn to is Linux Device Drivers (available both online and in book form), particularly chapter 15 which has a section on the topic.

In an ideal world, every system component would be able to map all the memory it ever needs to access. And this is the case for processes on Linux and most operating systems: a 32-bit process can only access a little less than 2^32 bytes of virtual memory (in fact about 3GB on a typical Linux 32-bit architecture). It gets difficult for the kernel, which needs to be able to map the full memory of the process whose system call it's executing, plus the whole physical memory, plus any other memory-mapped hardware device.

So when a 32-bit kernel needs to map more than 4GB of memory, it must be compiled with high memory support. High memory is memory which is not permanently mapped in the kernel's address space. (Low memory is the opposite: it is always mapped, so you can access it in the kernel simply by dereferencing a pointer.)

When you access high memory from kernel code, you need to call kmap first, to obtain a pointer from a page data structure (struct page). Calling kmap works whether the page is in high or low memory. There is also kmap_atomic which has added constraints but is more efficient on multiprocessor machines because it uses finer-grained locking. The pointer obtained through kmap is a resource: it uses up address space. Once you've finished with it, you must call kunmap (or kunmap_atomic) to free that resource; then the pointer is no longer valid, and the contents of the page can't be accessed until you call kmap again.

Thanks Gilles for the answer.. But, i am not still able to get the whole concept. Could you please be a bit more simpler without reducing the information in it? — Navaneeth Sen, Dec 18 '10 at 11:15

score 20 · Answer 3 · edited Oct 27 '14 at 19:36

This is relevant to the Linux kernel; I'm not sure how any Unix kernel handles this.

The High Memory is the segment of memory that user-space programs can address. It cannot touch Low Memory.

Low Memory is the segment of memory that the Linux kernel can address directly. If the kernel must access High Memory, it has to map it into its own address space first.

There was a patch introduced recently that lets you control where the segment is. The tradeoff is that you can take addressable memory away from user space so that the kernel can have more memory that it does not have to map before using.

Additional resources:

jp48 · Answer 4 · 2021-01-28T00:52:03.660

For the people looking for an explanation in the context of Linux kernel memory space, beware that there are two conflicting definitions of the high/low memory split (unfortunately there is no standard, one has to interpret that in context):

"High memory" defined as the totality of kernel space in VIRTUAL memory. This is a region that only the kernel can access and comprises all virtual addresses greater or equal than PAGE_OFFSET. "Low memory" refers therefore to the region of the remaining addresses, which correspond to the user-space memory accessible from each user process.

For example: on 32-bit x86 with a default PAGE_OFFSET, this means that high memory is any address ADDR with ADDR ≥ 0xC0000000 = PAGE_OFFSET (i.e. higher 1 GB).

This is the reason why in Linux 32-bit processes are typically limited to 3 GB. Note that PAGE_OFFSET cannot be configured directly, it depends on the configurable VMSPLIT_x options (source).

To summarize: in 32-bit archs, virtual memory is by default split into lower 3 GB (user space) and higher 1 GB (kernel space).

For 64 bit, PAGE_OFFSET is not configurable and depends on architectural details that are sometimes detected at runtime during kernel load.

On x86_64, PAGE_OFFSET is 0xffff888000000000 for 4-level paging (typical) and 0xff11000000000000 for 5-level paging (source). For ARM64 this is usually 0x8000000000000000. Note though, if KASLR is enabled, this value is intentionally unpredictable.

"High memory" defined as the portion of PHYSICAL memory that cannot be mapped contiguously with the rest of the kernel virtual memory. A portion of the kernel virtual address space can be mapped as a single contiguous chunk into the so-called physical "low memory". To fully understand what this means, a deeper knowledge of the Linux virtual memory space is required. I would recommend going through these slides.

From the slides:

This kind of "high/low memory" split is only applicable to 32-bit architectures where the installed physical RAM size is relatively high (more than ~1 GB). Otherwise, i.e. when the physical address space is small (<1 GB) or when the virtual memory space is large (64 bits), the whole physical space can be accessed from the kernel virtual memory space. In that case, all physical memory is considered "low memory".

It is preferable that high memory does not exist at all because the whole physical space can be accessed directly from the kernel, which makes memory management a lot simpler and efficient. This is especially important when dealing with DMAs (which typically require physically contiguous memory).

See also the answer by @gilles

score 8 · Answer 5 · answered Jan 29 '13 at 20:04

HIGHMEM is a range of kernel's memory space, but it is NOT memory you access but it's a place where you put what you want to access.

A typical 32bit Linux virtual memory map is like:

0x00000000-0xbfffffff: user process (3GB)
0xc0000000-0xffffffff: kernel space (1GB)

(CPU-specific vector and whatsoever are ignored here).

Linux splits the 1GB kernel space into 2 pieces, LOWMEM and HIGHMEM. The split varies from installation to installation.

If an installation chooses, say, 512MB-512MB for LOW and HIGH mems, the 512MB LOWMEM (0xc0000000-0xdfffffff) is statically mapped at the kernel boot time; usually the first so many bytes of the physical memory is used for this so that virtual and physical addresses in this range have a constant offset of, say, 0xc0000000.

On the other hand, the latter 512MB (HIGHMEM) has no static mapping (although you could leave pages semi-permanently mapped there, but you must do so explicitly in your driver code). Instead, pages are temporarily mapped and unmapped here so that virtual and physical addresses in this range have no consistent mapping. Typical uses of HIGHMEM include single-time data buffers.

score 4 · Answer 6 · answered Dec 17 '10 at 11:00

4

As far as I remember, "High Memory" is used for application space and "Low Memory" for the kernel.

Advantage is that (user-space) applications can't access kernel-space memory.

answered Dec 17 '10 at 11:00

Gert

9,994

Zheng Gao · Answer 7 · 2016-10-12T18:01:13.750

Many people have said that the low memory is for the operating system. This is usually true but does not have to be. High memory and low memory are just two parts of the memory space, but in Linux system low memory is only for kernel and high memory for user processes.

According to the "Dinosaur book (operating system concepts)", we can place the operating system in either low memory or high memory. The major factor affecting this decision is the location of the interrupt vector. Since the interrupt vector is often in low memory, programmers usually place the operating system in low memory as well.

What are high memory and low memory on Linux?

7 Answers7

Linked