Pages

· vm

Last time: the kernel gives you a table, and that table cannot have a row for every byte. A 64-bit keyspace, one entry per byte, and the table is larger than the memory it describes. So we don't translate bytes. We translate chunks. That's the grid ASLR was snapping to. (A Table)

Pages

Let's talk about the size of the thing we're translating, because it's the first design decision and it constrains everything downward. You cannot translate individual bytes. Do the arithmetic: a 64-bit address space, one table entry per byte, and the table is unfathomably larger than the memory it describes. It's absurd. So we chunk them up.

In the virtual world, those chunks are called pages. In physical memory, the equally-sized slots they land in are called frames, sometimes "page frames," or "physical pages," which is the term CS:APP prefers. Not buffers. A buffer is a different thing entirely. Page in, frame out. Same size on both sides, which is what makes the mapping work: a page slots into any frame, Fully associative.

How big is the page ? Typically 4 KiB, which is the historical default and the number every textbook quotes. Apple Silicon uses 16 KiB, which is exactly the grid ASLR was snapping to, 0x104720000 and 0x104c54000 were both multiples of 0x4000. Modern systems also support "huge pages" of 2 MiB or 1 GiB for workloads that touch enormous contiguous regions and would otherwise drown in translation overhead.

But why that size range? Two competing forces, pushing in from opposite directions. Pages want to be big, because bookkeeping cost is per-page. Halve the page size and you double the number of table entries you have to store and walk. Big pages also amortize disk transfers: reading from disk has a huge fixed cost per operation, so once you've paid to go there at all, you want to come back with a lot.

Pages want to be small, because you waste the leftovers. Allocate a page for a 40-byte object and the remaining 16,344 bytes are gone. And when memory is tight, you'd rather evict at fine granularity than throw out 16 KiB to reclaim space for 40 bytes.

Four to sixteen kilobytes is where those pressures balance out for typical workloads. That band is not a folklore number someone copied from a textbook. People actually measured it, and the honest result is that there is no one globally optimal page size. Alanko and Verkamo (1983) ran production programs through a working-set simulator and got two kinds of jobs: some want small pages and a large window, some want the reverse. What Intel shipped on the 386 was still 4 KiB, because that sat in the 512 B–4 KiB range machines of that era already used. IBM's System/360 Model 67 — one of the first production DAT / virtual-memory machines — already used 4096-byte pages in the mid-1960s. Later work keeps arguing the base should move with RAM: Weisberg and Wiseman (2009) simulated SPEC2000 and recommended 16 KiB, which is exactly what Apple Silicon did. A 2022 SIGARCH note makes the same case for raising the minimum, not just bolting on huge pages.

Now, splitting an address. This part is genuinely elegant.

If a page is 16 KiB, that's $2^{14}$ bytes, so the low 14 bits of any address are the position within its page. Not by convention — arithmetically. Those 14 bits count 0 through 16,383, which is exactly one page's worth of bytes. Everything above them identifies which page.

So every virtual address has two fields:

   virtual address
   ┌──────────────────────────┬────────────────────┐
   │          VPN             │        VPO         │
   │   virtual page number    │   offset in page   │
   └──────────────────────────┴────────────────────┘

VPN: virtual page number. VPO: virtual page offset. The CPU doesn't compute these, and the instruction doesn't encode them. They're just which bits you look at. The address 0x104720000 is a VPN and a VPO the same way the number 1,432 is "fourteen hundreds and thirty-two."

And on the physical side, the identical split: PPN (physical page number) and PPO (physical page offset). PPO always equals VPO. Always. Translation only ever changes which page, never where in the page, because pages are aligned and the same size on both sides. Byte 900 of a page is byte 900 of that page no matter which frame it's sitting in.

Which means the entire operation is: look up the VPN to get a PPN, then glue the untouched VPO onto the end.

Page table & kernel

That lookup needs somewhere to look. It's the page table: an array of page table entries, PTEs, one entry per virtual page in the address space. Each PTE is a small fixed-size record (8 bytes on x86-64). It always contains a valid bit (also called the Present bit). The rest of the entry is interpreted according to that bit:

that gives every virtual page one of three possible states at any moment:

The single most important fact about page tables is this:

Every process has its own private page table.

That is what makes the opening demo work. (Same address but two values) Parent and child each have a separate table. Both tables contain an entry for the virtual page that holds value. After the first write triggers copy-on-write, those two entries point at two different physical frames. Same VPN but different PPN. Same printed address and completely different memory.

The kernel owns the tables. It builds them when a process is created, updates a PTE when a page is brought in (or written for the first time under copy-on-write), and destroys the whole table when the process exits. The MMU does not construct mappings. It reads PTEs on every translation. (It does write two status bits later — Accessed and Dirty — so the kernel can tell what was used and what needs writeback. That's bookkeeping, not "the MMU decided your layout.")

So how does the MMU even find the table? The table itself lives in ordinary physical memory. A dedicated register holds the physical address of the root of the current process's page table. On x86-64 that register is CR3. The textbook name is PTBR (page-table base register). Apple Silicon's equivalent is TTBR.

Once the register is loaded, finding any PTE in this one-level picture is index arithmetic:

PTE address = PTBR + (VPN × size of a PTE)

Real 64-bit machines do not store one giant array like this. The table is a tree. That's the next problem. For now this is the whole mechanism: one array, VPN is the index.

And now the part that makes multitasking possible: the kernel rewrites that register when it switches to a different address space. Switching from your browser to your editor doesn't move any memory around. It changes one register, and instantly every virtual address in the machine means something different. Two threads of the same process share a table, so that switch does not touch CR3.

e.g..

Let's walk a real address through, with small numbers so the bit-fiddling is visible. This is the worked example from CS:APP.

Setup: 14-bit virtual addresses, 12-bit physical addresses, 64-byte pages.

64 bytes is $2^6$, so the low 6 bits are the offset. Everything above is the page number.

The address: 0x03d4.

In binary, across 14 bits:

   0 0 0 0 1 1 1 1 | 0 1 0 1 0 0
   └── VPN ───────┘ └─ VPO ────┘

The top 8 bits are 0b00001111 = 0x0F. That's the VPN. The low 6 bits are 0b010100 = 0x14. That's the VPO.

So: VPN = 0x0F, VPO = 0x14.

The MMU indexes the page table at entry 0x0F. Say that entry is valid and holds PPN 0x0D.

Now glue:

   PPN 0x0D        VPO 0x14
   0 0 1 1 0 1 | 0 1 0 1 0 0
   └──────────── 0x354 ─────────┘

Physical address: 0x354.

Notice that PPN is concatenated with VPO, shifted up into the high bits and the offset dropped in below. This trips me up constantly. 0x0D + 0x14 is 0x21, which is not the answer.

And that's it. A table lookup, followed by a bit-concatenation. This runs on every single memory access your machine performs, billions of times a second, so it has to be nearly free of any perf cost.

Right. Go home.

Except the page table lives in memory. To read one byte you first go to memory for the PTE, then go to memory for the data. Every access became two. That's Can be costly. That's next.