Project 4: Privilege Separation and System Calls

At the end of Project 3, your operating system ran several programs at once, shared the keyboard between them, and gave each one a window on the screen. But while this matches what actual desktop OSes in the 1980s and 1990s could do, there is zero security here: every process is exactly as powerful as the kernel. All processes run at the same privilege level (EL1), they can execute every instruction the kernel can, and they reach the kernel by calling straight into its functions. This isn’t a toy setup: early versions of Windows, Mac OS, and Linux all ran this way, and a single misbehaving program could take down the whole machine. But it’s not a good long-term design, and shared multi-user OSes had long solved this problem by the 1980s.

In this project you will make your kernel actually separate from userspace. You will move your processes to the CPU’s unprivileged level, EL0, where the hardware refuses to let them execute the instructions that control interrupts, timers, and exception vectors. Then you will give them ways to call into the kernel in a structured and secure way via system calls. The instruction you will use for this is called svc, short for “supervisor call”, which is the name IBM gave the very same idea on the System/360 in 1964 (!). Finally, you will build the kernel interface that Unix made famous in the 1970s: processes operate on resources provided by the kernel through file descriptors, small numeric identifiers that let the kernel abstract away the details. Your processes will also start, finish, and wait for other processes through system calls, bringing your OS closer to a modern Unix-like design (e.g., in modern macOS and Linux).

By the end, your OS will boot into a terminal again (this time on the UART console), and you will have a graphical terminal, too. From either terminal, you can start programs—text programs, graphical programs, and a game you did not even compile yourself—and a program that crashes or misuses privileged instructions will be stopped by your kernel rather than taking the machine down with it.

A note on what you already have. This project builds on your exception vector table, context switch, and timer interrupts from Project 2, and on the blocking, keyboard routing, and compositor from Project 3. Most of your Project 3 code survives unchanged; what changes is how processes reach that code. If your Project 3 interrupt handling or blocking is shaky, seek help first: in this project, every system call goes through the same exception path as your timer and UART interrupts, so bugs there will show up on system calls too.

Learning objectives

After completing this project, you will be able to:

Explain why an operating system needs privilege separation between user processes and the kernel. You will watch a handful of misbehaving programs take over or crash your OS when you start the project, and then change your OS to stop them.
Explain why a system call is a protected control transfer. A process can enter the kernel only through an entry point the kernel installed, never at an address of its own choosing.
Explain, step by step, how a user process enters the kernel through a system call and how the kernel returns to it—which registers the hardware changes, which ones your code saves, and where the result ends up.
Explain why a system call interface is a fixed contract (an Application Binary Interface (ABI)) between user programs and the kernel: both sides must agree on system call numbers, arguments, and return values. A program compiled against that contract runs on any kernel that implements the same system call ABI, and you will prove this by running a program that you don’t have source code for.
Explain why the kernel needs its own stack for each process, rather than trusting the stack pointer of a process that can set it to anything it likes.
Explain why pointers must not be passed blindly across the user/kernel boundary: you’ll understand why the kernel must check and copy the data behind every pointer a process gives it, and why it must never hand out its own pointers.
Explain how file descriptors let processes operate on kernel and machine resources safely, by naming them with small integers that the kernel interprets and checks on every use.

CS 1670 vs. 1690/2670

As in the previous projects, all students share the same core project, but students in CS 1690/2670 complete extra work and run their code on real hardware.

CS 1670	CS 1690/2670
Complete quests 1–4 in QEMU (emulated Raspberry Pi).	Complete quests 1–4 in QEMU (emulated Raspberry Pi).
	Complete quest 5 (pipes and a real shell).
	Run your OS on real hardware (Raspberry Pi).

Roadmap

The project has five quests, which build on each other. Quests 1–4 are required for everyone; quest 5 is for CS 1690/2670 only.

Task	What you build	What you learn
1. Who’s in charge?	Nothing, at first: you run programs that break your Project 3 OS. Then a kernel stack for every process, processes that run at EL0, and a kernel that kills a misbehaving process instead of panicking.	Why privilege separation matters; what the hardware enforces at EL0; why the kernel cannot use a process’s stack.
2. Knocking on the kernel’s door	User-side system call stubs (generated from your system call numbers), a system call dispatcher in the kernel, and your first system calls: `yield`, `ticks`, and `getpid`.	What happens, instruction by instruction, when a process makes a system call; what an ABI is.
3. File descriptors	Safe copying to and from user memory, a per-process file descriptor table, and `write`, `read`, and `peek` for the console—including a `read` that blocks inside the kernel.	Why the kernel must not trust user pointers; how file descriptors name kernel resources; what it means to block in the middle of a system call.
4. Tearing down the trampoline	System calls for everything the `F_BASE` trampoline used to do, `spawn`/`exit`/`wait`, terminals that launch programs (in CS 1670, on the serial console and in a window), and a game we compiled for you.	That a system call interface is a complete, stable contract; process creation and termination; zombies and `wait`.
5. A real shell (CS 1690/2670)	Pipes, a shell that runs both in a terminal emulator and on the serial console (talking to each through pipes), terminal emulators that you start from a shell (with a keyboard of their own), and shell pipelines.	How file descriptors compose; how inheritance and EOF make pipelines work.

Read the Background section before you start: it introduces the ideas that all quests rely on. Each quest then opens with its own, shorter, background material. Work through the quests in order; each depends on the one before it.

Assignment Installation

You will keep working in the same private GitHub repository you used for Projects 1–3.

To pull in the new files for this project (new empty source files, the provided programs, and an updated build system), run the following in your project directory:

git pull handout main

If this reports an error about an unknown remote, run:

git remote add handout git@github.com:csci1670/cs1670-f26-projects.git

and then git pull handout main again.

Merge conflicts. As in Projects 2 and 3, git pull may produce merge conflicts in files that contain both your code and ours. Resolve them by keeping your implementation and adding whatever we provide alongside it. The likely places are:

kernel/init.c. We have added extern declarations for the new programs (the misbehaving diab_* programs and snake). Keep them and your init() function.
kernel/memlayout.h. We have added a comment about this project’s new memory region, between the ARM local peripherals and the trampoline. This usually merges cleanly; if it does not, keep your existing definitions—you will remove some of them yourself in Quest 4.
kernel/aarch64.h. We have added constants for CNTKCTL_EL1 at the end of the file. Keep them and any changes of your own.

If you get stuck in a merge, git merge --abort gets you back to where you started; then come to office hours.

Code Structure

As before, you write all the significant code yourself; we provide a build system, a skeleton of (mostly empty) files, and a few programs. Here are the main files that are new or change in this project (each quest’s Files: line has the complete list):

.
├── Makefile                    updated: syscall number header, _start, prebuilt programs
├── kernel/
│   ├── memlayout.h, limits.h   Quests 1, 4: kernel stacks; removing the trampoline
│   ├── exceptions.S            Quest 1: vector entries for exceptions from EL0
│   ├── context.S               Quests 1, 3: SP_EL0; saved kernel contexts
│   ├── interrupts.c, .h        Quests 1-2, 4: exceptions from EL0; kill, don't panic
│   ├── elf.c                   Quest 1: a process's first saved context
│   ├── proc.c, proc.h          Quests 1, 3-5: kernel stacks, fd tables, spawn/exit/wait
│   ├── syscall.c, syscall.h    Quests 2-5: syscall numbers, dispatch, and handlers
│   ├── uaccess.c, uaccess.h    Quests 3-4: checking and copying user memory
│   ├── file.c, file.h          Quests 3-5: file descriptors and the console "file"
│   ├── console.c, console.h    Quests 1, 3-5: read and peek; keyboard focus; open_console
│   ├── compositor.c, .h        Quests 1, 4: windows on demand (init_canvas)
│   ├── scheduling.c, .h        Quests 1, 3-4: killed processes; blocking in a syscall; ZOMBIE
│   ├── klib.c, klib.h          Quest 4: no longer needed
│   ├── init.c                  Quests 1-5: which programs start at boot (boot_programs)
│   ├── userprogs.c, .h         generated by the build: every user program, and find_program()
│   └── pipe.c, pipe.h          Quest 5 (CS 1690/2670): pipes
└── user/
    ├── diabolical/diab_*.c     Quest 1: misbehaving programs (we provide these)
    ├── prebuilt/snake.elf      Quest 4: a program you did not compile (we provide this)
    ├── syscalls.S              Quests 2-5: system call stubs (generated by the build)
    ├── lib/
    │   ├── u_start.c           _start: calls main(), then exit() (we provide this)
    │   ├── u_syscalls.h        Quests 2-5: system call declarations
    │   ├── u_sterm.c, .h       Quest 4: a text console on the serial line (we provide this)
    │   └── u_common.c, .h      Quests 2-4: printf, getc, hasc on top of system calls
    ├── term.c                  Quest 4: your terminal starts programs
    └── ...                     you will also create basic_console.c (Quest 3);
                                gterm.c and sterm.c (CS 1670, Quest 4), or
                                gterm.c and sh.c, and copy vtty.c, gsnake.c,
                                lines.c, upper.c, and count.c (CS 1690/2670,
                                Quest 5)

Three important build system changes.

Auto-generated system call stubs. Previously, our makefile ran a script, user/gen_klib_header.py, that copied every trampoline function address (#define F_...) from your kernel/memlayout.h to user/u_klib.h, so that your user code would know the addresses for the trampoline functions. Now, we do something similar so that our user code can know about each system call number: the script user/gen_syscall_stubs.py finds every syscall number defined in kernel/syscall.h (concretely, every #define SYS_...) and creates a small assembly function user/syscalls.S, which is linked into every program.
Auto-generated user programs: Previously, our init.c contained a bunch of definitions for the start address of each user program (e.g., _binary_user_primecheck_elf_start), which we needed to add to init.c by hand each time we added a program. This was fine for running just a few programs at startup, but, starting in Quest 4, we’ll begin spawning new programs at runtime (i.e., from your terminal) while your OS is running! To do this, we need to be able to look up programs by name. So the build now runs a script, user/gen_user_prog_info.py, which generates kernel/userprogs.c and kernel/userprogs.h: a table of every program the build embeds (from user/, user/diabolical/, and user/prebuilt/), and a function find_program(name) that returns the program’s struct proginfo (its name and the start of its ELF image), or nullptr if there is no such program. You’ll use this in Quest 4 (#include "userprogs.h"). Both files are generated by the build (and ignored by git), so never edit them; and since every program is in the table automatically, you no longer need to add new programs anywhere.
Programs now start at _start, not at main. In actuality, main isn’t the first function of your C program that runs: most OSes wrap it in a function called _start. We now also do this: user/lib/u_start.c contains a three-line _start that calls your program’s main() and then passes its return value to exit(). (It declares main and exit itself, so it compiles whatever your user/lib/u_common.h says.) You will make exit() exit the process properly in Quest 4.

Building and running

Nothing changes: make qemu, make qemu-fb, make qemu-ws, their -gdb variants, and make clean work as they did in Project 3. However, the qemu-verbose target becomes much more useful in this project:

$ make qemu-verbose             # serial console, plus a log line for every exception taken
$ make qemu-verbose 2> int.log  # ...with the log saved to int.log instead

make qemu-verbose asks QEMU to log every exception, including every system call and every return to user mode. It helps you answer questions like “did my process actually run at EL0?” or “did that svc reach my kernel?” The log is long, and it goes to stderr, so the second form is usually more practical: your UART serial console stays in the terminal, and you can search int.log from another one. Appendix A shows you how to read the log.

Debugging in this project is mostly about the boundary between user code and kernel code. Appendix A collects some helpful information, including:

how to read the exception class from ESR_EL1,
what QEMU’s exception log looks like, and
gdb recipes for stepping across an svc and an eret.

Our Syscall ABI is the reference for the system call interface you will build.

Background

A process should not be as powerful as the kernel

Here is what any process on your Project 3 OS can do, because it runs at EL1:

Mask interrupts (msr DAIFSet, #2), and then never give up the CPU. Since no timer interrupt can preempt it, your scheduler never runs again.
Reprogram the timer, so that the interrupt that drives preemption and your compositor never fires.
Replace the exception vector table (msr VBAR_EL1, ...) with its own. From then on, every interrupt and every exception runs the process’s code, at EL1, instead of your kernel’s.
Crash, by executing an invalid instruction. Your Project 2 synchronous exception handler treats every exception as a kernel bug and panics, so one buggy program takes down every other program with it.
Point its stack pointer at nonsense. Your kernel saves an interrupted process’s registers on that process’s stack, so the next timer interrupt makes your kernel write wherever the process pointed sp.

Each of these is only one or two instructions, and you will run programs that do exactly these things and watch what happens to your OS.

The fix is the one we have been alluding to since Project 2: run user processes at EL0. The CPU then enforces rules on the instructions they execute, which no amount of cleverness can get around, because the rules are checked by the hardware on every instruction:

A process at EL0 tries to…	What the hardware does
execute `msr`/`mrs` on an EL1 system register (`VBAR_EL1`, `SCTLR_EL1`, `ELR_EL1`, …), or `eret`	refuses: undefined instruction exception, taken to EL1
mask interrupts by writing `DAIF`	refuses, unless the kernel allows it (ours won’t): trapped system register access, taken to EL1
program the timer (`CNTP_CTL_EL0`, `CNTP_CVAL_EL0`, …)	refuses, unless the kernel allows it in `CNTKCTL_EL1` (ours won’t): trapped system register access, taken to EL1
read the timer’s counter (`CNTPCT_EL0`)	refuses in the same way, unless the kernel allows it in `CNTKCTL_EL1` (ours will: telling the time is harmless)
raise its own privilege level	impossible: there is no such instruction, and `eret` can only reduce privilege

Notice what every row has in common: when a userspace (EL0) process tries something it is not allowed to do, the CPU stops it and transfers control to the kernel. The kernel then decides what to do: kill the process, say. A process cannot escape this with an instruction, because the instruction does not let it choose where in the kernel it lands. (Through memory it still can; see the box below.)

What EL0 does not protect, yet. The hardware rules above are all about instructions and system registers. They say nothing about memory: the Memory Management Unit (MMU) is still off, so a process at EL0 can still read and write any address it likes, including the kernel’s code and data, other processes’ memory, and the device registers.

So privilege separation in this project stops a process from taking over the machine with privileged instructions. It does not yet stop a determined attacker: by writing to memory, a process could overwrite an entry in your kernel’s tables, another process’s saved registers, or the code of your vector table, and get its own code run at EL1 anyway. It could also switch off the timer interrupt through the interrupt controller’s memory-mapped registers. Fixing that needs virtual memory, which you’ll implement in Project 5.

System calls: a protected control transfer

At EL0, a process can no longer call your kernel’s functions directly: they would run at EL0, too, and fail the first time they touched a privileged register. (You will see this happen in Quest 1.) So how does a process ask the kernel to do something for it?

The answer is the same mechanism that stops it: an exception. ARM64’s svc (“supervisor call”) instruction does nothing except take a synchronous exception to EL1. Because it is an exception, the hardware does everything it does for any other exception, which you learned in Project 2: it saves the return address in ELR_EL1 and PSTATE in SPSR_EL1, masks interrupts, switches to the EL1 stack pointer, and jumps to an entry in the exception vector table that the kernel installed.

This is called a protected control transfer: control moves from a less privileged program into more privileged code, but only at entry points chosen in advance by the more privileged code. A process can make a system call whenever it likes, with whatever arguments it likes, but it cannot choose which kernel code runs, and it cannot skip the kernel’s checks.

Going back is the reverse: the kernel restores the process’s saved registers and executes eret, which drops the CPU back to EL0 at the instruction after the svc. You have been writing this half since Project 2: it is restore_context.

Here is the whole round trip, which you will build in Quests 1 and 2:

   user process (EL0)                          kernel (EL1)
   ──────────────────                          ────────────
   n = write(1, buf, 5);
     │  C calls the stub write() with 1, buf, 5 in x0-x2
     ▼
   svc #2  ─── exception ──────────────────►  vector table, "lower EL, sync" entry (0x400)
               hardware: ELR_EL1 = next pc,     │  save all registers as a context
               SPSR_EL1 = PSTATE, mask IRQs,    │  on this process's *kernel* stack
               sp = SP_EL1, jump to VBAR+0x400  ▼
                                              handle_syscall(): ESR_EL1 says "svc #2"
                                                │  -> sys_write(ctx)
                                                │     reads x0-x2 from the saved context,
                                                │     checks them, does the work
                                                ▼
                                              ctx->x0 = 5            (the return value)
                                              restore_context(ctx)
   n == 5  ◄──── eret ──────────────────────  (back to EL0, at the instruction after svc)

From the process’s point of view, write() was a function call that returned 5. From the kernel’s point of view, an exception was taken, handled, and returned from—exactly like a timer interrupt, except that the process asked for it.

How is this different from the trampoline?

Since Project 1, your processes have called the kernel through function pointers stored below F_BASE. That is an unprotected control transfer: the process jumps to a kernel address, and the kernel code then runs with the process’s privilege, on the process’s stack, with whatever interrupt state the process had. Unfortunately, nothing stops a process from jumping into the middle of a kernel function, or to any other address it finds in the kernel.

A system call fixes all of this at once: the kernel decides where execution starts, the kernel runs at EL1 on its own stack with interrupts masked, and the process does not need to know where any kernel function lives. All the process knows is a system call number.

Quest 1: Who’s in charge?

Files: kernel/init.c, kernel/memlayout.h, kernel/limits.h, kernel/proc.h, kernel/proc.c, kernel/context.S, kernel/elf.c, kernel/aarch64.h, kernel/exceptions.S, kernel/interrupts.c, kernel/interrupts.h, kernel/scheduling.c, kernel/console.c, kernel/compositor.c, kernel/kernel.c, user/idle.c, user/diabolical/diab_*.c (provided).

Overview

In this quest, you will first break your OS from Project 3 on purpose, and then make sure the same programs can no longer break it. You will:

Run six misbehaving (“diabolical”) programs on your OS, and observe what each one does to it.
Give every process its own kernel stack, and start every process at EL0.
Handle hardware exceptions that come from EL0, and kill a process that misbehaves, instead of panicking.

At the end of the quest, the diabolical programs will be stopped by the hardware and killed by your kernel (or, in one case, simply made harmless), while the rest of the system keeps running. However, even most of your well-behaved programs will also be killed, for reasons that will set up the rest of the project.

Why are we doing this?

Hardware-assisted privilege separation is the foundation of all operating system protection mechanisms. A kernel can implement the cleverest scheduling policy or the most careful access checks, but they mean nothing if any program can itself just start behaving as the kernel and use privileged operations like switching off interrupts, replacing the exception vector table, or managing devices. The programs you will run in Part A are tiny, but each crashes your OS as it exists today.

What you need to do

Part A: misbehave

We provide six programs in user/diabolical/. Each one waits for a second, so that you can see the rest of your system running, then attempts its trick, and then (if it still can) reports what it did:

Program	What it tries to do
`diab_irqoff`	Masks IRQs, then computes forever without ever yielding.
`diab_timer`	Disables the ARM generic timer, then computes forever.
`diab_vectors`	Installs its own exception vector table, whose handlers announce that the program now owns the machine.
`diab_crash`	Executes an instruction that does not exist.
`diab_brk`	Executes a software breakpoint instruction (`brk`), which only a debugger should ever handle.
`diab_stack`	Points its stack pointer at an address with no memory behind it, then spins forever in a one-instruction loop that never touches memory.

Task: Run each diabolical program on your OS, and record what happens.

Load your usual Project 3 programs (idle, term, basic, clock, and pong) plus one diabolical program at a time, in init(). Run make qemu-fb (or make qemu-ws).
For each of the six programs, observe: what happens to the other programs, the screen, and the keyboard? Does your kernel print anything? And why, in terms of the mechanisms you built in Projects 2 and 3, does this happen? (Keep your notes—you will compare against them at the end of the quest.)

Hints

diab_irqoff and diab_timer look similar at first, but they are not: one of them still lets your keyboard interrupt through. Which one, and does that help anybody?
Read the source code of each program. They are short, and the comments explain what each instruction does.
If the screen freezes, check the serial console as well. Some of these programs announce themselves there.
diab_stack does nothing harmful itself. So who writes to memory next, and where?

🤖 AI use/coding: NOT allowed.

Why?

There is no code to write. The point of this task is for you to see how little it takes to defeat an OS without privilege separation, and to explain why this happens.

For context, see the principles around AI use in CS 1670.

Background: a kernel stack for every process

Since Project 2, your kernel has run on the stack of whichever process entered it: an interrupt saved the process’s context on the process’s stack, and kernel functions reached through the trampoline ran on it, too.

At EL0, this would be a disaster, for two reasons:

The process controls its stack pointer. A process can set sp to anything—the middle of your process table, your kernel’s code, or an address with no memory behind it—and then make a system call. If the kernel saved the process’s registers and ran its C code on that stack, the process would have chosen where the kernel writes. The kernel must never trust a stack pointer that a process can manipulate.
A process can be stopped in the middle of kernel code. In Project 3, a process blocked inside getc() with kernel stack frames on its stack. That will still happen (in Quest 3, inside read()), and while that process is blocked, other processes must be able to enter the kernel without overwriting its frames. So one kernel stack for the whole system is not enough: each process needs its own.

ARM64 makes the first part easy: each exception level has its own stack pointer. Code at EL0 uses SP_EL0; when an exception is taken to EL1, the hardware switches sp to SP_EL1 before your first instruction runs. The process’s SP_EL0 is left exactly as it was, and you save it like any other register.

So the design is: user code runs on the process’s user stack (via SP_EL0, at the highest address of the process’s memory slot, as before), and every entry into the kernel runs on the process’s kernel stack (via SP_EL1), a small, separate region that only the kernel uses. When a process is running at EL0, its kernel stack is empty, and SP_EL1 points at its highest address. When the process traps into the kernel, its registers land right at the highest address of its kernel stack, as its saved context (the same context_t you have used since Project 2, which also called it the “trapframe”):

   this process's kernel stack
   (only the kernel touches this)

   ┌──────────────────────────┬──────────────────────────────┐
   │ kernel C frames:         │ saved context: user's        │◄── beginning of stack
   │ ... handle_syscall()     │ x0-x30, SP_EL0, ELR_EL1,     │
   │                          │ SPSR_EL1                     │
   └──────────────────────────┴──────────────────────────────┘
   ▲
   SP_EL1 (while in the kernel)


   this process's memory slot
   (user code, data, canvas, user stack)

   ┌─────────┬────────┬─────────────┬────────────────────┐
   │ program │ canvas │             │ user stack         │◄── beginning of stack
   │         │        │     ...     │ (grows leftward)   │
   └─────────┴────────┴─────────────┴────────────────────┘
                                    ▲
                                    SP_EL0 (saved in context)

Recall that your restore_context(c) sets sp to c + sizeof(context_t) before its eret. When the kernel returns to a process by restoring the saved context at the highest address of its kernel stack, that sets SP_EL1 to exactly the highest address of the kernel stack—so the next time that process enters the kernel, the hardware is already pointing at an empty kernel stack. Hence, there is no special “switch to the kernel stack” code that you need to write!

With the MMU still off, a kernel stack protects your kernel from a process’s bad stack pointer. It does not protect the kernel stack from a process that deliberately writes to its memory (Project 5 will address that).

Part B: a kernel stack, and down to EL0

Task: Give every process a kernel stack, and start every process at EL0.

Split this work into two parts.

Per-process kernel stacks:

Pick a region of physical memory for NPROC kernel stacks of KSTACK_SIZE bytes each (16 KiB each is sufficient), define it in kernel/memlayout.h (or limits.h), and make sure it collides with nothing else. Record the location of the process’s kernel stack in its process descriptor, and set it when the kernel allocates a process.
Your context must now include the user stack pointer, since it will be different from the location where the saved context is. Think about what change you’ll have to make to your context switch code and context representation to also save and restore SP_EL0. (If you used our recommended layout from Project 2, the 34th slot in the context (offset 264) is a good place to use.)
Change the code that builds a new process’s initial context (in your ELF loader) so that the context sits at the beginning of the process’s kernel stack, not the highest address of its memory slot. Zero the whole context first: it no longer lives in the slot that your loader clears, and a kernel stack may still hold another process’s old kernel frames, which must never leak into a new process’s registers. Set the process’s saved ELR_EL1 to the entry point as before, its saved SP_EL0 to the highest address of the process’s memory slot (the user stack), and its saved SPSR_EL1 to a new constant, SPSR_USER: EL0t (EL0, using SP_EL0; SPSR_EL0t is already in kernel/aarch64.h) with IRQs unmasked and FIQ and SError masked.

EL0 exception handling:

Exceptions from EL0 arrive in the “lower EL, AArch64” group of your exception vector table: synchronous exceptions at offset 0x400, IRQs at 0x480. Make interrupts get handled exactly like your EL1 IRQ entry does. Make synchronous exceptions save a context and call a new C function (e.g., handle_el0_sync(context_t* ctx)). For now, this function should only print ESR_EL1, ELR_EL1, and FAR_EL1 and panic.
Decide what EL0 may do with the timer: somewhere in your startup code (before any process runs), write CNTKCTL_VALUE (from kernel/aarch64.h) to the CNTKCTL_EL1 system register. This lets processes at EL0 read the timer’s counter (CNTPCT_EL0), which is harmless, but not program the timer that preempts processes.
Make your idle process an empty loop for now, i.e., comment out the call to yield(). We’ll put the yield() back in Quest 2.
Load only idle and run make qemu-verbose. You should see a line like Exception return from AArch64 EL1 to AArch64 EL0 when your first process starts, and then a stream of timer interrupts ...from EL0 to EL1, each followed by a return to EL0. That is your idle process being preempted at EL0.

Hints

The kernel stack is where your saved contexts and kernel C frames live; the user stack is the highest address of the process’s memory slot, as before. Do not mix them up: a kernel stack inside the process’s slot would be memory that the process is entitled to write even once Project 5 enforces memory protection.
CHECK() that each kernel stack top is 16-byte aligned, as the ARM64 calling convention requires. Our boot code leaves the CPU’s own stack alignment check off, so a bad stack alignment never gives you a clean “stack alignment fault”. With the MMU off, though, every 8-byte load or store must be 8-byte aligned: a top that is not a multiple of 8 makes your kernel take a data abort (EC 0x25, with FAR_EL1 inside your kernel stacks) as soon as your ELF loader writes the first saved context there.
Your idle process cannot call yield() because the code it calls uses privileged instructions that cause an exception. (Unfortunately, it cannot use wfi to sleep at EL0, either: a wfi that would actually wait traps to EL1.)
The 0x200 and 0x280 entries (“current EL”) now only fire for exceptions taken while the kernel is running. Your kernel runs with interrupts masked, so 0x280 should never fire; add a CHECK() in handle_interrupt() that the interrupted context came from EL0 (the mode bits of its saved SPSR_EL1 are SPSR_EL0t).

🤖 AI use/coding: auxiliary use allowed. You may use AI to help with the inline assembly for CNTKCTL_EL1, to look up SPSR bit layouts, and to debug. You may not have it write the kernel stack setup, the initial saved context, or the vector table entries for you.

Why?

Each of these is a few lines, and each is a decision you must be able to explain: where the saved context lives, which stack pointer is which, and how the CPU gets from EL0 to your handler and back. We may ask about kernel stacks in your whiteboard discussion.

For context, see the principles around AI use in CS 1670.

Task: Update your memory layout diagram to show the kernel stacks.

🤖 AI use/coding: NOT allowed. Why? As in the previous projects, we want you to keep your own mental picture of your OS’s memory layout.

Part C: kill, don’t panic

Your OS now stops a misbehaving process at EL0 by panicking. That is no better than Project 3: one bad process still takes down the whole machine! An exception from EL0 is not a kernel bug. Instead, it’s a process doing something it isn’t allowed to do, and the kernel should simply remove that process and carry on with the others.

Task: Kill a process that causes a synchronous exception at EL0, and keep running everything else.

In handle_el0_sync(), read ESR_EL1 and extract the exception class (bits 31:26). For now, every class is a fault (Quest 2 adds system calls, which will be a “benign” form of exception).
Print a diagnostic on the UART serial console. This should include:

which process caused the exception (PID and name),
the exception class,
ELR_EL1 (the user PC), and
for instruction and data aborts (exception classes 0x20–0x25), FAR_EL1, the address that the process tried to access. (For other exception classes, FAR_EL1 is meaningless and may still hold the address from an earlier fault.)

(A short table that turns the common exception classes into strings, as described in Appendix A, makes these messages much easier to read.)

Kill the process: mark it UNUSED, and make sure nothing else in your kernel still thinks it is alive—in particular, if it held the keyboard focus, move the focus on (to “nobody”, if no other process reads the keyboard), and if it had a window, the compositor must stop drawing it. Then call your scheduler.
Leave the 0x200 handler alone: a synchronous exception in the kernel is still a kernel bug, and still panics.
Run each diabolical program again, this time all six at once together with idle, and compare against your notes from Part A.

Hints

Read ESR_EL1 first thing in your handler. It describes the most recent exception, so if anything goes wrong in your kernel before you read it, you will be looking at the wrong exception.
Your scheduler has never had to deal with a process vanishing while in RUNNING or BLOCKED state. Make sure pick_next() cannot pick a dead process, and that your kernel does not try to save a context into it.
You will reuse this “kill the current process” code in Quest 4, when processes can also exit voluntarily. Put it in a function of its own.
You cannot fully test the keyboard-focus and window parts of the cleanup yet: in Quests 1 and 2, every program that reads the keyboard or draws dies before it gets that far. Write them anyway, though, as Quests 3 and 4 will exercise this functionality.
If every diabolical program dies right away, at the same small offset in its slot and with EC 0x18, look that address up: it may be in diab_delay() (in user/diabolical/diab_delay.h), whose read of the timer’s counter causes an exception if your kernel doesn’t correctly set CNTKCTL_EL1 (Part B, step 5).

🤖 AI use/coding: auxiliary use allowed. You may use AI to generate the table of exception class names, and to help you debug. Write the code that decides to kill the process, and the cleanup, yourself.

Why?

Turning numbers into strings is not OS content! But deciding what a fault in a process means, and making sure every part of your kernel forgets a dead process, is important OS content.

For context, see the principles around AI use in CS 1670.

Check your work

Load idle and all six diabolical programs, and run make qemu. You should see five of them killed, with messages along these lines (your format and addresses will differ):

[kernel] killed pid 1 (diab_irqoff): EC 0x18 (trapped system register access), ELR 0x21000a0
[kernel] killed pid 2 (diab_timer): EC 0x18 (trapped system register access), ELR 0x22000a0
[kernel] killed pid 3 (diab_vectors): EC 0x00 (undefined instruction), ELR 0x2300ff0
[kernel] killed pid 4 (diab_crash): EC 0x00 (undefined instruction), ELR 0x24000a0
[kernel] killed pid 5 (diab_brk): EC 0x3c (brk instruction), ELR 0x25000a0

and then nothing more: your idle process carries on, and the kernel does not panic. Compare this with your notes from Part A! Then check these things:

Each diabolical program died at its trick, not somewhere else. Look up each ELR in the program’s own symbols (e.g., aarch64-elf-objdump -d obj/diab_timer.elf if using the native macOS setup; aarch64-gnu-linux-objdump -d obj/diab_timer.elf for the container setup), remembering that the program was loaded at the start of its memory slot (in the example above, slot N starts at 0x2000000 + N * 0x100000).
diab_stack is never killed, and harms nobody. It keeps spinning, and is preempted like any other process: the timer interrupt saves its registers on its kernel stack, not at the address it chose. On your OS prior to this quest, the same interrupt crashed the machine!
Your well-behaved programs die too. Load hello and counter. They are killed, with an ELR that points into your kernel’s code: the first time they call printf() or yield() through the trampoline, kernel code runs at EL0 and trips over the very protection you just turned on. The trampoline is an unprotected interface: it only worked because your processes ran in EL1. Quests 2–4 will replace it!
Some trampoline functions still “work”. Not every kernel function reached through the trampoline uses a privileged instruction. If your getpid() wrapper or your get_canvas() wrapper is called at EL0, it reads kernel memory, and that succeeds.
You are really at EL0. Load only idle, run make qemu-gdb, break in idle’s loop (or press Ctrl-c in gdb) and p/x $cpsr. (With diab_stack loaded, idle never runs: your scheduler always prefers a runnable process.) The low four bits are 0 (EL0t), not 5 (EL1h).

Quest 2: Knocking on the kernel’s door

Files: kernel/syscall.c, kernel/syscall.h, kernel/interrupts.c, kernel/init.c, kernel/drivers/timer.c, user/lib/u_syscalls.h, user/lib/u_common.c, user/lib/u_common.h, user/idle.c, user/hello.c, user/counter.c, user/squares.c, user/primecheck.c, user/pi.c.

Overview

Your processes are now locked out of the kernel. In this quest, you will give them a way back in. You will:

Set up the user side of a system call: a two-instruction stub for each system call, which our build generates from your system call numbers, and a C function declaration for each.
Write the kernel side: recognize a system call in your EL0 exception handler, find out which one it is, call the right handler, and return its result to the process.
Implement the first three system calls, yield(), ticks(), and getpid(), and make your programs survive long enough to use them.

Why are we doing this?

Every interaction between a program and the operating system—every file read, every network packet, every process created, every byte printed—goes through a system call. On Linux, a busy server makes millions of system calls per second through code that looks a lot like the code you are about to write!

Background: the system call ABI

A system call interface is a contract between two pieces of software that are compiled separately, possibly years apart, by different people. Both sides must agree on:

How to “name” the operation. Every system call has a fixed number. The system call numbers for our course (and your OS) are listed in the Syscall ABI: for example, yield is 0 and ticks is 1.
How to pass the number. Different OSes here make different design choices: Linux passes the syscall number in a register, but we recommend that you use the svc instruction’s 16-bit immediate operand: svc #0 is a yield, svc #1 is a ticks. When the exception is taken, the hardware copies the immediate into the low 16 bits of ESR_EL1 (part of the instruction-specific syndrome, ISS), where your kernel reads it.
How to pass the arguments and the result. Arguments go in registers x0 to x5, in order, and the result comes back in x0—the same registers the ARM64 C calling convention uses for an ordinary function call. This is why the user-side stubs are only two instructions long. On an error, a system call returns -1.
What each call means. Processes and kernel must have the same idea of a syscall’s semantics: for example, they must agree what read() does when there is no input yet, what “a tick” is, and which errors are possible.

Together, these are called an Application Binary Interface (ABI). An ABI is stronger than an API: it is not enough that the source code agrees on a function name; the machine code of a compiled program must agree with the kernel on every number and every register. Once a program has been compiled against an ABI, the kernel can change anything it likes internally, but it can never change the ABI without breaking that program. (Linux takes this so seriously that its key rule for kernel developers is “we do not break user space”: statically linked programs compiled potentially decades ago still run on today’s kernels.)

You will put this to the test in Quest 4, when you run a program that we compiled against the ABI specification. If your kernel honors the contract, it will just work.

Why numbers, and not function addresses?

The trampoline exposed the addresses of your kernel’s functions to every program. That is an implementation detail: recompile your kernel, and the addresses change. A system call number, by contrast, is an interface: it says “the operation called write”, and the kernel maps this to whichever internal function implements the write syscall (which may change). The kernel’s dispatch table—an array of function pointers indexed by system call number—is where syscall numbers get translated into code locations, and this table lives inside the kernel and gets generated as part of it. (Once Project 5 protects the kernel’s memory, no process will be able to read or change this table either.)

Where the arguments are, inside the kernel. By the time your C code runs, the process’s registers are in its saved context on its kernel stack. So x0 is ctx->regs[0] (or whatever you called it), x1 is ctx->regs[1], and so on. To return a value, write it into the saved context’s x0: when you restore_context(ctx), the eret returns to the process with that value in x0. This is where the C calling convention expects a function’s return value, so it will just work. Note also that ELR_EL1 already points at the instruction after the svc, so returning to the process continues in the right place (after the syscall).

What you need to do

Part A: the user side

The user-side stub for a system call is the function a program actually calls: yield(), ticks(), and so on. While these look like C functions to user programs, stubs are written in assembly. Our build system generates them for you: user/gen_syscall_stubs.py reads every #define SYS_<name> <number> line in kernel/syscall.h, and writes an assembly function called <name> into user/syscalls.S, which is linked into every program. For #define SYS_ticks 1, it generates:

  .global ticks
  .type ticks, %function
ticks:
  svc #1
  ret
  .size ticks, . - ticks

Two instructions are all a stub needs, because our ABI uses the same registers as an ordinary C function call. When a program calls ticks()—or write(1, buf, 5)—the C compiler has already put the arguments into x0, x1, …, and it will look for the result in x0 afterwards. So the stub only has to execute svc #1, which enters the kernel, and then ret to its caller, with the kernel’s result in x0. (The .global line makes the function visible to C code; the .type and .size lines tell gdb and objdump that it is a function, and how long it is.)

To the C compiler, a stub is just a function defined somewhere else, so each one needs an ordinary C declaration, which you write in user/lib/u_syscalls.h: for example, uint64 ticks(void);.

Task: Define your first system call numbers, and declare the system calls on the user side.

In kernel/syscall.h, define the numbers from our Syscall ABI for the first two system calls: #define SYS_yield 0 and #define SYS_ticks 1. Add each of the other numbers from the table when you implement that system call. Run make, and read the stubs in the generated user/syscalls.S.
In user/lib/u_syscalls.h, declare yield() and ticks() with the signatures from our ABI, and #include "u_syscalls.h" in user/lib/u_common.h, so that every program sees the declarations. Declare the other system calls as you get to them.
Remove the trampoline-based yield() wrapper from user/lib/u_common.c, and its declaration from user/lib/u_common.h: the generated yield stub replaces it. (The linker refuses to link two functions with the same name, and the compiler refuses two conflicting declarations.) Move the declaration of exit() from user/lib/u_common.h into user/lib/u_syscalls.h, too.
Fix exit(). Eventually, you will have an exit syscall that takes an int status and never returns: [[noreturn]] void exit(int status);. But we won’t get there until Quest 4, so keep exit as a C function in user/lib/u_common.c, with the body (void)status; for (;;) { yield(); }. (In Project 3, your exit() may have returned. Now that _start calls it, returning would send the process off into whatever code happens to follow _start, or to address 0.) Do not define SYS_exit yet: its generated stub would clash with this function. The new signature breaks the exit(); at the end of hello.c, counter.c, squares.c, primecheck.c, and pi.c: replace each with return 0;, which _start passes on to exit().

Hints

You can check your stubs by looking in user/syscalls.S
Why are the stubs in assembly, rather than C? The system call number is part of the instruction: svc #1 encodes the 1 in its machine code, so it cannot come from a variable. A stub needs exactly that instruction and nothing else—no stack frame, no moving registers around. C has no way to say this without inline assembly.
Why does ret work after svc? ret jumps to the address in x30, which the caller’s bl ticks put there. Taking an exception does not change x30, and your kernel restores every register from the saved context before it returns, so x30 still holds the caller’s return address.
A system call without a SYS_ number has no stub, so a program that calls it fails to link with “undefined reference”. In this case, the number is missing rather than the declaration.
user/syscalls.S is regenerated whenever kernel/syscall.h changes. Never edit it by hand: your changes would be overwritten.
The generator turns every #define SYS_... line into a stub, so do not use the SYS_ prefix for anything else in kernel/syscall.h (a SYS_MAX, say).

🤖 AI use/coding: auxiliary use allowed. There is little to write here: a few #defines and declarations. You may use AI to help you read the generated assembly, but you must be able to explain every instruction in a stub yourself.

Why?

The stub is the user-side half of a protected control transfer, and it is two instructions long. You should be able to say what each instruction does, which register holds what, and how the system call number gets to the kernel.

For context, see the principles around AI use in CS 1670.

Part B: the kernel side

Task: Dispatch system calls in your kernel.

In handle_el0_sync(), check the exception class: if it is 0x15 ("SVC instruction executed in AArch64 state"; ESR_EL1_EC_SVC_CAUSE in aarch64.h), this is a system call. Extract the system call number from the low 16 bits of ESR_EL1 (ESR_EL1_IMM_MASK) and call a system call handler you create (e.g., handle_syscall(ctx, number) in kernel/syscall.c). Any other exception class is still a fault that kills the process.
Write your syscall handler. It should:
- record ctx as the process’s saved context (so that, if the process gets descheduled during the system call, it resumes in user mode at the right place);
- look up the handler for the system call number in a dispatch table: an array of function pointers, indexed by system call number, each of type int64 (*)(context_t* ctx);
- if the number is out of range or has no handler, return -1 to the process (a process asking for a system call that does not exist is not a reason to kill it, nor to panic);
- otherwise, call the handler, store its return value in the saved context’s x0, and restore_context(ctx).

Hints

C’s initializer syntax can help make the dispatch table readable: [SYS_yield] = sys_yield, [SYS_ticks] = sys_ticks, .... Entries you have not written yet are NULL.
handle_syscall() never returns: like your interrupt handler, it ends in restore_context(), or in the scheduler if the process gave up the CPU. Mark it [[noreturn]].
Each handler reads its own arguments from ctx. That keeps handle_syscall() independent of how many arguments a system call has.
An int argument arrives in the low 32 bits of its register; the upper 32 bits are not guaranteed to be anything in particular. Read a int as (int)ctx->regs[0] (and similarly for other 32-bit or smaller values), never as the whole 64-bit register.

🤖 AI coding: NOT allowed for handle_el0_sync()’s dispatch and the system call handler. Auxiliary use allowed for debugging.

Why?

This is the heart of the project: the kernel half of the protected control transfer. It is about thirty lines, which are all important: from the exception class to the value the process finds in x0 when it resumes.

For context, see the principles around AI use in CS 1670.

Part C: the first system calls

You now have a mechanism and no system calls. Let’s add three, and see them work. Since your processes still call printf() through the trampoline (which kills them), we’ll first take printf out of the picture: until Quest 3, programs will compute in silence.

Task: Implement yield(), ticks(), and getpid(), and get your programs running at EL0.

Silence printf. Make the user-side printf() in user/lib/u_common.c do nothing at all, for now. (Keep your kernel-side printf() intact, though.)
yield(). Write a syscall-specific handler in the kernel (e.g., sys_yield(ctx)), which lets the scheduler run another process and returns 0 to the caller once the caller runs again.
ticks(). Write a second syscall-specific handler (e.g., sys_ticks(ctx)), which returns the number of milliseconds since the machine booted.
getpid(). Add SYS_getpid (it is 5 in our ABI) to kernel/syscall.h and declare getpid() in user/lib/u_syscalls.h. Write a syscall-specific handler in the kernel, which returns the caller’s PID, and delete the trampoline-based getpid() wrapper from user/lib/u_common.c (and its declaration from user/lib/u_common.h).
Put yield() back into your idle process’s loop.
Load idle, hello, counter, and primecheck. Since they currently print nothing, check that they run and switch between each other: add a DEBUG() print to sys_yield that shows which process yields, or use GDB (see “Check your work” below).

Hints

sys_yield never returns to handle_syscall(): it ends in your scheduler, which resumes some process with restore_context(). So think about how your could store the return value 0 before you give up the CPU. Remember that when the scheduler eventually resumes the process, it restores the saved context you recorded in handle_syscall().
Which kernel function should sys_yield call? The one that marks the current process RUNNABLE and runs the scheduler (what your Project 2 handle_yield calls after it records the context)—not yield_entry. handle_syscall() has already recorded the saved context as the process’s context; yield_entry would save a second, kernel-mode context on top of it.
Why milliseconds, when your timer counts ticks at TIMER_HZ? Because the ABI must not depend on your kernel’s internals: a program compiled today cannot know what TIMER_HZ a kernel developer chose. (The name ticks() is a leftover from Project 3’s tick counter.)
Compute ticks() from the hardware counter (CNTPCT_EL0 * 1000 / CNTFRQ_EL0, which your kernel at EL1 may still read), not from the number of timer interrupts you have taken. Timer interrupts are delayed whenever your kernel runs with them masked—during a long write() in Quest 3, say, or while your compositor copies—so an interrupt count falls behind real time, especially on the Pi.
clock.c and pong.c read CNTPCT_EL0 directly. Despite what their comments say, that still works at EL0, because your kernel lets EL0 read the counter (Quest 1, Part B). In Quest 4, you will still switch clock to ticks(), which ties it to the ABI rather than to the hardware.
To see ticks() in action before you can print, add a temporary DEBUG() print of its result to sys_ticks, and have a program call it in a loop.
DEBUG() prints only appear in a build with DEBUG_LVL=1 (e.g., make qemu DEBUG_LVL=1). Switching between that and a plain make rebuilds everything once.

🤖 AI use/coding: mixed.

NOT allowed for sys_yield.
Auxiliary use allowed for sys_ticks, sys_getpid, and debugging.

Why?

There is little code here, but sys_yield contains a subtle point—a system call that does not return to its caller in the ordinary way—that you should work through yourself.

For context, see the principles around AI use in CS 1670.

Check your work

With idle, hello, counter, and primecheck loaded, nothing appears on the serial console (other than your kernel’s messages), and nobody gets killed. To see the system calls happen:

(gdb) break handle_syscall
(gdb) continue
(gdb) p number                  # 0 = yield (or whatever you called the argument)
(gdb) p/x ctx->elr_el1          # the user PC to return to ...
(gdb) x/2i ctx->elr_el1 - 4     # ... right after the process's `svc #0x0`
(gdb) p/x ctx->spsr_el1         # low four bits 0: the call came from EL0t
(gdb) p/x $sp                   # inside the current process's kernel stack

and make qemu-verbose shows each one as an exception with class 0x15:

Taking exception 2 [SVC] on CPU 0
...from EL0 to EL1
...with ESR 0x15/0x56000000
...
Exception return from AArch64 EL1 to AArch64 EL0 PC 0x...

The 0x56000000 is ESR_EL1: exception class 0x15 in its top six bits, and the system call number in its bottom sixteen. (Bit 25, “IL”, is also set, because svc is a 32-bit instruction; that is why the value starts with 0x56 rather than 0x54.) Make a ticks() call and you will see 0x56000001.

Also check that:

Preemption still works. primecheck never yields; hello and counter still run.
ticks() keeps time. Two calls made a second apart (by your watch) should return values about 1000 apart.
A bogus system call returns -1. Add asm volatile("svc #999") to a program, and check (in gdb) that x0 is -1 afterwards and the program keeps running.

Bonus Quests (Extra Credit)

Important note: All extra credit work requires you to meet with an instructor at office hours to discuss your work in order to get credit. There will be no exceptions to this, since extra credit is strictly optional.

Quest 2-E1: What does a system call cost? (easy–moderate)

Task (extra credit): Measure how long a system call takes on your OS, compared with an ordinary function call. Call getpid() a million times in a loop, and time the loop with ticks(); then do the same with a user-space function that does the same amount of work. Do it in QEMU and on a real Raspberry Pi.

Explain where the time goes: what does your kernel do on every system call that a function call does not?

🤖 AI use/coding: allowed, including agentic coding. We generally allow AI for extra credit quests, but you must understand your implementation fully. To get credit, you will need to discuss your work with an instructor in person.

Quest 3: File descriptors

Files: kernel/uaccess.c, kernel/uaccess.h, kernel/file.c, kernel/file.h, kernel/proc.h, kernel/proc.c, kernel/syscall.c, kernel/syscall.h, kernel/console.c, kernel/console.h, kernel/scheduling.c, kernel/context.S, kernel/kernel.c, kernel/init.c, user/lib/u_syscalls.h, user/lib/u_common.c, user/lib/u_common.h, user/basic_console.c, kernel/drivers/uart.c.

Before you start: fix a race in your UART interrupt handler from Project 3. In Project 3, we told you to drain the UART’s receive FIFO in uart_interrupt(), and then to acknowledge the interrupt by writing to the ICR device register. That order has a race condition:

Suppose a byte arrives after your handler last found the FIFO empty, but before it writes to the ICR device register. The byte raises the receive interrupt after the FIFO is fully drained, but your write to ICR to acknowledge the interrupt clears it straight away.
The byte then sits in the FIFO, and nothing will ever tell your kernel about it as no more interrupts happen.

On a real Pi, the UART’s receive-timeout interrupt (if you unmasked it) eventually fires for the stranded byte, so this isn’t a problem on hardware. But QEMU’s emulated UART device raises its receive interrupt only when the FIFO goes from empty to non-empty and doesn’t model the timeout-driven interrupt. So under QEMU, your kernel never receives another keystroke!

When this happens, there is no panic and no error message: your programs keep running, but the keyboard is dead. The race is rare when you type slowly, but likely when input arrives in a burst (when you paste a line, say). You should fix this!.

Fix this by acknowledging the interrupt first, then draining the FIFO. In your UART interrupt handler, first write the ICR device register to clear the receive interrupts; then read the FIFO until it is empty. A byte that arrives after your last look at the FIFO now raises a fresh interrupt that nothing clears, so your interrupt handler runs again as soon as userspace re-enables interrupts. (Acknowledging the interrupt while bytes are still in the FIFO is fine, as you’ll read them all.)

Go to your UART driver and make this change now. It most likely only requires you to reorder two lines!

Overview

Your processes can now yield and tell the time, but they cannot print or read the keyboard. In this quest you will add the two most important system calls in Unix: write() and read(). These system calls are powerful and general because they rely on file descriptors, another key Unix abstraction. To support both, you will:

Write helper functions that check, and safely copy, data between a process’s memory and the kernel’s. (This helps your kernel defeat confused deputy attacks.)
Give each process a table of file descriptors, make 0 and 1 refer to the UART console, and implement write(). Once you have that, you’ll gain a working printf() again.
Implement a read() syscall to read data from the keyboard, and a peek() syscall to determine if there is data to read. Supporting read() requires your kernel to be able to block inside a system call until input arrives, so you’ll add that functionality too.

At the end of the quest, all of your text-mode programs work again: hello, counter, primecheck, pi, keytest, and BASIC on the serial console.

Why are we doing this?

write(fd, buf, n) and read(fd, buf, n) are the interface through which programs on Unix-like operating systems talk to nearly everything: terminals, files, pipes, network connections, and devices. Importantly, the program does not need to know which kind of resource it is talking to—only the kernel does. In this project, we’ll only support one kind of resource for now: the UART console. CS 1690/2670 students will add a second resource, pipes, and later projects will add more.

These system calls are also the first ones whose arguments are pointers, which forces you to think carefully about what the kernel can and cannot trust.

What you need to do

Part A: safe copies

Background: never trust a pointer from user space

Consider write(1, buf, n). The process passes a pointer, buf, and asks the kernel to read n bytes from it. What if buf does not point into the process’s own memory, but into the kernel’s—say, at your process table, or the kernel’s own code?

The kernel runs at EL1 and can read any memory it likes, so a naive sys_write would happily print the kernel’s data to the console. With read(0, buf, n), it is worse: a naive kernel would write keystrokes into its own memory, wherever the process pointed it. The process does not need any privilege of its own for this: it borrows the kernel’s. This is a classic security bug called a confused deputy: a privileged program is tricked into using its authority on behalf of someone who does not have it.

How do we defend against this? For each system call, we should follow a safety property: before touching user memory, the kernel checks that it belongs to that process. Then it copies the data across the boundary with a dedicated helper, and works only with its own copy. You will write three helpers:

user_range_ok(const void* uaddr, uint64 n) returns whether [uaddr, uaddr + n) lies entirely inside the current process’s memory.
copy_from_user(void* kdst, const void* usrc, uint64 n) checks [usrc, usrc + n) with user_range_ok(), and if it is fine, copies n bytes into the kernel buffer kdst.
copy_to_user(void* udst, const void* ksrc, uint64 n) checks [udst, udst + n) the same way, and copies from the kernel buffer ksrc into the process’s memory.

The two copy functions return 0 on success and -1 (copying nothing) if the check fails. Linux has functions with exactly these names, which work the same way!

On each system call, the kernel should check all of its pointer arguments before it does anything else—e.g., before it prints, blocks, or takes input out of a buffer—and return -1 at once if any of them is bad. That way, a call with a bad pointer has no effect at all, rather than doing half of its work first and then having to back out (which would be complex).

Why copy, rather than check the pointer and then use it directly? The data behind a user pointer belongs to the process, which can change it at any time—for example, in a more advanced OS that supports threads (one possible final project for you!), another thread on another CPU core might change the data after the kernel checked it and before the kernel uses it. Copying first means the kernel only ever acts on data that the process can no longer change. (Your kernel is single-core and runs with interrupts masked, so this can only happen right now across a point where a system call blocks and lets other processes run, and those processes modify the target memory, which none of our programs do. But you should follow the rule of “copy once, then use only the copy” anyway, as you might extend your OS in ways that make it necessary to have this protection.)

The rule applies in the other direction as well: the kernel should never return its own pointers to a process. In Project 3, we suggested that ps should not receive pointers to struct proc. Now that we have a real user/kernel boundary, we must follow this rule: system calls must return copies of kernel data (via copy_to_user), never addresses inside the kernel.

Note: This isn’t the whole picture. Since we have not yet implemented virtual memory (and therefore memory permissions), a process can still simply read or write into the kernel’s memory, even without system calls. Therefore, your helpers prevent the kernel from being an “accomplice” to any shenanigans, but we haven’t yet fully solved the problem yet. We will fix this in Project 5, and the protections you implement now are necessary complements to the virtual memory protection you implement then.

First, let’s put in place some infrastructure to help us securely access pointers passed into the kernel from userspace.

Task: Write user_range_ok(), copy_from_user(), and copy_to_user() in kernel/uaccess.c, with the signatures and behavior described above.

The valid range for a process is its own memory slot, i.e., addresses from the start of its slot to the start of the next process’s slot.
Reject a range that starts inside the slot but extends beyond it, and a range whose end overflows (wraps around past 0xFFFF'FFFF'FFFF'FFFF).
Test your implementation to make sure that it works correctly.

Hints

In (2), the other edge case, n == 0, is allowed, since it would not actually touch any memory.
addr + n < addr is a sufficient overflow test for unsigned integers here, since there is no multiplication involved.
To test your implementation, you can write a few CHECK()-based tests in kernel_main, before you start any processes (you will need to point current_process at a slot, temporarily). For example, you might want to test an address range inside the slot, a range that straddles its end, a range in another process’s slot, a kernel address, and an overflowing range.

🤖 AI coding: NOT allowed.

Why?

These functions are short (about ten lines each), but important: they are your kernel’s defense against a class of security bugs that exist even when your OS has proper memory protection.

For context, see the principles around AI use in CS 1670.

Part B: a file descriptor table, and `write()`

Next, we’re going to need file descriptors, so that you can implement the write() and read() system calls.

Background: file descriptors

A process that wants to print needs to have a place to send its output. More generally, a write() syscall must specify where to write the data. This can’t be specified as a pointer to the resource—kernel pointers do not cross the kernel/user boundary. Instead, the kernel keeps, for each process, a small table of the kernel objects that the process may use, and the process names an object by its index in that table. That index is a file descriptor number:

For example, here’s how a process might write to stdout (the console) via file descriptor number 1:

     process 3 (user)              struct proc for pid 3 (kernel)        kernel objects
     ────────────────              ──────────────────────────────        ──────────────
                                   fds[0] ──────────────┐
     write(1, "hi", 2) ─────────►  fds[1] ──────────────┴─────►  struct file: CONSOLE
                                   fds[2]      NULL               (reads come from the
                                   ...                             keyboard, writes go
                                   fds[7]      NULL                to the UART)

On every system call, the kernel checks that the descriptor is in range and refers to an open object, then uses the object it finds there. A process can pass any integer it likes; if the integer isn’t a valid file descriptor number, the kernel must return an error. And because the table lives inside the kernel, the process cannot modify it (especially once we have virtual memory protection).

By convention, file descriptor number 0 is the process’s standard input (stdin) and file descriptor number 1 is its standard output (stdout). A program that reads from 0 and writes to 1 does not know or care what is behind them: the console in this quest, a pipe to another program in Quest 5, or a file in a future project. That is what makes file descriptors so powerful!

In the next step, you’ll add file descriptors and implement write(), so that your processes can print again.

The typical way to implement file descriptors in a kernel is to add two pieces:

A file descriptor table for each process, which is an array of pointers to kernel objects (the I/O resources the process can use). The index into this array is the file descriptor number. Usually, you’d store this array in the process descriptor (e.g., struct proc), or store a pointer to it there. If you go with the pointer-based approach, you will need a separate global array of arrays of file descriptor tables, one per process for your maximum number of processes.
A file object for each open resource, which is a kernel structure that represents the resource. For now, you will only have one kind of file object: the console. Later, you will add more kinds of file objects (e.g., pipes, files, etc.).

Task: Add file descriptors to your kernel.

Define yourself a file object structure that represents an open kernel object (e.g., in kernel/file.h): for now, this struct only needs to contain fields to indicate (1) the resource type (e.g., via an enum that for now only contains type FILE_CONSOLE) and (2) a reference count (how many processes use this resource).
Create a single, global file object to represent the UART console (e.g., in kernel/file.c).
Add a file descriptor table to struct proc: an array of pointers to struct file (with a maximum number of possible FDs specified as a constant, e.g., MAX_NUM_FDS).
Whenever your kernel sets up a new process, make file descriptors 0 and 1 refer to the UART console, and set the rest to null pointers. Clear the file descriptor table when a process exits or is killed.

Hints

A kernel-side helper that turns a file descriptor number into a pointer to its file object (or a null pointer if the number is out of range or the file descriptor is not open), will be useful in every file-related system call.

🤖 AI use/coding: Auxiliary use allowed for the file descriptor table.

Why?

We want you to understand how file descriptors are represented on the kernel side (they are numbers in userspace). Implementing an FD table is one of this quest’s core ideas, but you’re welcome to seek help with C syntax or implementation issues.

For context, see the principles around AI use in CS 1670.

Task: Implement write(), and bring back printf().

Write a system call handler, such as sys_write(ctx), for the write() syscall (see our Syscall ABI for the arguments and details; in short, write(int fd, const void* buf, size_t n)). Check the descriptor and the whole range [buf, buf + n) for validity, and return -1 before writing anything if either is bad. Only then copy the user’s data into a kernel buffer in chunks (128 bytes at a time, say) with copy_from_user(). Depending on the type of object the file descriptor for the FD number passed to write() represents, invoke functions to write the data. So far, we only support the UART console: for this resource, send each chunk to the UART. Return the number of bytes written.
On the user side, add SYS_write to kernel/syscall.h (which generates the write() stub) and declare write() in user/lib/u_syscalls.h.
Then, rewrite printf() to format into a local buffer (with a size-bounded vsnprintf; see the hints) and call the write() syscall to write the local buffer to file descriptor number 1 (stdout).
Load the idle, hello, counter, primecheck, and pi programs. They should all print again!

Hints

The user-side printf needs a va_list version of your sprintf from Project 3 (a vsnprintf), since printf receives its arguments as .... Use a size-bounded version: an unbounded sprintf into a fixed buffer is setting you up for buffer overflow attacks.
Your kernel runs with interrupts masked, so a long write() keeps the timer interrupt waiting until the UART has sent every byte. This takes about 90 ms for a kilobyte at 115,200 baud, but that is fine for us. Think about how a real kernel avoids it (Project 3’s interrupt-driven transmit extra credit is one half of the answer!).
Your printf from Project 2 had to mask interrupts to keep lines from interleaving. Does the new one still need to?

🤖 AI use/coding: mixed.

Allowed, including agentic coding, for the user-side printf and vsnprintf, and for declaring write().
Auxiliary use allowed for sys_write.

Why?

A formatted-printing routine is a standard piece of code you have already written once. The way sys_write treats its arguments, on the other hand, is part of this quest’s core ideas: you should be able to explain what the kernel checks, why, and in which order.

For context, see the principles around AI use in CS 1670.

Part C: `read()`, `peek()`, and blocking in a system call

The read() syscall is more complicated than write() because it must wait for input to arrive. To do so without hanging other processes in the meantime requires blocking in the kernel (similar to what getc() did before). In this part, you will learn how to block a process in the middle of a system call.

Background: blocking in the middle of a system call

read(0, buf, 1) must wait until a key is pressed. You did this in Project 3: block() marks the process BLOCKED, saves a context, and runs the scheduler; when the key arrives, wakeup() makes the process runnable again, and its context is restored inside getc().

The difference now is where the waiting happens. In Project 3, getc() was called directly by a user process that also ran at EL1, and the kernel stack frames were on the process’s own stack. Now, sys_read() runs in the kernel, on the process’s kernel stack. This means the kernel stack frames are at lower addresses than the context saved when the process made the system call—its saved user context. Blocking and resuming requires saving another context: the saved kernel context. Hence, a process blocked in read() has two saved contexts on its kernel stack:

     SPSR = EL1h, IRQs masked:                                          SPSR = EL0t: resumes
     resumes the *kernel*,                                              the *user program*
     inside block()
     ▼                                                                  ▼
   ┌───────────────────────┬───────────┬────────────┬──────────────────┬────────────────────┐
   │ saved kernel context  │ block()'s │ sys_read's │ handle_syscall's │ saved user context │◄── beginning (highest address)
   │ (saved by block())    │ frame     │ frame      │ frame            │ (at svc)           │
   └───────────────────────┴───────────┴────────────┴──────────────────┴────────────────────┘
   ▲
   p->context points here while blocked

When the scheduler picks the process again, it restores p->context, the saved kernel context: execution resumes inside block() in the kernel (at EL1), on the kernel stack, as if block() had simply returned. sys_read() then finds the incoming key press, copies it to the user’s buffer, and returns to handle_syscall(), which restores the saved user context and returns to the user process (at EL0). Both cases rely on the same restore_context(). What differentiates them is the value in the saved SPSR_EL1: EL0t for a saved user context, EL1h for a saved kernel context.

Every way into your kernel—an interrupt, a system call, a fault—now masks interrupts in hardware, and every saved kernel context resumes with interrupts masked. So your kernel always runs with interrupts masked, and user code always runs with them enabled, without exception. In particular, the lost-wakeup race from Project 3 cannot happen in your kernel any more: no interrupt can arrive between checking the input buffer and blocking. (Do keep the while loop around block(), though: a woken process must always re-check its condition.)

In addition to a blocking read(), we also need a syscall to check if there is new data on a file descriptor without blocking (in other words, whether read() would return new data if called, without calling it, as doing so might block). In our OS, this happens via a syscall called peek(). In Linux and other Unix-like OSes, this is done via a non-blocking read, or the select() and poll() syscalls (which can even operate over multiple file descriptors at once), but these strategies are much more complex to implement.

Task: Implement read() and peek() syscalls and support them for the UART console.

This requires five steps:

Saved kernel contexts. Change the code that block() uses to save a context (in Project 3, yield_entry) so that the context it saves resumes in the kernel: its saved SPSR_EL1 must be SPSR_KERNEL (EL1h, interrupts masked, already in aarch64.h), not the user SPSR.
Implement a read() syscall. Write a syscall handler, such as sys_read(ctx), and support the spec in the Syscall ABI (in short, read(int fd, void* buf, size_t n)). Like write, this syscall first checks the validity of the file descriptor and the whole range [buf, buf + n), and returns -1 at once if either is bad. For the console, it then waits (with block(), as in Project 3) until the calling process’s input buffer holds at least one byte, takes up to n bytes—as many as are available, but no more—into a kernel buffer, copies them to the user’s buffer with copy_to_user(), and returns how many it copied. A read of 0 bytes returns 0 immediately.
Implement a peek(int fd) syscall. Write a handler, such as sys_peek(ctx), which returns how many bytes a read() on this descriptor could return right now without blocking—0 if none—and never blocks.
Keep your Project 3 keyboard focus logic: reading or peeking at the console marks the process as a keyboard reader, as getc() and hasc() did before.
Add the userspace side. Add SYS_read and SYS_peek to kernel/syscall.h, so that userspace stubs get generated for them. Then declare read() and peek() in the user-side C header (user/lib/u_syscalls.h).

Hints

Some common bugs to watch out for: (1) If a process blocked in read() resumes at EL0 instead of inside block() (and immediately crashes or behaves oddly), your kernel context was saved with the user SPSR. (2) If it resumes inside block() with interrupts enabled, it was saved with your Project 3 SPSR_PROC.
Why check the user’s buffer before blocking? Otherwise, read(0, bad_pointer, 1) would wait for a keystroke, take it out of the buffer, and only then fail—and the keystroke would be lost.
Where does sys_read get the bytes from? From the calling process’s input buffer—the per-process buffers you built in Project 3 Quest 2 Part E stay as they are.

🤖 AI use/coding: mixed.

NOT allowed for the saved kernel context change and for the blocking logic in sys_read.
Allowed, including agentic coding, for the user-side declarations.
Auxiliary use allowed for everything else.

Why?

That a process can block in the middle of a system call, with two saved contexts on its kernel stack, is a key concept in system call handling. The code that implements it is also short, but requires thinking. We’d like you to do that thinking, but it’s okay to get AI help with some of the more mechanical plumbing work.

For context, see the principles around AI use in CS 1670.

Task: Now rebuild getc() and hasc() on top of your new syscalls.

Rewrite getc() and hasc() in user/lib/u_common.c on top of the new syscalls (getc() reads one byte from file descriptor number 0, stdin).
Make a program work: copy our original, serial-console version of BASIC (basic.c from our repository of provided programs, which you started from in Project 3) to user/basic_console.c. In its main(), replace the endless while(1) { loop(); } with loop(); return 0;, so that BASIC’s BYE command ends the program (this will matter in Quest 4). Then load idle, keytest, and basic_console, and use Shift+Tab to switch between them.

🤖 AI use/coding: mixed.

Allowed, including agentic coding, for the user-side declarations and the new getc()/hasc() wrappers.
Auxiliary use allowed for everything else.

Why?

Most of the user-side plumbing work is mechanical, but you should understand how the getc() function a user process calls now differs from the version you had in Project 3.

For context, see the principles around AI use in CS 1670.

Check your work

With idle, hello, counter, primecheck, pi, keytest, and basic_console loaded (one after the other, not all at once), make qemu should result in the following:

Printing works again, from every program, and a line from one program is never interrupted by another.
Input works again, and goes to the process with the focus. Shift+Tab still moves the focus; a half-typed BASIC line is still there when you come back to it.
A blocked reader costs nothing. With only idle, keytest, and basic_console loaded and nobody typing, your idle process should be the only thing that runs (use the rate-limited printf trick from Project 3 to check).
Bad pointers are refused. Write a small test program that calls write(1, (void*)0x80000, 16) (your kernel’s code), write(1, buf, 100000000), read(0, (void*)0x80000, 1), and write(5, buf, 1). Each should return -1, and your kernel should neither print its own memory nor crash.

And in gdb, while a process is blocked in read(), check the output of the following commands:

(gdb) p processes[N].state           # should show: BLOCKED
(gdb) p/x processes[N].context       # should show an address in N's kernel stack
(gdb) p/x processes[N].context->spsr_el1   # the low four bits should be 5: EL1h, a saved kernel context

Bonus Quests (Extra Credit)

Quest 3-E1: A write() that does not hold up the machine (moderate difficulty)

Task (extra credit): Make the console’s write() return as soon as the data is in a kernel buffer, and send it from the UART’s transmit interrupt (Project 3’s Quest 1-E1), blocking the writer only when the buffer is full. Measure how long the timer interrupt is delayed while a process prints a long line, before and after.

Quest 3-E2: sleep_ms() as a system call (moderate difficulty)

Task (extra credit): Add a sleep_ms(ms) system call (numbered 100 or higher; see our ABI specification) that blocks the calling process until at least ms milliseconds have passed, without using the CPU in the meantime. Use it in programs that currently spin on ticks(). Think about how often your kernel has to check whether a sleeper’s time is up, and what that means for how precise sleep_ms(1) can be.

If you did Project 3’s Quest 2-E2, port your implementation to use system calls: what changes now that the sleeper blocks inside a system call, on its kernel stack? Tell us when you claim credit.

Quest 4: Tearing down the trampoline

Files: kernel/memlayout.h, kernel/klib.c, kernel/klib.h, kernel/syscall.c, kernel/syscall.h, kernel/proc.h, kernel/proc.c, kernel/file.c, kernel/file.h, kernel/scheduling.c, kernel/compositor.c, kernel/compositor.h, kernel/console.c, kernel/interrupts.c, kernel/init.c, kernel/kernel.c, kernel/uaccess.c, kernel/uaccess.h, user/lib/u_syscalls.h, user/lib/u_common.c, user/lib/u_common.h, user/lib/u_start.c (provided), user/lib/u_sterm.c and user/lib/u_sterm.h (provided), user/term.c, user/gterm.c and user/sterm.c (CS 1670), user/logo.c, user/basic.c, user/clock.c, user/pong.c, user/prebuilt/snake.elf (provided).

Overview

Your system call interface is still incomplete: there are function calls that the trampoline provided, but which don’t have corresponding syscalls yet. Your kernel also starts every process at boot, while real OSes allow users to start processes at runtime. In this quest, you will:

Replace every remaining trampoline function with a system call, and then delete the trampoline, so that the system call interface is the only way into your kernel.
Add system calls to create processes (spawn), end them (exit), and wait for them to end (wait).
Teach your terminal to start programs. If you are in CS 1670, you will then make a copy of it that runs on the serial console, so that you have a terminal in a window and one on the serial console.
Run a program that we compiled, and run it on top of your kernel without ever seeing its source code.

At the end, your OS boots into a terminal (on the serial console, if you are in CS 1670) from which you can start any program—text or graphical, well-behaved or diabolical.

Why are we doing this?

As long as the trampoline exists, your system call interface is optional: a process could bypass it. Instead we want the system call table to be the complete, official interface between your kernel and the programs it runs. (Project 5 will make it the only possible one, by protecting the kernel’s memory.) So, we will remove the trampoline, and make sure that all of its functionality is available via system calls.

The spawn, exit, and wait syscalls complete the life cycle of a process: for the first time, processes in your OS can start other processes, and processes can end and wait for each other. This turns out to be enough to create a terminal application that functions like the shell you’re used to.

What you need to do

Part A: no more shortcuts

The trampoline still provides whatever else you added to it in Project 3. In our reference implementation, that is procinfo() (for your terminal’s ps), a tick count (for its uptime), get_canvas(), and wait_for_vsync(); your names and your set of trampoline calls may differ. Each trampoline call now becomes a system call, or is replaced by one you already have. One of them, get_canvas(), needs to change shape also: since you will soon be able to start processes at any time, not just at boot, the kernel can no longer hard-code whether a process receives a canvas/window in init(). Instead, a process that wants to draw asks for a window with init_canvas().

Task: Replace the remaining trampoline functions with system calls, then delete the trampoline.

Here’s what you need to do for the remaining functions that prior projects definitely required you to add (you may have more in addition):

To get process information, add a process_info(struct pinfo* buf, int max) syscall. This syscall fills in up to max entries of a user-supplied array with information about the processes that currently exist (at least the PID, the parent’s PID, the state, and the name), and returns how many entries it filled in. Check the whole array first, and use copy_to_user() to set its contents.
To give a process a window, add an init_canvas() syscall that gives the calling process a window and a canvas, if it does not have one yet. Find a free cell in your 2x2 window grid, set up the struct canvas in the process’s memory slot (moving this code out of init()), and return the canvas’s address—an address in the process’s own memory. Return NULL if there is no display or no free cell. (When a process exits, its window cell becomes free again.)
wait_for_vsync() works as in Project 3, but now becomes a system call. A caller that has no canvas gets 0 back immediately (see the ABI specification).

In the next step, let’s make sure we can’t cheat by removing the trampoline:

Delete the trampoline by removing the code that installs the F_... pointers (your klib_init(), if you made one, or wherever you install them in kernel_main), the F_... definitions in kernel/memlayout.h, FB_INFO_BASE if you still have it, and the trampoline wrappers in user/lib/u_common.c. Move anything else that lives in klib.c/klib.h (such as struct pinfo) to syscall.c/syscall.h first. Any other trampoline function you added in Project 3 becomes a system call numbered 100 or higher, or goes away. Your kernel should no longer publish anything at a fixed address for user programs to find!
Port your programs: term, logo, basic, clock, and pong get their canvas from init_canvas() instead of get_canvas(); term’s ps uses process_info() and its uptime uses ticks(); and clock’s time function in its wiring section uses ticks() instead of reading the timer’s counter.
Load idle, term, basic, clock, and pong, and run make qemu-fb (or make qemu-ws). Your Project 3 desktop should be back, now running entirely in userspace (at EL0)!

Hints

struct pinfo is part of your ABI: the kernel and term.c must agree on its layout exactly, so define it once in each and keep them in sync (The ABI provides a suggested layout, though note that you need to account for alignment and padding).
init_canvas() returns a pointer, which may look like it breaks the rule that kernel pointers do not go to user processes. It doesn’t, though: the canvas lives in the process’s memory, and the kernel keeps its own trusted copy of where it is (struct proc), exactly as in Project 3. It’s okay for the kernel to return pointers into a process’s memory.
After this step, grep -rnwE 'F_[A-Z_]+|klib_init|FB_INFO_BASE' kernel/ user/*.c user/lib/ should find nothing that refers to the trampoline.
Wherever clock.c and pong.c call get_canvas() (once at startup, and on every frame in their frame-done functions if you did Project 3’s Quest 5), call init_canvas() once at startup instead, and keep the pointer.

🤖 AI use/coding: mixed.

Allowed, including agentic coding, for porting the programs (step 2) and for deleting the trampoline.
Auxiliary use allowed for the new system calls. Design the struct pinfo yourself.

Why?

Porting programs to a new interface is mechanical. Deciding what a process is allowed to learn about other processes, and how a window comes into existence, is interface design for your kernel—and, as in Project 3, you should be on top of it since you’ll need to work with this interface.

For context, see the principles around AI use in CS 1670.

Part B: spawn, exit, and wait

In the next step, we add support for your OS to start processes at runtime. We won’t implement this in the standard Unix way—fork() and exec()—to keep things simpler and avoid having to rewrite this code once you implement virtual memory in Project 5. Instead, your OS will support a spawn() syscall that does both steps at once (interestingly, some researchers believe that this is what should replace fork() in any case). The new process is a child of the caller, and the caller can wait for it to finish with wait(). A process can also end itself with exit(), returning an exit status to its parent.

Background: the life cycle of a process

Until now, your kernel created every process at boot, and no process ever ended voluntarily. Real systems need three more operations:

Creating a process. Our spawn(name, stdin_fd, stdout_fd) syscall creates a new process running the program called name (one of the programs embedded in your kernel image), and returns its PID. The new process is a child of the caller, its parent. The child’s file descriptors 0 and 1 refer to the same kernel objects as the parent’s descriptors at the numbers passed as stdin_fd and stdout_fd arguments. Hence, a parent process can decide where its child’s input comes from and its output goes to: so far, the console is our only option, but this will change later.
Ending a process. exit(status) ends the calling process, and records a small integer, the exit status, for its parent. By convention, 0 means success. A process killed by your kernel for misbehaving gets an exit status of -1.
Waiting for a process. wait(pid) blocks the caller until its child pid has exited, and returns the child’s exit status.

There is a subtle consequence of wait(). A process that has exited cannot disappear completely, because its parent may not have called wait() yet, and the exit status has to be kept somewhere until it does. So an exited process lingers in a new state, ZOMBIE: it does not run, it has closed its file descriptors and given up its window, but its process table entry (with its exit status) stays allocated until its parent waits for it. Only then is the entry freed. If a parent exits before its children, nobody will ever wait for them, so the kernel frees their entries itself when they exit. (On Unix, this is what the init process, PID 1, is for: it adopts orphans and waits for them.)

What about fork() and exec()?

Unix creates processes in two steps: fork() makes a copy of the calling process, and exec() replaces the copy’s program with a new one. In between, the child can rearrange its own file descriptors before the new program starts. This is elegant, but fork() copies a whole address space, which requires code that you’d need to rewrite when you add virtual memory (Project 5). spawn() does both steps at once, and takes the descriptor arrangement as arguments instead. Windows creates processes this way (CreateProcess), and POSIX offers it too (posix_spawn).

Who gets the keyboard? In Project 3, keystrokes go to the process with the focus, and Shift+Tab moves it. When your terminal starts BASIC and waits for it, you want your typing to go to BASIC without pressing Shift+Tab—but when it starts the clock, which never reads input, the focus should stay where it is. So we will refine the focus rule: a process takes the focus the first time it reads from the console, if its parent has the focus at that moment, and when a process that has the focus exits, the focus goes back to its parent (if the parent reads the keyboard). The Syscall ABI gives the complete rules.

Our next few syscalls will deal with starting and terminating processes.

The stencil code now provides an auto-generated program table. Currently, your init.c contains a bunch of definitions for the start address of each user program (e.g., _binary_user_primecheck_elf_start). You had to add one such definition to init.c by hand each time you added a program. This was fine for running just a few programs at startup, but you’ll now begin spawning new programs at runtime (i.e., from your terminal) while your OS is running! To do this, you need to be able to look up programs by name.

Our build system now runs a script, user/gen_user_prog_info.py, which generates kernel/userprogs.c and kernel/userprogs.h. These contain a table of every program the build embeds (i.e., programs from user/, user/diabolical/, and user/prebuilt/), and a function find_program(name) that returns the program’s struct proginfo (its name and the start of its ELF image), or nullptr if there is no such program.

In this quest, you’ll remove a lot of code from your init.c and use find_program() instead.

Task: Implement spawn() to start new processes. This requires three steps:

Your process descriptor now needs to track the parent process’s PID (or a pointer to its process descriptor) and a process’s exit status. You also need to add ZOMBIE to your process states.
Make sure your code for allocating new processes (e.g., allocproc()) really finds a free entry anywhere in the process table, that it returns a null pointer instead of panicking when the table is full, and that a process’s memory slot is determined by its table index. Before this project, no entry was ever freed, so this code has never been tested for re-use of PIDs and process slots!
Implement a spawn() syscall with signature spawn(const char* name, int stdin_fd, int stdout_fd) (see the Syscall ABI). This syscall should copy the process name (see hint below), look the program up with find_program() (from the generated kernel/userprogs.h; it returns nullptr if there is no such program), check both descriptors, allocate a process, load the program, give it a kernel stack and first saved context as in Quest 1, point its file descriptors 0 and 1 at whatever is in the caller’s file descriptor numbers passed as stdin_fd and stdout_fd (incrementing any relevant reference counts), record the new process’s parent, and make the new process runnable. Return the child’s PID, or -1 on any error.
On the userspace side, add SYS_spawn and declare the system call.
Change your init() function to no longer iterate over the executables array, but instead load only idle and term via find_program(), and have the user use term to spawn the other programs. You can also now remove the executables array and the declarations of symbols for your user programs from init.c.
In place of executables, list the programs that init() starts by name, in a global array called boot_programs, with idle first (it must be PID 0). For example:
```
// The programs init() starts at boot, in this order
const char* boot_programs[] = {"idle", "term"};

void init() {
  for (uint64 i = 0; i < sizeof(boot_programs) / sizeof(boot_programs[0]);
       i++) {
    struct proginfo* prog = find_program(boot_programs[i]);
    if (prog == nullptr) {
      panic("init: there is no program called '%s'", boot_programs[i]);
    }
    // ... start a process as you did for each entry of executables[] before ...
  }
  // ...
}
```
Please use this exact name (boot_programs), and keep the array outside any function. Our build system’s user/gen_user_symbols.py reads boot_programs from your kernel to tell GDB which program runs as which PID (see Using GDB with user programs). Without it, every build warns that it cannot find boot_programs or executables, and GDB has no symbols for the programs your kernel starts at boot.

Hints

To copy a program’s name, you’ll need to add a helper, copy_str_from_user(char* kdst, const char* usrc, size_t max), which copies a NUL-terminated string of at most max bytes (including the NULterminator) from the process, checking every byte’s address. It returns -1 if the string is not terminated within max bytes or leaves the process’s memory.
Test spawn before you have a terminal: have init() load a small test program that spawns squares, waits for it, and prints the exit status (it should be 0). Then spawn diab_crash and check that wait returns -1.

🤖 AI use/coding: mixed.

Auxiliary use allowed for spawn(), its helpers, and the bookkeeping in step 1.
Allowed, including agentic coding, for the user-side declarations and for the init() changes in step 5.

Why?

spawn mostly reuses code you already have for loading processes at boot, but you should still understand how it works.

For context, see the principles around AI use in CS 1670.

At this point, you’ve stopped using the executables array and only start one program that will eventually start all other programs! You can also now write a program that spawns other programs, and use it to test your syscall (load that program and the idle process only, and check that your spawned program runs).

Task: Implement exit() and wait(), and connect things up.

For the two new syscalls, you need to implement the following (bear the specificaitons in the Syscall ABI in mind):

exit() closes the process’s file descriptors, give up the exiting process’s window if it has one and (as described above) the keyboard focus (i.e., foreground process status), records the exit status, and becomes a ZOMBIE. You can free the process table entry right away if the parent process has already exited. It must also free any of the process’s own children that are already zombies, wake up the parent process, and run the scheduler.
wait() returns -1 if pid is not a child of the caller. Otherwise, it blocks until the child is a ZOMBIE, then frees the child’s process table entry and returns its exit status.

Then, some other changes are needed to bring everything together:

A process getting killed can reuse the process exit code you just wrote. Change your Quest 1 fault handler to end the process with your exit code, with status -1, so that a parent waiting for a child that crashes is woken up.
On the userspace side, add SYS_exit, and SYS_wait, declare the two new system calls, and delete your exit() from user/lib/u_common.c: from now on, the generated exit stub is the real system call. At this point, make sure to read user/lib/u_start.c (see the yellow box below): returning from main() now ends a process properly because _start calls the exit() syscall.
To ensure the keyboard focus doesn’t get lost when processes exit, implement the focus rules from the Syscall ABI in your console code.

Programs now start at _start, not at main. In actuality, main isn’t the first function of your C program that runs: most OSes wrap it in a function called _start. We now also do this: user/lib/u_start.c contains a three-line _start that calls your program’s main() and then passes its return value to exit().

Note: After you implement spawn, you’ll need to run one extra command when starting up GDB, if you want to debug code in any user programs. See Using GDB with user programs in Appendix A for details.

Hints

In exit(), your kernel is running on the kernel stack of the process that is exiting. That is fine, as long as nothing can reuse that kernel stack before you have switched away from it in the scheduler. Why is that guaranteed in your kernel?
What should wait() block on? The parent needs to be woken when one of its children exits; hence, the parent’s own struct proc is a perfectly good wait channel.
A wait in a loop (while (child is not a ZOMBIE) block(...)) realizes the same pattern as you implemented for read, i.e., the woken parent re-checks its condition.
Search your kernel for != UNUSED and == UNUSED, and decide for each occurrence what should happen for a ZOMBIE process. In particular, a zombie must never get the keyboard focus again, and must not keep its window. And if the foreground process with the keyboard focus dies while no other process reads the keyboard, the focus should return to a “nobody” state.
Put the focus check where a process first becomes a keyboard reader (in Project 3, the code that records that a process reads the keyboard). Checking on every read() or peek() is wrong, as a child that polls the keyboard would take the focus back every time the user presses Shift+Tab, and the parent can never be reached again.
For further thought: a killed child’s exit status is -1. But this is also what wait() returns on an error, so your terminal cannot tell “crashed” from “not my child”. How does Unix’s waitpid(pid, &status, 0) avoid this? (You don’t need to implement anything for this.)

🤖 AI use/coding: mixed.

NOT allowed for exit(), wait() (including the ZOMBIE logic), making your fault handler go through exit (step 3), and the keyboard focus rules (step 6).
Allowed, including agentic coding, for the user-side declarations.

Why?

exit and wait are where the process life cycle gets subtle: who frees what, when, and who gets woken up. This requires only about forty lines of code, but these lines are important and force you to engage with concepts like when a process really ends.

For context, see the principles around AI use in CS 1670.

Part C: terminals that run programs

Your terminal from Project 3 (user/term.c) can only run its built-in commands, not arbitrary programs. In this part, you will teach it to start programs. Then, if you are in CS 1670, you will make a second copy of it that runs on the UART console, so that your OS has a graphical terminal and a text-based one. (If you are in CS 1690/2670, you will get there differently: in Quest 5, you will split your terminal into a shell and a terminal emulator, and run the same shell both in a window and on the serial console.)

Background: where does a program’s output go?

When your terminal runs a program, the new process spawns with the terminal’s own file descriptors 0 and 1. The terminal got those from init(), so they refer to the console. Therefore, a program that you start from your graphical terminal reads the keyboard and prints on the UART serial console—rather than sending its output back to the graphical and drawing text in that process’s window. A text program like squares therefore prints its output on the UART console, not in your graphical terminal’s window, and a program that reads the keyboard (BASIC, say) takes the keyboard focus when it first reads, because its parent (your terminal) has it.

If you are in CS 1690/2670, you will fix this in Quest 5, which will introduce pipes that allow programs spawned from a graphical terminal to send their output back to the terminal’s window, and to read input from the graphical terminal’s input processing code.

If you are in CS 1670, your graphical terminal will continue to have this limitation: while its built-in commands print to the terminal’s window, any program you start from it prints to the UART console. Your serial console terminal, however, will be able to run programs and print the output directly.

Task (both CS 1670 and CS 1690/2670): Make your terminal run programs.

Keep the commands from Project 3 in user/term.c (help, clear, echo, ps, and uptime), and add:
- <PROGRAM>: any command line that is not a built-in command is the name of a program. Spawn the program with the terminal’s own file descriptors 0 and 1, wait for it, and print its exit status.
- <PROGRAM> &: spawn the program, print its PID, and don’t wait for it.
- run <PROGRAM> and run <PROGRAM> &: the same, for a program whose name is also the name of a built-in command.
- wait <pid>: wait for a child started with &, and print its exit status.
Make ps show each process’s parent and its state, including ZOMBIE.
Change init() to load only idle and term. You can now start every other program from the initial terminal.

Hints

Try squares & a few times, then look at ps. Why are they still there, and what makes them go away?
What happens if you start basic & (the graphical one) and then type? Where does your input go, and how do you get back to the terminal?
If a program you start never reads the keyboard and never exits (counter, say), your terminal waits for it forever. That is what Unix’s Ctrl-C is for; see the extra credit.

🤖 AI use/coding: allowed, including agentic coding.

Important note: you must still understand what happens in the kernel when your terminal runs squares—from the spawn system call to the wait returning—and be able to explain it in your whiteboard discussion.

Why?

The terminal an application that parses a command line and calls your three new system calls. While we want you to understand how it works, the details of the code (e.g., command line parsing) are okay to speed through with AI if you prefer.

For context, see the principles around AI use in CS 1670.

If you are in CS 1690/2670, skip the next task. Your term.c should stay as-is, and you should run Part D and the checks below from that terminal application. In Quest 5, you will split term.c into a shell (sh.c) and a terminal emulator (gterm.c), and run your shell on the serial console, too.

We provide you with user/lib/u_sterm.c and u_sterm.h, which implement a text console on the serial line, in the style of your Project 3 u_vterm library: they provide functions sterm_init(), sterm_clear(), sterm_putc(), sterm_puts(), and sterm_printf(). These functions write to file descriptor 1 (stdout) with the write() system call, and use ANSI escape sequences to clear the screen and move the cursor.

Task (CS 1670 only): Turn your terminal into two: a graphical terminal in a window (gterm), and a text-based terminal on the serial console (sterm).

Rename user/term.c to user/gterm.c using git mv. This is your graphical terminal, and you can leave it as-is.
Copy user/gterm.c to user/sterm.c, and switch the copy from your graphical text console to the sterm_* functions (in user/lib/u_sterm.h): this terminal will no longer ask for a window, and its output goes through the sterm_* functions. Read input with getc() as before.
Change init() to load idle and sterm: your OS now boots into a terminal on the UART serial console. Start your graphical terminal from there with gterm &; Shift+Tab moves the keyboard between the two.

Hints

sterm_putc() turns '\n' into "\r\n", which the serial console needs; the programs you start need to print "\r\n" themselves.
When sterm starts gterm &, why does the keyboard go to gterm? (Look at the focus rules in the Syscall ABI.)
Arrow keys arrive as escape sequences (ESC [ A and so on). If you support them (e.g., because you did the early extra credit), your text-based terminal needs to handle or ignore them, as they otherwise end up in your command line as [A.
You now have the same command code in two programs. That is fine for this project, but if you want to make things neater, you could factor the code out into a library in user/lib/.

🤖 AI use/coding: allowed, including agentic coding.

Why?

Porting your terminal from one text console library to another is a largely mechanical task.

For context, see the principles around AI use in CS 1670.

Part D: run a program you did not compile

We provide you with user/prebuilt/snake.elf, which is a text-based Snake game implemented atop your OS’s syscall ABI. We compiled the program, from source code you have not seen, against our Syscall ABI, with its own copy of every system call stub. It uses write() to draw with ANSI escape sequences, peek() and read() to read the arrow keys (or WASD) without waiting, ticks() to pace itself, yield() while it has nothing to do, and exit() when the game is over.

Task (CS 1670 and CS 1690/2670): Run snake from your terminal (sterm in CS 1670, term in CS 1690/2670).

The build system already embeds snake in your kernel image, like your own programs, and lists it in the generated program table, so spawn can find it without any changes.
Type snake in your terminal, and play. Snake draws on the serial console, even if you start it from a terminal in a window, and takes the keyboard while it runs (see above).

If your kernel implements the ABI correctly, it works without any change. If it doesn’t, Snake will tell you a lot about your system calls: use make qemu-verbose and gdb (on handle_syscall) to find out which system call it makes, with which arguments, and how your kernel handles it.

🤖 AI use/coding: allowed, including agentic coding—though there should be nothing to write. If you need to debug, AI may help you interpret what you see.

Why?

This is the payoff of learning objective 4: your OS now supports a binary executable compiled against a fixed interface runs on any kernel that implements that ABI correctly.

For context, see the principles around AI use in CS 1670.

Check your work

Run make qemu-fb (or make qemu-ws). If you are in CS 1670, your OS should boot into sterm on the serial console, with only idle and sterm running; if you are in CS 1690/2670, into your graphical term, with only idle and term running (the output of text programs appears on the serial console). Then check that:

Text programs run, and return. squares and pi print their output, finish, and the terminal prints their exit status (0) and gives you a new prompt. basic_console works, with your typing going to BASIC, until you type BYE. Programs that never finish (hello, counter, primecheck, keytest) work with &.
Graphical programs run in windows. clock &, pong &, basic &, and logo & each open a window, until all four cells of the screen are taken (a graphical terminal takes one, too); one more logo & then finds no free cell and prints a message rather than drawing over someone else.
Snake works. snake, play a game, and come back to your terminal when it ends.
Misbehaving programs are stopped. diab_irqoff, diab_timer, diab_vectors, diab_crash, and diab_brk: each is killed, your terminal reports an exit status of -1, and your terminal, your windows, and your keyboard keep working. diab_stack & keeps spinning without harming anyone. Compare this with your notes from Quest 1, Part A!
Zombies come and go. After squares & finishes, ps shows it as a ZOMBIE until you wait for it.
Nothing is left of the trampoline. No F_BASE, no code that installs trampoline pointers, no FB_INFO_BASE: nothing in your kernel puts data or pointers at a fixed address for programs to find.
(CS 1670 only) Two terminals exist. gterm & in sterm opens your graphical terminal and gives it the keyboard; Shift+Tab moves between the two, and programs started from either one work.

Task (CS 1670 only): Save a transcript of a session in sterm in which you start at least squares, snake, two graphical programs, and one diabolical program, and run ps, as docs/terminal.txt in your projects repository, and push it. This is part of your project submission. (CS 1690/2670 students record theirs at the end of Quest 5, on the serial console.)

🤖 AI use/coding: NOT allowed. Why? The transcript is evidence of what your OS does: it must be the unedited output of a real session (copy it from your terminal, or record it with script or tee on your computer).

Congratulations—you have separated kernel and userspace and can start and exit programs on your OS! Your processes can no longer use privileged instructions to take over the CPU, the timer, or the exception vectors, and the system call interface is the only deliberate way into your kernel. (We still need virtual memory to complete the security mechanisms of your OS, and we’ll add that in Project 5.) If you are in CS 1670, you’re done with the project, so scroll down and hand it in! If you are in CS 1690/2670, Quest 5 turns your terminal into a real shell, which runs both in a window and on the serial console.

Bonus Quests (Extra Credit)

Quest 4-E1: kill() and Ctrl-C (moderate difficulty)

Task (extra credit): Add a kill(pid) system call that ends another process (with exit status -1), and add a kill <pid> command to your terminal. Then make Ctrl-C (byte 0x03) end the program that currently has the keyboard focus, as it does on Unix—unless that program is the terminal itself.

Think about who should be allowed to kill whom, and what happens if the victim is blocked in the middle of a system call.

Quest 4-E2: Arguments for programs (moderate difficulty)

Task (extra credit): Let spawn pass a list of string arguments to the new program, so that main(int argc, char** argv) works: run echo hello world should start a program that prints its arguments. You will need to decide how the arguments travel from the parent’s memory to the child’s, where in the child’s memory they go, and how _start finds them.

Quest 5: A real shell (CS 1690/2670)

Files: kernel/pipe.c, kernel/pipe.h, kernel/file.c, kernel/file.h, kernel/console.c, kernel/syscall.c, kernel/syscall.h, kernel/proc.c, kernel/init.c, user/lib/u_syscalls.h, user/term.c, user/gterm.c, user/sh.c, user/basic.c, and user/vtty.c, user/gsnake.c, user/lines.c, user/upper.c, and user/count.c (from our repository of provided programs).

This quest is required for CS 1690/2670 students only. CS 1670 students are welcome to do it, but it is not graded for you.

Overview

Your terminal from Quest 4 does two jobs at once: it is a terminal emulator (it reads keys and draws characters) and a shell (it understands commands and starts programs). Real systems separate the two: the terminal emulator on your laptop and the shell running inside it are different programs, connected by a kernel object. In this quest, you will:

Add pipes to your kernel: a kernel buffer with a write end and a read end, each a file descriptor.
Split your terminal from Quest 4 into a terminal emulator and a shell, which talk to each other through two pipes, and run the same shell on the serial console, through a virtual terminal for the serial console that we provide (vtty). A new system call, open_console(), gives a terminal emulator that a shell started access to the keyboard.
Support pipelines in your shell: lines | upper runs two programs, with the first one’s output feeding the second one’s input.

At the end, your OS boots into a shell on the serial console. From there, you can open terminals in windows, and more terminals from those, each running a shell of its own.

Why are we doing this?

Pipes build on the file descriptor abstraction you created earlier in the project. A program that reads file descriptor 0 and writes file descriptor 1 never knows whether it is talking to the console, a pipe, or (in Project 6) a file, so small programs can be combined in ways their authors never planned. This composability is at the core of the Unix philosophy, and you will see that it needs very little new mechanism: a buffer, two kinds of file descriptor, and careful reference counting are sufficient.

Background: pipes, inheritance, and end-of-file

A pipe(int fds[2]) syscall creates a pipe and two new descriptors in the calling process: fds[0] is the read end, and fds[1] is the write end. Bytes written to the write end come out of the read end, in order. A pipe has a fixed-size buffer inside the kernel (your Project 3 ring buffer is sufficient for this):

read on the read end blocks while the pipe is empty; write on the write end blocks while it is full.
When the pipe is empty and no write end is open anywhere, read returns 0. This is how the reading program learns that its input is over. peek then returns -1 (and not 0, which means “empty, but more may come”), so that a program that never blocks can detect a pipe being closed.
When no read end is open anywhere, write returns -1: nobody will ever read the data.
close(fd) closes a file descriptor. When the last file descriptor that refers to an end of the pipe is closed—in any process—that end is closed.

The word “anywhere” above is important. Through spawn, two processes can share a descriptor’s underlying struct file, which is why you need reference counts. A shell pipeline such as lines | upper works like this:

 the shell runs:                        which sets up:

 pipe(p);                               ┌───────┐ fd 1       ┌──────────────┐       fd 0 ┌───────┐
 a = spawn("lines", 0, p[1]);           │ lines ├──────────► │ pipe buffer  ├──────────► │ upper │
 b = spawn("upper", p[0], 1);           └───────┘  (p[1])    │  (in kernel) │   (p[0])   └───────┘
 close(p[0]); close(p[1]);                                   └──────────────┘
 wait(a); wait(b);                      lines's stdin and upper's stdout are the shell's own

                                        the shell's copies of p[0] and p[1] must be closed!

When lines exits, its file descriptor 1 is closed. If that was the last open write end, upper’s next read returns 0, and upper also exits. But if the shell forgot to close its copy of p[1], the write end is still open, upper waits for input forever, and the shell waits for upper forever. Forgetting to close descriptors is a big source of common pipe bugs!

What you need to do

You’ll first add pipes to your OS, then split your terminal into a shell and a terminal emulator, and finally implement pipelines in your shell.

Part A: pipes

Task: Implement the pipe() and close() syscalls according to the Syscall ABI.

Define a pipe (e.g., in kernel/pipe.c). The pipe should be represented by a struct that consists of a ring buffer, the number of open read ends and open write ends, and whatever wait channels you need. Pipes need memory that outlives any single system call, so add a fixed-size table of pipes in the kernel (akin to the processor descriptor list).
Add two new struct file types, the read end and the write end of a pipe, and make your read, write, and peek system calls work on them, with the semantics described above.
Implement close(fd), and make your struct file reference counts work: spawn increments them, close and exit decrement them, and a pipe end is closed when its count reaches zero. Closing the console is fine, but the console itself never goes away.
Copy three small programs to test with from our repository of provided programs into user/: lines.c, which prints a few lines of text and exits; and two filters, which read file descriptor 0 until end-of-file and write to file descriptor 1: upper.c (converts letters to upper case) and count.c (prints the number of lines and characters it read, at the end). These programs use your printf(), read(), and write(). The build adds each new program to the generated program table, so spawn finds it automatically.

Hints

A writer blocked on a full pipe and a reader blocked on an empty pipe wait for different events; use different wait channels, or wake everybody and let them re-check.
When a pipe end is closed, wake up anyone waiting on the other end: they need to see end-of-file, or an error.
Test without a shell first: write a test program that creates a pipe, spawns lines with the write end as stdout and upper with the read end as stdin, closes both ends, and waits for both.
A filter program run on its own, with your terminal as its input (e.g., just upper), should never finish: our terminals have no way to send end-of-file, as Ctrl-D does on Unix. You can only use filter programs in pipelines.
Size your pipe table generously: every terminal will hold two pipes for as long as it runs, so you’ll want at least 32 pipes.

🤖 AI use/coding: auxiliary use allowed. You should design your pipe struct and its reference counting logic yourself, but you may use AI to help you write the code and to adapt the read, write, and peek system calls.

Why?

The important learning goal here is to understand how a pipe interacts with the file descriptor abstraction and what a kernel must do to implement its semantics. The implementation details are less important than the design here.

For context, see the principles around AI use in CS 1670.

Part B: a terminal emulator and a shell

Now, we’ll add a proper shell. Your shell will run in two places: in a window, behind your graphical terminal emulator (gterm); and on the serial console, behind a virtual terminal (vtty) for the UART serial console that we provide (vtty.c, in our repository of provided programs). Like getty on Unix, vtty makes two pipes, starts sh between them, and then copies everything typed on the serial console into the shell’s input pipe, and everything that comes out of the shell’s output pipe onto the serial console. (If the shell ever exits, vtty starts a new one.)

Both terminals are deliberately thin: they pass every keystroke onwards immediately and echo nothing. So echoing what the user types, and editing the command line, is the shell’s job. This is also true in real shells such as bash, whose line editor is a library (readline) inside the shell. This means that every program sees the same raw keystrokes behind either terminal as on the bare serial console: BASIC, which echoes its own input, and snake, which wants every key the moment it is pressed, work behind vtty just as they did in Quest 4.

Does this match real OSes? Yes, for the most part. The pipe design between terminal emulator and shell is the same as on Unix-like OSes (e.g., Linux, macOS).

The main difference is that Unix’s getty does line editing and echoing, while your vtty does not. Similarly, macOS brings a library called termios that handles line editing and echoing for programs that want it, while your shell does it itself. This matches what happens with the popular bash and zsh shells, though: they both disable the “line discipline” in getty/termios, and do their own line editing and echoing in the shell process. However, unlike your shell, they turn line discipline back on before running a program, so that cat and less work as expected. You won’t do that here, but you could.

There is some complexity to deal with in order for things to work as expected with graphical terminal emulators, though. You will start gterm from a shell: gterm & on the serial console, or gterm in another gterm. Then gterm inherits the shell’s file descriptors 0 and 1, which are pipes from the other terminal. If it read its keystrokes from file descriptor 0, it would get them second-hand, through a terminal that keeps the keyboard focus, and its own window could never have the focus. Hence, gterm asks the kernel for a file descriptor of its own that allows it to access the console directly. This relies on a new system call, open_console(), and any graphical program that wants the keyboard can do the same.

Task: Split your terminal into a shell and a terminal emulator.

The shell, user/sh.c, contains your terminal’s command logic and its line editing, but does no drawing. The shell reads keystrokes from file descriptor 0, one at a time, echoes them to file descriptor 1 and handles backspace as it goes, and writes everything else to file descriptor 1, too. The shell also starts programs with spawn, passing on its own file descriptors 0 and 1. It supports the built-in commands from Quest 4, Part C (help, clear, echo, ps, uptime, wait, and run). You should also add exit, which ends the shell, and treat any other command line as the name of a program to run (i.e., squares means the same as run squares).
The terminal emulator, user/gterm.c, is your graphical term.c, without commands or line editing (delete term.c once you have both). It calls init_canvas() and open_console() (step 4), and closes the file descriptor 0 it was started with. Then gterm creates two pipes and spawns sh with one pipe’s read end as its stdin and the other pipe’s write end as its stdout. Finally, gterm closes the pipe ends it won’t use, and loops: every keystroke from the keyboard (the descriptor that open_console() returned) goes into the shell’s input pipe as soon as it arrives; and whatever gterm reads from the shell’s output pipe is drawn on the canvas.
Use peek() to find out whether the keyboard or the shell’s output pipe has data, so that the terminal emulator (gterm) never blocks on one pipe while the other has data for it, and yield() when neither pipe has data. When peek() on the shell’s output pipe returns -1 (end-of-file), the shell has exited, so wait() for the shell process and exit.
Open the console in gterm. Implement the kernel and userspace parts of the open_console() syscall (number 51, as specified in the Syscall ABI). This syscall puts a console “file” in the caller’s lowest free descriptor, and makes the caller a keyboard reader at once. Add SYS_open_console to kernel/syscall.h, and declare the system call in user/lib/u_syscalls.h. Then extend your keyboard focus rules from a process’s parent to its ancestors, as the Syscall ABI describes. A program that a shell started then still takes the focus from the terminal that the shell runs in, and gives it back to that terminal when it exits.

Hints

Programs started by the shell inherit the shell’s file descriptors, so their output appears in the terminal emulator’s window (or on the serial console, behind vtty), and their input comes from the terminal emulator—without the shell having to do anything.
Use the sterm_* functions for the shell’s output: they write to file descriptor 1 and turn '\n' into "\r\n", which the serial console needs and your graphical text console can handle. sterm_clear() sends the ANSI escape sequences ESC [ 2 J (clear the screen) and ESC [ H (cursor home), which vtty passes to your host computer’s terminal; in gterm, you need to implement those two escape sequences and ignore any other escape sequence. (Programs that draw with escape sequences, like snake, won’t work in gterm.)
If the shell hangs after it starts a program, check which pipe ends are still open in which process.

🤖 AI use/coding: mixed.

Allowed, including agentic coding, for moving the command logic and line editing into sh.c, and for the terminal emulator’s drawing.
Auxiliary use allowed for everything else. Write the code in the terminal emulator that creates the two pipes, spawns sh with the right ends, and closes the ends that the terminal emulator does not use yourself. Update your keyboard focus rules, too.

Why?

The two programs are application code built from parts you already have. The design—who owns which descriptor, and which ends must be closed—is the interesting part, so we want you to focus on that.

For context, see the principles around AI use in CS 1670.

Now, run your shell on the serial console too.

Task: Run vtty with your shell on the serial console, and test that it can start graphical terminals and graphical programs.

Copy vtty.c from our repository of provided programs into user/, and read it: it does for the serial console what your gterm does for a window, using your system calls and your printf(). Change init() to start only idle and vtty (boot_programs is now {"idle", "vtty"}): your OS now boots into a shell on the serial console. Type gterm & there to open a shell in a window, and gterm in that window to open another. Each new window takes the keyboard focus, and Shift+Tab moves it between terminals.
Test it with a game. Copy gsnake.c (a graphical snake game) from the same repository into user/. Like your gterm, this program opens the console for its keystrokes, and closes the descriptor 0 it was started with. It does its own drawing, so all it needs from your OS is system calls.
- Run gsnake in a gterm. Its window should take the keyboard focus, and every key should go tot he game: the shell in your gterm must not see any keystrokes (live or after gsnake exits).
- When you quit (q), the game should print your score in the gterm you ran it from, and the focus should go back to that window.
- Try gsnake & on the serial console, too, and move the focus around with Shift+Tab while it runs.
Test it with BASIC. Your graphical basic reads its keys from file descriptor 0, so behind vtty or in a gterm it would fight the shell for keystrokes. Change the WIRING section of your user/basic.c the way gsnake implements I/O:
- In basic_start(), after init_canvas(), open the console, and close file descriptor 0.
- In basic_getc() and basic_key_waiting(), read from and peek() at the file descriptor that open_console() returned, instead of using getc() and hasc().
- As you did for basic_console in Quest 3, replace the endless while(1) { loop(); } in main() with loop(); return 0;. Then BYE ends BASIC, which gives the focus back to the terminal you started it from.

Hints

Why must gterm close the file descriptor 0 it was started with? Try it without: start gterm & on the serial console, then type exit there, and see whether vtty notices that its shell has gone. (Which pipe ends are still open, and in which processes?)
After exit, your emulator sees end-of-file on the shell’s output pipe only once every write end is closed. A program the shell started with & inherited the shell’s file descriptor 1, so the pipe stays open until that program exits, too. (That includes a gterm, if it keeps its file descriptor 1 for error messages.) In that case, the emulator finds out when its next write() to the shell’s input pipe returns -1 (as long as nobody else holds that pipe’s read end). vtty checks both and starts a new shell (like getty would on Linux).

🤖 AI use/coding: allowed, including agentic coding, but there is no code to write except for bug fixes.

Why?

This task is about connecting things up: provided that you did the prior tasks correctly, everything should just work!

For context, see the principles around AI use in CS 1670.

Part C: pipelines

Now that we’ve used pipes to connect shell and terminal emulator, we can re-use them inside the shell to connect programs to each other. This enables shell pipelines, a core concept of Unix-like operating systems’ command line user experience.

Task: Support pipelines of two or more programs in your shell: lines | upper, lines | upper | count, squares | count, and so on (with or without run in front). The shell creates one pipe between each pair of neighbors, spawns every program with the right descriptors, closes its own copies of all pipe ends, and waits for every program in the pipeline.

🤖 AI use/coding: auxiliary use allowed. You may use AI to help with parsing the command line. Write the code that creates the pipes, spawns the programs, and closes descriptors yourself.

Why?

A shell pipeline requires invoking several system calls in exactly the right order. We discussed those system calls in our seminar session, except that spawn replaces fork and exec in your OS.

For context, see the principles around AI use in CS 1670.

Check your work

make qemu-fb (or make qemu-ws) should boot into a shell on the serial console (behind vtty), with idle, vtty, and sh running. Type gterm & there to open a shell in a window, too. Then, in each of the two terminals:

The shell works: help, echo, clear, ps, and squares behave as in Quest 4, with all output in that terminal, and backspace works as you type.
Pipelines work: lines | upper prints your text in upper case and returns to the prompt; lines | upper | count and squares | count print the counts. If a pipeline never returns to the prompt, some write end is still open.
ps while a pipeline runs shows every member of the pipeline, with the shell as their parent. Start a pipeline that never ends, such as counter | upper, in gterm, press Shift+Tab to get to the shell on the serial console, and run ps there.
Terminals start terminals, and exit ends them: gterm in a gterm opens another window, which takes the keyboard focus (its border lights up). exit there closes that window at once, and the focus goes back to the gterm you started it from. exit on the serial console makes vtty start a new shell.
Graphical programs get the keyboard: gsnake and basic, started in either terminal (with or without &), take the focus and get every key directly. They give the focus back to that terminal when you quit (q in gsnake, BYE in BASIC). gsnake’s score appears in that terminal.
Quest 4 still works on the UART serial console: basic_console until BYE, snake, graphical programs with &, and the diabolical programs.
It all works on the hardware: run all of this on your Raspberry Pi as well, including the diabolical programs from Quest 1.

Task: Save a transcript of a session on the UART serial console in which you start at least squares, snake, two graphical programs, one diabolical program, and a pipeline, and run ps, as docs/terminal.txt in your projects repository, and push it. This is part of your project submission.

Bonus Quests (Extra Credit)

Quest 5-E1: Background jobs and job control (moderate–high difficulty)

Task (extra credit): Add job control to your shell: cmd & runs a pipeline in the background, jobs lists the background pipelines and their state, fg brings one back to the foreground (so that the shell waits for it again), and a background job that finishes is reported at the next prompt. Ctrl-C (if you did Quest 4-E1) should end only the foreground job.

You will need a way for the shell to find out whether a child has exited without blocking. Think about whether that is a new system call, or a variant of wait.

Handing in & grading

You will submit your work via git, using our grading server, exactly as for the previous projects.

Before you submit, make sure that:

make clean && make qemu builds and boots into your serial terminal from a fresh checkout, with only idle and sterm started at boot (CS 1670), or idle and vtty (CS 1690/2670).
Your code is formatted (make format) and free of debugging output you did not intend to leave in. (Lines inside DEBUG() can stay: they cost nothing unless you build with DEBUG_LVL=1.)
Your memory layout diagram, including the kernel stacks, is up to date and committed as docs/memlayout-physical.pdf.
Your terminal transcript (from Quest 4 in CS 1670, from Quest 5 in CS 1690/2670) is committed as docs/terminal.txt.

Submitting your code (and checking on the grading server)

Log into the grading server with the password you received.
Connect your repository under the “Project 4: System Calls” category, if you have not already, and click it to fetch your repository from GitHub.
Check your code works: use the buttons below the commit list to test your code in our grading environment.
Set your grading commit with the “Grade this commit” button. You can reassign it at any time before the 72 late hour cutoff.
After submitting your work, sign up for a whiteboard discussion.

Extra requirements for CS 1690/2670 students

In addition to submitting the project, you must also check that your code works on the real hardware. Two parts of this project deserve particular attention on a real Raspberry Pi:

What EL0 may touch. Your kernel’s settings for CNTKCTL_EL1 (and the SCTLR_EL1 bits that control EL0 access, which your boot code sets) decide what a process at EL0 may do with the timer and the interrupt masks. QEMU resets some of these registers to convenient values that the hardware does not guarantee. Run all six diabolical programs on the Pi, and check that each one behaves as in QEMU.
The UART serial console as a terminal. Your terminal, Snake, and your shell all depend on escape sequences and keystrokes arriving over the real serial line, which is less forgiving than QEMU’s.

You can test with your own Raspberry Pi. Note that not all of these tests can be done on the remote-accessible Raspberry Pi, since some key combinations get captured by your web browser.

Congratulations—you have completed the project and made your OS a whole lot more realistic! 🎉

Grading breakdown

Your grade is based on:

Functionality: We will run your code on the grading server and check that it works as expected (i.e., processes run at EL0, misbehaving processes are killed, and your terminal can start text and graphical programs, including snake). This is worth 50% of your grade and impacts the “Projects” part of the course grade.
Whiteboard Discussion: You will meet with a course staff member in a small group to discuss the concepts in this project for 45 minutes. The discussion is worth 50% of your grade and impacts the “Whiteboard Discussions” part of the course grade.
If you are in CS 1690/2670, the additional work you must complete will contribute 1/5 of your overall grade (i.e., the contributions of functionality and whiteboard discussion will be scaled to 40% of your grade each, and the extra work will be another 20%).

We will grade according to our honest grading scale.

Project 4: Privilege Separation and System Calls

Learning objectives

CS 1670 vs. 1690/2670

Roadmap

Assignment Installation

Code Structure

Building and running

Background

A process should not be as powerful as the kernel

System calls: a protected control transfer

Quest 1: Who’s in charge?

Overview

Why are we doing this?

What you need to do

Part A: misbehave

Background: a kernel stack for every process

Part B: a kernel stack, and down to EL0

Part C: kill, don’t panic

Check your work

Quest 2: Knocking on the kernel’s door

Overview

Why are we doing this?

Background: the system call ABI

What you need to do

Part A: the user side

Part B: the kernel side

Part C: the first system calls

Check your work

Bonus Quests (Extra Credit)

Quest 3: File descriptors

Overview

Why are we doing this?

What you need to do

Part A: safe copies

Background: never trust a pointer from user space

Part B: a file descriptor table, and write()

Background: file descriptors

Part C: read(), peek(), and blocking in a system call

Background: blocking in the middle of a system call

Check your work

Bonus Quests (Extra Credit)

Quest 4: Tearing down the trampoline

Overview

Why are we doing this?

What you need to do

Part A: no more shortcuts

Part B: spawn, exit, and wait

Background: the life cycle of a process

Part C: terminals that run programs

Background: where does a program’s output go?

Part D: run a program you did not compile

Check your work

Bonus Quests (Extra Credit)

Quest 5: A real shell (CS 1690/2670)

Overview

Why are we doing this?

Background: pipes, inheritance, and end-of-file

What you need to do

Part A: pipes

Part B: a terminal emulator and a shell

Part C: pipelines

Check your work

Bonus Quests (Extra Credit)

Handing in & grading

Submitting your code (and checking on the grading server)

Extra requirements for CS 1690/2670 students

Grading breakdown

Part B: a file descriptor table, and `write()`

Part C: `read()`, `peek()`, and blocking in a system call