Appendix B: Debugging devices and displays

Everything in Project 1’s appendix (the gdb cheat sheet) and Project 2’s (context switches, and why timing bugs are different) still applies. This project adds two new categories of wrong:

A device that does not interrupt, or interrupts forever. The failure is usually three registers away from where you are looking.
A screen. The good news is that a display is the most informative debugging output you have ever had: it shows you a megabyte of your program’s state at a glance. The bad news is that when it is wrong, it is wrong in ways that look like nothing you have seen.

The CHECK() and DEBUG() macros from Project 2’s Appendix D earn their keep here more than anywhere so far. A CHECK() that a pixel coordinate is inside the framebuffer turns “the machine died and I have no idea why” into a one-line panic.

Failure modes: input

Symptom	Usual cause
Nothing at all happens when you type	In order: you did not write `IMSC`; you wrote it before enabling the UART in `CR`; you set the wrong bit in `ENABLE_IRQS_2`; you are testing the timer’s bit rather than the peripheral bit in the core interrupt source register.
The console floods, or the machine locks up, on the first keypress	An interrupt storm: you are not acknowledging in `ICR`. The UART keeps asserting its line, so you take the same interrupt forever.
The machine hangs silently on the first keypress	`handle_interrupt` fell off the end instead of finishing with `restore_context(trapframe)`. Control returns to the IRQ stub in `exceptions.S`, whose next instruction is the `b .` you put there in Project 2. Check with `Ctrl-c` in `gdb` and `x/4i $pc`.
Works perfectly in QEMU, dead on a real Pi	The `IFLS` receive trigger level. The FIFO does not raise `RXI` until it is half full, so single keystrokes produce nothing. Unmask the receive timeout interrupt as well, or lower the trigger level. QEMU interrupts on every byte and hides this completely.
The first few characters arrive, then silence	You are reading one byte per interrupt instead of draining the FIFO in a loop.
Characters vanish when you paste or hold a key	Your ring buffer overflowed (print the dropped counter—this is what it is for) or its wrap-around arithmetic is wrong.
A process blocks and never wakes	Three candidates: you are not calling `wakeup` from the interrupt handler; the waker and the sleeper are using different addresses as the wait channel; or you have a lost wakeup, because you did not mask interrupts across the check-and-block.
`panic: no runnable process`	Everything is blocked and you have no idle process yet, or your `pick_next` refuses to fall back to it.
`getc()` never returns, even after a keystroke arrives	You gave up the CPU with the kernel’s `yield()` rather than the `yield_entry` stub, so no context was ever saved and the process cannot be resumed.

Failure modes: the display

Symptom	Usual cause
The screen stays black	`fb_init` failed (check `mbox_call`’s return and the response code in `buf[1]`), or it succeeded and you forgot to mask the top two bits off the bus address. Print the address you got.
The image is sheared diagonally	You used `width` as the row stride instead of `pitch_w`. This is the single most common framebuffer bug and it is instantly recognizable once you have seen it once.
The colors are wrong—red where you wanted blue	A pixel is `0xAABBGGRR`. The last two hex digits are red. (If everything is wrong rather than swapped, check the pixel-order tag in your mailbox request.)
Text comes out mirrored	The font’s bit 0 is the leftmost pixel, not the rightmost.
Every character is a solid block	You are testing `glyph[row]` for non-zero instead of testing one bit of it.
Garbage appears in someone else’s window	In Quest 3: a clipping bug in your drawing library, or your kernel’s `struct fb_info` and the user-side copy in `u_fb.h` have drifted apart. In Quest 4: the kernel’s and the user side’s `struct canvas` have drifted apart, or your compositor’s copy uses the wrong row stride—a canvas’s rows are `width` pixels apart, the screen’s are `pitch_w`.
A window is sheared, or shows a slice of its neighbor	Same thing, in the compositor: one `memcpy` per row, each row starting `pitch_w` pixels below the last on the screen and `width` pixels further on in the canvas.
It still tears after you added double-buffering	Your hidden-half index is inverted: you are drawing into the half on the screen and telling the GPU to show the other one. Getting this backwards looks exactly like the tearing you were trying to fix.
A window flashes back to an older picture now and then	In Quest 4: you held back a mid-frame window without bringing the hidden half up to date from the visible one, so the flip showed a frame that is two refreshes old. In Quest 5: damage detection copied a change into only one half of the framebuffer; a change must reach both halves, one refresh after the other.
A window never updates	In Quest 4: its program is never blocked when a refresh comes round, so its window is always held back—check that it really calls `wait_for_vsync()` between frames. In Quest 5, also: its `changed` flag is never set—a program that draws without your library has to set it itself.
The machine dies shortly after the first refresh	Your refresh path is reusing the same mailbox buffer `fb_init` used. Give the flip its own 16-byte-aligned buffer; you are writing to it from an interrupt handler.

Not everything ugly is a bug. In Quest 3, a display that tears and shows objects cut in half is correct behaviour for the design you have—your processes are writing into the very memory the display hardware is reading from, with no coordination at all. Do not spend an afternoon trying to fix it in Quest 3. Quest 4 fixes it properly.

What is a bug at that stage: a stray pixel in a neighbour’s window, an image that is sheared rather than torn (tearing is horizontal; shearing is diagonal and static), or output that never appears at all.

Looking at the screen from outside

You do not have to squint at the display window. QEMU will hand you the framebuffer as an image file, which is useful for comparing two runs, for looking closely at a glyph you have just designed, and for putting a screenshot in your hand-in.

While your OS is running under make qemu-fb or make qemu-ws:

Press Ctrl-a then c to switch from the serial console to QEMU’s own monitor (this is the toggle from Project 1’s appendix). The prompt changes to (qemu).
Type:
```
screendump /tmp/shot.ppm
```
Press Ctrl-a then c again to switch back to your OS’s console. Your OS has been running throughout; the monitor does not pause it.

The result is a PPM file, which most image viewers open directly. If yours does not:

sips -s format png /tmp/shot.ppm --out /tmp/shot.png   # macOS
pnmtopng /tmp/shot.ppm > /tmp/shot.png                 # Linux (netpbm)

A trap worth knowing about. A screendump catches the framebuffer at one arbitrary instant. Before Quest 4, that instant may well fall between the moment your program erased something and the moment it drew it again—so a program that looks fine on screen can screendump as a blank rectangle, and vice versa. Even after Quest 4, QEMU’s display is not synchronized with your flips, so a very occasional dump can show a torn frame that your compositor never produced. If a screenshot surprises you, take three more before you believe it.

`gdb` recipes for this project

The framebuffer and the input path are both just memory and registers, so the debugger still works; you only need to know what to look at.

Goal	How
See what `fb_init` actually produced	`p (struct fb_info )FB_INFO_BASE` (substitute your address; after Quest 4, wherever the kernel now keeps it). Check `pitch` against `width * 4` while you are there.
See a process’s canvas	`p *processes[N].canvas`, or whatever you called it, then `x/8wx` its `pixels`. In Quest 5, check that `changed` goes to 1 when the program draws and back to 0 after a refresh.
Check the framebuffer really is where you think	`x/8wx fb->pixels` after writing a known color; you should see it repeated.
Watch a mailbox call	`break mbox_call`, then `x/36wx $x0` before it and again after `finish`. Compare the response words against the property interface wiki page.
Find out who is scribbling on your window	`watch (unsigned int )ADDR`, where `ADDR` is a pixel that keeps getting corrupted. `gdb` stops at the instruction that wrote it, and `bt` tells you which process. This is the fastest way to find a clipping bug.
Inspect the keyboard buffer	`p input_buf` (or, after Quest 2’s Part E, `p processes[1].input_buf`).
See why a process is asleep	`p processes[N].state` and `p/x processes[N].blocked_on`, then compare that address against the one your waker passes to `wakeup`.
Confirm a process really blocked rather than spun	`break block`, then `bt`. You should see your `getc` and the user program below it on one stack.
Catch an interrupt in the act	`break uart_interrupt`, press a key, then `p current_process->name`—it will be a different process each time, which is the point.

Techniques that work better than a debugger here

Decide first whether it is the kernel or the program. Almost every display bug is one or the other, and you can separate them in a minute: draw a rectangle directly from kernel_main, right after fb_init returns. If that appears, your driver and your fb_info are fine and the bug is in user space. If it does not, stop looking at your user programs.
Use a pixel as a printf. In code where printing is too slow or too disruptive—an interrupt handler, the middle of a refresh—write a single pixel a known color instead. One store to memory, no formatting, no UART. A row of eight such pixels makes a perfectly good status display, and you can read it at a glance while the machine runs.
Count, do not print, in interrupt handlers. Increment a global; print it from somewhere calm. A printf in a handler takes tens of thousands of cycles with interrupts masked and changes the timing of the very thing you are trying to observe. (This is the same advice as Project 2’s event-ring-buffer technique, and for the same reason.)
Keep the dropped-byte counter. It costs one uint32 and it is the difference between “characters sometimes go missing” and “my buffer overflowed 14 times”. Print it in your terminal’s ps output, or in the status bar if you do that extra credit.
Bounds-check while developing, even though it costs you. A CHECK(x < fb->width && y < fb->height) inside your pixel function catches the entire class of bug where bad geometry writes past the end of the framebuffer into whatever is next in physical memory. With no MMU, the alternative symptom is your kernel misbehaving several seconds later for no visible reason.
Reduce to one process. Most of this project’s bugs reproduce with a single program loaded, and a single program’s output is something you can predict exactly. Add the others back once it works.
Print fb_info once at boot and leave it there. width, height, pitch, and the framebuffer address are the four numbers that nearly every subsequent display bug turns out to relate to. Having them scroll past at every boot costs nothing and saves you re-deriving them.

When the machine hangs and gdb is no help: as in Project 2, make qemu-verbose passes -d int to QEMU and logs every exception taken. In this project the most useful thing it shows you is an interrupt you are not acknowledging—you will see the same exception repeating forever, which is not something you can easily tell from the inside.