Project 0: Setting up your development tools

This project will demonstrate how to set up the tools and development environment we will use for the remaining projects in this course. We will use a similar environment to grade your work.

If you have taken other systems courses at Brown, you have likely set up a Docker container on your system with the tools you need for the course. In this course, we will set up an environment with similar tools, but, depending on your system, you might not need a Docker container: because we will be developing code exclusively for a virtual machine (QEMU) or on separate hardware (a physical Raspberry Pi), you may actually need fewer tools installed on your computer than your other classes, so your environment can be simpler.

These instructions will describe how to set up your environment, and how to set up a docker container if needed (or if you prefer to develop this way).

Heads up! Even if these instructions look similar to course environments you’ve set up before, don’t skip steps! Our setup is a bit different from other courses, and we have some important guidance on how to avoid bugs and issues that may not have come up in your other courses, but may cause problems for us.

Clone your environment

In this step, we’ll download the course’s setup code and determine how to configure your system.

Initial instructions for Mac OS users

If you're running on macOS, you will need to install a set of Apple-recommended command-line tools via the following command:

xcode-select --install

This ensures that your computer has installed git, a program we’ll use below. Alternatively, you may also download and install git directly, following instructions here. After that, continue with the rest of these instructions.

Start your setup by cloning our development environment repo and checking your system. To do this:

  1. Open a terminal in a directory on your computer where you want to do your coursework:
    • ⚠️ Windows users: open a WSL terminal (e.g. terminal type “Ubuntu”) and change to your WSL home directory (type cd ~)

    • ⚠️ Mac users: We strongly recommend you use a folder that does not sync with iCloud or any other cloud storage, which can cause random issues throughout the semester. To do this, we recommend starting from your main user folder (i.e., the folder that holds your Documents/Downloads/Desktop folders). For example:

      # Example:  go to your home directory before starting
      you@your-mac:~$ cd ~
      

      Optionally, you can setup your environment in any other folder you like, so long as it doesn’t sync with iCloud.

  1. Then, enter the following command to download our development environment to the new subdirectory, YOUR_DIRECTORY. Choose your own name (e.g., cs1670) instead of our placeholder YOUR_DIRECTORY. To do so, replace YOUR_DIRECTORY in the git command below with whatever you choose:

    git clone https://github.com/csci1670/cs1670-f26-devenv.git YOUR_DIRECTORY
    
  2. Use cd to enter the directory you have just created (e.g., cd cs1670 if you called your directory cs1670).

  3. Run the script check-host-system, which will check your host’s OS version and see what tools you already have installed:

    ./check-host-system

    The script should print some information about your system, and some recommendations, like this:

    Host OS:         macOS v15.7.4 Sequoia
    Architecture:    ARM64
    Shell:           /bin/bash
    Homebrew:        Yes (/opt/homebrew/bin/brew)
    Docker:          Yes (v29.4.1)
    
    ===================================================================
    
    Your system looks OK to install the native macOS (homebrew) version

Take a look at the output from the script: if you see any warnings in red, the script identified a problem with your setup that needs to be fixed before continuing. To fix them, follow the instructions in the script; you can find more details about the issues in the Troubleshooting section.

Otherwise, if there are no warnings, you’re all set to install your tools!

Installing your tools

We offer two ways you can install our development tools – the check-host-system script (from the previous section) will recommend which version you should use. Before continuing with this guide, follow the instructions to set up your preferred version:

  • Native macOS version (aka the “homebrew version”, for macOS Sonoma and later): installs our development tools natively on your host system using the homebrew package manager. This is a lot simpler than the container version, but only works on the most recent macOS versions.
  • Container version (Windows systems, macOS Ventura and earlier, most Linux systems): this version sets up a Docker container on your system, similar to the environment you may have used in your other systems courses.
Extra guidance if you use Linux

If you are a Linux user, we officially recommend using the container version, as it’s the most reliable way to set up an environment that is consistent with our tools: depending on your distribution, your host system may use a different compiler version or different names for certain programs, which may cause issues with our makefiles and testing scripts.

However, if you would prefer not to use the container version (and are feeling adventurous), you can try working on your native system by installing the necessary tools on your host system. For a starting point, we recommend taking a look at the container setup scripts in the docker directory of the dev environment repo for examples of the packages you would need to install (though the exact names and versions may differ for you).

If you have questions about using Linux for our course, please feel free to post on Edstem or see the instructors during office hours!


Task: Set up your preferred environment using the instructions above. After you’re done, return to this guide to test your setup!

Testing your setup: building a kernel!

Now that your environment is set up, it’s time to build your first operating system!

To test your setup, we’ve provided a pre-compiled “demo” operating system image (i.e., a machine code file containing a kernel and userspace programs) that demonstrates many of the features you’ll be implementing throughout the course.

By the end of this project, you’ll add a small file that modifies the operating system, and run it in QEMU to see the changes. This will make sure that both your compiler and QEMU are working.

Running the demo OS (console mode)

To build the demo image and run it in QEMU, do the following:

  1. If you have not done so already, open a terminal in your preferred environment and cd to your setup repository.

  2. Create the image with make:

    make
    Wait, I thought this image was pre-compiled? What am I compiling?

    Indeed, there isn’t much “compiling” happening for this project: almost all of the C code for the OS is pre-compiled in the file kernel/kernel-qemu.elf. This project uses a special makefile (different from the rest of our projects), which takes this compiled file as input and transforms it into the format we need to load into QEMU.

    We’ll also modify the OS image slightly in the next section, which will use the compiler.


  3. Run QEMU in console mode, like this, which starts up a QEMU virtual machine (VM) using our image as the operating system.

    make qemu

    If all goes well, QEMU should begin printing text to the console! When run in this mode, QEMU is printing any data written to your terminal, which in emulation terms actually means that QEMU writes to the VM’s serial I/O port for its “console”, which is then mapped to your terminal. We’ll learn more about what this means in the first few lectures.

Exiting QEMU

Now let’s exit QEMU. You may notice that pressing Ctrl+C doesn’t work–this is because Ctrl+C sends a character that is normally interpreted by the operating system, and our demo OS doesn’t implement it! Therefore, it has no way to shut itself down!

Instead, we need to tell QEMU itself (which is a process on our computer) to forcibly exit the VM. Fortunately, QEMU has some helpful “monitor” commands to do this. To exit QEMU:

  1. In your OS terminal, press Ctrl+A, let go, then quickly press x. The Ctrl+A is called an “escape sequence”: QEMU watches for you to type this key, and then it looks at the next key you press to tell it what to do–in this case, e(x)it.

    If QEMU doesn’t exit at first, don’t just close your terminal! It might take you a few tries to get the exit command to work: this is a way of sending commands you’ve likely never seen before, and it can be hard to learn the right speed to press the keys the first time. Don’t give up!

    🤔 Note: In the rest of this project and all future documentation, we will use the following notation to refer to entering this command:

    Ctrl+a x

    This is the notation in practice for a command with an escape sequence. Remember it for later!

  2. Once you exit QEMU, repeat the process a few times: practice starting QEMU (make qemu) and exiting using Ctrl+a x a couple of times, until you get a feel for sending the command – you’ll do it a lot, so it will help you to practice!!

Running with graphics

Our demo OS doesn’t just print to the console, however – it also has fancy graphics that are displayed on QEMU’s video display, which emulates a monitor that you might connect to a computer via an HDMI port.

We won’t start using the graphical display until Project 3, but QEMU’s graphics support can be a bit tricky to use on systems, we’ll test it now to make sure it’s working.

Task: Follow the instructions to run the demo OS with the graphical display.

Since running QEMU in graphics mode can sometimes be buggy, we support two ways to run it: a window method and a browser method–you should choose which method based on the environment you set up:

  • If you are using the container environment (e.g., if you use Windows, older macOS), you should use the browser method. This method creates a web page that shows the display, which you can open in your browser.
  • If you are using macOS and used the native install (i.e., homebrew) environment, you should try the window method. This shows the display in a normal desktop window on your mac.

Browser method

The browser method shows the QEMU display (i.e., what would be on a screen if we were to connect one) on a web page that you can view in your browser. This method should work in any development environment (container, or native macOS).

Expand for instructions

To show the display, do the following:

  1. In your setup repository (or any other assignment directory), run the following command:

    make qemu-ws

    The qemu-ws starts QEMU with the display output to a websocket, which is a way to make connections to web browsers. (If you want to learn more about websockets, take CS1680!)

    You might see some messages that say “disconnected” when you first start QEMU, this is okay, as long as it doesn’t quit back to the terminal–QEMU is now waiting for you to connect to it!

  2. Now we need to start the web viewer, which is a separate program that will make a page you can view in your browser. This step will differ based on which environment you are using:

    • If you are using the native macOS environment:
      • Open a new terminal and cd to your dev environment repo (i.e., where you cloned your projects repo)
      • Run the following script:
        util/run-webviewer-native
    • If you are using the container environment:
      • Open a new terminal in your container
      • Run the following script:
        cs1670-scripts/run-webviewer-container

    If you see a message Serving HTTP on ..., and print a link your viewer is working! Leave this terminal open and continue.

  3. Open your browser and visit the link given by the viewer. If everything worked, this should open a viewer that shows you what’s on the VM’s display, which should say “Welcome to CS1670!” and show a box moving around the screen:

    Display showing 'Welcome to CS1670!'
  4. When you’re done using the viewer, go back to the terminal where you ran make qemu-ws and exit using Ctrl+a x (same as before). You can leave the web viewer running in your other terminal (you’ll need it for the next part!).

If you can see the VM display, you’ve now tested both parts of your setup, yay! Now skip ahead to the Exercise.

Window method (native macOS only)

The window method shows QEMU’s graphical display (i.e., what would appear on a screen if we were to connect one) in a regular window on your Mac. This is usually more convenient than the browser version, but it doesn’t work in the container environment, and may have some bugs. If you use this method, and run into issues, you can always fall back to the browser method.

Expand for instructions

To run QEMU in windowed mode, run the following command from your repo:

make qemu-fb
Aside: what does "fb" mean?

The fb stands for framebuffer, which is how the operating system interacts with the display hardware. We’ll learn what this means later in the course!

This should open a small window that shows “Welcome to CS1670!”

Warning: If your window is really small, you may need to resize it so that it shows the whole display:

Cut off display

To do this, click and drag the bottom-right corner–the window should also-resize to the correct size as soon as you start dragging.

Once you’ve resized your display, you should see a window that shows “Welcome to CS1670!” and a box bouncing around the screen, like this:

Display showing 'Welcome to CS1670!'

Some important things to note as you use the QEMU display:

  • When you click into the display, QEMU will take control of your mouse. To get it back, press Ctrl+Option+G (as shown in the titlebar)
  • You can quit QEMU by closing the display window, or by pressing Ctrl+a x in the terminal (like we did for the previous demo)

Exercise

At this point, your tools are mostly set up! Let’s do a quick practice to make sure you can compile code for your operating system. While you don’t have the code for our ready-made demo OS, we’ve set things up so that you can modify it: as practice, we’ll change the background color of the display. The results of this part are what you’ll hand in at the end.

Initial background: how displays work

Data on the display is represented as a grid of pixels, which represent the color value for a particular point on the display. You may have seen the concept of pixels in terms of raster-type image files (e.g., .jpg, .png, or .bmp), which store data in much the same way.

How are pixels (i.e., color values) stored? Many different formats have been developed over the years to support different types of hardware. We’ll learn about some of these when we talk about graphics and framebuffers in some later lectures.

In general, arbitrary colors are represented as a combination of primary colors – on computing devices, the three primary colors are red, green, and blue.

The graphics hardware for our Raspberry Pi stores each pixel as a 32-bit number (i.e., 4 bytes), with 8 bits (one byte) each representing components of red, green, and blue. On the Pi, the numbers are stored in “BGR” format–that is, if viewed in hex, the bytes are arranged like this:

0x00BBGGRR
    BB     - blue channel  (1 byte)
      GG   - green channel (1 byte)
        RR - red channel   (1 byte)

Thus, you can think of each primary color value as an 8-bit number (values 0-255), packed into a single 32-bit number. For example, the value 0x00000000 represents black (no color) and 0x00ffffff represents white (full intensity of all colors).

You’ve likely seen similar formats before: HTML colors, used on all web pages, use very similar format, except the bytes are arranged differently: HTML colors and other systems use RGB format, rather than the BGR format used by the Pi.

Want to see more examples? Try out this page. The boxes rgb# and bgr# show the color values in hex; the boxes labeled R, G, B show the decimal values of each component.

So how do we use this to set the background color? We’ve set up this project’s stencil to read the background color from a certain address in memory (0x0e0000). the rest of the code sets all the background pixels to this value. For our demo, we’ll set a color value at this address.

Task: Set the background of the display to your favorite color! Here’s how:

  1. Open up the file kernel/bgcolor.c, which starts out as a blank file.

  2. Declare a variable representing the color for one pixel and set it to your favorite color. You can name the variable anything you want. (Think: what type should it be?)

  3. The next step is a bit special: we need to tell the compiler to store the variable at address 0x0e0000, which is where the rest of the code expects to find it. First read the following.

You’ll do this by adding a compiler attribute to your variable, which is a special directive to the compiler to do something not normally part of the C language. (We’ll introduce a few of these throughout the course!) The directive to place a variable in a certain memory region looks like this (for some variable my_var):

// Tell the linker to store my_var in the ".bgcolor" section of memory
// NOTE:  Your variable will NOT be type char--what should it be??!
char __attribute__((section(".bgcolor"))) my_var = 0x00;

When the code is compiled, the linker will use this attribute to put my_var in the correct location.

  1. Add this directive to your variable declaration.

  2. Once you’ve defined your color, run your OS with graphical output, either using the browser method or the window method, and see it on the display! If the color doesn’t show up as you expect, make sure you’re using the BGR format described above.

Once you have your display showing your color of choice, read the next section to submit your work!

Submitting your work

Submitting on the grading server

Head over to the grading server. If you were registered for the course on the first day of classes, you should have received credentials by email. If you don’t see an email, please check your spam folder.

If you only recently registered or you’re shopping the course but haven’t registered, you won’t have an account yet. In that case, please complete this form and wait until you receive your credentials. We will create accounts based on the form entries around 8-9pm every day.

To submit your work:
  1. Log into the grading server with the password you received. Once you’re logged in, enter your GitHub username (at the top).

  2. Add the repository URL under “Project 0” category. Click “Project 0” to initiate a fetch of your repository from GitHub:

    Lab view on the grading server'

    Setup project view on the grading server

    • Note: If your GitHub account is not linked to your Brown email address, the grading server will give you a command to run to verify your repository.
  3. You should see your commit listed on the Labs page, as shown above. You’ll also see a button labeled “Checkoff”. Click this button to have your submission checked off.

    After your work passes the test, you should see a screen similar to this one (you might need to refresh the page to see it):

    Checkoff view on the grading server

Setup feedback form

In addition to the checkoff, please also fill out our background and setup form to help us tune the assignment for future offerings of the course.

You will not receive credit unless you complete the form (regardless of the grade shown on the grading server).


Congratulations, you’ve completed Project 0 and are set up for the course! 🎉

Resources and troubleshooting

We’ve compiled a list of FAQs and common issues for both development environments:

FAQs and common issues will be collected here!