CSCI 3411 - Operating Systems

Posted on August 22, 2024 by Gabriel Parmer

This course covers the fundamental concepts of operating systems, focusing on resource management and abstraction. This includes OS structure, processes and thread management, communication with peripherals (I/O), synchronization, deadlocks, memory management, Virtual Machines, cloud infrastructures, and abstractions for cloud computation. The workload for this class is heavy and programming intensive.

Welcome! Great ready for a deep dive into what’s under the water.

Instructional Team Contact info

Please do not directly contact your TA unless it is an issue specific to your lab. Instead ask your question publicly in Disord, or email Gabe if it requires privacy.

Class Time/Location:

Your Immediate TODO

  1. Accept the invitation to the class’ google drive. Find the class-information document which starts with a number of links that you can use to proceed through this list.
  2. Complete the “getting to know you” survey now. This will require you to read this and associated webpages, along with the class-information document. We cannot grade your work without this information.
  3. Find the Discord invite to our server, and join.
  4. Find a link to the first homework (HW0) via github classroom in the class-information document.
  5. Please read about installing the class’ infrastructure in the xv6 document. See the class resources repository, and #tech-support on discord if you have any issues. We’ll help with this if you have any pending issues in the first lab.

Course Overview

Learning Outcomes

Objectives - In completing this class, students will…

Structure - This class is broken into two main activities: lectures and lab.

Direct Instruction and Independent Learning

Each semester, you’re expected to spend at least:

Course Prerequisites and Student Responsibilities

Prerequisites:

Responsibilities - Students must

I also need your feedback for parts of the class that aren’t working for you. Class evaluations are of limited use since they only are given at the end of the class. Please provide feedback using the linked feedback form in the class organization google doc.

Your role!

Provide feedback. Many aspects of the class are new and experimental, thus they might require modifications throughout the course of the class. Because of this, all students must take the contents of the section below on “Professor and Instructional Staff Responsibilities” seriously: if something isn’t working for you, assume that the class needs a tweak, and contact the professor, or provide anonymous feedback (see the form in the class-information document). Gabe will take this feedback seriously.

Community and online social contract. As an increased part of the class is online, it is necessary that everyone conduct themselves respectfully and productively online. Please see the class’ “Online Social Contract”. The class will have a zero-tolerance approach toward disrespectful conduct, or harassment.

Take responsibility for your own education. Much of the work for this class will be more group-oriented than in normal instances of the class. If you burden your partner with most of the work, you are setting yourself up for failure. If you take on your partner’s responsibilities, you are setting them up for failure. The homeworks build on each other. The benefit of this is that by the end of the class you’ll laugh at how easy some of the earlier homeworks are as you’ve massively leveled up. The challenge is that if you don’t put in a genuine effort on each homework, you will find later homeworks increasingly intractable. If you don’t put effort in for each homework, you’ll create dis-proportionally more work for yourself in later homeworks. If you cannot carry your own weight, you will not be able to achieve the learning objectives for the class.

Professor and Instructional Staff Responsibilities

You should depend on the instructional staff (including Gabe) for the class to

These can be remembered as:

We’re here to make it clear what you should do, what you should know, and what to do if you don’t feel like you know what you’re supposed to do or know.

If at any point, you have concerns about any of these, of if we’re dropping the ball on any of them, please let us know and we’ll do better. There are trade-offs made in the class (for example, homeworks are not formally specified as I don’t want them to be 50 pages each), but you always encouraged to ask questions and clarify.

Further, it is our intent that students from all diverse backgrounds and perspectives be well-served by this course, that students’ learning needs be addressed both in and out of class, and that the diversity that the students bring to this class be viewed as a resource, strength and benefit. It is my intent to present materials and activities that are respectful of: gender identity, sexuality, disability, age, socioeconomic status, ethnicity, race, nationality, religion, and culture. Your suggestions are encouraged and appreciated. Please let me know ways to improve the effectiveness of the course for you personally, or for other students or student groups.

Above all, it is important that this class provides an environment for learning and self improvement. Gabe takes this very seriously, so if it is not adequately serving that purpose, don’t hesitate to let him know why (see class-information for an anonymous feedback form).

Class Technologies

How you comport yourself online in the class just adhere to our online social contract. For the class we’ll use the following technologies for the specified purposes:

Please see the discussion about Academic Integrity below. A rule of thumb is that you should not post code on any discussion medium with reach beyond your group. We do not use blackboard.

Class Policies

Absences

I worry if I don’t see you in class!

If you need to be absent, please send me an email to let me know that you can’t make it, and please let me know why. If you know you’re going to be absent (religious holiday), try and let me know in advance, otherwise, let me know as soon as you can. I’ll follow up and help you plan your involvement in the class (e.g. group projects). If you’re sick, keep me updated on how you’re feeling if you can.

Experiments

Part of keeping a course “fresh” and successful is trying new things, and keeping the things that work. Please keep in mind that I might not have fully ironed out the kinks in each of these, so your feedback is valuable. Given this, the experiments I’m running this year include:

I’ll call on random students to answer questions. Instead of bureaucratically tracking participation, I’ll ask questions in lecture to a random student! This is stressful, and I apologize for that. However, I want to normalize that it is OK if you don’t know the answer. Given this, any student asked a question can either

  1. answer if you have a good idea – even if you’re a little shaky with it,
  2. give it a guess which might be wrong, but will be a great point for me to explain the concept more,
  3. ask a question if your understanding is incomplete and you know what you don’t know,
  4. or call in a favor of another specific student.

In the last case, I’ll end up calling on you again sooner. It is absolutely fine, to not know an answer, or to give it a wild guess. Most of life we’re wrong about most things, but we aim, in the end to get a few things right.

Individual work, with collaboration. All homeworks (with the exception of the Project) will be submitted individually. You’ll each have your own github classroom repo, and will submit your own work. However, you’re allowed to pair up with another student in the class – your buddy. You can collaborate as you see fit. Unlike the normal academic honesty rules of the class, you’re allowed to share code. For any commits that share code with your buddy, you must acknowledge as such in the corresponding commits.

If your buddy stops contributing, procrastinates, or otherwise indicates (through action) that they don’t want to reasonably collaborate, you are responsible for your own code and submission. You must not wait for your buddy, and make progress on your own to meet the deadline. Further, no student has to collaborate with their buddy, but you do have to be clear with your communication – you cannot “go dark” and have to use clear comunication about your intentions for the collaboration.

Grading

Only talk about your grades with Gabe. No other instructional staff have access to your grades.

Grades will be assigned with the following proportions:

Category Percentage
Homeworks 65%
Tests/Quizzes 30%
Participation 5%

If you cannot make it to any aspect of the class, or cannot meet any deadline, please let me know as soon as possible.

Homework Grade Modifications:

The following modifies apply to your grade on all homeworks (but not the project).

Your grade can be minimum 0%, but your maximum grade can be higher than 100% (if there is extra credit). There is one exception:

I cannot emphasize this point enough: The only way people have failed this class is by not trying or by not doing their own work.

Late Policy:

The only acceptable excuses for late submissions are medical, family, and related issues. For assignments that are assigned for a large span of time (e.g. two weeks), a single two day disturbance (for example) should not inhibit your ability to complete the assignment. Generally unacceptable excuses include:

You get 0 credit for late work that is not explicitly excused by Gabe. However, once the grades are released, you can resubmit your code within seven days, rounded up to the closest midnight. For example, if the grades are released at 2:30pm on the 8th, resubmissions are due at midnight on the 15th (i.e. \(8 + 7 = 15\)). You will get 0 additional credit for re-submissions after this date.

We will grade the last commit before the resubmission deadline, and you will receive 50% of the credit on top of the original grade, rounded up to the nearest percent. For example, if you got a 43% on the initial deadline, and resubmit and get a 68%, your final grade (\(g\)) will be \(g = 43 + \lceil \frac{(68 - 43)}{2} \rceil = 56\).

We calculate homework \(i\)’s grade as \(g_i\), and the extra credit as \(e_i\):

where \(o_i\) is the original submission’s grade (and includes the negative modifiers above), \(r_i\) is the regrade (which includes any applicable negative modifiers, and always includes the “cheating” penalty), \(e_{early}\) is any early submission points on the normal deadline, \(e^r_i\) is the resubmission’s extra credit, and \(e^o_i\) is the original submissions grade.

Do not come to rely on the resubmission process. Homeworks are due every week or every other week, so it is very harmful to get behind on them, or to add more work to your schedule. Thus, our late policy is quite strict to discourage procrastination.

Please note that in terms of academic honesty, you cannot share your code with your peers even after it is due.

Testing your implementation:

When you are required to test your code, please note that our test cases will test every edge case in the implementation. If you do not write code to test the edge-cases of the specifications, then you will lose credit. This is the largest reason why students get lower grades than they believe they deserve. If you believe that an error was made in grading, then please explicitly address why you believe your test cases are sufficient to test all cases. See the discussion about testing and debugging below.

Style and Code Craftsmanship:

Any homework graded by demo (i.e. the project) must adhere to style guidelines, and you must curate your code. It must be simple, readable, and clean. See this post For some thoughts on how to make your code more readable, see the Composite Style Guide.

Group Work

Working in groups can be challenging. The core strategies to avoid conflict and acrimony all must focus on planning and clear communication. You must follow this guidance in your group work. This includes:

It must be cognizant of

Read the above guidance carefully.

Academic Honesty

Just as you can do a google search for code online, it is trivial for us to do the same. We have caught numerous people cheating in the past in this way. If you feel pressured about an assignment, please come see me instead of cheating.

You are not allowed to collaborate on the homeworks and the lab assignments. The group projects require collaboration within each group, but no collaboration between teams is permitted. Please refer to the academic integrity policy linked from the course web page. This policy will be strictly enforced. If you’re having significant trouble with an assignment, please contact me. If you have not signed the academic integrity statement for homework 0, you will not receive a passing grade for the class.

Summary: I want to be very clear:

In the past, the only way that people have failed this class is to either not try (not do homeworks), or to not do their own work (or share their work with others). There is almost no reason to do this. Come to me, and we can discuss where you’re at, and what you have to do to make it through this difficult class!

Note that you’re very much allowed to do any of the following:

Justification: Not doing your own work hurts your ability to compete for CS jobs. Everyone came to college to learn. You learn by doing work to reinforce class ideas. Understanding ideas from class is nowhere near as effective at learning as applying those ideas in an implementation. Thus, you need to do your own implementation to be a competitive CS student in the real-world!

Not doing your own work hurts your peers. Courses that are graded on a curve have the side-effect that if someone inflates their grade not by hard work, but by not doing your own work, your good grade (should the Professor not catch you) will hurt the grade of all of your colleagues that put the work in.

Not doing your own work also hurts the quality of the OS class in the long term, thus hurts all future generations of GW CS undergrads. If you don’t do your own work, especially using previous year student’s work, to the point where the Professor believes that they must develop new assignments every year, then those assignments will not be as strong. They will not meet learning objectives in a course like this. All future CS students will be less prepared for competing for the best jobs.

Please see additional information about university policies for academic integrity. The Professor, when unable to use the grading penalties above, will raise an academic honesty violation.

University Policies

Please find details about university policies regarding

Course Schedule

The detailed schedule can be found in the class-organization document. At a high-level:

Topic xv6 Book Chapters/Additional Reading1 Silberschatz Chapters2
OS Fundamentals: Hardware and Structure 0, 1, 3 and this post 1, 2
Processes, IPC, Threads, Isolation this post 3, 4
Scheduling and Critical Sections 5, 4 5, 6, 19
Mutexes, Condition Vars, Deadlock N/A 6, 7
Memory Management and Allocation N/A 8
Memory Protection 2 9
File Systems and Storage 6 10, 11, 12, 13
Security N/A 15

You can find the lectures in the google drive. By default, I’ll provide these after they are fully covered in class. I’ve found that people read ahead which can harm the learning processes (as quite a bit of information is not on the slides) as it removes the problem solving processes. If this policy hinders your learning process, talk to me and we can discuss options. For your reference, all of the lecture’s pdfs are on the following topics:

Course Material

Exercises. The following exercises will help you understand the corresponding course topics.

Course topic Exercise
System structure Understanding calling conventions
Device interaction: polling vs. interrupts faux_s, section “Polling vs. Interrupts”
Device interaction: DMA faux_s, section “DMA”
Processes and fork fork-onacci
Processes and process APIs Forking, blocking and non-blocking communication
Threads and pthread APIs thread-onacci
Locks and synchronization The lock dictatorship
Atomic instructions and lock-free data-structures The revolution overthrows the dictatorship: down with locks
Parallel programming models: fork/join, map/reduce Democracy prevails: Penny for pawsident!
Understanding a program’s virtual memory Memory: the final frontier
File system, hierarchical namespace API Growing trees in a green, new UNIX!

We’ll cover a subset of these in lab.

Required reading. You must read through the material at the repo for class resources. This includes some intuition as to how to develop C, how to think about debugging and testing, and how to think about generating good code.

Strongly encouraged (but optional) Text. This book is useful to help you understand the concepts that might not be clear from the lectures.

xv6 code documentation. If you have trouble understanding the xv6 source code, or want to better understand it, I recommend the following resources:

You can fork the xv6 source on github. (This is the same code that is the foundation of most of the homeworks.) When you do, test that you can run it with make, and make qemu.

In the end, you have to get comfortable walking through the code, and understanding it. Using a code indexing system will make this easier. ggtags or ctags for emacs does great, but there are corresponding technologies for nearly all editors.

If you’re having trouble with C, here is a list of references:

It is your responsibility to quickly get up to speed in C, so please use these resources.

Assignment Submission

All of your homeworks for this class will be submitted via github. The last commit that you make to the repo will be graded, and the lateness penalties of that commit (see “Late Policy” below) will apply. Please do not make pushes to your repo after the version you want graded. Please also note that it is necessary to issue a git push up to the github repo for submission. Local commits to your own repo do not update github, thus cannot be counted as submission.

All assignments are submitted on github by doing a push of your commits before the deadline. There are a huge number of errors that one can make, so I highly suggest that you do the following procedure. When you submit an assignment, I suggest that you follow this procedure:

  1. git status to make sure that you have git added all of the intended files.
  2. git diff and git diff --stat to make sure that you’re committing what you think you are. Of note, check to see if you’re committing any printfs that are intended for debugging, not for program output.
  3. git commit and git push to upload your code to github.
  4. Go to the associated webpage for the repository, and make sure that the most recent commit is present in your repository.
  5. Use git clone ... to get a fresh version of the repository, build it, and run your tests. Students who omit this step get bad grades for very avoidable reasons. Always make sure that your submission matches what you think you’re submitting.

Git Usage on Group Projects

We want to be able to use your git commit logs and diffs, along with partner feedback to assess work done in groups. We don’t expect the work to be exactly equal between team members, but we do expect each group member to pull their own weight. Thus, we expect you to follow some simple rules when using git:

  1. Don’t rebase. Don’t rebase your commits to clean up your commit history. I wanted to see your entire commit history, thus avoid squashing commits. If you don’t know what this is, then you likely don’t need to worry.
  2. Commit with your GWU email. Set your git email address to your GWU email address. Failure to do this will mean that we see you doing no commits!
  3. When pair programming, swap who’s driving, and commit often. If you program together in your group, that’s fine. However, you need to swap back and forth regularly. Set a timer.
  4. Notate commits that don’t compile. If you make a commit when the code doesn’t compile, include DOES NOT COMPILE in the commit message body. Many commits made when pair programming will use this.

Your commit logs will not feed directly into your grade. However, they will play a role in how we determine groups, and they might lead to meetings to discuss your contributions.

  1. Find the book online↩︎

  2. Silberschatz chapters are specified with respect to the 9th edition↩︎