Project

1 TL;DR

This is a summary. Read the entire document.

  • Teams of up to four people
  • Turn in your proposal by 6 Apr
  • Complete in Milestone 1 by 18 Apr
  • Complete Milestone 2 by 25 Apr
  • Turn in your final report by 6 May
    • Final report is an informal “blog post” explaining what you did
    • 6 May is a hard deadline with no extensions possible
  • Turn in code to accompany your report via linked repository

2 Teams

You may work on this project in teams of up to four people. The minimum size of a team is one person.

3 Scope

  • Identify a sequential decision problem to solve with an algorithm
  • The problem should include some uncertainty
  • The problem should be non-trivial (i.e., infeasible to solve with a search such as BFS or A*)

4 Deliverables

4.1 Scope Agreement:

  • You must propose a project to me and I must approve your proposal by 6 Apr. You should submit this proposal with ample time to account for possible revisions. You are welcome to submit it much earlier; I typically turn these around in 48 working hours. The earlier you scope your project, the earlier you can begin your project.
  • Identify in your proposal:
    • The problem you intend to solve
    • The uncertainties involved
    • Why the problem is non-trivial
    • Existing solution methods
    • Your plan for modeling and solving the problem: what is the state space, what is the action space, what are the observations, and what algorithms might you use?
  • Include a link to a git or mercurial repo you will use. If you would like to keep the repo private, you must add me as a collaborator; please use github dot com for this. If the repo is public, you can use the host of your choice.
  • Submit this to me via email. Only one submission per group is required.

4.2 Report:

  • The final report must be turned in by 11:55 PM on 6 May
    • This is hard deadline and I will not grant extensions. Extenuating circumstances will result in a course Incomplete.
  • You will submit an informal report upon conclusion of the project. The format of the report will be a blog post. Possible ways to do this:
    • Github pages with Jekyll
    • Quarto, which can embed Jupyter Notebooks into static websites
    • The literal README of your repo
  • The report must effectively communicate what you did for your project in a way that lets a technical bystander reproduce your work. Include:
    • Software and hardware requirements
    • Links to any data sources
    • Motivation for your project
    • Explanation of what you accomplished
    • How you measured your success (or failure)
  • I won’t grade you on spelling or grammar and ask that you write this yourself without the use of any AI tool or LLM.
    • LLMs often ‘hallucinate’ logical inconsistencies in technical writeups. If your report has logical inconsistencies, you’ll lose credit.
  • How to submit: Each team member should turn in a PDF linking the report and the code repository, and detailing that team member’s contributions to the project.

4.2.1 Report Milestones

Details for each milestone in the grading section

  1. State Space Specification (18 Apr)
  2. State Space Implementation (25 Apr)
  3. Final Deadline (6 May)

I will grade your intermediate milestones by looking at your commit history. There is no formal submission of these milestones. Nonetheless, if you haven’t completed those parts of the project on time, you won’t get credit for them.

4.3 Code

  • Write your project in Python 3
    • Talk to me if you think you have a good reason to use another language
  • Include code with your report in a Github or Gitlab or similar source repository.
  • Specify dependencies with a requirements.txt or pyproject.toml
  • Include enough details in your report for me to reproduce your work without perusing your source code. I recommend either:
    • Step-by-step instructions
    • A makefile and/or build/run scripts

5 Grading

  • You will be graded on how you apply your model and solution method to your problem.
  • I will treat the proposal as a ‘contract:’ if you accomplish the project we agreed on, with all milestones met, you’ll get 100% credit.
    • I will warn you if the proposal risks an inadequate project, with an opportunity to revise
    • I will warn you if the proposal risks an impossibly-difficult project, with an opportunity to revise

5.1 Grading Details

5.1.1 Project Scope (10%)

  • Scope document includes a stochastic sequential decision problem (5%)
  • Scope document proposes a feasible solution method for the problem (5%)

5.1.2 State Space Description (10%)

  • Natural language description of state space in draft report (5%)
  • Complete mathematical description of states, transitions, actions, and observations (5%)

5.1.3 State Space Implementation (15%)

  • State space model implemented in your program
    • Your model should generate successor states, given actions
    • Your model should return observations1 from state-action \(\rightarrow\) state transitions

5.1.4 Final Report (65%)

  • Describe your problem and solution in a blog post:
    • Problem statement (5%)
    • Related solutions to similar problems (5%)
    • State space, actions, transitions, and observations (previously completed)
    • Solution method (5%)
  • Implementation of solution method for problem (50%)
    • Optimally or approximately-optimally compute a decision given a sequence of states/observations
    • Implemented solution is consistent with report description.

6 Collaboration & Accountability

  • If you work in a team of two, three, or four people, I expect that each team member will contribute equal effort to the project. To incentivize this, each team member will submit their own separate statement (several sentences) describing how work was broken down amongst team members. I reserve the right to assign credit for the project unevenly if it is clear that work was not shared evenly.

  • If you would like this project to be related/tangential to work you’re doing (or have already done) for another class, I will approve if and only if you are transparent about the boundaries of what work is done for this class and what work is/was done for the other class, and obtain similar approval from the other instructor.

  • If I can’t reproduce your work, I won’t be able to give you credit for it.

  • If you use any code that you didn’t write yourself, make it clear:

    • If you use a package via import, the import statement is sufficient
    • If you copy a code snippet from stack overflow, documentation, or anywhere else, include a link in comments
    • If you use a “problem” (such as a game) and write the solution, thoroughly detail what parts of the code are yours and what parts are someone else’s
    • Don’t use any LLM-generated code for this project:
      • It probably won’t work
      • You might inadvertently copy a large chunk of code from someone else’s repo verbatim
      • I’m very serious about this