Extra Credit Homework

☕

This assignment is individual effort; adhere to the syllabus collaboration policy and ask me if you have any questions.

This is an extra credit homework assignment. It will be scored out of 50. Any points you earn on it will be applied to your lowest homework grade, and can cause that grade to exceed 100.

Q1 - Carl’s Diner (35 pts)

In South Cleric, AZ, Carl’s Diner serves the best coffee in town. The coffee is brewed at the nearby Ignatius Coffee company in Cleric. Carl hopes to optimize his coffee purchasing.

Customers

Carl gets about 400 customers who want coffee per day, and he has noticed that the number of customers on each day \(c_t\) seems to be related to the number of customers on the previous day, \(c_{t-1}\)

Specifically, the number of customers on each day is given by a Poisson distribution:

\(c_t \sim Poisson(\lambda=100+\frac{3*c_{t-1}}{4})\)

Coffee Consumption

Carl purchases coffee by the bag. Each bag provides enough coffee for 25 customers.

Coffee Roasting

Coffee is roasted by Ignatius every day, to order. Carl takes delivery of coffee bags and tests the coffee to see if it meets his standards. 5% of coffee bags received from Ignatius do not meet Carl’s standards.

Coffee Inventory

  • Once a bag of coffee is opened, the whole bag is used that day.
  • Unused coffee doesn’t last forever. For each bag of coffee left over at the end of the day, there is a 25% chance the coffee will be stale the next day.

Observations

Carl observes exactly how many customers came into the diner during the day, and knows exactly how many bags of coffee are left over at the end of the day.

Outcomes

  • Each bag of coffee costs Carl $10.
  • Carl makes $2 off of every cup of coffee sold
  • Carl values every unsatisfied customer at negative $1
    • An unsatisfied customer is any customer who wants coffee and can’t get it
  • Carl discounts such that $1 tomorrow is worth $0.95 today

Decisions

At the end of each day, after the last customer, Carl has to decide how much coffee to order for the following day. Ignatius will roast it overnight, and deliver it before Carl’s Diner opens the next morning.

Frame this problem as a Markov Decision Process and determine the optimal policy by which Carl orders coffee at the end of each day.

Include in your writeup how you framed the problem, and include whatever code you used to solve the problem.

Simplification

You should be able to solve the full problem, but if you can’t figure out how, you could simplify the problem and solve the simplified problem. Depending on the level of simplification (reduction in state space), you’ll get some partial credit.

Q2 - Feedback (15 pts)

For one homework question (from Homeworks 1-4), identify an area where the homework assignment was unclear, and write an edited description of the problem statement (can be just a few sentences/bullet points) that you think would make the problem more clear. This could also include additional boilerplate Python code.

How To Submit

Submit as a .tar with a single PDF writeup, plus any additional Python code as .py files. You can submit the final weights as .csv or .json.