The Problem

In the following course, we will refrain mainly to discrete time problems, that is, at any given point in time $T=1 \ldots N$ and only in those points the agent may perform one single action. If N is finite, we shall refer to these problems as having a finite horizon. When N is infinite, these problems will be referred to as having an infinite horizon. When discussing finite horizon problems, at time T=N the agent is not allowed to perform an action and instead will be rewarded with an immediate end point profit according to the position it is in.