Next: Example 1
Up: Infinite Horizon Problems
Previous: Infinite Horizon Problems
We will suggest three possible return functions for the infinite horizon problem:
The Return Function
- The expected sum of the immediate rewards, i.e.
Note that this return function may diverge.
- The expected discounted sum of the immediate rewards, i.e.
In this case, a suffice condition for convergence can be for example:
Under this condition we can find an upper bound to the return function:
Note that this bound is very sensative to the value of the paramter .
- The expected average reward
This limit does not always exist. A sutisfactory demand for the limit's existance may be
These conditions will be discussed further in a later lecture.
- S is finite
is Markovian and stationary
- the system is non periodic