Partially Observable MDP
Rather than observing the state we observe some function of the state.
Ob - Observable function.
a random variable for each states.
Problem: different states may “look” similar.
The optimal strategy might need to consider the history.
Example: (1) Ob(s) = s+noise. (2) Ob(s) = first bit of s.