A decision rule may need memory of the whole history to determine the most profitable course of action, or it may need only the current state. It may also be a deterministic rule (resulting in one singe operation) or a stochastic one (resulting in a distribution on a set of operations). We will define the following set of rules:
MD - a deterministic Markovian rule (having no memory):
HD - a deterministic history dependent rule:
Where we define the history as: H1=s , and ,
MR - a stochastic Markovian rule:
,where P(A) is the set of distributions on A.
HR - a stochastic history dependent rule:
We define a stationary rule to be a rule that is independent on time, i.e. , for some d.
SD - a deterministic, stationary rule.
SR - a stochastic, stationary rule.