Next: Choosing the parameters for Up: "Encoding" Functions Previous: "Encoding" Functions

### methods of encoding

1.
Linear Function We choose a weight function, , for which the value function is . Naturally, we cannot calculate every function this way, but in many cases it gives good results. Another advantage is the simplicity of calculating the derivative . The derivative is the encoding of the state, which is very convenient computation-wise.
2.
Neural Networks (figure )

Calculation of a gate (see figure ):
First, we compute . Then give as an argument to a non linear function.
1.
perceptron :
The problem: it isn't a continuous function.
2.
sigmoid function: a continuous perceptron approximation,

Note that, , and when , , and when , .

It is possible to connect a large number of such gates, each gate has its own weight vector wi. There are simple algorithms for computing the derivative by using the chain rule.
Basically, we are left with a learning problem: finding F(r,s) that corresponds to V*. We are interested in two things:
• equivalent to

Next: Choosing the parameters for Up: "Encoding" Functions Previous: "Encoding" Functions
Yishay Mansour
2000-01-17