All Packages Class Hierarchy This Package Previous Next Index
Class sim.errFun.ValueIteration
java.lang.Object
|
+----sim.errFun.ErrFun
|
+----sim.errFun.RLErrFun
|
+----sim.errFun.ValueIteration
- public class ValueIteration
- extends RLErrFun
Perform Value Iteration with a given Markov Decision
Process, function approximator, and gradient-descent algorithm. The derivative
calculations with respect to the inputs have not been fully implemented here.
This code is (c) 1997 Mance E. Harmon
<harmonme@aa.wpafb.af.mil>,
The source and object code may be redistributed freely.
If the code is modified, please state so in the comments.
- Version:
- 1.03, 21 July 97
- Author:
- Mance E. Harmon
-
dEdWeightsV1
- gradient of mean squared error wrt weights of maximum advantage in successor state
-
oldState
- A copy of the original state.
-
rnd
- The random number generator that will be used for this object.
-
ValueIteration()
-
-
BNF(int)
- Return the BNF description of how to parse the parameters of this object.
-
evaluate(Random, boolean, boolean, boolean)
- return the scalar output for the current dInput vector
-
findGradient()
- update the fGradient vector based on the current fInput vector
-
initVects(MDP, RLErrFun)
- Create inputs and state vectors.
-
parse(Parser, int)
- Parse the input file to get the parameters for this object.
-
unparse(Unparser, int)
- Output a description of this object that can be parsed with parse().
dEdWeightsV1
protected MatrixD dEdWeightsV1
- gradient of mean squared error wrt weights of maximum advantage in successor state
oldState
protected MatrixD oldState
- A copy of the original state.
rnd
protected Random rnd
- The random number generator that will be used for this object. This is a copy of the generator passed to evaluate()
ValueIteration
public ValueIteration()
BNF
public String BNF(int lang)
- Return the BNF description of how to parse the parameters of this object.
- Overrides:
- BNF in class ErrFun
unparse
public void unparse(Unparser u,
int lang)
- Output a description of this object that can be parsed with parse().
- Overrides:
- unparse in class ErrFun
- See Also:
- Parsable
parse
public Object parse(Parser p,
int lang) throws ParserException
- Parse the input file to get the parameters for this object.
- Throws: ParserException
- parser didn't find the required token
- Overrides:
- parse in class ErrFun
initVects
public void initVects(MDP mdp,
RLErrFun rl)
- Create inputs and state vectors. Also, create any vectors that might be specific to this module.
- Overrides:
- initVects in class RLErrFun
evaluate
public double evaluate(Random rnd,
boolean willFindDeriv,
boolean willFindHess,
boolean rememberNoise)
- return the scalar output for the current dInput vector
- Overrides:
- evaluate in class ErrFun
findGradient
public void findGradient()
- update the fGradient vector based on the current fInput vector
- Overrides:
- findGradient in class ErrFun
All Packages Class Hierarchy This Package Previous Next Index