All Packages Class Hierarchy This Package Previous Next Index
Class sim.errFun.QLearning
java.lang.Object
|
+----sim.errFun.ErrFun
|
+----sim.errFun.RLErrFun
|
+----sim.errFun.QLearning
- public class QLearning
- extends RLErrFun
Perform Q learning, either residual gradient, residual or direct, with a given Markov Decision
Process, function approximator, and gradient-descent algorithm. This code works
with both stochastic and deterministic systems.
This code is (c) 1996 Mance E. Harmon
<harmonme@aa.wpafb.af.mil>,
http://www.cs.cmu.edu/~baird
The source and object code may be redistributed freely provided
no fee is charged. If the code is modified, please state so
in the comments.
- Version:
- 1.03, 21 July 97
- Author:
- Mance E. Harmon
-
dEdInQ1
- gradient of mean squared error for 1 training example wrt inputs of original state
-
dEdWeightsQ1
- gradient of mean squared error for 1 training example wrt weights of original state
-
oldAction
- A copy of the original action.
-
oldState
- A copy of the original state.
-
rnd
- The random number generator that will be used for this object.
-
QLearning()
-
-
BNF(int)
- Return the BNF description of how to parse the parameters of this object.
-
evaluate(Random, boolean, boolean, boolean)
- return the scalar output for the current dInput vector
-
findGradient()
- update the fGradient vector based on the current fInput vector
-
initVects(MDP, RLErrFun)
- Create inputs, state, and action vectors.
-
parse(Parser, int)
- Parse the input file to get the parameters for this object.
-
unparse(Unparser, int)
- Output a description of this object that can be parsed with parse().
dEdWeightsQ1
protected MatrixD dEdWeightsQ1
- gradient of mean squared error for 1 training example wrt weights of original state
dEdInQ1
protected MatrixD dEdInQ1
- gradient of mean squared error for 1 training example wrt inputs of original state
oldState
protected MatrixD oldState
- A copy of the original state.
oldAction
protected MatrixD oldAction
- A copy of the original action.
rnd
protected Random rnd
- The random number generator that will be used for this object. This is a copy of the generator passed to evaluate()
QLearning
public QLearning()
BNF
public String BNF(int lang)
- Return the BNF description of how to parse the parameters of this object.
- Overrides:
- BNF in class ErrFun
unparse
public void unparse(Unparser u,
int lang)
- Output a description of this object that can be parsed with parse().
- Overrides:
- unparse in class ErrFun
- See Also:
- Parsable
parse
public Object parse(Parser p,
int lang) throws ParserException
- Parse the input file to get the parameters for this object.
- Throws: ParserException
- parser didn't find the required token
- Overrides:
- parse in class ErrFun
initVects
public void initVects(MDP mdp,
RLErrFun rl)
- Create inputs, state, and action vectors. Also, create any vectors that might be specific to this module.
- Overrides:
- initVects in class RLErrFun
evaluate
public double evaluate(Random rnd,
boolean willFindDeriv,
boolean willFindHess,
boolean rememberNoise)
- return the scalar output for the current dInput vector
- Overrides:
- evaluate in class ErrFun
findGradient
public void findGradient()
- update the fGradient vector based on the current fInput vector
- Overrides:
- findGradient in class ErrFun
All Packages Class Hierarchy This Package Previous Next Index