All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class sim.errFun.ValueIteration

java.lang.Object
   |
   +----sim.errFun.ErrFun
           |
           +----sim.errFun.RLErrFun
                   |
                   +----sim.errFun.ValueIteration

public class ValueIteration
extends RLErrFun
Perform Value Iteration with a given Markov Decision Process, function approximator, and gradient-descent algorithm. The derivative calculations with respect to the inputs have not been fully implemented here.

This code is (c) 1997 Mance E. Harmon <harmonme@aa.wpafb.af.mil>, The source and object code may be redistributed freely. If the code is modified, please state so in the comments.

Version:
1.03, 21 July 97
Author:
Mance E. Harmon

Variable Index

 o dEdWeightsV1
gradient of mean squared error wrt weights of maximum advantage in successor state
 o oldState
A copy of the original state.
 o rnd
The random number generator that will be used for this object.

Constructor Index

 o ValueIteration()

Method Index

 o BNF(int)
Return the BNF description of how to parse the parameters of this object.
 o evaluate(Random, boolean, boolean, boolean)
return the scalar output for the current dInput vector
 o findGradient()
update the fGradient vector based on the current fInput vector
 o initVects(MDP, RLErrFun)
Create inputs and state vectors.
 o parse(Parser, int)
Parse the input file to get the parameters for this object.
 o unparse(Unparser, int)
Output a description of this object that can be parsed with parse().

Variables

 o dEdWeightsV1
 protected MatrixD dEdWeightsV1
gradient of mean squared error wrt weights of maximum advantage in successor state

 o oldState
 protected MatrixD oldState
A copy of the original state.

 o rnd
 protected Random rnd
The random number generator that will be used for this object. This is a copy of the generator passed to evaluate()

Constructors

 o ValueIteration
 public ValueIteration()

Methods

 o BNF
 public String BNF(int lang)
Return the BNF description of how to parse the parameters of this object.

Overrides:
BNF in class ErrFun
 o unparse
 public void unparse(Unparser u,
                     int lang)
Output a description of this object that can be parsed with parse().

Overrides:
unparse in class ErrFun
See Also:
Parsable
 o parse
 public Object parse(Parser p,
                     int lang) throws ParserException
Parse the input file to get the parameters for this object.

Throws: ParserException
parser didn't find the required token
Overrides:
parse in class ErrFun
 o initVects
 public void initVects(MDP mdp,
                       RLErrFun rl)
Create inputs and state vectors. Also, create any vectors that might be specific to this module.

Overrides:
initVects in class RLErrFun
 o evaluate
 public double evaluate(Random rnd,
                        boolean willFindDeriv,
                        boolean willFindHess,
                        boolean rememberNoise)
return the scalar output for the current dInput vector

Overrides:
evaluate in class ErrFun
 o findGradient
 public void findGradient()
update the fGradient vector based on the current fInput vector

Overrides:
findGradient in class ErrFun

All Packages  Class Hierarchy  This Package  Previous  Next  Index