Publications on Hierarchical Methods

Effective Control Knowledge Transfer Through Learning Skill and Representation Hierarchies
International Joint Conference on Artificial Intelligence 2007 (pdf - 192 KB) Abstract:
Learning capabilities of computer systems still lag far behind biological systems. One of the reason...

Gradient-Based Reinforcement Learning: Learning Combinations of Control Policies
Technical Report 97.50, Dip. Elettronica e Informazione, Politecnico di Milano ( gzipped Postscript - 381 KB) Abstract:
This report presents two innovative model-based reinforcement learning algorithms for continuous st...

The MAXQ Method for Hierarchical Reinforcement Learning
Proceedings of the International Conference on Machine Learning, 1998 ( gzipped Postscript - 53Kb) Abstract:
This paper presents a new approach to hierarchical reinforcement learning based on the MAXQ decompo...

Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
journal version; under review ( gzipped Postscript - 192Kb) Abstract:
This paper presents a new approach to hierarchical reinforcement learning based on the MAXQ decompo...

E-mail: tgd@cs.orst.edu
State abstraction in MAXQ hierarchical reinforcement learning
unpublished ( gzipped Postscript - 102Kb) Abstract:
Many researchers have explored methods for hierarchical reinforcement learning (RL) with tempora...

Hierarchical reinforcement learning with the MAXQ value function decomposition.
unpublished ( gzipped Postscript - 360Kb) Abstract:
Note: Supersedes previous version with same title. This paper presents a new approach to hier...

A Hierarchical Approach to Efficient Reinforcement Learning in Deterministic Domains
AAMAS 2006 (PDF - 140KB) Abstract:
Factored representations, model-based learning, and hierar- chies are well-studied techniques for i...

Behaviors Coordination Using Restless Bandits Allocation Indexes
Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior (SAB98) ( gzipped Postscript - 166KB) Abstract:
In order to remain viable and to reproduce an animal has to continuously deal with the problem of ...

Generating Hierarchical Structure in Reinforcement Learning from State Variables
Springer-Verlag (copyright) as a volume in their Lecture Notes in Artificial Intelligence series. (2000) ( gzipped Postscript - 11 pages) Abstract:
This paper presents the CQ algorithm which decomposes and solves a Markov Decision Process (MDP) by ...

Module Based Reinforcement Learning for a Real Robot
Proceedings of the 6th European Workshop on Learning Robots, 22-32, 1997 ( gzipped Postscript - 755 Kb) Abstract:
This is the shortest version of our Module-Based RL paper. The behaviour of reinforcement learnin...

Generalized Dynamic Concept Model as a Route to Construct Adaptive Autonomous Agents
Neural Network World 3:353-360, 1995 ( gzipped Postscript - ) Abstract:
This is an early approach to introduce generalization in RL. A model of adaptive autonomous agent...

Module-Based Reinforcement Learning: Experiments with a Real Robot
Machine Learning ( gzipped Postscript - 755) Abstract:
The behavior of reinforcement learning (RL) algorithms is best understood in completely observable, ...

Parallel Q-Learning for a block-pushing problem
Conference proceedings (pdf - 679KB) Abstract:
Our approach is based on reinforcement learning algorithm (Q-Learning). We propose an original archi...

Macro-Actions in Reinforcement Learning: An Empirical Analysis
technical report ( gzipped Postscript - 440K) Abstract:
Several researchers have proposed reinforcement learning methods that obtain advantages in le...

Hierarchical Optimal Control of MDP's
Proceedings of the 10th Yale Workshop on Adaptive and Learning systems. ( gzipped Postscript - 600 KB) Abstract:
In this paper we survey a new approach to reinforcement learning in which high and low-level decis...

E-mail: munos@cs.cmu.edu
Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems
European Conference on Machine Learning, 1997 (compressed Postscript - 283Kb) Abstract:
This paper presents a reinforcement learning algorithm for generating an adaptive control for a ...

Reinforcement Learning with Hierarchies of Machines
NIPS'97 ( gzipped Postscript - ) Abstract:
We present a new approach to reinforcement learning in which the policies considered by the learning...

Hierarchical Control and Learning for Markov Decision Processes
PhD Thesis ( gzipped Postscript - 440 KB ) Abstract:
This dissertation investigates the use of hierarchy and problem decomposition as a means of solv...

A whole bunch of papers on hierarchical reinforcement learning (since 1990)
journal papers and conference papers (HTML - 100 KB) Abstract:
There is no teacher providing useful intermediate subgoals for our hierarchical reinforcement learni...

Efficient Structure Learning in Factored-state MDPs
AAAI 2007 (PDF - 115KB) Abstract:
We consider the problem of reinforcement learning in factored-state MDPs in the setting in which le...

Hierarchical Optimization of Policy-Coupled Semi-Markov Decision Processes
International Conference on Machine Learning (ICML-99) ( gzipped Postscript - 250 KB) Abstract:
Manufacturing is a challenging real-world domain for applying MDP-based reinforcement learning algo...

HQ-Learning: Discovering Markovian subgoals for non-Markovian reinforcement learning
Technical Report IDSIA-95-96, October 1996 ( gzipped Postscript - 111 KB) Abstract:
To solve partially observable Markov decision problems, we introduce HQ-learning, a hierarchical ex...

Reinforcement Learning with a Hierarchy of Abstract Models
Appears in Proceedings of the Tenth National Conference on Artificial Intelligence, 1992 ( gzipped Postscript - 105 KB) Abstract:
Reinforcement learning (RL) algorithms have traditionally been thought of as trial and error learni...

Finding Structure in Reinforcement Learning
Advances in Neural Information Processing Systems (NIPS) 7, 1995. ( gzipped Postscript - 149 KB) Abstract:
Reinforcement learning addresses the problem of learning to select actions$ maximize one...