Publications on Partially Observable Problems

An Architecture for Situated-learning Agents
Ph.D. Thesis, Monash University, Australia 2004. (PDF - 1.9 MB) Abstract:
This thesis looks at the problem of situated learning agents operating in real-world environments ...

Markov Decision Processes: the control of high-dimensional systems.
Ph.D. Thesis, Vrije Universiteit Amsterdam, 2002 (PDF - 1.1 MB) Abstract:
We develop algorithms for the computation of (nearly) optimal decision rules in high-dimensional sys...

Incremental pruning: A simple, fast, exact algorithm for partially observable Markov decision processes.
Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI--97), 1997 (Postscript - 120 KB) Abstract:
Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a for...

Acting Optimally in Partially Observable Stochastic Domains
Proceedings of the Twelfth National Conference on Artificial Intelligence, 1994 (gzipped Postscript - 104 KB) Abstract:
In this paper, we describe the partially observable Markov decision process (POMDP) approach to fin...

E-mail: kalmar@mindmaker.kfkipark.hu
Module-Based Reinforcement Learning for a Real Robot
Proceedings of the 6th European Workshop on Learning Robots, Lecture Notes in AI, to appear. 1998 ( PDF - 845 KB) Abstract:
The behaviour of reinforcement learning (RL) algorithms is best understood in completely observable,...

E-mail: kalmar@mindmaker.kfkipark.hu
Module Based Reinforcement Learning for a Real Robot
Proceedings of the 6th European Workshop on Learning Robots, 22-32, 1997 ( PDF - 845 Kb) Abstract:
This is the shortest version of our Module-Based RL paper. The behaviour of reinforcement learnin...

Efficient dynamic-programming updates in partially observable Markov decision processes
Brown University Technical Report CS-95-19 (Postscript - 1.2 MB) Abstract:
We examine the problem of performing exact dynamic-programming updates in partially observable Mark...

E-mail: mlittman@cs.duke.edu
Learning policies for partially observable environments: Scaling up
Proceedings of the Twelfth International Conference on Machine Learning (Postscript - 315K) Abstract:
Partially observable Markov decision processes (POMDPs) model decision problems in which an agent t...

Memoryless policies: Theoretical limitations and practical results
From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (Postscript - 416KB) Abstract:
One form of adaptive behavior is "goal-seeking" in which an agent acts so as to minimize the time i...

An optimization-based categorization of reinforcement learning environments
Abstract:
This paper proposes a categorization of reinforcement learning environments based on the optimizati...

Robot Navigation using Discrete-Event Markov Decision Process Models
unpublished ( gzipped Postscript - 200 kb) Abstract:
This paper describes a novel architecture for robot navigation based on semi-Markov decision proces...

E-mail: russell@cs.berkeley.edu
Approximating optimal policies for partially observable stochastic domains
Proceedings of the IJCAI, 1995 (Postscript - 157 KB) Abstract:
The problem of making optimal decisions in uncertain conditions is central to Artificial Intelligen...

REINFORCEMENT LEARNING AND POMDPs (dozens of papers on RL in partially observable environments since 1989)
Journal papers and conference papers (HTML - 100KB) Abstract:
Realistic environments are not fully observable. General learning agents need an internal state to m...

Learning Without State-Estimation in Partially Observable Markovian Decision Processes
Proceedings of the Eleventh International Machine Learning Conference ( gzipped Postscript - ) Abstract:
...