Note: This content is accessible to all versions of every browser. However, this browser does not seem to support current Web standards, preventing the display of our site's design details.


Stochastic optimal control and multi-agent systems.

In this presentation I will introduce a novel theory for stochastic optimal control in non-linear dynamical systems in continuous space-time. The main advantage of this approach is that it allows treatment by standard Bayesian inference methods, such as the junction tree, Monte Carlo simulation or belief propagation. We apply this theory to collaborative multi-agent systems. The agents evolve according to a given non-linear dynamics with additive Wiener noise. Each agent can control its own dynamics. The goal is to minimize the accumulated joint cost, which consists of a state dependent term and a term that is quadratic in the control. We focus on systems of non-interacting agents that have to distribute themselves optimally over a number of targets, given a set of end-costs for the different possible agent-target combinations. We show that optimal control is the combinatorial sum of independent single-agent single-target optimal controls weighted by a factor proportional to the end-costs of the different combinations. Thus, multi-agent control is related to a standard graphical model inference problem. The additional computational cost compared to single-agent control is exponential in the tree-width of the graph specifying the combinatorial sum times the number of targets. We illustrate the result by simulations of systems with up to 42 agents. We discuss possible extensions to apply approximate inference methods for larger number of agents.
Type of Seminar:
Public Seminar
Prof. Bert Kappen
Radboud University, Nijmegen, the Netherlands
Apr 19, 2006   17:15

ETH Zentrum, Gloriastrasse 35, Building ETZ, Room E6
Contact Person:

Prof. Joachim Buhmann
No downloadable files available.
Biographical Sketch:
Bert Kappen studied particle physics in Groningen, the Netherlands and completed his PhD in this field in 1987 at the Rockefeller University in New York. From 1987 until 1989 he worked as a scientist at the Philips Research Laboratories in Eindhoven, the Netherlands. Presently, he is full professor of physics at the University of Nijmegen, conducting research in machine learning and computational neuroscience. His particular emphasis is to develop efficient computation methods for probabilistic inference using methods from statistical physics. He is founder of two companies that commercialize machine learning technology.