Note: This content is accessible to all versions of every browser. However, this browser does not seem to support current Web standards, preventing the display of our site's design details.


Prosthetic control via thoughts: Reinforcement learning for state-based control in a brain-machine interface.

Brain-Machine Interface (BMI) is an active research topic with potential to improve the lives of individuals afflicted with motor neuropathies. Conceptually, a BMI creates a bridge between a user’s nervous system and a prosthetic. The function of this bridge may be to replace a damaged output pathway (e.g. spinal cord) or an incoming pathway (e.g. visual system). Research groups have demonstrated impressive BMI performance both in animal models and humans. However, this success has been mainly with able-bodied subjects in laboratory environments. Clinical implementations have been limited by problems intrinsic to prior learning architectures. Specifically, prior BMI relied on supervised learning to control prosthetics for reconstructing trajectories. This requires separate training (which could be difficult for clinical users) and testing (where there is no learning) phases. I addressed this limitation by shifting the BMI paradigm away from supervised learning; instead, framing prosthetic control as a reinforcement learning (RL) task for a computational agent (CA). To test this control scheme, I designed a goal-based task to parallel a clinical prosthetic control problem. This created a natural synergy where the CA and BMI user have shared goals and similar learning methods to achieve control. Additionally, both the CA and BMI user constantly learn from interactions with the environment. This talk will focus on prosthetic controller design and operation in the neurophysiology domain. This domain features high-dimensional, stochastic states and unknown environmental dynamics. These challenges were addressed using RL to adapt a neural network towards a state-action value function. This control scheme was demonstrated in three rats over 25 brain-control sessions. All rats achieved prosthetic control significantly above chance for multiple task difficulties without requiring physical movements. While the control scheme was effective, some neural network behaviors suggest that alternative control methods should be considered in future implementations.

Type of Seminar:
Public Seminar
Dr. Jack DiGiovanna
Institut für Automatik, ETHZurich
Apr 28, 2009   16:00 /

Physikstr. 3, Building ETL, Room K 25
Contact Person:

File Download:

Request a copy of this publication.
Biographical Sketch: