SA/BA/MA projects

_Discaimer_: please download the attached PDF for a better reading experience. Model predictive control (MPC) is a powerful model-based control technique that received significant attention both in industry and academia due to its ability to efficiently obtain control inputs that are explicitly optimized for a predefined objective and fulfill process constraints. At each time-step, the MPC controller utilizes the measured (or estimated) value of the system state to construct a future prediction of the state and input trajectory. Then, by solving an optimization problem, the future predicted trajectory is chosen to minimize a given objective while fulfilling constraints. Each predicted trajectory has finite length to ensure that the problem can be solved in real time, which can potentially lead to myopic planning and reduce the closed-loop performance of the controller. A well-established way to overcome this inherent suboptimality, is to approximate the infinite-horizon cost by including a _terminal cost_ in the MPC problem, as seen in Equation (1) in the pdf. The terminal cost, i.e., a cost applied to the last state of the predicted trajectory, is usually designed to approximate the _value function_ of the terminal state, that is, to approximate the total cost-to-go that one has to pay to steer the terminal state to the target state. The value function can be learned efficiently in many situations. Observe in Figure 1 of the PDF, for example, how an MPC equipped with a value function is able to outperform a nominal MPC in the pendulum swing-up task, in the same simulation settings. Despite the great performance benefits that a value function used as terminal cost can bring, this approach does not work well if the system dynamics changes over time. In this case, to retain satisfactory performance, the value function needs to be re-learned, resulting in significant computational effort. In this project, we wish to alleviate the computational effort required for re-learning by utilizing _meta-learning_, a novel strategy for effectively adapting controllers to various environments and reducing the amount of data needed for the training.

This project will focus on establishing a meta-learning algorithm that can quickly adapt the value function associated to an MPC controller if the environment dynamics change. To achieve this, the study will first focus on constructing a value function, for example following the method in [1]. Subsequently, the meta-learning algorithm will be adapt the value function to different dynamics. If time permits, the project could also address this problem from a theoretical perspective, for example by establishing guaranteed suboptimality bounds on the resulting MPC controller. All results will be tested in simulation. The goals of the project are as follows: 1. familiarize with the algorithms described in [1]; 2. construct and deploy the meta learning scheme; 3. establish theoretical properties of the scheme. **References** [1] Ugo Rosolia and Francesco Borrelli. Learning model predictive control for iterative tasks: A computationally efficient approach for linear system. IFAC-PapersOnLine, 50(1):3142–3147, 2017.

We are looking for motivated students with some prior knowledge about MPC, optimization (e.g. gradient descent) and Python. Please send your resume/CV (including lists of relevant projects/courses) and transcript of records in PDF format via email to rzuliani@ethz.ch.

_Note_: please read the attached PDF for a better reading experience. Safety is a critical concern in control applications, particularly in systems where constraint violations can lead to hazardous outcomes, such as in autonomous driving, robotics, or process control. Model Predictive Control (MPC) offers a powerful framework to ensure safety by explicitly accounting for system constraints over a receding prediction horizon. By leveraging an internal model of the system and repeatedly solving a constrained optimization problem online, MPC can proactively enforce safety limits on states and inputs, while anticipating future behavior. This predictive capability enables MPC to integrate safety requirements into the control synthesis process, making it a suitable tool for safety-critical applications, as highlighted in recent works emphasizing formal guarantees and robust constraint satisfaction under uncertainty. MPC performance can be significantly enhanced through hyperparameter optimization techniques, which systematically tune parameters such as constraints and cost weights to better suit specific control tasks. As demonstrated in recent work [1], such optimization can lead to improved closed-loop performance by balancing control accuracy and computational complexity. These methods move beyond manual trial-and-error tuning, offering scalable and automated strategies to deploy high-performance MPC in real-world applications. While hyperparameter optimization can significantly enhance the performance of MPC controllers, it may also introduce safety risks if not carefully constrained. Automated tuning methods often focus on improving performance metrics such as tracking accuracy or economic cost, potentially overlooking critical safety constraints or robustness margins. For instance, optimizing cost weights without explicitly accounting for constraint satisfaction can result in aggressive control actions that violate system limits or reduce stability margins (see the example in Figure 2). Therefore, it is crucial to integrate safety-aware mechanisms into the optimization process, to ensure that performance improvements do not come at the expense of system safety.

The objective of this project is to develop a tuning framework that enhances the performance of a Model Predictive Control (MPC) scheme while providing _anytime_ safety guarantees. That is, the controller must ensure that the closed-loop system remains within predefined safety constraints at all times, even if the optimization process is interrupted before convergence. To achieve this, the project will build on recent advances in closed-loop optimization methods [1], [2], which leverage the concept of _differentiable optimization_—the ability to compute derivatives of the solution maps of parametric optimization problems. These methods open new possibilities for systematically tuning the parameters of an MPC controller by incorporating gradient-based optimization techniques. Although the project is primarily theoretical, it will also include a simulation component to demonstrate and validate the proposed methods. The main tasks of the project are: 1. Study and understand the closed-loop optimization techniques presented in [1] and [2]; 2. Design a differentiable tuning scheme that guarantees anytime feasibility and safety of the MPC controller; 3. Implement the proposed scheme and validate its effectiveness through numerical simulations. **References** [1] R. Zuliani, E. C. Balta, and J. Lygeros. BP-MPC: Optimizing the closed-loop performance of MPC using BackPropagation. IEEE Transactions on Automatic Control, 2025. [2] R. Zuliani, E. C. Balta, and J. Lygeros. Closed-loop Performance Optimization of Model Predictive Control with Robustness Guarantees. arXiv preprint arXiv:2403.04655, 2024.

We are looking for motivated students with some prior knowledge about MPC, optimization and Python. Please send your resume/CV (including lists of relevant projects/courses) and transcript of records in PDF format via email to rzuliani@ethz.ch.

Adaptive control seeks to maintain desired system performance despite uncertainties or changing dynamics, making it essential in applications like robotics, aerospace, and power systems. Reinforcement learning (RL), particularly in the form of adaptive dynamic programming, has recently emerged as a promising approach to design adaptive controllers directly from data, bypassing the need for explicit system identification. At the heart of this challenge lies the linear quadratic regulator (LQR) problem, a benchmark task where the goal is to achieve both closed-loop stability and optimality by ensuring the adaptive controller converges to the optimal LQR policy. However, most existing RL-based adaptive control methods lack rigorous guarantees on stability, which limits their safe application in real-world systems. Moreover, questions of robustness under noise, disturbances, or model uncertainty remain largely open. This project aims to address these gaps by leveraging sequential stability analysis techniques to develop RL-based adaptive control methods with formal guarantees on stability, optimality, and robustness for linear time-invariant systems. The student will combine theoretical tools with practical algorithm development and will validate the proposed methods through detailed simulations on representative systems, ensuring both theoretical soundness and practical performance.

The student will: Learn foundational concepts, including adaptive control, reinforcement learning, and adaptive dynamic programming for LTI systems; Review and synthesize the current literature on RL for adaptive control, focusing on identifying gaps in stability guarantees; Develop and implement algorithms that integrate sequential stability analysis with RL-based adaptive control; Bring together the theory, and analyze the results; Promising results could be turned into a publication.

Dr. Feiran Zhao and Marcell Bartos, zhaofe@ethz.ch, mbartos@ethz.ch.

Volume control is a long-standing research problem aimed at improving the precision and robustness of automated pipetting systems. Traditional pipetting methods rely on controlling liquid volumes indirectly through plunger displacement and predefined lookup tables specific to individual liquids. In contrast, this project explores an innovative approach where the volume of liquid in the pipette tip is controlled directly by considering internal air pressure according to the ideal gas law. However, there are several challenges that need to be addressed before deploying the controller in the real-world system including technical challenges, such as stick-slip friction, Helmholtz pressure oscillations, and other physical phenomena such as highly varying viscosities, which have thus far limited the robustness of the system. Additionally, factors such as liquid evaporation—causing unpredictable changes in the air dead-volume—and liquid retention effects, characterized by thin-film formation that leaves residual liquid in pipette tips after dispensing, further complicate precise volume control. The project is offered in collaboration with our industry partner Hamilton Robotics. It is a division of Hamilton Company, specializes in developing and manufacturing precision measurement devices and automated liquid handling workstations for the scientific community. Their product portfolio includes platforms like Microlab® VANTAGE, Microlab® STAR, Microlab® NIMBUS, and Microlab® Prep, known for their reliability, performance, and flexibility. Founded in 1950 by Clark Hamilton, the inventor of the microliter syringe, the company has grown into a global leader in laboratory automation, employing approximately 3,000 individuals worldwide.

Mathematical Modeling: Developing a comprehensive mathematical representation of the precision pipette hardware (MagPip). Data for system identification will be acquired through pressure sensors for accurate internal air pressure measurement. Control Architecture Design: Designing a robust control system that utilizes the mathematical model for the precision volume control subject to nonlinearities and disturbances. Several methods will be explored and compared. Simulation and Real-world Experimentation: Implementing the controller on a real-world prototype.

Muhammad Zakwan (mzakwan@ethz.ch) Efe C. Balta (efe.balta@inspire.ch)