Although constraints on the states can often be "softened", e.g., into probabilistic constraints, hard constraints on control inputs are omnipresent. This feature should be taken into account at the control synthesis stage, which leads to nontrivial difficulties in controller synthesis techniques due to inevitable nonlinearities involved.

The objective is to provide a tractable, convex, and globally feasible solution to the finite-horizon stochastic linear quadratic (LQ) problem for with possibly unbounded additive noise and hard constraints on the control policy. Within this framework one has two immediate directions to pursue in terms of controller design, namely, a posteriori bounding the standard LQG controller, or employing certainty-equivalent MPC controller. While the former direction explicitly incorporates some aspects of feedback, the synthesis of the latter involves control constraints and implicitly incorporates the notion of feedback.

Our choice of feedback policies (reported in [HCL09] and [CHL09]) explores the middle ground between these two choices: we explicitly incorporate both the control bounds and feedback at the design phase. More specifically, we adopt a policy that is affine in certain bounded functions of the past noise inputs. The optimal control problem is lifted onto general vector spaces of candidate control functions from which the controller can be selected algorithmically by solving a convex optimization problem. Our novel approach does not require artificially relaxing the hard constraints on the control input to soft probabilistic ones (to ensure large feasible sets), and still provides a globally feasible solution to the problem. Minimal assumptions of the noise sequence being i.i.d and having finite second moment are imposed. The effect of the noise appears in the convex optimization problem as certain fixed cross-covariance matrices, which may be computed offline and stored.

Once tractability of the optimization problem is ensured, we employ the resulting control policy in a receding horizon fashion. The above scheme requires complete observation of the state. We have also addressed the case of partial state observation in the context of linear controlled systems with i.i.d Gaussian noise; this case admits the employment of a Kalman-like filter, and may be used constructively for MPC. Performance comparisons of our receding horizon policy against certainty-equivalent MPC and saturated LQG for benchmark systems are depicted in Fig.1 & 2.

Fig.1: Average cost plots of our policy against receding horizon MPC in a test plant.

Fig.2: Average cost plots of our policy and saturated LQG in a test plant.

Besides performance aspects, it is also of importance to investigate stability of the designed controllers when applied in a receding horizon fashion. In the deterministic setting, it is not possible to render a linear controlled system globally asymptotically stable with bounded control inputs if the system matrix is unstable; with Lyapunov stable systems, however, it is possible to do so. In the context of stochastic MPC, our investigations indicate that incorporating an appropriate constraint to the finite-horizon optimal control subproblem, it is possible to ensure mean-square boundedness of the resulting closed-loop system if the unexcited system is Lyapunov stable. This investigation is currently under way.

We focus on linear dynamical systems driven by stochastic noise and a control input, and consider the problem of finding a control policy that minimizes an expected cost function while simultaneously fulfilling constraints on the control input and on the state evolution. In general, no control policy exists that guarantees satisfaction of deterministic (hard) constraints over the whole infinite horizon. One way to cope with this issue is to relax the constraints in terms of probabilistic (soft) constraints. This amounts to requiring that constraints will not be violated with sufficiently large probability or, alternatively, that an expected reward for the fulfillment of the constraints is kept sufficiently large.

In stochastic MPC, at every time t a finite-horizon approximation of the infinite-horizon problem is solved, but only the first control of the resulting policy is implemented. At the next time t+1 a new problem is considered, the control policy is updated, and the process is repeated in a receding horizon fashion. (Under time-invariance assumptions, the finite-horizon optimal control problem is the same at all times, giving rise to a stationary optimal control policy that can be computed offline.)

Two considerations lead to the reformulation of an infinite horizon problem in terms of subproblems of finite horizon length. First, given any bounded set (e.g. a safe set), the state of a linear stochastic dynamical system is guaranteed to exit the set at some time in the future with probability one whatever the control policy. Therefore, soft constraints may turn the original (infeasible) hard-constrained optimization problem into a feasible problem only if the horizon length is finite. Second, even if the constraints are reformulated so that an admissible infinite-horizon policy exists, the computation of such a policy is generally intractable.

The aim of this research is to establish the convexity of certain stochastic finite-horizon control problems with soft constraints. Convexity is central for establishing the uniqueness of the optimal solution, and for the fast computation of the solution by way of numerical procedures.

We have developed different classes of convex relaxations of finite-horizon stochastic optimal control of linear systems with Gaussian disturbances subject to chance-constraints or integrated chance-constraints on the states and/or control inputs. The results are currently under review.

In its bare essentials, deterministic MPC consists of two steps: (i) solving a finite-horizon optimal control problem with constraints on the state and the controlled inputs to get an optimal policy, and (ii) applying a controller derived from the policy obtained in step (i) in a rolling-horizon fashion. In view of the close relationship of SMPC with applications, any satisfactory theory of stochastic MPC must necessarily take into account its practical aspects. In this context, an examination of a standard linear system with constrained controlled inputs affected by independent and identically distributed (i.i.d.) unbounded (e.g., Gaussian) disturbance inputs shows that no control policy can ensure that with probability one the state stays confined to a bounded safe set for all instants of time. This is because the noise is unbounded, and the samples are independent of each other. Although disturbances are not likely to be unbounded in practice, assigning an a priori bound seems to demand considerable insight.

In case a bounded-noise model is adopted, existing worst-case analysis techniques for controlling deterministic systems with bounded uncertainties may be applied. The central idea is to synthesize a controller based on the bounds of the noise such that the target set becomes invariant with respect to the closed-loop dynamics. However, since the optimal policy is based on a worst-case analysis, it usually leads to rather conservative controllers, or even infeasibility. Moreover, complexity of the optimization problem grows rapidly (typically exponentially) with the optimization horizon. An alternative is to replace the hard constraints by probabilistic (soft) ones. The idea is to find a policy that guarantees that the state constraints are satisfied with high probability over a sufficiently long time horizon. While this approach may improve feasibility aspects of the problem, it does not address the issue of what actions should be taken once the state violates the constraints.

In view of the above considerations, developing recovery strategies appears to be a necessary step for dealing with constraint violation in SMPC. Such a strategy is to be activated once the state violates the constraints, and to be deactivated whenever the system returns to the safe set. In general, a recovery strategy must drive the system quickly to the safe set while simultaneously meeting other performance objectives. In the context of MPC, two merits are immediate: (a) once the constraints are transgressed, appropriate actions can be taken to bring the state back to the safe set quickly and optimally, and (b) if the original problem is posed with hard constraints on the state, in view of (a) they may be relaxed to probabilistic ones to improve feasibility.

One possible recovery strategy may be formulated as an optimal control problem up to an entry time, variously known as pursuit problem, transient programming, first passage problem, stochastic shortest path problem, etc. We formulate the problem as the minimization of an expected discounted cost until the state enters the safe set. An almost customary assumption in the literature concerned with stochastic optimal control up to an exit time is that the target set is absorbing. That is, there exists a control policy that makes the target set invariant with respect to the closed-loop stochastic dynamics. This is rather restrictive for control problems---it is invalid, for instance, in the very simple and canonical case of a linear controlled system with i.i.d. Gaussian noise inputs. We do not make this assumption, for, as mentioned above, our primary motivation for solving this problem is precisely to deal with the case that the target set is not absorbing.

In several applications, such as area surveillance, the task is so complex that the need to decompose the problem into multiple subsystems (e.g. cameras in an indoor scenario) which coordinate their actions arises. Moreover, the operating conditions may vary randomly (moving targets, external disturbances, etc.), and relying on a centralized unit that computes the overall system actions may be computationally costly and critical in case of failures. In order to alleviate such problems, we adopt a Decentralized Stochastic MPC approach in which the complex global problem is divided into relatively simpler local problems that can be handled by each subsystem. However, this distribution of subtasks, if done arbitrarily, can compromise the ability to perform the required task and may lead to performance deterioration if not instability. Some crucial questions that naturally arise are the following:

• Is the decentralized scheme stable?
• How well does it perform with respect to some global performance index for the whole system?
• What kind of information exchange is to be carried out between the subsystems?
• In case the operational conditions vary, how do the subsystems exchange subtasks among them so that the overall performance is not compromised?