William M. McEneaney

With a view to generalizing existing max-plus and min-plus methods so as to use quadratic basis functions with a non-uniform Hessian, the underlying dual space representation of dynamic programming founded on the semiconvex transform is generalized via the quadratic transform. Using this generalization, an illustrative iteration is proposed as the foundation of a new max-plus method for the solution of a class of optimal control problems.

A class of finite time horizon optimal control problems with nonlinear dynamics and non-quadratic costs is considered. Stat-quad duality is used to transform the problem into a canonical form. A derivative-free numerical method that only uses fixed-point iterations is devised to solve it efficiently, the convergence of which is limited only by the existence of the staticizing control process (argstat). For problems with mild and low-dimensional nonlinearities, this leads to dimension reduction of the control space. A 4-D and a 25-D control problem are solved to demonstrate its accuracy and scalability.

A finite-horizon nonlinear optimal control problem is considered. Stat-quad duality is used to generate an equivalent problem with linear dynamics and a modification term in the running cost and two auxiliary controls processes. This problem form is used to obtain a representation of the value function as a staticization problem over a set of quadratic functions, where the coefficients of the quadratics consists of the solution to a differential Riccati equation, a linear ODE and an integral. This representation allows the value function to be evaluated independently at any time and any point in the state space. A specialized numerical method is proposed for solving the resulting staticization problem, which is able to leverage the low dimensionality of nonlinearity. A numerical example with five-dimensional state space is included.

A finite-horizon nonlinear optimal control problem is considered. Stat-quad duality is used to generate an equivalent problem with linear dynamics and running cost that is quadratic in state with an additional term that is nonlinear in newly introduced control state variables. The new problem form is used to obtain a representation of the value function in terms of staticization over a set of quadratic functions, where the coefficients of the quadratic functions consist of the solutions to certain ODEs. A novel numerical method is indicated for solution of the resulting staticization problem; the method leverages the low dimensionality of nonlinearity. An example is included.

A class of nonlinear, stochastic staticization control problems (including minimization problems with smooth, convex, coercive payoffs) driven by diffusion dynamics with constant diffusion coefficient is considered. The nonlinearities are addressed through stat duality. The second-order Hamilton–Jacobi partial differential equation (HJ PDE) is converted into a first-order HJ PDE in the dual variable, which, however, contains a correction term. Approximations to the correction term are indicated. A numerical example is included.

A finite horizon nonlinear optimal control problem is considered for which the associated Hamiltonian satisfies a uniform semiconcavity property with respect to its state and costate variables. It is shown that the value function for this optimal control problem is equivalent to the value of a min-max game, provided the time horizon considered is sufficiently short. This further reduces to maximization of a linear functional over a convex set. It is further proposed that the min-max game can be relaxed to a type of stat (stationary) game, in which no time horizon constraint is involved.

Abstract A supremum-of-quadratics representation for a class of extended real valued barrier functions is developed and applied in the context of solving a continuous time linear regulator problem subject to a single state constraint of bounded norm. It is shown that this very simple state constrained regulator problem can be equivalently formulated as an unconstrained two-player game. By demonstrating equivalence of the upper and lower values, and exploiting existence and uniqueness of the optimal actions for both players, state feedback characterizations for the corresponding optimal policies for both players are developed. These characterizations are illustrated by a simple example.