Question

Find the optimal paths of the control, state, and costate variables that will $$\begin{array}{ll} \text { Maximize } & \int_{0}^{8} 6 y d t \\ \text { subject to } & y^{\prime}=y+u \\ & y(0)=10 \quad y(8) \text { free } \\ \text { and } & u(t) \in[0,2] \end{array}$$

Short answer

The optimal control is determined by the costate, \( \nu \): use \( u=2 \) when \( \nu > 0 \) and \( u=0 \) otherwise.

Step-by-step solution

Define the Hamiltonian

To solve the problem using the Pontryagin Maximum Principle (PMP), we start by defining the Hamiltonian. The Hamiltonian, \( H \), is defined as follows: \[ H = 6y + u (y + u) \]where \( u \) is the costate variable.

Determine the Costate Equation

The costate equation is derived from the Hamiltonian as follows: \[ u' = -\frac{\partial H}{\partial y} = -6 - u \] This equation will be used to find the costate variable \( u \).

Solve State Equation

The state equation is given as: \[ y' = y + u \]We will use this equation to find the state variable \( y \) given the initial condition \( y(0) = 10 \).

Find the Optimal Control

The optimal control \( u(t) \) maximizes the Hamiltonian. The Hamiltonian depends linearly on \( u \), so we set \( u = 2 \) when \( u > 0 \) and \( u = 0 \) when \( u \leq 0 \). We solve the system to find \( u \) and apply these conditions for \( u \).

Solve Costate Equation

To find \( u \) over time, solve the differential equation \[ u' = -6 - u \]with suitable boundary conditions. The solution involves an exponential function since it is a first-order linear differential equation.

Characterize the Solution

Once \( u \) and \( y \) are found, along with the optimal \( u \), substitute back into the system to verify all conditions. The path of the state variables, control and costate should satisfy all constraints and PMP conditions.

Key concepts

Pontryagin Maximum Principle

The Pontryagin Maximum Principle (PMP) is a key concept in optimal control theory. It provides necessary conditions for an optimal control problem to reach its maximum or minimum value. In simple terms, it helps identify the control laws that optimize a particular objective function.

Generally, PMP involves three main components:

• The control variable, which we seek to optimize.
• The state variable, which describes the system dynamics.
• The costate variable, which acts as a Lagrange multiplier in continuous optimization problems.

In the given exercise, PMP is employed to determine functions that maximize the integral of a state variable, subject to specific dynamic and boundary conditions. It sets the framework for deriving the Hamiltonian and subsequently the costate and state equations.

Hamiltonian

The Hamiltonian serves as a pivotal component when applying Pontryagin Maximum Principle. It essentially merges the system dynamics with the optimization criterion under one umbrella.

In our problem, the Hamiltonian is defined as:\[ H = 6y + u(y + u) \]This equation integrates the state variable \( y \), the control \( u \), and the costate function. The role of the Hamiltonian is to encapsulate the performance index and system equations into a single entity.

The control \( u \) is chosen to maximize the Hamiltonian function at each point in time, guiding us towards the optimal solution. The linear dependence on \( u \) indicates checking boundary values to maximize it, crucial for the final optimization.

Costate Equation

The costate equation emerges from differentiating the Hamiltonian with respect to the state variable, crucial to the PMP approach. In this context, the costate equation is derived as:\[ u' = -\frac{\partial H}{\partial y} = -6 - u \]This equation links the dynamics of the costate variable to the state and control variables, creating an interaction network.

The costate equation provides a differential equation that must be solved alongside the state equation. This relationship ensures that both state and costate variables evolve in harmony, aligning with the optimal control objectives. Solving the costate differential equation typically involves standard techniques for first-order linear equations.

State Equation

The state equation describes how the state variable \( y \) changes over time, governed by the system dynamics and the control input. In the exercise, the state equation is given as:\[ y' = y + u \]This equation is a differential equation that you solve by substituting optimal control values and initial or boundary conditions provided, like \( y(0) = 10 \).

Solutions to the state equation reveal how the state evolves from its initial position through control interventions. It's a central part of the solution process in control problems, as it directly determines the effect of the control \( u \) over time. Together with the costate equation, it forms the core system solving for the full state and costate trajectories.