Question

Because we use the envelope theorem in constrained optimization problems often in the text, proving this theorem in a simple case may help develop some intuition. Thus, suppose we wish to maximize a function of two variables and that the value of this function also depends on a parameter, \(a: f\left(x_{1}, x_{2}, a\right) .\) This maximization problem is subject to a constraint that can be written as: \(g\left(x_{1}, x_{2}, a\right)=0\) a. Write out the Lagrangian expression and the first-order conditions for this problem. b. Sum the two first-order conditions involving the \(x^{\prime}\) s. c. Now differentiate the above sum with respect to \(a\) - this shows how the \(x\) 's must change as \(a\) changes while requiring that the first-order conditions continue to hold. A. As we showed in the chapter, both the objective function and the constraint in this problem can be stated as functions of \(a: f\left(x_{1}(a), x_{2}(a), a\right), g\left(x_{1}(a), x_{2}(a), a\right)=0 .\) Differentiate the first of these with respect to \(a\). This shows how the value of the objective changes as \(a\) changes while keeping the \(x^{\prime}\) s at their optimal values. You should have terms that involve the \(x^{\prime}\) s and a single term in \(\partial f / \partial a\) e. Now differentiate the constraint as formulated in part (d) with respect to \(a\). You should have terms in the \(x\) 's and a single term in \(\partial g / \partial a\) f. Multiply the results from part (e) by \(\lambda\) (the Lagrange multiplier), and use this together with the first-order conditions from part (c) to substitute into the derivative from part (d). You should be able to show that \\[ \frac{d f\left(x_{1}(a), x_{2}(a), a\right)}{d a}=\frac{\partial f}{\partial a}+\lambda \frac{\partial g}{\partial a} \\] which is just the partial derivative of the Lagrangian expression when all the \(x^{\prime}\) 's are at their optimal values. This proves the envelope theorem. Explain intuitively how the various parts of this proof impose the condition that the \(x\) 's are constantly being adjusted to be at their optimal values. g. Return to Example 2.8 and explain how the envelope theorem can be applied to changes in the fence perimeter \(P\) -that is, how do changes in \(P\) affect the size of the area that can be fenced? Show that in this case the envelope theorem illustrates how the Lagrange multiplier puts a value on the constraint.

Short answer

The envelope theorem provides insight into the relationship between the change in the parameter and the optimization problem by showing that as the parameter changes, the optimal values of the variables adjust in a way that keeps the partial derivative of the Lagrangian expression constant. This result helps to understand the impact of the parameter on the objective function without explicitly re-solving the entire optimization problem for each change in the parameter.

Step-by-step solution

Part a - Writing the Lagrangian and First-order Conditions

The given maximization problem is: \\[ \text{Maximize} \hspace{0.1in} f(x_1, x_2, a) \hspace{0.1in}\text{subject to} \hspace{0.1in} g(x_1, x_2, a) = 0. \\] We can solve it by using the Lagrangian with a single constraint. The Lagrangian is \\[ \mathcal{L}(x_1,x_2,\lambda, a) = f(x_1, x_2, a) + \lambda (g(x_1, x_2, a)), \\] To find the first-order conditions, take the partial derivatives of the Lagrangian with respect to \(x_1\), \(x_2\), and \(\lambda\) and set them equal to zero: 1. \(\frac{\partial \mathcal{L}}{\partial x_1} = \frac{\partial f}{\partial x_1} + \lambda \frac{\partial g}{\partial x_1} = 0\) 2. \(\frac{\partial \mathcal{L}}{\partial x_2} = \frac{\partial f}{\partial x_2} + \lambda \frac{\partial g}{\partial x_2} = 0\) 3. \(\frac{\partial \mathcal{L}}{\partial \lambda} = g(x_1, x_2, a) = 0\)

Part b - Summing First-order Conditions

To sum the two first-order conditions involving the variables \(x_1\) and \(x_2\), simply add the first two equations: 1+2. \(\frac{\partial f}{\partial x_1} + \frac{\partial f}{\partial x_2}+\lambda \left(\frac{\partial g}{\partial x_1}+\frac{\partial g}{\partial x_2} \right)=0\)

Part c - Differentiating the Sum of First-order Conditions

Now differentiate the sum of the first-order conditions with respect to \(a\): \(\frac{d}{da}\left[\frac{\partial f}{\partial x_1} + \frac{\partial f}{\partial x_2}+\lambda \left(\frac{\partial g}{\partial x_1}+\frac{\partial g}{\partial x_2} \right)\right]=0\)

Part d - Differentiating the Objective Function

Differentiate the objective function \(f\left(x_{1}(a), x_{2}(a), a\right)\) with respect to \(a\): \(\frac{d f\left(x_{1}(a), x_{2}(a), a\right)}{d a} = \frac{\partial f}{\partial a}+\frac{\partial f}{\partial x_1}\frac{dx_1}{da}+\frac{\partial f}{\partial x_2}\frac{dx_2}{da}\)

Part e - Differentiating the Constraint

The constraint is \( g\left(x_{1}(a), x_{2}(a), a\right)=0 \). Differentiate it with respect to \(a\): \(\frac{d g\left(x_{1}(a), x_{2}(a), a\right)}{d a} = \frac{\partial g}{\partial a}+\frac{\partial g}{\partial x_1}\frac{dx_1}{da}+\frac{\partial g}{\partial x_2}\frac{dx_2}{da}\)

Part f - Substitution and the Envelope Theorem

Multiply the result from part (e) by \(\lambda\): \(\lambda\frac{d g\left(x_{1}(a), x_{2}(a), a\right)}{d a} = \lambda\left(\frac{\partial g}{\partial a}+\frac{\partial g}{\partial x_1}\frac{dx_1}{da}+\frac{\partial g}{\partial x_2}\frac{dx_2}{da}\right)\) Now substitute the first-order conditions from part (c) into the derivative from part (d): \(\frac{d f\left(x_{1}(a), x_{2}(a), a\right)}{d a} = \frac{\partial f}{\partial a} + \lambda \frac{\partial g}{\partial a}\) This proves the envelope theorem. Intuitively, this result tells us that as the parameter \(a\) changes, the \(x_i(a)\) values are adjusted optimally so that the partial derivative of the Lagrangian expression remains constant.

Part g - Applying the Envelope Theorem in Example 2.8

In Example 2.8, we are maximizing the area of a rectangular fence (\(A = x_1x_2\)) subject to a constraint on the fence perimeter (\(P = 2(x_1+x_2)\)). Applying the envelope theorem, we want to see how changes in the perimeter \(P\) affect the area that can be fenced. Since we have found that \(\frac{d A\left(x_{1}(P), x_{2}(P), P\right)}{d P}=\frac{\partial A}{\partial P}+\lambda \frac{\partial g}{\partial P}\), the change in the area due to a change in the fence perimeter is equal to the sum of the partial derivative of the area with respect to the perimeter and the product of the Lagrange multiplier and the partial derivative of the constraint with respect to the perimeter. The Lagrange multiplier, in this case, puts a value on how restrictive the constraint is, and it shows how the optimal area changes in response to a change in the available perimeter.

Key concepts

Constrained Optimization

Constrained optimization is a fundamental concept in economics and mathematics. It refers to the process of optimizing an objective function subject to certain constraints. In many real-world scenarios, decisions are made to maximize or minimize an objective, such as profit, cost, or utility, while satisfying certain limitations or conditions.

In the context of the problem provided, the objective function is denoted by \( f(x_1, x_2, a) \), which is to be maximized subject to a constraint \( g(x_1, x_2, a) = 0 \). Here, the constraint represents a boundary within which the optimization must occur, such as a budget constraint in economics or a physical limitation in engineering.

This type of problem is prevalent in various fields like economics, where a firm might aim to maximize profit under resource constraints, or in engineering, where design optimizations are performed under material limitations. Constrained optimization problems are often solved using the method of Lagrange multipliers, which we will explore next.

Lagrangian Expression

A Lagrangian expression is a powerful tool used to solve constrained optimization problems. The Lagrangian transforms the problem by incorporating the constraint into the objective function using a Lagrange multiplier. This new function is then optimized without the explicit constraint.

For the given problem, the Lagrangian is expressed as:

- \( \mathcal{L}(x_1, x_2, \lambda, a) = f(x_1, x_2, a) + \lambda \, g(x_1, x_2, a) \),

where \( f(x_1, x_2, a) \) is the function we want to maximize and \( \lambda \) is the Lagrange multiplier that quantifies the impact of the constraint \( g(x_1, x_2, a) = 0 \).

The Lagrangian method converts a constrained problem into an unconstrained one by absorbing the constraint into the objective and adjusting through the multiplier \( \lambda \). This makes it easier to apply calculus techniques to find optimal solutions and is particularly useful when dealing with complex constraints.

First-Order Conditions

First-order conditions are critical in solving optimization problems. They refer to the partial derivatives of the Lagrangian expression, set to zero, which helps determine the stationary points of the function.

In our example, the first-order conditions require computing the derivatives of the Lagrangian with respect to \( x_1, x_2 \), and \( \lambda \), resulting in the following set of equations:

• \( \frac{\partial \mathcal{L}}{\partial x_1} = \frac{\partial f}{\partial x_1} + \lambda \frac{\partial g}{\partial x_1} = 0 \)
• \( \frac{\partial \mathcal{L}}{\partial x_2} = \frac{\partial f}{\partial x_2} + \lambda \frac{\partial g}{\partial x_2} = 0 \)
• \( \frac{\partial \mathcal{L}}{\partial \lambda} = g(x_1, x_2, a) = 0 \)

These conditions ensure that the calculated \( x_1 \) and \( x_2 \) provide the maximum or minimum value for the initial objective function \( f \), given the constraint \( g \). They are vital in the procedure of identifying points where the function is optimized, highlighting how changes in constraints affect the optimality.

Lagrange Multiplier

The Lagrange multiplier, denoted as \( \lambda \), plays a crucial role in constrained optimization. It represents the sensitivity of the objective function to changes in the constraint. Essentially, it measures how much the objective function's optimal value will change with a marginal increase in the constraint.

In our scenario, after differentiating the constraint and multiplying by \( \lambda \), we integrate it into the Lagrangian's derivative. This provides insight into changes due to the parameter \( a \), illustrating the envelope theorem:

\[ \frac{d f\left(x_{1}(a), x_{2}(a), a\right)}{d a}=\frac{\partial f}{\partial a}+\lambda \frac{\partial g}{\partial a} \]

Here, \( \lambda \frac{\partial g}{\partial a} \) shows how much the objective function’s value is expected to change if the constraint’s parameter changes.

The Lagrange multiplier is especially useful in economics, often interpreted as the "shadow price," indicating the value of relaxation of constraints, such as additional budget capacity. Understanding this multiplier is key to leveraging the Lagrangian function in practical applications.