Quant Academy Math · Probability · Quant Finance
Phase 3 - Lesson 3.5

Constrained Optimization and Lagrange Multipliers

Optimizing subject to equality constraints, the geometry of parallel gradients, and the multiplier as a shadow price.

⏱ 55 min● Advanced🔗 Prereqs: 3.4
↖ Phase 3 hub
Builds on: Gradients (3.2) and unconstrained extrema (3.4).
Leads to: Lagrangian duality generalizes this to inequalities in Phase 10; the minimum-variance portfolio uses it directly.

Learning Objectives

Click a status chip to cycle: Not started → In progress → Studied → Practiced → Needs review → Mastered.

Key Vocabulary

Constraint set
The feasible region \(\{x: g(x)=c\}\) over which \(f\) is optimized.
Lagrange multiplier
The scalar \(\lambda\) linking objective and constraint gradients at an optimum.
Lagrangian
The function \(\mathcal{L}(x,\lambda)=f(x)-\lambda\,(g(x)-c)\) whose stationarity encodes the conditions.
Shadow price
The rate \(d f^*/dc=\lambda\) at which the optimal value improves as the constraint loosens.
Binding constraint
One satisfied with equality at the optimum, so its gradient enters the stationarity condition.
Minimum-variance portfolio
Weights minimizing \(w^\top\Sigma w\) subject to \(\mathbf{1}^\top w=1\).

Intuition & Motivation

Intuition
Picture the contour map of \(f\) and draw the constraint curve \(g=c\) on top. Walking along the constraint, \(f\) rises and falls; you stop improving exactly where the constraint curve is tangent to a level curve of \(f\). Tangency means the two curves share a normal direction, i.e. the gradients are parallel: \(\nabla f=\lambda\nabla g\). The multiplier \(\lambda\) is the proportionality constant, and it doubles as a price: it tells you how much the best attainable value would change if you relaxed the constraint by one unit.

The Lagrange condition

Theorem - Lagrange multipliers (equality constraint)
Let \(f,g\) be \(C^1\) and suppose \(x^*\) optimizes \(f\) on \(\{g=c\}\) with \(\nabla g(x^*)\neq 0\). Then there is a scalar \(\lambda\) with \(\nabla f(x^*)=\lambda\,\nabla g(x^*)\). Equivalently, \((x^*,\lambda)\) is a stationary point of the Lagrangian \(\mathcal{L}=f-\lambda(g-c)\).

The recipe: form \(\mathcal{L}\), set \(\nabla_x\mathcal{L}=0\) and \(\partial_\lambda\mathcal{L}=0\) (the latter just restores the constraint), and solve the combined system. With several constraints, add one multiplier each: \(\nabla f=\sum_k\lambda_k\nabla g_k\).

Worked Example - Closest point on a line / smallest sum of squares
1
Minimize \(f=x^2+y^2\) subject to \(g=x+y=1\) (a stylized fully-invested constraint).
2
Lagrangian \(\mathcal{L}=x^2+y^2-\lambda(x+y-1)\). Stationarity: \(2x=\lambda,\ 2y=\lambda\), so \(x=y\).
3
Constraint \(x+y=1\) gives \(x=y=\tfrac12\) and \(\lambda=1\).
4
Minimum value \(f=\tfrac14+\tfrac14=\tfrac12\). The multiplier \(\lambda=1=df^*/dc\): nudging the budget \(c\) from 1 raises the minimal cost at unit rate. Verify: \(f^*(c)=c^2/2\), so \(df^*/dc=c=1\).

The multiplier as a shadow price

Envelope theorem: at the optimum, \(\dfrac{d f^*}{dc}=\lambda\). In portfolio terms \(\lambda\) is a marginal cost/benefit - the improvement in the objective per unit relaxation of the constraint. This is why multipliers are called shadow prices and reappear as dual variables in Phase 10.

Application: the minimum-variance portfolio

Let \(\Sigma\succ 0\) be the covariance matrix of \(n\) assets’ returns and \(\mathbf 1\) the all-ones vector. The minimum-variance portfolio solves:

\[\min_w\ \tfrac12\,w^\top\Sigma w\quad\text{s.t.}\quad \mathbf 1^\top w=1.\] (3.5)
Worked Example - Solving the min-variance program
1
Lagrangian \(\mathcal L=\tfrac12 w^\top\Sigma w-\lambda(\mathbf 1^\top w-1)\). Gradient in \(w\): \(\Sigma w-\lambda\mathbf 1=0\).
2
Solve \(w=\lambda\,\Sigma^{-1}\mathbf 1\) (valid since \(\Sigma\succ 0\) is invertible).
3
Impose \(\mathbf 1^\top w=1\): \(\lambda\,\mathbf 1^\top\Sigma^{-1}\mathbf 1=1\), so \(\lambda=1/(\mathbf 1^\top\Sigma^{-1}\mathbf 1)\).
4
Therefore \(w^*=\dfrac{\Sigma^{-1}\mathbf 1}{\mathbf 1^\top\Sigma^{-1}\mathbf 1}\) with variance \(w^{*\top}\Sigma w^*=1/(\mathbf 1^\top\Sigma^{-1}\mathbf 1)=\lambda\).
5
The multiplier again equals the optimal variance - the shadow price of the budget constraint. This closed form is a cornerstone of Markowitz portfolio theory (Phase 14).
Common Mistakes to Avoid
  • Forgetting the constraint equation - \(\partial_\lambda\mathcal L=0\) must be part of the system, or you lose it.
  • Assuming \(\lambda\) has a fixed sign; for equality constraints it can be positive or negative.
  • Applying the method when \(\nabla g=0\) at the candidate (constraint qualification fails).
  • Interpreting a Lagrange stationary point as automatically a minimum; check second-order/convexity as in 3.4.
Quant Practitioner Tips
  • Set gradients proportional, then use the constraint to pin down \(\lambda\) - two steps, always.
  • Read \(\lambda\) as a price: it is the marginal value of loosening the constraint, the dual variable of Phase 10.
  • For quadratic objective + linear constraints the system is linear; solve it as one matrix equation (the min-variance closed form).

Knowledge Check

Q1 Easy
The Lagrange condition for minimizing \(f\) on \(g=c\) states that at the optimum:
\(\nabla f=0\)
\(\nabla f=\lambda\nabla g\)
\(f=g\)
\(\nabla g=0\)
Q2 Medium
The Lagrange multiplier \(\lambda\) at the optimum equals:
The constraint value c
The sensitivity \(df^*/dc\) of the optimal value to the constraint level
The objective value
Always 1
Q3 Hard
For \(\min\ \tfrac12 w^\top\Sigma w\) s.t. \(\mathbf 1^\top w=1\) with \(\Sigma\succ0\), the solution is:
\(w=\Sigma^{-1}\mathbf 1\)
\(w=\Sigma^{-1}\mathbf 1/(\mathbf 1^\top\Sigma^{-1}\mathbf 1)\)
\(w=\mathbf 1/n\)
\(w=\Sigma\mathbf 1\)

Practical Exercise

Maximize \(f(x,y)=xy\) subject to \(x+y=10\) (allocate a fixed budget of 10 across two uses to maximize their product). Solve with a multiplier and interpret \(\lambda\).

▶ Show full solution

Lagrangian \(\mathcal L=xy-\lambda(x+y-10)\). Stationarity: \(y=\lambda,\ x=\lambda\), so \(x=y\).

Constraint \(x+y=10\) gives \(x=y=5\) and \(\lambda=5\), with maximum \(f=25\).

Interpretation: \(\lambda=5=df^*/dc\). Increasing the budget from 10 to \(c\) gives optimum \(f^*=(c/2)^2=c^2/4\), so \(df^*/dc=c/2=5\) at \(c=10\). Each extra unit of budget adds about 5 to the product at the margin. The symmetric split \(x=y\) is the classic AM–GM result.

After the reveal, answer for yourself: Notice the second-order check: on the constraint line the product is a downward parabola, confirming a maximum.

Lesson Summary

Equality-constrained optima occur where the objective’s and constraint’s gradients are parallel, \(\nabla f=\lambda\nabla g\), encoded by stationarity of the Lagrangian. The multiplier \(\lambda\) is a shadow price, \(df^*/dc\). Applied to \(\min\ w^\top\Sigma w\) with a budget constraint it yields the minimum-variance portfolio \(w^*=\Sigma^{-1}\mathbf 1/(\mathbf 1^\top\Sigma^{-1}\mathbf 1)\).

Formula Sheet Additions

Min-variance portfolio
\[w^*=\frac{\Sigma^{-1}\mathbf 1}{\mathbf 1^\top\Sigma^{-1}\mathbf 1}\]
Lagrange applied to a quadratic risk objective with a budget constraint.

Retrieval Practice

Close the lesson and answer from memory before checking. This is deliberate, effortful recall - the single highest-yield study action.

▶ Show retrieval prompts & answers
Q: What is the geometric meaning of \(\nabla f=\lambda\nabla g\)?
A: At the constrained optimum the constraint curve is tangent to a level set of f, so their gradients (normals) are parallel.
Q: What does the Lagrange multiplier represent economically?
A: A shadow price: the marginal change df*/dc in the optimal value per unit relaxation of the constraint.

Completion Checklist

Confidence / mastery rating
Personal notes

Source References

This lesson synthesizes and paraphrases concepts from the sources below. No copyrighted text, problem sets, or solutions are reproduced. Return to the originals for full depth.

← 3.4Next: 4.1 →