Convex processes #
A convex process from U to X is a multivalued map A : u ↦ A u with
A (u₁ + u₂) ⊇ A u₁ + A u₂, A (λ u) = λ (A u) for λ > 0, and 0 ∈ A 0. These conditions say
exactly that the graph {(u, x) | x ∈ A u} is a convex cone containing the origin, so that is what
ConvexProcess is: a bundled PointedCone ℝ (U × X), with A.eval u the u-slice of the cone
and every elementary property a Submodule fact in disguise. Processes sit between linear
transformations and convex bifunctions — ofLinearMap embeds the former, indicatorBifun embeds a
process into the latter — and carry the adjoint, sum, product and inverse of that algebra.
The adjoint A* is the polar of the graph with a sign flip on one factor: (y, v) ∈ graph A* iff
(-v, y) ∈ (graph A)° (mem_graph_adjointProcess_iff_mem_polarCone), the sign convention
adjointBifun uses, so everything topological about A* comes from the theory of polar cones.
Rockafellar carries "supremum oriented" and "infimum oriented" as extra data on a convex set and
defines the adjoint of an infimum-oriented process by reversing the inequality. Here the two are two
definitions, adjointProcess and coadjointProcess, and A** uses one of each: with
adjointProcess twice the sign flips add instead of cancelling, giving
{p | ∀ w ∈ K°, 0 ≤ ⟨p, w⟩} in place of the bipolar K°°. Reflection through the origin
(reflect) exchanges the two adjoints and commutes with sums and products, so every
infimum-oriented statement below — they carry a co in their names — is the supremum-oriented one
read back through it. The inner products are the exception: reflection flips the dual variable
only, and coBracket_eq_neg_bracket records the sign.
Main definitions #
ConvexProcess U X— a convex process, carried by its graph.eval,dom,range,image,invareA u,dom A,range A,A C,A⁻¹;compwith theAddandSMulinstances areB A,A₁ + A₂andλ A.ofLinearMapembeds a linear transformation, andindicatorBifunis the bifunction(F u)(x) = δ(x | A u).ConvexProcess.adjointProcess,coadjointProcess,reflect— the two adjoints and the reflection exchanging them;coBracketis the inner product⟨Au, x*⟩ = inf {⟨x, x*⟩ | x ∈ A u}of an infimum-oriented process.
Main results #
exists_linearMap_of_isBounded— a convex process with full domain and boundedA 0is a linear transformation (Theorem 39.1 in [^1]).isClosed_graph_adjointProcess,coadjointProcess_adjointProcess_eq_self_iff—A*is always closed,A** = cl A, andA** = Aexactly for closedA.bracket_indicatorBifunand the results beside it —⟨Au, x*⟩is the support function ofA u, hence closed convex inx*and concave inu, and⟨u, A* x*⟩is its convex closure inu(Theorem 39.3 in [^1]).adjointProcess_add,adjointProcess_comp,adjointProcess_smul—(A₁ + A₂)* = A₁* + A₂*,(BA)* = A* B*and(λ A)* = λ (A*)forλ > 0;isClosed_graph_add,graph_adjointProcess_add_eq_closureand thecompanalogues are the closed halves.conj_imageBifun_indicatorBifun—(Af)* = A*⁻¹ f*, the infimum attained (Theorem 39.7 in [^1]).isClosed_image—A Cis closed for closedA, nonempty closed convexC, and no non-zero vector ofA⁻¹ 0recedingC.exists_pairing_sandwich— Fenchel duality at a positively homogeneous pair: a concavepbelow a convexq, both straddling0at the origin, admit a linear⟨·, y⟩between them.
Implementation notes #
The adjoints of a sum and of a product take an IsExactSum hypothesis, one instance per dual
vector, where Rockafellar assumes ri (dom A₁) ∩ ri (dom A₂) ≠ ∅. IsExactSum also requires both
summands proper, so the hypothesis carried here is stronger than the book's.
References #
[^1]: R. T. Rockafellar, Convex Analysis, Princeton University Press, 1970, §39.
The definition #
A convex process from U to X: a multivalued map A : u ↦ Au with
A (u₁ + u₂) ⊇ A u₁ + A u₂,A (λ u) = λ (A u)forλ > 0,0 ∈ A 0,
which Rockafellar shows is the same thing as a convex cone in U × X containing the origin — that
is, a PointedCone ℝ (U × X), read as a relation.
- graph : PointedCone ℝ (U × X)
The graph of the process.
Instances For
The graph of a convex process, as a relation.
Instances For
The effective domain of a convex process: the u at which A u is nonempty.
Instances For
The image of a set under a convex process: A C = ⋃ {A u | u ∈ C}.
Instances For
Two convex processes with the same graph are equal.
A linear transformation, read as a convex process. Its graph is the graph of T, a linear
subspace and hence in particular a pointed convex cone.
Equations
Instances For
The inverse of a convex process, again a convex process.
Equations
Instances For
Elementary structure #
Membership of a nonnegative multiple, in the form the PointedCone coercion hides.
Each value A u of a convex process is a convex set.
A 0 is a cone: it is stable under multiplication by a nonnegative scalar.
A u + A 0 ⊆ A u, so A 0 is a set of directions along which every value of the process is
invariant.
The effective domain of a convex process is convex.
The image of a convex set under a convex process is convex.
The image A C is the projection of graph A ∩ (C × X) on the second factor. This is what
turns closedness of A C into a statement about the image of a convex set under a linear map.
The indicator bifunction #
The indicator function determines its set.
Proof idea: δ(· | S) takes the value 0 exactly on S and ⊤ off it, and 0 ≠ ⊤ in
EReal.
The indicator bifunction of a supremum-oriented convex process: (F u)(x) = δ(x | A u).
This is the dictionary entry making every result about processes one about bifunctions.
Equations
- A.indicatorBifun u = Tdaf.ConvexAnalysis.indicatorFn (A.eval u)
Instances For
The graph function of the indicator bifunction of A is the indicator function of the graph
of A.
A convex process is determined by its indicator bifunction. A process is its graph, and
the indicator function of a set determines the set, so this is indicatorFn_injective read through
graphFn_indicatorBifun. It is what makes the correspondence with bifunctions one-to-one.
The indicator bifunction of a convex process is a convex bifunction.
The effective domain of the indicator bifunction is the effective domain of the process.
Positive multiples do not move the graph of a convex process: it is a cone.
A (λ u) = λ (A u) for λ > 0, the positive-homogeneity axiom of a convex process.
Both inclusions are one application of smul_mem_graph, to a and to a⁻¹: a cone is stable
under both directions of a positive scaling, which is what turns the inclusion Rockafellar's
axiom would give into an equality.
The algebra of convex processes, and its bifunction dictionary #
The infimal convolute of two indicator functions is the indicator function of the sum of the
sets. Operations/InfConv.lean has only the special case of a singleton, so it is proved here.
The sum of two convex processes, (A₁ + A₂) u = A₁ u + A₂ u.
Equations
- One or more equations did not get rendered due to their size.
The effective domain of a sum is the intersection: dom (A₁ + A₂) = dom A₁ ∩ dom A₂.
The scalar multiple λ A, defined by (λ A) u = λ (A u).
The graph {(u, λ x) | (u, x) ∈ graph A} is a pointed convex cone for every real λ, so the
definition needs no positivity; λ > 0 is needed only from adjointProcess_smul on, where the
inverse scaling is used. The action is a MulAction — 1 • A = A and (ab) • A = a • (b • A)
both hold — but it is not additive: 2 • A and A + A differ.
Equations
- One or more equations did not get rendered due to their size.
(λ A) u = λ (A u), the defining equation of the scalar multiple.
The product of convex processes, (B A) u = B (A u).
Equations
Instances For
The inverse of a product is the product of the inverses: (B A)⁻¹ = A⁻¹ B⁻¹.
The indicator bifunction of A₁ + A₂ is the infimal convolute F₁ □ F₂.
The indicator bifunction of B A is the product G F of the indicator bifunctions.
The adjoint of a convex process #
The adjoint of a supremum-oriented convex process:
A* x* = {u* | ⟨u, u*⟩ ≥ ⟨x, x*⟩ for every x ∈ A u and every u}. It is a convex process from Y
to V, and it is infimum oriented.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The adjoint of an infimum-oriented convex process: the same definition with the inequality
reversed. Keeping the two apart is what makes A** = cl A come out right.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The graph of A* is the polar of the graph of A, up to the sign flip on the first factor
that the adjoint of a bifunction carries.
The adjoint of the indicator bifunction of A is the indicator bifunction of A*. A*
carries the opposite orientation, which is why the indicator appears negated: an infimum-oriented
set is identified with -δ(· | ·).
The graph of A** is the bipolar of the graph of A. The two sign flips cancel, which is why
the second adjoint must be the infimum-oriented one.
The adjoint of a scalar multiple: (λ A)* = λ (A*) for λ > 0.
Rockafellar deduces this from the adjoint formula for Fλ. It is cheaper here as a direct
computation on cones: (y, v) ∈ graph (λA)* says λ⟨x, y⟩ ≤ ⟨u, v⟩ on graph A, which is
(y, λ⁻¹ v) ∈ graph A* after dividing by λ, and that is exactly v ∈ λ (A* y). Positivity of
λ is used only to divide.
The adjoint of a scalar multiple, for an infimum-oriented process: (λ A)* = λ (A*), with
the adjoint taken in the reversed sense. The proof is adjointProcess_smul with both inequalities
turned round.
A* is closed, and A** = cl A #
The adjoint of a convex process is a closed convex process, being an intersection of homogeneous closed half-spaces.
A** = cl A. Read through graph_coadjointProcess_adjointProcess, this is exactly the
bipolar theorem K°° = cl K for the graph.
A convex process is closed exactly when it is its own second adjoint.
The two inner products #
⟨Au, x*⟩ is the bracket of the indicator bifunction of A, and ⟨u, A* x*⟩ is the concave
bracket of its adjoint. Both are ordinary extremum problems over the values of a process: a
maximisation of ⟨·, x*⟩ over A u, and a minimisation of ⟨u, ·⟩ over A* x*.
The inner product ⟨Au, x*⟩ is the support function of the convex set A u. Every clause
about the x* variable below is then a property of support functions; the identity itself is
supportFn_eq_conj_indicatorFn read backwards, since ⟨Fu, ·⟩ is by definition the conjugate of
F u = δ(· | A u).
The first of Rockafellar's two extremum problems: ⟨Au, x*⟩ = sup {⟨x, x*⟩ | x ∈ A u}.
⟨Au, ·⟩ is positively homogeneous, being a support function.
⟨Au, ·⟩ is convex, being a conjugate.
⟨A ·, x*⟩ is positively homogeneous. This is the one clause that uses the definition of a
convex process rather than the general theory of brackets: A (λ u) = λ (A u) (eval_smul_arg),
and the support function of a positive multiple of a set is the corresponding multiple of the
support function.
⟨A ·, x*⟩ is concave, the bracket of a convex bifunction being concave in its first
variable.
The inner product ⟨Au, x*⟩ vanishes at the origin, because A 0 contains 0 and the
support function of a nonempty set is 0 at 0.
⟨Au, ·⟩ is closed as well as convex and positively homogeneous.
The second of Rockafellar's two extremum problems: ⟨u, A* x*⟩ = inf {⟨u, u*⟩ | u* ∈ A* x*}.
Together with bracket_indicatorBifun_apply this is a dual pair of linear programs; the two values
differ only by a closure in u (concaveBracket_adjointBifun_indicatorBifun_eq_partialCl₁). The
proof is the definition of the concave bracket plus adjointBifun_indicatorBifun: the indicator of
A* x* turns the unrestricted infimum into a restricted one.
⟨u, A* x*⟩ = cl_u ⟨Au, x*⟩: the two inner products differ by a closure in u.
The concave closure in the first variable is the only difference between them, for every convex
process — closedness of A plays no part. This is concaveBracket_adjointBifun_eq_partialCl₁
applied to the indicator bifunction.
The adjoint of a sum of processes #
The concave mirror of infConv_indicatorFn: the supremal convolute of two negated indicator
functions is the negated indicator function of the sum of the sets. Infimum-oriented convex sets
are carried by -δ(· | ·) (see ConvexProcess.adjointBifun_indicatorBifun), so this is the form
in which the adjoint of an infimal convolute speaks about processes.
Proof idea: supConv is infConv conjugated by negation, so the two negations inside cancel and
infConv_indicatorFn applies verbatim.
The adjoint of a sum is the sum of the adjoints: (A₁ + A₂)* = A₁* + A₂*.
Where the book assumes ri (dom A₁) ∩ ri (dom A₂) ≠ ∅, the hypothesis here is exactness of the
sum of the two support functions u ↦ ⟨Aᵢ u, y⟩, one instance per y. Since IsExactSum also
requires both summands proper, this is stronger than the book's hypothesis.
The adjoint of a product of processes #
A linear sandwich. If a concave p lies below a convex q, the two add exactly, and each
straddles 0 at the origin from its own side, then some ⟨·, y⟩ runs between them.
This is Fenchel's duality theorem read at a positively homogeneous pair. The two extrema are
p*(y) ≤ 0 and q*(y) ≥ 0 for free, so the attained dual value p*(y) - q*(y) ≥ 0 forces both to
vanish, and p*(y) = 0 and q*(y) = 0 say precisely p ≤ ⟨·, y⟩ and ⟨·, y⟩ ≤ q. No convexity
hypothesis appears: IsExactSum already carries it.
The inclusion that costs nothing: A* B* ⊆ (BA)*.
A pair (z*, u*) that factors through B* and then A* chains the two defining inequalities,
⟨z, z*⟩ ≤ ⟨x, x*⟩ ≤ ⟨u, u*⟩, for every factorisation u ↦ x ↦ z in BA.
The adjoint of a product is the product of the adjoints: (BA)* = A* B*.
Read directly this is a sandwich: (x*, u*) ∈ (BA)* says the concave x ↦ ⟨Bx, z*⟩ lies below
the convex x ↦ ⟨u*, A⁻¹x⟩, and a factorisation through A* and B* is exactly a linear
functional running between them, which exists_pairing_sandwich supplies.
Where the book assumes ri (range A) ∩ ri (dom B) ≠ ∅, the hypothesis here is that condition in
IsExactSum form, one instance per (z*, u*) — the two effective domains involved are
range A and dom B. As IsExactSum also requires both summands proper, it is stronger than the
book's hypothesis.
The conjugate of an image under a convex process #
The indicator bifunction of A⁻¹ is the indicator bifunction of A read backwards: both
values are δ(· | ·) of the same membership (u, x) ∈ graph A.
(A⁻¹)* = A*⁻¹. The inverse of a supremum-oriented process is infimum oriented, so its
adjoint is the coadjointProcess; with that reading the identity is an unfolding, both sides
being {(v, y) | ⟨x, y⟩ ≤ ⟨u, v⟩ for every (u, x) ∈ graph A}.
The F⁎* entry of the process/bifunction dictionary: the lower adjoint of the indicator
bifunction of A is the indicator bifunction of A*⁻¹.
This is adjointBifun_indicatorBifun with the two negations of lowerAdjointBifun cancelling
against the negation the opposite orientation of A* puts on its indicator, and the reversal
indicatorBifun_inv absorbing the transposition. It is the last thing (Af)* = A*⁻¹ f* needs.
The indicator bifunction of a convex process is finite at the origin, 0 being in A 0.
This is the "finite somewhere" side condition the closed-image results ask of F.
(Af)* = A*⁻¹ f*, where Af is the image Ff of f under the indicator bifunction F
of A, i.e. (Af)(x) = inf {f u | x ∈ A u}.
This specialises the conjugate of an image under a bifunction; the only work is the dictionary
lowerAdjointBifun_indicatorBifun. Rockafellar's ri (dom f) ∩ ri (dom A) ≠ ∅ becomes the
IsExactSum of conj_imageBifun, dom F being dom A.
The infimum defining (A*⁻¹ f*)(x*) is attained. This is exists_conj_imageBifun_eq read
through the same dictionary.
Bounded values force a linear transformation #
A 0 is a convex cone containing the origin, so if it is bounded it is {0}.
If A 0 is bounded and A has full domain, every value of A is a single point.
A convex process whose domain is everything and whose value at the origin is bounded is (the graph of) a linear transformation.
Linear transformations are exactly the convex processes all of whose values are nonempty and
bounded; the converse direction is ConvexProcess.dom_ofLinearMap together with
ConvexProcess.eval_ofLinearMap.
Closedness of the image #
If A is a closed convex process, C is a nonempty closed convex set, and no non-zero
vector of A⁻¹ 0 recedes C, then A C is closed. In particular this holds when C is
bounded, since then 0⁺C = {0}.
This is the recession-cone criterion for a linear image, at the projection (u, x) ↦ x: A C is
the image of graph A ∩ (C × X), whose recession cone is graph A ∩ (0⁺C × X) — the graph is its
own recession cone, being a pointed convex cone — and whose intersection with the kernel of the
projection is {(v, 0) | v ∈ A⁻¹ 0 ∩ 0⁺C}. No duality is needed.
The image of a nonempty compact convex set under a closed convex process is closed, the
bounded case of isClosed_image.
The reflected process, and the dictionary between the two orientations #
The reflection of a convex process: the process whose graph is the reflection of
graph A through the origin, so that (A.reflect) u = -(A (-u)).
Reflection is what exchanges the two orientations: reversing the inequality in the definition of
the adjoint is the same as reflecting the graph, so
adjointProcess Bu Bx A.reflect = coadjointProcess Bu Bx A (adjointProcess_reflect). Read the
other way round (coadjointProcess_eq_reflect_adjointProcess) it puts the reflection on the
conclusion, where reflect_add and reflect_comp cancel it, so the infimum-oriented mirrors
carry the supremum-oriented hypotheses unchanged.
Equations
Instances For
Reflection bundled: an additive automorphism of the convex processes from U to X.
It is involutive (reflect_involutive) and additive (reflect_add), so .injective, .eq_iff
and .toPerm come from the bundled form instead of being re-proved.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Reversing the inequality is reflecting the graph: the supremum-oriented adjoint of the reflected process is the infimum-oriented adjoint of the original one.
The mirror of adjointProcess_reflect: reflection also carries the infimum-oriented adjoint
back to the supremum-oriented one.
The infimum-oriented adjoint is the reflection of the supremum-oriented one. Together with
adjointProcess_reflect this is the whole content of "the adjoint of an infimum-oriented process
is defined in the same way, except that the inequality is reversed".
Reflection preserves closedness: it is a preimage under the homeomorphism p ↦ -p.
The adjoint of an infimum-oriented convex process is closed, as in the supremum-oriented case.
Taking the infimum-oriented adjoint and then the supremum-oriented one also returns cl A.
This is graph_coadjointProcess_adjointProcess_eq_closure read through
adjointProcess_reflect; the two sign flips still cancel, and it is what turns the closed halves
of the sum and product theorems into corollaries of their open halves.
A convex process is closed exactly when the supremum-oriented adjoint of its infimum-oriented adjoint is itself.
Sums and products for infimum-oriented processes #
The adjoint of a sum, for two infimum-oriented processes: (A₁ + A₂)* = A₁* + A₂*.
Rockafellar states the result for two processes "with the same orientation" and leaves the
infimum-oriented case implicit. It is the supremum-oriented theorem read through
coadjointProcess_eq_reflect_adjointProcess, reflection distributing over sums; the hypothesis is
adjointProcess_add's own, verbatim.
The adjoint of a product, for two infimum-oriented processes: (BA)* = A* B*.
Like coadjointProcess_add, this reflects the adjoints rather than the processes, so the
hypothesis is adjointProcess_comp's own; reflect_comp is what makes the product come out in
the same order.
The closed halves of the sum and product theorems #
The sum of two closed convex processes is the infimum-oriented adjoint of the sum of their adjoints, provided the two adjoints add exactly. This is the identity both closed-half results about sums come from.
The sum of two closed convex processes is closed. It is an adjoint, and an adjoint is always closed.
(A₁ + A₂)* is the closure of A₁* + A₂*, for closed A₁ and A₂.
The product of two closed convex processes is the infimum-oriented adjoint of the product of their adjoints. This is the identity both closed-half results about products come from.
The product of two closed convex processes is closed.
(BA)* is the closure of A* B*, for closed A and B.
The two inner products for infimum-oriented processes #
For an infimum-oriented process the indicator bifunction is the concave function -δ(· | A u),
and the inner product ⟨Au, x*⟩ is its concave conjugate: an infimum over A u rather than a
supremum. As with adjointProcess/coadjointProcess, the two orientations are separate
definitions rather than two branches of one flag, and the whole mirror is driven by the single
sign dictionary coBracket_eq_neg_bracket.
The inner product ⟨Au, x*⟩ of an infimum-oriented convex process:
⟨Au, x*⟩ = inf {⟨x, x*⟩ | x ∈ A u}.
This is the concave conjugate of the concave indicator -δ(· | A u), exactly as
bracket _ A.indicatorBifun u is the convex conjugate of δ(· | A u).
Equations
- Tdaf.ConvexAnalysis.ConvexProcess.coBracket Bx A u y = ⨅ x ∈ A.eval u, ↑((Bx x) y)
Instances For
The first of the two extremum problems, in infimum-oriented form:
⟨Au, x*⟩ = inf {⟨x, x*⟩ | x ∈ A u}.
The sign dictionary between the two orientations: the infimum-oriented inner product is
minus the supremum-oriented one, read at the reflected dual vector. Note that only the dual
variable is reflected — the primal variable u is untouched, because reversing the orientation
of a process does not reverse its argument.
The infimum-oriented inner product is minus a support function, read at the reflected dual vector.
The negative of ⟨Au, ·⟩ is the supremum-oriented inner product composed with the linear
reflection x* ↦ -x*. This is the form in which the sign dictionary feeds the convexity and
closedness lemmas for a composition with a linear map.
⟨Au, ·⟩ is positively homogeneous, in the infimum-oriented mirror.
⟨Au, ·⟩ is concave in the infimum-oriented mirror, being a concave conjugate.
⟨A ·, x*⟩ is positively homogeneous, in the infimum-oriented mirror.
⟨A ·, x*⟩ is convex in the infimum-oriented mirror, where in the supremum-oriented case
it is concave. Reversing the orientation of a process exchanges convexity and concavity in both
variables at once.
The inner product vanishes at the origin, in the infimum-oriented mirror.
⟨Au, ·⟩ is a closed concave function in the infimum-oriented mirror, the reflection
x* ↦ -x* being a homeomorphism.
The second of the two extremum problems, in infimum-oriented form:
⟨u, A* x*⟩ = sup {⟨u, u*⟩ | u* ∈ A* x*}, where A* is now the infimum-oriented adjoint
coadjointProcess, whose values are the reflections of those of adjointProcess.
⟨u, A* x*⟩ = cl_u ⟨Au, x*⟩, the infimum-oriented mirror.
The closure is now the ordinary convex closure clFn, because ⟨A ·, x*⟩ is convex rather than
concave (convexFn_coBracket_arg); in the supremum-oriented statement
concaveBracket_adjointBifun_indicatorBifun_eq_partialCl₁ it is the concave closure clConcave
packaged as partialCl₁. As there, closedness of A plays no part.