NEWELL-LITTLEWOOD NUMBERS III: EIGENCONES AND GIT-SEMIGROUPS

SHILIANG GAO, GIDON ORELOWITZ, NICOLAS RESSAYRE, AND ALEXANDER YONG

ABSTRACT. The Newell-Littlewood numbers are tensor product multiplicities of Weyl mod-ules for the classical groups in the stable range. Littlewood-Richardson coefficients forma special case. Klyachko connected eigenvalues of sums of Hermitian matrices to the sat-urated LR-cone and established defining linear inequalities. We prove analogues for thesaturated NL-cone:• an eigenvalue interpretation;• a minimal list of defining linear inequalities;• a description by Extended Horn inequalities, as conjectured in part II of this series; and• a factorization of NL-numbers, on the boundary.

1. INTRODUCTION

This is the third installment in a series [9, 10] about the Newell-Littlewood numbers [21, 19]

(1) Nλ,µ,ν =∑α,β,γ

cλα,βcµβ,γc

νγ,α;

the indices are partitions in Parn = (λ1, λ2, . . . , λn) ∈ Zn≥0 : λ1 ≥ λ2 ≥ . . . ≥ λn. In(1), cλα,β is the Littlewood-Richardson coefficient. The Littlewood-Richardson coefficients arethemselves Newell-Littlewood numbers: if |ν| = |λ|+|µ| thenNλ,µ,ν = cνλµ. The goal of thisseries is to establish analogues of results known for Littlewood-Richardson coefficients.This paper proves NL-generalizations of breakthrough results of Klyachko [14].

Fix n ∈ N := 1, 2, 3, . . .. The paper [9] investigated

NL-semigroup(n) = (λ, µ, ν) ∈ (Parn)3 : Nλ,µ,ν > 0.Indeed, NL-semigroup is a finitely generated semigroup [9, Section 5.2]. A good approxi-mation of it is the saturated semigroup:

NL-sat(n) = (λ, µ, ν) ∈ (ParQn )3 : ∃t > 0 Ntλ,tµ,tν 6= 0,

where ParQn = (λ1, . . . , λn) ∈ Qn : λ1 ≥ . . . ≥ λn ≥ 0. Our main results give descriptionsof NL-sat(n), including with a minimal list of defining linear inequalities.

Fixm ∈ N and consider the symplectic Lie algebra sp(2m,C). The irreducible sp(2m,C)-representations V (λ) are parametrized by their highest weight λ ∈ Parm (see Section 3.1for details). The tensor product multiplicities multmλ,µ,ν are defined by

V (λ)⊗ V (µ) =∑

ν∈Parm

V (ν)⊕multmλ,µ,ν .

Since sp(2m,C)-representations are self-dual, multmλ,µ,ν is symmetric in its inputs. Thesupports of these multiplicities (and more generally when sp(2m,C) is replaced by any

Date: July 5, 2021.

1

semisimple Lie algebra) are of significant interest (see, e.g., the survey [18] and the refer-ences therein). Consider the finitely generated semigroup

sp -semigroup(m) = (λ, µ, ν) ∈ (Parm)3 : multmλ,µ,ν > 0,

and the cone generated by it

sp -sat(m) = (λ, µ, ν) ∈ (ParQm)3 : ∃t > 0 multmtλ,tµ,tν > 0.

For m ≥ n, by postpending 0’s, Parn embeds into Parm. Newell-Littlewood numbersare tensor product multiplicities for sp(2m,C) in the stable range [16, Corollary 2.5.3]:

(2) ∀(λ, µ, ν) ∈ (Parn)3 if m ≥ 2n then multmλ,µ,ν = Nλ,µ,ν .

Now, (2) immediately implies

(3) NL-sat(n) = sp -sat(m) ∩ (ParQn )3, for any m ≥ 2n.

Our first result says the relationship of NL-sat to sp -sat is independent of the stable range.

Theorem 1.1. For any m ≥ n ≥ 1,

NL-sat(n) = sp -sat(m) ∩ (ParQn )3.

Theorem 1.1 has a number of consequences. Define

LR-sat(n) = (λ, µ, ν) ∈ (ParQn )3 : ∃t > 0 ctνtλ,tµ > 0.

Klyachko [14] showed that LR-sat(n) describes the possible eigenvalues λ, µ, ν of threen × n Hermitian matrices A,B,C (respectively) such that A + B = C. Similarly, Theo-rem 1.1 shows that NL-sat(n) describes solutions to a more general eigenvalue problem;see Section 2.6 and Proposition 3.1.1

Another major accomplishment of [14] was the first proved description of LR-sat(n) vialinear inequalities. We have three such descriptions of NL-sat(n). We now state the first ofthese. Set [n] = 1, . . . , n. For A ⊂ [n] and λ ∈ Parn, let λA be the partition using the onlyparts indexed by A; namely, if A = i1 < · · · < ir then λA = (λi1 , . . . , λir). In particular,|λA| =

∑i∈A λi. Using the known descriptions of sp -sat(n) [1, 23, 25] we deduce from

Theorem 1.1 a minimal list of inequalities defining NL-sat(n):

Theorem 1.2. Let (λ, µ, ν) ∈ (Parn)3. Then (λ, µ, ν) ∈ NL-sat(n) if and only if

(4) 0 ≤ |λA| − |λA′ |+ |µB| − |µB′ |+ |νC | − |νC′ |

for any subsets A,A′, B,B′, C, C ′ ⊂ [n] such that

(1) A ∩ A′ = B ∩B′ = C ∩ C ′ = ∅;(2) |A|+ |A′| = |B|+ |B′| = |C|+ |C ′| = |A′|+ |B′|+ |C ′| =: r;(3) cτ

0(C,C′)

τ0(A,A′)∨[(2n−2r)r ] τ0(B,B′)∨[(2n−2r)r ] = cτ2(C,C′)

τ2(A,A′)∨[rr ] τ2(B,B′)∨[rr ]= 1.

Moreover, this list of inequalities is irredundant.

1We remark that since (λ, µ, ν) ∈ LR-sat(n) is also in NL-sat(n) if and only if |λ| + |µ| = |ν| (see [9,Lemma 2.2(II)]), the sln-eigencone is a facet of the sp2n-eigencone.

2

The definition of the partitions occurring in condition (3) is in Section 3.2.The proofs of Theorems 1.1 and 1.2 use ideas of P. Belkale-S. Kumar [1] on their de-

formation of the cup product on flag manifolds, as well as the third author’s work onGIT-semigroups/cones [23, 25]. We interpret NL-sat(n) from the latter perspective in Sec-tion 5 (see Proposition 5.2) by study of the truncated tensor cone. Our argument requiresus to generalize [1, Theorem 28] [23, Theorem B] (recapitulated here as Theorem 2.3); seeTheorem 5.1. As an application, we obtain Theorem 1.3 below, which is a factorizationof the NL-coefficients on the boundary of NL-sat(n). Let λ ∈ Parn and A,A′ ⊂ [n]. WriteA′ = i′1 < · · · < i′s and A = i1 < · · · < it and set

λA,A′ = (λi′1 , . . . , λi′s ,−λit , . . . ,−λi1) and λA,A′= λ[n]−(A∪A′), etc.

Theorem 1.3. Let A,A′, B,B′, C, C ′ ⊂ [n] such that

(1) A ∩ A′ = B ∩B′ = C ∩ C ′ = ∅;(2) |A|+ |A′| = |B|+ |B′| = |C|+ |C ′| = |A′|+ |B′|+ |C ′| =: r;(3) cτ

0(C,C′)

τ0(A,A′)∨[(2n−2r)r ] τ0(B,B′)∨[(2n−2r)r ] = cτ2(C,C′)

τ2(A,A′)∨[rr ] τ2(B,B′)∨[rr ]= 1,

as in Theorem 1.2. For (λ, µ, ν) ∈ (Parn)3 such that

(5) 0 = |λA| − |λA′ |+ |µB| − |µB′|+ |νC | − |νC′ |,

(6) Nλ,µ,ν = cν∗C,C′

λA,A′ ,µB,B′NλA,A′ ,µB,B′ ,νC,C′ .

Theorem 1.3 is an analogous to [5, Theorem 7.4] and [13, Theorem 1.4] for cνλ,µ.

Knutson-Tao’s celebrated Saturation Theorem [15] proves, inter alia, that LR-sat(n) is de-scribed by Horn’s inequalities (see, e.g., Fulton’s survey [7]). This posits a generalization:

Conjecture 1.4 (NL-Saturation [9, Conjecture 5.5]). Let (λ, µ, ν) ∈ (Parn)3. Then Nλ,µ,ν 6= 0if and only if |λ|+ |µ|+ |ν| is even and there exists t > 0 such that Ntλ,tµ,tν 6= 0.

Theorem 1.2 permits us to prove Conjecture 1.4 for n ≤ 5, by computer-aided calcula-tion of Hilbert bases; see Section 6. This is the strongest evidence of the conjecture to date;previously, [9, Corollary 5.16] proved the n = 2 case, by combinatorial reasoning.

The conditions parametrizing the inequalities occurring in the Horn’s inequalities forLR-sat(n) are inductive, as they depend on LR-sat(n′) for n′ < n. Theorem 1.2 is notinductive. In [10], a conjectural description of NL-sat(n), somewhat closer to the Hornconjecture for LR-sat(n), was given. More precisely, the formulation [10, Conjecture 1.4]subsumes both Conjecture 1.4 and a description of NL-sat using extended Horn inequalities[10, Definition 1.2]. Our last result is a proof the latter part of the conjecture.

Let λ1, . . . , λs ∈ Parn for s ≥ 3. Treat the indices 1, . . . , s as elements of Z/sZ. Weintroduce the multiple Newell-Littlewood number as

(7) Nλ1,...,λs =∑

(α1,...,αs)∈(Parn)s

∏i∈Z/sZ

cλiαi αi+1.

When s = 3, we recover Newell-Littlewood numbers.2 To A = i1 < · · · < ir ⊂ [n],associate the partition

(8) τ(A) = (ir − r ≥ · · · ≥ i1 − 1).

2Nλ1,...,λs also has a representation-theoretic interpretation. Discussion may appear elsewhere.

3

Theorem 1.5. Let (λ, µ, ν) ∈ (ParQn )3. Then (λ, µ, ν) ∈ NL-sat(n) if and only if

(9) 0 ≤ |λA| − |λA′|+ |µB| − |µB′ |+ |νC | − |νC′ |

for any subsets A,A′, B,B′, C, C ′ ⊂ [n] such that

(1) A ∩ A′ = B ∩B′ = C ∩ C ′ = ∅;(2) |A|+ |A′| = |B|+ |B′| = |C|+ |C ′| = |A′|+ |B′|+ |C ′|;(3) Nτ(A′),τ(B),τ(C′),τ(A),τ(B′),τ(C) 6= 0.

As explained in Section 10.1, the inequalities of Theorem 1.5 are easily equivalent to theextended Horn inequalities. However, the reformulation of Theorem 1.5 is more compact.The main part of the proof is to show that the inequalities of Theorem 1.2 are all instancesof extended Horn inequalities. That argument, which occupies Sections 7–10, requiresheavy-lifting in tableau combinatorics. Along the way, we establish the relationship be-tween the demotion algorithm [9, Claim 3.5] and the RSK correspondence (Theorem 8.2), aswell as a nonvanishing result about multiple Newell-Littlewood numbers (Theorem 9.1).

2. GENERALITIES ON THE TENSOR CONES

2.1. Finitely generated semigroups. Γ ⊆ Zn is a semigroup if ~0 ∈ Γ and is closed underaddition. A finitely generated Γ generates a closed convex polyhedral cone ΓQ ⊆ Qn:

ΓQ = x ∈ Qn : ∃t ∈ Z>0 tx ∈ Γ.

The subgroup of Zn generated by Γ is

ΓZ = x− y : x, y ∈ Γ.

The semigroup Γ is saturated if Γ = ΓZ ∩ ΓQ.

2.2. GIT-semigroups. We recall the GIT-perspective of [23]. Let G be a complex reduc-tive group acting on an irreducible projective variety X . Let PicG(X) be the group ofG-linearized line bundles. Given L ∈ PicG(X), let H0(X,L) be the space of sections of L;it is a G-module. Let H0(X,L)G be the subspace of invariant sections. Define

GIT-semigroup(G,X) = L ∈ PicG(X) : H0(X,L)G 6= 0.

This is a semigroup sinceX being irreducible says the product of two nonzeroG-invariantsections is a nonzero G-invariant section. The saturated version of it is

GIT-sat(G,X) = L ∈ PicG(X)⊗Q : ∃t > 0 H0(X,L⊗t)G 6= 0.

2.3. The tensor semigroup. Let g be a semisimple complex Lie algebra, with fixed Borelsubalgebra b and Cartan subalgebra t ⊂ b. Denote by Λ+(g) ⊂ t∗ the semigroup of thedominant weights. It is contained in the weight lattice Λ(g) ' Zr, where r is the rank ofg. Given λ ∈ Λ+(g), denote by Vg(λ) (or simply V (λ)) the irreducible representation of gwith highest weight λ. Let V (λ)∗ be the dual representation. Consider the semigroup

g -semigroup = (λ, µ, ν) ∈ (Λ+(g))3 : V (ν)∗ ⊂ V (λ)⊗ V (µ),

and the generated cone g -sat in (Λ(g)⊗Q)3. When g = sp(2m,C) we have V (ν)∗ ' V (ν)and g -semigroup is what we denoted by sp-semigroup(m) in the introduction. The set

4

g -semigroup spans the rational vector space (Λ(g)⊗Q)3, or equivalently, the cone g -sat hasnonempty interior. The group (g -semigroup)Z is well-known (see, e.g., [22, Theorem 1.4]):

(g -semigroup)Z = (λ, µ, ν) ∈ (Λ(g))3 : λ+ µ+ ν ∈ ΛR(g),where ΛR(g) is the root lattice of g.

We now interpret g -semigroup in terms of Section 2.2. Consider the semisimple, simply-connected algebraic group G with Lie algebra g. Denote by B and T the connected sub-groups of G with Lie algebras b and t respectively. The character groups X(B) = X(T ) =Λ(g) of B and T coincide. For λ ∈ X(T ), Lλ is the unique G-linearized line bundle on theflag variety G/B such that B acts on the fiber over B/B with weight −λ.

AssumeX = (G/B)3. Then PicG(X) identifies withX(T )3. For (λ, µ, ν) ∈ X(T )3, defineL(λ,µ,ν). By the Borel-Weil Theorem,

(10) H0(X,L(λ,µ,ν)) = V (λ)∗ ⊗ V (µ)∗ ⊗ V (ν)∗.

In particular, GIT-semigroup(G,X) ' g -semigroup.Given three parabolic subgroups P,Q and R containing B, we consider more generally

X = G/P × G/Q × G/R. Then PicG(X) identifies with X(P ) ×X(Q) ×X(R) which is asubgroup of X(T )3. Moreover,

GIT-semigroup(G,X) = GIT-semigroup(G, (G/B)3) ∩ (X(P )×X(Q)×X(R)).

2.4. Schubert calculus. We need notation for the cohomology ring H∗(G/P,Z); P ⊃ Bbeing a parabolic subgroup. Let W (resp. WP ) be the Weyl group of G (resp. P ). Let ` :W −→ N be the Coxeter length, defined with respect to the simple reflections determinedby the choice of B. Let W P be the minimal length representatives of the cosets in W/WP .

For a closed irreducible subvariety Z ⊂ G/P , let [Z] be its class in H∗(G/P,Z), of degree2(dim(G/P ) − dim(Z)). For v ∈ W P , set τv = [BvP/P ] (dim(BvP/P ) = `(v)). Let w0 bethe longest element of W and w0,P be the longest element of WP . Set v∨ = w0vw0,P andτv = τ v

∨ ; τ v and τv are Poincare dual.Let ρ be the half sum of the positive roots of G. To any one-parameter subgroup τ :

C∗ → T , associate the parabolic subgroup (see [20])

P (τ) = g ∈ G : limt→0

τ(t)gτ(t−1) exists.

Fix such a τ such that P = P (τ).For v ∈ W P , define the BK-degree of τ v ∈ H∗(G/P,Z) to be BK-deg(τ v) := 〈v−1(ρ)−ρ, τ〉.

Let v1, v2 and v3 in W P . By [1, Proposition 17], if τ v3 appears in the product τ v1 · τ v2 then

(11) BK-deg(τ v3) ≤ BK-deg(τ v1) + BK-deg(τ v2).

In other words, the BK-degree filters the cohomology ring. Let 0 denote the associatedgraded product on H∗(G/P,Z).

2.5. Well-covering pairs. LetX be a projective variety. In [23], GIT-sat(G,X) is describedin terms of well-covering pairs. When X = (G/B)3, it recovers the description made byBelkale-Kumar [1]. We now discuss the case X = G/P × G/Q × G/R is the product ofthree partial flag varieties of G.

Let τ be a dominant one-parameter subgroup of T . The centralizer Gτ of the image ofτ in G is a Levi subgroup. Moreover, P (τ) is the parabolic subgroup generated by B and

5

Gτ . Let C be an irreducible component of the fixed set Xτ of τ in X . It is well-known thatC is the (Gτ )3-orbit of some T -fixed point:

(12) C = Gτu−1P/P ×Gτv−1Q/Q×Gτw−1R/R,

with u ∈ WP\W/WP (τ) and similarly for v and w. Set

C+ = P (τ)u−1P/P × P (τ)v−1Q/Q× P (τ)w−1R/R.

Then the closure of C+ is a Schubert variety (for G3) in X . By [23, Proposition 11], the pair(C, τ) is well-covering if and only if

(13) [PuP (τ)/P (τ)]0 [QvP (τ)/P (τ)]0 [RwP (τ)/P (τ)] = [pt] ∈ H∗(G/P (τ),Z).

It is said to be dominant if

(14) [PuP (τ)/P (τ)] · [QvP (τ)/P (τ)] · [RwP (τ)/P (τ)] 6= 0 ∈ H∗(G/P (τ),Z).

In this paper, the reader can take these characterizations as definitions of well-coveringand dominant pairs. They are used in [23] to produce inequalities for the GIT-cones:

Proposition 2.1. Let (λ, µ, ν) ∈ X(P ) × X(Q) × X(R) be dominant such that L(λ,µ,ν) ∈GIT-sat(G,X). Let (C, τ) be a dominant pair. Then,

(15) 〈uτ, λ〉+ 〈vτ, µ〉+ 〈wτ, ν〉 ≤ 0.

Here (u, v, w) are determined by C and equation (12).

In Proposition 2.1 and below, 〈·, ·〉 denotes the pairing between one parameter sub-groups and characters of T . Among the inequalities given by Proposition 2.1, those asso-ciated to well-covering pairs are sufficient to define the GIT-cone (see [23, Proposition 4]):

Proposition 2.2. Let (λ, µ, ν) ∈ X(P )×X(Q)×X(R) be dominant. Then,L(λ,µ,ν) ∈ GIT-sat(G,X)if and only if for any dominant one-parameter subgroup τ of T and any well-covering pair (C, τ),

(16) 〈uτ, λ〉+ 〈vτ, µ〉+ 〈wτ, ν〉 ≤ 0.

Here (u, v, w) are determined by C and equation (12).3

In the case P = Q = R = B, there is a more precise statement. The fact that theinequalites define the cone is Belkale-Kumar [1, Theorem 28]. The irredundancy is [23,Theorem B]. Let α be a simple root of G. Denote by Pα the associated maximal parabolicsubgroup of G containing B. Denote by $α∨ the associated fundamental one-parametersubgroup of T characterized by 〈$α∨ , β〉 = δβα for any simple root β.

Theorem 2.3 ([1, Theorem 28],[23, Theorem B]). Here X = (G/B)3. Let (λ, µ, ν) ∈ X(T )3

be dominant. Then, L(λ,µ,ν) ∈ GIT-sat(G,X) if and only if for any simple root α, for any u, v, win W Pα such that

(17) [BuPα/Pα]0 [BvPα/Pα]0 [BwPα/Pα] = [pt] ∈ H∗(G/Pα,Z),

(18) 〈u$α∨ , λ〉+ 〈v$α∨ , µ〉+ 〈w$α∨ , ν〉 ≤ 0.

Moreover, this list of inequalities is irredundant.

3Proposition 2.2 implies Proposition 2.1 is also a characterization of GIT-sat(G,X), but we have statedthe weaker form of the latter to emphasize it is an easier result.

6

Theorem 2.3 can be obtained from Proposition 2.2 by showing that it is sufficient toconsider the one-parameter subgroups τ equal to $α∨ for some simple root α. See theproof of Theorem 5.1 below for a similar argument.

2.6. The eigencone. A relationship between g -sat and projections of coadjoint orbits wasdiscovered by Heckman [12]. Theorem 2.4 below interprets g -sat in terms of eigenvalues.

Fix a maximal compact subgroup U of G such that T ∩U is a Cartan subgroup of U . Letu and t denote the Lie algebras of U and T respectively. Let t+ be the Weyl chamber oft corresponding to B. Let

√−1 denote the usual complex number. It is well-known that√

−1t+ is contained in u and that the map

(19)t+ −→ u/Uξ 7−→ U.(

√−1ξ)

is a homeomorphism. Here U acts on u by the adjoint action. Consider the set

Γ(U) := (ξ, ζ, η) ∈ (t+)3 : U.(√−1ξ) + U.(

√−1ζ) + U.(

√−1η) 3 0.

Let u∗ (resp. t∗) denote the dual (resp. complex dual) of u (resp. t). Let t∗+ denote thedominant chamber of t∗ corresponding to B. By taking the tangent map at the identity,one can embed X(T )+ in t∗+. Note that this embedding induces a rational structure onthe complex vector space t∗. Moreover it allows to embed the tensor cone g -sat in (t∗+)3.

The Cartan-Killing form allows to identify t+ and t∗+. In particular Γ(U) also embedsin (t∗+)3; the so obtained subset of (t∗+)3 is denoted by Γ(U) to avoid any confusion. Thefollowing result is well-known; see e.g., [17, Theorem 5] and the references therein.

Theorem 2.4. The set Γ(U) is a closed convex polyhedral cone. Moreover, g -sat is the set of therational points in Γ(U).

3. THE CASE OF THE SYMPLECTIC GROUP

3.1. The root system of type C. Let V = C2n with the standard basis ~e1, ~e2, . . . , ~e2n. LetJn be the n×n “anti-diagonal” identity matrix and define a skew-symmetric bilinear form

ω(•, •) : V × V → C using the block matrix Ω :=

[0 Jn−Jn 0

]. By definition, the symplectic

group G = Sp(2n,C) is the group of automorphisms of V that preserve this bilinear form.Given a n × n matrix A = (Aij)1≤i,j≤n define TA by (TA) = An+1−j n+1−i, obtained from

A by reflection across the antidiagonal. The Lie algebra sp(2n,C) is the set of matricesM ∈ Mat2n×2n(C) such that tMΩ + ΩM = 0; namely,

(20) sp(2n,C) =

(A BC TA

):A,B,C of size n× n,TB = B and TC = C

which has complex dimension 2n2 + n. The Lie algebra u(2n,C) of the unitary groupU(2n,C) is the set of anti-Hermitian matrices. Thus, (20) gives

(21) sp(2n,C) ∩ u(2n,C) =

(A B−tB TA

): tA = A and TB = B

which has real dimension 2n2 + n. As a consequence, U(2n) ∩ Sp(2n,C) is a maximalcompact subgroup of Sp(2n,C).

7

Let B be the Borel subgroup of upper triangular matrices in G. Let

T = diag(t1, . . . , tn, t−1n , . . . , t−1

1 ) : ti ∈ C∗

be the maximal torus contained in B. For i ∈ [1, n], let εi denote the character of T thatmaps diag(t1, . . . , tn, t

−1n , . . . , t−1

1 ) to ti; then X(T ) = ⊕ni=1Zεi. Here

Φ+ = εi ± εj : 1 ≤ i < j ≤ n ∪ 2εi : 1 ≤ i ≤ n,∆ = α1 = ε1 − ε2, α2 = ε2 − ε3, . . . , αn−1 = εn−1 − εn, αn = 2εn, andX(T )+ =

∑ni=1 λiεi : λ1 ≥ . . . ≥ λn ≥ 0 = Parn.

For i ∈ [1, 2n], set i = 2n+1−i. The Weyl groupW ofGmay be identified with a subgroupof the Weyl group S2n of SL(V ). More precisely,

W = w ∈ S2n : w(i) = w(i) ∀i ∈ [1, 2n].

Observe that T ∩ U(2n,C) has real dimension n and is a maximal torus of U(2n) ∩

Sp(2n,C). The bijection (19) implies that any matrix M1 =

(A B−tB TA

)in sp(2n,C) ∩

u(2n,C) (see (21)) is diagonalizable with eigenvalues in√−1R. Moreover, ordering the

eigenvalues by nonincreasing order, we get

λ(√−1M1) ∈ (λ1 ≥ . . . ≥ λn ≥ −λn ≥ . . . ≥ −λ1) : λi ∈ R.

For λ = (λ1, . . . , λn) ∈ Parn, set λ = (λ1, . . . , λn,−λn, . . . ,−λ1). Now, Theorems 1.1 withm = n and Theorem 2.4 give an interpretation of NL-sat(n) in terms of eigenvalues:

Proposition 3.1. Let λ, µ, ν ∈ Parn. Then (λ, µ, ν) ∈ NL-sat(n) if and only if there exist threematrices M1,M2,M3 ∈ sp(2n,C) ∩ u(2n,C) such that M1 +M2 +M3 = 0 and

(λ, µ, ν) = (λ(√−1M1), λ(

√−1M2), λ(

√−1M3)).

3.2. Isotropic Grassmannians and Schubert classes. Our reference for this subsection is[25, Section 5]. For r = 1, . . . , n, the one-parameter subgroup $α∨r is given by

$α∨r (t) = diag(t, . . . , t, 1, . . . , 1, t−1, . . . , t−1),

where t and t−1 occur r times.A subspace W ⊆ V is isotropic if for all ~v,~v′ ∈ W , ω(~v,~v′) = 0. Given an r-subset

I ⊂ [2n], we set FI = Span(~ei : i ∈ I). Clearly, FI is isotropic if and only if I ∩ I = ∅,where I = i : i ∈ I. Now, Pαr is the stabilizer of the isotropic subspace F1,...,r. Thus,G/Pαr = Grω(r, 2n) is the Grassmannian of isotropic r-dimensional vector subspaces of V .

Let S(r, 2n) denote the set of subsets of 1, . . . , 2nwith r elements. Set

Schub(Grω(r, 2n)) := I ∈ S(r, 2n) : I ∩ I = ∅.

If I = i1 < · · · < ir ∈ Schub(Grω(r, 2n)), let ik := ik for k ∈ [r], and ir+1 < · · · < ir+1 =

[2n] − (I ∪ I). Therefore wI = (i1, . . . , i2n) ∈ S2n is the element of W Pαr corresponding toFI ; that is, FI = wIP

αr/Pαr .Set

Schub′(Grω(r, 2n)) :=

(A,A′) : A ∈ S(a, n), A′ ∈ S(a′, n) for some a and a′ s.t.

a+ a′ = r and A ∩ A′ = ∅

.

8

a′

a

2n− r

n− a′ n− a

τ(A′)

τ(A)

τ(A) rotated 180

τ(I)

τ(A′)

FIGURE 1. τ(I), τ(A) and τ(A′).

This map is a bijection:

(22) Schub(Grω(r, 2n)) −→ Schub′(Grω(r, 2n))I 7−→ (I ∩ [n], I ∩ [n]).

Recall from the introduction the definition of τ(I) and hence τ(A) and τ(A′). The relation-ship between these three partitions is depicted in Figure 1.

Definition 3.2. For subsets X ⊆ Y ⊆ [2n], define

χ(X,Y ) :[2n]→ 0, 1, 2

i 7→

1 if i ∈ X2 if i ∈ Y \X0 if i /∈ Y

Treat χ(X,Y ) as a word of length 2n with letters in 0, 1, 2. Removing all the 2’s in theword, we obtain the characteristic function χ2

(X,Y ) of some subset I2 of [2n−|Y |+ |X|] with|X| elements. Similarly, removing all 0’s in the word, and then, replacing all the 2’s with0’s, gives the characteristic function χ0

(X,Y ) of some subset I0 of [|Y |] with |X| elements.

Given I ∈ Schub(Grω(r, 2n)) for some 1 ≤ r ≤ n, we apply this construction to I ⊂([2n]− I). One obtains I2 ∈ S(r, 2r) and I0 ∈ S(r, 2n− r).

Definition 3.3. For a partition λ = (λ1, . . . , λk) ⊆ (ab), i.e., the rectangle with a columns andb rows. Define λ∨ with respect to (ab) to be the partition (a− λb, a− λb−1, . . . , a− λ1) wherewe set λi = 0 for i > k. We will denote this by λ∨[ab].

Now,

(23) codim(BFI) = |τ(I0)∨[(2n−2r)r]|+ 1/2(|τ(I2)∨[rr]|+ |I ∩ [n]|).Moreover,

(24) dim(Grω(r, 2n)) = r(2n− 2r) +r(r + 1)

2.

Let I ∈ Schub(Grω(r, 2n)) and A = I ∩ [n], A′ = I ∩ [n] be the corresponding pair inSchub′(Grω(r, 2n)). Set

(25) τ 0(A,A′) = τ(I0), τ 2(A,A′) = τ(I2).

9

While the above discussion defines τ 0(A,A′), τ 2(A,A′) through the bijection (22), weemphasize that these partitions from Theorem 1.2 can be defined explicitly:

Definition-Lemma 3.4. Set a = |A| and a′ = |A′|. Write A = α1 < · · · < αa and A′ =α′1 < · · · < α′a′. Then

τ 2(A,A′)k = a+ |A′ ∩ [αk;n]| ∀k = 1, . . . , a;τ 2(A,A′)l+a = |A ∩ [α′a′+1−l]| ∀l = 1, . . . , a′;τ 0(A,A′)k = n− a− a′ + |[αk;n]− (A ∪ A′)| ∀k = 1, . . . , a;τ 0(A,A′)l+a = |[α′a′+1−l]− (A ∪ A′)| ∀l = 1, . . . , a′.

Proof. Write I = A′ ∪ A = i1 < · · · < irwith r = a+ a′. By definition,

τ 2(A,A′)k := τ(I2)k = |I ∩ [ia+a′+1−k]|, for 1 ≤ k ≤ a+ a′.

If k ≤ a, ia+a′+1−k ∈ A ⊂ [n + 1; 2n], ia+a′+1−k = αk and I ∩ [ia+a′+1−k] = A ∪ A′ ∩ [αk;n].The first assertion follows.

If k = a + l for some positive l, ia+a′+1−k ∈ A′ ⊆ [n], ia+a′+1−k = α′a′+1−l and I ∩[ia+a′+1−k] = A ∩ [α′a′+1−l].

Similarly,

τ 0(A,A′)k := τ(I0)k = |[ia+a′+1−k] ∩ ([2n]− (I ∪ I)|, for 1 ≤ k ≤ a+ a′.

If k ≤ a, [ia+a′+1−k]∩ ([2n]− (I ∪ I)) = ([n]− (A∪A′))∪ [αk;n]− (A ∪ A′) (a disjoint union).This proves the third claim.

If k = a+ l with some positive l, α′a′+1−l = ia+a′+1−k ∈ [n]; last assertion follows.

3.3. The parabolic subgroup P0. Fix m ≥ n. Let P0 be the subgroup of Sp(2m,C) ofmatrices

(26)

T1 ∗ ∗0 A ∗0 0 T2

,

where T1 and T2 are n× n upper triangular matrices and A is a matrix in Sp(2m− 2n,C).P0 is the standard parabolic subgroup of Sp(2m,C) corresponding to the simple rootsαn+1, . . . , αm. A character λ =

∑mi=1 λiεi ∈ X(T ) extends to P0 if and only if λn+1 =

· · · = λm = 0. Thus the set of dominant characters of X(P0) identifies with Parn. Hence

(27) sp -sat(m) ∩ (ParQn )3 = GIT-sat(Sp(2m,C), (Sp(2m,C)/P0)3).

Let SchubP0(Grω(r, 2m)) be the set of I ∈ Schub(Grω(r, 2m)) such that the Schubert vari-ety BFI is P0-stable. One checks (details omitted) that

(28) I ∈ SchubP0(Grω(r, 2m)) ⇐⇒ I ∩ [n+ 1; 2m− n] = [k; 2m− n] for some k ≥ m+ 1.

4. PROOF OF THEOREMS 1.1 AND 1.2

Proposition 4.1. The inequalities (4) in Theorem 1.2 characterize sp -sat(n).

Proof. Since sp -sat(n) = GIT-sat(Sp(2n), (Sp(2n)/B)3 (see Section 2.3), we may apply The-orem 2.3. Let (λ, µ, ν) ∈ (Parn)3. Write λ =

∑i λiεi and similarly for µ and ν.

10

Fix 1 ≤ r ≤ n and α = αr ∈ ∆. Given I ∈ Schub(Grω(r, 2n)), from the description of$α∨r and wI it is easy to check that

(29) 〈wI$α∨r , λ〉 =∑i∈I∩[n]

λi −∑i∈I∩[n]

λi

Then (4) is obtained from (18) associated to the triple of Schubert classes (I, J,K) ∈Schub(Grω(r, 2n))3 by setting

A = I ∩ [n], A′ = I ∩ [n];

B = J ∩ [n], B′ = J ∩ [n];

C = K ∩ [n], C ′ = K ∩ [n].

Since the map (22) is bijective, it suffices to show (17) from Theorem 2.3 is equivalent to

(1) |A′|+ |B′|+ |C ′| = r, and(2) cτ

0(C,C′)

τ0(A,A′)∨[(2n−2r)r ],τ0(B,B′)∨[(2n−2r)r ] = cτ2(C,C′)

τ2(A,A′)∨[rr ],τ2(B,B′)∨[rr ]= 1.

By [25, Theorem 19], condition (17) is equivalent to

(1) codim(BFI) + codim(BFJ) + codim(BFK) = dim(Grω(r, 2n)), and(2) cτ(K0)

τ(I0)∨[(2n−2r)r ],τ(J0)∨[(2n−2r)r ] = cτ(K2)

τ(I2)∨[rr ],τ(J2)∨[rr ]= 1.

By definition, the two conditions involving Littlewood-Richardson are the same. Assum-ing these two Littlewood-Richardson coefficients equal to one, it remains to prove thatcodim(BFI)+codim(BFJ)+codim(BFK) = dim(Grω(r, 2n)) if and only if |A′|+|B′|+|C ′| =r. This directly follows from (23) and (24).

Proof of Theorem 1.1: By Theorem 2.4, the inclusion sp -sat(n) ⊂ sp -sat(m) is equivalentto the inclusion Γ(Sp(2n,C) ∩ U(2n,C)) ⊂ Γ(Sp(2m,C) ∩ U(2m,C)). Here we use thesymplectic form defined in Section 3.1 to embed Sp(2n,C) in GL(2n,C).

Clearly, the following map is well-defined

Lie(Sp(2n,C) ∩ U(2n,C)) −→ Lie(Sp(2m,C) ∩ U(2m,C))

M =

(A BC D

)7−→ M =

A 0 B0 0 0C 0 D

where A, B, C and D are square matrices of size n, and the matrices of these Lie algebrasare described by (21).

Let (h1, h2, h3) ∈ Γ(Sp(2n,C) ∩ U(2n,C)). Let

(M1,M2,M3) ∈ (Sp(2n,C) ∩ U(2n,C))3.(√−1h1,

√−1h2,

√−1h3)

such that M1 +M2 +M3 = 0.The fact that M1 + M2 + M3 = 0 implies that (h1, h2, h3) ∈ Γ(Sp(2m,C) ∩ U(2m,C)),

where h1, h2, h3 are viewed as elements of Parm is the spectrum of M1.To obtain the converse inclusion

GIT-sat(Sp(2m,C), (Sp(2m,C)/P0)3) = sp -sat(m) ∩ (ParQn )3 ⊂ sp -sat(n),

we have to prove that any inequality (4) from Proposition 4.1 is satisfied by the points ofsp -sat(m) ∩ (ParQn )3; here we have used (27). Fix such an inequality (A,A′, B,B′, C, C ′).

11

Set I = A′ ∪ A ⊂ [2n], J = B′ ∪ B ⊂ [2n] and K = C ′ ∪ C ⊂ [2n]. Similarly for m, setI = A′∪2m+1−i : i ∈ A, J = B′∪2m+1−i : i ∈ B and K = C ′∪2m+1−i : i ∈ C;these are subsets of [2m]. Set also a′ = |A′|, b′ = |B′| and c = |C|.

Notice that (I2)0, (J2)0, (K2)0 ⊆ 0r = ∅ . Thus, trivially,

(30) cτ((K2)0)

τ((I2)0)∨,τ((J2)0)∨= c∅∅,∅ = 1.

Also, (I2)2 = I2, (J2)2 = J2, (K2)2 = K2. Since τ(I2) = τ(I2) := τ 2(A,A′), τ(J2) =τ(J2) := τ 2(B,B′) and τ(K2) = τ(K2) := τ 2(C,C ′), we have

(31) cτ((K2)2)

τ((I2)2)∨[rr ],τ((J2)2)∨[rr ]= c

τ(K2)

τ(I2)∨[rr ],τ(J2)∨[rr ]= c

τ2(C,C′)

τ2(A,A′)∨[rr ] τ2(B,B′)∨[rr ]= 1.

We apply [25, Theorem 8.2] to Grω(r, 2r) and the triple I2, J2, K2. The equations (30) and(31) mean that condition (iii) of said theorem holds. Hence by part (ii) of ibid.,

(32) [BFI2 ] · [BFJ2 ] · [BFK2 ] = [pt] ∈ H∗(Grω(r, 2r),Z).

One can easily check that

τ(K0) = [2(m− n)]c + τ(K0),

τ(I0)∨[(2m−2r)r] = [2(m− n)]a′+ τ(I0)∨[(2n−2r)r],

τ(J0)∨[(2m−2r)r] = [2(m− n)]b′+ τ(J0)∨[(2n−2r)r].

The assumption a′ + b′ = c and the semigroup property of LR-semigroup implies that

(33) cτ(K0)

τ(I0)∨[(2m−2r)r ],τ(J0)∨[(2m−2r)r ] 6= 0.

Next we apply [25, Proposition 8.1] to the I , J , K and the space Grω(r, 2m); equations (32)and (33) mean that condition (iii) holds. Hence by (i) of ibid. and (28),

(34) [P0FI ]0 [P0FJ ]0 [P0FK ] = d[pt] ∈ H∗(Grω(r, 2m),Z),

for some nonzero d. Now use Proposition 2.1, which shows that (4) is a case of (15) whichholds on GIT-sat(Sp(2m,C), (Sp(2m,C)/P0)3) = sp -sat(m) ∩ (ParQn )3, as desired.

Proof of Theorem 1.2: This follows from Theorem 1.1 and Proposition 4.1.

Example 4.2. Let n = 4, r = 3. Let

A = B′ = C ′ = ∅, A′ = 2, 3, 4, B = 1, 2, 4, C = 1, 3, 4,giving a triple ((A,A′), (B,B′), (C,C ′)) in (Schub′(Grω(3, 8)))3 satisfying conditions (1) and(2) from Theorem 1.2. The corresponding triple in Schub(Grω(3, 8)) is

I = 2, 3, 4, J = 5, 7, 8, K = 5, 6, 8 ⊆ [8].

Thusτ(I) = (1, 1, 1), τ(J) = (5, 5, 4), τ(K) = (5, 4, 4) ⊆ (53);

The three associated characteristic functions χI⊂[2n]−I , χJ⊂[2n]−J , χK⊂[2n]−K respectively are

21110002, 00201211, 02001121.

Thus the characteristic functions areχI2 = 111000, χJ2 = 000111, χK2 = 000111;χI0 = 01110, χJ0 = 01011, χK0 = 01101.

12

Now,I2 = 1, 2, 3 τ(I2) = 000 J2 = K2 = 4, 5, 6 τ(J2) = τ(K2) = 333I0 = 2, 3, 4 τ(I0) = 111J0 = 2, 4, 5 τ(J0) = 221 K0 = 2, 3, 5 τ(K0) = 211

The reader can check that

cτ0(C,C′)

τ0(A,A′)∨[(2n−2r)r ],τ0(B,B′)∨[(2n−2r)r ] = cτ(K0)

τ(I0)∨[(2n−2r)r ],τ(J0)∨[(2n−2r)r ] = c(2,1,1)(1,1,1),(1) = 1,

cτ2(C,C′)

τ2(A,A′)∨[rr ],τ2(B,B′)∨[rr ]= c

τ(K2)

τ(I2)∨[rr ],τ(J2)∨[rr ]= c

(3,3,3)(3,3,3),(0,0,0) = 1.

Hence, by Theorem 1.2, −λ2 − λ3 − λ4 + µ1 + µ2 + µ4 + ν1 + ν3 + ν4 ≥ 0 is one of theinequalities defining sp -sat(4).

5. THE TRUNCATED TENSOR CONE

In this section, we characterize the truncated tensor cone of sp -sat(m), that is, sp -sat(m)∩(ParQn )3 where m > n. By (3), this implies another set of inequalities for NL-sat(n).

We first need the following result, a generalization of Theorem 2.3:

Theorem 5.1. Here X = G/P × G/Q × G/R. Let (λ, µ, ν) ∈ X(P ) × X(Q) × X(R) bedominant. Then L(λ,µ,ν) ∈ GIT-sat(G,X) if and only if for any simple root α, for any

(u, v, w) ∈ WP\W/WPα ×WQ\W/WPα ×WR\W/WPα

such that

(35) [PuPα/Pα]0 [QvPα/Pα]0 [RwPα/Pα] = [pt] ∈ H∗(G/Pα,Z),

(36) 〈u$α∨ , λ〉+ 〈v$α∨ , µ〉+ 〈w$α∨ , ν〉 ≤ 0.

Proof. GIT-sat(G,X) is characterized by Proposition 2.2; let (C, τ) and a choice of

(u′, v′, w′) ∈ WP\W/WP (τ) ×WQ\W/WP (τ) ×WR\W/WP (τ)

be as in that proposition. Since every inequality (36) appears in Proposition 2.2 withτ = $α∨ , it suffices to show that (16) is implied by the inequalities in Theorem 5.1.

Writeτ =

∑α∈∆

nα$α∨ ,

where ∆ is the set of simple roots. Since τ is dominant the nα’s are nonnegative. Set

Supp(τ) := α ∈ ∆ : nα 6= 0.

Fix any α ∈ Supp(τ), Pα := P ($α∨) contains P (τ). Let

π : G/P (τ) −→ G/Pα

denote the associated projection. By [27, Theorem 1.1 and Section 1.1] (see also [24]),condition (13) implies there are

(u, v, w) ∈ WP\W/WPα ×WQ\W/WPα ×WR\W/WPα ,

such that condition (35) holds and such that (u, v, w) and (u′, v′, w′) define the same cosetsin WP\W/WPα ×WQ\W/WPα ×WR\W/WPα . Therefore, inequality (36) is the same as

〈u′$α∨ , λ〉+ 〈v′$α∨ , µ〉+ 〈w′$α∨ , ν〉 ≤ 0.

13

Therefore each of the inequalities (16) can be written as a linear combination of (36).Hence the inequalities of the theorem imply and are implied by the inequalities of Propo-sition 2.2, so the result follows.

We now deduce from Theorem 5.1 the following statement.

Proposition 5.2. Let (λ, µ, ν) in Parn and m ≥ n. Then (λ, µ, ν) ∈ sp -sat(m) if and only if

(37) |λI∩[n]| − |λI∩[n]|+ |µJ∩[n]| − |µJ∩[n]|+ |νK∩[n]| − |νK∩[n]| ≤ 0,

for any 1 ≤ r ≤ m and (I, J,K) ∈ SchubP0(Grω(r, 2m))3 such that

(1) |I ∩ [m]|+ |J ∩ [m]|+ |K ∩ [m]| = r, and(2) cτ(K0)

τ(I0)∨[(2m−2r)r ],τ(J0)∨[(2m−2r)r ] = cτ(K2)

τ(I2)∨[rr ],τ(J2)∨[rr ]= 1.

Proof. We already observed (29) that inequality (37) is inequality (16) in our context. Re-garding Theorem 5.1, the only thing to prove is that condition (13) associated to (I, J,K)is equivalent to the two conditions of the proposition. This is [25, Theorem 8.2].

A priori, Proposition 5.2 could contain redundant inequalities. In view of Theorem 1.2,an affirmative answer to this question would imply irredundancy:

Question 1. Does any (I, J,K) ∈ SchubP0(Grω(r, 2m))3 occurring in Proposition 5.2 satisfy

(1) I ∩ [n+ 1, 2m− n] = J ∩ [n+ 1, 2m− n] = K ∩ [n+ 1, 2m− n] = ∅;(2) cτ(K0)

τ(I0)∨[(2n−2r)r ],τ(J0)∨[(2n−2r)r ]= 1,

where I = I ∩ [n] ∪ i− 2(m− n) : i ∈ I ∩ [m+ 1, 2m], and, J and K are defined similarly.

Proof of Theorem 1.3. Fix an inequality (A,A′, B,B′, C, C ′) from (4). It is minimal for thefull-dimensional cone NL-sat(n) = sp -sat(2n) ∩ (Parn)3 ⊂ R3n. Thus, it has to appearin Proposition 5.2 for m = 2n. Let (I , J , K) ∈ SchubP0(Grω(r, 4n))3 be the associatedSchubert triple. Set A′ = I ∩ [2n], A = I ∩ [2n], etc. Since (I , J , K) ∈ SchubP0(Grω(r, 4n))3,A′, B′, C ′ ⊂ [n] (by (28)). Thus, comparing (4) and (37), we have

A = A ∩ [n] B = B ∩ [n] C = C ∩ [n]

A′ = A′ ∩ [n] = A′ B′ = B′ ∩ [n] = B′ C ′ = C ′ ∩ [n] = C ′.

Now, Proposition 5.2(1) and Theorem 1.2(2) imply that r = r. In particular, |A|+ |A′| =|A|+ |A′| = r and A = A. Similarly, B = B and C = C.

Let α be the simple root of Sp(4n,C) associated to r. Observe that the Levi subgroupof Pα has type Ar−1 × C2n−r. Let u, v, w ∈ W Pα corresponding to (A′, A), (B′, B) and(C ′, C), respectively. Proposition 5.2 and its proof show that (35) holds with P = Q =R = P0. In particular, one can apply the reduction rule proved in [28, Theorem 3.1] or [26,Theorem 1]: mult2n

λ,µ,ν is a tensor multiplicity for the Levi subgroup of Pα of type Ar−1 ×C2n−r. The factor c

ν∗C,C′

λA,A′ ,µB,B′in the theorem corresponds to the factor of type Ar−1. Adding

zeros, consider λ as an element of Par2n. Then the dominant weights to consider for thefactor C2n−r are λ[2n]−(A∪A′), µ[2n]−(B∪B′), ν[2n]−(C∪C′). Since these partitions have length atmost n−r, the tensor multiplicity for the factor ofC2n−r is a Newell-Littlewood coefficient.The theorem follows.

14

6. APPLICATION TO CONJECTURE 1.4

Corollary 6.1 (of Theorem 1.2). Conjecture 1.4 holds for n ≤ 5.

The proof is computational and uses the software Normaliz [4].Fix n ≥ 2 and consider the cone sp -sat(n). Consider the two lattices Λ = Z3n and

Λ2 = (λ, µ, ν) ∈ (Zn)3 : |λ|+ |µ|+ |ν| is even.Then NL-semigroup(n) ⊂ Λ2∩sp -sat(n). Conjecture 1.4 asserts that the converse inclusionholds. The set Λ2 ∩ sp -sat(n) is a semigroup of Λ2 defined by a family of linear inequal-ities (explicitly given by Theorem 1.2). Using Normaliz [4] one can compute (for small n)the minimal set of generators, i.e., the Hilbert basis, for this semigroup. Hence, to proveCorollary 6.1 one can proceed as follows:

(1) Compute the list of inequalities given by Theorem 1.2.(2) Compute the Hilbert basis of Λ2 ∩ sp -sat(n) using Normaliz.(3) Check Nλ,µ,ν > 0 for any (λ, µ, ν) in the Hilbert basis.

The table below summarizes our computations; see [8].

n # facets # EHI # rays # Hilb Λ2 ∩ sp -sat # Hilb Λ ∩ sp -sat

2 6+18 18 12 13 203 9+93 100 51 58 934 12+474 662 237 302 4515 15+2 421 5 731 1 122 1 598 2 171

In the column “# facets” there are the number of partition inequalities (like λ1 ≥ λ2)plus the number of inequalities (4) given by Theorem 1.2. The next column counts theinequalities (9) given by applying Theorem 1.5. The number of extremal rays of the conesp -sat(n) is also given. The two last column are the cardinalities of the Hilbert bases ofthe two semigroups Λ2 ∩ sp -sat(n) and Λ ∩ sp -sat(n).

7. PRELIMINARIES ON LITTLEWOOD-RICHARDSON COEFFICIENTS

We recall standard material [6] on Littlewood-Richardson coefficients.

7.1. Representations of GL(n,C). The irreducible representations V (λ) of GL(n,C) areindexed by their highest weight

λ ∈ Λ+n = (λ1 ≥ · · · ≥ λn) : λi ∈ Z ⊃ Parn.

In this setting, the Littlewood-Richardson coefficients appear because

V (λ)⊗ V (ν) =⊕ν

V (ν)⊕cνλ,µ .

The dual representation V (λ)∗ has highest weight λ∗ = (−λn ≥ · · · ≥ −λ1) ∈ Λ+n .

Moreover, for any a ∈ Z,V (λ+ an) = (det)a ⊗ V (λ).

Consequently, for any λ, µ, ν ∈ Λ+n ,

(38) cνλ,µ = cν+(a+b)n

λ+an,µ+bn = cν∗+(a+b)n

λ∗+an,µ∗+bn ;

15

this is [3, Theorem 4]. Let ν ′ be the transpose of ν. Since cνλ,µ = cν′

λ′,µ′ , by (38),

(39) cνλ,µ = cν′

λ′,µ′ = c(ν′)∨[(n+m)a+b]

(λ′)∨[na+b],(µ′)∨[ma+b]= c

(ν∨[(a+b)n+m])′

(λ∨[(a+b)n])′,(µ∨[(a+b)m])′= cν

∨[(a+b)n+m]

λ∨[(a+b)n],µ∨[(a+b)m] ,

for any m ≥ `(µ).

7.2. Littlewood-Richardson tableaux. A tableau filling T of shape ν/λ by N is semistan-dard if it is weakly increasing along rows and strictly increasing along columns; T is fur-thermore standard if the filling bijects the boxes of ν/λwith 1, 2, . . . , |ν/λ|. The content ofT is cont(T ) = (i1, i2, . . .) where ik is the number of k’s that appear in T . Let SSYT(ν/λ) bethe set of semistandard tableaux of shape ν/λ. Let SSYT(ν/λ, µ) be the subset consistingof those tableaux of content µ. Finally, SYT(ν/λ) is the subset of standard fillings.

Define rowword(T ) = (w1, w2, . . . , w|ν/λ|) to be the reading word obtained by readingright to left along rows and from top to bottom. Define revrowword(T ) to be the reverseof rowword(T ). Either rowword(T ) or revrowword(T ) are ballot if, for every i ≥ 2, k ≥ 1, thenumber of i − 1 that appear among the initial subword w1, w2, . . . , wk is weakly greaterthan the number of i that appear in that subword. We say T is ballot if rowword(T ) is ballot.

We will make use of a number of (re)formulations of the Littlewood-Richardson rule:

Theorem 7.1 (Littlewood-Richardson rule, version 1).

cνλ,µ = #T ∈ SSYT(ν/λ, µ) : T is ballot.

If λ and µ are straight-shape, let λ ? µ be the skew shape obtained by placing λ and µcorner to corner with the former southwest relative to the latter.

Theorem 7.2 (Littlewood-Richardson rule, version 2).

cνλ,µ = #T ∈ SSYT(λ ? µ, ν) : T is ballot.

We review basics of the theory of jeu de taquin. An inner corner x of ν/λ is a maximallysoutheast box of λ. For T ∈ SSYT(ν/λ), a jeu de taquin slide jdtx(T ) is obtained as follows.Place • in x, and apply one of the following slides, depending on how T looks near x:

(J1) • ab7→ b a• (if b ≤ a, or a does not exist)

(J2) • ab7→ a •

b(if a < b, or b does not exist)

Repeat use of (J1) or (J2) on the new box x′ where • lands. Terminate when • arrives at abox y of λ that has no labels south or east of it. Then jdtx(T ) is the result after erasing •.

A rectification of T ∈ SSYT(ν/λ) is defined iteratively. Pick an inner corner x0 of ν/λand determine T1 := jdtx0(T ) ∈ SSYT(ν(1)/λ(1)). Let x1 be an inner corner of ν(1)/λ(1) andcompute T2 := jdtx1(T1) ∈ SSYT(ν(2)/λ(2)). Repeat |λ| times, arriving at Rectxi(T ).

Theorem 7.3 (First fundamental theorem of jeu de taquin). For T ∈ SSYT(ν/λ), Rectxi(T )is independent on the choice of inner corners xi.

Thus we can unambiguously refer to the rectification Rect(T ).Knuth equivalence on words is the equivalence relation ≡K generated by the relations:

(. . . , b, c, a, . . .) ≡K (. . . , b, a, c, . . .) if a < b ≤ c,

16

(. . . , a, c, b, . . .) ≡K (. . . , c, a, b, . . .) if a ≤ b < c.

The following fact is well-known, and in any case, easy to see from the local relations:

Theorem 7.4. Suppose w, u are two words such that w ≡K u. Then w is reverse-ballot if andonly if u is reverse-ballot.

Jeu de taquin preserves Knuth word and reverse-ballotness:

Theorem 7.5. Let T ∈ SSYT(ν/λ) and T ′ = jdtx(T ).

(I) revrowword(T ) ≡K revrowword(T ′)(II) T is ballot if and only if T ′ is ballot.

Let Yµ be the super-semistandard tableau of shape µ, the tableau using only i’s in row i.

Theorem 7.6 (Littlewood-Richardson rule, version 3). Fix U ∈ SSYT(µ). Then

(40) cνλ,µ = T ∈ SSYT(ν/λ) : Rect(T ) = U.Moreover, if U = Yµ, then T is in the set given in (40) if and only if T is ballot.

The final claim follows from Theorem 7.5(II) and the fact that Yµ is ballot.

7.3. RSK correspondence. A biword of length m is

a =

(qp

)=

(q1 q2 . . . qmp1 p2 . . . pm

),

such that q1 ≤ q2 ≤ · · · ≤ qm and if qi = qj with i ≤ j then pi ≤ pj .The Robinson-Schensted-Knuth (RSK) correspondence bijectively maps biwords a to pairs

of semistandard tableaux (P (a), Q(a)) of the same shape. We refer to ibid. for details ofthis correspondence. Here, we just state that we use row insertion, so that the contentsof the insertion tableau P (a) is p1, . . . , pm, and the contents of the recording tableau Q(a)is q1, . . . , qm. Whereas RSK : a 7→ (P (a), Q(a)), RSK−1 : (P (a), Q(a)) 7→ a is defined byreverse row insertion.

If p = (p1, . . . , pm) is a word, RSK2(p) = Q(a) where a is the biword whose bottom rowis p and top row is 1 through m. RSK1(p) = P (a).

Proposition 7.7.

(I) revrowword(P (a)) ≡K p.(II) if a ≡K a′ then P (a) = P (a′).

(III) In each Knuth equivalence class C, there is a unique straight-shape semistandard tableauT such that revrowword(T ) ∈ C.

(IV) Suppose a =(qp

)and a′ =

(q′

p′

)are two biwords. Assume min(q′) > max(q). Then

Q(aa′)>max(q) is a skew semistandard tableaux such that Rect(Q(aa′)>max(q)) = Q(q′).

Proof. (I), (II) and (III) are well-known. For (IV), see [6, Section 5.1, Proposition 1].

We recall a construction in [6, Chapter 5]. Fix V0 ∈ SSYT(ν) and U0 ∈ SSYT(µ). Define

T (λ, µ, V0) = T ? U ∈ SSYT(λ ? µ) : Rect(T ? U) = V0and

S(ν/λ, U0) = S ∈ SSYT(ν/λ) : Rect(S) = U0.

17

Fix T0 ∈ SSYT(λ) where we use entries in Z<0. Define

ξ : S(ν/λ, U0)→ T (λ, µ, V0)

as follows. For any S ∈ S(ν/λ, U0), define T0 ∪ S to be the tableau of shape ν obtained byplacing T0 in λ and S in ν/λ. Consider the biword

(41) RSK−1(V0, T0 ∪ S) =

(t1 . . . tn v1 . . . vmx1 . . . xn w1 . . . wm

),

where n = |λ|,m = |µ|. There exist T ∈ SSYT(λ) and U ∈ SSYT(µ) such that

(42) (T, T0) = RSK

((t1 . . . tnx1 . . . xn

)), (U,U0) = RSK

((v1 . . . vmw1 . . . wm

)).

Let ξ(S) := T ? U .

Proposition 7.8 ([6, Chapter 5]). The map ξ : S(ν/λ, U0) → T (λ, µ, V0) is a bijection for anyU0 ∈ SSYT(µ) and V0 ∈ SSYT(ν). The cardinality of either set is cνλ,µ.

7.4. Some consequences. Given ν ∈ Parn, a ≥ ν1 and ` ∈ Z>0, we set

a` ∪ ν = (a, . . . , a︸ ︷︷ ︸`

, ν1, . . . , νn).

Lemma 7.9. Let λ, µ, ν ∈ Parn. Let a, b, k ∈ Z≥0 such that a + b ≥ ν1 − k, a ≥ µ1 − k andb ≥ λ1 − k. Then

c(a+b+k)n∪νλ+an,µ+bn = cν+kn

λ,µ

Proof. By (38),

(43) cν+kn

λ,µ = c(ν+kn)∗+(a+b+2k)n

λ∗+(b+k)n,µ∗+(a+k)n = c(ν+kn)∨[(a+b+2k)n]

λ∨[(b+k)n],µ∨[(a+k)n] = cν∨[(a+b+k)n]

λ∨[(b+k)n],µ∨[(a+k)n] .

Combining with (39), we obtain

cν+kn

λ,µ = cν∨[(a+b+k)n]

λ∨[(b+k)n]),µ∨[(a+k)n] = c(ν∨[(a+b+k)

n])∨[(a+b+k)2n]

(λ∨[(b+k)n]))∨[(a+b+k)n],(µ∨[(a+k)n])∨[(a+b+k)n] = c(a+b+k)n∪νλ+an,µ+bn .

Lemma 7.10. If T ∈ SSYT(ν/λ) and Rectxi(T ) = S ∈ SSYT(µ) (for some choice of innercorners xi) then cνλ,µ > 0.

Proof. Immediate from Theorem 7.6.

For λ = (λ1, λ2, . . . , λm) ∈ Parm and any k ∈ [m], write λ≤k = (λ1, λ2, . . . , λk), andλ>k = (λk+1, λk+2, . . . , λm). If T is a tableau of shape λ, let T≤k (respectively, T>k) be thesubtableau of T of shape λ≤k (respectively, λ>k).

Proposition 7.11. Let λ, µ, ν ∈ Parn such that cνλ,µ > 0. For any k ∈ [n − 1], there existpartitions α, β such that

cν≤kλ≤k,α

> 0, cν>kλ>k,β

> 0 and cµα,β > 0.

Proof. If ν≤k/λ≤k = ν/λ, we are done by using α = µ and β = ∅. We can then assumethat the skew partition ν/λ is non-empty below row k. By Theorem 7.1, there exists aballot tableau T ∈ SSYT(ν/λ, µ). Since T is ballot and semistandard, clearly T≤k is ballotand semistandard. As a result, the content α of T≤k is a partition. Therefore, cν≤kλ≤k,α

> 0.

18

Since T is semistandard, T>k is also semistandard. Set T ′ ∈ SSYT(β) to be the jeu de taquinrectification of T>k ∈ SSYT(ν>k/λ>k). Hence, by Lemma 7.10, cν>kλ>k,β

> 0.

Let T be the filling of the skew shape β ? α where we use a super-semistandard fillingYα in the α part and place T ′ ∈ SSYT(β) in the β part; clearly T ∈ SSYT(β ? α).

By Theorem 7.5(I),

revrowword(Yα) ≡K revrowword(T≤k)

revrowword(T ′) ≡K revrowword(T>k).

Hence

revrowword(T ) = revrowword(T ′) · revrowword(Yα)

≡K revrowword(T>k) · revrowword(T≤k)

= revrowword(T ).

Since revrowword(T ) is reverse-ballot, by Theorem 7.4, revrowword(T ) is reverse-ballot. ThusT is ballot. Hence by Theorem 7.2, T ∈ SSYT(β ? α, µ) witnesses cµβ,α = cµα,β > 0.

Lemma 7.12. [11, Theorem 3.1] Let λ, µ, ν be partitions with m,n, k ∈ Z≥0 and m ≥ n. Then

cνλ,µ ≤ cν+(km)

λ+(kn),µ+(km−n).

8. DEMOTION

8.1. Definition. We relate the demotion algorithm [9, Claim 3.5] to the Robinson-Schensted-Knuth (RSK) correspondence. This is used to prove Theorem 8.15 (needed in Theorem 9.1).

Fix a ballot tableau S ∈ SSYT(ν/λ, µ) and a corner box c0 of λ. Let λ↓ ⊆ λ be obtainedby removing c0. Demotion is defined as follows. Place a 1 in c0. Find the first 1 afterc0 (if it exists) in the column reading order. If it does not exist, terminate the process.Otherwise let c1 be the box containing this 1 and turn that into a 2. Similarly, find the first2 (if it exists, say in c2) in the column reading word order after c1 and change that to a 3.Terminate and output S↑ when, after replacing the k − 1 in ck−1 with k, there is no later kin the column reading order.

Proposition 8.1 ([9, Claim 3.5]). S↑ ∈ SSYT(ν/λ↓, µ↑) for some µ ⊂ µ↑. S↑ is ballot.

For a corner c of λ, let demote(c) ∈ N be the row index of µ↑/µ resulting from usingdemotion starting with c = λ/λ↓.

8.2. Relationship between demotion and RSK. We fixed a ballot S ∈ SSYT(ν/λ, µ). Alsofix T0 ∈ SSYT(λ) where we use entries in Z<0 as in Section 7.3.

Theorem 8.2 (Demotion-RSK relationship). Let c0 be the box in λ that is occupied by the largestentry in T0 (or the rightmost of such entries, if there are repeats), then demote(c0) = xn wheredemotion is applied to (a ballot tableau) S, and xn is defined in (41) where V0 = Yν .

If U ∈ SSYT(γ/δ), let shape(U) = γ/δ. Let α ⊆ γ be two partitions and let S1, S2 ∈SSYT(γ/α). Define S1 and S2 to L-R correspond by V0 if

shape(Rect(S1)) = shape(Rect(S2)) = β

19

for some partition β andξ1(S1) = ξ2(S2)

where ξ1 and ξ2 are ξ defined by U0 = Rect(S1) and U0 = Rect(S2) respectively:

S(γ/α,Rect(S1))ξ1−→ T (α, β, V0)

ξ2←− S(γ/α,Rect(S2)).

(There is another “T0” used in the definition of ξ1 and ξ2 that we fix, but will play no rolein our argument.)

Define S1 and S2 to be Q-equivalent if

(44) RSK2(revrowword(S1)) = RSK2(revrowword(S2)).

Proposition 8.3 ([6, Chapter A.3]). S1 and S2 L-R correspond by some V0 ∈ SSYT(β) if andonly if S1 and S2 are Q-equivalent.

Lemma 8.4 (Pieri property of RSK). Let W = (w1, . . . , wn) be a word, and x, y ∈ N. Letcol(x), col(y) be the column indices of the added boxes when inserting x, y during computation ofRSK1(Wxy). Then col(x) < col(y) if and only if x ≤ y.

Proof. (⇒) We prove the contrapositive. Suppose x > y. Let

κ =

(−n −n+ 1 . . . −1 1 2w1 w2 . . . wn x y

).

Let S = Q(κ)>0. By Proposition 7.7(IV),

Rect(S) = Q

((1 2x y

))= 1

2.

Therefore, by Theorem 7.5, S is a ballot; this means col(x) ≥ col(y).(⇐) If x ≤ y, then by the same reasoning as in the “⇒” argument:

Rect(S) = Q

((1 2x y

))= 1 2 ,

which means S is not a ballot, and thus col(x) < col(y).

Define

(45) ω := RSK−1(Yν , T0 ∪ S) =

(t1 . . . tn v1 . . . vmx1 . . . xn w1 . . . wm

).

Lemma 8.5. The P -tableau of any final segment of ω is super-semistandard.

Proof. By Proposition 7.7 (I),

x1 . . . xnw1 . . . wm ≡K revrowword(Yν).

Hence, by Theorem 7.4, x1 . . . xnw1 . . . wm is a reverse-ballot (that is wmwm−1 . . . x1 is bal-lot). As a result, any final segment f of it is also reverse-ballot. By Proposition 7.7 (I),revrowword(RSK1(f)) ≡K f ; moreover by Theorem 7.4, revrowword(RSK1(f)) is reverse-ballot. This immediately implies that RSK1(f) super-semistandard, as desired.

Lemma 8.6. Let Y = (y1, . . . , yp) and x, y, z ∈ Z such that x < y ≤ z, then

(46) RSK1(Y yxz) = RSK1(Y yzx).

Also, x is inserted into the same box in both the computation of RSK1(Y yxz) and RSK1(Y yzx).

20

Proof. Since x < y ≤ z, Y yxz≡KY yzx. Therefore by Proposition 7.7(II), (46) holds.For the remaining assertion, let

π1 =

(−p −(p− 1) . . . −1 1 2 2y1 y2 . . . yp y x z

)and π2 =

(−p −(p− 1) . . . −1 1 1 2y1 y2 . . . yp y z x

).

By Lemma 8.4, col(x) < col(z) in RSK1(Y yxz) = P (π1) = P (π2) = RSK1(Y yzx). Thus, it isenough to show that col(x) < col(z) in RSK(π2). Notice that

RSK

(1 1 2y z x

)=

(x zy

, 1 12

).

Thus, by Proposition 7.7(IV) and Theorem 7.5(II), Q(π2)>0 is a ballot tableau with content(2, 1). By Lemma 8.4, col(z) > col(y) in P (π2). We then conclude, by ballotness, thatcol(x) ≤ col(z) in P (π2). Since x and z are inserted to the same set of two boxes in π1 andπ2, col(x) < col(z) in P (π2).

Lemma 8.7. Let Y = y1 . . . yp and suppose x ≤ y < z ∈ Z. Let col(x), col(y), col(z), respec-tively, be the column indices of the added box when inserting x, y, z during the computation ofRSK1(Y zxy). Similarly, let col′(x), col′(y), col′(z) be the indices when computing RSK1(Y xzy).

(1) If col(x) < col(z) < col(y) then col′(y) < col′(x) < col′(z).(2) If col(x) < col(y) < col(z) then col′(x) < col′(y) < col′(z).

Proof. Consider the biwords

η1 =

(−p −(p− 1) . . . −1 1 2 3y1 y2 . . . yp z x y

)and η2 =

(−p −(p− 1) . . . −1 1 2 3y1 y2 . . . yp x z y

).

Letα = shape(RSK1(Y )), β = shape(RSK1(zxy)), γ = shape(RSK1(Y zxy)).

Also, setS1 = Q(η1)>0, S2 = Q(η2)>0 ∈ SSYT(γ/α).

Since Y xzy≡KY zxy, P (η1) = P (η2) (by Proposition 7.7(II)); let Vγ be this tableau. SetT0 = Q(η1)<0 = Q(η2)<0 ∈ SSYT(α) and let

ξ1 : S(γ/α,Rect(S1)) −→ T (α, β, Vγ)

andξ2 : S(γ/α,Rect(S2)) −→ T (α, β, Vγ)

be the maps as defined in Proposition 7.8. Since

RSK−1(Vγ, Q(η1)<0 ∪ S1) = η1 and RSK−1(Vγ, Q(η2)<0 ∪ S2) = η2,

we obtain

ξ1(S1) = RSK1(Y ) ? RSK1(zxy) and ξ2(S2) = RSK1(Y ) ? RSK1(xzy).

Since xzy≡Kzxy, we have RSK1(xzy) = RSK1(zxy) and thus ξ1(S1) = ξ2(S2). Notice fur-ther that we have shape(Rect(S1)) = β = shape(Rect(S2)). Therefore S1 and S2 L-R corre-spond by Vγ . By Proposition 8.3, S1 and S2 are also Q-equivalent.

In (1), since col(x) < col(z) < col(y), we know that revrowword(S1) = 213, and thus

RSK2(revrowword(S1)) = 1 32

.

21

Since x, y < z, by Lemma 8.4, we have col′(x), col′(y) < col′(z). Hence, revrowword(S2) =132 if col′(x) < col′(y) and revrowword(S2) = 312 if col′(y) < col′(x). Now,

RSK2(132) = 1 23

and RSK2(312) = 1 32

.

Since S1 and S2 are Q-equivalent, (44) holds; this implies we are in case col′(y) < col′(x).Thus col′(y) < col′(x) < col′(z), as desired.

In (2), since col(x) < col(y) < col(z), revrowword(S1) = 231. Hence

RSK2(revrowword(S1)) = 1 23

.

Since x, y < z, by Lemma 8.4, col′(x), col′(y) < col′(z). By the same reasoning as in (1)

RSK2(132) = 1 23

,

we see that col′(x) < col′(y) < col′(z).

For 1 ≤ j ≤ `(µ), let ij be the largest index such that vij = j and declare i0 = 0.

Lemma 8.8. Each ij (for 1 ≤ j ≤ `(µ)) is finite, and

wij−1+1 . . . wij = 1, . . . , 1︸ ︷︷ ︸µj−µj+1

, 2, . . . , 2︸ ︷︷ ︸µj+1−µj+2

, . . . , `(µ)− j + 1, . . . , `(µ)− j + 1︸ ︷︷ ︸µ`(µ)

.

Proof. Let U0 = Rect(S). Since S is assumed to be ballot, it follows from Theorem 7.5(II)that U0 = Yµ for some µ. Comparing (45) and (41) we conclude from (42) that

(U, Yµ) = RSK

((v1 . . . vmw1 . . . wm

)).

The first claim follows, since the multiset of labels of v is the same as than in Yµ. Also, byLemma 8.5, we also haveU = Yµ, which implies the second claim, for the same reason.

Lemma 8.9. For any 1 ≤ j ≤ xn − 1, there exists a largest kj such that(vkjwkj

)=(

jxn−j

).

Proof. By Lemma 8.5,

P

((tn v1 . . . vmxn w1 . . . wm

))and P

((v1 . . . vmw1 . . . wm

))are super-semistandard. Thus, it follows that µxn−1 > µxn . By Lemma 8.8, inwij−1+1 . . . wij ,there are µxn−1 − µxn > 0 many xn − j’s. Let ij−1 + 1 ≤ kj ≤ ij be the index of any suchoccurrence. Since vij−1+1, vij−1+2, . . . , vij = j, we are done.

Let S↑ ∈ SSYT(ν/λ↓, µ↑) be the ballot tableau from demoting S at c0. Define

ωj =

(t1 . . . tn v1 . . . vijx1 . . . xn w1 . . . wij

), for j ∈ [xn − 1].

Define

ω′j =

(t1 . . . tn−1 v1 . . . vk1−1 1 vk1+1 . . . vk2−1 2 vk2+1 . . . vij j + 1x1 . . . xn−1 w1 . . . wk1−1 xn wk1+1 . . . wk2−1 wk1 wk2+1 . . . wij wkj

),

this is the biword obtained from ωj by replacing(vkiwki

)with

(i

wki−1

)for 0 < i ≤ j (where

wk0 = xn), removing(tnxn

)and appending

(j+1wkj

)to the right-end.

22

Lemma 8.10. ω′j is a biword.

Proof. Since vki+1 > i ≥ vki−1 for 0 < i ≤ j and vij = j < j + 1, the first row of ω′j isweakly increasing. For the second row, first recall that by definition i = vki ≥ vki−1. If theinequality is strict there is nothing to show, so we may assume i = vki = vki−1. Since ω isa biword, wki ≥ wki−1. Since xn = wk0 > wk1 > wk2 > · · · > wkj , we have

wki−1> wki ≥ wki−1 for all 0 < i ≤ j.

Therefore ω′j is a biword.

By definition,RSK(ωj) = (P (ωj), T0 ∪ S≤j).

For each such j, define (S≤j)↑ ∈ SSYT(ν/λ↓, µ↑≤j) to be the ballot tableau obtained byapplying demotion to S≤j at the tn entry in T0, and T ↓0 ∈ SYT(λ↓) the tableau obtainedfrom T0 by removing the tn entry.

Proposition 8.11. For each j ∈ [xn − 1],

RSK(ω′j) = (P (ωj), T↓0 ∪ (S≤j)↑).

Proof of Proposition 8.11: We proceed by induction on j. For the base case j = 1, set

ω1 =

(t1 t2 . . . tn−1 tn 1 1 . . . 1x1 x2 . . . xn−1 xn w1 w2 . . . wi1

).

Since ω is a biword, by definition, w1 ≤ w2 ≤ . . . ≤ wi1 . Now,

ω′1 =

(t1 t2 . . . tn−1 1 . . . 1 1 1 . . . 1 2x1 x2 . . . xn−1 w1 . . . wk1−1 xn wk1+1 . . . wi1 wk1

).

Claim 8.12. Suppose tn appears in box c0 of T0 ∪ S≤1 and all 1’s of T0 ∪ S≤1 occur weakly westof c0. Then xn > wi1 .

Proof. Let d1, d2, . . . , di1 be boxes of the 1’s in S≤1, as listed from right to left (they form ahorizontal strip). Let P (1) be the tableau obtained by reverse insertion of P (0) := P (ω1) atd1. P (2) is obtained by reverse inserting P (1) at d2. Likewise define P (3), P (4), . . . , P (i1). Bydefinition of RSK, the output of the reverse inserting P (k) at dk+1 iswi1−k, for 0 ≤ k ≤ i1−1.

The box c0 in λ ⊂ ν = shape(T0∪S≤1) = shape(P (ω1)) also appears in each of shape(P (k))for 0 ≤ k ≤ i1. Thus, define rk to be the result of reverse inserting P (k) at c0, for 1 ≤ k ≤ i1(r0 does not make sense if d1 is directly below c0).

Now, by Lemma 8.4,

(47) r1 > wi1 .

We claim:

(48) ri1 ≥ ri1−1 ≥ ri1−2 ≥ · · · ≥ r1

Combining this with (47) gives us xn = ri1 ≥ r1 > wi1 , as desired.

Hence it remains to prove (48). Let 1 = y1 = P (1)(d2) (i.e., the entry of P (1) at boxd2). When reverse inserting y1 into the previous row, suppose y2 is reverse bumped etc.Suppose that yu is reverse inserted into the row of c0 and yu+1 is reverse bumped. By theLemma’s hypothesis, c0 is the rightmost box in its row. Hence there are two cases:

23

Case 1: (yu+1 < P (1)(c0)) By definition of RSK reverse row insertion, yu+i < P (1)(c0) fori ≥ 2. If b is any box of P (2) in a row strictly above that of c0 then

(49) P (1)(b) ≤ P (2)(b)

and

(50) P (1)(b) 6= P (2)(b) =⇒ P (2)(b) < P (1)(c0).

By (49) and (50) combined, the boxes used in reverse inserting P (1) starting at c0 are thesame when reverse inserting P (2) at c0. Therefore r2 ≥ r1.Case 2: (yu+1 = P (c0)) Consider the RSK reverse insertion path P starting at d2 that outputswi1−1; these are the boxes of P (1) that have been changed. Our assumption implies Pincludes c0. Notice that when computing r2 (by reverse inserting P (2) at c0) we use a pathP’ that is weakly right (in each row) of the subpath of P starting at c0. Since each of theboxes in P boxes contain a larger label in P (2) than in P (1), and P (2) is semistandard, itagain follows that r2 ≥ r1.

Having established r2 ≥ r1, we consider the reverse insertion of P (2) at d3. The analysisabove applies, mutatis mutandis, to show r3 ≥ r2. Repeating this, we obtain the claim.

Claim 8.13. The base case j = 1 follows from the special case where k1 = i1.

Proof. For i ∈ [k1, i1], set

θ′i :=

(t1 . . . tn−1 1 . . . 1 1 1 . . . 1 2x1 . . . xn−1 w1 . . . wk1−1 xn wk1+1 . . . wi wk1

),

and let

S := Q

((t1 . . . tn 1 . . . 1x1 . . . xn w1 . . . wk1

))>0

.

Suppose the base case is correct when k1 = i1. This implies that the unique 2 in Q(θ′k1)is in the same box as the rightmost 1 of S that is weakly west of c0. By the contrapositivestatement of Claim 8.12, shape(Q(θ′k1)) contains all the boxes occupied by 1 in S that areweakly west of c0. Hence the unique 2 in Q(θ′k1) will be in the same box as the rightmost1 weakly left of the tn in T0 ∪ S≤1. Consider

θ′i :=

(t1 . . . tn−1 1 . . . 1 1 1 . . . 1 2 3x1 . . . xn−1 w1 . . . wk1−1 xn wk1+1 . . . wi wk1 wi+1

).

It is clear that the unique 2 in Q(θ′i) is in the same box as the unique 2 in Q(θ′i) for anyi ∈ [k1, i1− 1]. Since wk1 < wi ≤ wi+1, by Lemma 8.6 we know that the unique 2 in Q(θ′i) isalso in the same box as the unique 2 in Q(θ′i+1). As a result, the 2 in Q(θ′k1) is in the samebox as the 2 in Q(θ′i1) = T ↓0 ∪ (S≤1)↑ = Q(ω′1) which proves the base case.

Now by Claim 8.13, it is enough to prove the base case with the assumption that k1 = i1.Since the second rows of ω1 and ω′1 are Knuth equivalent, we get P (ω′1) = P (ω1).

Now we show Q(ω′1) = T ↓0 ∪ (S≤1)↑. If some 1 appears immediately below the tn inT0∪S≤1, then we are done since T ↓0 ∪ (S≤1)↑ is semistandard. We can thus assume there isat most one box in each column of T ↓0 ∪ (S≤1)↑ that is filled by 1 or 2. For i ∈ [k1], consider

θi =

(t1 . . . tn−1 1 . . . 1 1 2 . . . 2x1 . . . xn−1 w1 . . . wi−1 xn wi . . . wk1

).

24

Let coli(wk) and coli(xn) be the column index where wk and xn is inserted in the com-putation of Q(θi) respectively. Notice that colk1(wk) = coli(wk) for all i > k. Also sincex1 · · ·xnw1 · · ·wk−1 ≡K x1 · · ·xn−1w1 · · ·wi−1xnwi · · ·wk−1 it follows that

(51) col1(wk) = coli(wk) for all i < k.

By Lemma 8.7(2), if coli(wi) < coli(wi+1) < coli(xn), then

(52) coli(wi) = coli+1(wi) < coli(wi+1) = coli+1(wi+1) < coli(xn) = coli+1(xn).

Let ` be the index such that w` is inserted to the rightmost 1 that is weakly west of c0 inthe computation of Q(ω1) = T0 ∪ S≤1. By (52) and a simple induction argument,

col1(w`) = col`(w`) < col1(xn) = col`(xn) < col`(w`+1).

Hence, by Lemma 8.7(1), we have

(53) col1(w`) = col`+1(w`+1) < col`+1(w`) < col`+1(xn) = col`(w`+1).

By (51) and Lemma 8.4,

(54) col`+1(w`+2) = col1(w`+2) > col1(w`+1) = col`(w`+1).

Thus by (53) and (54),

col`+1(w`+1) < col`+1(xn) < col`+1(w`+2).

Hence, by Lemma 8.7(1) and induction, for any k ∈ [`, k1 − 1], we have

colk(wk) = col`(w`) < colk(xn) < colk(wk+1).

Thus by Lemma 8.7(1), we get

col1(w`) = colk+1(wk+1) < colk+1(wk) < colk+1(xn).

As a result, col1(w`) = colk1(wk1) and the unique 2 in Q(θk1) = T ↓0 ∪ (S≤1)↑ is in the samebox as the rightmost 1 in the columns weakly left of the tn in T0 ∪ S≤1 as we desired. Thiscompletes the proof of the base case j = 1 of Proposition 8.11.

We assumed nothing about the inner semistandard tableau T0 in the proof of the basecase. In addition, notice that the largest entry in T ↓0 ∪ (S<j)↑ is the final j entry that resultsfrom demotion of T0 ∪ S<j at c0. Therefore the same argument we used in the base caseholds for any j ∈ [xn− 1] by replacing T0 with T ↓0 ∪ (S<j)↑. As a result we have RSK(ω′j) =

(P (ωj), T↓0 ∪ (S≤j)↑) as we wanted. This completes the proof of Proposition 8.11.

Conclusion of the proof of Theorem 8.2: By Proposition 8.11, RSK(ω′xn−1) = (P (ωj), T↓0 ∪

(S<xn)↑) and so demote(c0) ≥ xn. The xn-entries in S occupy shape(P (ωxn)/P (ωxn−1)).

In addition, the leftmost entry in P (ωxn)/P (ωxn−1) is added by inserting(

xnwixn−1+1

)to

P (ωxn−1) by RSK. Since wkxn−1 = 1 ≤ wixn−1+1 and P (ω′xn−1) = P (ωxn−1), by Lemma 8.4,the unique xn entry in T ↓0 ∪ (S<xn)↑ is strictly left of all xn entries in S. By definition,demotion terminates and we have demote(c0) = xn.

Let ω′ be the biword obtained from ω′xn−1 by appending(vixn−1+1 vixn−1+2 . . . vmwixn−1+1 wixn−1+2 . . . wm

)25

at the end. Since vixn−1+1 = xn, the first row of ω′ is weakly increasing. Since wkxn−1 =1, wkxn−1 ≤ wixn−1+1 and thus ω′ is a biword. Let S↑ be the ballot tableau obtained byapplying demotion to S at the tn entry in T0.

Corollary 8.14. RSK(ω′) = (Yν , T↓0 ∪ S↑)

Proof. By Proposition 8.11, ω′xn−1 ≡K ωxn−1 and thus ω′ ≡K ω. Therefore P (ω′) = Yν . ByTheorem 8.2, demote(c0) = xn and thus S↑ = (S≤xn−1)↑∪S≥xn . Since P (ω′xn−1) = P (ωxn−1),

Q(ω′) = Q(ω′xn−1) ∪ S≥xn = T ↓0 ∪ (S≤xn−1)↑ ∪ S≥xn = T ↓0 ∪ S↑.

The important consequence of Corollary 8.14 for Section 9 is:

Theorem 8.15. Let λ, µ, ν be three partitions, a′ and ` two positive integers. Assume that `(λ) ≤` and there exists a ballot tableau T ∈ SSYT(ν/µ, λ) with all entries in rows below ` at most a′.

Then, there exists a partition ρ such that

cνρ,λ≤a′ > 0, cρµ,λ>a′ > 0, µ>`−a′ = ρ>`−a′ .

8.3. Proof of Theorem 8.15. We first recall tableau infusion (also known as tableau switch-ing) following [2]. For T ∈ SSYT(ν/λ) and S ∈ SSYT(λ), let x0 to be the corner of the λoccupied by the largest entry in stdcont(S)(S). Define T (1) ∈ SSYT(ν(1)/λ(1)) = jdtx0(T ) andS1 ∈ SSYT(λ(1)) the subtableau of S obtained by removing the entry in x0. Fill in the boxν/ν(1) by S(x0), the entry of x0 in S, and denote this filling S(1).

For the triple T (i) ∈ SSYT(ν(i)/λ(i)), ∅ 6= Si ∈ SSYT(λ(i)) and S(i), let xi be the corner ofλ(i) occupied by the largest entry in stdcont(Si)(Si). Set T (i+1) = jdtxi(T

(i)) ∈ SSYT(ν(i+1)/λ(i+1))

and Si+1 ∈ SSYT(λ(1)) the subtableau of Si obtained by removing the entry in xi. Let usobtain S(i+1) from S(i) by attaching the cell ν(i)/ν(i+1) and fill in Si(xi).

For T and S as above, define

Infusion(S, T ) = (Infusion1(S, T ), Infusion2(S, T )) := (T (n), S(n))

where n = |λ|. Observe that T (n) = Rectxi(T ).

Theorem 8.16 ([2, Theorem 2.2 and Theorem 3.1]). S(n) ∈ SSYT(ν/ν(n)) and

Infusion(T (n), S(n)) = (S, T ).

For the ballot T ∈ SSYT(ν/µ, λ) as in the statement of Theorem 8.15, define

Inf(T ) := Infusion2(Yµ, T ).

Claim 8.17. The number of entries in the first column of Inf(T )≤`−a′ is at most `− `(λ).

Proof. Our proof is by contradiction. Suppose there are more than ` − `(λ) entries in thefirst column of Inf(T )≤`−a′ . Let k be the label in matrix coordinate (` + 1, 1) of Inf(T ) ∈SSYT(ν/λ). Then

(55) `− `(λ) < k ≤ `− a′,where the strict inequality is the by column strictness Inf(T ).

By Theorem 8.16, since Infusion1(Yµ, T ) = Yλ,

(56) Infusion(Yλ, Inf(T )) = (Yµ, T ).

26

Let xi be the box in λ occupied by the i-th largest entry in stdλ(Yλ) and set

(57) Inf(T )(i) = jdtxi jdtxi−1· · · jdtx1(Inf(T )) ∈ SSYT(ν(i)/λ(i)).

Let ik be the index such that xik is the leftmost box in row ` + 1 − k of λ. By the upperbound in (55), `+ 1− k > a′. Since, by hypothesis, all entries strictly south of the `-th rowof T are ≤ a′, by (56), Inf(T ) and Inf(T )(1), Inf(T )(2), . . . , Inf(T )(ik−1) coincide below row `.In particular,

Inf(T )(ik−1)(`+ 1, 1) = k.

By our choice of (partial) rectification order (57), in column 1 of ν, the rows ` + 2 − kthrough `+ 1 are all in Inf(T )(ik−1). Thus, since Inf(T )(ik−1) is semistandard,

(58) Inf(T )(ik−1)(`+ 1− j, 1) = k − j, for 0 ≤ j < k.

Thus, in the computation Inf(T )(ik) = jdtxik(Inf(T )(ik−1)), (58) implies jeu de taquin uses

only (J1) as its first k slides. This combined with (55) says that `+ 1− k > a′ appears in arow > ` in T . This is a contradiction of our hypothesis about T .

Claim 8.18. Let c0 be the southmost corner of λ. If demotion is applied to S ∈ SSYT(ν/λ, µ) withrespect to the corner box c0, then demote(c0) = m where m ∈ Z>0 is the smallest number that isnot in the first column of S.

Proof. Since c0 is a corner of the bottom row of λ, the first 1 after c0 in the column readingword, if exists, must be the rightmost 1 in row `(λ) + 1 of S. Therefore we have

S(`(λ) + 1, 1) = 1 and c1 is in row `(λ1) + 1.

Similarly, using that S is semistandard, and an easy induction, if ck exists, it is in row`(λ)+k and S(`(λ)+k, 1) = k. So demote(c0) ≤ m. On the other hand, since m−1 appearsin that first column, it follows cm−1 exists and so demote(c0) ≥ m, and we are done.

Label the boxes of the southmost row of λ by d1 through di, from right to left. FixU1 = Uto be a ballot semistandard tableau. Let demote(dj) be computed by applying demotionto Uj with respect to to dj , where Uj is the result of Uj−1 bumped at dj−1 (if j > 1).

Claim 8.19. demote(d1) ≥ demote(d2) ≥ · · · ≥ demote(di).

Proof. Let mi ∈ Z>0 be the smallest label not appearing in the first column of Ui. Bydefinition of demotion, m1 ≥ m2 ≥ · · · ≥ mi. Now apply Claim 8.18.

Let R := `(λ)− a ≥ 0. If R = 0 then `− a′ = 0 and Theorem 8.15 is trivial since we cantake ρ = µ. Therefore we may assumeR > 0. Now, iteratively apply demotion, beginningwith Inf(T ), using the boxes qiLi=1 in λ taken from right to left, bottom to top. Call theresult Inf(T )′ ∈ SSYT(ν/λ≤a′ , ρ). This completes the proof of Theorem 8.15:

Claim 8.20. The partition ρ satisfies:

(i) cνλ≤a′ ,ρ > 0;(ii) cρλ>a′ ,µ > 0; and

(iii) ρ>`−a′ = µ>`−a′ .

27

Proof of Claim 8.20: (i): Since demotion converts one ballot tableau into another, Inf(T )′ ∈SSYT(ν/λ〈c〉, ρ) is ballot. Hence cνλ≤a′ ,ρ = cνρ,λ≤a′ > 0.

(ii): Let T0 ∈ SYT(λ) be the superstandard tableau using negative entries. Let S =Inf(T ) ∈ SSYT(ν/λ, µ). Let

(59) ω := RSK−1(Yν , T0 ∪ S) =

(t1 . . . tn v1 . . . vmx1 . . . xn w1 . . . wm

).

Let ` ∈ [n] be the index such that

(60) (T ′, (T0)≤a′) = RSK

((t1 . . . t`x1 . . . x`

))for some T ′. By Corollary 8.14, we can find v′1, . . . , v

′m+n−` and w′1, . . . , w

′m+n−` such that

(Yν , (T0)≤a′ ∪ Inf(T )′) = RSK

((t1 . . . t` v′1 . . . v′m+n−`x1 . . . x` w′1 . . . w′m+n−`

)),

where

(Yρ, Yρ) = RSK

((v′1 . . . v′m+n−`w′1 . . . w′m+n−`

)).

Write

(T , T0) = RSK

((t`+1 . . . tnx`+1 . . . xn

)),

and let π be the shape of T . Let T := RSK1(x1x2 · · ·xn) ∈ SSYT(λ). Observe that T ′ ?T ∈ T (λ≤a′ , π, T ), by Proposition 7.7(III) and Theorem 7.5(I), since revrowword(T ′ ? T ) =x1x2 · · ·xn. Since cλλ≤a′ ,π = |T (λ≤a, π, T )| > 0, we get π = λ>a′ .

Notice by Lemma 8.5 and Corollary 8.14, we have x`+1 . . . xnw1 . . . wm≡K revrowword(Yρ).Therefore T ? Yµ ∈ T (λ>a′ , µ, Yρ), again by Proposition 7.7(III) and Theorem 7.5(I). Hence,cρλ>a′ ,µ = |T (λ>a′ , µ, Yρ)| > 0.

(iii): For 1 ≤ t ≤ L+1, let Inf(T )[t] be the (ballot) tableau obtained by iteratively applyingdemotion at q1, . . . , qt−1. Suppose ρ[t] = cont(Inf(T )[t]); hence

µ = ρ[1] ( ρ[2] ( ρ[3] ( · · · ( ρ[L+1] = ρ.

Call ρ[t] good if (ρ[t])i = µi for i > b; we will show all ρ[t] are good.Let r1, . . . , rR be the indices such that qrj is the rightmost box of the j-th southmost row

of λ; thus qr1 = q1. Suppose 1 ≤ u ≤ R. By Claim 8.17 and the construction of demotion,(Inf(T )[ru])≤`−a

′ has at most `− `(λ) + (u− 1) = `− a′−R+ (u− 1) < `− a′ elements in thefirst column. Hence ρ[ru+1] is good by Claim 8.18. Claim 8.19 either says that ρ[L+1] = ρ isgood if u = R (and we are done), or ρ[ru+1] is good if u < R.

9. A NONVANISHING RESULT ABOUT MULTIPLE NEWELL-LITTLEWOOD NUMBERS

To prove Theorem 1.5, we need:

28

ν−

λ+

µ−

ν+

µ+

λ−

α3

α1

α2 α5

α4

α6

FIGURE 2

c′

a′

b′

n− r

1© 2© 3©

4© 5© 6©

7©

8© 9©

FIGURE 3. λ+ = ∅

Theorem 9.1. Let 1 ≤ r < n and λ, µ, ν ⊂ (n − r)r be such that cν∨λ,µ > 0. For a′, b′, c′ ∈ Z≥0

such that a′ + b′ + c′ = r, set

λ− = (λ≤a′)∨[(n−r)a′ ] λ+ = λ>a′

µ− = (µ≤b′)∨[(n−r)b′ ] µ+ = µ>b′

ν− = (ν≤c′)∨[(n−r)c′ ] ν+ = ν>c′

.

Then Nλ−,µ+,ν−,λ+,µ−,ν+ > 0.

9.1. Proof of Theorem 9.1. We must show that there exist α1, . . . , α6 ∈ Parr such that theLittlewood-Richardson coefficients corresponding to each of the six triangles in Figure 2are positive. Set a = b′ + c′, b = a′ + c′, c = a′ + b′. We begin with a special case:

9.1.1. The case when λ+ is empty. Since λ+ = ∅, we need α2 = α3 := ∅ to get cλ+α2,α3> 0.

Also, α1 := µ− and α4 := ν− for cµ−α1,α2, cν

−α3,α4

> 0. It remains to find α5, α6 such that

(61) cν+

µ−,α6, cµ

+

ν−,α5, cλ

−

α5,α6> 0.

Consider Figure 3. We numbered the regions by 1© through 9© (some regions could beempty). We declare that µ is region 1© 4© 7©. Also ν is region 3© 6© 9© rotated 180 degrees,denoted as rotate( 3© 6© 9©). Thus ν∨/µ is region 2© 5© 8©.

29

For any partitions α5, α6 ⊆ (n− r)a′ , by (39), we have

cν+

µ−,α6= c

(ν+)∨[(n−r)c]

(µ−)∨[(n−r)b′],α∨[(n−r)a′ ]6

(62)

cµ+

ν−,α5= c

(µ+)∨[(n−r)b]

(ν−)∨[(n−r)c′],α∨[(n−r)a′ ]5

(63)

cλ−

α5,α6= c

(λ−)∨[(n−r)2a′ ]

α∨[(n−r)a′ ]5 ,α

∨[(n−r)a′ ]6

= c(n−r)a′∪(λ−)∨[(n−r)

a′ ]

α∨[(n−r)a′ ]5 ,α

∨[(n−r)a′ ]6

(64)

Since λ+ = ∅,(λ−)∨[(n−r)a′ ] = λ.

Combining this with (64) gives

(65) cλ−

α5,α6= c

(n−r)a′∪λ

α∨[(n−r)a′ ]5 ,α

∨[(n−r)a′ ]6

.

Observe thatµ+ is 4© 7©ν+ is rotate( 3© 6©)µ− is rotate( 2© 3©)ν− is 7© 8©.

Thus (ν+)∨[(n−r)c]/(µ−)∨[(n−r)b′ ] is 2© 4© 5©, and (µ+)∨[(n−r)b]/(ν−)∨[(n−r)c′ ] is rotate( 5© 6© 8©).By (62), (63) and (65), if there exists

(i) a ballot U ∈ SSYT( 2© 4© 5©, α) with α ∈ Para′ ;(ii) a ballot V ∈ SSYT( 5© 6© 8©, β) with β ∈ Para′ such that

(iii) c(n−r)a′∪λα,β > 0,

then α6 = α∨[(n−r)a′ ] and α5 = β∨[(n−r)a′ ] will satisfy (61). (In (ii) we have used the LR-symmetry cπθ,κ = cθ

∨

π∨,κ.)Two cases occur:

Case 1 (region 5© contains a column of length a′): Since λ+ = ∅, therefore the longest columnof 2© 5© 8© is also of length a′. Now, choose any such column; say its index is col1. Noticethat such a column divides region 5© into two parts: namely, 5©’ = the part of region 5©weakly left of col1 and 5©” = the remainder. Now by Proposition 7.11 (and its proof), thereexist partitions ρ, π where ρ is the content of a ballot tableau U ′ whose shape is region2© 5©” and π is the content of a ballot tableau V ′ whose shape is region 5©’ 8© such that

(66) cλρ,π > 0.

In particular, ρ, π ∈ Para′ .

Since (ν+)∨[(n−r)c]/(µ−)∨[(n−r)b′ ] (region 2© 4© 5©) is obtained by adding a rectangle (col1)a′

to the left end of region 2© 5©”, we can obtain a ballot tableau

U ∈ SSYT( 2© 4© 5©, ρ+ (col1)a′)

from U ′ by filling in each column of the rectangle (col1)a′ with [a′]. This is (i).

30

Similarly, since region 5© 6© 8© is obtained by adding a rectangle (n − r − col1)a′ to the

right end of region 5©’ 8©, we can construct a ballot tableau

V ∈ SSYT( 5© 6© 8©, π + (n− r − col1)a′)

from V ′ by filling in each column of (n− r − col1)a′ by [a′]. This is (ii).

By (66) and Lemma 7.9 where we set k = 0,

c(ρ1+π1)c∪λρ+(πa

′1 ),π+(ρa

′1 )> 0.

Now since π1 ≤ col1 and ρ1 ≤ n − r − col1, it follows that by setting α = ρ + (cola′

1 ) andβ = π + (n− r − col1)a

′ ,

c(n−r)a′∪λα,β > 0,

as required by (iii).Case 2 (no column in region 5© has length a′): Let col1 = (ν∨)b′+a′ and col2 = µb′+1. InFigure 4(A), col1 is the index of the right-most column that does not intersect region 6©. Ifno such column exists, then col1 = 0. Similarly, col2 is the index of the right-most columnthat intersect region 4©. If no such column exists, then col2 = 0.

Set k = col2 − col1. Since cν∨λ,µ > 0, by Lemma 7.12,

(67) cν∨+(k(a

′+b′))

µ+(kb′ ),λ+(ka′ )> 0.

The skew shape (ν∨ + (k(a′+b′)))/(µ + (kb′)) is depicted in Figure 4(B). Set 5©’ to be the

region in the new region 5© whose column is weakly less than col′2(= col2 = col′1 =col1 + k) and 5©” to be the remaining of the new region 5©. By (67) there is a ballotV ∈ SSYT( 2© 5©′ 5©′′ 8©, λ + (ka

′)). As in Case 1, by Proposition 7.11, we can find two

partitions ρ, π where ρ is the content of a ballot tableau U ′ whose shape is region 2© 5©”and π is the content of a ballot tableau V ′ whose shape is region 5©’ 8© such that

(68) cλ+(ka′)

ρ,π > 0.

Since (ν+)∨[(n−r)c]/(µ−)∨[(n−r)b′ ] (region 2© 4© 5©) is obtained by attaching a rectangle (cola′

1 )to the left end of region 2© 5©”, we can construct a ballot tableau

U ∈ SSYT( 2© 4© 5©, ρ+ (col1)a′)

from U ′ by filling in each column of the rectangle (cola′

1 ) by [a′]. This is (i).

Similarly, since region 5© 6© 8© is obtained by adding a rectangle (n − r − col2)a′ to the

right end of region 5©’ 8©, we obtain a ballot tableau

V ∈ SSYT( 5© 6© 8©, π + (n− r − col2)a′)

from V ′ by filling in each column of (n− r − col2)a′ by [a′]. This is (ii).

Notice that since ρ1 ≤ n− r − col1 and π1 ≤ col2, we have

ρ1 − k = ρ1 − col2 + col1 ≤ n− r − col2 and π1 − k = π1 − col2 + col1 ≤ col1.

Furthermore, sinceλ1 ≤ n− r = (n− r − col2) + col1 + k,

31

col1

col2

(A) Region 5©

col′2

5©’

col′1

5©”

2©

8©

(B) New region 5©

FIGURE 4. Case 2

we can apply Lemma 7.9 to (68) to see

c(n−r)a′∪λρ+(cola

′1 ),π+(n−r−col2)a′

> 0.

Thus (iii) holds by setting α = ρ+ (cola′

1 ) and β = π + (n− r − col2)a′ .

9.1.2. The general case. Since cν∨λ,µ > 0, by Theorem 7.1, there is a ballot T ∈ SSYT(ν∨/µ, λ).

Let T = T≤a′∪T≤a′+b′ be the subtableau of T that keeps only the first c = a′+b′ rows, to-

gether with the boxes of T whose entries are at most a′. Notice that T ∈ SSYT(ν(1)/µ, λ(1))is ballot, where ν(1), λ(1) are some partitions with λ(1) ∈ Para′+b′ , ν(1) ⊂ ν∨ and λ(1) ⊂ λ.

By Theorem 8.15, we can find a partition ρ such that

(69) cν(1)

ρ,(λ(1))≤a′> 0, cρ

µ,(λ(1))>a′> 0, ρ>b′ = µ>b′ .

Set

(70) α1 = (ρ≤b′)∨[(n−r)b′ ] and α2 = (λ(1))>a′ .

Since ρ/µ = ρ≤b′/µ≤b′ , by the second non-vanishing in (69),

(71) cµ−

α1,α2> 0.

Consider T>a′ ⊂ T , the tableau obtained from T by only keeping the entries that aregreater than a′. Since T is ballot, for any k > a′ and any initial segment of the column (orrow) reading word of T , k appears at least as much as k + 1. Thus k appears at least asmuch as k+ 1 in any initial segment of the column (or row) reading word of T>a′ . Hence,T>a′ = the tableau obtained from T>a

′ by subtracting a′ from every entry, is ballot.

LetU1 ∈ SSYT(ρ/µ, (λ(1))>a′) be a ballot tableau that witnesses the second non-vanishingin (69). Let U2 be the tableau of shape ν∨/ν(1) where for each box (i, j) ∈ ν∨/ν(1), we set

U2(i, j) = T (i, j)− a′.

Since T (i, j) ≥ a′ for all (i, j) ∈ ν∨/ν(1),

U2(i, j) = T>a′(i, j).

32

Let (T>a′)≤b′+a′ and (T>a′)>b′+a′ be the first b′+a′ and bottom c′ rows of T>a′ respectively.Notice in particular that (T>a′)>b′+a′ = U2, and thus

(72) revrowword((T>a′)>b′+a′) = revrowword(U2).

Since T>a′ is ballot, so is (T>a′)≤b′+a′ . Moreover, by the definition of λ(1), we have

cont((T>a′)≤b′+a′) = (λ(1))>a′ = cont(U1).

Since U1 is also ballot,

(73) revrowword((T>a′)≤b′+a′) ≡K revrowword(U1).

By ballotness of T>a′ , we have

revrowword(Yλ>a′ ) ≡K revrowword(T>a′) = revrowword((T>a′)>b′+a′)·revrowword((T>a′)≤b′+a′).

Combining with (72) and (73), we reach

revrowword(Yλ>a′ ) ≡K revrowword(U2) · revrowword(U1).

As a result, by Theorem 7.5, U2 ? U1 is ballot and cont(U2 ? U1) = λ>a′ = λ+.Then, by the construction in Proposition 7.11, there exists β such that

cν∨

ν(1),β > 0 and cλ+

(λ(1))>a′ ,β> 0.

Setting α3 = β, one gets

(74) cλ+

α2 α3> 0

Set now

(75) α4 = (ν(1))>c.

Since (ν(1))≤c = (ν∨)≤c, we have ν−/α4 = ν∨/ν(1) and

(76) cν−

α3,α4> 0.

We just constructed α1, . . . , α4 such that the three LR-coefficents associated to λ+, µ−and ν− do not vanish. Consider now three alternate partitions

λ = λ≤a′ , µ = ρ and ν∨[(n−r)r] = ν(1),

and define λ+, λ−, µ+, µ−, ν+, ν− as in Theorem 9.1 accordingly. In fact, we have

λ− = λ− λ+ = ∅µ− = α1 µ+ = µ+

ν− = α4 ν+ = ν+

.

By (69),

cν∨[(n−r)r ]

λ,µ> 0.

We can now apply Section 9.1.1 to get

Nλ−,µ+,ν−,λ+,µ−,ν+ > 0.

In particular, we haveα1 = α1, α2 = ∅, α3 = ∅, α4 = α4.

33

Therefore,

cν+

α1,α6= cν

+

α1,α6> 0,

cµ+

α4,α5= cµ

+

α4,α5> 0,

cλ−

α5,α6= cλ

−

α5,α6> 0.

Combined with (71), (74) and (76), we showed that the sextuple (α1, α2, α3, α4, α5, α6)make all six Littlewood-Richardson coefficients in Figure 2 non-zero.

10. EXTENDED HORN INEQUALITIES AND THE PROOF OF THEOREM 1.5

10.1. Extended Horn inequalities. We recall the following notion from [10]:

Definition 10.1. An extended Horn inequality on Par3n is

(77) 0 ≤ |λA| − |λA′ |+ |µB| − |µB′ |+ |νC | − |νC′ |

where A,A′, B,B′, C, C ′ ⊆ [n] satisfy

(I) A ∩ A′ = B ∩B′ = C ∩ C ′ = ∅(II) |A| = |B′|+ |C ′|, |B| = |A′|+ |C ′|, |C| = |A′|+ |B′|

(III) There exists A1, A2, B1, B2, C1, C2 ⊆ [n] such that:(i) |A1| = |A2| = |A′|, |B1| = |B2| = |B′|, |C1| = |C2| = |C ′|

(ii) cτ(A′)τ(A1),τ(A2), c

τ(B′)τ(B1),τ(B2), c

τ(C′)τ(C1),τ(C2) > 0

(iii) cτ(A)τ(B1),τ(C2), c

τ(B)τ(C1),τ(A2), c

τ(C)τ(A1),τ(B2) > 0.

Definition 10.2. The extended Horn cone is:

(78) EH(n) := (λ, µ, ν) ∈ (ParQn )3 : inequalities (77) are satisfied.

LetEH(n) = EH(n) ∩ (λ, µ, ν) ∈ (Parn)3 : |λ|+ |µ|+ |ν| is even.

Conjecture 10.3 ([10, Conjecture 1.4]). If (λ, µ, ν) ∈ EH(n) then Nλ,µ,ν > 0.

We will prove a weakened version of Conjecture 10.3:

Theorem 10.4 (cf. [10, Conjecture 1.4]). EH(n) = NL-sat(n).

Consequently, we are able to answer an issue raised in [10, Section 1]:

Corollary 10.5. Conjecture 1.4 implies Conjecture 10.3.

Corollary 10.5 is analogous to the situation in Zelevinsky’s [29], before [15].The following shows that Theorem 1.5 is equivalent to Theorem 10.4.

Lemma 10.6. A sextuple (A,A′, B,B′, C, C ′) of subsets of [n] parametrizes an extended Horninequality if and only if it appears in Theorem 1.5.

Proof. Definition 10.1 implies

cτ(A′)τ(A1),τ(A2)c

τ(B)τ(A2),τ(C1)c

τ(C′)τ(C1),τ(C2)c

τ(A)τ(C2),τ(B1)c

τ(B′)τ(B1),τ(B2)c

τ(C)τ(B2),τ(A1) > 0.

Since τ(A1), τ(A2) . . . have length at most n, this implies Nτ(A′),τ(B),τ(C′),τ(A),τ(B′),τ(C) 6= 0.

34

Conversely, ifNτ(A′),τ(B),τ(C′),τ(A),τ(B′),τ(C) 6= 0, there exists α1, α2, . . . , α6 ∈ Parn such that

cτ(A′)α1,α2

cτ(B)α2,α3

cτ(C′)α3,α4

cτ(A)α4,α5

cτ(B′)α5,α6

cτ(C)α6,α1

> 0.

Set a′ = |A′|. Then, the Young diagram of τ(A′) is contained in the rectangle a′ × (n −a′). But the nonvanishing of cτ(A′)

α1,α2 implies that α1 ⊂ τ(A′). Hence there exists A1 ⊆ [n]such that τ(A1) = α1. Similarly, we can pick A2, B1, B2, C1, C2 ⊆ [n] such that τ(A2) =α2, τ(C1) = α3 etc. that satisfy Definition 10.1.

10.2. Proof of Theorem 1.5. (⇒) By Lemma 10.6, EH(n) is the cone defined by the in-equalities in Theorem 1.5. Now, NL-sat(n) ⊆ EH(n) is immediate from [10, Theorem 1].

(⇐) Fix an inequality (4) associated to (A,A′, B,B′, C, C ′) appearing in Theorem 1.2.We prove that this inequality appears in Theorem 1.5; that is

(79) Nτ(A′),τ(B),τ(C′),τ(A),τ(B′),τ(C) 6= 0.

Set r = |A| + |A′|. Let I ∈ Schub(Grω(r, 2n)) be associated to (A,A′) ∈ Schub′(Grω(r, 2n)),under (22). Similarly define J and K. Now apply Theorem 9.1 with λ = τ(I)∨, µ = τ(J)∨

and ν = τ(K)∨, and a′ = |A′|, b′ = |B′| and c′ = |C ′| (refer to Figure 1 to see, τ(A′) = λ−,τ(B) = µ+ etc.). The assumption a′ + b′ + c′ = r comes from condition (2) in Theorem 1.5.The assumption cν

∨

λ,µ > 0 is implied by condition (3) in Theorem 1.2 combined with thesemigroup property of LR coefficients. Specifically, we are claiming, from the definitionsthat τ 0(C,C ′) + τ 2(C,C ′) = τ(K) = ν∨ etc. By Theorem 9.1, (79) holds, since, λ− =τ(A′), λ+ = τ(A), etc., as desired.

ACKNOWLEDGEMENTS

We thank Winfried Bruns and the Normaliz team. SG, GO, and AY were partially sup-ported by NSF RTG grant DMS 1937241. SG was partially supported by an NSF graduateresearch fellowship. AY was supported by a Simons collaboration grant and UIUC’s Cen-ter for Advanced Study.

REFERENCES

[1] Prakash Belkale and Shrawan Kumar. Eigenvalue problem and a new product in cohomology of flagvarieties. Invent. Math., 166(1):185–228, 2006.

[2] Georgia Benkart, Frank Sottile, and Jeffrey Stroomer. Tableau switching: algorithms and applications.J. Combin. Theory Ser. A, 76(1):11–43, 1996.

[3] Emmanuel Briand, Rosa Orellana, and Mercedes Rosas. Rectangular symmetries for coefficients ofsymmetric functions. The Electronic Journal of Combinatorics, 22(3), Jul 2015.

[4] W. Bruns, B. Ichim, C. Soger, and U. von der Ohe. Normaliz. algorithms for rational cones and affinemonoids. Available at https://www.normaliz.uni-osnabrueck.de.

[5] Harm Derksen and Jerzy Weyman. The combinatorics of quiver representations. Ann. Inst. Fourier,61(3):1061–1131, 2011.

[6] William Fulton. Young tableaux, volume 35 of London Mathematical Society Student Texts. CambridgeUniversity Press, Cambridge, 1997. With applications to representation theory and geometry.

[7] William Fulton. Eigenvalues, invariant factors, highest weights, and Schubert calculus. Bull. Amer.Math. Soc. (N.S.), 37(3):209–249, 2000.

[8] Shiliang Gao, Gidon Orelowitz, Nicolas Ressayre, and Alexander Yong. Sage programs and data onthe NL-semigroup. http://math.univ-lyon1.fr/~ressayre/Info/computedDatumNL.tar.gz, 2021.

[9] Shiliang Gao, Gidon Orelowitz, and Alexander Yong. Newell-Littlewood numbers, 2020.[10] Shiliang Gao, Gidon Orelowitz, and Alexander Yong. Newell-Littlewood numbers II: extended Horn

inequalities, 2020.

35

[11] Christian Gutschwager. On multiplicity-free skew characters and the Schubert calculus. Ann. Comb.,14(3):339–353, 2010.

[12] G. J. Heckman. Projections of orbits and asymptotic behavior of multiplicities for compact connectedLie groups. Invent. Math., 67(2):333–356, 1982.

[13] Ronald C. King, Christophe Tollu, and Frederic Toumazet. Factorisation of Littlewood-Richardsoncoefficients. J. Combin. Theory Ser. A, 116(2):314–333, 2009.

[14] Alexander A. Klyachko. Stable bundles, representation theory and Hermitian operators. Selecta Math.(N.S.), 4(3):419–445, 1998.

[15] Allen Knutson and Terence Tao. The honeycomb model of GLn(C) tensor products. I. Proof of thesaturation conjecture. J. Amer. Math. Soc., 12(4):1055–1090, 1999.

[16] Kazuhiko Koike and Itaru Terada. Young-diagrammatic methods for the representation theory of theclassical groups of type Bn, Cn, Dn. J. Algebra, 107(2):466–511, 1987.

[17] Shrawan Kumar. A survey of the additive eigenvalue problem. Transform. Groups, 19(4):1051–1148,2014. With an appendix by M. Kapovich.

[18] Shrawan Kumar. Additive eigenvalue problem. Eur. Math. Soc. Newsl., 98:20–27, 2015.[19] D. E. Littlewood. Products and plethysms of characters with orthogonal, symplectic and symmetric

groups. Canadian J. Math., 10:17–32, 1958.[20] David Mumford, John Fogarty, and Frances Kirwan. Geometric Invariant Theory. Springer Verlag, New

York, 3d edition, 1994.[21] M. J. Newell. Modification rules for the orthogonal and symplectic groups. Proc. Roy. Irish Acad. Sect.

A, 54:153–163, 1951.[22] B. Pasquier and N. Ressayre. The saturation property for branching rules—examples. Exp. Math.,

22(3):299–312, 2013.[23] Nicolas Ressayre. Geometric invariant theory and generalized eigenvalue problem. Invent. Math.,

180:389–441, 2010.[24] Nicolas Ressayre. Multiplicative formulas in Schubert calculus and quiver representation. Indag. Math.

(N.S.), 22(1-2):87–102, 2011.[25] Nicolas Ressayre. A cohomology-free description of eigencones in types A, B, and C. Int. Math. Res.

Not. IMRN, (21):4966–5005, 2012.[26] Nicolas Ressayre. Reductions for branching coefficients. J. Lie Theory, 31(3):885–896, 2021.[27] Edward Richmond. A multiplicative formula for structure constants in the cohomology of flag vari-

eties. Michigan Math. J., 61(1):3–17, 2012.[28] Mike Roth. Reduction rules for Littlewood-Richardson coefficients. Int. Math. Res. Not. IMRN,

(18):4105–4134, 2011.[29] Andrei Zelevinsky. Littlewood-Richardson semigroups. In New perspectives in algebraic combinatorics

(Berkeley, CA, 1996–97), volume 38 of Math. Sci. Res. Inst. Publ., pages 337–345. Cambridge Univ. Press,Cambridge, 1999.

DEPT. OF MATHEMATICS, UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN, URBANA, IL 61801

Email address: sgao23@illinois.edu

DEPT. OF MATHEMATICS, UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN, URBANA, IL 61801

Email address: gidono2@illinois.edu

INSTITUT CAMILLE JORDAN (ICJ), UMR CNRS 5208, UNIVERSITE CLAUDE BERNARD LYON I, 43BOULEVARD DU 11 NOVEMBRE 1918, F - 69622 VILLEURBANNE CEDEX

Email address: ressayre@math.univ-lyon1.fr

DEPT. OF MATHEMATICS, UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN, URBANA, IL 61801

Email address: ayong@illinois.edu

36