on the restriction of deligne-lusztig characters

On the Restriction of Deligne-Lusztig CharactersAuthor(s): Mark ReederSource: Journal of the American Mathematical Society, Vol. 20, No. 2 (Apr., 2007), pp. 573-602Published by: American Mathematical SocietyStable URL: http://www.jstor.org/stable/20161332 .

Accessed: 17/06/2014 15:56

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

American Mathematical Society is collaborating with JSTOR to digitize, preserve and extend access to Journalof the American Mathematical Society.

http://www.jstor.org

This content downloaded from 188.72.126.108 on Tue, 17 Jun 2014 15:56:03 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=ams

http://www.jstor.org/stable/20161332?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


JOURNAL OF THE AMERICAN MATHEMATICAL SOCIETY Volume 20, Number 2, April 2007, Pages 573-602 S 0894-0347(06)00540-6 Article electronically published on July 14, 2006

ON THE RESTRICTION OF DELIGNE-LUSZTIG CHARACTERS

MARK REEDER

1. Introduction

This paper was motivated by the following restriction problem for representations of finite orthogonal groups.

Let # be an algebraic closure of a finite field f of cardinality q, a power of a prime p > 2. Let G =

SO(V) be the special orthogonal group of a (2n + l)-dimensional ?-space V with nondegenerate quadratic form Q. Assume V and Q are defined over f, and let F denote the corresponding Probenius endomorphisms of V and G. Fix v e VF with Q(v) t? 0 and let H be the stabilizer of v in G.

Let 7T e 1tt(Gf), a e lrr(HF) be complex irreducible cuspidal representations of the respective groups GF and HF of f-rational points. The problem is to compute the multiplicity

(7T,a)HF =

dimHom#F(7r, a)

of a in the restriction of tt to HF.

Using unpublished work of Bernstein and Rallis (independently) on p-adic or

thogonal groups, it can be shown that

(7r,a)HF = 0 or 1.

In this paper, we compute (tt, g)hf exactly, when tt and a are irreducible cuspidal Deligne-Lusztig representations [8]. We do not rely on the above-mentioned work of Bernstein and Rallis. Our calculation follows from a qualitative study of restrictions of Deligne-Lusztig characters for general simple algebraic groups, to be described later in this introduction.

To state our multiplicity result for orthogonal groups, we first recall the inducing data. Let T C G, S C H be F-stable anisotropic tori in G and H. There are unique partitions A = (jAj), /x = (jH) of n (here \j,/j,j are the number of parts equal to

j) such that

j j

where, for any d > 1, f^ = $F is the extension of f in # of degree d, and

f^ is the

kernel of the norm mapping f^ ?? f *. The number of parts J2j Vj *s even ^ H is

split, and odd if H is nonsplit.

Received by the editors June 17, 2005.

2000 Mathematics Subject Classification. Primary 20C33. The author was supported by NSF grant DMS-0207231.

?2006 American Mathematical Society Reverts to public domain 28 years from publication

573



574 MARK REEDER

Let x G Irr(TF) and rj e Irr(5F) be irreducible characters of TF and SF which are regular in the sense that \ and r? have trivial stabilizers in the respective Weyl groups Wq(T)f and Wh(S)f. We may write

X = ?jXi, V = ?jVj,

where

Xj=Xji?~-?Xj\j eIrr((f2,)A0> each Xjk is a character of

f^, and likewise for rj. Let I^j ^ Z/2jZ be the Galois

group of f2j/f.

Definition 1.1. We say that x and 77 intertwine if rjjk' is a I^j-conjugate of Xjk for some 1 < j < n, 1 < k < Aj, 1 < kf < ?ij.

Note that x and 77 can intertwine even if T qk S. However, if ? and 77 have no

common parts, that is, if \j?ij = 0 for all j, then % and 77 do not intertwine.

By Deligne-Lusztig induction, we have virtual representations Rj, of GF and

Rg of HF', respectively. By the regularity assumptions on x and 77, these are

actually irreducible characters, up to sign. In fact, we have

(-l)rkGi??x e Ivt(Gf), (-l)rkH?"? G Irr(#F). These two irreducible characters are cuspidal, since T and 5 are anisotropic. We

prove:

Theorem 1.2. Let T and S be anisotropic F-stable maximal tori in G and H,

respectively, and let x G ITr(TF), 77 e Irr(5F) be regular characters. Then

/jxrkG+rktf/?G RH v = f? *fv,X intertwine,

'x' ,v 1 1 z/ry, x do ^oi intertwine.

If T and 5 are arbitrary jF-stable maximal tori, but \ an(i n are s^iH regular, then the multiplicity is either zero or a power of two; see (9.9) below.

The multiplicity result 1.2 is used in [12] to verify some cases of the conjectures of [11] describing restrictions from p-adic 502n+i to S02n in terms of symplectic local root numbers and the parameterization of depth-zero supercuspidal L-packets

given in [7]. As already mentioned, Theorem 1.2 follows from a qualitative result, in a general

setting, on multiplicities of Deligne-Lusztig representations. Let G be a a connected simple algebraic group defined over f, and let H be a

connected reductive f-subgroup of G. Fix F-stable maximal tori T C G and S C H,

along with arbitrary characters x ? Irr(TF) and 77 G Irr(SF). Prom this data Deligne and Lusztig [8] construct virtual characters i?!f % and

Rg on GF and HF, respectively. Let ( , )hf be the canonical pairing on virtual

characters of HF. We are interested in the multiplicity

(Rt,x^S,t])hf^

where R!f is viewed as a virtual character of HF, by restriction.

Let B and Bh be Borel subgroups of G and H, respectively, and let ? be the mini

mum codimension of a J3#-orbit in G/B. The invariant 5 is called the complexity of the H-variety G/B. The theory of complexity was first studied for reductive

groups over fields of characteristic zero (cf. [1] and the references therein). In that



ON THE RESTRICTION OF DELIGNE-LUSZTIG CHARACTERS 575

setting, it is proved in [1] that ? governs the growth of multiplicities in restric

tions of algebraic representations. We will show that ? also governs the growth of

multiplicities in restrictions of Deligne-Lusztig representations. Because we are in nonzero characteristic, we need to make an assumption. Let

g, v) be the Lie algebras of G and H.

Assumption 1.3. There is an Ad(H)-stable decomposition fl = ?) 0 m, and a

nondegenerate symmetric bilinear form B on m, invariant under Ad(H).

This assumption holds if p is a good prime for g and the Killing form of g is

nondegenerate on \) [24, 1.5.3]. For G = SOn+i,H

= SOn, our assumption holds

for p > 2. For an integer v > 1, let ATj : TF" ?? TF be the norm map, and let

Under Assumption 1.3, we prove the following.

Theorem 1.4. There is a polynomial of degree at most ?:

M(t) = At6 + ---

eQ[t],

whose coefficients depend on \ and W> and an integer m > 1 such that

for all positive integers v = 1 mod m. The degree ? is optimal: if q is sufficiently

large, there exist x, V such that the leading coefficient A is nonzero.

We also give an explicit formula for the leading term A in Theorem 1.4 (see Proposition 7.4). For G =

SOn+i, H = SOn, we have ? = 0, and our explicit formula for A leads to Theorem 1.2 (see Section 9). Even if ? > 0 one can sometimes use Theorem 1.4 to compute exact multiplicities, by exploiting the polynomial nature of M(t). In Section 10 we illustrate this for G = S Or, H = G2, where

5=1.

Our formula for A also allows us to show, for general G and H, and "very regular" X (see section 8), that the multiplicity

(RTx,S?h)hf

of the Steinberg representation Stn is a monic polynomial in q of degree ?, while the multiplicity of the trivial representation

(R^x,1h)hf is a polynomial in q of degree strictly less than ?. In particular, for G =

502n+i and H ?

S02n, we have

for very regular x

To prove Theorem 1.4 we use a method introduced by Thoma [27] for the study of the restriction of irreducible representations from GLn(f) to GLn_i(f) (where again ? = 0). In that situation, the Green's functions giving the character on unipotent elements were explicitly known. Hagedorn [13], in his 1994 Ph.D. thesis, showed how some of Thoma's methods could be generalized to Deligne-Lusztig characters for other pairs of classical groups, where the Green's functions are less explicit. The



576 MARK REEDER

abstract results of Hagedorn gave me the courage to attempt such calculations for

general groups and to obtain closed multiplicity formulas for orthogonal groups. It is a pleasure to thank Dick Gross for initiating the work in [12] which led to

this paper, for helpful remarks on an earlier version, and for aquainting me with

Hagedorn's thesis. The referee read the original version of this paper with care and insight, made

valuable comments and simplified some of the arguments. In particular, the proof of Lemma 3.1 given below is due to the referee and is much shorter than the original one.

Some general notation: The cardinality of a finite set X is denoted by \X\. Equivalence classes are generally denoted by [ ], sometimes with ornamentation. If g is an element of a group G, we write Ad(g) for the conjugation map Ad(g) : x i? gxg~x, and also write 9T := gTg~x for a subgroup T C G. The center of G is denoted Z(G) and the centralizer of g G G is denoted CG(g).

We write ( , )# for the pairing on the space of class functions on a finite group H, for which the irreducible characters of H are an orthonormal basis. l?G,Gf D H are finite overgroups of H and i?, ip' are class functions on G, G' respectively, then

(^^')h is understood to mean (^\h^'\h)h^ where \h denotes restriction to H.

2. Remarks on maximal tori

Let G be a connected reductive algebraic J-group. We assume G is defined over

f and has Frobenius F. If T is a maximal torus in G we denote its normalizer in G by NG(T) and write WG(T) = NG(T)/T for the Weyl group of T in G. If T is F-stable, we have

W(T)F = NG(T)F/TF,

by the Lang-Steinberg theorem. The reduction formula for Deligne-Lusztig characters (recalled in section 4 below)

involves a sum over the following kind of subset of GF. Fix an F-stable maximal torus T C G, and let s be a semisimple element in GF. We must sum over the set

NG(s,T)F:={j GF: *" T}.

Note that NG(s, T)F, if nonempty, is a union of GF x NG(T)F double cosets, where

Gs := CG(s)? is the identity component of the centralizer CG(s) of s in G. To say that s^ G T is to say that 7TcGs, so determining the GF x NG(T)F

double cosets in NG(s,T)F amounts to determining the Gf-conjugacy classes of

F-stable maximal tori in Gs which are contained in a given GF-conjugacy class.

Such classes of tori are parameterized by twisted conjugacy classes in Weyl groups of Gs and G.

The aim of this section is to parameterize the GF x NG(T)F double cosets in

NG(s, T)F in terms of the fiber of a natural map between twisted conjugacy classes in the Weyl groups of Gs and G. This parameterization will be fundamental to our

later calculations with Deligne-Lusztig characters. We begin by recalling the classification of F-stable maximal tori in G. See [5,

chap. 3] for more details in what follows. Fix an F-stable maximal torus To in

G contained in an F-stable Borel subgroup of G, and abbreviate NG = Ng(Tq),

Wg = WG(T0).




Let T(G) denote the set of all F-stable maximal tori in G. Then T(G) is a finite union of GF-orbits. For any T G T(G), let

[T\g := {7T : 7 G GF} denote the GF-orbit of T. There is g G G such that T = 9T0. Since T is F-stable, we have g~1F(g) G Afc- This gives an element

w := g~1F(g)T0 G WG.

The map Ad(#)t =

<7?#_1 is an f-isomorphism

Ad(g):{T0,wF)?+(T,F), where the second component denotes the action of Frobenius under an f-structure.

For any finite group A with F-action, we let H1 (F, A) denote the set of F

conjugacy classes in A. These are the orbits of the action of A on itself via (a, b) i?

abF(a)~1. Let [b] G H1 (F, A) denote the F-conjugacy class of an element b G A. For g,T,w as above, the F-conjugacy class of w is independent of the choice of

g. Hence we have a well-defined class

c\(T,G):=[w]eH1(F,WG). For each u G H1 (F, WG), the set

%(G):={TeT(G): cl(T,G)=u;} is a single GF-orbit in T(G), and all GF-orbits are of this form. Thus, the partition of the set of F-stable maximal tori into GF-orbits is given by

T(G) = JJ %{G).

Let s e GF be semisimple, and let Ts be an F-stable maximal torus of Gs contained in an F-stable Borel subgroup of Gs, and let WGs be the Weyl group of

Ts in Gs. The partition of T(GS) into Gf-orbits is given, as above, by

T(GS) = ?J %(GS). veH^F^Wos)

If T e T(G), the set of F-stable maximal tori in Gs which are GF-conjugate to T is a finite union (possibly empty) of Gf-orbits. We want to describe this union in terms of F-conjugacy classes in WGs. That is, given u G H1 (F, WG), we have

(2.1) TUJ(G)nT(Gs) = j} %(GS)

veMw

for some subset M^ ? Hl(F, WGs), and our task is to find Mw. The first point is that Ts is generally not contained in an F-stable Borel subgroup

of G. Let g G G be such that 9TS = T0, and let ys := gF(g)~l have image ys G WG. Then

d(Ts,G) = [ys}eH1(F,WG), and Ad(#) is an f-isomorphism

Ad(g):(Ts,F)?+ (T0,ysF).

Now To is also a maximal torus in Ad(^)Gs, whose Weyl group

WG3 := Ad(g)WGa

is a subgroup of WG, stable under Ad(ys) o F.



578 MARK REEDER

Define jGs : H1 (F, WGs) ?> H1 (F, WG) to be the composition of maps

(2.2) jGs : H\F, WGa) ^ H\yBF, W'Gm) ^ H^y.F, WG) ̂ H^F, Wg), where the middle map is induced by the inclusion WG

^ WG and rVs is the

twisting bijection given by rVs[x] =

[xys]. Now let T be an arbitrary F-stable maximal torus in Gs. Write T =

hTs, with

h e Gs, so that h~lF(h) G cl(T,Gs). For g G G as above, we have T = ^_1T0.

Since

gh-'Fihg-1) =

g{h-lF{h))g-1 gF(g)-\ it follows that

(2.3) cl(r,G)=jG.(cl(r,G.)). This proves:

Lemma 2.1. For each uo G H1 (F, WG) and T G TU(GS), we have

TUG)nT(Gs)= J} %(GS). vej^li")

We can also parameterize the Gf-orbits in [T]G n T(GS) via the mapping

(2.4) NG(s, T)F := {7 e GF : se ^T} ?> [T]G n T(GS), 7 ~ 7T.

Note that Gf acts on NG(s,T)F by left multiplication, and that (2.4) factors

through the quotient

(2.5) ?G(s,T)F:=GF\NG(s,T)F. The action of NG(T)F on NG(s, T)F by right multiplication commutes with the

Gf-action, hence factors through an action on ?G(s,T)F, where TF acts trivially. This gives an action of WG(T)F on ?G(s, T)F, whose orbits are the Gf x N(T)F double cosets in N(s,T)F.

Lemma 2.2. The mapping (2.4), sending 7 1?> 1T, induces a bijection

?G(s,T)F/WG(T)F ^ GF\([T]G n T(GS)) with the property that the stabilizer in WG(T)F of the class 7 G ?(s,T)F is iso

morphic, via Ad(7), to V^gs(7T)f.

Proof The bijectivity is straightforward and left to the reader. Let w G WG(T)F, and let w G NG(T)F be a representative of w. Then

7.^ = 7 & Gf-yw =

GFj & Ad(>y)we NGaCT)

This implies the assertion about the stabilizer. D

Combining Lemmas 2.1 and 2.2, we get an explicit formula for \?G(s,T)F\.

Corollary 2.3. Let u G Hl(F,WG) and T e %(G). Then the set NG(s,T)F is

nonempty if and only if the fiber jGl(w) is nonempty, in which case we have

|iVG(s'T) '" ? ?w^OT where, for each v G Jq1^), the torus Tv is chosen arbitrarily in TV(GS).




3. ON THE CENTRALIZER OF A SEMISIMPLE ELEMENT

Let s e GF be semisimple. In the previous section we parameterized the set of

Gf-conjugacy classes of maximal tori in Gs which are contained in a given GF

conjugacy class, in terms of fibers of the map jGs : HX(F,WGs) ?>

Hl(F,WG). To compute this map jGs concretely, we must find an element ys G WG such that

cl(Ts,G) =

[ys], where Ts G T(GS) is contained in an F-stable Borel subgroup of

Gs. This amounts to finding the f-isomorphism class of the connected centralizer

Gs.

An elegant formula for ys was given by Carter [6], using the Brauer complex. Here we explain a different method that is suited to our later computations; namely we show how the class [ys] can be determined from the effect of F on a "diagonal ized" G-conjugate of s. Unfortunately, both the present method, as well as that of

[6] require that CG(s) be connected. That is, we must assume that Gs = CG(s).

This holds for any semisimple s G G if G has simply-connected derived group. Our method generalizes that of Gross [10], who determined CG(s) when this group is a

torus (over an arbitrary field). Let $ denote the set of roots of To in G. Let ? denote the automorphisms of 3>

and WG induced by F. For a G $ with corresponding reflection sa G WG, we have

aoF = qf?~1-a, u(sa)

= s#.a.

Here is our recipe for finding cl(Ts, G). Let t G T0 be a G-conjugate of s, and let

$t = {ae$: a(t)

= 1}.

Since t has a conjugate in GF, there is w G W (not necessarily unique) such that

(3.1) F(t)=tw.

Choose such a w arbitrarily. From (3.1) it follows that

(3.2) w$$t = <S>t.

Now choose any positive system $f C $t- Then (3.2) implies that w? $f is another positive system in $*. Being the Weyl group of $t, the group WGt acts

simply transitively on positive systems in $?, so there is a unique x e WGt such that

(3.3) w?-$?=x>$?.

Setting y = x~lw, we see that w can be factored uniquely as

(3.4) w = xy,

where x G WGt and yd 3>+ = $f. Since CG(t) is connected, the group WGt is the full stabilizer of t in WG. This

means that a different choice of w satisfying (3.1) will change x, but not y.

Lemma 3.1. With y constructed as above, we have

cl(Ts,G) = [y]eH1(F,WG).

Proof The following proof was provided by the referee; it is shorter than the original proof. Choose g e G such that y = g~lF(g) e NG is a representative of y. Then

yF(t) =tX=t,



580 MARK REEDER

which implies that 9t G GF. Since CG(s) is connected, any element of GF which is

G-conjugate to s is in fact GF-conjugate to s. Hence, by multiplying g on the left

by an element of GF, we may assume that s = 9t.

By definition of y, there is an Ad (y) F-stable Borel subgroup Bt C Gt contain

ing T0. Hence 9Bt is an F-stable Borel subgroup of Gs, containing the F-stable maximal torus T's := 9T0. Since T's is Gf-conjugate to Ts, it follows that

cl(Ts,G) = [g-1F(g)} = [y],

as claimed. D

4. Deligne-Lusztig characters

Let T G T(G) be an F-stable maximal torus in G, and let x G Irr(TF). The

Deligne-Lusztig character R^a

has the following reduction formula [8]: For u unipo tent in Gf, we have

(4.1) R$a(su) =

?2 xd-'s^Q^-du) ?eNG(s,T)F

The summation is over the set ?G(s,T)F defined in (2.5), and for any reductive

f-group H, and S G T(H), the Green function Qg on the unipotent set of HF is defined by

Q%(u) =

R%A(u). In this section we describe the summation over ?G(s, T)F in (4.1) in terms of fibers of the map jGs studied in the previous two sections.

Breaking the sum (4.1) into W<3(T)F-orbits, we have

(4.2) ??lX(~)= E ?rjWE^l. vej?la) ^?v

where u = cl(T, G), Tv is any torus in TV(GS), and Ov is the WG(T)F-orbit in

?G(s,T)F corresponding to v e jgX(<jo) as in Lemma 2.2.

By the stabilizer assertion in Lemma 2.2, the inner sum in (4.2) can be written as

follows. For any 7 e NG(s, T)F and x' G Irr(TF), the value at s of the transported character

V:=x'oAd(7-1)eIrrrTF)

depends only on the image 7 G ?G(s,T)F. We have

(4.3) E X(7-V) = IWg \t )F| E 7*X(*)>

where 7 on the right side of (4.3) is an arbitrary element of NG(s,T)F such that

7 G Ov. In our later computations with ?^ it will be useful to let 5 vary in GF in such a

way that Gs is unchanged. Let Z(GS) denote the center of Gs. For v G i^1(F, Wgs), the function

(4.4) Xv := ? 7X 7 0?




is well-defined on Z(GS)F, and we have

(4.5) RtJ )= E Qt:(u)Xv(z), i?Gz = Gs.

5. Multiplicity as a polynomial

In this section we begin the proof of Theorem 1.4 and will show that the multi

plicity is given by a polynomial function. Let G be a connected reductive algebraic group over f. Let H C G be a connected reductive f-subgroup of G, and let S be an F-stable maximal torus of H.

5.1. Summation on HF. Suppose we are given a function / : HF ?> C, invariant under conjugation by HF, with the property that if h G HF has Jordan decompo sition h ? su, then f(h)

= 0 unless the conjugacy class Ad(i?TF) s meets S. Our first aim is to express the sum of / over HF as a sum of rational functions in q over an index set which does not depend on q.

Let Hss and Hupt be the sets of semisimple and unipotent elements of H. Let

S(HF) and U(HF) be the sets of Ad(#F)-orbits in (HSS)F and (i?uPt)F, respec

tively. By the vanishing assumption on /, we have

(5.1)

1 ' h?HF

' ' s (#ss)FuE(#supt)F

1 ' sesF ' v y ' [u]eu(HF)

The map 7^57 induces a bijection

CH(s)F\NH(s,S)F ^ Ad(HF) -snS, so that

Recalling that |^h(s)^ I

?H(s,S)F = H?\NH(s,S)F,

we get

(5-2) ^S/w\?^^M?n^?^?

5.2. A partition of S. To this point, the overgroup G has not played a role. Now G is used to partition the sum over SF in (5.2), as follows. Let I(S) be an index set for the set of subgroups

{Gs : se S}. Note that each element of I(S) is determined by a subset of the roots of S in G; hence I(S) is finite. For i e I(S) let GL be the corresponding connected centralizer, and let

SL:={seS: Gs = GJ.



582 MARK REEDER

Thus, S is finitely partitioned as

S= II ?v tei(S)

The F-action on S induces a permutation of I(S), and we let I(S)F be the F-fixed

points in I(S). Note that if Sf is nonempty, then i e I(S)F. For t e I(S), we set

HL:=(HnGL)?,

which is none other than Hs for any s G SL. Note that if s G SL, then s G S fl Ch(s), which implies that

(5.3) s e HL C G?.

Returning to our sum (5.2), we now have

1 f(su) (5-4) ijjFi E /W- Z^ ?^ 2^ \?u(s S)F\ \Cm (u)F\' 1 Uef^ Lei(S)F [u]eu(HF) sesF liV^5'^ I l?^W I

5.3. Restriction of Deligne-Lusztig characters. We now consider the function

/ arising in our multiplicity formula. Let H, S be as above, let T be an F-stable maximal torus of G, and let x G Irr(TF), rj G Irr(5F) be arbitrary characters.

Using the function / : HF ?? C given by

(5.5) f(h) = R!fJh).R?v(h), we have

M) (B?^R^H^-^r ? /W 1 ' heHF The map

jGs:H\F,WGa)^H\F,WG) defined in (2.2) depends only on Gs, so we set

3Gt ~3G8, for any s e SL.

We have an analogous map

jHt : H\F,WHi) ? tf1^,^).

Likewise, the sets ?G(s,T)F and ?jj(s,S)F depend only on ?, so we now write

?G(i,T)F :=?G(s,T)F, ?h(l,S)f := ?h(s,S)f, for s E SF.

Using (4.5) for G and H, along with (5.4), our multiplicity formula becomes

[u]eU(H?)

where the middle sum runs over v e .^(c^T, G)) and ? G j]I1(cl(S,H)). The character sums x^ and 77c are as defined in (4.4).




5.4. Green functions. We digress from our multiplicity formula (5.7) to recall

the polynomial nature of Green functions ?f?, defined on the unipotent set of GF, for a connected reductive f-group G with Frobenius F and F-stable maximal torus

TinG. For u =

1, we have

(5.8) Q$(l)=eG(w)[GF:TF}p,, where [GF : TF]P' is the maximal divisor of the index [GF : TF] which is prime to p, w e cl(T, G) and eG : WG

? {?1} is the sign character of WG. Note that

eG(W) =

(-l)rkC?+rkT ^ 7^2]. For u t? 1, the Green functions Q^(u) can be expressed as polynomials which are

known explicitly by tables for exceptional groups [3], [18] and for classical groups

by recursive formulas [19] which can be implemented on a computer [9]. It will

suffice for us to know the leading terms of these Green polynomials, which can be

expressed in a uniform way.

Let BG be the variety of Borel subgroups of G, and let Bq be the variety of u

fixed points in BG. The irreducible components of Bq all have the same dimension, and we set

dG(u):=dimB%.

Steinberg proved that

(5.9) 2dG(u) = dimCG(u) -

rkG,

where rk G is the absolute rank of G. Assume that p is a good prime for G. For each unipotent class [u] G U(GF) and

twisted conjugacy class [w] = ? G H1 (F, WG), there is a polynomial

of degree at most dG(u), such that

Qt(u) = QwAo) if cl(T, G)

= [w] (see [20] and the references therein).

The coefficient of tdG^ in Qw,u(t) is

tr[w,H2dG(u)(B%)],

where w acts on the ?-adic cohomology of Bq via the Springer construction (see [21], [14], [16]).

If we take u = 1, then dG(l) = N is the number of positive roots of G and

(5.10) <22,i(0 =

eG(w)tN + lower powers of t,

which is easily seen to be consistent with (5.8). Suppose now that we replace F by Fv for some v > 1. The GF -class of T is

then represented by

(w$y .$~v eWG,

where ? is the automorphism of WG induced by F. Suppose v = 1 mod m, where m is a positive integer divisible by the exponent of the finite group WG xi (?). This

implies that Fv = F on WG and that (wdf <d~v = w for all w G WG. It follows that ?f^F, WG)

= Hl(Fv, WG) and that the class cl(T, G) is the same with respect

to F or Fv.



584 MARK REEDER

Likewise, the class of u in GF or Gpu is determined by the G-conjugacy class

C C G containing u, together with a class in H1(F,AG(C)) or H1 (Fu, AG(C)), where AG(C) is the component group of the centralizer of some F-fixed element in G. As in the preceding paragraph, we may take m sufficiently divisible so that

Fv = F on AG(C) and that the class of u in GF or GF corresponds to the same

class in //^(F, AG(C)). We may choose m so that this holds for every G, since

there are finitely many unipotent classes. Let Qji v be the Green function for T on GF . For m sufficiently divisible as in

the previous two paragraphs and v = 1 mod m we have

QtAu) = QwA<f)

(Note the difficulty with the exceptional class in Fg is avoided since our conditions on m imply that v is odd; see [20, Remark 6.2].)

5.5. A progression of powers of Frobenius. The indices of and terms of the summations in (5.7) depend on F, and we wish to remove this dependence for

infinitely many powers of F, in order to represent the sum in (5.7) as the value of a rational function.

There is a positive integer m such that Fm acts trivially on the finite set I(S) and

the divisibility conditions on m from the previous section hold when G is replaced

by GL or HL for every i G I (S). In particular, m is divisible by the orders of the component groups AL (u) of the

centralizers in HL of all unipotent elements u G HF for every i G I(S)F', and Fm is

the identity automorphism on AL(u) for all such i and u. This implies that for each i e I(S)F and [u] G U(HF), there is a polynomial PLlU(t) G Z[t], of degree equal to

dimG//t(u), such that

(5.11) \CHL(u)F"\=PL,u(qn

for all v = 1 modm. Moreover, each polynomial PL,u(t) is of the form |A,(^)| times a monic polynomial in Z[t].

The above conditions on m also ensure that the indices in the outer two sum

mations in (5.7), as well as the quantity |iV#(?, S)F\ are unchanged if F is replaced

by Fv for v = 1 mod m.

To handle the inner sum, we add more conditions: in the next section we will

define certain subgroups Zj of S, indexed by subsets J C I(S)F. We also insist that m be divisible by \Zj/Z}\ and that Fm act trivially on Zj/Z?j for each J C I(S)F.

5.6. Character sums. In order to interpret the inner sum of (5.7) as a rational

function, we shall replace each summand SF by the group ZF, where

(5.12) ZL:=Z(GL)nS.

It is easy to check that

Zt C Z(HL),

(5.13) S.CZ.C S,

c = cG(zLy.




Let Xv and ry? be the character sums appearing in (5.7). Our aim is to express the sum

(5.14) T^pT ]T Xv(s)v<;(s) ' l

'sesF

as the value of a rational function. Define a partial ordering on I(S) by

t'<i ^ GLCGL>.

Equivalently, we have

tf < t & ZL* ? ZL.

Let

Y, := ZL -

SL

be the complement of SL in ZL.

Lemma 5.1. For every t G I (S) we have

Proof. Let s G YL. Then s e SL> for some ?/ G I(S), with ?' ̂ ?, so s G ZLi. Since

Y?, C ZM we have

Gl = C0g(Zl)cC0q(Yl)?Gs

= Gl,, so ?/ < i.

Conversely, let s G Zt>, with ?/ < i. Note that s G Zb. If s ? YL, then s e SL. This implies that

Gv = Cg(Zli)? cGs = GL,

contradicting i! < i. This proves the lemma. D

For a subset J ? I(S), let

Zj= f)zL,.

There is a polynomial fj e C[t] of degree dimZj such that

fj(Ql/) ? \Zj \, for all v = lmodm.

For v > 1, let

NT- TF" _> TF Ns< SF" _> SF

be the norm mappings. These are surjective. Set

XP = Xv ? N?\ V^:=rkoN^.

Assume that t e J C I(S)F. Then Zj is F-stable and

ZfcZf aZ{GL)Fr\Z{HL)F. Both Xi; and 7/? are defined on the latter group (see (4.4)), so we may restrict them to Zj

. Our conditions on m at the end of section 5.5 ensure that the restricted norm mapping

N*< : Zf -? ZF

is also surjective. This implies, for all integers v = lmodm, that



586 MARK REEDER

Hence for each J, we have

(5.15) E X%\zH"\z) = {Xv,rk)zf fA<t) zezr

Let I(l, S) := {d /(5) : d < ?}. It now follows from M?bius inversion that the rational function

fj(t) JJ f (t)

JCI(l,S)f JLX }

has the property that

(5.16) et,?,?(i):=<X?,r/?)zf + ? (-l)'",|(x?,?fc>z*-i

eW?(<n = 7^7 E xPWvirKs), K ls6sr

for all v = 1 modm. Since dimZj < dimZt for all J ? /(?, 5)F, we have

(5.17) deget?V,?<0.

5.7. Multiplicity as a polynomial. We return to our multiplicity formula (5.7). We have shown that

(5.18) (R^x, R^)hf = E *?fa)?a(<z),

a

where a runs over quadruples a = (?, u, v, ?), with

(5.19) l?I{S)f, [u} U(HF), vej?(cl(T,G)), ? G j?(cl(S,H)), Qa(t)

= ?L,v,?(t) is the rational function defined in (5.16) and ^a(t) is the rational

function defined by

(5.20) ..(?> = /.<?) -?&<"?&<?? iJvH(i,s)i'iip?(()r

Here Q^Lu(t) and Q^Lu(t) are the Green polynomials from section 5.4 and PhU(t) is the polynomial from (5.11).

If F is replaced by Fv with z/ = lmodm, where m is as in section 5.5, the summation indices a are unchanged, so that the rational function

(5.21) M(t):=5^*a(?)eQ(?) a has the property that

(5-22) {R^xM,R^)H^=M(qn, for all v = 1 modm. In particular, M(qu) is an integer for all v = 1 modm.

We next observe that the numerator of each term in M(t) belongs to Z[t], and the denominator of each term in M(t) is an integer times a monic polynomial in

Z[i\. Hence there is a G Z such that

where /(i) and g(i) are in Z[i] and <?(i) is monic. We can therefore write

aM(t)=p(t)+r(t),




where p(t) G Z[t] and r(t) is a rational function of negative degree. On the other

hand,

r(g") = aM{qv)-p{qv) = a(R^xM,R^M)H^ -p{qv) Z

for all v = 1 modm. Since r(^) ?> 0 as i/ ? oo, we must have r(t) =

0, so

M(i) = ip(i). This shows that M(i) is a polynomial, as claimed.

6. Complexity and the degree of M(t)

From now on, the algebraic group G is simple. That is, the center Z(G) is finite and contains every normal subgroup of G. Recall that the complexity S is the minimum codimension of a i?#-orbit in G/B. In this section we will complete the

proof of the first assertion of Theorem 1.4 by showing that 6 is an upper bound on

the degree of the multiplicity polynomial M(t) defined in (5.21).

6.1. A formula for the complexity. In this section we show that ? has the

simplest conceivable formula. Let g and i) be the Lie algebras of G and H. We are

assuming that g is simple. We also invoke Assumption 1.3. That is, we assume that

0 = ()?m, stable under Ad(H), and that there is a nondegenerate Ad(?f)-invariant symmetric form B on m. Hence Ad restricts to a homomorphism

Ad : H ?-> SO(m).

Lemma 6.1. Assume that H ^ G. Then

ker[Ad : H -> S O (m)] = Z(G) H H.

Proof Containment "D" is clear. We prove containment "C". Set

N := ker[Ad : H -* S O (m)] and let n be the Lie algebra of N. We have

n = ker[ad : i) ?

so(m)],

so n is an ideal in i). But [n, m] =0, so n is in fact an ideal in g. Since g is simple and not equal to ?), we have n = 0. Hence N is a finite normal algebraic subgroup of H. By [4, 22.1], iV is central in H, hence Ad(iV) acts trivially on (), as well as on m. It follows that N is central in G. This completes the proof. D

Let B and Bh be Borel subgroups of G and H, respectively. Let U and V be their respective unipotent radicals. After conjugating, we may assume that

BH = SV, B = TU

with

S C T, V CU.

Proposition 6.2. The complexity 6 is given by

f dim G/B - dim BH if H ^ G,

~|0 ifH = G.



588 MARK REEDER

Proof If H = G, the fact that ? = 0 is clear from the Bruhat decomposition. Assume from now on that H =? G. We must show that Bh has an orbit in G/B with finite stabilizers. Let w be the element of WG(T) such that WB Ci B = T.

Then every element of UwB/B can be uniquely expressed as uwB for u eU. For v e V, s e S, we have

vs uwB = v(sus~1)wB.

By uniqueness of expression, vs fixes uwB if and only if v = i/sw-1s_1. It follows

that the projection J3# ? 5 gives an isomorphism from the B#-stabilizer of uwB

to the 5-stabilizer of u~1V in the quotient variety U/V. We will show there exists u eU such that the latter stabilizer is finite.

Denote the Lie algebras of U, V, T, S by u, 0, i,s. The tangent space to U/V at

eV is u/t). We have

a/If = t/s?u/oe?/?, where ? = Ad(w)u is the opposite nilradical of u and ? is the opposite nilradical of t).

Since ker[Ad : S ?> GL(g/\))) is finite by Lemma 6.1, it follows that ker[Ad : S ?? GL(u/ti)] is finite. This latter kernel is the set of common zeros of the roots

$(S, U/V) of S in u/t) (see [4, 8.17]). We have

U/u = (U/u)5 0 Yl (U/f)a. a $(S,i//V)

A vector in u/t> whose a-component is nonzero for every a G $(S, U/V) will there

fore have finite stabilizer in 5. Proposition 6.2 now follows from a basic result:

Lemma 6.3. Let k be an algebraically closed field. Suppose a k-torus S acts on a

smooth irreducible affine k-variety X, fixing a point x G X, so that S acts on the

tangent space TXX at x. If there exists v G TXX having finite stabilizer Sv C 5, then there exists y X having finite stabilizer Sy C S.

This lemma can be proved as follows. Since the torus S acts completely reducibly on the coordinate ring k[X], the argument of Lemme 1 in [15] shows that there is an

5-equivariant morphism ip : X ?> TXX such that <p(x) = 0, and whose differential

d(px : TXX ?

TXX is bijective. The set U of points in TXX with finite stabilizers is

open, and nonempty by hypothesis. Since <p is dominant, the preimage (f~1(U) is

nonempty. If y G (p~1(U), then Sy C S^y), and the latter stabilizer is finite.

Lemma 6.3 can also be proved using a T-equivariant embedding of X in a linear

representation of T. D

6.2. Degree of *<*(?) We return now to our rational function

..c)-/.?- Q?"m"M

\NH(t,S)F\\PL,u(t)\' We have

(6.1) deg|Pt|t4(t)| = dim CH?, deg|/t(t)| =dimZ?.

From section 5.4 and equation (5.9) we find that

deg ̂a(t) < dim ZL + dGt (u) + d#t (i?) -

dimChl(u) (6'2) = dim ZL + \ [dim CGl (u)

- dim G^ (u) - rk G - rk ff]

.




The fixed point spaces gs, f)s,ms are the same for any s e SL; we denote them

by gL, f)?,ttx?. Thus we have an Ad(i2")-stable decomposition

gb = t)L 0 m,

and

dim CGl (u) - dim Chl (u) = dim m^

(6.3) <dimm, =

dimGGt(l)-dimGHt(l).

Define

SL := dim ZL + dim BGt ? dim Bhl

? dim S

(6'4) = dim Z? + |[dimm? - rkG - rk H].

For example, if ?o is the minimal element of I(S), then

(6.5) Gi0 = G, ZL0

= Z(G) n 5 and m?0

= m.

Since Z(G) is finite, Proposition 6.2 implies that

(6.6) 6L0 =

|[dimm - rkG - vkH]

= ?, ifH^G.

Lemma 6.4. We have deg^a(t) < ?L, with equality only if u = 1.

Proof The inequality follows from (6.2) and (6.3), and the last assertion follows from section 6.1. D

We now seek a bound on deg \fra which is independent of ?. We will show that

6L < 6, and that equality holds only in rather special circumstances. Let m[ be the sum of the eigenspaces of Ad(s) in m with eigenvalues ^ 1, for

any s e SL. Since det Ad(H) = 1 on m, the dimension dimm' is even. We have

m = m? 0 m[, the form B is nondegenerate on m[, and Ad : H ?> S O (m) restricts to a homomorphism Ad6 : HL ?> SO(xn[).

Lemma 6.5. For every i G I(S) we have 6L < ?. Moreover, if H ^ G, then the

following are equivalent:

(1) SL = 6; (2) dim(ZL) = \dimm,L; (3) AdL(ZL) is a maximal torus in SO(m[).

When these hold, the derived group of HL acts trivially on m[.

Proof. lfH = G, then 6 = 0 and

6L =

dimZ?-dimS<0.

From now on assume H ^ G. From (6.4) and (6.6), we have

(6.7) 6 - 6L = \ dimt< -

dimZ,.

Now, the group

NL := ker[Ad, : Zt ?. SO(m[)} is finite. Indeed, since ZL C Z(GL), it follows that NL centralizes m?, as well as m[.

Hence we have

NL C ker[Ad : H ?> S O (xa)] = Z(G) O E, the latter equality being from Lemma 6.1. Hence NL C Z(G), and the latter is finite since G is simple.



590 MARK REEDER

Since Ad(Z?) is a torus in SO(m[) and \ dimm^ is the dimension of a maximal torus in SO(m[), this proves that both sides of (6.7) are > 0 and that (l)-(3) are

equivalent.

For the last assertion, recall that ZL c Z(HL). If (l)-(3) hold, then Ad?(i^) centralizes a maximal torus in SO(m[), hence is contained in that torus. D

With this lemma, the first assertion of Theorem 1.4 has been proved.

6.3. A remark on the multiplicity formula. The formula (5.21), as written, contains more terms than are necessary. For, if we write

*a(t)Qa(t) = Pa(t) + Ra(t), where Pa(t) is a polynomial and deg?a(?) < 0, then

M(t) = Y, Pa(t) and J2 R^ = ?> a a

since M(t) is a polynomial. From (5.17) and Lemma 6.4 we have degPa < SL, where a =

(t, u, v, ?). It follows that

(6-8) <i??x, RIv)hf = M(q) =

? Pa(q),

where the sum is over just those a ? (i, u, v, ?) such that 5L > 0.

7. The leading term of M(t)

We have shown that the multiplicity polynomial M(t) has the form

M(t) = At6 + (lower powers of t).

In this section we find an explicit and effective formula for the leading term A of

M(t). Recall from (5.21) that

M(t) = J2^a(t)ea(t),

where a runs over quadruples (t, u,v,?) as in (5.19),

..W-/.M- Q':'{t)Q"m \?H{L,S)F\\PL,u{t)\

and

e?(t) = (Xv,rk)zF+ E (-1)'J|(x.,^)zjy^. JCI(l,S)f Jl^ )

By Lemmas 6.4 and 6.5, only quadruples a with u = 1 and 5L = ? contribute to the leading term; henceforth we assume a is of this form. As a power series in t, we then have

tya(t) = Aat? + (lower degree terms),

where

Aa = [ZF : Zf] {-^ rk(Gt)+rk(T)+rk(i/t)+rk(S)

(7-1} "?-^ "' ' \?H(t,S)F\

At first glance, each function ?a(t) could contribute many terms to A, coming from various ?/ < ? with dimZ?/ = dimZL, since ZL may be disconnected. We now

show that in fact @a(t) contributes only one term.




Lemma 7.1. If 5L = 6 and ?/ < i, then dimZL> < dimZ?.

Proof. If H = G, we have ?L = dim ZL ? dim S < 0 = 6 with equality iff ZL = S.

The lemma holds since S is connected. Now assume H ^ G. Suppose ?L = ? and ?/ < t, yet dimZ^ = dimZL. Then

(7.2) Z? ? ZL, C Z?.

From Lemma 6.5, the image AdL(ZL) is a maximal torus in SO(m[). It follows that

Ad6(Z?) = Ad(Z?).

Thus, for each z G ZL there is z0 G Z? such that

z1 := ^^o"1 G ker[Adt : Z6 -> SO(m[)].

By Lemma 6.1, we have z\ G Z(G) D H. Hence

(7.3) z = zqZ! GZ?-(Z(G)n#).

We have shown that

(7.4) ZL = Z?.{Z(G)nH). Now SL is stable under multiplication by Z(G) D H. Moreover, SL is open in ZL, so SL meets some connected component of ZL in an open dense set. But then (7.4) implies that SL meets every connected component of ZL in an open dense set.

Likewise, SL' meets some component of ZLi in an open dense set. By (7.2), every such component of ZL> is also a component of Zt. Therefore SL and SL* meet a common component of ZL in a dense open set. This implies that SL fl Sy is

nonempty; hence i ? l', contradicting ?' < t. D

As an aside, we mention the following consequence of (7.4) which simplifies our

eventual formula for A when G is adjoint.

Lemma 7.2. Suppose G is simple adjoint. If ?L = ?, then ZL is connected.

Return now to 6a(t). For each J C I(l,S), the subgroup Zj is contained in some ZL> with i' < i. Lemma 7.1 implies that

deg/j(f)<deg/A(t), which shows that the leading term of 0a(t) has the following simple form.

Corollary 7.3. Let a = (l, l,v,?) be a quadruple appearing in M(t) with 6L = 6.

Then

9a(oo) =

(xviVjzr

From (7.1) and Corollary 7.3 we get the following expression for the leading term A.

Proposition 7.4. The leading term A of M(t) in Theorem 1.4 is given by A =

^2L AL, where i runs over those t G I(S)F with ?L = 6, and

L~{ ' \?h(l,s)\ y*-?

*?

In the last summation, v and ? run over Jq1(c\(T^G)) and j^1 (cl(S, H)), respec tively.



592 MARK REEDER

As a simple illustration of Proposition 7.4, we show how it reduces to the Deligne Lusztig inner-product formula [8, thm. 6.8] when G = H. For t e I(S), we have then

5L = dimZ(G?) - dimS < 0 = 6

with equality iff GL = S = HL. This means t is the maximal element of I(S)F, and

M(t) = A = AL is the inner product (Rt,x, ̂ s,t/)gf

By Proposition 7.4, if T is not GF conjugate to S, then j?1(cl(T, G)) = 0, so

A = Q. Otherwise we may take S = T, and the fiber of jGc over cl(5, G) is the

singleton {v} corresponding to the class of S in itself. We have

Xv= ? WX, Vv= ? wr), ?G(L,S)F = WG(Sy wewG(s)F wewG(S)F

\F

and the result:

(Xv,riv)sF

\WG(S)F\ is the original Deligne-Lusztig formula for

(Rsai Rgv)GF.

8. Optimality

Recall that G is simple. In this section we show that the degree S is optimal. We

may assume that H ^ G. Let T C G and S C H be arbitrary F-stable maximal tori. We will show that for sufficiently large q, there are characters \ G Irr(TF) and rj G Irr(5F) such that the leading coefficient A is nonzero. In fact, we can take

77 to be the trivial character. For each t G I(S)F with AL ^ 0, the fiber Jq1(c\(T, G)) is nonempty. This means

that ZL is GF-conjugate to a subgroup ZL C T. There are only finitely many of these subgroups Z?. Recall from (6.5) that ?0 G I(S)F is the minimal element, for which Zi0

= Z(G) H S. If dimZt = 0 and SL = S, then Lemma 7.1 implies that

i = ?o- Hence, if 1 ̂ to, the torus T/Z?L has strictly smaller dimension than that of

T, so that Irr ?TF/Z0LF J is a polynomial in q of degree strictly less than dimT.

Hence for sufficiently large q there are characters x G Irr(TF) which are trivial on

ZF and nontrivial on every ZF for t ̂ to. We call these \ very regular. For very regular \ and t such that AL ^ 0, we have

(8-1) <X?,l)zr = \l ^7?' [0 if t ̂ ?o

It follows that for \ verv regular, and 77 = 1, the coefficient A of t? in M(t) is

given by

(8.2) A = eG(x)eH(y), where x G cl(T, G) and y G cl(5, H).

Let 1? be the automorphism of Wh induced by F and let ip be the character of an irreducible representation of (#) x W/j. For each y G W#, choose an F-stable torus Sy in H such that y G cl(Sy,H). We have then a class function i?if of HF defined by

1 H| yew?




For example, the trivial (1) and sign (e#) characters of W# extend to (#) ix Wh

(trivially on #). It is known (cf. [5, 7.6]) that ?f = 1H and RfH

= StH are

the trivial and Steinberg characters of HF, respectively. For very regular x> (8.2) implies that

(8.3) (i?^x, R%)hf = eG(x)(eH, ̂)wH ^ + (lower powers of <?).

In particular, we have

(eG(x)R%x, Stn)HF =

Q6 + (lower powers of <?),

while (eG(x)R!j,x, Ih)hf has degree < ?. This last result is to be expected, in view

of the results in [2] and [17].

9. RESTRICTION FROM S02n+1 TO 502n.

We return to the situation at the beginning of the introduction. So p > 2 and

(V,Q) is a (2n + l)-dimensional quadratic #-space, defined over f, with Frobenius F. Fix v e VF with Q(v) ^ 0, and let U be the orthogonal space of v in V. We take

G = SO(V), H = GV = SO(U), with f-structure on both groups induced from that on V. Assumption 1.3 holds: we may identify the quadratic spaces (m, B)

= (U, Q).

Let T, S be F-stable maximal tori in G and H respectively, and let \ G Irr(TF), rj G Irr(5F) be characters, which for the moment are arbitrary.

We have

? = dimBG ? dim Bh

? dimrkff = n2 ? (n2 ?

n) ? n = 0.

From now on, we only consider t G I(S)F with ?? = 0. Since G is adjoint, each such ZL is connected, by Lemma 7.2. Proposition 7.4 gives the multiplicity formula

/ i\rk(Gt)+rk(Ht)

(9.1) {-l)* s(B99X,Rgtfl)H,= ? (

7^ , ̂ , -E<X?.tt>*f,

where v and ? run over JqX(c\(T, G)) and ^^(c^S, i/)), respectively.

The connectedness of ZL implies that ?1 is not an eigenvalue of any s e SL. The last assertion of Lemma 6.5 implies that s e SL has distinct eigenvalues on V/Vs. It follows that

(9.2) GL = SO(Vs) x ZL, HL = SO(Us) x ZL.

Note that dirnF3 is odd, say dim Vs ? 2a + 1. The decompositions (9.2) imply that if two F-stable maximal tori in GF are

GF-conjugate, then they are GF-conjugate, and likewise for H. In other words, we have

\j^(ci(T,G))\-\j^(d(S,H))\<l. Hence the inner sum of (9.1) has at most one term.

To make this precise, we recall that tori in orthogonal groups are described

by pairs (A, A7) of partitions. We write partitions as A = (jXj), meaning that



594 MARK REEDER

A has Xj parts equal to j, and set |A| =

^ jAj- We have pairs of partitions

(v,v'), (A, A'), (/x, //) such that

z? - IM )"'x (& )"'. m + m=? - ?> 3

(9.3) ^^rj^X)Ajx^)Ai'' |A| + |A'| = n,

3 We have \j^(cl(T,G))\ |j^1(cl(S',i?))|

= 1 precisely when

(9.4) Vj < Xj,fij and i/j

< A^,/^

for all j. We assume (9.4) holds from now on. Note that if T and S are anisotropic, then Xj

= ?ij

= i/j

= 0 for all j. We count the number of t in the sum (9.1) giving rise to a fixed pair of partitions

(v, v'). For 5 G SF, consider the components (s^i,..., SjIXj ; s'jX,..., s\' , ) of s in the

jth block

(?rx(f/5csp. Then ? is determined by the pair of subsets

{k e [l,Hj] : sjk = 1}, {*' e [1,1$ : s'jk, = 1}.

It follows that there are

elements ? G I(S)F giving rise to (z/, */), where

?=-n(s> (::)=n? From equations (2.3) and \j'^1(c\(S,H))\

= lwe have

p.? i?.hs)'i - Jgaa

- Q (?) n^Do/ix^r-i^)-;.

Using Lemma 3.1 for GL and HL, we find that

(9.6) /_1\rkGi+rkift = /^rkG+rktf+E^

(One can also arrive at (9.6) by decomposing UF into irreducible fZF-modules, and

calculating discriminants.) Finally, we must calculate the pairing (Xv,W?)zF- We may conjugate T and S

to arrange that ZL C T D S. Then

Xv = \Wr (T)F\ ?~< (Xx)\z? Vc = W?T(SW\ ̂ (^)l^ ]VVG^ }

lxewG(T)F \vyHt\o)

\yeWH{s)F

We now assume that \ and rj are regular, in the sense that they have trivial stabilizers in WG(T)F and Wh(S)f, respectively.

On the jth block (f*)^ x (f^)^ of SF, we have

77 = ffoi ? ? r]jfik ?Vji?'"? Vjv'k




Likewise, on the jth block (f*)A^ x (f^)^

of TF, we have

X = Xji ? ' ? Xj\j ? Xji ? ' ' * ?

Xja;. Define

?9 7) /j = {^ G [1, fij] : rjjk G Tj {xj7, Xj?-1}, for some l G [1, A,]},

I'j =

{k'e [1, /xJ] : r/jk,

= T2j

- x^, for some l G [1, Aj]}.

For every pair of subsets

\k\, , k1/j} C Ij, {k-l,--' , kvi} C Ij,

each of the

conjugates of the character

Vjk, ? ? r?^. ? ^fc/ ? ? rjjKj

contributes exactly once to the pairing (Xvj%)zfj by the regularity assumption

(1.3). It follows that

(9.8) (Xv,rk)zr = u (lIJl) (l5l)(^)!(^)!(2j)^(2j)^. j ^ 3 / \ j /

Set

e ._

/_-j\rkG+rkT+rkH+rkS

Inserting (9.5), (9.6) and (9.8) into (9.1), and summing over all (v, v') satisfying (9.4), we get

v,v' j ^ ^ / \ j; /

(9.9) =n 1^1 /ir l\\ /I'i

?? S-K'l) ^=0 ^ "'

J 2r if

/j is empty for all j,

10 otherwise,

where

3

If either T or 5 is anisotropic, then r = 0. This proves Theorem 1.2.

10. RESTRICTION FROM S07 TO G2

The previous situation had ? = 0. We now consider a case where <J = 1. The

simplest such case is G = G2, H = SL3, which we leave to the reader. Here we take G = SO7, H = G2, embedded in G via the irreducible 7-dimensional

representation V of G2. We have

? = 9-8 = 1.

We assume p > 5.



596 MARK REEDER

We will calculate the multiplicities (Rt,xi Rs,t])hf, using the formula of section 6.3. We do not need any detailed knowledge of Green functions, beyond the general facts about their degrees and leading terms that we have already used.

Let a, ? be simple roots of a maximal f-split torus So in H, with a short. The nonzero weights of H in V are the short roots of So- We view the maximal f-split tori T0 and So as

T0 = {(x,y,z)e33 : xyz^O}, S0 = {(x,y,z) G# : xyz = l},

in such a way that the coordinate functions ei,e2,e3 on T0 restrict to the roots 2a + ?, ?a, ?a ?

? on So- In this realization, the simple co-roots of So in H are

a(t) = (t,t~2,t), ?(t) = (l,t,t~1), and the corresponding simple reflections ra, r? in the Weyl group Wh act by

ra ' (ti,t2,ts)

= (t% ,t^ ,f{ ), r? (ti,t2,t3)

= (ti,t3,t2).

Since So contains regular elements in T0, it follows that Wh is a subgroup of WG. If WG is realized as the group of the cube, then Wh is the subgroup preserving a

diagonal of the cube; as coset representatives for Wg/Wh we may take the identity and each coordinate sign change.

Let T, S be F-stable maximal tori in G and H, corresponding to the conjugacy classes of x G WG and y G Wh, respectively. Let x G Irr(TF), rj G Irr(SF).

We will use the refined multiplicity formula of section 6.3. We first tabulate the

pairs (t, u) in H, with t G I (So) and u G HL, for which

(10.1) dim ZL + dGt (u) + dHi (u) - dim CHc (u) > 0.

We find four types as shown:

type

(1.1.1) (U,*-1), t* = t??i

(M,*-1), t<* = t~^?i regular

1, u0

The middle column shows a typical element in So for each type of t. There can be more than one t of the same type. Here uo G HF is a long root element, which has Jordan partition 1322 on V.

From equation (5.18), we have

M(q) = Ma(q) + Mb(q) + Mc(q) + Md(q), where each term on the right is the sum

over those a whose t component is ??-conjugate to an t of the corresponding type a, b, c, d in the table above. Taking the polynomial part of each sum, as in section

6.3, we have

(10.2) M(q) - Pa(q) + Pb(q) + Pc(q) + Pd(q). We now calculate each of the four terms on the right side of (10.2).

Type a: The maps jGt and jnL are the identity. The centralizers of uq in G and H are both connected. Let <\>G, </># be the Springer representations of WG and

Wh corresponding to u0, let pG, pn be the reflection representations, and let eG, en be the sign representations.




From [5, 13.3], we find that 4>G is the unique two-dimensional representation of

WG = S$ k {?l}3 which is irreducible on S3 and nontrivial on {?1}3, and </># is

the one-dimensional character of Wh given by </>#(ra) = ?1, <t>H(r?)

= +1. In the following calculation we write R\ ~ R2 for rational functions Ri such that

deg(fii -

R2) < 0. We have

M ,?x _ Qg(l)Qf(l) , Q^{uo)Q^(uo) a[Q) \HF\ +

\CH(uo)F\ '

The uo term has degree zero, so we may replace it by its leading term:

^)^l^f(1)+M^(?) Since

\TF\ =

defcfa -

x) =

q3 -

pG(x)q2 + ,

\SF\ =

det(q -y)=q2- pH(y)q + ,

it follows that

tG{x)eH{y)-j-^pj-

-

-j^|

= (g6-l)(g4-l)(92-l) q6\TF\\SF\

~q + pa{x) + PH{y) We get

(10.3) Pa(q) = eG(x)eH(y) [q + Pg{x) + pH(y)} + MtfMv)

Type b: For i ? I(S)F of type b, the elements of SF are ii-conjugate to elements

of the form s = (1, t, f1) ? S0 for some t ? fx, t2 ̂ 1. We have

C = S03 x GL2, Ht = GL2.

The stabilizer W'H of s in Wff is generated by the reflection

r0 := rar?rar?ra ? WH.

The number of 1 ? I(S)F of type b is given by

'3 i?y = l, 1 if [y] = N, 0 otherwise,

where [ ] denotes a conjugacy class in Wh- We have

(10.4) vh(y) :=

\nh(l,s)f\ ?"'

\cWH(y)\ 2

for y G {l,r0}. The roots of T0 vanishing on s are e\,e2 + e$- The corresponding reflections

ri, T2 G WG generate the stabilizer WfG of s in W^. For s e Sf, the element ys of

section 2 is ys = 1. Hence, the mapping

of (2.2) is induced by the inclusion W'G c-^ WG; its image consists of the four

classes in WG represented by elements in WG , as shown in the following table:



598 MARK REEDER

H\(F,W'G) H^Wg) [I3,-]

r2

[12,-]

rir2

[2,1] Set

Wb:=W^t xW'Hl.

Using section 7 and taking (10.4) into account, it follows that

(10.5) Pb(q) = \eG(x)eH(y)(Xx, Vy)zF if (x,y) is (WG x W#)-conjugate to an element of Wb, and Pb(q)

= 0 otherwise.

Here we have written \x instead of \v> where {v} =

i?1([x]), and likewise for r\y.

Type c: For t G I(S)F of type c, the elements of SF are ii-conjugate to elements of the form s = (1, t, t~x) G So for some t G fy, t2 ̂ 1. We have

GL = S03 x U2, HL = U2.

The groups

W?t =

<n,r2), W'H? =

{r0)

are as in type b. The number of t G I(S)F of type c is given by

(10.6)

As before, we have

My) = {

My)

if 2/ = -1, if [y] = Ko], otherwise.

\Cw>?Sv)\ _

\?H{i,S)F\ Vc{y) \CWH{y)\ now for y G {?1,

? ro}.

Let r be the reflection about e2 ? es. For s G SF, the element ?/s of section 2 is

ys = r. Hence, the mapping

jG?:Hl(F,WG?)^H\F,WG)

is induced by the map x \?> xr, as shown in the following table:

Hl(F,W'Gi) Hl(F,WG) [12,-]

T\

[2,1] r2 7*17*2

h, ld

Set

Wc := W?tr

x W^r.

Using section 7 and taking (10.6) into account, it follows that

(10.7) Pc(q) = ?eG(z)e^y)<X*,%)zf if (x,y) is (Wg x W#)-conjugate to an element of Wc, and Pc(q)

= 0 otherwise.

Again, we have written \x instead of \v, where {v} =

jGl([x\), and likewise for rjy. Type d: In this case SL contains regular elements in G, so HL = ZL = S, and

GL = CG(S) is a maximal torus in G. Hence Pd(q) 7^ 0 only if T is GF-conjugate to CG(S). The mapping S h-? CG(S) is given, in terms of conjugacy classes in Wh and WG, in the following table:

Cg(S) "F7=r [2,1] [12,-]

-1

[3,-] h,3]




Denote the embedding WH ^Wg by yy-^y.

Let

Wd = {(y, y): ye WH} cWGx WH.

For (x, y) G W?, we may assume that T = CG(S). Then

e?(?) ~ E (WX,vri)s* = \WH(S)F\ E (WX,ri)sr wewG(T)F wewG(T)F

vewH{S)F

Since

*a(?) - \wH(sy

it follows that

(10.8) Pd(q) = E <WX, ^ = (X? v)sF wewG{T)F

if (x,2/) is (WG x W/f)-conjugate to an element of Wd, and Pd(<?) = 0 otherwise.

10.1. Cuspidal multiplicities. From now on we assume that our tori T^S are

anisotropic, and that the characters \ and rj are regular. We will make the multi

plicities computed above more precise. The elliptic classes in W(Bn) are those of the form [?, A], where A is a partition

of n. So in WG we have three elliptic classes:

[-,3], [-,12], [-13]. The first of these is the Coxeter class and the last is {?1}.

In Wh we also have three elliptic classes, represented by the powers

cox, cox2, cox3 = ? 1

of a Coxeter element cox = rar?. Via the embedding Wh ^-> WG, the elements

?1, cox of Wh are also the ?1 and Coxeter elements of WG. Let x e cl(T, G), y G cl(S, H). Note that Pb(q)

= 0. Combining formulas (10.3), (10.7), (10.8), we get the multiplicity formula

(10.9) -<?r,x, RIv)hf =q + Pg(x) + pH(v) -

<t>G(x)(l>H(y) + a(x, y) where

'|<x-i>r?-i)zf -

(x-wn)zg if x = y = -i,

-(Xcox, rj)ZF ]fx = y

= cox,

0 otherwise. (10.10) a(x,y)

= <

Here we have written Zc for ZL, when t has type c, and likewise for Z? = S. The numbers

(X-i,r7_i)ZF, (x-i,r)-i)ZF, (Xcox,rj)ZF are calculated explicitly in (10.11), (10.13) and (10.16) below.

We calculate pG(x) -j- Ph(v) ?

<t>G(%)<?>H(y) from the character tables of WG and

Wh and then set A(x) =

a(x,x) for x G {?l,cox} to arrive at the multiplicities in Table 1.

Note that S_i has regular characters only for q > 5 (which we have already assumed), and T_i has regular characters only for q>7. For g = 5we get Table 2,



600 MARK REEDER

y\x hiH-i] 12] -,3] cox

[coxa] = [-1] q-7 + A(-l) 9-3 q-\

cox* 9-2 9-2 9-2

cox 9-4 9 + 2 - A(cox) Table 1.

-{R^^R^h? for 9 > 7

[-,13] = [-!][[-, 12] I [-,3] = [cox] y\x [coxa] = [-1]

cox"

cox 6 or 7

Table 2. -(R$xa,R** )hf f0rq = b

where the dichotomy in the (cox, cox) entry arises from the fact that A (cox) = 0 or

1, depending on \ and rj (see below). The rest of this section is devoted to the explicit calculation of A(?1) and A (cox). A(-l): We identify

Tf! =

t?)3, SF, =

{(*, y, z) e TF, : xyz = 1},

X = Xi ? X2 (8) X3, 77 = Res5Fi (771 (8) 772 ? ^3), with x*, ^ G Irr(f^).

Recall that

^(-1) =

\(X-i^-i)zc -

(x-i,r/)Zd.

We have ZF = {(1,M_1) : ? e f?} and

1 (10.11) (x-i,f?-i>zf =2X; ? + ?+X^ +

XiXj ' 772 r/3 771 /fi

For ? of type d, we have Z?? = S_i. The W(3-orbit of x breaks up into four

Wh -orbits:

(10.12)

Oo = VJ/tf-(xi<S>X2?X3),

?i=WH-(xi?X2?X3),

<92 = WH-(xi?X2?X3),

03 = WH '(Xi?X2?X3).

These restrict to Wjj-orbits ?0,...,?3 in Irr(S^). Even though the orbits Oi consist of W^-regular characters, the characters in Oi need not be W#-regular.

Moreover, it can happen that Oi = Oj for i ̂ j. In any case, formula (10.8) gives

(10.13) (X-uV)zF = |{* G [0,3] : 77 e OJ|.

We illustrate with g = 7. The unique regular Wa-orbit in In^T^) contains the

character x ? C ? C2 ? C3, where ? is a faithful character of f?

? Ms- There are

two Wh -orbits of regular characters 77,7/ in In^S;^), distinguished as follows: 77




belongs to the restrictions ?\ ? ?3, and 77'

= Res5F (( <g> 1 ? ( 2) does not extend

to a regular character of TFX. Formulas (10.11) and (10.13) give

(X-uV-i)zF = 12, (x-i,TM>Zf

= 2>

(X-i,V-i)zF = 10, (x-i,<i>zf=0.

From Table 1 we get

(1015) -<A?_liX,Af_1,,>^

= 7-7+?.12-2 = 4,

-(?t-lx? R"-i?')hf = 7 - 7 + i 10 - 0 = 5.

A (cox): Here the only relevant type is d. We identify

2?=?=Mf.x?=*f3x]. ^x=(f?r+i, W(2cox)F

= W(Scox)F is cyclic of order six, and acts on TFox and SFox via Gal(fe/f).

If X G Irr(TCQX) and 77 e Irr(S^x) are both regular, and 77 appears in the restriction of some Galois conjugate of x, then the restriction map Irr(TCQX) ?>

Irr(S^x) maps WG

- x bijectively onto Wh rj- Formula (10.8) gives

/lnlfiN a/_n , ?v ?1 if ̂ = Res(^x) for some w G Gal(f6/f), (10.16) A(cox)

= (xcox, r\)zF

= \ n ., . d 10 otherwise.

Hence the (cox, cox) entry in Table 1 is made precise:

(10.17) iuG uH v _ ( 9 + 1 if f? = Res(^x) for some w G Gal(f6/f),

-<?r-,x'???,i|)H'-< +2 otherwise

REFERENCES

[1] D. Akhiezer and D. Panyushev, Multiplicities in the branching rules and the complexity of ho

mogeneous spaces, Moscow Math. Jour., 2 no. 1, (2002) pp. 17-33. MR1900582 (2003d:14057)

[2] E. Bannai, N. Kawanaka, and S.Y. Song, The character table of the Hecke algebra

K(GL2n(Fq),5p2n(F?)), J. Algebra, 129 (1990), no. 2, pp. 320-366. MR1040942 (91d:20052) [3] M. Beynon and N. Spaltenstein, Tables of Green Polynomials for exceptional groups, War

wick computer science centre report no. 23 (1986).

[4] A. Borel, Linear algebraic groups, second enlarged edition, Graduate Texts in Mathematics, vol. 126, Springer-Verlag, New York, 1991. MR1102012 (92d:20001)

[5] R. Carter, Finite groups of Lie type. Conjugacy classes and complex characters, John Wiley & Sons, Ltd., Chichester, 1993. MR1266626 (94k:20020)

[6] _, Semisimple conjugacy classes and classes in the Weyl group, Jour, of Alg., 260

(2003) pp. 99-110. MR1973577 (2004b:20071) [7] S. DeBacker and M. Reeder, Depth-Zero Supercuspidal L-packets and their Stability, preprint

2004.

[8] P. Deligne and G. Lusztig, Representations of reductive groups over finite fields, Ann. of

Math., 103 (1976) pp. 103-161. MR0393266 (52:14076) [9] M. Geck, G. Hiss, F. L?beck, G. Malle, and G. Pfeiffer, CHEVIE - A system for comput

ing and processing generic character tables for finite groups of Lie type, Weyl groups and

Hecke algebras, Appl. Algebra Engrg. Comm. Comput., 7 (1996) pp. 175-210. MR1486215

(99m:20017) [10] B. Gross, On the centralizer of a regular, semi-simple, stable conjugacy class, Electronic J.

of Representation Theory (2005). MR2133761 (2006a:20089) [11] B. Gross and D. Prasad, On the decomposition of a representation of SOn when restricted

to SOn-!, Canad. J. Math., 44 (1992), pp. 974-1002. MR1186476 (93j:22031)

[12] B. Gross and M. Reeder, From Laplace to Langlands via representations of orthogonal groups, Bull. Amer. Math. Soc, 43 (2006), pp. 163-205.



602 MARK REEDER

[13] T. Hagedorn, Multiplicities in Restricted Representations of GLn(Fq), Un(F 2), SOn(Fq), Ph.D. thesis, Harvard University (1994).

[14] D. Kazhdan, Proof of Springer's hypothesis, Israel J. Math., 28 (1977), pp. 272-286.

MR0486181 (58:5959) [15] D. Luna, Sur les orbites ferm?es des groupes alg?briques r?ductifs, Invent. Math., 16 (1972),

pp. 1-5. MR0294351 (45:3421) [16] G. Lusztig, Green functions and character sheaves, Ann. Math., 131 (1990), pp. 355-408.

MR1043271 (91c:20054) [17] _, Symmetric spaces over a finite field, Grothendieck Festschrift, III Birkh?user(1990),

pp. 57-81. MR1106911 (92e:20034) [18] T. Shoji, On the Green polynomials of a Chevalley group of type F4, Comm. in Algebra, 10

(1982), pp. 505-543. MR0647835 (83d:20030) [19] _, On the Green polynomials of classical groups, Invent. Math., 74 (1983), pp. 239-267.

MR0723216 (85f:20032) [20] _, Green functions of reductive groups over a finite field, Proc. Symp. Pure Math.,

(47), Amer. Math. Soc. (1987), pp. 289-301. MR0933366 (88m:20014)

[21] T. A. Springer, Trigonometric sums, Green functions of finite groups and representations of

Weyl groups, Invent. Math., 36 (1976), pp. 173-207. MR0442103 (56:491)

[22] _, A construction of representations of Weyl groups, Invent. Math., 44 (1978), pp. 279

293. MR0491988 (58:11154) [23] _, A purity result for fixed point varieties in flag manifolds, J. Fac. Sei. Univ. Tokyo

Sect. IA Math. , 31 (1984), pp. 271-282. MR0763421 (86c:14034)

[24] T. A. Springer and R. Steinberg, Conjugacy Classes, Seminar in algebraic groups and related

finite groups, Lecture Notes in Math., 131 (1970), pp. 167-266. MR0268192 (42:3091)

[25] R. Steinberg, On the desingularization of the unipotent variety, Invent. Math., 36 (1976), pp. 209-224. MR0430094 (55:3101)

[26] B. Srinivasan, Green polynomials for finite classical groups, Comm. in Algebra, 5 (1976),

pp. 1241-1259. MR0498889 (58:16905) [27] E. Thoma, Die Einschr?nkung der Charaktere von GL(n,q) auf GL(n-l,q), Math. Z., 119

(1971), pp. 321-338. MR0288190 (44:5388)

Department of Mathematics, Boston College, Chestnut Hill, Massachusetts 02467

E-mail address: reedermafibc.edu



on the restriction of deligne-lusztig characters

Documents