high-resolution analysis of the least distortion of fixed ... · least distortion of fixed-rate...
TRANSCRIPT
Z-1
Hig
h-R
esol
utio
n A
naly
sis
of th
e
Leas
t Dis
tort
ion
of F
ixed
-Rat
e V
ecto
r Q
uant
izer
s:
Zad
or's
For
mul
a
We
wis
h to
use
hig
h-re
solu
tion
anal
ysis
to d
eter
min
e an
app
roxi
mat
e fo
rmul
afo
r th
e di
stor
tion
of a
n op
timal
k-d
imen
sion
al q
uant
izer
with
dim
ensi
on k
and
size
M,
assu
min
g M
is
larg
e. T
hat i
s, w
e w
ish
to fi
nd a
n ap
prox
imat
efo
rmul
a fo
r th
e O
PT
A fu
nctio
n δ
(k,M
) w
hen
M i
s la
rge.
To
do th
is, w
e be
gin
with
the
Ben
nett
inte
gral
app
roxi
mat
ion
to d
isto
rtio
n:
D
≅
1
M2/
k
⌡⌠
m(x
)λ2/
k
(x) f
X(x
) dx
A fi
rst t
houg
ht is
to fi
nd fu
nctio
ns λ
( x)
and
m(x
) th
at m
inim
ize
Ben
nett'
sin
tegr
al, a
nd th
en s
ubst
itute
thes
e in
to th
e in
tegr
al to
find
the
leas
t pos
sibl
edi
stor
tion.
Thi
s is
inde
ed a
rea
sona
ble
appr
oach
, whi
ch w
e sh
all t
ake.
How
ever
, we
need
to ta
ke in
to a
ccou
nt th
e fa
ct th
at λ
(x)
and
m(x
) a
re n
otar
bitr
ary
func
tions
. R
athe
r th
ey r
epre
sent
a q
uant
izat
ion
dens
ity a
nd a
nin
ertia
l pro
file,
res
pect
ivel
y. T
hus,
we
mus
t mak
e su
re to
min
imiz
e B
enne
tt's
inte
gral
ove
r th
e se
t of f
unct
ions
λ(x
) a
nd m
(x)
that
are
pot
entia
l qua
ntiz
atio
nde
nsiti
es a
nd in
ertia
l pro
files
, res
pect
ivel
y. O
n th
e on
e ha
nd it
is e
asy
to s
ayw
hat f
unct
ions
are
pot
entia
l qua
ntiz
atio
n de
nsiti
es is
--
any
non
nega
tive
Z-2
func
tion
that
inte
grat
es to
one
is a
pot
entia
l qua
ntiz
atio
n de
nsity
. O
n th
e ot
her
hand
, it i
s m
ore
chal
leng
ing
to s
ay w
hat f
unct
ions
are
pot
entia
l ine
rtia
l pro
files
.W
e w
ould
obv
ious
ly li
ke m
(x)
to b
e as
sm
all a
s po
ssib
le.
But
it is
not
eas
y to
say
how
sm
all i
t can
be.
In w
hat f
ollo
ws
we
first
sho
w th
at th
e be
st in
ertia
l pro
file
is a
con
stan
t. W
ene
xt fi
nd th
e be
st q
uant
izat
ion
dens
ity.
Fin
ally
we
subs
titut
e th
ese
into
Ben
nett'
s in
tegr
al to
det
erm
ine
wha
t is
calle
d Z
ador
's fo
rmul
a as
an
appr
oxim
atio
n to
the
OP
TA
func
tion
δ(k
,M).
Z-3
Bes
t Ine
rtia
l Pro
file
The
follo
win
g fa
ct a
nd it
s pr
oof w
ere
disc
usse
d in
cla
ss.
It sh
ould
be
follo
wed
with
a d
iscu
ssio
n le
adin
g to
m* k.
How
ever
, I h
ave
foun
d a
mor
e di
rect
app
roac
hto
det
erm
inin
g th
e be
st in
ertia
l pro
file
and
m* k.
It r
epla
ces
Fac
t 1 w
ith F
act 2
.T
houg
h fo
r co
mpl
eten
ess,
I ha
ve le
ft F
act 1
her
e, o
ne m
ay s
kip
ahea
d di
rect
ly to
the
"Mor
e D
irect
App
roac
h".
• F
act 1
: Le
t f X
(x)
and
λ(x
) b
e k-
dim
ensi
onal
pro
babi
lity
and
quan
tizat
ion
dens
ities
, res
pect
ivel
y.
The
n fo
r al
l suf
ficie
ntly
larg
e in
tege
r M
, ev
ery
k-
dim
ensi
onal
qua
ntiz
er w
ith s
ize
M,
quan
tizat
ion
dens
ity λ
, an
d sm
alle
stm
ean-
squa
red
erro
r fo
r th
e pr
obab
ility
den
sity
f h
as c
onst
ant i
nert
ial p
rofil
e.
We
can
conc
lude
from
this
fact
that
the
best
iner
tial p
rofil
e is
a c
onst
ant.
•S
ketc
h of
Pro
of1 :
We'
ll ar
gue
that
if a
qua
ntiz
er's
iner
tial p
rofil
e m
(x)
is n
otco
nsta
nt, t
hen
the
quan
tizer
can
be
impr
oved
, i.e
. it i
s do
es n
ot h
ave
the
smal
lest
pos
sibl
e M
SE
for
the
give
n k
, M, λ
and
f X.
The
con
trap
ositi
ve o
f thi
sst
atem
ent i
s: I
f a q
uant
izer
has
sm
alle
st p
ossi
ble
MS
E, t
hen
its in
ertia
l pro
file
is a
con
stan
t, w
hich
is th
e de
sire
d fa
ct.
If th
e in
ertia
l pro
file
m(x
) o
f a q
uant
izer
is n
ot a
con
stan
t, th
en th
ere
exis
tspo
ints
xb
and
xg
suc
h th
at m
(xb)
> m
(xg)
. M
oreo
ver,
sin
ce m
is
a sm
ooth
func
tion,
ther
e ex
ists
a s
mal
l reg
ion
B s
uch
that
m(x
) >
m(x
g) f
or a
ll x
∈B
.("
b" is
for
"bad
' and
"g"
is fo
r "g
ood"
.) T
he id
ea o
f thi
s pr
oof i
s to
impr
ove
the
1 Thi
s fa
ct h
as n
ot b
een
rigor
ousl
y pr
oven
.
Z-4
quan
tizer
by
repl
acin
g th
e co
nfig
urat
ion
of q
uant
izat
ion
poin
ts a
nd c
ells
in B
with
thos
e fr
om a
con
grue
nt r
egio
n ne
ar x
g, b
ecau
se th
ose
poin
ts a
nd c
ells
have
a b
ette
r no
rmal
ized
mom
ent o
f ine
rtia
.
We
will
ass
ume
that
M i
s so
larg
e th
at th
e nu
mbe
r M
b o
f qua
ntiz
atio
npo
ints
/cel
ls in
B i
s its
elf l
arge
, and
can
be
appr
oxim
ated
in te
rms
of th
equ
antiz
atio
n de
nsity
as
Mb
≅ λ
(xb)
|B| M
We
now
wis
h to
cho
ose
Mb
qua
ntiz
atio
n po
ints
/cel
ls in
the
neig
hbor
hood
of
x g.
Let
G b
e a
regi
on s
urro
undi
ng x
g th
at is
con
grue
nt to
B a
nd th
at c
on-
tain
s M
b p
oint
s/ce
lls.
The
num
ber
of p
oint
s/ce
lls in
the
G i
s ap
prox
imat
ely
λ(x g
) |G
| M .
Equ
atin
g th
is to
the
form
ula
for
Mb
det
erm
ines
the
volu
me
of G
|G|
≅ |B
| λ(x b
)
λ(x g
)
We
will
ass
ume
that
G i
s so
sm
all t
hat
m(x
) ≅
m(x
g) f
or x
∈G
. (W
e ca
nas
sum
e th
is b
ecau
se w
e co
uld
have
cho
sen
B v
ery
smal
l and
M v
ery,
ver
yla
rge.
)
We
now
"co
py"
the
poin
ts a
nd c
ells
in G
and
tran
slat
e an
d sc
ale
them
so
they
just
fit i
nto
B,
repl
acin
g th
e or
igin
al p
oint
s an
d ce
lls in
B.
(We
don'
t wor
ryab
out c
ells
on
the
boun
dary
of
G b
ecau
se th
ese
cons
titut
e a
negl
igib
le
Z-5
frac
tion.
We
leav
e th
e po
ints
and
cel
ls in
G u
ncha
nged
.) T
he r
esul
t is
that
with
in B
, w
e no
w h
ave
the
sam
e nu
mbe
r of
poi
nts
as o
rigin
ally
(an
dco
nseq
uent
ly th
e sa
me
poin
t den
sity
), b
ut w
e ha
ve c
ells
with
nor
mal
ized
mom
ent o
f ine
rtia
app
roxi
mat
ely
m(x
g),
whi
ch is
sm
alle
r th
an b
efor
e.
To
see
that
this
has
the
desi
red
effe
ct, c
onsi
der
the
effe
ct o
n di
stor
tion.
Obv
ious
ly, t
he e
ffect
is o
nly
in th
e re
gion
B, w
hich
we
now
con
side
r.O
rigin
ally
, the
dis
tort
ion
in B
was
⌡⌠ B
||x-Q
(x)|
|2 fX(x
) dx
≅
⌡⌠ B
m
(x)
M2/
k λ2/k (x
) f X(x
) dx
≅
m(x
b)M
2/k λ2/
k (xb)
f X(x
b)
Afte
r th
e ch
ange
, the
dis
tort
ion
in B
is
⌡⌠ B
||x-Q
(x)|
|2 fX(x
) dx
≅
⌡⌠ B
m
(xg)
M2/
k λ2/k (x
) f X(x
) dx
≅
m(x
g)M
2/k λ2/
k (xb)
f X(x
b)
Sin
ce m
( xg)
< m
(xb)
, th
e ne
w q
uant
izer
has
less
dis
tort
ion,
and
thus
the
orig
inal
qua
ntiz
er c
ould
not
hav
e be
en o
ptim
al, w
hich
is th
e co
ntra
posi
tive
ofth
e de
sire
d fa
ct.
Hav
ing
show
n th
at th
e be
st in
ertia
l pro
files
are
con
stan
ts, w
e co
uld
now
pro
ceed
to s
how
that
the
cons
tant
is th
e sa
me
for
all p
roba
bilit
y an
d qu
antiz
atio
nde
nsiti
es a
nd is
the
min
imum
all
valid
iner
tial p
rofil
es, a
ll so
urce
and
qua
ntiz
atio
nde
nsiti
es a
nd a
ll po
ints
x.
How
ever
, in
hind
sigh
t, th
ere
is a
mor
e di
rect
appr
oach
, whi
ch w
e no
w p
rese
nt.
Z-6
Mor
e D
irect
App
roac
h
Def
initi
on:
For
k-d
imen
sion
al q
uant
izat
ion,
a fu
nctio
n m
(x)
: ℜk →
[0,∞
) is
sai
dto
be
a va
lid in
ertia
l pro
file
if f
or a
ll su
ffici
ently
larg
e M
the
re a
re q
uant
izer
sw
ith d
imen
sion
k,
size
M,
and
iner
tial p
rofil
e m
.
We
asse
rt th
at a
ny n
onne
gativ
e fu
nctio
n th
at in
tegr
ates
to o
ne is
a v
alid
quan
tizat
ion
dens
tiy.
And
we
asse
rt th
at v
alid
poi
nt d
ensi
ties
and
iner
tial p
rofil
esar
e co
ntin
uous
func
tions
or,
at l
east
, pie
cew
ise
cont
inuo
us.
Def
initi
on:
The
k-d
imen
sion
al s
hape
fact
or 2 ,
den
oted
m* k,
is th
e m
inim
umva
lue
of a
ny v
alid
iner
tial p
rofil
e fo
r an
y po
int d
ensi
ty.
Tha
t is,
m* k
=
min
valid
k-d
ime
nsi
on
al
ine
rtia
l pro
file
s m
min x
m(x
)
Late
r w
e'll
disc
uss
wha
t's k
now
n ab
out t
he v
alue
s of
fin
d m
* k.
• F
act 2
: Le
t f X
(x)
and
λ(x
) b
e k-
dim
ensi
onal
pro
babi
lity
and
quan
tizat
ion
dens
ities
, res
pect
ivel
y.
The
n fo
r al
l suf
ficie
ntly
larg
e in
tege
rs M
, ev
ery
k-
dim
ensi
onal
qua
ntiz
er w
ith s
ize
M,
quan
tizat
ion
dens
ity λ
and
sm
alle
stm
ean-
squa
red
erro
r fo
r th
e pr
obab
ility
den
sity
f h
as in
ertia
l pro
file
m(x
) ≅
m* k
for
all
x
2 Can
you
thin
k of
a b
ette
r na
me?
Z-7
Not
e: T
he b
est i
nert
ial p
rofil
e do
es n
ot d
epen
d on
the
prob
abili
ty d
istr
ibut
ion
of th
e ra
ndom
vec
tor
to b
e qu
antiz
ed.
Ske
tch
of P
roof
3 : T
he p
roof
is v
ery
sim
ilar
to th
at o
f Fac
t 1.
We'
ll ar
gue
that
ifa
quan
tizer
's in
ertia
l pro
file
m(x
) d
oes
not a
ppro
xim
atel
y eq
ual
m* k
for a
ll x
,th
en th
e qu
antiz
er c
an b
e im
prov
ed, i
.e. i
t is
does
not
hav
e th
e sm
alle
stpo
ssib
le M
SE
for
the
give
n k
, M, λ
, and
fX.
The
con
trap
ositi
ve o
f thi
sst
atem
ent i
s: I
f a q
uant
izer
has
sm
alle
st M
SE
, the
n its
iner
tial p
rofil
e eq
uals
m* k,
whi
ch is
the
desi
red
fact
.
Acc
ordi
ngly
, con
side
r a
k-di
men
sion
al q
uant
izer
Q w
ith la
rge
size
M a
ndpo
int d
ensi
ty λ
who
se in
ertia
l pro
file
m(x
) is
gre
ater
than
m* k
at a
poi
nt x
b.S
ince
m i
s pi
ecew
ise
cont
inuo
us, t
here
exi
sts
a sm
all r
egio
n B
sur
roun
ding
x b s
uch
that
m(x
) ≅
m(x
b) >
m* k,
for
all
x ∈
B.
The
idea
now
is to
impr
ove
the
quan
tizer
by
repl
acin
g th
e co
nfig
urat
ion
ofqu
antiz
atio
n po
ints
and
cel
ls in
B w
ith th
ose
from
a q
uant
izer
who
se in
ertia
lpr
ofile
equ
als
m* k
in s
ome
regi
on.
By
the
defin
ition
of
m* k,
the
re m
ust e
xist
aqu
antiz
er Q
' w
ith la
rge
size
who
se in
ertia
l pro
file
m'(x
) e
qual
s m
* k a
t som
epo
int
x g.
("b"
is fo
r "b
ad' a
nd "
g" is
for
"goo
d".)
We
will
ass
ume
that
M i
s so
larg
e th
at th
e nu
mbe
r M
b o
f qua
ntiz
atio
npo
ints
/cel
ls in
B i
s its
elf l
arge
, and
can
be
appr
oxim
ated
in te
rms
of th
equ
antiz
atio
n de
nsity
as
3 Thi
s fa
ct h
as n
ot b
een
rigor
ousl
y pr
oven
.
Z-8
Mb
≅ λ
(xb)
|B| M
We
now
wis
h to
cho
ose
Mb
qua
ntiz
atio
n po
ints
/cel
ls in
the
neig
hbor
hood
of
x g.
Let
G b
e a
regi
on s
urro
undi
ng x
g th
at is
con
grue
nt to
B a
nd th
at c
on-
tain
s M
b p
oint
s/ce
lls.
We
will
ass
ume
that
G i
s so
sm
all t
hat
m'(x
) ≅
m* k
for
x ∈
G.
We
can
assu
me
this
bec
ause
we
coul
d ha
ve c
hose
n B
ver
y sm
all a
ndM
ver
y, v
ery
larg
e.
We
now
"co
py"
the
poin
ts a
nd c
ells
in G
and
tran
slat
e an
d sc
ale
them
so
they
just
fit i
nto
B,
repl
acin
g th
e or
igin
al p
oint
s an
d ce
lls in
B.
(We
don'
t wor
ryab
out c
ells
on
the
boun
dary
of
G b
ecau
se th
ese
cons
titut
e a
negl
igib
lefr
actio
n.)
The
res
ult i
s th
at w
ithin
B,
we
now
hav
e th
e sa
me
num
ber
of p
oint
sas
orig
inal
ly (
and
cons
eque
ntly
the
sam
e qu
antiz
atio
n de
nsity
), b
ut w
e ha
vece
lls w
ith n
orm
aliz
ed m
omen
t of i
nert
ia a
ppro
xim
atel
y m
* k, w
hich
is s
mal
ler
than
bef
ore.
To
see
that
this
has
the
desi
red
effe
ct, c
onsi
der
the
dist
ortio
n, w
hich
isaf
fect
ed o
nly
in th
e re
gion
B.
Orig
inal
ly, t
he d
isto
rtio
n in
B w
as
⌡⌠ B
||x-Q
(x)|
|2 fX(x
) dx
≅
⌡⌠ B
m
(x)
M2/
k λ2/k (x
) f X(x
) dx
≅
m(x
b)M
2/k λ2/
k (xb)
fX(x
b)
Afte
r th
e ch
ange
, the
dis
tort
ion
in B
is
⌡⌠ B
||x-Q
(x)|
|2 fX(x
) dx
≅
⌡⌠ B
m
* kM
2/k λ2/
k (x) f X
(x) d
x ≅
m
* kM
2/k λ2/
k (xb)
fX(x
b)
Z-9
Sin
ce m
( xb)
> m
* k, t
he n
ew q
uant
izer
has
less
dis
tort
ion.
The
refo
re, t
heor
igin
al q
uant
izer
cou
ld n
ot h
ave
been
opt
imal
, whi
ch is
the
cont
rapo
sitiv
e of
the
desi
red
fact
.
Z-1
0
Bes
t Poi
nt D
ensi
ty
Hav
ing
foun
d th
at th
e be
st in
ertia
l pro
file
is a
con
stan
t, th
e be
st p
oint
den
sity
is th
atw
hich
min
imiz
es th
e re
mai
ning
term
s of
Ben
nett'
s in
tegr
al.
⌡⌠
f X(x
)λ2/
k (x
) dx
(*)
It co
uld
be fo
und
with
cal
culu
s of
var
iatio
ns, b
ut w
e w
ill u
se H
olde
r's in
equa
lity.
Z-1
1
Hol
der's
ineq
ualit
y:
Giv
en fu
nctio
n's
f a
nd g
, th
en fo
r an
y q
,r >
0 s
uch
that
1 q +
1 r =
1,
⌡⌠
|f(x)
g(x
)| dx
≤
⌡⌠
|f(x
)|q d
x1/
q
⌡⌠
|g(x
)|r d
x1
/r
with
equ
ality
iff
for
som
e c
, |f
(x)|q =
c |g
(x)|
r , a
ll x
Our
Str
ateg
y: C
hoos
e f,
g, q
and
r s
o th
at
|f(x)
|q =
f X(x
)λ(
x)2/
k
an
d ⌡⌠
|g(x)
|r dx
= 1
.
The
n ⌡⌠
f X(x
)λ(
x)2/
k
dx
=
⌡⌠
|f(x)|
q dx
≥
⌡⌠
|f(x
) g(x
)| dx
q
⌡⌠
|g(x
)|r d
x-q
/r =
∫
|f(x
) g(x
)| d
xq
with
equ
ality
hap
pens
if a
nd o
nly
if th
ere
is c
onst
ant
c s
uch
that
|f(
x)|q =
c |g
(x)|
r .
If it
turn
s ou
t tha
t the
inte
gral
on
the
far
right
doe
s no
t dep
end
on λ
, th
en w
e ha
ve a
low
er b
ound
to th
e in
tegr
al w
e ar
e m
inim
izin
g. A
nd w
e ca
n m
inim
ize
the
inte
gral
by
choo
sing
λ t
o sa
tisfy
the
cond
ition
that
giv
es e
qual
ity in
the
low
er b
ound
.
Z-1
2
Our
cho
ices
: q =
k+2 k,
r = k+
22
⇒
1 q + 1 r
=
k k+2 +
2 k+
2 = 1
f( x) =
f X
(x)
λ2/k
(
x)k/
(k+
2)an
d
g(x
) =
λ2/(k
+2)
(x
)
The
n as
des
ired,
|
f(x)
|q =
f X(x
)λ2/
k
(x)
and
⌡⌠
|g(x
)|r d
xq/
r =
⌡⌠
λ(x
) dx
q/r =
1
The
refo
re,
⌡⌠
f X(x
)λ2/
k
(x)
dx
≥
⌡⌠
|f(x
) g(x
)| d
xq =
⌡⌠
fk/(k
+2)
X(x
) d
x(k
+2)
/k
whe
re, f
ortu
nate
ly, t
he r
ight
-han
d si
de d
oes
not d
epen
d on
λ, a
nd w
here
equ
ality
hold
s iff
ther
e is
a c
onst
ant c
suc
h th
at
f X(x
)λ2/
k
(x) =
c λ
(x)
, i.e
. λ(
x) =
c' f
k/(k
+2)
X(x
)
whe
re c
' is
cho
sen
to m
ake
λ(x
) in
tegr
ate
to o
ne.
We
conc
lude
that
the
inte
gral
(*)
is m
inim
ized
by
the
poin
t den
sity
λ* k(x)
=
fk/(k
+2)
X(x
)
⌡⌠
fk/(k
+2)
X(x
') dx
'
and
the
resu
lting
min
imum
val
ue is
⌡⌠
fk/(k
+2)
X(x
) dx
(k+
2)/k
Z-1
3
Hav
ing
that
the
optim
al q
uant
izer
s w
ith d
imen
sion
k h
ave
iner
tial p
rofil
e m
( x)
≅ m
* k a
nd q
uant
izat
ion
dens
ity λ
(x)
≅ λ
* k, w
e m
ay s
ubst
itute
thes
e in
toB
enne
tt's
inte
gral
to o
btai
n th
e di
stor
tion
of a
n op
timal
qua
ntiz
er w
ithdi
men
sion
k a
nd la
rge
size
M fo
r a
rand
om v
ecto
r X
with
fX(x
). T
hat i
s,w
e fin
d th
e fo
llow
ing
appr
oxim
ate
form
ula
for
the
opta
func
tion:
δ(k,
M)
≅
1
M2/
k
⌡⌠
m
* kλ* k
2/k
(x
) fX(x
) dx
=
1M
2/k
m
* k ⌡⌠
1
(fk/
(k+
2)X
(x)/
c)2/
k
fX(x
) dx
,
w
here
c =
⌡⌠
fk/(k
+2)
X(x
) dx
=
1M
2/k
m
* k c2/
k
⌡⌠
fk/(k
+2)
X(x
) dx
=
1
M2/
k
m* k
c1+2/
k
=
1M
2/k
m
* k ( ⌡⌠
f X(x
)k/(k
+2)
d
x )(k
+2)
/k
Z-1
4
We
sum
mar
ize
in th
e fo
llow
ing.
Zad
or'
s T
heo
rem
4
Whe
n M
is
larg
e, th
e le
ast d
isto
rtio
n of
fixe
d-ra
te, k
-dim
'l V
Q w
ith M
poi
nts
is
δ(k,
M)
≅
Z(k
,M)
whe
re
Z(k
,M)
∆ = σ
2 β k
m* k
1M
2/k
is c
alle
d Z
ador
's fu
nctio
n
σ2 =
so
urce
var
ianc
e =
1 k ∑ i=1k v
ari
an
ce(X
i)
β k
=
1 σ2 ( ⌡⌠
f X(x
)k/(k
+2)
d
x )(k
+2)/
k =
"Z
ador
's fa
ctor
"
(dep
ends
on
"sha
pe"
of f
X(x
); i
nvar
iant
to a
sca
ling)
m* k
=
smal
lest
val
ue a
ttain
ed b
y an
y va
lid in
ertia
l pro
file
4 P. L
. Zad
or, "
Dev
elop
men
t and
eva
luat
ion
of p
roce
dure
s fo
r qu
antiz
ing
mul
tivar
iate
dis
trib
utio
ns,"
Ph.
D. D
isse
rtat
ion,
Sta
nfor
d, 1
963.
Als
o, P
.L. Z
ador
,"T
opic
s in
asy
mpt
otic
qua
ntiz
atio
n of
con
tinuo
us r
ando
m v
aria
bles
," B
ell L
ab. T
ech.
Mem
o, 1
966.
Z-1
5
Eq
uiv
alen
t S
tate
men
ts o
f Z
ado
r's
Th
eore
m
In te
rms
of r
ate:
Whe
n R
is
larg
e, th
e be
st fi
xed-
rate
, k-d
imen
sion
al V
Q's
with
rat
e R
hav
e M
SE
δ(k,
R)
≅ σ
2 βk
m* k
2-2R
∆ = Z
(k,R
)
In te
rms
of S
NR
:
Whe
n R
is
larg
e, th
e be
st fi
xed-
rate
, k-d
imen
sion
al V
Q's
with
rat
e R
hav
e S
NR
S(k
,R)
≅ 1
0 lo
g 10
σ2
Zk(
R)
= 6
.02
R -
10 lo
g 10
m* k
βk
Not
e:
SN
R in
crea
ses
at 6
dB
per
bit
for
optim
al q
uant
izer
s.
Rat
e in
term
s of
dis
tort
ion:
Whe
n D
is
smal
l, th
e be
st fi
xed-
rate
, k-d
imen
sion
al V
Q's
with
MS
E D
has
rat
e
γ (k,
R)
≅ 1 2 l
og (
m* k σ
2 βk)
- 1 2 lo
g 2(D
)
Z-1
6
Zad
or d
id n
ot d
eriv
e hi
s th
eore
m fr
om B
enne
tt's
inte
gral
. R
athe
r he
foun
d a
dire
ct p
roof
of t
he fa
ct th
at fo
r an
y di
men
sion
k,
ther
e is
a c
onst
ant
α k s
uch
that
for
any
prob
abili
ty d
ensi
ty f
X(x
)
lim M→
∞ M
2/k
δ
(k,M
) =
αk ( ⌡⌠
f X(x
)k/(k
+2)
d
x )(k
+2)
/k
Zad
or d
id n
ot e
quat
e α
k w
ith m
* k a
s w
e ha
ve d
efin
ed it
. B
ut h
e di
d gi
ve s
ome
boun
ds to
it.
Z-1
7
How
larg
e m
ust
M o
r R
be
in o
rder
for
the
form
ulas
to b
e ac
cura
te?
Exa
mpl
e: G
auss
-Mar
kov
Sou
rce,
cor
r. c
oeff.
ρ =
.9
0510152025
01
23
4R
ate
SNR, dBk=
2
k=4
k=1
VQ
's d
esig
ned
byLB
G a
lgor
ithm
.
Str
aigh
t lin
es a
reZ
ador
's fu
ntio
nZ
(k,R
).
m* 4
is e
stim
ated
. Z-1
8
Dat
a fo
r th
e pr
evio
us p
lot
Gau
ss-M
arko
v S
ourc
e, ρ
= .9
, S
NR
's in
dB
k =
1k
= 2
k =
4R
ate
Act
'lP
red' d
Diff
.A
ct'l
Pre
d'd
Diff
.A
ct'l
Pre
d'd
Diff
.
0.0
0.0
-4.3
54.
350.
00.
6-0
.56
0.00
3.33
-3.3
3
0.5
4.0
3.6
0.47
6.55
6.34
0.21
1.0
1.68
4.4
2.72
7.9
6.6
1.35
10.2
29.
350.
87
1.5
10.8
9.6
1.21
13.0
412
.36
0.68
2.0
7.70
9.3
1.60
13.5
12.6
0.92
15.8
115
.37
0.44
2.5
16.3
15.6
0.65
18.6
618
.38
0.27
3.0
13.7
214
.62
0.90
19.0
18.6
0.41
3.5
21.9
21.6
0.26
4.0
19.7
420
.22
0.48
24.8
24.6
0.16
The
pre
dict
ed v
alue
at
R =
0 i
s -
10 lo
g 10
m* k
β k
Z-1
9
05101520
01
23
4R
ate
SNR, dBk=
2
k=4
k=1
Exa
mpl
e 2:
IID
Gau
ssia
n S
ourc
e
VQ
's d
esig
ned
by L
BG
algo
rithm
.
Str
aigh
t lin
es a
re fr
omZ
ador
's fu
nctio
nZ
(k,R
).
m* 4
is e
stim
ated
.
Z-2
0
Dat
a fo
r th
e pr
evio
us p
lot
IID G
auss
ian
Sou
rce,
SN
R's
in d
B
k =
1k
= 2
k =
4R
ate
Act
'lP
red' d
Diff
.A
ct'l
Pre
d' dD
iff.
Act
'lP
red' d
Diff
.
0.0
0.0
-4.3
54.
350.
00-2
.95
2.95
0.00
-2.0
12.
01
0.5
1.66
0.06
1.60
1.89
1.00
0.89
1.0
1.68
4.4
2.72
4.39
3.07
1.32
4.60
4.01
0.60
1.5
6.96
6.08
0.88
7.34
7.02
0.32
2.0
7.70
9.3
1.60
9.64
9.09
0.55
10.1
810
.03
0.15
2.5
12.4
212
.10
0.32
13.2
113
.04
0.17
3.0
13.7
214
.62
0.90
15.2
715
.11
0.16
16.1
816
.05
0.14
3.5
18.1
718
.12
0.05
4.0
19.7
420
.22
0.48
21.1
221
.13
-0.0
1
The
pre
dict
ed v
alue
at
R =
0 is
-10
log 1
0 m
* k β k
Z-2
1
Rul
es o
f Thu
mb:
•Z
-G is
acc
urat
e fo
r R
~ ≥ 3
( M
~ ≥ 2
3k ).
•F
or a
giv
en R
, ac
cura
cy in
crea
ses
with
dim
ensi
on k
.
Z-2
2
Th
e V
alu
es o
f m
* k
How
to fi
nd th
e va
lue
of m
* k? S
uppo
se b
y so
me
mea
ns, w
e ca
n fin
d th
e O
PT
Afu
nctio
n δ(
k,M
) fo
r so
me
part
icul
ar p
df f
X(x
).
The
n by
Zad
or's
theo
rem
δ(k
,M)
≅ σ
2 β k
m* k
1M
2/k
whi
ch c
an b
e so
lved
for
m* k:
m* k
≅ M
2/k
δ
(k,M
) 1
σ2 βk
Mor
e pr
ecis
ely,
sup
pose
we
can
find
lim M→
∞ M
2/k
δ(
k,M
). T
hen,
m* k
=
lim M→
∞ M
2/k
δ(
k,M
)
1σ2 β
k
If w
e w
ish
to u
se th
is a
ppro
ach,
wha
t pdf
sho
uld
we
cons
ider
? B
y fa
r th
eea
sies
t typ
e of
to w
ork
with
is a
uni
form
. T
hat i
s, w
e ta
ke f
X(x
) to
be
aco
nsta
nt o
n so
me
regi
on, e
.g. a
cub
e, a
nd z
ero
else
whe
re.
Her
e's
wha
t's b
een
foun
d w
ith th
is a
ppro
ach:
Z-2
3
•k
= 1:
F
or s
cala
r qu
antiz
ers,
m* 1
= m
(inte
rval
) =
1 12 =
.08
33
•k
= 2:
For
two-
dim
ensi
onal
qua
ntiz
ers
m* 2
= m
(hex
agon
) =
5√3
/108
= .
0802
Thi
s is
bas
ed o
n th
e w
ork
L. F
ejes
Tot
h5 , w
hich
was
red
eriv
edin
depe
nden
tly b
y D
. New
man
6 in
a si
mpl
er fa
shio
n.
•k
≥ 3:
F
or k
≥ 3
, th
is a
ppro
ach
has
not y
ield
ed a
n an
swer
. T
hus
for
k ≥
3,th
e va
lue
of m
* k is
not
kno
wn.
How
ever
, the
re a
re s
ever
al b
ound
s, b
oth
uppe
r an
d lo
wer
. S
ince
the
uppe
r an
d lo
wer
bou
nds
are
fairl
y cl
ose,
we
get
a pr
etty
goo
d ap
prox
imat
ion
to m
* k .
5 L. F
ejes
Tot
h, "
Sur
la r
epre
sent
atio
n d'
une
popu
latio
n in
finie
par
un
nom
bre
fini d
'ele
men
ts,"
Act
a M
ath.
Aca
d. S
ci. H
ung.
, vol
. 10,
pp.
76-
81, 1
959.
6 D
.J. N
ewm
an, "
The
hex
agon
theo
rem
," B
ell L
ab. T
ech.
Mem
o, 1
964.
Als
o pu
blis
hed
late
r in
IEE
E T
rans
. Inf
orm
. The
ory,
vol
. 28,
pp.
137
-139
, Mar
.1
98
2.
Z-2
4
•Lo
wer
bou
nd to
m* k
m* k
≥ m
(k-d
imen
sion
al s
pher
e)
By
defin
ition
, m
* k is
the
leas
t nm
i of a
ny v
alid
iner
tial p
rofil
e. S
ince
no
k-di
men
sion
al p
artit
ion
can
have
any
cel
l with
nm
i les
s th
an th
at o
f a k
-dim
en-
sion
al s
pher
e, a
ny v
alid
iner
tial p
rofil
e is
low
er b
ound
ed b
y th
e nm
i of a
k-
dim
ensi
onal
sph
ere.
•A
noth
er lo
wer
bou
nd is
con
ject
ured
in th
e bo
ok S
pher
e P
acki
ngs,
Lat
tices
and
Gro
ups,
by
J.H
. Con
way
and
N.J
.A. S
loan
e', p
. 59-
62.
It is
tigh
ter
than
the
sphe
re lo
wer
bou
nd.
Z-2
5
•U
pper
bou
nds
to m
* k
We
find
uppe
r bo
unds
to m
* k b
y fin
ding
upp
er b
ound
s to
the
OP
TA
func
tion
δ(k,
M)
of a
uni
form
. (
Mor
e pr
ecis
ely,
we
find
uppe
r bo
unds
tolim M→
∞ M
2/k
δ c
(k,M
).)
How
do
we
find
uppe
r bo
unds
to th
e O
PT
A fu
nctio
n?
We
do th
is b
y fin
ding
the
dist
ortio
n of
the
best
qua
ntiz
ers
that
we
can
thin
kof
for
the
unifo
rm p
df.
The
OP
TA
func
tion
will
be
less
than
the
dist
ortio
n of
any
actu
al q
uant
izer
.
Z-2
6
•T
esse
latio
n up
per
boun
d.
A s
ubse
t G
of
ℜ2/
k
is
said
to te
ssel
ate
if th
ere
is a
par
titio
n S
of
ℜ2/
k
, al
lof
who
se c
ells
are
tran
slat
ions
and
or
rota
tions
of
G.
The
par
titio
n S
is
said
to b
e a
tess
elat
ion.
Som
e ex
ampl
es o
f tes
sela
tions
are
sho
wn
belo
w.
a.b.
c.d.
Con
side
r a
k-di
men
sion
al p
df th
at is
con
stan
t on
a se
t H
, i.e
.
f X(x
) = 1
/|H
|,
x∈H
0, el
se
and
cons
ider
the
quan
tizer
who
se p
artit
ion
S i
s a
tess
elat
ion
base
d on
the
set
G.
Thi
s qu
antiz
er h
as q
uant
izat
ion
dens
ity
λ(x)
= 1
/|H
|,
x∈H
0, el
se
and
iner
tial p
rofil
e m
(x)
= m
(G).
(W
e as
sum
e th
e qu
antiz
atio
n po
ints
are
inth
e sa
me
rela
tive
posi
tion
in e
ach
cell,
and
we
supp
ress
the
nota
tion
for
such
poi
nts.
)
Z-2
7
Fro
m B
enne
tt's
inte
gral
, we
have
D ≅≅≅≅
1M
2/k
⌡⌠
m(x
)λ2/
k
(x) f
X(x)
dx
=
1
M2/
k
⌡⌠ H
m
(G)
(1/|H
|)2/
k
(x)
1 |H| d
x
=
1M
2/k
m
(G)
|H|2/
k
= m
(G)
|G|2/
k
,
s
ince
|H
| ≅ M
|G|
Sin
ce th
e di
stor
tion
of th
is q
uant
izer
is a
t lea
st la
rge
as th
e O
PT
A fu
nctio
n,fo
r an
y te
ssel
atin
g ce
ll G
,
δ(k,
M)
≤ m
(G)
|G|2/
k
.
For
this
uni
form
sou
rce
σ2 βk
=
⌡⌠
f X(x
)k/(k
+2)
d
x (k
+2)/k
=
⌡⌠ H
1 |H
|k/(k
+2)
dx
(k+2
)/k =
|H
|2/k
Now
sub
stitu
ting
the
valu
e of
σ2 β
k a
nd th
e up
per
boun
d to
δ(k
,M),
we
find
m* k
=
lim M→
∞ M
2/k
δ(
k,M
)
1σ2 β
k
≤
lim M→
∞ M
2/k
m
(G)
|G|2/
k
1
|H|2
/k
= m
(G)
si
nce
|H
| ≅ M
|G|
Z-2
8
We
now
con
clud
e th
at m
* k is
upp
er b
ound
by
the
nmi o
f the
tess
elat
ing
set
G w
ith le
ast n
mi,
i.e.
m* k
≤ m
* T,k =∆
m
inte
ssel
atin
g G
m(G
)
Ger
sho
has
conj
ectu
red
that
this
bou
nd is
tigh
t for
k≥3
. B
ut it
is n
ot k
now
n if
this
true
.
In fa
ct, h
e co
njec
ture
d th
at th
e ce
lls o
fan
opt
imal
qua
ntiz
er w
ith m
any
poin
tsar
e, lo
cally
, tes
sela
tions
. T
his
cert
ainl
yse
ems
to b
e th
e ca
se fo
r k
= 1
,2.
For
exam
ple,
an
LBG
des
igne
d op
timal
k=2
,M
=256
qua
ntiz
er fo
r an
IID
pai
r of
Gau
ssia
n va
riabl
es is
sho
wn
to th
e rig
ht.
Not
e th
at s
ince
the
optim
al q
uant
izer
for
a no
nuni
form
will
ord
inar
ily h
ave
cells
of d
iffer
ent s
izes
, its
par
titio
n w
illno
t be
a te
ssel
atio
n. H
owev
er, i
n sm
all
regi
ons
the
tess
elat
ion
will
be
appa
rent
,i.e
. loc
ally
it is
app
roxi
mat
ely
ate
ssel
atio
n.
Z-2
9
•La
ttice
upp
er b
ound
.
A p
artit
ion
S i
s sa
id to
be
a la
ttice
tess
elat
ion
if it
is a
tess
elat
ion
such
that
all c
ells
are
tran
slat
ions
of e
ach
othe
r. (
Rot
atio
ns a
re n
ot a
llow
ed.
All
latti
ces
are
tess
elat
ions
, but
not
vic
e ve
rsa.
) E
xam
ples
c.
and
d.
give
npr
evio
usly
are
latti
ces.
The
oth
ers
are
not.
Man
y ex
ampl
es o
f lat
tice
are
know
n. F
or e
xam
ple,
see
the
book
by
Con
way
and
Slo
ane.
Ind
eed
ther
e m
ost k
now
n te
ssel
atio
ns a
re la
ttice
. B
yth
e sa
me
argu
men
t as
for
tess
elat
ions
,
m* k
≤
m* L,k
=∆
min
latt
ice
s S
ba
sed
on
Gm
(G)
Of c
ours
e m
* T,k ≤
m* L,k .
In
fact
, the
se m
ight
be
equa
l, bu
t for
k ≥
3 n
o on
ekn
ows.
(F
or k
=1 o
r 2,
the
y ar
e eq
ual.)
Z-3
0
•P
erio
dic
latti
ce u
pper
bou
nd
A p
artit
ion
S i
s sa
id to
be
a pe
riodi
c la
ttice
if th
ere
is a
fini
te c
olle
ctio
n of
cells
Mo
cel
ls G
1,...
,G M
o s
uch
that
the
rem
aini
ng c
ells
of t
he p
artit
ion
are
otai
ned
by tr
ansl
atin
g th
is g
roup
of c
ells
(as
a g
roup
). A
ll of
the
exam
ples
give
n pr
evio
usly
(a.
-d.)
are
per
iodi
c la
ttice
s. T
he fo
llow
ing
are
exam
ples
of
perio
dic
tess
elat
ions
that
are
not
latti
ces
nor
an o
rdin
ary
tess
elat
ions
.
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
xxx
7
We
may
con
stru
ct a
qua
ntiz
er fo
r a
unifo
rm p
df in
ess
entia
lly th
e sa
me
way
as fo
r a
tess
elat
ion.
Thi
s le
ads
to th
e up
per
boun
d
m* k
≤ m
*P
L,k
=∆
min
pe
rio
dic
latt
ice
s
~ m(G
1,.
..,G
Mo)
7 Thi
s is
a 3
-dim
ensi
onal
per
iodi
c la
ttice
with
2 ty
pes
of c
ells
, cal
led
the
Wea
ire-P
hela
n pa
rtiti
on.
It is
form
ed b
y te
ssel
atin
g a
fund
amen
tal g
roup
con
sist
ing
of t
wo
pent
agon
al d
odec
ahed
ra (
12 fa
ces,
eac
h is
5-s
ided
) an
d si
x 14
-hed
ra (
2 he
xago
nal f
aces
, 4 p
enta
gona
l fac
es o
f one
kin
d an
d 6
pent
agon
al fa
ces
ofan
othe
r ki
nd).
Z-3
1
whe
re
~ m i
s th
e av
erag
e no
rmal
ized
mom
ent o
f ine
rtia
of a
gro
up o
f cel
ls,
as d
efin
ed a
t the
end
of t
he le
ctur
e no
tes
on B
enne
tt's
inte
gral
.
Of c
ours
e m
*P
L,k
≤ m
* T,k ≤
m* L,k .
In
fact
, the
se m
ight
be
equa
l, bu
t for
k ≥
3no
one
kno
ws.
(F
or k
=1 o
r 2,
the
y ar
e eq
ual.)
Tho
ugh
it ha
s no
t bee
n pr
oven
, it s
eem
s qu
ite li
kely
to m
e th
at th
is b
ound
istig
ht.
Thu
s, fo
r th
e re
mai
nder
of t
his
cour
se w
e w
ill a
ssum
e
m* k
= m
*P
L,k
Z-3
2
•k
= 3
: T
he b
est k
now
n pe
riodi
c te
ssel
atio
n in
thre
e di
men
sion
s is
the
latti
ce g
ener
ated
by
the
trun
cate
d oc
tahe
dron
, tw
o of
whi
ch a
re il
lust
rate
dbe
low
. It
is k
now
n th
at th
is is
the
best
latti
ce.
But
it is
not
bee
n pr
oven
that
this
is th
e be
st te
ssel
atio
n or
per
iodi
c te
ssel
atio
n. H
owev
er, i
t see
ms
likel
yth
at it
is.
The
nm
i of t
he tr
unca
ted
octa
hedr
on is
m =
.078
543.
The
nm
i of a
3-
dim
ensi
onal
sph
ere
is .
0770
. T
here
fore
,
.077
0 ≤
m* 3 ≤
.07
85
It is
inte
rest
ing
that
the
basi
c gr
oup
of th
e W
eaire
-Phe
lan
part
ition
(sh
own
earli
er)
has
nmi .
0787
35 w
hich
is v
ery
near
ly a
s sm
all a
s th
at o
f the
trun
cate
d oc
tade
hedr
on.
Z-3
3
Pro
perti
es o
f m
* T,k
•It
is n
ot k
now
n if
m* k+
1 ≤
m* k
for
all
k.
The
re is
no
know
n pr
oof n
or c
ount
er e
xam
ple.
•T
houg
h m
* k m
ight
not
be
decr
easi
ng w
ith k
, as
sum
ing
Ger
sho'
s co
njec
ture
, it
can
be s
how
n to
hav
e a
kind
of d
ecre
asin
g tr
end
calle
d "s
ubad
ditiv
ity",
mea
ning
that
for
any
k,l
m* k+
l ≤
k k+
l m* k
+ l k+
l m* l
whi
ch im
plie
s (w
ith a
ppro
pria
te u
se o
f the
abo
ve)
m* 1
≥ m
* k ≥
m* 2k
It al
so c
an b
e sh
own
that
sub
addi
tivity
impl
ies
m* ∞ =∆
lim k→
∞ m
* k =
inf
k m
* k
•A
pro
of th
at s
ubad
ditiv
ity im
plie
s li
m =
inf
can
be fo
und
inG
alla
ger's
info
rmat
ion
theo
ry b
ook,
Lem
ma
2, p
p. 1
12,1
13.
The
pro
of o
f sub
addi
tivity
dep
ends
on
the
next
fact
:
Z-3
4
•F
act:
If
S ⊂
Rk
and
T ⊂
Rl ,
the
n
M(S
×T)
= v
ol(S
) M
(T)
+ v
ol(T
) M
(S)
whe
re M
(S)
= ∫ S |
|x||
2 dx
= M
I
•P
roo
f o
f F
act:
M(S
×T)
=
⌡⌠
S×T
||x
||2 d
x =
⌡⌠ S
⌡⌠ T
(||x
||2 +||
y||2 d
x dy
=
⌡⌠ S
(M(T
) +
||y|
|2 vol
(T))
dy
= M
(T)
vol(S
) +
M(S
) vo
l(T)
Pro
of
of
Su
bad
dit
ivit
y:
Ass
umin
g G
ersh
o's
conj
ectu
re, l
et S
and
T b
e te
ssel
atin
g po
lyhe
dra
with
unit
volu
mes
that
ach
ieve
m* k
and
m* l,
res
pect
ivel
y. T
hen
S×T
is
also
ate
ssel
atin
g po
lyhe
dron
. A
nd v
ol(S
×T)
= 1.
The
refo
re, a
pply
ing
the
Fac
t,
m* k
≤ m
(S×T
) =
M
(S×T
)
(k+
l)vol
(S×T
)(k+
l+2)
/(k+
l)
=
M(T
) vol
(S)
+ M
(S) v
ol(
T)
k+l
=
1 k+
l (M(T
) +
M(S
))
=
1 k+l (
l m(T
) + k
m(S
) + ))
=
k k+
l m* k
+ l k+
l m* l
Z-3
5
I bel
ieve
that
a p
roof
of s
ubad
ditiv
ity th
is s
ort c
ould
be
writ
ten
assu
min
g th
ew
eake
r co
njec
ture
that
the
smal
lest
iner
tial p
rofil
e co
rres
pond
s to
a p
erio
dic
tess
elat
ion.
•S
pher
e Lo
wer
bou
nd:
m* k
≥ m
(k-d
im s
pher
e) =
(Vk)
-2/k
k+2
→
1 2πe =
.058
5 ≅
1 17
→
1 2πe
whe
re V
k =
vol
. of k
-dim
. sph
ere
w r
adiu
s 1
•m
* k →
1 2πe
= .0
585
≅ 1 17
a
s k
→∞.
Pro
ved
by Z
amir
and
Fed
er 1
996.
•U
pper
bou
nds:
m
* k ≤
m(S
) fo
r be
st k
now
n te
ssel
atio
n.
Suc
h bo
unds
may
con
tinue
to im
prov
e as
peo
ple
lear
n of
bet
ter
tess
elat
ing
poly
hedr
a.
•T
here
is a
con
ject
ured
low
er b
ound
in C
onw
ay a
nd S
loan
e's
book
, p. 5
9-62
. It
is ti
ghte
r th
an th
e sp
here
low
er b
ound
•S
umm
ary:
m
* k d
ecre
ases
with
k (
thou
gh n
ot n
ecce
ssar
ily m
onot
onic
ally
)fro
m 1
/12
= .0
833
at
k =
1 to
m* ∞
= 1/
2πe
= .0
585
≅ 1/
17,
whi
ch r
epre
sent
s a
gain
of
1.53
dB
.
•T
he b
ook
by C
onw
ay a
nd S
loan
e ha
s a
sum
mar
y of
wha
t is
know
n ab
out t
he Z-3
6
best
tess
elat
ing
poly
tope
s. S
loan
e al
so h
as a
web
site
that
may
con
tain
furt
her
upda
tes.
•W
e ne
ed a
goo
d na
me
for
m* k
. A
ny s
ugge
stio
ns?
Z-3
7
The
Bes
t Kno
wn
Tes
sela
ting
Pol
ytop
es
dim
ensi
on p
olyt
ope
m* k
best
know
n,(u
pp
er
boun
d)
conj
'dlo
wer
boun
d
sphe
relo
wer
boun
d
gain
(dB
)10
log
m* 1/
m* k
1in
terv
al.0
833'
.083
30
2he
xago
n.0
802'
.079
6.1
6
3un
know
n.0
785'
trun
cate
doc
tahe
dron
.077
875
.077
0.2
6
4"
.076
60.
0761
'.0
750
.39
5"
.075
60.
0747
'.0
735
.47
6"
.074
20.
0735
'.0
723
.55
7"
.073
10.
0725
'.0
713
.60
8"
.071
70.
0716
'.0
705
.66
12"
.070
10.
0692
'.0
691
.81
16"
.068
30.
0676
'.0
666
.91
24"
.065
80.
0656
'.0
647
1.10
50"
.062
3'1.
26
100
".0
608'
1.37
Z-3
8
200
".0
599'
1.43
300
".0
595'
1.46
very
larg
esp
here
.058
5'.0
585
1.53
Z-3
9
dim
ensi
on
gain dB
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
110
100
1000
SNR
gai
ns d
ue to
dec
reas
ing
M*k
know
n va
lues
of
M*k
sphe
res
low
er b
ound
bas
ed o
n
C&
S co
njec
ture
1.53
dB l
imit
know
n la
ttic
e te
ssel
atio
ns
uppe
r bo
und
base
d on
Z-4
0
Pro
perti
es o
f the
Zad
or F
acto
r β k
(1)
If Y
= a
X +
b,
whe
re a
≠ 0
, the
n β
Y,k
= β
X,k
. T
his
show
s β
k d
epen
ds o
n th
esh
ape
of th
e de
nsity
, and
is in
varia
nt to
sca
ling
or s
hifti
ng.
Der
ivat
ion:
Thi
s ca
n be
der
ived
dire
ctly
, or
from
Pro
pert
y (2
) w
ith A
bei
ng a
diag
onal
mat
rix w
ith a
's o
n th
e di
agon
al a
nd |
A| =
ak ,
and
with
σ2 Y =
a2 σ
2 X.
(2)
If Y
= A
X +
b,
whe
re A
is
a no
nsin
gula
r sq
uare
mat
rix, t
hen
β Y,k
= σ2 X σ2 Y
|A|2/
k
βX
,k
Not
e th
at β
is
not a
ffect
ed b
y th
e ad
ditio
n of
the
cons
tant
b.
Der
ivat
ion:
W
e ha
ve
f Y(y
) =
|A
|-1 f X
(A-1
(y-b
)).
The
refo
re,
β Y,k
= 1 σ2 Y
∫
f Y(y
)k/k+
2
dy
(k+2
)/k =
1 σ2 Y
∫
(|A
|-1 f
X(A
-1 (y
-b))
)k/k+
2
dy
(k+
2)/k
= 1 σ2 Y
∫
(|A
|-1 f
X(A
-1 (y
-b))
)k/k+
2
dy
(k+
2)/k
= 1 σ2 Y
|A|-1
∫
f X(x
)k/k+
2
|A| d
x(k
+2)/
k , w
ith x
= A
-1 (y
-b),
y=A
x+b,
dy
= |A
| dx
= 1 σ2 Y
|A|2/
k
∫
f X(x
)k/k+
2
dx
(k+2
)/k
=
1 σ2 Y
|A|2/
k
σ2 X β
X,k
Z-4
1
(3)
If Y
= A
X +
b a
nd A
is
a k
×k o
rthog
onal
mat
rix (
i.e.
A-1
= A
t ),
then
β Y
,k =
βX
,k .
It sh
ould
be
intu
itive
that
the
opta
for
Y i
s th
e sa
me
as fo
r X
, be
caus
e on
e ca
nro
tate
and
tran
slat
e an
opt
imal
VQ
for
X t
o ge
t a V
Q w
ith th
e sa
me
perf
orm
ance
for
Y,
and
vice
ver
sa.
Der
ivat
ion:
For
an
orth
ogon
al m
atrix
||A
x||
= ||
x||
for
all
x. T
here
fore
,
σ2 Y =
1 k E||Y
-EY
||2 =
1 k E
||A(X
-EX
)||2
= 1 k E
||X-E
X||2
= σ
2 X
Nex
t |A
| = ∏ i=
1k λ
i =
1, w
here
the
λi's
are
the
eige
nval
ues,
whi
ch a
ll ha
ve
mag
nitu
de o
ne, b
ecau
se A
x =
λx i
mpl
ies
||x|
| = ||
Ax|
| = ||
λx||
⇒ |
λ| =
1.
(4)
If Y
= A
X +
b,
whe
re A
is a
k×k
dia
gona
l mat
rix w
ith d
iago
nal e
lem
ents
a1,
…,a
k,th
en
β Y,k =
∏ i=
1k a
2 i1/
k ∑ i=
1k σ
2 X,i
∑ i=1k a
2 iσ2 X
,i βX,k
=
∏ i=
1k a
2 i1/
k
1 k∑ i=
1k a
2 i
βX,k
if
the
σ2 X,i
's a
re a
ll th
e sm
ae
Z-4
2
(5)
If X
1,…
,Xk
are
inde
pend
ent,
then
β k =
1 σ2 X
∏ i=1k
∫
f i(x
)k/(k
+2
)
dx
(k+
2)/k
Der
ivat
ion:
βX,k =
1 σ2 X
∫
f 1(x
1)k/
k+2
…
f k(x
k)k/
k+2
d
x(k
+2)
/k
=
1 σ2 X
∫
f 1(x
1)k/
k+2
dx
1…∫
f k(x
k)k/
k+2
dx
k(k
+2)
/k
=
1 σ2 X
∏ i=1k
∫
f i(x
)k/k+
2
dx(k
+2)
/k
(6)
X1,
…,X
k in
depe
nden
t and
iden
tical
(IID
) w
ith v
aria
nce
σ2
β X,k
=
1 σ2
∫
f 1(x
)k/k+
2
dxk+
2
(7)
If X
1,…
,Xk
and
Y1,
…,Y
k h
ave
the
sam
e m
argi
nal d
istr
ibut
ions
, but
Y1,
…,Y
kar
e in
depe
nden
t, th
en
β x,k
≤ β
Y,k
w
ith e
qual
ity if
f the
Xi's
are
inde
pend
ent.
Thi
s ill
ustr
ates
how
dep
ende
nce
amon
g th
e X
i's (
equi
vale
ntly
mem
ory
in th
eso
urce
) r
educ
es th
e va
lue
of β
k.
Z-4
3
(8)
Sup
pose
X1,
...,X
k a
re G
auss
ian
(a)
Inde
pend
ent c
ase
β X,k
= 2
π (k+
2 k)(
k+2)
/2
∏ i=
1k σ
2 i 1/
k
σ2 X
Der
ivat
ion:
Fro
m (
5)
β X,k
=
1 σ2 X
∏ i=1k
⌡⌠
1 √2π
σ2 i
exp
-
x2 2σ2 i
k/(k
+2)
dx
(k+
2)/k
= 1 σ2 X
∏ i=1k
⌡⌠
1
√2πσ
2 i k
+2 k
exp
{-
x2
2σ2 i
k+
2 k}k
(2
πσ2 i
k+
2 k)1
/2
(2π
σ2 i)k/
(2(k
+2)
)
dx
(k+
2)/k
=
1 σ2 X
∏ i=
1k
2π
σ2 i1
/(k+
2) (
k+2 k)1/
2(k
+2)
/k
= 2
π (k+
2 k)( k
+2)
/2
∏ i=
1k σ
2 i 1/
k
σ2 X
Z-4
4
(b)
IID
Gau
ssia
n ca
se
β k =
2π
k+2 k
(k+2
)/2 →
β ∞
= 2
πe =
17.
1 as
k→
∞
In fa
ct, t
he β
k's d
ecre
ase
mon
oton
ical
ly u
p to
β∞
Not
e:
k+2 k
(k+
2)/2
= e
xp{k+
22
ln k+
2 k}
= e
xp{k+
22
ln (
1+ 2 k)
}
≅
ex
p{k+
22
2 k} =
exp
{k+2 k} →
e
Z-4
5
(c)
Cor
rela
ted
Gau
ssia
n ra
ndom
vec
tor
with
cov
aria
nce
mat
rix K
β k =
2π
k+2 k
(k+
2)/2
|K|1
/k
σ2
Der
ivat
ion:
We
assu
me
E X
= 0
bec
ause
the
mea
n do
es n
ot a
ffect
β.
We
find
βk
by
tran
sfor
min
g X
to
an in
depe
nden
t vec
tor
U v
ia a
n or
thog
onal
tran
sfor
m.
Fro
m (
3),
β k =
βU
,k;
β U,k
can
be
foun
d fr
om (
a) a
bove
.
Acc
ordi
ngly
, let
U =
A X
, w
here
A is
the
Kar
hune
n-Lo
eve
tran
sfor
m, i
.e. i
tsro
ws
z1,
…,z
k a
re a
n or
thon
orm
al s
et o
f eig
enve
ctor
s fo
r K
. Le
t λ 1
,…,λ
k b
eth
e co
rres
pond
ing
eige
nval
ues.
The
n
KU =
E U
Ut
= E
AX
Xt A
t =
A E
X X
t At
= A
KX A
t
= A
[λ1
z 1 …
λk
z k]
=
λ
1
.
.
λk
from
whi
ch w
e se
e th
at U
is
unco
rrel
ated
and
, als
o, in
depe
nden
t sin
ce it
isG
auss
ian.
Usi
ng th
e fa
ct th
at A
is
orth
onor
mal
and
(a)
abo
ve, w
e ha
ve
β X,k
= β
U,k
= 2
π (k+
2 k)(
k+2)
/2
∏ i=
1k σ
2 i 1/
k
σ2 X
= 2
π
k+
2 k ( k
+2)
/2 |Κ
|1/k
σ2
Z-4
6
(9)
Uni
form
den
sity
(a)
Inde
pend
ent
(eac
h X
i un
iform
on
som
e in
terv
al)
β k =
12
( ∏ i=1k σ
2 i)1/
k
σ2
(b)
IID
uni
form
on
an in
terv
al
β k =
12
Not
ice
that
it is
the
sam
e fo
r al
l k.
(c)
Uni
form
on
an a
rbitr
ary
k-di
men
sion
al s
et B
β k =
vol(B
)2/k
σ2
(10)
Lapl
acia
n de
nsity
(
f X(x
) =
1 √2 e
- √2|
x|
, σ2
= 1
)
(a)
Inde
pend
ent
(b)
IID β k
= 2
k+2 k
k+2 →
β ∞
= 2
e2
= 14
.8
as
k →
∞
Z-4
7
Asy
mpt
otic
Pro
pert
ies
of O
ptim
al Q
uant
izer
s
Let
Sx
den
ote
the
cell
cont
aini
ng x
.
•C
ell
volu
me
|Sx|
≅
1M
λ* k(
x) =
c
M f X
(x)k/
(k+
2)
Sm
alle
r w
here
f i
s la
rger
, whi
ch is
not
sur
pris
ing.
•C
ell
pro
bab
ility
Pr(
Sx)
≅
f X(x
) |S
x| ≅
fX(x
) c M
fX(x
)-k/(
k+2)
=
c M f
X(x
)2/(k
+2)
Larg
er w
here
f i
s la
rger
Z-4
8
•C
ell
dis
tort
ion
1 k ⌡⌠ Sx ||
x-Q
(x')|
|2 f X
(x')
dx'
≅ 1 k
f X(x
) ⌡⌠ Sx ||
x'-Q
(x')|
|2 d
x'
= 1 k
f X(x
) k m
* k |S
x|(k
+2)
/k
=
f X(x
) m* k
c M f X
(x)(-
k/k+
2)
(k+
2)/k
= m
* k c M
Sam
e fo
r al
l x;
i.e
. all
cells
con
trib
ute
the
sam
e to
the
dist
ortio
n.
•C
on
dit
ion
al c
ell
dis
tort
ion
1 k ⌡⌠ Sx ||
x-Q
(x')|
|2 f X
(x'|X
∈S
x) d
x' =
1 k ⌡⌠ Sx ||
x-Q
(x')|
|2
f X(x
')P
r(S
x) d
x'
=
1P
r(S
x) m
* k c M
= M c
f X(x
)-2/(
k+2)
m* k
c M
Inve
rsel
y pr
opor
tiona
l to
cell
prob
abili
ty.
Sm
alle
r w
here
f i
s la
rger
.