cognitive dynamic systems
TRANSCRIPT
1
s
COST2102, Spain, Sept. 2009 (Haykin)
Cognitive Dynamic System
Simon HaykinCognitive Systems Laboratory
McMaster, UniversityHamilton, Ontario, Canada
email: [email protected] site: http://soma.mcmaster.ca
2
ed
rks
COST2102, Spain, Sept. 2009 (Haykin)
Outline of The Lecture
1. Cognition
2. New Generation of Engineering Systems Enablwith Cognition
3. Exponential Growth of Cognitive Radio
4. Cognitive Radio Signal-processing Cycle
5. Spectrum Sensing
6. Transmit-power Control
7. Emergent Behaviour of Cognitive Radio Netwo
8. Final Remarks
3
dynamic system
e cognitive it must have
COST2102, Spain, Sept. 2009 (Haykin)
1. Cognition
The Human Brain
• The human brain is the most powerful cognitivein existence.
• For an “artificial” dynamic system to assume thcapabilities of the human brain, at the minimumthe capacity to perform the following tasks:
(i) learning and memory;(ii) planning;(iii) attention; and(iv) interaction with the world (environment)
4
ceptionensing)eption
Observations
COST2102, Spain, Sept. 2009 (Haykin)
The Perception-Action Cycle
Figure 1:
The nvironment
er(S
ction(Control)
Feedback Channel FFeedback Channel
Action Perc
The Environment
5
:
COST2102, Spain, Sept. 2009 (Haykin)
Explanatory Notes on the Perception-Action Cycle
1. Perception of the Enviornment involves:
• learning and memory;• attention
2. Action performed on the environment involves:
• planning; and• control
6
f interactions of
Prefrontalcortex
Polymodallassociationcortex
Motorhierarchy
Sensoryhierarchy
Working memory
COST2102, Spain, Sept. 2009 (Haykin)
Distributed Feedback:Fundamental Principle of Biology
This principle is embodied in the cybernetic cycle othe brain with its environment:
Figure 2: Cybernetic cycle of the brain(This figure is adapted from J.M. Fuster, 2003).
Environment
Premotorcortex
Primarymotorcortex
Unimodalassociationcortex
Primarysensorycortex
Actions on theEnvironment
Observations ofthe Environment
7
down into a
e world
COST2102, Spain, Sept. 2009 (Haykin)
Message taken from the brain’s cybernetic cycle
(i) Engineering paradigm:“Divide and Conquer”
whereby a highly complex problem is brokennumber of simpler ones.
(ii) Extraction of features of features of the outsid(Selfridge, 1958).
(iii) Hierarchical structure.
(iv) Global as well as local feedback.
Concluding Remark:Global Feedback is the Facilitator of Cognition
8
g Systems
y retain its
the designat will have
COST2102, Spain, Sept. 2009 (Haykin)
2. New Generation of EngineerinEnabled with Cognition
In the context of engineering systems,
• adaptivity was a hallmark of the 20th century• in the 21st century, local feedback will naturall
engineering importance, however:
It will be cognition that will play center stage inof the next generation of engineering systems thsocietal impacts on many fronts.
9
COST2102, Spain, Sept. 2009 (Haykin)Examples of Cognitive Dynamic Systems
(i) In my Cognitive Systems Laboratory atMcMaster University:
Cognitive Mobile Assistants:Memory-impaired patientsSocial networking
Systems:Cognitive radioCognitive radarCognitive energy systems
(ii) Other Examples:
Cognitive ComputingCognitive Car
10
tive Radio
COST2102, Spain, Sept. 2009 (Haykin)
3. Exponential Growth of Cogni
Figure 3:(Scanning the Issue, Proc. IEEE, April 2009).
11
ons commissionhe ATSC-digital devices.
for the creation
radios.
an-made noise (e.g., ignition-engine
COST2102, Spain, Sept. 2009 (Haykin)
Cognitive Radio is fast becoming a reality:
Digital Television Band
In November 2008, the Federal Communicati(FCC) in the United States ruled that access to ttelevision (DTV) band be permitted for wireless
For the first time ever, the way has been opened
of white spaces1 for use by low-power cognitive
1. In reality, the spectrum holes are not white spaces due to the unavoidable presence of interferers and/or the generation of mnoise).
12
sing Cycle
umor
Received signal
Receiver
nvironmental information
COST2102, Spain, Sept. 2009 (Haykin)
4. Cognitive Radio Signal-proces
Figure 4:
}Users
Radio Environment (Forward channel)
Dynamicspectrum manager
Spectr sens
Feedback link
. . .
Transmittedsignal
E
Transmitter
Compressedversion ofenvironmentaldata
13
sed on
timator:
COST2102, Spain, Sept. 2009 (Haykin)
5. Spectrum Sensing:
Parametric methods: Binary hypothesis-testing ba
• Pilot detection;• energy detection; and• cyclostationarity
Nonparametric (model-independent) Spectrum Es
• Multitaper method
14
ment
fficient,g
COST2102, Spain, Sept. 2009 (Haykin)
Spectrum Sensing: Problem State
Objective:
Design a spectrum sensor that is computationally eembodying the three essential dimensions of sensin
• time;
• frequency; and
• space
in a coherently integrated fashion
15
Sensing
ce spectral
interferers;um;kelihood sense;tionarity
d
gent
COST2102, Spain, Sept. 2009 (Haykin)
Desirable Attributes of Spectrum
• nonparametric formulation;accurate identification of spectrum holes, henresolution;
• accurate estimation of the angles of arrival ofhence improved utilization of the radio spectr
• near-optimal performance in the maximum-li• signal classification by exploiting the cyclosta
property of communication signals;• regularization, hence inmproved stability; an• quick computation in a cost-effective manner.
The multi-taper method satisfies these rather strinrequirements, (Thomson, 1982).
16
the
eentratione sample-
signalrty.
COST2102, Spain, Sept. 2009 (Haykin)
Important Property of Slepian Sequences on whichmulti-taper method (MTM) is based:
The Fourier transform of a Slepian sequenc(window) has the maximal energy concinside a prescribed bandwidth under a finitsize constraint.
Simply put, there is no other window in the digital processing (DSP) literature that satisfies this prope
17
25 30
olution units
Hamming
Slepian
COST2102, Spain, Sept. 2009 (Haykin)
0 5 10 15 20
Frequency in Rayleigh Res
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Slepian and Hamming Spectral Windows, dB
Figure 5: Comparison
with the Hammingof the Slepian Window
Window
18
in an“automatic”
nents:g inside the
utside this
antify tradeoff
ing on theer of data points
form of
COST2102, Spain, Sept. 2009 (Haykin)
Attributes of Multi-taper Spectral Estimation
(i) The multi-taper spectral estimator is applicablefashion.
(ii) The bias is decomposed into two quantifiable compo• local bias, due to frequency components residin
band f - W to f + W;• broadband bias, due to frequency components o
band.(iii) Multi-taper spectral estimator offers an easy-to-qu
between bias and variance.(iv) The degrees of freedom vary from six to ten, depend
time-bandwidth product NW, where N is the numband 2W is the bandwidth
(v) Multi-taper spectral estimation may be viewed as aregularization.
19
es between the
ve theory assumingstationarity
rier theory assumingostationarity
COST2102, Spain, Sept. 2009 (Haykin)
Figure 6: Illustrating the one-to-one correspondencLoéve and Fourier theories for cyclostationarity
Crosscorrelatorwithaveragingperformedoverthe set ofK Slepian tapers
Estimateof Loèvespectralcorrelation
γL(f1, f2)^
Slepian tapervk(t)
Slepian tapervk(t)exp(-j2πf1t)
Time series x(t) .
exp(-j2πf2t)
Multi-taper method
Multi-taper method
Mid-band frequency f
exp(-jπαt)
.
exp(jπαt)
T2
< t < T2
__
_
Narrow-band filter
Narrow-band filter
XT (f + α/2)α
XT (f - α/2)α
Cross-correlatorwithaveragingperformedovertime
FourierspectralcorrelationSα (f)
Sk(f1)
Sk(f2)
intervalT
Estimate ofCyclostationary signal x(t):
(a) Loè non
(b) Fou cycl
20
dner’s framework,ary processes with(having the sameof cyclostationary
omson’s framework
ses; andlemma through the
the time-frequencyity to adapt to the
COST2102, Spain, Sept. 2009 (Haykin)
Contrasting Two Theories on Cyclostationarity
The traditional treatment of cyclostationarity follows Garrooted in the traditional Fourier-transform theory of stationan important modification: introduction of parameter αdimension as frequency) in the statistical characterizationprocesses.
The more rigorous treatment of cyclostationarity follows Thwhich combines the following two approaches:
• the Loève transform for dealing with nonstationary proces• the multitaper method for resolving the bias-variance di
use of Slepian sequences.
This two-pronged mathematically rigorous theory foranalysis of nonstationary processes has a built-in capabilunderlying statistical periodicity of the signal under study.
Simply put, it is nonparametric and therefore robust.
21
pping curves: theue)
6 8 10ffset in MHz
COST2102, Spain, Sept. 2009 (Haykin)
Experimental Results: Digital TV
When the figure is expanded, we see two closely overlaarithmetic mean (upper black) and geometric mean (bl
0 2 4Frequency o
-90
-80
-70
-60
-50
-40
-30
-20
-10
Min, GM, AM, Max Spectrum Estimates, dB
Figure 7: Multi-taper spectralestimates of real-life ATSC-DTVdata.
The lower (green) and upper (blue)curves represent the minimum andmaximum estimates over 20 sections,with each section containing 2,200samples (i.e., 110 microseconds).
22
operativeization
andinty set.
ving the
COST2102, Spain, Sept. 2009 (Haykin)
6. Transmit-power Control
Objective:
Design a transmit-power controller within a non-cogame-theoretic framework (i.e., set of convex optimproblems):
• max-min optimization for robustification;• worst-case analysis on a specified uncerta
Iterative water-filling (IWA) is one approach of solproblem.
23
COST2102, Spain, Sept. 2009 (Haykin)Figure 8: Resource allocation results
gains are changed randomly todisappears, and interference
of robust iterative water-fillingalgorithm (IWFA), when 2 new usersjoin a network of 5 users, a subcarrier
address the mobility of the users.
24
ersion
COST2102, Spain, Sept. 2009 (Haykin)
Figure 9: Comparison of the two IWF algorithms and its new fast v (a) classical version
(b) fast version (new)
(b)
(a)
25
ive
and time- involves:
erized by a
COST2102, Spain, Sept. 2009 (Haykin)
7. Emergent Behaviour of CognitRadio Networks
The cognitive radio network is a hybrid, nonlinearvarying closed-loop feedback control system, which
• continuous dynamics, and
• discrete events.
Above all, the network is a complex system charactcomplicated and irreducible phenomenon, namely:
Emergent Behaviour
26
rs and other
s.
ive-radio users.
n across ther) and the feed-
COST2102, Spain, Sept. 2009 (Haykin)
Limiting (critical) factors affecting the emergentbehaviour of the network:
(i) Nonstationary character of spectrum holes.
(ii) Interfering signals from primary (legacy) usesecondary (cognitive radio) users.
(iii) Time-varying number of cognitive radio user
(iv) Unpredictable human behaviour of the cognit
(v) Transmission delay of wireless communicatioforward channel (from transmitter to receive
back link (from receiver to transmitter).2
2. The use of diversity is assumed to tame the notoriously unreliable behaviour of the channels.
27
rk
ient utilization of thedary (cognitive radio)
condary user, leading
e., traffic jams, chaos,
s of a large number of
nore the underlying
COST2102, Spain, Sept. 2009 (Haykin)
Two Kinds of Behaviour in a Self-organized Netwo
(i) Positive Emergent Behaviour
This first kind of behaviour embodies a harmonious and efficradio spectrum by all primary (legacy) users as well as seconusers with minimal coordination.
Underlying Protocol: Best response strategy adopted by each seto a Nash equilibrium.
(ii) Negative Emergent Behaviour
This second kind of behaviour is characterized by disorder (i.and wasted subbands of the radio spectrum).
It can arise due to one of two factors:
• inadequate number of spectrum holes to cope with the needsecondary users; or
• one or more greedy secondary users who, knowingly, igprotocol of the network.
28
s of a model of
cognitive radioperformance of
COST2102, Spain, Sept. 2009 (Haykin)
Study of the Emergent Behaviour of the Network
(i) Theoretical Study:
The emergent behavior is being studied on the basithe network, using control theory.
(ii) Experimental Study using Software Testbed3:
In the final analysis, emergent behaviour of anetwork is the litmus test for assessing the overallthe network.
3. The testbed has been completed, and it’s ready for testing. .
29
amice large-scaleant
at the globaloptimal, andiven limited
aily basis.
COST2102, Spain, Sept. 2009 (Haykin)
8. Final Remarks
Large-scale Engineering Applications:
Cognition is the key to a new generation of dynsystems, which will open the way for innovativengineering applications that will make significdifferences to our daily lives.
Overall Sub-optimality as the Design Objective:
At the local level we may seek optimality, butlevel we may have to settle on the “best” sub-reliable solution for the application at hand, gresources.
This is precisely what the human brain does on a d
30
physics for new
uman brain for
ystems aimed at
COST2102, Spain, Sept. 2009 (Haykin)
Looking into the Past and the Future
In the 20th century, we looked to mathematics andand innovative ideas.
In the 21st century, we will be looking to the hinspiring ideas, supported by• signal processing;• control theory;• information theory;• mathematics;• physics;• biology;• evolutionary computation; and• the computer and computational thinkingfor the theory and design of cognitive dynamic slarge-scale engineering applications.
31
ith cognition asan brain.
e new cognitivesome functions
COST2102, Spain, Sept. 2009 (Haykin)
Interplay between Cognitive Dynamic Systemsand the Human Brain:
In pursuing a new generation of dynamic systems wthe enabler, we are naturally motivated by the hum
In the course of time, we may well find that thdynamic systems so engineered help us understandof the brain itself.
32
COST2102, Spain, Sept. 2009 (Haykin)New Book:
Simon Haykin, Cognitive Dynamic SystemsCambridge University Press, 2010.