research article fast channel selection strategy in...

Research ArticleFast Channel Selection Strategy in CognitiveWireless Sensor Networks

Yong Sun and Jian-sheng Qian

School of Information and Electrical Engineering (SIEE), China University of Mining and Technology (CUMT),Xuzhou 221008, China

Correspondence should be addressed to Yong Sun; [email protected] and Jian-sheng Qian; [email protected]

Received 25 April 2015; Revised 12 June 2015; Accepted 14 June 2015

Academic Editor: Ana Alejos

Copyright © 2015 Y. Sun and J.-s. Qian.This is an open access article distributed under the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

In order to meet the practical requirement for Cognitive Wireless Sensor Networks applications, this paper proposes innovativefast channel selection algorithm to solve the shortcomings of original Experience-Weighted Attraction algorithm’s complexity,higher energy consuming, and the nodes’ hardware restrictions of real-time data processing capabilities. Research is conductedby comparing channel selection differences and timeliness with traditional Experience-Weighted Attraction learning. Thoughnot as stable as traditional Experience-Weighted Attraction learning, fast channel selection algorithm has effectively reduced thecomplexity of the original algorithm and has superior performance than Q learning.

1. Introduction

Traditionally, the licensed radio spectrum allocations areregulated by official authorities. The public and governmentuse of radio spectrum is managed by the National Telecom-munications and Information Administration (NTIA) andthe Federal Communications Commission (FCC) is in chargeof commercial radio resources, respectively, in theUSA.Withmore and more applications of wireless devices, the rapidincreasing requisition for radio spectrum licensing has ledto current shortage of radio spectrum allocations and puttheir governing bodies into trouble. In fact, FCC’s recentresearch has shown that these fixed static frequency channelsare always idle or not occupied most of the time. Spectrumbands are not efficiently used or under utilization either ata temporal or on a geographical level. By seeking “spectrumholes” (unused frequency channels), Cognitive Radio (CR)can greatly improve the use efficiency of spectrum resourcesand solve these problems presented above in a “secondaryutilization” (with lower priority than legacy users) way.First introduced by Mitola III [1], Cognitive Radio (CR)is often considered as an extension and expansion of SoftRadio (SR), which is equipped by general hardware and

capable of programming to transmit and receive various radiowaves.

There has already been lots of research in many aspectsof CR. In sensing, Panahi and Ohtsuki [2] present a FuzzyQ Learning (FQL) based scheme for channel sensing in CRnetworks. Zhang et al. [3] proposed a novel detection algo-rithm in which the fractal box dimension is used when theSignal to Noise Ratio (SNR) is high, while the improved TCCalgorithm is used when the SNR is low, and Khalaf [4] formu-lated the detection problembased on the eigendecompositiontechnique. Hossain et al. [5] evaluated the performance ofcooperative spectrum sensingwith the hard combinationOR,AND, and MAJORITY rules. Bkassiny et al. [6] presented anautonomous CR architecture, referred to as the Radiobot, todetect and identify the sensed signals. Lunden et al. [7] alsodistributed multiuser multiband spectrum sensing policiesfor CR networks based onmultiagent reinforcement learningwhile Reinforcement Learning-Based Cooperative Sensing(RLCS) method was proposed to address the cooperationoverhead problem and improve cooperative gain in CRad hoc networks. [8] In channel allocation, Gallego et al.[9] presented a game theoretic solution for joint channelallocation and power control in CR networks analyzed under

Hindawi Publishing CorporationInternational Journal of Distributed Sensor NetworksVolume 2015, Article ID 171357, 6 pageshttp://dx.doi.org/10.1155/2015/171357

2 International Journal of Distributed Sensor Networks

the physical interference model. In channel access, Teng etal. [10] demonstrated a reinforcement learning-based doubleauction algorithm aiming to improve the performance ofdynamic spectrum access in CR networks. In security, Wanget al. [11] proposed a Four-Dimensional Continuous TimeMarkov Chain model to analyze the communication perfor-mance of normal Secondary Users under PUEAs, typicallyaffected by SMUs, and compared several PUEA detectionschemes.

As revolutionary development of Intelligent Radio (IR),CR implements Soft Radio by adding Knowledge Base, Rea-soning Engine, and Learning Engine to be an independentCognitive Engine (CE), which makes the radio capable oflearning and adapting to the surrounding radio environment[12]. Knowledge Base which stores variety of cases, relations,and rules can be seen as memory in human’s brain and isvery common in Artificial Intelligence (AI) logic planning.Just like expert system in Artificial Intelligence, ReasoningEngine executes all kinds of state information for referenceof Knowledge Base by logic thinking and then generatesprocessed results or actions to drive Soft Radio changingsetting parameters to adapt to changing environment. Asthe core component and key feature for CR implementation,Learning Engine is in charge of keeping Knowledge Baseupdated by accumulating new environmental experience intonew knowledge extension, which is what differentiates CRfrom traditional preprogrammed ones.

There are varieties of learning algorithms available forCR, including neural networks, genetic models, and hiddenMarkov algorithms [13]. Bkassiny et al. characterized thelearning problem in CR and state the importance of ArtificialIntelligence in achieving real cognitive communications sys-tems [14] and proposed a Bayesian nonparametric signal clas-sification approach for spectrum sensing in CR [15]. Bizhaniand Ghasemi [16] used Multiresponse Learning Automata(MRLA) to control how Secondary Users should accessthe licensed primary channels in CR networks. Tsagkariset al. [17] used neural network-based learning to predictdata bit rate of CR. Galindo-Serrano and Giupponi [18]proposed a form of real-time decentralized Q learning tomanage the aggregated interference generated by multipleWRAN systems. Li [19] applied Multiagent ReinforcementLeaning (MARL) for the Secondary Users to learn goodstrategies of channel selection. Chen et al. [20] presented anintelligent policy based on reinforcement learning to acquirethe stochastic behavior of Primary Users (PUs). Zhang andLiu [21] obtained the capability of iteratively online learningenvironment performance by using Reinforcement Learning(RL) algorithm after observing the variability and uncertaintyof the heterogeneous wireless networks. Gallego et al. [9]provided no-regret learning algorithms to perform the jointchannel and power allocation and overcome the convergencelimitations of the local game. Zhu et al. [22] employedReinforcement Learning (RL) approach to find a near-optimal policy under undiscovered environment. Torkestaniand Meybodi [23] proposed the learning automata-basedCR to address the spectrum scarcity challenges in wirelessad hoc networks. Yang and Grace [24] improved channelassignment in multicast terrestrial communication systems

with distributed channel occupancy detection by using intel-ligence based on reinforcement learning and transmitterpower adjustment. Zhou et al. [25] designed a robust dis-tributed power control algorithm with low implementationcomplexity for CR networks through reinforcement learning,which does not require the interference channel and powerstrategy information among Secondary Users (SUs) and fromSUs users to PUs.

However, as known with our best effort till now, littlefocus has been placed on implementing Learning Engine ofCR with Experience-Weighted Attraction (EWA) algorithms.The innovative proposed channel selection algorithm basedon EWA learning [26, 27] allows cognition to learn radioenvironment communication channel characteristics online.By accumulating the history channel experience, it can pre-dict, select, and change the current optimal communicationchannel, dynamically ensure the quality of communicationlinks, and finally reduce system communication outage prob-ability. The effectiveness of this algorithm has been validatedby simple probability method [26] and with handoff scheme[27] in our preliminary studies. However, it is not appli-cable for processing capability and power-restricted nodesof Wireless Sensors Networks (WSNs) due to original EWAalgorithm’s high complexity and energy consuming. Based onour lots of earlier research, the study focus has been shifted tofast channel selection algorithm EWAS with low complexityand green energy. The rest of this paper is presented asfollows. In Section 2, EWAS algorithms will be introducedin full detail; then the simulation results comparison andanalysis are presented in Section 3. In the end, the conclusioncomes in Section 4.

2. Fast Cognitive Channel Selection Model

In the problem of radio communication channel selection,different wireless channels should have different channelavailabilities; that is, the idle probabilities 𝛼 of differencechannel should not be the same for CR. Assuming radiopropagation environment can be divided into 𝑛 channels,then the idle probability of channel 𝑖 (1 ≤ 𝑖 ≤ 𝑛) can beexpressed as 𝛼

𝑖, or Α = [𝛼

1, 𝛼2, . . . , 𝛼

𝑛−1, 𝛼𝑛] in vector form.

Let 𝛽𝑖be the successful transmission probability of channel

𝑖 (1 ≤ 𝑖 ≤ 𝑛); then Β = {𝛽1, 𝛽2, . . . , 𝛽

𝑛−1, 𝛽𝑛}. Think of the

radio channel characteristics change over time; the channelidle probability and successful transmission probability ofchannel 𝑖 (1 ≤ 𝑖 ≤ 𝑛) should not be the same at differenttime 𝑡; then the forms of probabilities after introducing timeparameter 𝑡 are Α(𝑡) = {𝛼

1(𝑡), 𝛼2(𝑡), . . . , 𝛼

𝑛−1(𝑡), 𝛼𝑛(𝑡)} and

Β(𝑡) = {𝛽1(𝑡), 𝛽2(𝑡), . . . , 𝛽

𝑛−1(𝑡), 𝛽𝑛(𝑡)}, respectively.

To reduce the complexity of channel selection strategybased on EWA learning algorithm, exponential operationshould be firstly avoided. Next the fast algorithm shouldsimplify the calculation procedure and optimize and updatethe objective function directly in ideal. This paper calculatesand carries iterative operation directly on channel selectionprobabilities and innovatively proposes a fast simplifiedcognitive channel selection algorithm EWAS.

International Journal of Distributed Sensor Networks 3

Define the probability of selecting channel 𝑗 in channelpreferable selection policy 𝑠

𝑗

𝑖at time 𝑡 as 𝑃

𝑗

𝑖(𝑡); then the

mathematical expression of 𝑃𝑗𝑖(𝑡) is

𝑃𝑗

𝑖(𝑡 + 1) =

1 − 𝜎

1 − 𝜎 ⋅ {1 − 𝐼 [𝑠𝑗

𝑖, 𝑠𝑖 (𝑡)]}

⋅⟨

(1 − 𝜏) ⋅ 𝑃𝑗

𝑖(𝑡)

1 − 𝜎 ⋅ {1 − 𝜋𝑖[𝑠𝑗

𝑖, s−𝑖 (

𝑡)]}

+ 𝜋𝑖[𝑠𝑗

𝑖, s−𝑖 (

𝑡)]

⋅ 𝜏⟩+ 𝐼 [1, 𝑥 (𝑗)] ⋅ 𝐼 [𝑠𝑗𝑖, 𝑠𝑖 (𝑡)] ⋅ 𝜋𝑖

[𝑠𝑗

𝑖, s−𝑖 (

𝑡)] ⋅ 𝜎,

(1)

where

𝑥 (𝑗) =

{

{

{

0, Transmission failure on channel 𝑗,

1, Successful transmission on channel 𝑗,

𝜋𝑖[𝑠𝑗

𝑖, s−𝑖 (

𝑡)] =

{

{

{

0, channel 𝑗 is sensed busy,

1, channel 𝑗 is sensed idle

(2)

and 𝐼[⋅] is the indicator function, which is defined as follows:

𝐼 (𝑥, 𝑦) =

{

{

{

1, 𝑥 = 𝑦,

0, 𝑥 = 𝑦.

(3)

Parameters 𝜎 and 𝜏 are attenuation coefficients of proba-bility and 𝜎 < 𝜏 ∈ (0, 1). As can be seen through in-depthanalysis of (1), in the period of radio environment sensingof CR, when perceiving the current state of the channel 𝑗being busy (strong electromagnetic noise over interferencethreshold for transmission), the state flag status is set to0 (unavailable), and the strategy of selecting channel 𝑗 fortransmission channel will get no payoff, or the award functionvalue of 𝜋

𝑖[𝑠𝑗

𝑖, s−𝑖(𝑡)] is 0 and channel selection probability

declines to (1 − 𝜏) ⋅ 𝑃𝑗

𝑖(𝑡); while perceiving the current state

of the channel 𝑗 being idle (electromagnetic noise belowinterference threshold for transmission), the state flag statusis set to 1 (available), and the strategy of selecting channel 𝑗for transmission channel will get the payoff of 𝜋

𝑖[𝑠𝑗

𝑖, s−𝑖(𝑡)],

respectively. In addition, the value of 𝜋𝑖[𝑠𝑗

𝑖, s−𝑖(𝑡)] is assumed

to equal 1 and channel selection probability is updated to(1 − 𝜏) ⋅ 𝑃

𝑗

𝑖(𝑡) + 𝜏.

These available channels are candidate channels for chan-nel selection of CR, and the candidate channel with thehighest probability (if more than one channel reaches thehighest selection probability, then one of these channels willbe selected randomly) of channel selection will be chosenfor transmission. After successful transmission, this channelselection probability will go up to (1−𝜎)⋅[(1−𝜏)⋅𝑃𝑗

𝑖(𝑡)+𝜏]+𝜎.

But if the transmission is unfortunately failure, the channelselection probability will be (1 − 𝜎) ⋅ [(1 − 𝜏) ⋅ 𝑃

𝑗

𝑖(𝑡) + 𝜏].

At this point, it can be seen that the complexity of EWASfast channel selection algorithm is O(𝑛). Due to exponentia-tion operation, EWA’s complexity isO(𝑛2) eventually.

3. Results and Discussion

Assume the number of channels in simulation environmentis 5, or 𝑛 = 5. For the coefficients, 𝜏 is set to the defaultvalue 0.1 according to general experience. Since the value of𝜎 should be lower than parameter 𝜏, we pick half value of𝜏𝛿 for coefficient 𝜎 in this paper; that is, 𝜎 = 𝜏/2 = 0.05.While there shall be some differences between each channel,the idle probabilities of these channelswill not be the same. Toreflect the general channels’ available probabilities, uniformdistribution vector in the range of 0 to 1 will be selected for theidle probability of each channel; that is, the initial channel idleprobability vectorΑ0 = {0.4, 0.9, 0.6, 0.5, 0.7}, while the initialchannel successful transmission probability vector Β0 =

{3/4, 8/9, 5/6, 4/5, 6/7}. Then the initial channel availableprobability vector Γ0 = Α0 ⋅ Β0 = {0.3, 0.8, 0.5, 0.4, 0.6}. Inorder to verify that this intelligent algorithm is capableof deciding and guiding CR real-time switch to the newtransmission channel with the highest available probabilityonline accurately, the channel idle probability vector willchange to Α1 = {0.6, 0.4, 0.7, 0.9, 0.5}, and the channel suc-cessful transmission probability vector will change to Β1 =

{5/6, 3/4, 6/7, 8/9, 4/5} after 33 rounds during the simulationprocess. Therefore the channel available probability vectorwill be Γ1 = Α1 ⋅ Β1 = {0.5, 0.3, 0.6, 0.8, 0.4} after thesimulation environment change. Taking suddenness and ran-domness of the above parameters under actual wireless envi-ronment into account, the value generated in each simulationround meets exponential distribution of the correspondingparameter above followed by the general rule.

In this paper, a simple repeated experimental method isapplied to verify the effectiveness of probability of channelselection algorithm based on EWA learning. That is, Turn-Based Strategy (TBS), a single uniformly distributed randomnumber within range [0, 1], is generated in each round. Ifthis number is less than the channel available probability𝛼𝑖, channel 𝑖 is judged as idle available state; else it is busy

unavailable state. Idle channel with the highest selectionprobability will be the preferable communication channel inthe current round. If more than one channel reaches thehighest probability of channel selection, then one of thesechannels will be selected randomly. After algorithm selectspreferable channel 𝑗, a single uniformly distributed randomnumberwithin range [0, 1] is also generated. Communicationchannel transmission is successful if this number is less thanthe probability of successful data transfer completion 𝛽

𝑗;

otherwise it fails.After the parameters above are set, the track records of

channel selection probability based on EWA learning areshown in Figure 1.

In Figure 1, EWAS learning algorithm randomly selectschannel 5 as the access channel in the condition of the sameinitial channel selection probabilities. After short initializa-tion process, EWAS learning algorithm can successfully trackand lock channel 2 as its preferable channel and its selectionprobability fluctuates slightly around 0.87. For the reason ofchannel availability, probability changes after 36th round andthe selection probability of channel 2 falls dramatically, whilethe selection probability of channel 4 increases, respectively,


0 20 40 60 80 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Channel selection probability based on EWAS learning

Iteration number

Chan

nel s

elect

ion

prob

abili

ty

Channel 1Channel 2Channel 3

Channel 4Channel 5

Figure 1: Channel selection probability based on EWA learning.

and steadily overtakes the selection probability of channel 2after 40 rounds. Channel 4 eventually replaces channel 2 tobecome optimal access channel under new channel availableprobability states.

In order to highlight better performance of channelselection algorithm based on EWA learning than othertraditional radio with fixed transmission channel, the timesof availability to access the channel and successful completionof transmissions in 100 rounds are collected in 3 scenes:fixed channel 2 as transmission channel, fixed channel 4 astransmission channel, and channel selection algorithm basedon Q learning channel selection algorithm based on EWAlearning and channel selection algorithm based on EWASlearning. The statistical data is compared in Figure 2.

The number of availabilities to access the channel withfixed channel 2 as transmission channel is 63, and the numberof successful completion of transmissions with fixed channel2 as transmission channel is 54; the number of availabilitiesto access the channel with fixed channel 4 as transmissionchannel is 74, and the number of successful completion oftransmissions with fixed channel 4 as transmission channelis 65.The numbers of availabilities to access the channel withchannel selection algorithm based on Q, EWA, and EWASlearning are the same as 100with no block, but the numbers ofsuccessful completion of transmissions are 80 for Q learning,81 for EWAS, and 82 for EWA learning, respectively. Finally,the probability of successful completion of transmission withchannel selection algorithm based on EWA learning is 81%,much higher than that of channel 2 (54%) and channel 4(65%). By evident statistical comparison, channel selectionalgorithm based on EWAS learning can greatly improve theprobabilities of successful channel access and transmissioncompletion, much the same as Q (80%) and EWA (82%)

Chan

nel2

Chan

nel4

Q le

arni

ng

EWA

S

EWA

0

20

40

60

80

100

120

Tim

es

Comparison between EWA/EWAS learning and reference ploys

SuccessAvailable

54

63 65

74

80

100

81

100

82

100

Figure 2: Comparison between EWA/EWAS learning and referenceploys.

0 10 20 30 40 50 60 70 80 90 1000

1

2

3

4

5

6

Channel selection based on Q and EWA/EWAS learning

Iteration number

Chan

nel n

umbe

r

Q learningEWA learning

EWAS learning

Figure 3: The tracks of channel selection based on EWA learningand Q learning.

learning. But its advantage has more intuitive reflection onthe comparison chart of real-time channel selection below.

In order to highlight better performance of channelselection algorithmbased onEWAS learning thanQ learning,the channel selection tracks based on both learning algo-rithms are recorded under the same initial states and radioenvironments. The results are illustrated in Figure 3.

Note that channel number 0 indicates that full channelblocking occurs, which means all the channels are in busystates and are not available for communication which isthe situation in the 21st round. The differences betweenEWAS learning and EWA learning algorithms are mainly

International Journal of Distributed Sensor Networks 5

Table 1: The probability table of channel selection based on EWAS learning.

Iteration number 85 86 87 88 89 90 91Channel 1 0.5304 0.5774 0.6197 0.6577 0.6919 0.7227 0.6505Channel 2 0.4355 0.3920 0.3528 0.3175 0.3858 0.4472 0.4025Channel 3 0.7985 0.8187 0.8449 0.8674 0.8867 0.9031 0.8171Channel 4 0.8341 0.8131 0.8318 0.8486 0.8638 0.8774 0.8897Channel 5 0.4589 0.4130 0.3717 0.3345 0.4011 0.4610 0.4149

Table 2: The probability table of channel selection based on EWA learning.

Iteration number 85 86 87 88 89 90 91Channel 1 0.1802 0.1862 0.1887 0.1909 0.1900 0.1892 0.1862Channel 2 0.1731 0.1717 0.1677 0.1641 0.1659 0.1674 0.1668Channel 3 0.2033 0.2075 0.2080 0.2084 0.2057 0.2032 0.1985Channel 4 0.2690 0.2617 0.2669 0.2716 0.2718 0.2719 0.2811Channel 5 0.1745 0.1729 0.1687 0.1650 0.1667 0.1682 0.1675

in the transition period (36th–45th round) of switchingchannel from former selected channel 2 to new optimalchannel 4, and this reflects the differences between thesetwo different algorithms. However sudden channel change offast algorithm in the 86th round arouses our big interest. Inorder to analyze the reason of this phenomenon, the channelselection probability records after each round are derived andshown in Tables 1 and 2.

Table 1 records channel selection probabilities valuescalculated by EWAS algorithm after each round, while Table 2presents channel selection probabilities values based onEWA learning. It can be seen from Table 1 that selectionprobabilities of channel 3 and channel 4 are very close toeach other from the 85th round to the 91st round. Aftertransmission failure of preferred channel 4 in the 85th round,selection probability of current channel 4 falls from 0.8341 to0.8131, while selection probability of channel 3 increases from0.7985 up to 0.8187 and weakly overtakes channel 4 to be newselected transmission channel by EWAS fast algorithm. Evenif the same trend in the probability changes, the selectionprobability of channel 4 calculated by EWA learning is stillthe largest of all in the 86th round and channel 4 beingthe optimal transmission channel remains unchanged. Insummary, channel selection based on EWAS fast algorithmhas the same performance in fast tracking, locking, andswitching to the current optimal channel from changingcommunication environment and is superior to Q learningalgorithmevennot asmuch stable as original EWAalgorithm.

4. Conclusion

In this paper, an innovative fast channel selection algorithmEWAS is proposed to solve the shortcomings of originalEWA algorithm’s complexity, higher energy consuming, andthe nodes’ hardware restrictions of real-time data processingcapabilities in order to meet the practical requirement forCognitive Wireless Sensor Networks (CWSNs) application.Research is conducted by comparing channel selection dif-ferences and timeliness with traditional Q learning and

EWA algorithm. Though not as stable as EWA learning, fastchannel selection algorithm EWAS has effectively reducedthe complexity of the original algorithm and has superiorperformance than Q learning. However, EWAS algorithm isof passive channel detection and access; future research islying on active channel state prediction and reallocation inapplication of Wireless Sensor Network (WSN).

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper.

Acknowledgment

This work was supported by the National High TechnologyResearch andDevelopment Program of China (863 Program)(no. 2012AA062103).

References

[1] J. Mitola III, “Cognitive radio for flexible mobile multimediacommunications,”Mobile Networks and Applications, vol. 6, no.5, pp. 435–441, 2001.

[2] F. H. Panahi and T. Ohtsuki, “Optimal channel-sensing schemefor cognitive radio systems based on fuzzy q-learning,” IEICETransactions on Communications, vol. 97, no. 2, pp. 283–294,2014.

[3] D. Zhang, K. Li, and L. Xiao, “An improved cognitive radio spec-trum sensing algorithm,” TELKOMNIKA Indonesian Journal ofElectrical Engineering, vol. 11, no. 2, pp. 583–590, 2013.

[4] G. A. Khalaf, “An optimal sinsing algorithm for multibandcognitive radio network,” International Journal of Informationand Network Security, vol. 2, no. 1, pp. 60–67, 2013.

[5] M. S. Hossain, M. I. Abdullah, and M. A. Hossain, “Hardcombination data fusion for cooperative spectrum sensing incognitive radio,” International Journal of Electrical and Com-puter Engineering, vol. 2, no. 6, pp. 811–818, 2012.

[6] M. Bkassiny, S. K. Jayaweera, Y. Li, and K. A. Avery, “Widebandspectrum sensing and non-parametric signal classification for


autonomous self-learning cognitive radios,” IEEE Transactionson Wireless Communications, vol. 11, no. 7, pp. 2596–2605, 2012.

[7] J. Lunden, S. R. Kulkarni, V. Koivunen, and H. V. Poor,“Multiagent reinforcement learning based spectrum sensingpolicies for cognitive radio networks,” IEEE Journal on SelectedTopics in Signal Processing, vol. 7, no. 5, pp. 858–868, 2013.

[8] B. F. Lo and I. F. Akyildiz, “Reinforcement learning for cooper-ative sensing gain in cognitive radio ad hoc networks,”WirelessNetworks, vol. 19, no. 6, pp. 1237–1250, 2013.

[9] J. R. Gallego, M. Canales, and J. Ortın, “Distributed resourceallocation in cognitive radio networks with a game learningapproach to improve aggregate system capacity,” Ad Hoc Net-works, vol. 10, no. 6, pp. 1076–1089, 2012.

[10] Y. L. Teng, F. R. Yu, K. Han, Y. F. Wei, and Y. Zhang,“Reinforcement-learning-based double auction design fordynamic spectrum access in cognitive radio networks,”WirelessPersonal Communications, vol. 69, no. 2, pp. 771–791, 2013.

[11] S.-S. Wang, X.-G. Luo, and B.-N. Li, “Primary user emulationattacks analysis for cognitive radio networks communication,”TELKOMNIKA Indonesian Journal of Electrical Engineering,vol. 11, no. 7, pp. 3905–3914, 2013.

[12] A. Bantouna, V. Stavroulaki, Y. Kritikou, K. Tsagkaris, P.Demestichas, and K.Moessner, “An overview of learningmech-anisms for cognitive systems,” EURASIP Journal on WirelessCommunications and Networking, vol. 2012, article 22, 2012.

[13] L. Gavrilovska, V. Atanasovski, I. Macaluso, and L. A. DaSilva,“Learning and reasoning in cognitive radio networks,” IEEECommunications Surveys and Tutorials, vol. 15, no. 4, pp. 1761–1777, 2013.

[14] M. Bkassiny, Y. Li, and S. K. Jayaweera, “A survey on machine-learning techniques in cognitive radios,” IEEE CommunicationsSurveys and Tutorials, vol. 15, no. 3, pp. 1136–1159, 2013.

[15] M. Bkassiny, S. K. Jayaweera, and Y. Li, “Multidimensionaldirichlet process-based non-parametric signal classification forautonomous self-learning cognitive radios,” IEEE TransactionsonWireless Communications, vol. 12, no. 11, pp. 5413–5423, 2013.

[16] H. Bizhani and A. Ghasemi, “Joint admission control andchannel selection based on multi response learning automata(MRLA) in cognitive radio networks,” Wireless Personal Com-munications, vol. 71, no. 1, pp. 629–649, 2013.

[17] K. Tsagkaris, A. Katidiotis, and P. Demestichas, “Neuralnetwork-based learning schemes for cognitive radio systems,”Computer Communications, vol. 31, no. 14, pp. 3394–3404, 2008.

[18] A. Galindo-Serrano and L. Giupponi, “Distributed Q-learningfor aggregated interference control in cognitive radio networks,”IEEE Transactions on Vehicular Technology, vol. 59, no. 4, pp.1823–1834, 2010.

[19] H. S. Li, “Multiagent Q-learning for aloha-like spectrum accessin cognitive radio systems,”Eurasip Journal onWireless Commu-nications and Networking, vol. 2010, Article ID 876216, 15 pages,2010.

[20] X. F. Chen, Z. Zhao, H. Zhang, and T. Chen, “Reinforcementlearning enhanced iterative power allocation in stochasticcognitive wireless mesh networks,” Wireless Personal Commu-nications, vol. 57, no. 1, pp. 89–104, 2011.

[21] W. Z. Zhang and X. C. Liu, “Centralized dynamic spectrumallocation in cognitive radio networks based on fuzzy logic andq-learning,” China Communications, vol. 8, no. 7, pp. 46–54,2011.

[22] J. Zhu, J. Wang, T. Luo, and S. Li, “Adaptive transmissionscheduling over fading channels for energy-efficient cognitive

radio networks by reinforcement learning,” TelecommunicationSystems, vol. 42, no. 1-2, pp. 123–138, 2009.

[23] J. A. Torkestani andM.R.Meybodi, “A learning automata-basedcognitive radio for clustered wireless ad-hoc networks,” Journalof Network and SystemsManagement, vol. 19, no. 2, pp. 278–297,2011.

[24] M. F. Yang and D. Grace, “Cognitive radio with reinforcementlearning applied to multicast downlink transmission withpower adjustment,” Wireless Personal Communications, vol. 57,no. 1, pp. 73–87, 2011.

[25] P. Zhou, Y. S. Chang, and J. A. Copeland, “Reinforcementlearning for repeated power control game in cognitive radionetworks,” IEEE Journal on Selected Areas in Communications,vol. 30, no. 1, pp. 54–69, 2012.

[26] Y. Sun and J.-S. Qian, “Cognitive radio channel selectionstrategy based on experience-weighted attraction learning,”TELKOMNIKA Indonesian Journal of Electrical Engineering,vol. 12, no. 1, pp. 149–156, 2014.

[27] Y. Sun and J. S. Qian, “EWA selection strategy with channelhandoff scheme in cognitive radio,” Sensors & Transducers, vol.173, no. 6, pp. 68–74, 2014.

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation http://www.hindawi.com

Journal ofEngineeringVolume 2014

Submit your manuscripts athttp://www.hindawi.com

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014

SensorsJournal of


Modelling & Simulation in EngineeringHindawi Publishing Corporation http://www.hindawi.com Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks


research article fast channel selection strategy in...

Documents