Download - Lecture 8: Reliable, Low-Power Routing
June 2008 WEI Short Course - L8 Routing 1
Wireless Embedded Inter-Networking
Foundations Ubiquitous Sensor Networks
Self-Organized Network - Routing and Forwarding
David E. CullerUniversity of California, Berkeley
June 2008 WEI Short Course - L8 Routing 2
Embedded IP Architecture - Network
Ne
two
rk
ForwarderUnicast
Tra
nL
ink
Router
Forwarding Table
Default
Routing Table
Media Management ControlLink StatsRemote Media Local Media
Data
Queue
UDP
Ap
p
Autoconf
DHCPv6
TCP
Neighbor Table
Addr Period Phase Pending RSSI Success
StatelessAutoconf
Sample Period
Sample Phase
ICMPv6
Ack
Multicast
Buffer
SendManager
Ph
y
SendManager
RoutingAlgorithm
6LoW
PA
N A
daptation
June 2008 WEI Short Course - L8 Routing 3
The Basic Communication Primitive
• Transmit a packet
• Received by a set of nodes– Dynamically determined
– Depends on physical environment at the time
– and what other communication is on-going
– And further constraints by the link layer
• Each selects whether to retransmit– Potentially after modification
• And if so, when
June 2008 WEI Short Course - L8 Routing 4
Wireless Multihop Communication
• Upon each transmission, one of the recipients retransmits
– determined by source, by receiver, by …
– on the ‘edge of the cell’
June 2008 WEI Short Course - L8 Routing 5
Why Multihop Communication?
• Power!– to transmit D grows as D3 or worse
– to route distance D grows linearly
• Bandwidth (spatial multiplexing)– With n nodes in a single cell, each
gets at most 1/n bandwidth
– Many small cells => many simultaneous transmissions.
• Reliability (spatial diversity)– Individual links experience
interference, obstacles, and multipath effects
– Even short-range “wireless wires” require human nurturing
» IRDA, Bluetooth, WiFi, Cell
– Provides spatial diversity and receiver diversity
» rather than antenna diversity
– Protocol level reliability
0
20
40
60
80
100
120
140
0 1 2 3 4 5
June 2008 WEI Short Course - L8 Routing 6
WSN Communication Requirements
• Local neighbor communication (1 to few)
• Dissemination (1 to many)
• Data Collection and Aggregation (many to 1)
• Point-to-point Transfers (1 to 1)
• Reliably over lossy links
• At low energy (E = P*T)– Idle listening, management, monitoring
• Adapting to changing conditions
• With very light memory footprint
June 2008 WEI Short Course - L8 Routing 7
Embedded Network Organization
Border Router
Sensor Node
IP Device
IP Network(powered)
IP Sensornet
June 2008 WEI Short Course - L8 Routing 8
Network Architecture – What forms the IP Link?
• IP Protocols assume certain link properties
• Many assume a full-broadcast domain– “Everyone can communicate with each other”
– Reflexive and transitive reachability
– Not ad-hoc, wireless networks
• Examples– IPv6 Neighbor Discovery
– IPv6 Address Autoconfiguration
– ICMPv6 Redirect
– DHCPv6
June 2008 WEI Short Course - L8 Routing 9
Network Architecture – PAN = IP Link
• PAN IPv6 Link-Local Scope
• Emulate reflexive and transitive reachability
• Conceivable to run existing IP-based protocols unmodified
• No IP-level visibility into wireless topology
• Must define subnetwork functionality
Personal Area Network (PAN) Single IP Link
June 2008 WEI Short Course - L8 Routing 10
Network Architecture – Local Link = IP Link
• Radio Range IPv6 Link-Local Scope
• IP-level visibility into link topology
• Routing metrics across other link technologies
• Utilize functionality defined by IP
• Non-reflexive and non-transitive reachability
Personal Area Network (PAN) Multiple IP Links
June 2008 WEI Short Course - L8 Routing 11
Multi-Hop Communication
• Short-range radios & Obstructions => Multi-hop Communication is often required
– i.e. Routing and Forwarding
– That is what IP does!
• “Mesh-under”: multi-hop communication at the link layer– Still needs routing to other links or other PANs
• “Route-over”: IP routing within the PAN
• 6LoWPAN supports both
PAN
June 2008 WEI Short Course - L8 Routing 12
IP-Based Multi-Hop• IP has always done “multi-hop”
– Routers connect sub-networks to one another
– The sub-networks may be the same or different physical links
• Routers utilize routing tables to determine which node represents the “next hop” toward the destination
• Routing protocols establish and maintain proper routing tables– Routers exchange messages with neighboring routers
– Different routing protocols are used in different situations
– RIP, OSPF, IGP, BGP, AODV, OLSR, …
• IP routing over 15.4 links does not require additional header information at 6LoWPAN layer
• Vast body of tools to support IP routing– Diagnosis, visibility, tracing, management
– These need to be reinvented for meshing
• IP is widely used in isolated networks too– Broad suite of security and management tools
June 2008 WEI Short Course - L8 Routing 13
Terminology
• There is no single-hop routing!• IP routing allows hosts in one network (IP Link)
to communicate with hosts in another• Topology Formation: determining the
connectivity graph, i.e., the network.• Routing: Protocols and Process for establishing
what paths are used in communicating over that graph and setting up tables.
• Forwarding: process of receiving messages on one interface, looking up the next hop, and transmitting them on that interface
• Meshing: some combination of formation, routing, and forwarding that occurs at the link layer (L2) transparently to the network layer
June 2008 WEI Short Course - L8 Routing 14
Meshing vs Routing• Conventional IP link is a full broadcast domain
– Routing connects links (i.e, networks)
• Many IP links have evolved from a broadcast domain to a link layer “mesh” with emulated broadcast
– ethernet => switched ethernet
– 802.11 => 802.11s
• Utilize high bandwidth on powered links to maintain the illusion of a broadcast domain
• 802.15.4 networks are limited in bandwidth and power so the emulation is quite visible.
• Routing at two different layers may be in conflict
• On-going IETF work in ROLL working group– Routing Over Low-Power and Lossy networks
June 2008 WEI Short Course - L8 Routing 15
Classical View of Routing
• Connectivity between nodes defines the network graph.
– Topology formation
• A Routing algorithm determines the sub-graph that is used for communication between nodes.
– Route formation, path selection
• Packets are forwarded from source to destination over the routing subgraph
– At each node in the path, determine the recipient of the next hop
• The selection at each hop is made based on the information at hand
– Sender address, current address, destination address, information in the packet, information on the node.
– Table-driven, source based, algorithmic, …– Who knows the route? Do you determine it as you go?
June 2008 WEI Short Course - L8 Routing 16
Basic Approaches
• Link state– Nodes shout (send) and listen (receive) to determine neighbor
connectivity.– Each floods this information throughout (Link State
Advertisement) so every node has a map (Link state data base) of the network.
– Any node can determine the path or the next hop.– management protocol deals with changes in connectivity– Classic Example: OSPF
• Distance vector– Nodes maintain routing information about “distance” and
“direction” to destinations– Choose next hop by comparing the cost of routing through
neighbors» Cost(dest D, neighbor b) = linkCost(b) + pathCost(b,D)
– Management propagates routing information» Sequence numbers, etc.
– Classic Example: RIP
June 2008 WEI Short Course - L8 Routing 17
What’s different in WSN?
• There is no a priori network graph– It is discovered by sending packets and seeing who receives
them.
– The link relationship is not binary.
» pairs of nodes communicate with some probability that is determined by many of factors.
– It is not static.
• The embedding of the “network” in space is important.
– Need to get information to travel between particular physical places.
– But the “communication range” is not a simple function of distance.
• addressing & naming – Flat EUID? Hierarchical IP? Topologically meaningful?
Spatially meaningful?
June 2008 WEI Short Course - L8 Routing 18
Topology Formation
• Much of the “paper protocols” define connectivity graph with unit disk model
– Link(A,B) iff dist(A,B) ≤ R
• OK for rough calculations, but not for protocol design
– Nearby nodes may not be able to communicate.
– Far away nodes may be able to communicate.
– Nodes that communicated in the past may not be able to communicate in the future.
– Nodes may have intermittent communication depending on external factors.
• Connectivity is determined by communication– If B receives packet reasonably reliably from A, then A B
– If A receives packet reasonably reliably from B, then A B
– And if both are true, A B
R
June 2008 WEI Short Course - L8 Routing 19
Wireless Routing Protocols
• Many wireless protocols in the IP context have been development in the IETF MANET (Mobile Ad Hoc Networking) working group in the context of 802.11 links carrying traditional TCP/IP point-to-point traffic.
– AODV – ad hoc on-demand distance vector
– OLSR – Optimized link state Routing
– DSDV - Destination Sequenced Distance Vector
– DSR – Dynamic Source Routing
– TDRPF - Topology Dissemination Based on Reverse-Path Forwarding
• Assume a fairly “classic” view of connectivity– Naïve radio
June 2008 WEI Short Course - L8 Routing 20
Neighbor Communication
0
11
June 2008 WEI Short Course - L8 Routing 21
Fundamental Primitive
• Transmit to whatever receivers happen to hear it
• This is the fundamental primitive that is buried underneath complex protocols like Bluetooth, but not made available.
• It is what make it possible to build higher level protocols on the link, especially IP.
• To determine connectivity,– Local broadcast– Respond– on-going protocol to estimate quality of the link
» Packet reliability (sequence numbers, acks)• Note 802.15.4 acks only from a specific destination
» RSSI, LQI, …
June 2008 WEI Short Course - L8 Routing 22
Neighbor Communication in IPv6
• Route-over: this is simply transmission to the Link Local All Nodes multicast address – FF02::1
• Mesh-under: the entire PAN is a single IP link (i.e., mesh under), this concept is lost.
• At the link layer, the broadcast address is well-defined
• But, in a low-power link, this is no longer the most basic communication primitive.
– Only receive if listening
– Link local broadcast requires either more wake up or more coordination than link local unicast
• Unicast/Multicast distinction remains
June 2008 WEI Short Course - L8 Routing 23
Simple Address-Free Flooding Protocol
• Root broadcasts a “new” message to local neighborhood
• Each node performs a simple ruleif (“new” incoming msg) then
take local action
retransmit modified msg
• No underlying routing structure required– The connectivity over physical space determines it.
June 2008 WEI Short Course - L8 Routing 24
Route-Free “Flood”
0
11
June 2008 WEI Short Course - L8 Routing 25
“Flooding”
• Route free dissemination is extremely useful in its own right
– Disseminate information
» Router advertisements, solicitaitons, …
– Network-wide discovery
– Join, …
• It is also the network primitive that most “ad hoc” protocols used to determine a route
– Flood from source till destination is reached.
– Each node records the source of the flood packet
» This is the parent in the “routing tree”
– Reverse the links to form the path back
June 2008 WEI Short Course - L8 Routing 26
Data Collection in concept
0
112
2
2
22
June 2008 WEI Short Course - L8 Routing 27
The Problems
• Flood causes tremendous contention– Many good links missed because of collistions
– Huge amount of noise
• Many links are not symmetric
June 2008 WEI Short Course - L8 Routing 28
Flood Dynamics• Experimental Setup
– 13x13 grid of nodes– separation 2ft– flat open surface– Identical length antennas,
pointing vertically upwards.– Fresh batteries on all nodes– Identical orientation of all
nodes– The region was clean of
external noise sources.• Range of signal strength
settings• Log many runs
Ganesan, Krishnamachari, Woo, Culler, Estrin and Wicker, Complex Behavior at Scale: An Experimental Study of Low-Power Wireless Sensor Networks , UCLA Computer
Science Technical Report UCLA/CSD-TR 02-0013
June 2008 WEI Short Course - L8 Routing 29
Final Tree
June 2008 WEI Short Course - L8 Routing 30
Factors
• Long asymmetric links are common– Many children
• Nodes out of range may have overlapping cells– hidden terminal effect
• Collisions => these nodes hear neither ‘parent’– become stragglers
• As the tree propagates – folds back on itself
– rebounds from the edge
– picking up these stragglers.
• Redundancy – Geometric overlap => <41% additional area
Ni, S.Y., Tseng, Y.C., Chen, Y.S., Sheu, J.P.: The broadcast storm problem in a mobile ad hoc network. MobiCom'99
June 2008 WEI Short Course - L8 Routing 31
Topology Reinforcement
0
112
2
2
22
June 2008 WEI Short Course - L8 Routing 32
Network Layer
• Router– Populate the routing table with candidates– By listening to Router Advertisement (RA) messages
» In LoWPAN Link quality is advertised too– Manages entries in the forwarding table
• Forwarder– Recv datagram on interface, lookup next hop in FT, request
transmission to that– Default route
• In IPv6, topology reinforcement is by RA– Plus whatever passive observation and link level piggy backing
Ne
two
rk
ForwarderUnicast
Router
Forwarding Table
Default
Routing Table
Queue
Autoconf
StatelessAutoconf
ICMPv6
Multicast
Buffer
SendManager
SendManager
RoutingAlgorithm
June 2008 WEI Short Course - L8 Routing 33
Routing – Baseline
• Routing state very limited. Routing protocol messages very infrequent.
• Reduce problem by:– Egress routing (default routes)
– Ingress routing (source routing)
– Any-to-any through border router or scoped discovery
Border RouterIP Device
IP Network(powered)
June 2008 WEI Short Course - L8 Routing 34
Routing – Egress Routes
• Distance-vector protocol rooted at Border Router
• Piggyback on ICMPv6 Router Advertisements– Hop-count or other routing metric
– Version number to clear routing state
• List of next-hop candidates– Top candidate used to configure default routes
– Select a random next-hop if default route fails
fd00::4
fd00::5
fd00::3
fd00::2
fd00::1
June 2008 WEI Short Course - L8 Routing 35
Routing – Ingress Routes
• Record route sent to subnet router anycast
• Forward using IPv6 Routing extension header– Use 16-bit short compressed form to minimize overhead
fd00::4
fd00::4
fd00::5
fd00::3
fd00::2
fd00::1
1 | 3 | fd00::4
June 2008 WEI Short Course - L8 Routing 36
Routing – Any-to-any Routes
• Route through border router by default
• Utilize scoped discovery to for nearby neighbors– Worst-case stretch (2D) with neighboring nodes
– New stretch 2D/(s+1), s = discovery scope
fd00::4
fd00::5
fd00::3
fd00::2
fd00::1
1 | 3 | fd00::4
fd00::4
Worst-case Stretch
June 2008 WEI Short Course - L8 Routing 37
Achieving Reliability in the Face of Uncertainty
June 2008 WEI Short Course - L8 Routing 38
Abstracting Uncertainty
Signal
Noise
Distance
June 2008 WEI Short Course - L8 Routing 39
Which node do you route through?
June 2008 WEI Short Course - L8 Routing 40
What does this mean?
• Always routing through nodes “at the hairy edge”
– Wherever you set the threshold, the most useful node will be close to it
• Topology determination is a continuous process of discovery and validation
– and it must be done politely
• Connectivity is determined by communication
– If B receives packet reasonably reliably from A, then A B
– If A receives packet reasonably reliably from B, then A B
– And if both are true, A B
June 2008 WEI Short Course - L8 Routing 41
Reliability Techniques
• The IP end-to-end architecture assumes IP links are 99% reliable
– Best effort says don’t try to push it to %99.999 on every link, put E2E reliability protocols over it when you need it, i.e., TCP
– Not that every link should send and forget.
• Good Low-power links are often <%90 reliable– 0.9h drops awfully quickly
• Need link-level acks– 15.4 acks not even good enough
– Filters out asymmetric links
• Need hop-by-hop retransmission
• Piggy-back net acks on link level
• Retransmission and rerouting
• Provides the cross-layer visibility
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
1 2 3 4 5 6 7 8 9 10
hops
P
June 2008 WEI Short Course - L8 Routing 42
RoutingSelecting Bi-Directional Links and Forming Routes
Default RoutesDefault RoutesDefault RoutesDefault Routes
Discovering LinksDiscovering LinksDiscovering LinksDiscovering LinksICMPv6 HdrICMPv6 HdrICMPv6 HdrICMPv6 Hdr Router AdvRouter AdvRouter AdvRouter Adv MHop InfoMHop InfoMHop InfoMHop Info
Inferring a Connectivity GraphInferring a Connectivity GraphInferring a Connectivity GraphInferring a Connectivity GraphRouting Routing TableTable
Routing Routing TableTablePrefix Next
Low Routing CostLow Routing Cost
High Routing CostHigh Routing Cost
High ConfidenceHigh Confidence
Low ConfidenceLow Confidence
Selecting a Default RouteSelecting a Default RouteSelecting a Default RouteSelecting a Default RouteRouting Routing TableTable
Routing Routing TableTablePrefix Next
Forwarding Forwarding TableTable
Forwarding Forwarding TableTablePrefix Next
Default
•Top candidateTop candidate•Dynamic re-Dynamic re-routingrouting•Increasing Increasing confidenceconfidence
June 2008 WEI Short Course - L8 Routing 43
High Quality Selection
• Sort by:– Link quality estimate confidence
– Path cost (including link quality)
• Link success rate requires state– RSSI for insertion (no prior state)
• Link success computed by link for each transmission– Dynamically alter default route to:
» Refresh link quality estimates
» Discover lower path costs
» Re-routing naturally does this
June 2008 WEI Short Course - L8 Routing 44
Building Neighborhoods & Routes
• Node transmits to some unknown set
• Candidate nbrs are sources of incoming packets
• Estimate of inbound link reliability
• Occasionally announce inbound link states– Provides reverse link estimate to outbound neighbors
– Basis for cost-based routing
• Cost-based Parent Selection– depth(me) = MIN nbr(me) depth(i)
– loss(me) = MIN nbr(me) loss(i)*est(me,i)
– trans(me) = MIN nbr(me) trans(i)+etrans(me,i)
• What about nbrs that don’t fit in the table?– FIFO, LRU, Frequency
Taming the Challenges of Reliable Multihop Routing in Sensor Networks, Alec Woo and David Culler, SenSys. 2003.
June 2008 WEI Short Course - L8 Routing 45
Pragmatics
• Maintain at least two (preferably three) candidate parents
– Link level retransmission and rerouting
– Use acks to determine quality of the link
– Throw in a new candidate from time to time.
• Do not record list of children– Insufficient memory to build the table.
• Route update message on a trickle schedule
June 2008 WEI Short Course - L8 Routing 46
Adding up the pieces - footprint
ROM RAM
CC2420 Driver 3149 272
802.15.4 Encryption 1194 101
Media Access Control 330 9
Media Management Control 1348 20
6LoWPAN + IPv6 2550 0
Checksums 134 0
SLAAC 216 32
DHCPv6 Client 212 3
DHCPv6 Proxy 104 2
ICMPv6 522 0
Unicast Forwarder 1158 451
Multicast Forwarder 352 4
Message Buffers 0 2048
Router 2050 106
UDP 450 6
TCP 1674 50
24038 ROM
3598 RAM
(including runtime)
* Production implementation on TI msp430/cc2420
June 2008 WEI Short Course - L8 Routing 47
… and reliabilityApplication Power Model
Data Rate Sensitivity (Router)
Data Rate Sensitivity (Router)
Data Rate Sensitivity (Edge)
Data Rate Sensitivity (Edge)
Deployment Duty CycleDeployment Duty Cycle Deployment ReliabilityDeployment Reliability
June 2008 WEI Short Course - L8 Routing 48
Discussion
• Upcoming– Trickle protocol
– Extension of IPv6 to support this sort of routing