adsa: balanced trees/12 1 241-423 advanced data structures and algorithms objectives – –discuss...

ADSA: Balanced Trees/12 1

241-423 Advanced Data Structures and Algorithms

Objectives– discuss various kinds of balanced search trees:

AVL trees, 2-3-4 trees, Red-Black trees,

Semester 2, 2012-2013

12. Balanced Search Trees


Contents

1. What is a Balanced Binary Search Tree?

2. AVL Trees

3. 2-3-4 Trees

4. Red-Black Trees


1.What is a Balanced Binary Search Tree?

• A balanced search tree is one where all the A balanced search tree is one where all the branches from the root have almost the branches from the root have almost the same height.same height.

balanced

unbalancedcontinued


• As a tree becomes more unbalanced, search running time decreases from O(log n) to O(n)– because the tree shape turns into a list

• We want to keep the binary search tree balanced as nodes are added/removed, so searching/insertion remain fast.


1.1. Balanced BSTs: AVL Trees

• An AVL tree maintains height balance– for each node, the difference in height of its two

subtrees is in the range -1 to 1


1.2. 2-3-4 Trees

o A multiway tree where each node has at most 4 children, and a node can hold up to 3 values.

o A 2-3-4- tree can be perfectly balanced• no difference in height between branches• requires complex nodes and links


o A red-black tree is a binary version of a 2-3-4 tree• the nodes have a 'color' attribute: BLACK or RED

• drawn in Ford and Topp (and here) in white and gray!!

• the tree maintains a balance measure called the BLACK height

BLACK

RED

1.3. Red-Black Trees1.3. Red-Black Trees


1.4. B-Trees1.4. B-Trees

• A multiway tree where each node has at A multiway tree where each node has at most m children, and a node can hold up to most m children, and a node can hold up to m-1 valuesm-1 values– a more general version of a 2-3-4 treea more general version of a 2-3-4 tree

• B-Trees are most commonly used in B-Trees are most commonly used in databases and filesystemsdatabases and filesystems– most nodes are stored in secondary storage such most nodes are stored in secondary storage such

as hard drivesas hard drives


2. AVL Trees• For each AVL tree node, the difference between the heights of its left

and right subtrees is either -1, 0 or +1– this is called the balance factor of a node

• balanceFactor = height(left subtree) - height(right subtree)

– if balanceFactor > 1 or < -1 then the tree is too unbalanced, and needs 'rearranging' to make it more balanced

L - R


• Heaviness– if the balanceFactor is positive, then the node is

"heavy on the left"• the height of the left subtree is greater than the

height of the right subtree

– a negative balanceFactor, means the node is "heavy on the right"

continued


root isheavy on the right,but still balanced

root isheavy on the left,but still balanced

root isheavy on the right,but still balanced

L – R= 0-1

L – R= 2-1

L – R= 1-2


2.1. The AVLTree Class


Using AVLTreeString[] stateList = {"NV", "NY", "MA", "CA", "GA"}; AVLTree<String> avltreeA = new AVLTree<String>();for (int i = 0; i < stateList.length; i++) avltreeB.add(stateList[i]);

System.out.println("States: " + avltreeA);

int[] arr = {50, 95, 60, 90, 70, 80, 75, 78};AVLTree<Integer> avltreeB = new AVLTree<Integer>();for (int i = 0; i < arr.length; i++) avltreeB.add(arr[i]);

// display the treeSystem.out.println(avltreeB.displayTree(2));avltreeB.drawTree(2);


ExecutionStates: [CA, GA, MA, NV, NY]

70(-1) 60(1) 90(1) 50(0) 78(0) 95(0) 75(0) 80(0)

(1)

(-1) root isheavy on the right,but still balanced


The AVLTree Node

• An AVLNode contains the node's value, references to the node's two subtrees, and the node height.

height(node) = max ( height(node.left), height(node.right) ) + 1;

nodeValue

height

left right

AVTTreeNodeobject

continued


private static class AVLNode<T>{ public T nodeValue; // node data

public int height;

// child links public AVLNode<T> left, right; public AVLNode (T item) { nodeValue = item; height = 0; left = null; right = null; }}

nodeValue

height

left right

Use of public is bad;the coding style is due to Ford & Topp


• The addition of a node may cause the tree to go out of balance. – addNode() adds a node and may reorder nodes as

it returns from the adding back to the root• reordering is the new idea in AVL trees

– the reordering is done using single and double rotations

2.2. Adding a Node to the Tree


2.2.1. Too Heavy on the Left

1212

2525

3030

55 2020

1111

(2)

1212

2525

3030

55 2020

2222

(2)node Pnode P

or

left

left

left

right

Two cases

outside grandchild inside grandchild

left or right branch -- doesn't matter

branch doesn't matter

continued

L – R = 3-1


• Inserting a node in the left subtree of P (e.g. adding 11 or 22) may cause P to become "too heavy on the left" – balance factor == 2

• The new node can either be in the outside or inside grandchild subtree:– outside grandchild = left-left

– inside grandchild = left-right

continued


2.2.2. Too Heavy on the Right

1212

2525

3030

2727 4545

4040

(-2)

1212

2525

3030

2727 4545

2929

(-2)node Pnode P

or

right

left

right

right

Two cases

outside grandchild inside grandchild



continued

L – R = 1-3


• Inserting a node in the right subtree of P (e.g. adding 29 or 40) may cause P to become "too heavy on the right"– balance factor == -2

• The new node can either be in the outside or inside grandchild subtree:– outside grandchild = right-right– inside grandchild = right-left

continued


2.3. Single Rotations

• When a new item is added to the subtree for an outside grandchild, the imbalance is fixed with a single right or left rotation

• Two cases:– left outside grandchild (left-left) -->

single right rotation– right outside grandchild (right-right) -->

single left rotation


2.3.1. Single Right Rotation

• A single right rotation occurs when a new element is added to the subtree of the left outside grandchild (left-left)

continued


add

add

cut

cut

left outside grandchild (left-left)

continued


• A single right rotation rotates the left child (LC) to replace the parent– the parent becomes the new right child

• The right subtree of LC (RGC) is attached as a left child of P– ok since the nodes in RGC are greater than LC but

less than P


singleRotateRight()

// single right rotation on pprivate static <T> AVLNode<T> singleRotateRight( AVLNode<T> p){ AVLNode<T> lc = p.left;

p.left = lc.right; // 1 & 4 on slide 26 lc.right = p; // 2 & 3

p.height = max(height(p.left), height(p.right)) + 1; lc.height = max(height(lc.left), height(rc.right)) + 1;

return lc;}


private static <T> int height(AVLNode<T> t){ if (t == null) return -1; else return t.height;}


2.3.2. Single Left Rotation

• A single left rotation occurs when a new element is added to the subtree of the right outside grandchild (right-right).

• The rotation exchanges the parent (P) and right child (RC) nodes, and attaches the subtree LGC as the right subtree of P.

continued


add

add

cut

cut

right outside grandchild (right-right)


singleRotateLeft()

// single left rotation on pprivate static <T> AVLNode<T> singleRotateLeft( AVLNode<T> p){ AVLNode<T> rc = p.right;

p.right = rc.left; // 1 & 4 on slide 31 rc.left = p; // 2 & 3

p.height = max(height(p.left),height(p.right)) + 1; rc.height = max(height(rc.left), height(rc.right)) + 1;

return rc;}


2.4. Double Rotations

• When a new item is added to the subtree for an inside grandchild, the imbalance is fixed with a double right or left rotation– a double rotation is two single rotations

• Two cases:– left inside grandchild (left-right) -->

double right rotation– right inside grandchild (right-left) -->

double left rotation


Single left rotationabout LC

Single right rotationabout P

balanced

2.4.1. A Double Right Rotation

left insidegrandchild(left-right)

Watch RGCrise to the top


doubleRotateRight()

private static <T> AVLNode<T> doubleRotateRight( AVLNode<T> p)/* double right rotation on p is left rotation, then right rotation */{ p.left = singleRotateLeft(p.left); return singleRotateRight(p);}


Single right rotationabout RC

Single left rotationabout P

balanced

2.4.2. A Double Left Rotation

right insidegrandchild(right-left)

P P

RC

RGC

LGC

BA

LC LC RC

LGC

RGCB

A

P

LC

RC

LGC

RGC

BA

Watch LGCrise to the top


doubleRotateLeft()

private static <T> AVLNode<T> doubleRotateLeft( AVLNode<T> p)/* double left rotation on p is right rotation, then left rotation */{ p.right = singleRotateRight(p.right); return singleRotateLeft(p);}


2.5. addNode()

• addNode() recurses down to the insertion point and inserts the node.

• As it returns, it visits the nodes in reverse order, fixing any imbalances using rotations.

• It must handle four cases:– balance height == 2: left-left, left-right– balance height == -2: right-left, right-right


Basic addNode()

private Node<T> addNode(Node<T> t, T item)

{

if (t == null) // found insertion point

t = new Node<T>(item);

else if (((Comparable<T>)item).compareTo(t.nodeValue) < 0) {

t.left = addNode( t.left, item); // visit left subtree

else if (((Comparable<T>)item).compareTo(t.nodeValue) > 0 ) {else if (((Comparable<T>)item).compareTo(t.nodeValue) > 0 ) {

t.right = addNode(t.right, item); // visit rightt.right = addNode(t.right, item); // visit right

else else

throw new IllegalStateException(); // duplicate errorthrow new IllegalStateException(); // duplicate error

return t;return t;

} // end of addNode()} // end of addNode()

No AVL rotation code added yetwas P in earlier slides


private AVLNode<T> addNode(AVLNode<T> t, T item){ if(t == null) // found insertion point t = new AVLNode<T>(item);

else if (((Comparable<T>)item).compareTo(t.nodeValue) < 0) { // visit left subtree: add node then maybe rotate t.left = addNode( t.left, item); // add node, then...

if (height(t.left) - height(t.right) == 2 ) { //too heavy on left if (((Comparable<T>)item).compareTo(t.left.nodeValue) < 0) // problem on left-left t = singleRotateRight(t); else // problem on left-right t = doubleRotateRight(t); // left then right rotation } } :

continued

AVL rotation code added


else if (((Comparable<T>)item).compareTo(t.nodeValue) > 0 ) { // visit right subtree: add node then maybe rotate t.right = addNode(t.right, item ); // add node, then...

if (height(t.left)-height(t.right) == -2){ //too heavy on right if (((Comparable<T>)item).compareTo(t.right.nodeValue) > 0) // problem on right-right t = singleRotateLeft(t); else // problem on right-left t = doubleRotateLeft(t); // right then left rotation } } else // duplicate; throw IllegalStateException throw new IllegalStateException();

// calculate new height of t t.height = max(height(t.left), height(t.right)) + 1;

return t;} // end of addNode()


public boolean add(T item){ try { root = addNode(root, item); // start from root } catch (IllegalStateException e) { return false; } // item is a duplicate

// increment the tree size and modCount treeSize++; modCount++;

return true; // node was added ok}

add() public interface for inserting an item


2.6. Building an AVL Tree

left outsidegrandchild(left-left)

continued

gray node istoo heavy


right outsidegrandchild

(right-right)

continued

45


right insidegrandchild(right-left)

double rotate left(right then left rotation)

continued


left insidegrandchild(left-right)

double rotate right(left then right rotation)

continued


2.7. Efficiency of AVL Tree Insertion

• Detailed analysis shows:int(log2n) height < 1.4405 log2(n+2) - 1.3277

• So the worst case running time for insertion is O(log2n).

• The worst case for deletion is also O(log2n).

ADSA: Balanced Trees/12 48AVL Trees 48

2.8. Deletion in an AVL Tree

• Deletion can easily cause an imbalance– e.g delete 32

44

17

7832 50

8848

62

54

44

17

7850

8848

62

54

before deletion of 32 after deletion


3. 2-3-4 Trees

• In a 2-3-4 tree:– a 2‑node has 1 value and a max of 2 children– a 3-node has 2 values and a max of 3 children– a 4-node has 3 values and a max of 4 children

The numbers refer to themaximum number of

branches that can leavethe node

same as a binarytree node


3.1. Searching a 2-3-4 Tree

• To find an item:– start at the root and compare the item with all

the values in the node;– if there's no match, move down to the

appropriate subtree;– repeat until you find a match or reach an empty

subtree


Search ExampleTry finding 9 and 30


3.2. Inserting into a 2-3-4 Tree3.2. Inserting into a 2-3-4 Tree• Search to the bottom for an insertion nodeSearch to the bottom for an insertion node

– 2-node at bottom: convert to 3-node2-node at bottom: convert to 3-node– 3-node at bottom: convert to 4-node3-node at bottom: convert to 4-node– 4-node at bottom: ??4-node at bottom: ??


Splitting 4-nodesSplitting 4-nodes

• Transform tree on the way down:Transform tree on the way down:– ensures last node is not a 4-nodeensures last node is not a 4-node– local transformation to split a 4-nodelocal transformation to split a 4-node

Insertion at the bottom is now easy since it's not a 4-nodeInsertion at the bottom is now easy since it's not a 4-node


ExampleExample• To split a 4-node. move middle value up.To split a 4-node. move middle value up.


3.3. Building

continued

insert 4

This 4-node will be split duringthe next insertion.



insert 10

Insertions happen at the bottom.


ADSA: Balanced Trees/12 57continued

The insertion point is at level 1, so the new 4-node at level 0 is not split during this insertion.

insert 55


2 8 1 0 1 5 3 5 5 5

2 54

1 2

2 8 1 0 1 1 1 5 3 5 5 5

2 54

1 2

S p lit 4-n o d e (4, 12, 25) In s e rt 11

insert 11



Another ExampleAnother Exampleinsert

insertThe search missed

the 4-nodes onthe left, so

not changed.


3.4. Efficiency of 2-3-4 Trees

• Searching for an item in a 2-3-4 tree with n elements:– the max number of nodes visited during the search

is int(log2n) + 1

• Inserting an element into a 2‑3‑4 tree:– requires splitting no more than int(log2n) + 1

4-nodes• normally requires far fewer splits

fast!


3.5. Drawbacks of 2-3-4 Trees

• Since any node may become a 4-node, then all nodes must have space for 3 values and 4 links– but most nodes are not 4-nodes– lots of wasted memory, unless impl. is fancier

• Complex nodes and links– slower to process than binary search trees


4. Red-Black Trees

• A red-black tree is a binary search treewhere each node has a 'color' – BLACK or RED

• A red-black tree is a binary version of a 2-3-4 tree, using different color combinations to represent 3-nodes and 4-nodes.– a 2-node is already a binary node


BLACK

RED

BLACK and RED are drawn in Ford and Topp

(and here) in white and gray!!


4.1. From 2-3-4 Tree Nodes to Red-Black Nodes

• A 2-node is already a binary node so doesn't need to change its shape.

• The color of a 2-node is always BLACK (drawn as white in these slides).

continued

2-node Conversion


• A 4-node has it's middle value become a BLACK (white) parent and the other values become RED (gray) children.

BLACK

RED

continued

4-node Conversion


• Represent a 3-node as:– a BLACK parent and a smaller RED left child or– a BLACK parent and a larger RED right child

A B

S T U

3-n o d e (A , B)in a 2-3-4 T re e

A

BS

(a ) Re d -b la c k t re e re p re s e n ta t io nA is a b la c k p a re n t ; B is a re d rig h t c h ild

T U

B

A

S T

U

(b ) Re d -b la c k t re e re p re s e n ta t io nB is a b la c k p a re n t ; A is a re d le ft c h ild

OR

3-node Conversion


4.2. Changing a 2-3-4 Tree into a Red-Black Tree

changethis node

continued


changethis node

continued


changethese nodes


4.3. Three Properties of a Red-Black Tree

• 1. The root must always be BLACK (white in our pictures)

• 2. A RED parent never has a RED child– in other words: there are never two successive

RED nodes in a path

continued

that must always be true for the tree to be red-black


• 3. Every path from the root to an empty subtree contains the same number of BLACK nodes– called the black height

• We can use black height to measure the balance of a red-black tree.


Check the Example Properties


4.4. Inserting a Node

• 1. Search down the tree to find the insertion point, splitting any 4-nodes (a BLACK parent with two RED children) by coloring the children BLACK and the parent RED– called a color flip

• Splitting a 4-node may involve additional rotations and color changes– there are 4 cases to consider (section 4.4.1)

continued

Three thingsto do.


• 2. Once the insertion point is found, add the new item as a RED leaf node (section 4.4.2)– this may create two successive RED nodes

• again use rotation and recoloring to reorder/rebalance the tree

• 3. Keep the root as a BLACK node.


4.4.1. Four Cases for Splitting a 4-Node

1 2 3 4

LL/black parent LR/black parent LL/red parent LR/red parent(also mirror

case, RR)

GL

L

GL

GL

GL

R L R

(also mirror case, RL)

(also mirror case, RR)

(also mirror case, RL)


• If the parent is BLACK, only a color flip is If the parent is BLACK, only a color flip is needed.needed.

Case 1 (LL/black P): An Example

1

LG

LG

LG

LG

L LL


Case 2 (LR/black P): Insert 55

2

Only a color flip is required for same reason as case 1

LG

LG

LG

LG

R RR


The color-flip creates two successive red nodes-- this breaks property 2, so must be fixed

Case 3 (LL / red parent)3

L

L

L

L


• To fix the red color conflict, carry out a single left or To fix the red color conflict, carry out a single left or right rotation of node right rotation of node PP::• LL LL right rotation of P right rotation of P

• RR (mirror case) RR (mirror case) left rotation of P left rotation of P

• Also change the colors of nodes Also change the colors of nodes PP and G. and G.

continued


and P and G color changes

3 3

LL of G → right rot of P RR of G → left rot of P

L

L

R

R


Case 3 (LL / red P) as a 2-3-4 Tree

3

X P G

A B C D

L

L


Case 4 (LR / red parent)Case 4 (LR / red parent)

The color-flip creates two successive red nodes-- this breaks property 2, so must be fixed

4

L

R

L

R


• To fix the red color conflict, carry out a To fix the red color conflict, carry out a doubledouble left or right rotation of node left or right rotation of node XX::

• LR LR double right rotation of X double right rotation of X– left then right rotationsleft then right rotations

• RL (mirror case) RL (mirror case) double left rot of X double left rot of X – right then left rotationsright then left rotations

• Also change the colors of nodes Also change the colors of nodes XX and G. and G.


LR Example

4

X

P

X and Grecoloured

L

R

left rotationof X

right rotationof X


4

Same Example, with 2-3-4 Views

L

R


4.4.2. Inserting a New Item• Always add a new item to a tree as a RED leaf node

– this may create two successive RED nodes, which breaks property 2

• Fix using single / double rotation and color flip as used for splitting 4-nodes:– LL/RR single right/left rotation of parent (P)– LR/RL double right/left rot of new node (X)


Example: Insert 14Example: Insert 14

single leftrotation of 12 and color flip

R

R


10

12

Insert 10 instead of 14Insert 10 instead of 14

R

L

single leftrotation and

color flip

single rightrotation of 10

RL = double left rotation of node 10(right then left)


4.5. Building a Red-Black Tree

continued

single right rotationof 20 and color flip

L

L

ADSA: Balanced Trees/12 90continued

1 2 3these numbers are from section 4.4.these numbers are from section 4.4.

Insert 35

Insert 25 now


Insert 30

L

R

LR = double right rotation of node 30(left then right)


4.6. Search Running Time

• The worst‑case running time to search a red-black tree or insert an item is O(log2n)

– the maximum length of a path in a red-black tree with black height B is 2*B-1

but this cannothappen if theinsertion rulesare followed


4.7. Deleting a Node• Deletion is more difficult than insertion!

– must usually replace the deleted node– but no further action is necessary when the

replacement node is RED

78

1 0 0807050

9060

75 78

1 0 0807050

9060

D e le te 75Re p la c e me n t n o d e 78 is RED

continued

replacement(next biggest)


• But deletion requires recoloring and rotations when the replacement node is BLACK.

7 8

1 0 08 07 05 0

9 06 0

7 5 7 5

1 0 07 87 05 0

8 06 0

D elete 9 0R ep lacem en t n o d e i s B L A C K

7 8

8 07 05 0

1 0 06 0

7 5

R ep lace 9 0w i th 1 0 0

R ig h t ro tat io n w i thp iv o t 8 0 an d

reco lo rin g

replacement


4.8. The RBTree Class

• RBTree implements the Collection interface and uses RBNode objects to create a red-black tree.

nodeValue

color

left right

RBNodeobject

parent


class RBTree<T> implements Collection<T> ds.util

Constructor

RBTree() Creates an empty red-black tree.

Methods

String displayTree(int maxCharacters) Returns a string that gives a hierarchical view of the tree. An asterisk (*) marks red nodes.

void drawTree(int maxCharacters) Creates a single frame that gives a graphical display of the tree. Nodes are colored.

String drawTrees(int maxCharacters) Creates of the action of the function and any return value.

String toString() Returns a string that describes the elements in a comma-separated

list enclosed in brackets.


• Ford and Topp's tutorial, "RBTree Class.pdf", provides more explanation of the RBTree class– local copy at

• http://fivedots.coe.psu.ac.th/Software.coe/ ADSA/Ford%20and%20Topp/

• Includes:– a discussion of the private data– explanation of the algorithms for splitting a 4‑node and

performing a top‑down insertion


Using the RBTree Class

import ds.util.RBTree;

public class UseRBTree{ public static void main (String[] args) { // 10 values for the red-black tree int[] intArr = {10, 25, 40, 15, 50, 45, 30, 65, 70, 55}; RBTree<Integer> rbtree = new RBTree<Integer>();

// load the tree with values for(int i = 0; i < intArr.length; i++) { rbtree.add(intArr[i]); rbtree.drawTrees(4); // display on-going tree } // in a JFrame

:


// display final tree to stdout System.out.println(rbtree.displayTree(2));

/* // remove red-node 25 rbtree.remove(25); rbtree.drawTrees(4); // tree shown in JFrame

// remove black-node root 45 rbtree.remove(45); rbtree.drawTree(3); // tree shown in JFrame*/ } // end of main()

} // end of UseRBTree class


ExecutionExecution

Tree changes are shown graphically by drawTrees() and drawTree()


JFrame Output: Add 10 ValuesJFrame Output: Add 10 Values

25 40

15 50

split 25; color change 25

single left rot of 25

continued

BLACK RED

a

b

started red, thenflipped to black


45 30

65

double right rotation of 45

continued

c d

split 45


70

55

single left rotation of 65

split 65; left rotation of 45

e

f

adsa: balanced trees/12 1 241-423 advanced data structures and algorithms objectives – –discuss...

Documents