21scs147l17replacementpolicy4-7[1]

7/27/2019 21SCS147L17ReplacementPolicy4-7[1]

1/60

Cache memory Replacement

Policy, Virtual Memory

Prof. Sin-Min Lee

Department of Computer Science


2/60

Associative Cache

Access orderA0 B0 C2 A0 D1 B0 E4 F5 A0

C2 D1 V0 G3 C2 H7 I6 A0 B0

Tc = 10 ns

Tp = 60 ns

FIFO

h = 0.389

TM = 40.56 ns


3/60

FIFO

MEMORY

FIFO is similar to LRU except that because FIFO method doesn't move

the most recently used block up in the queue, it ends up discarding ablock that was used recently instead of the least recently used. In thisexample, when the memory is full A is discarded though it was recentlyused.

Hit ratio: 7/18

A is removed from memory, although

it was not the least recently used.

A B C A D B E F A C D B G C H I A B

0 A B C C D D E F F F F F G G H I A B

1 A B B C C D E E E E E F F G H I A

2 A A B B C D D D D D E E F G H I

3 A A B C C C C C D D E F G H

4 A B B B B B C C D E F G

5 A A A A A B B C D E F6 A A B C D E

7 A B C D


4/60

LRU

A B C A D B E F A C D B G C H I A B

0 A B C A D B E F A C D B G C H I A B

1 A B C A D B E F A C D B G C H I A

2 A B C A D B E F A C D B G C H I

3 B C A D B E F A C D B G C H

4 C A D B E F A A D B G C

5 C C D B E F F A D B G

6 E E F A D D

7 E F F F

MEMORY

MEMORY FULL

E is LRU and isremoved from memory

LRU (Least Recently Used) is a method that keeps track of how oftena data block is used.

Hit ratio: 9/18

The bottom block is the LeastRecently Used, which gets

replaced (discarded) in favor of thenew one, when the memory is full.


5/60

Direct-Mapped Cache

Access orderA0 B0 C2 A0 D1 B0 E4 F5 A0

C2 D1 V0 G3 C2 H7 I6 A0 B0

Tc = 10 ns

Tp = 60 ns

h = 0.167

TM = 50.67 ns


6/60

2-Way Set Associative Cache

Access orderA0 B0 C2 A0 D1 B0 E4 F5 A0C2 D1 V0 G3 C2 H7 I6 A0 B0

Tc = 10 ns

Tp = 60 ns

LRU

h = 0.31389

TM = 40.56 ns


7/60

Associative Cache

(FIFO Replacement Policy)

Data A B C A D B E F A C D B G C H I A B

C

A

C

HE

A A A A A A A A A A A A A A A I I I

B B B B B B B B B B B B B B B A A

C C C C C C C C C C C C C C C B

D D D D D D D D D D D D D D

E E E E E E E E E E E E

F F F F F F F F F F F

G G G G G G

H H H H

Hit? * * * * * * *

Hit ratio = 7/18

A0 B0 C2A0 D1 B0 E4 F5A0 C2 D1 B0 G3 C2 H7 I6A0 B0


8/60

Two-way set associative cache

(LRU Replacement Policy)

Hit ratio = 7/18



C

A

C

H

E

0 A-0 A-1 A-1 A-0 A-0 A-1 E-0 E-0 E-1 E-1 E-1 B-0 B-0 B-0 B-0 B-0 B-1 B-0

0 B-0 B-0 B-1 B-1 B-0 B-1 B-1 A-0 A-0 A-0 A-1 A-1 A-1 A-1 A-1 A-0 A-1

1 D-0 D-0 D-0 D-1 D-1 D-1 D-0 D-0 D-0 D-0 D-0 D-0 D-0 D-0

1 F-0 F-0 F-0 F-1 F-1 F-1 F-1 F-1 F-1 F-1 F-1

2 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-1 C-1 C-12 I-0 I-0 I-0

3 G-0 G-0 G-1 G-1 G-1 G-1

3 H-0 H-0 H-0 H-0

Hit? * * * * * * *


9/60

Associative Cache with 2 byte line size (FIFO

Replacement Policy)

Hit ratio = 11/18


A and J; B and D; C and G; E and F; and I and H


C

A

C

H

E

A A A A A A A A A A A A A A I I I I

J J J J J J J J J J J J J J H H H H

B B B B B B B B B B B B B B B A A

D D D D D D D D D D D D D D D J J

C C C C C C C C C C C C C C C B

G G G G G G G G G G G G G G G D

E E E E E E E E E E E E

F F F F F F F F F F F F

Hit? * * * * * * * * * * *


10/60

Direct-mapped Cache

with line size of 2 bytes

Hit ratio 7/18


C

A

C

H

E

0 A B B A B B B B A A B B B B B B A B

1 J D D J D D D D J J D D D D D D J D

2 C C C C C C C C C C C C C C C C

3 G G G G G G G G G G G G G G G G

4 E E E E E E E E E E E E

5 F F F F F F F F F F F F

6 I I I I

7 H H H H

Hit? * * * * * * *




11/60

Two-way set Associative Cache

with line size of 2 bytes

Hit ratio = 12/18


C

A

C

H

E

0 A-0 A-1 A-1 A-0 A-1 A-1 E-0 E-0 E-1 B-0 B-0 B-0 B-0 B-0 B-0 B-0 B-1 B-0

1 J-0 J-1 J-1 J-0 J-1 J-1 F-0 F-0 F-1 D-0 D-0 D-0 D-0 D-0 D-0 D-0 D-1 D-0

0 B-0 B-0 B-1 B-0 B-0 B-1 B-1 A-0 A-0 A-1 A-1 A-1 A-1 A-1 A-1 A-0 A-1

1 D-0 D-0 D-1 D-0 D-0 D-1 D-1 J-0 J-0 J-1 J-1 J-1 J-1 J-1 J-1 J-0 J-1

2 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-0 C-1 C-1 C-1 C-1

3 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-0 G-1 G-1 G-1 G-1

2 I-0 I-0 I-0 I-0

3 H-0 H-0 H-0 H-0

Hit? * * * * * * * * * * * *




12/60

Page Replacement - FIFO

FIFO is simple to implement When page in, place page id on end of list

Evict page at head of list

Might be good? Page to be evicted has been inmemory the longest time

But?

Maybe it is being used We just dont know

FIFO suffers from Beladys Anomaly fault ratemay increase when there is more physical

memory!


13/60


14/60

Parkinson's law : "Programs expand to fill the memory available to

hold them"

Idea : Manage the storage available efficiently between the available

programs.


15/60


16/60


17/60


18/60


19/60


20/60


21/60


22/60


23/60


24/60


25/60


26/60


27/60


28/60


29/60


30/60


31/60

Before VM

Programmers tried to shrink programs to fit tiny

memories

Result: Small

Inefficient Algorithms


32/60

Solution to Memory Constraints

Use a secondary memory such as disk

Divide disk into pieces that fit memory (RAM)

Called Virtual Memory


33/60

Implementations of VM

Paging

Disk broken up into regular sized pages

Segmentation Disk broken up into variable sized segments


34/60

Memory Issues

Idea: Separate concepts of

address space Disk

memory locations RAM

Example:

Address Field = 216 = 65536 memory cells

Memory Size = 4096 memory cells

How can we fit

the Address Space

into Main Memory?


35/60

Paging

Break memories into Pages

NOTE: normally Main Memory has thousands of

pages

page

pagepage

1 page = 4096 bytes

New Issue: How to manage addressing?


36/60

Address MappingMapping 2ndary Memory addresses to

Main Memory addresses

page

page page

1 page = 4096 bytes

physical addressirtual address

Add M i


37/60

Address Mapping

Mapping 2ndary Memory (program/virtual)

addresses to Main Memory (physical) addresses

pagepage page

1 page = 4096 bytes

physical addressused by hardwarevirtual addressused by program

4095 8191

0 4096

virtualphysical


38/60

Paging

page

pagepage

4095 8191

04096

virtualphysical

0

4095 /04095/0

Illusion that Main Memory isLarge

Contiguous

Linear

Size(MM) = Size(2ndry M)

Transparent to Programmer


39/60

Paging Implementation

Virtual Address Space (Program) & Physical

Address Space (MM)

Broken up into equal pages(just like cache & MM!!)

Page size Always a power of 2

Common Size:512 to 64K bytes


40/60

Paging Implementation

Page Frames

Page Tables

Programs use Virtual Addresses


41/60

Memory Mapping

Note: 2ndry Mem = 64K; Main Mem = 32K

Page Frame:

home ofVM pages in MMPage Table:

home ofmappings for VM pages

Page # Page Frame #


42/60

Memory Mapping

Memory Management Unit (MMU):Device that performs virtual-to-physical mapping

MMU

MMU

15-bitPhysicalAddress

32-bit VMAddress


43/60

Memory Management Unit

32-bit Virtual Address

Broken into 2 portions

20-bit 12-bit

Virtual page # offset in page

(since our pages are 4KB)

How to determine if page is in

MM?Present/Absent Bit

in Page Table Entry

MMU

D d P i


44/60

Demand Paging

Possible Mapping

of pages

Page Fault:

Requested page isnot in MM

Demand Paging:

Page is demanded

by program

Page is loaded into

MM

D d P i


45/60

Demand Paging

Possible Mapping

of pages

Page Fault:

Requested page isnot in MM

Demand Paging:

Page is demanded

by program

Page is loaded into

MM

But What to bring in for a program on start up?

W ki S t


46/60

Working Set

Set of pages used by a process

Each process has a unique memory map

Importance in regards to a multi-tasked OS

At time t, there is a set of all

k recently usedpages

References tend to cluster on a small number ofpages

Put this set to Work!!! Store & Load it during Process Switching


47/60

Page Replacement Policy

Working Set:

Set of pages used actively & heavily

Kept in memory to reduce Page Faults

Set is found/maintained dynamically by OS

Replacement: OS tries to predict which

page would have least impact on the

running program

Common Replacement Schemes:Least Recently Used (LRU)

First-In-First-Out (FIFO)


48/60

Replacement Policy

Placement Policy

Which page is replaced?

Page removed should be the page least likely tobe referenced in the near future

Most policies predict the future behavior on the

basis of past behavior


49/60

Basic Replacement Algorithms

Least Recently Used (LRU)

Replaces the page that has not been referenced

for the longest timeBy the principle of locality, this should be the

page least likely to be referenced in the near

future

Each page could be tagged with the time of last

reference. This would require a great deal of

overhead.


50/60


51/60


52/60


53/60

SRAM DRAM

DRAMs use only

one transistor, plusa capacitor.

DRAMs are smallerand less expensive

because SRAMs aremade from four to sixtransistors (flip flops)

per bit.

SRAMs don't requireexternal refresh

circuitry or other workin order for them tokeep their data intact.

SRAM is faster thanDRAM


54/60

It has been discovered that for about 90%ofthe time that our programs execute only 10%of our code is used!

This is known as the Locality PrincipleTemporal Locality

When a program asks for a location in memory , it willlikely ask for that samelocation again very soonthereafter

Spatial Locality When a program asks for a memory location at a memory

address (lets say 1000) It will likely need a nearbylocation1001,1002,1003,10004 etc.


55/60

fastest possible access (usually 1 CPU cycle)Registers


56/60

We established that the Locality principlestates that only a small amount of Memory isneeded for most of the programs lifetime

We now have a Memory Hierarchy that placesvery fast yet expensive RAM near the CPU andlargerslowercheaper RAM further away

The trick is to keep the data that the CPUwants in the small expensive fast memoryclose to the CPU and how do we do

that???


57/60

Hardware and the Operating System are

responsible for moving data throughout theMemory Hierarchy when the CPU needs it.

Modern programming languages mainly assume

two levels of memory, main memory and diskstorage.

Programmers are responsible for moving data

between disk and memory through file I/O.

Optimizing compilers are responsible for

generating code that, when executed, will cause

the hardware to use caches and registers

efficiently.


58/60

A computer program or a hardware-maintainedstructure that is designed to manage a cache ofinformation

When the smaller cache is full, the algorithmmust choose which items to discard to makeroom for the new data

The "hit rate" of a cache describes how often a

searched-for item is actually found in the cache The "latency" of a cache describes how long

after requesting a desired item the cache canreturn that item


59/60

Each replacement strategy is a compromise between hit rate andlatency.

Direct Mapped Cache The direct mapped cache is the simplest form of cache and the easiest to check for a

hit.

Unfortunately, the direct mapped cache also has the worst performance, becauseagain there is only one place that any address can be stored.

Fully Associative Cache The fully associative cache has the best hit ratio because any line in the cache can

hold any address that needs to be cached.

However, this cache suffers from problems involving searching the cache

A replacement algorithm is used usually some form of a LRU "least recently used"algorithm

N-Way Set Associative Cache The set associative cache is a good compromise between the direct mapped and set

associative caches.


60/60

Virtual Memory is basically the extension ofphysical main memory (RAM) into a lowercost portion of our Memory Hierarchy (letssay Hard Disk)

A form of the Overlay approach, managed bythe OS, called Paging is used to swap pages

of memory back and forth between the Diskand Physical Ram.

Hard Disks are huge, but to you remember howslow they are??? Millions of times slower that

21scs147l17replacementpolicy4-7[1]

Documents