Distributed Shared Memory Systems

Arush Nagpal (101303034)

Ankit Gupta (101303023)

What is a Distributed System?

What is DSM?

The distributed shared memory (DSM) implements the shared memory model in Distributed Systems, which have no physical shared memory.

The shared memory model provides a virtual address space shared between all nodes.

This overcomes the high cost of communication in distributed systems. DSM systems move data to the location of Access.

Memory

MappingManager

Memory

MappingManager

Memory

MappingManager

Shared Memory

NODE 1 NODE 2 NODE 3

Purpose Of DSM Research

Building less expensive parallel machines Building larger parallel machines Eliminating the programming difficulty of Massively

Parallel Processing and Cluster architectures

Distributed Shared Memory Models

Object Based DSM Variable Based DSM Page Based DSM Structured DSM Hardware Supported DSM

Object Based Probably the simplest way to implement DSM

Shared data must be encapsulated in an object

Shared data may only be accessed via the methods in the object

Variable Based Delivers the lowest distribution granularity

Closely integrated in the compiler

May be hardware supported

Hardware Based DSM Uses hardware to eliminate software overhead

May be hidden even from the operating system

Usually provides sequential consistency

May limit the size of the DSM system

Page Based DSM Involves virtual memory and paging

Advantages of DSM (Distributed Shared Memory)

Data sharing is implicit, hiding data movement (as opposed to ‘Send’/‘Receive’ in message passing model)

Passing data structures containing pointers is easier (in message passing model data moves between different address spaces)

Moving entire object to user takes advantage of locality difference

Less expensive to build than tightly coupled multiprocessor system: off-the-shelf hardware, no expensive interface to shared physical memory

Very large total physical memory for all nodes: Large programs can run more efficiently

Programs written for shared memory multiprocessors can be run on DSM systems with minimum changes

Issues faced in development of DSM

Granularity Structure of Shared memory Memory coherence and access

synchronization Data location and access Replacement strategy Thrashing

Granularity

Granularity is the amount of data sent with each update

If granularity is too small and a large amount of contiguous data is updated, the overhead of sending many small messages leads to less efficiency

If granularity is too large, a whole page (or more) would be sent for an update to a single byte, thus reducing efficiency

Structure of Shared Memory

Structure refers to the layout of the shared data in memory.

Dependent on the type of applications that the DSM system is intended to support. By datatype By database No structuring (simple linear array)

Replacement Strategy

If the local memory of a node is full, a cache miss at that node implies not only a fetch of accessed data block from a remote node but also a replacement.

Data block must be replaced by the new data block.- Example: LRU with access modes

Private (local) pages to be replaced before shared ones

Private pages swapped to diskShared pages sent over network to ownerRead-only pages may be discarded (owners

have a copy)

Thrashing

Thrashing occurs when network resources are exhausted, and more time is spent in validating data and sending updates than is used doing actual work.

Based on system specifics, one should choose write-update or write-invalidate to avoid thrashing.

Memory Coherence and Access Synchronization

In a DSM system that allows replication of shared data item, copies of shared data item may simultaneously be available in the main memories of a number of nodes.

To solve the memory coherence problem that deal with the consistency of a piece of shared data lying in the main memories of two or more nodes.

DSM are based on- Replicated shared data objects- Concurrent access of data objects at many nodes

Coherent memory: Mechanism that control/synchronizes accesses is needed to maintain memory coherence

Strict Consistency: when value returned by read operation is the expected value (e.g., value of most recent write) Write-Invalidate Write-update

Sequential consistency: A system is sequentially consistent if- The result of any execution of operations of all processors is the

same as if they were executed in sequential order, and- The operations of each processor appear in this sequence in the

order specified by its program

Replicated consistency:- All copies of a memory location (replicas) eventually contain

same data when all writes issued by every processor have completed

Algorithms for implementing DSM

The Central Server Algorithm The Migration Algorithm The Read-Replication Algorithm The full-Replication Algorithm

The Central Server Algorithm

- Central server maintains all shared dataRead request: returns data itemWrite request: updates data and returns

acknowledgement message- Implementation

A timeout is used to resend a request if acknowledgment fails

Associated sequence numbers can be used to detect duplicate write requests

If an application’s request to access shared data fails repeatedly, a failure condition is sent to the application

The Migration Algorithm- Operation

Ship (migrate) entire data object (page, block) containing data item to requesting location

Allow only one node to access a shared data at a time- Advantages

Takes advantage of the locality of reference DSM can be integrated with Virtual Memory at each node

To locate a remote data object: Use a location server Maintain hints at each node Broadcast query

- Issues Only one node can access a data object at a time Thrashing can occur: to minimize it, set minimum time data

object resides at a node

The Read-Replication Algorithm Replicates data objects to multiple nodes DSM keeps track of location of data objects Multiple readers-one writer protocol After a write, all copies are invalidated or updated Examples of implementations:

IVY(Integrated shared Virtual memory at Yale): owner node of data object knows all nodes that have copies

MIRAGE: Developed at UCLA, kernel modified to support DSM operation.

Clouds: Georgia Institute of Technology Advantage

The read-replication can lead to substantial performance improvements if the ratio of reads to writes is large

The Full-Replication Algorithm

- Extension of read-replication algorithm: multiple nodes can read and multiple nodes can write (multiple-readers, multiple-writers protocol)

- Issue: consistency of data for multiple writers- Solution: use of gap-free sequencer

• All writes sent to sequencer• Sequencer assigns sequence number and sends write

request to all sites that have copies• Each node performs writes according to sequence numbers• A gap in sequence numbers indicates a missing write

request: node asks for retransmission of missing write requests

List of References The Distributed Shared Memory System (Brian N.

Bershad, Matthew J. Zekauskas, Wayne A. Sawdon) March 1993

Techniques for Reducing Consistency-Related Communication in Distributed Shared-Memory Systems (JOHN B. CARTER, University of Utah and JOHN K. BENNETT and WILLY ZWAENEPOEL , Rice University )

Distributed Shared Memory (Ajay Kshemkalyani and Mukesh Singhal)

Any Questions?