A Scalable Virtual Registry Service for jGMA

Matthew Grove

DSG Seminar 3rd May 2005

2

Outline

• Introduction,

•Changes made to the jGMA design from last time (November 2004),

•The architecture of the Virtual Registry (VR),

•An implementation of the VR using IRC,

•Benchmarking jGMA,

•Summary.

3

Introduction

• jGMA is a pure Java reference implementation of the GGF's Grid Monitoring Architecture,

• The jGMA design philosophy is to reuse existing implementations rather than reinvent them,

• To complete the implementation jGMA requires a registry component which must:

– Be scalable,– Store sufficient information to be GMA compliant,– Be secure, and prevent unauthorised access to the data,– Need minimal configuration,– Ideally have no single point of failure,– Be robust and tolerant of network failure,– Efficient query routing between VRs.

4

Old jGMA Architecture

Tomcat

Consumer

Search/Registration

Tomcat

Producer

jGMARegistry

Producer/Consumer

Servlet

Producer/Consumer

Servlet

Search/Registration

Producer/ConsumerMessage Path

AdministrativeBoundary

Registration/Publish

5

Architecture Changes

• We combined the old Registry and PC Servlets into one service called the Mediator,

• The Mediator provides hooks for a distributed Virtual Registry,

• The client-side architecture has remained untouched.

6

New jGMA Architecture

7

Implementation Changes

•Moved from Tomcat[4|5] to Jetty,

•Changed the internal API to cleanly encapsulate external communications code,

•Rewrote all code to make better use of features of Java 1.5 including the new concurrency package.

8

The Mediator

The Mediator replaces the PC Servlet and the Registry Servlet.

9

The jGMA Library and API

A choice of two local registries have been implemented: Simple (volatile) and MySQL.

The Mediator did not require changes to the client code.

10

Virtual Registry Services

•Bootstrapping: Joining the jGMA network with minimal hardwiring,

•Communications: Efficient routing of queries between VR peers,

•Caching: Keeping a temporary local copy of some information to reduce the amount of communications between peers.

The API is designed to allow different

implementations of these core services to be

mixed and matched.

11

IRC VR Service

• It is an attempt at meeting the criteria of the jGMA VR by building a service which uses the Internet Relay Chat protocol.

• IRC networks have servers connected via a graph topology, some networks manage thousands of users. Simultaneous connections yesterday on some major networks:

–Efnet – 89,817–UnderNet - 75,191–Freenode – 21,953

12

IRC VR Service Continued

– The Bot joins a channel and announces itself to other Bots,

– The Bot publishes search queries to the IRC channel,

– Other Bots receive the queries and pass them onto the Registry backend,

– The Bots reply to the queries via a private message over IRC.

Each Mediator has an “IRC Bot” which connects to a chosen IRC network (the IRC networks maintain a DNS pool providing fault tolerant bootstrapping).

All messages are routed by the IRC network.

13

IRC VR Walk-Through

14

IRC VR Strengths

• Established distributed network (IRC was ‘born’ in 1988 and the RFC was published in 1993),

• Efficient / scalable routing of messages,• Establishing client liveliness is handled by the IRC

protocol,• Fault tolerant bootstrapping via DNS,• Various degrees of security can be provided by

combinations of:–Leveraging functionality built into the IRC protocol

(locking the channel, private messages),–Using features of the IRC daemons such as encrypted

routing for IRC messages,–Running your own IRC daemons rather than using a

public network.

15

IRC VR Implementation

•Done:–Bots join the IRC network,

–Bots can post and read jGMA queries to channels,

–Bots can receive query responses via IRC private messages.

•TODO:–The IRC Bots do not interact with the Mediator yet,

–Bots need to be able to join a locked channel,

–There is no caching (this is a generic VR service, not just for the IRC VR).

16

IRC VR Issues / Questions

•What are the limits on individual channels, is there a need to have multiple channels and route messages between them?

•Using IRC requires an extra port be accessible to the mediator (out going port 6667).

17

Future Work

•Short term:–Complete the IRC VR,

–Implement a generic caching service for the VR,

–Write a simple P2P VR which can be used in place of the IRC VR.

–Benchmark jGMA against Naradabroker.

• Longer term:–Do another binary release of jGMA,

–Develop an application or library on top of jGMA.

18

Summary

•Current work is focusing on the implementation of IRC VR,

•The design of the jGMA architecture is hopefully complete.

19

•Project Web page:–http://dsg.port.ac.uk/projects/jGMA/

•The DSG Web page:–http://dsg.port.ac.uk/

Links