introducing tibco gridserver?? · 2014. 11. 14. · † gridserver service-oriented integration...

Two-Second Adv

TIBCO GridServer®

Introducing TIBCO GridServer®

Software Release 6.2November 2014

antage®

Important Information

SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCH EMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (OR PROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THE EMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANY OTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE.

USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS AND CONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTED SOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THE CLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOAD OR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE) OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USER LICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE “LICENSE” FILE(S) OF THE SOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, AND YOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BE BOUND BY THE SAME.

This document contains confidential information that is subject to U.S. and international copyright laws and treaties. No part of this document may be reproduced in any form without the written authorization of TIBCO Software Inc.

TIBCO, Two-Second Advantage, GridServer, FabricServer, GridClient, GridBroker, FabricBroker, LiveCluster, VersaUtility, VersaVision, SpeedLink, Federator, and RTI Design are either registered trademarks or trademarks of TIBCO Software Inc. in the United States and/or other countries.

EJB, Java EE, J2EE, and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries.

TIBCO products may include some or all of the following:

Software developed by Terence Parr.

Software developed by the Apache Software Foundation (http://www.apache.org/).

This product uses c3p0. c3p0 is distributed pursuant to the terms of the Lesser General Public License. The source code for c3p0 may be obtained from http://sourceforge.net/projects/c3p0/. For a period of time not to exceed three years from the Purchase Date, TIBCO also offers to provide Customer, upon written request of Customer, a copy of the source code for c3p0.

Software developed by MetaStuff, Ltd.

Software licensed under the Eclipse Public License. The source code for such software licensed under the Eclipse Public License is available upon request to TIBCO and additionally may be obtained from http://eclipse.org/.

Software developed by Info-ZIP.

This product includes Javassist licensed under the Mozilla Public License, v1.1. You may obtain a copy of the source code from http://www.jboss.org/javassist/

This product includes software licensed under the Common Development and Distribution License (CDDL) version 1.0. The source code for such software licensed under the Common Development and Distribution License (CDDL) version 1.0 is available upon request to TIBCO.

Software developed by Jason Hunter & Brett McLaughlin.

Software developed by JSON.org.

Software developed by QOS.ch.

Software developed by the OpenSymphony Group (http://www.opensymphony.com/).

This product includes WSDL4J software which is licensed under the Common Public License, v1.0. The source code for this software may be obtained from TIBCO’s software distribution site.

Software developed by the Indiana University Extreme! Lab (http://www.extreme.indiana.edu/).

Software developed by Jean-loup Gailly and Mark Adler.

All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only.

THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOT ALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASED AT THE SAME TIME. SEE THE README FILE FOR THE AVAILABILITY OF THIS SOFTWARE VERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM.

THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.

THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCO SOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME.

THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY OR INDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES.

This Product is covered by U.S. Patent No. 6,757,730, 7,093,004, 7,093,004, and patents pending.

Copyright © 1999-2014 TIBCO Software Inc. ALL RIGHTS RESERVED.

TIBCO Software Inc. Confidential Information

| vii

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiTIBCO GridServer Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiOther Documentation and Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiii

Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiv

Connecting with TIBCO Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiHow to Join TIBCOmmunity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiHow to Access All TIBCO Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiHow to Contact TIBCO Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Chapter 1 GridServer Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Product Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Grid Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

GridServer Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Operating Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Chapter 2 GridServer Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Clients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Service Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Service Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Service benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Binary-level Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Chapter 3 Inside the Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

The Engine Daemon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Engine Instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Engine Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Engine Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21


viii |

Chapter 4 Inside the GridServer Director. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Director Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Configuration Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Database Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Resource Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Chapter 5 Inside the Broker. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Broker Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Engine and Driver Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Resource Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Chapter 6 Inside GridServer Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Topography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Direct Data Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 7 Inside GridCache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

General Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Cache Configuration and Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Data Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Consistency/Synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Cache Loaders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Disk/Memory Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Chapter 8 Inside the Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Client Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49


| ix

Chapter 9 Inside the GridServer Batch Processing Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . .51

Parts of a Batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Batch Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Batch Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Scheduling Batch Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Administering Batches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55


x |


| xi

Preface

TIBCO GridServer® is a highly scalable software infrastructure that enables application services to operate in a virtualized fashion, unattached to specific hardware resources. Client applications submit requests to the Grid environment and GridServer dynamically provisions services to respond to the request. Multiple client applications can submit multiple requests in parallel and GridServer dynamically creates multiple service instances to handle requests in parallel on different Grid nodes. This architecture is therefore highly scalable in both speed and throughput. For example, a single client will see scalable performance gains in the processing of multiple requests, and many applications and users will see scalable throughput of the aggregate load.

Topics

• Related Documentation, page xii

• Typographical Conventions, page xiv

• Connecting with TIBCO Resources, page xvii


xii |

Related Documentation

This section lists documentation resources you may find useful.

TIBCO GridServer DocumentationThe following documentation is included with GridServer in Adobe Acrobat (PDF) format. To view the guides, log in to the Administration Tool and go to Admin > Documentation. The PDF files are also on the Manager at livecluster/admin/docs. From the Documentation page, you can also search all documentation for a phrase or keywords. The following documents form the GridServer documentation set::

• Introducing GridServer Introduces GridServer and key concepts and terms such as work, Engines, Directors, and Brokers. Read this first if you are new to GridServer.

• GridServer Administration Guide Tells the system administrator how to operate a GridServer installation. It describes scheduling, fault-tolerance, failover, performance and tuning, and other concepts and procedures.

• GridServer Installation Guide Describes how to install GridServer for Windows and Unix, including Managers, Engines, and pre-installation planning.

• GridServer Developer’s Guide Provides information on writing applications for GridServer. Subjects include Service Domains, using Services, PDriver (the Batch-oriented GridServer Client), the theory behind development with the GridServer API, and concepts needed to write and adapt applications.

• GridServer Speedlink Guide Describes SpeedLink, a high throughput, low latency compute paradigm implemented using GridServer, and provides information on how to use SpeedLink for both developers and administrators.

• GridServer Service-Oriented Integration Tutorial Tutorial on developing applications for GridServer using Services such as Java, .NET, native, or binary executable Services.

• GridServer PDriver Tutorial Tutorial on using PDriver, the Parametric Service Driver, to create and run Services with GridServer.

• GridServer COM Tutorial Tutorial explaining how client applications in Windows can use COMDriver, GridServer’s COM API, to work with services on GridServer.


| xiii

Other Documentation and HelpAdditional sources of help include:

• GridServer Administration Tool Help Click the context-sensitive help on any page of the GridServer Administration Tool to see online help.

• API Reference For information on the GridServer API, see the GridServer SDK in the docs directory. Java API information is in JavaDoc format; C++ documentation is in HTML; and .NET API help is in HTMLHelp. You can view and search the API documentation from the GridServer Administration Tool, also: log in to the Administration Tool and go to Admin > Documentation.


xiv |

Typographical Conventions

The following typographical conventions are used in this manual.

Table 1 General Typographical Conventions

Convention Use

TIBCO_HOME

DS_INSTALL

DS_MANAGER

DS_DATA

Many TIBCO products must be installed within the same home directory. This directory is referenced in documentation as TIBCO_HOME. The default value of TIBCO_HOME depends on the operating system. For example, on Windows systems, the default value is C:\tibco.

TIBCO GridServer® installs into a datasynapse directory within TIBCO_HOME. This directory is referenced in documentation as DS_INSTALL. The default value of DS_INSTALL depends on the operating system. For example, on Windows systems, the default installation directory is C:\tibco\datasynapse.

The Manager directory contains the read-only software files; by default, it is a directory within DS_INSTALL named manager, and is referred to as DS_MANAGER. For example, on Windows systems, the default Manager directory is C:\tibco\datasynapse\manager.

The data directory is the location of all volatile files used by the application server, such as server properties and configuration. By default, it is a directory within DS_INSTALL named manager-data, and is referred to as DS_DATA. For example on Windows systems, the default data directory is C:\tibco\datasynapse\manager-data

code font Code font identifies commands, code examples, filenames, pathnames, and output displayed in a command window. For example:

Use MyCommand to start the foo process.

bold code

font Bold code font is used in the following ways:

• In procedures, to indicate what a user types. For example: Type admin.

• In large code samples, to indicate the parts of the sample that are of particular interest.

• In command syntax, to indicate the default parameter for a command. For example, if no parameter is specified, MyCommand is enabled: MyCommand [enable | disable]


| xv

italic font Italic font is used in the following ways:

• To indicate a document title. For example: See TIBCO ActiveMatrix BusinessWorks Concepts.

• To introduce new terms. For example: A portal page may contain several portlets. Portlets are mini-applications that run in a portal.

• To indicate a variable in a command or code syntax that you must replace. For example: MyCommand PathName

Key combinations

Key names separated by a plus sign indicates keys pressed simultaneously. For example: Ctrl+C.

Key names separated by a comma and space indicate keys pressed one after the other. For example: Esc, Ctrl+Q.

The note icon indicates information that is of special interest or importance, for example, an additional action required only in certain circumstances.

The tip icon indicates an idea that could be useful, for example, a way to apply the information provided in the current section to achieve a specific result.

The warning icon indicates the potential for a damaging situation, for example, data loss or corruption if certain steps are taken or not taken.

Table 1 General Typographical Conventions (Continued)

Convention Use

Table 2 Syntax Typographical Conventions

Convention Use

[ ] An optional item in a command or code syntax.

For example:

MyCommand [optional_parameter] required_parameter

| A logical OR that separates multiple items of which only one may be chosen.

For example, you can select only one of the following parameters:

MyCommand param1 | param2 | param3


xvi |

{ } A logical group of items in a command. Other syntax notations may appear within each logical group.

For example, the following command requires two parameters, which can be either the pair param1 and param2, or the pair param3 and param4.

MyCommand {param1 param2} | {param3 param4}

In the next example, the command requires two parameters. The first parameter can be either param1 or param2 and the second can be either param3 or param4:

MyCommand {param1 | param2} {param3 | param4}

In the next example, the command can accept either two or three parameters. The first parameter must be param1. You can optionally include param2 as the second parameter. And the last parameter is either param3 or param4.

MyCommand param1 [param2] {param3 | param4}

Table 2 Syntax Typographical Conventions

Convention Use


| xvii

Connecting with TIBCO Resources

How to Join TIBCOmmunityTIBCOmmunity is an online destination for TIBCO customers, partners, and resident experts, a place to share and access the collective experience of the TIBCO community. TIBCOmmunity offers forums, blogs, and access to a variety of resources, including a GridServer-specific community. To register, go to http://www.tibcommunity.com.

How to Access All TIBCO DocumentationAfter you join TIBCOmmunity, you can access the documentation for all supported product versions here:

http://docs.tibco.com/

How to Contact TIBCO SupportFor comments or problems with this manual or the software it addresses, please contact TIBCO Support as follows.

• For an overview of TIBCO Support, and information about getting started with TIBCO Support, visit this site:

http://www.tibco.com/services/support

• If you already have a valid maintenance or support contract, visit this site:

https://support.tibco.com

Entry to this site requires a user name and password. If you do not have a user name, you can request one.


http://www.tibco.com/services/support

https://support.tibco.com

http://docs.tibco.com/

http://www.tibcommunity.com

xviii |


| 1

Chapter 1 GridServer Overview

This guide is your complete introduction for learning about DataSynapse GridServer concepts. It includes a complete overview of GridServer fundamentals, and is meant to be read first, before installation or development.

Topics

• Product Overview, page 2

• Grid Services, page 3

• GridServer Architecture, page 5

• Operating Environment, page 7


2 |

Product Overview

GridServer is a highly scalable software infrastructure that allows application services to operate in a virtualized fashion, unattached to specific hardware resources. Client applications submit requests to the Grid environment and GridServer dynamically provisions services to respond to the request. Multiple client applications can submit multiple requests in parallel and GridServer dynamically creates multiple service instances to handle requests in parallel on different Grid nodes. This architecture is therefore highly scalable in both speed and throughput. For example, a single client will see scalable performance gains in the processing of multiple requests, and many applications and users will see scalable throughput of the aggregate load.

A scalable architecture should provide linear gains in performance with the addition of more hardware resources, while making no impact on the manageability of the distributed application platform. This means that the system should be as manageable with ten nodes as it is with a thousand nodes. GridServer offers this type of operating environment—incremental resources added to the system impact performance in a linear fashion, but have no effect on management—including deployment of application services; configuration and administration; runtime management and monitoring; and historical reporting and usage statistics.

This chapter is a high-level overview of the GridServer platform. It will cover three separate areas: Grid Services, the GridServer architecture, and management and operations of the infrastructure.


| 3

Grid Services

The ability to integrate a new technology into a variety of legacy environments is a core requirement of any strategic technology. GridServer offers support for multiple languages and different integration strategies to meet this goal.

With GridServer, Grid clients (applications that use the Grid) can interoperate with any Service hosted on the Grid—regardless of the language or system that the components are running. For example, an application can make Grid Service calls that are implemented in any language and system, including the following:

Any Service can be accessed by Grid client applications in a number of ways, including the following:

Grid Service Implementations

Java Classes

.NET Classes

C++ Classes

C Functions in a DLL/SO

Excel (extension package)

Any executable or script

R functions

Grid Client Interfaces

Java/J2EE

.NET API (VB.NET, C#)

COM Object

C++/C

Batch Clients

R functions


4 |

GridServer enables a truly Service-oriented approach to Grid computing. The Grid client creates a Service instance and calls operations (methods) on the Service. These calls are virtualized and invoked on multiple Grid nodes. This provides limitless scalability for hosting these Services and offers a simple programming model for invoking these Services.

Services are described in more detail in Chapter 2, GridServer Services, on page 9.


| 5

GridServer Architecture

DataSynapse GridServer has four different components in its architecture:

• Grid Clients — The components that submit service requests to the Grid. Also called Drivers.

• Engines — The processes that host and run services on the Grid nodes.

• Brokers — The component that provides request queuing, scheduling and load-balancing. Brokers are also responsible for managing Engines.

• Directors — The component that assigns Grid Clients and Engines to Brokers based on policy. They manage and load balance Engines across available Brokers.

Figure 1 Engines and Grid Clients log in to the Director and are authenticated; the Director then routes Engines and Grid Clients to available Brokers.


6 |

Grid Clients submit requests through the Broker. Engines receive work requests from the Broker, and in most cases, exchange data directly with the Grid Clients. This allows the system to be highly scalable. Since the Brokers manage all work requests, load balancing is optimal and resilience is built into the system.

Figure 2 Brokers manage Engines and Grid Clients, and schedule work via lightweight messages. Grid Clients and Engines exchange work data directly when using Direct Data Transfer.


| 7

Operating Environment

GridServer includes a highly manageable operating environment featuring a web-based console, the GridServer Administration Tool, and a SOAP-based administration interface. Both allow you to deploy Services, manage the workloads running on the Grid, and configure the GridServer environment. The screenshot below shows the home page in the GridServer Administration Tool.

Deploying and registering Services and applications is made available throughout the Administration Tool. Uploading binaries and controlling versions is straightforward.

Figure 3 The GridServer Administration Tool.


8 |

The Administration Tool has a number of graphical runtime monitors to quantify Grid usage and current operations. Diagnostic tools enable remote debugging, logging, and error extraction.

Management of the GridServer platform is described in more detail in the GridServer Administration Guide.

Figure 4 The Broker Monitor.


| 9

Chapter 2 GridServer Services

A Service is a self-contained application or business function that is distributed to remote computing resources. A client application makes requests, which are routed by GridServer to those resources, which then return results to the client. One use of a Service is to load balance an application, whereby the client application receives requests and submits them asynchronously to the Service, which then executes these multiple, unrelated computations in parallel. Another use is to execute a single, larger computation. The client splits the computation into multiple, independent pieces, submits them asynchronously, and combines the returned results.

Topics

• Services, page 10

• Binary-level Integration, page 13


10 |

Services

The Service-Oriented method of defining work in GridServer is a standards-based model. It uses a thin client model, which promotes easy integration of an existing implementation. It also promotes language interoperability, as clients written in different languages can invoke methods in Service Implementations written in the same or other languages.

The Service-Oriented method uses two components: Clients and Service Implementations. Descriptions of both follow.

ClientsA Service Session is created on a client for a given named Service Type. The client invokes methods using implementation resources that are distributed on Engines.

You can create a Service client in different ways:

• A client-side API in Java, COM, C++, R, or .NET.

• A service proxy of Java or .NET client stubs generated by GridServer.

Figure 5 The relationship between Service Clients and Service Implementations.


| 11

Service ImplementationService Implementations are libraries or executables that are deployed to Engines and that process requests from clients. They process data and return results back to the client. Service Implementations are associated with Service Types by registering them on the Director. When a client makes a client request, it sends the request to a Manager instead of directly requesting an Engine to do the work. This one-to-many relationship provides fault tolerance and scalability for Services.

Service Implementations can be constructed with any of the following:

• Arbitrary Java classes

• Arbitrary .NET classes

• A Dynamic Library (.so, .DLL) with methods that conform to a simple input-output string interface

• R functions

• A command, such as a script or binary executable

Integration as a Service in most cases requires minimal changes to the client application.

Service SessionA running service is a Service Session. This includes the Service Client, Service Implementation, and Service state on all components. After a client creates a Service and the Service Implementation is running on Engines, this is collectively called the Service Session.

Service benefitsThere are many advantages to Services:

• Cross-language — Client and Service can be in different languages

• Dynamic — Method names can be determined dynamically, or use generated proxies for type safety

• Flexible — Use synchronous or asynchronous invocation patterns; can use client proxies generated by GridServer

• Virtual — Client-Engine correspondence is not one-to-one; Service requests are adaptively load balanced

• Stateful — Despite being virtual, stateful Services can be handled

• Standards — Standards-compliant


12 |

For more information on Services, see the GridServer Developer’s Guide.


| 13

Binary-level Integration

PDriver, or the Parametric Job Driver, is a Driver that executes existing command-line programs as a parallel processing Service using the GridServer environment without using the API, taking full advantage of the parallelism and fault tolerance of GridServer.

PDriver achieves parallelism by running the same program on Engines several times with different parameters. A script is used to define how these parameters change. For example, a distributed search mechanism using the grep command could conduct a brute-force search of a network-attached file system, with each task in the Service being given a different directory or piece of the file system to search.

PDriver uses its scripting language, PDS, to define Services. PDS scripts can also set options for a PDriver Service, such as remote logging and exit code checking.

For more information on PDriver, see the GridServer Developer’s Guide.


14 |


| 15

Chapter 3 Inside the Engine

This chapter describes the GridServer Engine.

Topics

• Overview, page 16

• The Engine Daemon, page 17

• Engine Instances, page 18

• Engine Modes, page 19

• Installation and Configuration, page 20

• Engine Life Cycle, page 21


16 |

Overview

The GridServer Engine is a lightweight container that hosts and runs Services within its process. In a sense, GridServer is a distributed, scalable application service layer. GridServer allows you to host business objects within a middle tier, composed of GridServer Engines.

The Engines create Service instances on demand, based on the scheduling decisions that the Broker makes. The first time it creates the Service, it is because a client has requested an operation on that Service type. In this case, it will create the Service, run the operation (method), and then store the Service in its cache. The scheduler is aware of the contents of this cache and will route other requests for that Service to the Engine.

By default, the Engines are single-threaded workers performing only one Service operation at a time. Often, more than one Engine instance will be running on a host. These processes are started and managed by an agent, the Engine Daemon, that runs on the host.

The Engine Daemon is capable of starting and stopping Engines based on pre-defined policy. These policies are based on CPU utilization of the host, user activity (for example, if it is a user’s desktop), or time of day. In most cases, the Engine Daemon starts Engine instances, monitors them and restarts them if there are failures or reconfigurations.

As used in this chapter, the terms Engine and Engine instance are synonymous. We use Engine instance to avoid confusion with Engine Daemon.

Figure 6 The GridServer Engine.


| 17

The Engine Daemon

The Engine Daemon manages Engines on resource computers. It is the program you install on a computer to maintain Engines on that machine. It runs as a separate process from the Engine Instances.

One Engine Daemon will run per machine. It starts and stops one or more Engines based on configuration and machine state.The Engine Daemon provides configuration for Engines. Engine Daemons log in to Directors so that they can be administered and configured. When Engine configuration changes are made by an administrator on a Director, they are automatically propagated to Engines through the Engine Daemon.


18 |

Engine Instances

The Engine Instance is the process that performs the actual processing of work. Typically, one Engine instance is run per CPU on a machine. By default, each Engine instance will run in its own process. You can also run all Engines in a single process to allow sharing of Service Session data across instances, at the cost of requiring that those instances only work on the same Session at the same time.

GridServer Engines report to the Manager when they are available for work. After logging in and synchronizing resources, they accept work assignments, run Services, and notify the Manager when results are ready.

Because the Engine Daemon controls the state of configuration on each Engine Instance and the Engine configuration is controlled centrally in the GridServer Administration Tool, it is very easy to control and configure Engine Instances across the Grid, changing when and how Engine Instances run.

Figure 7 Multiple Engine Instances using one Engine Daemon on a multiprocessor computer.


| 19

Engine Modes

Engines can be configured to run in a variety of modes, depending on the type of machines on which they will be installed. Dedicated Engines are configured to run continuously. This mode is for computing resources that are devoted to full-time processing on a Grid. Non-Dedicated mode is used for machines that are used part-time on the Grid, and also used for other purposes, such as desktop PCs.

Engines configured in the Non-Dedicated mode determine when to run based on two different modes. In the UI Idle mode, a non-dedicated Engine will start running after there has been no user activity for a set amount of time on the installed PC. The Engine stops immediately when the user returns. In CPU Idle mode, the Engine runs when CPU utilization is low.


20 |

Installation and Configuration

Engines are only installed on a machine once; their configuration is centrally managed, and they are automatically upgraded with a later version of GridServer when it is available.

An installer is provided for Windows Engines. Engines can be manually installed, or automatically installed across a network in conjunction with SMS. An archive file is provided for Linux and Solaris.

Once installed, the Engine configuration can be changed from anywhere on the network with the GridServer Administration Tool. It’s also possible to create configuration profiles which are used by multiple machines to synchronize configurations.

Figure 8 The Engines > Engine Install page in the DataSynapse GridServer Administration Tool.


| 21

Engine Life Cycle

The following briefly describes the life cycle of an Engine:

1. The Engine Daemon, which has been running on the computer, determines that an Engine Instance should be running, based on the Engine Mode and computer’s state.

2. The Engine Instance establishes a connection with a Broker via a Director.

3. The Engine Instance requests a task.

4. The Broker assigns a task to the Engine Instance. The Engine Instance downloads the task data from the Driver if Direct Data Transfer (DDT) is enabled, or the Broker if it isn’t.

5. The Engine Instance runs the task.

6. The Engine Instance sends a message to the Broker again to indicate that it has finished the task and request another; if DDT is enabled, it includes the data location; otherwise, it includes the actual data.

If the Engine is interrupted or fails gracefully, it sends a message to the Broker logging out. If an Engine fails unexpectedly, the Engine Monitor on the Broker will log off the Engine. In either case, the Broker reschedules the Engine’s task.

For more information on installing Engines, see the GridServer Installation Guide.


22 |


| 23

Chapter 4 Inside the GridServer Director

The GridServer Director is part of a GridServer Manager, the controlling mechanism of the Grid. Directors route Engines and Drivers to Brokers, and manage Brokers and Engine Daemons. Directors balance the load among their Brokers and provide Broker fault tolerance.

Topics

• Director Roles, page 24

• Configuration Management, page 26

• Database Management, page 27

• Resource Management, page 28


24 |

Director Roles

The Director is responsible for three basic roles: routing, authentication, and load balancing.

RoutingOne of the GridServer Director’s important roles within the Manager is to serve as the routing component. Both Engines and Clients log in via the Director, and the Director routes them to the appropriate Brokers.

AuthenticationThe Director, as part of routing, handles Client authentication.

Load BalancingBecause Directors take a central role with both Client and Engine traffic, they are a key object in configuring the load balancing of components. Directors allocate Clients and Engines to available Brokers.

The default policy for load balancing by Directors is by relative weight; essentially, a Director will assign Clients and Engines to Brokers so that they are equally distributed.

Directors balance load by using the following techniques:

• Clients are routed based on the roles that are assigned to the user running the Client.

Figure 9 Engines and Grid Clients log in to the Director and are authenticated; the Director then routes Engines and Grid Clients to available Brokers.


| 25

• Each Broker can be configured with weight settings for Clients and Engines which balance the number of clients given to that Broker by the Director.

• Idle Engines will be relocated to other Brokers if and when needed.

• An Engine configuration can be set to have a Home Broker. A Broker can be set up to have a set of Shared Brokers. Any Engine that uses that configuration is routed only to its Home Broker or any Broker that its Home shares to.


26 |

Configuration Management

The Director is used to configure many settings of a GridServer Manager, including users and passwords, Driver profiles, routing properties, and Engine configurations. These settings are configured with the web-based GridServer Administration Tool.


| 27

Database Management

GridServer can use an external reporting database to store data such as User, Engine, Driver, and Broker information. The reporting database is an enterprise-grade JDBC database that you provide that is used to log events and statistics. The configuration of the database used is changed on the Director from the GridServer Administration Tool.


28 |

Resource Management

Service Deployment files that are used by Engines are centrally managed, starting at the Director. The resources centrally located on the Director are then synchronized to Brokers, which then synchronize them with Engines.

Resources are distributed with Grid Libraries. A Grid Library is an archive containing a set of resources and properties necessary to run a Grid Service, along with configuration information that describes how those resources are to be used. Grid Libraries can contain JARs, native libraries, configuration files, environment variables, and hook files or alternate JREs needed to run a Service. Grid Libraries offer the flexibility to do such things as package a Service for use on multiple OSes from a single archive, upgrade resources without interrupting running Sessions, and change Grid Library variables without redeploying a Grid Library.


| 29

Chapter 5 Inside the Broker

The GridServer Broker, like the Director, is an integral part of a GridServer Manager. Brokers schedule work requests from Drivers to Engines, and manage both types of components to ensure service and return results.

Topics

• Broker Roles, page 30

• Resource Management, page 33


30 |

Broker Roles

A Broker’s roles include scheduling, Engine and Driver management, communication, and resource management.

SchedulingBrokers schedule tasks submitted by Drivers onto available Engines. To determine the scheduling of tasks to Engines, Brokers take into consideration Engine state and Service priority.

The Engine state is based on whether that Engine has loaded Grid Libraries needed by that Service, and whether that Session is cached on the Engine. This minimizes the amount of initialization work involved in running a Service on an Engine.

When the Classic Scheduler is used, the priority of a Service is considered when scheduling. Service priority enables more important work to take more resources, and urgent priority work can preempt less important work. The priority of a Service Session is set when created, can be modified at runtime.

When the SLA scheduler is used, a Session is set to be a member of a predefined SLA group. Such groups are defined on the Broker. The SLA values for groups are either defined as a percentage of Engines on the Broker, or a number of actual Engines. Within a group, Engines will be equally distributed to all Sessions in that group. The scheduler will guarantee that those SLA values are met, provided there are enough Engines available.

Brokers have several other methods used to control scheduling of tasks:

Figure 10 Brokers manage Engines and Drivers, and schedule work with lightweight messages. Drivers and Engines exchange work data directly when using Direct Data Transfer.


| 31

• Discriminators can be used to select or reject a subset of Engines when scheduling specific tasks or Services. For example, you can use discriminators to target Service requests to Engines running specific operating systems which contain their compiled code.

• Engine Blacklisting provides a method of preventing Engines from taking tasks from a Service if they have previously failed on a task from the same Service.

• The dependency mechanism enables the construction of workflow on a Broker without the need for an active Grid client to manage dependencies (wait for completed tasks, submit more based on successful outcome, and so on). When a Session is submitted, one or more Tasks or entire Services can be required to be completed prior to the Service being scheduled. These dependencies can be Sessions already submitted, or ones that have not yet been created. This way, multiple Sessions in different Services can be submitted, but they will not be eligible for scheduling until certain conditions are met—namely the successful completion of specific Sessions.

Brokers handle rescheduling of tasks when Engines go offline. When there is a clean reset of an Engine, such as when its host PC becomes too busy with user activity, it will notify the Broker; the Broker will determine if an Engine has failed because it is monitoring its connection. In either case, any outstanding work will be rescheduled to other Engines.

Engine and Driver ManagementAlthough authentication and assignment of Engines and Drivers is handled by the Director, the Broker manages clients through the remainder of their life cycle. This involves determining the availability of clients by monitoring their connections, and coordinating these results with the scheduling effort undertaken by the Broker.

CommunicationThe GridServer architecture combines the central control of a hub and spoke architecture with the efficient communication of a peer-to-peer architecture. This is made possible by the flexibility in configuring communication between the Broker, Engines, and Drivers.

Engines and Drivers communicate with Brokers using lightweight transaction messages. Direct Data Transfer (DDT) enables Engines and Drivers to exchange work data directly. This peer-to-peer communication eliminates any “heavy lifting” between the Broker and clients, improving its performance.


32 |

DDT can be disabled in situations where network layout requires all communication to go through a central point like a Broker. This will cause both transaction messages and data to be transferred through the Broker, creating a more traditional hub and spoke architecture between Brokers and clients.


| 33

Resource Management

Brokers maintain a copy of deployed resources from the Primary Director, and Engines synchronize these resources from the Broker


34 |


| 35

Chapter 6 Inside GridServer Topology

The GridServer Manager loosely consists of two entities: the GridServer Director and the GridServer Broker:

A minimal configuration of DataSynapse GridServer would consist of a single Manager configured with a Primary Director and a single Broker. Additional Brokers or Directors can be added to address three primary concerns: redundancy, volume, and application segregation.

Topics

• Redundancy, page 36

• Volume, page 37

• Topography, page 38

• Direct Data Transfer, page 39


36 |

Redundancy

Given a minimal configuration of a single Director and single Broker, Engines and Drivers will log in to the Director, but failure of the Director (such as in the case of hardware or network failure) would mean a Driver or Engine not logged in would no longer be able to establish a connection.

To prevent this, redundancy can be built into the GridServer architecture. One method is to run a second Manager with a Secondary Director, and configure Engines and Drivers with the address of both Directors. If the Primary Director fails, Engines and Drivers will contact the Secondary Director, which contains identical Engine configurations and will route Engines and Drivers to Brokers in the same manner as the Primary Director. The figure above shows an implementation with two Managers:

In addition to redundant Directors, a Broker can have a backup on another Manager. A Broker can be designated a Failover Broker on a Manager during installation or in the Manager Configuration page. A failover Broker acts as a backup Broker for times when other Brokers are offline. Under normal operation, they are idle.

For more information on redundancy, see the GridServer Administrator’s Guide.


| 37

Volume

In larger Grids, the volume of Engines in the Grid may require more capability than offered by a single Broker. To distribute load, additional Brokers are added to other Managers at installation. For example, the figure below shows a two Manager system with two Brokers


38 |

Topography

Several other factors may influence how you will integrate DataSynapse GridServer with your computing environment. These include:

• Instead of using one Grid for all types of Services, you may wish to divide different subsets of Services (for example, by size or priority) to different Brokers.

• Your network may dictate how your Manager environment should be planned. For example, if you have offices in two parts of the country and a relatively slow extranet but a fast intranet in each location, you could install a Manager in each location.

• Different Managers can support data used for different types of Services. For example, one Manager can be used for Services accessing a SQL database, and a different Manager can be used for Services that don’t access the database.

With this flexibility, it’s possible to architect a Manager model to provide a work space that will facilitate your workload and traffic. For more information on how to best design your Manager environment, contact our Integration Services staff, and we can help you determine how to best configure your installation.


| 39

Direct Data Transfer

By default, GridServer uses Direct Data Transfer, or peer-to-peer communication, to optimize data throughput between Drivers and Engines. Without Direct Data Transfer, all task inputs and outputs must be sent through the Manager. Sending the inputs and outputs through the Manager will result in higher memory and disk use on the Manager, and may lower throughput overall.

With Direct Data Transfer, only lightweight messages are sent though the Manager, and the “heavy lifting” is done by the Driver and Engine nodes themselves


40 |


| 41

Chapter 7 Inside GridCache

This chapter describes GridCache, a data access and caching tier available for Grid Services and clients.

Topics

• Overview, page 42

• General Capabilities, page 43


42 |

Overview

GridCache is a data access and caching tier available for Grid Services and clients. In general, it allows applications that have heavy data access requirements to run efficiently in a Grid environment. Cache clients can write or put objects into the cache for retrieval by other Grid nodes, or cache loaders can be installed that provide access to external data resources such as relational databases. By doing so, GridCache serves as binary cache in front of a relational database, improving read access efficiency by many orders of magnitude. In addition, GridCache allows the software architecture to enforce clear separation of data access logic from the business logic contained in the Service implementation.


| 43

General Capabilities

GridCache data is partitioned in unique regions of the cache. Data can be serializable Java and .NET objects or byte arrays. The cache of data can be arbitrarily large, limited only by the amount of disk space or RAM on the Manager or Engines. Each cache client locally caches only the data that has been requested by users on that component. The local cache of each client, designed to speed up access to frequently used data, is in-memory with the option to overflow to disk. The size of the local cache is configurable through the Engine distribution configuration or the Driver properties file. The Broker’s cache size is limited only by the amount of disk space available.

APIAccess to the GridCache is through a client API available on Drivers and Engines. The API is available in Java, .NET, and C++, with cross-platform access to data provided where appropriate. That is, a C++ application will have access to a byte array stored by a Java application but will not have access to a Java object.

ModesGridCache operates in one of the following modes:

• Local – This mode allows a client to cache data locally by putting elements into the cache. There is no synchronization between clients that are accessing local cache regions with the same name. This is similar to having a local hashtable with LRU and eviction based on time-to live from creation time.

• Local with loader – This mode enables a client to access data from the cache, which is in turn retrieved from a pluggable loader resident on the client. A loader is a general interface that takes an object key as an argument and returns a data object. A developer must provide the implementation of the loader, which in most cases accesses a relational database or XML repository. Puts (cache writes) are not allowed in this mode.

• Global – Clients put data into the cache which is then available globally. Full automatic synchronization occurs in this mode. All components have access to a synchronized view of all entries.

• Global with loader – This mode is similar to Local with loader, except the loader is located on the Grid Manager. Full automatic synchronization occurs in this mode.


44 |

Cache Configuration and AccessCache regions are configured through the GridServer Administration Tool. Region names or regular expressions are defined with a set of attributes. A Cache access method provides access to the region if it already exists or creates the region with the mapped attributes.

Data StorageBy default, all data put to a global cache is stored in a persistent backend datastore on the Broker’s file system. The Broker’s file cache can be size limited, although the default is no limit.

AttributesAttributes are defined in schemas and applied to newly created regions. The following types of attributed can be defined:

• TimeToLive — Data that has been in the cache for longer than the time to live attribute will be evicted, and the user will be forced to reload that data from the backend datastore.

• Local/GlobalLoader — Used for loading data from a backend datastore. See Cache Loaders, below.

• KeepAlive — Specifies how long the client will keep the region and its keys in its local cache after the last reference to the region goes away.

Consistency/SynchronizationCache synchronization is performed by propagating update notifications to all clients listening on a region. These notifications occur any time the region is changed. Specifically, they occur on a put when an element already existed, clear, or remove. Engines are guaranteed to receive all update notifications by the time they take the next task. There are no synchronization guarantees for the Driver; they receive notification messages the next time they perform a request, poll the server for results, or send a heartbeat.


| 45

Cache LoadersLoaders provide an optional mechanism for loading data into the cache from a backend datastore, such as a relational database. Users can implement and associate Cache Loaders with a region of the cache. These Cache Loaders can be associated with the region locally on the specific client (Driver or Engine) or globally, but not both.

A Global CacheLoader, deployed on a Broker, is used for synchronized regions from which all clients can access data. When other clients get access to that region, they automatically are using that loader. If a get is performed and data is not found, the loader will then attempt to load for that key by calling the loader’s load method.

A Local loader, deployed on Engines and Drivers, is essentially used to cache data locally from an external database that many clients can share. Removing an item is not allowed; instead, you invalidate items, which will cause that item to be removed from other client’s caches.

NotificationGridCache provides an optional mechanism to the user whereby the user can implement a class that will listen for update notifications. An update is defined to be either an invalidation call on a loaded object or on a put call on a key that exists in the cache already. The user can then take any action desired such as updating local copies of the object or data to the new version or ignoring the update completely. The next time that the data is requested from the cache, GridCache will fetch and locally cache the most current version of that data.

Disk/Memory CachingCache puts (or writes) when the cache is full pushes the oldest element out of the cache, possibly into the backing disk cache or possibly removed entirely. Any access to a cache element that has to get the element from the disk cache brings the element into the memory cache.


46 |


| 47

Chapter 8 Inside the Client

Grid clients connect and submit requests to GridServer through a GridServer Driver. There are many different options and the intent of the flexibility is to allow rapid integration with existing systems.

The GridServer Drivers provide simple API access to Grid Services. The invocations of these Services are virtualized, meaning requests are submitted to a queue, and the Service is eventually performed by a GridServer Engine. Many requests can be submitted simultaneously with GridServer performing the requests in parallel.

Topics

• Client Types, page 48

• Installation and Configuration, page 49


48 |

Client Types

Several types of Client interfaces are available for use with different applications.

Client Interface Description

JDriver The Java Driver, also known as JDriver, consists of a JAR file used with your Java application code.

CPPDriver The C++ Driver consists of DLLs and include files that are linked with your GridServer application.

.NETDriver The .NET Driver consists of a DLL that includes a .NET Assembly used with your .NET code. The .NET Driver is only available for Windows.

COMDriver The COM Driver enables an application using the Windows Component Object Model architecture to access GridServer. The COM Driver is only available for Windows.

R Driver The R Driver enables R applications to access Services. The R Driver is available for Windows and Linux.

PDriver PDriver, or the Parametric Job Driver, is a Driver that executes command-line programs as a parallel processing service using the GridServer environment. This enables you to take a single program, run it on several Engines, and return the results to a central location, without writing a program in Java or C++.


| 49

Installation and Configuration

Drivers are distributed in the GridServer SDK The software development kit includes Drivers, API documentation, and code examples. The SDK is downloaded from the GridServer Administration Tool.

Drivers use a local file called driver.properties to define properties used by the Driver at startup, such as the names of the Directors it uses, and what log files are generated. You must also specify the Driver’s username and password, must be a valid GridServer user. (Kerberos can also be used instead of including a password.) The user’s roles may limit what Brokers the Driver is allowed to be routed to.

For more information on Driver Installation, see the GridServer Developer’s Guide.


50 |


| 51

Chapter 9 Inside the GridServer Batch Processing Facility

GridServer comes with a built-in batch scheduling facility that makes it easy to schedule Grid workflows from a Web browser. Services can be scheduled from this facility, as well as Grid administrative functions. For example, a user could schedule all the Engines to reconfigure each night at a specific time. Another example would be to re-assign Engines to Brokers throughout the day. The most common use, however, is scheduling fixed Grid workloads at specific times throughout the day or week.

Batch events and commands are extensible through a Java API. A programmer can create arbitrary events that trigger processing, such as a database event or messaging event from an external system. GridServer comes with a FileEvent that watches a file to see if it's been modified. Once modified, it will continue processing the batch plan.

An interface is defined for commands that allow the developer to perform any action in Java. This action can then be included in a defined workflow. GridServer comes with commands for emailing notification, as well as running services defined in the Service Runner Registry.

Topics

• Parts of a Batch, page 52


52 |

Parts of a Batch

A Batch Definition contains instructions in the form of components that define both the schedule and the components that will be executed. When the Batch Definition is scheduled on the Manager, it creates a Batch Entry, which typically waits until its scheduled time, then creates a Batch Execution. The Batch Execution then creates Service Sessions, PDriver Jobs, or other Grid workloads that are executed on the Manager.

Figure 11 A Batch Definition contains Batch Components, each of which define what the Batch Definition does. When a Batch Definition is scheduled, it creates a Batch Entry, which is an instantiated Batch Definition, and will run as defined by the Batch Components. When it runs, it creates a Batch Execution, which then executes the components according to the definition.


| 53

Batch DefinitionsBatch Definitions are created using the GridServer Administration Tool, with a GUI-based Batch Editor. The Batch Editor contains a list of Batch Definitions resident on the Manager, and enables you to edit, delete, rename, or schedule any of these Batch Definitions. When you select a Batch Definition to be edited, a window displaying all of the Batch Components, their properties, and the order in which they are executed is opened, as shown to the right.

Batch ComponentsBy default, a Batch Definition contains a Batch component, and that contains a Schedule component. Each component contains parameters, and values are entered in text boxes or are changed with list controls. The editor provides built-in help to explain components, parameters, and their function.

The Batch Editor page enables you to change the order that Batch components will be executed, and add or remove components from a Batch Definition.

Batch components on the Batch Editor page can define when Batches will start, run Services, change Engine weights, send email messages, wait for an event to occur, test conditionals, and much more.

Figure 12 The Batch Editor


54 |

Scheduling Batch Definitions

The Batch Registry page lists all of the Batch Definitions on the Manager. You can schedule the Batch Definition, creating a Batch Entry, and open the Batch Schedule page.


| 55

Administering Batches

Batch Entries on a Manager are listed and administered on the Batch Admin page. All Batch Entries resident on the Manager are listed. You can suspend, resume, remove, or edit an existing Batch Entry.

For more information on the Batch Facility, see the GridServer Administrator’s Guide.


56 |


introducing tibco gridserver?? · 2014. 11. 14. · † gridserver service-oriented integration...

Documents