iriondq | requirements

Installation Requirements | IrionDQ

IrionDQ | Requirements IrionDQ 2016 - Installation Requirements

IrionDQ | Installation Requirements


1. AIM OF THIS DOCUMENT ................................................................................................................................... 4

2. WHAT IS IRIONDQ .............................................................................................................................................. 4

3. ARCHITECTURE ................................................................................................................................................... 6

3.1 SOFTWARE ............................................................................................................................................................ 6

3.2 HARDWARE ........................................................................................................................................................... 6

3.3 COMMUNICATION BETWEEN THE INTERNAL COMPONENTS ............................................................................................. 7

3.4 CONNECTIONS WITH EXTERNAL DATA SOURCE .............................................................................................................. 7

3.5 CONFIGURATION .................................................................................................................................................... 7

4. ARCHITECTURAL COMPONENTS ......................................................................................................................... 9

4.1 CLIENT SOFTWARE .................................................................................................................................................. 9

Workflow Service ....................................................................................................................................................... 9

Catalog Controller Service .......................................................................................................................................... 9

Web Api Service........................................................................................................................................................ 10

Catalog ..................................................................................................................................................................... 10

4.2 HARDWARE ......................................................................................................................................................... 10

Client ........................................................................................................................................................................ 10

Workflow Server ....................................................................................................................................................... 11

Catalog Controller Server ......................................................................................................................................... 11

WebApi Server .......................................................................................................................................................... 11

Catalog Server .......................................................................................................................................................... 11

5. INSTALLATION ...................................................................................................................................................12

5.1 INSTALLATION COMPLETION ................................................................................................................................... 12

5.2 SETUP MAINTENANCE MODE ................................................................................................................................. 12

5.3 INSTALLATION BY SCRIPTING ................................................................................................................................... 12

5.4 THE ADMINISTRATION COMMANDS ......................................................................................................................... 12

6. HW AND SW REQUIREMENTS ............................................................................................................................14

6.1 HARDWARE REQUIREMENTS ................................................................................................................................... 14

6.2 SOFTWARE REQUIREMENTS .................................................................................................................................... 15

6.3 SECURITY AND PERMISSIONS .................................................................................................................................. 16

6.4 GESTIONE FIREWALL ............................................................................................................................................. 16

7. LEGAL INFORMATION ........................................................................................................................................17


1. Document Aim

This document describes application architecture of IrionDQ, its components and the requirements needed for installing and managing the platform.

This document does not describe how to use the framework, or how to implement specific projects and solutions using IrionDQ.

Most of the information is explained from a high-level perspective and does not refer to technical requirements of a specific IrionDQ solution. For example, the ODBC drivers to install on clients or servers, the estimated volumes of the database occupation or the configuration of possibly existing third-parties components.

Hardware sizing is also described, but must be reviewed and adapted to the specific project, data volumes, and number of users or operating usage modes.

2. What is IrionDQ

IrionDQ is an innovative platform for developing data management solutions, strongly oriented to solve

specific business problems.

With an all in one approach, IrionDQ addresses all the typical data processing phases in an integrated way: from

data source connecting, to controlling, transforming and reporting phases.

In this context, IrionDQ has distinctive characteristics that make data management simpler and highly business

oriented.

IrionDQ provides, in a uniformed and integrated way, the following set of features for solution development:

Access to heterogeneous data sources

Information Standardization

Transformation and enrichment

Control

Data cleaning and integration/correction

Analysis

Consolidation

Export / Archiving

Reporting & Dashboards

Publishing

IrionDQ allows integrating and working data, regardless of where it comes from, as if it were in a single virtual

database; defining data mats, data warehouses, etc is not required.

IrionDQ does not keep data copies, except for temporary caches or dedicated modules for persistent data

storage. Each processing can import data from the original source, and editing tables or traced upstream

applications do not impact IrionDQ structures, instead they adapt automatically, after upgrading the

configuration of the solution.


All processing and controls within IrionDQ are automatically documented in a natural language so that the

used data and rules are traceable for auditing and reconstructing processes.

IrionDQ is a Framework within which rule systems, called Packages, are implemented for modelling business

needs.

IrionDQ also allows for defining Workflows, meaning it is possible to organize commands and processing into

automated and schedulable workflows. Execution can also be distributed on different machines and with

different accounts.

In addition, the Long Term Storage feature allows creating long-term storage repositories (DataShelves and

AuditShelves), within the solution itself, rather than on external Database servers, creating closer integration with

Packages.


3. Architecture

3.1 Software IrionDQ is developed with Microsoft .NET technology and operates in a Windows environment. Its architecture

is organized into different layers that include:

A Windows Client User Interface component for interactive use, solution developing and system administration.

A Workflow component, consisting of services for managing batches and scheduled executions.

A data web publishing system (Web Api Service).

A shared repository (on Microsoft SQL Server) that is used to handle temporary data and caches during processing and to store Package definitions, log information and Long Term Storage and Audit data.

The windows client User Interface component has a Client/Server architecture and, therefore, direct access to the DB Server.

3.2 Hardware The abovementioned software components must be hosted on one or more machines.

The choice and sizing for hardware and its architecture directly depends on the expected load and

requirements for performance, efficiency, reliability, availability and security in the environment of the specific

solution implemented.

Port: 8982

Catalog Server

(Database)

Client IrionDQ

RDP/ICA/

Port: 8981

Port: 8980

Port: 8980

Port: 8980

Web Api Server (1..n)

Remote Delivery

Server (e.g. Citrix)

Third party Web App

Client RDP

Catalog Controller Server

Workflow Server (1..n)

Figure 1 - IrionDQ 2016 Architecture


3.3 Communication between internal components All communications between system components are done with TCP/IP connections. The addresses and ports

to be used can be configured during installation and are editable through administration procedures.

The accesses to the Catalog Server use MS SQL-Server connections. These use, by default, the TCP port 1433

(and/or1434) as described in Microsoft documentation, and can be modified.

Communication with other IrionDQ processes (Controller, Workflow Agent, Web Api) is done through custom

protocols on TCP ports, defined during installation. In the above schema the default values of the ports are

displayed; if needed, they can be modified through administration procedures.

3.4 Connections with external data sources IrionDQ can access several external data sources, through a set of connectors based on standardized protocols

Third-party components, for example, ODBC, must be installed and configured on each machine that will access

the data sources, in particular on:

Clients

Remote Delivery Servers (in case of a virtualized Client)

Workflow Servers

WebApi Servers.

3.5 Configuration The IrionDQ architecture components can reside on the same machine or on different machines (DB Server,

Workflow Server, Web Api Server, Client) depending on the architectural choice, company policies, space and

elaborative sizing considerations.

For details regarding installation, please refer to the dedicated documentation.

The IrionDQ Client component has a WinForm based interface. Multi-user operating environments are

supported, and centralize and extend deployment and upgrade management (e.g. Citrix or Microsoft Terminal

Server).


DataBase Server:- IrionDQ Catalog

Remote Delivery Server (e.g. Citrix)

Client RDP

Client IrionDQ

Application Server:- Workflow Service- Catalog Controller Service- Web Api Service

Server

Figure 2 Example of a typical installation


4. Architectural Components

A short description of the architectural components:

4.1 Client Software The IrionDQ.UI.exe application interface is designed for using the interactive features. It can be recalled from the menu start->All programs->Irion ->IrionDQ.

It is a WinForm/WPF mixed technology application that allows creating, maintaining and using IrionDQ solutions.

The application is installed at Setup.

Workflow Service

The role of this component is to execute Workflows at the request of the Catalog Controller service.

The service is installed with auto-start and a user.

If, due to company policies, the service must be configured with an account other than a one, it must have at least the privileges listed in the dedicated documentation (Grants and privileges).

For each Workflow start request, the service instantiates a separate process, with the ownership of the user associated to the Agent or to the specific workflow, if differently configured.

The architecture can consider multiple Workflow services hosted on a dedicated Server, to balance loading in case of heavy operations.

If not necessary (i.e. when using IrionDQ is exclusively interactive), the installation and the configuration of this component can be omitted .

The service is installed at Setup.

Catalog Controller Service

The roles of this component, which require minimal resource usage, are:

Cyclical and automatic maintenance of the Catalogs.

Scheduled Workflows activation.

Monitoring and notification management for Workflows in execution.

The service is installed with auto-start and a user.

The Controller service is associated to one or more Catalogs within the Catalog Server.

Each catalog must be associated to a controller.

It is possible to associate the Controller service to one or more Catalogs during Setup.

In terms of network visibility, the Controller Server must access:

The Catalogs for which it is responsible to perform maintenance.

All the monitored Workflow Servers.

The Client can also interact with the Controller Server to request the execution and monitor the Workflows.



Web Api Service

The role of this component, at the heart of the SOA architecture, is to provide IrionDQ system data and

elaboration functions through a HTTP / OData v4 protocol.

To carry out this role, the service must be able to communicate with the IrionDQ Catalog.


Catalog

An IrionDQ Catalog is a physical set of MS SQL Server Databases. Setup creates (or updates if already existent)

IrionDQ component installations on the DB Server. It is also possible to create or update the database by using

the dedicated administrative cmdlets.

On each DB Server it is possible to define one or more Catalogs.

In order to create a new Catalog a username and password are required with the appropriate permissions for

creating a new SQL user and Databases.

The username and relative password are never stored, but used only during the Catalog creation.

The following are created on the DB Server:

an encrypted password. As stated in the requirements,

be edited by a dedicated command at the end of the installation. For further information refer to the

dedicated documentation).

The databases for operating IrionDQ.

The necessary DBs are created for each Catalog that insists on the server.

The Catalog is identified by a suffix that identifies the database name (when no suffix is specified, it is named

Default Catalog).

For the DB backup and maintenance policies, refer to the product documentation.

In order to avoid incoherence, the permissions on the created IrionDQ DBs must not be added or removed

manually.

db

owner group.

4.2 Hardware

Client

The Client is the workstation on which the executable IrionDQ.Ui.exe has been installed. It provides access to

the IrionDQ interface; if requested, the Client can be virtualized by RDP or Citrix systems, in order to centralize

and simplify the distribution of the service. In this case, it is called a distributed Client.


Workflow Server

The Workflow Server is installed and enabled. It corresponds to the executable IrionDQ.WorkflowAgent.exe.

In case of heavy tasks, to balance the workload it is possible to install and configure more machines with the Workflow Server role, whose jobs are coordinated by the Catalog Controller Service.

Catalog Controller Server

The Catalog Controller Server enabled. It corresponds to the executable IrionDQ.ControllerService.exe.

Because the service has low resource consumption, it is not normally required to use an exclusive machine. One Workflow Server on which the Catalog Controller service is installed and configured can perform it.

WebApi Server

The WebApi Server is the server on which the Web Api Windows services is installed and enabled. It corresponds to the executable IrionDQ.WebApiServer.exe.

The architecture can consider the installation of more than one Web Api Server machine in parallel, increasing

system power.

Catalog Server

The Catalog Server is the DB Server on which the IrionDQ Catalog resides. On this machine, MS SQL-Server is

installed, with specific maintenance and backup policies.


5. Installation

IrionDQ must be installed with a certain Setup procedure.

During the installation, it is possible to choose to remove (or not) pre-existing installations.

The 2016 Release can coexist with previous versions of IrionDQ.

After the installation, the Setup proceeds with configuration steps in which it is possible to define/create the

referred Catalog and activate the services.

5.1 Installation Completion After completing the Setup, the license must be activated.

The license key resides on the Catalog: no Client activation is therefore required.

A dedicated administrative interface allows managing licenses and users.

5.2 Setup Maintenance Mode It is possible to access setup maintenance mode by re-launching the same version of the Setup, or by selecting either the button

In this mode, it is possible to remove the product, repair the installation if any components are damaged, or modify the optional components list by adding or removing items from it.

Service editing is allowed through the dedicated administrative cmdlets.

5.3 Installation by scripting In order to satisfy very specific needs, for example, a wide scale application deploy (through Microsoft SMS or

similar), the details of the files to copy, the COM components registers, the configuration files and the database

scripts are available for each specific release.

The upgrading script subset to execute depends on the original database version. The setup and the Admin

commands make this selection automatically.

5.4 The Administration Commands The system is administrated through the use of Powershell cmdlets and can execute Catalog management tasks. The main features are:

Installation administrative tasks

o Catalog database Installation/Upgrade

o Service configuration

o Licenses and privileges management

Solution deploy tasks

o Catalog objects Export and Import

o Configuration editing

o Permissions management


With these commands, integration with pre-existing change management, deployment processes, backup, disaster recovery systems and so on is very easy.


6. HW and SW Requirements

This section explains the hardware and software requirements for IrionDQ components of a typical installation;

the flexibility of the system allows a stand-alone installation and scalable and distributed structures, which can

handle big data elaborations. The requirements can change according to the specific project/solution and to

the adopted architectural pattern.

If the chosen configuration places more than one component on the same machine, the best practice is to at

least, size the component with larger requirements.

6.1 Hardware Requirements Catalog Server (Database):

The Catalog Server resides, normally, on a server different than the IrionDQ installation server. In this case the Server requirements must be considered in relation to the Catalog Server, while the machine that hosts the related IrionDQ installation must be sized according to the task stress expected for the processes that will be executed (including the scheduled Workflow).

The hardware requirements for the catalog server (database) are:

Quad-core processors, according to the server segment market standards;

8 GB RAM;

Disk space sufficient to contain the data in function of the usage type;

IrionDQ intensively uses a SQL Server engine; therefore, this hardware must be also configured. Special attention should be paid to the number of disks, the access velocity, the throughput, and so on.

Considering the stress intensity that IrionDQ delegates to the Database, it is suggested to use a dedicated DB Server.

Catalog Controller Server Requirements

A Dual-core processor according to the market standard for the server segment;

At least 4GB RAM;

About 300 MB free disk space for the application.

Since the resource usage is low, the service can coexist with any other application and is normally installed on the Workflow Server.

Workflow Server Requirements


At least 4GB RAM;


Web Api Server Requirements


At least 4GB RAM;



Client Requirements

A Dual-core processor according to the market standard for the desktop segment;

At least 4GB RAM;


Monitor with 1024 x 768 dpi minimal resolution (at least 1200 x 800 dpi recommended).

Client on application server (Citrix or TS) Requirements

For a typical usage of 4-6 IrionDQ clients per server, the minimal configuration must include:

A Quad-core processor according to the market standard for the server segment;

At least 2GB RAM for each connected client;


6.2 Software Requirements Operating System:

One of the following operating system versions (or more recent):

Windows Server 2008 (64 bit)

Windows Server 2012 (64 bit)

Windows 7 (64 bit)

Windows 8 (64 bit)

Windows 8.1 (64 bit)

Windows 10 (64 bit)

All the 32 bit versions are also admitted but deprecated.

Database:

Servers on which the Catalog instances reside require a Microsoft SQL Server Standard Edition installation (or a superior one).

Microsoft SQL Server 2008 R2 / 2012 / 2014 Standard version, Business intelligence or Enterprise.

Beginning with the Update 6, IrionDQ 2016 supports also SQL Server 2016.

case insensitive ; it is recommended collation

when installing the Db server.

Database Driver:

IrionDQ integrates the connectivity to MS-SQL Server and Oracle. The system administrators carry out the

installation of other connection protocols. ADO.Net drivers are always preferable where available.

Other Requirements:

IrionDQ requires the Microsoft.NET framework, version 4.6.2.

For compatibility with the previous versions, version 4.6.1 is also supported, but has lower performance results. The version to install can be downloaded from: https://www.microsoft.com/it-it/download/details.aspx?id=53344

Microsoft Excel version XP or a more recent one (2010 or 2013 is recommended). If the Packages to develop do not use an integration with Excel, this requirement can be omitted.

https://www.microsoft.com/it-it/download/details.aspx?id=53344

https://www.microsoft.com/it-it/download/details.aspx?id=53344


The Bloomberg application, inclusive of the API, is mandatory in order to use the relative module.

Microsoft Powershell 4.0.

In case of a remote environment installation (e.g. Citrix, Terminal Server) check possibly existing patches/quick-fixes.

6.3 Security and Permissions During installation and updates, the Setup requires administrative privileges.

SQL Server must be set with exclusive Windows authentication.

In some cases, the Windows DEP (Data Execution Prevention) locks the execution of the executable IrionDQ.FileNormalizerRunner.exe; it is suggested to add it to the check exclusion list.

Note:

To operate, IrionDQ requires that the properties set during installation remain unchanged. In particular:

The SQL login is created with Owner permission (DBOwner) on the Catalog database. This user is indispensable, and it is not sufficient for it to only belong to the DBO group.

The user name can be eventually renamed by an administrative procedure.

Data transfer among the DBs is best performed by the application itself. Thus, during the installation it is necessary to enable and not subsequently modify the option, as well as the user.

IrionDQ uses the SQL Server broker functionalities, enabled for some of its databases during the installation.

6.4 Firewall configuration When the architectural choice consists of different instances for the Client, Controller, Workflow and WebAPI, a

firewall configuration is mandatory.

The standard configuration is:

The Workflow service stays listening on port 8981.

The Catalog Controller stays listening on port 8980.

The WebApi Service can be accessed through port 8982.

For each of the three services it is possible to change the default ports by running the administration procedure.

Irion supports the SQL Server Availability System in Always On Failover Cluster Instances mode.


7. Legal Information

This document is distributed with the software that includes an end user agreement. As well as the software

described, it is licensed and can be used and copied, only in accordance with the terms of such license.

Excepting what may be permitted by such license, no part of this document can be reproduced, stored in a

retrieval system or transmitted in any form or by any means, electronic, mechanical, recording, or other, without

the prior written consent of Irion. Copyright laws, though not distributed with software supplied with the end

user license agreement, protect the content of this document. The contents of this document are provided for

informational purposes only, and it is supposed to change without notice and does not imply any commitment

to Irion. Irion disclaims any responsibility for any errors or inaccuracies that may appear in this document.

Irion, IrionDQ and IDQ are registered marks or trademarks by Irion in Italy and/or in other countries.

Microsoft, MS, Windows, Windows 7, Windows 8, Windows 8.1, Windows 10, Windows Server 2008, Windows

Server 2012 e PowerShell, are registered marks or trademarks by Microsoft Corporation in the United States

and/or in other countries. The respective owners could recognize all the other mentioned product names as

marks.

www.iriondq.com

©2016 irion srl. All rights reserved. Irion® and IrionDQ® are trademarks of Irion and may be registered in the U.S. Patent and Trademark Office and in other countries

Irion via Livorno, 60 10144, Torino Italy [email protected]

iriondq | requirements

Documents