Download - IrionDQ | Requirements
Installation Requirements | IrionDQ
Page 1
IrionDQ | Requirements IrionDQ 2016 - Installation Requirements
IrionDQ | Installation Requirements
Page 2
Installation Requirements | IrionDQ
Page 3
1. AIM OF THIS DOCUMENT ................................................................................................................................... 4
2. WHAT IS IRIONDQ .............................................................................................................................................. 4
3. ARCHITECTURE ................................................................................................................................................... 6
3.1 SOFTWARE ............................................................................................................................................................ 6
3.2 HARDWARE ........................................................................................................................................................... 6
3.3 COMMUNICATION BETWEEN THE INTERNAL COMPONENTS ............................................................................................. 7
3.4 CONNECTIONS WITH EXTERNAL DATA SOURCE .............................................................................................................. 7
3.5 CONFIGURATION .................................................................................................................................................... 7
4. ARCHITECTURAL COMPONENTS ......................................................................................................................... 9
4.1 CLIENT SOFTWARE .................................................................................................................................................. 9
Workflow Service ....................................................................................................................................................... 9
Catalog Controller Service .......................................................................................................................................... 9
Web Api Service........................................................................................................................................................ 10
Catalog ..................................................................................................................................................................... 10
4.2 HARDWARE ......................................................................................................................................................... 10
Client ........................................................................................................................................................................ 10
Workflow Server ....................................................................................................................................................... 11
Catalog Controller Server ......................................................................................................................................... 11
WebApi Server .......................................................................................................................................................... 11
Catalog Server .......................................................................................................................................................... 11
5. INSTALLATION ...................................................................................................................................................12
5.1 INSTALLATION COMPLETION ................................................................................................................................... 12
5.2 SETUP MAINTENANCE MODE ................................................................................................................................. 12
5.3 INSTALLATION BY SCRIPTING ................................................................................................................................... 12
5.4 THE ADMINISTRATION COMMANDS ......................................................................................................................... 12
6. HW AND SW REQUIREMENTS ............................................................................................................................14
6.1 HARDWARE REQUIREMENTS ................................................................................................................................... 14
6.2 SOFTWARE REQUIREMENTS .................................................................................................................................... 15
6.3 SECURITY AND PERMISSIONS .................................................................................................................................. 16
6.4 GESTIONE FIREWALL ............................................................................................................................................. 16
7. LEGAL INFORMATION ........................................................................................................................................17
IrionDQ | Installation Requirements
Page 4
1. Document Aim
This document describes application architecture of IrionDQ, its components and the requirements needed for installing and managing the platform.
This document does not describe how to use the framework, or how to implement specific projects and solutions using IrionDQ.
Most of the information is explained from a high-level perspective and does not refer to technical requirements of a specific IrionDQ solution. For example, the ODBC drivers to install on clients or servers, the estimated volumes of the database occupation or the configuration of possibly existing third-parties components.
Hardware sizing is also described, but must be reviewed and adapted to the specific project, data volumes, and number of users or operating usage modes.
2. What is IrionDQ
IrionDQ is an innovative platform for developing data management solutions, strongly oriented to solve
specific business problems.
With an all in one approach, IrionDQ addresses all the typical data processing phases in an integrated way: from
data source connecting, to controlling, transforming and reporting phases.
In this context, IrionDQ has distinctive characteristics that make data management simpler and highly business
oriented.
IrionDQ provides, in a uniformed and integrated way, the following set of features for solution development:
Access to heterogeneous data sources
Information Standardization
Transformation and enrichment
Control
Data cleaning and integration/correction
Analysis
Consolidation
Export / Archiving
Reporting & Dashboards
Publishing
IrionDQ allows integrating and working data, regardless of where it comes from, as if it were in a single virtual
database; defining data mats, data warehouses, etc is not required.
IrionDQ does not keep data copies, except for temporary caches or dedicated modules for persistent data
storage. Each processing can import data from the original source, and editing tables or traced upstream
applications do not impact IrionDQ structures, instead they adapt automatically, after upgrading the
configuration of the solution.
Installation Requirements | IrionDQ
Page 5
All processing and controls within IrionDQ are automatically documented in a natural language so that the
used data and rules are traceable for auditing and reconstructing processes.
IrionDQ is a Framework within which rule systems, called Packages, are implemented for modelling business
needs.
IrionDQ also allows for defining Workflows, meaning it is possible to organize commands and processing into
automated and schedulable workflows. Execution can also be distributed on different machines and with
different accounts.
In addition, the Long Term Storage feature allows creating long-term storage repositories (DataShelves and
AuditShelves), within the solution itself, rather than on external Database servers, creating closer integration with
Packages.
IrionDQ | Installation Requirements
Page 6
3. Architecture
3.1 Software IrionDQ is developed with Microsoft .NET technology and operates in a Windows environment. Its architecture
is organized into different layers that include:
A Windows Client User Interface component for interactive use, solution developing and system administration.
A Workflow component, consisting of services for managing batches and scheduled executions.
A data web publishing system (Web Api Service).
A shared repository (on Microsoft SQL Server) that is used to handle temporary data and caches during processing and to store Package definitions, log information and Long Term Storage and Audit data.
The windows client User Interface component has a Client/Server architecture and, therefore, direct access to the DB Server.
3.2 Hardware The abovementioned software components must be hosted on one or more machines.
The choice and sizing for hardware and its architecture directly depends on the expected load and
requirements for performance, efficiency, reliability, availability and security in the environment of the specific
solution implemented.
Port: 8982
Catalog Server
(Database)
Client IrionDQ
RDP/ICA/
Port: 8981
Port: 8980
Port: 8980
Port: 8980
Web Api Server (1..n)
Remote Delivery
Server (e.g. Citrix)
Third party Web App
Client RDP
Catalog Controller Server
Workflow Server (1..n)
Figure 1 - IrionDQ 2016 Architecture
Installation Requirements | IrionDQ
Page 7
3.3 Communication between internal components All communications between system components are done with TCP/IP connections. The addresses and ports
to be used can be configured during installation and are editable through administration procedures.
The accesses to the Catalog Server use MS SQL-Server connections. These use, by default, the TCP port 1433
(and/or1434) as described in Microsoft documentation, and can be modified.
Communication with other IrionDQ processes (Controller, Workflow Agent, Web Api) is done through custom
protocols on TCP ports, defined during installation. In the above schema the default values of the ports are
displayed; if needed, they can be modified through administration procedures.
3.4 Connections with external data sources IrionDQ can access several external data sources, through a set of connectors based on standardized protocols
Third-party components, for example, ODBC, must be installed and configured on each machine that will access
the data sources, in particular on:
Clients
Remote Delivery Servers (in case of a virtualized Client)
Workflow Servers
WebApi Servers.
3.5 Configuration The IrionDQ architecture components can reside on the same machine or on different machines (DB Server,
Workflow Server, Web Api Server, Client) depending on the architectural choice, company policies, space and
elaborative sizing considerations.
For details regarding installation, please refer to the dedicated documentation.
The IrionDQ Client component has a WinForm based interface. Multi-user operating environments are
supported, and centralize and extend deployment and upgrade management (e.g. Citrix or Microsoft Terminal
Server).
IrionDQ | Installation Requirements
Page 8
DataBase Server:- IrionDQ Catalog
Remote Delivery Server (e.g. Citrix)
Client RDP
Client IrionDQ
Application Server:- Workflow Service- Catalog Controller Service- Web Api Service
Server
Figure 2 Example of a typical installation
Installation Requirements | IrionDQ
Page 9
4. Architectural Components
A short description of the architectural components:
4.1 Client Software The IrionDQ.UI.exe application interface is designed for using the interactive features. It can be recalled from the menu start->All programs->Irion ->IrionDQ.
It is a WinForm/WPF mixed technology application that allows creating, maintaining and using IrionDQ solutions.
The application is installed at Setup.
Workflow Service
The role of this component is to execute Workflows at the request of the Catalog Controller service.
The service is installed with auto-start and a user.
If, due to company policies, the service must be configured with an account other than a one, it must have at least the privileges listed in the dedicated documentation (Grants and privileges).
For each Workflow start request, the service instantiates a separate process, with the ownership of the user associated to the Agent or to the specific workflow, if differently configured.
The architecture can consider multiple Workflow services hosted on a dedicated Server, to balance loading in case of heavy operations.
If not necessary (i.e. when using IrionDQ is exclusively interactive), the installation and the configuration of this component can be omitted .
The service is installed at Setup.
Catalog Controller Service
The roles of this component, which require minimal resource usage, are:
Cyclical and automatic maintenance of the Catalogs.
Scheduled Workflows activation.
Monitoring and notification management for Workflows in execution.
The service is installed with auto-start and a user.
The Controller service is associated to one or more Catalogs within the Catalog Server.
Each catalog must be associated to a controller.
It is possible to associate the Controller service to one or more Catalogs during Setup.
In terms of network visibility, the Controller Server must access:
The Catalogs for which it is responsible to perform maintenance.
All the monitored Workflow Servers.
The Client can also interact with the Controller Server to request the execution and monitor the Workflows.
The service is installed at Setup.
IrionDQ | Installation Requirements
Page 10
Web Api Service
The role of this component, at the heart of the SOA architecture, is to provide IrionDQ system data and
elaboration functions through a HTTP / OData v4 protocol.
To carry out this role, the service must be able to communicate with the IrionDQ Catalog.
The service is installed at Setup.
Catalog
An IrionDQ Catalog is a physical set of MS SQL Server Databases. Setup creates (or updates if already existent)
IrionDQ component installations on the DB Server. It is also possible to create or update the database by using
the dedicated administrative cmdlets.
On each DB Server it is possible to define one or more Catalogs.
In order to create a new Catalog a username and password are required with the appropriate permissions for
creating a new SQL user and Databases.
The username and relative password are never stored, but used only during the Catalog creation.
The following are created on the DB Server:
an encrypted password. As stated in the requirements,
be edited by a dedicated command at the end of the installation. For further information refer to the
dedicated documentation).
The databases for operating IrionDQ.
The necessary DBs are created for each Catalog that insists on the server.
The Catalog is identified by a suffix that identifies the database name (when no suffix is specified, it is named
Default Catalog).
For the DB backup and maintenance policies, refer to the product documentation.
In order to avoid incoherence, the permissions on the created IrionDQ DBs must not be added or removed
manually.
db
owner group.
4.2 Hardware
Client
The Client is the workstation on which the executable IrionDQ.Ui.exe has been installed. It provides access to
the IrionDQ interface; if requested, the Client can be virtualized by RDP or Citrix systems, in order to centralize
and simplify the distribution of the service. In this case, it is called a distributed Client.
Installation Requirements | IrionDQ
Page 11
Workflow Server
The Workflow Server is installed and enabled. It corresponds to the executable IrionDQ.WorkflowAgent.exe.
In case of heavy tasks, to balance the workload it is possible to install and configure more machines with the Workflow Server role, whose jobs are coordinated by the Catalog Controller Service.
Catalog Controller Server
The Catalog Controller Server enabled. It corresponds to the executable IrionDQ.ControllerService.exe.
Because the service has low resource consumption, it is not normally required to use an exclusive machine. One Workflow Server on which the Catalog Controller service is installed and configured can perform it.
WebApi Server
The WebApi Server is the server on which the Web Api Windows services is installed and enabled. It corresponds to the executable IrionDQ.WebApiServer.exe.
The architecture can consider the installation of more than one Web Api Server machine in parallel, increasing
system power.
Catalog Server
The Catalog Server is the DB Server on which the IrionDQ Catalog resides. On this machine, MS SQL-Server is
installed, with specific maintenance and backup policies.
IrionDQ | Installation Requirements
Page 12
5. Installation
IrionDQ must be installed with a certain Setup procedure.
During the installation, it is possible to choose to remove (or not) pre-existing installations.
The 2016 Release can coexist with previous versions of IrionDQ.
After the installation, the Setup proceeds with configuration steps in which it is possible to define/create the
referred Catalog and activate the services.
5.1 Installation Completion After completing the Setup, the license must be activated.
The license key resides on the Catalog: no Client activation is therefore required.
A dedicated administrative interface allows managing licenses and users.
5.2 Setup Maintenance Mode It is possible to access setup maintenance mode by re-launching the same version of the Setup, or by selecting either the button
In this mode, it is possible to remove the product, repair the installation if any components are damaged, or modify the optional components list by adding or removing items from it.
Service editing is allowed through the dedicated administrative cmdlets.
5.3 Installation by scripting In order to satisfy very specific needs, for example, a wide scale application deploy (through Microsoft SMS or
similar), the details of the files to copy, the COM components registers, the configuration files and the database
scripts are available for each specific release.
The upgrading script subset to execute depends on the original database version. The setup and the Admin
commands make this selection automatically.
5.4 The Administration Commands The system is administrated through the use of Powershell cmdlets and can execute Catalog management tasks. The main features are:
Installation administrative tasks
o Catalog database Installation/Upgrade
o Service configuration
o Licenses and privileges management
Solution deploy tasks
o Catalog objects Export and Import
o Configuration editing
o Permissions management
Installation Requirements | IrionDQ
Page 13
With these commands, integration with pre-existing change management, deployment processes, backup, disaster recovery systems and so on is very easy.
IrionDQ | Installation Requirements
Page 14
6. HW and SW Requirements
This section explains the hardware and software requirements for IrionDQ components of a typical installation;
the flexibility of the system allows a stand-alone installation and scalable and distributed structures, which can
handle big data elaborations. The requirements can change according to the specific project/solution and to
the adopted architectural pattern.
If the chosen configuration places more than one component on the same machine, the best practice is to at
least, size the component with larger requirements.
6.1 Hardware Requirements Catalog Server (Database):
The Catalog Server resides, normally, on a server different than the IrionDQ installation server. In this case the Server requirements must be considered in relation to the Catalog Server, while the machine that hosts the related IrionDQ installation must be sized according to the task stress expected for the processes that will be executed (including the scheduled Workflow).
The hardware requirements for the catalog server (database) are:
Quad-core processors, according to the server segment market standards;
8 GB RAM;
Disk space sufficient to contain the data in function of the usage type;
IrionDQ intensively uses a SQL Server engine; therefore, this hardware must be also configured. Special attention should be paid to the number of disks, the access velocity, the throughput, and so on.
Considering the stress intensity that IrionDQ delegates to the Database, it is suggested to use a dedicated DB Server.
Catalog Controller Server Requirements
A Dual-core processor according to the market standard for the server segment;
At least 4GB RAM;
About 300 MB free disk space for the application.
Since the resource usage is low, the service can coexist with any other application and is normally installed on the Workflow Server.
Workflow Server Requirements
A Dual-core processor according to the market standard for the server segment;
At least 4GB RAM;
About 300 MB free disk space for the application.
Web Api Server Requirements
A Dual-core processor according to the market standard for the server segment;
At least 4GB RAM;
About 300 MB free disk space for the application.
Installation Requirements | IrionDQ
Page 15
Client Requirements
A Dual-core processor according to the market standard for the desktop segment;
At least 4GB RAM;
About 300 MB free disk space for the application.
Monitor with 1024 x 768 dpi minimal resolution (at least 1200 x 800 dpi recommended).
Client on application server (Citrix or TS) Requirements
For a typical usage of 4-6 IrionDQ clients per server, the minimal configuration must include:
A Quad-core processor according to the market standard for the server segment;
At least 2GB RAM for each connected client;
About 200 MB free disk space for the application.
6.2 Software Requirements Operating System:
One of the following operating system versions (or more recent):
Windows Server 2008 (64 bit)
Windows Server 2012 (64 bit)
Windows 7 (64 bit)
Windows 8 (64 bit)
Windows 8.1 (64 bit)
Windows 10 (64 bit)
All the 32 bit versions are also admitted but deprecated.
Database:
Servers on which the Catalog instances reside require a Microsoft SQL Server Standard Edition installation (or a superior one).
Microsoft SQL Server 2008 R2 / 2012 / 2014 Standard version, Business intelligence or Enterprise.
Beginning with the Update 6, IrionDQ 2016 supports also SQL Server 2016.
case insensitive ; it is recommended collation
when installing the Db server.
Database Driver:
IrionDQ integrates the connectivity to MS-SQL Server and Oracle. The system administrators carry out the
installation of other connection protocols. ADO.Net drivers are always preferable where available.
Other Requirements:
IrionDQ requires the Microsoft.NET framework, version 4.6.2.
For compatibility with the previous versions, version 4.6.1 is also supported, but has lower performance results. The version to install can be downloaded from: https://www.microsoft.com/it-it/download/details.aspx?id=53344
Microsoft Excel version XP or a more recent one (2010 or 2013 is recommended). If the Packages to develop do not use an integration with Excel, this requirement can be omitted.
IrionDQ | Installation Requirements
Page 16
The Bloomberg application, inclusive of the API, is mandatory in order to use the relative module.
Microsoft Powershell 4.0.
In case of a remote environment installation (e.g. Citrix, Terminal Server) check possibly existing patches/quick-fixes.
6.3 Security and Permissions During installation and updates, the Setup requires administrative privileges.
SQL Server must be set with exclusive Windows authentication.
In some cases, the Windows DEP (Data Execution Prevention) locks the execution of the executable IrionDQ.FileNormalizerRunner.exe; it is suggested to add it to the check exclusion list.
Note:
To operate, IrionDQ requires that the properties set during installation remain unchanged. In particular:
The SQL login is created with Owner permission (DBOwner) on the Catalog database. This user is indispensable, and it is not sufficient for it to only belong to the DBO group.
The user name can be eventually renamed by an administrative procedure.
Data transfer among the DBs is best performed by the application itself. Thus, during the installation it is necessary to enable and not subsequently modify the option, as well as the user.
IrionDQ uses the SQL Server broker functionalities, enabled for some of its databases during the installation.
6.4 Firewall configuration When the architectural choice consists of different instances for the Client, Controller, Workflow and WebAPI, a
firewall configuration is mandatory.
The standard configuration is:
The Workflow service stays listening on port 8981.
The Catalog Controller stays listening on port 8980.
The WebApi Service can be accessed through port 8982.
For each of the three services it is possible to change the default ports by running the administration procedure.
Irion supports the SQL Server Availability System in Always On Failover Cluster Instances mode.
Installation Requirements | IrionDQ
Page 17
7. Legal Information
This document is distributed with the software that includes an end user agreement. As well as the software
described, it is licensed and can be used and copied, only in accordance with the terms of such license.
Excepting what may be permitted by such license, no part of this document can be reproduced, stored in a
retrieval system or transmitted in any form or by any means, electronic, mechanical, recording, or other, without
the prior written consent of Irion. Copyright laws, though not distributed with software supplied with the end
user license agreement, protect the content of this document. The contents of this document are provided for
informational purposes only, and it is supposed to change without notice and does not imply any commitment
to Irion. Irion disclaims any responsibility for any errors or inaccuracies that may appear in this document.
Irion, IrionDQ and IDQ are registered marks or trademarks by Irion in Italy and/or in other countries.
Microsoft, MS, Windows, Windows 7, Windows 8, Windows 8.1, Windows 10, Windows Server 2008, Windows
Server 2012 e PowerShell, are registered marks or trademarks by Microsoft Corporation in the United States
and/or in other countries. The respective owners could recognize all the other mentioned product names as
marks.
www.iriondq.com
©2016 irion srl. All rights reserved. Irion® and IrionDQ® are trademarks of Irion and may be registered in the U.S. Patent and Trademark Office and in other countries
Irion via Livorno, 60 10144, Torino Italy [email protected]