elastic stack solution on dell emc vxflex family...000060 white paper elastic stack solution on dell...

000060

White Paper

Elastic Stack Solution on Dell EMC VxFlex family

Abstract This white paper highlights the deployment and best practices of Elastic Stack on the Dell EMC VxFlex family.

March 2020

Revisions

2 Elastic Stack Solution on Dell EMC VxFlex family | 000060

Revisions Date Description

March 2020 Initial release

Acknowledgements This paper was produced by the following:

Author: Kailas Goliwadekar

Support: Keith Quebodeaux and Shashikiran Chidambara

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright © 2020 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be trademarks of their respective owners. [3/16/2020] [White Paper] [000060]

Table of contents


Table of contents Revisions............................................................................................................................................................................. 2

Acknowledgements ............................................................................................................................................................. 2

Table of contents ................................................................................................................................................................ 3

Executive summary ............................................................................................................................................................. 5

1 Introduction ................................................................................................................................................................... 6

1.1 Objective ............................................................................................................................................................. 6

1.2 Audience ............................................................................................................................................................. 6

1.3 Terminology ........................................................................................................................................................ 7

2 Product overview .......................................................................................................................................................... 8

2.1 VxFlex family ...................................................................................................................................................... 8

2.1.1 VxFlex integrated rack................................................................................................................................ 8

2.1.2 VxFlex appliance ........................................................................................................................................ 9

2.1.3 VxFlex Ready Nodes.................................................................................................................................. 9

2.1.4 VxFlex OS .................................................................................................................................................. 9

2.1.5 VxFlex Manager ....................................................................................................................................... 10

2.1.6 VxFlex OS management .......................................................................................................................... 10

2.2 Elastic Stack ..................................................................................................................................................... 11

3 Elastic Stack architecture overview ............................................................................................................................ 12

3.1 Elasticsearch .................................................................................................................................................... 12

3.2 Logstash ........................................................................................................................................................... 13

3.3 Kibana ............................................................................................................................................................... 14

3.4 Beats ................................................................................................................................................................. 14

4 Solution architecture ................................................................................................................................................... 15

4.1 Logical architecture .......................................................................................................................................... 15

4.2 Network architecture ......................................................................................................................................... 16

5 Testing and Validation ................................................................................................................................................ 17

5.1 Test methodology ............................................................................................................................................. 17

5.2 Test results ....................................................................................................................................................... 18

6 Best practices ............................................................................................................................................................. 20

6.1 VxFlex integrated rack ...................................................................................................................................... 20

6.2 VxFlex OS ......................................................................................................................................................... 20

6.3 VxFlex network ................................................................................................................................................. 20

6.4 Elasticsearch .................................................................................................................................................... 20

7 Deployment of Elastic Stack on VxFlex ..................................................................................................................... 21

Table of contents


8 Conclusion .................................................................................................................................................................. 22

A Appendix A: Detailed results ...................................................................................................................................... 23

A.1 Use case1: EventData track ............................................................................................................................. 23

A.2 Use case1: Http_logs track ............................................................................................................................... 25

B Appendix B: Configuration details .............................................................................................................................. 28

C Technical support and resources ............................................................................................................................... 29

C.1 Related resources............................................................................................................................................. 29

C.2 Additional resources ......................................................................................................................................... 29

Executive summary


Executive summary Teams responsible for managing large-scale and rapidly growing data center infrastructure constantly meet challenges in terms of scalability and flexibility needs. Enterprises are transitioning from the traditional data center to a hyperconverged infrastructure (HCI) approach to address these challenges. HCI based offerings give enterprises the ability to support modern workload and provides a simplified solution to manage their infrastructure needs. Organizations are switching to public cloud security and log analytics to provide uninterrupted operations with minimum downtime and SLAs with huge amount of data being generated daily. Since many organizations are switching to cloud, the necessity for public cloud security tools and log analytics platforms is increasing rapidly.

A substantial amount of this data consists of logs like web server, http, application logs, and so on. With careful and detail log analysis, organizations can tap on various business opportunities and threats surrounding it. The Elastic Stack solution on Dell EMC VxFlex family provides complete log analysis that helps in searching, analyzing, visualizing data from different machines, in addition helping to identify any potential security threats.

This white paper provides guidelines and best practices for optimal deployment of Elastic Stack on Dell EMC VxFlex family.

The Dell EMC VxFlex family for Elastic Stack solution offers unique advantages. The results from Elastic stack performance benchmarking tests, executed using benchmarking tool Rally clearly showed that VxFlex can support high performance requirements of Elastic stack. Other key benefits of deploying Elastic stack on VxFlex include:

• Flexibility of deployment options. • Faster indexing and searching performance. • Easily scalable solution to multiple Elastic master and data nodes

Introduction


1 Introduction This white paper outlines the design consideration of deploying Elastic Stack on VxFlex family. The primary focus of this paper is to provide the reader with details of an Elastic Stack solution on VxFlex family. This paper also provides more details on the best practices while deploying Elastic stack on VxFlex along with some performance benchmarking results using Rally tool.

1.1 Objective This white paper demonstrates

• VxFlex family and Elastic Stack Overview • Elastic Stack Architecture • Deployment of Elastic Stack on VxFlex integrated rack • Validating Elastic Stack on VxFlex integrated rack • Conclusion

1.2 Audience This white paper is intended for Elastic Stack administrators, system engineers, partners, and members of Dell EMC and partner professional service community who are looking to make real-time decisions, and analysis of data to make the right marketing decisions.

The reader of this document must have a working knowledge of the following technologies:

• Dell EMC VxFlex portfolios • Elastic Stack • VMware vSphere

Introduction


1.3 Terminology The following table defines acronyms and terms that are used throughout this document:

Terminology

Term Definition

HCI Hyperconverged Infrastructure

OOB Out-of-band management

SVM Storage Virtual Machine

SDS Storage Data Server

SDC Storage Data Client

VLAN Virtual Local Area Network

TOR Top of the Rack Switch

Product overview


2 Product overview

2.1 VxFlex family VxFlex family helps transforming from a traditional three-tier architecture to a modern data center without any trade-offs, meeting business requirements without compromise. The central software layer for VxFlex family is VxFlex OS, scale-out block storage service that enables customers to create a scale-out Server SAN and/or hyperconverged infrastructure. The VxFlex family currently include VxFlex Ready Nodes, VxFlex appliance, and VxFlex integrated rack.

VxFlex family

2.1.1 VxFlex integrated rack VxFlex integrated rack is a rack-scale engineered system, with integrated networking, that provides linear scalability and enterprise-grade availability. VxFlex integrated rack is engineered, manufactured, managed, supported, and sustained as one system for single end-to-end life cycle support.

VxFlex integrated systems create a server-based SAN by combining virtualization software, which is known as VxFlex OS, with Dell EMC PowerEdge servers to deliver flexible, scalable performance and capacity on demand. Local storage resources are combined to create a virtual pool of block storage with varying performance tiers. An important building block of VxFlex integrated rack is the compute part, which consumes the storage volumes from the system. The VxFlex integrated rack enables you to scale from a small environment to enterprise scale with over a thousand nodes. In addition, it provides enterprise grade data protection, multi-tenant capabilities, and add-on enterprise features such as QoS, thin provisioning, and snapshots. VxFlex systems deliver the performance and time-to-value required to meet the demands of the modern enterprise data center.

Product overview


VxFlex integrated rack benefits

2.1.2 VxFlex appliance VxFlex appliance is a preconfigured and validated for fast, easy deployment, VxFlex appliance offers a turnkey experience in an economic form factor. With VxFlex appliance, customers benefit from a smaller starting point, with massive scale potential, without having to compromise on performance and resiliency.

2.1.3 VxFlex Ready Nodes VxFlex Ready Nodes combine Dell EMC PowerEdge servers that are powered by Intel® Xeon® Scalable Processors and VxFlex OS software to create scalable, reliable, and easy-to-deploy building blocks for hyperconverged or server SAN architecture, multi-hypervisor or bare metal environments, and high-performance databases.

2.1.4 VxFlex OS VxFlex OS is software that creates a server and IP-based SAN from direct-attached storage to deliver flexible and scalable performance and capacity on demand. As an alternative to a traditional SAN infrastructure, VxFlex OS combines HDD, SSD, and NVMe media to create virtual pools of block storage with varying performance tiers. VxFlex OS provides enterprise-grade data protection, multi-tenant capabilities, and add-on enterprise features such as QoS, thin provisioning, and snapshots. VxFlex OS supports physical and virtualized servers, has been proven to deliver significant TCO savings vs. traditional SAN.

Product overview


VxFlex OS consists of three primary components: Meta Data Manager (MDM), Storage Data Client (SDC) and Storage Data Server (SDS). The MDMs work the brain of the system and are responsible for managing metadata and core functions such as automated rebuild and rebalance, which ensure data access as media and servers fail. The SDC runs like an agent or daemon on a server and acts like a highly available HBA to connect to storage cluster to consume the storage as required by the application workload. The SDSs are installed on every compute node and are daemons that contribute storage to the storage cluster. SDCs communicate directly with the SDSs. When an SDC gets an I/O request from the application, it sees the cached metadata map, and sends the request directly to the SDS, which contains the requested data.

2.1.5 VxFlex Manager VxFlex Manager is a unified management and automation solution for the VxFlex integrated rack and the VxFlex appliance that enables template-based provisioning of infrastructure and workloads. It provides monitoring, alerting, and health status of hardware and services. It simplifies and automates lifecycle management of diverse and heterogeneous IT resources. It enables users to respond rapidly and reliably to dynamic business needs.

VxFlex Manager is a comprehensive IT Operations Management (ITOM) software purpose that is built for VxFlex to automate and simplify implementation, expansion, and lifecycle management.

VxFlex Manager brings together multiple management consoles, workflow automation, and an intuitive interface that allows customers to monitor, manage, deploy, and maintain physical and virtual resources with the click of a button. Key tenets of the VxFlex Manager architecture include:

• System assurance: compliancy and non-disruptive remediation • Insights: monitoring, alerting, and health checks • Implementation simplification: simplified and automated system deployment and workflows • Node serviceability: single-button take node out of and back into service • Hypervisor update: single-button upgrade of hypervisor

2.1.6 VxFlex OS management VxFlex OS management is available using a GUI, CLI, and REST clients. There is a VMware vSphere® plug-in that allows VMware admins to deploy, upgrade, configure, and manage VxFlex OS in an ESXi environment within VMware vSphere.

Product overview


2.2 Elastic Stack Elastic Stack is a group of open source products from Elastic. It was earlier known as the ELK Stack. It is an amazing and powerful collection of three open source projects namely Elasticsearch, Logstash, and Kibana. These three technologies work well with each other despite being separate projects.

Elastic Stack components

Elastic Stack is an end-to-end log analysis solution that helps in deep searching, analyzing, and visualizing the logs generated from different machine. The Elastic Stack includes numerous capabilities that help users easily build solutions focused in three areas:

• Enterprise Search -- application search, site search, and workplace search. • Observability -- logging, APM, and metrics. • Security -- SIEM, endpoint, and security analytics.

Value added features such as machine learning, alerting, maps services, canvas, cross cluster replication, cross cluster search, field and document level security, and cluster authentication can all be leveraged to help most of the information about your data. Many of these capabilities are free and others can be licensed through an Elastic Subscription that includes support. For more information, contact Elastic team.

https://www.elastic.co/subscriptions

https://www.elastic.co/contact?storm=global-header-en

Elastic Stack architecture overview


3 Elastic Stack architecture overview Elastic stack is designed to help users to import data with any format and source; to search, analyze, and visualize imported data in real time. The following figure displays the components available in Elastic Stack:

• Beats are agents that ship the data from different systems.

• Logstash collects logs, and events data. It parses, filters, and transforms the data.

• Elasticsearch stores, searches, and analyzes the data.

• Kibana is used to visualize the data.

Elastic Stack architecture

3.1 Elasticsearch Elasticsearch is a NoSQL database that is based on Lucene search engine and provides a restful interface for different operations. It is easy to scale, flexible, schema-less distributed search and analytics engine. It provides simple deployment maximum reliability and uses indexes for data storage. Elasticsearch can be easily scaled horizontally. It is used to perform detailed analysis and stores the data centrally for quick search of the documents.

An Elasticsearch cluster is a collection of one or more nodes (servers) that together holds the entire data and provides federated indexing and search capabilities across all nodes. A node is a single server that is part of the cluster, stores the data, and participates in the cluster’s indexing and search capabilities.



An index is a collection of documents that have similar characteristics. A document is a basic unit of information that can be indexed. When an index is created, the number of shards can be defined. Each shard is a fully functional and independent "index" that can be hosted on any node in the cluster. Sharding is important for two primary reasons:

• It allows horizontally split or scale the content volume. • It allows to distribute and parallelize operations across shards (potentially on multiple nodes) thus

increasing performance/throughput.

For more information about Elasticsearch, see Elastic Stack and Product Documentation .

3.2 Logstash Logstash is the data collection pipeline tool through which input is taken from different sources and output to different data sources. The data is cleansed through filter options and transformed before sending it to Elasticsearch. Logstash has different adapters to handle different applications, and it normalizes the data into different destinations. Processing in Logstash is organized into one or more pipelines.

Logstash

These pipelines receive data and place it in internal queue. The processing threads then use filter plugins, and once the data is processed its send to the output plug-in.

https://www.elastic.co/guide/en/elasticsearch/



3.3 Kibana Kibana is an open-source analytics software data visualization tool. With this tool, different visualizations, charts, maps, histograms, and dashboards can be created. Kibana reads data from Elasticsearch and helps developers to have an immediate insight into it. Kibana enables creating and saving custom dashboards. It can be used for search, view, and interact with data that is stored in Elasticsearch directories. There are different methods for performing searches on the data.

A sample Kibana dashboard is shown here:

Sample Kibana Dashboard

The most common search types in Kibana are Free text searches, Field-level searches, Logical statements, and Proximity searches.

Kibana offers loads of advantages like easy visualizing, real-time analysis, debugging capabilities, saving and managing multiple dashboards, and it is completely integrated with Elasticsearch.

3.4 Beats Beats are open-source data shippers that are installed as agents on the servers and send operational data to Elasticsearch. They are lightweight and can send the data from hundreds or thousands of machines to Elastic Common Schema (ECS) that normalizes the data. Each beat is a separately installable product.

For example, Metricbeat is used to collect metrics for memory usage, CPU usage, and disk space, whereas Filebeat is used to send file data such as logs.

Solution architecture


4 Solution architecture For this paper, the Elastic Stack solution is deployed on Dell EMC VxFlex integrated rack. The VxFlex integrated is configured in a two-layer configuration meaning that servers are either compute nodes, providing the ESXi hypervisor or storage nodes running VxFlex OS.

4.1 Logical architecture The below diagram shows logical architecture diagram of Elastic Stack on Dell EMC VxFlex integrated rack two-layer setup that had four SDS and three SDC. The setup of VxFlex two-layer was configured using VxFlex Manager which automates the entire deployment process.


From VxFlex OS standpoint, there was a single VxFlex OS cluster with a protection domain, consisting of four RHEL storage only nodes that played the role of SDS. Each host is populated with ten 1.92 TB Toshiba SSD drives. These 40 disks are used to create a storage pool from which the usable storage volumes are created.

The VxFlex OS client software (SDC) is installed on each ESXi host in order to access the volumes created in the storage pool, a VMware Datastore was created on each of these volumes. The Elastic Cluster VMs were deployed onto these datastores. The Elastic cluster that is included one Master VM and two Data VMs.

On each of the three SDCs, a single CentOS VM was created. The first to host Logstash, the second Elasticsearch and the third Kibana. All these three VMs could communicate with each other.

For detailed configuration of VxFlex integrated nodes, see Appendix 9.2 Configuration.



4.2 Network architecture The following figure demonstrates high-level network architecture of the two-layer set up on VxFlex integrated rack system:

Network architecture

VxFlex integrated rack networking details Components Description

Cisco Nexus 93180YC-EX 10 Gbps & 25 Gbps TOR switches Cisco Nexus 9332PQ 25 Gbps Aggregation switches Cisco Nexus 3172TQ 1 Gbps & 10 Gbps Management switches VxFlex storage traffic 2 x 25 Gbps links

Testing and Validation


5 Testing and Validation This section provides a detailed summary and description of the tests that are performed to validate the Elastic Stack hosted on VxFlex integrated rack with benchmarking tool Rally.

5.1 Test methodology Unit testing through System testing methods were used to perform various tests on Elastic Stack. Elastic team used Rally, a benchmarking tool to run their nightly benchmarking tests.

Rally acts as load generator tool for the Elasticsearch. It also builds, sets up and tears down the Elasticsearch cluster. This makes Rally a suitable tool to run in a basic environment. Rally also can be used on an existing Elasticsearch cluster to manage benchmark configurations, compare results, and find any performance issues. For more information about Rally, see Getting Started with Rally.

For this paper, most prevalent use cases of Elastic were chosen that demonstrates the VxFlex Family is one of the best platforms to host Elasticsearch.

Use Case 1: This use case is based on the rally-eventdata-track that simulates event-based data use-cases. This track can be used to create more complex and realistic simulations and benchmarks. More information about this track use can be found at out at https://github.com/elastic/rally-eventdata-track.

Use Case 2: This use case is based on the webserver logs that are collected from a big event in 1998. These are real logs and demonstrate elastic search indexes these logs and can be visualized in Kibana. More information about this track can be found out at https://github.com/elastic/rally-tracks/tree/master/http_logs.

https://esrally.readthedocs.io/en/stable/quickstart.html

https://github.com/elastic/rally-eventdata-track

https://github.com/elastic/rally-tracks/tree/master/http_logs



5.2 Test results Rally was run on a separate host to benchmark the Elasticsearch cluster hosted on a two-layer VxFlex cluster. The two-layer configuration included three compute nodes and four storage nodes. The Elasticsearch cluster includes one Master Node and two Data Nodes. The following figure shows the test set up for Rally:

Test setup

Use Case 1: A challenge ‘lasticlogs-1bn-load from EventData track was chosen to benchmark existing Elasticsearch cluster. This challenge indexes 1 billion events into several indexes of two primary shards each and results in around 200 GB of indexes being generated on disk. It can be used give an idea of how max indexing performance behaves over an extended period. The benchmarking was done using the following command:

esrally --track=eventdata --target-hosts=192.168.105.71:9200,192.168.105.72:9200,192.168.105.73:9200 --track-repository=eventdata --challenge=elasticlogs-1bn-load --pipeline=benchmark-only



Results for challenge elasticlogs-1bn-load

Use Case 2: http_logs track was chosen and run using Rally. This is rather a simple track based on the webserver logs from the 1998 Football world cup. This track has five shards, and eight bulk indexing requests are issued when the test is running. The benchmarking was done using the following command:

esrally --track=pmc --target-hosts=192.168.105.71:9200,192.168.105.72:9200,192.168.105.73:9200 --pipeline=benchmark-only

The key results for track http_logs are represented in the following table:

Results for track http_logs

For detailed results, see Appendix A.1.

Parameters Value

Median Throughput for index-append-1000-elasticlogs_q_write 93,673 docs/s

Median Throughput for node_storage 10.02 ops/s

Error Rate 0%

Total time to run the tests 10,844 s

Average CPU Utilization on each Data Node 58%

Memory Utilization on each Data Node 32%

Network Utilization on each Data Node 96,000 KBps

Parameters Value

Index Throughput 302,283 docs/s

Cumulative indexing time of primary shards 146 min

Total Young Gen GC 140.62 s

Total time to run the tests 90.31 min

Average CPU Utilization on each Data Node 30%

Memory Utilization on each Data Node 25%

Best practices


6 Best practices The following sections outline the best practices followed in this solution.

6.1 VxFlex integrated rack • Minimum number of nodes for production workload is seven for a VxFlex two-layer set up. • Homogenous node types are recommended for predictable performance. • Maximum number of devices in a storage pool is 300. • Maximum number of nodes in a protection domain is 32. • Change the passwords for all default accounts. • Use secure communication – HTTPS (TCP port 443) to remotely access VxFlex nodes. • Ensure VxFlex rack is compliant to an RCM.

6.2 VxFlex OS • Configure high-performance profile for MDM, SDS, and SDC. • Disable Read Flash cache and Read RAM cache for all flash clusters. • Check with VxFlex platform team to increase the per device queue length value to 256 per host for

improving the I/O concurrency. • Ensure that the customize power plan is set to High Performance.

6.3 VxFlex network • Confirm with VxFlex platform team to enable Jumbo frames for Windows VM, SVM, and at ESXi host. • Use the Para virtual SCSI (PVSCSI) controller on guest VMs to achieve high performance. • Enable secure network protocol options only (for example, HTTPS and Secure Shell (SSH). • Separate management and control traffic from production application traffic. You can provide this

separation by using VLANs. • Separate VMware vSphere vMotion traffic from production traffic according to VxFlex standard.

6.4 Elasticsearch • RAM to be 32 GB or 64 GB for a virtual machine. • JVM Heap size to 50% of RAM, max up to 32 GB. Keep at least 50% RAM available. • If Elasticsearch must scale vertically, then add more vCPUs. • Start with a proof of concept, and then test, optimize, iterate, and scale. • Elasticsearch recommends increasing ulimit of File descriptors to 65536. • Disable swap file. • Increase the refresh interval of index from 1 s to more. For example, 20 s. This interval depends

on case to case basis. • Benchmarking VM where Rally is supposed to run on a different VM which is not part of the

Elasticsearch cluster. • Link followed for benchmarking https://www.elastic.co/blog/seven-tips-for-better-elasticsearch-

benchmarks.

Note: These best practices were used for the Elasticsearch configuration on a virtual machine and few choices made were subjective.

https://www.elastic.co/blog/seven-tips-for-better-elasticsearch-benchmarks

https://www.elastic.co/blog/seven-tips-for-better-elasticsearch-benchmarks

Deployment of Elastic Stack on VxFlex


7 Deployment of Elastic Stack on VxFlex

Elastic Stack can be deployed on different operating systems. For this white paper, Elastic Stack is deployed on CentOS 7 VM on Dell EMC VxFlex integrated rack.

Elastic cluster deployment had one Master Node, and two Data Nodes. All these clusters were deployed as Virtual Machines with CentOS 7 on different ESXi hosts.

1. Installed Elasticsearch 7.3 by following the procedure in the following link https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html

2. Once Elasticsearch is installed, then Logstash was installed using the procedure in the following link https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

3. Kibana was installed to access the dashboards using the procedure in the following link

https://www.elastic.co/guide/en/kibana/current/rpm.html

4. Installed other VMs with Filebeat to push log file into the Elasticsearch. Beats were installed using the procedure in the following link https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html

https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

https://www.elastic.co/guide/en/kibana/current/rpm.html

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html

Conclusion


8 Conclusion

The solution provided in this paper, demonstrates how you can deploy Elastic Stack on VxFlex family to meet performance, resiliency, and scale. In addition, it states the best practices for deployment of VxFlex system and the Elastic Stack. The solution showcases how VxFlex can address some of the key Elasticsearch application challenges, for example the benchmark results demonstrate the capability of indexing one billion documents in approximately 3 hours, highlighting the extreme performance capabilities of VxFlex.

Elastic customers are looking for extreme performance and scalability and flexibility to be deployed in multiple operating environments such as Windows, Linux, Virtualized or Bare Metal. VxFlex provides the scalable compute and storage that is required for Elastic stack with utmost flexibility.

Besides, Elastic is a GCP partner, available through the GCP Marketplace and VxFlex is already validated as the infrastructure for Google Anthos solution.

Appendix A: Detailed results


A Appendix A: Detailed results

A.1 Use case1: EventData track

The challenge elasticlogs-1bn-load was run on EventData track and the results are captured in this section.

VxFlex Dashboard during test run

CPU Utilization (60%) on one of the Data Nodes, the other data node had similar CPU Utilization.



Memory Utilization (47%) on Data Node

Network Utilization on Data Node

Iostat on Data Node



A.2 Use case1: Http_logs track

The http_logs_track was run with Rally, and the results are captured in this section.

VxFlex Dashboard during test run

CPU Utilization on one of the Data Nodes, the other data node had similar CPU Utilization



Memory Utilization on Data Node

Network Utilization



Iostat on Data Node

Appendix B: Configuration details


B Appendix B: Configuration details Configuration details

Component Description

VxFlex integrated rack Compute Nodes

3 x VxFlex Nodes (R640 servers):

• VxFlex OS version: R3_0.1 • ESXi version: 6.7 U2 • CPU: 2 x Intel(R) Xeon(R) Gold 6140 CPU @ 2.30 GHz, 18 cores • Memory: 384-GB RAM ((12 x 32-GB DIMMs)

4 x VxFlex Nodes (R640 servers):

• VxFlex OS version: R3_0.1 • VMware ESXi version: 6.7 U2 • CPU: 2 x Intel(R) Xeon(R) Gold 6126 CPU @ 2.60 GHz, 12 Cores • Memory: 192-GB RAM (12 x 16-GB DIMMs)

Network 2 NIC cards, each having 2 ports 25 GbE connection

Elastic Stack • Elasticsearch: 7.3.2 • Kibana: 7.3.2 • Logstash:7.3.2 • FileBeat: 7.3.2 • Rally: 1.3

Elasticsearch Master VM • Operating system version: CentOS7 • vCPU: 16 • Memory: 32 GB • HeapSize: 16 GB

Elasticsearch Data Node VM • Operating system version: CentOS7 • vCPU: 16 • Memory: 64 GB • HeapSize: 32 GB

Rally VM • Operating system version: CentOS7 • vCPU: 8 • Memory: 16 GB

Technical support and resources


C Technical support and resources

C.1 Related resources See the following referenced or recommended resources that are related to this document:

Note: The following links are open to customers although some may require registration for access.

• VxFlex OS blog: https://blog.dellemc.com/en-us/tag/VxFlex OS/ • Elasticsearch: https://www.elastic.co/products/elasticsearch • Rally: https://esrally.readthedocs.io • Elasticsearch Tuning: https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-

indexing-speed.html • Deployment of Elastic Stack: https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-

centos-7/

C.2 Additional resources Referenced or recommended publications:

• Dell.com/support is focused on meeting customer needs with proven services and support. • Dell EMC VxFlex product overview and White papers on DellEMC.com provides expertise that helps

to ensure customer success on Dell EMC VxFlex family.

https://blog.dellemc.com/en-us/tag/scaleio/

https://www.elastic.co/products/elasticsearch

https://esrally.readthedocs.io/

https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html

https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-centos-7/

https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-centos-7/

http://www.dell.com/support

https://www.dellemc.com/en-us/hyperconverged-infrastructure/vxflex.htm

elastic stack solution on dell emc vxflex family...000060 white paper elastic stack solution on dell...

Documents