Mastering VMware NSX® for vSphere®Elver Sena Sosa

Copyright © 2020 by John Wiley & Sons, Inc., Indianapolis, Indiana

Published simultaneously in Canada

ISBN: 978-1-119-51354-4ISBN: 978-1-119-51351-3 (ebk)ISBN: 978-1-119-51353-7 (ebk)

Manufactured in the United States of America

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechani-cal, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for per-mission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at https://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or website is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read.

For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at https://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Control Number: 2019956689

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. VMware NSX and vSphere are registered trademarks of VMware, Inc. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

About the AuthorsElver Sena Sosa is a Data Center Solutions Architect who specializes in Software Defined Data Center technologies. Over the past 20 years, Elver has driven the presales, design, and deploy-ment of projects throughout APAC, EMEA, and APAC. Elver has been the go-to partner for helping VMware evangelize NSX, vSAN, and VCF, VMware’s Software Defined products and solutions. Elver is a skilled communicator who enjoys sharing his experience on the interdepend-encies of technology to audiences around the world. Elver has continued working in SDDC with his company, Hydra 1303, Inc, where he published his first book, the NSX exam study guide, VCP6-NV Official Cert Guide, and the YouTube vSAN Architecture 100 series.

Trey McMahon is based out of Richmond, Virginia, and is a Cloud Data Engineer on the Hydra 1303 team. Trey has been in networking since 1997, writing authorized courses and exams for Cisco, developing instructor readiness programs and labs for EMC and Cisco, teaching network engineering in over 30 countries, and supporting VMware customer enablement. These days at Hydra 1303, he specializes in cloudy things.

Zac Smith is a lead Data Center Solutions Engineer at Hydra 1303. He specializes in providing automated data center solutions. Zac has been in the IT industry for 20 years and has been a part of many enterprise solution designs and deployments. Zac has also been involved in writing numerous courses for VMware and Cisco, as well as providing partner and customer enablement sessions on a global scale.

About the Technical EditorShane Weinbrecht has been in the technology industry for the past 20 years, working as a systems administrator for enterprise companies such as IBM and The Adidas Group; for the past 10 years, he has been on the vendor side currently employed by Nutanix as a senior systems engineer covering healthcare. Most importantly, Shane is happily married and the proud father of two amazing boys and enjoys spending time with his family and friends, photography, Obstacle Course Racing, and Krav Maga.

AcknowledgmentsA special thanks goes to Luciana de Padua, a key member of our team here at Hydra 1303 that we rely on for . . . well, everything. She sets a high bar for excellence with her ninja-level PKS and NSX-T skills, positive energy, and ability to make all of this fun, while loving what she does. Always in demand, both by VMware internally and our direct customers, she’s never in one time zone for very long. This book wouldn’t have been written if we didn’t have Lu leading Hydra 1303’s European engagements throughout the process.

Thanks also to the talented editors at Wiley Publishing: Tom Cirtin, Kim Cofer, Shane Weinbrecht, Kathyrn Duggan and Athiyappan Lalith Kumar. Your suggestions were consistently dead on and helped to improve the clarity every time.

Contents at a GlanceIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Chapter 1 • Abstracting Network and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 2 • NSX Architecture and Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Chapter 3 • Preparing NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 4 • Distributed Logical Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Chapter 5 • Marrying VLANs and VXLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Chapter 6 • Distributed Logical Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Chapter 7 • NFV: Routing with NSX Edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Chapter 8 • More NVF: NSX Edge Services Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Chapter 9 • NSX Security, the Money Maker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Chapter 10 • Service Composer and Third-Party Appliances . . . . . . . . . . . . . . . . . . . . 223

Chapter 11 • vRealize Automation and REST APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Appendix • The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

ContentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Chapter 1 • Abstracting Network and Security . . . . . . . . . . . . . . . . . . . . . . . .1Networks: 1990s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Colocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Workload-to-Server Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Inefficient Resource Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3The Long Road to Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Data Centers Come of Age . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Data Center Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Workloads Won’t Stay Put . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6What Is Happening in There? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Portability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Virtualize Away . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Extending Virtualization to Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Virtual Networking and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9NSX to the Rescue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Chapter 2 • NSX Architecture and Requirements . . . . . . . . . . . . . . . . . . . . . .15NSX Network Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Planes of Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16NSX Manager Role and Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18ESXi Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19vCenter Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20vSphere Distributed Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21NSX VIBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Competitive Advantage: IOChain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24IOChain Security Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24NSX Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25NSX Controller Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26NSX Controller Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26NSX Edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28ESG Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

NSX Role-Based Access Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Overlay and Underlay Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Replication Modes for Traffic Going to Multiple Destinations . . . . . . . . . . . . . . . . . . 34

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Xii | Contents

Chapter 3 • Preparing NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39NSX Manager Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Open Ports and Name Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Minimum Resource Requirements for NSX Data Center Appliances . . . . . . . . . . . . . 40vSphere HA and DRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41IP Addressing and Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Installing the Client Integration Plug-in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Installing NSX Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Associating NSX Manager to vCenter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Adding AD/LDAP to NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Linking Multiple NSX Managers Together (Cross- vCenter NSX) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Multi-site Consistency with Universal Components . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Primary and Secondary NSX Managers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Preparing ESXi Clusters for NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Creating a Universal Transport Zone on the Primary NSX Manager . . . . . . . . . . . . . . . . 56vSphere Distributed Switches Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57Adding Secondary NSX Managers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Chapter 4 • Distributed Logical Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61vSphere Standard Switch (vSS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Traffic Shaping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Understanding Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64NIC Teaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Ensuring Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Virtual Distributed Switch (vDS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67Virtual eXtensible LANs (VXLANs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Employing Logical Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Three Tables That Store VNI Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Collecting VNI Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Centralized MAC Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75VTEP Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

We Might as Well Talk about ARP Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Filling In the L2 and L3 Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Switch Security Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Understanding Broadcast, Unknown Unicast, and Multicast . . . . . . . . . . . . . . . . . . . . . . 83Layer 2 Flooding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Replication Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Deploying Logical Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Creating a Logical Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Chapter 5 • Marrying VLANs and VXLANs . . . . . . . . . . . . . . . . . . . . . . . . . . .87Shotgun Wedding: Layer 2 Bridge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Contents | Xiii

Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Under the Hood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102Layer 2 VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102NSX Native L2 Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Hardware Switches to the Rescue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Hardware VTEPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104Under the Hood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Chapter 6 • Distributed Logical Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Distributed Logical Router (DLR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Control Plane Smarts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Logical Router Control Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Understanding DLR Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Another Concept to Consider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Let’s Get Smart about Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117OSPF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119Border Gateway Protocol (BGP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120Oh Yeah, Statics Too . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Deploying Distributed Logical Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Chapter 7 • NFV: Routing with NSX Edges . . . . . . . . . . . . . . . . . . . . . . . . . 137Network Function Virtualization: NSX Has It Too . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

This Is Nice: Edge HA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138Adding HA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Let’s Do Routing Like We Always Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140Deploying the Edge Services Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144Configuring BGP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151Configuring OSPF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154Configuring Static Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Routing with the DLR and ESG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156Using CLI Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156Default Behaviors to Be Aware Of . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157Equal Cost Multi-Path Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Chapter 8 • More NVF: NSX Edge Services Gateway . . . . . . . . . . . . . . . . . . 163ESG Network Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163Network Address Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Configuring Source NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166Configuring Destination NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166Configuring SNAT on the ESG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167Configuring DNAT on the ESG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

XiV | Contents

ESG Load Balancer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171Configuring an ESG Load Balancer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Layer 2 VPN (If You Must) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178Secure Sockets Layer Virtual Private Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Split Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180Configuring SSL VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

Internet Protocol Security VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187Understanding NAT Traversal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188Configuring IPsec Site-to-Site VPN with the ESG . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Round Up of Other Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190DHCP Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191Configuring the ESG as a DHCP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192DHCP Relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194Configuring the DLR for DHCP Relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196DNS Relay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198Configuring DNS Relay on the ESG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Chapter 9 • NSX Security, the Money Maker . . . . . . . . . . . . . . . . . . . . . . . . 203Traditional Router ACL Firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203I Told You about the IOChain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Slot 2: Distributed Firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206Under the Hood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Adding DFW Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210Segregating Firewall Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214IP Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215Gratuitous ARP Used in ARP Poisoning Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Why Is My Traffic Getting Blocked? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218Great, Now It’s Being Allowed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219Identity Firewall: Rules Based on Who Logs In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Distributing Firewall Rules to Each ESXi Host: What’s Happening? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Chapter 10 • Service Composer and Third-Party Appliances . . . . . . . . . . 223Security Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Dynamic Inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225Static Inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226Static Exclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226Defining a Security Group through Static Inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 227Defining a Security Group through Dynamic Inclusion . . . . . . . . . . . . . . . . . . . . . . . 229Customizing a Security Group with Static Exclusion . . . . . . . . . . . . . . . . . . . . . . . . . 231Defining a Security Group Using Security Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231Adding to DFW Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Service Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236IOChain, the Gift that Keeps on Giving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Layer 7 Stuff: Network Introspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Guest Introspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Contents | XV

Service Insertion Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238Security Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Creating Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Enforcing Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

Chapter 11 • vRealize Automation and REST APis . . . . . . . . . . . . . . . . . . . 247vRealize Automation Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247vRA Editions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249Integrating vRA and NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

vRealize Automation Endpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250Associating NSX Manager with vRealize Automation . . . . . . . . . . . . . . . . . . . . . . . . 252Network Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253vRA External, Routed, and NAT Network Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . 255Reservations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

vRealize Orchestrator Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261Creating a Blueprint for One Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261Adding NSX Workflow to a Blueprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264Creating a Request Service in the vRA Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Configuring an Entitlement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

Deploying a Blueprint that Consumes NSX Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271REST APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

NSX REST API GET Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275NSX REST API POST Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275NSX REST API DELETE Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

Appendix • The Bottom Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279Chapter 1: Abstracting Network and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279Chapter 2: NSX Architecture and Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280Chapter 3: Preparing NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280Chapter 4: Distributed Logical Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281Chapter 5: Marrying VLANs and VXLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283Chapter 6: Distributed Logical Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284Chapter 7: NFV: Routing with NSX Edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286Chapter 8: More NVF: NSX Edge Services Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287Chapter 9: NSX Security, the Money Maker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289Chapter 10: Service Composer and Third-Party Appliances . . . . . . . . . . . . . . . . . . . . . . 290Chapter 11: vRealize Automation and REST APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

Introduction

The advantages of server virtualization in data centers are well established. From the beginning, VMware has led the charge with vSphere. Organizations migrating physical servers to virtual immediately see the benefits of lower operational costs, the ability to pool CPU and memory resources, server consolidation, and simplified management.

VMware had mastered compute virtualization and thought, “Why not do the same for the entire data center?” Routers, switches, load balancers, firewalls . . . essentially all key physical networking components, could be implemented in software, creating a Software-Defined Data Center (SDDC). That product, VMware NSX, is the subject of this book.

In 1962, Sir Arthur Clarke published an essay asserting three laws. His third law stated, “Any sufficiently advanced technology is indistinguishable from magic.” If you’re not familiar with NSX, the abilities you gain as a network administrator almost seem like magic at first, but we’ll dive into the details to explain how it all works. It doesn’t matter if you don’t have a background in vSphere. There are plenty of analogies and examples throughout, breaking down the underly-ing concepts to make it easy to understand the capabilities of NSX and how to configure it.

The way NSX provides network virtualization is to overlay software on top of your existing physical network, all without having to make changes to what you have in place. This is much like what happens with server virtualization. When virtualizing servers, a hypervisor separates and hides the underlying complexities of physical CPU and memory resources from the software components (operating system and application), which exist in a virtual machine. With this separation, the server itself just becomes a collection of files, easily cloned or moved. An immedi-ate benefit gained is the time and effort saved when deploying a server. Instead of waiting for the order of your physical servers to arrive by truck, then waiting for someone to rack and stack, then waiting for someone else to install an operating system, then waiting again for network connectivity, security, installation, and configuration of the application . . . you get the picture. Instead of waiting on each of those teams, the server can be deployed with a click of a button.

NSX can do the same and much more for your entire data center. The agility NSX provides opens new possibilities. For instance, a developer comes to you needing a temporary test server and a NAT router to provide Internet connectivity. The admin can use NSX to deploy a virtual machine (VM) and a virtual NAT router. The developer completes the test, the VM and NAT router are deleted, and all of this occurs before lunch. NSX can do the same thing for entire networks.

xvIII | INTRODUCTION

The same developer comes to you in the afternoon requesting a large test environment that mimics the production network while being completely isolated. She needs routers, multiple subnets, a firewall, load balancers, some servers running Windows, others running Linux: all set up with proper addressing, default gateways, DNS, DHCP, and her favorite dev tools installed and ready to go. It’s a good bet that setting this up in a physical lab would take a lot of time and may involve several teams.

With NSX, that same network could be deployed by an administrator with a few clicks, or even better, it can be automated completely, without having to involve an administrator at all. VMware has a product that works with NSX called vRealize Automation (vRA) that does just that. It provides our developer with a catalog portal, allowing her to customize and initiate the deployment herself, all without her needing to have a background in networking.

If you’re a security admin, this might seem like chaos would ensue, with anyone being able to deploy whatever they want on the network. NSX has that covered as well. As a security adminis-trator, you still hold the keys and assign who can do what, but those keys just got a lot more powerful with NSX.

Imagine if you had an unlimited budget and were able to attach a separate firewall to every server in the entire network, making it impossible to bypass security while significantly reducing latency. Additionally, what if you didn’t have to manage each of those firewalls individually? What if you could enter the rules once and they propagate instantly to every firewall, increasing security dramatically while making your job a lot easier and improving performance. It’s not magic; that’s the S in NSX.

The N in NSX is for networking, the S is for security. The X? Some say it stands for eXtensibil-ity or eXtended, but it could just as well be a way to make the product sound cool. Either way, the point is that both networking and security get equal treatment in NSX, two products in one. At the same time, instead of these additions adding more complexity to your job, you’ll find just the opposite. With the firewall example or the example of the developer deploying the large test network, as a security administrator, you set the rules and permissions and you’re done. Automation takes care of the tedious legwork, while avoiding the typical mistakes that arise when trying to deploy something before having your morning coffee. Those mistakes often lead to even more legwork with more of your time drained troubleshooting.

Wait, the title of the book says NSX-V. What does the V for? Since NSX is tightly integrated with vSphere, its legal name is NSX for vSphere, but we’ll just refer to it as NSX for short. NSX-V has a cousin, NSX-T, with the T standing for transformers. In a nutshell, that product is made to easily integrate with environments using multiple hypervisors, Kubernetes, Docker, KVM, and OpenStack. If all that sounds like a lot to take in, not to worry, we’ll save that for another book.

Welcome to NSX.

What Does This Book Cover?Chapter 1: Abstracting Network and Security We often learn how to configure something new without really understanding why it exists in the first place. You should always be asking, “What problem does this solve?” The people armed with these details are often positioned to

INTRODUCTION | xIx

engineer around new problems when they arise. This chapter is a quick read to help you understand why NSX was created in the first place, the problems it solves, and where NSX fits in the evolution of networking, setting the stage for rest of the book’s discussions on virtualization.

Chapter 2: NSX Architectures and Requirements This chapter is an overview of NSX operations. It details the components that make up NSX, their functions, and how they communicate. Equally important, it introduces NSX terminology used throughout the book, as well as virtualization logic.

Chapter 3: Preparing NSX In this chapter, you will find out everything you need to have in place before you can deploy NSX. This includes not only resources like CPU, RAM, and disk space, but it also covers ports that are necessary for NSX components to communicate, and prepping your ESXi hosts for NSX.

Chapter 4: Distributed Logical Switch It’s helpful if you are already familiar with how a physical switch works before getting into the details of a Distributed Logical Switch. Don’t worry if you’re not. In this chapter, we’ll look at how all switches learn, and why being distributed and logical is a dramatic improvement over centralized and physical. You’ll also find out how NSX uses tunnels as a solution to bypass limitations of your physical network.

Chapter 5: Marrying VLANs and VXLANs On the virtual side, we have VMs living on VXLANs. On the physical side, we have servers living on VLANs. Rather than configuring lots of little subnets and routing traffic between logical and physical environments, this chapter goes into how to connect the two (physical and logical), making it easy to exchange information without having to re-IP everything.

Chapter 6: Distributed Logical Router In Chapter 4, we compared a physical switch and a Distributed Logical Switch. We do the same in this chapter for physical routers vs. Distributed Logical Routers, covering how they work, how they improve performance while making your job easier, and the protocols they use to communicate.

Chapter 7: NFV: Routing with NSX Edges In this chapter, we talk about network services beyond routing and switching that are often provided by proprietary dedicated physical devices, such as firewalls, load balancers, NAT routers, and DNS servers. We’ll see how these network functions can be virtualized (Network Function Virtualization, or NFV) in NSX.

Chapter 8: More NFV: NSX Edge Services Gateway This chapter focuses on the Edge Services Gateway, the Swiss Army knife of NSX devices, that can do load balancing, Network Address Translation (NAT), DHCP, DHCP Relay, DNS Relay, several flavors of VPNs, and most importantly, route traffic in and out of your NSX environment.

Chapter 9: NSX Security, the Money Maker When it’s said that NSX provides better security, you’ll find out why in this chapter. Rather than funneling traffic through a single-point physical firewall, it’s as if a police officer were stationed just outside the door of every home. The NSX Distributed Firewall provides security that is enforced just outside the VM, making it impossible to bypass the inspection of traffic in or out. We also look at how you can extend NSX functionality to incorporate firewall solutions from other vendors.

Chapter 10: Service Composer and Third-Party Appliances This chapter introduces Service Composer. This built-in NSX tool allows you to daisy-chain security policies based on what is happening in real time. You’ll see an example of a virus scan triggering a series of security

xx | INTRODUCTION

policies automatically applied, eventually leading to a virus-free VM. You’ll also learn how to tie in services from other vendors and explain the differences between guest introspection and network introspection.

Chapter 11: vRealize Automation and REST APIs Saving the best time-saving tool for last, this chapter covers vRealize Automation (vRA), a self-service portal containing a catalog of what can be provisioned. If a non-admin needs a VM, they can deploy it. If it needs to be a cluster of VMs running Linux with a load balancer and NAT, they can deploy it. As an admin, you can even time bomb it, so that after the time expires, vRA will keep your network clean and tidy by removing what was deployed, automatically. You will also see how administra-tive tasks can be done without going through a GUI, using REST APIs.

Additional ResourcesHere’s a list of supporting resources that augment what is covered in this book, including the authorized VCP6-NV NSX exam guide, online videos, free practice labs, helpful blogs, and supporting documentation.

VCP6-NV Official Cert Guide (NSX exam #2V0-642) by Elver Sena Sosa:

www.amazon.com/VCP6- NV- Official- Cert- Guide- 2V0- 641/dp/9332582750/ref=sr_1_1?keywords=elver+sena+sosa&qid=1577768162&sr=8- 1

YouTube vSAN Architecture 100 Series by Elver Sena Sosa:

www.youtube.com/results?search_query=vsan+architecture+100+series

Weekly data center virtualization blog posts from the Hydra 1303 team:

www.hydra1303.com

Practice with free VMware NSX Hands-on Labs (HOL):

www.vmware.com/products/nsx/nsx-hol.html

VMUG – VMware User Group:

www.vmug.com

VMware NSX-V Design Guide:

www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/nsx/vmw- nsx- network- virtualization- design- guide.pdf

VMware authorized NSX classes (classroom and online):

mylearn.vmware.com/mgrReg/courses.cfm?ui=www_edu&a=one&id_ subject=83185

How to Contact the PublisherIf you believe you’ve found a mistake in this book, please bring it to our attention. At John Wiley & Sons, we understand how important it is to provide our customers with accurate content, but even with our best efforts, an error may occur.

In order to submit your possible errata, please email it to our Customer Service Team at wileysupport@wiley.com with the subject line “Possible Book Errata Submission.”

Chapter 1

In this chapter, we will examine the evolution of Data Center Networking and Security from the 1990s to the present in order to better understand how network virtualization in today’s data centers provides solutions that reduce costs, greatly improve manageability, and increase security.

Most IT professionals are familiar with server virtualization using virtual machines (VMs). A virtual machine is purely software. An abstraction layer creates a way to decouple the physical hardware resources from that software. In doing so, the VM becomes a collection of files that can be backed up, moved, or allocated more resources without having to make changes to the physical environment.

We will delve into how VMware NSX is the next step in data center evolution, allowing virtualization to extend beyond servers. Routers, switches, firewalls, load balancers, and other networking components can all be virtualized through NSX. NSX provides an abstraction layer that decouples these components from the underlying physical hardware, which provides administrators with new solutions that further reduce costs, improve manageability, and increase security across the entire data center.

IN THIS CHAPTER, YOU WILL LEARN ABOUT:

◆ The evolution of the modern data center

◆ How early networks created a need for data centers

◆ Colocation: the sharing of provider data centers

◆ Challenges in cost, resource allocation, and provisioning

◆ VMware server virtualization

◆ VMware storage virtualization

◆ VMware NSX: virtual networking and security

Networks: 1990sThe 1990s brought about changes to networking that we take for granted today. We shifted from the original Ethernet design of half-duplex communication, where devices take turns sending data, to full duplex. With full duplex, each device had a dedicated connection to the network that allowed us to send and receive simultaneously, while at the same time reducing collisions on the wire to zero (see Figure 1.1). The move to full duplex effectively doubled our throughput.

Abstracting Network and Security

2 | CHAPTER 1 AbstrActing network And security

Figure 1.2 colocated space rented in provider data centers

100 Mbps Ethernet connections became possible and the technology was given the unfortu-nate name Fast Ethernet, a label that has not aged well considering that available 100 GB ports of today are 1000 times faster than the ’90s version of “Fast.”

The ’90s also ushered in our first cable modems, converging data and voice with VoIP, and of course, the Internet’s explosion in popularity. As Internet businesses started to boom, a demand was created for a place to host business servers. They needed reliable connectivity and an environment that provided the necessary power and cooling along with physical security. They needed a data center. Although it was possible for an organization to build its own dedicated data centers, it was both costly and time-consuming, especially for online startups booming in the ’90s.

ColocationAn attractive solution, especially for startups, was colocation. Many providers offered relatively inexpensive hosting plans, allowing businesses to move their physical servers and networking devices to the provider’s ready-made data center. With colocation, organizations were essentially renting space, but they still maintained complete control over their physical devices (see Figure 1.2). The organization was still responsible for installing the operating system, upgrades, and backups. The only real difference was that the location of their compute resources had changed from locally hosted to the provider site.

ONE WAY

TWO WAY

TWO WAY

SimplexOne direction, one channel

Half DuplexTwo directions, one channel

Full DuplexTwo directions, two channels

Figure 1.1 simplex, half duplex, and full duplex compared

networks: 1990s | 3

The Internet boom of the ’90s meant that due to web computing, a massive amount of data was being generated, which created a need for storage solutions such as Fibre Channel, iSCSI, and NFS. One major benefit in having these resources together in a data center was centralized management.

Not all data centers looked as impressive in the ’90s as they do today. Google’s first data center was created in 1998, and was just a 7 × 4 foot cage with only enough space for 30 servers on shelves.

Workload-to-Server RatioThe general design choice at the time was that each server would handle a single workload in a 1:1 ratio. To support a new workload, you bought another server, installed an operating system, and deployed it. There were numerous issues with this plan.

Inefficient Resource AllocationThere was no centralized management of CPU and memory. Each server was independent and had its own dedicated resources that could not be shared. This led to one of two choices:

◆ The simplistic approach was to allocate servers with a fixed amount of CPU and RAM, giving wide berth for future growth. This strategy meant that resources were largely underutilized. For example, servers on average used less than 20 percent of their CPU.

◆ The alternative was to micromanage the resources per machine. Although compute resources were better utilized, the administrator’s time was not. Spikes in usage some-times created emergency situations with applications failing due to a lack CPU or memory.

The Long Road to ProvisioningRolling out a new server involved numerous teams: the infrastructure team would install the server; the network team would allocate an IP subnet, create a new VLAN, and configure routing; the server team would install the operating system and update it; the database team would establish a database for the workload, and the team of developers would load their applications; and finally, the security team would modify the firewall configuration to control access to and from the server (see Figure 1.3). This process repeated for every new workload.

Infra

structure

Secu

rity

Database

Serv

er

Network

Provisioning

Figure 1.3 traditional provisioning involves numerous teams and is time-consuming.

4 | CHAPTER 1 AbstrActing network And security

storage network firewall

PhysicalServer 1

Physical Environment

App

OS

PhysicalServer 2

App

OS

PhysicalServer 3

App

OS

PhysicalServer 4

App

OS

Figure 1.4 The move to company- built data centers

The time to fully provision a new workload, from the moment the server was purchased to the point where the application was ready to use, could often take months, greatly impacting business agility. Hand in hand with the slow rollouts was cost. When dealing entirely in the physical realm with hardware, it’s almost impossible to automate the process. Many teams had to be directly involved and, typically, the process could not move forward until the tasks of the previous team had been completed.

Data Centers Come of AgeAs companies grew, many reached a point where colocation was no longer cost-effective due to the amount of rented space required, and they built out their own data centers. Some organiza-tions were unable to take advantage of colocation at all due to compliance regulations, and they built their own data centers for this reason (see Figure 1.4).

Data Center WorkloadsA typical data center would consist of the physical servers, each with its own operating system connected to network services.

Rather than relying on leveraging a lot of local disks for permanent storage, most enterprises liked having their data all in one place, managed by a storage team. Centralized storage services made it easier to increase data durability through replication and to enhance reliability with backup and restore options that did not rely solely on tape backups. The storage team would carve out a Logical Unit of space, a LUN, for each operating system.

To control access, firewall services would protect the applications and data.

dAtA centers come of Age | 5

Workloads Won’t Stay PutHaving centralized resources and services only solved part of a problem. Although being in one place made them easier to control, so much still could only be accomplished manually by personnel. Automation allows the organization to be much more agile; to be able to quickly react when conditions change. Data centers during the ’90s lacked that agility.

Consider this analogy. Imagine you are the civil engineer for what will someday be Manhattan, New York. You design the layout for the roads. Going from design to a fully func-tional road will take considerable time and resources, but an even greater issue is looming in the future. The grid design for Manhattan was developed in 1811 (see Figure 1.5). The design supported the 95,000 residents of the time and took into consideration growth, but not enough to cover the 3.1 million people who work and live there now. The point is that trying to alleviate traffic congestion in New York is very difficult because we lack the ability to move the roads or to move the traffic without making the problem worse. Any time we are dealing with the physical world, we lack agility.

Figure 1.5 manhattan city grid designed in 1811

6 | CHAPTER 1 AbstrActing network And security

HARDWAREHARDWARE

OPERATING SYSTEM VIRTUALIZATION LAYER

APPLICATIONSOS

APP

OS

APP

OS

APP

Figure 1.6 The hypervisor is a virtualization layer decoupling software from the underly-ing hardware.

The data centers of the ’90s were heavily reliant on dedicated physical devices. If congestion occurred in one part of the data center, it was possible that a given workload could be moved to an area of less contention, but it was about as easy as trying to move that city road. These data center management tasks had to be done manually, and during the transition, traffic was negatively impacted.

VMwareIn 2005, VMware launched VMware Infrastructure 3, which became the catalyst for VMware’s move into the data center. It changed the paradigm for how physical servers and operating systems coexist. Prior to 2005, there was a 1:1 relationship: one server, one operating system.

VirtualizationVMware created a hypervisor (what we now refer to as ESXi) that enabled installing multiple operating systems on a single physical server (see Figure 1.6). By creating an abstraction layer, the operating systems no longer had to have direct knowledge of the underlying compute services, the CPU and memory.

The separate operating systems are what we now call virtual machines. The problem of trying to decide between provisioning simplicity and micromanaging resources immediately disap-peared. Each virtual machine has access to a pool of CPU and memory resources via the abstrac-tion layer, and each is given a reserved slice. Making changes to the amounts allocated to a virtual machine is something configured in software.

What Is Happening in There?Virtualization decoupled the software from the hardware. On the software side, you had the operating system and the application; on the hardware side, you had the compute resources. This bifurcation of physical and software meant that on the software side, we finally had agility instead of being tied to the railroad tracks of the physical environment (see Figure 1.7).

Consider the analogy of a physical three-drawer metal filing cabinet vs. a Documents folder on your laptop (Figure 1.8). They may both contain the same data, but if the task is to send all your records to your lawyer, sending or copying the contents of the papers within the metal filing cabinet is a giant chore. In comparison, sending the files from your Windows Documents folder may take a few clicks and 45 seconds out of your day.

VmwAre | 7

The point is we can easily move things in software. It is a virtual space. Moving things in physical space takes vastly more effort, time, and almost always, more money. VMware’s decoupling of the two opened a whole new world of possibilities.

App App App

Operating System

VMware Virtualization Layer

CPU Memory NIC Disk

x86 Architecture

App App App

Operating System

Figure 1.7 Virtualization creates an abstraction layer that hides the complexities of the physical layer from the software.

Figure 1.8 Physically storing data

8 | CHAPTER 1 AbstrActing network And security

PortabilityA key VMware feature that really leveraged the decoupling of physical and software is vMotion (see Figure 1.9). With vMotion, we can move a running workload to a different physical host. But portability and the option to move workloads is only the first step. Once an entity is wholly contained in software, you can automate it.

Virtualize AwayvMotion works in concert with the Distributed Resource Scheduler (DRS). The DRS actively monitors the CPU and memory resource utilization for each virtual machine. If multiple virtual machines located on the same physical server spike in CPU or memory usage to the point where there is a contention for these resources, DRS can detect the issue and automatically leverage vMotion to migrate the virtual machine to a different server with less contention (see Figure 1.10).

VMware ESX

App

OS

App

OS

App

OS

VMware ESX

App

OS App

OS

App

OS

VMware ESX

Physical Servers

Resource Pool

App

OS

App

OS

App

OS

Figure 1.10 in the event of a physical host failing, the workload can be moved to other hosts in the cluster.

ESX Server ESX Server

vMotion Technology

HardwareHardware

App

OS

App

OSOSOSS OS

AppFigure 1.9 Vmware vmotion is a benefit that would not be possible without virtualization.