resources.docs.pivotal.ioprerequisites for a bring your own topology deployment to nsx-t data center...

TanzuKubernetesGridIntegratedEdition(TKGI)v1.8

Documentation

v1.8

Published:June30,2020

Copyright©2020VMware,Inc.AllRightsReserved.

Note:ThecontentsofthisPDFmayhavefallenoutofdate.Forcurrentdocumentation,seehttps://docs.pivotal.io/pks/1-8

https://docs.pivotal.io/pks/1-8

VMwareTanzuKubernetesGridIntegratedEdition

In this topic

Overview

WhatTanzuKubernetesGridIntegratedEditionAddstoKubernetes

Features

TanzuKubernetesGridIntegratedEditionPrerequisites

Page last updated:

VMwareTanzuKubernetesGridIntegratedEdition(TKGI)enablesoperatorstoprovision,operate,andmanageenterprise-gradeKubernetesclustersusingBOSHandOpsManager.

Overview

TanzuKubernetesGridIntegratedEditiondeploysKubernetestoBOSH andOps Manager ,andusestheOn-Demand Brokertodynamicallyinstantiate,deploy,andmanagehighly-availableKubernetesclusterson-premisesoronapubliccloud.

AfteroperatorsinstallTKGI,developerscanusetheTKGICommandLineInterface(TKGICLI)toprovisionKubernetesclusters,andruncontainer-basedworkloadsontheclusterswiththeKubernetesCLI,kubectl.

OperatorsinstallTKGIasatileontheOpsManagerInstallationDashboard,orfromtheTKGIManagementConsoleonvSphere.

YoucanrunTKGIstandaloneoralongsideVMwareTanzuApplicationServiceforVMsonOpsManager.

WhatTanzuKubernetesGridIntegratedEditionAddstoKubernetes

ThefollowingtabledetailsthefeaturesthatTanzuKubernetesGridIntegratedEditionaddstotheKubernetesplatform.

Feature Included inK8sIncluded in Tanzu Kubernetes Grid IntegratedEdition

Singletenantingress ✓ ✓

Securemulti-tenantingress ✓

Statefulsetsofpods ✓ ✓

Multi-containerpods ✓ ✓

Rollingupgradestopods ✓ ✓

Rollingupgradestoclusterinfrastructure ✓

Podscalingandhighavailability ✓ ✓

Clusterprovisioningandscaling ✓

MonitoringandrecoveryofclusterVMsandprocesses

✓

Note:Asofv1.8,EnterprisePKShasbeenrenamedtoVMwareTanzuKubernetesGridIntegratedEdition.Somescreenshotsinthisdocumentationdonotyetreflectthechange.

Copyright©2020VMware,Inc.AllRightsReserved. 2 1.8

https://bosh.iohttps://docs.pivotal.io/platformhttps://docs.pivotal.io/svc-sdk/odb/index.html

Persistentdisks ✓ ✓

Securecontainerregistry ✓

Embedded,hardenedoperatingsystem ✓

Features

TanzuKubernetesGridIntegratedEditionhasthefollowingfeatures:

Kubernetes compatibility:ConstantcompatibilitywithcurrentstablereleaseofKubernetes

Production-ready:Highlyavailablefromapplicationstoinfrastructure,withnosinglepointsoffailure

BOSH advantages:Built-inhealthchecks,scaling,auto-healingandrollingupgrades

Fully automated operations:Fullyautomateddeploy,scale,patch,andupgradeexperience

Multi-cloud:Consistentoperationalexperienceacrossmultipleclouds

TanzuKubernetesGridIntegratedEditionPrerequisites

ForinformationabouttheresourcerequirementsforinstallingTanzuKubernetesGridIntegratedEdition,seethetopicthatcorrespondstoyourcloudprovider:

vSphere Prerequisites and Resource Requirements

vSphere with NSX-T Version RequirementsandHardware Requirements for Tanzu Kubernetes Grid Integrated Edition onvSphere with NSX-T

GCP Prerequisites and Resource Requirements

AWS Prerequisites and Resource Requirements

Azure Prerequisites and Resource Requirements

[email protected].


mailto:[email protected]

ReleaseNotes

In this topic

TKGIv1.8.0ProductSnapshot

UpgradePath

Features

BugFixes

KnownIssues

TKGIManagementConsole1.8.0Features

ProductSnapshot

UpgradePath

KnownIssues

Page last updated:

ThistopiccontainsreleasenotesforTanzuKubernetesGridIntegratedEdition(TKGI)v1.8.

TKGIv1.8.0

Release Date:June30,2020

ProductSnapshot

Release Details

Version v1.8.0

Releasedate June30,2020

Component Version

Kubernetes v1.17.5

Docker v19.03.5

On-DemandBroker v0.38.0

CoreDNS v1.6.2

NCP v3.0.1

UAA v74.5.15


warning:BeforeinstallingorupgradingtoTanzuKubernetesGridIntegratedEditionv1.8,reviewtheBreaking Changesbelow.


PerconaXtraDBCluster(PXC) v0.22.0

MetricsServer v0.3.6

etcd v3.4.3

kubo-release v1.8.0

Compatibilities Versions

OpsManager

AWS,Azure,GCP:SeeVMware Tanzu Network vSphere v7.0:OpsManagerv2.9.3+vSphere v6.7orv6.5:OpsManagerv2.9.3+,v2.8.2+,v2.7.15+

vSphere SeeVMware Product Interoperability Matrices

NSX-T v3.0 ,v2.5.1,v2.5.0

Xenialstemcells SeeVMware Tanzu Network

WindowsstemcellsvSphere v7.0:v2019.15vSphere v6.7orv6.5:v2019.15andlater

Harbor v2.0,v1.10.3

CNSforvSphere v1.0.2

BackupandRestoreSDK v1.18.0

ExcludingVCF4;seeVCF 4 and Converged VDS v7 Not Supported in TKGI v1.8.

TKGIsupportsNSX-Tv3.0asabetaintegration.UpgradingNSX-Ttov3.0isnotrecommendedforproductionorlarge-scaleTKGIenvironments.FormoreinformationaboutNSX-Tv3.0support,seeNSX-T v3.0 Compatibilitybelow.

SeeTKGI v1.8 With NSX-T and NCP v3.0.1 Not Compatible With Xenial Stemcells 621.76 and Later.

UpgradePath

ThesupportedupgradepathstoTanzuKubernetesGridIntegratedEditionv1.8.0arefromEnterprisePKSv1.7.0andlaterpatches.

Features

ThissectiondescribesnewfeaturesandchangesinVMwareTanzuKubernetesGridIntegratedEditionv1.8.0.

EnterprisePKSRenamedtoTanzuKubernetesGridIntegratedEdition

EnterprisePKShasbeenrenamedtoTanzuKubernetesGridIntegratedEdition(TKGI).

Whathaschanged:

TheTanzuKubernetesGridIntegratedEditionv1.8tileusesthenewname.

TanzuKubernetesGridIntegratedEditionv1.8includestwodownloadsoftheCLI,theTKGICLIandPKSCLI.SeePKS CLIRenamed to TKGI CLIbelow.

Whathasnotchanged:

Internalcomponentscontinuetousetheoldnameanditsalternatives,suchas PKS , pks ,and pivotal-container-service .Thisincludes,butisnotlimitedto,BOSHnames,UAAroles,andtextstringscontainingtheproductnameinTKGIcomponentsand

*

†

‡

*

†

‡


https://network.pivotal.io/products/pivotal-container-servicehttps://www.vmware.com/resources/compatibility/sim/interop_matrix.php#interop&356=&175=&1=https://network.pivotal.io/products/pivotal-container-service

TKGI-provisionedclusters.

IfyouintendtocontinueusingthePKSCLIinTKGIv1.8,noactionisrequired.However,futurereleasesofTKGIwilldeprecateandremovethePKSCLI.

PKSCLIRenamedtoTKGICLI

Tosupporttheproduct name change,TanzuKubernetesGridIntegratedEditionv1.8isdistributedwithaTKGICLIinadditiontoaPKSCLI.

BothCLIsworkidenticallyandacceptthesamecommandsandarguments.TorunaTKGICLIcommand,substitute tkgi whereyoupreviouslyused pks .Formoreinformation,seeTKGI CLI .

TodownloadtheTKGICLIorthePKSCLI,seeVMware Tanzu Network .

vSpherev7Compatibility

TKGIv1.8canrunonvSpherev7.

NSX-Tv3.0Compatibility

OnvSphere,TKGIcanrunwithNSX-Tv3.0containernetworking.

TKGIControlPlaneandAPI

TheTKGIAPIVMnolongerstoresacopyofthecontrolplanedatabasethatthev1.7upgrademigratedtotheDatabaseVM.ThisdeletionfreesinternalmemoryintheTKGIAPIVM.Asaresult,usersmaynoticeimprovedcontrolplaneperformance.

ThePKS 1.7.x Upgrade - MySQL Clone erranderrandhasbeenremovedfromtheTKGItileErrandspane.

KubernetesControlPlane

OnAzure,TKGIsupportsdisablingthecreationofadefaultoutboundSNATruleforclusters.SeeKubernetes Cloud ProviderforhowtodisablethedefaultSNATrule.

TKGIMonitoringandLogging

AllTKGIcomponentsuseTLSv1.2withstrongciphers,includingthe metrics-server component. sslscan onametrics-server overport 443 nowreportsonlyTLSv1.2+ciphers.

CustomerExperienceImprovementProgram(CEIP)andTelemetry

ThelegacyTelemetryDBhasbeenremovedfromtheTKGIDatabase.

ComponentUpdates

Thefollowingcomponentshavebeenupdated:

warning:TKGIsupportsNSX-Tv3.0asabetaintegration.IntermittentupgradefailuresandscaleproblemsmayoccurifyouupgradetoNSX-Tv3.0.UpgradingyourNSX-Tenvironmenttov3.0inaproductionorlarge-scaledeploymentisnotrecommendeduntilapatchresolvingtheseissueshasbeenreleased.


https://docs-pcf-staging.cfapps.io/tkgi/1-8/cli/https://network.pivotal.io/products/pivotal-container-service/

BumpsKubernetestov1.17.5.

BumpsNCPtov3.0.1.

BumpsUAAtov74.5.15.

BugFixes

TKGIv1.8.0includesthefollowingbugfixes:

tkgi tasks returnsvalidoutputforallclusters.

tkgi upgrade-cluster errandnolongertimesoutwhenstopping dockerd processes.

tkgi get-credentials worksforclustersthathavenotbeenupgraded.

tkgi update-cluster retainsthe compute_profile valuewhenchangingsettingsforclusterscreatedwithaComputeProfile.

KnownIssues

TKGIv1.8.0hasthefollowingknownissues:

VCF4andConvergedVDSv7NotSupportedinTKGIv1.8

ForinstallationsonvSpherev7withNSX-Tv3.0integration,TKGIv1.8supportsonlyN-VDSforNSX-Ttraffic.Itdoesnotsupport:

ConvergedVirtualDistributedSwitch(C-VDS)v7,whichletsyouusethesameVDSforbothvSphereandNSX-Ttraffic

VMwareCloudFoundation(VCF)v4.x,whichusesonlyVDSmodewithNSX-Tv3.0

Formoreinformation,seeConfigure vSphere Networking for ESXi HostsinInstallingandConfiguringNSX-TDataCenterv3.0forTanzuKubernetesGridIntegratedEdition.

TKGIv1.8WithNSX-TandNCPv3.0.1NotCompatibleWithXenialStemcells621.76andLater

TKGIwithNSX-TandNCPv3.0.1iscompatiblewithLinuxUbuntuXenialstemcellv621.75,butnotwithstemcellversionsv621.76andlater.

TKGIv1.8(Windows)onvSphereNotCompatiblewithOpsManagerv2.9

TKGIv1.8installationswithWindowsworker-basedKubernetesclustersonvSphere(Flannel)arenotcompatiblewithOpsManagerv2.9.IfyoudonotintendtodeployandrunWindowsworker-basedKubernetesclusters,youcanuseOpsManagerv2.9withTKGIv1.8.

ForOpsManagercompatibilityinformation,seeVMware Tanzu Network .

PingingWindowsWorkersDoesNotWork

TKGI-provisionedWindowsworkersinheritaKuberneteslimitationthatpreventsoutboundICMPcommunicationfromworkers.Asaresult,pingingWindowsworkersdoesnotwork.

Forinformationaboutthislimitation,seeLimitations > Networking intheWindowsinKubernetesdocumentation.

TMCIntegrationNotSupportedonGCP

TKGIonGoogleCloudPlatform(GCP)doesnotsupportTanzuMissionControlintegration,whichisconfiguredintheTanzu


https://network.pivotal.io/products/pivotal-container-servicehttps://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#networking-1

Kubernetes Grid Integrated Editiontile>theTanzu Mission Control (Experimental)pane.

IfyouintendtorunTKGIv1.8onGCP,skipthispanewhenconfiguringtheTanzuKubernetesGridIntegratedEditiontile.

502BadGatewayAfterOIDCLogin

Symptom

Youexperiencea“502BadGateway”errorfromtheNSXloadbalancerafteryoulogintoOIDC.

Explanation

AlargeresponseheaderhasexceededyourNSX-Tloadbalancermaximumresponseheadersize.Thedefaultmaximumresponseheadersizeis10,240charactersandshouldberesizedto50,000.

Workaround

Ifyouexperiencethisissue,manuallyreconfigureyourNSX-T request_header_size and response_header_size to50,000characters.ForinformationaboutconfiguringNSX-Tdefaultheadersizes,seeOIDC Response Header Overflow intheKnowledgeBase.

OnePlanIDLongerthanOtherPlanIDs

Symptom

OneofyourplanIDsisonecharacterlongerthanyourotherplanIDs.

Explanation

InTKGI,eachplanhasauniqueplanID.AplanIDisnormallyaUUIDconsistingof32alphanumericcharactersand4hyphens.However,thePlan 4IDconsistsof33alphanumericcharactersand4hyphens.

Solution

YoucansafelyconfigureandusePlan 4.ThelengthofthePlan 4IDdoesnotaffectthefunctionalityofPlan 4clusters.

IfyourequireallplanIDstohaveidenticallength,donotactivateorusePlan 4.

NSX-TPre-CheckErrandFailsDuetoEdgeNodeConfiguration

Symptom

YouhaveconfiguredyourNSX-TEdgeNodeVMas medium size,andtheNSX-TPre-CheckErrandfailswiththefollowingerror:“ERROR:NSX-TPrecheckfailedduetoEdgeNode…noofcpucoresislessthan8”.

Explanation

TheNSX-TPre-CheckErrandiserroneouslyreturningthe“cpucoresislessthan8”error.

Solution

YoucansafelyconfigureyourNSX-TEdgeNodeVMsas medium sizeandignoretheerror.


https://community.pivotal.io/s/article/OIDC-Response-Header-overflow

DifficultyChangingProxyforWindowsWorkers

YoumustconfigureaglobalproxyintheTanzuKubernetesGridIntegratedEditiontile>NetworkingpanebeforeyoucreateanyWindowsworkersthatusetheproxy.

YoucannotchangetheproxyconfigurationforWindowsworkersinanexistingcluster.

CharacterLimitationsinHTTPProxyPassword

ForvSpherewithNSX-T,theHTTPProxypasswordfielddoesnotsupportthefollowingspecialcharacters: & or ; .

TKGIManagementConsole1.8.0

Release Date:June30,2020

Features

TanzuKubernetesGridIntegratedEditionManagementConsolev1.8.0updatesinclude:

SupportforvSphere7

SupportforNSX-T3.0

RebrandingtoTanzuKubernetesGridIntegratedEditionManagementConsole

SpecifyFQDNfortheOpsManagerVMduringupgrade

ProductSnapshot

Element Details

Version v1.8.0

Releasedate June30,2020

InstalledTanzuKubernetesGridIntegratedEditionversion v1.8.0

InstalledOpsManagerversion v2.9.0

InstalledKubernetesversion v1.17.5

CompatibleNSX-Tversions v3.0,v2.5.1,v2.5.0

InstalledHarborRegistryversion v2.0,v1.10.3

Windowsstemcells v2019.20andlater

UpgradePath

ThesupportedupgradepathtoTanzuKubernetesGridIntegratedEditionManagementConsolev1.8.0isfromTanzuKubernetesGridIntegratedEditionv1.7.0andlater.

KnownIssues

Note:TanzuKubernetesGridIntegratedEditionManagementConsoleprovidesanopinionatedinstallationofTKGI.ThesupportedversionsmaydifferfromorbemorelimitedthanwhatisgenerallysupportedbyTKGI.


TheTanzuKubernetesGridIntegratedEditionManagementConsolev1.8.0hasthefollowingknownissues:

vSphereHAcausesManagementConsoleovfenvDataCorruption

Symptom

IfyouenablevSphereHAonacluster,iftheTKGIManagementConsoleapplianceVMisrunningonahostinthatcluster,andifthehostreboots,vSphereHArecreatesanewTKGIManagementConsoleapplianceVMonanotherhostinthecluster.DuetoanissuewithvSphereHA,the ovfenv dataforthenewlycreatedapplianceVMiscorruptedandthenewapplianceVMdoesnotbootupwiththecorrectnetworkconfiguration.

Workaround

InthevSphereClient,right-clicktheapplianceVMandselectPower>Shut Down Guest OS.

Right-clicktheapplianceagainandselectEditSettings.

SelectVM OptionsandclickOK.

VerifyunderRecentTasksthata Reconfigure virtual machine taskhasrunontheapplianceVM.

PowerontheapplianceVM.

Base64encodedfileargumentsarenotdecodedinKubernetesprofiles

Symptom

SomefileargumentsinKubernetesprofilesarebase64encoded.WhenthemanagementconsoledisplaystheKubernetesprofile,somefileargumentsarenotdecoded.

Workaround

Run echo"$content"|base64--decode

Networkprofilesnotimmediatelyselectable

Symptom

IfyoucreatenetworkprofilesandthentrytoapplythemintheCreateClusterpage,thenewprofilesarenotavailableforselection.

Workaround

Logoutofthemanagementconsoleandlogbackinagain.

Real-TimeIPinformationnotdisplayedfornetworkprofiles

Symptom

Intheclustersummarypage,onlydefaultIPpool,podIPblock,nodeIPblockvaluesaredisplayed,ratherthanthereal-timevaluesfromtheassociatednetworkprofile.

Workaround


None

[email protected].



TanzuKubernetesGridIntegratedEditionConcepts

Page last updated:

ThistopicdescribesVMwareTanzuKubernetesGridIntegratedEditionconcepts.Seethefollowingsections:

Tanzu Kubernetes Grid Integrated Edition Architecture

About Tanzu Kubernetes Grid Integrated Edition Upgrades

TKGI API Authentication

Load Balancers in Tanzu Kubernetes Grid Integrated Edition

VM Sizing for Tanzu Kubernetes Grid Integrated Edition Clusters

Telemetry

Sink Architecture in Tanzu Kubernetes Grid Integrated Edition

[email protected].




TanzuKubernetesGridIntegratedEditionArchitecture

In this topic

TanzuKubernetesGridIntegratedEditionOverview

TKGIControlPlaneOverviewTKGIAPIVM

TKGIDatabaseVM

AvailabilityZones

WindowsWorker-BasedKubernetesCluster(Beta)HighAvailability

Page last updated:

ThistopicdescribeshowVMwareTanzuKubernetesGridIntegratedEditionmanagesthedeploymentofKubernetesclusters.

TanzuKubernetesGridIntegratedEditionOverview

AnTanzuKubernetesGridIntegratedEditionenvironmentconsistsofaTKGIControlPlaneandoneormoreworkloadclusters.

TanzuKubernetesGridIntegratedEditionadministratorsusetheTKGIControlPlanetodeployandmanageKubernetesclusters.Theworkloadclustersruntheappspushedbydevelopers.

ThefollowingillustratestheinteractionbetweenTanzuKubernetesGridIntegratedEditioncomponents:



AdministratorsaccesstheTKGIControlPlanethroughtheTKGICommandLineInterface(TKGICLI)installedontheirlocalworkstations.

WithintheTKGIControlPlanetheTKGIAPIandTKGIBrokeruseBOSHtoexecutetherequestedclustermanagementfunctions.ForinformationabouttheTKGIControlPlane,seeTKGI Control Plane Overviewbelow.ForinstructionsoninstallingtheTKGICLI,seeInstalling the TKGI CLI.

KubernetesdeploysandmanagesworkloadsonKubernetesclusters.AdministratorsusetheKubernetesCLI, kubectl ,todirectKubernetesfromtheirlocalworkstations.Forinformationabout kubectl ,seeOverview of kubectl intheKubernetesdocumentation.

TKGIControlPlaneOverview

TheTKGIControlPlanemanagesthelifecycleofKubernetesclustersdeployedusingTanzuKubernetesGridIntegratedEdition.

ThecontrolplaneprovidesthefollowingviatheTKGIAPI:

Viewclusterplans

Createclusters


https://kubernetes.io/docs/reference/kubectl/overview/

Viewinformationaboutclusters

Obtaincredentialstodeployworkloadstoclusters

Scaleclusters

Deleteclusters

CreateandmanagenetworkprofilesforVMwareNSX-T

Inaddition,theTKGIControlPlanecanupgradeallexistingclustersusingtheUpgrade all clustersBOSHerrand.Formoreinformation,seeUpgrade Kubernetes ClustersinUpgradingTanzuKubernetesGridIntegratedEdition(FlannelNetworking).

TKGIControlPlaneishostedonapairofVMs:

TheTKGI API VMhostsclustermanagementservices.

TheTKGI Database VMstoresclustermanagementdata.

TKGIAPIVM

TheTKGIAPIVMhoststhefollowingservices:

UserAccountandAuthentication(UAA)

TKGIAPI

TKGIBroker

BillingandTelemetry

ThefollowingsectionsdescribeUAA,TKGIAPI,andTKGIBrokerservices,theprimaryserviceshostedontheTKGIAPIVM.

UAA

WhenauserlogsintoorlogsoutoftheTKGIAPIthroughtheTKGICLI,theTKGICLIcommunicateswithUAAtoauthenticatethem.TheTKGIAPIpermitsonlyauthenticateduserstomanageKubernetesclusters.Formoreinformationaboutauthenticating,seeTKGI APIAuthentication.

UAAmustbeconfiguredwiththeappropriateusersanduserpermissions.Formoreinformation,seeManaging Tanzu KubernetesGrid Integrated Edition Users with UAA.

TKGIAPI

ThroughtheTKGICLI,usersinstructtheTKGIAPIservicetodeploy,scaleup,anddeleteKubernetesclustersaswellasshowclusterdetailsandplans.TheTKGIAPIcanalsowriteKubernetesclustercredentialstoalocalkubeconfigfile,whichenablesuserstoconnecttoaclusterthrough kubectl .

OnAWS,GCP,andvSpherewithoutNSX-TdeploymentstheTKGICLIcommunicateswiththeTKGIAPIwithinthecontrolplaneviatheTKGIAPILoadBalancer.OnvSpherewithNSX-TdeploymentstheTKGIAPIhostisaccessibleviaaDNATrule.ForinformationaboutenablingtheTKGIAPIonvSpherewithNSX-T,seetheShare the TKGI API EndpointsectioninInstallingTanzuKubernetesGridIntegratedEditiononvSpherewithNSX-TIntegration.

TheTKGIAPIsendsallclustermanagementrequests,exceptread-onlyrequests,totheTKGIBroker.

TKGIBroker


WhentheTKGIAPIreceivesarequesttomodifyaKubernetescluster,itinstructstheTKGIBrokertomaketherequestedchange.

TheTKGIBrokerconsistsofanOn-Demand Service Broker andaServiceAdapter.TheTKGIBrokergeneratesaBOSHmanifestandinstructstheBOSHDirectortodeployordeletetheKubernetescluster.

ForTanzuKubernetesGridIntegratedEditiondeploymentsonvSpherewithNSX-T,thereisanadditionalcomponent,theTanzuKubernetesGridIntegratedEditionNSX-TProxyBroker.TheTKGIAPIcommunicateswiththeTKGINSX-TProxyBroker,whichinturncommunicateswiththeNSXManagertoprovisiontheNodeNetworkingresources.TheTKGINSX-TProxyBrokerthenforwardstherequesttotheOn-DemandServiceBrokertodeploythecluster.

TKGIDatabaseVM

TheTKGIDatabaseVMhostsMySQL,proxy,andotherdata-relatedservices.Thesedata-relatedfunctionspersistTKGIControlPlanedataforthethefollowingservices:

TKGIAPI

UAA

Billing

Telemetry

AvailabilityZones

TanzuKubernetesGridIntegratedEditionusesAvailabilityZones(AZs)toprovidehighavailabilityforKubernetesclusterworkers.

WhenanoperatorcreatesPlansfordevelopers,theyassignAZstothePlans.AssigningmultipleAZstoaPlanallowsdeveloperstoprovidehigh-availabilityfortheirworkerclusters.Whenaclusterhasmorethanonenode,OpsManagerbalancesthosenodesacrosstheAvailabilityZonesassignedtothecluster.

Public-cloudIaaSessuchasAWSandAzureprovideAZsaspartoftheirservice.InvSpherewithNSX-T,youdefineandcreateAZsusingvCenterclustersandresourcepools.SeeStep 4: Create Availability ZonesinConfiguringBOSHDirectorwithNSX-TforTanzuKubernetesGridIntegratedEditionforhowtocreateAZsinNSX-T.

ForinstructionsonselectingAZsforyourTanzuKubernetesGridIntegratedEditionPlans,seePlansinInstallingTanzuKubernetesGridIntegratedEditiononvSpherewithNSX-T.

ForinstructionsonselectingtheAZfortheTanzuKubernetesGridIntegratedEditioncontrolplane,seeAssign AZs and NetworksinInstallingTanzuKubernetesGridIntegratedEditiononvSpherewithNSX-T.

WindowsWorker-BasedKubernetesCluster(Beta)HighAvailability

Windowsworker-basedcluster(beta)Linuxnodescanbeconfiguredineitherstandardorhighavailabilitymodes.

Instandardmode,asingleMaster/etcdnodeandasingleLinuxworkermanageacluster’sWindowsKubernetesVMs.

Inhighavailabilitymode,multipleMaster/etcdandLinuxworkernodesmanageacluster’sWindowsKubernetesVMs.

ThefollowingillustratestheinteractionbetweentheTanzuKubernetesGridIntegratedEditionManagementPlaneandWindowsworker-basedKubernetesclusters:


https://docs.pivotal.io/svc-sdk/odb/index.html

ToconfigureTanzuKubernetesGridIntegratedEditionWindowsworker-basedclustersforhighavailability,setthesefieldsinthePlanpaneasdescribedinPlansinConfiguringWindowsWorker-BasedKubernetesClusters(Beta):

Enable HA Linux workers

Master/ETCD Node Instances

Worker Node Instances

[email protected].



AboutTanzuKubernetesGridIntegratedEditionUpgrades

In this topic

Overview

DecidingBetweenFullandTwo-PhaseUpgrade

WhatHappensDuringFullTKGIandTKGIControlPlaneUpgrades

WhatHappensDuringClusterUpgrades

Page last updated:

ThistopicprovidesconceptualinformationaboutTanzuKubernetesGridIntegratedEditionupgrades,includingupgradingtheTKGIcontrolplaneandTKGI-provisionedKubernetesclusters.

Forstep-by-stepinstructionsonupgradingTanzuKubernetesGridIntegratedEditionandTKGI-provisionedKubernetesclusters,see:

Upgrading Tanzu Kubernetes Grid Integrated Edition (Flannel Networking)

Upgrading Tanzu Kubernetes Grid Integrated Edition (NSX-T Networking)

Upgrading Clusters

Overview

AnTanzuKubernetesGridIntegratedEditionupgrademodifiestheversionofTanzuKubernetesGridIntegratedEdition,forexample,fromv1.7.xtov1.8.0orfromv1.8.0tov1.8.1.

Bydefault,TanzuKubernetesGridIntegratedEditionissettoperformafullupgrade,whichupgradesboththeTKGIcontrolplaneandallTKGI-provisionedKubernetesclusters.

However,youcanchoosetoupgradeTanzuKubernetesGridIntegratedEditionintwophasesbyupgradingtheTKGIcontrolplanefirstandthenupgradingyourTKGI-provisionedKubernetesclusterslater.

BoththefullupgradeandtheTKGIcontrolplaneupgradeareperformedthroughtheTanzuKubernetesGridIntegratedEditiontileonly.WhenupgradingTKGI-provisionedKubernetesclusters,youcanuseeithertheTanzuKubernetesGridIntegratedEditiontileortheTKGICLI.Seethetablebelow.

Upgrade typeUpgrade method

TKGI Tile TKGI CLI

FullTKGIupgrade ✔ ✖

TKGIcontrolplaneonly ✔ ✖

Kubernetesclustersonly ✔ ✔

Typically,ifyouchoosetoupgradeTKGI-provisionedKubernetesclustersonly,youwillupgradethemthroughtheTKGICLI.



DecidingBetweenFullandTwo-PhaseUpgrade

WhendecidingwhethertoperformthedefaultfullupgradeortoupgradetheTKGIcontrolplaneandTKGI-provisionedKubernetesclustersseparately,consideryourorganizationneeds.

Forexample,ifyourorganizationrunsTKGI-provisionedKubernetesclustersinbothdevelopmentandproductionenvironmentsandyouwanttoupgradeonlyoneenvironmentfirst,youcanachieveyourgoalbyupgradingtheTKGIcontrolplaneandTKGI-provisionedKubernetesseparatelyinsteadofperformingafullupgrade.

ExamplesofotheradvantagesofupgradingTanzuKubernetesGridIntegratedEditionintwophasesinclude:

FasterTanzuKubernetesGridIntegratedEditiontileupgrades.IfyouhavealargenumberofclustersinyourTanzuKubernetesGridIntegratedEditiondeployment,performingafullupgradecansignificantlyincreasetheamountoftimerequiredtoupgradetheTanzuKubernetesGridIntegratedEditiontile.

Moregranularcontroloverclusterupgrades.Inadditiontoenablingyoutoupgradesubsetsofclusters,theTKGICLIsupportsupgradingeachclusterindividually.

Notamonolithicupgrade.Thishelpsisolatetherootcauseofanerrorwhentroubleshootingupgrades.Forexample,whenacluster-relatedupgradeerroroccursduringafullupgrade,theentireTanzuKubernetesGridIntegratedEditiontileupgrademayfail.

WhatHappensDuringFullTKGIandTKGIControlPlaneUpgrades

YoucanperformfullTKGIupgradesandTKGIcontrolplaneupgradesonlythroughtheTanzuKubernetesGridIntegratedEditiontile.

AfteryouaddanewTanzuKubernetesGridIntegratedEditiontileversiontoyourstagingareaontheOpsManagerInstallationDashboard,OpsManagerautomaticallymigratesyourconfigurationsettingsintothenewtileversion.

Formoreinformation,see:

Full TKGI Upgrades

TKGI Control Plane Upgrades

FullTKGIUpgrades

Duringafull TKGI upgrade,theTanzuKubernetesGridIntegratedEditiontiledoesthefollowing:

1. UpgradestheTKGIcontrolplane,whichincludestheTKGIAPIandUAAserversandtheTKGIdatabase.ThiscontrolplaneupgradecausestemporaryoutagesasdescribedinControl Plane Outagesbelow.

2. UpgradesTKGI-provisionedKubernetesclusters.

UpgradingTKGI-provisionedKubernetesclustersiscontrolledbytheUpgrade all clusters errandintheTanzuKubernetesGridIntegratedEditiontile.Theclusterupgradeprocessrecreatesallclusters,whichmaycauseclusteroutages.Formoreinformation,seeWhatHappens During Cluster Upgradesbelow.

warning:IfyoudisablethedefaultfullupgradeandupgradeonlytheTKGIcontrolplane,youmustupgradeallyourTKGI-provisionedKubernetesclustersbeforethenextTanzuKubernetesGridIntegratedEditiontileupgrade.DisablingthedefaultfullupgradeandupgradingonlytheTKGIcontrolplanecausetheTKGIversiontaggedinyourKubernetesclusterstofallbehindtheTanzuKubernetesGridIntegratedEditiontileversion.IfyourTKGI-provisionedKubernetesclustersfallmorethanoneversionbehindthetile,TanzuKubernetesGridIntegratedEditioncannotupgradetheclusters.


TKGIControlPlaneUpgrades

WhenupgradingtheTKGI control plane only,theTanzuKubernetesGridIntegratedEditiontilefollowstheprocessdescribedinFull TKGI Upgradesabove,step1.ItdoesnotupgradeTKGI-provisionedKubernetesclusters,step2.

ControlPlaneOutages

UpgradingtheTanzuKubernetesGridIntegratedEditioncontrolplanetemporarilyinterruptsthefollowing:

LoggingintotheTKGICLIandusingall tkgi commands

UsingtheTKGIAPItoretrieveinformationaboutclusters

UsingtheTKGIAPItocreateanddeleteclusters

UsingtheTKGIAPItoresizeclusters

TheseoutagesdonotaffecttheKubernetesclustersthemselves.DuringaTKGIcontrolplaneupgrade,youcanstillinteractwithclustersandtheirworkloadsusingtheKubernetesCommandLineInterface, kubectl .

FormoreinformationabouttheTKGIcontrolplane,seeTKGI Control Plane OverviewinTanzuKubernetesGridIntegratedEditionArchitecture.

CanaryInstances

TheTanzuKubernetesGridIntegratedEditiontileisaBOSHdeployment.

BOSH-deployedproductscansetanumberofcanaryinstancestoupgradefirst,beforetherestofthedeploymentVMs.BOSHcontinuestheupgradeonlyifthecanaryinstanceupgradesucceeds.Ifthecanaryinstanceencountersanerror,theupgradestopsrunningandotherVMsarenotaffected.

TheTanzuKubernetesGridIntegratedEditiontileusesonecanaryinstancewhendeployingorupgradingTanzuKubernetesGridIntegratedEdition.

WhatHappensDuringClusterUpgrades

UpgradingTKGI-provisionedKubernetesclustersupdatestheirKubernetesversiontotheversionincludedwiththeTanzuKubernetesGridIntegratedEditiontile.ItalsoupdatestheTKGIversiontaggedinyourclusterstotheTanzuKubernetesGridIntegratedEditiontileversion.

YoucanupgradeTKGI-provisionedKubernetesclusterseitherthroughtheTanzuKubernetesGridIntegratedEditiontileortheTKGICLI.Seethetablebelow.

This method Upgrades

TheUpgrade all clusters errandintheTanzu Kubernetes Grid Integrated Editiontile>Errands

Allclusters.Clustersareupgradedserially.

tkgi upgrade-cluster Onecluster.

tkgi upgrade-clusters Multipleclusters.Clustersareupgradedseriallyorinparallel.

DuringanupgradeofTKGI-provisionedclusters,TanzuKubernetesGridIntegratedEditionrecreatesyourclusters.Thisincludesthefollowingstagesforeachclusteryouupgrade:


1. Masternodesarerecreated.

2. Workernodesarerecreated.

Dependingonyourclusterconfiguration,theserecreationsmaycauseMaster Nodes OutageorWorker Nodes Outageasdescribedbelow.

MasterNodesOutage

WhenTanzuKubernetesGridIntegratedEditionupgradesasingle-mastercluster,youcannotinteractwithyourcluster,use kubectl ,orpushnewworkloads.

Toavoidthislossoffunctionality,VMwarerecommendsusingmulti-masterclusters.

WorkerNodesOutage

WhenTanzuKubernetesGridIntegratedEditionupgradesaworkernode,thenodestopsrunningcontainers.Ifyourworkloadsrunonasinglenode,theywillexperiencedowntime.

Toavoiddowntimeforstatelessworkloads,VMwarerecommendsusingatleastoneworkernodeperavailabilityzone(AZ).Forstatefulworkloads,VMwarerecommendsusingaminimumoftwoworkernodesperAZ.

[email protected].

Note:WhentheUpgrade all clusters errandisenabledintheTanzuKubernetesGridIntegratedEditiontile,updatingthetilewithanewLinuxorWindowsstemcellrollseveryLinuxorWindowsVMineachKubernetescluster.ThisautomaticrollingensuresthatallyourVMsarepatched.Toavoidworkloaddowntime,usetheresourceconfigurationrecommendedinMasterNodes OutageandWorker Nodes OutageaboveandinMaintaining Workload Uptime.



TKGIAPIAuthentication

In this topic

AuthenticationofTKGIAPIRequests

RoutingtotheTKGIAPIVM

Page last updated:

ThistopicdescribeshowtheVMwareTanzuKubernetesGridIntegratedEditionAPIworkswithUserAccountandAuthentication(UAA)tomanageauthenticationandauthorizationinyourTanzuKubernetesGridIntegratedEditiondeployment.

AuthenticationofTKGIAPIRequests

BeforeuserscanloginandusetheTKGICLI,youmustconfigureTKGIAPIaccesswithUAA.Formoreinformation,seeManagingTanzu Kubernetes Grid Integrated Edition Users with UAAandLogging in to Tanzu Kubernetes Grid Integrated Edition.

YouusetheUAACommandLineInterface(UAAC)totargettheUAAserverandrequestanaccesstokenfortheUAAadminuser.Ifyourrequestissuccessful,theUAAserverreturnstheaccesstoken.TheUAAadminaccesstokenauthorizesyoutomakerequeststotheTKGIAPIusingtheTKGICLIandgrantclusteraccesstoneworexistingusers.

WhenauserwithclusteraccesslogsintotheTKGICLI,theCLIrequestsanaccesstokenfortheuserfromtheUAAserver.Iftherequestissuccessful,theUAAserverreturnsanaccesstokentotheTKGICLI.WhentheuserrunsTKGICLIcommands,forexample, tkgiclusters ,theCLIsendstherequesttotheTKGIAPIserverandincludestheuser’sUAAtoken.

TheTKGIAPIsendsarequesttotheUAAservertovalidatetheuser’stoken.IftheUAAserverconfirmsthatthetokenisvalid,theTKGIAPIusestheclusterinformationfromtheTKGIbrokertorespondtotherequest.Forexample,iftheuserruns tkgiclusters ,theCLIreturnsalistoftheclustersthattheuserisauthorizedtomanage.

RoutingtotheTKGIAPIVM

TheTKGIAPIserverandtheUAAserverusedifferentportnumbersontheAPIVM.Forexample,ifyourTKGIAPIdomainisapi.tkgi.example.com ,youcanreachyourTKGIAPIandUAAserversatthefollowingURLs:

Server URL

TKGIAPI api.tkgi.example.com:9021

UAA api.tkgi.example.com:8443

RefertoOps Manager>Tanzu Kubernetes Grid Integrated Edition tile>TKGI API>API Hostname (FQDN)foryourTKGIAPIdomain.

Loadbalancerimplementationsdifferbydeploymentenvironment.ForTanzuKubernetesGridIntegratedEditiondeploymentsonGCP,AWS,orvSpherewithoutNSX-T,youconfigurealoadbalancertoaccesstheTKGIAPIwhenyouinstalltheTanzuKubernetesGridIntegratedEditiontile.Forexample,seeConfiguring TKGI API Load Balancer.



ForoverviewinformationaboutloadbalancersinTanzuKubernetesGridIntegratedEdition,seeLoad Balancers in TanzuKubernetes Grid Integrated Edition Deployments without NSX-T.

[email protected].



LoadBalancersinTanzuKubernetesGridIntegratedEdition

In this topic

LoadBalancersinTanzuKubernetesGridIntegratedEditionDeploymentswithoutNSX-TAbouttheTKGIAPILoadBalancer

AboutKubernetesClusterLoadBalancers

AboutWorkloadLoadBalancers

LoadBalancersinTanzuKubernetesGridIntegratedEditionDeploymentsonvSpherewithNSX-TResizingLoadBalancers

Page last updated:

ThistopicdescribesthetypesofloadbalancersthatareusedinVMwareTanzuKubernetesGridIntegratedEditiondeployments.Loadbalancersdifferbythetypeofdeployment.

LoadBalancersinTanzuKubernetesGridIntegratedEditionDeploymentswithoutNSX-TForTanzuKubernetesGridIntegratedEditiondeploymentsonGCP,AWS,orvSpherewithoutNSX-T,youcanconfigureloadbalancersforthefollowing:

TKGI API:ConfiguringthisloadbalancerenablesyoutorunTKGICommandLineInterface(TKGICLI)commandsfromyourlocalworkstation.

Kubernetes Clusters:ConfiguringaloadbalancerforeachnewclusterenablesyoutorunKubernetesCLI(kubectl)commandsonthecluster.

Workloads:Configuringaloadbalancerforyourapplicationworkloadsenablesexternalaccesstotheservicesthatrunonyourcluster.

Thefollowingdiagram,applicabletoGCP,AWS,andvSpherewithoutNSX-T,showswhereeachoftheaboveloadbalancerscanbeusedwithinyourTanzuKubernetesGridIntegratedEditiondeployment.



IfyouuseeithervSpherewithoutNSX-TorGCP,youareexpectedtocreateyourownloadbalancerswithinyourcloudproviderconsole.Ifyourcloudproviderdoesnotofferloadbalancing,youcanuseanyexternalTCPorHTTPSloadbalancerofyourchoice.

AbouttheTKGIAPILoadBalancer

TheTKGIAPIloadbalancerenablesyoutoaccesstheTKGIAPIfromoutsidethenetworkonTanzuKubernetesGridIntegratedEditiondeploymentsonGCP,AWS,andonvSpherewithoutNSX-T.Forexample,configuringaloadbalancerfortheTKGIAPIenablesyoutorunTKGICLIcommandsfromyourlocalworkstation.

ForinformationaboutconfiguringtheTKGIAPIloadbalanceronvSpherewithoutNSX-T,seeConfiguring TKGI API Load Balancer.

AboutKubernetesClusterLoadBalancers

WhenyoucreateanTanzuKubernetesGridIntegratedEditionclusteronGCP,AWS,andonvSpherewithoutNSX-T,youmustconfigure


externalaccesstotheclusterbycreatinganexternalTCPorHTTPSloadbalancer.TheloadbalancerenablestheKubernetesCLItocommunicatewiththecluster.

Ifyoucreateaclusterinanon-productionenvironment,youcanchoosenottousealoadbalancer.Toenablekubectltoaccesstheclusterwithoutaloadbalancer,youcandooneofthefollowing:

CreateaDNSentrythatpointstothecluster’smasterVM.Forexample:

my-cluster.example.com A 10.0.0.5

Ontheworkstationwhereyourunkubectlcommands,addthemasterIPaddressofyourclusterand kubo.internal tothe/etc/hosts file.Forexample:

10.0.0.5 kubo.internal

Formoreinformationaboutconfiguringaclusterloadbalancer,seethefollowing:

Creating and Configuring a GCP Load Balancer for Tanzu Kubernetes Grid Integrated Edition Clusters

Creating and Configuring an AWS Load Balancer for Tanzu Kubernetes Grid Integrated Edition Clusters

Creating and Configuring an Azure Load Balancer for Tanzu Kubernetes Grid Integrated Edition Clusters

AboutWorkloadLoadBalancers

ToenableexternalaccesstoyourTanzuKubernetesGridIntegratedEditionapponGCP,AWS,andonvSpherewithoutNSX-T,youcaneithercreatealoadbalancerorexposeastaticportonyourworkload.

Forinformationaboutconfiguringaloadbalancerforyourappworkload,seeDeploying and Exposing Basic Linux Workloads.

IfyouuseAWS,youmustconfigureroutingintheAWSconsolebeforeyoucancreatealoadbalancerforyourworkload.Youmustcreateapublicsubnetineachavailabilityzone(AZ)whereyouaredeployingtheworkloadandtagthepublicsubnetwithyourcluster’suniqueidentifier.

SeetheAWS PrerequisitessectionofDeployingandExposingBasicLinuxWorkloadsbeforeyoucreateaworkloadloadbalancer.

DeployYourWorkloadLoadBalancerwithanIngressController

AKubernetesingresscontrollersitsbehindaloadbalancer,routingHTTPandHTTPSrequestsfromoutsidetheclustertoserviceswithinthecluster.Kubernetesingressresourcescanbeconfiguredtoloadbalancetraffic,provideexternallyreachableURLstoservices,andmanageotheraspectsofnetworktraffic.

IfyouaddaningresscontrollertoyourTanzuKubernetesGridIntegratedEditiondeployment,trafficroutingiscontrolledbytheingressresourcerulesyoudefine.VMwarerecommendsconfiguringTanzuKubernetesGridIntegratedEditiondeploymentswithbothaworkloadloadbalancerandaningresscontroller.

ThefollowingdiagramshowshowtheingressroutingcanbeusedwithinyourTanzuKubernetesGridIntegratedEditiondeployment.


TheloadbalanceronTanzuKubernetesGridIntegratedEditiononvSpherewithNSX-TisautomaticallyprovisionedwithKubernetesingressresourceswithouttheneedtodeployandconfigureanadditionalingresscontroller.

ForinformationaboutdeployingaloadbalancerconfiguredwithingressroutingonGCP,AWS,Azure,andvSpherewithoutNSX-T,seeConfiguring Ingress Routing.ForinformationaboutingressroutingonvSpherewithNSX-T,seeConfiguring Ingress Resourcesand Load Balancer Services.

LoadBalancersinTanzuKubernetesGridIntegratedEditionDeploymentsonvSpherewithNSX-TTanzuKubernetesGridIntegratedEditiondeploymentsonvSpherewithNSX-TdonotrequirealoadbalancerconfiguredtoaccesstheTKGIAPI.TheyrequireonlyaDNATruleconfiguredsothattheTKGIAPIhostisaccessible.Formoreinformation,seeShare theTanzu Kubernetes Grid Integrated Edition EndpointinInstallingTanzuKubernetesGridIntegratedEditiononvSpherewithNSX-TIntegration.

NSX-Thandlesloadbalancercreation,configuration,anddeletionautomaticallyaspartoftheKubernetesclustercreate,update,anddeleteprocess.WhenanewKubernetesclusteriscreated,NSX-Tcreatesandconfiguresadedicatedloadbalancertiedtoit.Theloadbalancerisasharedresourcedesignedtoprovideefficienttrafficdistributiontomasternodesaswellasservicesdeployedonworkernodes.Eachapplicationserviceismappedtoavirtualserverinstance,carvedoutfromthesameloadbalancer.Formoreinformation,seeLogical Load Balancer intheNSX-Tdocumentation.

Virtualserverinstancesarecreatedontheloadbalancertoprovideaccesstothefollowing:

Kubernetes API and UI services on a Kubernetes cluster.Thisenablesrequeststobeloadbalancedacrossmultiplemasternodes.

Ingress controller.ThisenablesthevirtualserverinstancetodispatchHTTPandHTTPSrequeststoservicesassociatedwithIngressrules.

type:loadbalancer services.ThisenablestheservertohandleTCPconnectionsorUDPflowstowardexposedservices.

Loadbalancersaredeployedinhigh-availabilitymodesothattheyareresilienttopotentialfailuresandabletorecoverquicklyfromcriticalconditions.


https://docs.vmware.com/en/VMware-NSX-T/2.1/com.vmware.nsxt.admin.doc/GUID-46567C8D-A5C5-4793-8CDF-858E58FDE3C4.html

ResizingLoadBalancers

WhenanewKubernetesclusterisprovisionedusingtheTKGIAPI,NSX-Tcreatesadedicatedloadbalancerforthatnewcluster.Bydefault,thesizeoftheloadbalancerissettoSmall.

Withnetworkprofiles,youcanchangethesizeoftheloadbalancerdeployedbyNSX-Tatthetimeofclustercreation.Forinformationaboutnetworkprofiles,seeUsing Network Profiles (NSX-T Only).

FormoreinformationaboutthetypesofloadbalancersNSX-Tprovisionsandtheircapacities,seeScaling Load BalancerResources intheNSX-Tdocumentation.

[email protected].

Note:The NodePort ServicetypeisnotsupportedforTanzuKubernetesGridIntegratedEditiondeploymentsonvSpherewithNSX-T.Only type:LoadBalancer ServicesandServicesassociatedwithIngressrulesaresupportedonvSpherewithNSX-T.


https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.admin.doc/GUID-19B12230-8BF4-4AF7-9EB7-3701B0A0A439.htmlmailto:[email protected]

VMSizingforTanzuKubernetesGridIntegratedEditionClusters

In this topic

Overview

MasterNodeVMSize

WorkerNodeVMNumberandSizeExampleWorkerNodeRequirementCalculation

CustomizeMasterandWorkerNodeVMSizeandType

Page last updated:

ThistopicdescribeshowVMwareTanzuKubernetesGridIntegratedEditionrecommendsyouapproachthesizingofVMsforclustercomponents.

Overview

WhenyouconfigureplansintheTanzuKubernetesGridIntegratedEditiontile,youprovideVMsizesforthemasterandworkernodeVMs.Formoreinformationaboutconfiguringplans,seethePlanssectionofInstallingTanzuKubernetesGridIntegratedEditionforyourIaaS:

vSphere

vSphere with NSX-T Integration

Google Cloud Platform (GCP)

Amazon Web Services (AWS)

Azure

Youselectthenumberofmasternodeswhenyouconfiguretheplan.

ForworkernodeVMs,youselectthenumberandsizebasedontheneedsofyourworkload.ThesizingofmasterandworkernodeVMsishighlydependentonthecharacteristicsoftheworkload.Adapttherecommendationsinthistopicbasedonyourownworkloadrequirements.

MasterNodeVMSize

ThemasternodeVMsizeislinkedtothenumberofworkernodes.TheVMsizingshowninthefollowingtableispermasternode:

TocustomizethesizeoftheKubernetesmasternodeVM,seeCustomize Master and Worker Node VM Size and Type.


Note:Iftherearemultiplemasternodes,allmasternodeVMsarethesamesize.Toconfigurethenumberofmasternodes,seethePlanssectionofInstallingTanzuKubernetesGridIntegratedEditionforyourIaaS.


Number of Workers CPU RAM (GB)

1-5 1 3.75

6-10 2 7.5

11-100 4 15

101-250 8 30

251-500 16 60

500+ 32 120

DonotoverloadyourmasternodeVMsbyexceedingtherecommendedmaximumnumberofworkernodeVMsorbydownsizingfromtherecommendedVMsizingslistedabove.TheserecommendationssupportbothatypicalworkloadmanagedbyaVMandthehigherthanusualworkloadmanagedbytheVMwhileotherVM’sintheclusterareupgrading.

WorkerNodeVMNumberandSize

Amaximumof100podscanrunonasingleworkernode.TheactualnumberofpodsthateachworkernoderunsdependsontheworkloadtypeaswellastheCPUandmemoryrequirementsoftheworkload.

TocalculatethenumberandsizeofworkerVMsyourequire,determinethefollowingforyourworkload:

Maximumnumberofpodsyouexpecttorun[ p ]

Memoryrequirementsperpod[ m ]

CPUrequirementsperpod[ c ]

Usingthevaluesabove,youcancalculatethefollowing:

Minimumnumberofworkers[ W ]= p / 100

MinimumRAMperworker= m * 100

MinimumnumberofCPUsperworker= c * 100

Thiscalculationgivesyoutheminimumnumberofworkernodesyourworkloadrequires.Werecommendthatyouincreasethisvaluetoaccountforfailuresandupgrades.

Forexample,increasethenumberofworkernodesbyatleastonetomaintainworkloaduptimeduringanupgrade.Additionally,increasethenumberofworkernodestofityourownfailuretolerancecriteria.

ThemaximumnumberofworkernodesthatyoucancreateforaplaninanTanzuKubernetesGridIntegratedEdition-provisionedKubernetesclusterissetbytheMaximum number of workers on a clusterfieldinthePlanspaneoftheTanzuKubernetesGridIntegratedEditiontile.TocustomizethesizeoftheKubernetesworkernodeVM,seeCustomize Master and Worker Node VM Sizeand Type.

ExampleWorkerNodeRequirementCalculation

Anexampleapphasthefollowingminimumrequirements:

Numberofpods[ p ]=1000

warning:UpgradinganoverloadedKubernetesclustermasternodeVMcanresultindowntime.


RAMperpod[ m ]=1GB

CPUperpod[ c ]=0.10

TodeterminehowmanyworkernodeVMstheapprequires,dothefollowing:

1. Calculatethenumberofworkersusing p / 100 :

1000/100 = 10 workers

2. CalculatetheminimumRAMperworkerusing m * 100 :

1 * 100 = 100 GB

3. CalculatetheminimumnumberofCPUsperworkerusing c * 100 :

0.10 * 100 = 10 CPUs

4. Forupgrades,increasethenumberofworkersbyone:

10 workers + 1 worker = 11 workers

5. Forfailuretolerance,increasethenumberofworkersbytwo:

11 workers + 2 workers = 13 workers

Intotal,thisappworkloadrequires13workerswith10CPUsand100GBRAM.

CustomizeMasterandWorkerNodeVMSizeandType

YouselecttheCPU,memory,anddiskspacefortheKubernetesnodeVMsfromasetlistintheTanzuKubernetesGridIntegratedEditiontile.MasterandworkernodeVMsizesandtypesareselectedonaper-planbasis.Formoreinformation,seethePlanssectionoftheTanzuKubernetesGridIntegratedEditioninstallationtopicforyourIaaS.Forexample,Installing Tanzu Kubernetes GridIntegrated Edition on vSphere with NSX-T.

WhilethelistofavailablenodeVMtypesandsizesisextensive,thelistmaynotprovidetheexacttypeandsizeofVMthatyouwant.YoucanusetheOpsManagerAPItocustomizethesizeandtypesofthemasterandworkernodeVMs.Formoreinformation,seeHowto Create or Remove Custom VM_TYPE Template using the Operations Manager API intheKnowledgeBase.

[email protected].

warning:DonotreducethesizeofyourKubernetesmasternodeVMsbelowtherecommendedsizeslistedinMaster NodeVM Size,above.UpgradinganoverloadedKubernetesclustermasternodeVMcanresultindowntime.


https://community.pivotal.io/s/article/how-to-create-or-remove-custom-vmtype-template-using-the-ops-manager-apimailto:[email protected]

Telemetry

In this topic

OverviewParticipationLevels

ConfigureCEIPandTelemetry

SystemComponents

DataDictionary

SampleReports

Page last updated:

ThistopicdescribestheVMwareCustomerExperienceImprovementProgram(CEIP)andtheTelemetryProgramusedintheTanzuKubernetesGridIntegratedEditiontile.

Overview

TheCEIPandTelemetryprogramallowsVMwaretocollectdatafromcustomerinstallationstoimproveyourTanzuKubernetesGridIntegratedEditionexperience.CollectingdataatscaleenablesustoidentifypatternsandalertyoutowarningsignalsinyourTanzuKubernetesGridIntegratedEditioninstallation.

ParticipationLevels

YoucanconfigureTanzuKubernetesGridIntegratedEditiontouseoneofthefollowingCEIPandTelemetryparticipationlevels:

None:Thisleveldisablesdatacollection.

Standard:(Default)Thislevelcollectsdataanonymously.YourdataisusedtoinformtheongoingdevelopmentofTanzuKubernetesGridIntegratedEdition.

Enhanced:ThislevelenablesVMwaretowarnyouaboutsecurityvulnerabilitiesandpotentialissueswithyoursoftwareconfigurations.Formoreinformation,seeBenefits of the Enhanced Participation Levelbelow.

BenefitsoftheEnhancedParticipationLevel

BenefitsyoureceivewiththeEnhancedparticipationlevelincludebutarenotlimitedtothefollowing:

Usage data:ThisgivesyouaccesstodataaboutKubernetespodandclusterusageinyourTanzuKubernetesGridIntegratedEditioninstallation.Seesample reportsbelowformoredetails.

Access to your telemetry data:ThisgivesyouaccesstoconfigurationandusagedataaboutyourTanzuKubernetesGridIntegratedEditioninstallation.Seesample reportsbelowformoredetails.


Note:TanzuKubernetesGridIntegratedEditiondoesnotcollectanypersonallyidentifiableinformation(PII)ateitherparticipationlevel.ForalistofthedataTanzuKubernetesGridIntegratedEditioncollects,seeData Dictionary.


Proactive support:ThisenablesVMwaretoproactivelywarnyouaboutunhealthypatterns.

Benchmarks:ThisisyourusagerelativetotherestoftheTanzuKubernetesGridIntegratedEditionuserbase.

ThetablebelowcomparestheStandardandEnhancedparticipationlevels.

Benefit Standard Level Enhanced Level

Usagedata Rawdata Reportsandtrendanalysis

Accesstoyourtelemetrydata No Yes

Proactivesupport No Yes

Benchmarks No Yes

ConfigureCEIPandTelemetry

Video:ForinformationaboutconfiguringCEIPandTelemetryparticipation,seetheCEIP Opt-In Walkthrough video onYouTube.

ToconfigureCEIPandTelemetry,seetheCEIPandTelemetry sectionoftheinstallationtopicforyourIaaS:

Installing Tanzu Kubernetes Grid Integrated Edition on vSphere

Installing Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX-T

Installing Tanzu Kubernetes Grid Integrated Edition on AWS

Installing Tanzu Kubernetes Grid Integrated Edition on Azure

Installing Tanzu Kubernetes Grid Integrated Edition on GCP

ProxyCommunication

Ifyouuseaproxyserver,theTanzuKubernetesGridIntegratedEditionproxysettingsapplytooutgoingtelemetrydata.

ToconfigureTanzuKubernetesGridIntegratedEditionproxysettingsforCEIPandTelemetryandothercommunications,seethefollowing:

ForAWS,seeUsing Proxies with Tanzu Kubernetes Grid Integrated Edition on AWS.

ForvSphere,seeNetworkinginInstallingTanzuKubernetesGridIntegratedEditiononvSphere.

ForvSpherewithNSX-T,seeUsing Proxies with Tanzu Kubernetes Grid Integrated Edition on NSX-T.

SystemComponents

TheCEIPandTelemetryprogramsusethefollowingcomponentstocollectdata:

Telemetry Server:ThiscomponentrunsontheTKGIcontrolplane.TheserverreceivestelemetryeventsfromtheTKGIAPIandmetricsfromTelemetryagentpods.Theserversendseventsandmetricstoadatalakeforarchivingandanalysis.

Telemetry Agent Pod:ThiscomponentrunsineachKubernetesclusterasadeploymentwithonereplica.AgentpodsperiodicallypolltheKubernetesAPIforclustermetricsandsendthemetricstotheTelemetryserver.

Thefollowingdiagramshowshowtelemetrydataflowsthroughthesystemcomponents:

Note:VMwarereservestherighttochangethebenefitsassociatedwiththeEnhancedparticipationlevelatanytime.


https://www.youtube.com/embed/RTyq9ODUbU4

DataDictionary

ForinformationaboutTKGITelemetrycollectionandreporting,seetheTKGI Telemetry Data spreadsheet,hostedonGoogleDrive.

SampleReports

Video:SeetheSample Report: Create Cluster Duration videoonYouTube.

YoucanviewtheinteractiveversionoftheSample Workbook withTableau Reader (freetouse).Clickonthelinksbelowtoseestaticscreenshotsofthereports.

1. Consumption :AsanOperatorofTKGI,IneedawaytomonitorpodconsumptionacrossmyTKGIenvironmentsovertime,soIcan:

SeewhichenvironmentsandclustersgettheheaviestuseSeetemporalpatternsinpodconsumptionScalecapacityaccordinglyShowandchargebackusersofTKGIwithinmyorganization

2. API heartbeats + Cluster heartbeats :AsanOperatorofTKGIIneedawaytoseetheversionofTKGIeachofmyenvironmentswasrunningovertime,soIcan:

KeeptrackofallmyTKGIenvironmentsandclustersIdentifyenvironmentsandclustersinneedofupgrading

3. Cluster creation events :AsanOperatorofTKGIIwanttoseehowoftenclustercreationsucceedsacrossmyTKGIenvironments,soIcan:

Identifyenvironmentsthatencounterrepeatedfailuresanddebugorinterveneasappropriatetoavoidfrustrationfor


file:///Users/pspinrad/workspace/pdfer/html/docs-pcf-staging.cfapps.io/tkgi/1-8/images/telemetry-data-flow.pnghttps://drive.google.com/open?id=18UCd1kbhR3xV_XOl6KcEU64GI6ySdkRa3iG_8QAROl8#gid=1858241440https://www.youtube.com/embed/Q41g7uWBvhAhttp://bit.ly/sampleworkbookhttps://www.tableau.com/products/readerhttp://bit.ly/consumptionreporthttp://bit.ly/apiheartbeatshttp://bit.ly/clusterheartbeatshttp://bit.ly/clustercreate

clusteradminsandusers

4. Cluster creation duration :AsanOperatorofTKGIIwanttoseehowlongittakestocreateclusters,soIcan:

Intervenewhenclustercreationsignificantlymoretimethanexpected,andadjustmyplanandnetworkconfigurationasappropriate

5. Cluster creation errors :AsanOperatorofTKGI,IwanttoseewhaterrorsarebeingencounteredmostfrequentlyduringclustercreationsoIcan:

Quicklyidentifywidespreadproblemsandremediate(e.g.NSXerrors)

6. Container images :AsanOperatorofTKGI,IwanttoseewhichcontainerimagesareinuseacrossmyTKGIinstallationssoIcan:

ConductanauditofcontainerimagesandidentifyprohibitedorproblematicimagesInferwhichworkloadsarerunningonTKGI,toinformmyplanning,resourcing,andoutreach

[email protected].


http://bit.ly/createdurationhttp://bit.ly/createerrorshttp://bit.ly/containerimagesmailto:[email protected]

InstallingTanzuKubernetesGridIntegratedEdition

Page last updated:

TanzuKubernetesGridIntegratedEditionManagementConsole(vSphereOnly)

SeethefollowingdocumentationfortheManagementConsole,whichistherecommendedmethodforinstallingTanzuKubernetesGridIntegratedEditiononvSphere:

Install Tanzu Kubernetes Grid Integrated Edition on vSphere with the Management Console

Formoreinformation,seeWhen Should I Use Tanzu Kubernetes Grid Integrated Edition Management Console?.

TanzuKubernetesGridIntegratedEditiononOpsManager

SeethefollowingdocumentationforhowtomanuallyinstallTanzuKubernetesGridIntegratedEdition,usingOpsManager,onOpsManager:

vSphere with Flannel

vSphere with NSX-T

Google Cloud Platform

Amazon Web Services

Azure

[email protected].


Note:TanzuKubernetesGridIntegratedEditionsupportsair-gappeddeploymentsonvSpherewithorwithoutNSX-Tintegration.



InstallingTanzuKubernetesGridIntegratedEditiononvSphere

In this topic

Overview

WhenShouldIUseTanzuKubernetesGridIntegratedEditionManagementConsole?

Page last updated:

ThistopicdescribesoptionsforinstallingTanzuKubernetesGridIntegratedEditiononvSphere.

Overview

YoucaninstallTanzuKubernetesGridIntegratedEdition(TKGI)onvSphereinthreeways,dependingonwhetheryouusetheTKGIManagementConsole,andwhichcontainernetworkingoverlayyouuse:

Install Tanzu Kubernetes Grid Integrated Edition on vSphere with the Management Console

Install Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX-T Using Ops Manager

Install Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX-T Using Ops Manager

Wherepossible,VMwarerecommendsusingthemanagementconsoletoinstallTanzuKubernetesGridIntegratedEditiononvSphere.Formoreinformation,seeWhen Should I Use Tanzu Kubernetes Grid Integrated Edition Management Console?,below.

WhenShouldIUseTanzuKubernetesGridIntegratedEditionManagementConsole?

TanzuKubernetesGridIntegratedEditionManagementConsolegreatlysimplifiestheprocessofdeployingTanzuKubernetesGridIntegratedEdition,especiallyinlesscomplexenvironments.However,ifyourequiremoreflexibilityininconfiguringyourdeployment,especiallyincomplexNSX-TDataCenterdeployments,itmightbemoreappropriatetoperformtheinstallationmanually.Forinformationaboutthesupportedtopologiesforamanualinstallation,seeNSX-T Deployment Topologies for TanzuKubernetes Grid Integrated Edition.

BeforeusingTanzuKubernetesGridIntegratedEditionManagementConsoletodeployTanzuKubernetesGridIntegratedEdition,considerthefollowingfactors:

IfyouwanttodeployTanzuKubernetesGridIntegratedEditionManagementConsoletoaNo-NATtopologywithanNSX-TDataCenterlogicalswitch,youmustperformaBYOTdeployment.

DeploymentstoaMulti-Tier-0topologyaresupportedinBYOTdeploymentsonlyandrequireadditionalconfiguration.Forinformationabouttheadditionalconfigurationrequired,seeTanzu Kubernetes Grid Integrated Edition Management ConsoleCannot Retrieve Cluster Data in a Multi-Tier0 TopologyinTroubleshootingTanzuKubernetesGridIntegratedEditionManagementConsole.

DeploymentstoaNo-NATtopologywithavSphereStandardSwitchoravSphereDistributedSwitcharenotsupportedinanycase.

Multi-Foundationdeploymentsarenotsupportedinanycase.

HowusersdeployTanzuKubernetesGridIntegratedEditionoftendependsonwhethertheyalreadyhaveacustominstallationofthe



OpsManagertool,whichTKGIuses:

UsersonvSpherewhodonothaveacustomOpsManagerinstallationmaypreferthemanagementconsole.

UserswhoalreadyhaveOpsManagerinstalledforotheruses,forexampletorunVMware Tanzu Application Service for VMs ,mayprefertoinstallTKGImanually.

[email protected].


https://docs.pivotal.io/platform/application-service/overview/intro.htmlmailto:[email protected]

InstallTanzuKubernetesGridIntegratedEditiononvSpherewiththeManagementConsole

Page last updated:

VMwareTanzuKubernetesGridIntegratedEditionManagementConsoleprovidesaunifiedinstallationexperiencefordeployingVMwareTanzuKubernetesGridIntegratedEditiontovSphere.ThemanagementconsoleisprovidedasavirtualappliancethatyoudeploytovSpherebyusinganOVAtemplate.ThemanagementconsoleprovidesagraphicaluserinterfacethatassistsyouwiththeconfigurationwhendeployingTanzuKubernetesGridIntegratedEditiontovSphere:

ConfiguresnetworkingforTanzuKubernetesGridIntegratedEdition

DeploysOpsManager

GeneratesandregistersSSLcertificates

DeploysBOSHDirector

DeploysTanzuKubernetesGridIntegratedEdition

DeploysHarborRegistry

TanzuKubernetesGridIntegratedEditionManagementConsoleiseasytouse.IfyouareexperiencedwithinstallingTanzuKubernetesGridIntegratedEditiononvSphere,thehelpandthetooltipsintheinstallerUIshouldbeenoughtocompletetheprocess.IfyouarenewtoTanzuKubernetesGridIntegratedEdition,refertothisdocumentationasneededtoassistwiththeinstallation.

Seethefollowingtopics:

Prerequisites for Tanzu Kubernetes Grid Integrated Edition Management Console Deployment

Deploy the Tanzu Kubernetes Grid Integrated Edition Management Console

Deploy Tanzu Kubernetes Grid Integrated Edition from the Management Console

AfteryouhavedeployedTanzuKubernetesGridIntegratedEditiononvSphere,youcanusethemanagementconsoletodeployKubernetesclustersandmanagetheirlifecycle,andmonitorandmanagetheoperationofyourTanzuKubernetesGridIntegratedEditiondeployment.Forinformationabouthowtousethemanagementconsoleafterdeployment,seethefollowingtopics:

Create and Manage Clusters in the Management Console

Monitor and Manage Tanzu Kubernetes Grid Integrated Edition in the Management Console

Troubleshooting Tanzu Kubernetes Grid Integrated Edition Management Console

[email protected].




PrerequisitesforTanzuKubernetesGridIntegratedEditionManagementConsoleDeployment

In this topic

NetworkConfigurations

Page last updated:

VMwareTanzuKubernetesGridIntegratedEditionManagementConsoleisprovidedasanOVAtemplatethatrequiresataminimumthevSphereresourcesdescribedinVirtual Infrastructure Prerequisites.

Formoreinformation,seeWhen Should I Use Tanzu Kubernetes Grid Integrated Edition Management Console?

NetworkConfigurations

TanzuKubernetesGridIntegratedEditionManagementConsoleprovides3networkconfigurationoptionsforyourTanzuKubernetesGridIntegratedEditiondeployments.Eachnetworkconfigurationoptionhasspecificprerequisites.

Bring your own topology:DeployTanzuKubernetesGridIntegratedEditiontoanexistingNSX-TDataCenternetworkthatyouhavefullyconfiguredyourself.SeePrerequisites for a Bring Your Own Topology Deployment to NSX-T Data Center.

Automated NAT deployment:DeployTanzuKubernetesGridIntegratedEditiontoanexistingNSX-TDataCenternetworkthatyouhavenotfullysetup,thatTanzuKubernetesGridIntegratedEditionManagementConsolehelpstoconfigureforyou.SeePrerequisites for an Automated NAT Deployment to NSX-T Data Center.

Flannel:DeployTanzuKubernetesGridIntegratedEditiontoaFlannelnetworkthatTanzuKubernetesGridIntegratedEditionManagementConsoleprovisionsforyou.SeePrerequisites for a Flannel Network.

ForthelistoffirewallportsthatmustbeopenforanTanzuKubernetesGridIntegratedEditionManagementConsoledeployment,seeFirewall Ports and Protocols Requirements for Tanzu Kubernetes Grid Integrated Edition Management Console.

WhenyourenvironmentmeetstheprerequisitesforvSphereandforyourchosentypeofnetworking,youcanDeploy the TanzuKubernetes Grid Integrated Edition Management Console.

[email protected].




VirtualInfrastructurePrerequisites

Page last updated:

ThevSphereenvironmenttowhichyoudeploythemanagementconsoleOVArequiresthefollowingconfiguration:

CPU:2

RAM:8GB

Disk:40GB

VirtualNIC(vNIC)shouldbeassignedtoanetworkwithconnectivitytovCenterandNSXDatacenterManager,ifyouareusingNSX-TDataCenterasthecontainernetworkinginterfaceforTanzuKubernetesGridIntegratedEdition.

ThefollowingvSphereclustersmustexistinthetargetvCenterServerdatacenterbeforeyoucandeployTanzuKubernetesGridIntegratedEditionfromthemanagementconsole:

ManagementclusterforTKGIManagementPlanecomponents.

AtleastonecomputeclusterforKubernetesClusternodes,withtherecommendationbeingtodeploymorethanone,forhigh-availabilitypurposes.

ForinformationaboutthesupportedversionsofvSphere,seetherelease notes.

[email protected].


Note:TheOVArequirementsdescribedherearetheminimumsupportedconfiguration.



FirewallPortsandProtocolsRequirementsforTanzuKubernetesGridIntegratedEditionManagementConsole

Page last updated:

Firewallsandsecuritypoliciesareusedtofiltertrafficandlimitaccessinenvironmentswithstrictinter-networkaccesscontrolpolicies.

Appsfrequentlyrequiretheabilitytopassinternalcommunicationbetweensystemcomponentsondifferentnetworksandrequireoneormoreconduitsthroughtheenvironment’sfirewalls.FirewallrulesarealsorequiredtoenableinterfacingwithexternalsystemssuchaswithenterpriseappsorappsanddataonthepublicInternet.

ForTanzuKubernetesGridIntegratedEditiononvSphere,itisrecommendedtodisablesecuritypoliciesthatfiltertrafficbetweenthenetworkssupportingthesystem.TosecuretheenvironmentandgrantaccessbetweensystemcomponentswithTanzuKubernetesGridIntegratedEdition,useoneofthefollowingmethods:

EnableaccesstoappsthroughstandardKubernetesload-balancersandingresscontrollertypes.Thisenablesyoutodesignatespecificportsandprotocolsasafirewallconduit.

EnableaccessusingtheNSX-Tloadbalancerandingress.Thisenablesyoutoconfigureexternaladdressesandportsthatareautomaticallymappedandresolvedtointernal/localaddressesandports.

Ifyouareunabletoimplementyoursecuritypolicyusingthesemethods,refertothetablebelow,whichidentifiestheflowsbetweenthesystemcomponentsinanTanzuKubernetesGridIntegratedEditionManagementConsoledeployment.

Source Component Destination Component DestinationProtocolDestinationPort Service

ManagementConsoleVM

AllSystemComponents TCP 22 ssh

ManagementConsoleVM

AllSystemComponents TCP 80 http

ManagementConsoleVM

AllSystemComponents TCP 443 https

ManagementConsoleVM

CloudFoundryBOSHDirector TCP 25555boshdirectorrestapi

ManagementConsoleVM

DNSvalidationforOpsManager TCP 53 netcat


Notes:TheSourceComponentisIPaddressoftheTanzuKubernetesGridIntegratedEditionManagementConsoleVM.

InastandardTanzuKubernetesGridIntegratedEditiondeployment,itisassumedthatOpsManagerandBOSHarealreadydeployedbeforeyoudeployTanzuKubernetesGridIntegratedEdition.ThisisnotthecasewithTanzuKubernetesGridIntegratedEditiondeploymentsfromthemanagementconsole,inwhichyoudonotknowtheIPaddressesinthedeploymentnetworkthatwillbeassignedtoTKGIAPIVM,BOSHVM,andOpsManagerVM.Asaconsequence,itisrecommendedtocreateafirewallrulethatallowsaccessbythemanagementconsoleVMtotheentiredeploymentsubnet.


ManagementConsoleVM

KubernetesClusterAPIServer-LBVIP TCP 8443 httpsca

ManagementConsoleVM

PivotalCloudFoundryOperationsManager

TCP 22 ssh

ManagementConsoleVM

PivotalCloudFoundryOperationsManager

TCP 443 https

ManagementConsoleVM

TKGIController TCP 9021 tkgiapiserver

ManagementConsoleVM

vCenterServer TCP 443 https

Source Component Destination Component DestinationProtocolDestinationPort Service

[email protected].



PrerequisitesforaBringYourOwnTopologyDeploymenttoNSX-TDataCenter

In this topic

GeneralRequirements

NSX-TDataCenterConfigurationRequirements

Proof-of-ConceptDeployments

Page last updated:

AbringyourowntopologyenvironmentisanNSX-TDataCenterinstancethatyouhavefullyconfiguredyourselfforusewithTanzuKubernetesGridIntegratedEdition.Forexample,anNSX-TDataCenterinstancethatyouhaveusedinapreviousdeploymentofTanzuKubernetesGridIntegratedEdition.Thefollowingobjectsmustbeinplacebeforeyoustartaproductiondeployment.

3NSXManagerNodesdeployed

NSXManagementClusterformed

VirtualIPaddressassignedforManagementClusterorloadbalancer

ForinformationaboutthesupportedversionsofNSX-TDataCenter,seetherelease notes.

GeneralRequirementsAnactive/activeTier-0Routercreated.

AlogicalswitchonanNSX-TVirtualDistributedSwitch(N-VDS)forusebytheTKGImanagementplaneisprepared.TheswitchmustbeeitherundertheTier-0router,orundertheTier-1routeriftheTier-1routerisdirectlyundertheTier-0router.

EdgeClusterwithatleast2NSX-TDataCenterEdgeNodesdeployedinactive/standbymode,withconnectivitytoanuplinknetworkconfigured.

OverlayTransportZonecreated,withtheedgenodesincluded.

VLANTransportZonecreated,withtheedgenodesincluded.

MTUofalltransportnodesandphysicalinterfacesconfiguredto1600ormore.

IfyourNSX-TDataCenterenvironmentusescustomcertificates,obtaintheCAcertificateforNSXManager.


Notes:DonotusethenetworkonwhichyoudeploytheTanzuKubernetesGridIntegratedEditionManagementConsoleVMasthenetworkforthemanagementplanewhenyoudeployTanzuKubernetesGridIntegratedEdition.UsingthesamenetworkforthemanagementconsoleVMandthemanagementplanerequiresadditionalNSX-TDataCenterconfigurationandisnotrecommended.

IfNSX-TDataCenterusescustomcertificatesandyoudonotprovidetheCAcertificateforNSXManager,TanzuKubernetesGridIntegratedEditionManagementConsoleautomaticallygeneratesoneandregistersitwithNSXManager.ThiscancauseotherservicesthatareintegratedwithNSXManagernottofunctioncorrectly.

InBYOTmode,TanzuKubernetesGridIntegratedEditionManagementConsoleautomaticallyretrievesthetier0HAmodefrom


NSX-TDataCenterConfigurationRequirementsVirtualIPfortheTier-0Routerconfigured

FloatingIPPoolconfigured

PodIPBlockIDcreated

NodeIPBlockIDcreated

LogicalSwitchconfiguredforTKGIManagementPlane

Tier-1RouterconfiguredandconnectedtotheTier-0Router

RoutingforTKGIFloatingIPsconfiguredtopointtotheTier-0HAVirtualIP


Therequirementsaboveareforproductionenvironments.Inproof-of-conceptdeploymentsoneNSXManagernodeissufficient.TheNSXmanagementclusterandloadbalancerarealsooptionalforproof-of-conceptdeployments.

[email protected].

yourNSX-TDataCenterenvironmentandcreatesNATrulesonthetier0ortier1router.

IfyouaredeployingTanzuKubernetesGridIntegratedEditioninamultiple-tier0topology,additionalpost-deploymentconfigurationofthemanagementconsoleVMisrequired.Forinformation,seeTanzu Kubernetes Grid Integrated EditionManagement Console Cannot Retrieve Cluster Data in a Multi-Tier0 TopologyinTroubleshootingtheManagementConsole.



PrerequisitesforanAutomatedNATDeploymenttoNSX-TDataCenter

In this topic

GeneralRequirements


Page last updated:

AnunpreparedenvironmentisanNSX-TDataCenterinstancethatyouhavenotalreadyconfiguredforusewithTanzuKubernetesGridIntegratedEdition.TanzuKubernetesGridIntegratedEditionManagementConsolehelpsyoutocompletetheconfigurationofanunpreparedenvironmentonvSphere,buttheenvironmentmustmeetcertaininfrastructureprerequisites.

3NSXManagerNodesdeployed

NSXManagementClusterformed

VirtualIPaddressassignedfortheManagementClusterorloadbalancer

ForinformationaboutthesupportedversionsofNSX-TDataCenter,seetherelease notes.

GeneralRequirementsEdgeClusterwithatleast2NSX-TDataCenterEdgeNodesdeployedandconnectivitytoanuplinknetworkconfiguredandverified

OverlayTransportZonecreated,withtheedgenodesincluded

VLANTransportZonecreated,withtheedgenodesincluded

MTUofalltransportnodesandphysicalinterfacesconfiguredto1600ormore

ObtainthefollowingIPaddressesfortheuplinknetworktouse:

Subnet,subnetmask,gateway,andVLANIDoftheuplinknetworkAddresseswithintheuplinksubnetfortheTier0uplinksAddresstousefortheHAVirtualIPontheTier-0router

ObtainthefollowingIPadditionaladdresses:

CIDRrangestousefordeployment,pods,andnodes.ThisrangeofIPaddressesmustnotbeinconflictwithanyotherworkloads.IPaddressesofDNSandNTPserversArangeof5availablefloatingIPaddresses

IfyourNSX-TDataCenterenvironmentusescustomcertificates,obtaintheCAcertificateforNSXManager



Note:IfNSX-TDataCenterusescustomcertificatesandyoudonotprovidetheCAcertificateforNSXManager,TanzuKubernetesGridIntegratedEditionManagementConsoleautomaticallygeneratesoneandregistersitwithNSXManager.ThiscancauseotherservicesthatareintegratedwithNSXManagernottofunctioncorrectly.


Therequirementsaboveareforproductionenvironments.Inproof-of-conceptdeploymentsoneNSXManagernodeissufficient.TheNSXmanagementclusterandloadbalancerarealsooptionalforproof-of-conceptdeployments.OneNSX-TDataCenterEdgenodeissufficientforproof-of-conceptdeployments.

[email protected].



PrerequisitesforaFlannelNetwork

Page last updated:

YoucanselecttheoptionforTanzuKubernetesGridIntegratedEditionManagementConsoletoprovisionaFlannelcontainernetworkinginterfaceforyouduringTanzuKubernetesGridIntegratedEditiondeploymentonvSphere.

ObtainthefollowingIPaddressestousefordeploymenttoaFlannelnetwork:

DNSserver,subnet,subnetmask,andgatewayofthenetworkonwhichtodeployTanzuKubernetesGridIntegratedEdition

DNSserver,subnet,subnetmask,andgatewayoftheFlannelservicenetwork

SubnetrangeandsubnetmaskfortheKubernetespodandKubernetesservicenetworks

[email protected].




DeploytheTanzuKubernetesGridIntegratedEditionManagementConsole

In this topic

Prerequisites

Step1:DeploytheOVATemplate

Step2:LogIntoTanzuKubernetesGridIntegratedEditionManagementConsole

NextSteps

Page last updated:

ThistopicdescribeshowtodeploytheVMwareTanzuKubernetesGridIntegratedEditionManagementConsolefromtheOVAtemplate.

IfyouhavedeployedapreviousversionofVMwareTanzuKubernetesGridIntegratedEditionManagementConsole,youcanusethemanagementconsoletoupgradeittoanewerversion.Forinformationaboutupgrading,seeUpgrade Tanzu Kubernetes GridIntegrated Edition Management Console.

PrerequisitesDownloadtheOVAtemplatefromhttps://downloads.vmware.com .

UseanaccountwithvSphereadministratorprivilegestologintovSphereusingthevSphereClient.

ThevCenterServerinstancemustbecorrectlyconfiguredforTanzuKubernetesGridIntegratedEditionManagementConsoledeployment.ForinformationaboutthevCenterServerrequirements,seeVirtual Infrastructure Prerequisites.

Step1:DeploytheOVATemplate

TodeploytheTanzuKubernetesGridIntegratedEditionManagementConsoletovSphere,theprocedureisasfollows:

1. InthevSphereClient,right-clickanobjectinthevCenterServerinventory,selectDeploy OVF template,selectLocal file,andclickBrowsetonavigatetoyourdownloadoftheOVAtemplate.

2. FollowtheinstallerpromptstoperformbasicconfigurationofthemanagementconsoleandtoselectthevSphereresourcesforittouse.

AcceptormodifythemanagementconsoleVMnameSelectthedestinationdatacenterorfolderSelectthedestinationclusterorresourcepoolforthemanagementconsoleVMAccepttheenduserlicenseagreements(EULA)SelectthediskformatanddestinationdatastoreforthemanagementconsoleVM

3. OntheSelect Networkspage,selectanetworkportgrouptowhichtoconnectthemanagementconsoleVM.


important:IfyouintendtodeployTanzuKubernetesGridIntegratedEditioninabringyourowntopologyNSX-TData


https://downloads.vmware.com

4. OntheCustomize templatepage,expandAppliance Configuration.

SettherootpasswordforthemanagementconsoleVM.SettingtherootpasswordfortheVMismandatory.OptionallyuncheckthePermit Root Logincheckbox.

Therootpasswordistheonlymandatoryoption.Ifyouwanttouseauto-generatedcertificates,DHCPnetworking,andyoudonotwanttointegratewithVMwarevRealizeLogInsight,clickNexttostarttheOVAdeployment.Otherwise,completetheremainingstepsinthisprocedure.

5. ConfigurethemanagementconsoleVMcertificate,thatisusedbyalloftheservicesthatruninthemanagementconsoleVMtoauthenticateconnections.Touseauto-generated,self-signedcertificates,leavetheAppliance TLS Certificate,Appliance TLS Certificate Key,andCertificate Authority Certificatetextboxesblank.Touseacustomcertificate:PastethecontentsoftheservercertificatePEMfileintheAppliance TLS Certificatetextbox.

-----BEGINCERTIFICATE-----appliance_certificate_contents-----ENDCERTIFICATE-----

PastethecontentsofthecertificatekeyintheAppliance TLS Certificate Keytextbox.ThemanagementconsoleVMsupportsunencryptedPEMencodedformatsforTLSprivatekeys.

-----BEGINPRIVATEKEY-----appliance_private_key_contents-----ENDPRIVATEKEY-----

PastethecontentsoftheCertificateAuthority(CA)fileintheCertificate Authority Certificatetextbox.

-----BEGINCERTIFICATE-----root_CA_certificate_contents-----ENDCERTIFICATE-----

TouseacertificatethatusesachainofintermediateCAs,pasteintotheCertificate Authority CertificatetextboxthecontentsofacertificatechainPEMfile.ThePEMfilemustincludeachainoftheintermediateCAsallthewaydowntotherootCA.

-----BEGINCERTIFICATE-----intermediate_CA_certificate_contents-----ENDCERTIFICATE----------BEGINCERTIFICATE-----intermediate_CA_certificate_contents-----ENDCERTIFICATE----------BEGINCERTIFICATE-----root_CA_certificate_contents-----ENDCERTIFICATE-----

Centerenvironment,donotusethenetworkonwhichyoudeploytheTanzuKubernetesGridIntegratedEditionManagementConsoleVMasthenetworkforthemanagementplanewhenyoudeployTanzuKubernetesGridIntegratedEdition.UsingthesamenetworkforthemanagementconsoleVMandthemanagementplanerequiresadditionalNSX-TDataCenterconfigurationandisnotrecommended.

Note:Ifyouuncheckthecheckbox,youcanpermitrootloginlaterbyeditingthesettingsofthemanagementconsoleVM.


6. ExpandNetworking PropertiesandoptionallyconfigurethenetworkingforthemanagementconsoleVM.TouseDHCP,leavethesepropertiesblank.

TosetastaticIPaddressonthemanagementconsoleVM,settheNetwork IP Address,Network Netmask,andDefaultGatewaysettings.ToconfigureDNSservers,settheDomain Name Servers,andDomain Search Pathsettings.Tospecifyafullyqualifieddomainname(FQDN)forthemanagementconsoleVM,settheFQDNsetting.Ifnecessary,updateDocker Container Network SubnetandDocker Container Network Gateway.

ServicesinthemanagementconsoleVMaredeployedasDockercontainersonaDockerbridgenetwork.UpdatethesevaluesifthedefaultsubnetCIDR172.18.0.0/16andgatewayaddress172.18.0.1forthisbridgenetworkconflictwithexistingnetworks.

7. OptionallyenterthehostnameandportforVMwarevRealizeLogInsightintheLog Insight Server Host/IPandLog InsightServer Porttextboxes.vRealizeLogInsightgatherslogsfromtheTanzuKubernetesGridIntegratedEditionManagementConsoleVMitself.ForvRealizeLogInsighttogatherlogsfromyourTanzuKubernetesGridIntegratedEditiondeployments,youmustconfiguretheconnectionwhenyoudeployTanzuKubernetesGridIntegratedEditionfromTanzuKubernetesGridIntegratedEditionManagementConsole.

8. ClickNexttoreviewthesettingsthatyouhavemade.

9. ClickFinishtodeploytheTanzuKubernetesGridIntegratedEditionManagementConsole.

UsetheRecentTaskspanelatthebottomofthevSphereClienttocheckthestatusoftheOVAimportanddeploymentofthemanagementconsoleVM.ThemanagementconsoleVMtakesafewminutestodeploy.

IfthemanagementconsoleVMfailstodeploy,seeTroubleshooting.

Step2:LogIntoTanzuKubernetesGridIntegratedEditionManagementConsole

WhentheOVAdeploymenthascompletedsuccessfully,youcanaccessthemanagementconsole.

1. InthevSphereClient,right-clickthemanagementconsoleVMandselectPower>Power On.

2. WhenthemanagementconsoleVMhasbooted,gototheSummarytabfortheVMandcopyitsIPaddress.

3. EnterthemanagementconsoleVMIPaddressinabrowser.

4. AttheVMwareTanzuKubernetesGridIntegratedEditionloginpage,enterusername root andtherootpasswordthatyousetwhenyoudeployedtheOVAtemplate.

NextSteps

YoucannowuseTanzuKubernetesGridIntegratedEditionManagementConsoletodeployorupgradeTanzuKubernetesGridIntegratedEditioninstances,eitherbyusingtheconfigurationwizardorbyimportinganexistingYAMLconfigurationfile.

Deploy Tanzu Kubernetes Grid Integrated Edition from the management console

Upgrade Tanzu Kubernetes Grid Integrated Edition Management Console

[email protected].



DeployTanzuKubernetesGridIntegratedEditionfromtheManagementConsole

Page last updated:

YoucandeployanewVMwareTanzuKubernetesGridIntegratedEditioninstanceonvSphereeitherbyusingtheVMwareTanzuKubernetesGridIntegratedEditionManagementPortalconfigurationwizardtoguideyouthroughtheconfigurationprocess,orbyimportinganexistingYAMLconfigurationfileintotheYAMLeditor.

Deploy Tanzu Kubernetes Grid Integrated Edition by Using the Configuration Wizard

Deploy Tanzu Kubernetes Grid Integrated Edition by Importing a YAML Configuration File

IfyoudeployTanzuKubernetesGridIntegratedEditionwithplansthatuseWindowsworkernodes,furtherconfigurationisrequired.SeeEnable Plans with Windows Worker NodesforinformationabouthowtoinstallaWindowsServerstemcellandothernecessaryconfigurationactionsthatyoumustperformafteryoudeployTanzuKubernetesGridIntegratedEdition.

[email protected].




DeployTanzuKubernetesGridIntegratedEditionbyUsingtheConfigurationWizard

In this topic

Prerequisites

Step0:LaunchtheConfigurationWizard

Step1:ConnecttovCenterServer

Step2:ConfigureNetworkingConfigureanAutomatedNATDeploymenttoNSX-TDataCenter

ConfigureaBringYourOwnTopologyDeploymenttoNSX-TDataCenter

ConfigureaFlannelNetwork

Step3:ConfigureIdentityManagementUseaLocalDatabase

UseanExternalLDAPServer

UseaSAMLIdentityProvider

OptionallyConfigureUAAandCustomCertificates

Step4:ConfigureAvailabilityZones

Step5:ConfigureResourcesandStorage

Step6:ConfigurePlans

Step7:ConfigureIntegrationsConfigureaConnectiontoVMwareTanzuMissionControl

ConfigureaConnectiontoWavefront

ConfigureaConnectiontoVMwarevRealizeOperationsManagementPackforContainerMonitoring

ConfigureaConnectiontoVMwarevRealizeLogInsight

ConfigureaConnectiontoSyslog

Step8:ConfigureHarbor

Step9:ConfigureCEIPandTelemetry

Step10:GenerateConfigurationFileandDeployTanzuKubernetesGridIntegratedEdition

NextSteps

Page last updated:

ThistopicdescribeshowtousetheconfigurationwizardtodeployTanzuKubernetesGridIntegratedEditiononvSphere.

ForinformationabouthowtodeployTanzuKubernetesGridIntegratedEditionfromaYAML,seeDeploy Tanzu Kubernetes GridIntegrated Edition by Importing a YAML Configuration File.

Forinformationabouthowtoupgradeanexistingdeploymenttothisversion,seeUpgrade Tanzu Kubernetes Grid IntegratedEdition Management Console.

PrerequisitesDeploy the Tanzu Kubernetes Grid Integrated Edition Management ConsoletovCenterServer.



ThevCenterServerinstancemustbecorrectlyconfiguredforTanzuKubernetesGridIntegratedEditiondeployment.ForinformationaboutthevCenterServerrequirements,seeVirtual Infrastructure Prerequisites.

Dependingonthetypeofnetworkingyouwanttouse,yourinfrastructuremustmeettheappropriateprerequisites.Forinformationaboutnetworkingprerequisites,seethefollowingtopics:

Prerequisites for an Automated NAT Deployment to NSX-T Data CenterPrerequisites for a Bring Your Own Topology Deployment to NSX-T Data CenterPrerequisites for a Flannel Network

-Log in to Tanzu Kubernetes Grid Integrated Edition Management Console.

Step0:LaunchtheConfigurationWizard1. OntheVMwareTanzuKubernetesGridIntegratedEditionlandingpage,clickInstall.

View a larger version of this image

2. ClickStart Configuration.


file:///Users/pspinrad/workspace/pdfer/html/docs-pcf-staging.cfapps.io/tkgi/1-8/images/console/console-install-upgrade.png

View a larger version of this image

Togethelpinthewizardatanytime,clickthe?iconatthetopofthepageandselectHelp,orclicktheMore Info…linksineachsectiontoseehelptopicsrelevanttothatsection.Clicktheiiconsfortipsabouthowtofillinspecificfields.

Step1:ConnecttovCenterServer1. EntertheIPaddressorFQDNforthevCenterServerinstanceonwhichtodeployTanzuKubernetesGridIntegratedEdition.

2. EnterthevCenterSingleSignOnusernameandpasswordforauseraccountthathasvSphereadministratorpermissions.

3. ClickConnect.

4. SelectthedatacenterinwhichtodeployTanzuKubernetesGridIntegratedEditionfromthedrop-downmenu.

5. ClickNexttoconfigurenetworking.

Step2:ConfigureNetworking

ProvideconnectioninformationforthecontainernetworkinginterfacetousewithTanzuKubernetesGridIntegratedEdition.TanzuKubernetesGridIntegratedEditionManagementConsoleprovides3networkconfigurationoptionsforyourTanzuKubernetesGridIntegratedEditiondeployments.Eachnetworkconfigurationoptionhasspecificprerequisites.

warning:Ideally,donotdeployTGKIfromthemanagementconsoletoadatacenterthatalsoincludesTKGIinstancesthatyoudeployedmanually.IfdeployingmanagementconsoleandmanualinstancesofTKGItothesamedatacentercannotbeavoided,makesurethattheTKGIinstancesthatyoudeployedmanuallydonotusethefoldernamesBoshVMFolder:pks_vms , BoshTemplateFolder:pks_templates , BoshDiskPath:pks_disk .Ifamanualinstallationusesthesefolder

names,theVMsthattheycontainwillbedeletedwhenyoudeleteaTKGIinstancefromthemanagementconsole.


resources.docs.pivotal.ioprerequisites for a bring your own topology deployment to nsx-t data center...

Documents