NSX Advance Load Balancer

Welcome to NSX Advance Load Balancer (ALB) lab.

Through this lab, you will learn how to integrate the VMware NSX ALB with the latest version of VMware Cloud Director (VCD) 10.2 and test the features and functionality of the ALB with the integration.

Topics that will be covered are:

  • Pre-requisite before integration
    • vCenter pre-requisites
    • NSX-T pre-requisites
    • VMware Cloud Director pre-requisites
  • NSX ALB integration
    • NSX ALB startup
    • Integrating with NSX-T Manager
    • Generating ALB certificate for VCD use
    • Integrating NSX ALB with VCD
  • Load Balancing tenant workloads
    • Adding Server Pool
    • Adding Virtual Service
    • Test Load Balanced Servers

Overview

This section provides a high-level overview of the integration between NSX ALB, NSX-T and VMware Cloud Director.

If you would want to proceed directly to the lab, please click >>HERE<<

Infrastructure Overview

The image above shows the communication path for the NSX ALB integration.  
NSX ALB will add NSX-T manager as a "cloud" and within that integration, it will also link to the vCenter that NSX-T had integrations with.
VMware Cloud Director will then add the NSX ALB Controller and the respective Service Engine Groups (that are prepared for VCD use).

Dataplane Overview

This section will elaborate on the dataplane topology implementation for VCD's integration with NSX ALB.
There are two types of implementation:

  • Dedicated Service Engines
  • Shared Service Engines

Dedicated Service Engine Group

The diagram above depicts the topology if a dedicated ALB Service Engine Group is created for a tenant . For this implementation, Service Engines are dedicated to a particular tenant and no other tenants will share these SEs. This is a good use case for tenants that want dedicated performance for load balancing their services or when they want total data isolations (e.g. financial institutions or government requirements).

Shared Service Engine Group

The diagram above depicts the topology if shared Service Engine Group is created for multitenant utilization. This implementation is a recommended implementation as it allows shared service engine resources to be fully utilized. We also do not need to worry too much on over-sizing as NSX ALB can auto-scale the service engines in the service engine groups on demand.

Pre-Requisites Before Integration

Before the integration is being done, there are pre-requisites that are to be met:

  • NSX-T already integrated with VCD
  • VCD running on version 10.2 and above
  • NSX-T running on version 3.0 and above
  • NSX ALB running on version 20.1.1 (versions higher are not yet tested)
  • vSphere content library has to be created
  • A management segment has to be created for the ALB Service Engines (SE) with DHCP enabled
  • VCD's NSX-T Network Pool Transport Zone has to be same as the Transport Zone selected on NSX ALB

vCenter Pre-Requisites

In this section, we will prepare the pre-requisites on vCenter to integrate with NSX ALB.

Login to the vCenter

Access the webUI of vcsa-01a and key in the credentials to access vCenter:

  1. Click on the vCenter shortcut on the browser
  2. Username: administrator@corp.local
  3. Password: VMware1!
  4. Click Login

Creating A Content Library

When integrating NSX ALB, you will be required to create a Content Library. This is to allow NSX ALB Controllers to place the image files of the Service Engines so that it can request on-demand creations of Service Engines to the vCenter

Go to Content Libraries

From vCenter's main page:

  1. Click on Menu
  2. Select Content Libraries
Create New Content Library

On the Content Library page click on +Create.

Provide Name and Location for Content Library

On the first page of creating a content library:

  1. Provide content library name of alb-content-lib
  2. Click Next
Provide Content Library Configuration

Leave all settings default and click Next.

Select storage for Content Library

To select the storage:

  1. Select RegionA01-ISCSI01-COMP01
  2. Click Next
Review New Content Library Creation

After reviewing the configuration to be correct, click on Finish.

NSX-T Pre-Requisites

In this section, we will need to prepare NSX-T for the NSX ALB integration. We will need to create the Tier-1 router (in orange) as the management segment gateway for the Service Engines that will be created by the ALB Controller. This will allow management plane communication between the ALB Controller and the Service Engines.

This segment will also have to be enabled with DHCP in order for the SEs to automatically obtain IPs when provisioned. In our lab, we will utilize the DHCP service from NSX-T.

Do note that the SE Management Segment has to be advertised from Tier-0 to the upstream network so that they will know how to reach it.

Login to NSX-T Manager

To access the NSX-T Manager:

  1. Access the browser and click on the NSX-T shortcut
  2. Username: admin
  3. Password: VMware1!VMware1!
  4. Click Log In

Creating NSX-T Tier-1 Gateway

The NSX-T Tier-1 Gateway will be used as the gateway for the NSX ALB Service Engine's gateway. This is an infrastructure requirement from NSX ALB integration with NSX-T. This NSX-T Tier-1 gateway will then be connected to a Tier-0 gateway which can be the same Tier-0 router as the tenant's external network or it can be a separate Tier-0 router (if you would like to have a segregation of management and tenant networks. In this lab, we'll utilize the same Tier-0 router as the tenant's external network to conserve resources.

Adding NSX-T Tier-1 Gateway

To add a Tier-1 gateway:

  1. Go to Networking
  2. Select Tier-1 Gateways
  3. Click on Add Tier-1 Gateway
Fill in the Tier-1 details

Configure these settings:

  1. Tier-1 Gateway Name: nsxalb-se-mgmt
  2. Linked Tier-0 Gateway: nextgen-gw
  3. Edge Cluster: nextgen-edge-cluster
  4. Click Save
Continue Configuring Tier-1 Gateway
  1. Click Yes to continue configuring nsxalb-se-mgmt Tier-1 router
Set DHCP Server on nsxalb-se-mgmt Tier-1
  1. On the DHCP setting option, click on Set DHCP Configuration
Set DHCP Configuration
  1. Select DHCP Server from the Type dropdown menu
  2. DHCP Server Profile, click on the 3 dots (menu) button
  3. Select  Create DHCP Profile  
Create DHCP Profile
  1. Profile Name: DHCP-Profile
  2. Edge Cluster: nextgen-edge-cluster
  3. Click Save

TIPS: We can leave Server IP Address blank and it would default to 100.96.0.1

Save DHCP Server Configuration
  1. Click Save to save the DHCP configuration

To advertise the nsxalb-se-mgmt connected segments:

  1. Expand Route Advertisement
  2. Toggle All Connected Segments & Service Ports
  3. Click Save
Close Editing Mode
  1. Click Close Editing at the bottom of the edit dialog to exit edit mode and return to the list of Tier-1 Gateways.

Create NSX ALB Service Engine Segment

The NSX ALB Service Engines management vnic will need to connect to an NSX-T Segment. This Segment will be connected to the nsxalb-se-mgmt NSX-T Tier-1 router that was created in the previous step, then given a gateway IP.

Adding a Segment

To create a Segment:

  1. Go to Networking
  2. Select Segments on the left pane
  3. Click on Add Segment
Fill in the details

Configure these settings:

  1. Segment Name: nsxalb-se-seg
  2. Select nsxalb-se-mgmt Tier-1 Router
  3. Select TZ-HOL-Overlay Transport Zone
  4. Subnets: 192.168.160.1/24
  1. Scroll down a little
  2. Click Save
Continue Configuring Segment

It will prompt you on whether you want to continue the Segment configuration:

  1. Click Yes
Setting up DHCP for NSX ALB Management Segment
  1. To set up DHCP for this particular segment, click Set DHCP config
Set DHCP config
  1. Select DHCP Type to be Gateway DHCP Server from dropdown
  2. Ensure DHCP Profile is DHCP-Profile
  3. Enable DHCP Config
  4. Set DHCP Ranges to 192.168.160.11-192.168.160.200
  5. Click Apply
Save the Configuration
  1. Scroll down a little
  2. Click on Save
Close Editing for Segment
  1. Finally, Close Editing at the bottom

VMware Cloud Director Pre-Requisites

For VCD pre-requisite, we must note down the Transport Zone used for the NSX-T network pool. This is required when integrating NSX ALB with NSX-T to ensure successful integration with VCD.

Login to VMware Cloud Director

To access the VMware Cloud Director provider portal:

  1. Access the browser and click on the vCD - Provider shortcut
  2. Username: admin
  3. Password: VMware1!
  4. Click Sign In

Review Network Pool Configuration

To review the configuration:

  1. Go to Resources
  2. Ensure it is on Cloud Resources
  3. On the left pane, select Network Pools
  4. Click on regionA name

Note the Transport Zone used

  1. Note down the Transport Zone: TZ-HOL-Overlay

NSX Advance Load Balancer (ALB) Bringup and Integration with NSX-T

In this lab, we have already deployed the NSX ALB controller from the ova file to save time. The ova deployment will request for the infrastructure details e.g. which ESXi host to install on and what IP address to use for the management IP.

You will begin the NSX ALB from first bootup, then integrate it with NSX-T manager.

NSX ALB Startup

At startup, NSX ALB will request for more configuration settings to complete the installation.  

Create ALB Controller Administrator Account

Access the NSX ALB webUI via the browser:

  1. Click on the Avi Controller shortcut
  2. Password & Confirm Password: VMware1!
  3. Click on Create Account

Provide System Settings

Provide system settings for:

  1. DNS Resolver(s): 192.168.110.10
  2. DNS Search Domain: corp.local
  3. Backup Passphrase & Confirm Backup Passphrase: VMware1!

TIPS: 192.168.110.10 is the vPod's AD/DNS server.

  1. Scroll down to the end
  2. Replace the first NTP Server to 192.168.100.1 (from 0.us.pool.ntp.org)
  3. Click Next

TIPS: 192.168.100.1 is the vPod Router which is the vPod's ntp server.

Provide Email/SMTP settings

  1. Leave everything default and click Next

Orchestrator Integration

  1. Select No Orchestrator

INFO: NSX-T Integration selection is not visible here yet. It'll be done at a later stage.

TIPS: You could also select 'VMware', but it'll be a redundant configuration because using NSX-T Integration will include vCenter integration.

Choose Support of Multi-Tenancy

  1. Select Yes for multiple tenant support

TIPS: It doesn't really matter whether it is Yes or No with VCD integration because VCD will be the multi-tenant portal, however we still choose Yes here because we don't want to take away the option to support multi-tenant from NSX ALB level when choose to do so in the future.

Tenant Settings

Under the multi-tenant settings:

  1. Select Per tenant IP
  2. Select Service Engines are managed within the tenant context, not shared across tenants
  3. Click on Complete

TIPS: Again, it doesn't really matter which one you choose in the context of VCD integration, but if there wasn't any VCD integration, you will select as such to provide self-service multi-tenant load balancing infrastructure to your tenants.

Integrating with NSX-T

After completing the NSX ALB startup, you'll be greeted with the NSX ALB Dashboard. You'll need access the NSX-T integration page

  1. Click on the top left side corner menu
  2. Select Infrastructure

Creating the NSX-T Cloud

To create the NSX-T cloud:

  1. Click on Clouds
  2. Click on Create
  3. Select NSX-T Cloud

Configuring ALB NSX-T Cloud

  1. Name: nsxt-cloud
  2. Object Name Prefix: alb

Configuring ALB NSX-T Cloud (cont.)

Under NSX-T Credentials section:

  1. Scroll down a little
  2. NSX-T Manager Address: nsx-mgr.corp.local
  3. Click on the 3 dots (options menu)
  4. Click on Create
Add NSX-T Credentials

To create NSX-T credentials for ALB's usage:

  1. Name: nsxt-cred
  2. Credentials Type select NSX-T
  3. Username: admin
  4. Password: VMware1!VMware1!
  5. Click Save

Connect to NSX-T Manager

  1. Click on Connect to connect to NSX-T Manager

Select Transport Zone

  1. Select TZ-HOL-Overlay

WARNING! This MUST be the same transport zone as the one used in VCD. We have already noted this down in the VCD Pre-requisite step.

Select Management Network Segment

  1. Select earlier created Tier1 Logical Router ID: nsxalb-se-mgmt
  2. Select earlier created Segment ID: nsxalb-se-seg
  3. Click Add
  4. Select same Tier1 router: nsxalb-se-mgmt
  5. Select same Segment:  nsxalb-se-seg

Configure vCenter Servers

  1. Scroll down a little more
  2. Under vCenter Server(s), click Add
Adding vCenter Server
  1. Name: vcsa-01a
  2. Select 192.168.110.22 on the dropdown menu
  3. Click on the 3 dots (options menu)
  4. Click on Create
Add vCenter Credentials
  1. Name: vcsa-01a-cred
  2. From the dropdown menu, select vCenter as the Credentials Type
  3. Username: administrator@corp.local
  4. Password: VMware1!
  5. Click on Save
Connect to vCenter
  1. Click on Connect to view the Content Library
Select Content Library
  1. Select the alb-content-lib created in the vCenter Pre-requisite
  2. Click on Done

Finish Setting Up NSX-T Cloud

  1. Click Save to finish

Generate ALB Certificate for VMware Cloud Director

By default, the NSX ALB uses a default SSL/TLS security certificate in which VCD will reject. You can also use your organization's valid signed certificate, but for simplicity of the lab, we will create a new self-signed certificate (that meets VCD's requirements).

Access the Security Page UI

To access the certificate page, you will need to:

  1. Click on top left corner menu
  2. Select Templates
  3. Select Security tab
  4. Click on Create
  1. Select Controller Certificate

Add Certificate (SSL/TLS)

Key in these configuration:

  1. Name: avicorplocal
  2. Common Name: avi-controller
  3. Organization Unit: HOL
  4. Organization: VMware
  5. Locality or City: Palo Alto
  6. State Name or Province: California
  7. Country: US

Add Certificate (SSL/TLS) (cont.)

  1. Scroll down to the end
  2. Subject Alternate Name (SAN): avi-controller.corp.local
  3. Click on Save

Apply New Self-Signed Certificate

After creating the new self-signed certificate, we will need to apply it:

  1. Click on top left corner menu of the webUI
  2. Select Administration
  3. Select the Settings tab
  4. Choose Access Settings sub-tab
  5. Click on the pencil (edit) button on the right

Replace the SSL/TLS Certificate

Delete the 2 existing SSL/TLS Certificate:

  1. Delete System-Default-Portal-Cert
  2. Delete System-Default-Portal-Cert-EC256

Replace the SSL/TLS Certificate (cont.)

Add the newly created self-signed certificate:

  1. From SSL/TLS Certificate dropdown menu, select avicorplocal
  2. Click on Save

INFO: After clicking on Save, we will have to refresh our web browser because the SSL certificate that we are using has changed.

Create Service Engine Group for VCD

To access the Service Engine Group configuration:

  1. Click on top left corner menu
  2. Select Infrastructure
  3. Select the Service Engine Group tab
  1. Select Cloud you will select nsxt-cloud
  2. Click on Create on the right

Configure Service Engine Group Settings

This is where you configure the Service Engine Group. Here we are trying to configure a group for VCD to use as a shared service engine mode

  1. Service Engine Group Name: VCD-Shared-SE-Group
  2. Max Number of Service Engines: 2
  3. Click on Save

INFO: We only select 2 for Max number of SEs because we don't want it to accidentally grow too huge in our resource-constraint lab.

TIPS: VCD will use one SE group per shared or dedicated SE pool assignment. You will need to create more if you want provide few dedicated SE pools or separate shared SE pools.

TIPS: Under the Advance tab, you will see Buffer Service Engines which is set to 1. Hence, when first Virtual Service is created, 2 Service Engines will be created immediately, because 1 is in use, another 1 is for buffer.

VMware Cloud Director Integration

After setting up the infrastructure, we move on to integrating the VMware Cloud Director with NSX ALB.

Login to VMware Cloud Director

To access the VMware Cloud Director provider portal:

  1. Access the browser and click on the vCD - Provider shortcut
  2. Username: admin
  3. Password: VMware1!
  4. Click Sign In

Add NSX ALB Controllers to VCD

To add an NSX ALB Controller to VCD:

  1. Go to Resources
  2. Select the Infrastructure Resources tab
  3. Click on Controllers under the NSX-ALB on the left pane
  4. Click on Add

Add Controller

Key in the configurations for the NSX ALB Controller:

  1. Name: avi-controller
  2. URL: https://avi-controller.corp.local
  3. Username: admin
  4. Password: VMware1!
  5. Enterprise License toggle green
  6. Click on Save

TIPS: If you do not toggle the enterprise license, the NSX ALB will only allow the basic edition features. For this lab, we will use Enterprise License

Trust NSX ALB Certificate

VCD will ask you whether you want to trust this NSX ALB Controller:

  1. Click on Trust

Add NSX-T Cloud from NSX ALB

To add an NSX-T Cloud from NSX ALB:

  1. Select NSX-T Clouds under the NSX-ALB on the left pane
  2. Click on Add

Add NSX-T Cloud

Configure these settings:

  1. Select avi-controller from the dropdown menu
  2. Name: avi-controller
  3. Select nsxt-cloud from the Available Clouds
  4. Click on Add

Add Service Engine Group from NSX ALB

To configure the available Service Engine Groups from NSX ALB into VCD:

  1. Go to the Service Engine Groups under NSX-ALB on the left pane
  2. Click Add

Add Service Engine Group

Configure these settings:

  1. Select avi-controller from the NSX-T Cloud dropdown menu
  2. Reservation Model select Shared
  3. Name: VCD-Shared-SEGroup-01
  4. Select VCD-Shared-SE-Group under the Available Service Engine Groups
  5. Click on Add

TIPS: Reservation Shared will allow multiple tenants utilize the same Service Engine group. This will allow providers to overprovision the service engines for best ROI. Reservation Dedicated will only allow one tenant to utilize that service engine group - hence dedicated

Load Balancing Workloads

The topology above depicts how the current setup is. There is an organization "e-corp" with their organization VDC of e-corp-nextgen-paygo that has 3 virtual machines already deployed on the NSX-T backed Provider VDC (nextgen-resources). Their requirement is to load balance those 3 web VMs and front it with the virtual IP of 100.100.3.10.

INFO: 100.100.3.0/24 subnet has been allocated to tenant e-corp as an VCD External Network

Accessing the Tenant Portal

To access the e-corp tenant:

  1. Click on Resources on the top menu
  2. Select Cloud Resources on the tab below
  3. Select Organizations on the left pane
  4. Click on the popup menu beside "e-corp"

Accessing the Organization VDC

  1. The tenant portal will pop up and you will need to click on the organization VDC e-corp-nextgen-paygo

Configuring Load Balancer under Edge Gateway

To access the load balancer configuration, you must access the edge gateway:

  1. Select Edges on the left pane
  2. Select the existing Edge Gateway ecorp-gw

Enable Load Balancer Service

Before using the load balancer, we'll need to enable it first:

  1. Click on General Settings under Load Balancer
  2. Select Edit
Edit Load Balancer General Settings
  1. Toggle Load Balancer State to Active
  2. Click on Save
What Happens on the Backend?

TIPS: Service Network Specification is to define the IP address range for the internal VIPs. If we Use Default, VCD will assign the 192.168.255.0/25 range. This segment will attach to the Tier-1 gateway of the tenant. You can choose to change this if the default subnet clashes with an existing network of the tenant

Adding Service Engine Group to Tenant Org VDC

After enabling the Load Balancer service, we will see more options under Load Balancer. Next, we have to select the Service Engine Group to be assigned to this tenant:

  1. Click on Service Engine Groups
  2. Click on Add
Add Service Engine Group

This window will allow the tenant to consume the earlier added Service Engine Group. Maximum allowed virtual services is to allow the tenant to create a maximum number of virtual services. Reserved is the number of Service Engines reserved for this tenant from the pool.

  1. Select VCD-Shared-SEGroup-01
  2. Maximum Allowed: 2
  3. Reserved: 0

INFO: This will allow the tenant to configure maximum of 2 virtual services and 0 service engines will be reserved for this tenant. This is inline with provider over-provisioning concepts.

Adding Server Pool

To load balance workloads, we must first create the pool in which the servers in that pool will be load balanced.

  1. Click on Pools
  2. Click on Add to add a new server pool
General Settings for Load Balancer Pool
  1. Name: web-cities-pool
  2. Load Balanacer Algorithm: Round Robin
  3. Active Health Monitor: Add Monitor
  4. Select HTTP
Members for Load Balancer Pool
  1. Click on Members Tab
  2. Click on Add
  3. Address: 172.16.30.21
  4. Port: 80
  5. Do the same for 172.16.30.22 and 172.16.30.23
  6. Click Save

Adding Virtual Service

Virtual Service is where we configure the Virtual IP of the service we want to load balance. To configure the virtual service:

  1. Click on Virtual Services
  2. Click on Add
Add Virtual Service for the Load Balancer
  1. Name: web-cities-vs
  2. Service Engine Group: VCD-Shared-SEGroup-01
  3. Load Balancer Pool: web-cities-pool
  4. Virtual IP: 100.100.3.10
  5. Service Type: HTTP
  6. Click Save
What Happens on the Backend?

Because this is an as-a-service service, after completing the creation of the virtual service, you will see the service engines being provisioned on vCenter.

From NSX ALB Infrastructure > Service Engine > nsxt-cloud you will see the same thing

From topology point of view, the blue Tier-1 will create a static route for VIP 100.100.3.10 towards the specific Service Engine that holds the VIP via the 192.168.255.x VCD LB segment network.

Verifying Service is Up from NSX ALB

From the NSX ALB webUI

  1. Click on the top left corner menu
  2. Select Applications
  3. Go to Virtual Services tab

We should see the corresponding VIP (100.100.3.10) and the Health is Green and the

Verifying Services is Up from VCD Tenant Portal

From the VCD Tenant Portal:

  1. Select the e-corp-nextgen-paygo on Datacenter tab
  2. Click on Edges
  3. Select ecorp-gw
  4. Click on Virtual Services

We should be able to see the Health is Up

Testing Load Balanced Servers

Once all the Service Engines are up and healthy, we should be able to access the Virtual IP.

  1. From the browser, go to http://100.100.1.30
  2. Click on the browser Refresh button

You should be able to see the IP address of the corresponding backend server and a picture (New York City, Singapore or Paris) every time we press the refresh.

View Virtual Service Statistics

Under the ecorp-gw

  1. Click on the arrow button

We should be able to see the virtual service statistics just like NSX ALB UI itself