Best Practices: vSphere Admin Best Practice Guide for Carbon Black Cloud Workload appliance

Overview

Introduction

The vSphere Admin Best Practice Guide is intended to provide vSphere admins a Best Practice guide when installing, operating, and maintaining the Carbon Black Cloud Workload appliance. The Carbon Black Cloud Workload appliance provides integration between vCenter, Carbon Black Cloud and the VM workload fleet. This Best Practice Guide has been developed in collaboration with other VMware customers that have implemented the Carbon Black Cloud Workload appliance. It will provide insights and helpful best practices for every size enterprise and installation.

Purpose of This Guide

This vSphere Admin Best Practice Guide takes you through the Best Practices for securing, operating, and maintaining the Carbon Black Cloud Workload appliance. You should be familiar with the reference architecture for the Carbon Black Cloud Workload appliance. The assumption is made that the Carbon Black Cloud Workload appliance has been installed properly and is fully functional. For more detailed information, see the Carbon Black Cloud Workload Overview. For additional installation resources see the Additional Resources section of this guide.

The Best Practice Guide will provide best practices in three key areas security, operations, and maintenance.

Audience

This Best Practice Guide is intended for experienced vSphere administrators who are familiar with VMware vSphere and VMware vCenter Server. Familiarity with networking and storage in a virtual environment is necessary and assumed. Knowledge of other technologies, such as IP Addressing, DNS Configuration, and a Linux familiarity is also helpful.

Security Best Practices

Introduction

Listed below are several security best practices that should be adopted when installing, operating, and maintaining the Carbon Black Cloud Workload appliance. While this list is not comprehensive, these best practices should be considered as the minimal security best practice for any installation of the Carbon Black Cloud Workload appliance in a vSphere environment.

Management Interface Segmentation

The management interface (the IP address assigned during appliance creation) for the Carbon Black Cloud Workload appliance should be connected to the segmented management VLAN in the vSphere environment.

This is considered a best practice for any management interface within a vSphere environment.

This exact method of this segmentation will depend on the customer environment. vSS (virtual Standard Switch), vDS (virtual Distributed Switch), or even NSX may provide the needed segmentation. Additionally, access control lists (ACL) that restrict access to authorized administrators and authorized IP addresses should be implemented at the layer three interface to the management VLAN.

Required Connectivity

 Appliance must have HTTPS (443) connectivity to communicate with the vCenter Server and the Carbon Black Cloud. (prod.cwp.carbonblack.io). Connectivity can be verified using the Carbon Black Cloud console (Settings> API Access > Appliance Details) or by connecting to the appliance using SSH and issuing the following curl commands.

curl -v telnet://<carbonblack_prod_url>:443 (Carbon Black Cloud)

curl -v telnet://<vcsa*_on_vc>:443 (vCenter)

_{*vcsa = vCenter Server Appliance}

NTP Access

NTP must be configured on the Carbon Black Cloud Workload appliance. The appliance can use any valid NTP source. This source could be external or internal to a customer’s environment. NTP is configured on the appliance management interface using the IP address or DNS name configured during the appliance deployment.

By default, the appliance is configured for the UTC time zone. The time zone can be changed via a console connection to the appliance VM via vCenter. vCenter and the Carbon Black Cloud Workload appliance must be configured for the same time zone

Appliance Passwords

The password for the appliance expires in 90 days after you deploy the appliance for the first time.

You must reset the password before getting expired. For details, refer to Reset Appliance Password.

You can also extend the password expiration time manually or deactivate the password expiration permanently. For details, refer to Extend Password Expiration Time for Appliance

 

Operations Best Practices

Introduction

This section will highlight some of the operational best practices that are recommended by VMware in collaboration with our customer base.

The Carbon Black Cloud Workload appliance is deployed on Photon OS. A lightweight and extensible operating system, Photon OS is an open-source minimalist Linux operating system from VMware that is optimized for cloud computing platforms, VMware vSphere deployments, and applications native to the cloud. Photon OS is a Linux container host optimized for vSphere.

Command Line Interface

Most virtual appliances provide a management interface GUI for configuration and management. However, the console connection, or a SSH session, are possible alternatives for managing the Carbon Black Cloud Workload appliance. The interface in this case will be a simple command line interface (CLI). The console can be accessed via the vSphere Client, utilizing the web console or remote console functionality. SSH access to the Carbon Black Cloud Workload appliance can be utilized.

Becoming familiar with the command line functionality of Photon OS is a best practice for vSphere admins since this is a common platform for multiple VMware appliances.

Some of the functions available that are available via the CLI are:

Network Configuration: Allow the user to display the current configuration and set the virtual appliance default gateway, host name, DNS, proxy server and configure IP address allocation (DHCP/static). Use the command,

opt/vmware/share/vami/vami_config_net

Process Monitoring: The ‘top’ utility is available on the Carbon Black Cloud Workload appliance. The ‘top’ tool monitors system resources, workloads, and performance. It can unmask problems caused by processes or applications overconsuming CPUs, time, or RAM.

User Account Management: Reset appliance passwords, extend password expiration time for appliance, or change the password complexity requirement.

See Reset Appliance Password

See Extend Password Expiration Time for Appliance

Lifecycle Operations: Check firewall rules, reboot, power off, restart network, etc. Example commands:

sudo iptables --list
​​​​​​​sudo reboot
sudo shutdown
systemctl restart systemd-networkd.service

 

Vulnerability Management

As a vSphere administrator, you need to have visibility of the known vulnerabilities in your environment to understand your security posture and schedule maintenance windows for patching and remediation. This is one of the most important value add features that the Carbon Black Cloud Workload appliance provides. With the help of the vulnerability assessment capability within the Carbon Black Cloud Workload appliance, you can proactively determine the risk in your environment.

You can now monitor known vulnerabilities from the Carbon Black Cloud Workload Plug-in in vCenter. You can discover vulnerabilities from the plug-in Summary tab or from the Vulnerabilities tab and coordinate with your teams to schedule maintenance windows for patches or updates.

To view the vulnerability assessment feature, you must enable Carbon Black in your data center. After enabling Carbon Black, you can typically view vulnerability data within a few minutes. Utilizing this vulnerability functionality should be at the core of a viable vulnerability management program. Successful customers have developed a framework and cadence for utilizing this information. The entirety of the vulnerability information including affected assets can be exported for further clarification and manipulation.

The ability to understand the dynamic nature of certain vulnerabilities is critical to ensuring the security of the vSphere infrastructure. VMware has partnered with Kenna Security to provide this dynamic capability. The vulnerabilities are assigned a scoring number that is known as the Kenna Risk Score. 

Kenna Risk Score

Unlike the Common Vulnerability Scoring System (CVSS) and other static scoring methods, Kenna Security provides the context required to understand the true level of risk that vulnerabilities pose to an organization. Kenna Security ingests, aggregates, and processes billions of pieces of data from internal and external sources, including more than 18 threat and exploit intelligence feeds. Kenna Security then automates the analysis of this data using proven data science algorithms to deliver an accurate, quantifiable risk score for every vulnerability.

For VMware Carbon Black Cloud Workload users, these scores are then immediately reflected within the “Vulnerability” tab of the Carbon Black Cloud Workload Plug-in and are dynamically updated to correspond with changes in the threat landscape. The result is a real-time scoring method that considers a comprehensive set of internal and external data sources to provide full context into the specific amount of risk for every vulnerability, enabling security analysts to truly understand the level of risk and therefore effectively prioritize which vulnerabilities to remediate first. The Kenna Risk Score takes into account what is happening in real-time, in the wild, for each vulnerability. The score then provides an estimate of the likelihood of exploitation to deliver a rank ordering of the probability of exploitation using that particular attack vector.

Inventory and Deployment

After enabling the Carbon Black Cloud Workload Plug-in in vCenter, you can view the inventory that is enabled for Carbon Black protection and view the inventory that is not enabled for Carbon Black protection.

You can now easily monitor and protect the data center workloads from the Carbon Black Cloud Workload Plug-in. The Carbon Black Cloud Workload Plugin provides deep visibility into your data center inventory and end-to-end life-cycle management for the Carbon Black components. Carbon Black protection can be easily enabled for unprotected workloads from the Inventory tab by selecting the workload under the Not Enabled tab. You can enable Carbon Black on Windows and Linux VMs. Eligibility for this functionality is based the version of VMware tools and the operating system.

Windows Virtual Machines: For Windows VMs, the Carbon Black launcher is packaged with VMware Tools. To receive the launcher for your workloads, you must install or upgrade VMware Tools to version 11.2 or later.

Linux Virtual Machines: For Linux VMs, you must manually install the launcher available at VMware Tools Operating System Specific Packages (OSPs). Download and install Carbon Black launcher for your guest operating system from the package repository at Linux CB Launcher. 

Maintenance

Introduction

In this maintenance section there are several best practices for maintaining the Carbon Black Cloud Workload appliance and associated components. There are best practices for the appliance itself and for Carbon Black software.

Appliance Maintenance

Since a virtual appliance is built inside a virtual machine, it is important to create and configure the virtual machine according to best practices. Most virtual appliances are built to deliver a specific server-class application.  As such, the virtual machine itself does not need access to a wide variety of virtual hardware. It is best practice to remove any virtual hardware that your appliance does not need (floppy, CD, USB, etc.).  If you do choose to support these devices, it is best to configure your appliance to start with those devices disconnected. All best practice concerning snapshots is applicable to the Carbon Black Cloud Workload appliance VM.

You can view the overall health status of the Carbon Black Cloud Workload appliance on the Carbon Black Cloud Workload Plug-in. Appliance Worker, vSphere Worker, Gateway, and Access Control Service are appliance services. You can also view the connectivity status of each appliance service on the Carbon Black Cloud Workload Plug-in. You can also view service-wise health status on the Carbon Black Cloud Workload appliance dashboard. The appliance can have one of the following health statuses:

Connected: The appliance is connected.

Disconnected: The appliance is disconnected. If the status is disconnected, make sure that the appliance VM is powered-on. Go to the appliance Registration tab and verify the configurations.

Unhealthy: The appliance is connected, but one of the services is down. The individual appliance services can have Connected or Disconnected status. When the appliance status is Unhealthy, look for individual service statuses. For the disconnected appliance service, you can restart the service as follows.

1. SSH to the Carbon Black Cloud Workload appliance using the admin credentials.
2. Switch to the root user using the sudo su command.
3. Use the appropriate command for the service that you want to restart.

   • systemctl restart cwp-appliance-worker

   • systemctl restart cwp-access-control-service

   • systemctl restart cwp-vsphere-worker

   • systemctl restart cwp-appliance-gateway.service

4. Verify the appliance service status again.

If the Carbon Black Cloud Workload appliance is placed on thin-provisioned or lazy-zeroed thick-provisioned storage, the appliance startup might be slower than it would be if placed on eager-zeroed thick-provisioned storage. 

Carbon Black Maintenance

Appliance and Plug-in

You can upgrade the Carbon Black Cloud Workload appliance automatically by scheduling the upgrade frequency. When a new upgrade bundle becomes available, your appliance is upgraded based on the selected day and time.

1. From your browser, log in to the Carbon Black Cloud Workload appliance at https://<appliance IP address> using the admin credentials.
2. Go to the Appliance > Upgrade page.
3. Select the required day and time for the upgrade.

If not set, the default time zone for the appliance is UTC. Upgrade occurs in the appliance time zone.

After the appliance is upgraded, the Carbon Black Cloud Workload Plug-in is upgraded as well. You can view the updated version or the build number on the appliance dashboard

 

Carbon Black on Virtual Machines

You can quickly update Carbon Black sensors on the virtual machines (VM) where your workloads are running.

To update Carbon Black on all enabled VMs.

1. Log in to the vSphere Client using your administrator credentials.
2. In the left navigation pane, click Carbon Black.
3. Go to the Inventory > Enabled tab.
4. Select one or more VMs for which you want to update Carbon Black, and then click Update.

A confirmation dialog box appears.

5. Click OK.

Carbon Black is updated to the latest available sensor version.

You can also update Carbon Black for the individual VMs. Go to the VM (Windows or Linux) where you want to update, and on the Summary tab, scroll down to the Carbon Black panel. Alternatively, you can also use the Configure > Carbon Black > Security tab.

You can view the sensor version on the Carbon Black panel.

Additional Resources

Installation Video

Installation process video. Installing CBC Workload Appliance in under 15 minutes

Installation Documents

Complete installation documentation Deploy and Configure Carbon Black Cloud Workload appliance

 

Additional Resources

For more information about Carbon Black Cloud Workload, you can explore the following resources:

• VMware Carbon Black Cloud Workload Guide
• Carbon Black Tech Zone
• Photon OS Documentation

Changelog

The following updates were made to this guide:

Date	Description of Changes
10/1/21	Initial draft by DEM

About the Author and Contributors

Dale McKay is a technology evangelist and strategist with deep expertise in security, virtualization, and networking. He has extensive knowledge of a variety of technologies for meeting the strategic and tactical needs of clients. He has strong real-world, hands-on skills in cybersecurity, with his focus being on implementing policies and operating procedures that help his customers address their cybersecurity demands. He is an experienced leader in determining client needs, delivering solutions, and building relationships.

• Dale McKay, Senior Technical Marketing Architect, Network and Advanced Security Business Group, VMware