Monitoring VMware

VMware vSphere provides an innovative mechanism for organizing and viewing any virtual infrastructure built on its platform. Using a unique combination of physical and logical components, this mechanism effectively and efficiently fulfills the VMware vision of the modern virtual infrastructure.

Foglight for VMware accommodates customers of all sizes that leverage the VMware virtualization platform by examining and enhancing eminently knowledgeable VMware view of the virtual world.

VMware vCenter allows for the configuration of a hierarchical organizational structure that resides primarily within the virtual domain. This enables an organization to easily configure physical VMware ESX Servers and virtual machines to reside in logical groups that dictate the various aspects of the virtual infrastructure (like physical object location, resource allocations and limitations for virtual machines, and high availability settings for physical and virtual components).

This section covers the following key areas:

To access the Cloud Manager dashboard, click Monitor VMware on the Welcome page or select VMware from the left navigation pane.

Object roles

Before we get too far into discussing the layout and capabilities of Foglight for VMware, we must understand the different roles the various physical and virtual objects play within the overall virtual infrastructure. The vCenter Server and VMware ESX Servers provide the physical foundation for the vSphere infrastructure. Virtual machines on the other hand are classified as virtual components for the purpose of management and monitoring, even though they have many of the same characteristics (like direct network and storage access) as physical systems. At any given time, a virtual machine must be contained within a single VMware ESX Server. The particular ESX Server in which a given virtual machine is contained may change of course over the lifetime of the virtual machine through the use of unique VMware technologies such as VMware vSphere vMotion or VMware vSphere High Availability (VMware HA).

Physical objects

The physical objects within the VMware virtual infrastructure are those with which you can physically interact. The virtual components or objects that make up the virtual environment cannot exist without the presence of underlying physical components. A VMware ESX Server is an example of a physical component. To have Foglight for VMware monitor a virtual infrastructure, the virtual infrastructure must consist of at least one vCenter Server that is used to manage the virtual infrastructure and at least one ESX Server that is used to runvirtual machines.

Each ESX Server that is used to run virtual machines must have its own managing vCenter Agent installed on it.

ESX Server Hosts

An ESX Server Host is the single physical component required to begin building a virtual infrastructure. An ESX Server provides a hypervisor based architecture for controlling and managing resources for the virtual machines that run on it. The virtual machines running on the host share the resources it provides. Should resources become over-committed, the ESX Server hypervisor determines which virtual machines have priority access to the shared resources (based on manual virtual machine configurations) and distributes the available resources accordingly.

Each ESX Server is managed by a single vCenter Server instance, and can be configured to exist logically within either a datacenter or cluster virtual object within the overall virtual infrastructure.

VMware vCenter Server

Although a vCenter Server can technically exist as a virtual machine, it is considered a physical component within the VMware virtual infrastructure.

VMware vCenter is the software tool used to manage virtual environments that are built on the VMware virtualization platform. vCenter creates a hierarchical structure of virtual objects that enables a system administrator to logically lay out his virtual infrastructure configuration. vCenter also introduces other advanced VMware functionality such as Distributed Resource Scheduling (DRS), VMotion, and High Availability (HA) that can be used to enhance the benefits of a virtual infrastructure.

vCenter provides a robust WSDL that Foglight for VMware leverages to capture and manipulate key characteristics and performance metrics of the various object types and objects found within the virtual infrastructure configuration. Each vCenter instance that is to be monitored using this product must have a VMware Performance Agent configured for it that points to the Web service interface. As mentioned in the Foglight for VMware Installation Guide, this agent can be installed on the vCenter Server itself because all of the required components for the proper operation of the agent come pre-configured.

A single vCenter Server can monitor approximately 100 VMware ESX Servers and 1500 virtual machines before performance and scalability challenges demand the introduction of a second vCenter Server. Multiple vCenter instances can be disbursed geographically to localize the management of large, distributed vSphere implementations.

Virtual objects

Virtual objects can exist only within the confines of the virtual infrastructure. With the exception of virtual machines, virtual objects are logical and are used for organizing VMware ESX Servers and virtual machines, either geographically or by function. In addition, virtual objects allow for the advanced configuration of resource management and of high availability settings.

Virtual machines

The creation and subsequent use of virtual machines is the primary purpose for building and maintaining a virtual infrastructure. Virtual machines share many of the characteristics of physical systems (like storage and network interaction), but they do not have direct access to the hardware that is used to process their information and they are considered virtual components within the virtual infrastructure.

A virtual machine encompasses more than just a guest operating system like Microsoft Windows. A virtual machine also contains specific configurations that help to define it, such as the number of processors and the amount of memory it can leverage.

All of the resource utilization for a particular virtual machine on a VMware ESX Server is scheduled through that Server’s hypervisor. The efficient tracking and analysis of this scheduling of resources at both the virtual machine and the ESX Server Host level is a key function provided by Foglight for VMware. At any given time a virtual machine must reside on a single VMware ESX Server, but it can be moved across physical ESX Servers, typically without downtime, through the use of key vCenter functionality called VMotion.

VMotion provides a method for proactively moving a virtual machine from one ESX Server to another while avoiding the downtime that can arise from having to perform actions like patching a physical host server. VMotion also provides a manual method a system administrator can use to better balance virtual machine workloads based on resource utilization trends.

A feature called Migration Modeler provides a method for analyzing the impact of using vMotion to move a virtual machine between two VMware ESX Servers in a cluster. Migration Modeler provides this functionality without you actually having to move the virtual machine.

Foglight for VMware also provides a mechanism that tracks the life cycle of the virtual machines within the virtual infrastructure. This enables you to quickly and easily view a history of a virtual machine’s performance metrics and a history of its logical location within the virtual infrastructure.

VMware vCenter offers some additional valuable features that customers may wish to use including the VMware Distributed Resource Scheduling (DRS) feature for automating the process of balancing VMware ESX Server utilization and the VMware High Availability (HA) feature for recovering from host failure within a cluster.

Datacenters

A datacenter is the topmost virtual object within a vCenter Server implementation and is required before any VMware ESX Server Hosts can be added to a vCenter. A datacenter is most commonly used to identify the physical boundaries within which an ESX Server Host can exist. In most implementations these boundaries constitute a single physical location that contains a large number of ESX Server Hosts. There is no hard and fast rule stating that a datacenter must exist entirely at just one physical location, but other datacenter implementations are atypical of most virtual infrastructures. Within the boundaries of a datacenter, objects of the same type cannot have the same name. For example, it is not possible to configure two ESX Server Hosts with the same name to reside within the same datacenter. The same goes for virtual machines, clusters, resource pools and any other objects that can be created and configured to reside within a datacenter. Objects of the same type can have identical names as long as they are located in different datacenters.

Datastores

The management of datastores is carried out at the both the datacenter and the ESX Server levels.

Each datastore is contained within a datacenter and must be uniquely named within its containing datacenter.

A datastore represents a storage location for virtual machine files. The storage location can be a local file system path, a Virtual Machine File System Storage (VMFS) volume, or a Network Attached Storage directory.

ESX Server Hosts can be configured to mount a set of network drives (or datastores). For each storage location within a datacenter there is only one datastore, so multiple hosts may be configured to point to the same datastore.

Whenever an ESX Server Host accesses a virtual machine or file within a datacenter it must use the appropriate datastore path. Each datastore object keeps a record of ESX Server Hosts that have mounted it, and a datastore object can be removed only if no hosts are currently mounting that datastore.

Datastores are host-independent and platform-independent. Therefore, they do not change in any way when the virtual machines contained within them are moved from one ESX Server to another.

VSAN datastores

Virtual SAN (VSAN) is a software component running on the ESXi hypervisor. It collects storage resources associated with a cluster and creates a storage pool that is accessible to all hosts on the cluster. When Virtual SAN is enabled on a cluster, a VSAN datastore is created in your environment. VSAN datastores are collections of storage elements that are available to the hosts.

Clusters

A cluster object is a group of VMware ESX Servers that share common storage resources and network configurations. A cluster represents a pool of the combined resources of all of the ESX Server Hosts assigned to the cluster. For example, if four ESX Servers are added to a cluster and each ESX Server has 2x2 GHz processors with 4 GB of memory, the cluster represents a pool of 16 GHz of CPU processing power and 16 GB of memory that is available for use by virtual machines.

A cluster also serves as the boundary for virtual machine migration activity through the VMware vMotion or VMware HA features. When using either of these technologies for virtual machine migration it is critical that the participating ESX Server Hosts have identical storage resource and network configurations, and this is guaranteed within a cluster by the very definition of a cluster.

vApp

A vApp is a group of virtual machines that can be managed as a single object. vApps simplify management of complex, multi-tiered applications that run on multiple interdependent virtual machines. vApps have the same basic operations as virtual machines and resource pools. With vApps, you can set the order in which the virtual machines in the vApp power on, automatically assign IP addresses to virtual machines in the vApp, and provide application level customization.

vApps offer:

  • Container for multiple virtual machines.
  • Resource controls for the VMs inside the container.
  • Network configurations contained inside.
  • Portability of the vApp such that everything can be contained and transferred to another virtual infrastructure.
  • Entire vApps can be powered on, powered off, suspended, or shutdown.
  • Entire vApps can be cloned.

Resource pools

Resource pools enable an administrator to fine tune resource allocations within a cluster. A resource pool can be configured to leverage a portion of the overall available resources within a cluster and then virtual machines can be assigned to that resource pool. This enables an administrator to prioritize virtual machines—to either limit or guarantee certain resources to a particular virtual machine or group of virtual machines.

Resource pools can be configured in many ways, from simple to complex. For a simple example, two resource pools are configured within a cluster; one is named Production Virtual Machines and the other is named Development Virtual Machines. The Production resource pool is configured with a “High” share priority and the Development resource pool is configured with the default “Normal” share priority. In this case any virtual machine residing in the Production resource pool is automatically given twice the priority, in terms of access to system resources during periods of contention, of any virtual machine residing in the Development resource pool.

To better demonstrates the true potential of using resource pools, the following is an advanced example. Four ESX Servers are added to a cluster and each ESX Server has 2x2 GHz processors with 4 GB of memory. The cluster therefore represents a pool of 16 GHz of CPU processing power and 16 GB of memory that is available for use by virtual machines. The figure below illustrates that the Production Cluster resource that resides in the Chicago datacenter has 16 GHz of processing power and 16 GB of memory. A resource pool is created for a CRM Application that has access to 8 GHz of the cluster’s total CPU resources and 6 GB of the cluster’s total memory.

By drilling down further from there you see that within the CRM Application resource pool there are two more resource pools (Database and Web). The existence of the Database resource pool ensures that key database virtual machines have access to the resources necessary to perform their highly transactional operations. The web servers have access to a smaller portion of the overall resources—just enough to provide the necessary end-user responsiveness from a web transaction perspective without impacting the key backend database infrastructure.

To assist with the understanding of these nested relationships of virtualized objects, Foglight for VMware provides both a Topological and a Hierarchical view of the entire virtual infrastructure as well as resource pool mapping functionality for maximum flexibility in tracking advanced virtual infrastructure configurations.

Virtual switches

Your VMware environment uses virtual switches to distribute network traffic. A stand-alone ESX host typically uses a standard virtual switch to manage network traffic to and from virtual machines running on that host. Distributed virtual switches manage configuration of proxy switches, enabling communication between a virtual machine using the distributed switch and any of these components:

  • any other virtual machines on any ESX hosts that use the same distributed switch
  • virtual machines that use a standard virtual switch
  • remote systems on a physical network connected to the same ESX host

In addition to standard and distributed virtual switches, your monitored environment may also include one or more Cisco virtual switches. A Cisco virtual switch is a third-party distributed virtual switch that manages network traffic between virtual machines and other components in your integrated virtual infrastructure.

Folders

Folders are hierarchical components that exist within a vCenter and they enable an administrator to more easily organize the virtual environment for manageability. There are three different types of folders that can exist within the various layers of the virtual infrastructure hierarchy. The following table lists the available types of folders, and explains the levels at which they can exist and the objects they can contain.

Folder Type Level at Which It Can Exist Objects It Can Contain
Datacenter vCenter Root Datacenters
Virtual Machine Datacenter Virtual Machines and Templates
Compute Resources Datacenter Hosts and Clusters

Folders may contain nested folders of the same type, but not of other types. It is not possible, for example, to create a virtual machine folder within a datacenter folder.

Folders are provided strictly for organizational and management purposes. They offer a way for an administrator to classify objects that is not tied to (and therefore bound by) the virtual/physical relationship framework. For example, two datacenter folders are created at a vCenter root; one folder is labeled Primary Datacenters and the other is labeled Disaster Recovery Datacenters. An administrator can configure multiple primary datacenters containing production ESX Servers, place those datacenters in the Primary Datacenters folder, and then assign the necessary permissions to that folder to allow standard users to perform management tasks for the entire primary virtual infrastructure. The administrator can then configure multiple disaster recovery datacenters containing disaster recovery ESX Servers, place those datacenters in the Disaster Recovery Datacenters folder, and assign a different set of permissions to that folder. This prevents standard users from building virtual machines that may take over resources that are necessarily dedicated to HA-configured disaster failover virtual infrastructure components.

Using Foglight for VMware, you can observe either a Topology view that does not use folders and presents a logical breakdown of the virtual infrastructure by component, or a Hierarchy view that uses folders and presents the familiar interface that is found within the vCenter management server.