TrueSight Capacity Optimization (TSCO) - Discussion of VMware/ESX memory utilization measures (Consumed vs. Active Memory)

Version 38
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    BMC Performance Assurance for Servers


    APPLIES TO:

    TrueSight Capacity Optimization 11.5, 11.3.01, 11.0, 10.7, 10.5, 10.3, 10.0 ; BMC Performance Assurance 9.5



    PROBLEM:

     

    Discussion of VMware/ESX memory utilization measurements

    Applies To:

    TrueSight Capacity Optimization 11.5, 11.3.01, 11.0, 10.7, 10.5, 10.3, 10.0
    BMC Performance Assurance 9.5, 9.0, 7.5.10

     

     


    SOLUTION:

     

    Here are some references about ESX VM guest memory.

    The vSphere Resource Management Guide, section on "Managing Memory Resources", explains the meanings of the two memory counters:

    1. When memory is overcommitted, ESX estimates the working set size of each VM, to allocate machine memory among different virtual machines. A VM that is actively using its memory tends to be given more machine memory. This estimated "actively used" memory is measured by the "Active memory" counter.

    2. The machine memory that is being used for a given VM is counted by the "Consumed memory" counter.

    So, the consumed memory corresponds to the portion of the machine memory capacity that is used by a VM. This is why BMC is recommending that as a good measure for capacity management. The "Active memory" counter is more meaningful for ESX's internal algorithms.


    As for why it's still necessary to look at Active memory:

    Active memory is useful primarily for understanding the detailed behavior of a single VM. An application may have periods of large working sets, during which ESX will be forced to give it larger allocations of machine memory.

      

     Unfortunately, "consumed" memory or any other metric visible to vSphere, does not tell you how much of the memory in a guest is actually being used. If you are trying to find the right amount of memory to allocate to the VM, then it is important to understand the characteristics of the processing inside the guest.

      
       
    • Some operating systems tend to assign the entire amount of memory they see in the "machine" for various kinds of caches, so as to get the best possible performance: filesystem caches are popular.
      
       
    • Some database and similar workloads tend to grab as much memory as they can get, so that they can use large buffers and do most of their work in memory. Again, the purpose is to maximize their performance by using as much memory as the "machine" has.
      

    The above kinds of behaviors are virtualization-unfriendly, because to the hypervisor, this makes it look as though all of the allocated memory is being used. The problem is, vSphere cannot tell whether the guest workload is using this kind of greedy policy or not.

      

    It is possible to experiment with guests, starving them of memory gradually to find out when they start misbehaving, but this kind of trial and error exploration is not foolproof, because the business demand is varying, and because users don't appreciate it when performance suffers due to an experiment.

      

    The only way to find out the memory demand of a virtual machine is to:

      
       
    • Understand its internal structure, so that you know which performance indicators to track;
    •  
    • Collect these indicators by instrumenting the OS or application as needed.
      

    =====================================================

      

     Here is a description of the metrics from the VMware API:

      

     

                                                                                     

    usage

    absolute

    percent

    1 (4)

    average (minimum) (maximum) (none)

    Memory Usage (Average)
    Memory Usage (Minimum)
    Memory Usage (Maximum)
    Memory Usage (None)

    ο

    ο

    Memory usage as percentage of total configured or available memory.

          
           
    • Virtual machine:   Percentage of configured virtual machine “physical” memory:
      active ÷ virtual machine configured size
    •      
    • Host:   Percentage of available machine memory:
      consumed ÷ machine-memory-size
    •      
    • Cluster:  memory usage = memory consumed + memory overhead ÷ effectivemem
    •     
      

     

      

     And definitions from www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference/memory_counters.html#active:

      

     

                                                                                                                                                                                                                                   
    COUNTERSTATSTYPEUNITLEVELSROLLUPTYPESLABELS     ENTITY
    VHRC
    activeabsolutekiloBytes2 (4)average
    (minimum)
    (maximum)
    Memory Active (Average)
    Memory Active (Minimum)
    Memory Active (Maximum)
    Amount of memory that is actively used, as estimated by VMkernel based on recently touched memory pages.      
           
    • Virtual machine:   Amount of guest “physical” memory actively used.
    •      
    • Host:  Sum of all active metrics for all powered-on virtual machines plus vSphere services (such as COS, vpxa) on the host.
    •     
    consumedabsolutekiloBytes1 (4)average
    (minimum)
    (maximum)
    Memory Consumed (Average)
    Memory Consumed (Minimum)
    Memory Consumed (Maximum)
    Amount of memory consumed by a virtual machine, host, or cluster.      
           
    • Virtual machine:   Amount of guest physical memory consumed by the virtual machine for guest memory. Consumed memory does not include overhead memory. It includes shared memory and memory that might be reserved, but not actually used. Use this metric for charge-back purposes. 

      vm consumed memory = memory granted - memory saved due to memory sharing 

       
    •      
    • Host:   Amount of machine memory used on the host. Consumed memory includes Includes memory used by the Service Console, the VMkernel, vSphere services, plus the totalconsumed metrics for all running virtual machines. 

      host consumed memory = total host memory - free host memory 

       
    •      
    • Cluster:   Amount of host machine memory used by all powered on virtual machines in the cluster. A cluster's consumed memory consists of virtual machine consumed memory and overhead memory. It does not include host-specific overhead memory, such as memory used by the service console or VMkernel.
    •     
      

     

      

    ===================================================

      

    BMC CAPACITY PLANNING AND PERFORMANCE TOOLS

      

    TrueSight Capacity Optimization (TSCO) Virtual Farm dashboard uses Cluster Consumed memory as its basis.

    All of the Visualizer Cluster and Host graphics use Consumed memory as the basis for memory utilization, and thus also for calculating how many additional VMs could be supported.  This is consistent with VMware's choice for memory usage for clusters and hosts. 

      

    The ratio of Consumed and Active memory is reported on the Visualizer VMware Memory Cluster graphic, the "Active/Consumed" metric.  If you want to see the difference between using Active or Consumed for capacity planning, you can see results for both on the Visualizer VMware Memory Capacity Cluster graphic -- the top line shows calculations using Consumed memory as a basis, the bottom line shows the same calculations, but using Active memory as the basis.

      

    Perceiver shows both metrics in the VMware Virtual Machine Summary by Resource: Memory Profile.

    Virtualization Planning bases all ESX memory calculations for VM guests on Consumed memory.

     

      

    -------------------------------

      

    Perceiver (in VMware Host Summary by Resource, Memory Profile:  Memory Utilization by VM) and Visualizer (VMware -> VMware Memory Host/VM) also report and compare the consumed memory utilization for the host (based on consumed memory) with the sum of the VM memory utilization (based on active memory).  Generally you would expect to see a large gap between the host and VM measurements, since they are using completely different basis for what memory is considered utilized.  Again, this reflects the definitions chosen by VMware.

      
    Related Products:  
       
    1. BMC Performance Assurance for Servers

     


    Article Number:

    000321666


    Article Type:

    Solutions to a Product Problem



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles