This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.
BMC Performance Assurance
TrueSight Capacity Optimization TSCO
(1) IBM/AIX API reporting less "Virtual" memory than "Real" memory, as reported using the BPA/Performance Assurance Agent
(2) What does "Virtual" memory represent for a Linux process?
LP: BMC Performance Assurance 9.5.00 (and earlier)
DR: BMC Performance Assurance 9.5.00
Details: On the 'Graphics->Process->Process Memory Metrics' view in Visualizer the value of the 'virtual memory metric' is lower than the value of the 'real memory metric' on AIX systems for a given process, which is not possible.
Issue Summary: Real / virtual memory metrics look wrong on 'process memory graphic' in Visualizer for AIX systems.
First, let's look at the mechanics of how and is being reported for an process in Visualizer, exactly which metric is that, and finally what does it represent.
In the Visualizer database, these metrics are from the CAXPROCD (PROCD) table:
The Visualizer chart is being used to display the values from the database.
The Analyze component assigns the values for these database fields using the udr metrics and , respectively. As always, the processes selected for reporting via the Visualizer database are the highest-consuming CPU processes (30) during the selected interval.
So in order to interpret what the reported values mean, you need to know for each specific operating system platform (and possibly version), which API metrics are used to supply the values for RSS and SIZE and how the operating system vendor documents those metrics.
AIX PLATFORM EXAMPLE
The reported observation was that Virtual is smaller than Real Memory for an AIX process. When results for another AIX server were checked, Virtual was always larger than Real Memory.
For the server in question, selecting one oracle process 10616916 on 31 March 2014 at 12:00 AM, the values from the Visualizer graphic are:
Real Memory Total MB 71.59
Virtual Memory MB 161.68
These correspond exactly with these Process_Statistics metrics (in KB instead of MB):
Other memory metrics for this process are:
Private RSS 3288
Shared RSS 70016
For this process, RSS is smaller than SIZE.
So what are the specific definitions of where RSS and SIZE udr metrics are gathered for AIX ?
Looking at the AIX header file for the SIZE metric of Process_Statistics, it says:
unsigned long pi_size; /* size of image (pages) */
which is then converted from pages to KB by the collector.
Using standard unix commands, and we can compare the values reported for RSS and SZ by the AIX with those from the data collector:
$ ps v
PID TTY STAT TIME PGIN SIZE RSS LIM TSIZ TRS %CPU %MEM COMMAND
352404 pts/1 A 0:00 2 672 956 32768 240 284 0.0 0.0 /bin/ksh
610392 pts/1 A 0:00 0 652 936 32768 240 284 0.0 0.0 -ksh
651306 pts/1 A 0:00 0 740 844 32768 79 104 0.0 0.0 ps v
$ ps -l
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
200001 A 357 352404 610392 1 60 20 7ba0b400 908 pts/1 0:00 ksh
200001 A 357 512242 352404 6 63 20 7174a400 820 pts/1 0:00 ps
240001 A 357 610392 569448 0 60 20 c309c400 888 pts/1 0:00 ksh
Look is used to dynamically report the values from the data collector at about the same time as ps was run. For example for process with PID 352404:
PID = 352404
RSS = 956
SIZE = 908
The data collection results match those of ps -l (RSS) and ps v (SZ).
The documentation for these metrics (from is:
- (v flag) The real-memory (resident set) size of the process (in 1 KB units).
- (-l and l flags) The size in 1 KB units of the core image of the process.
- (v flag) The virtual size of the data section of the process (in 1 KB units).
The conclusion is that the AIX data collector is reporting exactly what ps is reporting, and for some processes, SZ is less than RSS. Logically this wouldn't be expected since SZ should be the maximum amount of memory that the process could ever use, so should always be larger than the amount of RSS (real/resident) memory that's recorded.
The next step we are recommending is to escalate this question to IBM, since BMC has verified that we are collecting what we expect to, and using an IBM API to obtain the data values. For the sample udr data, about 25% of the processes are reporting less "Virtual" memory than "Real" memory. This should be useful for documenting your question to IBM.
LINUX PLATFORM EXAMPLE
Checking the results for java process 44286 on 26 January 2015 at 12:05 AM, the values from the Visualizer graphic are:
Real Memory Total MB 525.16
Virtual Memory MB 1981.32
These correspond to these Process_Statistics metrics (in KB instead of MB):
Other memory metrics for this process are:
Private RSS 438784
Shared RSS 98984
So what is the specific definition of the SIZE udr metric for Linux?
From man ps on Linux we find these definitions
SZ size in physical pages of the core image of the process. This includes text, data, and stack space. Device mappings are currently excluded; this is subject to change.
VSZ virtual memory size of the process in KB (1024-byte units). Device mappings are currently excluded; this is subject to change.
This command can be used to display both of these metrics
ps -o pid,rss,trss,sz,vsz,comm
The Linux data collector use the /proc/[pid] files to get the by process memory metrics:
(1) In /proc/[pid]/statm we find
/proc/[pid]/statm Provides information about memory usage, measured in pages. The columns are: size (1) total program size (same as VmSize in /proc/[pid]/status) resident (2) resident set size (same as VmRSS in /proc/[pid]/status) share (3) shared pages (i.e., backed by a file) text (4) text (code) lib (5) library (unused in Linux 2.6) data (6) data + stack dt (7) dirty pages (unused in Linux 2.6)
rss %ld Resident Set Size: number of pages the process has in real memory. This is just the pages which count toward text, data, or stack space. This does not include pages which have not been demand-loaded in, or which are swapped out.
vsize %lu Virtual memory size in bytes.
- BMC Performance Assurance