Share:|

Are you using BPPM 9.0x or earlier?
If so are you seeing duplicate devices?

 

The instances of duplication in BPPM have been on the rise here in the Support community. The BPPM Console is displaying duplicate devices. You’re not seeing double and please do not adjust your monitor; this is a known issue with BPPM and it is documented as to how to fix it. We also recommend upgrading to BPPM 9.5 as soon as possible to avoid the issue.

 

So what is this duplicate device stuff anyhow? Let’s take a deeper look at the issue:

 

The issue has to do mainly with the way devices are handled in BPPM. See BMC ProactiveNet Server defines an entity that it can monitor as a device. Because BMC ProactiveNet Server collects data from many sources by using different methods, monitored entities are presented to BMC ProactiveNet Server as devices in multiple ways. BMC ProactiveNet Server has algorithms to convert the information presented about entities into devices, to determine whether two or more pieces of data refer to the same entity, and to consolidate those entities into a single device. However, due to the divergent nature of the data collection process, BMC ProactiveNet Server may not always determine that multiple entities are in fact referring to the same device. As a result, duplicate devices can show up in BMC ProactiveNet Server, but you can prevent most of such scenarios by using configuration settings.

 

A wide range of data sources can create devices in BMC ProactiveNet Server, and each source can be configured in different methods and can interpret data in a different style. This means there are many causes of duplicate devices so please take the time to review them all. This results in a situation where data flows into BMC ProactiveNet Server about a given device in multiple formats such as IP address, hostname, and FQDN. BMC ProactiveNet Server uses the best possible algorithm to identify that it is the same device that is presented in multiple formats by comparing the UUIDs if the device is a virtual machine in VMware, reconciliation IDs if the device is from BMC Atrium CMDB, and FQDNs if the device is across data sources. The algorithm can be summarized as follows:

  • If information about the two devices comes in through the BMC PATROL Adapter, compare the FQDN property presented by the BMC PATROL Agent. If the strings match, the devices are consolidated.
  • If the two devices are VMware virtual machines, compare their UUIDs. If the strings match, the devices are consolidated.
  • If information about the two devices comes in from BMC Atrium CMDB, compare their CMDB reconciliation IDs.  If the strings match, the devices are consolidated.
  • If information about one device comes from BMC PATROL Agent and that about another from BMC Atrium CMDB, compare their component alias values. If the strings match, the devices are consolidated.
  • If information about one device comes from BMC PATROL Agent and that about another from BMC Performance Manager Portal, compare their hostname and FQDN values. If the strings match, the devices are consolidated.

 

Tips for Avoiding Duplicate Devices:

 

  • Ensure that the /AgentSetup/integration/publishHostName variable is set as the FQDN of the host
  • Because BMC ProactiveNet Server is case sensitive, ensure that the host names are set in the same case similar to FQDN
  • Use FQDN of a device rather than the short name of it.

 

We see many different scenarios where duplicate devices are detected within BPPM. Here are the main areas we see issues:

 

 

Scenarios where multiple BMC PATROL AGENTS collect data for the same device

BMC PATROL Agent presents the details of each computer on which it is running as a device. If any of these computers is also monitored by another BMC PATROL Agent, a scenario arises where information about the same device comes to BMC ProactiveNet Server through different agents. The deployment architecture for this scenario to occur can be as follows:

  1. A BMC PATROL Agent is running on a virtual machine. The virtual machine is also monitored using another BMC PATROL Agent, which is running on another computer.
  2. A BMC PATROL Agent is running on a computer. This computer is also remotely monitored using another BMC PATROL Agent, which is running on another computer by using the remote monitoring capability of Windows KM or Unix KM.
  3. A BMC PATROL Agent is monitoring a virtual machine, and another BMC PATROL Agent is remotely monitoring the same virtual machine by using the remote monitoring capability of Windows KM or Unix KM remote.


Devices with Hostname and FQDN
BMC ProactiveNet Server displays duplicate devices when the hostname and FQDN (for example, xyz and xyz.acme.com) for the same device are found. It can happen when a device is represented with the short name through one route and with FQDN through another route. BMC PATROL Agent versions earlier than 9.0.00 publish themselves with the short name (hostname). For versions 9.0.00 or later, BMC PATROL Agent behavior can be changed to publish FQDN by using /AgentSetup/integration/useFQDNHostName variable and set its value to yes. Also, if a computer is configured with no DNS configuration but just the hostname, the device is represented with the short name.

 

Device with different DNS subdomains
BMC ProactiveNet Server displays duplicate devices when devices with different DNS subdomains are found, for example, xyz.acme.com and xyz.sub.acme.com.

 

Input value of BMC ProactiveNet Integration Service is different from the value returned by the BMC PATROL Agent
BMC ProactiveNet Server displays duplicate devices when the input value of BMC ProactiveNet Integration Service is different from the value returned by BMC PATROL Agent through the /AgentSetup/integration/publishHostName variable (BMC PATROL Agent is installed on BMC ProactiveNet remote agent with integration service enabled).

 

Token ID value is different for the same device name
BMC ProactiveNet Server displays duplicate devices when the token ID values are different for the same device.

 

Remote Monitoring: VSM KM monitoring a virtual machine and BMC PATROL Agent installed on the virtual machine
BMC ProactiveNet Server displays duplicate devices when the BMC PATROL Agent and the VSM KM running on a different BMC PATROL Agent present the same name in different formats (short name or FQDN).

 

Same device names (short name or fqdn) presented in different cases (for example: uppercase, lowercase, or mixed case)
BMC ProactiveNet Server displays duplicate devices when devices with the same name (short name or FQDN) are presented in different cases.

 

Different device DNS names for the same device, before and after powering on the virtual machine
This scenario was observed in BMC ProactiveNet Server version 8.5 SP4 but not in BMC ProactiveNet Server version 8.6 SP2. BMC ProactiveNet Server displays duplicate devices when the devices have different DNS name before and after the virtual machine is powered on.

 

BMC PATROL Agent added on different ports in manual mode using automatic registration with integration
BMC ProactiveNet Server displays duplicate devices when BMC PATROL Agent is added in manual mode and using automatic registration with integration.

 

Scenarios where BMC PATROL Agents and BMC Performance Manager Portal collect data for a device
As a general best practice, it is best to add elements to BMC Performance Manager Portal for monitoring by using FQDN. This ensures that when a device is imported to BMC ProactiveNet Server, it uses the FQDN for device creation. If the information about the device comes from another route (for example, BMC PATROL Agent) by using FQDN, the device is consolidated.

 

Device names from BMC PATROL agent/BMC Performance Manager Portal and alias name from BMC Atrium CMDB are different

BMC ProactiveNet Server displays duplicate devices when device names from BMC PATROL Agent/BMC Performance Manager Portal and the alias name for the same device in BMC Atrium CMDB are different.

 

Save device names (short name or FQDN) presented in different cases (for example: uppercase, lowercase, or mixed case)
BMC ProactiveNet Server displays duplicate devices when devices with the same name (short name or FQDN) are presented in different cases.

 

Scenarios where BMC PATROL Agents, BMC Performance Manager Portal, or SCOM and BMC Atrium CMDB collect data for a device

Device having different reconciliation IDs during BMC ProactiveNet Server and BMC Atrium CMDB sync
A device can have different reconciliation IDs mostly due to bad reconciliation. BMC ProactiveNet Server displays duplicate devices when a device has different reconciliation IDs during BMC ProactiveNet Server and BMC Atrium CMDB sync. The duplicate reconciliation IDs most likely exist in BMC.ASSET dataset.

 

In BMC Atrium CMDB, reconciliation ID is used to identify the same CI across multiple datasets. The reconciliation ID is set during the identification phase of reconciliation in BMC Atrium CMDB. If the identification rules in BMC Atrium CMDB fail to identify a CI that already exists, BMC Atrium CMDB can generate a new reconciliation ID for the CI, which results in duplicate CIs in the BMC.ASSET dataset with different reconciliation ID. In this case, the problem is with the identification rules in BMC Atrium CMDB or the provider of that data to BMC Atrium CMDB such as BMC Atrium Discovery and Dependency Mapping, SCOM, or BMC Server Automation.

 

Devices created in BMC Atrium CMDB first and then synced through BMC PATROL Agent Adapter
The following table explains different scenarios that can create duplicate devices:

BMC Atrium CMDB

BMC PATROL Agent

Result

FQDN

IP Address

Creates duplicate device

FQDN

Hostname

Creates duplicate device

Hostname

IP Address

Creates duplicate device

Hostname

FQDN

Creates duplicate device

 

Devices created in BMC Atrium CMDB first and then synced through BMC Performance Manager Portal Adapter
The following table explains different scenarios that can create duplicate devices:

BMC Atrium CMDB

BMC Performance Manager Portal

Result

FQDN

IP Address

Creates duplicate device

FQDN

Hostname

Creates duplicate device

Hostname

IP Address

Creates duplicate device

Hostname

FQDN

Creates duplicate device

 

Devices created in BMC Atrium CMDB first and then synced through SCOM Adapter
The following table explains different scenarios that can create duplicate devices:

BMC Atrium CMDB

SCOM

Result

FQDN

IP Address

Creates duplicate device

FQDN

Hostname

Creates duplicate device

Hostname

IP Address

Creates duplicate device

Hostname

FQDN

Creates duplicate device

 

Devices with different component aliases in BMC Performance Manager Portal and BMC Atrium CMDB
BMC ProactiveNet Server displays duplicate devices when a device has different component aliases in BMC Performance Manager Portal and BMC Atrium CMDB.

 

Guidelines to prevent duplicate CIs during data synchronization between BMC Atrium Discovery and Dependency Mapping and BMC Atrium CMDB
The following guidelines must be followed to ensure that the data synchronized from BMC Atrium Discovery and Dependency Mapping to BMC Atrium CMDB does not generate duplicate CIs in BMC.ASSET dataset or duplicate devices in BMC ProactiveNet Server: Configure CI filtering in BMC Atrium Discovery and Dependency Mapping to ensure that the quality data is synchronized. Only synchronize CIs from BMC Atrium Discovery and Dependency Mapping to BMC Atrium CMDB that do not have null values for attributes that are used in the identification rules of reconciliation process. Ensure that the BMC Atrium Discovery and Dependency Mapping resolve FQDNs of its devices against the primary DNS domain (xyz.acme.com) because BMC ProactiveNet Server is configured to resolve FQDNs against the primary DNS domain.

 

You can create custom synchronization mappings in BMC Atrium Discovery and Dependency Mapping to automatically create component aliases within BMC Atrium CMDB, for example, converting the FQDN to lowercase, uppercase, or mixed case, constructing new component aliases from other dynamic CI attributes, or even setting static component aliases.

 

Best Practices for BMC Atrium Discovery and Dependency Mapping to avoid duplicate devices
The information of a normal computer system coming from BMC Atrium Discovery and Dependency Mapping has the name of the server and domain as a separate field, and BMC ProactiveNet Server consolidates the entries based on it. You can also use an alias entry or entries with host name and domain for the manual computer system CIs.

For example, some attributes for the discovered server at acme.com are as follows:

 

For the network-attached storage devices, if the ComponentAliases field is blank, the best practice is to be consistent with the host name (for example, xyz) and the domain name (for example, acme.com) of the discovered server.

To see examples of the scenarios mentioned above or for the entire document regarding Duplicate Devices, please download it from here- http://documents.bmc.com/supportu/documents/88/88/298888/298888.pdf

 

How to clean up duplicate devices in BPPM

BMC has created a stale cleaner script which will clean up the devices from the database level. The stale cleaner modifies the BPPM database and identify and remove duplicate devices. The stale cleaner requires stopping the BPPM server and starting only the BPPM database and the cell.

 

 

Here are the complete details from the readme file for the stale cleaner script so you can be familiar with the steps needed to run the script. The script applies to both Sybase and Oracle and can be run on UNIX or Windows. The script should only be run at the recommendation of BMC Support. Therefore, if you see issue with duplication, please let us know so we can recommend the best way to manage the type of duplication being seen in BPPM. The stale cleaner location will be sent to you on an as needed bases. Given the power of the script Support needs to review the situation and determine if the script should be run.

 

*** NOTE *** This script is not applicable for BPPM 9.5 and must not be run against that version ***
** NOTE ** If BPPM is integrated with CMDB then only stale instance cleanup is supported.**

  1. To find stale instances and stale autodiscovered devices, the following can be executed with BPPM server running:

a. Windows BPPM server:

- If a Sybase DB is used:
launchScript.bat -object Instance
launchScript.bat -object Device
- If an Oracle DB is used:
launchScript.bat -object Instance -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT>  -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE

               launchScript.bat -object Device -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT>  -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE              

Note: In the single Oracle database instance, need to specify the SID or ServiceName of the instance in <SID>. In the Oracle RAC system, need to specify a ServiceName which exists for the entire Oracle RAC system in <SID>.

b. Solaris/Linux BPPM server:
- Source the BPPM environment:
- Bourne/Bash
. /usr/pw/pronto/bin/.tmprofile
- C shell:
source /usr/pw/pronto/bin/.tmcsh

   - Make the script executable:
chmod 755 launchScript.sh

- If a Sybase DB is used:
./launchScript.sh -object Instance
./launchScript.sh -object Device

  - If an Oracle DB is used:

      ./launchScript.sh -object Instance -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT> -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE

      ./launchScript.sh -object Device -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT> -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE              

Note: In the single Oracle database instance, need to specify the SID or ServiceName of the instance in <SID>. In the Oracle RAC system, need to specify a ServiceName which exists for the entire Oracle RAC system in <SID>.

2. The output will be written to the following files:
a. BPPM-DiscvStaleObjects-Inst.txt - contains a list of the stale instances
b. BPPM-DiscvStaleObjects-Dev.txt - contains a list of the stale autodiscovered devices

3. To cleanup stale instances and stale autodiscovered devices, please perform the following:
a. Stop BPPM server with 'pw sys stop' command.

b. Once BPPM server is stopped (verify with 'pw p l' command), please take a backup of the pw\server\var\ directory. If Sybase database is used, take a backup copy of the pronto.log and storm*.db files in pw\sybase directory. If Oracle database is used, contact the DBA to ensure a backup is performed of the database.

c. Start just the database and the cell. The cell can be started with 'pw p s mcell' command. If Sybase database is used, it can be started with 'pw p s dbsrv' command. Verify that both the cell and database are running with 'pw p l' command.

d. Windows BPPM server:
- If a Sybase DB is used:
launchScript.bat -object Instance –cleanup
launchScript.bat -object Device -cleanup

   - If an Oracle DB is used:

               launchScript.bat -object Instance -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT>  -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE -cleanup

               launchScript.bat -object Device -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT>  -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE -cleanup

Note: In the single Oracle database instance, need to specify the SID or ServiceName of the instance in <SID>. In the Oracle RAC system, need to specify a ServiceName which exists for the entire Oracle RAC system in <SID>.

e. Solaris/Linux BPPM server:
- Source the BPPM environment:
- Bourne/Bash
. /usr/pw/pronto/bin/.tmprofile

      - C shell:
source /usr/pw/pronto/bin/.tmcsh

   - If a Sybase DB is used:
./launchScript.sh -object Instance –cleanup
./launchScript.sh -object Device -cleanup

   - If an Oracle DB is used:

      ./launchScript.sh -object Instance -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT> -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE -cleanup

      ./launchScript.sh -object Device -host <ORACLE_SERVER_HOSTNAME> -port <ORACLE_SERVER_PORT> -sid <SID> -user <USERNAME> -password <PASSWORD> -dbtype ORACLE -cleanup

Note: In the single Oracle database instance, need to specify the SID or ServiceName of the instance in <SID>. In the Oracle RAC system, need to specify a ServiceName which exists for the entire Oracle RAC system in <SID>.

f. Once the script has completed, please start BPPM server with 'pw sys start' command.


If you find any discrepancies after executing the script:
Stop BPPM server with 'pw sys stop' command.
Restore the backups taken in step 3b above:

a. the database
b. the pw\server\var directory
Start BPPM server with 'pw sys start' command.

 

Newly Published Knowledge Articles - New Content has been added over the last month!

KA414984 - With BPPM 9.5, is it possible to suppress the header when running a pw command?

KA415240 - Global records lost after upgrading cell from version 9.0 to version 9.5.

KA415434 - The Data Collection & Consistency Status Problem Report shows devices that have a filter policy applied.

KA415590 - After upgrading to BPPM 9.5, ALARM events are not visible in the ops console. Events for other event classes are seen ok.

KA415613 - The BPPM database size is very large

KA415588 - What happens to an Incident raised via IBRSD if the causal event is closed and then reopened?

KA415563 - Why is there no BPPM 9.5 Monitor the Monitor KM (MTM KM) ?

KA415372 - Receiving "Unable to process Application Class "PMG_GLOBAL_SETTINGS" from Patrol Agent" after configuring PATROL Agent in BPPM CMA 9.5.00.

KA414761 - Where can I find the PCIG utility documentation and files?

 

Popular Knowledge Articles

KA329164 - Is there a way to manually delete the auto-synced groups in ProactiveNet Server 8.x.xx?

KA356762 - How to check the data collection in BPPM database  

KA350291 -The tunnelproxy process is not running.

KA361627 - The installation of the ProactiveNet CMDB Extensions from BPPM 8.6.02 and 8.6.02.01 failed on Atrium (AR) version 7.6.04 SP2 

KA329355 - Will the data from the PATROL Adapter be deleted from the ProactiveNet database (Sybase) when the PATROL Adapter is deleted? 

 

 

DON’T MISS A GREAT OPPORTUNITY : Upcoming Webinars

Check out the details here: BMC ProactiveNet Performance Management (BPPM) 9.5 Best Practices Webinar Series


Upcoming Webinar topics include:

•    Configuring for Advanced Thresholds and Abnormalities (September 2014)

•    Configuring Simple Service Modeling & PCA (October 2014)

•    High Availability & Disaster Recovery (November 2014)

 

Sign up now by accessing the link above. Then click on the topic of interest and sign up for the Webinar. These are 90 minute sessions designed to help you get the most of out the product.

 

Looking for a previous blog posting?
Find them here: BMC TrueSight Support Blogs

 

Feedback1.jpg

Feedback:

Help us determine how we are doing. Use the rating system at the bottom of this blog to rate this post! Feel free to comment as well. We want to hear from you!