This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.
CMDB sync takes too long
Example: With ~6000 hosts, it took approximately 20 hours to sync to the CMDB
Some things to look at are shown below ... See the Solution portion.
1) Look at the "Details" page for your CMDB Sync Connection
2) Look at the "Status" page to see if there are errors. Especially see if there are timeouts or connection failures.
3) Look at the Atrium Core Console (Normalization page) to see the settings for the BMC.ADDM dataset
Root Cause 1: Inline Normalization for BMC.ADDM dataset will cause slow performance.
By default, when creating a new Dataset in the Atrium Core Console (Reconciliation page), Inline Normalization is enabled for that dataset.
Inline Normalization enabled means that for each CI synchronized to BMC.ADDM Dataset, a request for normalization is triggered.
This is a large drag on performance.
Go to the AtriumCore Console->Normalization->Edit Configuration.
Check that Inline Normalization is disabled for BMC.ADDM dataset
With Inline Normalization disabled, you need to run Normalization as a batch job, and it should be run before Reconciliation.
The screenshot above (in the Problem statement) shows 2 items that can be unchecked for BMC.ADDM dataset so that performance is better.
You don't need the "Impact Normalization Feature" because ADDM CMDB Sync creates its own impact relationships (assuming you are using the default data model).
Root Cause 2: The specified RPC queue is too busy
Symptoms of this problem would be connection errors and timeouts.
The default RPC queue is 30696: the CMDB general administration queue.
Other RPC queues that you can specify are: 390698 (Reconciliation queue), 390699 (Normalization queue)
None of these RPC queues is a dedicated queue only for Discovery CMDB Sync.
In a future release of Discovery (combined with a later version AR Server), there will be additional queues you can specify, so that you can use a dedicated queue for CMDB Sync.
See this documentation about the CMDB Sync queues: https://docs.bmc.com/docs/display/DISCO113/Configuring+CMDB+synchronization+through+a+private+queue
Solution 2: You can change the RPC queue in the GUI to use 390698 or 390699, but care must be taken so that CMDB Sync is not "competing" with Normalization and/or Reconciliation resources.
Root Cause 3: The "Concurrent Workers" on the "Details" page of the CMDB Sync connection in Discovery GUI is set too low or too high.
1 Concurrent Worker may be too low.
More than 3 Concurrent Workers may be too high, depending on how many threads are available to the RPC queue that is specified (see Root Cause 2 for discussion of RPC queues), and depending on how busy that RPC queue on the AR side. i.e. none of those queues is dedicated to Discovery CMDB Sync.
If you have many connection errors and timeouts during CMDB Sync, then the Concurrent Workers is probably set too high.
If there are no connection errors or timeouts, then you can try setting Concurrent Workers to a higher value. (more is not always better)
What is a Concurrent Worker? A Concurrent Worker is equivalent to approximately 5-10 threads.
Solution 3: Change the number of Concurrent Workers, using the information provided above.
Root Cause 4: The "Batch Size" could be too high or too low
If you have timeouts and connection errors to the CMDB, the "Batch Size" could be too high.
If there are no timeouts and no connection errors, you may be able to increase the "Batch Size" to get better performance.
As a general rule, you probably should not increase the "Batch Size" higher than 100. (There are exceptions, of course).
Solution 4: change "Batch Size" on the Details tab
Root Cause 5: Slow CMDB Sync or slow SQL on the AR Server side, caused by configuration settings or customizations on the AR Server side.
Examples could be: AR Escalations, AR Filters, an overlay on a BMC.CORE form (not recommended!)
Another example is that you may need to add some indexes to the database to improve SQL performance.
Solution 5: You should explore the performance issues with Atrium Core or AR Server team
(They would want to look at these AR side logs: arsql, arfilter, arescl, arapi)
Root Cause 6: If you are on CMDB 8.1 or CMDB 9.1, there is a patch for the CMDB for "slow Mark As Deleted performance". (Be sure you have this patch if you are on CMDB 8.1 or 9.0).
The problem does not exist in other versions of CMDB.
Solution 6: Get patch from the CMDB/Atrium Core team.
The patch numbers are:
For CMDB 9.0, it is SW00495641
For CMDB 8.1, it is SW00495598
- BMC Atrium CMDB Suite
- BMC Atrium Discovery and Dependency Mapping