4 Replies Latest reply on Sep 21, 2018 7:47 AM by Rob Presland

    Concurrent Workers Best Practice?

    Rob Presland
      Share This:

      We're trying to improve the sync times between Discovery 11.x and Remedy 8.1 (to be 9.1 soon). We are working with BMC, but wanted to ask to see what others have done with Concurrent Workers settings. There is a statement in the docs "The best practice is to start from the default and gradually increase the number up to when sync rate keeps up with discovery rate.". Does this mean increase the number until the sync takes no longer than the discovery? We are running continuous sync. We recently completed a test increasing the Concurrent Workers to 2 and saw a 50% decrease in sync time. Obviously there is a max of Concurrent Workers based on the server resources, but what are others doing?


      Thanks in advance.


      (we got into this situation because we are looking at syncing the discovered packages which dramatically increases the sync times)

        • 1. Re: Concurrent Workers Best Practice?
          Andrew Waters

          There is a CMDB sync queue. If this continues growing then you need to increase the number. if it briefly increases but then decreases, for example when a scan has been initiated, then you are okay. What you don't want is the queue to always contain lots of things to sync because then it has fallen behind Discovery scanning.

          1 of 1 people found this helpful
          • 2. Re: Concurrent Workers Best Practice?
            Duncan Grisby

            Also, note that the first time you sync a host (or other root node like NetworkDevice), CMDB sync will have to create a lot of new CIs and relationships in the CMDB. In subsequent syncs, it only needs to send updates to existing CIs, and create any new ones. There is therefore a lot more CMDB activity on the first sync than on subsequent ones. It is fine if sync lags behind discovery for new data. It is only in the business-as-usual case where most activity is small updates that you should watch to make sure the sync is keeping up.


            We always recommend that people do not sync package information, because it tends to overwhelm the CMDB with a lot of low-value information. What is the use case for the package information in the CMDB?  Maybe you can send a smaller amount of data that allows you to address the same requirement.

            1 of 1 people found this helpful
            • 3. Re: Concurrent Workers Best Practice?
              Rob Presland

              Yes, the initial sync will always be larger and slower, especially when introducing packages. It wouldn't be our choice to sync packages but this is one option to satisfy our client's requirements, and one way to mitigate the problem of BMC Discovery missing software installed but not running. It may indeed be true that this is more the solution rather than the requirement. In any case, if there is performance improvement we can implement we are ahead of the game in the end.

              • 4. Re: Concurrent Workers Best Practice?
                Rob Presland

                For anyone following this thread, wanted to provide an update...


                We are successfully sync'ing packages from BMC Discovery to CMDB, with the help of a private queue on the Atrium side (we are using the Normalization queue to sync from BMC Discovery, this queue is separate from the primary Atrium queue), and the concurrent workers settings (we are using up to 3 for our environment). It has been a good experience in learning how to tune these settings to ensure a good performance and a reasonable sync time.


                Thanks to all.