7 Replies Latest reply on Jan 27, 2020 6:49 AM by Ondrej Kieler

    How to stop CMDB sync services

    Ondrej Kieler
      Share This:

      Hello,

       

      Is it possible to stop only CMDB sync services (BMC Discovery 11.3) and leave all other services running? Discovery is probably running quite extensive SQL queries to CMDB which overloading the CMDB and we would like to stop the services to be sure that the queries are really started by BMC Discovery tool and avoid outage of whole consolidator system. We already tried to disable sync but consolidator is still trying to connect to CMDB.

       

      Thanks,

       

      Ondrej Kieler

        • 2. Re: How to stop CMDB sync services
          Ondrej Kieler

          Hello Rodrigo,

           

          Thanks for the answer. I know all of those actions. I tried them but even when sync is disabled the consolidator is trying to contact the server and still running on it the queries which are searching for duplicates in BaseElement form.

           

          So this is not helping.

           

          Regards,

           

          Ondrej

          • 3. Re: How to stop CMDB sync services
            Ondrej Kieler

            Please see the logs bellow (both CMDB sync is disabled (CMDB Main and CMDB UAT)):

             

            tail  tw_cmdbsync_exporter.log

                    at com.bmc.cmdb.api.CMDBUtil.CMDBGetVersions(Unknown Source)

                    at com.tideway.integrations.cmdbsync.SyncServerMonitor.loadServerInfo(SyncServerMonitor.java:128)

                    at com.tideway.integrations.cmdbsync.SyncServerMonitor.getServerInfo(SyncServerMonitor.java:67)

                    at com.tideway.integrations.cmdbsync.SyncProvider.testCredentialId(SyncProvider.java:239)

                    at CMDBSyncCORBA.GraphSyncPOA._invoke(GraphSyncPOA.java:84)

                    at org.jacorb.poa.RequestProcessor.invokeOperation(RequestProcessor.java:297)

                    at org.jacorb.poa.RequestProcessor.process(RequestProcessor.java:591)

                    at org.jacorb.poa.RequestProcessor.run(RequestProcessor.java:734)

            RequestProcessor-16: 2020-01-24 13:19:06,711: com.bmc.arsys.apitransport.connection.c: ERROR: Connects to <IP address:Port> through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out (Connection timed out) <IP address>

            RequestProcessor-14: 2020-01-24 13:23:21,303: com.bmc.arsys.apitransport.connection.c: ERROR: Connects to <IP address:Port> through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out (Connection timed out) <IP address>

             

            tail tw_svc_cmdbsync_exporter.out

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

            [jacorb.poa.controlle] WARN : Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max (currently: 20)

             

            tail tw_svc_cmdbsync_transformer.log

            140111844771584: 2020-01-24 12:51:39,677: cmdb_sync.target: ERROR: CMDB Main: Error retrieving server details - CORBA.TIMEOUT(omniORB.TIMEOUT_CallTimedOutOnClient, CORBA.COMPLETED_MAYBE)

            140111844771584: 2020-01-24 12:51:39,677: cmdb_sync.target: INFO: CMDB Main: Failed to contact server. 134 attempts so far.

            140111836378880: 2020-01-24 13:04:59,622: cmdb_sync.target: ERROR: CMDB UAT: Error retrieving server details - CORBA.TIMEOUT(omniORB.TIMEOUT_CallTimedOutOnClient, CORBA.COMPLETED_MAYBE)

            140111836378880: 2020-01-24 13:04:59,623: cmdb_sync.target: INFO: CMDB UAT: Failed to contact server. 135 attempts so far.

            140111844771584: 2020-01-24 13:04:59,777: cmdb_sync.target: ERROR: CMDB Main: Error retrieving server details - CORBA.TIMEOUT(omniORB.TIMEOUT_CallTimedOutOnClient, CORBA.COMPLETED_MAYBE)

            140111844771584: 2020-01-24 13:04:59,777: cmdb_sync.target: INFO: CMDB Main: Failed to contact server. 135 attempts so far.

            140111836378880: 2020-01-24 13:18:19,721: cmdb_sync.target: ERROR: CMDB UAT: Error retrieving server details - CORBA.TIMEOUT(omniORB.TIMEOUT_CallTimedOutOnClient, CORBA.COMPLETED_MAYBE)

            140111836378880: 2020-01-24 13:18:19,721: cmdb_sync.target: INFO: CMDB UAT: Failed to contact server. 136 attempts so far.

            140111844771584: 2020-01-24 13:18:19,877: cmdb_sync.target: ERROR: CMDB Main: Error retrieving server details - CORBA.TIMEOUT(omniORB.TIMEOUT_CallTimedOutOnClient, CORBA.COMPLETED_MAYBE)

            140111844771584: 2020-01-24 13:18:19,877: cmdb_sync.target: INFO: CMDB Main: Failed to contact server. 136 attempts so far.

            • 4. Re: How to stop CMDB sync services
              Duncan Grisby

              Those logs make it look as though the CMDB is not being contacted at all.

               

              To be really sure Discovery is not querying the CMDB, even for a liveness check, a simple thing to do is temporarily break the credentials. Tell Discovery the username is different, then it will not be able to connect.

               

              You can stop the CMDB sync services for a temporary test, but you will start seeing errors from other services that are unhappy they can't talk to the CMDB ones:

               

              tw_service_control --stop cmdb_sync_transformer

              tw_service_control --stop cmdb_sync_exporter

               

              ...still running on it the queries which are searching for duplicates in BaseElement form.

              Discovery doesn't run queries to search for duplicates. What exactly are the queries you are seeing?

              2 of 2 people found this helpful
              • 5. Re: How to stop CMDB sync services
                Ondrej Kieler

                App log is showing this one:

                <SQL > <TID: 0000011976> <RPC ID: 0022797762> <Queue: List      > <Client-RPC: 390696   > <USER: <SYNCUSER>                             > <Overlay-Group: 1 > /* Fri Jan 24 2020 14:40:24.3550 */SELECT  * FROM ( SELECT T548.C1,C179,C400127400,C400129200 FROM T548 WHERE (T548.C400127400 = '<DATASETID>') ORDER BY 1 ASC ) WHERE ROWNUM <= 2

                 

                DB log this:

                select * from (select RequestId, InstanceId, DatasetId, ReconciliationIdentity  from bmc_core_bmc_baselement where (Datasetid = '<DATASETID>')  order by 1 ASC)  where rownum <=2

                 

                After stopping of services these queries did not run at all.

                • 6. Re: How to stop CMDB sync services
                  Duncan Grisby

                  That query is run just once when CMDB sync starts up, or a new connection is created, to verify whether the dataset has any contents or not. It will only be run more than once if it fails. Does it never succeed?

                  2 of 2 people found this helpful
                  • 7. Re: How to stop CMDB sync services
                    Ondrej Kieler

                    It succeded for sure as we already transferred quite a lot of CI's to CMDB from this consolidator using that particular Sync job. But we do not have any updates for this particular dataset for quite a long time. Port which is used for sync is opened from consolidator to CMDB but we are constantly receiving the errors mentioned above.