14 Replies Latest reply on Feb 29, 2020 12:48 AM by Gajanand Patil

    Delete & Purge activity

    Lalit Joshi
      Share This:

      Hi CMDB experts,

       

      Can anyone advice me on this, what should be the correct process of re-adding CIs which were previously deleted from import dataset and currently presents in Target dataset as Mark As Deleted = Yes and status = delete. What happens when there is a relationship exists in any dataset at the time of deletion?

       

      Thanks & Regards

      Lalit Joshi

        • 1. Delete & Purge activity

          For weak relationships the MarkAsDeleted attribute in CI is propogated to relationship as well and those CI are also set for deletion.The only option is to re-populate them in Import Dataset.

          1 of 1 people found this helpful
          • 2. Delete & Purge activity

            That is probably the easiest way to get them back into the import dataset.  However, there may be some identification issues to resolve as a result.  If I am not mistaken, you will probably need to reset the ReconciliationID for all affected records in BMC.ASSET to '0' so Identification can take place.

             

            There is another consideration as well...  does your import data set contain the relationship mappings.  If it does, they you should be good to go.  If not, then the only way I can think of (painful) would be to open the backend forms for each CI class, set up a search which will give you all the records marked for deletion, remove the 'marked for deletion flag' and export them.  You'll have to get all the relationship class forms as well.  Once you have all these, edit each file by performing a search and replace, changing BMC.ASSET to the dataset name for your import dataset, then import the main CI classes, followed by the relationship (BaseRelationship???) file.

             

            As I said, painful, but you don't loose the reconciliation ID, nor the InstanceID, which will make Recon go much smoother.

            2 of 2 people found this helpful
            • 3. Delete & Purge activity
              Lalit Joshi

              I think the above details are really helpful to work on this area. To move further I am explaining the actual scenario where discovered CIs will may have relationships as well because before deletion many of them were related.

               

              They are using only three classes BMC_ComputerSystem, OS and Product followed by relationships ComSys2OS & ComSys2Product. When AIE pumps up CIs into the Import Datasets (they have four different dataset) their initial Status comes as "Received". 4 Standard Recon Jobs are in place to identify, merge and delete the data however Identification is restricted using a global qualification group (AssetLifeCycleStatus = Deployed). Asset Team is dedicated to identify the required CIs and promote them for reconciling by updating Status to "Deployed" as per their internal process only "Deployed" CIs and their relationships (if any) should go to BMC Asset.

               

              When re-adding old deleted CIs they are facing unexpected results because CIs are exists in BMC Asset with status = Delete and they have a Precedence group where AssetLifeCycleStatus always have a height precedence then value coming from all four datasets. What process they should choose to overcome from this problem. Please advice.

               

              Thanks & Regards

              Lalit Joshi

              1 of 1 people found this helpful
              • 4. Delete & Purge activity
                Lalit Joshi

                Thanks Crowell, You have mentioned that need to reset the ReconciliationID forall affected records in BMC.ASSET to '0'. I didn't get you here. Why need to reset therecon id in production dataset? May be I am missing something

                 

                I saw when I re-added the Primary CI (Computer System) again into the Import dataset all Childs (product & OS) were not merged to BMC Asset? But the expectation was as they have a weak relation so all CIs and relationships should get merged to BMC Asset.

                 

                Thanks & Regards

                Lalit Joshi

                • 5. Re: Delete & Purge activity

                  I have, in the past, made the mistake of deleting and purging my import dataset then tried reimporting and saw it fail, sa you describe.

                   

                  I learn that after a recon id is assigned - to both identified import ci as well as th bmc.asset ci, this is what is  used for future recons.  Your future updates to import data dont modify this.  So if an asset ci has a reconID, it will expect the same in the source.

                   

                  So since your import source is discovery with relationships, try clearing the reconID, qualification (temporarily) and run the recon jobs.

                   

                  From my Android phone on T-Mobile. The first nationwide 4G network.

                  1 of 1 people found this helpful
                  • 6. Re: Delete & Purge activity
                    Carey Walker

                    Lalit

                     

                    Can you clarify the sequence of events here?? It sounds like the typical issue of CIs disappearing out of the discovery scans and being 'aged' out, eventually, of the import dataset.  So the following is the case???

                     

                    1. CI is initially discovered and the reconciliation process eventually gets it into BMC.ASSET.

                     

                    2. Some time later, the discovery scan no longer sees the CI and after several missing scans, it sets the Mark As Deleted attribute in the discovery dataset, which gets reconciled through to BMC.ASSET.

                     

                    3. A purge job removes the CI from the discovery dataset (and hopefully it is configured to check that the CI is also set as MarkAsDeleted in the main dataset BMC.ASSET!). You typically don't want to run Purge jobs in BMC.ASSET because it's required to keep the CIs there, but reflected as deleted (or possibly Disposed etc).

                     

                    4. Some time later, the CI which had stopped scanning, is somehow discovered again and comes back into the discovery dataset?

                     

                    If this is the case, then the rediscovered CI in the discovery dataset will start its second time around life in the discovery dataset with no recon. id. When you run the next recon. identification activity it will likely match the BMC.ASSET CI (even though it now has a Deleted status) and the discovered dataset CI will be assigned the recon. id from BMC.ASSET.

                     

                    I don't think you should ever reset recon. ids in BMC.ASSET back to zero. Remember that there could be several datasets out there that have already been identified against BMC.ASSET and share that recon. id. It's OK to reset recon. ids. in a discovery dataset to 0 to force a re-identification if you really need to do that, but I'd be very wary of doing that in BMC.ASSET.

                     

                    Is this exactly what is happening for you?? If it is, the only thing we need to look at is how to get these CIs with 'Received' status, once changed by the asset team to 'Deployed' to still get merged correctly into BMC.ASSET. If I understand the scenario correctly, the CI in the discovery dataset won't be considered for recon. until the asset team have reviewed and set to Deployed (this is a good process!). However, once this is done, the CI should still merge OK, it's just that it will stay as Deleted status in BMC.ASSET because of your precedence rules. Is this all correct??

                    • 7. Re: Delete & Purge activity
                      Lalit Joshi

                      Hi  Careyw<https://communities.bmc.com/communities/people/careyw>,

                       

                      Thanks for replying. You are almost correct at all pointes, steps are the same. Only change in place is instead of purge I have used delete activity to remove CIs from Import Dataset (is that a wrong way to do it?) and we are not soft deleting CIs from Asset as customer wants them to keep remain for future use. The problem I am facing now is when CIs got discovered in Import Dataset again and then recon engine has auto assigned them the existing identity from Asset (which is a correct) but on other hand Company and status should be updated but CIs are gone to BMC Asset with the Status “Received”. Which is incorrect.

                      The expectation was recon job should not identify and merge any new discovered CI where status is not Deployed and reconciliation identify is zero. As per customer process AM team is supposed to manually update the Company and status Deployed then only they should promote them to Asset. But it’s not happening.

                       

                      Thakns & Regards

                       

                      Lalit Joshi

                      2 of 2 people found this helpful
                      • 8. Re: Delete & Purge activity
                        Carey Walker

                        OK, so you are NOT purging the import

                         

                        dataset (which would only remove soft deleted items), but you are DELETING the dataset which removes ALL items. Usually it's best to leave the CIs in the discovery dataset specifically to allow the import process to see when an item is no longer being discovered (i.e. it can correctly process adds, changes and deletes in the discovery dataset, whereas if you delete the data totally, every time you load it's all new. Have I understood this correctly??

                         

                        Also, if you are deleting not purging, how did the CIs in the target dataset (BMC.ASSET I assume), get soft deleted?? They would normally only get soft deleted when reconciliation processed them from the import dataset after they had been sift deleted there?? So are you deleting the import dataset AFTER some CIs have been sift deleted and then reconciled?? I guess that is what is happening.

                         

                        Finally, if you have a qualification on the identification activity that says only identify in the import dataset if the status=Deployed (which I thought was already in place??), no CIs should ever get  a recon. id until they are reviewed by the asset team and set to Deployed. I'm a bit confused that you say they are then getting merged?? A CI with no recon. id won't be merged. And finally, even more confusing, even if the Received CI did somehow get identified, you said you had precedence rules to stop the status being updated in BMC.ASSET?? So again, how did the CI get merged (if you had a qualification to stop it until reviewed) and how did it get to update the Status in the target dataset (if the precedence rule was defined to stop that)?

                        1 of 1 people found this helpful
                        • 9. Re: Delete & Purge activity
                          Lalit Joshi

                          Yes, We have four import datasets created in CMDB. AIE interface is deployed to populate CIs from following data sources ALTIRIS, SCCM, NETEISS and SAP into four separate import datasets. ALTIRIS and SCCM is supposed to handle Computer System, OS and Product CI and relationships as I already explained in previous threads. In NETEISS we have only Computer System CIs. Case should be very simple.

                           

                          Requirement:

                           

                          1.       There should be a one scheduled Recon job for each dataset which can only make updates in BMC asset. All new CIs should be rejected. To make this possible I made auto identify to No for all three classes and it’s not identifying any new CI, however merge is happening where any update comes. Other activity is Delete where we have a qualification Mark as Deleted = Yes only delete such instances from Import Dataset. when required its deleting the record from import dataset and keeping the same record remain in BMC Asset with Status = Delete.

                           

                          2.       Another recon job is used promote new CIs to BMC asset after manual update of Company. That’s the reason all CIs in Import dataset are coming with their status = Received. Later Customer can decide which CI they want to promote to Asset. They can simply update the company and status to Deployed and trigger the job. There is an identification qualification being used ($AssetLifeCycleStatus$=”Deployed” and $ReconciliationIdentify$=”0”) for new CIs. This job only have Identify and Merge activity.

                           

                          3.       We also have a Precedence rule used in all the jobs where higher priority has been assigned to ‘AssetLifeCycleStatus’ and ‘PrimaryCapability’ attributes for BMC.Asset. This is logical and stops the wrong update, example: all CIs existing or new comes in import dataset with Status = Received all the time.

                           

                           

                          If I will use purge instead of delete for the same case? Will that help me? I need to test this.

                          1 of 1 people found this helpful
                          • 10. Delete & Purge activity
                            Gustavo del Gerbo

                            Hi Lalit,

                             

                            Your implementation is a very good one, and by reading your points above it was very clear to me how you manage the whole "import to ASSET" process.

                            Regarding the Purge vs. Delete question: In the above scenario a Purge job would work great and it will also save you a lot of processing time.

                             

                            Delete + Re-create VS. Purge + Update is a scenario where we always recommend to go with the later.

                             

                            Regards,

                            Gustavo.

                            1 of 1 people found this helpful
                            • 11. Re: Delete & Purge activity
                              Lalit Joshi

                              Thanks for your comments Gustavo.

                              I will try using the Purge activity instead of Delete and retest the jobs again.

                               

                              Regards

                               

                              Lalit Joshi

                              2 of 2 people found this helpful
                              • 12. Re: Delete & Purge activity

                                totally agree to Gustavo's points. Delete activity deletes CI regardless of whether they are "marked as delete" or not. So its always safe to use purge activity which will only and only delete records which are "marked as delete" or you can say which are "soft deleted".

                                 

                                When you mark a CI as deleted you need to mark the child CI as deleted as well, otherwise you will end up with orphan relationship CIs. If you wish to remove orphan relationship Cis you can use the cmdbdiag utility. opt/BMC/AtriumCore/cmdb/cmdbdiag.

                                 

                                once you have removed orphan relationship Cis from the asset ds you can reconcile the relationships to asset from the import ds, which will refer to the recon id of the parent ci to build the relationships newly.  do not set recon id for asset cis to 0. wonder what wil happen if you do that and hit save. yikkesss !!

                                 

                                 

                                -Amey.

                                2 of 2 people found this helpful
                                • 13. Re: Delete & Purge activity
                                  Lalit Joshi

                                  Thanks for your reply Amey. I have replaced Delete with Purge and its working fine. Now I will test it by soft deleting CIs with weaker relationships in Import Datasets.

                                  1 of 1 people found this helpful
                                  • 14. Re: Delete & Purge activity
                                    Gajanand Patil

                                    Good discussion on the recon job with its purge and delete activity.

                                     

                                    Regards,

                                    Gajanan