In TSCO, when two different ETLs are mistakenly sharing the same sysid for an entity is it possible to divide them?

Version 1
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    TrueSight Capacity Optimization


    TrueSight Capacity Optimization 11.5, 11.3.01, 11.0, 10.7, 10.5


    Two scenarios:

    Scenario A:

    In our environment we have two different ETLs which are importing entities.  One is importing entities from a production environment (PROD) and the other is importing entities from a disaster recovery environment (DR).  It turns out that the computer names in both environments are the same so when we ran a reconcilation using option to "compare entities by" "entity name and type" the entities from the PROD and DR environments were mistakenly reconciled together because they had the same hostname.

    Scenario B:

    In our environment we have two different ETLs which are importing entities.  One ETL is importing entities from the TSOM database and the other ETL is importing system data for the same systems from the TSCO Gateway Server VIS parser ETL.  Since both of these ETLs are reporting similar metric we want to split the entities and keep them separate so each ETL can import its version of data separately.

    Under Administration -> ETL & System Tasks -> Entity catalogs -> [Select Entity Catalog] there is a button calls 'Divide entities'.  Is it possible to use that option to re-split the entities that have been mistakenly reconciled?  If not, is there any way to split the entities that have been reconciled?


    The ability to split an entity after an invalid reconciliation is limited to a few scenarios within the product.  Scenarios where the entity split is possible are:
      (A) The reconciled entity was reconciled across two different ETLs (different ETL IDs)
      (B) The reconciled entity did not import the same metrics across those two different ETLs

    Scenarios where the entity split is not possible include:
      * The reconciled entity was reconciled within the same ETL (so entity A from ETL 1 and entity B from ETL 1 were reconciled together)

    Scenarios where the entity split will not be clean include:
      * The reconciled entity was reconciled across two different ETLs but there were overlapping metric imports (say, CPU_UTIL) across both ETLs

    The problem with entities being reconciled together that have the same metrics (regardless of whether those metrics cover the same time period or not) is that metrics are merged into a single "Data Series" ID (SYSOBJID in the TSCO database).  This means that although TSCO knows that there is a relationship between the ETL importing data and the series ID, it does not know which data points were imported by which ETL.

    So, a scenario where the 'Divide entities' functionality would be applicable:

    • One ETL is importing system configuration information and one ETL is importing system statistics data.  In this case the Divide Entities would work as the data is coming from separate ETLs and there is no overlap between the metrics being imported.
    Scenarios where the 'Divide entities' functionality would not be applicable:  
    • A VMware host or guest entity had been mistakely reconciled across two different vCenter Extractor Service ETLs.  The problem in this scenario is that there will be a direct overlap between the metrics being reported by the two different ETLs.  So, the reconciliation collapses the data (for example for CPU_UTIL) into a single dataseries and thus it isn't possible to accurately split the series back along the lines of which ETL imported which data points (even if there is not specific overlap in the times of the data points).  This means that each entity will receive the dataseries once split, but it would not be clear which ETL data import that series had originally been imported by.
    So, it is critical that the initial reconciliation of entities is carefully considered and only the appropriate entities are reconciled together.  In most typical scenarios the 'Divide entities' functionality will not be able to clearly separate the reconciled entities and there would be no way to cleanly split them without restoring the TSCO database. 

    In the scenario where a single ETL is importing data for multiple unrelated entities to the same sysid most likely with overlapping data (what we would call an 'entity collision') there is no mechanism to split the entity in the TSCO product itself. 

    Scenarios that can cause an entity collision include:  
    1.  Someone executing an entity reconciliation (for example, a 'Reconcile' by NAME and TYPE when a separate STRONG LOOKUP is required to correctly maintain separate unique entities).
    3.  A mis-configured ETL which is not properly defining STRONGLOOKUKPFIELDS or WEAKLOOKUPFIELDS resulting in an invalid reuse of an existing sysid for a non-unique entity
    When there has been an entity collision within a single ETL or with overlapping metrics there is no way to split the existing entity in TSCO -- but what you could do is: 
      (A) Delete the entity with the lookup collision and purge it from TSCO. The next execution of the ETL would then create a new entity which, assuming the strong and weak lookup fields are sufficiently unique, will re-create separate entities. 
     - or - 
      (B) Remove or rename all (or all but one) of the Lookup values associated with the problem entities and let the next ETL execution create new entities for the unique lookup values. 

    When there has been an entity collision with overlapping metrics it is not possible to determine which real-world entity any of the performance data associated with this sysid actually represents (and it could change from interval to interval since it would just be whichever entity loaded data last that would win). 

    Whenever there is an entity collision some key things to consider are:  
    • Are the strong and weak lookup fields associated with this ETL sufficient to uniquely represent the entity?
    • If so, how did all theses separate entities collide under this same sysid?
    In many cases he 'purge the entities and re-import' problem remediation option is probably the best.  This is particularly true in a scenario where the conf/performance data associated with the collided entity has been compromised because it is all overlapping data so it is impossible to determine which source entity any given sample pertains to. 


    Article Number:


    Article Type:


      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles