Troubleshooting TrueSight Capacity Optimization (TSCO) event alert with TrueSight Operations Management (TSOM) integration

Version 1
    Share This:

    This document contains official content from the BMC Software Knowledge Base. It is automatically updated when the knowledge article is modified.


    PRODUCT:

    TrueSight Capacity Optimization


    COMPONENT:

    Capacity Optimization


    APPLIES TO:

    TrueSight Capacity Optimization 11.5, 11.3.01, 11.0, 10.7, 10.5, 10.3



    QUESTION:

    TSCO is configured to send event alerts to TSOM. The connection is successful but TSOM seems to be receiving all the capacity alerts even if the Action setting is configured to send a SNMP trap.


    ANSWER:

     

    The first place to check is under the Administration -> 'Resource Monitor' -> Alert log section to validate that the Alert has been asserted.

    Note that per the TrueSight Capacity Optimization (TSCO) documentation, only a subset of the Resource Monitors support alert integration with TSOM:
      https://docs.bmc.com/docs/capacityoptimization/btco115/integrating-with-truesight-operations-management-830156586.html#IntegratingwithTrueSightOperationsManagement-ToconfigureanOptimizerruletosendeventstoTrueSightOperationsManagement

    When an alert is edited there should be an option available under the 'Actions' heading in 'Edit' mode with the actions to perform when that alert asserts.  Check to make sure that the action to forward the Alert to TSOM is selected.  If the alert is configured with an action to forward to TSOM take a screenshot of that and send it to Technical Support along with a screenshot of the screen showing that alert has asserted.

    Additionally, Check the TSCO $CPITBASE/scheduler/log/cpit.log for messages related to the publish of Capacity Alerts to TSOM.  For example, here are messages indicating that TSCO tried to forward a Capacity Alert to TSOM but was unable to find a matching system with which to associate the alert:

    2017-06-06 09:51:25,452 OUTPUT [taskid=18]- Login Succeeded with the server 'http://labtruesight.in.lab:8080/tsws/10.0/api/' for user 'admin' for tenant 'BmcRealm'.
    2017-06-06 09:51:25,460 INFO  [taskid=18]- Successfully connected to the BPPM Webservice URL
    2017-06-06 09:54:24,286 INFO  [taskid=18]- Processing data to raise an event for system =hostname1
    2017-06-06 09:54:24,289 INFO  [taskid=18]- Processing data to raise an event for metric  =CPU
    2017-06-06 09:57:24,294 INFO  [taskid=18]- [BPPMEventDetails] Cannot get the toekn ID lookupField
    2017-06-06 09:57:24,311 INFO  [taskid=18]- [BPPMEventDetails] generating tokenID
    2017-06-06 09:57:25,345 ERROR [taskid=18]- BCO_PUB_ERR112: http://labtruesight.in.lab:8080/tsws/10.0/api/Device/VI-UUID:420eef24-6d43-3831-250c-ae9ad3adb33f/event?idType=URI,HTTP response code:404,The requested resource is not available,please check id resource is available.
    2017-06-06 09:57:25,414 INFO  [taskid=18]- Processing data to raise an event for system =hostname2
    2017-06-06 09:57:25,416 INFO  [taskid=18]- Processing data to raise an event for metric  =CPU
    2017-06-06 09:57:25,439 INFO  [taskid=18]- [BPPMEventDetails] Cannot get the toekn ID lookupField
    2017-06-06 09:57:25,485 INFO  [taskid=18]- [BPPMEventDetails] generating tokenID
    2017-06-06 09:57:25,958 ERROR [taskid=18]- BCO_PUB_ERR112: http://labtruesight.in.lab:8080/tsws/10.0/api/Device/VI-UUID:420e9940-5f0f-2eb0-b7d4-13c00ac0ffee/event?idType=URI,HTTP response code:404,The requested resource is not available,please check id resource is available.
    2017-06-06 09:57:25,959 INFO  [taskid=18]- Processing data to raise an event for system =hostname3
    2017-06-06 09:57:25,964 INFO  [taskid=18]- Processing data to raise an event for metric  =CPU
    2017-06-06 09:57:25,967 INFO  [taskid=18]- [BPPMEventDetails] Cannot get the toekn ID lookupField
    2017-06-06 09:57:25,968 INFO  [taskid=18]- [BPPMEventDetails] generating tokenID
    2017-06-06 09:57:26,146 ERROR [taskid=18]- BCO_PUB_ERR112: http://labtruesight.in.lab:8080/tsws/10.0/api/Device/VI-UUID:420eedaa-fc4a-84bd-85b6-a83bbbaa12345/event?idType=URI,HTTP response code:404,The requested resource is not available,please check id resource is available.
    < -- cut about 100 more blocks of these same errors -- >

    Q: What is this error message trying to say has gone wrong when forwarding the alert to TSOM?
    A: The error basically is indicating that the entity for which alert has been raised is not available in TSOM.

    Q: What is the purpose of the "token ID lookupField" being generated?
    A: To identify entities uniquely in TSOM a unique identifier is required, if the entity data is imported from TSOM, that unique identifier is imported into TSCO for the entity, but for VMware entities data isn't coming from TSOM so it is necessary to create the tokenID on the TSCO side and then lookup that entity using that tokenID on the TSOM side.  In some cirsumstances even when the data is imported from TSOM a unique identifier is not available in that case TSCO will construct the tokenID it uses for entity lookup on the TSOM side.

    Q: Is it a requirement that an entity be defined already on the TSOM side for an Alert to be asserted into TSOM for it or can Alerts be asserted into TSOM even if that entity isn't already known within TSOM?
    A: Entity needs to be in TSOM for the Alert to be asserted, when the alert is raised to TSOM, TSOM tries to associate the event to the entity.

    Q: Is there a log file on the TSOM side that might provide more detail on why the alert forward into TSOM is failing?
    A: The alert is failing on the TSOM end as the resource/entity for which the event is raised  is not available, in the event where the existence of the entity has been confirmed and the resource is available then a good next step would be to ask the TSOM Administrator to provide the logs from the TSOM side for further troubleshooting.

    Also if the entities are type VMware Cluster data cannot be associated with that entity type and the same error will be generated

    This is described here:
      https://docs.bmc.com/docs/display/btco107/Integrating+with+TrueSight+Operations+Management#IntegratingwithTrueSightOperationsManagement-TointegrateeventsfromTrueSightCapacityOptimizationintoTrueSightOperationsManagement

    The last line for this document mentions the same.: "Events for a cluster entity are sent to TrueSight Operations Management, but they do not get associated with a cluster entity at TrueSight Operations Management."

    Expanding more on how to interpret those error messages, every entity on the TSOM side has an ID associated with it which is constructed via a standard method.  TSCO can either get that Token ID by querying TSOM for it or by building it using a standard method.  The reason that events are being dropped is that TSCO is trying to register an event against "VI-UUID:420eef24-6d43-3831-250c-ae9ad3adb33f" which TSCO thinks should be the Token ID of VMware guest "hostname1" but that doesn't match what is on the TSOM side (or that entity doesn't exist).

    On the TSOM side, one can also see the event being reported in the eventws.log.# log:

    eventws.log.2:INFO  06/08 09:28:41 [Thread-24,EXECUTOR_POOL::Dummy_SERVER:1496845626178::] DefaultEventConverter  Method: doSlotMapping, Sending event without any mapping for default event case: Event id: null, severity: null, date reception: null, status: null, {{msg=CPU will saturate in 13 days, severity=WARNING, utilization_measurment_unit=PCT, days_to_reach_threshold=13, cluster_name=, mc_object_uri=http://eng-bmapp02:8000/console/bcogw.do?UUID=420eef24-6d43-3831-250c-ae9ad3adb33f, unique_identifier=VI-UUID:420eef24-6d43-3831-250c-ae9ad3adb33f##CPUUTIL, mc_host_address=, CLASS=BCO_EV, datastore_name=, file_system_name=, mc_smc_alias=VI-UUID:420eef24-6d43-3831-250c-ae9ad3adb33f, event_type=VM_CPU_UTIL_EXCEEDS_THRESHOLD, mc_host=hostname1, is_predictive=TRUE, metric_utilization=, metric=CPU, threshold_measurment_unit=PCT, threshold_value=85.00, VM_name=hc-c16dwa01, entity=VM, partition_name=, host_name=, mc_tool_uri=}}

    There is also an audit message generated on the TSOM side that includes both the hostname and that VI-UUID:

    AdminAudit.log.1:INFO  06/06 10:29:15 AdminAudit           [Timer-3] 600002 User admin: mo.util.UpdateMO -action update -moTypeId 1 -moInstId 1391 -parentMOTypeId 0 -parentMOInstId 0 -cmdbReconId   -cmdbInstId   -cmdbDataSet   -doNotUpdateCell true -attrs 448  192 public 449 VI-UUID:420eef24-6d43-3831-250c-ae9ad3adb33f 450  451  452  453  454 BMC_ComputerSystem:172.20.53.34 455 0 456  457  458 labtsim.in.lab_1391 459  460 -1 461 [] 462 [] 463 [Full Access] 464 [Full Access] 465 0 466 0 467 Microsoft Windows Server 2008 R2 (64-bit) 468  469  470 hostname.in.lab@hostname1wa01 292 HOSTNAME.in.lab 433 4 434 22 435  436  244 2 437  245 2 438  439  440 1 441 HOSTNAME.in.lab 249 1 442 978 187 1391 443 2 251 1 188 172.20.53.34 444 172.20.12.126: 445 172.20.12.126 446 HOSTNAME.in.lab 190 10 447 2 191 172.20.53.34

    The following SQL can be executed by your TSOM Administrator can run to check the TSOM UUID of entity "hostname1":
      select * from DEVICE_CNTL where DEVICEDNSNAME like '%hostname1%' 

    If that SQL doesn't return anything or doesn't return the expected Token_ID value that would explain why the Alert isn't being associated.  If nothing is returned that would indicate that the VMware entity doesn't exist on the TSOM side (TSCO will only associate Capacity Alerts with existing entities on the TSOM side).  If a different Token_ID value is returned that could indicate that the entity in TSOM doesn't come from a VMware KM but is coming via another KM (such as a collector running within the guest).

    It is possible to look up the entity within the TrueSight Infrastructure Manager (TSIM) UI.

    To find the CI in the TSIM UI , once we login in the left pane we have different tabs and one is Find CI(marked), user can use that to find the CI specify the name of entity here and select the graph view on the top right hand side.

    At the bottom in the right pane there is Details view which has details of the device.

    The Advanced Tab has the details of the Aliases for the entity.

    Screenshot of TSIM

    So, for example, what one would look for in the output is to check the UUID of any 'hostname1' entities that exist on the TSOM side.  In particular one would be interested if the UUID reported for the 'VMware Virtual Machine' entity of that name matches the UUID that TSCO is trying to use ("VI-UUID:420eef24-6d43-3831-250c-ae9ad3adb33f") for the Alert parent.  If it does then that would indicate the problem is that although the entity exists within TSOM with that UUID it isn’t being reported in the DEVICE_CNTL table which is why the alert can be associated with it.  If the problem can be narrowed to that level it would indicate that the problem exists on the TSOM database side.

    Q: Does TSCO require the entity name to be fully qualified on the TSOM side in order to successfully look it up?
    A: In the scenario where TSCO is attempting to associate an alert with an entity that wasn't imported from TSOM It depends on the tokenID that is constructed on the TSCO side how an entity will be looked up.  So, we might see an error like this on the TSCO side:
      [pool-5991-thread-1]- BCO_PUB_ERR112: https://tsom.domain.com:8043/tsws/10.0/api/Device/hostname:domain.com/event?idType=URI,HTTP response code:404

    In that event TSCO is attempting to find an entity named 'hostname.domain.com' on the TSOM side. If the entity name is just 'hostname' on the TSOM side then it will not be successfully identified via that query and the Capacity Alert will not be mapped to it.

     


    Article Number:

    000138652


    Article Type:

    FAQ/Procedural



      Looking for additional information?    Search BMC Support  or  Browse Knowledge Articles