6 Replies Latest reply on Jul 8, 2020 11:36 AM by Reinaldo Chacon

    Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?

    Sheena George
      Share This:

      Is there any way to allow only particular devices to be displayed in TSCO while using the TSOM 10.1/10.5/10.7/11.0/11.3 extractor ETL?

        • 1. Re: Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?
          Ben Davies

          Sheena George  I had to read the question a few time realize what the question is...

           

          It seems you can to the platform level...

          you could also manipulate the 'Skip entity creation' to not allow this ETL to make new devices, effectively ignoring devices that are not already in TSCO, which may give the result you wish.

           

           

           

          https://docs.bmc.com/docs/capacityoptimization/btco2002/bmc-truesight-operations-management-10-1-10-5-10-7-11-0-11-3-ext…

          • 2. Re: Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?
            Sheena George

            Hi Ben,

            Thanks for the reply.

            My question is , if suppose i have 100 devices (same OS) added to TSPS. I need only 20 devices to be displayed in my TSCO workspace when i run the ETL TSOM 10.1/10.5/10.7/11.0/11.3 extractor. Where i can filter it ? i dont find any option to whitelist or blacklist the devices while creating the ETL.

             

            Also, this ETL will gather only those devices which has patrol agent on?

            • 3. Re: Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?
              Ben Davies

              Simple answer to the exact question "With 100 devices on the TSPS, can I use the ETL TSOM 10.1/10.5/10.7/11.0/11.3 extractor to get only 20 of those?"   Implying no other factors. No.  There is no opportunity to white list or black list to get only the 20 subject systems within that ETL.

               

              More Complex answer to the situation in general.

               

              The ETL engine can get any data from any data source provided that the data is accessible and addressable, that is to say has a path and at least semi structured.  This structure can be in the form of log data with predictable format up to strictly structured database data.  The 'trick' is for each step of the ETL (Extract Transform Load) to be addressed.  Using the 100 devices question as the scenario, here are a couple ways to get only the subject 20.

               

              The extraction step needs a network path, credentials, permissions, and query that get the expected data.

              The Transform step need to match and gap the metrics.  What does the source call the metric and its format vs. what does TSCO have defined for that metric name and format.

              The Load step needs to take the results of the first two steps and place it into the TSCO database (via the Data Warehouse).

               

              Using the built in ETL 'TSOM 10.1/10.5/10.7/11.0/11.3 extractor', the query and match and gap steps are done but there is no opportunity to white list.  There is, however, an opportunity to not let the ETL make (load) new devices.  So if the 20 subject devices were already in TSCO this switch would only populate those items, effectively acting as a white list.

               

              These devices could be made with a separate custom ETL from some data source such as a CMDB (Configuration Management DataBase); then the TSOM ETL run without the ability to make new devices.

               

              These devices could also be made with a custom ETL but that would leave you to reproduce the extractor query, match and gap, but then allows for a white list such as a "where in ()"  statement, then load the metrics.

               

              You could also just add the 100 devices to TSCO (Capacity Optimization) and use a 'tag' to identify the subject 20 devices for reporting in the workspace of TSCO as well as the TSPS (Presentation Service).  This last option allows for the 'other 80' to be reported on as well. In my experience TSCO provides the best value when reporting on complete subsets of data, and is common design feature in built in ETLs (that is to say get all capacity data for all devices from a data source).  This is, in my opinion, the most correct of these options (based on the data we have). There are however, non trivial impacts to this simple to implement solution.

               

              We can help you with this.

               

              So we put the two architecture drawings together (BMC TrueSight Capacity Optimization and BMC TrueSight Operations Management) to help clarify the parts and pieces if additional conversation is needed.

              1 of 1 people found this helpful
              • 4. Re: Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?
                Sheena George

                Hi Ben,

                 

                Thanks for the detailed explanation. 100 devices was just an example. We have more than 10,000 devices and i don't think, pulling the entire devices is the right option in order to take just 50 out of this.

                But i liked the idea of the adding those 50 devices already in TSCO so that the ETL will not make any new devices. I will give a try of this.

                 

                Thanks again for this idea and i hope there will be a filter option added to this ETL soon.

                • 5. Re: Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?
                  Ben Davies

                  Sheena George if you would mark the answer you found 'most correct' that would be helpful

                   

                  Then as a general note for others writing questions, it would be helpful to keep the proportions correct even if the numbers are not.  So the question was "I have 100 and wish 20"  but a better question for your real life proportion of 10,000+ vs 50 would have been 1,000 vs 5.   The suggestion to use a CMDB to populate the TSCO then use the ETL without allowing it to make devices IS the most correct answer with these proportions,  where adding 80 devices that are potentially report able in the future, so that you avoid the whole mess is arguably a correct answer. However, this suggestion falls apart quickly as the counts grow.

                   

                  Anyway, the take away from this conversation is that there are tools and techniques that can be combined to achieve a good result.  Creatively, within the confines of the tool requirements and limitations, gives way more flexibility than is first apparent.

                  2 of 2 people found this helpful
                  • 6. Re: Is there any way to filter/whitelist devices in TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL?
                    Reinaldo Chacon

                    Hello Sheena,

                     

                    Currently there isn't a blacklist/whitelist feature available on the TrueSight Operations Management 10.1/10.5/10.7/11.0/11.3 extractor ETL. However the feature of Platform filtering allows you to filter systems as follows:

                    • Standalone (any OS)
                    • AIX Power VM/WPAR
                    • MS HyperV KVM
                    • Solaris LDOM/Zone

                     

                    Hope this helps.

                    Reinaldo C.