6 Replies Latest reply: Aug 30, 2012 8:16 AM by Alexander Kreußler RSS

    How to automically add a new system to an already existing derived system

    Alexander Kreußler

      Hello colleagues,

      in our BCO perimeter we work a lot with derived systems to aggregate resource consumption. I now wonder how we can automatically make a newly discovered object, e.g. a new server, become a member of an already existing derived system.

       

      Example: A BPA ETL populates our BCO Unix domain with server metrics. In BCO we have a derived system representing all non-clustered Unix servers. New hosts in BPA are properly discovered by BCO automatically, but adding the new hosts to the derived system remains as a manual task. How can I automate this.

       

      Any input is appreciated,

      Regards,

      Thanks and have a nice weekend,

      Alexander

        • 1. Re: How to automically add a new system to an already existing derived system
          Renato Bonomini

          Hello Alexander,

          I found that the easiest way to achieve your goal is to create what we informally refer to as an 'Auto-ETL', which means an ETL that computes data out of BCO itself.

           

          This is the workflow

          • you identify the criteria of aggregation
          • you code this criteria in SQL (e.g. all operating systems like 'Solaris%')
          • you set-up a SQL Extractor
            • input is the BCO database (Public Views help in this sense)
            • the criteria are the ones identifies above
            • the output is a 'logical' system, equivalent to your derived system

           

          For example, using this procedure we have been able often to represent 'All Unix Machines', 'All Solaris Machines', 'Whole datacenter' systems, summing up and representing the complete aggregate.

          This gives a nice representation of the total computation power of the IT infrastructure (if you are getting data from BPA, you can also compute aggregations weighted on specint ratings!).

           

          Adding an Object-Relationship ETL would also enable you to fully mimic the same behaviour of the derived entity mechanism, which as far as I am aware of cannot be automated as-is.

           

          Please let me highlight that if all if you need to perform is reporting (e.g. no forecasting), you can do a very similar aggregation in the analysis itself.

          Screen Shot 2012-08-24 at 08.04.54.png

           

          Notes

           

          The nice alternative closer to using this mechanism is a 'Hierachy Rule', as the one named 'Typical Physical Machine for Application Layer', but I don't see any that, as shipped, will fit your needs as you describe.

           

          Cheers,

          Renato

          • 2. Re: How to automically add a new system to an already existing derived system
            Alexander Kreußler

            Hello Renato,

            first of all thanks for the answer.

             

            I choosed to go for an OBJREL ETL as I use this functionaly in a similar way to sort new systems in domain folders according to a given criteria. But applying it on my "derived system" problem is still not successfull.

             

            Details: My derived system is called "Pool Privat Number CPUs", the systems to add are "xsph10a038w" and "xsph10c122k".                                                                   

             

            My "Entities query" returns

             

            ENTNM                              DS_ENTNM                                                        ENT

            -------------------------                ----------------------------------------                                  ---

            Pool Privat Number CPUs   Pool Privat Number CPUs                                     SYS

            xsph10a038w                     9A15C4494B44BD5483531674D8475FB8              SYS

            xsph10c122k                     9A2EADD55B7F9A33C44514D5C562C31F            SYS

             

            My relationship query returns

             

            ENT ENTNM                               DS_ENTNM                                                      ENT   DS_ENTNMPARENT

            --- -------------------------                     ----------------------------------------                                ---      -----------------------

            SYS Pool Privat Number CPUs   Pool Privat Number CPUs                                   SYS    _ROOTAPP_

            SYS xsph10a038w                     9A15C4494B44BD5483531674D8475FB8            SYS    Pool Privat Number CPUs

            SYS xsph10c122k                     9A2EADD55B7F9A33C44514D5C562C31F          SYS    Pool Privat Number CPUs

             

            Unfortunately nothing happens, the two servers are not added to the derived system. Any clue?

             

            Regards,

            Thanks

            Alexander

            • 3. Re: How to automically add a new system to an already existing derived system
              Renato Bonomini

              Hello Alexander,

              the OBJREL transaction will only associate parent-to-child but it will not modify the derived entity, whose aggregation is not managed by this relationship.

               

              My suggestion is to

              • Task 1: create your own 'equivalent' system by using a SQL Extractor that works exactly like the derived entity (in your case, a SQL Extractor summing xsph10a038w and xsph10c122k into Pool Privat Number CPUs)
              • Task 2 (optional) create the association with an OBJREL transaction

               

              Task 2 is merely 'graphical'.

               

              R.

              • 4. Re: How to automically add a new system to an already existing derived system
                Alexander Kreußler

                Hello Renato,

                OK, refering to your suggestion I still have a lack of understanding - maybe I am a bit slow on the uptake. You describe to define "input", "criteria" and "output" in the conf of an SQL extractor. Not sure how to map this on SYSNM, DS_SYSNM, OBJNM, SYSTYPEENM and VALUE columns of a generic database extractor SQL query.

                 

                Regards,

                Thanks,

                Alexander

                • 5. Re: How to automically add a new system to an already existing derived system
                  Renato Bonomini

                  Hello Alexander,

                  Input, criteria and output are terms I used to describe high level actions, not actual column/attribute names,

                  let me add some details to my answer with a full example.

                   

                  Please note that this 'manual' method is much more advanced and requires more coding than the derived entity as you will have full control (and responsibility) over the data extraction and aggregation.

                  For simplicity, it only aggregates CPU Utilization.

                  If you don't feel comfortable or you'd require more information, I would recommend involving Professional Services (BMC or partners such as Moviri where I work) or BMC support.

                   

                  • you identify the criteria of aggregation
                    • for example 'Aggregate CPU utilization from all machines belonging to domain X having Solaris as Operating System'
                  • you code this criteria in SQL (e.g. all operating systems like 'Solaris%')
                    • all machines that belong to Domain X: and pa.applname like 'X'
                    • all machines whose Operating System is Solaris: and psmconf.metric like 'OS_TYPE'  and pconf.value like 'Solaris%'
                    • aggregate CPU (hourly values): SUM(pdata.avgvalue*pdata.duration)/SUM(pdata.duration).
                  • you set-up a SQL Extractor
                    • input [data source] is the BCO database
                    • the criteria are the ones identifies above, coded in SQL
                    • the output is a 'logical' system, equivalent to your derived system - I make up a name based on the criteria concat(concat(concat('myequivalent_',pconf.value),'_in_'),pa.applname) as ds_sysnm

                   

                  Here below the minimal query to be used for SQLE. Please note

                  • I've added only the fields strictly required by the example (ts, ds_sysnm, objnm, duration, value), other fields such as sysnm,locationm,subobjnm might be added
                  • the aggregation logic is extremely simple here and it does not consider missing samples, 0 duration and other things. Let's say it's a simple example for now

                   

                  select

                    pdata.ts as ts,

                    concat(concat(concat('myequivalent_',pconf.value),'_in_'),pa.applname) as ds_sysnm,

                    'CPU_UTIL' as objnm,

                    avg(pdata.duration) as duration,

                    SUM(pdata.avgvalue*pdata.duration)/SUM(pdata.duration) as value

                  from

                    pv_application pa ,pv_appl_sys pas,pv_system ps,

                    pv_sys_metric psmconf, pv_sys_conf_last pconf,

                    pv_sys_metric psmdata, pv_sys_data_day pdata

                  where

                  /* join application to system and get all children */

                    pa.appid=pas.appid

                    and ps.sysid=pas.sysid

                    and pa.applname like 'E%'

                  /* get systems with Solaris OS */

                    and psmconf.sysid=ps.sysid

                    and psmconf.sysmetricid=pconf.sysmetricid

                    and psmconf.metric like 'OS_TYPE'

                    and pconf.value like 'Solaris%'

                  /* get CPU data */

                    and psmdata.sysmetricid=pdata.sysmetricid

                    and psmdata.sysid=ps.sysid

                    and psmdata.metric like 'CPU_UTIL'

                  /* valid configuration fields */

                    and pdata.ts between pconf.sincedate and pconf.todate

                  /* grouping */

                    group by pconf.value, pa.applname,ts

                    order by ts

                  • 6. Re: How to automically add a new system to an already existing derived system
                    Alexander Kreußler

                    Hello Renato,

                    OK, now got it. With the help of the SQL code example I was able to successfully reproduce this approach in our BCO environment.

                    Thank you very much for taking your time to go through this in detail.

                    Regards,

                    Alexander