4 Replies Latest reply on Jul 16, 2018 9:49 AM by Lee Foster

    Update to EOT Correlation Logic

    Stewart Crighton
    Stewart Crighton Feb 8, 2018 9:46 AM

      Greetings -

       

      This update was made to assist in requests we were getting for Correlation of many devices.  Either the requests were not specific on what the Cause and Effects are.  Or the requests were so large that keeping track of the changes would be challenging.  This attempts to reduce the number of entries required to track which events should be correlated.


      At the same time an update was made to allow the correlate logic to be dynamically driven.  The specific issue was that requests were also coming in to drive a new action, or update an existing action all the time.  So the correlate rule now drives its when decisions based on table entries.  It then drives the logic of what to update based on the match table entries.

       

      Any feed back on this to improve the process would be appreciated.  I have tested it at a base level, and it functions for my needs.  However; I would like to make it generic enough so that any new requests I get will not be an issue.

       

      Thank you;

      Stewart

       

      Attached is a document the goes over an update to the EOT logic for Same Domain Correlation.

       

      The out of the box EOT code was slightly modified.  Specifically the removal of the rules:

      same_domain_check

      same_domain_event

      everything

       

      The top two generate an event to assist in the correlation logic.  This did not work for our environment, so I removed them.  I had to add a rule that has the same logic as same_domain_check.  However; it searches the known Events in the system instead of the new event class.

       

      The everything rule was removed and replaced with two new rules.  The first rule is for the Effect Events, and the second rule is for the Cause Events.

       

      Finally a new rule was added to parse out information from events and attempt to make generic entries for correlation..

        • 1. Re: Update to EOT Correlation Logic
          Betty Neumann
          Betty Neumann Employee Apr 2, 2018 9:19 AM (in response to Stewart Crighton)

          Stewart, thanks for the great information! I hope that the Community does come through with any suggestions and tips. If not, please do open a case with Support or speak to your Account Representative regarding Professional Services review of this concept if needed. We do appreciate the post and hope to see feedback soon!

           

          Thanks

          Betty Neumann

          BMC Support

          • 2. Re: Update to EOT Correlation Logic
            Lee Foster
            Lee Foster Employee Jun 22, 2018 7:16 PM (in response to Stewart Crighton)

            I have to agree there is some work that needs to be done on the EMF cell.  Rick Floyd and I have been friends for years and he was the original author of it.  The last time I spoke to him was on the EMF cell.  When I describe some of the issues I saw in the design, some of the cases I have worked and some design additions that I saw that was missing.  He just told me it’s a framework and not a product.  If I saw a change that I thought was necessary then just make it.  He added that he has made numerous changes to the given cells since he left BMC and demoed some of them to me.

             

            As far as your observations I will agree that this section of code needs to be reworked.  I have worked a couple of cases that we had trouble with in regards to performance.  It’s on my list to rework the code in my lab environment.  Quickly reading through the document I noticed that you used policies.  I personally avoid using policies but I will go through the logic you have demonstrated as see if I can incorporate those concepts.  I will let you know after I find time to rework this section of code but it could be couple of months before I find the time to get everything in NORM cell reworked and tested.

            • 3. Re: Update to EOT Correlation Logic
              Stewart Crighton
              Stewart Crighton Jul 15, 2018 12:58 PM (in response to Lee Foster)

              Personally I am a big fan of the EMF.  Even if someone does not implement this ruleset, I would recommend being familiar with it.  The concepts used to accomplish many of the functions can be easily re-used in many other solutions.  To be honest, the logic used for the correlation rule, is really just re-applying ideas I found in other sections of the EMF ruleset.

               

              As for your statement that you avoid policies....  I am wondering why.  I always just thought of them as another basic type of table essentially.

               

              The only reason that I used it here was for the dynamic where statement.  I did not test to see if a separate Dynamic Data Table could be used instead....  so that may be an alternative solution.  I did test with the single DDT, but the rule would not trigger correctly.  I assume it has to do with being used in the WHEN and WHERE statements at the same time.

              • 4. Re: Update to EOT Correlation Logic
                Lee Foster
                Lee Foster Employee Jul 16, 2018 9:49 AM (in response to Stewart Crighton)

                Yes, you could.  Am coming from an older programming background where I started off with Fortran, Cobol, Pascal, Assembler, paper tape, punch cards and mag tapes.  The first computer I programmed on was a PDP 11/70 working on a medical diagnosing program for record keeping.  That been said, there isn’t a technical or performance reason for my choice.  It just conflicts with my programming style.  Using one of my old sayings from the day; “If you take a problem to a room of ten programmers, you will get 10 different answers”.  My view point is; I rather have the code in one place.  It’s easier to main maintain and I find the policies in themselves, limiting in effect.  If I use both it adds a level of complexity in both maintaining and troubleshooting.  Now, I can’t say I haven’t ever used policies but I just avoid them for the fore mentioned reasons.  In an initial cell where it is just a pass-through cell where some de-dup, limited filtering, etc is performed before the event is sent to a normalization cell; then yes on that cell I might use them but if I do then it will be policy only for the most part; ruling out possible filter rules in the MRL as the only exception.

                 

                As far as the tables; yes you could add data tables and match tables if you saw a need for it.  I have in fact been moving tables around the various cells where I am creating, combining, reorganization and dividing them as I see fit.  No one will look at them or even support my cell structure and design.  It just lives in the lab and it will never see the light of day as I can’t share it.  But I will share one change I added verbally.  I added in the normalization cell a new table that will classify the events based on the value of mc_object_class where it will set the application slot.  For example, if the mc_object_class is for the Oracle KM, it will set the bmcps_application slot to “Oracle”.  In the ENR cell where am writing code now, it will set the slots for ITSM ticketing in regards of where it goes; tier 1-3, assignment and so on.  When it gets to the TSIM cell it will have to perform model lookup to extract additional information that I don’t want to have to back populate in other cell tables.  I could setup a model (or copy of it really) on the ENR but at the moment I don’t see the use case for it.  But thinking about it, it would help with the notification of key events and it would help with CI-based blackouts to remove that load from the TSIM cell.  However, at the moment am following suite with the company I closely work with where I know their use case like the back of my hand.