12 Replies Latest reply on Dec 9, 2019 1:38 AM by Kaushik KM

    match_regex -- Retrieving 1st group of multiple matches from a string

    Kaushik KM
      Share This:

      Hi ALL,

       

      Need a quick help here,

      I am trying to retrieve the 1st group element of multiple match from a  string(msg slot),

      msg :

      108 OPEN Failure rate increase on 2 Services SERVICES Failure rate increase OPEN Problem 108 in environment ProductionProblem detected at: 00:07 (EST) 08.11.20192 impacted servicesWeb serviceCsrCustomerProfileControllerFailure rate increase5.2 requests/min impactedby a failure rate increase to 64 %Service method: All methods affectedWeb serviceInternalEStatementsControllerFailure rate increase1.6 requests/min impactedby a failure rate increase to 90 %Service method: All methods affectedRoot causeWeb serviceInternalEStatementsControllerFailure rate increase1.6 requests/min impactedby a failure rate increase to 67 %Service method: All methods affectedWeb serviceCsrCustomerProfileControllerFailure rate increase5.2 requests/min impactedby a failure rate increase to 64 %Service method: All methods affectedhttps:/

      sdfasdjdfkladsjkfl

      asjkdfasdjflkadsjfkladsjfladsjfl

      asjdfkjasldkfjadsklfjkladsjfuqeeyfssfkljdlk

       

       

       

      match_regex($EV.msg,impactedby a failure rate increase to (\d+) %,'',$FLDS);

       

      in the above text i want to match all possible matches and retrieve the digits in a LIST_OF STRING variable ($FLDS)

       

      desired output : $FLDS -- should contain ['64','90','67','64'] , Can someone please help me with correct regex  ?

       

      my regex retrieves only the first value. which is 64, in perl "/regex/g" does the work for us. but i am not able to give the option "/regex/g" (global match) in MRL.

       

      Any ideas are helpful.

      Thank you,

      Kaushik KM.

       

        • 2. Re: match_regex -- Retrieving 1st group of multiple matches from a string
          Brendan Murray

          Hi Kaushik,

           

          I don't know of a way to do a global regex match in MRL. Also, I try to avoid using match_regex because it is computationally expensive. Where possible, I use the standard cell string and list processing functions. You could solve your problem by breaking up your msg slot into a list with strtolist and then doing a listwalk to process each member of the list. The listwalk function is, as far as I know, the only loop-like programming structure in MRL.

           

          Here's an example that works for the sample event you provide (note that I am using the PEM_EV event class just for testing purposes):

           

          refine extract_values : PEM_EV ($EV)

          where [$EV.status != 'CLOSED' AND $EV.mc_object_class=='test']{

           

               # Convert the msg slot to a list. Use the percent sign as the separator.

               $FLDS = strtolist($EV.msg, '%');

           

               # For each element in the list of fields.

               $ELEM = listwalk($FLDS);

           

                    # Extract the numeric value from the list element as a string

                    # and strip any blanks off the beginning and end of the string.

                    $STRNUM = strip(strextract($ELEM,len($ELEM)-2,2));

           

                    # Store the numbers in the long message slot

                    # as a comma-separated list.

                    $EV.mc_long_msg = concat([$EV.mc_long_msg, ', ', $STRNUM]);

          }

          END

           

          refine trim_values : PEM_EV ($EV)

          where [$EV.status != 'CLOSED' AND $EV.mc_long_msg != '']{

           

               # Trim the unwanted characters off the ends of the mc_long_msg slot.

               $EV.mc_long_msg = strextract($EV.mc_long_msg, 3, len($EV.mc_long_msg)-6);

           

          }

          END

           

          The result is this:

           

          Detailed Message.png

           

          A few notes:

           

          1. I had to use two rules. One to do the initial parsing and a second to trim the unwanted characters off the ends of the string. The second rule is necessary because all statements after the listwalk are executed for each element of the list. I needed a separate rule to process the final result of the listwalk just once.

           

          2. When the listwalk is complete, the mc_long_message slot will have some unwanted characters at both ends. There will be a comma and a space at the beginning of the string and a comma and a space and two junk characters at the end of the string. The junk characters are from the last element of the $FLDS list which follows the final % sign. There will be no valid number in this last element, so we just get the last two characters at the end of the event. Hence the need for the second refine rule.

           

          3. My final result is a string data type, not a list as you specify above. However, you can easily convert it to a list with strtolist. Or, if you have an available list slot in your event class, you could use the add_to_list function to add each number directly to the slot.

           

          4. I have only the one sample event you provided to test the rules with. You will need to test it with a larger data set to see if the rule works for all variations of the event.

           

          5. My rules are simpler than they look. If you strip out the comments, they are actually pretty short.

           

          Let me know if this solves your problem.

           

          Regards,

           

          Brendan

          2 of 2 people found this helpful
          • 3. Re: match_regex -- Retrieving 1st group of multiple matches from a string
            Kaushik KM

            Hello Brendan Murray,

             

            The logic is Good! and thanks for listwalk, I will give this a try and see how it goes as a workaround.

            I still would like to see how to use the Global match regex in match_regex primitive, Havent seen any examples till now, I request you to check internally if possible.Similar question is unanswered in the post : match_regex issue/question .

             

            Thank you for explaining notes point 2 above, Waiting for examples,

             

            Thanks,

            Kaushik KM

            • 4. Re: match_regex -- Retrieving 1st group of multiple matches from a string
              Kaushik KM

              Hi Brendan Murray,

               

              Any specific function to convert from List_of string directly to List_of Integer ?

              i can use other logic anyways to convert and use the variable, but i was unable to find any direct function to do the above.

               

              Thanks,

              Kaushik KM

              • 5. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                Brendan Murray

                Hi Kaushik,

                 

                There is no MRL function to convert List_of string to List_of integer. You will have to convert the values to integer when you extract them from the list_of string.

                 

                For example:

                 

                $INTVAL = stringtoint(listgetelt($MYLIST,1))

                 

                Another approach would be to add a custom slot to your event class and make it a LIST_OF INTEGER type. Then you can extract the values as strings, convert them to integers and then use the add_to_list function to add them to the slot, one at a time. At least, that is the theory. I have not tried it myself.

                 

                Regards,

                 

                Brendan

                • 6. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                  Brendan Murray

                  Hi Kaushik,

                   

                  I am checking with R&D on your question regarding a global modifier for the match_regex function. I will post the answer here when I get it.

                   

                  Regards,

                   

                  Brendan

                  • 7. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                    Tim Bocardo

                    If you still want a regex, you can use this:

                              .*?(impactedby a failure rate increase to (\d+) %)+

                    so this:

                              match_regex($EV.msg,".*?(impactedby a failure rate increase to (\d+) %)+",$FLDS);

                    (I'm not sure if you need the double quotes... I didn't test the mrl)

                    By wrapping the repeating part in parens and adding the plus at the end turns it into a multiple-match-unknown-quantity expression.

                    After the match, your results are in the even-numbered elements of the list:

                    $FLDS[1] is "impactedby a failure rate increase to"

                    $FLDS[2] is "64"

                    $FLDS[3] is "impactedby a failure rate increase to"

                    $FLDS[4] is "90"

                    etc.

                     

                    I checked it on regex101.com (a regex tester) and it parsed out just as I said.

                    1 of 1 people found this helpful
                    • 8. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                      Kaushik KM

                      Hi Tim Bocardo,

                       

                      1)It dosent work as expected in MRL,

                      2)In 101regex :  /regex/g  will be enabled by default i guess.

                       

                      =======================================

                      refine catch_num : MYCLASS ($EV)

                      where [ $EV.mc_object_class hassubstring 'Failure rate']

                      {

                      ntadd($EV,"Inside where");

                      if (match_regex($EV.msg,".*?(impactedby a failure rate increase to (\d+) %)+",'',$FLDS))

                      then

                      {

                           ntadd($EV,"String matched");

                           $EV.mc_object = listgetelt($FLDS,2);

                           $EV.mc_parameter_threshold = listgetelt($FLDS,1);

                           $EV.mc_parameter = listlen($FLDS);

                      }

                      else

                      {

                           ntadd($EV,"String not matched");

                      };

                      }

                      END

                      ============================================

                       

                      output :

                      listlen --> is just 2 (which means it has not taken all the values)

                       

                       

                       

                      Hi Brendan Murray,

                      The above logic to pick last 2 digits before the % character. fails as we also get values sometimes is some decimal. example 5.75 % --> it will pick up 75 which is incorrect.

                      Just curious to know if at all any info from R&D regarding global match.

                       

                      Thanks,

                      Kaushik KM

                      • 9. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                        Brendan Murray

                        Hi Kaushik,

                         

                        In my example, I was trying to show you how to solve your problem without using match_regex. I only had the sample data you provided, which didn't have any decimal numbers in it. Now that we know there will be decimal numbers, you have two options:

                         

                        1. Use MRL string functions to solve the problem. There is almost always a way to use combinations of string functions to solve problems like your decimal numbers.
                        2. Use match_regex on each element of the list in the listwalk. Since there is only one value you want to extract in each element, the lack of a global search is no longer a problem.

                         

                        To use match_regex, simply replace this line in my code:

                         

                        $STRNUM = strip(strextract($ELEM,len($ELEM)-2,2));

                         

                        with a modified version of your original match_regex expression:

                         

                        match_regex($ELEM,impactedby a failure rate increase to (\d+) %,'',$FLDS);

                         

                        Just be aware that the combination of listwalk and match_regex may cause performance problems. I'm not saying it will, but keep an eye on performance (latency). On the positive side, you should no longer need my second rule, so it may balance out. Also, the final line of my first rule will probably need to be changed since the output of match_regex is a list. My code expects a string. I am sure you can make the necessary changes.

                         

                        By the way, I was also not able to get Tim's regex to work as desired in MRL. It works great on regex101, but I suspect you are correct that it automatically applies the /g modifier.

                         

                        I did ask our R&D organization for a definitive answer. They have not yet responded. I will try again.

                         

                        Regards,

                         

                        Brendan

                        • 10. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                          Kaushik KM

                          Hi Brendan Murray,

                           

                          Thank you for suggestion !! was able to get it working, below is what i tried,i had to modify regex a bit to capture 1st two decimal points. someone might find it helpful.

                          -------------------------------------------------------------------------------------------------------------------------------------

                          refine extract_values : MY_CLASS($EV)

                          where [ $EV.mc_host equals 'services' AND $EV.status not_equals 'CLOSED']

                          {

                          $FLDS = strtolist($EV.msg, '%');

                          $ELEM = listwalk($FLDS);

                           

                                  if (match_regex($ELEM,"impactedby a failure rate increase to (\d+(\.\d{1,2})?)",'',$VAL)) then

                                  {

                                    if((listgetelt($VAL,1))> 50 ) then

                                    {

                                    $EV.mc_parameter_value = "qualified";

                                    };

                                  };

                          if (($EV.mc_parameter_value == "qualified")) then

                          {

                                  $EV.status = "OPEN";

                                  $EV.my_incident_priority = PRIORITY_3;

                                  $EV.my_incident_severity = "MINOR";

                                  $EV.mc_priority = PRIORITY_3;

                                  $EV.severity = "MINOR";

                                  ntadd($EV,"NORMCELL: % Failure is greater than 50, Setting the severity to P3");

                          }

                          else {

                                  $EV.status = "CLOSED";

                                  ntadd($EV,"NORMCELL: % Failure is less than 50, closing the event");

                                  };

                          }

                          END

                          ==========================================================================

                          I will wait for the definitive answer for global modifier in match_regex to mark the answer.

                           

                          Thank you

                          Kaushik KM!

                          • 11. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                            Brendan Murray

                            Hi Kaushik,

                             

                            I have gotten confirmation from a developer that the global modifier is not supported by the regex libraries we use in the cell. The only modifiers supported are the ones listed in the documentation. Sorry to be the bearer of bad news.

                             

                            If you feel we should add the global modifier to the cell, please create an Idea on Communities. I can't guarantee we will add the feature, but if enough people vote it up, it will get the attention of our Product Management team.

                             

                            Regards,

                             

                            Brendan

                            • 12. Re: match_regex -- Retrieving 1st group of multiple matches from a string
                              Kaushik KM

                              Thank you Brendan Murray for clear confirmation.  

                              'Listwalk' primitive may become useful in this case, as you showed above.

                               

                              Thank you,

                              Kaushik KM