9 Replies Latest reply on Nov 27, 2019 2:50 AM by Andrew Waters

    Text file comparison

    Vishnu Parihar
      Share This:

      Hi, I have written a TPL to extract the data from text(WIndows) or rxt(Unix) file, that is working fine. That text file contains the data like:

      AppName=BMC Discovery

      OwnerName=John Wick

      OwnerContact=john.wick@gmail.com

       

      Filename : customdata.txt

       

      Now with the help of custom TPL we have extracted the data from Windows and Unix host, but the text file is free text file and any one can change the text file that will affect the CMDB data as well.

       

      Now I am planning to create different TPL to compare the old and new data and whatever difference/change is there, just trigger a mail to Discovery admin.

       

      Note: Filename will remain same. it will never change, only data will change inside the file.

       

      i.e after one month the server owner has changed from John to Edwin Loren, then server owner details will change from John to Edwin.

      Now I want Discovery should compare the data and send a mail to Discovery admin, if changes exist else exit.

       

      I also go through with below discussion, it is very useful but not in my case. Below article is comparing two different file but I want to compare same file during every scan..

       

      ADDM Experts - I want to use TPL to find a specific Unix host and diff two known files. If there is a difference then send an email warning that the file changed. If it does not find a difference then just exit gracefully out of the pattern. - Thank You

        • 1. Re: Text file comparison
          Brice-Emmanuel Loiseaux

          The only way to know the file content changed is to parse it, no? Why don't you enrich your initial pattern to email admin if you detect a change compared to what is currently stored on your Host.

          Alternatively, if a file checksum command is available on your target Host, you could run a file checksum, store it on the Host node and compare it next time you scan the Host before you parse the file.

          • 2. Re: Text file comparison
            Vishnu Parihar

            Thanks Brice for you suggestion.

            Below I attached the short code of custom data extraction from Windows, could you please help me to understand, how to compare ?

             

            tpl 1.15 module customdata;

             

             

            configuration ComplDiscCfg 3.0

            """Set Configuration options to control location of Compliance File Discovery"""

            "Common Windows File Locations"

            win_location := ["c:\\custom\\customdata.txt"];

            end configuration;

             

             

            pattern CustData 1.1

             

             

            triggers

            on host := Host created, confirmed where os_class = "Windows";

            end triggers;

             

            body

             

             

            for custdata in ComplDiscCfg.win_location do

            log.info ("Looking in directory %custdata%");

            instance_file := discovery.fileGet(host, custdata);

             

            if instance_file and instance_file.content then

            log.info ("Found the custdata file.");

             

            OwnerName := '';

            UsedByBusinessContact := '';

             

            det_type :=  "Windows Custom Data";

             

            OwnerName := text.strip(regex.extract(instance_file.content, regex 'OwnerName=(.*)', raw '\1'));

             

             

            if OwnerName <> "" then

            OwnerName := text.upper(OwnerName);

            log.debug ("OwnerName = %OwnerName%");

            else

            OwnerName := "Unknown";

            list.append(missingValuesList,"OwnerName");

            log.debug ("OwnerName not found");

            end if;

             

            det_name := "%host.hostname%";

            det_key := "%det_name%/%host.key%";

             

            //ADD a new detail to the host node

            detail := model.Detail(

            key  := det_key,

            name := det_name,

            type := det_type,

            OwnerName := OwnerName,

             

            model.rel.Detail(ElementWithDetail := host, Detail := detail);

            model.setRemovalGroup(detail, det_type);

            end if;

            end for;

            end body;

            end pattern;

            • 3. Re: Text file comparison
              Andrew Waters

              Basically you would need to find the Detail from a previous scan and check if the values have changed.

              details := search(in host traverse ElementWithDetail:Detail:Detail:Detail where type = %det_type%);

               

              if details then

                old_detail = details[0];

               

                // Compare the fields, e.g.

               

                if old_detail.OwnerName <> OwnerName then

                  ....

                end if;

              end if;

              1 of 1 people found this helpful
              • 4. Re: Text file comparison
                Bob Anderson

                add these attributes from instance_file to the Detail node.  Then compare them on every run:

                 

                instance_file.size

                instance_file.md5sum

                instance_file.last_modified

                 

                If any of these have changed since last discovery run, send email, then update the values in the Detail node

                 

                From ViewTaxonomy:

                1 of 1 people found this helpful
                • 5. Re: Text file comparison
                  Andrew Waters

                  This is true if you are sure the only changes there are modify the value or values you are interested in.

                  1 of 1 people found this helpful
                  • 6. Re: Text file comparison
                    Vishnu Parihar

                    Hi,

                     

                    I tried it with couple of ways....the pattern is working successfully but if we are doing first time Discovery then we are getting error like:

                    "RuleError on rule tpl_WindowsSoftDatabyText_body_1 due to: Error while executing a rule -- AttributeError: 'str' object has no attribute 'get' "

                     

                    Below is pattern:

                     

                    tpl 1.15 module customdata;

                    configuration ComplDiscCfg 3.0

                         """Set Configuration options to control location of Compliance File Discovery"""

                         "Common Windows File Locations"

                         win_location := ["c:\\custom\\customdata.txt"];

                    end configuration;

                     

                    pattern CustData 1.1

                     

                    triggers

                         on host := Host created, confirmed where os_class = "Windows";

                    end triggers;

                     

                    body

                         for custdata in ComplDiscCfg.win_location do

                         log.info ("Looking in directory %custdata%");

                         instance_file := discovery.fileGet(host, custdata);

                         if instance_file and instance_file.content then

                              log.info ("Found the custdata file.");

                                   ApplicationName := '';

                                   det_type :=  "Windows Data";

                     

                    ApplicationName := text.strip(regex.extract(instance_file.content, regex 'ApplicationName=(.*)', raw '\1'));

                     

                     

                    //Softdata comparision with new data....

                         log.debug("Windows_SoftData: Seeking if the softdata is already present...");

                         details := search(in host traverse ElementWithDetail:Detail:Detail:Detail where type = %det_type%);

                         if details then

                              old_detail := details[0];

                              log.debug("Windows_SoftData: Existing file found");

                              log.debug("Existing Application - %old_detail.ApplicationName%");

                         else

                              log.debug("Windows_Softdata: First time discovery of softdata");

                         end if;

                    //***************************************************************************************************************************

                         log.debug("Windows_SoftData: Starting softdata Discovery...");

                              if ApplicationName <> "" then

                                   if text.upper(ApplicationName) <> old_detail.ApplicationName then

                                        log.debug("Windows_SoftData: (if.2) Old ApplicationName & New ApplicationName is not same");

                                        list.append(updatedSoftdata, "ApplicationName - is changed from %old_detail.ApplicationName% to %ApplicationName%");

                                        log.debug("Windows_SoftData: (if.2) Data appended: ApplicationName - is changed from %old_detail.ApplicationName% to %ApplicationName%");

                                             if old_detail.ApplicationName = "Need to fill !" or "" then

                                                  old_detail.ApplicationName := "BLANK";

                                                  log.debug("Windows_SoftData: Old ApplicationName is NULL");

                                                  list.append(updatedSoftdata, "ApplicationName - is changed from %old_detail.ApplicationName% to %ApplicationName%");

                                                  log.debug("Windows_SoftData: (if.3)Data appended: ApplicationName - is changed from %old_detail.ApplicationName% to %ApplicationName%");

                                             end if;

                                        end if;

                                        ApplicationName := text.upper(ApplicationName);

                                        ApplicationName := text.upper(ApplicationName[0]) + text.lower(ApplicationName[1:]);

                                        log.debug ("Windows_SoftData: ApplicationName = %ApplicationName%");

                                   else

                                        if old_detail.ApplicationName = 'Need to fill !' then

                                             log.debug("ELSE: Old Application name need to fill");

                                             old_detail.ApplicationName := "";

                                        end if;

                                        appName := "BLANK";

                                        if text.upper(ApplicationName) <> old_detail.ApplicationName then

                                             log.debug("ELSE: Old ApplicationName & New ApplicationName is not same");

                                             list.append(updatedSoftdata, "ApplicationName - is changed from %old_detail.ApplicationName% to %appName%");

                                             log.debug("ELSE: Data appended - ApplicationName - is changed from %old_detail.ApplicationName% to %appName%");

                                                  if old_detail.ApplicationName = "" then

                                                       log.debug("ELSE: Old Application name NULL");

                                                       old_detail.ApplicationName := "BLANK";

                                                       list.append(updatedSoftdata, "ApplicationName - is changed from %old_detail.ApplicationName% to %appName%");

                                                       log.debug("ELSE: Data appended - ApplicationName - is changed from %old_detail.ApplicationName% to %appName%");

                                                 end if;

                                             end if;

                                             ApplicationName := "Need to fill !";

                                             list.append(missingValuesList,"ApplicationName");

                                             log.debug ("Windows_SoftData: ApplicationName not found");

                                        end if;

                     

                    det_name := "%host.hostname%";

                    det_key := "%det_name%/%host.key%";

                     

                    //ADD a new detail to the host node

                     

                    detail := model.Detail(

                    key  := det_key,

                    name := det_name,

                    type := det_type,

                    AppName := ApplicationName,

                    model.rel.Detail(ElementWithDetail := host, Detail := detail);

                    model.setRemovalGroup(detail, det_type);

                     

                    end if;

                    end for;

                    end body;

                    end pattern;

                    • 7. Re: Text file comparison
                      Bob Anderson

                      On the first discovery/inference of the Host, there is NO Detail node because it has not been created.  You do that later in the pattern

                       

                      You do a check for the Detail node, and even log a message about first time scan, however, you are using the Detail node as working variables for your pattern.  Since the first discovery does not have this detail yet, you are trying to reference an empty pointer, hence the RuleError message.

                       

                      Rather than using the Detail node as a working viariable, use a real local variable for the ApplicationName, then simply Create/Update the Detail node with the data from your local variables.

                       

                      NOTE: your code appears to be missing the closing parenthsis for the model.Detail call.

                      NOTE: your formatting makes it hard to determine what you are trying to do.  Perhaps better to attach the file rather than paste in the code.

                       

                      hth

                       

                      Bob

                      1 of 1 people found this helpful
                      • 8. Re: Text file comparison
                        Vishnu Parihar

                        Hi Bob,

                         

                        Thanks for the quick response.

                        I have attached the pattern.

                         

                        With the help of this pattern I am trying to do two things:

                                 1. Read the data from a text file and store it in Discovery.

                                 2. If the data is changed in the text file...trigger a mail.

                         

                        Now... point one is working file but point two is working partially as I mentioned earlier. If the data is already present in Discovery then we can compare and if this is first time Discovery of host then populating the error....

                         

                        Looking for your suggestion.

                         

                        Thanks

                        Vishnu Parihar

                        • 9. Re: Text file comparison
                          Andrew Waters

                          But this is still trying to compare the data with non-existing Detail nodes. If it is not present old_detail is not set and then you try to access it in

                          if text.upper(ApplicationName) <> old_detail.ApplicationName then

                          You need either always initialise old_detail and check if it is a node or check details before access old_deetail.

                          1 of 1 people found this helpful