7 Replies Latest reply on Jul 10, 2015 7:38 AM by Yanick Girouard

    Windows Patch Analysis Job taking long time to complete

    Rajkumar Rajpurohit

      Hi,

      We have recently upgraded our BSA environment from v8.2 SP4 to v8.5SP1 Patch5. We have upgraded the rscd agent to the latest version 8.5.1.304 on all the servers.

      Now, I am facing an issue with Windows Patch Analysis Job. The PAJ is taking 20-23minutes to complete on just 105 servers. Only 1 or 2 times it took below 2 minutes. I have run this for Windows 2012 R2 servers (win2k12 Patch catalog).

       

      Our production environment have 3 application servers (server profile type ALL for all 3 app servers) install on Windows 2008 R2 servers. DB is SQL Server 2008 R2.

       

      Could you please let me know how we can improving the performance of PAJ. Or what all things i can check in my environment to find the root cause and fix this slowness issue.

       

      Thanks

      Rajkumar.

        • 1. Re: Windows Patch Analysis Job taking long time to complete
          Bill Robinson

          Can you clarify if it’s the analysis phase or deploy phase that is taking a long time ?

           

          Where are these systems in relation to the appserver?  across a wan ? same network ?

          • 2. Re: Windows Patch Analysis Job taking long time to complete
            Rajkumar Rajpurohit

            Hi Bill,

            It is the analysis phase that is taking a long time.


            The systems are split across different datacenters. We are using advanced repeater for each of the DCs. But i think repeater will not be used for PAJ.


            One thing i noticed is that when the PAJ runs from one of the appservers (03 in our case) , it is completing within 2mins as expected. And when the same PAJ runs from either of the other two app servers (01 or 02), it is taking longer time. Below are the screen shots of a PAJ taking longer time:

            SS1.pngSS2.png

             

            Can you please let me know what configurations i can check in our environment which might cause this kind of delays.

             

            Thanks

            Rajkumar.

            • 3. Re: Windows Patch Analysis Job taking long time to complete
              Yanick Girouard

              I suggest you monitor the processes and CPU on one of the servers being affected, while it's running, and check if by any chance, BLPatchCheck2.exe is running at high CPU usage for too long.

               

              Does it even get pass the copying of the xml based on the logs or it doesn't show any logs at all for the targets that are running?

              • 4. Re: Windows Patch Analysis Job taking long time to complete
                Rajkumar Rajpurohit

                Hi Girouard,

                Thanks for the reply. Sure, i will check the CPU usage when i run the analysis and let you know what i find.

                 

                Yes, the copying of the xml does get completed on all the targets, but only after a lot of time (approx. 20-22mins). In the screen shots you can see targets that do not show any thing in the logs, those are the servers which are taking longer time.

                 

                But it is little strange that the analysis gets completed in expected time on all targets when the PAJ runs from one of the app servers(03). And it does not complete in expected time when it is run from any of the other 2 app servers.

                 

                -Rajkumar.

                • 5. Re: Windows Patch Analysis Job taking long time to complete
                  Bill Robinson

                  what do you mean 'runs from either of the other two appservers' - the job is picked up by those appservers or the WIT from the job are running on those appservers ?  in the 2m in case, are all the WIT for the job running on the 1st appserver ?

                   

                  are all 3 of these appservers in the same location ?  is the network path from the appserver to the target the same for all 3 appservers ?

                  • 6. Re: Windows Patch Analysis Job taking long time to complete
                    Rajkumar Rajpurohit

                    Yes, i meant the job being picked up by those appservers.

                     

                    All 3 appservers are in the same location and the network path from these app servers to the target servers is same as well.

                    And for the 2min case, the WIT for the job were running on all the 3 app servers.

                    • 7. Re: Windows Patch Analysis Job taking long time to complete
                      Yanick Girouard

                      I suggest you monitor exactly what is happening in terms of processes (use Task Manager with the added Command-Line column -- add it with the View menu) when each line is shown in the PAJ logs. Sometimes a line is printed to indicate it started a certain step, and it's completing it without printing something else before starting another task which it doesn't echo anything for in the logs... so it can be confusing.

                       

                      I.e. it says it starts copying the xml, and then that it's starts the analysis, but in between it had to decompress and decrypt the xml. The issue will seem like it' sthe copy, but the file probably got there within seconds, and it's the decryption that's taking forever.

                       

                      I know that shavlik's engine is relying on msxml to decrypt the HUGE xml files in order for hfcli to be able to read it. That takes a lot of CPU and on some servers, has been known to cause problems (and take forever to run).

                       

                      Did you by any chance, change the PRIORITY setting in the Windows options of the Patch Global Configuration panel?