12 Replies Latest reply on Oct 29, 2013 4:12 PM by Lavanya Bobba

    RHEL patching deploy options for faster patching

    Lavanya Bobba

      Please suggest deploy job options for RHEL update level patching for faster staging and patching?

       

      I am sure, we have to selecte stage and simulate in deploy options and wanted to make sure other options also.

        • 1. Re: RHEL patching deploy options for faster patching
          Bill Robinson

          what part is slow?

           

          your targets are on the same network as your appserver?

          • 2. Re: RHEL patching deploy options for faster patching
            Lavanya Bobba

            They are in same network, staging part took 3 + hours

            • 3. Re: RHEL patching deploy options for faster patching
              Bill Robinson

              there's a lot of reasons that could be the case:

               

              - how many targets in total?

              - how many targets in parallel?

              - how many jobs were generated ?

              - how many patches per target?

              - network speed between the appserver and targets?

              • 4. Re: RHEL patching deploy options for faster patching
                Lavanya Bobba

                One target only in remediation job.

                 

                Patches per target are -531rpms

                 

                here are traceroute and ping stats from target to app server

                $ traceroute blapphp01

                traceroute to blapphp01 (10.219.24.218), 30 hops max, 60 byte packets

                1  10.217.32.2 (10.217.32.2)  0.389 ms  0.372 ms  0.369 ms

                2  10.129.88.65 (10.129.88.65)  0.322 ms  0.308 ms  0.263 ms

                3  dcc9dis1-te-5-4.thehartford.com (10.129.88.102)  0.489 ms  0.629 ms  0.758 ms

                4  blapphp01.thehartford.com (10.219.24.218)  0.295 ms  0.271 ms  0.221 ms

                $ arp -a blapphp01

                arp: in 1 entries no match found.

                $ ping blapphp01

                PING blapphp01.thehartford.com (10.219.24.218) 56(84) bytes of data.

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=1 ttl=61 time=0.456 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=2 ttl=61 time=0.428 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=3 ttl=61 time=0.396 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=4 ttl=61 time=0.719 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=5 ttl=61 time=0.494 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=6 ttl=61 time=1.67 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=7 ttl=61 time=0.377 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=8 ttl=61 time=0.388 ms

                64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=9 ttl=61 time=0.564 ms

                ^C

                ping statistics ---

                9 packets transmitted, 9 received, 0% packet loss, time 8302ms


                • 5. Re: RHEL patching deploy options for faster patching
                  Lavanya Bobba

                  $ ping blapphp01

                  PING blapphp01.thehartford.com (10.219.24.218) 56(84) bytes of data.

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=1 ttl=61 time=0.456 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=2 ttl=61 time=0.428 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=3 ttl=61 time=0.396 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=4 ttl=61 time=0.719 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=5 ttl=61 time=0.494 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=6 ttl=61 time=1.67 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=7 ttl=61 time=0.377 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=8 ttl=61 time=0.388 ms

                  64 bytes from blapphp01.thehartford.com (10.219.24.218): icmp_seq=9 ttl=61 time=0.564 ms

                  ^C

                  --- blapphp01.thehartford.com ping statistics ---

                  9 packets transmitted, 9 received, 0% packet loss, time 8302ms

                  rtt min/avg/max/mdev = 0.377/0.610/1.671/0.389 ms

                  • 6. Re: RHEL patching deploy options for faster patching
                    Bill Robinson

                    it takes 3 hours to copy ~500 patches from one server to another on this network?

                     

                    your patch helper is on the same network as the appserver and the target ?

                    • 7. Re: RHEL patching deploy options for faster patching
                      Lavanya Bobba

                      # more bldeploy-a36ce03e797834afbff7f6fa426e42de.log

                      08/01/13 09:38:08.432 INFO     bldeploy - Bldeploy started for package: a36ce03e797834afbff7f6fa426e42de

                      08/01/13 09:38:08.433 DEBUG    bldeploy - Bldeploy Arguments:

                      08/01/13 09:38:08.433 DEBUG    bldeploy - /tech/bladelogic/NSH/sbin/bldeploy

                      08/01/13 09:38:08.433 DEBUG    bldeploy - a36ce03e797834afbff7f6fa426e42de

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -N="/var/tmp/stage/0ddb8c7ba2c73eecb0b9ef8ab9035e5e"

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -P=RHEL6TST

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -Q=params.txt

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -PKG="Remediation%20job%206.4-20959871%20@%202013-08-01%2009-35-14-689-0400-2229361.1"

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -DryRun

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -Xr

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -V1

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -Xp

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -js0

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -jr0

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -jc0

                      08/01/13 09:38:08.433 DEBUG    bldeploy - -StartNew

                      08/01/13 09:38:08.433 DEBUG    bldeploy - /tech/bladelogic/NSH/sbin/bldeploy  - 8.2.01.273

                      08/01/13 09:38:08.433 INFO     bldeploy - Reboot is not allowed

                      08/01/13 09:38:08.433 DEBUG    bldeploy - Cleanup staging area on failure

                      08/01/13 09:38:13.441 DEBUG    bldeploy - Utilizing configuration file: /var/tmp/stage/0ddb8c7ba2c73eecb0b9ef8ab9035e5e/a36ce03e797834afbff7f6fa426e42de.cfg

                      08/01/13 09:38:13.441 DEBUG    bldeploy - Fresh start of deployment, deleting any pre-existing configuration file

                      08/01/13 09:38:13.442 DEBUG    bldeploy - Could not open file "/tech/bladelogic/NSH/params.txt" for reading

                      08/01/13 09:38:13.442 DEBUG    bldeploy -

                      DEPLOYNAME = a36ce03e797834afbff7f6fa426e42de:

                      DEPLOYPATH = /var/tmp/stage/0ddb8c7ba2c73eecb0b9ef8ab9035e5e:

                      FQHOST = rhel6tst.thehartford.com:

                      HOST = rhel6tst:

                      HOSTIP = 10.217.35.152:

                      NAMESPACE = RHEL6TST:

                      ROLLBACKPATH = /tech/bladelogic/NSH/Transactions/a36ce03e797834afbff7f6fa426e42de:

                      RSCDIR = /tech/bladelogic/NSH:

                      XMLFILEPATH = /var/tmp/stage/0ddb8c7ba2c73eecb0b9ef8ab9035e5e/bldeploy.xml:

                      08/01/13 09:38:13.442 DEBUG    bldeploy - Full command line /tech/bladelogic/NSH/sbin/bldeploy a36ce03e797834afbff7f6fa426e42de -N="/var/tmp/stage/0ddb8c7ba2c73eecb0b9e

                      f8ab9035e5e" -P=RHEL6TST -Q=params.txt -PKG="Remediation%20job%206.4-20959871%20@%202013-08-01%2009-35-14-689-0400-2229361.1" -DryRun -Xr -V1 -Xp -js0 -jr0 -jc0

                      08/01/13 09:38:14.444 DEBUG    bldeploy - Validating existence of wait lock /tech/bladelogic/NSH/Transactions/locks/reboot.lck

                      08/01/13 09:38:14.444 DEBUG    bldeploy - Deployment running in single-job mode

                      08/01/13 09:38:16.450 DEBUG    bldeploy - Acquired appropriate lock: /tech/bladelogic/NSH/Transactions/locks/writer.lck

                      08/01/13 09:38:17.453 DEBUG    bldeploy - Creating directory /tech/bladelogic/NSH/Transactions/a36ce03e797834afbff7f6fa426e42de :access world

                      08/01/13 09:38:17.453 INFO     bldeploy - Bldeploy "Remediation job 6.4-20959871 @ 2013-08-01 09-35-14-689-0400" UUID(a36ce03e797834afbff7f6fa426e42de) started using fi

                      le /var/tmp/stage/0ddb8c7ba2c73eecb0b9ef8ab9035e5e/bldeploy.xml with rollback directory /tech/bladelogic/NSH/Transactions/a36ce03e797834afbff7f6fa426e42de

                      • 8. Re: RHEL patching deploy options for faster patching
                        Lavanya Bobba

                        Yes , helper server is on same network as on app server and target

                        • 9. Re: RHEL patching deploy options for faster patching
                          Bill Robinson

                          i'm not sure what the point of the bldeploy log was.  so you have a single target in you job w/ ~500 patches, and that one job takes 3 hours to run - including the stage and commit phases ?

                           

                          if that's the case i would start to investigate what particular phases take a long time - the copy, the install ?  a complete bldeploy log may help w/ that, but the one above seems to be incomplete.

                          • 10. Re: RHEL patching deploy options for faster patching
                            Lavanya Bobba


                            We ran the deploy job to see each phase timings, will update once we get the details on each phase.

                             

                            Single target , with 520 rpms missing

                            • 11. Re: RHEL patching deploy options for faster patching
                              Jim Wilson

                              Hi Lavanya Bobba

                               

                              Did this get resolved?

                              If so, please can you update the discussion thread with details so that is can be marked as answered.

                               

                              Thanks & Regards,

                              Jim (Community Manager/Facilitator)

                              • 12. Re: RHEL patching deploy options for faster patching
                                Lavanya Bobba

                                Issue seems to be with the offline catalog which was created , patch deployment was not successful with that catalog.

                                 

                                We have created new catalog and it worked fine