1 2 3 Previous Next 34 Replies Latest reply on Jul 1, 2010 8:26 PM by Bill Robinson

    Deploying code based on delta with a different destination

    Amir Khamis

      Hello,

      I am trying to code using NSH/BLCLI a deployment in a UNIX/Linux only environment that will deploy a delta between 2 large directories.  In principle I have 1 master server with all the code located under it and I have many target servers I like to deploy code to.  The code is usually in gigabytes and I only need to update the destination based on updated, added or removed files on the master i.e. the delta.  The target directory architecture is always different from the source or master server directory architecture for example: my source code will be under /code/WebSphere and my target will be /app/WebSphere etc...  The delta deployment need to run as a different UNIX user.  I also need to be able to rollback so using a BLPackage is a must.  I also need to execute commands before and after the delta deployment and the commands need to run as different users and in some cases execute as root.

      Here is in high level how I thought to skin this cat:

      1.       create an audit job to audit the target server with the target directory against the master source directory, for example audit target_host:/app/WebSphere against master_host:/code/WebShpere

      2.       package delta from the output audit job as a BL package

      3.       open that same delta BL package and add pre and post commands to get executed using various UNIX users, in addition add/update configuration files

      4.       create a deploy job to run the BL package

      5.       execute deploy job

      6.       run audit job from step 1 to return a perfect sync

       

      My questions are:

      What do you think about the above plan to tackle this problem?

      Obviously I do have a problem with the different target folder, how to overcome this obstacle?

      Can all the above be accomplished from the command line using BLCLI?

       

      Thanks

        • 1. Re: Deploying code based on delta with a different destination
          Bill Robinson

          This should mostly work.  You should use a component template to allow for path parameterization in the audit/snapshot.  in the blpackage you need to make sure those parameters get setup and not hardcoded.  when you run your commands you should run them like "su - ".  make sure your deploy user is mapped to root on the target.

          • 2. Re: Deploying code based on delta with a different destination
            Amir Khamis

            Thanks Bill,

            I was trying to simplify the deployment to not include component templates and components but I guess I have no choice but using it and utilize a local parameter in the template and use it when packaging the component using a BLPackage.

            I have a very good understanding and in fact a working script that will help me accomplish what I need done using BLCLI commands.  The script I have also utilizes a local parameter which I will use to set the destination path. 

            Can you please share with me in high level an example BLCLI execution scenario which basically audit a destination server folder against a source server folder, create a delta package?  From there on I will be all set since I have a basic script doing a simple deployment of some code on a source server to a destination server, so I just need to improve it to only deploy the delta and wondering what will be the best way to do that and if you can help me in the organization of the commands.

             

            Thanks

             

             

            • 3. Re: Deploying code based on delta with a different destination
              Bill Robinson

              do you need to automate the creation of the template also?

               

              one thing to check, can you do this manually - create a template and create a part that uses a property.  discover the template.  then do the audit and do the sync to master - does that use the property or does it hard code it?

              1 of 1 people found this helpful
              • 4. Re: Deploying code based on delta with a different destination
                Amir Khamis

                I created a component template and added one part to it: ??APP_PATH??/source.  APP_PATH is a local property and has a default value of /software.  my target server has /software/target folder that needs to be synced with /software/source from the master server.

                I ran a discovery on both servers, obviously still stuck with the same issue.   the target server does not have /software/source therefor it is showing with a red x on /software/source.  I tried to audit and sync to master with no luck.  if component templates my only hope to accomplish this perhaps I am not setting up the example you asked me to setup properly.  can you please elaborate some more on the manual test we are trying to perform using the console?

                thanks

                • 5. Re: Deploying code based on delta with a different destination
                  Bill Robinson

                  take out the default value of your local property APP_PATH.

                   

                  in the template/local properties area you need to create multipe instances.  create 1 instance for each possible path. 

                   

                  base your discovery on existance onf ??APP_PATH??, run discovery and then retry your audit.

                  1 of 1 people found this helpful
                  • 6. Re: Deploying code based on delta with a different destination
                    Amir Khamis

                    I created a new component template, I checked all under Allowed Operations.  I created a local property called APP_PATH of type Simple String.  Under instances I created 2 instances:

                    Source and made the value of APP_PATH = /software/source

                    Target and made the value of APP_PATH = /software/target

                    I added one part by choosing the source directory on the source server i.e. /software/source.

                    I edit the part name and changed it to ??APP_PATH?? Local parameter.

                    I ran the discovery on my source and destination server, it came back with:

                    1.       Source instance on source server (succeeded)

                    2.       Source instance on target server (failed with a red-x on the source directory)

                    3.       Target instance on source server (failed with a red-x on the source directory)

                    4.       Target instance on target server (succeeded)

                     

                    I tried to run an audit by choosing component 1 as the master and component 4 as the target with no luck.  I am sure I am doing something wrong.

                    • 7. Re: Deploying code based on delta with a different destination
                      Amir Khamis

                      Alright, I had a better luck but still having issues with sync with master.

                      Later after we resolve the basic audit process I like to know if we are able to generate components 1 & 4 from my previous step and eliminate the creation of components 2 and 3 which obviously failed?

                      So here is what I did different:

                      From the component templates workspace I selected my source server source directory component or component 1 from the previous thread I audited against the target server target directory or component 4 from the previous thread.  The audit ran perfectly and showed me master server with source directory on the left and target server with target directory on the right! And highlighted all the differences (added/removed/changed files) I tried to sync with master, a BL package with a yellow warning icon got created, the bl package used ??APP_PATH??  Local property on every asset it is changing, deleting or adding which is exactly what I need.  The package local property had the value of /software/target.  When I run the deploy job it fails in the commit stage, here is the error log:

                      Info       Jun 9, 2010 1:48:53 PM   Started running the deploy step job 'sync target with master->sync target with master' on application server 'blogic'(2,000,000) against target server 'blreports'                      

                      Info        Jun 9, 2010 1:49:00 PM   Package sync-target-with-master-2000684.1-2000195.1 started                 

                      Info        Jun 9, 2010 1:49:05 PM   Package sync-target-with-master-2000684.1-2000195.1 initialized, entering wait queue for processing                         

                      Info        Jun 9, 2010 1:49:05 PM   Package sync-target-with-master-2000684.1-2000195.1 processing instructions                 

                      Info        Jun 9, 2010 1:49:08 PM   Package sync-target-with-master-2000684.1-2000195.1 auto-rollback on failure started                 

                      Info        Jun 9, 2010 1:49:08 PM   Package sync-target-with-master-2000684.1-2000195.1 completed. exitCode = -4001 (Apply failed; auto-rollback successful)                               

                      Error      Jun 9, 2010 1:49:11 PM   APPLY failed for server blreports. Exit code = -4001                         

                      Info        Jun 9, 2010 1:49:11 PM   The job 'sync target with master->sync target with master' has failed on server blreports                               

                      Info        Jun 9, 2010 1:55:42 PM   Deployment job not in single-job mode. Reboot and single-user mode is not allowed

                                                     

                      Error      Jun 9, 2010 1:55:51 PM   Apply failed: id = 2

                                      2              /software/target/example_file2

                      Error      Jun 9, 2010 1:55:51 PM   Failed to change the owner for /software/target/example_file2 uid 0 gid 0: No such file or directory

                                      2              /software/target/example_file2

                      Info        Jun 9, 2010 1:55:51 PM   Processing asset FILE

                                      2              /software/target/example_file2

                      Info        Jun 9, 2010 1:55:51 PM   Processing asset FILE

                                      1              /software/target/example_file

                      Info        Jun 9, 2010 1:55:51 PM   Processing asset BLPACKAGE

                                      0             

                      Info        Jun 9, 2010 1:55:51 PM   Starting apply

                                                     

                      Info        Jun 9, 2010 1:55:51 PM   Bldeploy sync-target-with-master-2000684.1-2000195.1 started using file /var/tmp/stage/sync-target-with-master-2000684.1-2000195.1/bldeploy.xml with rollback directory /usr/nsh/Transactions/sync-target-with-master-2000684.1-2000195.1

                                                     

                      Warning               Jun 9, 2010 1:55:52 PM   Deploy failed. Cleaning up staging area.

                                                     

                      Warning               Jun 9, 2010 1:55:52 PM   Deploy failed. Cleaning up staging area.

                                                     

                      Info        Jun 9, 2010 1:55:52 PM   Undo succeeded

                                                     

                      Info        Jun 9, 2010 1:55:52 PM   Processing asset FILE

                                      1              /software/target/example_file

                      Warning               Jun 9, 2010 1:55:52 PM   FileDelete: FileName: /software/target/example_file2 does not exist

                                      2              /software/target/example_file2

                      Info        Jun 9, 2010 1:55:52 PM   Processing asset FILE

                                      2              /software/target/example_file2

                      Info        Jun 9, 2010 1:55:52 PM   Processing asset BLPACKAGE

                                      0             

                      Info        Jun 9, 2010 1:55:52 PM   Starting undo

                                                     

                      • 8. Re: Deploying code based on delta with a different destination
                        Daniel Goetzman

                        Not sure if I am following this completely, but Sync to Master needs "File Contents" checked in the template if the delta contents are "new" files to the target server. If you don't have the "File Contents" checked, I typically see these type of errors (trying to set owner/permission on files that are not yet on the target and in fact included in the delta deploy blpackage.

                        • 9. Re: Deploying code based on delta with a different destination
                          Amir Khamis

                          Thanks guys, after recreating the template, I made sure this time to include any option applicable for UNIX

                          like content etc.  now things seem to work better and I was able to do my audit and sync to master.

                          The fun starts now since I have to do the same from the command line!  got a quick question on blcli:

                          so overtime I need to start executing blcli commands I need to authenticate and that will cause a problem, in older versions we were able to use a file and that's it.  can you please explain this process to me and help me figure out how we can execute blcli commands without specifying the password in my script? here is how I am connecting to execute my code, can you help me understand exactly what I need and don't need, my goal is to not include the password in the script at all:

                          I execute this once per system:
                          blcred authprofile -add –profile profile_name -host bladelogicp:9840 -type SRP

                          and then everytime I need to run any code this is what I setup:
                          blcred cred -acquire -profile profile_name -username user_name -password ******
                          blcli_setoption serviceProfileName profile_name
                          export BL_AUTH_PROFILE_NAME=profile_name
                          blcli_setoption authType SRP
                          blcli_setoption appserver_host bladelogicp
                          blcli_setoption appserver_port 9829
                          blcli_setoption userName user_name
                          blcli_setoption password *******
                          blcli_setoption roleName BLAdmins
                          blcli_connect

                          thank you

                          • 10. Re: Deploying code based on delta with a different destination
                            Bill Robinson

                            w/ the blcli you should use the 'performance commands' - look in the blcli help for info on that.  but basically like:

                             

                            in nsh:

                            blcred cred -acquire -profile

                            blcli_connect

                             

                            then run all your blcli w/ blcli_execute, not blcli.

                            • 11. Re: Deploying code based on delta with a different destination
                              Amir Khamis

                              blcred cred -acquire –profile <profile name>  still prompts you.  According to the admin doc on page 213 I looked at:

                              Obtaining a session credential by referencing a keytab file

                              If you are running Network Shell or the BLCLI in batch mode and you need to obtain a session credential non-interactively, you can direct blcred to retrieve an SRP user name and password from an SRP keytab file, using a command like the following:

                              blcred cred -acquire -profile srpProfile -i /home/user/user_info.dat

                              I generate the file as described on page 214:

                              Generating a user information file

                              It seemed to work, but I was not able to find where did the user_info.dat file got created so I can reference it with the above command and hopefully my problem is solved.

                              I searched the system for user_info.dat, I was only able to find an older file under /root/.bladelogic/.user

                               

                              Any ideas on keytab files?

                               

                               

                              • 12. Re: Deploying code based on delta with a different destination
                                Bill Robinson

                                you use the bl_gen_user_info command in INSTALL/br or INSTALL/bin.  that will prompt you for the name of the file, you can name it whatever you want.  then use the:

                                blcred cred -acquire -profile <profile> -i <user_info.dat>

                                 

                                 

                                you can also do:

                                 

                                blcred cred -acquire -profile <profile> -username <user> -passwd <passwd>

                                 

                                but that's in the clear.

                                • 13. Re: Deploying code based on delta with a different destination
                                  Amir Khamis

                                  I kept going for now, hopefully you will be able to help me out with the keytab file, but I have a big issue right now and has a higher priority for me to get this project moving!

                                  I am unable to create a component using BLCLI based on a "Property Set Instance" which I disparately need!

                                  the option is available using the GUI, if you RMB on a template that has a custom local property with instances it will provide a drop down list for you to choose from.  I am uising the BLCLI online help constantly I am unable to find any indication on how to do that under Component/createComponent.  here is the command I am executing:

                                  blcli_execute Server getServerIdByName $SRC_HOST
                                  blcli_storeenv SRC_ID

                                  blcli_execute Component createComponent "$TEMPLATE_NAME (${SRC_HOST})" "$TEMPLATE_KEY" "$SRC_ID"
                                  blcli_storeenv SRC_COMP_KEY

                                  I need to be able to execute Component createComponent that will take a "Property Set Instace"?  can you please help me out with that?

                                   

                                  thanks

                                  • 14. Re: Deploying code based on delta with a different destination
                                    Bill Robinson

                                    why don't you run a discovery job?  that will create the component.

                                    1 2 3 Previous Next