1 2 Previous Next 21 Replies Latest reply on Dec 6, 2012 9:16 AM by ddd lll

    Slow File Copy via Bladelogic

      Hi,

      I have Bladelogic 7.6 in one of our Customer environment. We are using Bladelogic File deploy job to transfer 10GB file from one server in US DC to one Server in Singapore. Now Without using Bladelogic we have observed it takes 6-7 Hours to copy the same, But in case we use NSH Script job or File deploy Job it takes around 26-30 Hours to do the same.

      Not only this we tried copying a 100MB file from a server in DC to another server in same DC. Without Bladelogic it takes about 30 Seconds. But using bladelogic it took 6 minutes to copy the same. What can be the possible reason for having this kind of behaviour. We can go for a repository server in Singapore DC but using Bladelogic copy within Same DC is taking so much time.

        • 1. Re: Slow File Copy via Bladelogic

          Hi,

           

          You now:

          any copy operation involving the appserver is going to copy the files through  the appserver ...

           

          See e.g.

          https://communities.bmc.com/communities/message/201872#201872

           

           

           

          Cheers

          dinoel

          • 2. Re: Slow File Copy via Bladelogic

            But in case i need to deploy a file in singapore from one server to other and My Bladelogic Server is in US then would the file copy go via BL Appserver in US. If so then this may be problematic for us. Any method I can use to avoid the same problem.

            I don't want to use repeater as till 8.2 it is still a big failure.

            • 3. Re: Slow File Copy via Bladelogic

              e.g. you can use NSH script job with nexec for this.

               

              Something like:

               

               

              CMD=( cmd /c net use "${SHARE_BASE}" "/user:${SHARE_USER}" "${SHARE_PASSWD}" "&&" xcopy /Y /E /S /D "${SHARE_DIR}" "${STAGE_DIR}\\" )

               

              nexec "$TARGET" "$CMD[@]"

               

              ...of course you have to define all variables ..

               

              The idea is to use a windows xcopy via nexec.

              • 4. Re: Slow File Copy via Bladelogic
                Bill Robinson

                is the bsa appserver in the US ?

                 

                also, there is some overhead in the nsh to rscd connection as it's encrypted, it should be similar to using scp to copy the files.

                 

                when you do the copy without bladelogic what are you using to copy?

                • 5. Re: Slow File Copy via Bladelogic
                  Soundappan Shanmugam

                  In many cases Bill has clearly stated that BL cannot be a replacement of

                  native Copy and share.. and that too copying 20 GB file is too heavy..

                   

                  bill,

                  your comments on it

                  • 6. Re: Slow File Copy via Bladelogic

                    Hi,

                    Our Application Server is in US. I am not pointing the limitation of the tool but asking for a suggestion. Clearly a simple task of file copy has to be achived via BL. It is not acceptable that despite having Agents on the end point servers I have to use Credentials to do a file copy.  Even a open source tool gives this simple facility. Also there is no question of replacing the native file copy but it is my understanding with privledge user mapping a copy command from Windows Server can be launched and it can Copy the file on second server. For example If I log onto a Windows Server A using Administrator username and using cmd prompt try to use xcopy i can copy files accross the servers without providing password (provided i have same password on second windows Server as that of source) .Anyways I have already raised a case with BMC for same. Hoping to hear some logical solution for same,

                    • 7. Re: Slow File Copy via Bladelogic
                      Bill Robinson

                      bsa doesn’t use windows creds (it doesn’t have to) to initiate a connection to the target agent so I don’t see how that factors into the discussion.  What open source tool allows you to copy files from one server to another without credentials ?  your xcopy example doesn’t really count because you are providing credentials via your login.  have you tried to use an Automation Principal user instead of the default UPM when connecting w/ the agent, and initiating the copy/xcopy under those credentials (assuming they are the same across two boxes) ?  what about using the ‘network url’ type of software deploy?  what kind of files are you copying here ?

                      • 8. Re: Slow File Copy via Bladelogic

                        Piyush, another approach is to install NSH Utilities on your server (in US DC) that hosts the file which you want to copy.

                         

                        Then you can create a BLPackage Deploy Job and run it against your US DC server. The external command in your blpackage would be:

                         

                        cp /path/to/local/file //remote_singapure_server/path/to/folder

                         

                        When you run this job, since you have NSH Utilities installed on the US server, it will run the NSH cp command and copy the file directly from US server to Singapore server. This way you remove the overhead of copying the file through appserver. The command can be parameterized obviously, this was just a blue print of how to make this work.

                        • 9. Re: Slow File Copy via Bladelogic

                          Lazar,

                           

                          we are speaking about Windows here ...

                          That's why we can use only windows like commands but not unix/nsh like commands isn't it ?

                           

                          The next question:

                          What is the difference between NSH cp (your proposal) and Windows copy/ xcopy (my proposal) ?

                           

                          Cheers

                          dinoel

                          • 10. Re: Slow File Copy via Bladelogic

                            dinoel, If you install NSH Utilities on the system that hosts the files, then you can run nsh cp command, even if it's windows.

                             

                            about the differences, I suppose both ways have pros and cons.

                            Your way pros: no extra installation needed

                            My way pros: no need to maintain shares, and expose usernames/passwords

                             

                            Up to the user which way to use, just giving another approach since the topic was not closed yet : )

                            • 11. Re: Slow File Copy via Bladelogic

                              Lazar,

                               

                              OKay,  it's a matter of taste to use nsh cp or windows copy/ xcopy !

                              But i don't understand why "no need to maintain shares, and expose usernames/passwords" ?

                               

                              I think you have to use Windows commands/ script in the external command of your  BLPackage in this case, because you are working with windows systems here.

                               

                              That means whatever you use for file copying: nsh cp or windows copy/ xcopy or something else --- if it is necessary to mount a share on the remote server for copying of files you HAVE TO DO IT.

                               

                              Cheers

                              dinoel

                              • 12. Re: Slow File Copy via Bladelogic

                                dinoel, Yes it's a matter of taste what way to use, I am not forcing one way over another.

                                 

                                >>I think you have to use Windows commands/ script in the external command of your  BLPackage in this case, because you are working with windows systems here.

                                 

                                Not if you have NSH Utilities installed on the system from which you will run nsh cp command (US DC).

                                 

                                >>That means whatever you use for file copying: nsh cp or windows copy/ xcopy or something else --- if it is necessary to mount a share on the remote server for copying of files you HAVE TO DO IT.

                                 

                                Not if you have NSH Utilities installed on the system from which you will run nsh cp command against the target that has Agent installed. One thing I forgot to mention in the original proposal is that you may need to adjust the ACLs on the target so that US DC system has access to Singapore system.

                                 

                                 

                                Perhaps did not understand the approach I was proposing. Please test it in your lab and you'll see.

                                • 13. Re: Slow File Copy via Bladelogic

                                  Lazar,

                                   

                                  you mean it is not enough to have rscd-agents on the both servers. i have to install the NSH Utilities additionally.

                                  Where can i get the NSH Utilities ?

                                   

                                  Cheers

                                  dinoel

                                  • 14. Re: Slow File Copy via Bladelogic

                                    Yes, with my approach you need to install NSH Utilities on the system that contains your payload (US DC in this case), so that from that system you can execute nsh cp /local/file //remote_server/path/to/folder

                                     

                                    NSH Utilities are on EPD site among the rest of installers (appserver,agent, nsh, content, etc...)

                                    1 2 Previous Next