10 Replies Latest reply on Nov 24, 2011 9:09 AM by Jim Campbell

    Extended Objects and NSH Proxy Contexts

    Jim Campbell

      We had a problem with running out of NSH proxies when running NSH scripts that connected to large numbers of servers.  We've been able to fix this by adding an explicit diconnect $target inside whatever loop we're running to connect to servers.  This works fine for NSH script jobs but now we're seeing what I presume is the same problem with extended objects used in compliance.  When run on a small number of servers its fine, but with large numbers of servers we get failed compliance attempts and messages that look like:


      Error          Oct 13, 2011 9:05:33 AM          com.bladelogic.app.collector.AssetCollectionException: SSL_connect2


      Error in Initializing RBAC User and Role (SSO Proxy)

      Network Shell can be used for local access





      SSO Error: Error reading server greeting.

      nsh:1: operation canceled: //appserver01/script.nsh

      (component=Compliance_Job_Name_Here (target_server_here), selector=Extended Object:Script)


      The extended object in question is set up as a central execution script that is called with:


      nsh -c //appserver01/script.nsh ??TARGET.NAME??


      The script pushes a small vbscript to servers and retrieves a result after nexec'ing it on the target server.

        • 1. Extended Objects and NSH Proxy Contexts
          R V

          Besides that I couldn't solve your problem or even help you at the moment - are you really running out of "NSH proxies" (maybe it's just a matter of naming something).  To my knowledge NSH proxy is only used while connecting from (e.g.) some workstation running an NSH script. Or do you have seperate JobServers running over the NSH proxy? And - how do you "realize" that you do not have any more proxy connection?


          (I'm really just interested in the configuration behind your problem to get the picture).

          • 2. Extended Objects and NSH Proxy Contexts
            Jim Campbell

            Two appservers, each of which runs jobs,  client connections, and nsh proxy.  Each appserver is set up such that the nsh client uses the nsh proxy on the appserver (i.e. app01's nsh client is set up to use the app01 nsh proxy, app02's nsh client is set up to use app02's nsh proxy).  From what we were told this is requisite to allow for file deploy jobs and nsh script jobs to properly use automation principals (i.e. that the nsh client on the appserver has to be set up to use an NSH proxy).


            The nsh proxy contexts in Infrastructure Management  -> NSHProxyConnectionService -> Active NSH proxies goes to the value we've set for MaxNshProxyContexts (which is 200 for now).


            We had this problem (or at least what appears to be the same problem) when running NSH scripts on large numbers of servers in a loop that does pretty much the same thing (push vbscript to server, nexec vbscript).  I don't have the thread URL but there was a thread on the forums that explained the problem and that an explicit disconnect was required after ending communication with a server.  This fixed the problem for NSH scripts but it appears to be a problem with extended objects used in compliance.

            • 3. Extended Objects and NSH Proxy Contexts
              Bill Robinson

              Jim - you have the appserver configured to use a nsh proxy right? (eg, to use Automation Principals)


              can you post the EO you are using?


              also, in blasadmin, what is the maxnshproxythreads set to ?

              • 4. Extended Objects and NSH Proxy Contexts
                Jim Campbell

                This is the sort of general shell we use for this both with NSH scripts and extended objects:




                if [ ! -d "//$TARGET/c/temp/Bladelogic" ]


                mkdir -p //$TARGET/c/temp/Bladelogic



                cat > //$TARGET/c/temp/Bladelogic/script_file_name_here.vbs << EOF

                <<VB STUFF HERE>>



                resultstring=`nexec $TARGET cmd /c cscript //nologo c:\\\\temp\\\\bladelogic\\\\script_file_name_here.vbs`

                echo $resultstring


                disconnect $TARGET

                • 5. Re: Extended Objects and NSH Proxy Contexts
                  Stefan Kruth

                  Usually, you're right, the proxy is meant to being used interactively. The app server itself doesn't require a proxy to run NSH script jobs of type central execution. Obviously, this app server nsh is configured to use a proxy (profile and secure file) and the fact that it is invoked with nsh -c kicks off an nsh that utilises this proxy. Every command uses a proxy thread and if you fire a massive amount of them, you will run out of threads, even if the prune time is set to a very small value. This is AFAIR the timeout, when a thread is released from a connection and made available to the thread pool again.

                  Answer to the question could be: Don't use the proxy Script of type central execution without nsh -c should do it, or did I miss someting here (of course unconfigure the utilisation of proxy on the app server's NSH and you may want to think about the utilisation of the spawner, too


                  Cheers, Stefan

                  • 6. Extended Objects and NSH Proxy Contexts
                    Jim Campbell

                    Is there a way to force it to not utilize the proxy while retaining the use of the proxy when we need it for automation principal usage?


                    In the case of this extended object we don't need to use automation principals but there are many instances in which they are required for us.  My understanding was that when using the nsh client it was a binary use/don't use based on whats in the secure file but if there's a way to dynamically turn it on and off that would be ideal.

                    • 7. Extended Objects and NSH Proxy Contexts
                      Bill Robinson

                      you have to use the proxy to use the AP.


                      what is the maxproxytheards set to ?

                      • 8. Re: Extended Objects and NSH Proxy Contexts
                        Stefan Kruth

                        Sorry, wasn't aware that you're using automation principals. Unfortunately, there is no switch, sorry. You might want to check (in addition to Bill's recommendations) the nsh idle connection prune time. Lowering might also mitigate the impact.

                        • 9. Extended Objects and NSH Proxy Contexts

                          what are the values for :-


                          AppServer MaxNshProxyContexts

                          AppServer MaxNshProxyThreads

                          • 10. Extended Objects and NSH Proxy Contexts
                            Jim Campbell

                            Sorry about not responding on this thread and I just realized I had not.  This ended up being an error in communication.  It was a problem with a limited number of NSH proxy connections but the problem was fixed by limiting the number of concurrent targets for the compliance job (my understanding from talking to the guy running the jobs at the time of posting this was that it was not fixing the problem and that he was having to run the jobs against smaller subsets of targets).