5 Replies Latest reply on Feb 18, 2016 11:05 PM by L W

    How can I pass env variables when running nexec -e

    L W



      I have created many BL batch jobs, each residing at their own specified job location, which all executes multiple underlying batch jobs and jobs, which runs the same nsh script.  The nexec script is a generic wrapper that find out the job's name, copies a general script from the depot location that has similar path tree as the job name, to the client and does an nexec -e with the script.


      The issue I have is since the script it executes is a general script that performs a specific function pertaining to the job's name, it cannot be scripted specifically for that environment.  In our unix env, we usually set environmental variables, so any subshell or script that is called can recall those variables.  But with nexec, none of my variables can pass through.


      I am currently passing parameters to the nsh script, which passes all of them to the script via


      nexec -e SCRIPT $@


      But since the nexec script is used by all, I have to have imagine all flagged parameters that will ever be used and have them defined in the front  of param list (since nsh's getops doesn't know how to parse them properly when they are not in the front), and every job that calls that nsh script has to have all the unused flags/parameters turned off.  Plus now I have to have a different jobs to call the same nsh script, since the parameters passed to the nsh script is cuztomizable only at the job level, not at the batch job level.



      I did try to set env variables this way:


      nexec -e sh -c "env ENV1=something ENV2=something2 SCRIPT $@"


      But the passing of $@ to the subshell, especially quoted ones does not want the shell to expand (such as '*' char) is not handled properly.  Even if I parse $@ in the nsh script and pass them properly as quoted individual parameters, the end quote of sh -c still appears at the wrong place and things gets passed wrong.  nsh -ncq didnt help.


      Besides the parameter passing, my other workaround is to set have global and local env files in the depot location, and copy that over to the client.  But now I have to have all my scripts source those env files, and I have to keep all the multiple copies of env files in sync (ksh, csh, perl all have different formats, so they need different env files).



      What I really wanted was to have the environmental variables be set at the job or batch job level, and have it propagate to the nsh script, then to the eventual script it runs.  But that seems impossible.

        • 1. Re: How can I pass env variables when running nexec -e
          Bill Robinson

          so i have script.sh which echos $1,2,ENV1,ENV2.  then i have a wrapper script that does the nexec of

          sh -c "ENV1=foo ENV2=bar /path/to/script.sh $@"


          and that seems to work fine:

          % nsh test.nsh "1 2 3 4 5"

          ARGS:1 2 3 4 5





          • 2. Re: How can I pass env variables when running nexec -e
            L W

            This doesn't seem to work with arguments that are multi-words, which I frequently have to do, either to combine a bunch of options into one param (such as all the FS to check, or even commands to run, which by itself might include quotes.) instead of defining an arbitrary large number of parameters for the script to accept


            % test.nsh "a b" c

            nsh ARG1: a b

            nsr ARG2: c

            nsr ARGC: 2


            script COUNT: 2

            script $@: a b

            script  ARG1: a

            script ARG2: b


            The nsh script's nexec line is:

            nexec -e sh -c "./dewit.sh $@"


            debug shows this is what nsh is executing:

            nexec -D //serverabc/tmp -ncq -e sh -c './dewit.sh a b' c


            I tried it with all sorts of escaping the quotes and w/wo -nc and it doesn't seem to help:


            nexec -e sh -c "./dewit.sh \"$@\""


            +./test.nsh:7> nexec -D //serverabc/tmp -e sh -c './dewit.sh "a b' 'c"'

            c": -c: line 0: unexpected EOF while looking for matching `"'

            c": -c: line 1: syntax error: unexpected end of file



            Tried nexec -e sh -c with these (among others), to no availe

            "./dewit.sh "\""$@"\"""

            "./dewit.sh '"'$@'"'"


            I think it is nsh that is parsing the whole argument list and passing it to sh -c, and it seems to be mis-parsing it.

            • 3. Re: How can I pass env variables when running nexec -e
              L W

              The whole issue with quote parsing seems to be due to the nsh shell mid-interpreting the arguments passed to the nexec command.


              The only way I found to work is to re-parse the parameters passed to the nsh script, separately quote each one, and write the whole nexec command into a separate file, which will then be sourced by the nsh script.


              echo -n 'nexec -D //serverabc -e sh -c "' > /tmp/run.it

              echo -n "env USERID='$USERID' DATAPATH='$DATAPATH' /tmp/dewit.sh " >> /tmp/run.it

              until [[ $# -eq 0 ]];do

                 echo  -n " '$1'"


              done >> /tmp/run.it

              echo '"' >> /tmp/run.it

              . /tmp/run.it

              If I set USERID and DATAPATH and pass the required parameters, the file it wrote to now has the proper layout for nexec to understand:

              % export USERID=oracle

              % export DATAPATH=/var

              % ./test.nsh "a b" c


              % cat /tmp/run.it

              nexec -D //serverabc -e sh -c "env USERID='oracle' DATAPATH='/var' /tmp/dewit.sh  'a b' 'c'"


              The only problem now is that it is not possible to assign environmental variables at the jobs level so it gets inherited by the nsh script where it can reconstruct them in the nexec command string.  I might have to assign it as parameters, and when the nsh script encounter a parameter that consists of one or more word each having the pattern:




              the script would parse it, and echo each one after the  echo -n "env command.  This way, even if one job might need to set USERID and another job might need PORTID, each can be passed to their respective scripts by the nsh wrapper script

              • 4. Re: How can I pass env variables when running nexec -e
                Bill Robinson

                I’m not sure what you mean by ‘environment variables at the job level’.  you can pass parameters to the nsh script/job just like any other shell script.

                • 5. Re: How can I pass env variables when running nexec -e
                  L W

                  What I am trying to do is create a batch job to do perform a set of actions on a set of varied set of servers.  I want to re-use the underlying batch jobs, jobs and scripts as much as possible, with only differences in server list or if the servers between environments have differences such as UID or app location, thus I can pass what's unique as environment variables (or less preferably, parameters).


                  Example I would have used:


                  HRApp_Stop_Dev     - batch job to stop HRApp in dev env

                      HRApp_Web_Dev     - batch job, with a list of dev web servers to stop (e.g. webdev1, webdev2 )

                            HRApp_Web    - job that executes with common generic wrapper

                                 wrapper.nsh    - nsh script that finds parent job name, then finds script in depot (in same location as job (in /depot instead of /jobs)) with similar name and executes it

                                      HRApp_Web_stop.ksh    - script it found in depot

                      HRApp_Appsvr_Dev     - batch jobs with a list of dev app servers to stop (e.g. appdev1, appdev2, appdev3 )

                            HRApp_Appsvr   - job that executes with common generic wrapper

                                 wrapper.nsh    - same nsh script

                                      HRApp_Appsvr_stop.ksh     - script it found in depot



                  HRApp_Stop_Prod     - batch job to stop HRApp in prod env

                       HRApp_Web_Prod     - batch job, with a list of prod web servers to stop (e.g. webprod1, webprod2, webprod3 )

                            HRApp_Web    - same job as above, but with a different

                                 wrapper.nsh    - same nsh script

                                      HRApp_Web_stop.ksh    - same script it found in depot above



                  The ones in color shows that they use the same Job, nsh script or script.  Bold are unique objects.  The *_stop.ksh and wrapper.nsh scripts could be re-used for other Apps without modification. 


                  So I would have wanted the job HRApp_Web job to have env variable UID=httpddev in the Dev's batch job and UID=httpdprod in Prod's batch job passed to wrapper.nsh, which would pass it the script via nexec -e HRApp_Web_stop.ksh.  I cannot find a way to do that, so I had to pass it to the wrapper.nsh script as 2nd param (I reserved 1st param for -t, which is timeout param that some script would take as an option.  Since wrapper.nsh is a script used by all jobs, all jobs that executes wrapper.nsh will have that as an option, even if the script it executes might not use it).


                  But I can't find a way for the batch job to set parameters or env variables to pass to the job, which then would pass it to wrapper.nsh.  And a job's params passed to the wrapper.nsh cannot be overridden by its parent batch job (like server list can). I ending up creating separate jobs for Prod and Dev env, each calling wrapper.nsh, each with different param.  So now it looks like:




                            HRApp_Web_Dev - job specific to Dev env, with param 2 = httpddev (flagged param 1 is skipped)

                                 wrapper.nsh    - nsh script passes received param to script via $@

                                      HRApp_Web_stop.ksh    - script gets httpdev from $1 and uses it as UID




                            HRApp_Web_Prod - job specific to Prod env, with param 2 = httpdprod (flagged param 1 is skipped)

                                 wrapper.nsh    - nsh script passes received param to script via $@

                                      HRApp_Web_stop.ksh    - script gets httpdprod from $1 and uses it as UID


                  There could be multiple tens or over a hundred of Apps that might have similar steps, so re-use and non-environment specific jobs are desirable.  Having different jobs for different environment would double or triple the number of batch jobs I have to create.