1 2 Previous Next 15 Replies Latest reply on Oct 8, 2012 12:26 PM by Jim Wilson

    NSH Script job quietly quits mid-way, with Green status

      Hello,

       

      I have an odd behavior with NSH Script job.  The script job is executing on the BLApp server, running a snapshot job and then grabbing the run key and  results of the snapshot.  The problem though is, after the snapshot job executes to successful completion and then gets the job run key, the nsh script job just stops.  No errors in logs.  NSH script job reports successful completion, even though it didn't execute the next several hundred lines of nsh.

       

      #******************************************

      blcli_execute Job executeAndWait 32 "${SNAPJOB_DBKEY}"

       

      blcli_execute JobRun getJobRunKey

      ### NSH Script Job just STOPS here.

      blcli_storeenv SNAP_RUNKEY

      blcli_execute JobResult findJobResultKey "${SNAP_RUNKEY}"

      blcli_storeenv SNAP_RESULTKEY
      #*******************************************

       

      The last thing in the job log is the result of the getJobRunKey.  That DBKey that getJobRunKey returns is valid, as I've passed it to findJobResultKey manually in the nsh cmd line and I get a proper result back.

       

      I've tried executing the Snapshot job using other BLCLI commands, and the getJobRunKey portion quits out in the same way.

       

      This doesn't happen for every server.  We originally thought it was a function of a very large snapshot result, or perhaps a server that is geographically very far away.  But I have also seen this behavior from a relatively small snapshot that is closer geographically.

       

       

      What factors might contribute to this happening?  I've only seen it via NSH Script job executing a snapshot job and then grabbing the job run key (like Job executeJobAndWait or SnapshotJob executeJobGetJobResultKey or SnapshotJob executeJobAndWait....which are all more or less doing the same thing, which is why I'm now trying to break it out like the above.  Seems that the getJobRunKey is the offender....?

       

      BL 8.1.03.312

       

       

       

      Thank You!

        • 1. NSH Script job quietly quits mid-way, with Green status
          Bill Robinson

          i think you want to run this:

          blcli_execute SnapshotJob executeAndWait "${SNAPJOB_DBKEY}"

          blcli_storeenv SNAP_RUNKEY

          blcli_execute JobResult findJobResultKey "${SNAP_RUNKEY}"

          blcli_storeenv SNAP_RESULTKEY

          • 2. NSH Script job quietly quits mid-way, with Green status
            Bill Robinson

            also you can put this in the top of the script to get debug info:

             

            DEBUG_BLCLI=true

            export DEBUG_BLCLI

            • 3. NSH Script job quietly quits mid-way, with Green status

              Hi Bill,

               

              Thank you.  I've tried the SnapshotJob executeAndWait.  It returns the runkey and then just stops.

               

              Good note on the debug info.  Will retry with that and hopefully be illuminated.

              • 4. NSH Script job quietly quits mid-way, with Green status

                Is there anything I can pass to the SnapshotJob execution that will give such debug detail in the snapshot job logs as well?

                • 5. NSH Script job quietly quits mid-way, with Green status
                  Bill Robinson

                  there's a DEBUG_MODE_ENABLED property on the job in 8.1+ that will get you debug info from the job, but i don't think that is your problem.

                  • 6. NSH Script job quietly quits mid-way, with Green status

                    Bill, understood that the DEBUG_MODE_ENABLED might not be a help here, but it's good to know....thx.

                     

                    So here is the relevant log from the above-referenced nsh script job commands.  Please help me interpret the last couple of lines, as I'm not sure what I'm reading.  Again you can see that the nsh script job just halts (now i can see Exit Code 0 from the Debug flag), but there is still much more nsh in the script that is simply not executed:

                     

                    Info Executing Snap job. Please wait...

                    Error -- ENTERING bin_blcli_execute

                    Error -- ENTERING __blcli_execute

                    Error -- ENTERING __blcli_connect

                    Error -- ENTERING __init_blcli

                    Error -- ENTERING __init_blcli_jvm

                    Error -- EXITING  __init_blcli_jvm

                    Error -- EXITING  __init_blcli

                    Error -- EXITING  __blcli_connect

                    Error -- RETRIEVING method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;

                    Error -- FINISHED retrieving method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;

                    Error -- CREATING an array of Strings

                    Error -- CREATED an array of Strings

                    Error -- ADDING arguments to the array

                    Error -- ADDED arguments to the array

                    Error -- CALLING executeEmbeddedShellCommand

                    Error -- CALLED executeEmbeddedShellCommand

                    Error -- ENTERING __is_command_result_success

                    Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- RETRIEVING method: success with sig ()Z

                    Error -- FINISHED retrieving method: success with sig ()Z

                    Error -- EXITING  __is_command_result_success: 1

                    Error -- EXITING  __blcli_execute

                    Error -- EXITING  bin_blcli_execute

                    Error -- ENTERING bin_blcli_execute

                    Error -- ENTERING __blcli_execute

                    Error -- ENTERING __blcli_connect

                    Error -- ENTERING __init_blcli

                    Error -- ENTERING __init_blcli_jvm

                    Error -- EXITING  __init_blcli_jvm

                    Error -- EXITING  __init_blcli

                    Error -- EXITING  __blcli_connect

                    Error -- RETRIEVING method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;

                    Error -- FINISHED retrieving method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;

                    Error -- CREATING an array of Strings

                    Error -- CREATED an array of Strings

                    Error -- ADDING arguments to the array

                    Error -- ADDED arguments to the array

                    Info com.bladelogic.om.infra.model.job.jobrun.JobRunImpl@7d444b2c

                    Error -- CALLING executeEmbeddedShellCommand

                    Info SUCCESS: Executing Snapshot job

                    Info Converting job run to run key

                    Error -- CALLED executeEmbeddedShellCommand

                    Error -- ENTERING __is_command_result_success

                    Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- RETRIEVING method: success with sig ()Z

                    Error -- FINISHED retrieving method: success with sig ()Z

                    Error -- EXITING  __is_command_result_success: 1

                    Error -- EXITING  __blcli_execute

                    Error -- EXITING  bin_blcli_execute

                    Error -- ENTERING bin_blcli_storeenv

                    Error -- ENTERING __blcli_storeenv

                    Error -- ENTERING __is_command_result_success

                    Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- RETRIEVING method: success with sig ()Z

                    Error -- FINISHED retrieving method: success with sig ()Z

                    Error -- EXITING  __is_command_result_success: 1

                    Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult

                    Error -- RETRIEVING method: getReturnValue with sig ()Ljava/lang/Object;

                    Error -- FINISHED retrieving method: getReturnValue with sig ()Ljava/lang/Object;

                    Error -- RETRIEVING method: toString with sig ()Ljava/lang/String;

                    Error -- FINISHED retrieving method: toString with sig ()Ljava/lang/String;

                    Error -- EXITING  __blcli_storeenv

                    Error -- EXITING  bin_blcli_storeenv

                    Info DBKey:SJobRunKeyImpl:2492758

                    Error -- ENTERING bin_blcli_execute

                    Error -- ENTERING __blcli_execute

                    Error -- ENTERING __blcli_connect

                    Error -- ENTERING __init_blcli

                    Error -- ENTERING __init_blcli_jvm

                    Error -- EXITING  __init_blcli_jvm

                    Error -- EXITING  __init_blcli

                    Error -- EXITING  __blcli_connect

                    Error -- RETRIEVING method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;

                    Error -- FINISHED retrieving method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;

                    Error -- CREATING an array of Strings

                    Error -- CREATED an array of Strings

                    Error -- ADDING arguments to the array

                    Error -- ADDED arguments to the array

                    Error -- CALLING executeEmbeddedShellCommand

                    Info Exit Code 0

                    Info The job 'Build' has succeeded

                    ******************************************************

                     

                    Thank You.

                    • 7. NSH Script job quietly quits mid-way, with Green status

                      Update: From these logs, I can now see that the blcli_storeenv command is executed successfully.  It appears now that the problem is stemming from the  findJobResultKey routine.

                       

                      Again, the above log is from the commands referenced in my original post:

                      blcli_execute Job executeAndWait 32 "${SNAPJOB_DBKEY}"

                       

                      blcli_execute JobRun getJobRunKey

                       

                      blcli_storeenv SNAP_RUNKEY

                       

                      blcli_execute JobResult findJobResultKey "${SNAP_RUNKEY}"

                       

                      blcli_storeenv SNAP_RESULTKEY

                       

                       

                       

                      Thanks.

                      • 8. NSH Script job quietly quits mid-way, with Green status
                        Bill Robinson

                        can you change it to:

                        blcli_execute SnapshotJob executeAndWait "${SNAPJOB_DBKEY}"

                        blcli_storeenv SNAP_RUNKEY

                        blcli_execute JobResult findJobResultKey "${SNAP_RUNKEY}"

                        blcli_storeenv SNAP_RESULTKEY

                        • 9. NSH Script job quietly quits mid-way, with Green status
                          Bill Robinson

                          also - what is the purpose of this script ?

                          • 10. NSH Script job quietly quits mid-way, with Green status

                            FYI for others reading this, Bill's 1st command should be

                            blcli_execute SnapshotJob executeJobAndWait ...

                            • 11. NSH Script job quietly quits mid-way, with Green status

                              Here are the logs from a job run with the commands you indicated.  Things are roughly the same, still just quitting...

                               

                              Info Executing Snap job.  Please wait...
                              Error -- ENTERING bin_blcli_execute
                              Error -- ENTERING __blcli_execute
                              Error -- ENTERING __blcli_connect
                              Error -- ENTERING __init_blcli
                              Error -- ENTERING __init_blcli_jvm
                              Error -- EXITING  __init_blcli_jvm
                              Error -- EXITING  __init_blcli
                              Error -- EXITING  __blcli_connect
                              Error -- RETRIEVING method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;
                              Error -- FINISHED retrieving method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;
                              Error -- CREATING an array of Strings
                              Error -- CREATED an array of Strings
                              Error -- ADDING arguments to the array
                              Error -- ADDED arguments to the array
                              Error -- CALLING executeEmbeddedShellCommand
                              Error -- CALLED executeEmbeddedShellCommand
                              Error -- ENTERING __is_command_result_success
                              Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult
                              Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult
                              Error -- RETRIEVING method: success with sig ()Z
                              Error -- FINISHED retrieving method: success with sig ()Z
                              Error -- EXITING  __is_command_result_success: 1
                              Error -- EXITING  __blcli_execute
                              Error -- EXITING  bin_blcli_execute
                              Error -- ENTERING bin_blcli_storeenv
                              Error -- ENTERING __blcli_storeenv
                              Error -- ENTERING __is_command_result_success
                              Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult
                              Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult
                              Error -- RETRIEVING method: success with sig ()Z
                              Error -- FINISHED retrieving method: success with sig ()Z
                              Error -- EXITING  __is_command_result_success: 1
                              Error -- RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult
                              Error -- FINISHED RETRIEVING class: com/bladelogic/om/infra/cli/CommandResult
                              Error -- RETRIEVING method: getReturnValue with sig ()Ljava/lang/Object;
                              Error -- FINISHED retrieving method: getReturnValue with sig ()Ljava/lang/Object;
                              Error -- RETRIEVING method: toString with sig ()Ljava/lang/String;
                              Error -- FINISHED retrieving method: toString with sig ()Ljava/lang/String;
                              Error -- EXITING  __blcli_storeenv
                              Error -- EXITING  bin_blcli_storeenv
                              Error -- ENTERING bin_blcli_execute
                              Error -- ENTERING __blcli_execute
                              Error -- ENTERING __blcli_connect
                              Error -- ENTERING __init_blcli
                              Error -- ENTERING __init_blcli_jvm
                              Info DBKey:SJobRunKeyImpl:2495076
                              Error -- EXITING  __init_blcli_jvm
                              Info SUCCESS: Executing Snapshot job
                              Error -- EXITING  __init_blcli
                              Error -- EXITING  __blcli_connect
                              Error -- RETRIEVING method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;
                              Error -- FINISHED retrieving method: executeEmbeddedShellCommand with sig ([Ljava/lang/String;)Lcom/bladelogic/om/infra/cli/CommandResult;
                              Error -- CREATING an array of Strings
                              Error -- CREATED an array of Strings
                              Error -- ADDING arguments to the array
                              Error -- ADDED arguments to the array
                              Error -- CALLING executeEmbeddedShellCommand
                              Info Exit Code 0
                              Info The job 'Build' has succeeded

                              *************************************************************

                               

                              Ideas???

                               

                               

                              Thanks.

                              • 12. NSH Script job quietly quits mid-way, with Green status
                                Bill Robinson

                                can you put a set -x at the top of your script and re-run it ?  take out the DEBUG_BLCLI.

                                • 13. Re: NSH Script job quietly quits mid-way, with Green status

                                  Hi Bill,

                                   

                                  Replaced the BLCLI Debug with set -x.  New output below (...same behavior).  It's still got me thinking about the findJobResultKey command nested within whatever "job execute..." command called....

                                   

                                   

                                  Info Executing Snap job.  Please wait...

                                  Error +//@/data/apps/nsh/NSH/tmp/application_server/scripts/job__87185ede-199f-47ff-80e9-859692c78156/script_DBKey-SJobKeyImpl-2081623-7__2349c723-4dc1-46ec-8b19-1419d0afdaa2.2020724.1_ADAS_Build.nsh:2472> blcli_storeenv MASTERSNAP_RUNKEY

                                  Error +//@/data/apps/nsh/NSH/tmp/application_server/scripts/job__87185ede-199f-47ff-80e9-859692c78156/script_DBKey-SJobKeyImpl-2081623-7__2349c723-4dc1-46ec-8b19-1419d0afdaa2.2020724.1_ADAS_Build.nsh:2473> errorHandler 0 'Executing Snapshot job'

                                  Error +errorHandler:2> RETURN_CODE=0

                                  Error +errorHandler:3> MSG='Executing Snapshot job'

                                  Error +errorHandler:4> ((  0 > 0  ))

                                  Error +errorHandler:10> echo

                                  Error +errorHandler:11> echo 'SUCCESS: Executing Snapshot job'

                                  Error +//@/data/apps/nsh/NSH/tmp/application_server/scripts/job__87185ede-199f-47ff-80e9-859692c78156/script_DBKey-SJobKeyImpl-2081623-7__2349c723-4dc1-46ec-8b19-1419d0afdaa2.2020724.1_ADAS_Build.nsh:2480> blcli_execute JobResult findJobResultKey DBKey:SJobRunK

                                  Info DBKey:SJobRunKeyImpl:2495092

                                  Info SUCCESS: Executing Snapshot job

                                  Info Exit Code 0

                                  Info The job 'Build' has succeeded

                                  ***************************************************************************

                                   

                                  As a reminder, there's still plenty of nsh after the snapshot job is executed.  This script job just quits after the snapshot job is run.  Only happens on select servers.

                                   

                                  Do you think I'm heading towards a support ticket on this?

                                  1 2 Previous Next