7 Replies Latest reply on Jul 31, 2020 12:08 AM by Rajesh M Kumar

    auto rerun with a delay for failed cyclic jobs

    Rajesh M Kumar
      Share This:

      Hi All,

       

      We have jobs running every hour and want to set auto rerun so that job restarts after failure only once with a delay of 10 mins. These jobs don't have pre or post execution command option available. Any guidance in this regards would be much helpful.

        • 1. Re: auto rerun with a delay for failed cyclic jobs
          Mark Francome

          Within the pure Control-M job sphere, this is not easily done. One idea would be to use an exclusive resource and, when the job fails, force in another job that also uses the exclusive resource. The additional job just sleeps for 70 minutes.

           

          However, there is probably some clever API method of achieving this in a more reliable way.

          1 of 1 people found this helpful
          • 2. Re: auto rerun with a delay for failed cyclic jobs
            Bentze Perlmutter

            Hi Rajesh,

             

            I agree with Mark Francome that this functionality doesn't exist as an out-of-the-box feature in Control-M. (It would be nice if it did exist. Maybe you can request BMC to open an RFE for this?)

            I.e. whenever a job is rerun, via human/api/cli rerun, via it being cyclic or via a Do Rerun, the INTERVAL value, set on the Scheduling Tab under the Cyclic settings, is used to control the next run time.

             

            The workarounds I can suggest are:

            Option 1:

            1. Define a Shout to Program Destination

            2. The program will receive the Control-M Name and the OrderID of a job

            3. The program will use AAPI CLI to run: ctm run job::runNow <jobId>

            <jobId> is the <Control-M Name>:<Order ID>

            4. In your job definition define:

            On NOTOK

            Do Notify Shout Destination of #2 and in the message include %%DATACENTER:%%ORDERID (they will get passed to the program as $2 so it will need to retrieve it to use in the ctm run command)

             

            This way when the job fails, assuming it is cyclic and Control-M/Server System Parameter CYCLIC_MAXWAIT is set to NOTOK (which is the default), it will rerun based on the 60min interval but shout to program will bypass it and run it immediately.

            If you want a 10min wait so add that into the Program called by Shout. (but if you do this, the shout should call a parent program that calls the aapi program as a child and the parent should not wait for child, otherwise all Control-M/Server activities may be suspended for 10min.

            I.e. Shout to Program --> calls script1 --> calls script2 and script 1 ends --> script2 has 10min sleep and then the aapi command.

             

            Option 2:

            1. The Smart Folder this job is in should have a Smart Folder variable defined as: %%MYSLEEP=3000 (=50min)

            2. The cyclic job should have a PRECMD set to: sleep %%MYSLEEP

            3. The cyclic job should have an interval of 10MIN (instead of current 60MIN)

            4. The cyclic job should have these ON/DOs:

            On NOTOK DO Set Variable %%MYSLEEP=0

            On OK DO Set Variable %%MYSLEEP=3000

             

            This way when the job ends OK it will run after 10min but first have a sleep for 50min. (in GUI it will show as running after 10min but really the sleep will be happening for the first 50min)

            If the job ends NOTOK it will run after 10min but first have a sleep of 0min so effectively will run after 10min.

             

            Hope these ideas are helpful for what you need.

             

            Regards,

            Bentze

            1 of 1 people found this helpful
            • 3. Re: auto rerun with a delay for failed cyclic jobs
              Rajesh M Kumar

              Thanks Mark and Bentze,

               

              PRECMD option is not available as these job types were created using Application Integrator. Option 1 seems bit tricky but I'll definitely give it a try.

              • 4. Re: auto rerun with a delay for failed cyclic jobs
                Mark Francome

                Does AI allow for you to run the job as a command? If so, you could always consider putting conditional commands in -

                 

                if (whatever) = true then sleep 600 elsif echo 'run now' && your_script.sh

                • 5. Re: auto rerun with a delay for failed cyclic jobs
                  Bentze Perlmutter

                  Hi Rajesh,

                   

                  Although it is an AI Job Type, try to add the variable:

                  %%PRECMD = sleep 60

                   

                  The Pre Command field in the GUI resolves to the variable %%PRECMD so even if AI jobs don't display this field, they may still support the variable. I think it's worth a test.

                   

                  Regards,

                  Bentze

                  • 6. Re: auto rerun with a delay for failed cyclic jobs
                    Arun Prasanna Dhayalavel

                    Hello Rajesh,

                     

                    Try loading the sleep timer as a job while loading the job to re-run.

                     

                    As Job A(actual cyclic job) fails, need to be configured to load 2 jobs - Job B(with Sleep Command - sleep 600) and Job C(the non-cyclic copy of job A with dependency from Job B).

                     

                    This might affect the number of jobs being loaded in case of frequent failures though.

                     

                    Thanks,

                    Arun Prasanna D.

                    • 7. Re: auto rerun with a delay for failed cyclic jobs
                      Rajesh M Kumar

                      Thanks Arun,

                       

                      Unfortunately adding new job is not an option. There are many cyclic jobs that need this setup and we're at tight job based licensing.

                      For now client is ok with this setup only for non-cyclic jobs that we've implemented already.