4 replies to this topic

[Fromentin]

Hi,

we have to schedule our process every hour but if it fails, next instances should not run until the one who failes has been restarted succesfully.

We then created a script to disable the process if it fails and one to enable it when it ends in success.

Is there an easier way to do this ? (we have a lot of sessions and wnat to avoid to do this on all uproc)

Thanks.

[LeSlovene]

tooo! that's a tricky one we have here...
First question :Is it a multiple or a cyclic task?

In case of a cylic task with serial launch, I would suggest this, but I have not tested it :

- put in your uproc a dependence on itself with following properties : "Aborted, Excluded"
- add in your uproc a completion instructions of this type : purge uprocs' event for itself, status aborted.

With these settings, here is what should happen (to be confirmed by tests of course!!):
When an occurence falls in aborted, the next one will stay in event wait. Since this is a cyclic task with serial launch, no other occurence should ne launched.
You will have then to relaunch manually the aborted occurence, checking the "by pass condition check" in the launch window.
Assuming this execution is then successful, it will purge the previous event in aborted, enabling the execution in event wait to start right away, and the cycle to be back on normal state.

Beware, this solution is valid only for cyclic task with serial launch. Indeed, with multiple task or parallel launch the fact that an occurence stays in event wait will not prevent the next occurence to be launched, leading then to a queue of occurences in event wait as long as the aborted occurence has not sucess.

Another warning, I am not sure if the completion instruction can purge more than event with one instructions. If not, you will have troule in case of falilure of your relaunch (having then 2 aborted events to purge...)

Fromentin, on May 6 2010, 11:56 PM, said:

Hi,

we have to schedule our process every hour but if it fails, next instances should not run until the one who failes has been restarted succesfully.

We then created a script to disable the process if it fails and one to enable it when it ends in success.

Is there an easier way to do this ? (we have a lot of sessions and wnat to avoid to do this on all uproc)

Thanks.

Edited by LeSlovene, 07 May 2010 - 09:32 AM.

[men]

Hi,

I concur with our Slovène friend that a solution should normally be found through conditioning the Uproc on itself. But the exact settings depend on what you have to do.

When a job fails, do you want to cancel or postpone the next runs until the aborted job has been successfully recovered?

If you need to cancel them, you can use a fatal condition that forbids the job to run if an event "Aborted" exists for the same job. If you need to postpone, that gets trickier as mentioned by François!

One important thing is also to be aware of the failure ASAP: What mechanism/tool are in place? Are you using a supervision tool for this (HP OVO, BMC Performance Manager, Tivoli, Nagios or Orsyp "generic" supervision)? This would possibly limit the appearances of this situation.

Michel

[Fromentin]

Hi,

unfortunately we don't have any cyclic task, and to add some complexity, we have also to do this on some session if any its uproc fails.

I will go ahead with my scripts, it will just be long to implement.

Thanks anyway !

[ary]

Hi Fromentin,

Here is a neat trick that I use whenever I need the behavior you're describing. It serializes multiple executions of a session, meaning that an instance of this session won't start while another instance of itself has started, but not yet completed.

HEADER
   `-uproc1
		`-uproc2
			   `-TRAILER

With the following conditioning:
HEADER: dependent on itself being absent, same session, same mu, any processing date (you can set it as fatal if you want it to be refused if another instance of itself hasn't finished properly)
TRAILER: completion instruction to purge HEADER events, same session, same mu, any processing date

How does it work?
As soon as the header of the session starts, it posts an event corresponding to itself in the Job Events. The HEADER of any other instance of this session would see this event (not absent) and wait for it to disappear (via the completion instructions).

This way, you are sure that multiple instances of this session will execute sequentially, never in parallel.

Hope this helps!
A.

Edited by ary, 13 July 2010 - 10:39 PM.