FS
Documentation

Task3G/User Guide/Setting Up Jobs

This page was last modified 04:38, 30 May 2013.

From Documentation

< Task3G | User Guide(Difference between revisions)
Jump to: navigation, search
Revision as of 06:33, 18 April 2006
Daniels (Talk | contribs)

← Previous diff
Current revision
Mike (Talk | contribs)
(Before you run your first job)
Line 1: Line 1:
-This chapter describes the process of defining jobs, including:+This section describes the process of defining jobs, including:
-Jobs+*Jobs
-Tasks+*Tasks
-Queues+*Queues
-Dependencies+*Dependencies
-Roles and Access Capabilities+*Roles and Access Capabilities
-Before you run your first job+ 
-task3G forms the skeleton of a complete task control and batch job submission system.+<br>
-Your next task is to flesh out this skeleton by adding job components for the+=== Before you run your first job ===
-main jobs and procedures at your site. The Overview on page 15, explains what job+ 
-components are and how they are related.+Task3G forms the skeleton of a complete task control and batch job submission system. Your next task is to flesh out this skeleton by adding job components for the main jobs and procedures at your site. The [[Task3G/User Guide/Overview|Task3G Overview]] explains what job components are and how they are related.
-26 Setting Up Jobs+ 
-These details are stored centrally in task3G's database, rather than being duplicated+These details are stored centrally in task3G's database, rather than being duplicated in shell-scripts and applications throughout the network. This means that generalized components need only to be entered once and can be maintained in one place.
-in shell-scripts and applications throughout the network. This means that generalised+ 
-components need only to be entered once and can be maintained in one place.+You may not need to do all of the following steps, as some of the components and site-specific details will have been defined when task3G and COSmanager were installed.
-You may not need to do all of the following steps, as some of the components and+#If necessary define new job queues.
-site-specific details will have been defined when task3G and COSMmanager were+#Define tasks for any commands that will be used in a job.
-installed.+#Define jobs to carry out all your regular batch processing.
-1. If necessary define new job queues.+#Create dependencies within a job.
-2. Define tasks for any commands that will be used in a job.+ 
-3. Define jobs to carry out all your regular batch processing.+<br>
-4. Create dependencies within a job.+ 
-Planning your job control strategy+=== Planning your job control strategy ===
-There is no universal scheme that is best for all sites, and task3G does not try to+ 
-impose a standard strategy. However, task3G does make it simpler to automate your+There is no universal scheme that is best for all sites, and task3G does not try to impose a standard strategy. However, task3G does make it simpler to automate your job submission procedures, as it lets you build jobs from reusable components. task3G also provides a convenient means of documenting your procedures at any point, as information about all your jobs is stored centrally in task3G tables.
-job submission procedures, as it lets you build jobs from reusable components.+ 
-task3G also provides a convenient means of documenting your procedures at any+There are two main points to consider. First, given the current system environment, how best to handle the current processing workload. Second, planning to maintain service in the face of changes in processing requirements, both expected (increase in workload) and unexpected (system failure).
-point, as information about all your jobs is stored centrally in task3G tables.+ 
-There are two main points to consider. First, given the current system environment,+
-how best to handle the current processing workload. Second, planning to maintain+
-service in the face of changes in processing requirements, both expected (increase in+
-workload) and unexpected (system failure).+
Following is a list of factors that may influence your strategy: Following is a list of factors that may influence your strategy:
-What is the relative priority of interactive users and background processing?+*What is the relative priority of interactive users and background processing?
-How heavily is the system loaded? Are there cyclic peaks in demand, and if+*How heavily is the system loaded? Are there cyclic peaks in demand, and if so when? What is the average load and maximum load? How is system load projected to change in future?
-so when? What is the average load and maximum load? How is system load+*Does your site have performance measures or service guarantees that set targets in terms of system availability, terminal response time, or job turnaround?
-projected to change in future?+*What is the cost to the organisation if some jobs fail - e.g. database update transactions lost or delayed, delays in processing accounts or payroll, financial reports not distributed.
-Does your site have performance measures or service guarantees that set+*Are there procedures that should be followed to recover from a major system failure, and are staff aware of them? For example, how and when should interrupted jobs rerun?
-targets in terms of system availability, terminal response time, or job turnaround?+*Your current and future network configuration - e.g. which jobs will be run centrally and which on distributed hosts?
-What is the cost to the organisation if some jobs fail - e.g. database update+*What resources are available? Are there limitations in terms of disk capacity, I/O and network throughput, tape drives, etc.?
-transactions lost or delayed, delays in processing accounts or payroll, financial+*Business requirements - e.g. need for round-the-clock availability, daily/weekly/monthly/yearly processing cycles. What happens to scheduled jobs on public holidays?
-reports not distributed.+*Operations environment - e.g. shift coverage. How much time is available each day to process background jobs?
-Setting Up Jobs 27+ 
-Are there procedures that should be followed to recover from a major system+<br>
-failure, and are staff aware of them? For example, how and when should+ 
-interrupted jobs rerun?+=== Task3G Configuration ===
-Your current and future network configuration - e.g. which jobs will be run+ 
-centrally and which on distributed hosts?+The configuration screen is accessed via the COSmanager menu
-What resources are available? Are there limitations in terms of disk capacity,+ 
-I/O and network throughput, tape drives, etc.?+[[Image:Task img 4.jpg|none|frame|upright|Figure 5 — The COSmanager buttonbar (GUI)]]
-Business requirements - e.g. need for round-the-clock availability,+ 
-daily/weekly/monthly/yearly processing cycles. What happens to scheduled+ 
-jobs on public holidays?+By clicking the {{cnav|Config}} button, a sub menu appears containing all product configuration options.
-Operations environment - e.g. shift coverage. How much time is available+Access task3G Configuration by selecting the task3G configuration option. Select {{cnav|Product Configuration > task3G configuration}}.
-each day to process background jobs?+ 
-task3G Configuration+The task3G Job configuration console should appear. This screen shows all defined jobs as well as providing access for configuring all other aspects.
-The configuration screen is accessed via the COSMmanager menu+ 
-Figure 5 — The COSMmanager buttonbar (GUI)+<br>
-By clicking the Config button, a sub menu appears containing all product configuration+ 
-options.+== How to Define a Job ==
-Access task3G Configuration by selecting the task3G configuration option. Select+ 
-Product Configuration > task3G configuration+There are three types task3G jobs that can be defined. They are automatic jobs, manual jobs and subjobs.
-The task3G Job configuration console should appear. This screen shows all defined+ 
-jobs as well as providing access for configuring all other aspects.+;Automatic jobs: are jobs that are run by cron. An operator is not required to submit jobs of this type. Jobs in this mode need a schedule and time(s) at which the job will run. This is defined by the When and At Time(s) field on the job prompt form.
-How to Define a Job+;Manual jobs: are jobs that must be submitted by an operator. These jobs can also be scheduled but are never run automatically. The manual job uses the At Time(s) and On (day) fields for scheduling. However, this is optional, manual jobs need not have scheduling details as these can be provided while submitting a job.
-There are three types task3G jobs that can be defined. They are automatic jobs,+;Subjobs: are jobs within a job. These jobs cannot be scheduled. They are to be defined in another job as a task. When it is time for this task (subjob) to run, then the tasks of this subjob will execute. Subjobs help in the sense that larger, more complex jobs can be broken into smaller, simpler steps.
-manual jobs and subjobs.+{{note| Any job can be used as a subjob. However, jobs defined as subjobs cannot be run by themselves.}}
-Automatic jobs are jobs that are run by cron. An operator is not required to+ 
-submit jobs of this type. Jobs in this mode need a schedule and+<br>
-time(s) at which the job will run. This is defined by the When and+=== Configuration Console ===
-At Time(s) field on the job prompt form.+ 
-Manual jobs are jobs that must be submitted by an operator. These jobs can also+The task3G Job console is where all configuration concerning jobs is accomplished. Select {{cnav|Config > task3G Configuration}}.
-be scheduled but are never run automatically. The manual job uses+ 
-the At Time(s) and On (day) fields for scheduling. However, this is+{{cnav|Display > Job report}} menu option displays a full report of the last run.
-optional, manual jobs need not have scheduling details as these can+ 
-be provided while submitting a job.+The {{cnav|Maintain}} pulldown menu contains the standard Add, Change, Clone and Remove options which work the same way as maintaining other COSmanager tables.
-Subjobs are jobs within a job. These jobs cannot be scheduled. They are to+ 
-be defined in another job as a task. When it is time for this task+
-(subjob) to run, then the tasks of this subjob will execute. Subjobs+
-help in the sense that larger, more complex jobs can be broken into+
-smaller, simpler steps.+
-Note Any job can be used as a subjob. However, jobs defined as subjobs+
-cannot be run by themselves.+
-Note A step by step guide to defining a job can be found in Tutorial 2:+
-Creating a Job on page 84.+
-Note A step by step guide to defining a subjob can be found in Tutorial 3:+
-Creating a Subjob on page 86.+
-Configuration Console+
-The task3G Job console is where all configuration concerning jobs is accomplished.+
-Select Config > task3G Configuration.+
-Setting Up Jobs 29+
-Display > Job report menu option displays a full report of the last run.+
-The Maintain pulldown menu contains the standard Add, Change, Clone and+
-Remove options which work the same way as maintaining other COSMmanager+
-tables.+
The following is a list of actions for each menu item: The following is a list of actions for each menu item:
-Creating a new job is done by selecting Maintain > Add.+*Creating a new job is done by selecting {{cnav|Maintain > Add}}.
-Changing the details of a job is done by selecting Maintain > Change.+*Changing the details of a job is done by selecting {{cnav|Maintain > Change}}.
-Copying a job is done by selecting Maintain > Clone.+*Copying a job is done by selecting {{cnav|Maintain > Clone}}.
-Deleting a job is done by selecting Maintain > Remove.+*Deleting a job is done by selecting {{cnav|Maintain > Remove}}.
-Maintain > Tasks in job+ 
-selects the task3G Tasks in job console allowing you to view and+ 
-maintain the tasks in a chosen job.+[[Image:Task img 5.jpg|frame|Figure 6 — Configuration console, or ‘Maintain Jobs’ window]]
-Maintain > Create Duty+;{{cnav|Maintain > Tasks in job}}: selects the task3G Tasks in job console allowing you to view and maintain the tasks in a chosen job.
-for use with duty3G.+ 
-Maintain > Sync crontab+;{{cnav|Maintain > Create Duty}}: for use with duty3G.
-to ensure that the automatic jobs are correctly replicated in+ 
-COSMmanager crontab.+;{{cnav|Maintain > Sync crontab}}: to ensure that the automatic jobs are correctly replicated in COSmanager crontab.
-Tables > Tasks+ 
-selects the task3G Task details console allowing you to maintain+;{{cnav|Tables > Tasks}}: selects the task3G Task details console allowing you to maintain tasks
-tasks+ 
-Tables > Roles+;{{cnav|Tables > Roles}}: selects the task3G Roles and access capabilities console allowing you to view an maintain task3G roles and capabilities.
-selects the task3G Roles and access capabilities console allowing+ 
-you to view an maintain task3G roles and capabilities.+;{{cnav|Tables > Queues}}: selects the task3G Job queue details console allowing you to maintain the job queue.
-Tables > Queues+ 
-selects the task3G Job queue details console allowing you to+;{{cnav|Tables > Schedules}}: selects the task3G schedules console allowing you to view and maintain task3G schedules.
-maintain the job queue.+ 
-Tables > Schedules+;{{cnav|Tables > Datelists}}: selects the task3G datelists console allowing you to maintain datelists.
-selects the task3G schedules console allowing you to view and+ 
-maintain task3G schedules.+<br>
-Tables > Datelists+ 
-selects the task3G datelists console allowing you to maintain+=== To Define Information About a Job ===
-datelists.+ 
-30 Setting Up Jobs+Defining a job is split into two entry forms. The first comprises of the basic options needed for each task, the latter comprises of advanced options.
-.+ 
-Figure 6 — Configuration console, or ‘Maintain Jobs’ window+[[Image:Task img 6.jpg|frame|Figure 7 — Prompt form to add a job]]
-The above screen shows all defined jobs.+;Job: The job name identifies each individual job. This must be unique.
-To Define Information About a Job+ 
-Defining a job is split into two entry forms. The first comprises of the basic options+;Description: Appears in the job schedule, so make it meaningful to help users to choose the right job.
-needed for each task, the latter comprises of advanced options.+ 
-Job The job name identifies each individual job. This must be unique.+;Mode: The mode in which the job will run in:
-Description Appears in the job schedule, so make it meaningful to help users+:;''automatic'': if the job will run to a regular schedule and doesn't need to be manually submitted
-to choose the right job.+:;''manual'': if the job will be submitted by the operator
-Mode The mode in which the job will run in:+:;''subjob'': if the job will only run as a task in another job.
-automatic if the job will run to a regular schedule and doesn't need to be+ 
-Setting Up Jobs 31+ 
-manually submitted+==== When to initiate the job (automatic job) ====
-manual if the job will be submitted by the operator+ 
-subjob if the job will only run as a task in another job.+The When and At time(s) fields determine the scheduling details associated with the job.
-Figure 7 — Prompt form to add a job+ 
-When to initiate the job (automatic job)+;When: The frequency at which the job runs, whether its daily, weekly, monthly, every second week etc. This is used for automatic jobs only.
-The When and At time(s) fields determine the scheduling details associated with+ 
-the job.+;At time(s): Time at which the job will start. This is used for automatic jobs and is also optional for manual jobs.
-When The frequency at which the job runs, whether its daily, weekly,+ 
-monthly, every second week etc. This is used for automatic jobs+;On (day): The day in which the job will start. This is mandatory for manual jobs if a start time was specified above, i.e. the job is 'scheduled'. ‘Next 24 hrs’ (default) means that the job will run later on the day of submission, or if submitted after the ‘At time(s)’ on the following day.
-only.+ 
-At time(s) Time at which the job will start. This is used for automatic jobs and+ 
-is also optional for manual jobs.+The next three fields specify the access role; the host this job will run on; and the job queue it will be scheduled to.
-On (day) The day in which the job will start. This is mandatory for manual+ 
-jobs if a start time was specified above, i.e. the job is 'scheduled'.+;Access role: Who can run and control this job.
-‘Next 24 hrs’ (default) means that the job will run later on the day+;Run on host: On which host will the job run.
-32 Setting Up Jobs+;Queue: The job queue the job will be submitted to.
-of submission, or if submitted after the ‘At time(s)’ on the+ 
-following day.+ 
-The next three fields specify the access role; the host this job will run on; and the+==== Advanced options ====
-job queue it will be scheduled to.+ 
-Access role Who can run and control this job.+You can configure advanced options such as job parameters, pre- and post-conditions, concurrent tasks, priority, clear status and notes. Press {{cnav|Advanced}} to enter one or more of these command or options.
-Run on host: On which host will the job run.+ 
-Queue The job queue the job will be submitted to.+[[Image:Task img 7.jpg|frame|Figure 8 — Prompt form to add a job (advanced features)]]
-Advanced options+;Job parameters: Job parameters cause SHELL ENVIRONMENT variables to be set which are available to all tasks in the job. Commonly they would be included as $ variables which are passed as parameters to the commands in a task.
-You can configure advanced options such as job parameters, pre- and post-conditions,+ 
-concurrent tasks, priority, clear status and notes. Press Advanced to enter one+;Fixed Parameters: Job parameters can be set to fixed values in the job configuration. They should be a space separated list in the form VARIABLE=VALUE. The given VALUE will be assigned to the VARIABLE at the start of the job, and is available as an ENVIRONMENT VARIABLE to all tasks in the job.
-or more of these command or options.+ 
-Setting Up Jobs 33+;Prompt form: Job parameters can also be set at job submission via a PROMPT form. The full path must be specified, unless the form is in <tt>$APPL_HOME/local/prompt</tt>. As with the 'Fixed parameters' the value(s) specified in the form will be available as ENVIRONMENT VARIABLES to all tasks in the job.
-Figure 8 — Prompt form to add a job (advanced features)+ 
-Job parameters Job parameters cause SHELL ENVIRONMENT variables to be+;Pre- and Post-processing: This consists of commands that are executed before and/or after the job has run. The pre-processing consists of the at-submit command, the wait command and the pre-condition. These are executed in the order they appear. The post-processing consists of the at-end command.
-set which are available to all tasks in the job. Commonly they would+ 
-be included as $ variables which are passed as parameters to the+;At-submit command: An operating system command that executes when the job is submitted. This may be interactive if the job is manually submitted.
-commands in a task.+ 
-Fixed Parameters+;Wait condition: A boolean expression which can test the exit status of a job, exit status of a task or the return code from an arbitrary UNIX command or script. If defined, this expression must be satisfied (TRUE) before a job begins execution. The job will wait till the expression is satisfied. Form more information, see [[#Creating Wait Conditions]].
-Job parameters can be set to fixed values in the job configuration+ 
-Prompt form Job parameters can be set at job submission via a PROMPT form.+;Timeout (mins): This is the number of minutes before the wait condition times out. If the wait condition times out, the condition fails and the job is not started.
-The full path must be specified, unless the form is in+ 
-$APPL_HOME/local/prompt+;Pre-condition: A UNIX command, script or boolean expression which is executed before the job is run. The boolean expression is identical in syntax as the wait condition. However, a pre-condition returns simply SUCCESS or FAIL immediately, it does not wait. If the precondition fails, the job is not run. For more information, see [[#Creating Wait Conditions]].
-34 Setting Up Jobs+ 
-Pre- and Post-processing+;Check Syntax: Checks the syntax of the wait condition and pre-condition and notifies if the conditions are OK or have errors.
-This consists of commands that are executed before and/or after+ 
-the job has run. The pre-processing consists of the at-submit+;On failure: A job log file is created on the commencement of each job. This option specifies whether the log is kept or deleted if the wait condition or pre-condition fails.
-command, the wait command and the pre-condition. These are+ 
-executed in the order they appear. The post-processing consists of+;At-end command: An operating system command that executes when the job has ended. The ‘Status’ environment variable is set to the job’s exit status prior to calling this command. If the variable is 0, the job was successful.
-the at-end command.+ 
-At-submit command+;Clear status: When to clear the job status. Submit - every time the job is submitted. End - when the job finishes. Success - when a job has ended successfully. Never - the job status cannot be cleared other than manually from the Task monitor console, or programmatically via the FSclrstat program.
-An operating system command that executes when the job is+ 
-submitted. This may be interactive if the job is manually submitted.+;Concurrent tasks: The maximum number of tasks that can be run concurrently. If more than this number of tasks are ready to run, some will be queued. Leaving the field blank signifies that an unlimited number of tasks can be run concurrently.
-Wait condition A boolean expression which can test the exit status of a job, exit+ 
-status of a task or the return code from an arbitrary UNIX+;Priority: At what priority level the job will run at. This takes precedence over the queue priority.
-command or script. If defined, this expression must be satisfied+ 
-(TRUE) before a job begins execution. The job will wait till the+;Notes file: A file that contains extra information about a particular job which the operator can view. This field must contain a full path to the file, unless it is under <tt>$APPL_HOME/local/notes</tt>.
-expression is satisfied. Form more information,see Creating Wait+ 
-Conditions on page 49.+<br>
-Timeout (mins) This is the number of minutes before the wait condition times out.+ 
-If the wait condition times out, the condition fails and the job is+==== Notification Options ====
-not started.+Task3G can notify certain people either when this job requests operator assistance, or when it ends. For each of these, you can specify if and whom to notify, using the following fields:
-Pre-condition A UNIX command, script or boolean expression which is+;Notify: When and if to notify if the job fails or requires assistance. 'never' => don't perform this type of notification for this job; 'default' => notify the default people defined in the Task3G settings; 'specify' => notify the people defined in the 'Who' field below.
-executed before the job is run. The boolean expression is identical+;Who: Space separated list of people to notify. These can be COSmanager users, email addresses or mobile phone numbers (for SMS).
-in syntax as the wait condition. However, a pre-condition returns+ 
-simply SUCCESS or FAIL immediately, it does not wait. If the precondition+ 
-fails, the job is not run. For more information, see+{{note| If you specify a COSmanager user their notification method and address must be configured in the COSmanager 'Users' table}}
-Creating Wait Conditions on page 49.+{{note| The mechanics of the EMAIL and SMS notification should be configured in the COSmanager 'Notification Method' table}}
-Check Syntax Checks the syntax of the wait condition and pre-condition and+<br>
-notifies if the conditions are OK or have errors.+ 
-On failure A job log file is created on the commencement of each job. This+== How to Define a Task ==
-option specifies whether the log is kept or deleted if the wait+ 
-condition or pre-condition fails.+The building block of a job is the task. In effect, a task describes the 'what' and 'whether' of one component in a job. The 'What' part identifies an operating system command or script, and the name of the host on which it will run. The 'whether' part consists of pre- and post-conditions. These are UNIX commands or scripts that are run before and after the main UNIX command or script.
-At-end command+ 
-An operating system command that executes when the job has+Defining tasks can be accomplished from 2 areas, they are from the 'Task list' console and the 'Tasks in job' console.
-ended. The ‘Status’ environment variable is set to the job’s exit+ 
-status prior to calling this command. If the variable is 0, the job+Tasks defined in the 'Task list' console are placed in a pool of tasks which can be chosen when adding tasks to a job. Tasks defined in the 'Tasks in job' console are added to the job but are also placed in this pool, and so can later be added to other jobs.
-was successful.+ 
-Setting Up Jobs 35+<br>
-Clear status When to clear the job status. Submit - every time the job is+==== Configuration Console ====
-submitted. End - when the job finishes. Success - when a job has+ 
-ended successfully. Never - the job status cannot be cleared other+[[Image:Task img 8.jpg|frame|Figure 9 — Configuration console, or ‘Maintain Tasks’ window]]
-than manually from the Task monitor console, or+To access the 'Task list' console from the task3G configuration console select {{cnav|Tables > Tasks}}.
-programmatically via the FSclrstat program.+ 
-Concurrent tasks+To access the 'Tasks in job' console from the task3G Configuration console select {{cnav|Maintain > Tasks in Job}}.
-The maximum number of tasks that can be run concurrently. If+ 
-more than this number of tasks are ready to run, some will be+To access the 'task3G Jobs' console see [[#Task3G Configuration]].
-queued. Leaving the field blank signifies that an unlimited number+ 
-of tasks can be run concurrently.+The {{cnav|Maintain}} pulldown menu contains the standard Add, Change, Clone and Remove items which work the same way as maintaining other COSmanager tables. Generally, Double-Clicking an item automatically selects {{cnav|Change}}.
-Priority At what priority level the job will run at. This takes precedence+ 
-over the queue priority.+
-Notes file A file that contains extra information about a particular job which+
-the operator can view. This field must contain a full path to the file,+
-unless it is under $APPL_HOME/local/notes.+
-36 Setting Up Jobs+
-How to Define a Task+
-The building block of a job is the task. In effect, a task describes the 'what' and+
-'whether' of one component in a job. The 'What' part identifies an operating system+
-command or script, and the name of the host on which it will run. The 'whether'+
-part consists of pre- and post-conditions. These are UNIX commands or scripts+
-that are run before and after the main UNIX command or script.+
-Defining tasks can be accomplished from 2 areas, they are from the 'Task list' console+
-and the 'Tasks in job' console.+
-Tasks defined in the 'Task list' console are placed in a pool of tasks which can be+
-chosen when adding tasks to a job. Tasks defined in the 'Tasks in job' console are+
-added to the job but are also placed in this pool, and so can later be added to other+
-jobs.+
-Configuration Console+
-To access the 'Task list' console from the task3G configuration console select+
-Tables > Tasks.+
-To access the 'Tasks in job' console from the task3G Configuration console select+
-Maintain > Tasks in Job.+
-To access the 'task3G Jobs' console see COStask Configuration on page 27..+
-Note A step by step guide to defining a queue can be found in Tutorial 4:+
-Creating a Task on page 88.+
-Setting Up Jobs 37+
-Figure 9 — Configuration console, or ‘Maintain Tasks’ window+
-The Maintain pulldown menu contains the standard Add, Change, Clone and+
-Remove items which work the same way as maintaining other COSMmanager+
-tables. Generally, Double-Clicking an item automatically selects Change.+
The following is a list of actions for each menu item: The following is a list of actions for each menu item:
-Creating a new task is done by selecting Maintain > Add.+*Creating a new task is done by selecting {{cnav|Maintain > Add}}.
-Changing the details of a task is done by selecting Maintain > Changing.+*Changing the details of a task is done by selecting {{cnav|Maintain > Change}}. Double-Clicking on the task is the equivalent to {{cnav|Maintain > Change}}.
-Double-Clicking on the task is the equivalent to Maintain >+*Copying a task is done by selecting {{cnav|Maintain > Clone}}.
-Change.+*Deleting a task is done by selecting {{cnav|Maintain > Remove}}.
-Copying a task is done by selecting Maintain > Clone.+ 
-Deleting a task is done by selecting Maintain > Remove.+<br>
-38 Setting Up Jobs+ 
-To Define Information About a Task+=== To Define Information About a Task ===
-Defining a task is split into two entry forms. The first comprises of the basic options+ 
-needed for each task, the latter comprises of advanced options.+Defining a task is split into two entry forms. The first comprises of the basic options needed for each task, the latter comprises of advanced options.
-Figure 10 — Prompt form to add a task+ 
-Task name The task name identifies each individual task. This must be a+[[Image:Task img 9.jpg|frame|Figure 10 — Prompt form to add a task]]
-unique identifier.+;Task name: The task name identifies each individual task. This must be a unique identifier.
-Command This is the command the task executes. This can be a UNIX+ 
-command or a script. Also referred to as the main command.+;Command: This is the command the task executes. This can be a UNIX command or a script. Also referred to as the main command. Leave this field blank if the task is to be a subjob.
-Leave this field blank if the task is to be a subjob.+ 
-Run on host This is the host on which the command will run. If left blank it will+;Run on host: This is the host on which the command will run. If left blank it will run on the default host of the job.
-run on the default host of the job.+ 
-As user Under which UNIX user the command or script will execute. This+;As user: Under which UNIX user the command or script will execute. This is currently ignored on Windows NT hosts.
-is currently ignored on Windows NT hosts.+ 
-Subjob This is where you specify a pre-defined subjob to run within the+;Subjob: This is where you specify a pre-defined subjob to run within the task. The task will run the subjob, thus giving the ‘job within a job’ effect.
-task. The task will run the subjob, thus giving the ‘job within a job’+ 
-effect.+;Comments: A description or comment about the task.
-Comments A description or comment about the task.+ 
-Setting Up Jobs 39+<br>
-Advanced options+==== Advanced options ====
-You can configure advanced options such as pre- and post-conditions, time-outs,+ 
-recover actions and operator privileges. Press Advanced to enter one or more of+You can configure advanced options such as pre- and post-conditions, time-outs, recover actions and operator privileges. Press {{cnav|Advanced}} to enter one or more of these command or options.
-these command or options.+ 
-Figure 11 — Prompt form to add a task (advanced options)+[[Image:Task img 10.jpg|frame|Figure 11 — Prompt form to add a task (advanced options)]]
-Pre- and Post-processing+;Pre- and Post-processing: This consists of commands that are executed before and/or after the main command. These can be operating system commands or script.
-This consists of commands that are executed before and/or after+ 
-the main command. These can be operating system commands or+;Pre-condition: A UNIX command, script or boolean expression which is executed before the main command is run. The boolean expression is identical in syntax as the wait condition. However, a pre-condition returns simply SUCCESS or FAIL immediately, it does not wait. If it succeeds, the main command is run, if it fails, neither the main command nor any post-processing is done and the result is dependent on what job recovery action is specified for 'On pre-cond fail'. For more information, see [[#Creating Wait Conditions]].
-script.+ 
-40 Setting Up Jobs+;Check Syntax: Checks the syntax of the wait condition and notifies if the condition is OK or has an error.
-Pre-condition A UNIX command, script or boolean expression which is+ 
-executed before the main command is run. The boolean+;On-success cmd: The on-success command is executed after the main command succeeds, it will not execute if it fails. If the ‘On-success’ command succeeds, the task completes successfully. If it fails, the result is dependent on what job recovery action is specified for 'On error'.
-expression is identical in syntax as the wait condition. However, a+ 
-pre-condition returns simply SUCCESS or FAIL immediately, it+;On-error cmd: The on-error command is executed after the main command fails, it will not execute if it succeeds. If the ‘On-error’ command fails, the task fails. If it succeeds, the result is dependent on what job recovery action is specified for 'On-error'.
-does not wait. If it succeeds, the main command is run, if it fails,+ 
-neither the main command nor any post-processing is done and+;Override status: This field only applies if an ‘On-error command’ is specified. If this field is ‘yes’ and the On-error command succeeds, the task completes successfully. This will set the completed tasks status to SUCCESSFUL even though the main command FAILED. If ‘no’ the status of the task will be set to FAIL even if On-error completes successfully.
-the result is dependent on what job recovery action is specified for+ 
-'On pre-cond fail'. For more information, see Creating Wait+;Timeout: This is the number of minutes before the task times out. If the task executes for longer than this time, the task is killed and the result is dependent on what job recovery action is specified for 'On timeout'.
-Conditions on page 49.+ 
-Check Syntax Checks the syntax of the wait condition and notifies if the+<br>
-condition is OK or has an error.+ 
-On-success cmd+==== Job recovery actions ====
-The on-success command is executed after the main command+ 
-succeeds, it will not execute if it fails. If the ‘On-success’ command+These recovery actions are triggered on command and timeout failures. There are one of four possible actions: assist, stop, continue and ignore.
-succeeds, the task completes successfully. If it fails, the result is+ 
-dependent on what job recovery action is specified for 'On error'.+:;''assist'': enter assist mode on error. The operator then decides the action.
-On-error cmd The on-error command is executed after the main command fails,+:;''stop'': ends the task on error and don't run any of its dependencies. An error status is raised in the job.
-it will not execute if it succeeds. If the ‘On-error’ command fails,+:;''continue'': ends the task and runs the dependencies. An error status is raised in the job.
-the task fails. If it succeeds, the result is dependent on what job+:;''ignore'': ends the task and runs the dependencies. No error status is raised in the job.
-recovery action is specified for 'On-error'.+ 
-Override status This field only applies if an ‘On-error command’ is specified. If+;On error: This recovery action is triggered when post-processing fails. If no post-processing is specified, then the 'on error' recovery action will handle main command failures.
-this field is ‘yes’ and the On-error command succeeds, the task+ 
-completes successfully. This will set the completed tasks status to+;On pre-cond fail: This recovery action is triggered when the pre-condition fails.
-SUCCESSFUL even though the main command FAILED. If ‘no’+ 
-the status of the task will be set to FAIL even if On-error+;On kill: This recovery action is triggered when the task is killed via the task monitor console.
-completes successfully.+ 
-Timeout This is the number of minutes before the task times out. If the task+;On timeout: This recovery action is triggered when the task times out.
-executes for longer than this time, the task is killed and the result+ 
-is dependent on what job recovery action is specified for 'On+<br>
-timeout'.+==== Operator Privileges ====
-Setting Up Jobs 41+ 
-Job recovery actions+The following options allow or disallow the operator from performing certain functions on this task:
-These recovery actions are triggered on command and timeout failures. There are+ 
-one of four possible actions: assist, stop, continue and ignore.+;Can re-run: If the task goes into 'assist mode', this determines whether the operator can re-run the task.
-assist enter assist mode on error. The operator then decides the action.+ 
-stop ends the task on error and don't run any of its dependencies. An+;Can continue: If the task goes into 'assist mode', this determines whether the operator can allow the task's dependencies to run.
-error status is raised in the job.+ 
-continue ends the task and runs the dependencies. An error status is raised+;Can kill: Whether the operator can kill (cancel) this task after it has started.
-in the job.+ 
-ignore ends the task and runs the dependencies. No error status is raised+;Notes file: A file that contains extra information about a particular task which the operator can view. This field must contain a full path to the file, unless it is under <tt>$APPL_HOME/local/notes</tt>.
-in the job.+ 
-On error This recovery action is triggered when post-processing fails. If no+<br>
-post-processing is specified, then the 'on error' recovery action will+ 
-handle main command failures.+== Inserting Tasks into a Job ==
-On pre-cond fail+ 
-This recovery action is triggered when the pre-condition fails.+Populating a job with tasks and subjobs is done from the 'Tasks in job' console. Also from this console, dependencies and wait conditions can be defined. The console displays the number of tasks and subjobs contained within the job and its dependencies.
-On kill This recovery action is triggered when the task is killed via the task+ 
-monitor console.+
-On timeout This recovery action is triggered when the task times out.+
-Operator Privileges+
-The following options allow or disallow the operator from performing certain functions+
-on this task:+
-Can re-run If the task goes into 'assist mode', this determines whether the+
-operator can re-run the task.+
-Can continue If the task goes into 'assist mode', this determines whether the+
-operator can allow the task's dependencies to run.+
-Can kill Whether the operator can kill (cancel) this task after it has started.+
-42 Setting Up Jobs+
-Notes file A file that contains extra information about a particular task which+
-the operator can view. This field must contain a full path to the file,+
-unless it is under $APPL_HOME/local/notes.+
-Setting Up Jobs 43+
-Inserting Tasks into a Job+
-Populating a job with tasks and subjobs is done from the ''Tasks in job' console.+
-Also from this console, dependencies and wait conditions can be defined. The console+
-displays the number of tasks and subjobs contained within the job and its+
-dependencies.+
From the 'task3G Jobs' console, select Maintain > Tasks in job. From the 'task3G Jobs' console, select Maintain > Tasks in job.
-To access the 'task3G Jobs' console, see COStask Configuration on page 27.+ 
-Figure 12 — 'Tasks in Job' console+To access the 'task3G Jobs' console, see [[#Task3G Configuration]].
 + 
 +[[Image:Task img 11.jpg|frame|Figure 12 — 'Tasks in Job' console]]
The following is a list of the menus and an explanation of each: The following is a list of the menus and an explanation of each:
-Maintain This menu deals with creating and adding tasks to a job as well as+ 
-maintaining tasks.+;Maintain: This menu deals with creating and adding tasks to a job as well as maintaining tasks.
-Dependencies This menu deals with creating and checking dependencies as well+ 
-as creating wait conditions.+;Dependencies: This menu deals with creating and checking dependencies as well as creating wait conditions.
-View This menu deals with viewing jobs.+ 
-Adding Tasks to a Job and Task Maintenance.+;View: This menu deals with viewing jobs.
-Adding tasks to a job is done via the maintain menu. Selecting Maintain > Add,+ 
-opens a window containing all pre-defined tasks. From this window, selecting multiple+<br>
-tasks will add them to the job. If you want the new tasks to be dependent on an+=== Adding Tasks to a Job and Task Maintenance ===
-existing task, highlight that task before selecting Add.+ 
-The 'Tasks in job' console also allows the user to add new tasks to a job without+Adding tasks to a job is done via the maintain menu. Selecting {{cnav|Maintain > Add}}, opens a window containing all pre-defined tasks. From this window, selecting multiple tasks will add them to the job. If you want the new tasks to be dependent on an existing task, highlight that task before selecting {{cnav|Add}}.
-having to create them via the 'Tasks' console. This is done by selecting Maintain+ 
-> New. If you want the new tasks to be dependent on an existing task, highlight that+The 'Tasks in job' console also allows the user to add new tasks to a job without having to create them via the 'Tasks' console. This is done by selecting {{cnav|Maintain > New}}. If you want the new tasks to be dependent on an existing task, highlight that task before selecting {{cnav|New}}.
-task before selecting New.+ 
-Tasks can be changed or deleted by selecting Maintain > Change and Maintain+Tasks can be changed or deleted by selecting {{cnav|Maintain > Change}} and {{cnav|Maintain > Remove}} respectively. Double-Clicking on tasks is the equivalent to {{cnav|Maintain > Change}}.
-> Remove respectively. Double-Clicking on tasks is the equivalent to+ 
-Maintain > Change.+The 'Task' console can be accessed by selecting {{cnav|Maintain > Tasks}}.
-The 'Task' console can be accessed by selecting Maintain > Tasks.+ 
-Creating Dependencies+<br>
-With no dependencies defined, a task3G job would start as many of its tasks in parallel+ 
-as is possible. The maximum number of concurrent tasks can be defined in the+=== Creating Dependencies ===
-job definition. Usually you want to define dependencies so that a task is not started+ 
-until certain other 'parent' tasks have finished.+With no dependencies defined, a task3G job would start as many of its tasks in parallel as is possible. The maximum number of concurrent tasks can be defined in the job definition. Usually you want to define dependencies so that a task is not started until certain other 'parent' tasks have finished.
-Dependent tasks are displayed indented, under their parent with a '+' sign aligned+ 
-with their parent.+Dependent tasks are displayed indented, under their parent with a '+' sign aligned with their parent.
 + 
Once all tasks have been added to the job, creating dependencies is the next step. Once all tasks have been added to the job, creating dependencies is the next step.
-This is done using the following items found under the Dependencies menu:+This is done using the following items found under the {{cnav|Dependencies}} menu:
-Setting Up Jobs 45+ 
-Cut By selecting one or more tasks and then selecting+;Cut: By selecting one or more tasks and then selecting {{cnav|Dependencies > Cut}}, the task(s) are removed from the job and placed into a temporary buffer.
-Dependencies > Cut, the task(s) are removed from the job+ 
-and placed into a temporary buffer.+;Link: By selecting one or more tasks and then selecting {{cnav|Dependencies > Link}}, the task(s) are placed into a temporary buffer. They are not removed from the job.
-Link By selecting one or more tasks and then selecting+ 
-Dependencies > Link, the task(s) are placed into a+;Paste: By selecting {{cnav|Dependencies > Paste}}, the task(s) in the buffer are added to the job. If a task was highlighted prior to pasting, the buffered task(s) are pasted under the highlighted task, making them dependent on the highlighted task. The buffer is then cleared.
-temporary buffer. They are not removed from the job.+ 
-Paste By selecting Dependencies > Paste, the task(s) in the buffer+;Sequence: By highlighting multiple tasks and selecting {{cnav|Dependencies > Sequence}}, a sequence of dependent tasks is created. A window appears containing the selected tasks allowing you to put them in the desired order. This is achieved by 'dragging' the tasks up and down in this window. When the correct order has been achieved, push the {{cnav|Accept}} button and a sequence of dependent tasks is created.
-are added to the job. If a task was highlighted prior to pasting, the+ 
-buffered task(s) are pasted under the highlighted task, making+ 
-them dependent on the highlighted task. The buffer is then+'''Example 1: one task dependent on another'''
-cleared.+ 
-Sequence By highlighting multiple tasks and selecting Dependencies >+::[[Image:Task img 12.jpg|Task img 12.jpg]]
-Sequence, a sequence of dependent tasks is created. A window+#Add tasks <tt>Update_db</tt> (task1) and <tt>Reports</tt> (task2) to a job. The window contents of the 'Tasks in job' console should appear as the following:
-appears containing the selected tasks allowing you to put them in+#::<tt>Reports
-the desired order. This is achieved by 'dragging' the tasks up and+#::Update_db</tt>
-down in this window. When the correct order has been achieved,+#Next, create a dependency making task Reports dependent on task <tt>Update_db</tt>. Select Reports.
-push the Accept button and a sequence of dependent tasks is+#Select {{cnav|Dependencies > Cut}}
-created.+#Select <tt>Update_db</tt>
-Example 1: one task dependent on another.+#Select {{cnav|Dependencies > Paste}}. The window contents of the 'Tasks in job' console should appear as the following:
-1. Add tasks Update_db (task1) and Reports (task2) to a job.+#::<tt>Update_db
-The window contents of the 'Tasks in job' console should appear as the following:+#::+ Reports</tt>
-Reports+ 
-Update_db+:When run, task3G will start <tt>Reports</tt> once <tt>Update_db</tt> has completed.
-2. Next, create a dependency making task Reports dependent on task+ 
-Update_db. Select Reports.+ 
-46 Setting Up Jobs+'''Example 2: two tasks dependent on one'''
-3. Select Dependencies > Cut+ 
-4. Select Update_db+::[[Image:Task img 13.jpg|Task img 13.jpg]]
-5. Select Dependencies > Paste+#Add tasks <tt>Update_db</tt>, <tt>Reports</tt> and <tt>Cleanup</tt> to a job. The window contents of the 'Tasks in job' console should appear as the following:
-The window contents of the 'Tasks in job' console should appear as the following:+#::<tt>Cleanup
-Update_db+#::Reports
-+ Reports+#::Update_db</tt>
-When run, task3G will start Reports once Update_db has completed.+#Next, create a dependency making task <tt>Reports</tt> and <tt>Cleanup</tt> both dependent on task <tt>Update_db</tt>. Select both <tt>Reports</tt> and <tt>Cleanup</tt>.
-Example 2: two tasks dependent on one.+#Select {{cnav|Dependencies > Cut}}
-1. Add tasks Update_db, Reports and Cleanup to a job.+#Select Update_db
-The window contents of the 'Tasks in job' console should appear as the following:+#Select {{cnav|Dependencies > Paste}}. The window contents of the 'Tasks in job' console should appear as the following:
-Cleanup+#::<tt>Update_db
-Reports+#::+ Reports
-Update_db+#::+ Cleanup</tt>
-2. Next, create a dependency making task Reports and Cleanup both+ 
-dependent on task Update_db. Select both Reports and Cleanup.+:When run, task3G will start both <tt>Reports</tt> and <tt>Cleanup</tt> once <tt>Update_db</tt> has completed.
-3. Select Dependencies > Cut+ 
-4. Select Update_db+ 
-5. Select Dependencies > Paste+'''Example 3: one task dependent on two'''
-The window contents of the 'Tasks in job' console should appear as the followSetting+ 
-Up Jobs 47+::[[Image:Task img 14.jpg|Task img 14.jpg]]
-ing:+#Add tasks <tt>Update_db</tt>, <tt>Reports</tt> and <tt>Cleanup</tt> to a job. The window contents of the 'Tasks in job' console should appear as the following:
-Update_db+#::<tt>Cleanup
-+ Reports+#::Reports
-+ Cleanup+#::Update_db</tt>
-When run, task3G will start both Reports and Cleanup once Update_db has+#Next, create a dependency making task <tt>Cleanup</tt> dependent on both <tt>Update_db</tt> and <tt>Reports</tt> tasks. Select <tt>Cleanup</tt>.
-completed.+#Select {{cnav|Dependencies > Cut}}
-Example 3: one task dependent on two.+#Select <tt>Update_db</tt>
-1. Add tasks Update_db, Reports and Cleanup to a job.+#Select {{cnav|Dependencies > Paste}}
-The window contents of the 'Tasks in job' console should appear as the following:+#Select <tt>Cleanup</tt> again.
-Cleanup+#Select {{cnav|Dependencies > Link}}
-Reports+#Select <tt>Reports</tt>
-Update_db+#Select {{cnav|Dependencies > Paste}}. The window contents of the 'Tasks in job' console should appear as the following:
-2. Next, create a dependency making task Cleanup dependent on both+#::<tt>Update_db
-Update_db and Reports tasks. Select Cleanup.+#::+ Cleanup
-3. Select Dependencies > Cut+#::Reports
-4. Select Update_db+#::+ Cleanup</tt>
-5. Select Dependencies > Paste+ 
-6. Select Cleanup again.+:With the second <tt>Cleanup</tt> flagged as a link. When run, task3G will start <tt>Cleanup</tt> once both <tt>Update_db</tt> and <tt>Reports</tt> have completed.
-7. Select Dependencies > Link+ 
-8. Select Reports+ 
-9. Select Dependencies > Paste+'''Example 4: a sequence of dependent tasks'''
-48 Setting Up Jobs+ 
-The window contents of the 'Tasks in job' console should appear as the following:+::[[Image:Task img 15.jpg|Task img 15.jpg]]
-Update_db+#Add tasks <tt>Update_db</tt>, <tt>Reports</tt> and <tt>Cleanup</tt> to a job. The window contents of the 'Tasks in job' console should appear as the following:
-+ Cleanup+#::<tt>Cleanup
-Reports+#::Reports
-+ Cleanup+#::Update_db</tt>
-With the second Cleanup flagged as a link. When run, task3G will start Cleanup+#Next, create a dependency making task <tt>Reports</tt> dependent on <tt>Update_db</tt> and <tt>Cleanup</tt> dependent on Reports tasks. Select all tasks.
-once both Update_db and Reports have completed.+#Select {{cnav|Dependencies > Sequence}}
-Example 4: a sequence of dependent tasks.+#Drag Update_db above <tt>Cleanup</tt> (top of the display)
-1. Add tasks Update_db, Reports and Cleanup to a job.+#Either drag <tt>Reports</tt> above <tt>Cleanup</tt> or drag <tt>Cleanup</tt> below <tt>Reports</tt>
-The window contents of the 'Tasks in job' console should appear as the following:+#Press {{cnav|Accept}} to save this ordering of tasks. The window contents of the 'Tasks in job' console should appear as the following:
-Cleanup+#::<tt>Update_db
-Reports+#::+ Reports
-Update_db+#::+ Cleanup</tt>
-2. Next, create a dependency making task Reports dependent on+ 
-Update_db and Cleanup dependent on Reports tasks. Select all tasks.+:When run, task3G will run <tt>Update_db</tt>, then <tt>Reports</tt>, then <tt>Cleanup</tt>.
-3. Select Dependencies > Sequence+ 
-4. Drag Update_db above Cleanup (top of the display)+<br>
-5. Either drag Reports above Cleanup or drag Cleanup below Reports+ 
-Setting Up Jobs 49+=== Checking Dependencies ===
-6. Press Accept to save this ordering of tasks.+ 
-The window contents of the 'Tasks in job' console should appear as the following:+When constructing complex dependencies that involve a large number of tasks, it is possible to unknowingly create deadlocks and/or unnecessary dependencies.
-Update_db+ 
-+ Reports+Deadlocks are basically cyclic errors. Deadlocks occur when two or more tasks are dependent on each other, eg. task3 is dependent on task2, which is dependent on task1, which is dependent on task3. It is impossible for task1, task2 or task3 to start.
-+ Cleanup+ 
-When run, task3G will run Update_db, then Reports, then Cleanup.+Unnecessary dependencies are basically redundant dependencies, they are dependencies that are not needed.
-Checking Dependencies+ 
-When constructing complex dependencies that involve a large number of tasks, it is+These conditions will cause the offending task to appear in the monitor console colored either red or orange.
-possible to unknowingly create deadlocks and/or unnecessary dependencies.+ 
-Deadlocks are basically cyclic errors. Deadlocks occur when two or more tasks are+To fix these conditions, select {{cnav|Dependencies > Optimise}}.
-dependent on each other, e.g. task3 is dependent on task2, which is dependent on+ 
-task1, which is dependent on task3. It is impossible for task1, task2 or task3 to start.+<br>
-Unnecessary dependencies are basically redundant dependencies, they are dependencies+ 
-that are not needed.+=== Creating Wait Conditions ===
-These conditions will cause the offending task to appear in the monitor console colored+ 
-either red or orange.+'Wait For' conditions are defined against a task in a job. To create a wait for condition, select a task then select {{cnav|Dependencies > Wait for}}.
-To fix these conditions, select Dependencies > Optimise.+ 
-Creating Wait Conditions+[[Image:Task img 16.jpg|frame|Figure 13 — Wait for condition]]
-'Wait For' conditions are defined against a task in a job. To create a wait for condition,+;Wait condition: A boolean expression which can test the exit status of a job, exit status of a task or the return code from an arbitrary UNIX command or script. If defined, this expression must be satisfied (TRUE) before a task begins execution. The task will wait till the expression is satisfied.
-select a task then select Dependencies > Wait for.+ 
-Note A step by step guide to defining a wait condition can be found in+ 
-Tutorial 6: Creating Wait Conditions on page 94.+
-50 Setting Up Jobs+
-Figure 13 — Wait for condition+
-Wait condition A boolean expression which can test the exit status of a job, exit+
-status of a task or the return code from an arbitrary UNIX+
-command or script. If defined, this expression must be satisfied+
-(TRUE) before a task begins execution. The task will wait till the+
-expression is satisfied.+
The wait condition uses the following symbols: The wait condition uses the following symbols:
-Normal precedence for boolean expressions applies and brackets can be used. Primary+ 
-expressions can be:+:{| border="0" cellpadding="3" cellspacing="0"
-• A job or task name of the form+! align="left" width="75" style="border-bottom:1px solid grey;border-right:1px solid grey;" | Symbol
-Symbol Meaning+! align="left" width="250" style="border-bottom:1px solid grey;" | Meaning
-|| OR+|-
-&& AND+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | &brvbar;&brvbar;
-== EQUAL TO+| style="border-bottom:1px solid grey;" | OR
-!= NOT EQUAL TO+|-
-> GREATER THAN+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | &&
-< LESS THAN+| style="border-bottom:1px solid grey;" | AND
->= GREATER THAN OR EQUAL TO+|-
-<= LESS THAN OR EQUAL TO+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | ==
-! NOT+| style="border-bottom:1px solid grey;" | EQUAL TO
-Setting Up Jobs 51+|-
-$name where 'name' is a task in the current job.+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | !=
-$job where 'job' is another task3G job.+| style="border-bottom:1px solid grey;" | NOT EQUAL TO
-$job:name where 'name' is a task in job 'job'.+|-
-You can extend the above form by testing for a particular exit status, using+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | >
-the numeric operators listed above.+| style="border-bottom:1px solid grey;" | GREATER THAN
-Example: $step3 == 5+|-
-This expression will wait for the task or job to finish, then either return+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | <
-TRUE or FALSE.+| style="border-bottom:1px solid grey;" | LESS THAN
-An command (including a shell test [ ] command). This will be run periodically+|-
-until it returns TRUE (zero exit status).+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | >=
-Note that if a shell command returns a non-zero status, the condition will+| style="border-bottom:1px solid grey;" | GREATER THAN OR EQUAL TO
-WAIT for 60 seconds and try again. It will not return FALSE.+|-
-task3G uses the following convention for both task and job statuses:+| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | <=
 +| style="border-bottom:1px solid grey;" | LESS THAN OR EQUAL TO
 +|-
 +| valign="top" style="border-right:1px solid grey;" | !
 +| NOT
 +|}
 + 
 + 
 +Normal precedence for boolean expressions applies and brackets can be used. Primary expressions can be:
 +*A job or task name of the form
 +*:'''''<tt>$name'''''&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt> where 'name' is a task in the current job.
 +*:'''''<tt>$job'''''&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt> where 'job' is another task3G job.
 +*:'''''<tt>$job:name'''''&nbsp;</tt> where 'name' is a task in job 'job'.
 + 
 +*You can extend the above form by testing for a particular exit status, using the numeric operators listed above.
 +*:Example: <code>$step3 == 5</code>
 +*:This expression will wait for the task or job to finish, then either return TRUE or FALSE.
 + 
 +*An command (including a shell test <tt>[ ]</tt> command). This will be run periodically until it returns TRUE (zero exit status).
 +{{note| that if a shell command returns a non-zero status, the condition will WAIT for 60 seconds and try again. It will not return FALSE.}}
 + 
 + 
 +Task3G uses the following convention for both task and job statuses:
 + 
 +:{| border="0" cellpadding="3" cellspacing="0"
 +! align="left" width="150" style="border-bottom:1px solid grey;border-right:1px solid grey;" | Numeric Exit Status
 +! align="left" width="150" style="border-bottom:1px solid grey;" | Symbol
 +|-
 +| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | 0
 +| style="border-bottom:1px solid grey;" | OK
 +|-
 +| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | 1
 +| style="border-bottom:1px solid grey;" | ERR[OR]
 +|-
 +| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | 2
 +| style="border-bottom:1px solid grey;" | WARN[ING]
 +|-
 +| valign="top" style="border-bottom:1px solid grey;border-right:1px solid grey;" | 10
 +| style="border-bottom:1px solid grey;" | KILL[ED]
 +|}
 + 
 + 
The following are some examples of wait conditions: The following are some examples of wait conditions:
-Example. $step3 == 0+ 
-Will wait for task 'step3' in the current job to finish, and when it does it will+:Example: <code>$step3 == 0</code>
-return TRUE or FALSE depending whether its exit status is 0.+::Will wait for task 'step3' in the current job to finish, and when it does it will return TRUE or FALSE depending whether its exit status is 0.
-Example. $step3 == KILLED+ 
-Will wait for task 'step3' in the current job to finish, and when it does it will+:Example: <code>$step3 == KILLED</code>
-return TRUE or FALSE depending whether its exit status is 10 (killed).+::Will wait for task 'step3' in the current job to finish, and when it does it will return TRUE or FALSE depending whether its exit status is 10 (killed).
-Example. $step3 == ERR+ 
-Will wait for task 'step3' in the current job to finish, and when it does it will+:Example: <code>$step3 == ERR</code>
-Numeric Exit Status Symbol+::Will wait for task 'step3' in the current job to finish, and when it does it will return TRUE or FALSE depending whether its exit status is 1 (error).
-0 OK+ 
-1 ERR[OR]+:Example: <code>$job2:step3 < 2</code>
-2 WARN[ING]+::Will wait for task 'step3' in job 'job2' to finish, then check its exit status is less than 2.
-10 KILL[ED]+ 
-52 Setting Up Jobs+:Example: <code>$job2:step3 && [ -f /tmp/in_data ]</code>
-return TRUE or FALSE depending whether its exit status is 1 (error).+::Will wait for task 'step3' in job 'job2' to finish and for a file <tt>/tmp/in_data</tt> to exist.
-Example. $job2:step3 < 2+ 
-Will wait for task 'step3' in job 'job2' to finish, then check its exit status is+ 
-less than 2.+A task_name (or job_name) by itself will simply wait for the task or job to terminate, then return TRUE.
-Example. $job2:step3 && [ -f /tmp/in_data ]+ 
-Will wait for task 'step3' in job 'job2' to finish and for a file /tmp/in_data to+[[Image:Task img 17.jpg|frame|Figure 14 — Check syntax report]]
-exist.+;Check Syntax: Checks the syntax of the wait condition and notifies if the condition is OK or has an error.
-A task_name (or job_name) by itself will simply wait for the task or job to terminate,+ 
-then return TRUE.+;On failure: If the wait condition fails, i.e. evaluates to FALSE, the task will either go into assist mode.
-Check Syntax Checks the syntax of the wait condition and notifies if the+:;'''''assist''''': enter assist mode on failure. The operator then decides the action.
-condition is OK or has an error.+:;'''''stop''''': stops the task on failure and doesn't run any dependencies. An
-Figure 14 — Check syntax report+:;'''''error''''': status is raised in the job.
-On failure If the wait condition fails, i.e. evaluates to FALSE, the task will+ 
-either go into assist mode.+<br>
-assist enter assist mode on failure. The operator then decides the action.+ 
-stop stops the task on failure and doesn't run any dependencies. An+=== Viewing other Jobs and Subjobs ===
-error status is raised in the job.+ 
-Setting Up Jobs 53+Using the View menu items, other jobs and subjobs can be selected for viewing via the 'Tasks in job' console. The following is an explanation of each item:
-Viewing other Jobs and Subjobs+ 
-Using the View menu items, other jobs and subjobs can be selected for viewing via+;Parent job: when viewing a subjob, selecting {{cnav|View > Parent}} job will view the tasks within the parent job.
-the 'Tasks in job' console. The following is an explanation of each item:+;Subjob: selecting a subjob and selecting {{cnav|View > Subjob}}, will view the tasks within the subjob.
-Parent job when viewing a subjob, selecting View > Parent job will view+;Another job: selecting {{cnav|View > Another job}} will view the tasks within the next job in alphabetical order.
-the tasks within the parent job.+;Show notes: selecting {{cnav|View > Show notes}} will view the task notes.
-Subjob selecting a subjob and selecting View > Subjob, will view the+ 
-tasks within the subjob.+<br>
-Another job selecting View > Another job will view the tasks within the+ 
-next job in alphabetical order.+== How to Define a Job Queue ==
-Show notes selecting View > Show notes will view the task notes.+ 
-54 Setting Up Jobs+Task3G jobs can be submitted to the 'at' queue, 'batch' queue, 'task' queue or to a user defined job queue. The 'cron' queue is reserved for cron jobs, it is not available for job submission.
-How to Define a Job Queue+ 
-task3G jobs can be submitted to the 'at' queue, 'batch' queue, 'task' queue or to a+Job queues are used to stream jobs with similar characteristics. For each queue, you can define a default priority and the maximum number of concurrent jobs. The queue will only release up to this number of jobs to run at the same time. Once one of these jobs finishes, the next job in that queue is started. The priority determines the job's relative share of the CPU; it equates to the UNIX nice value.
-user defined job queue. The 'cron' queue is reserved for cron jobs, it is not available+ 
-for job submission.+Each queue is identified by a letter, for example 'a' for the AT_queue, 'b' for the BATCH_queue, 't' for the TASK queue. d - z are available for user-defined queues. 'c' is reserved for cron jobs.
-Job queues are used to stream jobs with similar characteristics. For each queue, you+{{note| Some operating systems do not support all queues d-z. Also some operating systems do not support specifying the maximum number of concurrent jobs.}}
-can define a default priority and the maximum number of concurrent jobs. The+ 
-queue will only release up to this number of jobs to run at the same time. Once one+<br>
-of these jobs finishes, the next job in that queue is started. The priority determines+=== Configuration Console ===
-the job's relative share of the CPU; it equates to the UNIX nice value.+ 
-Each queue is identified by a letter, for example 'a' for the AT_queue, 'b' for the+The task3G Job Queue Details console is where all configuration concerning queues is accomplished. This is access via the task3G Jobs console. From the task3G Jobs console, select {{cnav|Tables > Queues}}.
-BATCH_queue, 't' for the TASK queue. d - z are available for user-defined queues.+ 
-'c' is reserved for cron jobs.+[[Image:Task img 18.jpg|frame|Figure 15 — The task3G Job Queue Details]]
-Note Some operating systems do not support all queues d-z. Also some+To access the task3G Jobs console, see [[#Task3G Configuration]].
-operating systems do not support specifying the maximum number of+ 
-concurrent jobs.+Figure 15 shows all defined queues.
-Note A step by step guide to defining a queue can be found in Tutorial 1:+ 
-Creating a Queue on page 83.+The {{cnav|Maintain}} pulldown menu contains the standard Add, Change, Clone and Remove items which work the same way as maintaining other COSmanager tables. Generally, Double-Clicking an item automatically selects {{cnav|Change}}.
-Configuration Console+ 
-The task3G Job Queue Details console is where all configuration concerning queues+
-is accomplished. This is access via the task3G Jobs console. From the task3G Jobs+
-console, select Tables > Queues+
-Setting Up Jobs 55+
-To access the task3G Jobs console, see COStask Configuration on page 27.+
-Figure 15 — The task3G Job Queue Details+
-The above screen shows all defined queues.+
-The Maintain pulldown menu contains the standard Add, Change, Clone and+
-Remove items which work the same way as maintaining other COSMmanager+
-tables. Generally, Double-Clicking an item automatically selects Change.+
The following is a list of actions for each menu item: The following is a list of actions for each menu item:
-Creating a new queue is done by selecting Maintain > Add.+*Creating a new queue is done by selecting {{cnav|Maintain > Add}}.
-Changing the details of a queue is done by selecting Maintain >+*Changing the details of a queue is done by selecting {{cnav|Maintain > Change}}. Double-Clicking on the task is the equivalent to {{cnav|Maintain > Change}}.
-Change. Double-Clicking on the task is the equivalent to Maintain >+*Copying a queue is done by selecting {{cnav|Maintain > Clone}}.
-Change.+*Deleting a queue is done by selecting {{cnav|Maintain > Remove}}.
-Copying a queue is done by selecting Maintain > Clone.+ 
-Deleting a queue is done by selecting Maintain > Remove.+ 
-To define a new queue:+==== To define a new queue ====
-Figure 16 — Defining a task3G queue+ 
-Queue This is the unique name of the queue which is used as the+[[Image:Task img 19.jpg|frame|Figure 16 — Defining a task3G queue]]
-identifier.+;Queue: This is the unique name of the queue which is used as the identifier.
-Description A description or comment about the queue.+;Description: A description or comment about the queue.
-UNIX queue The UNIX at queue name to be used with this task3G queue. You+;UNIX queue: The UNIX at queue name to be used with this task3G queue. You can normally use up to 25 job queues. Jobs can be submitted to the standard system queues provided, such as the at queue (a), batch queue (b) and the task queue (t). Another 22 queues may be defined with the letters, d through to z, though some operating systems may restrict this range.
-can normally use up to 25 job queues. Jobs can be submitted to the+{{note| Jobs can't be submitted to the c queue as it is reserved for use by cron.}}
-standard system queues provided, such as the at queue (a), batch+{{note| The number of queues that can be defined may be restricted in some versions of UNIX.}}
-queue (b) and the task queue (t). Another 22 queues may be+;Concurrent jobs: The maximum number of jobs that can run concurrently in this queue.
-defined with the letters, d through to z, though some operating+;Re-scheduled time: The number of seconds between checks to determine if new jobs can be launched.
-systems may restrict this range.+;Priority: The priority determines the job's relative share of the CPU; it equates to the UNIX nice value.
-Note Jobs can't be submitted to the c queue as it is reserved for use by cron.+{{note| Remember that the lower the number, the higher the priority. -19 is the highest priority, +19 being the lowest.}}
-Note The number of queues that can be defined may be restricted in some+ 
-versions of UNIX.+<br>
-Concurrent jobs The maximum number of jobs that can run concurrently in this+ 
-queue.+== Adding a Schedule ==
-Setting Up Jobs 57+ 
-Re-scheduled time+Schedules define processing cycles for running jobs. There are also used by other COSmanager applications. A schedule specifies the day which a job is to be run. You choose from the list when defining an automatic job. You can add new schedules based on your own requirements to the schedules provided with task3G.
-The number of seconds between checks to determine if new jobs+ 
-can be launched.+<br>
-Priority The priority determines the job's relative share of the CPU; it+=== Defining schedule dates ===
-equates to the UNIX nice value.+ 
-Note Remember that the lower the number, the higher the priority. -19 is+The base schedule is simply a list of days and months. The prompt form for adding schedules has three fields, Day(s) of Month, Month(s) of Year, and Day(s) of Week. If all the conditions in these fields are met, then the duty is scheduled for today. For
-the highest priority, +19 being the lowest.+example, to schedule duties for the first Monday and Friday of every month, you would fill in the fields like this:
-58 Setting Up Jobs+ 
-Adding a Schedule+:Day(s) of Month:1-7
-Schedules define processing cycles for running jobs. There are also used by other+:Month(s) of Year:1-12
-COSMmanager applications. A schedule specifies the day which a job is to be run.+:Day(s) of Week:1,5
-You choose from the list when defining an automatic job. You can add new schedules+ 
-based on your own requirements to the schedules provided with task3G.+Leaving a field’s default value means ‘always true’, so in this example every month is included.
-Defining schedule dates+ 
-The base schedule is simply a list of days and months. The prompt form for adding+This base schedule can be further refined by defining ‘include’ and ‘exclude’ datelists. Finally, you can also enter a UNIX command which must return TRUE (status 0) if the current date is in the schedule.
-schedules has three fields, Day(s) of Month, Month(s) of Year, and Day(s) of Week.+ 
-If all the conditions in these fields are met, then the duty is scheduled for today. For+<br>
-example, to schedule duties for the first Monday and Friday of every month, you+ 
-would fill in the fields like this:+=== Adding a schedule ===
-Day(s) of Month:1-7+ 
-Month(s) of Year:1-12+Select {{cnav|Maintain > Schedules}} from the ‘task3G jobs’ console.
-Day(s) of Week:1,5+ 
-Leaving a field’s default value means ‘always true’, so in this example every month is+Select {{cnav|Maintain > Add}}.
-included.+ 
-This base schedule can be further refined by defining ‘include’ and ‘exclude’ datelists.+[[Image:Task img 20.jpg|frame|Figure 17 — The task3G Job Queue Details]]
-Finally, you can also enter a UNIX command which must return TRUE (status+;Order: task3G allocates an order number, by default ten more than the previous highest. The order number determines the order in which the list of outstanding scheduled jobs is presented to the user. If you want to keep the new time with a group of similar schedule times, change the order to an available number close to that of the group.
-0) if the current date is in the schedule.+ 
-Adding a schedule+;Schedule name: Enter a descriptive term, consistent with the names displayed in the list of schedules.
-Select Maintain > Schedules from the ‘task3G jobs’ console.+ 
-Select Maintain > Add.+The next three fields control which days a duty will be scheduled. By default the fields return ‘true’ for every day of the month, every month of the year and every day of the week. You restrict the scheduled days by limiting one or more of the fields to certain values or ranges.
-Setting Up Jobs 59+ 
-Order task3G allocates an order number, by default ten more than the+;Day(s) of month: Enter a list of days of then month. Example: first week of the month: 1-7
-previous highest. The order number determines the order in which+ 
-the list of outstanding scheduled jobs is presented to the user. If+;Month(s) of year: Enter a list of months. Example: first month of every quarter: 1,4,7,10
-you want to keep the new time with a group of similar schedule+ 
-times, change the order to an available number close to that of the+;Day(s) of week: Enter the days of the week (0=Sunday, 6=Saturday). Example: Monday to Wednesday and Friday: 1-3,5
-group.+ 
-Schedule name Enter a descriptive term, consistent with the names displayed in+;Start time, Period Length: Do not change. They are for use with other COSmanager applications.
-the list of schedules.+ 
-The next three fields control which days a duty will be scheduled. By default the+;And in list: Only schedule the job for dates that are defined above AND are in the datelist selected.
-fields return ‘true’ for every day of the month, every month of the year and every+ 
-day of the week. You restrict the scheduled days by limiting one or more of the+;And not in list: Only schedule the job for dates that are defined above and NOT included in the datelist selected.
-fields to certain values or ranges.+ 
-Day(s) of monthEnter a list of days of then month. Example: first week of the+;Schedule command: Enter a command which will return ‘true’ only if today’s date is when the job should be run.
-month: 1-7+ 
-Month(s) of YearEnter a list of months. Example: first month of every quarter:+;Use with cron: Must be ‘yes’ for task3G.
-1,4,7,10+ 
-60 Setting Up Jobs+Press {{cnav|Accept}} to complete the definition of the scheduled time.
-Day(s) of Week Enter the days of the week (0=Sunday, 6=Saturday). Example:+ 
-Monday to Wednesday and Friday: 1-3,5+<br>
-Start time, Period LengthDo not change. They are for use with other+ 
-COSMmanager applications.+=== Synchronising crontab ===
-And in list Only schedule the job for dates that are defined above AND are in+ 
-the datelist selected.+To ensure that the automatic jobs are correctly replicated in COSmanager crontab, use {{cnav|Maintain > Sync crontab}} on the ‘task3G jobs’ console. Carry out this task if problems are experienced with automatic jobs in task3G which may be caused by inconsistencies between the tables.
-And not in list Only schedule the job for dates that are defined above and NOT+ 
-included in the datelist selected.+On selecting the {{cnav|Sync crontab}} option, task3G displays a dialogue box, which indicates what is about to happen and provides the option to exit or to proceed.
-Schedule commandEnter a command which will return ‘true’ only if today’s date is+ 
-when the job should be run.+Press {{cnav|Accept}} to synchronise the tables; task3G will return a message indicating the outcome of the task.
-Use with cron Must be ‘yes’ for task3G.+{{note| This function does not alter the root crontab in any way; the only table amended is the COSmanager (ie cosmos) crontab.}}
-Press Accept to complete the definition of the scheduled time.+ 
-Synchronising crontab+<br>
-To ensure that the automatic jobs are correctly replicated in COSMmanager+
-crontab, use Maintain > Sync crontab on the ‘task3G jobs’ console. Carry+
-out this task if problems are experienced with automatic jobs in task3G which may+
-be caused by inconsistencies between the tables.+
-On selecting the Sync crontab option, task3G displays a dialogue box, which+
-indicates what is about to happen and provides the option to exit or to proceed.+
-Press Accept to synchronise the tables; task3G will return a message indicating the+
-outcome of the task.+
-Note: this does not alter the root crontab in any way; the only table amended+
-is the COSMmanager crontab.+

Current revision

This section describes the process of defining jobs, including:


Contents

Before you run your first job

Task3G forms the skeleton of a complete task control and batch job submission system. Your next task is to flesh out this skeleton by adding job components for the main jobs and procedures at your site. The Task3G Overview explains what job components are and how they are related.

These details are stored centrally in task3G's database, rather than being duplicated in shell-scripts and applications throughout the network. This means that generalized components need only to be entered once and can be maintained in one place.

You may not need to do all of the following steps, as some of the components and site-specific details will have been defined when task3G and COSmanager were installed.

  1. If necessary define new job queues.
  2. Define tasks for any commands that will be used in a job.
  3. Define jobs to carry out all your regular batch processing.
  4. Create dependencies within a job.


Planning your job control strategy

There is no universal scheme that is best for all sites, and task3G does not try to impose a standard strategy. However, task3G does make it simpler to automate your job submission procedures, as it lets you build jobs from reusable components. task3G also provides a convenient means of documenting your procedures at any point, as information about all your jobs is stored centrally in task3G tables.

There are two main points to consider. First, given the current system environment, how best to handle the current processing workload. Second, planning to maintain service in the face of changes in processing requirements, both expected (increase in workload) and unexpected (system failure).

Following is a list of factors that may influence your strategy:


Task3G Configuration

The configuration screen is accessed via the COSmanager menu

Figure 5 — The COSmanager buttonbar (GUI)
Figure 5 — The COSmanager buttonbar (GUI)


By clicking the Config button, a sub menu appears containing all product configuration options. Access task3G Configuration by selecting the task3G configuration option. Select Product Configuration > task3G configuration .

The task3G Job configuration console should appear. This screen shows all defined jobs as well as providing access for configuring all other aspects.


How to Define a Job

There are three types task3G jobs that can be defined. They are automatic jobs, manual jobs and subjobs.

Automatic jobs
are jobs that are run by cron. An operator is not required to submit jobs of this type. Jobs in this mode need a schedule and time(s) at which the job will run. This is defined by the When and At Time(s) field on the job prompt form.
Manual jobs
are jobs that must be submitted by an operator. These jobs can also be scheduled but are never run automatically. The manual job uses the At Time(s) and On (day) fields for scheduling. However, this is optional, manual jobs need not have scheduling details as these can be provided while submitting a job.
Subjobs
are jobs within a job. These jobs cannot be scheduled. They are to be defined in another job as a task. When it is time for this task (subjob) to run, then the tasks of this subjob will execute. Subjobs help in the sense that larger, more complex jobs can be broken into smaller, simpler steps.
Note
Note
Any job can be used as a subjob. However, jobs defined as subjobs cannot be run by themselves.


Configuration Console

The task3G Job console is where all configuration concerning jobs is accomplished. Select Config > task3G Configuration .

Display > Job report menu option displays a full report of the last run.

The Maintain pulldown menu contains the standard Add, Change, Clone and Remove options which work the same way as maintaining other COSmanager tables.

The following is a list of actions for each menu item:


Figure 6 — Configuration console, or ‘Maintain Jobs’ window
Figure 6 — Configuration console, or ‘Maintain Jobs’ window
Maintain > Tasks in job
selects the task3G Tasks in job console allowing you to view and maintain the tasks in a chosen job.
Maintain > Create Duty
for use with duty3G.
Maintain > Sync crontab
to ensure that the automatic jobs are correctly replicated in COSmanager crontab.
Tables > Tasks
selects the task3G Task details console allowing you to maintain tasks
Tables > Roles
selects the task3G Roles and access capabilities console allowing you to view an maintain task3G roles and capabilities.
Tables > Queues
selects the task3G Job queue details console allowing you to maintain the job queue.
Tables > Schedules
selects the task3G schedules console allowing you to view and maintain task3G schedules.
Tables > Datelists
selects the task3G datelists console allowing you to maintain datelists.


To Define Information About a Job

Defining a job is split into two entry forms. The first comprises of the basic options needed for each task, the latter comprises of advanced options.

Figure 7 — Prompt form to add a job
Figure 7 — Prompt form to add a job
Job
The job name identifies each individual job. This must be unique.
Description
Appears in the job schedule, so make it meaningful to help users to choose the right job.
Mode
The mode in which the job will run in:
automatic
if the job will run to a regular schedule and doesn't need to be manually submitted
manual
if the job will be submitted by the operator
subjob
if the job will only run as a task in another job.


When to initiate the job (automatic job)

The When and At time(s) fields determine the scheduling details associated with the job.

When
The frequency at which the job runs, whether its daily, weekly, monthly, every second week etc. This is used for automatic jobs only.
At time(s)
Time at which the job will start. This is used for automatic jobs and is also optional for manual jobs.
On (day)
The day in which the job will start. This is mandatory for manual jobs if a start time was specified above, i.e. the job is 'scheduled'. ‘Next 24 hrs’ (default) means that the job will run later on the day of submission, or if submitted after the ‘At time(s)’ on the following day.


The next three fields specify the access role; the host this job will run on; and the job queue it will be scheduled to.

Access role
Who can run and control this job.
Run on host
On which host will the job run.
Queue
The job queue the job will be submitted to.


Advanced options

You can configure advanced options such as job parameters, pre- and post-conditions, concurrent tasks, priority, clear status and notes. Press Advanced to enter one or more of these command or options.

Figure 8 — Prompt form to add a job (advanced features)
Figure 8 — Prompt form to add a job (advanced features)
Job parameters
Job parameters cause SHELL ENVIRONMENT variables to be set which are available to all tasks in the job. Commonly they would be included as $ variables which are passed as parameters to the commands in a task.
Fixed Parameters
Job parameters can be set to fixed values in the job configuration. They should be a space separated list in the form VARIABLE=VALUE. The given VALUE will be assigned to the VARIABLE at the start of the job, and is available as an ENVIRONMENT VARIABLE to all tasks in the job.
Prompt form
Job parameters can also be set at job submission via a PROMPT form. The full path must be specified, unless the form is in $APPL_HOME/local/prompt. As with the 'Fixed parameters' the value(s) specified in the form will be available as ENVIRONMENT VARIABLES to all tasks in the job.
Pre- and Post-processing
This consists of commands that are executed before and/or after the job has run. The pre-processing consists of the at-submit command, the wait command and the pre-condition. These are executed in the order they appear. The post-processing consists of the at-end command.
At-submit command
An operating system command that executes when the job is submitted. This may be interactive if the job is manually submitted.
Wait condition
A boolean expression which can test the exit status of a job, exit status of a task or the return code from an arbitrary UNIX command or script. If defined, this expression must be satisfied (TRUE) before a job begins execution. The job will wait till the expression is satisfied. Form more information, see #Creating Wait Conditions.
Timeout (mins)
This is the number of minutes before the wait condition times out. If the wait condition times out, the condition fails and the job is not started.
Pre-condition
A UNIX command, script or boolean expression which is executed before the job is run. The boolean expression is identical in syntax as the wait condition. However, a pre-condition returns simply SUCCESS or FAIL immediately, it does not wait. If the precondition fails, the job is not run. For more information, see #Creating Wait Conditions.
Check Syntax
Checks the syntax of the wait condition and pre-condition and notifies if the conditions are OK or have errors.
On failure
A job log file is created on the commencement of each job. This option specifies whether the log is kept or deleted if the wait condition or pre-condition fails.
At-end command
An operating system command that executes when the job has ended. The ‘Status’ environment variable is set to the job’s exit status prior to calling this command. If the variable is 0, the job was successful.
Clear status
When to clear the job status. Submit - every time the job is submitted. End - when the job finishes. Success - when a job has ended successfully. Never - the job status cannot be cleared other than manually from the Task monitor console, or programmatically via the FSclrstat program.
Concurrent tasks
The maximum number of tasks that can be run concurrently. If more than this number of tasks are ready to run, some will be queued. Leaving the field blank signifies that an unlimited number of tasks can be run concurrently.
Priority
At what priority level the job will run at. This takes precedence over the queue priority.
Notes file
A file that contains extra information about a particular job which the operator can view. This field must contain a full path to the file, unless it is under $APPL_HOME/local/notes.


Notification Options

Task3G can notify certain people either when this job requests operator assistance, or when it ends. For each of these, you can specify if and whom to notify, using the following fields:

Notify
When and if to notify if the job fails or requires assistance. 'never' => don't perform this type of notification for this job; 'default' => notify the default people defined in the Task3G settings; 'specify' => notify the people defined in the 'Who' field below.
Who
Space separated list of people to notify. These can be COSmanager users, email addresses or mobile phone numbers (for SMS).


Note
Note
If you specify a COSmanager user their notification method and address must be configured in the COSmanager 'Users' table
Note
Note
The mechanics of the EMAIL and SMS notification should be configured in the COSmanager 'Notification Method' table


How to Define a Task

The building block of a job is the task. In effect, a task describes the 'what' and 'whether' of one component in a job. The 'What' part identifies an operating system command or script, and the name of the host on which it will run. The 'whether' part consists of pre- and post-conditions. These are UNIX commands or scripts that are run before and after the main UNIX command or script.

Defining tasks can be accomplished from 2 areas, they are from the 'Task list' console and the 'Tasks in job' console.

Tasks defined in the 'Task list' console are placed in a pool of tasks which can be chosen when adding tasks to a job. Tasks defined in the 'Tasks in job' console are added to the job but are also placed in this pool, and so can later be added to other jobs.


Configuration Console

Figure 9 — Configuration console, or ‘Maintain Tasks’ window
Figure 9 — Configuration console, or ‘Maintain Tasks’ window

To access the 'Task list' console from the task3G configuration console select Tables > Tasks .

To access the 'Tasks in job' console from the task3G Configuration console select Maintain > Tasks in Job .

To access the 'task3G Jobs' console see #Task3G Configuration.

The Maintain pulldown menu contains the standard Add, Change, Clone and Remove items which work the same way as maintaining other COSmanager tables. Generally, Double-Clicking an item automatically selects Change .

The following is a list of actions for each menu item:


To Define Information About a Task

Defining a task is split into two entry forms. The first comprises of the basic options needed for each task, the latter comprises of advanced options.

Figure 10 — Prompt form to add a task
Figure 10 — Prompt form to add a task
Task name
The task name identifies each individual task. This must be a unique identifier.
Command
This is the command the task executes. This can be a UNIX command or a script. Also referred to as the main command. Leave this field blank if the task is to be a subjob.
Run on host
This is the host on which the command will run. If left blank it will run on the default host of the job.
As user
Under which UNIX user the command or script will execute. This is currently ignored on Windows NT hosts.
Subjob
This is where you specify a pre-defined subjob to run within the task. The task will run the subjob, thus giving the ‘job within a job’ effect.
Comments
A description or comment about the task.


Advanced options

You can configure advanced options such as pre- and post-conditions, time-outs, recover actions and operator privileges. Press Advanced to enter one or more of these command or options.

Figure 11 — Prompt form to add a task (advanced options)
Figure 11 — Prompt form to add a task (advanced options)
Pre- and Post-processing
This consists of commands that are executed before and/or after the main command. These can be operating system commands or script.
Pre-condition
A UNIX command, script or boolean expression which is executed before the main command is run. The boolean expression is identical in syntax as the wait condition. However, a pre-condition returns simply SUCCESS or FAIL immediately, it does not wait. If it succeeds, the main command is run, if it fails, neither the main command nor any post-processing is done and the result is dependent on what job recovery action is specified for 'On pre-cond fail'. For more information, see #Creating Wait Conditions.
Check Syntax
Checks the syntax of the wait condition and notifies if the condition is OK or has an error.
On-success cmd
The on-success command is executed after the main command succeeds, it will not execute if it fails. If the ‘On-success’ command succeeds, the task completes successfully. If it fails, the result is dependent on what job recovery action is specified for 'On error'.
On-error cmd
The on-error command is executed after the main command fails, it will not execute if it succeeds. If the ‘On-error’ command fails, the task fails. If it succeeds, the result is dependent on what job recovery action is specified for 'On-error'.
Override status
This field only applies if an ‘On-error command’ is specified. If this field is ‘yes’ and the On-error command succeeds, the task completes successfully. This will set the completed tasks status to SUCCESSFUL even though the main command FAILED. If ‘no’ the status of the task will be set to FAIL even if On-error completes successfully.
Timeout
This is the number of minutes before the task times out. If the task executes for longer than this time, the task is killed and the result is dependent on what job recovery action is specified for 'On timeout'.


Job recovery actions

These recovery actions are triggered on command and timeout failures. There are one of four possible actions: assist, stop, continue and ignore.

assist
enter assist mode on error. The operator then decides the action.
stop
ends the task on error and don't run any of its dependencies. An error status is raised in the job.
continue
ends the task and runs the dependencies. An error status is raised in the job.
ignore
ends the task and runs the dependencies. No error status is raised in the job.
On error
This recovery action is triggered when post-processing fails. If no post-processing is specified, then the 'on error' recovery action will handle main command failures.
On pre-cond fail
This recovery action is triggered when the pre-condition fails.
On kill
This recovery action is triggered when the task is killed via the task monitor console.
On timeout
This recovery action is triggered when the task times out.


Operator Privileges

The following options allow or disallow the operator from performing certain functions on this task:

Can re-run
If the task goes into 'assist mode', this determines whether the operator can re-run the task.
Can continue
If the task goes into 'assist mode', this determines whether the operator can allow the task's dependencies to run.
Can kill
Whether the operator can kill (cancel) this task after it has started.
Notes file
A file that contains extra information about a particular task which the operator can view. This field must contain a full path to the file, unless it is under $APPL_HOME/local/notes.


Inserting Tasks into a Job

Populating a job with tasks and subjobs is done from the 'Tasks in job' console. Also from this console, dependencies and wait conditions can be defined. The console displays the number of tasks and subjobs contained within the job and its dependencies.

From the 'task3G Jobs' console, select Maintain > Tasks in job.

To access the 'task3G Jobs' console, see #Task3G Configuration.

Figure 12 — 'Tasks in Job' console
Figure 12 — 'Tasks in Job' console

The following is a list of the menus and an explanation of each:

Maintain
This menu deals with creating and adding tasks to a job as well as maintaining tasks.
Dependencies
This menu deals with creating and checking dependencies as well as creating wait conditions.
View
This menu deals with viewing jobs.


Adding Tasks to a Job and Task Maintenance

Adding tasks to a job is done via the maintain menu. Selecting Maintain > Add , opens a window containing all pre-defined tasks. From this window, selecting multiple tasks will add them to the job. If you want the new tasks to be dependent on an existing task, highlight that task before selecting Add .

The 'Tasks in job' console also allows the user to add new tasks to a job without having to create them via the 'Tasks' console. This is done by selecting Maintain > New . If you want the new tasks to be dependent on an existing task, highlight that task before selecting New .

Tasks can be changed or deleted by selecting Maintain > Change and Maintain > Remove respectively. Double-Clicking on tasks is the equivalent to Maintain > Change .

The 'Task' console can be accessed by selecting Maintain > Tasks .


Creating Dependencies

With no dependencies defined, a task3G job would start as many of its tasks in parallel as is possible. The maximum number of concurrent tasks can be defined in the job definition. Usually you want to define dependencies so that a task is not started until certain other 'parent' tasks have finished.

Dependent tasks are displayed indented, under their parent with a '+' sign aligned with their parent.

Once all tasks have been added to the job, creating dependencies is the next step. This is done using the following items found under the Dependencies menu:

Cut
By selecting one or more tasks and then selecting Dependencies > Cut , the task(s) are removed from the job and placed into a temporary buffer.
Link
By selecting one or more tasks and then selecting Dependencies > Link , the task(s) are placed into a temporary buffer. They are not removed from the job.
Paste
By selecting Dependencies > Paste , the task(s) in the buffer are added to the job. If a task was highlighted prior to pasting, the buffered task(s) are pasted under the highlighted task, making them dependent on the highlighted task. The buffer is then cleared.
Sequence
By highlighting multiple tasks and selecting Dependencies > Sequence , a sequence of dependent tasks is created. A window appears containing the selected tasks allowing you to put them in the desired order. This is achieved by 'dragging' the tasks up and down in this window. When the correct order has been achieved, push the Accept button and a sequence of dependent tasks is created.


Example 1: one task dependent on another

Task img 12.jpg
  1. Add tasks Update_db (task1) and Reports (task2) to a job. The window contents of the 'Tasks in job' console should appear as the following:
    Reports
    Update_db
  2. Next, create a dependency making task Reports dependent on task Update_db. Select Reports.
  3. Select Dependencies > Cut
  4. Select Update_db
  5. Select Dependencies > Paste . The window contents of the 'Tasks in job' console should appear as the following:
    Update_db
    + Reports
When run, task3G will start Reports once Update_db has completed.


Example 2: two tasks dependent on one

Task img 13.jpg
  1. Add tasks Update_db, Reports and Cleanup to a job. The window contents of the 'Tasks in job' console should appear as the following:
    Cleanup
    Reports
    Update_db
  2. Next, create a dependency making task Reports and Cleanup both dependent on task Update_db. Select both Reports and Cleanup.
  3. Select Dependencies > Cut
  4. Select Update_db
  5. Select Dependencies > Paste . The window contents of the 'Tasks in job' console should appear as the following:
    Update_db
    + Reports
    + Cleanup
When run, task3G will start both Reports and Cleanup once Update_db has completed.


Example 3: one task dependent on two

Task img 14.jpg
  1. Add tasks Update_db, Reports and Cleanup to a job. The window contents of the 'Tasks in job' console should appear as the following:
    Cleanup
    Reports
    Update_db
  2. Next, create a dependency making task Cleanup dependent on both Update_db and Reports tasks. Select Cleanup.
  3. Select Dependencies > Cut
  4. Select Update_db
  5. Select Dependencies > Paste
  6. Select Cleanup again.
  7. Select Dependencies > Link
  8. Select Reports
  9. Select Dependencies > Paste . The window contents of the 'Tasks in job' console should appear as the following:
    Update_db
    + Cleanup
    Reports
    + Cleanup
With the second Cleanup flagged as a link. When run, task3G will start Cleanup once both Update_db and Reports have completed.


Example 4: a sequence of dependent tasks

Task img 15.jpg
  1. Add tasks Update_db, Reports and Cleanup to a job. The window contents of the 'Tasks in job' console should appear as the following:
    Cleanup
    Reports
    Update_db
  2. Next, create a dependency making task Reports dependent on Update_db and Cleanup dependent on Reports tasks. Select all tasks.
  3. Select Dependencies > Sequence
  4. Drag Update_db above Cleanup (top of the display)
  5. Either drag Reports above Cleanup or drag Cleanup below Reports
  6. Press Accept to save this ordering of tasks. The window contents of the 'Tasks in job' console should appear as the following:
    Update_db
    + Reports
    + Cleanup
When run, task3G will run Update_db, then Reports, then Cleanup.


Checking Dependencies

When constructing complex dependencies that involve a large number of tasks, it is possible to unknowingly create deadlocks and/or unnecessary dependencies.

Deadlocks are basically cyclic errors. Deadlocks occur when two or more tasks are dependent on each other, eg. task3 is dependent on task2, which is dependent on task1, which is dependent on task3. It is impossible for task1, task2 or task3 to start.

Unnecessary dependencies are basically redundant dependencies, they are dependencies that are not needed.

These conditions will cause the offending task to appear in the monitor console colored either red or orange.

To fix these conditions, select Dependencies > Optimise .


Creating Wait Conditions

'Wait For' conditions are defined against a task in a job. To create a wait for condition, select a task then select Dependencies > Wait for .

Figure 13 — Wait for condition
Figure 13 — Wait for condition
Wait condition
A boolean expression which can test the exit status of a job, exit status of a task or the return code from an arbitrary UNIX command or script. If defined, this expression must be satisfied (TRUE) before a task begins execution. The task will wait till the expression is satisfied.


The wait condition uses the following symbols:

Symbol Meaning
¦¦ OR
&& AND
== EQUAL TO
 != NOT EQUAL TO
> GREATER THAN
< LESS THAN
>= GREATER THAN OR EQUAL TO
<= LESS THAN OR EQUAL TO
 ! NOT


Normal precedence for boolean expressions applies and brackets can be used. Primary expressions can be:

Note
Note
that if a shell command returns a non-zero status, the condition will WAIT for 60 seconds and try again. It will not return FALSE.


Task3G uses the following convention for both task and job statuses:

Numeric Exit Status Symbol
0 OK
1 ERR[OR]
2 WARN[ING]
10 KILL[ED]


The following are some examples of wait conditions:

Example: $step3 == 0
Will wait for task 'step3' in the current job to finish, and when it does it will return TRUE or FALSE depending whether its exit status is 0.
Example: $step3 == KILLED
Will wait for task 'step3' in the current job to finish, and when it does it will return TRUE or FALSE depending whether its exit status is 10 (killed).
Example: $step3 == ERR
Will wait for task 'step3' in the current job to finish, and when it does it will return TRUE or FALSE depending whether its exit status is 1 (error).
Example: $job2:step3 < 2
Will wait for task 'step3' in job 'job2' to finish, then check its exit status is less than 2.
Example: $job2:step3 && [ -f /tmp/in_data ]
Will wait for task 'step3' in job 'job2' to finish and for a file /tmp/in_data to exist.


A task_name (or job_name) by itself will simply wait for the task or job to terminate, then return TRUE.

Figure 14 — Check syntax report
Figure 14 — Check syntax report
Check Syntax
Checks the syntax of the wait condition and notifies if the condition is OK or has an error.
On failure
If the wait condition fails, i.e. evaluates to FALSE, the task will either go into assist mode.
assist
enter assist mode on failure. The operator then decides the action.
stop
stops the task on failure and doesn't run any dependencies. An
error
status is raised in the job.


Viewing other Jobs and Subjobs

Using the View menu items, other jobs and subjobs can be selected for viewing via the 'Tasks in job' console. The following is an explanation of each item:

Parent job
when viewing a subjob, selecting View > Parent job will view the tasks within the parent job.
Subjob
selecting a subjob and selecting View > Subjob , will view the tasks within the subjob.
Another job
selecting View > Another job will view the tasks within the next job in alphabetical order.
Show notes
selecting View > Show notes will view the task notes.


How to Define a Job Queue

Task3G jobs can be submitted to the 'at' queue, 'batch' queue, 'task' queue or to a user defined job queue. The 'cron' queue is reserved for cron jobs, it is not available for job submission.

Job queues are used to stream jobs with similar characteristics. For each queue, you can define a default priority and the maximum number of concurrent jobs. The queue will only release up to this number of jobs to run at the same time. Once one of these jobs finishes, the next job in that queue is started. The priority determines the job's relative share of the CPU; it equates to the UNIX nice value.

Each queue is identified by a letter, for example 'a' for the AT_queue, 'b' for the BATCH_queue, 't' for the TASK queue. d - z are available for user-defined queues. 'c' is reserved for cron jobs.

Note
Note
Some operating systems do not support all queues d-z. Also some operating systems do not support specifying the maximum number of concurrent jobs.


Configuration Console

The task3G Job Queue Details console is where all configuration concerning queues is accomplished. This is access via the task3G Jobs console. From the task3G Jobs console, select Tables > Queues .

Figure 15 — The task3G Job Queue Details
Figure 15 — The task3G Job Queue Details

To access the task3G Jobs console, see #Task3G Configuration.

Figure 15 shows all defined queues.

The Maintain pulldown menu contains the standard Add, Change, Clone and Remove items which work the same way as maintaining other COSmanager tables. Generally, Double-Clicking an item automatically selects Change .

The following is a list of actions for each menu item:


To define a new queue

Figure 16 — Defining a task3G queue
Figure 16 — Defining a task3G queue
Queue
This is the unique name of the queue which is used as the identifier.
Description
A description or comment about the queue.
UNIX queue
The UNIX at queue name to be used with this task3G queue. You can normally use up to 25 job queues. Jobs can be submitted to the standard system queues provided, such as the at queue (a), batch queue (b) and the task queue (t). Another 22 queues may be defined with the letters, d through to z, though some operating systems may restrict this range.
Note
Note
Jobs can't be submitted to the c queue as it is reserved for use by cron.
Note
Note
The number of queues that can be defined may be restricted in some versions of UNIX.
Concurrent jobs
The maximum number of jobs that can run concurrently in this queue.
Re-scheduled time
The number of seconds between checks to determine if new jobs can be launched.
Priority
The priority determines the job's relative share of the CPU; it equates to the UNIX nice value.
Note
Note
Remember that the lower the number, the higher the priority. -19 is the highest priority, +19 being the lowest.


Adding a Schedule

Schedules define processing cycles for running jobs. There are also used by other COSmanager applications. A schedule specifies the day which a job is to be run. You choose from the list when defining an automatic job. You can add new schedules based on your own requirements to the schedules provided with task3G.


Defining schedule dates

The base schedule is simply a list of days and months. The prompt form for adding schedules has three fields, Day(s) of Month, Month(s) of Year, and Day(s) of Week. If all the conditions in these fields are met, then the duty is scheduled for today. For example, to schedule duties for the first Monday and Friday of every month, you would fill in the fields like this:

Day(s) of Month:1-7
Month(s) of Year:1-12
Day(s) of Week:1,5

Leaving a field’s default value means ‘always true’, so in this example every month is included.

This base schedule can be further refined by defining ‘include’ and ‘exclude’ datelists. Finally, you can also enter a UNIX command which must return TRUE (status 0) if the current date is in the schedule.


Adding a schedule

Select Maintain > Schedules from the ‘task3G jobs’ console.

Select Maintain > Add .

Figure 17 — The task3G Job Queue Details
Figure 17 — The task3G Job Queue Details
Order
task3G allocates an order number, by default ten more than the previous highest. The order number determines the order in which the list of outstanding scheduled jobs is presented to the user. If you want to keep the new time with a group of similar schedule times, change the order to an available number close to that of the group.
Schedule name
Enter a descriptive term, consistent with the names displayed in the list of schedules.

The next three fields control which days a duty will be scheduled. By default the fields return ‘true’ for every day of the month, every month of the year and every day of the week. You restrict the scheduled days by limiting one or more of the fields to certain values or ranges.

Day(s) of month
Enter a list of days of then month. Example: first week of the month: 1-7
Month(s) of year
Enter a list of months. Example: first month of every quarter: 1,4,7,10
Day(s) of week
Enter the days of the week (0=Sunday, 6=Saturday). Example: Monday to Wednesday and Friday: 1-3,5
Start time, Period Length
Do not change. They are for use with other COSmanager applications.
And in list
Only schedule the job for dates that are defined above AND are in the datelist selected.
And not in list
Only schedule the job for dates that are defined above and NOT included in the datelist selected.
Schedule command
Enter a command which will return ‘true’ only if today’s date is when the job should be run.
Use with cron
Must be ‘yes’ for task3G.

Press Accept to complete the definition of the scheduled time.


Synchronising crontab

To ensure that the automatic jobs are correctly replicated in COSmanager crontab, use Maintain > Sync crontab on the ‘task3G jobs’ console. Carry out this task if problems are experienced with automatic jobs in task3G which may be caused by inconsistencies between the tables.

On selecting the Sync crontab option, task3G displays a dialogue box, which indicates what is about to happen and provides the option to exit or to proceed.

Press Accept to synchronise the tables; task3G will return a message indicating the outcome of the task.

Note
Note
This function does not alter the root crontab in any way; the only table amended is the COSmanager (ie cosmos) crontab.