Task3G/User Guide/Overview

From Documentation

task3G is a job control and batch job scheduling application that provides networkwide job execution with comprehensive inter-task dependencies. task3G runs under the COSMmanager system administration environment, for more information about COSMmanager, consult the COSMmanager user guide. task3G builds on the standard UNIX batch facilities - at, cron and batch. It provides both a graphical (GUI) and character (CUI) interface for managing those facilities and introduces a job control and inter-task dependency mechanism to give full production control facilities. task3G provides a flexible mechanism for controlling the flow of job execution. The execution of a job may be conditional on the availability of resources or the outcome of another job. The job control mechanism enables a complex network of tasks to be set up and safely run unattended. 16 Overview Jobs A job consists of a set of steps or tasks that are run depending on their relationship to one another. Jobs which are submitted via task3G can be run to a regular schedule or at any time. Jobs are assembled by selecting details from tables of predefined components, such as: • Task group (who can run the job). • Which job queue it will run in. • Which host it will run on. • Number of tasks that can run concurrently within a job. Subjobs Subjobs are jobs within a job. Subjobs are useful in avoiding unnecessary duplication of groups of tasks. By defining a group of tasks as a subjob, only a subjob need be included in each relevant job instead of all the tasks in that group. Also subjobs can be used to simplify complex jobs by grouping together relevant tasks thus making the job more readable and easier to understand. Because any job can be used as a subjob, a top-level controlling job can be built replacing traditional facilities like JCL (Job Control Language). Tasks Tasks are the building blocks of jobs. Each task executes an OS command and/or script. In addition to this main command there are also pre- and post-processing commands. The pre-processing command, executed before the main command, can check such things as the availability of certain system resources. The post-processing, executed after the main command, consists of two different commands, on-success processing and on-error processing. One of these post-processing commands is triggered depending on the outcome of the main command, whether it succeeded or returned an error. The pre- and post-processing commands are not mandatory. Each task has a unique name. This acts as the identifier for each task. Overview 17 Note Jobs cannot contain the same task twice as each task is identified by a unique name. A task must be cloned and renamed if an identical task needs to be included in the same job. Tasks to be inserted in a job can be chosen from a pre-defined list. This allows tasks to be easily reused in many different jobs. A time-out can be specified for each task, which if exceeded will cause the task to be killed. Dependencies and the Wait Condition task3G has a very flexible scheme for controlling the flow of execution among tasks in a job and within a network of related jobs. The relationships between tasks within a job are called dependencies. In other words, the execution of a task is dependent on the completion of one or more tasks. This allows an arbitrary 'graph' of tasks to be defined. More complex conditions and dependencies of tasks in other jobs are achieved by an optional 'wait' boolean expression. This expression must be satisfied (TRUE) before a task begins execution. The boolean expression can include such conditions as the state of a job, state of a task within another job, state of a task within the same job or the return code from an arbitrary UNIX command or script. Each task can be configured with such an expression. Scheduling and Running Jobs Jobs can be run in one of two modes: manual or automatic. Automatic jobs are run by 'cron' at regular times, e.g. 11pm every Monday, while the operator must submit manual jobs. Jobs can be submitted to run immediately (background) or at some time later (scheduled). Active jobs can be monitored through the job monitor, for more information on the job monitor, see Submitting and Managing Jobs on page 61. If needed, pre-configured scheduling information (such as start time, queue, etc.) can be overridden when the job is submitted. Job Queues and Priorities Job queues are used to stream jobs that have similar requirements and a similar relative priority. You can use up to 25 job queues. Jobs can be submitted to the standard system queues provided, such as the 'at' queue (a), 'batch' queue (b) and the 'task' queue (t). Note Jobs can't be submitted to the c queue as it is reserved for use by cron. Note Some operating systems may restrict the number of job queues. Refer to the 'at' man page. The priority determines the job's share of available CPU time relative to other system and user processes. The priority only applies once the job is running; it has no effect on which among a queue of waiting jobs will be the next to run. It equates to the UNIX nice value. You can limit the number of concurrently running jobs that can be released from a queue. For example, if you have three tape drives, you can set up a 'Tape jobs' queue from which a maximum of three jobs can run at any one time. Other jobs in the queue must wait until one of the running jobs ends and a tape drive is released. Roles and Access Capabilities task3G users are assigned to one or more 'roles', based on their responsibilities and their level of access to jobs. New roles may also be defined with different capabilities. Some predefined roles provided are: Managers who have access to all task3G functions such as being able to run or kill any job belonging to any role. Overview 19 Administrators who have access to most task3G functions such as creating new jobs and tasks. Operators who have access to basic task3G functions such as submitting and controlling jobs. By default, roles do with a higher level of access generally inherit all the capabilities of lower access users. For example, managers inherit administrator capabilities and administrators inherit operator capabilities. Note Generally, capabilities in roles do overlap as the predefined ones do, but this may not be the case with user defined roles. Another feature provided is the ability to associate a job with a role. Only users who have been assigned to that role can submit or control the job. Parameter Prompt Forms A manually submitted job can display a parameter prompt form to the operator just before it is submitted. If you need to change one or more arguments each time the job is run, this is a convenient way of passing the variable information to the job without having to edit the command field. Prompt forms are simply data entry screens, such as the ones you use in task3G to add or change jobs, tasks, and queues. For example, for a job to produce a monthly report, you could define a prompt form with a single field, 'Month'. Log Files and Audit Trail All jobs and tasks are logged individually within task3G. Each time the job and task runs, a new log file is created, thus keeping separate records of the same job or task. By default all output of a task's command is sent to the task's log file. An audit trail is also kept that logs task3G's activity. Notes by operators may also be inserted into the audit trail. Periodically the audit trail is archived to save space and a 20 Overview new one is created. You can define how often this is done, and how many archived audit trails to keep. This COSMmanager Framework provides this audit trail cycling. Job Status The word 'status' has two distinct meanings in task3G: • The numeric 'exit status' of an operating system command. • The information task3G keeps about an active or recently completed job and its tasks. The task3G 'job status' records the exit statuses of all the tasks that have completed in the job, as well as current state of job and running tasks. Note task3G will not allow a job to start unless its job status is clear. This can be use to prevent accidental job re-run.