AutoSys Scheduler: Create & Manage Jobs using AutoSys Job Scheduler

AutoSys is used for defining, scheduling and monitoring jobs. These jobs can be a UNIX script, java program or any other program which can be invoked from shell. Before starting we assume that user has already setup an AutoSys environment. This environment consists of autosys server and autosys client.

AutoSys System components

1. Event server (AutoSys database)
2. Event processor
3. Remote agent

Event Server

The event server is a AutoSys database which stores all system information and events as well as all job, monitor, and report definitions. Sometimes this database is also called as a data server, which actually describes a server instance. That is, it is either a UNIX or Windows process, and it is associated data space (or raw disk storage), that can include multiple databases or tablespaces.

Event Processor

This is main component of the autosys system. This processes all the events it reads from dataserver. The event processor is the program, running either as a UNIX process or as a Windows service that actually runs AutoSys. It schedules and starts jobs. When you start the event processor it continually scans the database for events to be processed. When it finds one, it checks whether the event satisfies the starting conditions for any job in the database.

Remote Agent

On a UNIX machine, the remote agent is a temporary process started by the event processor to perform a specific task on a remote (client) machine. On a Windows machine, the remote agent is a Windows service running on a remote (client) machine that is directed by the event processor to perform specific tasks.
The remote agent starts the command specified for a given job, sends running and completion information about a task to the event server, then exits. If the remote agent is unable to transfer the information, it waits and tries again until it can successfully communicate with the database.

Basic functionality of AutoSys

Below is the diagram which explains the basic functionality, please check the explanation.

Explanation

1. The event processor scans the event server for the next event to process. If no event is ready, the event processor scans again in five seconds.
2. The event processor reads from the event server that an event is ready. If the event is a STARTJOB event, the job definition and attributes are retrieved from the Event Server, including the command and the pointer (full path name on the client machine) to the profile file to be used for the job. In addition, for jobs running on Windows machines, the event processor retrieves from the database the user IDs and passwords required to run the job on the client machine.
3. The event processor processes the event. If the event is a STARTJOB, the event processor attempts to establish a connection with the remote agent on the client machine, and passes the job attributes to the client machine.
The event processor sends a CHANGE_STATUS event marking in the event server that the job is in STARTING state.
4. On a UNIX machine, the inetd invokes the remote agent. On a Windows machine, the remote agent logs onto the machine as the user defined as the job’s owner, using the user IDs and passwords passed to it from the event processor.
5. The remote agent sends an acknowledgment back to the event processor indicating that it has received the job parameters. The socket connection is terminated. At this point, the event processor resumes scanning the event server database, looking for events to process.
6. The remote agent starts a process and executes the command in the job definition.
7. The remote agent issues a CHANGE_STATUS event marking in the event server that the job is in RUNNING state.
8. The client job process runs to completion, then returns an exit code to the remote agent and quits.

Defining autosys job

There are various parameters to define autosys job. Starting from profile, timezone, start time, starting condition and so on. There are the two methods you can use to create job definitions:
1. Using the AutoSys Graphical User Interface (GUI).
2. Using the AutoSys Job Information Language (JIL) through a command-line interface.
In this tutorial we will use JIL language to create autosys jobs.
JIL stands for Job Information Language. Using this you can instruct autosys to save job definitions. This information saved in autosys database. You can also create a jil file which contains job definition. You can then pass this jil file to autosys.
Essential attributes for defining job
1. Job Name
JIL Keyword : insert_job. Name used to identify the job.
2. Job Type
a. JIL Keyword : job_type. The job type is one of job types: command (c), file watcher (f) or box (b).
3. Owner
a. JIL Keyword : owner
The job owner specifies whose user ID the command will be run under on the client machine. This attribute is automatically set to the user who invoked jil or the GUI to define the job, and cannot be changed except by the edit superuser.
Other job attributes:
1. command: The command attribute can be the name of any command, executable, UNIX shell script or batch file, and its arguments.
2. machine: This attribute specifies the client machine on which the command should be run.
3. date_condition: The start date/time dependencies attribute is a toggle, which specifies whether or not there are date, time, or both, conditions required for starting the job.
4. days_of_week: The days of the week attribute specifies the days on which the job should be run.
Sample jil file for command job echoJob.jil

insert_job:echoJob

machine:unixMachine

owner:username

command:echo “Hello this is command job”

 

To add this job in atosys db. Run following command from unix:

jil < echoJob.jil

 

Commands to control the job

Start job command

sendevent –E FORCE_STARTJOB -J <job_name>

 

sendevent -E STARTJOB -J <job_name>

 

To put jobs on OFF ICE or ON ICE

sendevent -E OFF_ICE -J <job_name>

 

sendevent -E ON_ICE -J <job_name>

 

sendevent -E KILLJOB –J "Job Name Here"

 

 

 

 

How to install second hard drive in Ubuntu Linux

So you’ve been using linux for awhile now, and it’s time to install another hard drive for some more storage? Maybe it’s finally time to wipe out a Window’s or NTFS partition.

In any event, I wrote this post so you would have a little help through this step. I’m going to assume that you already know how to install a hard drive. I’m also going to assume that you knew how to make it a master or slave, you’ve checked to make sure that it shows up in bios, and that it was intalled properly. It also assumes you’ve already formatted your drive in linux ext3 format, using a tool like gParted, or something similar.

If you’ve done all of these things, then boot up your system – and let’s get going. I’m using Ubuntu linux – formerly Breezy 5.10, but I’ve recently upgraded to Dapper 6.06.

Open up a terminal window and run the following command:
$ sudo fdisk -l

You should get a listing of the hard drives installed on your computer. There will be a little paragraph for each one that will look like this:


Disk /dev/hda: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 4678 37576003+ 83 Linux
/dev/hda2 4679 4865 1502077+ 5 Extended
/dev/hda5 4679 4865 1502046 82 Linux swap / Solaris

In windows disk drives are assigned an alphabet letter, and traditionally – floppy disk drives were a: and / or b:, and the main hard drive was c:/. Then the first cdrom or dvdrom was d:/, and any additional drives would be e:, f:, and so on. In linux it’s kind of the same, but in a different format. All hard drives installed are listed in the ‘device’ or /dev directory. All drives start with the appendage “hd” (I think for ‘hard drive’).

So, if you have 2 hard drives and one cdrom – then you have 3 devices. You have a hda, hdb, and hdc. The number of partitions comes next. If your main hard drive is linux – and you have 3 partitions, then you’ll have a hda1, hda2, and hda5. The partition numbers aren’t in a logical order – hda5 is always the swap partition.

So know, if you see your hard drive listed in the sudo fdisk -l, then you know you can mount it. Your computer has a startup file that tells it what to mount when it boots. We need to edit this file and add the new drive.

But first we have to create a directory. We have to create what’s called a “mount point”. A mount point is a virtual directory. It’s like saying – mount that hard drive from this directory.

The most logical place to create the mount point for the new hard drive is either the /mnt or /media folder. Many would say that /mnt is the only place it should be – the mount folder. However, Ubuntu always mounts all dvd, cd, and removable media in /media. I chose to make my mount point here for just that very reason.

Think of what you want to call the new mount point name. Just make sure you don’t use any special characters or spaces in the name. I called mine linuxstore. Now, in terminal run the following command substituting my mount point name for yours:
$ sudo mkdir /media/linuxstore

Next, here’s one of the most important things….and somethng that I didn’t find in any of the articles on the web when I was trying to figure out how to do this. You have to make the mount point directory ‘writable’. In other words, you have to give it writable permissions. They have to be world-writable permissions since you aren’t a member of the ‘root’ group in which all mount points are owned.

So, now you want to run the following command (again substituting my mount point name for yours):
$ chmod 777 /media/linuxstore

If you want to mount your drive right away, and you don’t care if it’s mounted automatically every time you boot – then in terminal run the following command:
$ mount /dev/hdd1 /media/linuxstore

There! Now you are temporarily mounted. But…if you want it to be permanent, you need to edit your filesystem tab file. Run the following command in terminal:
$ gedit /etc/fstab

The text editor window will appear with the fstab file loaded up. You will see something that looks kind of like this:

# /etc/fstab: static file system information.
#
# 
proc /proc proc defaults 0 0
/dev/hda1 / ext3 defaults,errors=remount-ro 0 1
/dev/hda5 none swap sw 0 0
/dev/hdb1 /media/hdb1 ext3 defaults 0 0
/dev/hdc /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0

All you have to do is add a new line for the new drive…
I will add the following line to my fstab for my new drive:

/dev/hdd1 /media/linuxstore ext3 defaults 0 0

Just be sure to substitute both the name of my hard drive for yours (mine is hdd1, is yours hdc1 or another name?), and my mount point hame for yours. Then save the file.

Now you will have the new hard drive mounted and writable both every time you boot. In Ubuntu, you should find your new drive listed under your ‘Places’ menu. To make the hard drive show up right now, without rebooting – just reload your fstab file with the following command:
$ sudo mount -a

Now you’re done! Enjoy your new storage drive in Linux!

Basic sed tricks

1. What is sed? - sed is stream editor, a Unix tool for working with streams of text data. See the awful truth about sed.
2. How do you substitute strings with sed? - Use ’s/old/new’ command, so sed ’s/hello/goodbye/’ would substitute the occurrence of the word hello to goodbye.
3. How do you inject text with sed? - & in the substitution string defines the pattern found in the search string. As an example, here’s us trying to find a word ‘hello’ and replacing it with ‘hello and how are you’:
        echo ‘hello there’ | sed ’s/^hello/& and how are you/’
4. Can I find several patterns and refer to them in the replacement string? - Yes, use (pattern) and then refer to your patterns as \1, \2, \3 and so on.
5. If the string is ‘old old old’ and I run ’s/old/new’, I get ‘new old old’ as the result. I need ‘new new new‘. - You forgot the global modifier, which would replace every occurrence of the pattern with the substitution. ’s/old/new/g‘ will work.
6. But I want ‘old old new’ from the previous example. - Just use the numeric modifier saying you want the third occurrence to be replaced. ’s/old/new/3‘ will work.
7. I wrote a rather complex sed script. How do I save and run it? - Assuming that your file is named myscript1.sed, you can invoke sed -f myscript1.sed.
8. How do I delete trailing whitespaces from each line? - sed ’s/[ \t]*$//’ Here we’re replacing any occurrence of a space or a tab with nothing. Check sed one-liners for more examples.
9. How do you print just a few first lines of the file? - sed 1q will give you just the first line, sed 10q the first 10 lines.
10. How do you replace a pattern only if it’s found, so that it’s executed faster? - Nest the replacement statement: sed ‘/old/ s/old/new/g’ file.txt

Linux network administrator questions

1. Give an example of set of shell commands that will give you the number of files in a directory
2. How do you tell what process has a TCP port open in Linux
3. On a Red Hat Linux Variant how do you control whether a service starts when the system boots
4. How do you tell the amount of free disk space left on a volume
5. Give an example of a set of shell commands to tell how many times “bob” has logged on to the system this month
6. Give an example of a recursively copying a directory from one location to another.
7. How do you modify the IP and Net mask of a system running a Red Hat Variant of Linux
8. Give an example of a set of shell commands that will give you the number of “httpd” processes running on a Linux box.
9. On CentOS or Fedora based system using the package management application, how do you tell what package provided the file “libnss_ldap.so”
10. What is the difference between VTP client, server, and transparent
11. What is the maximum length of CAT6
12. How does one set up a layer two link to share VLANs
13. How does one implement redundant links at Layer 2
14. What is the difference between a hub, switch, and a router? What are the security advantages of switch vs. hub?
15. Show an example of using telnet to learn the headers of an http server.
16. In what OSI layer does PPP exist
17. What’s the difference between TCP and UDP
18. Given a DNS server that has just started (with an empty cache) and host contacting this DNS server (using it’s OS setting) to learn an address for google.com, list the steps the DNS server will take to learn it with IP addresses (each step will have multiple possible IP addresses – you need choose only one per step).
19. Why are layer 2 loops bad, and what protocol was designed to prevent them
20. Given a radius server at 10.0.0.2 and a shared key of ‘abc123′ show the IOS commands necessary to authenticate switch users against the radius server, while still allowing the use of local username / password pairs