How to see hidden files in Linux via terminal

A filename that begins with a period is called an invisible filename (or an invisible file or sometimes a hidden file) because ls does not normally display it. The command ls –a displays all filenames, even invisible ones.

$ls –a
--shows all the files ….

Names of startup files usually begin with a period so that they are invisible and do not clutter a directory listing. The .plan file  is also invisible. Two special invisible entries—a single and double period (. and . .)—appear in every directory.

Options with the commands

 

--help – help
-a all
-c create
-f forced / file read or write
-i  interactively
-l long display
-r recursively
-q quiet
-v verbose

grep: Finds a String

The grep (global regular expression print) utility searches through one or more files to see whether any contain a specified string of characters. It does not change the file it searches but simply displays each line that contains the string.

Originally this utility's name was a play on an ed—an original UNIX editor, available on Linux—command: g/re/p. In this command the g stands for global, re is a regular expression delimited by slashes, and p means print.

Example

$ cat memo

Helen:In our meeting on June 6 we discussed the issue of credit.Have you had any further thoughts about it? Alex

$ grep 'credit' memo

discussed the issue of credit.

The grep command above searches through the file memo for lines that contain the string credit and displays a single line that meets this criterion. If memo contained such words as discredit, creditor, or accreditation, grep would have displayed those lines as well because they contain the string it was searching for. The –w option causes grep to match only whole words. You do not need to enclose the string you are searching for in single quotation marks, but doing so allows you to put SPACEs and special characters in the search string.

The grep utility can do much more than search for a simple string in a single file.

lpr, lpq and lprm : Prints a File

The lpr (line printer) utility places one or more files in a print queue for printing. Linux provides print queues so that only one job is printed on a given printer at a time. A queue allows several people or jobs to send output simultaneously to a single printer with the expected results. On machines with access to more than one printer, you can use the –P option to instruct lpr to place the file in the queue for a specific printer, including one that is connected to another machine on the network. The following command prints the file named report:

$ lpr report

Because this command does not specify a printer, the output goes to the default printer, which is the printer when you have only one printer.

The next command line prints the same file on the printer named mailroom:

$ lpr -Pmailroom report

lpq - Queue of print jobs

You can see what jobs are in the print queue by using the lpq utility:

$ lpq

lp is ready and printing

Rank  Owner   Job Files                 Total Size

active alex    86 (standard input)        954061 bytes

In this example, Alex has one job that is being printed; no other jobs are in the queue.

lprm – Remove the items from print queue

You can use the job number (86 in this case) with the lprm utility to remove the job from the print queue and stop it from printing:

$ lprm 86

You can send more than one file to the printer with a single command. The following command line prints three files on the printer named laser1:

$ lpr -Plaser1 05.txt 108.txt 12.txt

mv: Changes the Name of a File

The mv (move) utility can rename a file without making a copy of it. The mv command line specifies an existing file and a new filename using the same syntax as cp:

mv existing-filename new-filename

CP copies the file

$ ls
memo

$ cp memo memo.copy

$ ls
memo memo.copy

MV renames the file
$ ls
memo

$ mv memo memo.0130
$ ls
memo.0130

Moving the files from 1 directory to other

The mv utility can be used for more than changing the name of a file.

When used to move one or more files to a new directory, the mv command has this syntax:

mv existing-file-list directory
If the working directory is /home/alex, Alex can use the following command to move the files - names and temp from the working directory to the literature directory:

$ mv names temp literature

This command changes the absolute pathnames of the names and temp files from /home/alex/names and /home/alex/temp to /home/alex/literature/names and /home/alex/literature/temp, respectively. Like most Linux commands, mv accepts either absolute or relative pathnames.

Moving directories from 1 Directory to other

Just as it moves ordinary files from one directory to another, so mv can move directories. The syntax is similar except that you specify one or more directories, not ordinary files, to move:

mv existing-directory-list new-directory
If new-directory does not exist, the existing-directory-list must contain just one directory name, which mv changes to new-directory (mv renames the directory). Although directories can be renamed using mv, their contents cannot be copied with cp unless you use the – r option. Refer to the explanations of tar and cpio for other ways to copy and/or move directories.

Caution: mv can destroy a file

Just as cp can destroy a file, so can mv. Also like cp, mv has a –i (interactive) option.

cp: Copies a File

The cp (copy) utility makes a copy of a file. This utility can copy any file, including text and executable program (binary) files. You can use cp to make a backup copy of a file or a copy to experiment with.

The cp command line uses the following syntax to specify source and destination files:

cp source-file destination-file







The source-file is the name of the file that cp will copy. The destination-file is the name that cp assigns to the resulting (new) copy of the file.

Caution: cp can destroy a file

If the destination-file exists before you give a cp command, cp overwrites it. Because cp overwrites (and destroys the contents of) an existing destination-file without warning, take care not to cause cp to overwrite a file that you need. The cp – i (interactive) option prompts you before it overwrites a file.

The following example assumes that the file named orange.2 exists before you give the cp command. The user answers y to overwrite the file:

$ cp – i orange orange.2

cp: overwrite 'orange.2'? y

The cp command line in Figure 3-2 copies the file named memo to memo.copy. The period is part of the filename—just another character. The initial ls command shows that memo is the only file in the directory. After the cp command, a second ls shows two files in the directory, memo and memo.copy.

Sometimes it is useful to incorporate the date in the name of a copy of a file. The following example includes the date January 30 (0130) in the copied file:

$ cp memo memo.0130

Although it has no significance to Linux, the date can help you find a version of a file that you created on a certain date. It can also help you avoid overwriting existing files by providing a unique filename each day.

Use scp  or ftp  when you need to copy a file from one system to another on a common network.

Getting the help for commands on Linux

Most GNU utilities have a ––help option that displays information about the utility.

 

$ cat --help

Usage: cat [OPTION] [FILE]...
Concatenate FILE(s), or standard input, to standard output.
  -A, --show-all           equivalent to -vET
  -b, --number-nonblank    number nonblank output lines
  -e                       equivalent to -vE
  -E, --show-ends          display $ at end of each line

...

If the information that ––help displays runs off the screen, send the output through the less pager (page 31) using a pipe:

$ ls --help | less

Aborting Execution of jobs in Linux / Unix

To terminate a program from a character-based display, press the interrupt key (CONTROL-C or sometimes DELETE or DEL). When you press this key, the Linux operating system sends a terminal interrupt signal both to the program you are running and to the shell. Exactly what effect this signal has depends on the program. Some programs stop execution immediately; others ignore the signal. Some programs take other actions. When it receives a terminal interrupt signal, the shell displays a prompt and waits for another command.

If these methods do not terminate the program, try stopping the program with the suspend key (typically CONTROL-Z), giving the jobs command to verify the job number of the program, and using kill to abort the program. The job number is the number within the brackets at the left end of the line that jobs displays ([1]). The kill command sends a signal to the job specified as its argument. You must precede the job number with a percent sign (%1):

$ bigjob
^Z
[1]+  Stopped                 bigjob

$ jobs
[1]+  Stopped                 bigjob

$ kill %1
[1]+  Stopped                 bigjob

$ RETURN
[1]+  Killed                  bigjob

By default kill sends a software termination signal (–TERM). When this signal does not work, try using a kill (–KILL) signal:

$ kill -KILL %1

A running program cannot ignore a kill signal—it is sure to abort the program. The kill command returns a prompt; press RETURN again to see the confirmation message.

What are emulators for Linux?

Linux supports programs, called emulators, that run code intended for other operating systems. By using emulators you can run some DOS, Windows, and Macintosh programs under Linux. Wine (www.winehq.com) is an open-source implementation of the Windows API on top of X and UNIX/Linux; QEMU (fabrice.bellard.free.fr/qemu) is a CPU-only emulator that executes x86 Linux binaries on non-x86 Linux systems.

What do GNU stands for?

GNU, which stands for Gnu's Not UNIX, is the name for the complete UNIX-compatible software system which I am writing so that I can give it away free to everyone who can use it.

List of Environment variables in Linux

Variable name	Stored information
DISPLAY	used by the X Window system to identify the display server
DOMAIN	domain name
EDITOR	stores your favorite line editor
HISTSIZE	size of the shell history file in number of lines
HOME	path to your home directory
HOSTNAME	local host name
INPUTRC	location of definition file for input devices such as keyboard
LANG	preferred language
LD_LIBRARY_PATH	paths to search for libraries
LOGNAME	login name
MAIL	location of your incoming mail folder
MANPATH	paths to search for man pages
OS	string describing the operating system
OSTYPE	more information about version etc.
PAGER	used by programs like man which need to know what to do in case output is more than one terminal window.
PATH	search paths for commands
PS1	primary prompt
PS2	secondary prompt
PWD	present working directory
SHELL	current shell
TERM	terminal type
UID	user ID
USER(NAME)	user name
VISUAL	your favorite full-screen editor
XENVIRONMENT	location of your personal settings for X behavior
XFILESEARCHPATH	paths to search for graphical libraries

Which shell am I using?

 

To find out which shell you are currently using, type the following command:

echo $SHELL

The result of that command will tell you what your current shell is and may look something like:

/bin/bash

That would mean that you are using the bash shell. On a side note, you would also realise that this shell resides in the bin directory.

Preserve permissions and ownership when copying files using cp

We use the -p flag to tell cp to preserve the permissions and ownership of the files involved. So if we wanted to copy a folder recursively preserving permissions and ownership of the folder and the files contained in it, we would use the following command (preferably as root):

cp -R -p /source/folder/ /destination/folder/

All the files in the source folder would be copied to the destination folder and the permissions and owners would be preserved.

If we had a single file it would be similar:

cp -p /path/to/file-a /path/to/file-b

To check that the owner and permissions have been preserved, we can simply use:

ls -altp /destination/folder/

The result of the above command would show us the permissions that the files in the destination folder have.

AutoSys Scheduler: Create & Manage Jobs using AutoSys Job Scheduler

AutoSys is used for defining, scheduling and monitoring jobs. These jobs can be a UNIX script, java program or any other program which can be invoked from shell. Before starting we assume that user has already setup an AutoSys environment. This environment consists of autosys server and autosys client.

AutoSys System components

1. Event server (AutoSys database)
2. Event processor
3. Remote agent

Event Server

The event server is a AutoSys database which stores all system information and events as well as all job, monitor, and report definitions. Sometimes this database is also called as a data server, which actually describes a server instance. That is, it is either a UNIX or Windows process, and it is associated data space (or raw disk storage), that can include multiple databases or tablespaces.

Event Processor

This is main component of the autosys system. This processes all the events it reads from dataserver. The event processor is the program, running either as a UNIX process or as a Windows service that actually runs AutoSys. It schedules and starts jobs. When you start the event processor it continually scans the database for events to be processed. When it finds one, it checks whether the event satisfies the starting conditions for any job in the database.

Remote Agent

On a UNIX machine, the remote agent is a temporary process started by the event processor to perform a specific task on a remote (client) machine. On a Windows machine, the remote agent is a Windows service running on a remote (client) machine that is directed by the event processor to perform specific tasks.
The remote agent starts the command specified for a given job, sends running and completion information about a task to the event server, then exits. If the remote agent is unable to transfer the information, it waits and tries again until it can successfully communicate with the database.

Basic functionality of AutoSys

Below is the diagram which explains the basic functionality, please check the explanation.

Explanation

1. The event processor scans the event server for the next event to process. If no event is ready, the event processor scans again in five seconds.
2. The event processor reads from the event server that an event is ready. If the event is a STARTJOB event, the job definition and attributes are retrieved from the Event Server, including the command and the pointer (full path name on the client machine) to the profile file to be used for the job. In addition, for jobs running on Windows machines, the event processor retrieves from the database the user IDs and passwords required to run the job on the client machine.
3. The event processor processes the event. If the event is a STARTJOB, the event processor attempts to establish a connection with the remote agent on the client machine, and passes the job attributes to the client machine.
The event processor sends a CHANGE_STATUS event marking in the event server that the job is in STARTING state.
4. On a UNIX machine, the inetd invokes the remote agent. On a Windows machine, the remote agent logs onto the machine as the user defined as the job’s owner, using the user IDs and passwords passed to it from the event processor.
5. The remote agent sends an acknowledgment back to the event processor indicating that it has received the job parameters. The socket connection is terminated. At this point, the event processor resumes scanning the event server database, looking for events to process.
6. The remote agent starts a process and executes the command in the job definition.
7. The remote agent issues a CHANGE_STATUS event marking in the event server that the job is in RUNNING state.
8. The client job process runs to completion, then returns an exit code to the remote agent and quits.

Defining autosys job

There are various parameters to define autosys job. Starting from profile, timezone, start time, starting condition and so on. There are the two methods you can use to create job definitions:
1. Using the AutoSys Graphical User Interface (GUI).
2. Using the AutoSys Job Information Language (JIL) through a command-line interface.
In this tutorial we will use JIL language to create autosys jobs.
JIL stands for Job Information Language. Using this you can instruct autosys to save job definitions. This information saved in autosys database. You can also create a jil file which contains job definition. You can then pass this jil file to autosys.
Essential attributes for defining job
1. Job Name
JIL Keyword : insert_job. Name used to identify the job.
2. Job Type
a. JIL Keyword : job_type. The job type is one of job types: command (c), file watcher (f) or box (b).
3. Owner
a. JIL Keyword : owner
The job owner specifies whose user ID the command will be run under on the client machine. This attribute is automatically set to the user who invoked jil or the GUI to define the job, and cannot be changed except by the edit superuser.
Other job attributes:
1. command: The command attribute can be the name of any command, executable, UNIX shell script or batch file, and its arguments.
2. machine: This attribute specifies the client machine on which the command should be run.
3. date_condition: The start date/time dependencies attribute is a toggle, which specifies whether or not there are date, time, or both, conditions required for starting the job.
4. days_of_week: The days of the week attribute specifies the days on which the job should be run.
Sample jil file for command job echoJob.jil

insert_job:echoJob

machine:unixMachine

owner:username

command:echo “Hello this is command job”

 

To add this job in atosys db. Run following command from unix:

jil < echoJob.jil

 

Commands to control the job

Start job command

sendevent –E FORCE_STARTJOB -J <job_name>

 

sendevent -E STARTJOB -J <job_name>

 

To put jobs on OFF ICE or ON ICE

sendevent -E OFF_ICE -J <job_name>

 

sendevent -E ON_ICE -J <job_name>

 

sendevent -E KILLJOB –J "Job Name Here"

 

 

 

 

How to install second hard drive in Ubuntu Linux

So you’ve been using linux for awhile now, and it’s time to install another hard drive for some more storage? Maybe it’s finally time to wipe out a Window’s or NTFS partition.

In any event, I wrote this post so you would have a little help through this step. I’m going to assume that you already know how to install a hard drive. I’m also going to assume that you knew how to make it a master or slave, you’ve checked to make sure that it shows up in bios, and that it was intalled properly. It also assumes you’ve already formatted your drive in linux ext3 format, using a tool like gParted, or something similar.

If you’ve done all of these things, then boot up your system – and let’s get going. I’m using Ubuntu linux – formerly Breezy 5.10, but I’ve recently upgraded to Dapper 6.06.

Open up a terminal window and run the following command:
$ sudo fdisk -l

You should get a listing of the hard drives installed on your computer. There will be a little paragraph for each one that will look like this:


Disk /dev/hda: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 4678 37576003+ 83 Linux
/dev/hda2 4679 4865 1502077+ 5 Extended
/dev/hda5 4679 4865 1502046 82 Linux swap / Solaris

In windows disk drives are assigned an alphabet letter, and traditionally – floppy disk drives were a: and / or b:, and the main hard drive was c:/. Then the first cdrom or dvdrom was d:/, and any additional drives would be e:, f:, and so on. In linux it’s kind of the same, but in a different format. All hard drives installed are listed in the ‘device’ or /dev directory. All drives start with the appendage “hd” (I think for ‘hard drive’).

So, if you have 2 hard drives and one cdrom – then you have 3 devices. You have a hda, hdb, and hdc. The number of partitions comes next. If your main hard drive is linux – and you have 3 partitions, then you’ll have a hda1, hda2, and hda5. The partition numbers aren’t in a logical order – hda5 is always the swap partition.

So know, if you see your hard drive listed in the sudo fdisk -l, then you know you can mount it. Your computer has a startup file that tells it what to mount when it boots. We need to edit this file and add the new drive.

But first we have to create a directory. We have to create what’s called a “mount point”. A mount point is a virtual directory. It’s like saying – mount that hard drive from this directory.

The most logical place to create the mount point for the new hard drive is either the /mnt or /media folder. Many would say that /mnt is the only place it should be – the mount folder. However, Ubuntu always mounts all dvd, cd, and removable media in /media. I chose to make my mount point here for just that very reason.

Think of what you want to call the new mount point name. Just make sure you don’t use any special characters or spaces in the name. I called mine linuxstore. Now, in terminal run the following command substituting my mount point name for yours:
$ sudo mkdir /media/linuxstore

Next, here’s one of the most important things….and somethng that I didn’t find in any of the articles on the web when I was trying to figure out how to do this. You have to make the mount point directory ‘writable’. In other words, you have to give it writable permissions. They have to be world-writable permissions since you aren’t a member of the ‘root’ group in which all mount points are owned.

So, now you want to run the following command (again substituting my mount point name for yours):
$ chmod 777 /media/linuxstore

If you want to mount your drive right away, and you don’t care if it’s mounted automatically every time you boot – then in terminal run the following command:
$ mount /dev/hdd1 /media/linuxstore

There! Now you are temporarily mounted. But…if you want it to be permanent, you need to edit your filesystem tab file. Run the following command in terminal:
$ gedit /etc/fstab

The text editor window will appear with the fstab file loaded up. You will see something that looks kind of like this:

# /etc/fstab: static file system information.
#
# 
proc /proc proc defaults 0 0
/dev/hda1 / ext3 defaults,errors=remount-ro 0 1
/dev/hda5 none swap sw 0 0
/dev/hdb1 /media/hdb1 ext3 defaults 0 0
/dev/hdc /media/cdrom0 udf,iso9660 user,noauto 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0

All you have to do is add a new line for the new drive…
I will add the following line to my fstab for my new drive:

/dev/hdd1 /media/linuxstore ext3 defaults 0 0

Just be sure to substitute both the name of my hard drive for yours (mine is hdd1, is yours hdc1 or another name?), and my mount point hame for yours. Then save the file.

Now you will have the new hard drive mounted and writable both every time you boot. In Ubuntu, you should find your new drive listed under your ‘Places’ menu. To make the hard drive show up right now, without rebooting – just reload your fstab file with the following command:
$ sudo mount -a

Now you’re done! Enjoy your new storage drive in Linux!

Basic sed tricks

1. What is sed? - sed is stream editor, a Unix tool for working with streams of text data. See the awful truth about sed.
2. How do you substitute strings with sed? - Use ’s/old/new’ command, so sed ’s/hello/goodbye/’ would substitute the occurrence of the word hello to goodbye.
3. How do you inject text with sed? - & in the substitution string defines the pattern found in the search string. As an example, here’s us trying to find a word ‘hello’ and replacing it with ‘hello and how are you’:
        echo ‘hello there’ | sed ’s/^hello/& and how are you/’
4. Can I find several patterns and refer to them in the replacement string? - Yes, use (pattern) and then refer to your patterns as \1, \2, \3 and so on.
5. If the string is ‘old old old’ and I run ’s/old/new’, I get ‘new old old’ as the result. I need ‘new new new‘. - You forgot the global modifier, which would replace every occurrence of the pattern with the substitution. ’s/old/new/g‘ will work.
6. But I want ‘old old new’ from the previous example. - Just use the numeric modifier saying you want the third occurrence to be replaced. ’s/old/new/3‘ will work.
7. I wrote a rather complex sed script. How do I save and run it? - Assuming that your file is named myscript1.sed, you can invoke sed -f myscript1.sed.
8. How do I delete trailing whitespaces from each line? - sed ’s/[ \t]*$//’ Here we’re replacing any occurrence of a space or a tab with nothing. Check sed one-liners for more examples.
9. How do you print just a few first lines of the file? - sed 1q will give you just the first line, sed 10q the first 10 lines.
10. How do you replace a pattern only if it’s found, so that it’s executed faster? - Nest the replacement statement: sed ‘/old/ s/old/new/g’ file.txt

Linux network administrator questions

1. Give an example of set of shell commands that will give you the number of files in a directory
2. How do you tell what process has a TCP port open in Linux
3. On a Red Hat Linux Variant how do you control whether a service starts when the system boots
4. How do you tell the amount of free disk space left on a volume
5. Give an example of a set of shell commands to tell how many times “bob” has logged on to the system this month
6. Give an example of a recursively copying a directory from one location to another.
7. How do you modify the IP and Net mask of a system running a Red Hat Variant of Linux
8. Give an example of a set of shell commands that will give you the number of “httpd” processes running on a Linux box.
9. On CentOS or Fedora based system using the package management application, how do you tell what package provided the file “libnss_ldap.so”
10. What is the difference between VTP client, server, and transparent
11. What is the maximum length of CAT6
12. How does one set up a layer two link to share VLANs
13. How does one implement redundant links at Layer 2
14. What is the difference between a hub, switch, and a router? What are the security advantages of switch vs. hub?
15. Show an example of using telnet to learn the headers of an http server.
16. In what OSI layer does PPP exist
17. What’s the difference between TCP and UDP
18. Given a DNS server that has just started (with an empty cache) and host contacting this DNS server (using it’s OS setting) to learn an address for google.com, list the steps the DNS server will take to learn it with IP addresses (each step will have multiple possible IP addresses – you need choose only one per step).
19. Why are layer 2 loops bad, and what protocol was designed to prevent them
20. Given a radius server at 10.0.0.2 and a shared key of ‘abc123′ show the IOS commands necessary to authenticate switch users against the radius server, while still allowing the use of local username / password pairs