Friday, November 22, 2013

Command Redirection

Output Redirection:

The output from a command normally intended for standard output can be easily can be diverted to a file instead. This capability is known as output redirection: this is useful to
If the notation > file is appended to any command that normally writes its output to standard output, the output of that command will be written to file instead of your terminal:
Check following who command which would redirect complete output of the command in users file.

$ who > users
Notice that no output appears at the terminal. This is because the output has been redirected from the default standard output device (the terminal) into the specified file. If you would check users file then it would have complete content:
$ cat users
root     :0           Nov  7 12:11
root     pts/0        Nov  7 12:12 (:0.0)
root     pts/1        Nov  7 12:20 (:0.0)
$
If a command has its output redirected to a file and the file already contains some data, that data will be lost. Consider this example:
$ echo line 1 > users
$ cat users
line 1
$
You can use >> operator to append the output in an existing file as follows:
$ echo line 2 >> users
$ cat users
line 1
line 2
$

Input Redirection:

Just as the output of a command can be redirected to a file, so can the input of a command be redirected from a file. As the greater-than character > is used for output redirection, the less-than character < is used to redirect the input of a command.
The commands that normally take their input from standard input can have their input redirected from a file in this manner. For example, to count the number of lines in the file users generated above, you can execute the command as follows:
$ wc -l users
2 users
$
Here it produces output 2 lines. You can count the number of lines in the file by redirecting the standard input of the wc command from the file users:
$ wc -l < users
2
$
Note that there is a difference in the output produced by the two forms of the wc command. In the first case, the name of the file users is listed with the line count; in the second case, it is not.
In the first case, wc knows that it is reading its input from the file users. In the second case, it only knows that it is reading its input from standard input so it does not display file name.

Here Document:

here document is used to redirect input into an interactive shell script or program.
We can run an interactive program within a shell script without user action by supplying the required input for the interactive program, or interactive shell script.
The general form for a here document is:
command << delimiter
document
delimiter
Here the shell interprets the << operator as an instruction to read input until it finds a line containing the specified delimiter. All the input lines up to the line containing the delimiter are then fed into the standard input of the command.
The delimiter tells the shell that the here document has completed. Without it, the shell continues to read input forever. The delimiter must be a single word that does not contain spaces or tabs.
Following is the input to the command wc -l to count total number of line:
$wc -l << EOF
         This is a simple lookup program
         for good (and bad) restaurants
         in Cape Town.
EOF
3
$
You can use here document to print multiple lines using your script as follows:
#!/bin/sh

cat << EOF
This is a simple lookup program
for good (and bad) restaurants
in Cape Town.
EOF     
This would produce following result:
This is a simple lookup program
for good (and bad) restaurants
in Cape Town.
The following script runs a session with the vi text editor and save the input in the file test.txt.
#!/bin/sh

filename=test.txt
vi $filename <<EndOfCommands
i
This file was created automatically from
a shell script
^[
ZZ
EndOfCommands
If you run this script with vim acting as vi, then you will likely see output like the following:
$ sh test.sh
Vim: Warning: Input is not from a terminal
$
After running the script, you should see the following added to the file test.txt:
$ cat test.txt
This file was created automatically from
a shell script
$

Discard the output:

Sometimes you will need to execute a command, but you don't want the output displayed to the screen. In such cases you can discard the output by redirecting it to the file /dev/null:
$ command > /dev/null
Here command is the name of the command you want to execute. The file /dev/null is a special file that automatically discards all its input.
To discard both output of a command and its error output, use standard redirection to redirect STDERR to STDOUT:
$ command > /dev/null 2>&1
Here 2 represents STDERR and 1 represents STDOUT. You can display a message on to STDERR by redirecting STDIN into STDERR as follows:
$ echo message 1>&2


Redirection Commands:

Following is the complete list of commands which you can use for redirection:
Command
Description
pgm > file
Output of pgm is redirected to file
pgm < file
Program pgm reads its input from file.
pgm >> file
Output of pgm is appended to file.
n > file
Output from stream with descriptor n redirected to file.
n >> file
Output from stream with descriptor n appended to file.
n >& m
Merge output from stream n with stream m.
n <& m
Merge input from stream n with stream m.
<< tag
Standard input comes from here through next tag at start of line.
|
Takes output from one program, or process, and sends it to another.

Note that file descriptor 0 is normally standard input (STDIN), 1 is standard output (STDOUT), and 2 is standard error output (STDERR).

Tuesday, November 19, 2013

Delete Files Older Than n Days on Linux


The find utility on linux allows you to pass in a bunch of interesting arguments, including one to execute another command on each file. We’ll use this in order to figure out what files are older than a certain number of days, and then use the rm command to delete them.
Command Syntax
find /path/to/files* -mtime +5 -exec rm {} \;
Note that there are spaces between rm, {}, and \;
Explanation
  • The first argument is the path to the files. This can be a path, a directory, or a wildcard as in the example above. I would recommend using the full path, and make sure that you run the command without the exec rm to make sure you are getting the right results.
  • The second argument, -mtime, is used to specify the number of days old that the file is. If you enter +5, it will find files older than 5 days.
  • The third argument, -exec, allows you to pass in a command such as rm. The {} \; at the end is required to end the command.
This should work on Ubuntu, Suse, Redhat, or pretty much any version of linux.

Monday, October 14, 2013

Reading self path/name/parameter from bash

Below is script and test run to get parameter,path,script file name etc.

#!/bin/bash

echo
echo "arguments called with ---->  ${@}     "
echo "\$1 ---------------------->  $1       "
echo "\$2 ---------------------->  $2       "
echo "path to me --------------->  ${0}     "
echo "parent path -------------->  ${0%/*}  "
echo "my name ------------------>  ${0##*/} "
echo
exit

# ------------- Test script------------- #

$  
/home/myscript/my_setting.sh
"'My name is'" "'Ryan'"
# ------------- RESULTS ------------- # arguments called with ---> 'My name is' 'Ryan' $1 ----------------------> 'My name is' $2 ----------------------> 'Ryan' path to me --------------> /home/myscript/my_setting.sh parent path -------------> /home/myscript my name -----------------> my_setting.sh

Thursday, August 22, 2013

Script converting seconds to time format

Here is simple script to convert seconds to time (hh:mm:ss) format.

#!/bin/bash
convertsectotime() {
 ((h=${1}/3600))
 ((m=(${1}%3600)/60))
 ((s=${1}%60))
 printf "%02d:%02d:%02d\n" $h $m $s
}
#TIME1="36"
echo "enter seconds:"
read TIME1
echo $(convertsectotime $TIME1)



Wednesday, July 31, 2013

Operators in Bash script

1 String comparison operators

(1) s1 = s2
(2) s1 != s2
(3) s1 < s2
(4) s1 > s2
(5) -n s1
(6) -z s1

(1) s1 matches s2
(2) s1 does not match s2
(3) __TO-DO__
(4) __TO-DO__
(5) s1 is not null (contains one or more characters)
(6) s1 is null

2 Arithmetic operators

+
-
*
/
% (remainder)

3 Arithmetic relational operators

-lt (<)
-gt (>)
-le (<=)
-ge (>=)
-eq (==)
-ne (!=)

Tuesday, July 16, 2013

Splitting string using cut command in Linux

cut command can be used to split the string with specific split character

command:

echo "18:30:00"|cut -d ':' -f1
18

echo "18:30:00"|cut -d ':' -f2

30

echo "18:30:00"|cut -d ':' -f3

00

Sunday, July 14, 2013

split string in bash

here is example using sed command.

echo "my name is;chotelal" | sed -e 's/;/\n/g'
myname is
chotelal


File test option in shell

These are the options to be used in conditional statement to check file attributes.

-e
file exists

example:
      
      if [ -b "$filename" ]
      then
        echo "$filename exist."
      fi

-a
file exists
This is same in effect to –e, but now it is discouraged to use.
-f
file is a regular file (not a directory or device file)
-s
file is not zero size
-d
file is a directory
-b
file is a “block device”
-c
file is a “character device”
-p
file is a “pipe”
-h
file is a “symbolic link”

-L
file is a symbolic link
-S
file is a “socket”
-r
file has read permission (for the user running the test)
-w
file has write permission (for the user running the test)
-x
file has execute permission (for the user running the test)

-k
sticky bit set
Commonly known as the sticky bit, the save-text-mode flag is a special type of file permission. If a file has this flag set, that file will be kept in cache memory, for quicker access.  
-O
you are owner of file
-G
group-id of file same as yours
-N
file modified since it was last read
f1 -nt f2
file f1 is newer than f2
f1 -ot f2
file f1 is older than f2
f1 -ef f2
files f1 and f2 are hard links to the same file
!
"not" -- reverses the sense of the tests above (returns true if condition absent).


how to find files with specific date range

There seems to be no direct option in find command to specify the date and get the files with specified modified date, but find command provide -newer option which will find files newer than file specified. so, here is small tweak which worked for me.

Variable:
Sdate=07/02/2013
Stime=19:00:00
Edate=07/05/2013
Etime=18:00:00
WRKDIR=/data/log
TEMPDIR= /home/dba/temp

1. create a file with modified date as start date.

> touch -d "$Sdate $Stime" $TEMPDIR/start

2. create a file with modified date as end date

> touch -d "$Edate $Etime" $TEMPDIR/end

3. find command

> find $JIRDIR -newer $TEMPDIR/start -and -not -newer  $TEMPDIR/end >$TEMPDIR/tmp_jir.txt

"tmp_jir.txt" file contains file names between the specified date range.


Saturday, June 29, 2013

Script command in linux

Script command is very useful way to record logs of any command fired in shell, it logs anything printed on screen.

Example:
--------

script myfile.txt
Will log all results into the file myfile.txt. This will open a sub-shell and records all information through this session. The script ends when the forked shell exits (user types exit) or when CTRL-D is typed.

Killing child process in Linux

I had requirement to stop one of your stats collection process at 7 AM, the stats collection was running from one .sh file which was running a .sql file to collect stats.so, to kill entire process i had to find child process (here child process is .sql file).

I searched through the internet and found various tricks to do it, but most of them seems to be not supporting in my bash version. it tried it myself ,here is the code which is working for me.


export xpid2=`ps -ef|grep -v grep|grep collect_daily_stats2|awk '{print $2}'`
echo "parent:"$xpid2
export cpid2=`ps -ef|grep -v grep|grep "$xpid2"|awk '{print $2}'|sed -s '1d'`
echo "child:"$cpid2
kill -9 "$cpid2"
echo "daily stats2 stopped"
 mailx -s"Daily stats2 stopped successfully" teradatamanager@cvscaremark.com  < /dev/null
else


any suggestions are welcome.

Thursday, May 23, 2013

find string in file using find command


Here is the syntax to find a word in a file.

find . -type f -exec grep -l "word" {} +

Friday, March 15, 2013

List file name containing perticular text in Linux


In Linux Shell to find a particular text string in all the files from the current directory recursively, use find command
 find . -exec grep -l "World" {} \;
This command searches through all directories from the current directory recursively for the files that contain the string “World”.

Friday, March 8, 2013

How to get Linux version information

There are various ways we can get Linux version information  and commands below are commonly used in various Linux distribution.

1. by simply viewing file /proc/version using cat.


[root@localhost /]# cat /proc/version
Linux version 2.6.18-164.el5 (mockbuild@x86-002.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Tue Aug 18 15:51:54 EDT 2009


2. by using "uname" command.

[root@localhost /]# uname -a
Linux localhost.localdomain 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:54 EDT 2009 i686 i686 i386 GNU/Linux

3. by using "lsb_release" commands.

[root@localhost /]# lsb_release -a
LSB Version:    :core-3.1-ia32:core-3.1-noarch:graphics-3.1-ia32:graphics-3.1-noarch
Distributor ID: RedHatEnterpriseServer
Description:    Red Hat Enterprise Linux Server release 5.4 (Tikanga)
Release:        5.4
Codename:       Tikanga

4. by using "dmesg" command (display kernal message buffer) .

[root@localhost /]# dmesg |grep "Linux version"
Linux version 2.6.18-164.el5 (mockbuild@x86-002.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Tue Aug 18 15:51:54 EDT 2009

this command is very useful to get various configuration information from Linux system.