No I said this Way! IO Redirection

One of the most powerful tools in the Linux or Unix Scripting is that you are able to redirect StdIn StdOut and StdErr or even other Channels of IO Inter Process Communication
from and to Other Descriptors.

There are basicly 3 of them as i described above:
0 or the StdIn is the Standard Input descriptor.
1 or the StdOut is the standard output descriptor.
2 or the StdErr is the standard error descriptor.


You may use other descriptors like 3 or higher up to 255 for programming or scripting purposes, i.e when creating a script that uses zenity or dialog. Using

ls /dev/fd/

will reveal all descriptors you have available in the current shell.

Redirecting them is often necessary to get rid of nasty output or read from a file to read the output later as the output goes faster than your eye.
However you have to make sure that the noclobber option of bash is disabled to overwriting
existing files. Check this by using:

echo $SHELLOPTS

If you see noclobber here in the output you can disable this by using:

set +o noclobber

Basically you have these redirection options for the program myprogram

(1) myprogram > Output.log redirects StdOut to the file Output.log

(2) myprogram >> Output.log appends hate StdOut to the file Output.log

(3) myprogram 2> Output.log writes the StdErr to the Output.log. Respective the same for >>.

(4) myprogram &> Output.log writes StdErr and StdOut to the Output.log file.

Some administrators still using the older and more complected to write
myprogragramm > Output.log 2&>1 . These Construct basically appends the Stderr to StdOut
(2&>1) and than redirects both to the Output.log. Note that bash executes the descriptor redirection from the right hand side to the left hand side. In our example this was the redirection for StdErr appending to StdOut. You can use whatever Descriptor you want N&>M will always concatenate the descriptor N to descriptor M.

(5) myprogram < Input.lst the program reads its input from Input.lst

Remembering what i was writing about the noclobber option. Yeah actually i lied about this. You can force IO-Redicrection by using the >N| construct.

(6) myprogram >2| Output.log, will force the redirection to Output.log

Sometimes, if you write a demonized script for example you will find it useful l to close certain descriptors:

7) 2<&- will close the Input for StdErr.

And guess what

(8) 2>&- does? It closes the Output for StdtErr.

Again this does apply to any file descriptor you may bring to live or that already exists.

Here are two examples what else you can do with IO-Redirection:

If you don't have an Editor you may "create" one by using

cat > My.txt << EOF
foo
bar
EOF

to write foo and bar to the file My.txt. Clearing a file is often done by using
cat /dev/null > Output.log But using:
:> Output.log
is faster to write and does the same. This is because the : applies to nothing in bash. and :> overwrites the file with nothing.

Want Real Logging?

However if you want real logging in a script you have written you may want to use logger and create a proper facility which can be used by syslog or syslog-ng.

logger Script.Err mydaemon.sh

will send logging information to the syslog daemon. You can configure the logging target by editing the /etc/syslog.conf to determine the logging target.

The Art of Human Hacking

Hi Girls and Guys,

I just read this book:
http://www.social-engineer.org/social-engineering/the-art-of-human-hacking/

Not that i will encourage anybody to use these Techniques. But to be prepared against the weakest peace in a chain, the human is the best way to protect your infrastructure and yourself from security flaws. The CCC has some good videos form various congresses showing how easy it is to take over someones infrastructure:

here
and here

Have fun

Good Documentation Praxis

UPDATE (16.10.2011)

Hi girls and guys,

whenever there is something to reinstall or to recover there might be some additional tasks to fulfill to get everything back in the state it's supposed to be. Sure backing up a system often can safe you a lot of effort, but what's also necessary is to document your system and keep track of the things you are changing from the point on you are going away from a fresh vanilla setup.

There are several reasons why you should doing this and i will list them shortly:

Fixing Issues can often be a lot easier when you know exactly what has changed, which software was installed or upgraded recently and which configuration settings have changed.

Along comes the time where you have to get the hard track of a task that is very complex and a good documentation of your tracks can save you a lot of time.

On such systems where more than one administrator does the job documenting for others, so they are able to follow the steps your were working on, makes life for everyone a bit nicer.

Also it can be a good method to write down what you want to implement/deploy later on for your system.

So that nails it down to 4 very simple document entries in my personal Wiki:
Bugs --> where you describe errors and unusual behavior for yourself and others.

Howto --> where document stepwise what you are doing to fulfill a major task such as setting up a Web server for example.

Last action taken --> where you document whenever you change something on the system. Here you should also document whenever you update certain packages or change even big things.

Todo --> Tasks you not yet have managed to do, because of lack of time or maybe because you need to inform yourself how to get this done.

You can use a Web server based Wiki but you may want to give zim or rednotebook a try to accomplish this.

But using a wiki is only half the true. You also should comment your work. That means whenever you change a line in a configuration file, document it and whenever you are writing a programm or a script comment what the lines are meant to do, especially those line where you where sitting hours figuring out how to accomplish a particular task.

The next step is what is called a versioning system for your /etc folder. You may want to use the etckeeper. Since i use git for developing, i will use
git as version controling with etckeeper. I am not going in detail how to use
and install etckeeper since there are several blogs out there discussing this topic.

http://bryan-murdock.blogspot.com/2007/07/put-etc-under-revision-control-with-git.html
http://www.jukie.net/~bart/blog/20070312134706


You also should keep an eye on the history command. Some Editing may safe you time and work
in the future. So go on and edit your local or your global bashrc...
First expand your history from the default size of 1000 to maybe 100000 by changing or
definig these variables in your bashrc

HISTSIZE=1000000
HISTFILESIZE=1000000

If you want knowing when a specific command was executed, you may want to set the HISTTIMEFORMAT varibale, which is normaly not set. For example i set this to:

HISTTIMEFORMAT="%F-%M-%S --> "

So my History output looks like this:
996 2011-09-30-05-08 --> pull -u
997 2011-09-30-05-08 --> git pull -u
998 2011-09-30-05-08 --> cd

Preventing duplicated lines can be archived by
HISTCONTROL=ignoredups

Also you may want to set the HISTIGNORE variable:
HISTIGNORE="su *":"sudo *"

to avoid that somebody can determine which command were exacted by root.

that's all for today