Hi Girls and Guys,
today i will tell you something about anacron.
With anacron you are able to install and maintain a Crontab which runs periodically but asynchronous. Now what does this mean?
Imagine you have a Job for a laptop which you want to execute on regular basis but because you don't know on which times your computer actually is turned on and therefore it might be difficult and even impossible to setup such jobs on fixed times. Anacron does not assume that the machine you are working on is running 24/7 like a productive server.
I will guide you through setting up a job for your anacron having a virusscan with avgscan but first let me tell you a little bit more about anacron: First of all anacron is not an other cron daemon. Rather think of it as an extension to the already installed Cron Daemon. If you first
installing ancron for your distribution anacron sets up a script in /etc/cron.hourly to startup /usr/sbin/anacron every hour reading and executing the commands you have specified in your /etc/anacrontab. In openSUSE you can install anacron using zypper. Simply type:
zypper in cronie-anacron
as root to install it.
Now lets take a look at the /etc/anacrontab:
++++++++
SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
RANDOM_DELAY=45
START_HOURS_RANGE=18-23
#period in days delay in minutes job-identifier command
@daily 30 cron.avg_l1zard avgscan -W /home/l1zard/.wine -awbHpPcmj -r /var/log/avg_l1zard.report /home/l1zard/
@daily 30 cron.avg_tmp avgscan -awbHpPcmj -r /var/log/avg_tmp.report /tmp/
@daily 30 cron.avg_root avgscan -awbHpPcmj -r /var/log/avg_root.report /root/
++++++++The variable SHELL tells the anacron program which Shell is going to be used to execute the commands. The PATH variable is like the environment variable PATH in bash which tells the shell where to look for executable files. With MAILTO you can specify a list of users which will receive a system informing these users about success or failure regarding these jobs. With RANDOM_DELAY you can specify the maximum range of the random delay which is added to the base delay setup in column 2. The START_HOURS_RANGE is the most interesting part of this configuration file. Here you can setup the lapse of time per day in which anacron should run. This is useful if you use commands which are using a lot of system resources and you want anacron to this jobs in a time where you can effort such loss of performence to other tasks.
Now lets have a look at the more interesting table. The first column you can specify the period in days on which you wish running the jobs. You may also use macros such as @daily @monthly or @weekly to run jobs on daily, monthly or weekly basis. In the second column you say anacron the base delay. Anacron waits this amount of time plus the RANDOM_DELAY before it starts any jobs. This way you can be sure that anacron will give you some time to do other jobs after you have logged in. The 3rd column is the name of the spooler file and the id of the job. You will find a file called /var/spool/anacron/job_identifier.The last column of the jobtable is the job including all options and parameters given to this command. However you must ensure that the program you want to run is in the PATH of your configuration file.
So lets test our anacrontab as a final step. Executing
anacron -T && echo "ok"
will test the for us whether the anacrontab we just installed is valid. And with
anacron -fs
we can test whether the commands we have are doing what we want them to do for us. The -s option is used to tell anacron to execute jobs in a row rather than all together. Remember you get a mail where you will be informed about success or failure of the jobs you want to have anacron executed.
That's it for today folks.
Donnerstag, 10. November 2011
Run Jobs asynchronous with anacron
Mittwoch, 26. Oktober 2011
No I said this Way! IO Redirection
One of the most powerful tools in the Linux or Unix Scripting is that you are able to redirect StdIn StdOut and StdErr or even other Channels of IO Inter Process Communication
from and to Other Descriptors.
There are basicly 3 of them as i described above:
0 or the StdIn is the Standard Input descriptor.
1 or the StdOut is the standard output descriptor.
2 or the StdErr is the standard error descriptor.
You may use other descriptors like 3 or higher up to 255 for programming or scripting purposes, i.e when creating a script that uses zenity or dialog. Using
ls /dev/fd/
will reveal all descriptors you have available in the current shell.
Redirecting them is often necessary to get rid of nasty output or read from a file to read the output later as the output goes faster than your eye.
However you have to make sure that the noclobber option of bash is disabled to overwriting
existing files. Check this by using:
echo $SHELLOPTS
If you see noclobber here in the output you can disable this by using:
set +o noclobber
Basically you have these redirection options for the program myprogram
(1) myprogram > Output.log redirects StdOut to the file Output.log
(2) myprogram >> Output.log appends hate StdOut to the file Output.log
(3) myprogram 2> Output.log writes the StdErr to the Output.log. Respective the same for >>.
(4) myprogram &> Output.log writes StdErr and StdOut to the Output.log file.
Some administrators still using the older and more complected to write
myprogragramm > Output.log 2&>1 . These Construct basically appends the Stderr to StdOut
(2&>1) and than redirects both to the Output.log. Note that bash executes the descriptor redirection from the right hand side to the left hand side. In our example this was the redirection for StdErr appending to StdOut. You can use whatever Descriptor you want N&>M will always concatenate the descriptor N to descriptor M.
(5) myprogram < Input.lst the program reads its input from Input.lst
Remembering what i was writing about the noclobber option. Yeah actually i lied about this. You can force IO-Redicrection by using the >N| construct.
(6) myprogram >2| Output.log, will force the redirection to Output.log
Sometimes, if you write a demonized script for example you will find it useful l to close certain descriptors:
7) 2<&- will close the Input for StdErr.
And guess what
(8) 2>&- does? It closes the Output for StdtErr.
Again this does apply to any file descriptor you may bring to live or that already exists.
Here are two examples what else you can do with IO-Redirection:
If you don't have an Editor you may "create" one by using
cat > My.txt << EOF
foo
bar
EOF
to write foo and bar to the file My.txt. Clearing a file is often done by using
cat /dev/null > Output.log But using:
:> Output.log
is faster to write and does the same. This is because the : applies to nothing in bash. and :> overwrites the file with nothing.
Want Real Logging?
However if you want real logging in a script you have written you may want to use logger and create a proper facility which can be used by syslog or syslog-ng.
logger Script.Err mydaemon.sh
will send logging information to the syslog daemon. You can configure the logging target by editing the /etc/syslog.conf to determine the logging target.
from and to Other Descriptors.
There are basicly 3 of them as i described above:
0 or the StdIn is the Standard Input descriptor.
1 or the StdOut is the standard output descriptor.
2 or the StdErr is the standard error descriptor.
You may use other descriptors like 3 or higher up to 255 for programming or scripting purposes, i.e when creating a script that uses zenity or dialog. Using
ls /dev/fd/
will reveal all descriptors you have available in the current shell.
Redirecting them is often necessary to get rid of nasty output or read from a file to read the output later as the output goes faster than your eye.
However you have to make sure that the noclobber option of bash is disabled to overwriting
existing files. Check this by using:
echo $SHELLOPTS
If you see noclobber here in the output you can disable this by using:
set +o noclobber
Basically you have these redirection options for the program myprogram
(1) myprogram > Output.log redirects StdOut to the file Output.log
(2) myprogram >> Output.log appends hate StdOut to the file Output.log
(3) myprogram 2> Output.log writes the StdErr to the Output.log. Respective the same for >>.
(4) myprogram &> Output.log writes StdErr and StdOut to the Output.log file.
Some administrators still using the older and more complected to write
myprogragramm > Output.log 2&>1 . These Construct basically appends the Stderr to StdOut
(2&>1) and than redirects both to the Output.log. Note that bash executes the descriptor redirection from the right hand side to the left hand side. In our example this was the redirection for StdErr appending to StdOut. You can use whatever Descriptor you want N&>M will always concatenate the descriptor N to descriptor M.
(5) myprogram < Input.lst the program reads its input from Input.lst
Remembering what i was writing about the noclobber option. Yeah actually i lied about this. You can force IO-Redicrection by using the >N| construct.
(6) myprogram >2| Output.log, will force the redirection to Output.log
Sometimes, if you write a demonized script for example you will find it useful l to close certain descriptors:
7) 2<&- will close the Input for StdErr.
And guess what
(8) 2>&- does? It closes the Output for StdtErr.
Again this does apply to any file descriptor you may bring to live or that already exists.
Here are two examples what else you can do with IO-Redirection:
If you don't have an Editor you may "create" one by using
cat > My.txt << EOF
foo
bar
EOF
to write foo and bar to the file My.txt. Clearing a file is often done by using
cat /dev/null > Output.log But using:
:> Output.log
is faster to write and does the same. This is because the : applies to nothing in bash. and :> overwrites the file with nothing.
Want Real Logging?
However if you want real logging in a script you have written you may want to use logger and create a proper facility which can be used by syslog or syslog-ng.
logger Script.Err mydaemon.sh
will send logging information to the syslog daemon. You can configure the logging target by editing the /etc/syslog.conf to determine the logging target.
Montag, 24. Oktober 2011
The Art of Human Hacking
Hi Girls and Guys,
I just read this book:
http://www.social-engineer.org/social-engineering/the-art-of-human-hacking/
Not that i will encourage anybody to use these Techniques. But to be prepared against the weakest peace in a chain, the human is the best way to protect your infrastructure and yourself from security flaws. The CCC has some good videos form various congresses showing how easy it is to take over someones infrastructure:
here
and here
Have fun
I just read this book:
http://www.social-engineer.org/social-engineering/the-art-of-human-hacking/
Not that i will encourage anybody to use these Techniques. But to be prepared against the weakest peace in a chain, the human is the best way to protect your infrastructure and yourself from security flaws. The CCC has some good videos form various congresses showing how easy it is to take over someones infrastructure:
here
and here
Have fun
Sonntag, 16. Oktober 2011
Good Documentation Praxis
UPDATE (16.10.2011)
Hi girls and guys,
whenever there is something to reinstall or to recover there might be some additional tasks to fulfill to get everything back in the state it's supposed to be. Sure backing up a system often can safe you a lot of effort, but what's also necessary is to document your system and keep track of the things you are changing from the point on you are going away from a fresh vanilla setup.
There are several reasons why you should doing this and i will list them shortly:
Fixing Issues can often be a lot easier when you know exactly what has changed, which software was installed or upgraded recently and which configuration settings have changed.
Along comes the time where you have to get the hard track of a task that is very complex and a good documentation of your tracks can save you a lot of time.
On such systems where more than one administrator does the job documenting for others, so they are able to follow the steps your were working on, makes life for everyone a bit nicer.
Also it can be a good method to write down what you want to implement/deploy later on for your system.
So that nails it down to 4 very simple document entries in my personal Wiki:
Bugs --> where you describe errors and unusual behavior for yourself and others.
Howto --> where document stepwise what you are doing to fulfill a major task such as setting up a Web server for example.
Last action taken --> where you document whenever you change something on the system. Here you should also document whenever you update certain packages or change even big things.
Todo --> Tasks you not yet have managed to do, because of lack of time or maybe because you need to inform yourself how to get this done.
You can use a Web server based Wiki but you may want to give zim or rednotebook a try to accomplish this.
But using a wiki is only half the true. You also should comment your work. That means whenever you change a line in a configuration file, document it and whenever you are writing a programm or a script comment what the lines are meant to do, especially those line where you where sitting hours figuring out how to accomplish a particular task.
The next step is what is called a versioning system for your /etc folder. You may want to use the etckeeper. Since i use git for developing, i will use
git as version controling with etckeeper. I am not going in detail how to use
and install etckeeper since there are several blogs out there discussing this topic.
http://bryan-murdock.blogspot.com/2007/07/put-etc-under-revision-control-with-git.html
http://www.jukie.net/~bart/blog/20070312134706
You also should keep an eye on the history command. Some Editing may safe you time and work
in the future. So go on and edit your local or your global bashrc...
First expand your history from the default size of 1000 to maybe 100000 by changing or
definig these variables in your bashrc
HISTSIZE=1000000
HISTFILESIZE=1000000
If you want knowing when a specific command was executed, you may want to set the HISTTIMEFORMAT varibale, which is normaly not set. For example i set this to:
HISTTIMEFORMAT="%F-%M-%S --> "
So my History output looks like this:
996 2011-09-30-05-08 --> pull -u
997 2011-09-30-05-08 --> git pull -u
998 2011-09-30-05-08 --> cd
Preventing duplicated lines can be archived by
HISTCONTROL=ignoredups
Also you may want to set the HISTIGNORE variable:
HISTIGNORE="su *":"sudo *"
to avoid that somebody can determine which command were exacted by root.
that's all for today
Hi girls and guys,
whenever there is something to reinstall or to recover there might be some additional tasks to fulfill to get everything back in the state it's supposed to be. Sure backing up a system often can safe you a lot of effort, but what's also necessary is to document your system and keep track of the things you are changing from the point on you are going away from a fresh vanilla setup.
There are several reasons why you should doing this and i will list them shortly:
Fixing Issues can often be a lot easier when you know exactly what has changed, which software was installed or upgraded recently and which configuration settings have changed.
Along comes the time where you have to get the hard track of a task that is very complex and a good documentation of your tracks can save you a lot of time.
On such systems where more than one administrator does the job documenting for others, so they are able to follow the steps your were working on, makes life for everyone a bit nicer.
Also it can be a good method to write down what you want to implement/deploy later on for your system.
So that nails it down to 4 very simple document entries in my personal Wiki:
Bugs --> where you describe errors and unusual behavior for yourself and others.
Howto --> where document stepwise what you are doing to fulfill a major task such as setting up a Web server for example.
Last action taken --> where you document whenever you change something on the system. Here you should also document whenever you update certain packages or change even big things.
Todo --> Tasks you not yet have managed to do, because of lack of time or maybe because you need to inform yourself how to get this done.
You can use a Web server based Wiki but you may want to give zim or rednotebook a try to accomplish this.
But using a wiki is only half the true. You also should comment your work. That means whenever you change a line in a configuration file, document it and whenever you are writing a programm or a script comment what the lines are meant to do, especially those line where you where sitting hours figuring out how to accomplish a particular task.
The next step is what is called a versioning system for your /etc folder. You may want to use the etckeeper. Since i use git for developing, i will use
git as version controling with etckeeper. I am not going in detail how to use
and install etckeeper since there are several blogs out there discussing this topic.
http://bryan-murdock.blogspot.com/2007/07/put-etc-under-revision-control-with-git.html
http://www.jukie.net/~bart/blog/20070312134706
You also should keep an eye on the history command. Some Editing may safe you time and work
in the future. So go on and edit your local or your global bashrc...
First expand your history from the default size of 1000 to maybe 100000 by changing or
definig these variables in your bashrc
HISTSIZE=1000000
HISTFILESIZE=1000000
If you want knowing when a specific command was executed, you may want to set the HISTTIMEFORMAT varibale, which is normaly not set. For example i set this to:
HISTTIMEFORMAT="%F-%M-%S --> "
So my History output looks like this:
996 2011-09-30-05-08 --> pull -u
997 2011-09-30-05-08 --> git pull -u
998 2011-09-30-05-08 --> cd
Preventing duplicated lines can be archived by
HISTCONTROL=ignoredups
Also you may want to set the HISTIGNORE variable:
HISTIGNORE="su *":"sudo *"
to avoid that somebody can determine which command were exacted by root.
that's all for today
Mittwoch, 27. Juli 2011
My Backup Strategy Part I
today i will tell you something about the most important tasks of maintaining a system up and running. Today i will show you my backup solution.
In fact my backup solution does not involve technology you will find on server infrastructure such as raid or lvm. I was able to get all my system and private data backuped without this sophisticated technology. The reason why i don't use backup with raid and lvm is, that raid does not prevent you from doing stupid things. For example when you delete data on a RAID 1 array you also delete this data from all your backup disks as well.
I use only scripts and cron jobs and couple of extra hard disks. First thing i build in an extra hard disk of the same size which is mounted to /mnt/Backup.
This disk could be bigger than the disk where your operating system is sitting but it should not have less space.
For the systems backup you should realize that you do not need to save the data which were installed by rpm or debian packages. Instead to save time and space only save those files which are different from an installed or not installed by a system package. In suse you can use the yast backup tool which does exactly this. In debian you can create an iso image of installed packages from aptoncd program and find those files not installed and save this into a tar.gz file as the yast backup tool does.
With Yast you are able to easily set up a automated cronjob running once a week on friday 22 pm for example. It will save both the packages installed on your system and the list of packages you have installed. If you prefer a more stable backup you could use the script package_state and use the -s option to save the current state of packages in a tar.gz file.
In debian you can archive this by using:
dpkg -l|grep '^ii.*'|awk '{print $2}' >> installed.lst
apt-get install --download-only --from-file=installed.lst --targt=/Your/desired/directory
tar cvfz backuped_packages.tar.gz /Your/desired/directory
mv backuped_packages.tar.gz /mnt/Backup/system/
But i am not an Debian Expert. So there might be other ways for accomplishing this task.
Also i always make a copy of the whole /etc directory just in case. You could use etckeeper on debian systems. On rpm systems you have todo this task with the copy command:
cp -av /etc/* /mnt/Backup/system/etc/
With this solution when the backup disk fails you simply plug in a new disk a make a new backup. when the main disk fails, where your system is running on you plug in a new disk for system and recover your system from the packages you have previously installed.
And copying back the /etc directory and the files which were not installed
by any package.
Now for the users home directory i recommend to use separate /home partition so whenever installing a new system you dont need to delete these files. You can even backup the users files and settings wiht my rsync wrapper script
backup_userhome. You clearly need rsync to be installed. Afer that you can call the script with:
backup_userhome /mnt/Backup User
And restoration is just the other way around with restore_userhome
restore_userhome /mnt/Backup User
Be sure you delete all data from users home which where deployed during the insatllation, then have the restore script running and then login for the
first time. et vola all is back where it should be. you even can visit the files you were last working on by using the places in gnome.
thats it:
Part II will be written when gnu hurd ready or even earlier ;)
In fact my backup solution does not involve technology you will find on server infrastructure such as raid or lvm. I was able to get all my system and private data backuped without this sophisticated technology. The reason why i don't use backup with raid and lvm is, that raid does not prevent you from doing stupid things. For example when you delete data on a RAID 1 array you also delete this data from all your backup disks as well.
I use only scripts and cron jobs and couple of extra hard disks. First thing i build in an extra hard disk of the same size which is mounted to /mnt/Backup.
This disk could be bigger than the disk where your operating system is sitting but it should not have less space.
For the systems backup you should realize that you do not need to save the data which were installed by rpm or debian packages. Instead to save time and space only save those files which are different from an installed or not installed by a system package. In suse you can use the yast backup tool which does exactly this. In debian you can create an iso image of installed packages from aptoncd program and find those files not installed and save this into a tar.gz file as the yast backup tool does.
With Yast you are able to easily set up a automated cronjob running once a week on friday 22 pm for example. It will save both the packages installed on your system and the list of packages you have installed. If you prefer a more stable backup you could use the script package_state and use the -s option to save the current state of packages in a tar.gz file.
In debian you can archive this by using:
dpkg -l|grep '^ii.*'|awk '{print $2}' >> installed.lst
apt-get install --download-only --from-file=installed.lst --targt=/Your/desired/directory
tar cvfz backuped_packages.tar.gz /Your/desired/directory
mv backuped_packages.tar.gz /mnt/Backup/system/
But i am not an Debian Expert. So there might be other ways for accomplishing this task.
Also i always make a copy of the whole /etc directory just in case. You could use etckeeper on debian systems. On rpm systems you have todo this task with the copy command:
cp -av /etc/* /mnt/Backup/system/etc/
With this solution when the backup disk fails you simply plug in a new disk a make a new backup. when the main disk fails, where your system is running on you plug in a new disk for system and recover your system from the packages you have previously installed.
And copying back the /etc directory and the files which were not installed
by any package.
Now for the users home directory i recommend to use separate /home partition so whenever installing a new system you dont need to delete these files. You can even backup the users files and settings wiht my rsync wrapper script
backup_userhome. You clearly need rsync to be installed. Afer that you can call the script with:
backup_userhome /mnt/Backup User
And restoration is just the other way around with restore_userhome
restore_userhome /mnt/Backup User
Be sure you delete all data from users home which where deployed during the insatllation, then have the restore script running and then login for the
first time. et vola all is back where it should be. you even can visit the files you were last working on by using the places in gnome.
thats it:
Part II will be written when gnu hurd ready or even earlier ;)
Abonnieren
Posts (Atom)
Posts
