Author Archive

Linux User Group

Posted by on Tuesday, 7 April, 2009

Linux User Group’s are great places to get support, meet like minded people, and get free internet and hardware (sometimes :p)
New Zealand has a national LUG – the NZ Linux Usergroup – which is based in Auckland but has members from all around the country.

NZLUG Website

The NZLUG’s webpage has links to the other groups around the country – in Waikato Hawkes Bay, Wellington, Canterbury, Kapiti, Christchurch, Wairarapa and Duneden.
If you don’t have a LUG close to where you live, it might be a good idea to seek out some other pengiunheads and start one.

LUG’s tend to have a primary way of contact: electornic mailing lists.
These are special e-mail adresses that members “subscribe” to.
Then anyone who is subscribed can send an e-mail or reply to that address and it is sent to all the members. Kinda of like BCC’s but only better, because most have onlinen archives that are indexed by your favourite search engine (IE google).

Mailing List Ettiquite

When posting to a mailing list, please type in full, correct English (no MSN speak :p) and keep your message brief, polite and to the point (include more details if asked).
We also ask that you send e-mails in plain text rather than formatted HTML as HTML messages are likely to go straight into the trash without being read. Last-Modified: 2007-03-07 19:38:50


Cron

Posted by on Tuesday, 7 April, 2009

Technically, cron is just the clock daemon (/usr/sbin/cron or perhaps /usr/sbin/crond) that executes commands at specific times. However, a handful of configuration files and programs go into making up the cron package. Like many system processes, cron never ends.

The controlling files for cron are the cron-tables or crontabs. The crontabs are often located in /var/spool/cron/crontab. However, on SuSE you will find them in /var/spool/cron/tabs. The names of the files in this directory are the names of the users that submit the cron jobs.

Unlike other UNIX dialects, the Linux cron daemon does not sleep until the next cron job is ready. Instead, when cron completes one job, it will keep checking once a minute for more jobs to run. Also, you should not edit the files directly. You can edit them with a text editor like vi, though there is the potential for messing things up. Therefore, you should use the tool that Linux provides: crontab. (see the man-page for more details)

The crontab utility has several functions. It is the means by which files containing the cron jobs are submitted to the system. Second, it can list the contents of your crontab. If you are root, it can also submit and list jobs for any user. The problem is that jobs cannot be submitted individually. Using crontab, you must submit all of the jobs at the same time.

At first, that might sound a little annoying. However, lets take a look at the process of “adding” a job. To add a cron job, you must first list out the contents of the existing crontab with the -l option. If you are root and wish to add something to another user’s crontab, use the -u option followed by the user’s logname. Then redirect this crontab to a file, which you can then edit. (Note that on some systems crontab has -e (for “edit”), which will do all the work for you. See the man-page for more details.)

For example, lets say that you are the root user and want to add something to the UUCP user’s crontab. First, get the output of the existing crontab entry with this command:

crontab -l -u uucp >/tmp/crontab.uucp

To add an entry, simply include a new line. Save the file, get out of your editor, and run the crontab utility again. This time, omit the -l to list the file but include the name of the file. The crontab utility can also accept input from stdin, so you could leave off the file name and crontab would allow you to input the cronjobs on the command line. Keep in mind that any previous crontab is removed no matter what method you use.

The file /tmp/crontab.uucp now contains the contents of UUCPs crontab. It might look something like this:

39,9 * * * * /usr/lib/uucp/uudemon.hour > /dev/null
10 * * * * /usr/lib/uucp/uudemon.poll > /dev/null
45 23 * * * ulimit 5000; /usr/lib/uucp/uudemon.clean > /dev/null
48 10,14 * * 1-5 /usr/lib/uucp/uudemon.admin > /dev/null

Despite its appearance, each crontab entry consists of only six fields. The first five represent the time the job should be executed and the sixth is the actual command. The first five fields are separated by either a space or a tab and represent the following units, respectively:

  • minutes (0-59)
  • hour (0-23)
  • day of the month (1-31)
  • month of the year (1-12)
  • day of the week (0-6, 0=Sunday)

To specify all possible values, use an asterisk (*). You can specify a single value simply by including that one value. For example, the second line in the previous example has a value of 10 in the first field, meaning 10 minutes after the hour. Because all of the other four time fields are asterisks, this means that the command is run every hour of every day at 10 minutes past the hour.

Ranges of values are composed of the first value, a dash, and the ending value. For example, the fourth line has a range (1-5) in the day of the week column, meaning that the command is only executed on days 1-5, Monday through Friday.

To specify different values that are not within a range, separate the individual values by a column. In the fourth example, the hour field has the two values 10 and 14. This means that the command is run at 10 a.m. and 2 p.m.

Note that times are additive. Lets look at an example:

10 * 1,16 * 1-5 /usr/local/bin/command

The command is run 10 minutes after every hour on the first and sixteenth, as well as Monday through Friday. If either the first or the sixteenth were on a weekend, the command would still run because the day of the month field would apply. However, this does not mean that if the first is a Monday, the command is run twice.

The crontab entry can be defined to run at different intervals than just every hour or every day. The granularity can be specified to every two minutes or every three hours without having to put each individual entry in the crontab.

Lets say we wanted to run the previous command not at 10 minutes after the hour, but every ten minutes. We could make an entry that looked like this.:

0,10,20,30,40,50 * 1,16 * 1-5 /usr/local/bin/command

This runs every 10 minutes: at the top of the hour, 10 minutes after, 20 minutes after, and so on. To make life easier, we could simply create the entry like this:

*/10 * 1,16 * 1-5 /usr/local/bin/command

This syntax may be new to some administrators. (It was to me.) The slash (/) says that within the specific interval (in this case, every minute), run the command every so many minutes; in this case, every 10 minutes.

We can also use this even when we specify a range. For example, if the job was only supposed to run between 20 minutes after the hour and 40 minutes after the hour, the entry might look like this:

20-40 * 1,16 * 1-5 /usr/local/bin/command

What if you wanted it to run at these times, but only every three minutes? The line might look like this:

20-40/3 * 1,16 * 1-5 /usr/local/bin/command

To make things even more complicated, you could say that you wanted the command to run every two minutes between the hour and 20 minutes after, every three minutes between 20 and 40 minutes after, then every 5 minutes between 40 minutes after and the hour.

0-20/2,21-40/3,41-59/5 * 1,16 * 1-5 /usr/local/bin/command

One really nice thing that a lot of Linux dialects do is allow you to specify abbreviations for the days of the week and the months. Its a lot easier to remember that fri is for Friday instead of 5.

With the exception of certain errors in the time fields, errors are not reported until cron runs the command. All error messages and output is mailed to the users. At least that’s what the crontab man-page says and that is basically true. However, as you see in the previous examples, you are redirecting stdout to /dev/null. If you wanted to, you could also redirect stderr there and you would never see whether there were any errors.

Output is mailed to the user because there is no real terminal on which the cronjobs are being executed. Therefore, there is no screen to display the errors. Also, there is no keyboard to accept input. Does that mean you cannot give input to a cron job? No. Think back to the discussion on shell scripts. We can redefine stdin, stdout and stderr. This way they can all point to files and behave as we expect.

One thing I would like to point out is that I do not advocate doing redirection in the command field of the crontab. I like doing as little there as possible. Instead, I put the absolute path to a shell script. I can then test the crontab entry with something simple. Once that works, I can make changes to the shell script without having to resubmit the cronjob.

Keep in mind that cron is not exact. It synchronizes itself to the top of each minute. On a busy system in which you lose clock ticks, jobs may not be executed until a couple minutes after the scheduled time. In addition, there may be other processes with higher priorities that delay cron jobs. In some cases, (particularly on very busy systems) jobs might end up being skipped if they are run every minute.

Access is permitted to the cron facility through two files, both in /etc. If you have a file cron.allow, you can specify which users are allowed to use cron. The cron.deny says who are specifically not allowed to use cron. If neither file exists, only the system users have access. However, if you want everyone to have access, create an entry cron.deny file. In other words, no one is denied access.

It is often useful for root to run jobs as a different user without having to switch users (for example, using the su command). Most Linux dialects provide a mechanism in the form of the /etc/crontab file. This file is typically only writable by root and in some cases, only root can read it (which is often necessary in high security environments). The general syntax is the same as the standard crontabs, with a couple of exceptions.

The first difference is the header, which you can see here:

SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
59 *  * * *     root  rm -f /var/spool/cron/lastrun/cron.hourly
14 0  * * *     root  rm -f /var/spool/cron/lastrun/cron.daily
29 0  * * 6     root  rm -f /var/spool/cron/lastrun/cron.weekly
44 0  1 * *     root  rm -f /var/spool/cron/lastrun/cron.monthly

The SHELL variable defines the shell under which each command will run. The PATH variable is like the normal PATH environment variable and defines the search path. The MAILTO variable says who should get email messages, which includes error messages and the standard output of the executed commands.

The structure of the actual entries is pretty much the same with the exception of the user name (root in each case here). This way, the root users (or whoever can edit /etc/crontab) can define which user executes the command. Keep in mind that this can be a big security hole. If someone can write to this file, they can create an entry that runs as root and therefore has complete control of the system.

The next command in the cron “suite” is at. Its function is to execute a command at a specific time. The difference is that once the at job has run, it disappears from the system. As for cron, two files, at.allow and at.deny, have the same effect on the at program.

The batch command is also used to run commands once. However, commands submitted with batch are run when the system gets around to it, which means when the system is less busy, for example, in the middle of the night. Its possible that such jobs are spread out over the entire day, depending on the load of the system.

One thing to note is the behavior of at and batch. Both accept the names of the commands from the command line and not as arguments to the command itself. You must first run the command to be brought to a new line, where you input the commands you want execute. After each command, press Enter. When you are done, press Ctrl-D.

Because these two commands accept commands from stdin, you can input the command without having to do so on a new line each time. One possibility is to redirect input from a file. For example

at now +1 hour < command_list

where command_list is a file containing a list of commands. You could also have at (or batch) as the end of a pipe

cat command_list | at now + 1 hour

cat command_list | batch

Another interesting thing about both at and batch is that they create a kind of shell script to execute your command. When you run at or batch, a file is created in /usr/spool/cron/atjobs. This file contains the system variables that you would normally have defined, plus some other information that is contained in /usr/lib/cron.proto. This essentially creates an environment as though you had logged in.

Last-Modified: 2007-03-07 19:38:50


Sorting MP3s

Posted by on Tuesday, 7 April, 2009

#!/bin/bash
#
# My mp3s are always heniously unsorted and blowed if im gonna sit there spending hours sorting them
# Call me lazy or something but this definatly helps (even though i spent a couple days writing this and working out bugs)
#
# WARNING – THIS PROGRAM RELIES ON ID3 TAGS!
#
# There are a couple programs to get the ID3 tags sorted out available
# easytag, juk, rhythembox etc (those are all gui ones)
#
# Make sure mp3info is installed ( http://ibiblio.org/mp3info/ )
# just ./runthis and hey presto (make sure you edit where your mp3s are of course first)
#debug
#set -x

#The Dir where all your mp3s are
DIR=”/other/mp3s”

#The final destination dir for the mp3s – No trailing / needed
DESTINATION=”/other/mp3s”

#This is the thing that seperates fields (some people prefer _ some prefer a space however SPACES BREAK THINGS! use ‘1’ if you want spaces)
SPACER=”_”

#try not too edit to much below this

if [ -e moving.log ];then
rm -rf moving.log
fi

if [ ${SPACER} = “1” ];then
SPACER=”[:space:]”
fi

if [ $1 ];then

find ${DIR} -type f -name \*.[mM][Pp]3 | while read i
do

FINALDEST=”${DESTINATION}”
ARTIST=`mp3info -p %a “$i” | tr [:punct:][:blank:] ${SPACER} `
ALBUM=`mp3info -p %l “$i” | tr [:punct:][:blank:] ${SPACER} `
GENRE=`mp3info -p %g “$i” | tr [:punct:][:blank:] ${SPACER}`
SONGNAME=`mp3info -p %t “$i” | tr [:punct:][:blank:] ${SPACER}`
TRACKNUM=`mp3info -p %n “$i”`
FILENAME=`mp3info -p %f “$i” | tr [:punct:][:blank:][mM][Pp]3 ${SPACER} `

# sort the artists
function artistpath {
if [ ! “${ARTIST}” ];then
FINALDEST=”${FINALDEST}/Unknown/”
else
FINALDEST=”${FINALDEST}/${ARTIST}/”
fi
}
function albumpath {
#sort the albums
if [ ! “${ALBUM}” ];then
FINALDEST=”${FINALDEST}/Unknown/”
else
FINALDEST=”${FINALDEST}/${ALBUM}/”
fi
}
if [ $1 == “1” ];then
artistpath
albumpath
elif [ $1 == “2” ];then
albumpath
artistpath
elif [ $1 == “3” ];then
artistpath
elif [ $1 == “4” ];then
albumpath
fi

FINALDEST=`echo ${FINALDEST} | sed s/${SPACER}${SPACER}*/${SPACER}/g`
echo mkdir -p “${FINALDEST}”
echo mkdir -p “${FINALDEST}”>>moving.log
mkdir -p “${FINALDEST}”

unset NEWNAME

if [ “$2″ ];then

NEWNAME=`echo $2 | sed s/TRACKNUM/$TRACKNUM/g | sed s/ALBUM/$ALBUM/g | sed s/ARTIST/$ARTIST/g | sed s/SONGNAME/$SONGNAME/g`
else
NEWNAME=”${TRACKNUM}-${ALBUM}-${ARTIST}-${SONGNAME}”
fi

if [ “${TRACKNUM}” == “” ] && [ “${ALBUM}” == “” ] && [ “${ARTIST}” == “” ] && [ “${SONGNME}” == “” ];then
echo “Songname is non existant (no id3 tag?) – resorting to ${FILENAME} $i”
echo “Songname is non existant (no id3 tag?) – resorting to ${FILENAME} $i”>>moving.log
NEWNAME=”${FILENAME}”
fi
NEWNAME=`echo ${NEWNAME} | sed s/–*/-/g | sed s/^-// | sed s/^${SPACER}// `
FINALDEST=”${FINALDEST}${NEWNAME}.mp3″
FIXDEST=`echo ${FINALDEST} | sed s/${SPACER}${SPACER}*/${SPACER}/g `
echo moving $i to ${FIXDEST}
echo moving $i to ${FIXDEST} >>moving.log
mv -u “$i” “${FIXDEST}”
FINALDEST=”${DESTINATION}”
unset FIXDEST
done

# cleaning up empty dirs
#add this
#for i in `find ./ -type d ` ; do if [ “`find $i -type f`” ];then echo checking $i ; else echo $i is empty;rm -r “$i” ; fi ; done

for EMPTYDIR in ${DESTINATION}/*
do
if [ -d “${EMPTYDIR}” ];then
if [ “`find ${EMPTYDIR} -type f`” ];then
echo Checking ${EMPTYDIR} for cleanup
else
echo ${EMPTYDIR} is empty and being removed
echo ${EMPTYDIR} is empty and being removed >>moving.log
rm -r “${EMPTYDIR}”
#find “${EMPTYDIR}”
fi
else
echo ${EMPTYDIR} is a file

fi
done

# main else

else
echo “Useage: $0 {1|2|3|4} [PATTERN] ”
echo “1 sort by artist then album”
echo “2 sort by album then artist ”
echo “3 sort by artist only”
echo “4 sort by album only”
echo
echo “[PATTERN] – defaults to TRACKNUM-ARTIST-ALBUM-SONGNAME”
echo “ARTIST – Artist”
echo “ALBUM – Album Name”
echo “TRACKNUM – TrackNumber”
echo “SONGNAME – Songname”
fi

# Yes i realize some of the code is horiffic but it did the job at the time ;]
# Liz