New Zealand Linux

Sometimes files gets deleted accidentally, whilst they are still running. This also applies to things like flash videos or other things, usually temporary files or even when a server is exploited.
Sometimes you want to keep those files right? but they are deleted! but how can they be running when they are deleted?
\
This is what /proc is all about. Its a neat way of keeping track of information like file descriptors for each PID.
So, everyone knows what ps does, it shows you whats running .. like this

start
www-data 4146 2.0 2.9 62088 28292 ? S 00:00 0:02 \_ /usr/sbin/apache2 -k start
www-data 5287 0.0 0.4 42072 4536 ? S 00:01 0:00 \_ /usr/sbin/apache2 -k start
1005 27387 1.9 0.5 7940 5576 ? S Nov28 9:31 perl

Oh wait, what is that thing called ‘perl’ ? This is an example from a hacked box. I knew the application was not called ‘perl’, and since it had been sending spam i knew it was probably a bad file.
So, i wanted to find what files were open by the pid 27387 – i installed and used ‘lsof’ which gives an ls of open files.

# lsof -p 27387
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
perl 27387 ausername cwd DIR 202,1 4096 476065 /tmp
perl 27387 ausername rtd DIR 202,1 4096 2 /
perl 27387 ausername txt REG 202,1 1254016 100305 /usr/bin/perl
perl 27387 ausername mem REG 202,1 75472 344738 /lib/i686/nosegneg/libresolv-2.9.so
perl 27387 ausername mem REG 202,1 21976 344867 /lib/i686/nosegneg/libnss_dns-2.9.so
perl 27387 ausername mem REG 202,1 42504 344774 /lib/i686/nosegneg/libnss_files-2.9.so
perl 27387 ausername mem REG 202,1 38444 345027 /lib/i686/nosegneg/libnss_nis-2.9.so
perl 27387 ausername mem REG 202,1 87804 344872 /lib/i686/nosegneg/libnsl-2.9.so
perl 27387 ausername mem REG 202,1 30436 344758 /lib/i686/nosegneg/libnss_compat-2.9.so
perl 27387 ausername mem REG 202,1 19816 181707 /usr/lib/perl/5.10.0/auto/Socket/Socket.so
perl 27387 ausername mem REG 202,1 38296 345032 /lib/i686/nosegneg/libcrypt-2.9.so
perl 27387 ausername mem REG 202,1 1450372 344746 /lib/i686/nosegneg/libc-2.9.so
perl 27387 ausername mem REG 202,1 116294 345030 /lib/i686/nosegneg/libpthread-2.9.so
perl 27387 ausername mem REG 202,1 149328 344754 /lib/i686/nosegneg/libm-2.9.so
perl 27387 ausername mem REG 202,1 9676 345034 /lib/i686/nosegneg/libdl-2.9.so
perl 27387 ausername mem REG 202,1 117348 344891 /lib/ld-2.9.so
perl 27387 ausername 0r CHR 1,3 0t0 197031 /dev/null
perl 27387 ausername 1w CHR 1,3 0t0 197031 /dev/null
perl 27387 ausername 2w CHR 1,3 0t0 197031 /dev/null
perl 27387 ausername 3r REG 202,1 14782 2872816 /tmp/ (deleted)
perl 27387 ausername 4wW REG 202,1 0 2872817 /tmp/…

Edit: You can use lsof +L1 -p pid/27387 to only show deleted items – thanks Jeffrey Caughel

Sorry for verbosity there, but its required. Ok, now you can see the line that has (deleted) – this is what we want. The 4th line over tells me its using the file descriptor 3 (ignore the r after it for now)

So to access that file, i look at /proc/27387/fd/3 – often its best to copy that file elsewhere before it gets deleted (ie if program is closed). The first number is the pid, the fd is the file descriptor , and the last is the script itself which was deleted – in this case a spaming perl script.
Now i knew what that script did, it was not touching the filesystem just sending spam, so i knew it was safe to kill it.
This is also handy for saving youtube videos or when you accidentally rm -rf /etc or /bin when things are still running 🙂

We have some users who own servers who dont want to fork out for automated systems like Plesk or Virtualmin, but don’t really want to deal with adding domains and email addresses all the time (and sometimes get lost)

I decided today after one such user emailed us to add another 3 domains and bunch of email addresses to write something simple to help him out, and thought I would share them with you.

I put the following in a plain text file in /root/adddomain.sh

#!/bin/bash
if [ ! $1 ];then
echo "Usage: $0 domainname.com"
exit 0
fi
 
echo Adding the virtualhost to apache
cat >/tmp/httpd.tmp < < EOF
 
<VirtualHost *:80>
DocumentRoot /var/www/CHANGEME/html
ServerName CHANGEME
ServerAlias www.CHANGEME
<directory "/var/www/CHANGEME">
allow from all
Options +Indexes
</directory>
 
 
EOF
cat /tmp/httpd.tmp | sed s/CHANGEME/$1/g >> /etc/httpd/conf/httpd.conf
 
echo Making the directory at /var/www/$1
mkdir -p /var/www/$1/html
 
echo reloading apache
/etc/init.d/httpd reload
 
echo Adding domain to mail
echo $1 /etc/postfix/virtual_domain # this was his postfix virtual domain name list

Then run

chmod +x adddomain.sh

Now I can add domains like this very easily

[root@hostname ~]# ./adddomain.sh
Usage: ./adddomain.sh domainname.com
[root@hostname ~]# ./adddomain.sh domain.co.nz
Adding the virtualhost to apache
Making the directory at /var/www/domain.co.nz
reloading apache
Reloading httpd:                                           [  OK  ]
Adding domain to mail
[root@hostname ~]#

Please note: do not add the ‘www’ part onto the domain name. That is done in the script itself where required.

Since he had set up virtual hosting in postfix, i then created another text file at /root/addmailuser.sh – this was so he could add email addresses easily and quickly. The contents were

#!/bin/bash
 
if [ ! $2 ]; then
echo "Usage: $0 [username|destination] emailaddress"
exit 0
fi
 
if [ -z $(echo $1 | grep @) ];then
echo Looks like a username to me, adding the user
adduser -s /sbin/nologin $1
passwd $1
else
echo Looks like a redirect off site, adding it as such
fi
 
echo Adding the email address
echo $2  $1 >> /etc/postfix/virtual
 
echo Running postmap
postmap /etc/postfix/virtual
 
echo Reloading postfix
/etc/init.d/postfix restart

Again i run the chmod on it

chmod +x addmailuser.sh

This is how I can use it

[root@hostname ~]# ./addmailuser.sh
Usage: ./addmailuser.sh [username|destination] emailaddress 
[root@hostname ~]# ./addmailuser.sh julie.domain julie@domain.co.nz
Looks like a username to me, adding the user
Changing password for user julie.domain.
New UNIX password:
BAD PASSWORD: it is too short
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
Adding the email address
Running postmap
Reloading postfix
Shutting down postfix:                                     [  OK  ]
Starting postfix:                                          [  OK  ]
[root@hostname ~]#

Or I can use it to create an off site alias

[root@hostname ~]# ./addmailuser.sh james.someguy@gmail.com james@domain.co.nz
Looks like a redirect offsite, adding it as such
Adding the email address
Running postmap
Reloading postfix
Shutting down postfix:                                     [  OK  ]
Starting postfix:                                          [  OK  ]
[root@hostname ~]#

These were designed/written for Centos/RedHat based systems, let me know if you want it for Debian/Ubuntu based ones. Also, strictly speaking, things don’t need to be restarted, but it doesn’t hurt and is a good way of testing things work ok.
There is no error checking in either of these scripts, feel free to contribute patches/fixes 🙂

Occasionally you just want a bit of piece of mind about your server or Linux install. You may suspect there is somebody who has hacked your computer or even something changed by a package install that shouldnt have been.

Heres a couple of ideas on how to do a quick ‘health’ check on he md5sum of binary packages.

Debian based people should install dlocate and use that

apt-get install dlocate
dlocate -md5check openssh-server

To force a fail try something like this

mv /usr/share/man/man5/sshd_config.5.gz /usr/share/man/man5/sshd_config.5.gz-old
echo Boo > /usr/share/man/man5/sshd_config.5.gz
dlocate -md5check openssh-server

For Redhat/Centos etc based servers you can use yum

 rpm -qvV openssh

Again you can force a fail by changing a file

mv /usr/share/doc/openssh-4.3p2/CREDITS /usr/share/doc/openssh-4.3p2/CREDITS-old
echo Boo >/usr/share/doc/openssh-4.3p2/CREDITS
rpm -qvV openssh

For less verbosity just drop the lower case v (so its rpm -qV )