Posts Tagged howto

Linux Backups for Servers and Desktops

Posted by on Thursday, 18 June, 2009

Everyone wants to back up right? Well you will once you have totally lost the last years worth of work on a website and somebody breaks things severely!

Heres a quick and nasty backup HOWTO.

Database Dumps

mysqldump -u root -p mydatabase > mydatabase.sql

This dumps a database into a file, you can modify this to dump it offsite using ssh with this command.

su postgres -c "pg_dumpall" > pgdatabase.psql

If you use postgres you can change this to something like this

mysqldump -u root -p mydatabase > mydatabase.sql | ssh username@backup.comain.com “dd of=mydatabase.sql”

If you want to dump the entire database you can use

mysqldump -A -u root -p >entiredatabase.sql

This may take some time. To put this in a shell script and dump multiple copies and keep track of things you can use something similar to this

date=`date +%m-%h-%Y`

mysqldump -A -u root -p >${date}-fulldatabase.sql

This will expand to dump it into something like

06-Jun-2009-fulldatabase.sql

File Backup

FTP

To run a regular interactive FTP session:

lftp -u 'username,password' backup.yourdomain.com

To backup one or more files:

lftp -u 'username,password' backup.yourdomain.com -e "set ftp:ssl-protect-data true; mput /local/dir/files* /remotedir; exit"

You need to set ftp:ssl-protect-data else you will not be able to store the file. If you want to make this a default option, add it to the lftp.conf file. e.g. :

grep -qai "set ftp:ssl-protect-data true" /etc/lftp.conf || echo "set ftp:ssl-protect-data true" >> /etc/lftp.conf

To restore a file from the FTP server to your Machine:

lftp -u 'username,password' backup.yourdomain.com -e "set ftp:ssl-protect-data true;mget /remotedir/files* -O /localdir; exit" .

The -O option is not required it you wish to store to the current local directory.

To mirror a whole directory to the FTP server:

lftp -u 'username,password' backup.yourdomain.com -e "set ftp:ssl-protect-data true;mirror --reverse /local/dir/name remotedirname; exit" .

--reverse means that the ‘mirroring’ is going in the reverse direction than ‘normal’. i.e. from your server to the backup server. If you run man lftp there are a few other options to choose from. e.g. --delete to delete files on the backup server that do not exist locally. Or --continue to continue a mirror job. Or --exclude files to exclude certain files from the transfer.

To restore a whole directory from the FTP server to your machine:

lftp -u 'username,password' backup.yourdomain.com -e "set ftp:ssl-protect-data true;mirror remotedirname /local/dir/name;exit"

To create a nightly cronjob that uploads a directory to the backup FTP server, create a /etc/crond.daily/ftpbackup file like this:


#!/bin/bash
lftp -u 'username,password' backup.yourdomain.com -e "set ftp:ssl-protect-data true;mirror --reverse /local/dir/name remotedirname;exit" > /dev/null

Run

chmod +x /etc/cron.daily/ftpbackup .

Then check the files have been mirrored as you expect the next day.

Rsync

Rsync is a better option in some ways as it checks the MD5 of files and updates them if they are out of date, rather than re-copying the entire lot. Short but easy shell script to copy things over

#!/bin/bash
EXCLUDE=” –exclude *.tmp \
–exclude *.temp”
USER=username
HOST=backup.domain.com
BACKUPPATH=/backups

rsync –archive -vv –rsh=ssh $EXCLUDE $USER@$HOST:/etc/ $BACKUPPATH/$HOST/etc

Rdiff-backup

This is better again than rsync as it does versioning control and only backs up the difference in files.

To backup files

rdiff-backup /some/local-dir hostname.net::/whatever/remote-dir

To restore

rdiff-backup --restore-as-of now host.net::/remote-dir/file local-dir/file
rdiff-backup -r now host.net::/remote-dir/file local-dir/file

The -r command is the same as –restore-as-of

The main advantage of rdiff-backup is that it keeps version history. This command restores host.net::/remote-dir/file as it was 10 days ago into a new location /tmp/file .

rdiff-backup -r 10D host.net::/remote-dir/file /tmp/file

Other acceptable time strings include 5m4s (5 minutes and 4 seconds) and 2002-03-05 (March 5th, 2002). For more information, see the TIME FORMATS section of the manual page.

More examples can be found at http://www.nongnu.org/rdiff-backup/examples.html

This tutorial was compiled from several others, and props go out to http://rimuhosting.com and http://www.howtoforge.com


Tracking down io problems on your Linux box

Posted by on Tuesday, 16 June, 2009

I’m sure everyone has had problem at one time or another trying to figure out why your machine is going so slow, but nothing appears to be using the RAM or CPU at all.

The first option is to top ‘top’ and look for the line which has the ‘wa’

Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 96.7%id, 3.3%wa, 0.0%hi, 0.0%si, 0.0%st

mine says 3.3%wa – this is the wait time trying to write to disk. Now from there you can install the package (under most distros) called ‘sysstat’

sysstat – sar, iostat and mpstat – system performance tools for Linux

This contains several tools for trying to track down whats using the disk to write lots.

iotop – simple top-like I/O monitor. This is installed and can show you realtime whats writing to disk at any time and using what load
iostat – Report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).
sar – Collect, report, or save system activity information.

If there’s plenty of cache/buffers, and sar -W 1 0 shows lots of zeroes (and possibly the occasional blip) then the disk is getting thrashed, but it’s not swap.

Running iostat -dx 1 will show you all the partitions and how hard they’re working (look at %util). If %util is consistently at or around 100 for any partition of disk, you can definitively say that the disks are getting thrashed.

If the disk has high %util, but the actual throughput (rsec/s and wsec/s) is pretty low, then it’s possible you’ve got a hardware fault or RAID rebuild going on. A hardware error might show up on a smartctl run (smartctl -a /dev/sda or whatever), looking at things like the reallocated sector count, but SMART isn’t real, well, smart, so don’t trust it too much. A RAID rebuild should show up in your RAID management (you are monitoring your hardware RAID setup, aren’t you?). A software RAID rebuild will be shown in /proc/mdstat. (cat /proc/mdstat )


10 Free Linux Ebooks For Beginners

Posted by on Monday, 11 May, 2009

Thanks to the great guys at http://www.linuxhaxor.net/2009/05/10/10-free-linux-ebooks-for-beginners/ for this fantastic list! and Peter Reutemann from WLUG

1) Introduction to Linux – A Hands on Guide

Author: Machtelt Garrels
Format: HTML

2) Bash Guide for Beginners

Author: Machtelt Garrels
Format: HTML

3) Rute User’s Tutorial and Exposition

Author: Paul Sheer
Format: HTML

4) The Linux Starter Pack

Author: Paul Hudson
Format: PDF

5) FLOSS Manuals

Author: FSF
Format: HTML & PDF

6) The Easiest Linux Guide You’ll Ever Read

Author: Scott Morris
Format: PDF

7) Linux Knowledge Base and Tutorial

Author: James Mohr
Format: PDF

8) Ubuntu Pocket Guide and Reference

Author: Keir Thomas
Format: PDF

9) Linux Newbie Administrator Guide

Author: Peter and Stan Klimas
Format: HTML

10) Slackware Linux Basics

Author: Daniël de Kok
Format: HTML