Purpose: as a quick diagnostic tool, sometimes it’s nice to know exactly how fast two machines can (theoretically) be connected. I was having some slow downs on my NAS and using these command line tools, was able to determine that it was my NIC (and not my hard drives) that was causing the slow down. I learned this from a thread at slashdot.
How To: Automated Encrypted Incremental Backups on Amazon S3 with Duplicity (OS X or Ubuntu)
Purpose: setup an automatic encrypted off-site backup system that utilizes Amazon S3 with incremental backups by duplicity on the Mac (Leopard) or Ubuntu. Currently, I do have my own on-site backup system in place (nightly backups via rsync to external hard drive), but I am wary that some day my house may explode and I’ll have nothing left. Enter my new friend: the encrypted off-site backup.
Bash Script: Incremental Encrypted Backups with Duplicity (Amazon S3)
This bash script was designed to automate and simplify the remote backup process of duplicity on Amazon S3. After your script is configured, you can easily backup, restore, verify and clean (either via cron or manually) your data without having to remember lots of different command options and passphrases.
Most importantly, you can easily backup the script and your gpg key in a convenient passphrase-encrypted file. This comes in in handy if/when your machine ever does go belly up.
how to use
To get the bleeding edge very latest code in the script you can download a zip copy of the source or clone the git repository like so (a “stable” version 4 is coming soon):
git clone git://github.com/thornomad/dt-s3-backup.git
You’ll also need to have a number of things in place in order to utilize this script, specifically: gpg, duplicity, an Amazon S3 account, and (optionally) s3cmd. If you need help getting all these in order, I wrote another post about putting it all together. It’s not all that difficult, but does take a few pieces of the puzzle to be in order.
Once you have the script, you will need to fill out the foobar variables with your own specific information. I suggest testing the script on a small directory of files and a local directory for your destination first to make sure it is working.
change log
Here is a list of the changes so far.
Version Four (March 2010)
--list-filesoption--restore-fileoption--backupis a required option to actually run the script
Version Three (01/31/09)
- Added comment to explain why folks need to use s3+ and not s3: for Amazon buckets
- Used “unset” to remove the variables at end of the script (thanks: alvaro)
- Fixed a problem when the backup folder on S3 was nested inside another bucket (thanks John Kinsella)
- Changed the PASSPHRASE field to default to the actual passphrase, so one can easily backup the entire script and not have to worry about remembering the passphrase or where it’s kept.
- Added
--backup-this-scriptoption which will turn the script and the secret key into an encrypted tarball that can be kept somewhere safe for easy restores if the machine goes down. - Cleaned up the get_file_size function so it wouldn’t run when it wasn’t supposed to.
Version Two (12/03/08) [Download Version 2]
- added GPL license
- changed the cleanup feature to automatically force a full backup after (n) number of days as well as automatically cleanup after (n) number of days
- added option to force cleanup after (n) number of full backups (rather than by days)
- option to changed log file ownership
- runtime checks for installed required software and write permissions on log directory
- fixed formatting of logfile to be a little more consistent
- setup everything in clever functions
Version One (11/24/08) [Download Version 1]
- Initial release.
How to: Install Netatalk (AFP) on Ubuntu with Encrypted Authentication
Purpose: Install Netatalk (AFP) on Ubuntu with encrypted authentication (using OpenSSL), which is not enabled by default with the Ubuntu netatalk package. By default, the package installed from the Ubuntu universal repositories will transmit your password via clear text (you’ll know this because Mac OS X Tiger will throw a warning and Leopard won’t do anything useful at all).
How to: Mount a SFTP Folder (SSH + FTP) on Ubuntu Linux using SSHFS & Fuse
Purpose: to mount a remote directory on my local Ubuntu Linux Desktop system using SFTP (which is SSH in an FTP-like fashion). The goal is to easily gain access to a remote system’s files through another folder on my desktop. I used sshfs to accomplish this.
How to: Redirect Apache’s Default www or public_html Folder to a Directory in Your Home Folder
Purpose: The default installation of Apache (from a Ubuntu-Server installation) sets the base directory for the web documents as /var/www (on Ubuntu’s installation — this may be different if you are running Apache on another machine); this may not be where you want it, in the end, and certainly isn’t as easily accessible from a remote machine. One option is to change where it is Apache searches for its web documents folder in Apache’s configuration file; another way, which I chose, is to create a symbolic link in the default location’s place and have it point to a directory in my user’s home folder.
How To: Install a LAMP Server (Linux, Apache, Mysql, PHP) on Older Laptop with Ubuntu
The Purpose: Install a LAMP server (Linux, Apache, Mysql, PHP) on an older laptop to be accessed and maintained remotely (becasue the laptop has a broken monitor and unreliable video-output connections). Then, transfer web projects from the server on my PowerMac to the new linux server and free up my beast!