Comments on: Bash Script: Incremental Encrypted Backups with Duplicity (Amazon S3)

By: Kevin

Kevin — Sat, 13 Mar 2010 22:21:16 +0000

In case anyone runs across the same issue, it appears the offending directory is /home/virtfs. If you are running a WHM/CPanel setup and want to use duplicity to backup your server, then you’ll want to exclude /home/virtfs from your backups.

By: Damon

Damon — Sat, 13 Mar 2010 19:03:33 +0000

Duplicity's most basic command line structure is:

duplicity src dest

In our case, ROOT is our src ... you do not have to have an include or an exclude list (those are optional) but you do need a src and a dest ... (at least, as far as I know).

By: Kevin

Kevin — Sat, 13 Mar 2010 18:54:45 +0000

If INCLIST requires full paths, then what’s the point of the ROOT parameter?

By: Damon

Damon — Sat, 13 Mar 2010 15:29:21 +0000

I added an option to the latest version of the script to allow you to easily output the command without having to modify the source …

By: Damon

Damon — Fri, 12 Mar 2010 18:43:32 +0000

Sure is - check out this comment for how to do it.

By: Kevin

Kevin — Fri, 12 Mar 2010 18:22:46 +0000

Is there something I can add to the script myself to do that?

By: Damon

Damon — Fri, 12 Mar 2010 17:41:13 +0000

If you are generating an appropriate command line argument I would imagine it is a duplicity issue — I’m not active in duplicities development … probably a good question for launchpad.

Maybe I should add a way to generate the script “output command” so its easier to debug …

By: Kevin

Kevin — Fri, 12 Mar 2010 17:22:35 +0000

Well, I'm not getting the same errors now that I've upgraded to version 0.6.08b, but after an hour of backing up to S3, I get this. Any ideas?

# /usr/local/sbin/dt-s3-backup.sh --backup
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1239, in ?
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1232, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1205, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 416, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 294, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.4/site-packages/duplicity/gpg.py", line 279, in GPGWriteFile
    data = block_iter.next(min(block_size, bytes_to_go)).data
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 505, in next
    result = self.process(self.input_iter.next(), size)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 631, in process
    data, last_block = self.get_data_block(fp, size - 512)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 658, in get_data_block
    buf = fp.read(read_size)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 415, in read
    buf = self.infile.read(length)
  File "/usr/lib/python2.4/site-packages/duplicity/diffdir.py", line 384, in read
    buf = self.infile.read(length)
IOError: [Errno 22] Invalid argument

By: Kevin

Kevin — Fri, 12 Mar 2010 15:25:11 +0000

No kidding; a big thanks, indeed! Duplicity has been such a wonderful thing for our server backups, I can’t live without it!

Still in the middle of a new backup, but there aren’t any errors yet. That’s a good sign.

By: Patrick

Patrick — Fri, 12 Mar 2010 07:04:43 +0000

So, this problem seemed to have been caused by something in the Duplicity update. Upgrading to version 0.6.08b fixed the issue for me. Thanks to Damon and also to the the folks over on the Duplicity Launchpad site for all the help tracking this down!