Install the Android SDK

Go to Andoird SDK and download the SDK for the Mac: http://developer.android.com/sdk/index.html.
Unzip  it and move the entire folder to the Desktop. In the Mac Finder, browse to the: ~/Desktop/android-sdk-macosx/tools folder and double click the “android” unix file (which loads the Android SDK Manager). I  checked the 4.0.3 (API15) box and followed the steps to install it.

Here’s what it looked like after I was done:

Enable USB Debugging and Reboot into the Bootloader

You need to enable USB Debugging on your Galaxy Nexus. Head over to Settings>Developer options>USB debugging and shut off phone (and said goodbye to everything on it).

Unlock Using Fastboot

In Terminal.app on your Mac, head into your I went to my platform-tools folder and ran fastboot like so:

• cd ~/Desktop/android-sdk-macosx/platform-tools
• ./adb reboot bootloader
• ./fastboot oem unlock

Immediately after this, you should see a < waiting for device > warning. Go ahead and plug in the phone via USB and, like magic, it should ask you if really want to unlock it.  Which you do. Badly.

Using the volume keys to toggle my answer and the power button to select I updated the lock state.  (Terminal.app threw a usb_read error but it turned out to be nothing.)

Lastly, toggle the bootloader screen into a “start” button and reboot. Personally, I had to wait a good period of time (did two restarts) before I was in!

install custom recovery and superuser access

Download the latest custom recovery file from clockworkmod.com and a su.zip file (not sure where I got that actually). With the phone running and your usb cable plugged in run:

• ./adb push su.zip /sdcard/
• ./adb reboot bootloader
• ./fastboot flash recovery recovery-clockwork-touch-6.0.2.3-toro.img

Then boot into the recovery and install the su.zip file. Reboot.

Backup Tips and Next Steps

I ended up doing this part of the process twice because the first time the mistake (as I like to call it) I made was to latch my Google account to the phone and do auto-backup and restore of settings/apps immediately after it restarted.  Not so bad if I was getting ready to use the phone … however, the next step in my journey was to make a nandroid backup of the stock ROM.  And I didn’t want a bloated backup. So – if you are also planning to make your stock backup at this point, I wouldn’t let Google restore everything just yet.

The rest of the instructions for flashing a recovery, actually making the nandroid backup, and all else Galaxy Nexus can be found at Droid-Life. I followed these instructions exactly (except where they say copy files to the SDK’s /tools folder I copied them to /platform-tools).

Enjoy! I would recommend AOKP for your custom ROM. It’s what I’m using and has been swell.

a couple easy steps:

Log into your Site5 account via ssh and follow these steps (note, the version of the script doesn’t exist anymore, try getting it here):

• wget http://ilerner.3b1.org/vim/vim7-install.sh
• chmod +x vim7-install.sh
• ./vim7-install.sh -cache tmp/ -home

This will install the Vim executable at ~/bin/vim.

In order for it to run in lieu of the default Vim app, make an alias at the bottom of your ~/.bashrc file that points to it:

alias vim=~/bin/vim

That should be all you have to do.

how to set it up

The only way I have found to circumvent this annoyance is to play a little trick on iMovie. And that trick is to move the original ~/Movies folder under a different name and put a symlink in its place (to the network folder of your choice).

Here are the steps I took from the Terminal to do this (you will need to have Administrator privileges and you will need to know where your network folder is attached):

• sudo mv ~/Movies/ ~/Movies-original
• ln -s /Volumes/my-network-volume/Movies/ ~/Movies

Now, when I load up iMovie it will use the networked volume. If you don’t like this, you can always move your ~/Movies-original folder back where it was.

remaining issues

Unfortunately, this isn’t a perfect fix. There are two caveats or issues I have found.

One, is that now my hard drive (called Papa Bear) is listed twice in the Project Library:

And two, is that when you delete an event or project from within iMovie, it moves the project into a temporary folder rather than actually deleting it. The next time you open iMovie, it will show up again. I’m sure this is because it thinks it is working with a bona fide Trash system but its not (because it is network storage. See this example:

In order to get around this annoyance, I created a Python script which removes the temporary files for me and created a cron job to run the script from time to time (cleaning up the mess iMovie leaves behind).

#!/usr/bin/env python
'''
Simple script to remove all directories that match the name:
    /iMovie Temporary Items */  [note the star!]

'''
import os
import fnmatch
import shutil

# tuple of directories to search through (you can add more than one)
DIRS = ('/Volumes/leaker-damon/tps/Videos/',)

for directory in DIRS:
    for root, dirnames, filenames in os.walk(directory):
        for directory in dirnames:
            if fnmatch.fnmatch(directory,'iMovie Temporary Items *'):
                shutil.rmtree(os.path.join(root,directory))

make this more better

Am interested to hear how other people have been handling this — this approach certainly isn’t perfect, but it seems to work (for now).

As a self-taught Python user I am still looking for insight on the most pythonic and programmatically-friendly way of accomplishing a given task. In this case, I have written a script that will perform a “clean bzip2″ of a directory (or directories). Mac OS X (via AFP and netatalk, in my case) tends leaves a bunch of ugly files/directories hanging around and I would rather not include them in my compressed tar file.

In writing the script, though, I ran into some questions and I am not sure what the recommended approach would be. The script works, as it is, but I feel its a little hacked together and also a little limited in its application. There is something to be said for programs that “just work” (this does) but I want to take it a little further as an educational endeavor and would like it to appear robust, future-thinking, and pythonic.

My initial questions are:

1. Is there a better way to implement a --quiet flag? [using logging]
2. I am not very clear on the use of Exceptions (or even if I am using it in a good way here) — is what I have done the right approach?
3. Finally, in general: any feedback on how to improve this? (I am thinking, just now, that the script is only suitable for a command line usage, and couldn’t be imported by another script, for example.)

Any feedback is greatly appreciated. Writing a script like this is a good learning tool (for me, at least).

#! /usr/bin/env python

'''Script to perform a "clean" bzip2 on a directory (or directories).  Removes
extraneous files that are created by Apple/AFP/netatalk before compressing.
'''

import os
import tarfile
import logging
from optparse import OptionParser

# Default files and directories to exclude from the bzip tar
IGNORE_DIRS = ('.AppleDouble',)
IGNORE_FILES = ('.DS_Store',)

class DestinationTarFileExists(Exception):
    '''If the destination tar.bz2 file already exists.'''

def ignore_walk(directory, ignore_dirs=None, ignore_files=None):
    '''Ignore defined files and directories when doing the walk.'''

    # TODO: this does not currently take wild cards into account.  For example,
    # if you wanted to exclude *.pyc files ... should fix that.  Perhaps
    # consider moving this entirely into the below function (or making it more
    # reusable for other apps).
    for dirpath, dirnames, filenames in os.walk(directory):
        if ignore_dirs:
            dirnames[:] = [dn for dn in dirnames if dn not in ignore_dirs]
        if ignore_files:
            filenames[:] = [fn for fn in filenames if fn not in ignore_files]
        yield dirpath, dirnames, filenames

def tar_bzip2_directory(directory, ignore_dirs=IGNORE_DIRS,
                                   ignore_files=IGNORE_FILES ):
    '''Takes a directory and creates a tar.bz2 file (based on the directory
    name).  You can exclude files and sub-directories as desired.'''

    file_name = '-'.join(directory.split(' '))
    tar_name = file_name.replace('/','').lower() + ".tar.bz2"
    
    if os.path.exists(tar_name):
        msg = ("The file %s already exists. " + 
                "Please move or rename it and try again.") % tar_name
        raise DestinationTarFileExists(msg)
    
    tar = tarfile.open(tar_name, 'w:bz2')

    for dirpath, dirnames, filenames in ignore_walk(directory, ignore_dirs,
            ignore_files):
        for file in filenames:
            logging.info(os.path.join(dirpath, file))
            tar.add(os.path.join(dirpath, file))

    tar.close()

def main(args=None, callback=None):
    directories = []

    for arg in args:
        if os.path.isdir(arg):
            directories.append(arg)
        else:
            logging.ERROR("Ingoring: %s (it's not a directory)." % arg)

    for dir in directories:
        try:
            tar_bzip2_directory(dir)
        except DestinationTarFileExists, e:
            print e

if __name__ == "__main__":

    parser = OptionParser(usage="%prog [options: -q ] [dir1] [...dir2]")
    parser.add_option("-q", "--quiet", action="store_true", dest="quiet")
    options, args = parser.parse_args()

    if options.quiet:
        logging.basicConfig(level=logging.ERROR, format='%(message)s')
    else:
        logging.basicConfig(level=logging.INFO, format='%(message)s')

    main(args)

March 2011 Update: I haven’t done much with this in the past few months and there are some good changes/forks out there that need to be merged. Sorry for the delays. I intend to update it in April when I finally have some free time to work on it again. Thanks for all your input and ideas and changes. -Damon

This isn’t meant to be a full-fledged tracking application (see django-tracking) or a real analytic tool (try Google Analytics); rather, it’s meant to simply count the number of hits/view on an object-per-object basis.

How to install:

I find that the easiest way to work with django apps is to symbolically link them to my site-packages directory. It’s easier to update the apps with svn, git, or hg than it is to manually download the files and install by hand.

For me, this is what it looks like (you can cut and paste to make it easy):

• cd ~/src
• git clone git://github.com/thornomad/django-hitcount.git
• sudo ln -s `pwd`/django-hitcount/hitcount `python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`/hitcount

Test it works by loading python and checking the version (should not get an error):

>>> import hitcount
>>> hitcount.__version__
'0.1 alpha'
>>>

Adding to your django project:

Add the hitcount app to your INSTALLED_APPS tuple and run syncdb.

There are three additional settings you can add to your settings.py file:

HITCOUNT_KEEP_HIT_ACTIVE = { 'days': 7 }
HITCOUNT_HITS_PER_IP_LIMIT = 0
HITCOUNT_EXCLUDE_USER_GROUP = ( 'Editor', )

HITCOUNT_KEEP_HIT_ACTIVE: is the number of days, weeks, months, hours, etc (timedelta kwargs), that an Hit is kept ‘active’. If a Hit is ‘active’ a repeat viewing will not be counted. After the active period ends, however, a new Hit will be recorded. You can decide how long you want this period to last …

HITCOUNT_HITS_PER_IP_LIMIT: limit the number of ‘active’ hits from a single IP address. 0 means that it is unlimited. You may want to set this, or not.

HITCOUNT_EXCLUDE_USER_GROUP: don’t count any hits from certain logged in users. In the example above, I don’t want any of my editors inflating the total Hit count.

Adding to your urls.py:

You need to add one line to your urls.py file.

You can have this url, itself, point to anywhere you like, but you need to keep the name='hitcount_update_ajax' constant.

from django.conf.urls.defaults import *
from django.views.generic.list_detail import object_detail
from hitcount.views import update_hit_count_ajax

urlpatterns = patterns('',
    url(r'^ajax/hit/$', # you can change this url if you would like
        update_hit_count_ajax,
        name='hitcount_update_ajax'), # keep this name the same

    # other views, for example my object view is:

    url(r'^/video/(?P<object_id>\d+)$', object_detail,
        {   'queryset': Video.objects.all(),
            'template_name': "video/view.html"},
            name='video_detail_view'),

)

Edit your templates

Add the javascript to your object_detail templates (or any template that handles a single object) so that our hit counter is called after the document loads.

Here is what my head includes:

{% load hitcount_tags %}
<script src="/media/js/jquery-latest.js" type="text/javascript"></script>
<script type="text/javascript"><!--
    $(document).ready(function() {
        {% get_hit_count_javascript for object %}
    });
--></script>

When the template is rendered, it should turn into something like this:

 
$(document).ready(function() {

$.post( '/ajax/hit/',
	{ hitcount_pk : '3' },
	function(data, status) {
		if (data.status == 'error') {
			// do something for error?
		}
	},
	'json');

});

Display the hits!

The most exciting part, is actually displaying your hits. There are four different ways to do it:

    - Return total hits for an object: 
      {% get_hit_count for [object] %}
    
    - Get total hits for an object as a specified variable:
      {% get_hit_count for [object] as [var] %}
    
    - Get total hits for an object over a certain time period:
      {% get_hit_count for [object] within ["days=1,minutes=30"] %}

    - Get total hits for an object over a certain time period as a variable:
      {% get_hit_count for [object] within ["days=1,minutes=30"] as [var] %}

Of course, I googled it and found this post by Jim Beveridge — inside the comments were a suggestion to replace the capacitors.  So, I set about doing it.  I needed to replace the green ones (see image) and got everything at RadioShack.  Cost less than $20 (and I had to buy a soldering iron).

So, out went the green ones (which were bulging) and in went the jumbo-sized RadioShack ones (I would recommend buying them online so they actually fit, but hey: I take what I can get).

Case top won’t fit but a little masking tape did the trick and back to the basement it went — working like a charm!  Would have cost less than $5 if I had my own soldering iron …

Since this will be used mainly to store my files and do web development, I went with something cheap and, as much as possible, power efficient.  Here’s what I ended up with:

• Processor: AMD 4850e  (2.5ghz Dual-Processor, 45Watt)
• Motherboard: ECS A780GM-A AM2+ (780G)
• Memory: 4gb DDR-800
• Hard Drives: 3x Western Digital 1TB “Green Power” (WD10EADS)
• Power Supply: Antec 380 Watts Earth Power Supplies (EA380)

I am still waiting for the new hard drives to arrive (should be here today) — but so far, so good!  Hope to benchmark some power consumption as well as hard drive performance when I have the chance.  And also, get a virtual machine up and running for the web server so I can get rid of the two computers in my basement and just run one.

install nc and pv

If you are on a mac, you can use MacPorts to install this easily enough:

• $ sudo port install netcat pv

If you are on Ubuntu, you just need to get pv installed (nc is already there):

• $ sudo aptitude install pv

Once you have nc and pv installed, it’s really simple.  On one machine, run the following command:

• $ nc -ulp 5000 > /dev/null

On the second machine run the following command (you need the IP address of the first machine):

• $ pv < /dev/zero | nc -u ip.addy.of.other.machine 5000

And you should get some output with a little <=> sign moving across the screen that resembles this (static):

1.15GB 0:00:19 [ 218MB/s] [             <=>                  ]

This will show you the number of MB/s the connection is averaging; if you watch if for a bit, you can get an idea of where things stand. Here are some of the results I got between my PowerMac, PowerBook, and EEE PC:

localhost:                      218MB/s
gigabit Ethernet:               143MB/s
100MB ethernet:                 11.5MB/s
wireless (poor connection):     141kB/s

If you read the original post on slashdot, it seems you can us pv to measure a lot of other stuff as well — I haven’t played with it yet but should, one day.

before you begin

Before you can start backing things up off-site in a secure fashion, you’ll need to get a few pieces of the puzzle in place. Namely, you’ll need software installed (duplicity), a GPG key (for encryption), and an Amazon S3 account setup (for storage), and then use a backup script that can be run automatically (for laziness’s sake!).

Getting an Amazon S3 account is easy to do: head over to http://aws.amazon.com/s3/ and sign up; grab your “Access Key ID” and “Secret Access Key” and you are ready to go. There is a lot you can do with S3 (and a lot of ways to access it), but for our purposes, this is pretty much all you need.

Lastly, I would recommend spending a little time reading about duplicity (see the man page) as well as GnuPG (man page). There is a lot to consider, and I just picked the options I thought would work best for me.

Install the Software: Duplicity

For this to work we need duplicity installed with all the correct dependencies. The easiest way to do this on your Mac is to simply use MacPorts, which has an up-to-date version in the repositories (see Installing MacPorts if you don’t have it installed already). If you already have MacPorts installed, all you should have to do is run the following from the Terminal:

• $ sudo port install duplicity py25-socket-ssl py25-boto

If you are using Ubuntu, you could simply run sudo aptitude install duplicity to install the program (it is in the repositories); however, if you want to make sure you are using the latest version (which may not be available there yet), you can try this:

• $ sudo apt-get build-dep duplicity
• $ sudo aptitude install python-boto ncftp
• $ wget http://savannah.nongnu.org/download/duplicity/duplicity-0.5.07.tar.gz
• $ tar xvzf duplicity-0.5.07.tar.gz
• $ cd duplicity-0.5.07/
• $ sudo python setup.py install

If you ever want to upgrade again, just download and untar the latest version and run the last setup line again.  It will install the newest version for you.

If everything has installed correctly, you can do a test run pretty easily on your local machine by backing up a folder to another local folder (first command) and then restoring it to a different folder (second command). If you look inside this /test/backup-location/ you’ll see what duplicity looks like:

• $ duplicity --no-encryption /test/folder/ file:///test/backup-location/
• $ duplicity --no-encryption file:///test/backup-location/ /test/restore-location/

Setting Up Encryption

For duplicity to really shine, it needs to have a gpg key to encrypt your files. If you don’t already have one, you can create it by running the following (read the documentation for more information):

• $ gpg --gen-key

I used all the defaults when setting up my key and chose my own passphrase. Unfortunately, in order to make this work without user input (as an automatic cron job), the passphrase is going to have to be stored somewhere on your computer locally, so, I wouldn’t use one of your usual passwords (something really long and unique would be better).  Also, if you already have a gpg key (or want to use one for other purposes), I would recommend making a different one for the Amazon S3 backups — because, in the end, your password has to be stored somewhere on your computer for it to work auto-magically.

Once you have your gpg key created you can check it out by running:

• $ gpg --list-keys

This shows your new key, which probably looks something like this:

pub   1024D/CA4ZA320 2008-11-15
uid                  Damon Timm (thornomad) email@example.com
sub   2048g/C1E64A4F 2008-11-15

Note the public key identifier “CA4FA320″ (yours will be different); we will need that to go in our script.

final step: using a backup script

So, everything is on your system and, hopefully, working.  Now, to run a backup takes a lot of typing (on the command line) and the easiest way to avoid this chore is to run a backup script.  A script can store your Amazon and GPG key information and make it so you don’t have to type anything ever again!

Backing things up is a very personal task, and everyone is going to want to do it a little differently. I created my own backup script which you are happy to check out — if you have any neat features you add to suggestions, I would love to hear them and incorporate them.

Good luck!

This bash script was designed to automate and simplify the remote backup process of duplicity on Amazon S3. After your script is configured, you can easily backup, restore, verify and clean (either via cron or manually) your data without having to remember lots of different command options and passphrases.

Most importantly, you can easily backup the script and your gpg key in a convenient passphrase-encrypted file. This comes in in handy if/when your machine ever does go belly up. Code is hosted at github.

how to use

To get the latest latest code in the script you can download a zip copy of the source or clone the git repository like so:

• git clone git://github.com/thornomad/dt-s3-backup.git

You’ll also need to have a number of things in place in order to utilize this script, specifically: gpg, duplicity, an Amazon S3 account, and (optionally) s3cmd. If you need help getting all these in order, I wrote another post about putting it all together. It’s not all that difficult, but does take a few pieces of the puzzle to be in order.

Once you have the script, you will need to fill out the foobar variables with your own specific information.  I suggest testing the script on a small directory of files and a local directory for your destination first to make sure it is working.

Usage

From the README file:

COMMON USAGE EXAMPLES
=====================

* View help:
    $ dt-s3-backup.sh

* Run an incremental backup:
	$ dt-s3-backup.sh --backup

* Force a one-off full backup:
    $ dt-s3-backup.sh --full

* Restore your entire backup:
	$ dt-s3-backup.sh --restore 
    You will be prompted for a restore directory

	$ dt-s3-backup.sh --restore /home/user/restore-folder
    You can also provide a restore folder on the command line.

* Restore a specific file in the backup:
    $ dt-s3-backup.sh --restore-file
    You will be prompted for a file to restore to the current directory

    $ dt-s3-backup.sh --restore-file img/mom.jpg
    Restores the file img/mom.jpg to the current directory

    $ dt-s3-backup.sh --restore-file img/mom.jpg /home/user/i-love-mom.jpg
    Restores the file img/mom.jpg to /home/user/i-love-mom.jpg

* List files in the remote archive
	$ dt-s3-backup.sh --list-current-files

* Verify the backup
    $ dt-s3-backup.sh --verify

* Backup the script and gpg key (for safekeeping)
    $ dt-s3-backup.sh --backup-script

Changes

You can view the changelog at github.