2012-06-13

Motion Google Drive Uploader and Emailer

***** UPDATE: The script in this article no longer works due to Google authentication changes. Please use the new script described here: Motion Google Drive Uploader for OAuth 2.0



I'm using the brilliant Motion software with my Raspberry Pi to capture goings-on around my house. This blog post: Battery powered, Wireless, Motion detecting Raspberry Pi and this blog post: Raspberry Pi Webcam explain how.

I set up Lighttpd to point at the directory where Motion was creating video files so that I could go and check periodically to see if any new events had been captured. This works well - I just set up my home router so I could access the http over the Internet and then download the AVIs to my laptop or whatever. The down side to this is that you have to pro-actively look to see if there are new videos and secondly it puts additional strain on the Raspberry Pi to host the files for repeated download.

So, I have created a Python script to upload the AVI to Google Drive, get a URL to view the video on line and then email this using the same GMail account to an address. To keep the Raspberry Pi tidy the script then deletes the local file.

If you have the Google Drive Sync application running too you can see the files as they arrive which is fun:

Source


#!/usr/bin/python2
'''
Created on 6 Jun 2012

@author: Jeremy Blythe

Motion Uploader - uploads videos to Google Drive

Read the blog entry at http://jeremyblythe.blogspot.com for more information
'''

import smtplib
from datetime import datetime

import os.path
import sys

import gdata.data
import gdata.docs.data
import gdata.docs.client
import ConfigParser

class MotionUploader:
    def __init__(self, config_file_path):
        # Load config
        config = ConfigParser.ConfigParser()
        config.read(config_file_path)
        
        # GMail account credentials
        self.username = config.get('gmail', 'user')
        self.password = config.get('gmail', 'password')
        self.from_name = config.get('gmail', 'name')
        self.sender = config.get('gmail', 'sender')
        
        # Recipient email address (could be same as from_addr)
        self.recipient = config.get('gmail', 'recipient')        
        
        # Subject line for email
        self.subject = config.get('gmail', 'subject')
        
        # First line of email message
        self.message = config.get('gmail', 'message')
                
        # Folder (or collection) in Docs where you want the videos to go
        self.folder = config.get('docs', 'folder')
        
        # Options
        self.delete_after_upload = config.getboolean('options', 'delete-after-upload')
        self.send_email = config.getboolean('options', 'send-email')
        
        self._create_gdata_client()

    def _create_gdata_client(self):
        """Create a Documents List Client."""
        self.client = gdata.docs.client.DocsClient(source='motion_uploader')
        self.client.http_client.debug = False
        self.client.client_login(self.username, self.password, service=self.client.auth_service, source=self.client.source)
               
    def _get_folder_resource(self):
        """Find and return the resource whose title matches the given folder."""
        col = None
        for resource in self.client.GetAllResources(uri='/feeds/default/private/full/-/folder'):
            if resource.title.text == self.folder:
                col = resource
                break    
        return col
    
    def _send_email(self,msg):
        '''Send an email using the GMail account.'''
        senddate=datetime.strftime(datetime.now(), '%Y-%m-%d')
        m="Date: %s\r\nFrom: %s <%s>\r\nTo: %s\r\nSubject: %s\r\nX-Mailer: My-Mail\r\n\r\n" % (senddate, self.from_name, self.sender, self.recipient, self.subject)
        server = smtplib.SMTP('smtp.gmail.com:587')
        server.starttls()
        server.login(self.username, self.password)
        server.sendmail(self.sender, self.recipient, m+msg)
        server.quit()    

    def _upload(self, video_file_path, folder_resource):
        '''Upload the video and return the doc'''
        doc = gdata.docs.data.Resource(type='video', title=os.path.basename(video_file_path))
        media = gdata.data.MediaSource()
        media.SetFileHandle(video_file_path, 'video/avi')
        doc = self.client.CreateResource(doc, media=media, collection=folder_resource)
        return doc
    
    def upload_video(self, video_file_path):
        """Upload a video to the specified folder. Then optionally send an email and optionally delete the local file."""
        folder_resource = self._get_folder_resource()
        if not folder_resource:
            raise Exception('Could not find the %s folder' % self.folder)

        doc = self._upload(video_file_path, folder_resource)
                      
        if self.send_email:
            video_link = None
            for link in doc.link:
                if 'video.google.com' in link.href:
                    video_link = link.href
                    break
            # Send an email with the link if found
            msg = self.message
            if video_link:
                msg += '\n\n' + video_link                
            self._send_email(msg)    

        if self.delete_after_upload:
            os.remove(video_file_path)

if __name__ == '__main__':         
    try:
        if len(sys.argv) < 3:
            exit('Motion Uploader - uploads videos to Google Drive\n   by Jeremy Blythe (http://jeremyblythe.blogspot.com)\n\n   Usage: uploader.py {config-file-path} {video-file-path}')
        cfg_path = sys.argv[1]
        vid_path = sys.argv[2]    
        if not os.path.exists(cfg_path):
            exit('Config file does not exist [%s]' % cfg_path)    
        if not os.path.exists(vid_path):
            exit('Video file does not exist [%s]' % vid_path)    
        MotionUploader(cfg_path).upload_video(vid_path)        
    except gdata.client.BadAuthentication:
        exit('Invalid user credentials given.')
    except gdata.client.Error:
        exit('Login Error')
    except Exception as e:
        exit('Error: [%s]' % e)


Installation

If you haven't already got Python and Pip installed then do it now. On my Arch Linux Raspberry Pi I did this:
pacman -S python2
pacman -S python2-pip
Note: This script has been tested on Python 2.7.3
Next get the Google gdata library:
pip2 install gdata
Now download the two files above to somewhere on your machine. Make uploader.py executable:
chmod +x uploader.py
Open uploader.cfg to enter your settings:
[gmail]
# GMail account credentials
name = My Name
user = gmailusername
password = gmailpassword
sender = me@gmail.com

# Recipient email address (could be same as from_addr)
recipient = me@gmail.com

# Subject line for email
subject = Motion detected

# First line of email message
message = Video uploaded

[docs]
# Folder (or collection) in Docs where you want the videos to go
folder = motion

[options]
# Delete the local video file after the upload
delete-after-upload = true

# Send an email after the upload
send-email = true
Make sure you create a folder in your Google Drive to match the setting in the uploader.cfg file. This is the folder where the videos will go.
At this point it's worth testing the script e.g.
./uploader.py /etc/motion/uploader.cfg /srv/http/motion/cam1/85-20120608194940.avi
If all's well you can now change the Motion setting like so:
on_movie_end /root/py/uploader.py /etc/motion/uploader.cfg %f

Enjoy!

***** UPDATE: The script in this article no longer works due to Google authentication changes. Please use the new script described here: Motion Google Drive Uploader for OAuth 2.0


2012-06-05

Battery powered, Wireless, Motion detecting Raspberry Pi

Hardware

Software

  • Arch Linux ARM
  • Motion
  • Lighttpd

Set up

Update Arch

Before installing any new software it's a good idea to get up-to-date. I was caught out with an issue which seems to be catching others out too - when upgrading I got this:
error: failed to commit transaction (conflicting files)
hwids: /usr/share/hwdata/pci.ids exists in filesystem
hwids: /usr/share/hwdata/usb.ids exists in filesystem
Errors occurred, no packages were upgraded.
In this case, for these two files, it's OK to rename them and try again - it seems that there's been a change of which package supplies these files and it's not handled very gracefully.
[root@alarmpi ~]# mv /usr/share/hwdata/pci.ids /usr/share/hwdata/pci.ids.save
[root@alarmpi ~]# mv /usr/share/hwdata/usb.ids /usr/share/hwdata/usb.ids.save
[root@alarmpi ~]# pacman -Syu

Wifi

I chose the ASUS USB-N10 because of its size and the fact that it works out-of-the-box with Arch. It uses the r8712u module which is already installed with the distro. Getting the wireless to work well enough is not particularly straightforward. However there are some good instructions here: ArchWiki Wireless Setup. For the wireless management software I chose netcfg and wrote a script similar to the one given under "Intermittent Connection Failure" so I had complete control over what I wanted to do whenever I lost connection to my router. This was helpful for troubleshooting and useful when you're walking around the garden trying to find a good place to put the box.

Motion

Motion is a great piece of software. Just use pacman to install it and then follow the instructions on the Motion site for setting this up. My top tip would be to edit /etc/motion/motion.conf to set up the stream and control ports and switch off webcam_localhost and control_localhost. Then run it on the console "motion -s" and use the web control to fiddle with the settings - for some changes you'll have to restart motion. Sometimes you might need a mask file. I found that taking one of the snapshots, loading it in Gimp and painting over the top was the best way to do this. You can then export the file in pgm format. At 320x240 the Raspberry Pi was quite happy to run at 5fps with motion using about 20 to 25% CPU and low memory usage (I've still got 194M free).

Lighttpd

To get at the snapshots and movies that Motion has captured it's nice to be able to grab them from a web page. I'm going to write some Python to do this nicely, but in the mean time I've set up lighttpd to present the capture directory really quickly. Simply use pacman to install it and then edit /etc/lighttpd/lighttpd.conf following the quick instructions here: Tutorial Configuration

Note: I am now uploading my videos to Google Drive rather than keeping them on the Raspberry Pi. See: Motion Google Drive Uploader and Emailer

Results

I have been consistently getting nearly 12 hours running time from the battery pack on a full charge. That gave me enough time to leave the box out all day and catch some "wildlife". Now I just need to write some software at last! (More posts on that coming soon)