Django DB Mutex

.gitignore

@@ -0,0 +1,16 @@
+# Compiled python files
+*.pyc
+
+# Vim files
+*.swp
+*.swo
+
+# Coverage files
+.coverage
+
+# Setuptools distribution folder.
+/dist/
+
+# Python egg metadata, regenerated from source files by setuptools.
+/*.egg-info
+/*.egg

.travis.yml

@@ -0,0 +1,15 @@
+language: python
+python:
+  - '2.7'
+env:
+  - DJANGO=1.6.1 DB=postgres
+install:
+  - pip install -q Django==$DJANGO
+  - pip install -r requirements.txt
+before_script:
+  - find . | grep .py$ | grep -v /migrations | xargs pep8 --max-line-length=120
+  - find . | grep .py$ | grep -v /migrations | grep -v __init__.py | xargs pyflakes
+  - psql -c 'CREATE DATABASE db_mutex;' -U postgres
+script:
+  - coverage run --source='db_mutex' --branch --omit 'db_mutex/migrations/*' manage.py test
+  - coverage report --fail-under=100

LICENSE

@@ -2,20 +2,19 @@ The MIT License (MIT)
 
 Copyright (c) 2014 Ambition
 
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
 
 The above copyright notice and this permission notice shall be included in all
 copies or substantial portions of the Software.
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

MANIFEST.in

@@ -0,0 +1,3 @@
+include db_mutex/VERSION
+include README.md
+include LICENSE

README.md

@@ -1,4 +1,79 @@
+[![Build Status](https://travis-ci.org/ambitioninc/django-db-mutex.png)](https://travis-ci.org/ambitioninc/django-db-mutex)
 django-db-mutex
 ===============
 
-Acquire a mutex via the DB in Django
+Provides the ability to acquire a mutex lock from the database in Django.
+
+## A Brief Overview
+For critical pieces of code that cannot overlap with one another, it is often necessary to acquire a mutex lock of some sort. Many solutions use a memcache lock strategy, however, this strategy can be brittle in the case of memcache going down or when an unconsistent hashing function is used in a distributed memcache setup.
+
+If your application does not need a high performance mutex lock, Django DB Mutex does the trick. The common use case for Django DB Mutex is to provide the abilty to lock long-running periodic tasks that should not overlap with one another. Celery is the common backend for Django when scheduling periodic tasks.
+
+## How to Use Django DB Mutex
+The Django DB Mutex app provides a context manager and function decorator for locking a critical section of code. The context manager is used in the following way:
+
+        from db_mutex import db_mutex, DBMutexError, DBMutexTimeoutError
+
+        # Lock a critical section of code
+        try:
+            with db_mutex('lock_id'):
+                # Run critical code here
+                pass
+        except DBMutexError:
+            print 'Could not obtain lock'
+        except DBMutexTimeoutError:
+            print 'Task completed but the lock timed out'
+
+You'll notice that two errors were caught from this context manager. The first one, DBMutexError, is thrown if the lock cannot be acquired. The second one, DBMutexTimeoutError, is thrown if the critical code completes but the lock timed out. More about lock timeout in the next section.
+
+The db_mutex decorator can also be used in a similar manner for locking a function.
+
+        from db_mutex import db_mutex, DBMutexError, DBMutexTimeoutError
+
+        @db_mutex('lock_id')
+        def critical_function():
+            pass
+
+        try:
+            critical_function()
+        except DBMutexError:
+            print 'Could not obtain lock'
+        except DBMutexTimeoutError:
+            print 'Task completed but the lock timed out'
+
+## Lock Timeout
+Django DB Mutex comes with lock timeout baked in. This ensures that a lock cannot be held forever. This is especially important when working with segments of code that may run out of memory or produce errors that do not raise exceptions.
+
+In the default setup of this app, a lock is only valid for 30 minutes. As shown earlier in the example code, if the lock times out during the execution of a critical piece of code, a DBMutexTimeoutError will be thrown. This error basically says that a critical section of your code could have overlapped (but it doesn't necessarily say if a section of code overlapped or didn't).
+
+In order to change the duration of a lock, set the DB_MUTEX_TTL_SECONDS variable in your settings.py file to a number of seconds. If you want your locks to never expire (beware!), set the setting to None.
+
+## Usage with Celery
+Django DB Mutex can be used with celery's tasks in the following manner.
+
+    from celery import Task
+    from abc import ABCMeta, abstractmethod
+
+    class NonOverlappingTask(Task):
+        __metaclass__ = ABCMeta
+
+        @abstractmethod
+        def run_worker(self, *args, **kwargs):
+            """
+            Run worker code here.
+            """
+            pass
+
+        def run(self, *args, **kwargs):
+            try:
+                with db_mutex(self.__class__.__name__):
+                    self.run_worker(*args, **kwargs):
+            except DBMutexError:
+                # Ignore this task since the same one is already running
+                pass
+            except DBMutexTimeoutError:
+                # A task ran for a long time and another one may have overlapped with it. Report the error
+                pass
+
+## License
+MIT License (see the LICENSE file included in the repository)

db_mutex/VERSION

@@ -0,0 +1 @@
+0.1

db_mutex/__init__.py

@@ -0,0 +1 @@
+from .db_mutex import DBMutexError, DBMutexTimeoutError, db_mutex

db_mutex/db_mutex.py

@@ -0,0 +1,126 @@
+from datetime import datetime, timedelta
+import functools
+
+from django.conf import settings
+from django.db import transaction, IntegrityError
+
+from .models import DBMutex
+
+
+class DBMutexError(Exception):
+    """
+    Thrown when a lock cannot be acquired.
+    """
+    pass
+
+
+class DBMutexTimeoutError(Exception):
+    """
+    Thrown when a lock times out before it is released.
+    """
+    pass
+
+
+class db_mutex(object):
+    """
+    An object that acts as a context manager and a function decorator for acquiring a
+    DB mutex lock.
+
+    Args:
+        lock_id: The ID of the lock one is trying to acquire
+
+    Raises:
+        DBMutexError when the lock cannot be obtained
+        DBMutexTimeoutError when the lock was deleted during execution
+
+    Examples:
+        This context manager/function decorator can be used in the following way
+
+        from db_mutex import db_mutex
+
+        # Lock a critical section of code
+        try:
+            with db_mutex('lock_id'):
+                # Run critical code here
+                pass
+        except DBMutexError:
+            print 'Could not obtain lock'
+        except DBMutexTimeoutError:
+            print 'Task completed but the lock timed out'
+
+        # Lock a function
+        @db_mutex('lock_id'):
+        def critical_function():
+            # Critical code goes here
+            pass
+
+        try:
+            critical_function()
+        except DBMutexError:
+            print 'Could not obtain lock'
+        except DBMutexTimeoutError:
+            print 'Task completed but the lock timed out'
+    """
+    mutex_ttl_seconds_settings_key = 'DB_MUTEX_TTL_SECONDS'
+
+    def __init__(self, lock_id):
+        self.lock_id = lock_id
+        self.lock = None
+
+    def get_mutex_ttl_seconds(self):
+        """
+        Returns a TTL for mutex locks. It defaults to 30 minutes. If the user specifies None
+        as the TTL, locks never expire.
+        """
+        return getattr(settings, self.mutex_ttl_seconds_settings_key, timedelta(minutes=30).total_seconds())
+
+    def delete_expired_locks(self):
+        """
+        Deletes all expired mutex locks if a ttl is provided.
+        """
+        ttl_seconds = self.get_mutex_ttl_seconds()
+        if ttl_seconds is not None:
+            DBMutex.objects.filter(creation_time__lte=datetime.utcnow() - timedelta(seconds=ttl_seconds)).delete()
+
+    def __call__(self, func):
+        return self.decorate_callable(func)
+
+    def __enter__(self):
+        self.start()
+
+    def __exit__(self, *args):
+        self.stop()
+
+    def start(self):
+        """
+        Acquires the db mutex lock. Takes the necessary steps to delete any stale locks.
+        Throws a DBMutexError if it can't acquire the lock.
+        """
+        # Delete any expired locks first
+        self.delete_expired_locks()
+        try:
+            with transaction.atomic():
+                self.lock = DBMutex.objects.create(lock_id=self.lock_id)
+        except IntegrityError:
+            raise DBMutexError('Could not acquire lock: {0}'.format(self.lock_id))
+
+    def stop(self):
+        """
+        Releases the db mutex lock. Throws an error if the lock was released before the function finished.
+        """
+        if not DBMutex.objects.filter(id=self.lock.id).exists():
+            raise DBMutexTimeoutError('Lock {0} expired before function completed'.format(self.lock_id))
+        else:
+            self.lock.delete()
+
+    def decorate_callable(self, func):
+        """
+        Decorates a function with the db_mutex decorator by using this class as a context manager around
+        it.
+        """
+        def wrapper(*args, **kwargs):
+            with self:
+                result = func(*args, **kwargs)
+            return result
+        functools.update_wrapper(wrapper, func)
+        return wrapper

db_mutex/migrations/0001_initial.py

@@ -0,0 +1,34 @@
+# -*- coding: utf-8 -*-
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        # Adding model 'DBMutex'
+        db.create_table(u'db_mutex_dbmutex', (
+            (u'id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('lock_id', self.gf('django.db.models.fields.CharField')(unique=True, max_length=256)),
+            ('creation_time', self.gf('django.db.models.fields.DateTimeField')(auto_now_add=True, blank=True)),
+        ))
+        db.send_create_signal(u'db_mutex', ['DBMutex'])
+
+
+    def backwards(self, orm):
+        # Deleting model 'DBMutex'
+        db.delete_table(u'db_mutex_dbmutex')
+
+
+    models = {
+        u'db_mutex.dbmutex': {
+            'Meta': {'object_name': 'DBMutex'},
+            'creation_time': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'lock_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'})
+        }
+    }
+
+    complete_apps = ['db_mutex']

db_mutex/models.py

@@ -0,0 +1,9 @@
+from django.db import models
+
+
+class DBMutex(models.Model):
+    """
+    Models a mutex lock with a lock ID and a creation time.
+    """
+    lock_id = models.CharField(max_length=256, unique=True)
+    creation_time = models.DateTimeField(auto_now_add=True)

manage.py

@@ -0,0 +1,10 @@
+#!/usr/bin/env python
+import os
+import sys
+
+if __name__ == '__main__':
+    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'test_project.settings')
+
+    from django.core.management import execute_from_command_line
+
+    execute_from_command_line(sys.argv)

requirements.txt

@@ -0,0 +1,9 @@
+coverage
+django-dynamic-fixture==1.6.5
+django-nose==1.1
+pep8
+psycopg2==2.4.5
+pyflakes
+south==0.7.6
+freezegun==0.1.13
+# Note that Django is a requirement, but it is installed in the .travis.yml file in order to test against different versions

setup.py

@@ -0,0 +1,26 @@
+import os
+from setuptools import setup
+
+
+setup(
+    name='django-db-mutex',
+    version=open(os.path.join(os.path.dirname(__file__), 'db_mutex', 'VERSION')).read().strip(),
+    description='Acquire a mutex via the DB in Django',
+    long_description=open('README.md').read(),
+    url='http://github.com/ambitioninc/django-db-mutex/',
+    author='Wes Kendall',
+    author_email='[email protected]',
+    packages=[
+        'manager_utils',
+    ],
+    classifiers=[
+        'Programming Language :: Python',
+        'License :: OSI Approved :: BSD License',
+        'Operating System :: OS Independent',
+        'Framework :: Django',
+    ],
+    install_requires=[
+        'django>=1.6',
+    ],
+    include_package_data=True,
+)