Table of Contents¶

django-issue Documentation¶

Overview¶

Sometimes things go wrong in production; if it is a repeating or ongoing error, it makes sense to represent and track this in some way. This is the purpose of the Issue class. When an error is detected, an Issue can be created to store details about it.

Possible advantages of this:

When something goes wrong, corrective actions that are taken often should not be repeated.
It gives admins an ability to view at a glance a history of actions taken by the system to address the issue.

Once an Issue is created, it is often desirable to act on it. For this, django-issue provides a Responder model. A Responder specifies a pattern to match against Issue`s; when a pattern matches for an :class:`Issue to a ‘Responder’, the Responder executes some configured action.

How are Issues created? They can be easily created by any bit of code. Alternatively, you can use the Assertion. The goal of an Assertion is to provide a means for detecting when certain properties of your system no longer hold true.

Think of it as a cross between the classic assert statement available in many programming langauge and traditional software monitoring systems like Nagios.

Philosophy¶

It’s often the case that you know how your system should behave (you built it). The problem is, your system doesn’t know how it’s supposed to behave. So when it missbehaves (due to bugs, unexpected edge-cases, user error, malware, a cat climbing on a keyboard), it doesn’t realize that something is amiss and continues on it’s way doing something terribly wrong. If the system had a more explicit notion of it’s expected behavior, then it could try to correct deviations, or at the very least, to escalate to a human and ask for help.

django-issue is an initial exploration into these ideas; how can we detect when things go wrong, represent the fact that they have gone wrong, and then respond to them?

Examples¶

Representing the fact that something is amiss¶

Suppose an error occurs in the middle of the night that needs to be addressed in the morning (but is not pressing enough to wake someone up). We could do something like this:

from isssue.models import Issue

try:
    // a problem occurs
except ValueError as ve:
    Issue.objects.create(name='That *impossible* edge case finally happened...', details=str(ve))

How do you know if something has gone wrong?¶

Enter the Assertion class. Suppose you have a model that tracks a heartbeat from some external service/software components. If, after some amount of time, the sytem does not receive a heartbeat, a human should be notified. Suppose you have an app called ‘heartbeat’ and your models.py file looks like this:

# Models.py
from datetime import datetime, timedelta

from django.db import models


class HeartbeatKeeper(model.Model):
    last_heartbeat = models.DateTimeField(auto_now=True)


def check_for_recent_heartbeat(**kwargs):
    """
    Returns (True, None) when all is well.
    Returns (False, None) otherwise.
    """
    delta = timedelta(minutes=30)
    interval = (datetime.utcnow() - HeartbeatKeeper.objects.get().last_heartbeat)
    return (interval < delta, None)

Now you create an Assertion to call your check_for_recent_heartbeat() function and create an Issue when it returns a tuple beginning with False:

Assertion.objects.create(target_function='heartbeat.models.check_for_recent_heartbeat', name='Check for heartbeat')

When the check_for_recent_heartbeat function returns a False tuple, then an Issue is created with the name ‘Check for heartbeat’).

There is a special type of a Assertion called a ModelAssertion. A ModelAssertion is designed to ensure that certain properties hold true for the models in your database.

Suppose you have a Profile model for your Users. After 5 days of signing up, you want to be notified if the user hasn’t created a profile pic yet. You have an app, ‘profile’, and your models.py file looks like this:

# Models.py
from datetime import datetime, timedelta

from django.contrib.auth.models import Group, User
from django.db import models


class Profile(model.Model):
    user = models.ForeignKey(User)
    # Note: selfies are preferred
    pic_url = models.URLField(null=True, blank=True)


def should_have_pic_by_now(record, **kwargs):
    """
    Check if the specified user has a pic or still has time for one.
    """
    interval = timedelta(days=5)

    has_pic = record.pic_url is not None
    date_joined = record.user.date_joined

    okay = has_pic or (datetime.utcnow() - date_joined) < interval

    return (okay, None)

Now you create an ModelAssertion to call your check_for_recent_heartbeat() function and create an Issue when it returns a tuple beginning with False:

from django.contrib.contenttypes.models import ContentType

from profile.models import Profile

ModelAssertion.objects.create(
    target_function='profile.models.should_have_pic_by_now', name='Check for pic', model_type=ContentType.get_for_model(Profile))

Now whenever a Uer account (and associated Profile) is created, an Issue is created if the user does not set a profile pic within 5 days.

Addressing an ongoing problem¶

Now when this exception occurs, we will have a record in the database along with details about what happened and when. Now suppose we want an email notification when this happens. Well we could add the following:

from issue.models import Responder, ResponderAction


r = Responder.objects.create(watch_pattern='That \*impossible\* edge case finally happened')

ResponderAction.objects.create(responder=r, delay_sec=30, target_function='issue.actions.email',
    function_kwargs={
        'subject': 'Doh!',
        'recipients': 'john.smith@example.com',
    })

There is a helper function, build_responder() for constructing a Responder and one or more associated ResponderAction from json:

from issue.builder import build_responder


build_responder({
    'watch_pattern': 'That \'impossible\' edge case finally happened...',
    'actions': [
        {
            'target_function': 'issue.actions.email',
            'function_kwargs': {
                'subject': 'Doh!',
                'recipients': 'john.smith@example.com',
            },
            'delay_sec': 30,
        },
        {
            'target_function': 'issue.actions.email',
            'function_kwargs': {
                'subject': 'Doh-2!',
                'recipients': 'john.smith-boss@example.com',
            },
            'delay_sec': 1800,
        },
    ]})

The delay_sec may be ommitted; when this happens the ResponderAction will be executed as soon as the Responder matches against an Issue.

When do these checks happen?¶

Two management commands are provided, check_assertions and respond_to_issues which should be ran periodically.

Installation¶

To install the latest release, type:

pip install django-issue

To install the latest code directly from source, type:

pip install git+git://github.com/ambitioninc/django-issue.git

Code documentation¶

django_issue¶

Issue classes¶

Assertion Classes¶

Responder classes¶

Contributing¶

Contributions and issues are most welcome! All issues and pull requests are handled through github on the ambitioninc repository. Also, please check for any existing issues before filing a new one. If you have a great idea but it involves big changes, please file a ticket before making a pull request! We want to make sure you don’t spend your time coding something that might not fit the scope of the project.

Running the tests¶

To get the source source code and run the unit tests, run:

git clone git://github.com/ambitioninc/django-issue.git
cd django-issue
virtualenv env
. env/bin/activate
python setup.py install
coverage run setup.py test
coverage report --fail-under=100

While 100% code coverage does not make a library bug-free, it significantly reduces the number of easily caught bugs! Please make sure coverage is at 100% before submitting a pull request!

Code Quality¶

For code quality, please run flake8:

pip install flake8
flake8 .

Code Styling¶

Please arrange imports with the following style

# Standard library imports
import os

# Third party package imports
from mock import patch
from django.conf import settings

# Local package imports
from issue.version import __version__

Please follow Google’s python style guide wherever possible.

Building the docs¶

When in the project directory:

pip install -r requirements/docs.txt
python setup.py build_sphinx
open docs/_build/html/index.html

Release Checklist¶

Before a new release, please go through the following checklist:

Bump version in issue/version.py
Add a release note in docs/release_notes.rst
Git tag the version

Upload to pypi:

pip install wheel
python setup.py sdist bdist_wheel upload

Vulnerability Reporting¶

For any security issues, please do NOT file an issue or pull request on github! Please contact security@ambition.com with the GPG key provided on Ambition’s website.

Release Notes¶

v3.1.2¶

Fix django warning related to JSONField default value

v3.1.1¶

Remove 1.10 from setup file

v3.1.0¶

json field encoder (drop support for django 1.10)

v3.0.0¶

Add tox to support more versions
Upgrade to django’s JSONField

v2.0.0¶

Remove python 2.7 support
Remove python 3.4 support
Remove Django 1.9 support
Remove Django 1.10 support
Add Django 2.0 support

v1.4.0¶

Python 3.6 support
Django 1.10 support
Django 1.11 support
Remove Django 1.8 support

v1.3.0¶

Python 3.5 support, remove django 1.7 support

v1.2.0¶

Django 1.9 support

v1.1.0¶

Django 1.8 support and removal of 1.6 support

v1.0.5¶

Additional tweak to the behavior of maybe_open_issue

v1.0.4¶

Remove south as a dependency

v1.0.3¶

Tweak to the behavior of reopen_issue

v1.0.2¶

Tweak to the behavior of maybe_open_issue

v1.0.1¶

Added a helper method, maybe_open_issue, to the IssueManager class
This implements logic that’s begun to repeat in my Issue use cases

v1.0.0¶

Added Django 1.7 compatability
ModelAssertion and BaseAssertion’s .check() methods were renamed

to check_assertion()

v0.1¶

This is the initial release of django-issue.