Welcome to MongoElector’s documentation!

Contents:

MongoElector

About

The MongoElector project provides two pieces of distributed coordination; Distributed locks via ‘MongoLocker’ and master elections via ‘MongoElector’. MongoElector makes heavy use of MongoLocker, but the locking functionality within MongoLocker can be used separately.

• Free software: GPLv3
• Documentation: https://mongoelector.readthedocs.io.

Note

As of 0.3.0 release, the distributed master election functionality is working. Additional features and functionality as well as bug fixes and minor API changes will be ongoing over the next couple of minor releases.

Features

• Simple API to allow distributed master election
• Distributed locking via MongoDB
• Ensure/Verify a specific instance holds the lock
• TTL

Todo

• Cluster Health and Management within MongoElector

Installation

Stable release

To install MongoElector, run this command in your terminal:

$ pip install mongoelector

From sources

The sources for MongoElector can be downloaded from the Github repo.

You can either clone the public repository:

$ git clone git://github.com/zebpalmer/mongoelector

Or download the tarball:

$ curl  -OL https://github.com/zebpalmer/mongoelector/tarball/master

Once you have a copy of the source, you can install it with:

$ python setup.py install

Usage

To use MongoElector in a project:

import mongoelector

MongoElector

A few random examples of interacting with MongoElector

# Instantiate MongoElector object
elector = MongoElector('CleverName', dbconn, ttl=15, dbname='coolproj',
                       onmaster=self.onmaster, onmasterloss=self.onmasterloss)

# example callbacks

def onmasterloss():
    logging.info("Master Lost, shutting down task scheduler)
    sched.shutdown() # shutdown APScheduler

def onmaster():
    logging.info("Elected master, starting task scheduler)
    sched.start() # start APScheduler

# start mongoelector
elector.start()

# shutdown mongoelector, release master lock
elector.stop()

# log if master
logging.debug('master status: {}'.format(elector.ismaster))

# release the master lock, allowing another instance to take it
elector.release()

# log if master exists (on any node)
logging.debug('Cluster master is running: {}'.format(elector.master_exists))

MongoLocker

# create Pymongo dbconnection
dbconn = MongoClient(cfg.dbhost)
# create lock object with a ttl of 60.
# If the lock isn't refreshed within ttl seconds, it will auto-expire.
mlock = MongoLocker(self.key, dbconn, ttl=60, dbname='coolproj',
                    dbcollection='electorlocks')

# acquire the lock, raise AcquireTimeout after 30 seconds if not acquired.
mlock.acquire(timeout=30)

# release lock
mlock.release()

# check to see if lock is locked by any instance
print(mlock.locked())

# check to see if lock is owned by this instance
print(mlock.owned())

Code Documentation

class mongoelector.MongoLocker(key, db, dbcollection='mongolocker', ttl=600, timeparanoid=True)

Distributed lock object backed by MongoDB.

Intended to mimic standard lib Lock object as much as reasonable. This object is used by MongoElector, but is perfectly happy being used as a standalone distributed locking object.

Parameters:

• key (str) – Name of distributed lock
• dbconn (PyMongo db connection) – Pymongo client connection to mongodb
• dbname (str) – name of database (defaults to ‘mongoelector’)
• dbname – name of collection (defaults to ‘mongolocker’)
• ttl (int) – Lock will expire (ttl seconds) after acquired unless renewed or released
• timeparanoid (bool) – Sanity check to ensure local server time matches mongodb server time (utc)

acquire(blocking=True, timeout=None, step=0.25, force=False)

Attempts to acquire the lock, will block and retry indefinitely by default. Can be configured not to block, or to have a timeout. You can also force the acquisition if you have a really good reason to do so.

Parameters:

• blocking (bool) – If true (default), will wait until lock is acquired.
• timeout (int) – blocking acquire will fail after timeout in seconds if the lock hasn’t been acquired yet.
• step (float or int) – delay between acquire attempts
• force (bool) – CAUTION: will forcibly take ownership of the lock

get_current()

Returns the current (valid) lock object from the database, regardless of which instance it is owned by.

locked()

Returns current status of the lock, but does not indicate if the current instance has ownership or not. (for that, use ‘self.owned()’) This is a ‘look before you leap’ option. For example, it can be used to ensure that some process is owns the lock and is doing the associated work. Obviously this method does not guarantee that the current instance will be successful in obtaining the lock on a subsequent acquire.

Returns:Lock status Return type:bool

owned()

Determines if self is the owner of the lock object. This verifies the instance uuid matches the uuid of the lock record in the db.

Returns:Owner status Return type:bool

release(force=False)

releases lock if owned by the current instance.

Parameters:force – CAUTION: Forces the release to happen,

even if the local instance isn’t the lock owner. :type force: bool

status

touch()

Renews lock expiration timestamp

Returns:new expiration timestamp Return type:datetime

class mongoelector.MongoElector(key, db, ttl=15, onmaster=None, onmasterloss=None, onloop=None, app_version=None, report_status=True)

This object will do lots of awesome distributed master election coolness

Create a MongoElector instance

Parameters:

• key (str) – Name of the distributed lock that is used for master election. should be unique to this type of daemon i.e. any instance for which you want to run exactly one master should all share this same name.
• db – Connection to a MongoDB database
• ttl (int) – Time-to-live for the distributed lock. If the master node fails silently, this timeout must be hit before another node will take over.
• onmaster (Function or Method) – Function that will be run every time this instance is elected as the new master
• onmasterloss (Function or Method) – Function that will be run every time when this instance loses it’s master status
• onloop (Function or Method) – Function that will be run on every loop
• app_version (str) – Parent app version, if provided, will be included in node_status for monitoring

cluster_detail

ismaster

Returns True if this instance is master

master_exists

Returns true if an instance (not necessarily this one) has master

node_status

Status info for current object

poll()

Main polling logic, will refresh lock if it’s owned, or tries to obtain the lock if it’s available. Runs onloop callback after lock maintenance logic

In general, this should only be called by the elector thread

pollwait

An appropriate sleep time to wait before next poll

release()

Releases master lock if owned and calls onmasterloss if provided.

report_status()

running

Returns true if the elector logic is running

start(blocking=False)

Starts mongo elector polling on a background thread then returns. If blocking is set to True, this will never return until stop() is

Parameters:blocking (bool) – If False, returns as soon as the elector thread is started. If True, will only return after stop() is called i.e. by another thread.

stop()

Cleanly stop the elector. Surrender master if owned

Contributing

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

You can contribute in many ways:

Types of Contributions

Report Bugs

Report bugs at https://github.com/zebpalmer/MongoElector/issues.

If you are reporting a bug, please include:

• Your operating system name and version.
• Your python version
• MongoDB version
• Any details about your local setup that might be helpful in troubleshooting.
• Detailed steps to reproduce the bug.

Submit Feedback

The best way to send feedback is to file an issue at https://github.com/zebpalmer/MongoElector/issues.

If you are proposing a feature:

• Explain in detail how it would work.
• Keep the scope as narrow as possible, to make it easier to implement.
• Remember that this is a volunteer-driven project, and that contributions are welcome :)

Get Started!

Ready to contribute? Here’s how to set up mongoelector for local development.

1. Fork the mongoelector repo on GitHub.

2. Clone your fork locally:

   $ git clone git@github.com:your_name_here/mongoelector.git
   

3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:

   $ mkvirtualenv mongoelector
   $ cd mongoelector/
   $ python setup.py develop
   

4. Create a branch for local development:

   $ git checkout -b name-of-your-bugfix-or-feature
   

   Now you can make your changes locally.

5. When you’re done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:

   $ python setup.py test
   $ tox
   

   To get flake8 and tox, just pip install them into your virtualenv. You will also need an instance of MongoDB running, tests default connecting to localhost.

6. Commit your changes and push your branch to GitHub:

   $ git add .
   $ git commit -m "Your detailed description of your changes."
   $ git push origin name-of-your-bugfix-or-feature
   

7. Submit a pull request through the GitHub website.

Pull Request Guidelines

Before you submit a pull request, check that it meets these guidelines:

1. The pull request should include tests.
2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst.
3. The pull request should work for Python 2.7, 3.4 and 3.5. Check https://travis-ci.org/zebpalmer/MongoElector/pull_requests and make sure that the tests pass for all supported Python versions.

Tips

To run a subset of tests:

$ python -m unittest tests.test_mongoelector

Authors

Creator

• Zeb Palmer <zeb@zebpalmer.com>

Contributors

• Jesse Roberts

History

0.3.0 (2016-08-22)

• Initial working release of master election logic.

0.2.1 (2016-08-10)

• Distributed locking api & functionality largely complete & stable.

0.0.1 (2016-05-13)

• Hello World

Indices and tables

• Index
• Module Index
• Search Page