Welcome to MCL‘s documentation!

The multiprocess communication library (a.k.a MCL) provides software for developing low-latency message-passing systems. Message-passing is performed using IPv6 multicast. Two key concepts underpin the design philosophy of and use of MCL: the publish-subscribe paradigm and inter-process communication.

To ensure the success and longevity of a software project, it must be extensible and reliable. MCL encourages users to adopt a modular design when developing software projects. By breaking up software into modules of functionality, small portions of code can be independently developed and maintained - increasing flexibility, maintainability and extensibility. Information is transfered from one module to another using the publish-subscribe paradigm. This strategy allows event-driven code to be run on many independent processes. Message passing over the publish-subscribe network allows communication to occur transparently within a single computer, across multiple computers and across heterogeneous devices.

MCL was developed at The Intelligent Vehicles and Safety Systems Group at the Australian Centre for Field Robotics (ACFR).

The following sections provide an introduction to the MCL project:

Getting started

Operating system support

MCL has been developed in the GNU/Linux operating system. In particular, Debian based systems have been used. As a result, support for Windows and Mac OS X and other Unix systems have not been tested. Due to Python’s flexibility, MCL is likely to work on these platforms with a little more (undocumented) effort.

Obtaining the code

MCL can be checked out from the git repository using

git clone https://github.com/acfr/mcl <target>

where <target> is your local target directory.

Installation

This code supports installation using pip (via setuptools). To install from the cloned repository:

cd <target>
sudo pip install .

To uninstall the package:

pip uninstall mcl

Documentation

The MCL documentation can be read online. To build a local copy run:

cd <target>
make docs

The entry point of the documentation can then be found at:

<target>/doc/build/html/index.html

During compilation, Sphinx’ will attempt to apply the `Read the Docs theme. If this theme is not available, the default Sphinx theme will be used. To enable the Read the Docs theme, install the package via pip:

sudo pip install sphinx_rtd_theme

Contributing

The MCL code base is version controlled using Git.

Identify Yourself

If you are contributing to the code base, make sure your contact details have been correctly setup. Do not use an obscure identity in the MCL repositories - this will make it difficult to associate code changes with a contributor. An email address which identifies the author is ideal. This can be done by issuing

git config user.email "<name>@<domain>.com"

in the root of the repository or system wide for all repositories

git config user.email --global "<name>@<domain>.com"

Commit Workflow

When committing code to the MCL repository, good commit hygiene ensures that collaborators can monitor changes to the code base. Some guidelines for good development practise are:

  • Development including bug fixes and new features should be developed on a local branch based on ‘master’ (in general).
  • Make separate commits for logically separate changes. To reduce the amount of “patch noise”, do not submit trivial or minor changes.
  • The MCL project favours rebase rather than merge commits on the master branch. A clean linear history of code changes is preferred over traceability of development branches. Never abuse the rebase command to modify the history of commits that have been pushed into a public repository (the same goes for git commit --amend and git reset).

The following is an example of development adhering to this work-flow

  1. Get latest changes using:

    git checkout master
    git fetch origin
    git merge master
    

    or

    git checkout master
    git pull origin master
    
  2. Isolate development of feature (or bug fix) on a new branch:

    git checkout -b newfeature
    
  3. Work on feature

  4. Import commits from the remote repository and integrate with the feature branch:

    git fetch origin
    git rebase origin/master
    
  5. Repeat steps 3 & 4 until development has completed. Be sure to finalise development by completing step 4 before attempting to integrate your changes with the remote repository.

  6. Integrate changes with the remote repository:

    git checkout master
    git pull
    git rebase newfeature
    

    To interactively edit the sequence of commits on the feature branch which are about to be rebased use:

    git rebase -i newfeature
    

    Finally,

    git push
    

Commit Logs

Good commit hygiene also includes how commit messages are create. A good commit message will effectively communicate your changes to collaborators. Concise yet descriptive messages are important when viewing the commit history. Good commit messages contain:

  1. A short description (soft limit of 50 characters) of the commit on the first line.

    • The first line should not be terminated by a full stop.

    • The first line is often prefixed with the main location of development. For example:

      network: read multiple items from IPv6, UDP socket
      
  2. A blank line

  3. A more detailed description of the commit wrapped to 72 characters.

An example of a good commit message is included from the git documentation:

Short (50 chars or less) summary of changes

More detailed explanatory text, if necessary.  Wrap it to about 72
characters or so.  In some contexts, the first line is treated as the
subject of an email and the rest of the text as the body.  The blank
line separating the summary from the body is critical (unless you omit
the body entirely); tools like rebase can get confused if you run the
two together.

Further paragraphs come after blank lines.

 - Bullet points are okay, too

 - Typically a hyphen or asterisk is used for the bullet, preceded by a
   single space, with blank lines in between, but conventions vary here

Style Guide

This section describes the coding and documentation style used in MCL. These styles leverage community standards and should be familiar to pythonists, pythoneers, pythonistas’ and abusers of the word idiomatic. For experienced Python developers, this section outlines how the community standards have been used (and abused) in MCL. A brief read-through aught to be sufficient. For new Python users, this section serves a crash-course in Python coding and documentation convention. Further detail is provided externally via links.

Code

The coding convention adopted in the standard library of the main Python distribution is defined in PEP8: the Style Guide for Python Code. PEP8 warns:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is most important.

In short, when writing new code, take a few minutes to determine the style of the surrounding code. When contributing to the code base, attempt to ‘blend in’ and be consistent with its style.

Python is the main scripting language used at Google. The Google Python Style Guide largely complements PEP8. When in doubt, adopt the style of the surrounding code or defer to PEP8.

Help You Help Yourself

Your favourite IDE or editor can help you adhere to the PEP8 guidelines. This can be done by configuring a code checking tool to check your Python code. The tool pep8.py checks python code against some of the style conventions in PEP8. You can install, upgrade or uninstall pep8.py with these commands:

pip install pep8
pip install --upgrade pep8
pip uninstall pep8

The Pylint tool is a source-code quality and bug checker. Pylint provides:

  • checking line-code’s length,
  • checking if variable names are well-formed according to your coding standard
  • checking if imported modules are used

On Debian-based systems Pylint can be installed using:

sudo apt-get install pylint

Other (overlapping) code checking tools include flake8, pyflakes and pychecker.

Documentation

Docstrings

Python emphasises documentation through the use of docstrings:

A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition [1].

Properly documenting code not only makes the source more accessible to other programmers but it enables software such as the docstring processing system, Docutils and Sphinx to parse the documentation.

Sphinx

MCL documentation is generated using Sphinx. Sphinx uses reStructuredText as its markup language. Many of Sphinx‘s strengths come from the power and straightforwardness of reStructuredText and its parsing and translating suite - the Docutils [2]. Due to Sphinx‘s reliance on Docutils, the quality of Sphinx documentation depends on the completeness of the project’s docstrings.

MCL uses Google style docstrings. A comprehensive example of Google style docstrings can be found here. Support for Google style docstrings in Sphinx is provided through the Napoleon extension.

Warning

As of Sphinx 1.3, the napoleon extension will come packaged with Sphinx under sphinx.ext.napoleon. The sphinxcontrib.napoleon extension will continue to work with Sphinx <= 1.2.

Since Sphinx relies on reStructuredText, it is worth learning the markup. Examples are provided in:

API documentation is easier to traverse and more powerful at communicating if function, class, method or attribute definitions are linked when they are referenced. Sphinx provides markup allowing these constructs to be hyperlinked when they are referenced in the documentation.

Structure

sphinx-apidoc is a tool for automatic generation of Sphinx sources that, using the autodoc extension, document a whole package in the style of other automatic API documentation tools.

Although sphinx-apidoc provides an ‘automagic’ mechanism for documentation, it is difficult to override its behaviour and format the look and feel of the documentation it produces. To allow for more control over the presentation of the MCL documentation, the structure is specified explicitly. Although this approach requires more supervision, the ‘heavy lifting’ is still done by autodoc and autosummary.

The Sphinx structure is set out in the doc/source directory. The master document, doc/source/index.rst, serves as a welcome page and contains the root of the “table of contents tree” (or toctree) - linking to additional documentation. The Sphinx configuration can be found in doc/source/conf.py.

Learning how to extend the documentation is best done by studying the structure defined in the doc/source directory. There are two points within the documentation that require maintenance. The master document, doc/source/index.rst, needs to be updated every time a new package is created. If new modules are added to an existing package, the package __init__.py file must be updated to include the new module. Modifications to existing modules do not require any maintenance - these changes will be handled by autodoc and autosummary.

Footnotes

[1]PEP 257: Docstring Conventions
[2]Sphinx: Python Documentation Generator

MCL API documentation

The following packages are available in MCL:

mcl Root MCL namespace.
mcl.event Event-driven functionality of MCL.
mcl.logging Package for creating and handling network log files in MCL.
mcl.messages Modules for specifying and manipulating MCL message objects.
mcl.network Network communication services.

Indices and tables