Overview

The following is an overview of each piece of the MonkeyFarm build environment.

monkeyfarm.hub

The MonkeyFarm Hub is a the central service that everything interacts with. It is the only piece that talks directly to the database, and provides both the web frontend as well as the API interface. The monkeyfarm.hub is built on top of the TurboGears2 mega framework.

Currently the ‘web frontend’ is not entirely built out, as the primary focus for use is via command line. A mostly readonly frontend will be built out in future versions.

The API interface is a REST-like design, however because command line is the primary use all resources are identified by a unique ‘label’ or by ‘project+label’. It exposes the following resources via HTTP/HTTPS:

• arch
• bug_handler
• build
• build_handler
• build_type
• distro
• group
• package
• package-branch
• project
• release
• system
• tag
• target
• task
• user
• util
• vcs_handler

Each resource responds to (at least) the following:

• get_one.json

  • example: GET /user/john.doe/get_one.json

• get_all.json

  • example: GET /user/get_all.json

• post.json

  • example: POST /user/post.json?label=john.doe&other-param=blah...

• put.json

  • example: PUT /user/john.doe/put.json?id=XX&label=john.doe&other-param=blah...

• delete.json

  • example: DELETE /user/john.doe/delete.json?confirm_delete=True

Some resources have additional functions exposed. For a full example of working with the API, see the MonkeyFarm Python Interface Library.

monkeyfarm.interface

The MonkeyFarm Interface is the library bindings for different programming languages. Currently, the only interface included in the MonkeyFarm source tree is for Python.

The interface libraries are intended to be lightweight and have limited external dependencies. The only external dependency of the Python library is zope.interface used to implement the MFRequestHandler Interface. The interface library do not rely on any other piece of the MonkeyFarm source to function.

monkeyfarm.core

The MonkeyFarm Core is simply the core application for the command line and daemon tools. It is built on top of the Cement CLI Application Framework and is required by all MonkeyFarm utilities (except the hub). It provides the command ‘mf’.

The Core piece sets up a base application for the client and other tools to build off of.

monkeyfarm.client

The MonkeyFarm Client provides the command line functions to use and interact with the hub. It exposes commands and options to the ‘mf’ command line tool extending its functionality.

monkeyfarm.regulator

The MonkeyFarm Regulator is primarily a daemon process that runs on top of monkeyfarm.core. It’s primary purpose is to cycle through any functions exposed to the ‘regulator_hook’. Any operations that need to be run constantly should be tied into the regulator_hook. Essentially, the regulator cycles through these hooks constantly. The regulator spawns a new process every time a regulator is started, meaning if one regulator takes minutes to run, it does not hold up other regulators that might take seconds to run. The regulator is configurable to determine cycle time, and the number of regulator_hooks to run simultaneously.

The regulator_hook should not be used for operations that are only meant to run occasionally. For example, an operation that should only run once a day should be configured via Cron and not via the regulator_hook.

The regulator is also responsible for the initial registration of build_handlers, vcs_handlers, and bug_handlers. These are internal pieces that provide a plugin interface to extend functionality. Their use is pretty straight forward:

build_handler

Determine how builds are handled

vcs_handler

Determines how to checkout source code from a VCS repo (Git, Bzr, etc)

bug_handler

Determines how to interface with a bug tracker (bugzilla, github, etc)

monkeyfarm.worker

The MonkeyFarm Worker is primarily a daemon process that runs on top of the monkeyfarm.core. It is the process that handles building tasks and is generally run on dedicated ‘builder’ systems.

The worker relies on the ‘build_handler’ system for it to be able to function. Plugins provide build_handler that tell MonkeyFarm how to build a task. For example, the monkeyfarm.mock plugin provides the ‘mock’ build_handler. With this plugin installed, the monkeyfarm.regulator process registers the build_handler on the hub, but it does not enable it at first. It waits for a worker to register that is also has that build_handler installed. Once a worker checks in with that build_handler, it is then enabled by the regulator and can be used for builds. When a worker pulls down a task, it uses the build_handler to build the task.