Charm Helpers Documentation

The charmhelpers Python library is an extensive collection of functions and classes for simplifying the development of Juju Charms. It includes utilities for:

  • Interacting with the host environment
  • Managing hook events
  • Reading and writing charm configuration
  • Installing dependencies
  • Much, much more!

Getting Started

For a video introduction to charmhelpers, check out this Charm School session. To start using charmhelpers, proceed with the instructions on the remainder of this page.

Installing Charm Tools

First, follow these instructions to install the charm-tools package for your platform.

Creating a New Charm

$ cd ~
$ mkdirs -p charms/precise
$ cd charms/precise
$ charm create -t python mycharm
INFO: Generating template for mycharm in ./mycharm
INFO: No mycharm in apt cache; creating an empty charm instead.
Symlink all hooks to one python source file? [yN] y
INFO:root:Loading charm helper config from charm-helpers.yaml.
INFO:root:Checking out lp:charm-helpers to /tmp/tmpPAqUyN/charm-helpers.
Branched 160 revisions.
INFO:root:Syncing directory: /tmp/tmpPAqUyN/charm-helpers/charmhelpers/core -> lib/charmhelpers/core.
INFO:root:Adding missing __init__.py: lib/charmhelpers/__init__.py

Let’s see what our new charm looks like:

$ tree mycharm/
mycharm/
├── charm-helpers.yaml
├── config.yaml
├── hooks
│   ├── config-changed -> hooks.py
│   ├── hooks.py
│   ├── install -> hooks.py
│   ├── start -> hooks.py
│   ├── stop -> hooks.py
│   └── upgrade-charm -> hooks.py
├── icon.svg
├── lib
│   └── charmhelpers
│       ├── core
│       │   ├── fstab.py
│       │   ├── hookenv.py
│       │   ├── host.py
│       │   └── __init__.py
│       └── __init__.py
├── metadata.yaml
├── README.ex
├── revision
├── scripts
│   └── charm_helpers_sync.py
└── tests
    ├── 00-setup
    └── 10-deploy

6 directories, 20 files

The charmhelpers code is bundled in our charm in the lib/ directory. All of our python code will go in hooks/hook.py. A look at that file reveals that charmhelpers has been added to the python path and imported for us:

$ head mycharm/hooks/hooks.py -n11
#!/usr/bin/python

import os
import sys

sys.path.insert(0, os.path.join(os.environ['CHARM_DIR'], 'lib'))

from charmhelpers.core import (
    hookenv,
    host,
)

Updating Charmhelpers Packages

By default, a new charm installs only the charmhelpers.core package, but other packages are available (for a complete list, see the API Documentation). The installed packages are controlled by the charm-helpers.yaml file in our charm:

$ cd mycharm
$ cat charm-helpers.yaml
destination: lib/charmhelpers
branch: lp:charm-helpers
include:
  - core

Let’s update this file to include some more packages:

$ vim charm-helpers.yaml
$ cat charm-helpers.yaml
destination: lib/charmhelpers
branch: lp:charm-helpers
include:
  - core
  - contrib.storage
  - fetch

Now we need to download the new packages into our charm:

$ ./scripts/charm_helpers_sync.py -c charm-helpers.yaml
INFO:root:Loading charm helper config from charm-helpers.yaml.
INFO:root:Checking out lp:charm-helpers to /tmp/tmpT38Y87/charm-helpers.
Branched 160 revisions.
INFO:root:Syncing directory: /tmp/tmpT38Y87/charm-helpers/charmhelpers/core -> lib/charmhelpers/core.
INFO:root:Syncing directory: /tmp/tmpT38Y87/charm-helpers/charmhelpers/contrib/storage -> lib/charmhelpers/contrib/storage.
INFO:root:Adding missing __init__.py: lib/charmhelpers/contrib/__init__.py
INFO:root:Syncing directory: /tmp/tmpT38Y87/charm-helpers/charmhelpers/fetch -> lib/charmhelpers/fetch.

A look at our charmhelpers directory reveals that the new packages have indeed been added. We are now free to import and use them in our charm:

$ tree lib/charmhelpers/
lib/charmhelpers/
├── contrib
│   ├── __init__.py
│   └── storage
│       ├── __init__.py
│       └── linux
│           ├── ceph.py
│           ├── __init__.py
│           ├── loopback.py
│           ├── lvm.py
│           └── utils.py
├── core
│   ├── fstab.py
│   ├── hookenv.py
│   ├── host.py
│   └── __init__.py
├── fetch
│   ├── archiveurl.py
│   ├── bzrurl.py
│   └── __init__.py
└── __init__.py

5 directories, 15 files

Next Steps

Now that you have access to charmhelpers in your charm, check out the Examples or API Documentation to learn about all the great functionality that charmhelpers provides.

Examples

If you’d like to contribute an example (please do!), please refer to the Contributing page for instructions on how to do so.

Interacting with Charm Configuration

The charmhelpers.core.hookenv.config(), when called with no arguments, returns a charmhelpers.core.hookenv.Config instance - a dictionary representation of a charm’s config.yaml file. This object can be used to:

  • get a charm’s current config values
  • check if a config value has changed since the last hook invocation
  • view the previous value of a changed config item
  • save arbitrary key/value data for use in a later hook

For the following examples we’ll assume our charm has a config.yaml file that looks like this:

options:
  app-name:
    type: string
    default: "My App"
    description: "Name of your app."

Getting charm config values

# hooks/hooks.py

from charmhelpers.core import hookenv

hooks = hookenv.Hooks()

@hooks.hook('install')
def install():
    config = hookenv.config()

    assert config['app-name'] == 'My App'

Checking if a config value has changed

Let’s say the user changes the app-name config value at runtime by executing the following juju command:

juju set mycharm app-name="My New App"

which triggers a config-changed hook:

# hooks/hooks.py

from charmhelpers.core import hookenv

hooks = hookenv.Hooks()

@hooks.hook('config-changed')
def config_changed():
    config = hookenv.config()

    assert config.changed('app-name')
    assert config['app-name'] == 'My New App'
    assert config.previous('app-name') == 'My App'

Saving arbitrary key/value data

The Config object maybe also be used to store arbitrary data that you want to persist across hook invocations:

# hooks/hooks.py

from charmhelpers.core import hookenv

hooks = hookenv.Hooks()

@hooks.hook('install')
def install():
    config = hookenv.config()

    config['mykey'] = 'myval'

@hooks.hook('config-changed')
def config_changed():
    config = hookenv.config()

    assert config['mykey'] == 'myval'

Managing Charms with the Charm Framework

Traditional charm authoring is focused on implementing hooks. That is, the charm author is thinking in terms of “What hook am I handling; what does this hook need to do?” However, in most cases, the real question should be “Do I have the information I need to configure and start this piece of software and, if so, what are the steps for doing so?” The charm framework tries to bring the focus to the data and the setup tasks, in the most declarative way possible.

Stop Thinking about Hooks

Consider a charm that has a config option specifying the version of software to install, and which requires both a database and a messaging queue server (such as RabbitMQ) for the software to function. In which hook do you install the software?

If your config option has a default value, you could possibly use the install hook, but then you’d still be duplicating logic in config-changed. And even if you delay install until config-changed, you can’t start the software in start, since you have to wait for the database and messaging servers. And you can’t reliably determine which of those hooks to put your logic in, either, since they could be connected in any order.

Thus, a good design pattern is to have all the logic in a common location, which checks all of the requirements and performs the appropriate actions if they are satisfied. And every hook ends up reduced to:

#!/bin/env python
import common
common.manage()

The Charm Framework then gives you a declarative way to describe what your requirements and actions to take are, as well as what data you provide to other services. It also provides helpers around relations and config to make implementing that logic as easy and self-documenting as possible.

Overview

Charms using the Framework declare one or more components, providing the information mentioned above: what is provided by the component, what is required by the component, and what callbacks should be run to setup (start) and cleanup (stop) the service. (The definition format is fully documented in the Manager class reference.)

An example implementation might be:

from charmhelpers.core import hookenv
from charmhelpers.core.charmframework import Manager
from charmhelpers.core.charmframework import helpers

class HttpRelation(helpers.Relation):
    relation_name = 'website'

    def provide_data(self, remote_service, ready):
        return {
            'hostname': hookenv.unit_get('private-address'),
            'port': 8080,
        }

components = [
    {
        'name': 'WordPress',
        'provides': [HttpRelation],
        'requires': [
            helpers.config_not_default('password'),
            helpers.Relation(
                relation_name = 'db',
                required_keys = ['host', 'user', 'password', 'database'],
            ),
        ],
        'callbacks': [
            actions.install_frontend,
            helpers.template(source='wp-config.php.j2',
                             target=os.path.join(WP_INSTALL_DIR, 'wp-config.php'))
            helpers.template(source='wordpress.upstart.j2',
                             target='/etc/init/wordpress'),
            helpers.service_restart('wordpress'),
            helpers.open_ports(8080),
        ],
        'cleanup': [
            helpers.close_ports(8080),
            helpers.service_stop('wordpress'),
        ],
    },
]

def manage():
    Manager(components).manage()

Each time a hook is fired, the conditions will be checked (in this case, that the password has been set, and that the db relation is connected and has all of the required keys) and, if met, the appropriate actions taken (correct front-end installed, config files written / updated, and the Upstart job (re)started).

Relation Data Providers

If your charm provides any relations for other charms to use, this is where you will define the data that is sent to them. You will almost certainly want to use the charmhelpers.core.charmframework.helpers.Relation base class, and then define a provide() method to generate or collect the data to be sent.

A more complicated example, which relies on the name of the remote service might be:

class DatabaseRelation(Relation):
    relation_name = 'db'

    def provide(self, remote_service, all_ready):
        if not all_ready:
            return None  # not yet ready to provide our service
        if database_exists(remote_service):
            password = get_user_password(remote_service)
        else:
            password = create_user(remote_service)
            create_database(remote_service)
        return {
            'host': hookenv.unit_get('private-address'),
            'user': remote_service,
            'password': password,
            'database': remote_service,
        }

Alternatively, an example that involves bi-directional data exchange might be:

class NamedDatabaseRelation(Relation):
    relation_name = 'named-db'
    required_keys = ['database']

    def provide(self, remote_service, all_ready):
        if not all_ready:
            return None  # not yet ready to provide our service
        data = self.filtered_data(remote_service).values()[0]
        database = data['database']
        if not database_exists(database):
            create_database(database)
        password = get_or_create_user(remote_service)
        return {
            'host': hookenv.unit_get('private-address'),
            'user': remote_service,
            'password': password,
            'database': database,
        }

Requirement Predicates

Chances are, your charm depends on some external source of data before it can properly configure itself. The most common dependencies are on user config options, and relations to other charms.

Requirement predicates are responsible for checking those dependencies and verifying that they are satisfied so that the charm can proceed with its setup. Again, you can use the Relation class, or any of the other predicate helpers defined in charmhelpers.core.charmframework.helpers to make these predicates easy.

(Note that Relation subclasses will generally cover both sides of the relation, provides and requires, so that all of the logic for a relation interface protocol is encapsulated in a single class. These classes can then be re-used among multiple Charms, and participating in a relation becomes as easy as using a pre-defined class.)

Ready Callbacks

These are just simple callbacks that will be triggered once all of the requires dependencies are met. It is highly recommended that you split these into logical, easily unit-testable chunks.

The charmhelpers.core.charmframework.helpers module has helpers for common actions, such as opening ports, rendering templates, and restarting services.

The callbacks can use any of the standard charm-helpers methods for performing their work, but the recommended way to access relation data for relations in the requires dependencies is via any_ready_unit() or all_ready_units() helpers, which use the data stored in charmhelpers.core.unitdata, since some relations may have units connected which do not yet satisfy the requirements to participate with the charm.

Cleanup Callbacks

These callbacks are triggered if the charm is being stopped, or if any of the requires dependencies change such that the requirements can no longer be met.

The charmhelpers.core.charmframework.helpers module has helpers for common actions, such as closing ports, and stopping services.

Conclusion

By using this framework, it is easy to see what the preconditions for the charm are, and there is never a concern about things being in a partially configured state. As a charm author, you can focus on what is important to you: what data is mandatory, what is optional, and what actions should be taken once the requirements are met.

Contributing

All contributions, both code and documentation, are welcome!

Source

The source code is located at https://code.launchpad.net/charm-helpers. To submit contributions you’ll need to create a Launchpad account if you do not already have one.

To get the code:

$ bzr branch lp:charm-helpers

To build and run tests:

$ cd charm-helpers
$ make

Submitting a Merge Proposal

Run make test and ensure all tests pass. Then commit your changes and push them to a personal branch:

bzr ci -m "Description of your changes"
bzr push lp:~<launchpad-username>/charm-helpers/my-feature

Note that the branch name (‘my-feature’ in the above example) can be anything you choose - preferably something descriptive.

Once your branch is pushed, open it in a web browser, e.g.:

https://code.launchpad.net/~<launchpad-username>/charm-helpers/my-feature

Find and click on the ‘Propose for merging’ link, and on the following screen, click the ‘Propose Merge’ button.

Note

Do not set a value in the ‘Reviewer’ field - it will be set automatically.

Open Bugs

If you’re looking for something to work on, the open bug/feature list can be found at https://bugs.launchpad.net/charm-helpers.

Documentation

If you’d like to contribute to the documentation, please refer to the HACKING document in the root of the source tree for instructions on building the documentation.

Contributions to the Examples section of the documentation are especially welcome, and are easy to add. Simply add a new .rst file under charmhelpers/docs/examples.

Getting Help

If you need help you can find it in #juju on the Freenode IRC network. Come talk to us - we’re a friendly bunch!

API Documentation

charmhelpers.core package

charmhelpers.core.charmframework

Manager Manage all hooks, based on a set of charm component definitions.
helpers.HttpRelation Relation subclass for handling relations using the http interface protocol.
helpers.MySQLRelation Relation subclass for handling relations using the mysql interface protocol.
helpers.Relation Base class for relation handler classes.
helpers.all_ready_units Returns the unit name and data for all ready untis on the given relation, sorted by unit name.
helpers.any_ready_unit Returns the unit name and data for any single ready unit on the given relation.
helpers.close_ports Callback helper that will close the given ports when called.
helpers.config_eq Predicate helper that asserts that the given config option is equal to the given value.
helpers.config_is_set Predicate helper that asserts that the given config option is set to a non-empty / non-None value.
helpers.config_ne Predicate helper that asserts that the given config option is not equal to the given value.
helpers.config_not_default Predicate helper that asserts that the given config option is not equal to its default value.
helpers.open_ports Callback helper that will open the given ports when called.
helpers.render_template Callback helper that will render a Jinja2 template.
helpers.service_reload Callback helper that will reload the given service.
helpers.service_restart Callback helper that will restart the given service.
helpers.service_stop Callback helper that will stop the given service.
class charmhelpers.core.charmframework.base.Manager(components)

Bases: object

Manage all hooks, based on a set of charm component definitions.

Component definitions are dicts in the following format (all fields are optional):

{
    "name": <name of component>,
    "provides": <list of providers>,
    "requires": <list of predicates>,
    "callbacks": <list of callbacks>,
    "cleanup": <list of callbacks>,
}
Variables:
  • name (str) – Descriptive name
  • provides (list) – Data providers, most likely subclasses of Relation, although any object (or class) which has a relation_name attribute and a provide method which accepts the remote service name and a flag indicating whether the callbacks have been fired, and which returns a dict of data to send to that remote service.
  • requires (list) – Predicates, all of which must pass for the callbacks to be fired. Relation subclasses or instances can be used as predicates in addition to simple no-argument functions which return a bool. Any Relation subclass or instance will have its store_data() method called. See helpers for several helpers that can be used to construct predicates.
  • callbacks (list) – Callbacks that are fired if all requires predicates pass. They will be passed no arguments. See helpers for several helpers that can be used to construct callbacks.
  • cleanup (list) – Callbacks that are fired when stopping the service.

Examples:

The following registers a component which depends on a mongodb relation and a password option being changed from its default value, and which runs a custom db_migrate function, adds a system user, renders a couple of templates, starts the system service ‘myservice’, and finally opens a couple of ports:

from charmhelpers.core import charmframework
from charmhelpers.core import host
from functools import partial

class MongoRelation(charmframework.helpers.Relation):
    relation_name = 'mongo'
    required_keys = ['hostname', 'port']

manager = charmframework.Manager([
    {
        'provides': [
            HttpRelation,
        ],
        'requires': [
            MongoRelation,
            charmframwork.helpers.config_not_default('password'),
        ],
        'callbacks': [
            db_migrate,
            partial(host.adduser, 'myuser'),
            charmframework.helpers.render_template(source='myservice.conf'),
            charmframework.helpers.render_template(source='myservice.ini',
                                                   target='/etc/myservice.ini',
                                                   owner='myuser', perms=0400),
            charmframework.helpers.service_restart('myservice'),
            charmframework.helpers.open_ports(80, 443),
        ],
        'cleanup': [
            charmframework.helpers.close_ports(80, 443),
            charmframework.helpers.service_stop('myservice'),
        ],
    },
])
manager.manage()
fire_event(event_name, component)

Fire an actions, or cleanup event on a given component.

is_ready(component)

Determine if a registered component is ready, by checking its requires items.

manage()

Handle the current hook by doing The Right Thing with the registered components.

not_ready(*args, **kwargs)
provide_data()

Set the relation data for each provider in the provides list.

A provider can be a class or an instance, and must have a relation_name attribute, which indicates which relation is applies to, and a provide(remote_service, all_ready) method, where remote_service is the name of the Juju service on the other end of the relation, and all_ready is a boolean indicating whether all of the requirements for the component were satisfied and the callbacks were run.

Note that the providers are called after the requirements are checked and the callbacks (possibly) run, since the callbacks may generate some of the data to be provided.

run_callbacks()

Check all components’ requires predicates and run either callbacks or cleanup actions.

run_cleanup()

Unconditionally run all components’ cleanup actions.

save_lost(component)

Save an indicator that the given component is no longer ready.

save_ready(component)

Save an indicator that the given component is now ready.

store_data(component)
was_ready(component)

Determine if the given component was previously ready.

charmhelpers.core.charmframework.helpers
class charmhelpers.core.charmframework.helpers.HttpRelation(**kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

Relation subclass for handling relations using the http interface protocol.

As mentioned in Relation, all variables (including port) can be overridden when instantiating the class.

port = 80

Port upon which this service is listening.

provide(remote_service, all_ready)
relation_name = 'website'
required_keys = ['host', 'port']
class charmhelpers.core.charmframework.helpers.MySQLRelation(**kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

Relation subclass for handling relations using the mysql interface protocol.

As mentioned in Relation, all variables (including dsn_format) can be overridden when instantiating the class.

dsn_format = 'mysql://{user}:{password}@{host}/{database}'

In addition to the required_keys, a dsn value is constructed, based on this format string, and added to the filtered_data().

filtered_data(remote_service=None)

Get the filtered data, as per Relation.filtered_data(), and add a dsn entry constructed from the other values.

relation_name = 'db'
required_keys = ['host', 'user', 'password', 'database']
class charmhelpers.core.charmframework.helpers.Relation(**kwargs)

Bases: object

Base class for relation handler classes.

Note: The default constructor allows any pre-defined class attribute to be overridden via kwargs during instantiation. Thus, the above variables can be set or overridden when constructing the relation or a subclass. For example, the following will override both the optional flag and the port:

class MyRelation(Relation):
    relation_name = 'foo'
    port = 80

rel = MyRelation(optional=True, port=8080)
connected_units()

Returns a list of the names of all units connected on this relation.

filtered_data(remote_service=None)

Get the data from unfiltered_data() and filter it to only include units which have set all of the required_keys.

Parameters:remote_service (str) – If given, additionally filter to only include data for units of a single service (useful in provide() methods).
Returns:Mapping (dict) of unit name to that unit’s data dict.
is_ready()

Determine if this relation is “ready”.

A relation is ready if it is not optional and there is at least one unit that satisfies all of the required_keys. (Optional relations are always considered ready.)

Note: A Relation instance can also be evaluated directly (e.g., via bool()) to test its readiness.

optional = False

Whether the relation is optional. An optional relation always passes when used as a predicate, even if no units are connected. However, it is still useful to include them in the requires section, because it is both informative when reading the definition and allows the framework to automatically call store_data().

provide(remote_service, all_ready)

Provide data to the other side of the relation.

This should be implemented by subclasses.

Parameters:
  • remote_service (str) – Name of the remote service to which this relation will be providing data
  • all_ready (bool) – Whether all requires predicates for this Charm component have been met and the callbacks run
relation_name = None

Name (not interface) of the relation to manage.

required_keys = []

List of keys that must be set on the relation for it to be considered ready (i.e., pass when used as a requires predicate). If no keys are defined, then a relation is considered ready if a unit is attached.

store_data()

Store all filtered relation data is stored in unitdata under relations.ready.

The data can be retrieved using:

from charmhelpers.core import unitdata
data = unitdata.kv.get('relations.ready')[relation_name][unit_name]

However, it is generally recommended to use one of any_ready_unit() or all_ready_units().

This method is called automatically for all Relation subclasses as predicates in the required section of a definition.

unfiltered_data()

Get all relation data for any unit connected to this relation.

Returns:Mapping (dict) of unit name to that unit’s data dict.
charmhelpers.core.charmframework.helpers.all_ready_units(relation_name)

Returns the unit name and data for all ready untis on the given relation, sorted by unit name.

This requires that Relation.store_data() has been called for the given relation, which is done automatically by charmhelpers.core.charmframework.base.Manager.

Parameters:relation_name (str) – Name of the relation
Return list:List of tuples containing (unit_name, unit_data), or an empty list if no units are ready for the given relation
charmhelpers.core.charmframework.helpers.any_ready_unit(relation_name)

Returns the unit name and data for any single ready unit on the given relation.

If more than one unit is attached and ready, this is not guaranteed to return the same unit each time it is called. It is recommended that you use all_ready_units() if you ever expect more than one service or unit to be attached.

This requires that Relation.store_data() has been called for the given relation, which is done automatically by Manager.

Parameters:relation_name (str) – Name of the relation
Returns:Tuple containing (unit_name, unit_data), or (None, None) if no units are ready for the given relation
charmhelpers.core.charmframework.helpers.close_ports(*ports)

Callback helper that will close the given ports when called.

Roughly equivalent to:

lambda: map(hookenv.open_port, ports)

However, ports can be given as individual ports (close_ports(80, 443)), or a list of ports (close_ports([80, 443])).

charmhelpers.core.charmframework.helpers.config_eq(option, value)

Predicate helper that asserts that the given config option is equal to the given value.

Values of None are normalized to empty strings, since it is impossible to properly “unset” a config value once it has been set; it can only be set to an empty string.

For example:

Manager([
    {
        'requires': [config_eq('unstable', False)],
        'callbacks': [install_stable],
    },
    {
        'requires': [config_eq('unstable', True)],
        'callbacks': [install_unstable],
    },
])

In this case, only one of install_stable or install_unstable will ever be called, depending on the value of the ‘unstable’ config option.

charmhelpers.core.charmframework.helpers.config_is_set(option)

Predicate helper that asserts that the given config option is set to a non-empty / non-None value.

This is equivalent to config_ne(option, ''), but can be a bit more clear.

charmhelpers.core.charmframework.helpers.config_ne(option, value)

Predicate helper that asserts that the given config option is not equal to the given value.

Values of None are normalized to empty strings, since it is impossible to properly “unset” a config value once it has been set; it can only be set to an empty string.

charmhelpers.core.charmframework.helpers.config_not_default(option)

Predicate helper that asserts that the given config option is not equal to its default value.

Values of None are normalized to empty strings, since it is impossible to properly “unset” a config value once it has been set; it can only be set to an empty string.

charmhelpers.core.charmframework.helpers.open_ports(*ports)

Callback helper that will open the given ports when called.

Roughly equivalent to:

lambda: map(hookenv.open_port, ports)

However, ports can be given as individual ports (open_ports(80, 443)), or a list of ports (open_ports([80, 443])).

charmhelpers.core.charmframework.helpers.render_template(source, target, context=None, owner='root', group='root', perms=292)

Callback helper that will render a Jinja2 template.

The template is provided a context which contains these items by default:

Parameters:
  • source (str) – The template source file, relative to $CHARM_DIR/templates
  • target (str) – The target to write the rendered template to
  • context (dict) – Additional context to be provided to the template
  • owner (str) – The owner of the rendered file (default: root)
  • group (str) – The group of the rendered file (default: root)
  • perms (int) – The permissions of the rendered file (default 0o444)
charmhelpers.core.charmframework.helpers.service_reload(service_name)

Callback helper that will reload the given service.

Equivalent to:

functools.partial(hookenv.service_reload, service_name)
charmhelpers.core.charmframework.helpers.service_restart(service_name)

Callback helper that will restart the given service.

Equivalent to:

functools.partial(hookenv.service_restart, service_name)
charmhelpers.core.charmframework.helpers.service_stop(service_name)

Callback helper that will stop the given service.

Equivalent to:

functools.partial(hookenv.service_stop, service_name)

charmhelpers.core.decorators

charmhelpers.core.decorators.retry_on_exception(num_retries, base_delay=0, exc_type=<type 'exceptions.Exception'>)

If the decorated function raises exception exc_type, allow num_retries retry attempts before raise the exception.

charmhelpers.core.fstab

class charmhelpers.core.fstab.Fstab(path=None)

Bases: _io.FileIO

This class extends file in order to implement a file reader/writer for file /etc/fstab

DEFAULT_PATH = '/etc/fstab'
class Entry(device, mountpoint, filesystem, options, d=0, p=0)

Bases: object

Entry class represents a non-comment line on the /etc/fstab file

classmethod Fstab.add(device, mountpoint, filesystem, options=None, path=None)
Fstab.add_entry(entry)
Fstab.entries
Fstab.get_entry_by_attr(attr, value)
classmethod Fstab.remove_by_mountpoint(mountpoint, path=None)
Fstab.remove_entry(entry)

charmhelpers.core.hookenv

Config A dictionary representation of the charm’s config.yaml, with some
Hooks A convenient handler for hook functions.
Serializable Wrapper, an object that can be serialized to yaml or json
UnregisteredHookError Raised when an undefined hook is called
action_fail Sets the action status to failed and sets the error message.
action_get
action_set Sets the values to be returned after the action finishes
cached Cache return values for multiple executions of func + args
charm_dir Return the root directory of the current charm
charm_name
close_port Close a service network port
config
execution_environment A convenient bundling of the current execution context
flush Flushes any entries from function cache where the
hook_name The name of the currently executing hook
in_relation_hook Determine whether we’re running in a relation hook
is_relation_made
local_unit Local unit ID
log Write a message to the juju log
metadata
open_port Open a service network port
related_units
relation_for_unit
relation_get
relation_id The relation ID for the current relation hook
relation_ids
relation_set Set relation information for the current unit
relation_type The scope for the current relation hook
relation_types
relations
relations_for_id
relations_of_type
remote_unit The remote unit for the current relation hook
service_name The name service group this unit belongs to
unit_get
unit_private_ip Get this unit’s private IP address

Interactions with the Juju environment

class charmhelpers.core.hookenv.Config(*args, **kw)

Bases: dict

A dictionary representation of the charm’s config.yaml, with some extra features:

  • See which values in the dictionary have changed since the previous hook.
  • For values that have changed, see what the previous value was.
  • Store arbitrary data for use in a later hook.

NOTE: Do not instantiate this object directly - instead call hookenv.config(), which will return an instance of Config.

Example usage:

>>> # inside a hook
>>> from charmhelpers.core import hookenv
>>> config = hookenv.config()
>>> config['foo']
'bar'
>>> # store a new key/value for later use
>>> config['mykey'] = 'myval'


>>> # user runs `juju set mycharm foo=baz`
>>> # now we're inside subsequent config-changed hook
>>> config = hookenv.config()
>>> config['foo']
'baz'
>>> # test to see if this val has changed since last hook
>>> config.changed('foo')
True
>>> # what was the previous value?
>>> config.previous('foo')
'bar'
>>> # keys/values that we add are preserved across hooks
>>> config['mykey']
'myval'
CONFIG_FILE_NAME = '.juju-persistent-config'
changed(key)

Return True if the current value for this key is different from the previous value.

keys()
load_previous(path=None)

Load previous copy of config from disk.

In normal usage you don’t need to call this method directly - it is called automatically at object initialization.

Parameters:path – File path from which to load the previous config. If None, config is loaded from the default location. If path is specified, subsequent save() calls will write to the same path.
previous(key)

Return previous value for this key, or None if there is no previous value.

save()

Save this config to disk.

If the charm is using the Services Framework or :meth:’@hook <Hooks.hook>’ decorator, this is called automatically at the end of successful hook execution. Otherwise, it should be called directly by user code.

To disable automatic saves, set implicit_save=False on this instance.

class charmhelpers.core.hookenv.Hooks(config_save=True)

Bases: object

A convenient handler for hook functions.

Example:

hooks = Hooks()

# register a hook, taking its name from the function name
@hooks.hook()
def install():
    pass  # your code here

# register a hook, providing a custom hook name
@hooks.hook("config-changed")
def config_changed():
    pass  # your code here

if __name__ == "__main__":
    # execute a hook based on the name the program is called by
    hooks.execute(sys.argv)
execute(args)

Execute a registered hook based on args[0]

hook(*hook_names)

Decorator, registering them as hooks

register(name, function)

Register a hook

class charmhelpers.core.hookenv.Serializable(obj)

Bases: UserDict.UserDict

Wrapper, an object that can be serialized to yaml or json

json()

Serialize the object to json

yaml()

Serialize the object to yaml

exception charmhelpers.core.hookenv.UnregisteredHookError

Bases: exceptions.Exception

Raised when an undefined hook is called

charmhelpers.core.hookenv.action_fail(message)

Sets the action status to failed and sets the error message.

The results set by action_set are preserved.

charmhelpers.core.hookenv.action_get(*args, **kwargs)
charmhelpers.core.hookenv.action_set(values)

Sets the values to be returned after the action finishes

charmhelpers.core.hookenv.cached(func)

Cache return values for multiple executions of func + args

For example:

@cached
def unit_get(attribute):
    pass

unit_get('test')

will cache the result of unit_get + ‘test’ for future calls.

charmhelpers.core.hookenv.charm_dir()

Return the root directory of the current charm

charmhelpers.core.hookenv.charm_name(*args, **kwargs)
charmhelpers.core.hookenv.close_port(port, protocol='TCP')

Close a service network port

charmhelpers.core.hookenv.config(*args, **kwargs)
charmhelpers.core.hookenv.execution_environment()

A convenient bundling of the current execution context

charmhelpers.core.hookenv.flush(key)

Flushes any entries from function cache where the key is found in the function+args

charmhelpers.core.hookenv.hook_name()

The name of the currently executing hook

charmhelpers.core.hookenv.in_relation_hook()

Determine whether we’re running in a relation hook

charmhelpers.core.hookenv.is_relation_made(*args, **kwargs)
charmhelpers.core.hookenv.local_unit()

Local unit ID

charmhelpers.core.hookenv.log(message, level=None)

Write a message to the juju log

charmhelpers.core.hookenv.metadata(*args, **kwargs)
charmhelpers.core.hookenv.open_port(port, protocol='TCP')

Open a service network port

charmhelpers.core.hookenv.related_units(*args, **kwargs)
charmhelpers.core.hookenv.relation_for_unit(*args, **kwargs)
charmhelpers.core.hookenv.relation_get(*args, **kwargs)
charmhelpers.core.hookenv.relation_id()

The relation ID for the current relation hook

charmhelpers.core.hookenv.relation_ids(*args, **kwargs)
charmhelpers.core.hookenv.relation_set(relation_id=None, relation_settings=None, **kwargs)

Set relation information for the current unit

charmhelpers.core.hookenv.relation_type()

The scope for the current relation hook

charmhelpers.core.hookenv.relation_types(*args, **kwargs)
charmhelpers.core.hookenv.relations(*args, **kwargs)
charmhelpers.core.hookenv.relations_for_id(*args, **kwargs)
charmhelpers.core.hookenv.relations_of_type(*args, **kwargs)
charmhelpers.core.hookenv.remote_unit()

The remote unit for the current relation hook

charmhelpers.core.hookenv.service_name()

The name service group this unit belongs to

charmhelpers.core.hookenv.unit_get(*args, **kwargs)
charmhelpers.core.hookenv.unit_private_ip()

Get this unit’s private IP address

charmhelpers.core.host

ChecksumError
add_group Add a group to the system
add_user_to_group Add a user to a group
adduser Add a user to the system
chdir
check_hash Validate a file using a cryptographic checksum.
chownr
cmp_pkgrevno Compare supplied revno with the revno of the installed package
cpu_arch
file_hash Generate a hash checksum of the contents of ‘path’ or None if not found.
fstab_add Adds the given device entry to the /etc/fstab file
fstab_remove Remove the given mountpoint entry from /etc/fstab
get_nic_hwaddr
get_nic_mtu
lchownr
list_nics Return a list of nics of given type(s)
lsb_release Return /etc/lsb-release in a dict
mkdir Create a directory
mount Mount a filesystem at a particular mountpoint
mounts Get a list of all mounted volumes as [[mountpoint,device],[...]]
pwgen Generate a random pasword.
restart_on_change Restart services based on configuration files changing
rsync Replicate the contents of a path
service Control a system service
service_available Determine whether a system service is available
service_reload Reload a system service, optionally falling back to restart if
service_restart Restart a system service
service_running Determine whether a system service is running
service_start Start a system service
service_stop Stop a system service
set_nic_mtu Set MTU on a network interface
symlink Create a symbolic link
umount Unmount a filesystem
write_file Create or overwrite a file with the contents of a byte string.

Tools for working with the host system

exception charmhelpers.core.host.ChecksumError

Bases: exceptions.ValueError

charmhelpers.core.host.add_group(group_name, system_group=False)

Add a group to the system

charmhelpers.core.host.add_user_to_group(username, group)

Add a user to a group

charmhelpers.core.host.adduser(username, password=None, shell='/bin/bash', system_user=False, group=None, groups=None)

Add a user to the system

charmhelpers.core.host.chdir(*args, **kwds)
charmhelpers.core.host.check_hash(path, checksum, hash_type='md5')

Validate a file using a cryptographic checksum.

Parameters:
  • checksum (str) – Value of the checksum used to validate the file.
  • hash_type (str) – Hash algorithm used to generate checksum. Can be any hash alrgorithm supported by hashlib, such as md5, sha1, sha256, sha512, etc.
Raises ChecksumError:
 

If the file fails the checksum
charmhelpers.core.host.chownr(path, owner, group, follow_links=True)
charmhelpers.core.host.cmp_pkgrevno(package, revno, pkgcache=None)

Compare supplied revno with the revno of the installed package

  • 1 => Installed revno is greater than supplied arg
  • 0 => Installed revno is the same as supplied arg
  • -1 => Installed revno is less than supplied arg

This function imports apt_cache function from charmhelpers.fetch if the pkgcache argument is None. Be sure to add charmhelpers.fetch if you call this function, or pass an apt_pkg.Cache() instance.

charmhelpers.core.host.cpu_arch()
charmhelpers.core.host.file_hash(path, hash_type='md5')

Generate a hash checksum of the contents of ‘path’ or None if not found.

Parameters:hash_type (str) – Any hash alrgorithm supported by hashlib, such as md5, sha1, sha256, sha512, etc.
charmhelpers.core.host.fstab_add(dev, mp, fs, options=None)

Adds the given device entry to the /etc/fstab file

charmhelpers.core.host.fstab_remove(mp)

Remove the given mountpoint entry from /etc/fstab

charmhelpers.core.host.get_nic_hwaddr(nic)
charmhelpers.core.host.get_nic_mtu(nic)
charmhelpers.core.host.lchownr(path, owner, group)
charmhelpers.core.host.list_nics(nic_type)

Return a list of nics of given type(s)

charmhelpers.core.host.lsb_release()

Return /etc/lsb-release in a dict

charmhelpers.core.host.mkdir(path, owner='root', group='root', perms=365, force=False)

Create a directory

charmhelpers.core.host.mount(device, mountpoint, options=None, persist=False, filesystem='ext3')

Mount a filesystem at a particular mountpoint

charmhelpers.core.host.mounts()

Get a list of all mounted volumes as [[mountpoint,device],[...]]

charmhelpers.core.host.pwgen(length=None)

Generate a random pasword.

charmhelpers.core.host.restart_on_change(restart_map, stopstart=False)

Restart services based on configuration files changing

This function is used a decorator, for example:

@restart_on_change({
    '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
    })
def ceph_client_changed():
    pass  # your code here

In this example, the cinder-api and cinder-volume services would be restarted if /etc/ceph/ceph.conf is changed by the ceph_client_changed function.

charmhelpers.core.host.rsync(from_path, to_path, flags='-r', options=None)

Replicate the contents of a path

charmhelpers.core.host.service(action, service_name)

Control a system service

charmhelpers.core.host.service_available(service_name)

Determine whether a system service is available

charmhelpers.core.host.service_reload(service_name, restart_on_failure=False)

Reload a system service, optionally falling back to restart if reload fails

charmhelpers.core.host.service_restart(service_name)

Restart a system service

charmhelpers.core.host.service_running(service)

Determine whether a system service is running

charmhelpers.core.host.service_start(service_name)

Start a system service

charmhelpers.core.host.service_stop(service_name)

Stop a system service

charmhelpers.core.host.set_nic_mtu(nic, mtu)

Set MTU on a network interface

Create a symbolic link

charmhelpers.core.host.umount(mountpoint, persist=False)

Unmount a filesystem

charmhelpers.core.host.write_file(path, content, owner='root', group='root', perms=292)

Create or overwrite a file with the contents of a byte string.

charmhelpers.core.strutils

charmhelpers.core.strutils.bool_from_string(value)

Interpret string value as boolean.

Returns True if value translates to True otherwise False.

charmhelpers.core.sysctl

charmhelpers.core.sysctl.create(sysctl_dict, sysctl_file)

Creates a sysctl.conf file from a YAML associative array

Parameters:
  • sysctl_dict (str) – a YAML-formatted string of sysctl options eg “{ ‘kernel.max_pid’: 1337 }”
  • sysctl_file (str or unicode) – path to the sysctl file to be saved
Returns:

None

charmhelpers.core.templating

charmhelpers.core.templating.render(source, target, context, owner='root', group='root', perms=292, templates_dir=None, encoding='UTF-8')

Render a Jinja2 template.

Parameters:
  • source (str) – Path to template, relative to templates_dir.
  • target (str) – Path (generally absolute) to write template to. If None, no file will be written (in case only the returned rendered template string is desired).
  • context (dict) – Variables available to the template.
  • owner (str) – Passed through to write_file().
  • group (str) – Passed through to write_file().
  • perms (str) – Passed through to write_file().
  • templates_dir (str) – Base path for the templates. Defaults to $CHARM_DIR/templates.
  • encoding (str) – Encoding to use when writing the file.
Returns:

The rendered template.

Note: Using this requires python-jinja2; if it is not installed, calling this will attempt to use charmhelpers.fetch.apt_install() to install it.

charmhelpers.core.unitdata

Intro

A simple way to store state in units. This provides a key value storage with support for versioned, transactional operation, and can calculate deltas from previous values to simplify unit logic when processing changes.

Hook Integration

There are several extant frameworks for hook execution, including

  • charmhelpers.core.hookenv.Hooks
  • charmhelpers.core.services.ServiceManager

The storage classes are framework agnostic, one simple integration is via the HookData contextmanager. It will record the current hook execution environment (including relation data, config data, etc.), setup a transaction and allow easy access to the changes from previously seen values. One consequence of the integration is the reservation of particular keys (‘rels’, ‘unit’, ‘env’, ‘config’, ‘charm_revisions’) for their respective values.

Here’s a fully worked integration example using hookenv.Hooks:

from charmhelper.core import hookenv, unitdata

hook_data = unitdata.HookData()
db = unitdata.kv()
hooks = hookenv.Hooks()

@hooks.hook
def config_changed():
    # Print all changes to configuration from previously seen
    # values.
    for changed, (prev, cur) in hook_data.conf.items():
        print('config changed', changed,
              'previous value', prev,
              'current value',  cur)

    # Get some unit specific bookeeping
    if not db.get('pkg_key'):
        key = urllib.urlopen('https://example.com/pkg_key').read()
        db.set('pkg_key', key)

    # Directly access all charm config as a mapping.
    conf = db.getrange('config', True)

    # Directly access all relation data as a mapping
    rels = db.getrange('rels', True)

if __name__ == '__main__':
    with hook_data():
        hook.execute()

A more basic integration is via the hook_scope context manager which simply manages transaction scope (and records hook name, and timestamp):

>>> from unitdata import kv
>>> db = kv()
>>> with db.hook_scope('install'):
...    # do work, in transactional scope.
...    db.set('x', 1)
>>> db.get('x')
1
Usage

Values are automatically json de/serialized to preserve basic typing and complex data struct capabilities (dicts, lists, ints, booleans, etc).

Individual values can be manipulated via get/set:

>>> kv.set('y', True)
>>> kv.get('y')
True

# We can set complex values (dicts, lists) as a single key.
>>> kv.set('config', {'a': 1, 'b': True'})

# Also supports returning dictionaries as a record which
# provides attribute access.
>>> config = kv.get('config', record=True)
>>> config.b
True

Groups of keys can be manipulated with update/getrange:

>>> kv.update({'z': 1, 'y': 2}, prefix="gui.")
>>> kv.getrange('gui.', strip=True)
{'z': 1, 'y': 2}

When updating values, its very helpful to understand which values have actually changed and how have they changed. The storage provides a delta method to provide for this:

>>> data = {'debug': True, 'option': 2}
>>> delta = kv.delta(data, 'config.')
>>> delta.debug.previous
None
>>> delta.debug.current
True
>>> delta
{'debug': (None, True), 'option': (None, 2)}

Note the delta method does not persist the actual change, it needs to be explicitly saved via ‘update’ method:

>>> kv.update(data, 'config.')

Values modified in the context of a hook scope retain historical values associated to the hookname.

>>> with db.hook_scope('config-changed'):
...      db.set('x', 42)
>>> db.gethistory('x')
[(1, u'x', 1, u'install', u'2015-01-21T16:49:30.038372'),
 (2, u'x', 42, u'config-changed', u'2015-01-21T16:49:30.038786')]
API Reference
Delta Delta(previous, current)
DeltaSet
HookData Simple integration for existing hook exec frameworks.
Record
Storage Simple key value database for local unit state within charms.
kv
class charmhelpers.core.unitdata.Delta

Bases: tuple

Delta(previous, current)

current

Alias for field number 1

previous

Alias for field number 0

class charmhelpers.core.unitdata.DeltaSet

Bases: charmhelpers.core.unitdata.Record

class charmhelpers.core.unitdata.HookData

Bases: object

Simple integration for existing hook exec frameworks.

Records all unit information, and stores deltas for processing by the hook.

Sample:

from charmhelper.core import hookenv, unitdata

changes = unitdata.HookData()
db = unitdata.kv()
hooks = hookenv.Hooks()

@hooks.hook
def config_changed():
    # View all changes to configuration
    for changed, (prev, cur) in changes.conf.items():
        print('config changed', changed,
              'previous value', prev,
              'current value',  cur)

    # Get some unit specific bookeeping
    if not db.get('pkg_key'):
        key = urllib.urlopen('https://example.com/pkg_key').read()
        db.set('pkg_key', key)

if __name__ == '__main__':
    with changes():
        hook.execute()
class charmhelpers.core.unitdata.Record

Bases: dict

class charmhelpers.core.unitdata.Storage(path=None)

Bases: object

Simple key value database for local unit state within charms.

Modifications are automatically committed at hook exit. That’s currently regardless of exit code.

To support dicts, lists, integer, floats, and booleans values are automatically json encoded/decoded.

close()
debug(fh=<open file '<stderr>', mode 'w' at 0x7fc58b25d1e0>)
delta(mapping, prefix)

return a delta containing values that have changed.

flush(save=True)
get(key, default=None, record=False)
gethistory(key, deserialize=False)
getrange(key_prefix, strip=False)
hook_scope(*args, **kwds)

Scope all future interactions to the current hook execution revision.

set(key, value)
unset(key)
update(mapping, prefix='')
charmhelpers.core.unitdata.kv()

charmhelpers.contrib package

charmhelpers.contrib.ansible package

Charm Helpers ansible - declare the state of your machines.

This helper enables you to declare your machine state, rather than program it procedurally (and have to test each change to your procedures). Your install hook can be as simple as:

{{{
import charmhelpers.contrib.ansible


def install():
    charmhelpers.contrib.ansible.install_ansible_support()
    charmhelpers.contrib.ansible.apply_playbook('playbooks/install.yaml')
}}}

and won’t need to change (nor will its tests) when you change the machine state.

All of your juju config and relation-data are available as template variables within your playbooks and templates. An install playbook looks something like:

{{{
---
- hosts: localhost
  user: root

  tasks:
    - name: Add private repositories.
      template:
        src: ../templates/private-repositories.list.jinja2
        dest: /etc/apt/sources.list.d/private.list

    - name: Update the cache.
      apt: update_cache=yes

    - name: Install dependencies.
      apt: pkg={{ item }}
      with_items:
        - python-mimeparse
        - python-webob
        - sunburnt

    - name: Setup groups.
      group: name={{ item.name }} gid={{ item.gid }}
      with_items:
        - { name: 'deploy_user', gid: 1800 }
        - { name: 'service_user', gid: 1500 }

  ...
}}}

Read more online about playbooks and standard ansible modules.

A further feature os the ansible hooks is to provide a light weight “action” scripting tool. This is a decorator that you apply to a function, and that function can now receive cli args, and can pass extra args to the playbook.

e.g.

@hooks.action() def some_action(amount, force=”False”):

“Usage: some-action AMOUNT [force=True]” # <– shown on error # process the arguments # do some calls # return extra-vars to be passed to ansible-playbook return {

‘amount’: int(amount), ‘type’: force,

}

You can now create a symlink to hooks.py that can be invoked like a hook, but with cli params:

# link actions/some-action to hooks/hooks.py

actions/some-action amount=10 force=true

class charmhelpers.contrib.ansible.AnsibleHooks(playbook_path, default_hooks=None)

Bases: charmhelpers.core.hookenv.Hooks

Run a playbook with the hook-name as the tag.

This helper builds on the standard hookenv.Hooks helper, but additionally runs the playbook with the hook-name specified using –tags (ie. running all the tasks tagged with the hook-name).

Example:

hooks = AnsibleHooks(playbook_path='playbooks/my_machine_state.yaml')

# All the tasks within my_machine_state.yaml tagged with 'install'
# will be run automatically after do_custom_work()
@hooks.hook()
def install():
    do_custom_work()

# For most of your hooks, you won't need to do anything other
# than run the tagged tasks for the hook:
@hooks.hook('config-changed', 'start', 'stop')
def just_use_playbook():
    pass

# As a convenience, you can avoid the above noop function by specifying
# the hooks which are handled by ansible-only and they'll be registered
# for you:
# hooks = AnsibleHooks(
#     'playbooks/my_machine_state.yaml',
#     default_hooks=['config-changed', 'start', 'stop'])

if __name__ == "__main__":
    # execute a hook based on the name the program is called by
    hooks.execute(sys.argv)
action(*action_names)

Decorator, registering them as actions

execute(args)

Execute the hook followed by the playbook using the hook as tag.

register_action(name, function)

Register a hook

charmhelpers.contrib.ansible.apply_playbook(playbook, tags=None, extra_vars=None)
charmhelpers.contrib.ansible.install_ansible_support(from_ppa=True, ppa_location='ppa:rquillo/ansible')

Installs the ansible package.

By default it is installed from the PPA linked from the ansible website or from a ppa specified by a charm config..

If from_ppa is empty, you must ensure that the package is available from a configured repository.

charmhelpers.contrib.bigdata package

charmhelpers.contrib.bigdata.relations
charmhelpers.contrib.bigdata.relations.DataNode Relation which communicates DataNode info back to NameNodes.
charmhelpers.contrib.bigdata.relations.FlumeAgent
charmhelpers.contrib.bigdata.relations.HadoopClient
charmhelpers.contrib.bigdata.relations.Hive
charmhelpers.contrib.bigdata.relations.MySQL
charmhelpers.contrib.bigdata.relations.NameNode Relation which communicates the NameNode (HDFS) connection & status info.
charmhelpers.contrib.bigdata.relations.NameNodeMaster Alternate NameNode relation for DataNodes.
charmhelpers.contrib.bigdata.relations.NodeManager Relation which communicates NodeManager info back to ResourceManagers.
charmhelpers.contrib.bigdata.relations.ResourceManager Relation which communicates the ResourceManager (YARN) connection & status info.
charmhelpers.contrib.bigdata.relations.ResourceManagerMaster Alternate ResourceManager relation for NodeManagers.
charmhelpers.contrib.bigdata.relations.SpecMatchingRelation Relation base class that validates that a version and environment between two related charms match, to prevent interoperability issues.
class charmhelpers.contrib.bigdata.relations.DataNode(**kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

Relation which communicates DataNode info back to NameNodes.

relation_name = 'datanode'
required_keys = ['private-address']
class charmhelpers.contrib.bigdata.relations.FlumeAgent(**kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

provide(remote_service, all_ready)
relation_name = 'flume-agent'
required_keys = ['private-address', 'port']
class charmhelpers.contrib.bigdata.relations.HadoopClient(*args, **kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

provide(remote_service, all_ready)
relation_name = 'hadoop-client'
required_keys = ['hdfs-ready']
class charmhelpers.contrib.bigdata.relations.Hive(port=None)

Bases: charmhelpers.core.charmframework.helpers.Relation

provide(remote_service, all_ready)
relation_name = 'hive'
required_keys = ['private-address', 'port', 'ready']
class charmhelpers.contrib.bigdata.relations.MySQL(**kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

relation_name = 'db'
required_keys = ['host', 'database', 'user', 'password']
class charmhelpers.contrib.bigdata.relations.NameNode(spec=None, port=None)

Bases: charmhelpers.contrib.bigdata.relations.SpecMatchingRelation

Relation which communicates the NameNode (HDFS) connection & status info.

This is the relation that clients should use.

provide(remote_service, all_ready)
relation_name = 'namenode'
required_keys = ['private-address', 'port', 'ready']
class charmhelpers.contrib.bigdata.relations.NameNodeMaster(spec=None, port=None)

Bases: charmhelpers.contrib.bigdata.relations.NameNode

Alternate NameNode relation for DataNodes.

provide(remote_service, all_ready)
relation_name = 'datanode'
class charmhelpers.contrib.bigdata.relations.NodeManager(**kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

Relation which communicates NodeManager info back to ResourceManagers.

relation_name = 'nodemanager'
required_keys = ['private-address']
class charmhelpers.contrib.bigdata.relations.ResourceManager(spec=None, port=None)

Bases: charmhelpers.contrib.bigdata.relations.SpecMatchingRelation

Relation which communicates the ResourceManager (YARN) connection & status info.

This is the relation that clients should use.

provide(remote_service, all_ready)
relation_name = 'resourcemanager'
required_keys = ['private-address', 'port', 'ready']
class charmhelpers.contrib.bigdata.relations.ResourceManagerMaster(spec=None, port=None)

Bases: charmhelpers.contrib.bigdata.relations.ResourceManager

Alternate ResourceManager relation for NodeManagers.

get_ssh_key()
install_ssh_keys()
provide(remote_service, all_ready)
relation_name = 'nodemanager'
required_keys = ['private-address', 'ssh-key', 'ready']
class charmhelpers.contrib.bigdata.relations.SpecMatchingRelation(spec=None, *args, **kwargs)

Bases: charmhelpers.core.charmframework.helpers.Relation

Relation base class that validates that a version and environment between two related charms match, to prevent interoperability issues.

This class adds a spec key to the required_keys and populates it in provide(). The spec value must be passed in to __init__().

The spec should be a mapping (or a callback that returns a mapping) which describes all aspects of the charm’s environment or configuration that might affect its interoperability with the remote charm. The charm on the requires side of the relation will verify that all of the keys in its spec are present and exactly equal on the provides side of the relation. This does mean that the requires side can be a subset of the provides side, but not the other way around.

An example spec string might be:

{
    'arch': 'x86_64',
    'vendor': 'apache',
    'version': '2.4',
}
is_ready()

Validate the spec data from the connected units to ensure that it matches the local spec.

provide(remote_service, all_ready)

Provide the spec data to the remote service.

Subclasses must either delegate to this method (e.g., via super()) or include 'spec': json.dumps(self.spec) in the provided data themselves.

spec
charmhelpers.contrib.bigdata.handlers
charmhelpers.contrib.bigdata.handlers.apache.HDFS
charmhelpers.contrib.bigdata.handlers.apache.HadoopBase
charmhelpers.contrib.bigdata.handlers.apache.YARN
class charmhelpers.contrib.bigdata.handlers.apache.HDFS(hadoop_base)

Bases: object

configure_client()
configure_datanode()
configure_hdfs_base(host, port)
configure_namenode()
configure_secondarynamenode()

Configure the Secondary Namenode when the apache-hadoop-hdfs-secondary charm is deployed and related to apache-hadoop-hdfs-master.

The only purpose of the secondary namenode is to perform periodic checkpoints. The secondary name-node periodically downloads current namenode image and edits log files, joins them into new image and uploads the new image back to the (primary and the only) namenode.

create_hdfs_dirs()
format_namenode()
register_slaves()
start_datanode()
start_namenode()
start_secondarynamenode()
stop_datanode()
stop_namenode()
stop_secondarynamenode()
class charmhelpers.contrib.bigdata.handlers.apache.HadoopBase(dist_config)

Bases: object

configure_hadoop()
configure_hosts_file()

Add the unit’s private-address to /etc/hosts to ensure that Java can resolve the hostname of the server to its real IP address.

install(force=False)
install_base_packages()
install_hadoop()
install_java()

Run the java-installer resource to install Java and determine the JAVA_HOME and Java version.

The java-installer must be idempotent and its only output (on stdout) should be two lines: the JAVA_HOME path, and the Java version, respectively.

If there is an error installing Java, the installer should exit with a non-zero exit code.

is_installed()
run(user, command, *args, **kwargs)

Run a Hadoop command as the hdfs user.

Parameters:
  • command (str) – Command to run, prefixed with bin/ or sbin/
  • args (list) – Additional args to pass to the command
setup_hadoop_config()
spec()

Generate the full spec for keeping charms in sync.

NB: This has to be a callback instead of a plain property because it is passed to the relations during construction of the Manager but needs to properly reflect the Java version in the same hook invocation that installs Java.

class charmhelpers.contrib.bigdata.handlers.apache.YARN(hadoop_base)

Bases: object

configure_client()
configure_jobhistory()
configure_nodemanager()
configure_resourcemanager()
configure_yarn_base(host, port)
install_demo()
start_jobhistory()
start_nodemanager()
start_resourcemanager()
stop_jobhistory()
stop_nodemanager()
stop_resourcemanager()
charmhelpers.contrib.bigdata.utils
charmhelpers.contrib.bigdata.utils.DistConfig This class processes distribution-specific configuration options.
charmhelpers.contrib.bigdata.utils.TimeoutError
charmhelpers.contrib.bigdata.utils.disable_firewall Temporarily disable the firewall, via ufw.
charmhelpers.contrib.bigdata.utils.environment_edit_in_place Edit the /etc/environment file in-place.
charmhelpers.contrib.bigdata.utils.jps Get PIDs for named Java processes, for any user.
charmhelpers.contrib.bigdata.utils.normalize_strbool
charmhelpers.contrib.bigdata.utils.re_edit_in_place Perform a set of in-place edits to a file.
charmhelpers.contrib.bigdata.utils.read_etc_env Read /etc/environment and return it as a dict.
charmhelpers.contrib.bigdata.utils.run_as Run a command as a particular user, using /etc/environment and optionally capturing and returning the output.
charmhelpers.contrib.bigdata.utils.verify_resources Predicate for specific named resources, with useful rendering in the logs.
charmhelpers.contrib.bigdata.utils.wait_for_hdfs
charmhelpers.contrib.bigdata.utils.xmlpropmap_edit_in_place Edit an XML property map (configuration) file in-place.
class charmhelpers.contrib.bigdata.utils.DistConfig(filename='dist.yaml', required_keys=None)

Bases: object

This class processes distribution-specific configuration options.

Some configuration options are specific to the Hadoop distribution, (e.g. Apache, Hortonworks, MapR, etc). These options are immutable and must not change throughout the charm deployment lifecycle.

Helper methods are provided for keys that require action. Presently, this includes adding/removing directories, dependent packages, and groups/users. Other required keys may be listed when instantiating this class, but this will only validate these keys exist in the yaml; it will not provide any helper functionality for unkown keys.

Parameters:
  • filename (str) – File to process (default dist.yaml)
  • required_keys (list) – A list of keys required to be present in the yaml
Example dist.yaml with supported keys:

vendor: ‘<name>’ hadoop_version: ‘<version>’ packages:

  • ‘<package 1>’
  • ‘<package 2>’
groups:
  • ‘<name>’
users:
<user 1>:
groups: [‘<primary>’, ‘<group>’, ‘<group>’]
<user 2>:
groups: [‘<primary>’]
dirs:
<dir 1>:
path: ‘</path/to/dir>’ perms: 0777
<dir 2>:
path: ‘{config[<option>]}’ # value comes from config option owner: ‘<user>’ group: ‘<group>’ perms: 0755
ports:
<name1>:
port: <port> exposed_on: <service> # optional
<name2>:
port: <port> exposed_on: <service> # optional
add_dirs()
add_packages()
add_users()
exposed_ports(service)
path(key)
port(key)
remove_dirs()
remove_packages()
remove_users()
exception charmhelpers.contrib.bigdata.utils.TimeoutError

Bases: exceptions.Exception

charmhelpers.contrib.bigdata.utils.disable_firewall(*args, **kwds)

Temporarily disable the firewall, via ufw.

charmhelpers.contrib.bigdata.utils.environment_edit_in_place(*args, **kwds)

Edit the /etc/environment file in-place.

There is no standard definition for the format of /etc/environment, but the convention, which this helper supports, is simple key-value pairs, separated by =, with optionally quoted values.

Note that this helper will implicitly quote all values.

Also note that the file is not locked during the edits.

charmhelpers.contrib.bigdata.utils.jps(name)

Get PIDs for named Java processes, for any user.

charmhelpers.contrib.bigdata.utils.normalize_strbool(value)
charmhelpers.contrib.bigdata.utils.re_edit_in_place(filename, subs)

Perform a set of in-place edits to a file.

Parameters:
  • filename (str) – Name of file to edit
  • subs (dict) – Mapping of patterns to replacement strings
charmhelpers.contrib.bigdata.utils.read_etc_env()

Read /etc/environment and return it as a dict.

charmhelpers.contrib.bigdata.utils.run_as(user, command, *args, **kwargs)

Run a command as a particular user, using /etc/environment and optionally capturing and returning the output.

Raises subprocess.CalledProcessError if command fails.

Parameters:
  • user (str) – Username to run command as
  • command (str) – Command to run
  • args (list) – Additional args to pass to command
  • env (dict) – Additional env variables (will be merged with /etc/environment)
  • capture_output (bool) – Capture and return output (default: False)
class charmhelpers.contrib.bigdata.utils.verify_resources(*which)

Bases: object

Predicate for specific named resources, with useful rendering in the logs.

Parameters:*which (str) –

One or more resource names to fetch & verify. Defaults to all non-optional resources.
charmhelpers.contrib.bigdata.utils.wait_for_hdfs(timeout)
charmhelpers.contrib.bigdata.utils.xmlpropmap_edit_in_place(*args, **kwds)

Edit an XML property map (configuration) file in-place.

This helper acts as a context manager which edits an XML file of the form:

<configuration>
<property>
<name>property-name</name> <value>property-value</value> <description>Optional property description</description>

</property> ...

</configuration>

This context manager yields a dict containing the existing name/value mappings. Properties can then be modified, added, or removed, and the changes will be reflected in the file.

Example usage:

with xmlpropmap_edit_in_place(‘my.xml’) as props:
props[‘foo’] = ‘bar’ del props[‘removed’]

Note that the file is not locked during the edits.

charmhelpers.contrib.charmhelpers package

charmhelpers.contrib.charmhelpers.unit_info(service_name, item_name, data=None, unit=None)
charmhelpers.contrib.charmhelpers.wait_for_machine(num_machines=1, timeout=300)

Wait timeout seconds for num_machines machines to come up.

This wait_for... function can be called by other wait_for functions whose timeouts might be too short in situations where only a bare Juju setup has been bootstrapped.

Returns:A tuple of (num_machines, time_taken). This is used for testing.
charmhelpers.contrib.charmhelpers.wait_for_page_contents(url, contents, timeout=120, validate=None)
charmhelpers.contrib.charmhelpers.wait_for_relation(service_name, relation_name, timeout=120)

Wait timeout seconds for a given relation to come up.

charmhelpers.contrib.charmhelpers.wait_for_unit(service_name, timeout=480)

Wait timeout seconds for a given service name to come up.

charmhelpers.contrib.charmsupport package

charmhelpers.contrib.charmsupport.nrpe module

Compatibility with the nrpe-external-master charm

class charmhelpers.contrib.charmsupport.nrpe.Check(shortname, description, check_cmd)

Bases: object

run()
service_template = '\n#---------------------------------------------------\n# This file is Juju managed\n#---------------------------------------------------\ndefine service {{\n use active-service\n host_name {nagios_hostname}\n service_description {nagios_hostname}[{shortname}] {description}\n check_command check_nrpe!{command}\n servicegroups {nagios_servicegroup}\n}}\n'
shortname_re = '[A-Za-z0-9-_]+$'
write(nagios_context, hostname, nagios_servicegroups)
write_service_config(nagios_context, hostname, nagios_servicegroups)
exception charmhelpers.contrib.charmsupport.nrpe.CheckException

Bases: exceptions.Exception

class charmhelpers.contrib.charmsupport.nrpe.NRPE(hostname=None)

Bases: object

add_check(*args, **kwargs)
nagios_exportdir = '/var/lib/nagios/export'
nagios_logdir = '/var/log/nagios'
nrpe_confdir = '/etc/nagios/nrpe.d'
write()
charmhelpers.contrib.charmsupport.nrpe.add_haproxy_checks(nrpe, unit_name)

Add checks for each service in list

Parameters:
  • nrpe (NRPE) – NRPE object to add check to
  • unit_name (str) – Unit name to use in check description
charmhelpers.contrib.charmsupport.nrpe.add_init_service_checks(nrpe, services, unit_name)

Add checks for each service in list

Parameters:
  • nrpe (NRPE) – NRPE object to add check to
  • services (list) – List of services to check
  • unit_name (str) – Unit name to use in check description
charmhelpers.contrib.charmsupport.nrpe.copy_nrpe_checks()

Copy the nrpe checks into place

charmhelpers.contrib.charmsupport.nrpe.get_nagios_hostcontext(relation_name='nrpe-external-master')

Query relation with nrpe subordinate, return the nagios_host_context

Parameters:relation_name (str) – Name of relation nrpe sub joined to
charmhelpers.contrib.charmsupport.nrpe.get_nagios_hostname(relation_name='nrpe-external-master')

Query relation with nrpe subordinate, return the nagios_hostname

Parameters:relation_name (str) – Name of relation nrpe sub joined to
charmhelpers.contrib.charmsupport.nrpe.get_nagios_unit_name(relation_name='nrpe-external-master')

Return the nagios unit name prepended with host_context if needed

Parameters:relation_name (str) – Name of relation nrpe sub joined to
charmhelpers.contrib.charmsupport.volumes module

Functions for managing volumes in juju units. One volume is supported per unit. Subordinates may have their own storage, provided it is on its own partition.

Configuration stanzas:

volume-ephemeral:
  type: boolean
  default: true
  description: >
    If false, a volume is mounted as sepecified in "volume-map"
    If true, ephemeral storage will be used, meaning that log data
       will only exist as long as the machine. YOU HAVE BEEN WARNED.
volume-map:
  type: string
  default: {}
  description: >
    YAML map of units to device names, e.g:
      "{ rsyslog/0: /dev/vdb, rsyslog/1: /dev/vdb }"
    Service units will raise a configure-error if volume-ephemeral
    is 'true' and no volume-map value is set. Use 'juju set' to set a
    value and 'juju resolved' to complete configuration.

Usage:

from charmsupport.volumes import configure_volume, VolumeConfigurationError
from charmsupport.hookenv import log, ERROR
def post_mount_hook():
    stop_service('myservice')
def post_mount_hook():
    start_service('myservice')

if __name__ == '__main__':
    try:
        configure_volume(before_change=pre_mount_hook,
                         after_change=post_mount_hook)
    except VolumeConfigurationError:
        log('Storage could not be configured', ERROR)
exception charmhelpers.contrib.charmsupport.volumes.VolumeConfigurationError

Bases: exceptions.Exception

Volume configuration data is missing or invalid

charmhelpers.contrib.charmsupport.volumes.configure_volume(before_change=<function <lambda> at 0x7fc5863e3c08>, after_change=<function <lambda> at 0x7fc5863e3b90>)

Set up storage (or don’t) according to the charm’s volume configuration. Returns the mount point or “ephemeral”. before_change and after_change are optional functions to be called if the volume configuration changes.

charmhelpers.contrib.charmsupport.volumes.get_config()

Gather and sanity-check volume configuration data

charmhelpers.contrib.charmsupport.volumes.managed_mounts()

List of all mounted managed volumes

charmhelpers.contrib.charmsupport.volumes.mount_volume(config)
charmhelpers.contrib.charmsupport.volumes.unmount_volume(config)

charmhelpers.contrib.hahelpers package

charmhelpers.contrib.hahelpers.apache module
charmhelpers.contrib.hahelpers.apache.get_ca_cert()
charmhelpers.contrib.hahelpers.apache.get_cert(cn=None)
charmhelpers.contrib.hahelpers.apache.install_ca_cert(ca_cert)
charmhelpers.contrib.hahelpers.cluster module

Helpers for clustering and determining “cluster leadership” and other clustering-related helpers.

exception charmhelpers.contrib.hahelpers.cluster.CRMResourceNotFound

Bases: exceptions.Exception

exception charmhelpers.contrib.hahelpers.cluster.HAIncompleteConfig

Bases: exceptions.Exception

charmhelpers.contrib.hahelpers.cluster.canonical_url(configs, vip_setting='vip')

Returns the correct HTTP URL to this host given the state of HTTPS configuration and hacluster.

:configs : OSTemplateRenderer: A config tempating object to inspect for
a complete https context.
Vip_setting:str: Setting in charm config that specifies VIP address.
charmhelpers.contrib.hahelpers.cluster.determine_apache_port(public_port, singlenode_mode=False)

Description: Determine correct apache listening port based on public IP + state of the cluster.

public_port: int: standard public port for given service

singlenode_mode: boolean: Shuffle ports when only a single unit is present

returns: int: the correct listening port for the HAProxy service

charmhelpers.contrib.hahelpers.cluster.determine_api_port(public_port, singlenode_mode=False)

Determine correct API server listening port based on existence of HTTPS reverse proxy and/or haproxy.

public_port: int: standard public port for given service

singlenode_mode: boolean: Shuffle ports when only a single unit is present

returns: int: the correct listening port for the API service

charmhelpers.contrib.hahelpers.cluster.eligible_leader(resource)
charmhelpers.contrib.hahelpers.cluster.get_hacluster_config(exclude_keys=None)

Obtains all relevant configuration from charm configuration required for initiating a relation to hacluster:

ha-bindiface, ha-mcastport, vip

param: exclude_keys: list of setting key(s) to be excluded. returns: dict: A dict containing settings keyed by setting name. raises: HAIncompleteConfig if settings are missing.

charmhelpers.contrib.hahelpers.cluster.https()

Determines whether enough data has been provided in configuration or relation data to configure HTTPS . returns: boolean

charmhelpers.contrib.hahelpers.cluster.is_clustered()
charmhelpers.contrib.hahelpers.cluster.is_elected_leader(resource)

Returns True if the charm executing this is the elected cluster leader.

It relies on two mechanisms to determine leadership:
1. If the charm is part of a corosync cluster, call corosync to determine leadership. 2. If the charm is not part of a corosync cluster, the leader is determined as being “the alive unit with the lowest unit numer”. In other words, the oldest surviving unit.
charmhelpers.contrib.hahelpers.cluster.is_leader(resource)
charmhelpers.contrib.hahelpers.cluster.oldest_peer(peers)

Determines who the oldest peer is by comparing unit numbers.

charmhelpers.contrib.hahelpers.cluster.peer_ips(peer_relation='cluster', addr_key='private-address')

Return a dict of peers and their private-address

charmhelpers.contrib.hahelpers.cluster.peer_units(peer_relation='cluster')

charmhelpers.contrib.network package

charmhelpers.contrib.network.ovs package

Helpers for interacting with OpenvSwitch

charmhelpers.contrib.network.ovs.add_bridge(name)

Add the named bridge to openvswitch

charmhelpers.contrib.network.ovs.add_bridge_port(name, port, promisc=False)

Add a port to the named openvswitch bridge

charmhelpers.contrib.network.ovs.del_bridge(name)

Delete the named bridge from openvswitch

charmhelpers.contrib.network.ovs.del_bridge_port(name, port)

Delete a port from the named openvswitch bridge

charmhelpers.contrib.network.ovs.full_restart()

Full restart and reload of openvswitch

charmhelpers.contrib.network.ovs.get_certificate()

Read openvswitch certificate from disk

charmhelpers.contrib.network.ovs.set_manager(manager)

Set the controller for the local openvswitch

charmhelpers.contrib.network.ip module
charmhelpers.contrib.network.ip.format_ipv6_addr(address)

If address is IPv6, wrap it in ‘[]’ otherwise return None.

This is required by most configuration files when specifying IPv6 addresses.

charmhelpers.contrib.network.ip.get_address_in_network(network, fallback=None, fatal=False)

Get an IPv4 or IPv6 address within the network from the host.

Parameters:
  • (str) (fallback) – CIDR presentation format. For example, ‘192.168.1.0/24’.
  • (str) – If no address is found, return fallback.
  • (boolean) (fatal) – If no address is found, fallback is not set and fatal is True then exit(1).
charmhelpers.contrib.network.ip.get_bridge_nics(bridge, vnic_dir='/sys/devices/virtual/net')

Return a list of nics comprising a given bridge on the system.

charmhelpers.contrib.network.ip.get_bridges(vnic_dir='/sys/devices/virtual/net')

Return a list of bridges on the system.

charmhelpers.contrib.network.ip.get_host_ip(hostname, fallback=None)

Resolves the IP for a given hostname, or returns the input if it is already an IP.

charmhelpers.contrib.network.ip.get_hostname(address, fqdn=True)

Resolves hostname for given IP, or returns the input if it is already a hostname.

charmhelpers.contrib.network.ip.get_iface_addr(iface='eth0', inet_type='AF_INET', inc_aliases=False, fatal=True, exc_list=None)

Return the assigned IP address for a given interface, if any.

charmhelpers.contrib.network.ip.get_iface_from_addr(addr)

Work out on which interface the provided address is configured.

charmhelpers.contrib.network.ip.get_ipv6_addr(*args, **kwargs)
charmhelpers.contrib.network.ip.is_address_in_network(network, address)

Determine whether the provided address is within a network range.

Parameters:
  • (str) (network) – CIDR presentation format. For example, ‘192.168.1.0/24’.
  • address – An individual IPv4 or IPv6 address without a net mask or subnet prefix. For example, ‘192.168.1.1’.
Returns boolean:
 

Flag indicating whether address is in network.
charmhelpers.contrib.network.ip.is_bridge_member(nic)

Check if a given nic is a member of a bridge.

charmhelpers.contrib.network.ip.is_ip(address)

Returns True if address is a valid IP address.

charmhelpers.contrib.network.ip.is_ipv6(address)

Determine whether provided address is IPv6 or not.

charmhelpers.contrib.network.ip.no_ip_found_error_out(network)
charmhelpers.contrib.network.ip.ns_query(address)
charmhelpers.contrib.network.ip.sniff_iface(f)

Ensure decorated function is called with a value for iface.

If no iface provided, inject net iface inferred from unit private address.

charmhelpers.contrib.openstack package

charmhelpers.contrib.openstack.templates package
charmhelpers.contrib.openstack.alternatives module

Helper for managing alternatives for file conflict resolution

charmhelpers.contrib.openstack.alternatives.install_alternative(name, target, source, priority=50)

Install alternative configuration

charmhelpers.contrib.openstack.context module
class charmhelpers.contrib.openstack.context.AMQPContext(ssl_dir=None, rel_name='amqp', relation_prefix=None)

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.ApacheSSLContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

Generates a context for an apache vhost configuration that configures HTTPS reverse proxying for one or many endpoints. Generated context looks something like:

{
    'namespace': 'cinder',
    'private_address': 'iscsi.mycinderhost.com',
    'endpoints': [(8776, 8766), (8777, 8767)]
}

The endpoints list consists of a tuples mapping external ports to internal ports.

canonical_names()

Figure out which canonical names clients will access this service.

configure_ca()
configure_cert(cn=None)
enable_modules()
external_ports = []
get_network_addresses()
For each network configured, return corresponding address and vip
(if available).

Returns a list of tuples of the form:

[(address_in_net_a, vip_in_net_a),
(address_in_net_b, vip_in_net_b), ...]

or, if no vip(s) available:

[(address_in_net_a, address_in_net_a),
(address_in_net_b, address_in_net_b), ...]
interfaces = ['https']
service_namespace = None
class charmhelpers.contrib.openstack.context.BindHostContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.CephContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

Generates context for /etc/ceph/ceph.conf templates.

interfaces = ['ceph']
class charmhelpers.contrib.openstack.context.DataPortContext

Bases: charmhelpers.contrib.openstack.context.NeutronPortContext

class charmhelpers.contrib.openstack.context.ExternalPortContext

Bases: charmhelpers.contrib.openstack.context.NeutronPortContext

class charmhelpers.contrib.openstack.context.HAProxyContext(singlenode_mode=False)

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

Provides half a context for the haproxy template, which describes all peers to be included in the cluster. Each charm needs to include its own context generator that describes the port mapping.

interfaces = ['cluster']
class charmhelpers.contrib.openstack.context.IdentityServiceContext(service=None, service_user=None, rel_name='identity-service')

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.ImageServiceContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

interfaces = ['image-service']
class charmhelpers.contrib.openstack.context.LogLevelContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.NetworkServiceContext(rel_name='quantum-network-service')

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.NeutronAPIContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

Inspects current neutron-plugin-api relation for neutron settings. Return defaults if it is not present.

get_neutron_options(rdata)
interfaces = ['neutron-plugin-api']
class charmhelpers.contrib.openstack.context.NeutronContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

calico_ctxt()
interfaces = []
n1kv_ctxt()
network_manager
neutron_ctxt()
neutron_security_groups
nvp_ctxt()
ovs_ctxt()
packages
plugin
class charmhelpers.contrib.openstack.context.NeutronPortContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

NIC_PREFIXES = ['eth', 'bond']
resolve_ports(ports)

Resolve NICs not yet bound to bridge(s)

If hwaddress provided then returns resolved hwaddress otherwise NIC.

class charmhelpers.contrib.openstack.context.NotificationDriverContext(zmq_relation='zeromq-configuration', amqp_relation='amqp')

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.OSConfigFlagContext(charm_flag='config-flags', template_flag='user_config_flags')

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

Provides support for user-defined config flags.

Users can define a comma-seperated list of key=value pairs in the charm configuration and apply them at any point in any file by using a template flag.

Sometimes users might want config flags inserted within a specific section so this class allows users to specify the template flag name, allowing for multiple template flags (sections) within the same context.

NOTE: the value of config-flags may be a comma-separated list of
key=value pairs and some Openstack config files support comma-separated lists as values.
exception charmhelpers.contrib.openstack.context.OSContextError

Bases: exceptions.Exception

class charmhelpers.contrib.openstack.context.OSContextGenerator

Bases: object

Base class for all context generators.

interfaces = []
class charmhelpers.contrib.openstack.context.PhyNICMTUContext

Bases: charmhelpers.contrib.openstack.context.DataPortContext

class charmhelpers.contrib.openstack.context.PostgresqlDBContext(database=None)

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

interfaces = ['pgsql-db']
class charmhelpers.contrib.openstack.context.SharedDBContext(database=None, user=None, relation_prefix=None, ssl_dir=None)

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

interfaces = ['shared-db']
class charmhelpers.contrib.openstack.context.SubordinateConfigContext(service, config_file, interface)

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

Responsible for inspecting relations to subordinates that may be exporting required config via a json blob.

The subordinate interface allows subordinates to export their configuration requirements to the principle for multiple config files and multiple serivces. Ie, a subordinate that has interfaces to both glance and nova may export to following yaml blob as json:

glance:
    /etc/glance/glance-api.conf:
        sections:
            DEFAULT:
                - [key1, value1]
    /etc/glance/glance-registry.conf:
            MYSECTION:
                - [key2, value2]
nova:
    /etc/nova/nova.conf:
        sections:
            DEFAULT:
                - [key3, value3]

It is then up to the principle charms to subscribe this context to the service+config file it is interestd in. Configuration data will be available in the template context, in glance’s case, as:

ctxt = {
    ... other context ...
    'subordinate_config': {
        'DEFAULT': {
            'key1': 'value1',
        },
        'MYSECTION': {
            'key2': 'value2',
        },
    }
}
class charmhelpers.contrib.openstack.context.SysctlContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

This context check if the ‘sysctl’ option exists on configuration then creates a file with the loaded contents

class charmhelpers.contrib.openstack.context.SyslogContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

class charmhelpers.contrib.openstack.context.WorkerConfigContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

num_cpus
class charmhelpers.contrib.openstack.context.ZeroMQContext

Bases: charmhelpers.contrib.openstack.context.OSContextGenerator

interfaces = ['zeromq-configuration']
charmhelpers.contrib.openstack.context.config_flags_parser(config_flags)

Parses config flags string into dict.

This parsing method supports a few different formats for the config flag values to be parsed:

  1. A string in the simple format of key=value pairs, with the possibility of specifying multiple key value pairs within the same string. For example, a string in the format of ‘key1=value1, key2=value2’ will return a dict of: {‘key1’: ‘value1’,

    ‘key2’: ‘value2’}.

  2. A string in the above format, but supporting a comma-delimited list of values for the same key. For example, a string in the format of ‘key1=value1, key2=value3,value4,value5’ will return a dict of: {‘key1’, ‘value1’,

    ‘key2’, ‘value2,value3,value4’}

  3. A string containing a colon character (:) prior to an equal character (=) will be treated as yaml and parsed as such. This can be used to specify more complex key value pairs. For example, a string in the format of ‘key1: subkey1=value1, subkey2=value2’ will return a dict of: {‘key1’, ‘subkey1=value1, subkey2=value2’}

The provided config_flags string may be a list of comma-separated values which themselves may be comma-separated list of values.

charmhelpers.contrib.openstack.context.context_complete(ctxt)
charmhelpers.contrib.openstack.context.db_ssl(rdata, ctxt, ssl_dir)
charmhelpers.contrib.openstack.context.ensure_packages(packages)

Install but do not upgrade required plugin packages.

charmhelpers.contrib.openstack.neutron module
charmhelpers.contrib.openstack.neutron.determine_dkms_package()

Determine which DKMS package should be used based on kernel version

charmhelpers.contrib.openstack.neutron.headers_package()

Ensures correct linux-headers for running kernel are installed, for building DKMS package

charmhelpers.contrib.openstack.neutron.kernel_version()

Retrieve the current major kernel version as a tuple e.g. (3, 13)

charmhelpers.contrib.openstack.neutron.network_manager()

Deals with the renaming of Quantum to Neutron in H and any situations that require compatability (eg, deploying H with network-manager=quantum, upgrading from G).

charmhelpers.contrib.openstack.neutron.neutron_plugin_attribute(plugin, attr, net_manager=None)
charmhelpers.contrib.openstack.neutron.neutron_plugins()
charmhelpers.contrib.openstack.neutron.parse_bridge_mappings(mappings)

Parse bridge mappings.

Mappings must be a space-delimited list of provider:bridge mappings.

Returns dict of the form {provider:bridge}.

charmhelpers.contrib.openstack.neutron.parse_data_port_mappings(mappings, default_bridge='br-data')

Parse data port mappings.

Mappings must be a space-delimited list of bridge:port mappings.

Returns dict of the form {bridge:port}.

charmhelpers.contrib.openstack.neutron.parse_mappings(mappings)
charmhelpers.contrib.openstack.neutron.parse_vlan_range_mappings(mappings)

Parse vlan range mappings.

Mappings must be a space-delimited list of provider:start:end mappings.

Returns dict of the form {provider: (start, end)}.

charmhelpers.contrib.openstack.neutron.quantum_plugins()
charmhelpers.contrib.openstack.templating module
exception charmhelpers.contrib.openstack.templating.OSConfigException

Bases: exceptions.Exception

class charmhelpers.contrib.openstack.templating.OSConfigRenderer(templates_dir, openstack_release)

Bases: object

This class provides a common templating system to be used by OpenStack charms. It is intended to help charms share common code and templates, and ease the burden of managing config templates across multiple OpenStack releases.

Basic usage:

# import some common context generates from charmhelpers
from charmhelpers.contrib.openstack import context

# Create a renderer object for a specific OS release.
configs = OSConfigRenderer(templates_dir='/tmp/templates',
                           openstack_release='folsom')
# register some config files with context generators.
configs.register(config_file='/etc/nova/nova.conf',
                 contexts=[context.SharedDBContext(),
                           context.AMQPContext()])
configs.register(config_file='/etc/nova/api-paste.ini',
                 contexts=[context.IdentityServiceContext()])
configs.register(config_file='/etc/haproxy/haproxy.conf',
                 contexts=[context.HAProxyContext()])
# write out a single config
configs.write('/etc/nova/nova.conf')
# write out all registered configs
configs.write_all()

OpenStack Releases and template loading

When the object is instantiated, it is associated with a specific OS release. This dictates how the template loader will be constructed.

The constructed loader attempts to load the template from several places in the following order: - from the most recent OS release-specific template dir (if one exists) - the base templates_dir - a template directory shipped in the charm with this helper file.

For the example above, ‘/tmp/templates’ contains the following structure:

/tmp/templates/nova.conf
/tmp/templates/api-paste.ini
/tmp/templates/grizzly/api-paste.ini
/tmp/templates/havana/api-paste.ini

Since it was registered with the grizzly release, it first seraches the grizzly directory for nova.conf, then the templates dir.

When writing api-paste.ini, it will find the template in the grizzly directory.

If the object were created with folsom, it would fall back to the base templates dir for its api-paste.ini template.

This system should help manage changes in config files through openstack releases, allowing charms to fall back to the most recently updated config template for a given release

The haproxy.conf, since it is not shipped in the templates dir, will be loaded from the module directory’s template directory, eg $CHARM/hooks/charmhelpers/contrib/openstack/templates. This allows us to ship common templates (haproxy, apache) with the helpers.

Context generators

Context generators are used to generate template contexts during hook execution. Doing so may require inspecting service relations, charm config, etc. When registered, a config file is associated with a list of generators. When a template is rendered and written, all context generates are called in a chain to generate the context dictionary passed to the jinja2 template. See context.py for more info.

complete_contexts()

Returns a list of context interfaces that yield a complete context.

register(config_file, contexts)

Register a config file with a list of context generators to be called during rendering.

render(config_file)
set_release(openstack_release)

Resets the template environment and generates a new template loader based on a the new openstack release.

write(config_file)

Write a single config file, raises if config file is not registered.

write_all()

Write out all registered config files.

class charmhelpers.contrib.openstack.templating.OSConfigTemplate(config_file, contexts)

Bases: object

Associates a config file template with a list of context generators. Responsible for constructing a template context based on those generators.

complete_contexts()

Return a list of interfaces that have atisfied contexts.

context()
charmhelpers.contrib.openstack.templating.get_loader(templates_dir, os_release)

Create a jinja2.ChoiceLoader containing template dirs up to and including os_release. If directory template directory is missing at templates_dir, it will be omitted from the loader. templates_dir is added to the bottom of the search list as a base loading dir.

A charm may also ship a templates dir with this module and it will be appended to the bottom of the search list, eg:

hooks/charmhelpers/contrib/openstack/templates
Parameters:
  • (str) (os_release) – Base template directory containing release sub-directories.
  • (str) – OpenStack release codename to construct template loader.
Returns:

jinja2.ChoiceLoader constructed with a list of jinja2.FilesystemLoaders, ordered in descending order by OpenStack release.
charmhelpers.contrib.openstack.utils module
charmhelpers.contrib.openstack.utils.clean_storage(block_device)
Ensures a block device is clean. That is:
  • unmounted
  • any lvm volume groups are deactivated
  • any lvm physical device signatures removed
  • partition table wiped
Parameters:block_device – str: Full path to block device to clean.
charmhelpers.contrib.openstack.utils.config_value_changed(option)

Determine if config value changed since last call to this function.

charmhelpers.contrib.openstack.utils.configure_installation_source(rel)

Configure apt installation source.

charmhelpers.contrib.openstack.utils.ensure_block_device(block_device)

Confirm block_device, create as loopback if necessary.

Parameters:block_device – str: Full path of block device to ensure.
Returns:str: Full path of ensured block device.
charmhelpers.contrib.openstack.utils.error_out(msg)
charmhelpers.contrib.openstack.utils.get_matchmaker_map(mm_file='/etc/oslo/matchmaker_ring.json')
charmhelpers.contrib.openstack.utils.get_os_codename_install_source(src)

Derive OpenStack release codename from a given installation source.

charmhelpers.contrib.openstack.utils.get_os_codename_package(package, fatal=True)

Derive OpenStack release codename from an installed package.

charmhelpers.contrib.openstack.utils.get_os_codename_version(vers)

Determine OpenStack codename from version number.

charmhelpers.contrib.openstack.utils.get_os_version_codename(codename)

Determine OpenStack version number from codename.

charmhelpers.contrib.openstack.utils.get_os_version_install_source(src)
charmhelpers.contrib.openstack.utils.get_os_version_package(pkg, fatal=True)

Derive OpenStack version number from an installed package.

charmhelpers.contrib.openstack.utils.git_clone_and_install(projects_yaml, core_project)

Clone/install all specified OpenStack repositories.

The expected format of projects_yaml is:
repositories:
  • {name: keystone,

    repository: ‘git://git.openstack.org/openstack/keystone.git’, branch: ‘stable/icehouse’}

  • {name: requirements,

    repository: ‘git://git.openstack.org/openstack/requirements.git’, branch: ‘stable/icehouse’}

directory: /mnt/openstack-git http_proxy: http://squid.internal:3128 https_proxy: https://squid.internal:3128

The directory, http_proxy, and https_proxy keys are optional.

charmhelpers.contrib.openstack.utils.git_install_requested()

Returns true if openstack-origin-git is specified.

charmhelpers.contrib.openstack.utils.git_src_dir(projects_yaml, project)

Return the directory where the specified project’s source is located.

charmhelpers.contrib.openstack.utils.import_key(keyid)
charmhelpers.contrib.openstack.utils.openstack_upgrade_available(package)

Determines if an OpenStack upgrade is available from installation source, based on version of installed package.

Parameters:package – str: Name of installed package.
Returns:bool: : Returns True if configured installation source offers a newer version of package.
charmhelpers.contrib.openstack.utils.os_release(package, base='essex')

Returns OpenStack release codename from a cached global. If the codename can not be determined from either an installed package or the installation source, the earliest release supported by the charm should be returned.

charmhelpers.contrib.openstack.utils.os_requires_version(ostack_release, pkg)

Decorator for hook to specify minimum supported release

charmhelpers.contrib.openstack.utils.save_script_rc(script_path='scripts/scriptrc', **env_vars)

Write an rc file in the charm-delivered directory containing exported environment variables provided by env_vars. Any charm scripts run outside the juju hook environment can source this scriptrc to obtain updated config information necessary to perform health checks or service changes.

charmhelpers.contrib.openstack.utils.sync_db_with_multi_ipv6_addresses(database, database_user, relation_prefix=None)

charmhelpers.contrib.peerstorage package

charmhelpers.contrib.peerstorage.peer_echo(includes=None)

Echo filtered attributes back onto the same relation for storage.

This is a requirement to use the peerstorage module - it needs to be called from the peer relation’s changed hook.

charmhelpers.contrib.peerstorage.peer_retrieve(key, relation_name='cluster')

Retrieve a named key from peer relation relation_name.

charmhelpers.contrib.peerstorage.peer_retrieve_by_prefix(prefix, relation_name='cluster', delimiter='_', inc_list=None, exc_list=None)

Retrieve k/v pairs given a prefix and filter using {inc,exc}_list

charmhelpers.contrib.peerstorage.peer_store(key, value, relation_name='cluster')

Store the key/value pair on the named peer relation relation_name.

charmhelpers.contrib.peerstorage.peer_store_and_set(relation_id=None, peer_relation_name='cluster', peer_store_fatal=False, relation_settings=None, delimiter='_', **kwargs)

Store passed-in arguments both in argument relation and in peer storage.

It functions like doing relation_set() and peer_store() at the same time, with the same data.

@param relation_id: the id of the relation to store the data on. Defaults
to the current relation.
@param peer_store_fatal: Set to True, the function will raise an exception
should the peer sotrage not be avialable.

charmhelpers.contrib.python package

charmhelpers.contrib.python.debug module
charmhelpers.contrib.python.debug.set_trace(addr='0.0.0.0', port=4444)

Set a trace point using the remote debugger

charmhelpers.contrib.python.packages module
charmhelpers.contrib.python.packages.parse_options(given, available)

Given a set of options, check if available

charmhelpers.contrib.python.packages.pip_install(package, fatal=False, upgrade=False, **options)

Install a python package

charmhelpers.contrib.python.packages.pip_install_requirements(requirements, **options)

Install a requirements file

charmhelpers.contrib.python.packages.pip_list()

Returns the list of current python installed packages

charmhelpers.contrib.python.packages.pip_uninstall(package, **options)

Uninstall a python package

charmhelpers.contrib.python.rpdb module

Remote Python Debugger (pdb wrapper).

class charmhelpers.contrib.python.rpdb.Rpdb(addr='127.0.0.1', port=4444)

Bases: pdb.Pdb

do_EOF(arg)

Stop all operation on continue.

do_c(arg)

Stop all operation on continue.

do_cont(arg)

Stop all operation on continue.

do_continue(arg)

Stop all operation on continue.

do_exit(arg)

Stop all operation on continue.

do_quit(arg)

Stop all operation on continue.

shutdown()

Revert stdin and stdout, close the socket.

charmhelpers.contrib.python.version module
charmhelpers.contrib.python.version.current_version()

Current system python version

charmhelpers.contrib.python.version.current_version_string()

Current system python version as string major.minor.micro

charmhelpers.contrib.saltstack package

Charm Helpers saltstack - declare the state of your machines.

This helper enables you to declare your machine state, rather than program it procedurally (and have to test each change to your procedures). Your install hook can be as simple as:

{{{
from charmhelpers.contrib.saltstack import (
    install_salt_support,
    update_machine_state,
)


def install():
    install_salt_support()
    update_machine_state('machine_states/dependencies.yaml')
    update_machine_state('machine_states/installed.yaml')
}}}

and won’t need to change (nor will its tests) when you change the machine state.

It’s using a python package called salt-minion which allows various formats for specifying resources, such as:

{{{
/srv/{{ basedir }}:
    file.directory:
        - group: ubunet
        - user: ubunet
        - require:
            - user: ubunet
        - recurse:
            - user
            - group

ubunet:
    group.present:
        - gid: 1500
    user.present:
        - uid: 1500
        - gid: 1500
        - createhome: False
        - require:
            - group: ubunet
}}}
The docs for all the different state definitions are at:
http://docs.saltstack.com/ref/states/all/
TODO:
  • Add test helpers which will ensure that machine state definitions are functionally (but not necessarily logically) correct (ie. getting salt to parse all state defs.
  • Add a link to a public bootstrap charm example / blogpost.
  • Find a way to obviate the need to use the grains[‘charm_dir’] syntax in templates.
charmhelpers.contrib.saltstack.install_salt_support(from_ppa=True)

Installs the salt-minion helper for machine state.

By default the salt-minion package is installed from the saltstack PPA. If from_ppa is False you must ensure that the salt-minion package is available in the apt cache.

charmhelpers.contrib.saltstack.update_machine_state(state_path)

Update the machine state using the provided state declaration.

charmhelpers.contrib.ssl package

charmhelpers.contrib.ssl.service module
class charmhelpers.contrib.ssl.service.ServiceCA(name, ca_dir, cert_type='standard')

Bases: object

ca_cert
ca_conf
ca_key
create_certificate(common_name)
default_ca_expiry = '2190'
default_expiry = '730'
static get_ca(type='standard')
get_ca_bundle()
get_certificate(common_name)
get_conf_variables()
get_or_create_cert(common_name)
classmethod get_service_cert(type='standard')
init()
signing_conf
charmhelpers.contrib.ssl.generate_selfsigned(keyfile, certfile, keysize='1024', config=None, subject=None, cn=None)

Generate selfsigned SSL keypair

You must provide one of the 3 optional arguments: config, subject or cn If more than one is provided the leftmost will be used

Arguments: keyfile – (required) full path to the keyfile to be created certfile – (required) full path to the certfile to be created keysize – (optional) SSL key length config – (optional) openssl configuration file subject – (optional) dictionary with SSL subject variables cn – (optional) cerfificate common name

Required keys in subject dict: cn – Common name (eq. FQDN)

Optional keys in subject dict country – Country Name (2 letter code) state – State or Province Name (full name) locality – Locality Name (eg, city) organization – Organization Name (eg, company) organizational_unit – Organizational Unit Name (eg, section) email – Email Address

charmhelpers.contrib.storage package

charmhelpers.contrib.storage.linux package
charmhelpers.contrib.storage.linux.ceph module
class charmhelpers.contrib.storage.linux.ceph.CephBrokerRq(api_version=1)

Bases: object

Ceph broker request.

Multiple operations can be added to a request and sent to the Ceph broker to be executed.

Request is json-encoded for sending over the wire.

The API is versioned and defaults to version 1.

add_op_create_pool(name, replica_count=3)
request
class charmhelpers.contrib.storage.linux.ceph.CephBrokerRsp(encoded_rsp)

Bases: object

Ceph broker response.

Response is json-decoded and contents provided as methods/properties.

The API is versioned and defaults to version 1.

exit_code
exit_msg
charmhelpers.contrib.storage.linux.ceph.ceph_version()

Retrieve the local version of ceph.

charmhelpers.contrib.storage.linux.ceph.configure(service, key, auth, use_syslog)

Perform basic configuration of Ceph.

charmhelpers.contrib.storage.linux.ceph.copy_files(src, dst, symlinks=False, ignore=None)

Copy files from src to dst.

charmhelpers.contrib.storage.linux.ceph.create_key_file(service, key)

Create a file containing key.

charmhelpers.contrib.storage.linux.ceph.create_keyring(service, key)

Create a new Ceph keyring containing key.

charmhelpers.contrib.storage.linux.ceph.create_pool(service, name, replicas=3)

Create a new RADOS pool.

charmhelpers.contrib.storage.linux.ceph.create_rbd_image(service, pool, image, sizemb)

Create a new RADOS block device.

charmhelpers.contrib.storage.linux.ceph.delete_keyring(service)

Delete an existing Ceph keyring.

charmhelpers.contrib.storage.linux.ceph.delete_pool(service, name)

Delete a RADOS pool from ceph.

charmhelpers.contrib.storage.linux.ceph.ensure_ceph_keyring(service, user=None, group=None)

Ensures a ceph keyring is created for a named service and optionally ensures user and group ownership.

Returns False if no ceph key is available in relation state.

charmhelpers.contrib.storage.linux.ceph.ensure_ceph_storage(service, pool, rbd_img, sizemb, mount_point, blk_device, fstype, system_services=, []replicas=3)

NOTE: This function must only be called from a single service unit for the same rbd_img otherwise data loss will occur.

Ensures given pool and RBD image exists, is mapped to a block device, and the device is formatted and mounted at the given mount_point.

If formatting a device for the first time, data existing at mount_point will be migrated to the RBD device before being re-mounted.

All services listed in system_services will be stopped prior to data migration and restarted when complete.

charmhelpers.contrib.storage.linux.ceph.filesystem_mounted(fs)

Determine whether a filesytems is already mounted.

charmhelpers.contrib.storage.linux.ceph.get_ceph_nodes()

Query named relation ‘ceph’ to determine current nodes.

charmhelpers.contrib.storage.linux.ceph.get_osds(service)

Return a list of all Ceph Object Storage Daemons currently in the cluster.

charmhelpers.contrib.storage.linux.ceph.image_mapped(name)

Determine whether a RADOS block device is mapped locally.

charmhelpers.contrib.storage.linux.ceph.install()

Basic Ceph client installation.

charmhelpers.contrib.storage.linux.ceph.make_filesystem(blk_device, fstype='ext4', timeout=10)

Make a new filesystem on the specified block device.

charmhelpers.contrib.storage.linux.ceph.map_block_storage(service, pool, image)

Map a RADOS block device for local use.

charmhelpers.contrib.storage.linux.ceph.modprobe(module)

Load a kernel module and configure for auto-load on reboot.

charmhelpers.contrib.storage.linux.ceph.place_data_on_block_device(blk_device, data_src_dst)

Migrate data in data_src_dst to blk_device and then remount.

charmhelpers.contrib.storage.linux.ceph.pool_exists(service, name)

Check to see if a RADOS pool already exists.

charmhelpers.contrib.storage.linux.ceph.rbd_exists(service, pool, rbd_img)

Check to see if a RADOS block device exists.

charmhelpers.contrib.storage.linux.loopback module
charmhelpers.contrib.storage.linux.loopback.create_loopback(file_path)

Create a loopback device for a given backing file.

Returns:str: Full path to new loopback device (eg, /dev/loop0)
charmhelpers.contrib.storage.linux.loopback.ensure_loopback_device(path, size)

Ensure a loopback device exists for a given backing file path and size. If it a loopback device is not mapped to file, a new one will be created.

TODO: Confirm size of found loopback device.

Returns:str: Full path to the ensured loopback device (eg, /dev/loop0)
charmhelpers.contrib.storage.linux.loopback.loopback_devices()

Parse through ‘losetup -a’ output to determine currently mapped loopback devices. Output is expected to look like:

/dev/loop0: [0807]:961814 (/tmp/my.img)
Returns:dict: a dict mapping {loopback_dev: backing_file}
charmhelpers.contrib.storage.linux.lvm module
charmhelpers.contrib.storage.linux.lvm.create_lvm_physical_volume(block_device)

Initialize a block device as an LVM physical volume.

Parameters:block_device – str: Full path of block device to initialize.
charmhelpers.contrib.storage.linux.lvm.create_lvm_volume_group(volume_group, block_device)

Create an LVM volume group backed by a given block device.

Assumes block device has already been initialized as an LVM PV.

Parameters:volume_group – str: Name of volume group to create.
Block_device:str: Full path of PV-initialized block device.
charmhelpers.contrib.storage.linux.lvm.deactivate_lvm_volume_group(block_device)

Deactivate any volume gruop associated with an LVM physical volume.

Parameters:block_device – str: Full path to LVM physical volume
charmhelpers.contrib.storage.linux.lvm.is_lvm_physical_volume(block_device)

Determine whether a block device is initialized as an LVM PV.

Parameters:block_device – str: Full path of block device to inspect.
Returns:boolean: True if block device is a PV, False if not.
charmhelpers.contrib.storage.linux.lvm.list_lvm_volume_group(block_device)

List LVM volume group associated with a given block device.

Assumes block device is a valid LVM PV.

Parameters:block_device – str: Full path of block device to inspect.
Returns:str: Name of volume group associated with block device or None
charmhelpers.contrib.storage.linux.lvm.remove_lvm_physical_volume(block_device)

Remove LVM PV signatures from a given block device.

Parameters:block_device – str: Full path of block device to scrub.
charmhelpers.contrib.storage.linux.utils module
charmhelpers.contrib.storage.linux.utils.is_block_device(path)

Confirm device at path is a valid block device node.

Returns:boolean: True if path is a block device, False if not.
charmhelpers.contrib.storage.linux.utils.is_device_mounted(device)

Given a device path, return True if that device is mounted, and False if it isn’t.

Parameters:device – str: Full path of the device to check.
Returns:boolean: True if the path represents a mounted device, False if it doesn’t.
charmhelpers.contrib.storage.linux.utils.zap_disk(block_device)

Clear a block device of partition table. Relies on sgdisk, which is installed as pat of the ‘gdisk’ package in Ubuntu.

Parameters:block_device – str: Full path of block device to clean.

charmhelpers.contrib.templating package

charmhelpers.contrib.templating.contexts module

A helper to create a yaml cache of config with namespaced relation data.

charmhelpers.contrib.templating.contexts.dict_keys_without_hyphens(a_dict)

Return the a new dict with underscores instead of hyphens in keys.

charmhelpers.contrib.templating.contexts.juju_state_to_yaml(yaml_path, namespace_separator=':', allow_hyphens_in_keys=True, mode=None)

Update the juju config and state in a yaml file.

This includes any current relation-get data, and the charm directory.

This function was created for the ansible and saltstack support, as those libraries can use a yaml file to supply context to templates, but it may be useful generally to create and update an on-disk cache of all the config, including previous relation data.

By default, hyphens are allowed in keys as this is supported by yaml, but for tools like ansible, hyphens are not valid [1].

[1] http://www.ansibleworks.com/docs/playbooks_variables.html#what-makes-a-valid-variable-name

charmhelpers.contrib.templating.contexts.update_relations(context, namespace_separator=':')

Update the context with the relation data.

charmhelpers.contrib.templating.pyformat module

Templating using standard Python str.format() method.

charmhelpers.contrib.templating.pyformat.render(template, extra={}, **kwargs)

Return the template rendered using Python’s str.format().

charmhelpers.contrib.unison package

charmhelpers.contrib.unison.collect_authed_hosts(peer_interface)

Iterate through the units on peer interface to find all that have the calling host in its authorized hosts list

charmhelpers.contrib.unison.create_private_key(user, priv_key_path)
charmhelpers.contrib.unison.create_public_key(user, priv_key_path, pub_key_path)
charmhelpers.contrib.unison.ensure_user(user, group=None)
charmhelpers.contrib.unison.get_homedir(user)
charmhelpers.contrib.unison.get_keypair(user)
charmhelpers.contrib.unison.run_as_user(user, cmd, gid=None)
charmhelpers.contrib.unison.ssh_authorized_peers(peer_interface, user, group=None, ensure_local_user=False)

Main setup function, should be called from both peer -changed and -joined hooks with the same parameters.

charmhelpers.contrib.unison.sync_path_to_host(path, host, user, verbose=False, cmd=None, gid=None, fatal=False)

Sync path to an specific peer host

Propagates exception if operation fails and fatal=True.

charmhelpers.contrib.unison.sync_to_peer(host, user, paths=None, verbose=False, cmd=None, gid=None, fatal=False)

Sync paths to an specific peer host

Propagates exception if any operation fails and fatal=True.

charmhelpers.contrib.unison.sync_to_peers(peer_interface, user, paths=None, verbose=False, cmd=None, gid=None, fatal=False)

Sync all hosts to an specific path

The type of group is integer, it allows user has permissions to operate a directory have a different group id with the user id.

Propagates exception if any operation fails and fatal=True.

charmhelpers.contrib.unison.write_authorized_keys(user, keys)
charmhelpers.contrib.unison.write_known_hosts(user, hosts)

charmhelpers.fetch package

charmhelpers.fetch.archiveurl module

class charmhelpers.fetch.archiveurl.ArchiveUrlFetchHandler

Bases: charmhelpers.fetch.BaseFetchHandler

Handler to download archive files from arbitrary URLs.

Can fetch from http, https, ftp, and file URLs.

Can install either tarballs (.tar, .tgz, .tbz2, etc) or zip files.

Installs the contents of the archive in $CHARM_DIR/fetched/.

can_handle(source)
download(source, dest)

Download an archive file.

Parameters:
  • source (str) – URL pointing to an archive file.
  • dest (str) – Local path location to download archive file to.
download_and_validate(url, hashsum, validate='sha1')
install(source, dest=None, checksum=None, hash_type='sha1')

Download and install an archive file, with optional checksum validation.

The checksum can also be given on the source URL’s fragment. For example:

handler.install('http://example.com/file.tgz#sha1=deadbeef')
Parameters:
  • source (str) – URL pointing to an archive file.
  • dest (str) – Local destination path to install to. If not given, installs to $CHARM_DIR/archives/archive_file_name.
  • checksum (str) – If given, validate the archive file after download.
  • hash_type (str) – Algorithm used to generate checksum. Can be any hash alrgorithm supported by hashlib, such as md5, sha1, sha256, sha512, etc.
charmhelpers.fetch.archiveurl.splitpasswd(user)

urllib.splitpasswd(), but six’s support of this is missing

charmhelpers.fetch.archiveurl.splituser(host)

urllib.splituser(), but six’s support of this seems broken

charmhelpers.fetch.bzrurl module

exception charmhelpers.fetch.AptLockError

Bases: exceptions.Exception

class charmhelpers.fetch.BaseFetchHandler

Bases: object

Base class for FetchHandler implementations in fetch plugins

base_url(url)

Return url without querystring or fragment

can_handle(source)

Returns True if the source can be handled. Otherwise returns a string explaining why it cannot

install(source)

Try to download and unpack the source. Return the path to the unpacked files or raise UnhandledSource.

parse_url(url)
exception charmhelpers.fetch.SourceConfigError

Bases: exceptions.Exception

exception charmhelpers.fetch.UnhandledSource

Bases: exceptions.Exception

charmhelpers.fetch.add_source(source, key=None)

Add a package source to this system.

@param source: a URL or sources.list entry, as supported by add-apt-repository(1). Examples:

ppa:charmers/example
deb https://stub:key@private.example.com/ubuntu trusty main
In addition:
‘proposed:’ may be used to enable the standard ‘proposed’ pocket for the release. ‘cloud:’ may be used to activate official cloud archive pockets, such as ‘cloud:icehouse’ ‘distro’ may be used as a noop

@param key: A key to be added to the system’s APT keyring and used to verify the signatures on packages. Ideally, this should be an ASCII format GPG public key including the block headers. A GPG key id may also be used, but be aware that only insecure protocols are available to retrieve the actual public key from a public keyserver placing your Juju environment at risk. ppa and cloud archive keys are securely added automtically, so sould not be provided.

charmhelpers.fetch.apt_cache(in_memory=True)

Build and return an apt cache

charmhelpers.fetch.apt_hold(packages, fatal=False)

Hold one or more packages

charmhelpers.fetch.apt_install(packages, options=None, fatal=False)

Install one or more packages

charmhelpers.fetch.apt_purge(packages, fatal=False)

Purge one or more packages

charmhelpers.fetch.apt_update(fatal=False)

Update local apt cache

charmhelpers.fetch.apt_upgrade(options=None, fatal=False, dist=False)

Upgrade all packages

charmhelpers.fetch.configure_sources(update=False, sources_var='install_sources', keys_var='install_keys')

Configure multiple sources from charm configuration.

The lists are encoded as yaml fragments in the configuration. The frament needs to be included as a string. Sources and their corresponding keys are of the types supported by add_source().

Example config:
install_sources: |
install_keys: |
  • null
  • “a1b2c3d4”

Note that ‘null’ (a.k.a. None) should not be quoted.

charmhelpers.fetch.filter_installed_packages(packages)

Returns a list of packages that require installation

charmhelpers.fetch.install_from_config(config_var_name)
charmhelpers.fetch.install_remote(source, *args, **kwargs)

Install a file tree from a remote source

The specified source should be a url of the form:
scheme://[host]/path[#[option=value][&...]]

Schemes supported are based on this modules submodules. Options supported are submodule-specific. Additional arguments are passed through to the submodule.

For example:

dest = install_remote('http://example.com/archive.tgz',
                      checksum='deadbeef',
                      hash_type='sha1')

This will download archive.tgz, validate it using SHA1 and, if the file is ok, extract it and return the directory in which it was extracted. If the checksum fails, it will raise charmhelpers.core.host.ChecksumError.

charmhelpers.fetch.plugins(fetch_handlers=None)

charmhelpers.payload package

charmhelpers.payload.archive module

exception charmhelpers.payload.archive.ArchiveError

Bases: exceptions.Exception

charmhelpers.payload.archive.archive_dest_default(archive_name)
charmhelpers.payload.archive.extract(archive_name, destpath=None)
charmhelpers.payload.archive.extract_tarfile(archive_name, destpath)

Unpack a tar archive, optionally compressed

charmhelpers.payload.archive.extract_zipfile(archive_name, destpath)

Unpack a zip file

charmhelpers.payload.archive.get_archive_handler(archive_name)

charmhelpers.payload.execd module

charmhelpers.payload.execd.default_execd_dir()
charmhelpers.payload.execd.execd_module_paths(execd_dir=None)

Generate a list of full paths to modules within execd_dir.

charmhelpers.payload.execd.execd_preinstall(execd_dir=None)

Run charm-pre-install for each module within execd_dir.

charmhelpers.payload.execd.execd_run(command, execd_dir=None, die_on_error=False, stderr=None)

Run command for each module within execd_dir which defines it.

charmhelpers.payload.execd.execd_submodule_paths(command, execd_dir=None)

Generate a list of full paths to the specified command within exec_dir.

Tools for working with files injected into a charm just before deployment.

charmhelpers.cli package

charmhelpers.cli.host module

charmhelpers.cli.host.mounts()

List mounts

class charmhelpers.cli.CommandLine

Bases: object

argument_parser = None
formatter = None
run()

Run cli, processing arguments and executing subcommands.

subcommand(command_name=None)

Decorate a function as a subcommand. Use its arguments as the command-line arguments

subcommand_builder(command_name, description=None)

Decorate a function that builds a subcommand. Builders should accept a single argument (the subparser instance) and return the function to be run as the command.

subparsers = None
class charmhelpers.cli.OutputFormatter(outfile=<open file '<stdout>', mode 'w' at 0x7fc58b25d150>)

Bases: object

add_arguments(argument_parser)
csv(output)

Output data as excel-compatible CSV

format_output(output, fmt='raw')
json(output)

Output data in JSON format

py(output)

Output data as a nicely-formatted python data structure

raw(output)

Output data as raw string (default)

supported_formats
tab(output)

Output data in excel-compatible tab-delimited format

yaml(output)

Output data in YAML format

charmhelpers.cli.describe_arguments(func)

Analyze a function’s signature and return a data structure suitable for passing in as arguments to an argparse parser’s add_argument() method.

Indices and tables