Warning

As of 20 May 2018, this project has been split into the core framework mqttgateway, and the interfaces available, for example musiccast2mqtt. Head to those repos for the updates projects.

Welcome to MQTT_Gateways

mqtt_gateways is a python wrapper to build consistent gateways to MQTT networks.

What it does:

• it deals with all the boilerplate code to manage an MQTT connection, to load configuration and mapping data, and to create log handlers,
• it encapsulates the interface in a class that needs only 2 methods __init__ and loop,
• it creates an intuitive messaging abstraction layer between the wrapper and the interface,
• it isolates the syntax and keywords of the MQTT network from the internals of the interface.

Who is it for:

Developers of MQTT networks in a domestic environment, or smart homes, looking to adopt a definitive syntax for their MQTT messages and to build gateways with their devices that are not MQTT enabled.

Available gateways

The repository contains some already developed gateways to existing systems. The currently available gateways are:

• dummy: the template; check the mqtt_gateways.dummy documentation.
• entry: example used for the tutorial; check it here.

• C-Bus: gateway to the Clipsal-Schneider C-Bus system, via its PCI Serial Interface.

  Check the C-Bus documentation.

Contents

Overview

Objective

When setting up an IoT eco-system with a lot of different devices, it becomes quickly problematic to have them talking to each other smoothly. There are a number of choices to make in order for this to happen. This project assumes that some of those choices have been made: using MQTT as the messaging transport. What this project does is helping in the next set of choices to make: choosing a consistent messaging model and defining its implementation via a MQTT syntax.

If all the devices involved already communicate via MQTT, this project can only help with its proposed syntax for MQTT messages. If some devices can’t communicate natively via MQTT, then this project proposes a python wrapper that should facilitate writing the gateway between these devices and the MQTT network. This gateway can then run as a service on a machine that is connected to these devices via whatever interface is available: serial, bluetooth, TCP, or else.

Concepts

This project has 2 parts:

1. The definition of the messaging model: the project has its own model for messages which is adapted to domestic IoT environments. It is an abstraction layer that defines a message by many attributes and not only by destination and content.
2. The implementation of this model through a python wrapper to communicate via MQTT networks. The wrapper takes care of the addressing syntax and the commands translation via mapping data provided by the interface.

For a more in-depth information, go to Concepts.

Usage

This project is provided with the core application (the wrapper), and an example interface (the dummy interface) that does not interface with anything but shows how the system works. The developer can then write its own interface by using the dummy interface as a template. There are also some already developped interfaces available in the repository.

Installation

The installation involves a copy of the repository and the setting of some configuration parameters. The only dependency is the paho.mqtt library.

For the full installation guide, go to Installation.

Develop your interface

The interface is made of a class that has to define the 2 methods __init__ (to initialise the interface) and loop (called periodically to do whatever needs to be done to interact with the devices), very much like an Arduino script setup and loop functions.

The loop method communicates with the application via 2 lists of messages (an incoming and an outgoing one). It reads the incoming list for commands from the MQTT environment and writes into the outgoing list any updates on status or commands sent from the devices to the rest of the network.

In the most classic example, a serial interface is the only way to communicate with the device. The __init__ method would initialise the serial port and the loop method would write to the serial port any command received from the application through the incoming message list, and read the serial port for messages from the device to be forwarded to the application.

If needed, a mapping JSON file can store the correspondence between the MQTT keywords and the internal keywords. This feature is available in case the MQTT syntax needs to change.

For a complete guide on how to develop an interface, go to Tutorial.

For a more detailed description of the project, go to Project Description.

Installation

Copying the repository

To install this application just copy the github repository on your machine. More precisely:

If you use a command-line only linux system:

• change directory to the location you want this application to be. It could go under /usr/local/bin/ for example, or your home directory /home/your_username/. Let’s call this directory app_dir/ for future reference.
• download the zipped file from GitHub, unzip it and delete it:

wget http://github.com/ppt000/mqtt_gateways/archive/master.zip
unzip master.zip
rm master.zip

If you use a window system:

• point your browser to GitHub, making sure that you are on the master branch of the mqtt_gateways repository.
• use the Download Zip button or menu from that page to download the zipped file.
• Unzip the file into the location you want this application to be, let’s call it app_dir.

There is now under app_dir/ a directory called mqtt_gateways_master or something similar. Inside it, there are all the files needed for the application.

The only non-standard dependency is the paho.mqtt library. Please install it if you do not have it already in your environment, using pip for example.

The directory structure of the relevant files should look like this:

The core engine of the project is the gateway sub-package with the main module start_gateway.py that initialises everything and launches the main loop. The mqtt_map.py module defines a class for internal messages and a MsgMap class for translation methods between internal and MQTT messages. These methods rely on mapping data to be provided by the developer to be discussed later.

The utils sub-package is a set of utility functions.

The dummy sub-package is the first interface. It doesn’t do anything except helping to check the set-up and understand the inner workings of the application.

The data directory contains all the data files for all the interfaces. These are usually the configuration files and the mapping data files.

Configuration

The configuration file has a standard INI syntax, with sections identified by [SECTION] and options within sections identified by option=value. The interface of the gateway can use the [INTERFACE] section where any number of options can be inserted and will be made available to the application through a dictionary initialised with all the option:value pairs.

In the case of the dummy gateway the configuration file is just there to give the address of the MQTT broker. Edit the dummy2mqtt.conf file in the [MQTT] section:

[MQTT]
host: 127.0.0.1
#port: 1883

The address of the MQTT broker should be provided in the same format as expected by the paho.mqtt library, usually a raw IP address (192.168.1.55 for example) if the broker is on your local network, or an http address (not tested) if your broker is in the cloud. The default port is 1883, if it is different it can also be indicated in the configuration file.

Authentication is not available at this stage.

Note

The application is supposed to run in an MQTT enabled environment as the whole idea is to communicate with other devices via MQTT. Therefore there should be a MQTT broker. Ideally the MQTT broker is in the local network, even if a broker in the cloud is acceptable. If you are just testing this application and do not have an MQTT broker, you can use a public one, but the idea is that, down the line, you set up your own local broker.

For more details about the .conf file, defaults and command line arguments, go to Configuration.

Launch

The application should be launched from the root directory; in our case it is the first mqtt_gateways directory. From there, type:

python -m mqtt_gateways.dummy.dummy2mqtt ../data/

The ../data/ argument indicates where the configuration file is.

The application only outputs 1 line to start with: it indicates the location of the log file. Thereafter it only outputs errors, if any, so if nothing happens it is a good sign. More information can be found in the log file, which in our case is located inside the data directory, as long as the configuration file has been used as is. Let the process run a minute or so and check the log file. It should start with a banner message to indicate the application has started, then a list of the full configuration used. Logs from previous runs are kept so make sure to ‘start from the end’ of the file to read the latest logs. If the MQTT connection is successful it should say so as well as displaying the topics to which the application has subscribed. Thereafter, there should be some DEBUG level logs to indicate the messages received, if any (there should be none at this stage).

First Run

After the start-up phase, the dummy interface logs (at a DEBUG level) any MQTT messages it receives and emits a unique message every 30 seconds. Start your favourite MQTT monitor app (I use the excellent mqtt-spy). Connect to your MQTT broker and subscribe to the topic:

home/+/dummy/+/+/+/C

You should see the messages arriving every 30 seconds in the MQTT monitor, as well as in the log. Publish now a message from the MQTT monitor:

topic: home/lighting/dummy/office/undefined/me/C
payload: LIGHT_ON

You should see in the log that the message has been received by the gateway, and that it has been processed correctly, meaning that even if it does not do anything, the translation methods have worked.

The mapping data

The mapping data is the link between MQTT and the internal language of the interface. It maps every keyword in the MQTT vocabulary into the equivalent keyword in the interface. This mapping is a very simple one-to-one relationship for every keyword, and its use is only to isolate the internal code from any changes in the MQTT vocabulary. For the dummy interface, the mapping data is provided by the text file dummy_map.json in the data folder. It’s just there as a template, as, once again, the dummy interface really doesn’t do anything. Note that the map file also contains the topics that the interface should subscribe to.

Concepts

The message model

The primary use case for this project is a domestic environment with multiple devices of any type: lights, audio video components, security devices, heating, air conditioning, controllers, keypads, etc… For many (good) reasons, MQTT has been selected as the communication protocol. But only a few, if any, devices are MQTT enabled. Even for those devices that communicate natively through MQTT, agreeing on a syntax that make them exchange messages coherently is not easy.

Example

In the example below, our smart home has some lighting connected in four different rooms through a proprietary network, four audio-video devices connected through another proprietary network, and some other devices that are already MQTT-enabled, but which still need to speak a common language.

One of the objectives of this project is not only to define a common MQTT syntax, but also to make it as intuitive as possible. Ideally, a human should be able to write a MQTT message off-hand and operate successfully any device in the network.

Message Addressing

The first step of any message is to define its destination. A flexible addressing model should allow for a heuristic approach based on a combination of characteristics of the recipient, on top of the standard deterministic approach (e.g. a unique device id). Four characteristics are usually considered:

• the function of the device: lighting, security, audio-video, etc;
• its location;
• its gateway: which application is managing that device, if any;
• the name of the device.

In our example, a MQTT view shows how those four characteristics define all the devices in the network. The 2 gateways are also added.

Some considerations about those four characteristics:

• not all four characteristics need to be provided to address succesfully a device;
• the device name can be generic (e.g. spotlight) or specific and unique within the network (e.g. lightid1224); in the generic case, obviously other characteristics are needed to address the device.
• any device can have more than one value for each characteristics, particularly the function and device ones (it is unlikely for the gateway and location characteristics);
• the location is important and probably the most intuitive characteristic of all; preferably it should represent the place where the device operates and not where it is physically located (e.g. an audio amplifier might be in the basement but it powers speakers in the living room; the location should be the living room); the location might even not be defined (e.g. to address the security system of the house, or an audio network player that can broadcast to multiple channels).
• the gateway is the most deterministic characteristic (alongside a unique device id); this should be the chosen route for fast and unambiguous messaging.
• the function is another important intuitive characteristic; not only it helps in addressing devices (combined with a location for example), but it should also help to clarify ambiguous commands (ON with lighting or with audiovideo means different things). However things can get more complicated if a device has more than one function; this should be allowed, it is up to the gateway to make sure any ambiguity is resolved from the other characteristics.

Those four characteristics should ensure that the messaging model is flexible enough to be heuristic or deterministic. A gateway will decide how flexible it wants to be. If it has enough bandwidth, it can decide to subscribe to all lighting messages and then parse all messages received to check if they are actually addressed to it. Or it can subscribe only to messages addressed specifically to itself (through the gateway name), restricting access only to senders that know the name of that gateway (arguably not a very intuitive option).

Message Content

The content of a message in the context of domestic IoT can be modelled in many different ways. This project splits it into 3 characteristics:

• a type with 2 possible values: command for messages that are requiring an action to be performed, or status for messages that only broadcast a state;
• an action that indicates what to do or what the status is, or is referring to;
• a set of arguments that might complete the action characteristic.

The key characteristic here is the action, a string that can represent anything. Indeed the message content could be limited to it if the string contained all the information needed, like SWITCH_LIGHT_ON, CHANGE_VOLUME_+4, or REPORT_TEMPERATURE. However, separating those 3 elements eases the processing of internal messages in the code.

Message Source

The sender, which can be a device or another gateway for example, is an optional characteristic in our message model. It can be very useful in answering status requests in a targeted way, for example.

Bridging MQTT and the interface

There are therefore a total of 8 characteristics in our message model:

• function,
• gateway,
• location,
• device,
• type,
• action,
• argument of action,
• source.

They are all strings except type which can only have 2 predefined values. They are all the fields that can appear in a MQTT message, either in the topic or in the payload. They are all attributes of the internal message class that is used to exchange messages between the core of the application (the wrapper) and the interface being developed. They are all the characteristics available to the developer to code its interface.

The internal message class

The internal message class internalMsg defines the objects stored in the lists shared by the application core and the interface. It is supposed to be the most useful representation of a message for the interface code. All that the framework does is parse MQTT messages into internal ones, and back. The framework therefore defines the MQTT syntax by the way it converts the messages.

The conversion process

This conversion process happens inside the class msgMap with the methods MQTT2Internal() and Internal2MQTT(). These methods achieve 2 things:

• map the keywords for every characteristic between the MQTT vocabulary and the internal one; this is done via a simple dictionary initialised by a mapping file,
• define intrinsically the syntax of the MQTT messages in the way the various characteristics are positioned within the MQTT topic and payload.

The MQTT syntax

The topic is structured like this:

root/function/gateway/location/device/source/type

where root can be anything the developer wants (home for example) and type can be only C or S.

The payload is simply the action alone if there are no arguments:

action_name

or the action with the arguments all in a JSON string like this:

{"action":"action_name","arg1":"value1","arg2":"value2",...}

where the first action key is written as is and the other argument keys can be chosen by the developer and will be simply copied in the argument dictionary.

The mapping data

The conversion between MQTT keywords and internal ones is based on a simple one-to-one relationship table for each characteristic (all except type) . It ensures that whatever keyword is used in the interface code is not affected by any change in the MQTT vocabulary. For example, let’s assume a location name in the MQTT vocabulary is basement and is related to the internal constant BASEMENT used inside the interface code. If for some reason the name in the MQTT vocabulary needs to be changed to lowergroundfloor, this can be done in the mapping table without touching the interface code. It is a minor feature but it helps to really separate the MQTT world from the internal interface.

Currently the mapping data is provided by a JSON formatted file. The JSON schema mqtt_map_schema.json is available in the gateway package. New JSON mapping files can be tested against this schema (I use the online validation tool https://www.jsonschemavalidator.net/) The mapping file also contains the topics to subscribe to and the root token for all the topics.

As the schema shows, all the keyword types can (but do not have to) be mapped: function, gateway, location, device, source, action, argument keys and argument values. However, to give more flexibility, there are 3 mapping options available for each type:

• none: the keywords are left unchanged, so there is no need to provide the mapping data;
• strict: the conversion of the keywords go through a map, and any missing keyword raises an error, and the corresponding message is probably ignored;
• loose: like strict except that missing keywords do not raise any error but are passed unchanged.

Tutorial

Let’s go through a practical example, with a very simple protocol.

The Need

The gate of the house has an entry system, or intercom. Visitors push the bell button, and (if all goes well…) after a brief conversation someone let them in by pushing a gate release button in the house. Residents have a code to let themselves in: they enter the code and the system releases the gate.

It would be nice to receive messages about these events, so that other events can be triggered (like switching on lights). It would also be nice to trigger the gate release independently of the entry system.

The Solution

We assume the system exposes the electrical contacts that operate the bell and the gate. A micro-controller (an Arduino for example), can sense the electrical contacts going HIGH or LOW and can communicate these levels to a Raspberry Pi (or any other computer) via the USB connection in serial mode for example. The micro-controller can also be told to switch ON or OFF a relay to release the gate. We will call Entry System the combination of the actual entry system with the micro-controller physical interface. Note: of course a Raspberry Pi could sense directly the electrical contacts without being shielded by another board. However this use-case suits the tutorial, and is probably more reliable in the long run.

Implementation

The micro-controller is programmed to communicate with very simple messages for each event: each message is a pair of numbers (in ASCII), the first indicating which contact the message is about and the second indicating its state. With 2 contacts (the bell or the gate), and 2 states (ON or OFF), there are only 4 messages to deal with: 10, 11, 20 and 21.

More precisely, the micro-controller:

• sends a message when a contact goes ON (11 or 21) and another one when it goes off (10 or 20)
• can also receive and process messages, but only the one triggering the gate release makes sense (21); indeed, there is no need to process 11 or 10 as there is no need to operate the bell via MQTT, and there is no need to process the gate release OFF message (20) as the micro-controller turns the gate release OFF automatically after 3 seconds, for security.

The next step is therefore to code the interface for the computer connected to the micro-controller. The interface will be plugged into our wrapper to make the gateway to MQTT. Let’s call the gateway entry. This will be the label used in all the names in the project (packages, modules, folders, class, configuration and mapping files).

The map file

A good place to start before coding the interface is to write the map file as it forces to list the functionalities that we want to implement.

Here we want the gateway to broadcast the state changes of the bell (1) and the gate release (2), as well as open the gate when commanded to (3). Additionally, we would like the light at the gate to be switched on when the gate is opened (4) (this could be done in another application that receives the gate open broadcast, but it is useful to show how it can be done inside this gateway). We therefore have the following events to model with our message characteristics (see Concepts).

Model

Event	Function	Gateway	Location	Device	Type	Action
Bell Ring	Security	entry2mqtt	gate_entry	Bell	Status	BELL_ON
Bell End	Security	entry2mqtt	gate_entry	Bell	Status	BELL_OFF
Gate Open	Security	entry2mqtt	gate_entry	Gate	Status	GATE_OPEN
Gate Close	Security	entry2mqtt	gate_entry	Gate	Status or Command	GATE_CLOSE
Light On	Lighting	unknown	gate_entry	unknown	Command	LIGHT_ON
Light Off	Lighting	unknown	gate_entry	unknown	Command	LIGHT_OFF

There a few important points to make here:

• The status messages sent by this gateway need to be as unequivocal as possible. By having the Gateway characteristic set to this gateway name should already make those messages unique.
• Ideally these messages should also be overloaded with information, so that other applications have a range of possibilities for topics to subscribe to, depending of which keywords they happen to know. In the example above, specifying the Device gives an extra layer of identification that is redundant but gives more options for subscription topics.
• The command messages need to embody as much information as possible to ensure they reach their destination. However here we will assume that we do not know the Gateway or the Device that operates the lights. So we only specify the Function, the Location and the Action and hopefully the application in charge of the lighting will receive the message and execute the command.
• by creating this model, we are making some choices like defining 2 devices (Bell and Gate) even if in this case we could have defined the whole Entry System as the device, and let the actions indicate to that device what to do (e.g. GATE_OPEN). Being more specific is probably a better choice.

With this model we can write the map file. As a reminder, the map file is only there to insulate the MQTT keywords from our own interface keywords, so that if there are any changes in the MQTT vocabulary, one only needs to change the map file and not the code. If this gateway is an addition to an already existing MQTT eco-system with the same syntax, then some or maybe all of the keywords are already defined, so the idea of the map is to take those keywords and map them to their corresponding keyword from our code. If this gateway is the first one or if some keywords do not exist, this map file will create those new keywords in the MQTT vocabulary, and other applications will need to know about them to be able to communicate with this gateway.

For the sake of this tutorial, we will use MQTT keywords very similar to the ones used in our model above, apart from the fact that all letters are lowercase for the MQTT keywords (it’s a choice). The only keyword that we will assume already exists is the location corresponding to the gate. Here we assume it is already defined as frontgarden and we will use it for our MQTT equivalent keyword. We also need to add the subscription that we will need for our gateway. Here we only need to receive messages that request the gate to be opened. The topics to subscribe to have to be tight enough so that our gateway does not get flooded with messages that are not addressed to it, but also loose enough to be flexible and not too tied to a rigorous vocabulary.

The map file would then look like this:

{
        "root": "home",
        "topics": ["home/security/+/frontgarden/+/+/C",
        "home/+/entry2mqtt/+/+/+/C",
        "home/+/+/+/entrysystem/+/C"],
        "function": {
                "map": {
                        "security": "Security",
                        "lighting": "Lighting"
                },
                "maptype": "strict"
        },
        "gateway": {
                "map": {
                        "entry2mqtt": "entry2mqtt"
                },
                "maptype": "strict"
        },
        "location": {
                "map": {
                        "frontgarden": "gate_entry"
                },
                "maptype": "strict"
        },
        "device": {
                "map": {
                        "gate": "Gate",
                        "bell": "Bell"
                },
                "maptype": "strict"
        },
        "source": {
                "maptype": "none"
        },
        "action": {
                "map": {
                        "gate_open": "GATE_OPEN",
                        "bell_off": "BELL_OFF",
                        "bell_on": "BELL_ON",
                        "light_off": "LIGHT_OFF",
                        "light_on": "LIGHT_ON",
                        "gate_close": "GATE_CLOSE"
                },
                "maptype": "strict"
        },
        "argkey": {
                "maptype": "none"
        },
        "argvalue": {
                "maptype": "none"
        }
}

Enter all those lines in a file named entry_map.json to be created in the folder mqtt_gateways/data. That’s it for the map file.

The interface

The interface is a Python sub-package of the mqtt_gateways package. Let’s create it in a new folder mqtt_gateways\entry with an empty module __init__.py. In order not to start from scratch, let’s use the dummy interface as a template. Copy dummy_interface.py from the dummy package into the entry package, and change all the dummy instances into entry (in the name of the file as well as inside the file). The actual interface code has to be in the class entryInterface within the module entry_interface.py. It needs to have at least a constructor __init__ and a method called loop.

The constructor

The constructor receives 3 arguments: a dictionary of parameters, a pair of message lists, and the fullpath of the application (which is a non-essential argument but useful for logging purposes or to find extra files if needed).

The dictionary of parameters is loaded with whatever we put in the configuration file in the [INTERFACE] section. It’s up to us to decide what we put in there. Here we probably only need a port or device name in order to open the serial port. We will create the configuration file later, but for now we will assume that there will be an option port:whateveritis in the [INTERFACE] section, so we can retrieve it in our code.

The constructor will generally need to keep the message lists locally so that the loop method can access them, so they will be assigned to local members.

Finally, the constructor will have to initialise the serial communication.

Starting from the template copied above, the only thing to add is the opening of the serial port. Add at the top of the module:

import serial

(you need to have the PySerial library in your environment), and add the following line inside the constructor:

self._ser = serial.Serial(port=port, baudrate=9600, timeout=0.01)

The port variable is already defined in the template (check the code). The baudrate has to be the same as the one set by the micro-controller. Finally the timeout is fundamental. It has to be short enough so that the main loop is not delayed too much. Without timeout, all the serial exchanges will be blocking, which obviously can not work in this context as the loop method need to be processed as fast as possible.

The loop method

This method will be called by the main loop to let our interface to do whatever it needs to do. It needs to execute as fast as possible otherwise it will block the whole process. If really needed, one could implement separate threads here, but most of the time this is overkill.

The loop method should deal with the incoming messages first, execute them if necessary, then read its own system for events and stack them in the outgoing list if there are any.

Use the code in the template to read the incoming messages list and add this code to it to deal with the case where the message is a command to open the gate:

if msg.action == 'GATE_OPEN':
    try:
        self._ser.write('21')
    except serial.SerialException:
        self._logger.info('Problem writing to the serial interface')

Always try to catch any exception that should not disrupt the whole application. Most of them should not be fatal.

Then read the serial interface to see if there are any events:

try:
    data = self._ser.read(2)
except serial.SerialException:
    self._logger.info('Problem reading the serial interface')
    return
if len(data) < 2:
    return

If there is an event, convert it into an internal message and add it to the outgoing message list:

if data[0] == '1':
    device = 'Bell'
    if data[1] == '0':
        action = 'BELL_OFF'
    elif data[1] == '1':
        action = 'BELL_ON'
    else:
        self._logger.info('Unexpected code from Entry System')
        return
elif data[0] == '2':
    device = 'Gate'
    if data[1] == '0':
        action = 'GATE_CLOSE'
    elif data[1] == '1':
        action = 'GATE_OPEN'
    else:
        self._logger.info('Unexpected code from Entry System')
        return
msg = internalMsg(iscmd=False, # it is a status message
                  function='Security',
                  gateway='entry2mqtt',
                  location='gate_entry',
                  device=device,
                  action=action)
self._msgl_out.append(msg)

Finally, let’s send a command to switch on the light in case the gate was opened:

if data == '21':
    msg = internalMsg(iscmd=True,
                      function='Lighting',
                      location='gate_entry',
                      action='LIGHT_ON')
    self._msgl_out.append(msg)

That’s it. Of course one can improve the functionality by putting a timer to switch off the lights after a while for example.

Other coding strategies

The class can be defined as a subclass of Serial in this case. It might be more elegant and it reflects well what that is, i.e. a higher level serial interface to a specific device.

The conversion of the raw messages from the serial interface into internal messages can be done through lookup tables instead of nested ifs, in the same vein as the map file converts MQTT keywords into internal keywords. However that conversion can be more complex to represent, because, for example, a single internal message might need multiple events or commands to be sent to the interface. In this case it is quite simple, and we could have defined a dictionary to help the conversion.

Wrapping it all up

Once the interface is defined, all is left to do is to create the launch script and the configuration file. Those 2 steps are easy using the templates.

Copy the dummy project launch script dummy2mqtt.py and paste it into the entry directory. Change every instance of dummy into entry`, as in the interface steps. If all the naming steps have been respected, the script entry2mqtt.py just created should work.

Go to the mqtt_gateways/data directory, copy the configuration file dummy2mqtt.conf and paste it in the same folder with the name entry2mqtt.py. Edit the file and enter the port option under the [INTERFACE] section:

[INTERFACE]
port=/dev/ttyACM0

Obviously input whatever is the correct name of the port, the one shown is generally the one to use on a Raspberry Pi for the USB serial connection. If you are on Windows, your port should be something like COM3.

If you went through the installation process the MQTT parameters should already be set up, otherwise do so. Other parameters can be left as they are. Check the configuration guide for more details.

Launch

To launch the gateway, goto the root directory mqtt_gateways (the first one). This should be the working directory from where the following command should be run:

python -m mqtt_gateways.entry.entry2mqtt ../data

On Windows, use ..\data as argument.

Done!

Configuration

Note

Coming soon!

Project Decription

Note

This is a work in progress.

This is a more detailed description of the project.

mqtt_gateways package

Module contents

Root package for the mqtt_gateways project. There are no modules in this package.

Subpackages

mqtt_gateways.dummy package

Module contents

A dummy gateway to test the installation setup, the loading of the configuration files, and the basic operation of the core application.

There are 2 modules:

• dummy_interface.py defines the class dummyInterface;
• dummy2mqtt.py which is the application launcher.

Submodules

mqtt_gateways.dummy.dummy2mqtt module

Launcher script for the dummy gateway.

Use this as a template. If the name conventions have been respected, just change all occurrences of dummy into the name of your gateway.

mqtt_gateways.dummy.dummy_interface module

The dummy interface class definition. Use it as a template.

This module defines the class dummyInterface that will be instantiated by the main gateway module. Any other code needed for the interface can be placed here or in other modules, as long as the necessary imports are included of course.

class mqtt_gateways.dummy.dummy_interface.dummyInterface(params, msglist_in, msglist_out)

Bases: object

Doesn’t do anything but provides a template.

The minimum requirement for the interface class is to define 2 public methods:

• the constructor __init__ which takes 4 arguments,
• the loop method.

Parameters:params (dictionary of strings) – contains all the options from the configuration file This dictionary is initialised by the [INTERFACE] section in the configuration file. All the options in that section generate an entry in the dictionary. Use this to pass parameters from the configuration file to the interface, for example the name of a port, or the speed of a serial communication.

loop()

The method called periodically by the main loop.

Place here your code to interact with your system.

mqtt_gateways.gateway package

Module contents

The package representing the core of the application.

There are 4 modules:

• mqtt_client.py defines the child class of the official MQTT Client class of the paho library;
• mqtt_map.py defines the classes internalMsg and msgMap;
• start_gateway.py which contains the script for the application initialisation and main loop.
• configuration.py which contains the default configuration as a string.

Submodules

mqtt_gateways.gateway.mqtt_map module

This module represents the bridge between the internal representation of messages and the MQTT representation.

It defines 2 classes:

• internalMsg is the internal representation of a message
• msgMap is the conversion engine between the internal representation and the MQTT one.

As a reminder, we define the MQTT syntax as follows:

• topic: root/function/gateway/location/device/sender/type-{C or S}
• payload: action or status, in plain text or in query string, e.g. key1=value1&key2=value2&...

class mqtt_gateways.gateway.mqtt_map.internalMsg(iscmd=False, function=None, gateway=None, location=None, device=None, sender=None, action=None, arguments=None)

Bases: object

Defines all the characteristics of an internal message.

Parameters:

• iscmd (bool) – Indicates if the message is a command (True) or a status (False), optional
• function (string) – internal representation of function, optional
• gateway (string) – internal representation of gateway, optional
• location (string) – internal representation of location, optional
• device (string) – internal representation of device, optional
• sender (string) – internal representation of sender, optional
• action (string) – internal representation of action, optional
• arguments (dictionary of strings) – all values should be assumed to be strings, optional

copy()

docstring

str()

Helper function to stringify the class attributes.

reply(response, reason)

Formats the message to be sent as a reply to an existing command

This method is supposed to be used with an existing message that has been received by the interface

class mqtt_gateways.gateway.mqtt_map.MsgList

Bases: Queue.Queue, object

docstring

push(item)

Equivalent to self._list.append(item)

pull()

Equivalent to self._list.pop(0)

class mqtt_gateways.gateway.mqtt_map.mappedFields(function, gateway, location, device, sender, action, argkey, argvalue)

Bases: tuple

action

Alias for field number 5

argkey

Alias for field number 6

argvalue

Alias for field number 7

device

Alias for field number 3

function

Alias for field number 0

gateway

Alias for field number 1

location

Alias for field number 2

sender

Alias for field number 4

class mqtt_gateways.gateway.mqtt_map.msgMap(jsondict)

Bases: object

Contains the mapping data and the conversion methods.

Initialises the 5 maps from the argument mapdata, which is an object that must be readable line by line with a simple iterator. The syntax for mapdata is that each line has to start with one of 6 possible labels (topic, function, gateway, location, device, action) followed by : and then the actual data. If the label is topic then the data should be a valid MQTT topic string, otherwise the data should be a pair of keywords separated by a ,, the first being the MQTT representation of the element and the second being its internal equivalent.

To access the maps use: mqtt_token = maps.*field*.m2i(internal_token) Example: mqtt_token = maps.gateway.m2i(internal_token)

Parameters:jsondata (dictionary) – contains the map data in the agreed format; if None, the NO_MAP structure is used.

class tokenMap(maptype, mapdict=None)

Bases: object

Represents the mapping for a given token or characteristic.

Each instantiation of this class represent the mapping for a given token, and contains the type of mapping, the mapping dictionary if available, and the methods to convert the keywords back and forth between MQTT and internal representation.

m2i(mqtt_token)

docstring

i2m(internal_token)

docstring

sender()

docstring

mqtt2internal(mqtt_msg)

Converts the MQTT message into an internal one.

Parameters:mqtt_msg (mqtt.MQTTMessage) – a MQTT message. Returns:the conversion of the MQTT message Return type:internalMsg object Raises:ValueError – in case of bad MQTT syntax or unrecognised map elements

internal2mqtt(internal_msg)

Converts an internal message into a MQTT one.

Parameters:internal_msg (an internalMsg object) – the message to convert Returns:a full MQTT message where topic syntax is root/function/gateway/location/device/sender/{C or S} and payload syntax is either a plain action or a query string. Return type:a MQTTMessage object Raises:ValueError – in case a token conversion fails

mqtt_gateways.gateway.mqtt_map.test()

docstring

mqtt_gateways.gateway.start_gateway module

Defines the function that starts the gateway and the 3 MQTT callbacks.

This module exposes the main entry points for the framework: the gateway interface class, which is received as an argument by the main function startgateway(), and is instantiated after all the initialisations are done. Note that at the moment of instantiation, the configuration file should be loaded, so anything that is written inside the [INTERFACE] section will be passed on to the class constructor. This way custom configuration settings can be passed on to the gateway interface.

mqtt_gateways.gateway.start_gateway.startgateway(gateway_interface)

Initialisation and main loop.

Initialises the configuration and the logger, starts the interface, starts the MQTT communication then starts the main loop. The loop calls the MQTT loop method to process any message from the broker, then calls the gateway interface loop, and finally publishes all MQTT messages queued.

Notes on MQTT behaviour:

• if not connected, the loop and publish methods will not do anything, but raise no errors either.
• it seems that the loop method handles always only one message per call.

Notes on the loading of data: the configuration file is the only file that needs to be either passed as argument through the command line, or the default settings will be used (and probably fail as one needs at least a valid MQTT broker for the application to start). All other filenames will be in the configuration file itself. The configuration file name can be passed as the first argument in the command line. If the argument is a directory (i.err. ends with a slash) then it is appended with the default file name. If it is a path it is checked to see if it is absolute, and if it is not it will be prepended with the path of the calling script. The default file name is the name of the calling script with the .conf extension. The default directory is the directory of the calling script.

Parameters:

gateway_interface (class (not an instance of it!)) – the interface The only requirement is that it should have an appropriate constructor and a loop method.

Raises:

OSError – if any of the necessary files are not found.

The necessary files are the configuration file (which is necessary to define the MQTT broker, at the very least) and the map (for which there can not be any default). It tries to catch most other ‘possible’ exceptions. KeyboardInterrupt should work as there are a few pauses around. Finally, only errors thrown by the provided interface class will not be caught and could terminate the application.

mqtt_gateways.gateway.configuration module

The default configuration settings in a single string constant.

Given how the configuration loader works, only the sections and options declared already here will be considered in any external configuration file. If an external configuration file is read and contains sections and options not included in here, they will be ignored, except for the [INTERFACE] section. The section [INTERFACE] is reserved to the configuration parameters that might be needed by the interface being implemented.

Use this declaration as a template configuration file.

mqtt_gateways.utils package

Module contents

Utilities.

Submodules

mqtt_gateways.utils.throttled_exception module

An exception class that throttles events in case an error is triggered too often.

This exception can be used as a base class instead of Exception. It adds a counter and a timer that allow to silence the error for a while if desired. Only after a given period a trigger is set to indicate that a number of errors have happened and it is time to report them. It creates 2 members:

• trigger is a boolean set to True after the requested lag;
• report is a string giving some more information on top of the latest message.

The code using these exceptions can test the member trigger and decide to silence the error until it is True. At any point one can still decide to use these exceptions as normal ones and ignore the trigger and report members.

Usage:

try:
    some statements that might raise your own exception derived from ThrottledException
except YourExceptionError as err:
    if err.trigger:
        log(err.report)

exception mqtt_gateways.utils.throttled_exception.ThrottledException(msg=None, throttlelag=10, module_name=None)

Bases: exceptions.Exception

Exception class base to throttle events

Parameters:

• msg (string) – the error message, as for usual exceptions, optional
• throttlelag (int) – the lag time in seconds while errors should be throttled, optional
• module_name (string) – the calling module to give extra information, optional

mqtt_gateways.utils.init_logger module

Function to initialise a logger with pre-defined handlers.

mqtt_gateways.utils.init_logger.initlogger(logger, logfiledata, emaildata)

The logger passed as parameter should be sent by the ‘root’ module if hierarchical logging is the objective. The logger is then initialised with the following handlers:

• the standard ‘Stream’ handler will always log level WARN and above;
• a rotating file handler, with fixed parameters (max 50kB, 3 rollover files); the level for this handler is DEBUG if the parameter ‘log_debug’ is True, INFO otherwise; the file name for this log is given by the log_filepath parameter which is used as is; an error message is logged in the standard handler if there was a problem creating the file;
• an email handler with the level set to CRITICAL;

Parameters:

• logger – the actual logger object to be initialised;
• logfiledata (tuple) –

  [0] = logfilepath (string): the log file path,

  if None, file logging is disabled;

  [1] = log_debug (boolean): a flag to indicate

  if DEBUG logging is required, or only INFO;

  [2] = consolelevel (string): the level of log to be sent

  to the console (stdout), or NONE.

• emaildata (tuple) –

  [0] = host (string), [1] = port (int), [2] = address (string),

  if either of those 3 values are None or empty, no email logging is enabled;

  [3] = app_name (string).

Returns:

Nothing

Raises:

any IOErrors thrown by file handling methods are caught.

mqtt_gateways.utils.load_config module

Function to facilitate the loading of configuration parameters. Based on ConfigParser.

mqtt_gateways.utils.load_config.loadconfig(cfg_dflt_string, cfg_filepath)

The configuration is loaded with the help of the python library ConfigParser and the following convention.

The default configuration is represented by the string passed as argument cfg_dflt_string. This string is expected to have all the necessary sections and options that the application will need, with their default values. All options need to be listed there, even the ones that HAVE to be updated and have no default value. This default configuration will be usually declared in the caller module as a constant string, and will be used as a template for the actual configuration file.

The function ‘loads’ this default configuration, then checks if the configuration file is available, and if found it grabs only the values from the file that are also present in the default configuration. Anything else in the file is not considered, except for the [INTERFACE] section (see below). The result is a configuration object with all the necessary fields, updated by the file values if present, or with the default values if not. The application can therefore call all fields without index errors.

The exception to the above process in the [INTERFACE] section. The options of this section will be loaded ‘as is’ in the Config object. This section can be used to define ad-hoc options that are not in the default configuration.

Finally, the function updates the option ‘location’ in the section [CONFIG] with the full path of the configuration file used, just in case it is needed later. However it only updates it if it was present in the default configuration string, in the spirit of the above convention. It also ‘logs’ the error in the ‘error’ option of the same section, if any OS exception occurred while opening or reading the configuration file.

Parameters:

• cfg_dflt_string (string) – represents the default configuration.
• cfg_filepath (string) – the path of the configuration file; it is used ‘as is’ and if it is relative there is no guarantee of where it will actually point.

Returns:

object loaded with the parameters.

Return type:

ConfigParser.RawConfigParser object

mqtt_gateways.cbus package

Module contents

Interface to C-BUS via the PCI module from Clipsal-Schneider.

Submodules

mqtt_gateways.cbus.cbus2mqtt module

Starter module for the C-Bus gateway.

mqtt_gateways.cbus.cbus_interface module

This module defines the cbusInterface class.

mqtt_gateways.cbus.cbus_interface._checksum(hexstring)

Returns True if hexstring is a valid hexadecimal string and if the checksum is correct as specified by the C-Bus documentation, otherwise returns False.

mqtt_gateways.cbus.cbus_interface._PATTERN_MonitoredSAL = '05([0-9A-F]{2})3800([0-9A-F]+)'

RegEx patterns to match incoming messages from C-Bus; see docs for terminology.

mqtt_gateways.cbus.cbus_interface._PATTERN_CALReply = '86[0-9A-F]{4}00([0-9A-F]{2})0738([0-9A-F]{2})([0-9A-F]+)'

RegEx patterns to match incoming messages from C-Bus; see docs for terminology.

mqtt_gateways.cbus.cbus_interface._PATTERN_LevelRequest = '\\05FF00730738'

Level request pattern (it is just a string here) - it needs to be followed by the Block Start and then x0D.

mqtt_gateways.cbus.cbus_interface._CBUSMSGMAXLEN = 16

C-Bus actual maximum message length.

mqtt_gateways.cbus.cbus_interface._STATUS_REQUEST_FREQ = 60

Number of seconds between each Status Request - arbitrary.

mqtt_gateways.cbus.cbus_interface._BLOCK_START = ['00', '20', '40', '60', '80', 'A0', 'C0', 'E0']

The 8 different possibilities for _BLOCK_START.

mqtt_gateways.cbus.cbus_interface._MAXARGNUMBER = 2

Maximum number of arguments accepted in a command. Needed for error management.

mqtt_gateways.cbus.cbus_interface._INTERNAL2CBUS = 0

Index for the dictionaries.

mqtt_gateways.cbus.cbus_interface._CBUS2INTERNAL = 1

Index for the dictionaries.

class mqtt_gateways.cbus.cbus_interface.cbusInterface(params, msglist_in, msglist_out)

Bases: mqtt_gateways.cbus.cbus_serial.cbusSerial

Represents the higher layer of communication with the C-Bus interface.

Parameters:

• params (dictionary) – the parameters from the configuration file under the [INTERFACE] section.
• msglists (list) – pair of lists of internalMsg objects; the first list is for the inbound messages, the second one for the outbound messages.
• fullpath (string) – full absolute path of the application.

loop()

The compulsory method called periodically by the gateway.

The loop deals with inbound messages as expected, but also triggers a periodic status request per block as per C-Bus documentation. This is an extra feature to double check that no state change has been missed, but also to get the correct status at startup.

_execute_command(imsg)

Executes the command represented by the internal message.

The internal message might represent a single light to operate on or a whole set of lights (for a whole location for example), in which case there should be many commands to send. The algorithm concatenates the tokens depending on the number of arguments required by the action to create single commands (made of an action byte, an address byte and eventually a level byte), then concatenates those commands so that the maximum amount of messages are sent to the interface. According to the documentation, the maximum length of the code that can be sent to the serial interface is 21 bytes, excluding the leading x5C and the trailing x0D. This means 18 bytes for commands (excluding the ‘053800’ at the beginning). Therefore the length of concatenated commands is tested in order to avoid sending messages that are too long.

Parameters:

imsg (internalMsg object) – the message/command to execute

Raises:

• cbusConnectionError – in case of a problem writing to the serial interface.
• ValueError – when the message can not be converted in a C-Bus action.

_status_request()

Launch status request periodically.

Only one block is requested at a time. Depending on the frequency set, it might take some time to go through all the blocks. If all 8 blocks are used, the whole cycle takes 8 * _STATUS_REQUEST_FREQ to complete. As the initialisation code actually removes the blocks that do not contain any active lights, the actual cycle might be shorter.

_read_bus()

Reads one line from the CBus serial interface and converts it in a command list.

The line read is a string made of characters that are mostly hexadecimal with some special characters as well. It should always have at the end a <cr> and <lf>. The algorithm checks first for special situations (code too short, code ends with correct characters, code is an error or represent a restart request). Then it checks if it is a ‘Monitored SAL’ message, which means that an input unit (keypad or button) has sent a command to operate lights. Then it checks if the code is a ‘CAL Reply’ message, which means it is a reply to a status request and contains the status of one or more lights.

Raises:

• cbusConnectionError – in case of problems reading the serial interface.
• cbusInitError – in case of problems initialising the serial interface.

mqtt_gateways.cbus.cbus_serial module

This module defines the cbusSerial class representing the low-level communication layer with the C-Bus PCI serial interface.

mqtt_gateways.cbus.cbus_serial._s2p(hexs)

Helper to log properly the commands sent to C-BUS. It catches non printable characters and replaces them with hex string in the form ‘xFF’. Note: s2p = string to printable.

Parameters:hexs (string) – hexadecimal string. Returns:the argument with non printable characters replaced by their hex value. Return type:string

mqtt_gateways.cbus.cbus_serial._REQUESTS = [['Interface Options 1 Settings Live', '@2A3001', '8230301E', '~@A3300030', '3230009E'], ['Interface Options 1 Power Up Settings', '@1A4101', '8241300D', '@A3410030', '3241008D'], ['Interface Options 2', '@1A3E01', '823E0040', '@A33E0000', '323E0090'], ['Interface Options 3', '@1A4201', '82420F2D', '@A342000F', '3242008C']]

Hard coded data needed to request and set specific parameters of the C-Bus serial interface, as well as the values of these parameters that are needed for this interface to operate properly. More specifically:

• Options 1: SMART mode ON and MONITOR ON, all else OFF;
• Options 2: all OFF;
• Options 3: PCN, LOCAL_SAL, PUN and EXSTAT ON.

The format is: [Parameter Name for reference, Request code, Desired Answer, Set Command code, Acknowledge code]

mqtt_gateways.cbus.cbus_serial._PAUSE = 0.1

in seconds, to leave time to the PCI to process the commands; completely arbitrary, might be useless.

mqtt_gateways.cbus.cbus_serial._THROTTLELAG = 600

in seconds, lag to throttle communication error with the PCI.

exception mqtt_gateways.cbus.cbus_serial.cbusError

Bases: exceptions.Exception

Local Exception.

exception mqtt_gateways.cbus.cbus_serial.cbusInitError

Bases: exceptions.Exception

Initialisation of the Serial Interface Error.

exception mqtt_gateways.cbus.cbus_serial.cbusConnectionError(msg=None)

Bases: mqtt_gateways.utils.throttled_exception.ThrottledException

Connection Error for the Serial Interface.

class mqtt_gateways.cbus.cbus_serial.cbusSerial(port)

Bases: serial.serialposix.Serial

Represents the low-level communication layer with the C-Bus PCI serial interface.

It is an extension of the pySerial.Serial class. It allows to open the port specifically to communicate with the C-Bus interface, as well as initiate its parameters. The methods defined override the serial library ones; they mostly catch the exceptions and log the errors if any.

Any code using these methods must make sure that values returned are tested and that exceptions are being caught. In those cases probably the port is not working and need to be restarted or the application has to stop. This allows to write non-blocking code that deals with the bad connection only at higher levels, for example in the main loop, rather than at each write or read command.

The constructor tries to open the port but might not succeed. Therefore the class instance is always created but the port might be open or not. Use the serial library methods open() to later open the port in case of failure, and is_open() to test if it is opened already.

Parameters:

• port (string) – port name, passed as is to the Serial library
• full_path (string) – used to ‘hook’ to the root logger; should contain the application name

Raises:

cbusInitError – if the serial interface can not be opened.

readline()

Reads a full line from the serial interface. Overrides parent class method.

Raises:cbusConnectionError – in case of a SerialException from the interface

write(code)

Writes the code to the interface. Overrides parent class method.

Parameters:code (string) – characters or bytes to write to the serial interface Returns:number of characters or bytes written to the interface Return type:int Raises:cbusConnectionError – if there are any problems during the writing process

init_pci_options()

Initialises the parameters of the PCI interface.

This code relies on the _REQUESTS list of parameters. Going through every item of the list, the code asks for the status of a parameter ‘block’, checks the reply against what is expected, requests an update with the correct values in case they aren’t, and checks the reply to the update to make sure it has been accepted.

Raises:cbusInitError – if something fatal happens during this process

mqtt_gateways.cbus.cbus_data module

This module contains the data necessary to decode or encode C-Bus messages.

mqtt_gateways.cbus.cbus_data.LIGHTS = {'Bathroom_Spots': ['0D', 'Bathroom'], 'Bedroom_Bedside': ['0A', 'Bedroom'], 'Bedroom_Socket': ['0B', 'Bedroom'], 'Bedroom_Spots': ['0C', 'Bedroom'], 'DiningRoom_Pendant': ['04', 'DiningRoom'], 'DiningRoom_Socket': ['05', 'DiningRoom'], 'Kitchen_Spots': ['01', 'Kitchen'], 'Kitchen_UnderUnit': ['02', 'Kitchen'], 'LivingRoom_Spots': ['06', 'LivingRoom'], 'LivingRoom_Wall': ['07', 'LivingRoom'], 'Office_Socket': ['08', 'Office'], 'Office_Spots': ['09', 'Office'], 'TVRoom_Spots': ['03', 'TVRoom']}

Dictionary of lights names with their C-Bus codes and their location.

The dictionary is in the form 'internal name of device/light':['C-Bus address', 'location name']. This dictionary allows to build at once the devices dictionary as well as the location one. Obviously the lights names must be unique, as well as the corresponding C-Bus codes. This setup ensures every item (light name, location name, light C-Bus code) appears only once and there are no risks of duplicates.

mqtt_gateways.cbus.cbus_data.ACTIONS = [['LIGHT_ON', ['79']], ['LIGHT_OFF', ['01']], ['LIGHT_LVL', ['02', '%%']], ['RAMP_0S_LVL', ['02', '%%']], ['RAMP_4S_LVL', ['0A', '%%']], ['RAMP_8S_LVL', ['12', '%%']], ['RAMP_12S_LVL', ['1A', '%%']], ['RAMP_20S_LVL', ['22', '%%']], ['RAMP_30S_LVL', ['2A', '%%']], ['RAMP_40S_LVL', ['32', '%%']], ['RAMP_60S_LVL', ['3A', '%%']], ['RAMP_90S_LVL', ['42', '%%']], ['RAMP_2M_LVL', ['4A', '%%']], ['RAMP_3M_LVL', ['52', '%%']], ['RAMP_5M_LVL', ['5A', '%%']], ['RAMP_7M_LVL', ['62', '%%']], ['RAMP_10M_LVL', ['6A', '%%']], ['RAMP_15M_LVL', ['72', '%%']], ['RAMP_17M_LVL', ['7A', '%%']], ['TERMINATERAMP', ['09']], ['LIGHT_LOW', ['02', '55']], ['LIGHT_MEDIUM', ['02', 'AA']]]

List of local actions and their C-Bus hex code equivalent.

Only short commands are used here (as defined in C-Bus documentation) as long commands seem to be only needed for labels. The 3 last bits of short commands indicate the number of arguments required. In practice, only the ON and OFF commands (and TERMINATERAMP, rarely used) require one argument only (the Address) while all the others (the RAMP to LEVEL ones) require two arguments (the Address and the Level to reach). No other commands are allowed.

The list is made of pairs where the first element is the internal name of the action and the second element is another list made of the C-Bus codes representing this action. This list of C-Bus codes has one or two elements. The first one represents the actual action code in C-Bus (ON, OFF, RAMP, …) and the second one represents the LEVEL to reach in the RAMP case. In the case of standard actions, the argument is left variable and has to be communicated as an argument of the action; in this case the convention is to represent it with a %%.

The code allows to create different ways to execute the same action, e.g. switching a light to a medium level can be achieved with the action LIGHT_LVL and a parameter AA or with the custom made action LIGHT_MEDIUM. Add your own actions at the end of the list.

This data is represented in a list because the order here matters to build the reverse dictionary. The reverse dictionary will only contain the standard actions, which means that in the translation from C-Bus to MQTT, the custom-made actions are not taken into account.

mqtt_gateways.cbus.cbus_data.FUNCTIONS = {'Lighting': '38'}

Dictionary of Functions or Applications as defined in C-Bus.

The dictionary is in the form 'internal name':'C-Bus code'. This dictionary should not be modified but can be appended with additional Applications from C-Bus.

mqtt_gateways.cbus.cbus_data.LEVELS = {'55': 'F', '56': 'E', '59': 'D', '5A': 'C', '65': 'B', '66': 'A', '69': '9', '6A': '8', '95': '7', '96': '6', '99': '5', '9A': '4', 'A5': '3', 'A6': '2', 'A9': '1', 'AA': '0'}

Dictionary for the correspondence of levels definition in C-Bus status replies.

From C-Bus documentation. Do not change.

mqtt_gateways.musiccast package

Module contents

Gateway to Yamaha MusicCast devices.

Submodules

mqtt_gateways.musiccast.musiccast2mqtt module

mqtt_gateways.musiccast.musiccast_interface module

mqtt_gateways.musiccast.musiccast_system module

mqtt_gateways.musiccast.musiccast_comm module

Low-level communication module with the MusicCast system.

Reviewed on 16 May 2018 by Paolo.

mqtt_gateways.musiccast.musiccast_comm.set_socket(listen_port)

Instantiates a socket and binds it to the port provided.

Also initialises the 2 module ‘constants’ MCSOCKET and MCPORT.

Parameters:listen_port (int) – the local port to bind to the socket. Raises:CommsError – in case of failure to bind the socket to the port.

mqtt_gateways.musiccast.musiccast_comm.get_event()

Checks the socket for events broadcasted by the MusicCast devices.

TODO: check max length of the events and more than one event could arrive at once

The ‘body’ of the event (see below) is in the form: (‘{“main”:{“power”:”on”},”device_id”:”00A0DED57E83”}’, (‘192.168.1.44’, 38507)) or: (‘{“main”:{“volume”:88},”zone2”:{“volume”:0}, “device_id”:”00A0DED3FD57”}’, (‘192.168.1.42’, 46514))

class mqtt_gateways.musiccast.musiccast_comm.musiccastComm(host)

Bases: object

Manages the low-level calls to the MusicCast devices.

Every instance represents a single live connection to a given MusicCast device, represented simply by a host address.

Parameters:host (string) – the HTTP address for the host, as recognisable by the httplib library.

mcrequest(qualifier, mc_command)

Sends a single HTTP request and returns the response.

This method sends the request and read the response step by step in order to catch properly any error in the process. Currently the requests are always with method = ‘GET’ and version = ‘v1’.

Parameters:

• qualifier (string) – the token in the MusicCast syntax representing either a zone or a source, depending on the type of command sent;
• mc_command (string) – the command to send at the end of the request; it has to include any extra argument if there are any.

Raises:

commsError – in case of any form of Communication Error with the device.

Returns:

the dictionary equivalent of the JSON structure sent back as a reply

from the device.

Return type:

dictionary

mqtt_gateways.musiccast.musiccast_data module

Data for the MusicCast system.

mqtt_gateways.musiccast.musiccast_data.TRANSFORM_ARG = {'action': (<function <lambda> at 0x7fc8495841b8>, <function <lambda> at 0x7fc849584140>), 'input': (<function <lambda> at 0x7fc849584398>, <function <lambda> at 0x7fc849584320>), 'mute': (<function <lambda> at 0x7fc8495f0488>, <function <lambda> at 0x7fc8495f0e60>), 'power': (<function <lambda> at 0x7fc8495f0b90>, <function <lambda> at 0x7fc8495f0668>), 'preset': (<function <lambda> at 0x7fc849584410>, <function <lambda> at 0x7fc849584488>), 'source': (<function <lambda> at 0x7fc8495842a8>, <function <lambda> at 0x7fc849584230>), 'volume': (<function <lambda> at 0x7fc8495f0cf8>, <function <lambda> at 0x7fc8495f0ed8>)}

Transforms arguments from internal keyword to MusicCast keyword and back.

The value for each key is a pair of lambdas; the first one transforms its arguments from internal representation to Musiccast, and the second one does the reverse. The lambdas have to be called by a Zone object.

mqtt_gateways.musiccast.musiccast_data.ACTIONS = {'CD_BACK': <function <lambda> at 0x7fc849584c80>, 'CD_FORWARD': <function <lambda> at 0x7fc849584cf8>, 'CD_PAUSE': <function <lambda> at 0x7fc849584d70>, 'CD_PLAY': <function <lambda> at 0x7fc849584de8>, 'CD_STOP': <function <lambda> at 0x7fc849584e60>, 'GET_INPUTS': <function <lambda> at 0x7fc8495848c0>, 'GET_SOURCES': <function <lambda> at 0x7fc8495849b0>, 'MUTE_OFF': <function <lambda> at 0x7fc8495847d0>, 'MUTE_ON': <function <lambda> at 0x7fc849584758>, 'MUTE_TOGGLE': <function <lambda> at 0x7fc849584848>, 'NETRADIO_PRESET': <function <lambda> at 0x7fc849417140>, 'POWER_OFF': <function <lambda> at 0x7fc849584500>, 'POWER_ON': <function <lambda> at 0x7fc849584578>, 'SET_INPUT': <function <lambda> at 0x7fc849584938>, 'SET_SOURCE': <function <lambda> at 0x7fc849584a28>, 'SET_VOLUME': <function <lambda> at 0x7fc8495845f0>, 'SOURCE_CD': <function <lambda> at 0x7fc849584aa0>, 'SOURCE_NETRADIO': <function <lambda> at 0x7fc849584b18>, 'SOURCE_SPOTIFY': <function <lambda> at 0x7fc849584c08>, 'SOURCE_TUNER': <function <lambda> at 0x7fc849584b90>, 'SPOTIFY_BACK': <function <lambda> at 0x7fc849584f50>, 'SPOTIFY_FORWARD': <function <lambda> at 0x7fc849417050>, 'SPOTIFY_PLAYPAUSE': <function <lambda> at 0x7fc849584ed8>, 'TUNER_PRESET': <function <lambda> at 0x7fc8494170c8>, 'VOLUME_DOWN': <function <lambda> at 0x7fc8495846e0>, 'VOLUME_UP': <function <lambda> at 0x7fc849584668>}

The dictionary with all the data to process the various commands.

It has to be called from an instance of the class Zone.

mqtt_gateways.musiccast.musiccast_data.EVENTS = {'cd': {'play_info_updated': <function <lambda> at 0x7fc8494179b0>, 'play_time': <function <lambda> at 0x7fc849417938>, 'device_status': None}, 'clock': {'settings_updated': None}, 'device_id': None, 'dist': {'dist_info_updated': None}, 'main': {'power': <function <lambda> at 0x7fc8494171b8>, 'mute': <function <lambda> at 0x7fc849417320>, 'status_updated': <function <lambda> at 0x7fc849417398>, 'volume': <function <lambda> at 0x7fc8494172a8>, 'input': <function <lambda> at 0x7fc849417230>, 'signal_info_updated': None}, 'netusb': {'trial_time_left': None, 'play_error': None, 'list_info_updated': None, 'trial_status': None, 'preset_control': None, 'recent_info_updated': None, 'preset_info_updated': <function <lambda> at 0x7fc849417848>, 'play_info_updated': <function <lambda> at 0x7fc8494178c0>, 'multiple_play_errors': None, 'account_updated': None, 'play_time': <function <lambda> at 0x7fc8494177d0>, 'play_message': <function <lambda> at 0x7fc849417758>}, 'system': {'speaker_settings_updated': None, 'stereo_pair_info_updated': None, 'name_text_updated': None, 'location_info_updated': None, 'tag_updated': None, 'bluetooth_info_updated': None, 'func_status_updated': None}, 'tuner': {'play_info_updated': <function <lambda> at 0x7fc849417668>, 'preset_info_updated': <function <lambda> at 0x7fc8494176e0>}, 'zone2': {'power': <function <lambda> at 0x7fc849417410>, 'mute': <function <lambda> at 0x7fc849417578>, 'status_updated': <function <lambda> at 0x7fc8494175f0>, 'volume': <function <lambda> at 0x7fc849417500>, 'input': <function <lambda> at 0x7fc849417488>, 'signal_info_updated': None}, 'zone3': {}, 'zone4': {}}

Dictionary to decode incoming events.

Indices and Tables

• Index
• Module Index