Table Of Content¶

Technical Documentation¶

Developers Documentation¶

• Developing a plugin
• Developing a brain part
• Developing a user interface
• Core development
• Releasing tasks
• Tools

What is Domogik in few words ?¶

Domogik is a free home automation solution based on xPL. It allows to use several User Interfaces (you can even create your own interface if you want). Domogik is modular : you can add and remove plugins easily and create your own plugins. A plugin is based on a technology (or part of a technology).

How does it work ?¶

Domogik is made of several core components. Each core component has some dedicated tasks. All components interact between them thanks to a message queue : Zero MQ. As Domogik uses xPL for automation purpose, some core components are plugged to the xPL network which is used by the xPL plugins. There is also one database to store configuration and data of all the sensors.

Components overview¶

Why xPL ?¶

xPL is an open protocol intended to allow the control and monitoring of home automation devices. The primary design goal of xPL is to provide a rich set of features and functionalities, whilst maintaining an elegant, uncomplicated message structure. The protocol includes complete discovery and auto-configuration capabilities which support a fully ‘plug and play’ architecture essential to ensure a good end-user experience. xPL benefits from a strongly-specified message structure, required to ensure that xPL-enabled devices from different vendors are able to communicate without the risk of incompatibilities.

—From xPL official website

xPL is really a useful protocol for automation. It is quite easy to implement of small electronic devices like arduino and it is used by other automation systems. Thanks to xPL you may plug 2 (or more) automation systems together, for example Domogik and xPL-perl.

Security¶

• Check Domogik administration interface with Wapiti

0.5.x releasing tasks¶

In a 0.5-hotfix branch :

• If needed, update src/domogik/__init__.py (this should already be done!)
• Check the milestone is finished in the tracker
• Create a new release note docs/users/releases/
• Create a blog post in all languages
• Prepare some artwork for twitter and the website
• Update the website index and complete the screenshot pages if needed
• Add the new release note in docs/contents.txt and docs/index.txt and docs/users/installation/index.txt
• Update docs/users/installation/index.txt with the new packages path
• Tag
• Test the install
• Merge in master
• Merge in develop
• In 0.5-hotfix, change the number to the next release (0.5.2 => 0.5.3 for example)
• Website : update the download page
• Website : publish (blog post + download page + index + screenshots)
• Do the same steps if needed for Domoweb

Obsolete - Resources¶

• branching model to follow : http://nvie.com/posts/a-successful-git-branching-model/

Obsolete - 0.4.0 specific tasks¶

• apply the branching model and communicate on it with all developers! * master => release-0.4 * create develop from master
• write doc about it for plugins
• write doc about it for the core (add a dedicated section about core dev in the doc) : hotfixes, features branches, ...
• set up the new download manager

Obsolete - Before releasing¶

Here are the tasks to do before releasing a package:

• review the quality (pylint)
• review the security with Wapiti
• check the tickets about this release
• decide if this is alpha/beta/candidate/final
• create the release-0.4.0 branch (from master in 0.4.0, from develop after) which will be used for the final steps (this allow to continue developments for next releases in the develop branch)
• check the release doc page and fill it if necessary

Obsolete - Packaging¶

• update package.sh
• generate the package, check its content
• test the package (installation, basic tests)
• set the tag on git master branch
• upload the package
• add it on wordpress
• create the delivery articles
• create the mail for ML
• tweet the release : link to the website article

Purpose¶

Before releasing, we must check the security of the administration web interface thanks to Wapiti : http://wapiti.sourceforge.net/

Install¶

$ wget http://sourceforge.net/projects/wapiti/files/wapiti/wapiti-2.3.0/wapiti-2.3.0.zip
$ unzip wapiti-2.3.0.zip
$ cd wapiti-2.3.0/bin/
$ chmod u+x *
$ sudo python setup.py install

Do the test!¶

$ ./bin/wapiti  http://192.168.1.10:40406

Wapiti-2.2.1 (wapiti.sourceforge.net)
..
 Note
========
Cette session de scan a été enregistrée dans le fichier /home/fritz/scans/192.168.1.10:40406.xml
Vous pouvez l'utiliser pour lancer des attaques sans scanner à nouveau le siteweb avec le paramêtre "-k"
[*] Chargement des modules :
        mod_crlf, mod_exec, mod_file, mod_sql, mod_xss, mod_backup, mod_htaccess, mod_blindsql, mod_permanentxss, mod_nikto

[+] Lancement du module crlf

[+] Lancement du module exec

[+] Lancement du module file

[+] Lancement du module sql

[+] Lancement du module xss

[+] Lancement du module blindsql

[+] Lancement du module permanentxss

Rapport
------
Un rapport a été généré dans le fichier generated_report
Ouvrez generated_report/index.html avec un navigateur pour voir ce rapport.

$ wget http://sourceforge.net/projects/wapiti/files/wapiti/wapiti-2.3.0/wapiti-2.3.0.zip
$ unzip wapiti-2.3.0.zip
$ cd wapiti-2.3.0/bin/
$ chmod u+x *

$ ./wapiti-getcookie /tmp/cookies.txt http://192.168.1.10:40406/login
<Cookie session=eyJfaWQiOnsiIGIiOiJOakV6TUdFd09EUTBaV05tT0dWaFl6Z3hOR1ZtWlRFME56UmlOMlppTkdNPSJ9fQ.BchpcA.XHRIKY8BaXEcC_r8FJfiRujtiZk for 192.168.1.10/>
Please enter values for the following form:
url = http://192.168.1.10:40406/login
user (default) : admin
passwd (letmein) : 123
<Cookie session=eyJfZnJlc2giOnRydWUsIl9pZCI6eyIgYiI6Ik5qRXpNR0V3T0RRMFpXTm1PR1ZoWXpneE5HVm1aVEUwTnpSaU4yWmlOR009In0sInVzZXJfaWQiOjF9.BchpeA.-vOHvUD-cDyDahfuZ5AcLuI2Udw for 192.168.1.10/>

$ wapiti  http://192.168.1.10:40406 --cookie /tmp/cookies.txt -v 2 -x http://192.168.1.10:40406/logout

List of ports used by Domogik and related softwares¶

Calculated sensors¶

A calculated sensor is a sensor where the stored value is calculated. as an example lets use a power sensor that transmits the number of KWh used, and we want to calculate the cost for the electricity of that interval.

If we then create a calculated sensor with the below formula we can store the price for the power consumption during that time.

def calculate(value):
    result = value * 0,11
    return result

The above function wil return the value to store, note this function can contain any python code you want, but there are some rules to follow:

• the function name is always ‘calculate’
• it has one input parameter named ‘value’
• it has one return parameter, a float

Overview¶

Some tools are installed with Domogik, they all starts by dmg_. The others are available only in Domogik sources. In this page, the first ones will be describes.

See the sources tools for the other tools.

dmg_dump¶

dmg_dump is a tool to listen xPL messages on the server.

Example:

$ dmg_dump
HUB discovery > starting
HUB discovery > looking for the hub. I hope there is one hub, Domogik won't work without the hub!
HUB discovery > Received HBEAT echo, HUB detected
HUB discovery > hub found, configuration in progress
2013-01-26 15:15:16.384598 - xpl-trig
{
hop=1
source=domogik-xpl_time.ambre
target=*
}
datetime.basic
{
datetime=20130126151847
date=20130126
time=151847
format1=201301261518475
}

^CKeyBoardInterrupt

It can be used with several options. See the -h option to get the full list:

$ dmg_dump -h
Usage: dmg_dump [options]

Options:
  -h, --help      show this help message and exit
  -c              Diaply data in a compress way
  -t XPLTYPE      Filter messages on XPL message type
  -s XPLSOURCE    Filter messages on XPL source field
  -S XPLSCHEMA    Filter messages on XPL schema field
  -i XPLINSTANCE  Filter messages on XPL instance
  -V, --version   Display Domogik version.
  -f              Run the plugin in foreground, default to background.

dmg_send¶

dmg_send allows you to send xPL messages from the command line.

Example:

$ ./send.py xpl-cmnd x10.basic "device=a1,command=on"

dmg_version¶

This tool will display the current Domogik version:

$ dmg_version
REST_API_release : 0.6
Domogik_release : 0.3.0
Sources_release : default.5689 (0.2.0-alpha1) - 2013-01-26 14:56 +0100

pylint-check.sh¶

This is a script which check all .py files from a directory to get a note about their quality.

Example (we assert $DOMOGIK_HOME is the root of your Domogik repository):

$ cd $DOMOGIK_HOME/domogik/src/domogik/bin # for example
$ $DOMOGIK_HOME/domogik/src/tools/pylint-check.sh
[...]
Good files :
./cidmodem.py;9,49;OK
./datetimemgr.py;8,71;OK
./gagenda.py;9,80;OK
./__init__.py;10,00;OK
./ipx800.py;9,38;OK
./manager.py;8,03;OK
./mirror.py;9,17;OK
./onewire.py;8,31;OK
./plcbus.py;8,33;OK
./send.py;9,57;OK
./teleinfo.py;9,74;OK
./wol_ping.py;9,32;OK
./x10_heyu.py;8,18;OK
./xbmc_not.py;9,51;OK
Bad files :
./dawndusk.py;3,17;KO
./dbmgr.py;7,67;KO
./dump_xpl.py;7,50;KO
./module_sample.py;7,14;KO
./rest.py;7,18;KO
./test.py;-10,00;KO
./test_send.py;7,83;KO

By default, a good file has a note greater or equals to 8,00/10. You can change this value in the script.

What should a user interface for Domogik do ?¶

Well, basically, what you want! Just keep in mind that Domogik is released with an administration user interface. This interface is used to manage Domogik, the plugins and the devices. You don’t need to handle these features and, I think you shouldn’t.

User interfaces should focus on control and visualisation:

• display the devices and their features.
• allow to control these devices features.
• allow to view the history of these devices features.
• organize devices by category, rooms or whatever you want.

Steps to follow¶

F.A.Q¶

• Q: Is this xPL hub compliant with other xPL systems (like xpl-perl for example) ?
• R: Yes, of course. There was a bug in the hub included with Domogik 0.2/0.3 which blocked xpl-perl clients, but since Domogik 0.4, xpl-perl and Domogik can be used together with the Domogik xPL hub.
• Q: I want to use the C hub instead of the python hub. How can I do ?
• R: After the installation, you can switch by editing /etc/default/domogik and setting DOMOGIK_XPL_HUB to c (or python for the new hub)
• Q: Where are the hub log files ?
• R: The C hub has no log files. The python hub log files are in /var/log/xplhub/. You can also customize the loglevel in the hub config file : /etc/domogik/xplhub.cfg.

The python hub¶

General informations¶

Some parts of this documentation will refer to the plugins development documentation as there are a lot of things in common :

• the workflow
• the way to create a repository, to write the doc, to submit the package

Differences between a plugin and a brain package¶

A brain package is not something that is executed directly : the butler component will load each brain package it will find on the current host.

A brain package does not need to create any device, so all the related informations in the json file will be empty.

A brain can contain several translations : you can find all the available languages in the rs/ folder. Example :

./rs

./rs/nl_BE
./rs/nl_BE/DT_Temp.rive
./rs/nl_BE/...
./rs/nl_BE/DT_Humidity.rive

./rs/fr_FR
./rs/fr_FR/DT_Temp.rive
./rs/fr_FR/...
./rs/fr_FR/DT_Humidity.rive

./rs/en_US
./rs/en_US/DT_Temp.rive
./rs/en_US/...
./rs/en_US/DT_Humidity.rive

The RiveScript code is used as a basis. You can find more informations on the official website : http://www.rivescript.com/docs/tutorial

First, create a repository and write some specifications¶

See the plugin development documentation.

Branches management¶

See the plugin development documentation.

Brain package name¶

There is no restriction in the length of the brain package name (for plugins, you have a 8 characters limitation due to xPL).

Prepare the plugin¶

See the plugin development documentation.

Specific parts :

• Brain package file tree

Before coding : branching to ‘develop’¶

See the plugin development documentation.

Json file¶

See the plugin development documentation.

To sum up, for a brain, you need only to fill 3 parts. You have to let the others empty :

• identity
• configuration
• json_version

Example:

{
    "identity": {
        "author": "Fritz",
        "author_email": "fritz.smh at gmail.com",
        "tags": ["butler"],
        "dependencies": [],
        "description": "Butler package to use Wolfram Alpha",
        "domogik_min_version": "0.4.1",
        "name": "wolfram",
        "type": "brain",
        "version": "1.0"
    },
    "configuration": [
        {
            "default": "",
            "description": "To get a Wolfram Alpha API key, you need to register (freely) on https://developer.wolframalpha.com/portal/signin.html",
            "key": "api_key",
            "name" : "Wolfram alpha API key",
            "required": true,
            "type": "string"
        }
    ],
    "products": [],
    "commands": {},
    "xpl_commands": {},
    "sensors": {},
    "xpl_stats": {},
    "device_types": {},
    "json_version": 2
}

Rivescript part¶

Rules for writing the Rivescript part

Python part¶

Some brain parts may need a library file to interact with some internet services (for example). You can create a python library in the lib/ folder.

See the Rivescript rules for more informations.

Dedicated administration page¶

See the plugin development documentation.

Icon¶

See the plugin development documentation.

Documentation¶

See the plugin development documentation.

Tests¶

Your brain is ready ? Make some users test it¶

See the plugin development documentation.

Release a version of your brain¶

See the plugin development documentation.

Using python objects¶

Features¶

The butler is able to tell the user which features it can handle. To allow this, you must add a special comment and a corresponding trigger. Example with the tiem feature:

// ##feature## give the date
+ give the date
@ shortcut date

+ shortcut date
...

Notice that if your feature accept a parameter, you should describe it also. Example in english:

// ##feature## calculate parameter
+ calculate *
...

Example in french:

// ##feature## calculer parametre
+ calculer *
...

You must use only generic expressions to describe the feature and not use words like “my, your, ...”. This is needed to allow natural speaking when requesting the butler to learn some commands. You can look at the domogik-brain-core package for more details about learning.

Weight¶

In rivescript, the default weight is set to 1. But the rivescript included in Domogik is patched to use a default weight of 10.

If you plan to use weight in your scripts, don’t forget about this!

Special characters¶

All the below characters are replaced by spaces before processing : ?!.,

The “-” character is not replaced as it can be used for mathematic usages. If you have a trigger that may contain a “-” in it, please handle it with “[-]”

People mistakes¶

People make sadly a lot of mistakes, especially when writing... In the rules, you may try to take the most common mistakes in account.

0.4.1¶

• Complete doc for 0.4.1

0.4.0¶

• New plugin format for Domogik 0.4.

General informations¶

• What is new in 0.4 about Domogik plugins format ?
• Changelog of plugins format since Domogik 0.4
• Release numbers rules

First, create a repository and write some specifications¶

• Create the plugin repository on GitHub
• Write specifications

Branches management¶

• What is a branch in git ?
• The branches management is detailed in the workflow of a plugin development

Plugin name¶

Choose a name that describes the plugin as good as possible. Notice that a plugin can not use more than 8 characters and they should be all lower case. This restriction is linked to xPL and is not a Domogik limitation.

Prepare the plugin¶

See workflow, step 1

Now, you will have to create a few files for your plugin on the master branch.

• Plugin file tree
• Readme and changelog files
• .gitignore file

Before coding : branching to ‘develop’¶

See workflow, step 2

• Switch to the *develop* branch before starting your developments : follow the step 2 of the workflow
• If you want, you can create in one shot the main folders and files

Json file¶

See workflow, step 3

The info.json file it surely the most important part of a plugin. It will describe the configuration of the plugins, the features, the xPL messages to send and to listen, ... You should first read all this chapter, then make sure that you already defined the following part in your specifications :

• configuration parameters
• devices features, splitted in sensors and commands
• xPL messages used by these features
• the data to store in database

Starting to create the json file without being clear on these elements is useless!

• Json file overview
• Sample json file
• Convert the json file from Domogik 0.2/0.3 format to the new Domogik 0.4 format
• Identity part
• Configuration part
• Device types part
• Sensors part
• Xpl stats part
• Commands part
• Xpl commands part
• Resources :
  • Data types
  • Field types
  • Conversion files

Python part¶

See workflow, step 3

After creating a start script (used only by the developers), you will now create the python part of your plugin.

• start.sh file
• Python part
  • __init__.py files
  • Bin part
  • Library part
  • How to ...
    • Make the detected devices available in the user interface
    • ...

Dedicated administration page¶

See workflow, step 3

Some plugin may need some advanced configuration capabilities or some helpers feature that can be used from the administration interface. This is not mandatory.

• Add dedicated administration pages

Icon¶

See workflow, step 3

Your plugin should have an icon.

• Add an icon

Udev rules¶

See workflow, step 3

If needed, you can add some sample udev rules files for your plugin.

• Add some udev rules files

Documentation¶

See workflow, step 3

The documentation is a part important of a plugin! If it is not enough clear, the user may have some difficulties to use the plugin.

• Overview of the documentation process
• Doc tree
• How to write the documentation in *ReST* format (which is used by *sphinx*)
• What should contain the documentation ?
• The developers part
• Make your documentation published on ReadTheDocs.org

Tests¶

See workflow, step 3

Preparing some automated tests is something really important for a plugin. It will help you to test automatically the plugin after each commit and will help you to easily validate some contributions (pull requests).

• Tests
  • The testrunner tool
  • Travis CI (continuous integration)

Your plugin is ready ? Make some users test it¶

See workflow, step 4

Now, your plugin may be functional, documented and the tests are automated. It is time to release a version! But before, you should make some people test your plugin!

• Release your plugin by following the step 4 of the workflow

Release a version of your plugin¶

See workflow, step 5

• Release your plugin by following the step 5 of the workflow

..todo ::

• switch to develop
• create a branch per version for readthedoc ????
• update info.json version number
• update docs/conf.py version number
• do the dev/doc/tests
• switch to master when version is ready (see previous chapter)

Some other points¶

• Helpers
  • ...

• Data files

Some big changes¶

With Domogik 0.4, the plugin format changes a lot. But don’t be afraid, it is the last big change of the plugins format : as Domogik 0.4 is the new base for the next Domogik releases, the plugins format won’t change like it did from 0.1 to 0.2 and from 0.3 to 0.4!

The 0.4 release come with a new database model, more powerful, more efficient. This model will allow to handle multi addresses technologies, like KNX or Chacon devices with the RFXCOM (for which the address is unit+id). It will also allow a more accurate choice of data types.

There are several important changes about the plugins format:

• All the xml files url2xpl and stats are not needed anymore and all what a plugin needs to know to speak xpl is centralized in a json file, named info.json.
• In Domogik previous releases, the plugins file tree was complex : there were some files in src/domogik_packages, src/share and 2 sort of file tree depending on the installation mode. Now, whatever your installation will be done (from sources, from a packaged release, from a debian package, ...), all the plugins will be installed in the same way, in the same place. And moreover, each plugin will have its own dedicated folder!

The plugins are easier to install¶

To deploy a plugin from its author sources, you will just have to grab the sources or a zip file, extract it somewhere in your filesystem and just create a symbolic link in /var/lib/domogik/domogik_packages to the plugin directory! Nothing else to do! And there is a tool which will do that for you. :doc:More informations about packages installation in the dedicated chapter </users/packages/index>.

Overview¶

All the released versions of the plugins can be available on the packages repository : http://repo-public.domogik.org The developer of a plugin just need to submit its plugin (more details in step 5).

We want to avoid alpha/beta/candidates versions for a plugin :

• it may be too heavy to handle for some developers.
• it will fill the packages repository with non stable plugin releases.

So, we want each developer to release a plugin version only when the plugin release in progress is finished (you will find a checklist of what is expected in the step 5). If some people wish to use a plugin under development, they just need to use the GitHub (or whatever else) repository (more details in the step 4).

Focus on the branches¶

For any Domogik package on git, you should follow the branching model described on this page. This branching model is inspired from the one here : http://nvie.com/posts/a-successful-git-branching-model/

Basically :

• the master branch contains the last stable release
• the develop branch contains the release which is currently developed. It you have to do any upgrade or fix, you should do them in this branch.
• when a stable release is ready (in the develop branch), you create a tag (and version) on the GitHub repository and then merge it into the master branch.

Example for a new plugin¶

On the left, there is the master branch. On the right, the develop branch.

Step 1 - prepare the information files on the master branch¶

You will find the links for these actions in the index of the plugin development documentation

This step goal is to provide at least a README.md file in the master branch to help people find the plugin documentation even if no release has been released (and so the master branch contains no plugin yet).

Step 2 - switch to the develop branch¶

Before coding anything you must switch to the develop branch : as the master branch content must be the last stable release of your plugin, you must NEVER do any development in this branch (master). This branch will be filled only with merge of the tags of the stable releases done in the develop branch.

First, if this is not already done, commit the changes on the master branch (README.md, CHANGELOG, .gitignore) and push them:

$ git status
...
        .gitignore
        CHANGELOG
        README.md
...

$ git add .gitignore CHANGELOG README.md
$ git commit -m "Initial commit"
[master (commit racine) e86b189] Initial commit
3 files changed, 9 insertions(+)
...
$ git push
...
To git@github.com:fritz-smh/domogik-plugin-teleinfo.git
 * [new branch]      master -> master

At this point, the master branch will only include nearly nothing. This is the goal! The master branch will be populated with the plugin data only when a stable version will be released.

Then, create the new develop branch and push it to the remote repository (on GitHub):

$ git branch develop
$ git checkout develop
$ git push --set-upstream origin develop
...
To git@github.com:fritz-smh/domogik-plugin-teleinfo.git
 * [new branch]      develop -> develop
...

You can now start to develop your plugin on this branch!

Step 3 - Develop the plugin¶

You will find the links for these actions in the index of the plugin development documentation

You should stay on this step until a stable version of your plugin is not ready in the develop branch. Please don’t forget that a stable version means:

• the plugin is functional and stable
• the plugin code is clean
• the info.json file is ok
• the documentation is written and handled by ReadTheDoc
• if the plugin technology allows it, some tests have been written and are functional with Travis CI
• ...

You will find a complete checklist in a later step

Step 4 - Register a development version¶

Step 5 - Releasing a stable version¶

"identity": {
    ...
    "name": "teleinfo",
    "type": "plugin",
    "version": "1.0"
},

$ git pull --tags
$ git tag -l

$ git tag -a "1.0" -m "release 1.0"

$ git push --tags

$ git checkout master
$ git merge 1.0

$ git push

$ git checkout develop

Purpose¶

Since Domogik 0.4, the administration user interface is no more included in Domoweb.

Technical details¶

The administration user interface server is made with Flask, a python framework. Bootstrap is used for the rendering part.

Startup¶

On startup, the butler will check for installed brain packages in the Domogik package directory. Depending on the configured language, it will load the appropriate language related files.

Interacting with the butler¶

To interact with the butler, you just need to :

• publish your queries in an interface.input MQ message.
• subscribe for responses or notifications to an interface.output MQ message

{
  "text" : ... ,
  "media" : ... ,
  "location" : ... ,
  "identity" : ... ,
  "sex" : ... ,
  "mood" : ... ,
  "source" : ... ,
}

{
  "text" : ... ,
  "media" : ... ,
  "location" : ... ,
  "identity" : ... ,
  "sex" : ... ,
  "mood" : ... ,
  "reply_to" : ... ,
}

Purpose¶

The dbmgr component is used to do some actions on the database over the MQ. It allows to :

• get and set configuration parameters values for the plugins
• get the devices list for a client

Purpose¶

The manager component is installed on the main host of a Domogik system. It manages the clients list and details, start the clients (plugins installed on the hosts), ... On Domogik startup, the manager will try to start the other core components (excepting the administration user interfaces).

Packages discovery¶

To find which clients are available on the host, the manager will check for the installed packages on startup, and then each N seconds. If a package in find in the packages installation folder, /var/lib/domogik/domogik_packages/ by default, the manager will check if this is a valid package and if so, it will add this package installed on the host to the clients list.

Purpose¶

The xplgw component (xPL gateway) is the gateway between Domogik and xPL world. This component is listening the xPL network for sensor values and send xPL commands over xPL when needed.

How are sensor values caught ?¶

For each device feature a listener is created based on the content of the tables core_xplstat and core_xplstat_param. When the listener catch a message, it will extract the values from it, apply some transformations (history cleanup, round, ...) and store them in database. Some conversions may be applied depending on the device features.

How a xPL command is sent ?¶

When a user interface (or scenario, ...) need to send a xPL command, the following actions are done:

• the REST url /cmd is called with the appropriate parameters (for user interfaces).
• REST send a message on the MQ about the command request.
• xplgw catch the message on the MQ and build the xPL message to send from the parameters it get over the MQ and the contant of the tables core_xplcommand and core_xplcommand_param*.

Purpose¶

Domogik data type are used to define the availables values for a device type.

Where can I find the list of the data types ?¶

The data types allowed by Domogik are defined in the file src/domogik/common/datatypes.json in the sources. Here is for example the content of the file for the *master* branch

How to choose a data type ?¶

If the device can be used for various use cases, you should choose a parent type, for example DT_Bool for a True/False usage. If the device is specific to an usage, you can use a child type, for example, instead of DT_Bool, use DT_Switch for a light, DT_OpenClose for a door sensor.

Notice about widgets¶

A widget should handle both the parent and child type to be compliant with most of the plugins : an on/off light widget would handle both DT_Bool and DT_Switch.

Purpose¶

The manager is the one which will start, stop and monitor all the others components (dbmgr, rest, plugins, ...). It will also manage the packages (installation, ...).

You can have several managers on a multihost installation.

You can call it with the dmg_manager command.

Design¶

The plugins manager is a xPL plugin (inherited of domogik.xpl.common.plugin.XplPlugin). When it starts, it connects to the xPL network and wait for messages.

Depending on the options passed at startup, it will be able to start:

-h, --help            show this help message and exit
-d                    Start database manager if not already running.
-r                    Start scenario manager if not already running.
-p                    Activate background ping for all plugins.
-E                    This manager is the one who looks for hardware.
-t CUSTOM_PING_DURATION
                      Time for xpl ping duration (default : 10)
-w WAIT_TIME_BETWEEN_PING
                      Time between 2 xpl ping (default : 15)
-V, --version         Display Domogik version.
-f                    Run the plugin in foreground, default to background.

It will then wait and listen for domogik.system and domogik.package xPL command messages.

Auto-refresh the enabled plugins list¶

When the Domogik configuration file is updated, the manager reloads it (so there is no need to restart the manager after enabling a plugin).

xPL commands¶

See the dedicated page.

Multi host¶

On a multi host system, there are 2 sorts of managers :

• the master which is only on the main host.
• the secondaries which are on all the others hosts.

Library¶

For the Message Queue domogik is using the 0MQ (zero MQ) library. More info about this system can be found at http://www.zeromq.org/.

General informations¶

• pub/sub
• req/rep

Frame format¶

• msg id frame
• data = the data (json format) for the action

The Network¶

As domogik has a lot of different components that need to publish or subscribe messages a system is required to forward the messages.

The dmg_forwarder was build for this, the forwarder listens on 2 different ports, on for the publisher clients and one for the subscribed clients. So all clients (domogik components) that publish a messages will need to connect to the publisher listing port, the forwarder will then broadcast this message to all clients (domogik components) that are connected on the subscriber port.

The subscribing to certain messages is done on the client side, by using the subscribe socket of zmq.

The publisher¶

The MQPub class is used to publish messages onto the pub/sub network.

The class needs 2 parameters on init:

• context = is an instance of zmq.Context()
• caller_id = is name used to identify the client

For sending (publishing) messages there is a send_event method with 2 parameters:

• category = used to generate the message id
• content = a python object (mostly a dict) that well be json encoded to be used as the content of the message

The subscribers¶

Frame format¶

• null frame

• protocol (MDPC01 or MDPW01) = what protocol to use (Client or Worker)

• service = the service where we want the request to (rinor, dbmanager, host.pluginname, host.manager)

  • for services that can only run once (rest, dbmanager, scenario manager) its just there name
  • for services that can run multiple times (on multiple hosts) its in the format <host>.<service>

• action = the action to perform (config.get, config.set, plugin.start, plugin.stop, plugin.install, ...)

• data = the data (json format) for the action

Clients¶

There are 2 clients provided, an async and a sync client

import zmq
from zmq.eventloop.ioloop import IOLoop
from domogik.mq.reqrep.client import MQSyncReq
from domogik.mq.message import MQMessage

cli = MQSyncReq(zmq.Context())
msg = MQMessage()
msg.set_action('plugin.start.do')
msg.add_data('id', 'diskfree')
print cli.request('manager', msg.get(), timeout=10).get()

Workers¶

• all plugins => for helper commands
• dbmanager
• packagemanager
• manager => start/stop a plugin for example

anything that needs to act as req/rep worker can extend the MQRep class

the mdpworker needs to be initialized: MQRep.init(self, zmq.Context(), ‘dbmgr’) the first argument is the zmq context the second argument is the workers name

then you can override the on_mdp_request method def on_mdp_request(self, msg) the msg argument is an object of type MQMessage

the MQMessage has 2 main items: - the action field - the data file (python dict)

from domogik.mq.reqrep.worker import MQRep
from domogik.mq.message import MQMessage
from zmq.eventloop.ioloop import IOLoop
import zmq


class Rep(MQRep):

    def __init__(self):
        MQRep.__init__(self, zmq.Context(), 'test_rep')
        IOLoop.instance().start()

    def on_mdp_request(self, msg):
        # display the req message
        print(msg)
        # call a function to reply to the message depending on the header
        if msg.get_action() == "foo.this.do":
            self._mdp_reply_foo_this(msg)

    def _mdp_reply_foo_this(self, msg_received):
        # get the value for a key of the received message
        the_value = data.get_data('a_key')
        # send the reply
        msg = MQMessage()
        msg.set_action('foo.this.result')
        msg.add_data('another_key', 'another_value')
        print msg.get()
        self.reply(msg.get())

Examples¶

empty
MDPC01
dbmanager
**config.get**
**{ host = igor, plugin = velbus, key = protocol}**

Null
MDPW01
0x02
envelopp (used to generate the reply
empty
**config.get**
**{ host=igor, plugin = velbus, key = protocol}**

Null
MDPW01
0x03
envelop (copy from the receiving packet)
empty
**config.result**
**{ data }**

empty
MDPC01
dbmanager
**config.result**
**{ data }**

Purpose¶

In Domgik world a package is a category of module. There are packages for Domogik but also packages for Domoweb (icons, themes for example).

Currently there is only one package type in Domogik: the xPL plugins, called as plugins.

In Domogik 0.5, the only way to install a package is a command line tool : see the user documentation for more inforations.

In the next releases, there will be, as it is in Domogik 0.3, a way to install and find packages over the administration web interface.

Packages and clients¶

A package is just a group of files you can install on a Domogik system. When a package is installed on a Domogik server, it becomes a client : a client is a package instance on a server.

This is a very important thing to understand : a client is a package installed on a server, so you can have a package installed several times on a Domogik multi host system and all these installations are clients.

Purpose¶

xPL plugins packages, called also plugins are the first type of packages available on Domogik. They are used to interact with home automation devices or online services.

How to develop a plugin ?¶

Just read the documentation on how to develop a plugin.

Overview¶

Plugins developers don’t need to read this page to be able to develop some plugins. This page is mainly for core developers to understand internal features of the plugins.

Plugins status¶

During its life, a plugin will get various status. All these status are available:

• STATUS_STARTING : the client is started but not yet ready (waiting for configuration or still doing some init stuff)
• STATUS_NOT_CONFIGURED : the client is not configured and so can’t be started
• STATUS_ALIVE : the client is ready for use
• STATUS_STO_REQUEST : the client has been requested to stop itself
• STATUS_STOPPED : the client as been stopped in the appropriate way
• STATUS_DEAD : the client has disappeared
• STATUS_UKNOWN : the client is in an unknown status (client detected but status not set to anything : for a plugin it may be a syntax error)
• STATUS_INVALID : the client is invalid (for a plugin, this may be an invalid json file)

To get the associated values, please check in Domogik sources : src/domogik/xpl/common/plugin.py

Purpose¶

The /cmd entry in REST is used by User Interfaces to send commands to devices.

For a xPL plugin, the translation of the url in a xPL message is done based on the database content (the device feature options) and the info.json file of the plugin.

Url description¶

http://ip:port/cmd/<command id>?<key1>=<value1>&<key2>=<value2>&...

The goal is to be able to generate a xpl message based on a simple url.

The url is formatted like this :

• the first param is command id, this is a defined command in the DB
• the options are corresponding to the parameters of the command, for example the status to apply to a light (on, off) or the dim level to apply to the light. The values that can be used for a device feature are related to the device feature data type.

How the UI will get the status/value of the device feature ?¶

REST only sends http code as response. If http code = 200, then the user interface should request REST to get the last sensor value (a command is always linked to a sensor in the info.json file). To get (for example), the level of a dimmable device after a “+10” increasing command, UI will use /sensor url to get the last value.

How do I find the command id ?¶

To find the command id of a device command, you first need to query the /device url and parse the commands part.

I am a user interface developer and I don’t understand the values to use for the parameters in the url options!¶

To help you to understand with real examples, or to check you are building your /cmd urls in the correct way, you can go in Domogik admin and check the details of a device. You will see all the /cmd available urls for the commands and all /sensor available urls for the sensors.

Which values should be used ?¶

You device is related to a data type. Depending on the data type, you will be able to use some values.

DT_Switch
        labels:
                1 = On
                0 = Off

"DT_Switch": {
    "labels": {
        "0": "Off",
        "1": "On"
    },
    "parent": "DT_Bool",
    "childs": []
},

"DT_State": {
    "labels": {
        "0": "Inactive",
        "1": "Active"
    },
    "parent": "DT_Bool",
    "childs": []
},

Flow¶

Here is what is done when the /cmd/... is called:

Examples¶

The following data is in the DB:

id = 1
name = test
usage = ventilation
type = x10.relay

id = 10
name = command1
xpl_command_id = 12

id = 11
key = level

id = 12
device = 1
schema = lighting.device

command = 12
key = address
value = 12

lighting.device {
    level = 100
    address = 12
}

Overview¶

REST is a web server which handle request in REST format. It is used by the user interfaces to get informations about devices, sensor history, send commands, ...

How to test REST requests¶

$ wget -qO- http://127.0.0.1:40405/device/1

Configuration section in /etc/domogik/domogik.cfg¶

In Domogik configuration file, there are several options about REST in [rest] section:

• interfaces : the network interface to use : eth0, lo, ...
• port : the port for the rest service : 40405
• use_ssl : use ssl (True) or not (False)
• ssl_certificate : The ssl .crt file
• ssl_key : The ssl .key file
• clean_json : display a clean json result (indentation, carriage return) (True) or a raw result (False)

How to use REST with SSL?¶

Please read the installation documentation.

Status¶

Rest uses http status codes to identify the error:

get:

code = 200

data = the returned data from the get request (json)

delete:

code = 204

data = empty

put (update):

code = 200

data = the updated object (json)

post (create):

code = 201

data = the new object (json)

error:

code = 400

data = {msg: “string describing the error”}

action ok, no content returned:

code = 204

data = Empty

Purpose¶

No official xPL Schema exists for Plcbus. A dedicated schema has been made for this feature. Discussion on the official xPL Forum about this xPL schema: http://xplproject.org.uk/forums/viewtopic.php?f=2&t=908&

plcbus.basic Message specification¶

• Class = PLCBUS
• Type = BASIC

PLCBUS.BASIC
{
USERCODE=<plcbus network usercode>
DEVICE=<sensor name>
COMMAND=<value sent to the device>
[ACK=<0|1>]
[DATA1=<additional data>]
[DATA2=<additional data>]
}

PLCBUS.BASIC
{
USERCODE=<plcbus network usercode>
DEVICE=<device name>
COMMAND=<value sent to the device>
[DATA1=<additional data>]
[DATA2=<additional data>]
}

PLCBUS.BASIC
{
USERCODE=<plcbus network usercode>
DEVICE=<device name>
COMMAND=<value sent to the device>
ACK=1
[DATA1=<additional data>]
[DATA2=<additional data>]
}

Examples¶

Find below manual commands examples, assuming your usercode is “FF”:

Turn ON A1:

dmg_send xpl-cmnd plcbus.basic "device=A1,command=ON,usercode=FF"

Turn OFF A1:

dmg_send xpl-cmnd plcbus.basic "device=A1,command=OFF,usercode=FF"

Preselect Dimmer to 40% on A1:

dmg_send xpl-cmnd plcbus.basic "device=A1,command=PRESET_DIM,usercode=FF,data1=40"

Preselect Dimmer to 40% with 3 secondes Fade Rate on A1:

dmg_send xpl-cmnd plcbus.basic "device=A1,command=PRESET_DIM,usercode=FF,data1=40,data2=3"

Preselect No Fade Rate on A1:

dmg_send xpl-cmnd plcbus.basic "device=A1,command=PRESET_DIM,usercode=FF,data2=0"

Turn ALL Ligths OFF:

dmg_send xpl-cmnd plcbus.basic "command=ALL_LIGHTS_OFF"

Turn ALL Ligths ON:

dmg_send xpl-cmnd plcbus.basic "command=ALL_LIGHTS_ON"

Purpose¶

There is no xPL schema defined for the information of a (french) teleinfo system. This page defines this needed schema.

Teleinfo.basic Message specification¶

• Class = TELEINFO
• Type = BASIC

The teleinfo.basic schema is used to send the data of a teleinfo system from an electric counter.

teleinfo.basic
{
adco=<Adresse du compteur>
optarif=<Option tarifaire>
isousc=<Intensité souscrite>
base=<Index option base>
iinst=<Intensité instantanée>
imax=<Intensité maximale appelée>
motdetat=<Mot d'état du compteur>
[hchc=<Heures|creuses>]
[hchp=<Heures|pleines>]
[ejphn=<Heures|normales>]
[ejphpm=<Heures|de pointe>]
[bbrhcjb=<Heures|creuses jours bleus>]
[bbrhpjb=<Heures|pleines jours bleus>]
[bbrhcjw=<Heures|creuses jours blancs>]
[bbrhpjw=<Heures|pleines jours blancs>]
[bbrhcjr=<Heures|creuses jours rouges>]
[bbrhpjr=<Heures|pleines jours rouges>]
[pejp=<Préavis|début EJP (30min)>]
[ptec=<Période|tarifaire actuelle>]
[demain=<Couleur|du lendemain>]
[adps=<Avertissement|de dépassement>]
[papp=<Puissance|apparente>]
[hhphc=<Horaire|heure pleine/heure creuse>]
[ppot=<Présence|des potentiels>]
[iinst1=<Intensité|instantanée phase 1>]
[iinst2=<Intensité|instantanée phase 2>]
[iinst3=<Intensité|instantanée phase 3>]
[imax1=<Intensité|maximale phase 1>]
[imax2=<Intensité|maximale phase 2>]
[imax3=<Intensité|maximale phase 3>]
[pmax=<Puissance|maximale triphasée>]
}

Teleinfo.short Message specification¶

• Class = TELEINFO
• Type = SHORT

The teleinfo.short schema is used when the max intensity is reached on a 3 phasis installation.

teleinfo.short
{
adir1=<Dépassement d'intensité sur la phase 1>
adir2=<Dépassement d'intensité sur la phase 2>
adir3=<Dépassement d'intensité sur la phase 3>
adco=<Adresse du compteur>
iinst1=<Intensité instantanée phase 1>
iinst2=<Intensité instantanée phase 2>
iinst3=<Intensité instantanée phase 3>
}

Purpose¶

The dedicated administration pages are available in Domogik administration on a client page in the Advanced menu.

How it works¶

On startup, Domogik administration will look for each installed client if there is an admin/ folder. If so, the content of this folder will be handled by Flask to be available from the administration.

So, you just need to fill an admin/ folder in the root of your package repository.

Create a simple page (diskfree example)¶

First, create the files :

mkdir admin/
mkdir admin/templates/
touch admin/__init__.py
touch admin/templates/plugin_diskfree.html

Of course, change plugin_diskfree with the appropriate package id.

# -*- coding: utf-8 -*-

### common imports
from flask import Blueprint, abort
from domogik.common.utils import get_packages_directory
from domogik.admin.application import render_template
from domogik.admin.views.clients import get_client_detail
from jinja2 import TemplateNotFound

### package specific imports
import subprocess



### package specific functions
def get_df():
    df = subprocess.Popen(["df", "-h"], stdout=subprocess.PIPE)
    output = df.communicate()[0]
    #device, size, used, available, percent, mountpoint = output.split("\n")[1].split()
    if isinstance(output, str):
        output = unicode(output, 'utf-8')
    return output

### common tasks
package = "plugin_diskfree"
template_dir = "{0}/{1}/admin/templates".format(get_packages_directory(), package)
static_dir = "{0}/{1}/admin/static".format(get_packages_directory(), package)

plugin_diskfree_adm = Blueprint(package, __name__,
                        template_folder = template_dir,
                        static_folder = static_dir)

@plugin_diskfree_adm.route('/<client_id>')
def index(client_id):
    detail = get_client_detail(client_id)
    try:
        return render_template('plugin_diskfree.html',
            clientid = client_id,
            client_detail = detail,
            mactive="clients",
            active = 'advanced',
            df = get_df())

    except TemplateNotFound:
        abort(404)

{% extends theme("base/base.html") %}
{% import theme("base/wtf.html") as wtf %}

{% block content %}
{% include theme("client_menu.html") %}
<div class="container">
<h2>{% trans %}Result of the 'df -h' command{% endtrans %}</H2>
<p>{% trans %}You only need to create devices for the mounted pathes. The <em>df -h</em> result will show you all the mounted pathes that may be monitored.{% endtrans %}</p>
<p>{% trans %}Please notice that the plugin may not give exactly the same values! This is related to the way the size are calculated. {% endtrans %}</p>
<pre>{{ df }}</pre>
</div>
{% endblock %}

Use Domogik Message Queue (MQ)¶

If you need to use the MQ from your admin page, you can by using WebSockets in Javascript : a gateway is available in the admin.

Limitations¶

Because the conversion code is using eval() the conversion code in the file should be very simple. There should be no comments in the file and there should only be one function per conversion file. See the example below.

Example¶

Some conversion may be needed for some features of a plugin. For example, the velbus plugin has some level features. Velbus hardware handle these levels with values from 0 to 255. For such features, Domogik has a datatype named DT_Scaling which allows a range from 0 to 100. So when data is sent from the velbus plugin over xPL, the data is in the range [0, 255]. When the xplgw component received it, before storing it in database, it will convert the value for the needed Domogik datatype : from [0-255] to [0-100]. To allow this, the plugin is delivered with a file from_level_to_DT_Scaling.py which contains:

def from_level_to_DT_Scaling(x):
    # 0 - 255 translated to 0 - 100
    return round(int(x) / 255 * 100)

This conversion function is used by a sensor, defined in the info.json file:

"sensors": {
    ...
    "level_range": {
            "name": "level sensor",
            "data_type": "DT_Scaling",
            "conversion": "from_level_to_DT_Scaling",
            "incremental": false,
            "history": {
                "store": true,
                "duplicate": false,
                "max": 0,
                "expire": 0,
                "round_value": 0
            }
    },
    ...

So, each time a value is caught by xplgw for such a sensor, it will be translated thanks to the conversion function to the appropriate value.

Create the repository¶

Now the plugins sources are no more present in Domogik sources. Each plugin has its own GitHub (or elsewhere you want) repository. Thanks to this, everybody can now create its own plugin with version control without needing to request the Domogik team for an access.

You must name the repository like this : domogik-<type>-<name> :

• domogik : this will allow everybody to find easily your plugin.
• <type> : the package type. For a plugin, the package type is plugin.
• <name> : the package name. It will be your plugin name : onewire, ipx800, plcbus, .... The package name must have a max length of 8 characters and be lower case. If needed you can use an underscore in the name. This limit is related to the xpl protocol.

Go on http://github.com, log in and create an account by clicking on the icon highlighted in yellow :

You will access to the repository creation form :

Fill the repository name, let the access as Public, set .gitignore to None (we will create our own file) and click on Create repository.

The repository is now created:

Clone the repository to start developing your plugin¶

In the right column, you can easily copy a link to clone the repository. To clone over ssh, just click on the SSH link:

And then, click on the icon highlighted in yellow to get the url in your clipboard:

You will have something like : git@github.com:fritz-smh/domogik-plugin-diskfree.git

Let’s say you will work in you home directory. Clone your repository:

$ cd $HOME
$ git clone git@github.com:fritz-smh/domogik-plugin-diskfree.git

If you want, you can activate the plugin on Domogik side:

$ cd /var/lib/domogik/domogik_packages
$ ln -s /home/youruser/domogik-plugin-diskfree plugin_diskfree

Notice that you could wait for your plugin to have a valid json first and then use the dmg_package tool to install it :

$ dmg_package -i $HOME/domogik-plugin-diskfree

Assuming you cloned an already working plugin, Domogik will automatically detect the plugin after a few seconds and you will be able to configure it and start it.

Be carefull : the symbolic link must be named like this : /var/lib/domogik/domogik_packages/[package type]_[package name]! You must use an underscore because python will not be able to see your plugin as a package is you use a minus.

A structured documentation¶

Your documentation must be logic and structured. It should follow some logical steps:

• first, describe the plugin, what is its goal, what are its features
• then, don’t forget to list the compliant hardware devices or services
• add some photos of the hardware in order the user can be sure he has the same thing
• if there are some complex dependancies (like OWFS for onewire), explain how to install them or at least give url to the appropriate documentations
• explain how to configure the plugin (the global configuration, not the domogik devices configuration)
• explain how to create a device (not where the user should click on the user interface as there is already a documentation for this) :
  • which feature it used for which hardware device or service ?
  • what does a device address will look like
  • give sample values and explain them
• if your plugin has some helpers, explain their features and how to use them
• finally you may add any informations that seem important for the user

Prerequisites¶

Your doc is written :)

Prepare the configuration file for Sphinx¶

In the plugin repository, create a file docs/conf.py which contains:

import sys
import os

extensions = [
    'sphinx.ext.todo',
]

source_suffix = '.txt'

master_doc = 'index'

### part to update ###################################
project = u'domogik-plugin-diskfree'
copyright = u'2014, Fritz'
version = '0.1'
release = version
######################################################

pygments_style = 'sphinx'

html_theme = 'default'
html_static_path = ['_static']
htmlhelp_basename = project

Update the middle part depending on your plugin.

Create an account on RTD¶

If you don’t have an account, go on RTD and create an account

Add your plugin project to RTD¶

In the dashboard <https://readthedocs.org/dashboard/>, click on Import to add your plugin documentation.

Fill the form:

• Name : put the full plugin name : domogik-plugin-myplugin.
• Repo : put the github repository url : http://github.com/mygithublogin/domogik-plugin-myplugin.git
• Repository type : Git or something else if you don’t use git
• Description : your plugin description
• Language : English
• Documentation type : Sphinx html
• Project url : empty
• Canonical url : empty
• Tags : domogik

Click on Create.

The build will be automatically started for the branch master. It may fail with the reason:

Conf file not found.

This is normal as the master branch contains no documentation (and no plugin code) if you are working on the first release of your plugin!

You will now need to activate the develop branch for your RTD plugin project.

Generate documentation for the develop branch¶

From the dashboard, for your plugin project, click on the Admin button. In the menu, go in Versions.

Set the develop version as Active. Uncheck the Active checkbox for the latest version and click on Submit.

You are now on the Overview page for your plugin project. The only version available is develop. You can click on Build to start the build of the documentation for the develop branch.

In the Builds menu for your project, you can check if the build was successful or not. If not, you can click on the status to get the error details.

You can now see the built documentation at http://domogik-plugin-myplugin.readthedocs.org/en/develop/ (adapt myplugin to your plugin name)

Notice that the View Docs button will send you to the latest version of your plugin documentation, which does not exists yet! This version will exists when your plugin will be stable and so, when you will have merge the develop branch in the master branch.

Set the hook to build the doc for each commit pushed¶

In your github project, you can set a hook to launch the documentation build on RTD for each commit pushed.

Go on your Github project url. For example https://github.com/mygithublogin/domogik-plugin-myplugin

• Go in the Settings menu, then in Webhooks and services.
• In the Services part, click on Add service and choose ReadTheDocs.
• The ReadTheDocs service configuration page will be displayed. Check that Active is set and click on Add service.

Now, just do a documentation update, commit it and push it. You should see the doc updated a few minutes later.

Versions management¶

..todo ::

Continue :) * version management

i18n¶

..todo ::

TODO !

Read the doc :)¶

Your documentation is available at http://domogik-plugin-<plugin name>.readthedocs.org/en/<branch name>/. Example : http://domogik-plugin-teleinfo.readthedocs.org/en/develop/

The mandatory part¶

Only 3 files are mandatory:

• docs/index.txt
• docs/myplugin.txt : replace myplugin by your plugin name : diskfree, ozwave, ...
• docs/changelog.txt
• docs/dev.txt

The index.txt file will be the toctree of your documentation. It will contain only references to all the other files.

The myplugin.txt file is the main file for your plugin documentation. It will contain informations about plugin configuration, devices creation, ...

The changelog.txt file will contain the history of your package.

The dev.txt will contain some technical informations for the developers.

Example tree for a simple plugin¶

For a plugin named myplugin:

docs/index.txt
docs/myplugin.txt
docs/dev.txt
docs/changelog.txt
docs/*.png

For these plugins all the content of the user documentation will be directly written in myplugin.txt.

Example tree for a big plugin¶

docs/index.txt
docs/mybigplugin.txt
docs/aspecialpage.txt
docs/anotherspecialpage.txt
docs/dev.txt
docs/changelog.txt
docs/*.png

Of course you can use sub directories if you need.

Focus on index.txt¶

The index.txt file is very important! It must contain a link to all other txt files. Example:

.. _toc:

================
Table Of Content
================

.. toctree::

    /myplugin
    /dev
    /changelog

Purpose¶

These types are used to validate the input (on the ui) for certain fields. An example configuration parameters, command parameters, ....

Known Field Types¶

.gitignore file¶

For each plugin, a .gitignore file must be created in the root directory. Create it like this:

$ echo "*.pyc
*.swp
_build_doc" > .gitignore

This file is used by git to exclude some files from the repository.

Purpose¶

The Json file describe the package and the features of the package. There is one common part which is the same for all packages type and some optional parts that depends on the package type.

The common part¶

Package type :	Plugin	External
Is section required for package type :

The common part is like this :

{
    "json_version": 2,
    "identity": {
        "type": "plugin",
        "id": "onwire",
        "category": "onewire",
        "version": "0.1",
        "domogik_min_version": "0.2.0",
        "documentation": 'http://wiki.domogik.org/plugin_onewire',
        "description": 'Manage 1-wire devices...',
        "author": 'Domogik',
        "author_email": 'xx@xxx.fr',
        "changelog" : "0.1\n-Create plugin",
        "dependencies": [
            {
                "id": "owfs (>=2.8.4)",
                "type": "other"
            }
        ],
     },
    "files": [
        "src/share/domogik/design/plugin/onewire/icon.png",
        "src/share/domogik/plugins/onewire.json",
        ...
    ],
}

Optional parts¶

You must add the following parts in the json file when developing a package.

Technology¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"technology": {
    "description": "1-wire",
    "id": "onewire",
    "name": "1-wire"
},

Description¶

These data will be inserted in database during the package installation.

• technology : description of the package technology
  • id : technilogy id.
  • name : tehcnology name.
  • description : short description of the technology.

Device types¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"device_types": {
    "onewire.thermometer" {
        "description": "Thermometer",
        "id": "onewire.thermometer",
        "name": "Thermometer",
        "commands": ["set_level_bin"],
        "sensors": ["level"],
        "params": [
                {
                        "key": "channel",
                        "description": "The channel number",
                        "type": "integer",
                },
                ...
        ]
    },
    ...
}

Description¶

These data will be inserted in database during the package installation.

• device_types : dictionary of device types, indexed on device_type id

  • id : device type id

  • name : device type name

  • description : short description of the device type

  • commands : list of commands supported by this device type

  • sensors : list of sensors supported by this device type

  • params : list of parameters that are common to ALL xpl messages for this device type

    • key : the xpl key for the message
    • description : short description of this field
    • type : the value type for this field
    • depending on the type the are other possibilities (like min_value, max_value, ....)

Configuration elements¶

Package type :	Plugin	External
Is section required for package type :

These data are read by the manager.

Example¶

"configuration": [
    {
        "id": "0",
        "interface": "no",
        "key": "startup-plugin",
        "type": "boolean",
        "options": [],
        "default": "False",
        "description": "Automatically start plugin at Domogik startup",
        "optional": "no",
    },
    ...
]

Description¶

• configuration : list of the configuration parameters of a plugin

  • id : number of the configuration item. The display order will be related to the id.

  • interface : yes or no. If yes, group with all the following keys with interface = yes. This group of keys allow to configure N iterations of these keys.

  • type : the type of the value to set for the key.

    • string : the default type

    • number : a number

    • boolean : True, False

    • enum : a list of options. Example:

      "type" : "enum",
      "options": [
          "ipx800v1",
          "ipx800pro",
          "ipx800v2",
          "ipx800v3"
      ],
      

  • options : if type = enum, list the available options in this table.

  • default : suggested value.

  • description : parameter short description.

  • optional : yes or no : is the parameter optional or not ? It is used only for notification in the user interface.

Todo

detail enum

Udev rules¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"udev-rules": [
    {
        "description": "Usb DS9490R adaptator",
        "filename": "onewire.rules",
        "model": "DS9490R",
        "rule": "SUBSYSTEMS==\"usb\", ATTRS{idVendor}==\"04fa\", ATTRS{idProduct}==\"2490\", SYMLINK+=\"onewire\", MODE=\"0666\""
    }
]

Description¶

• udev-rules : list of the suggested udev rules. * description : short description of the rule. Indicate the related device model here. * filename : suggested filename to use for the rule. * model : related device model. * rule : the udev rule.

External members identification¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"external": {
    "device_id": "rgb",
    "vendor_id": "arduino"
},

Description¶

• external : description of the vendor and device id of the external member. This is used to identify a specific external member model. * device_id : device id. * vendor_id : vendor id.

Commands¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"commands": {
        "set_level_bin": {
                "name": "Switch On or Off",
                "return_confirmation": true,
                "params": [{
                        "key": "level",
                        "value_type": "binary",
                        "values": [0, 255]
                }],
                "xpl_command": "set_level_bin"
        },
        ...
}

Description¶

• Commands : this section describes the commands needed by this plugin, its a dictionary indexed by the command reference

  • name : the name of this command

  • return_confirmation : does rinor need to wait for a confirmation

  • params : the needed params to be able to complete this command (typically these are the values set by the widget)

    • key : the name of the parameter

    • value_type : what type of value we can receive

    • values : the possible values, the format depends on the value_type

      • value_type = binary : then values is a list containing the off and on value
      • value_type = range : then values contains a list of min and max value of the range

  • xpl_command : what xpl command is linked to this command (this field is optional)

Sensors¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"sensors": {
    "level": {
            "name": "level",
            "unit": "%",
            "value_type": "range",
            "values": [0, 100]
    },
    ...
},

Description¶

• Sensors : this section describes the sensors supported by this plugin, its a dictionary indexed by the command reference

  • name : the name of the sensor

  • unit : the unit of this value, needed for ui display

  • value_type: what type of value we can receive

  • values : the possible values, the format depends on the value_type

    • value_type = binary : then values is a list containing the off and on value
    • value_type = range : then values contains a list of min and max value of the range

Xpl_commands¶

Package type :	Plugin	External
Is section required for package type :

Example¶

"xpl_commands": {
     "set_level_bin": {
        "name": "blah",
        "schema": "lighting.basic",
        "xplstat_name": "get_level",
        "parameters": {
                "static": [
                    {
                        "key": "stat",
                        "value": "stat"
                    }
                ],
                "device": [
                    {
                        "key": "dummy",
                        "description": "a dummy param",
                        "type": "string"
                    }
                ]
            }
     },
     ...
},

Description¶

• xpl_command : this section describes the xpl_command to sent when a command is triggered

  • name : the name of the command

  • schema : the xpl schema to use

  • xplstat_name : the xplstat that will be expected as a confirmation

  • parameters : the key/value pairs for the xpl message

    • static : values that are static, these key/value pairs are always there and not changeable

      • key : xpl key
      • value : xpl value

    • device : parameter that can be changed on a per device basis, will be displayed in the create device interface

      • key : xpl key
      • description : the description that will be displayed
      • type : the value type

How to insert the data in database¶

This action is only for developers!

If you want to manually insert the data of the json in the database, launch this command :

cd src/tools/packages
./insert_data.py ../../share/domogik/plugins/<your plugin name>.json

Purpose¶

The commands section will quickly describe each command with:

• a name
• an xpl command to generate. Sess xpl commands for info about these.
• a list of parameters

Example 1¶

This example is the velbus plugin. See the device types documentation for more informations.

For each command, we define a name and the required parameters for this command.

"commands": {
    "set_level_bin": {
         "name": "Switch On or Off",
         "return_confirmation": true,
         "parameters": [{
             "key": "level",
             "data_type": "DT_Switch",
             "conversion": "from_DT_Switch_to_level"
         }],
         "xpl_command": "set_level_bin"
    },
    ...
}

Description¶

Each item has several properties:

• name : it is a string used to display this command on the UI

• return_confirmation : true or false depending if the plugin responds to the command

• xpl_command : what xpl command to generate

• parameters : a list of parameters that need to be filled in during command generation

  • key : the xpl-message key of the parameter
  • data_type : what datatype this parameter will be
  • conversion : the optional conversion function to run. See conversion function documentation

What happens if a command is received by the xpl gateway¶

1. Check that all parameters are provided in the mq message
2. Load the needed parameters from the xpl_command
3. Combine all parameters to generate the xpl message
4. Load the xpl_stat thats linked to the defined xpl command
5. Create an xpl listener for the xpl_stat/xpl_trigeer messages
6. Send out the xpl_command
7. Wait until the xpl_stat is received

Purpose¶

The configuration section describes the plugin configuration items. This part is only related to global configuration ! All the configuration elements which are related to a device are to be defined in the domogik device types part.

Example¶

For this example we assume that your plugin has this configuration item:

• an interval item : if not configured, the default value configured in the json will be used y the plugin

Here is the corresponding section:

"configuration": [
    {
        "default": 5,
        "description": "Interval between each poll (minute)",
        "key": "interval",
        "name": "Interval",
        "required": false,
        "type": "integer",
        ...
    }

Notice about the auto_startup parameter¶

One parameter is automatically added by Domogik to your configuration parameters : the auto_startup element. The following data is dynamically added by Domogik when loading the info.json file:

{
    "default": false,
    "description": "Automatically start the plugin at Domogik startup",
    "key": "auto_startup",
    "name" : "Start the plugin with Domogik",
    "required": true,
    "type": "boolean"
}

This element allows the user to enable a plugin to be started on Domogik startup (useful during a server reboot).

Description¶

Each item has several properties:

• key : the configuration key. This is used to store the value in database. The key must be lower case, limited to 255 characters. The only separator allowed in the underscore. This won’t be displayed on the user interface.
• name : a human readable equivalent of the key. This wil be displayed on the user interface.
• description : the configuration item description. If the item type is an integer, a float or something like this, you must specify the unity (minutes, seconds, ...) in the description.
• type : the configuration item type. You will find more informations below.
• default : the default value. If the configuration item is not set in database, this default value will be used by the plugin.
• required : is the configuration of this item required to run the plugin ? On the user interface, you may not be able to start a plugin if all the required = true items are not filled. If the required key doesn’t exist, we assume that the item is required and required = true.
• ... : some additional and optional parameters depending on the type. You will find more informations below.

Purpose¶

The device_types section describes all the Domogik device types that are handled by the plugin. A Domogikdevice type is a sort of device (for example an analog input, a digital input, a temperature sensor, a temperature and humidity sensor, ...).

A device type is linked to some commands and some sensors. For example, a temperature sensor will have only a temperature sensor. A temperature and humidity sensor will have 2 sensors : a temperature sensor and a humidity sensor. In some technologies, an on/off lighting device type will have 1 command : on/off (which may be named switch) and a dimmer device type will have 2 commands: on/off (switch) and dimmer (which could be simply named dimmer).

Finally a device type has some parameters. For simple device type, there will be only one parameter : the device address. For some complex device type (some KNX or Chacon devices for example), you will have several parameters : the address which is split in 2 parts (chacon devices), the actuator address which is not the same as the sensor one (KNX devices), ...

The known datatypes can be found in the file /var/lib/domogik/resources/datatypes.json.

Example 1¶

This example is the diskfree plugin. This plugin has only one device type named diskfree.disk_usage. This device type feature is to look at the disk space status on a filesystem. There are 4 informations which are available for this feature:

• the total space available
• the free space available
• the used space
• the percent of space used

These 4 informations are here given by 4 sensors.

To check these sensors the plugin need 2 information for the device type:

• the path to look at on the filesystem
• the interval between each check

"device_types": {
    "diskfree.disk_usage": {
        "description": "Disk usage",
        "id": "diskfree.disk_usage",
        "name": "Disk usage",
        "commands": [],
        "sensors": ["get_total_space", "get_percent_used", "get_free_space", "get_used_space"],
        "parameters": [
            {
                "key": "device",
                "xpl": true,
                "description": "The path to look at.",
                "type": "string"
            },
            {
                "key": "interval",
                "xpl": false,
                "description": "The time in minutes between each check.",
                "type": "integer"
            }
        ]
    }
}

Purpose¶

The identity section is the identity card of the plugin.

Example¶

"identity": {
    "author": "John",
    "author_email": "john@dummy.com",
    "tags": ["computer"],
    "dependencies": [],
    "description": "A plugin description which may be on \n several lines",
    "domogik_min_version": "0.4.0",
    "name": "myplugin",
    "type": "plugin",
    "version": "0.1"
},

Some optional fields can be added in identity:

"identity": {
    ...
    "xpl_clients_only": true,
    ...
},

"identity": {
    ...
    "compliant_xpl_clients" : ["rfxcom-lan"],
    ...
},

Description¶

Mandatory :

• type : the package type. For a plugin, the value will always be plugin
• name : the plugin name. Example : ipx800, diskfree, ...
• version : the plugin version. It must respect the rules about version numbers
• domogik_min_version : the minimum Domogik compliant release. It can’t be lower than 0.4.0
• description : the plugin description
• author : nickname, full name, society or anything to identify the author of the plugin
• author_email : an email to contact the package author. If you want you can write the email like this : john at dummy dot com.
• tags : a list of tags. You will find some examples below.
• dependencies : a list of dependencies for the package. You will find more informations below.

Optionnal :

• xpl_clients_only : true|false. Default : false. This key is to set to true only for the plugins with no python part. These are special plugins that will be used only for adding compatibility to external xPL clients to Domogik. Example : the generic plugin.
• compliant_xpl_clients : an array of <vendor_id>-<device_id> identifiers. This array is to used when a plugin with a python part can also handle some external xpl clients. For example, the rfxcom plugin can handle both usb and lan models. The usb one needs python to work. The lan one just need the json. All xpl clients detected as corresponding to one of the <vendor_id>-<device_id> identifiers will be visually linked to the plugin in the Domogik administration interface.

"dependencies": [
    {
        "id": "pyserial (>=2.5)",
        "type": "python"
    }
],

"dependencies": [
    {
        "id": "cron",
        "type": "plugin"
    }

"dependencies": [
    {
        "id": "owfs (>=2.8.4)",
        "type": "other"
    }

Overview¶

Purpose¶

This document describes the step to follow for upgrading the plugin json file from version 1 to 2. With json version 2 the url2xpl and the stats xml files are deprecated, the content of these files are moved to the plugin json file (sensors and commands parts). This document only describes how to upgrade.

Configuration part¶

The following keys had been removed:

• id : just define the options in the appropriate order in the table
• interface (as now the way of handling multiple devices will change)

In the json file, the following keys are renamed:

• startup-plugin => auto_startup. And this key is also removed! You don’t need to add it in the json anymore, it is dynamically added by Domogik as all plugins needs it.

The following way of writing have changed:

• minus => underscore in keys
• yes/no => true/false
• empty => null
• optionnal => required
• the option part is optional if not required by the type.

The following items has been added:

• name : a human readable name for the configuration item.

Files part¶

There is no more files part as now the plugin files are contained in the same root directory. So when the package is created, all files are included.

Identity part¶

The following keys had been removed:

• no more changelog

The following way of writing have changed:

• category => tags
• id => name

Sample¶

# Here part of json declaration

    "device_types": {                                           # device type declaration who appears in user interface list to create device
        "fooplugin.device_name": {
            "id": "fooplugin.device_name",                          # device id for internal reference (no Xpl)
            "description": "A short description",
            "name": "The device type name",
            "commands": ["cmd_one", "cmd_two"],                     # link to list of commands
            "sensors": ["sensor_one", "sensor_two"],                # link to list of sensors
            "params": [                                             # parameters that are common to ALL xpl messages for this device type
                {
                    "key": "key1_devicetype",                       # key copied to the xpl messages, this a static key like device address
                    "description": "A short description",
                    "type": "string"                                # key appears in the xpl as : {key1_devicetype : this a string type}
                    "xpl": True                                     # if the params is in the xpl messages : True. For example, an interval will not be in a xpl message
                    "default": "A default value"                    # optionally a default value for this parameter
                },
                {
                    "key": "key2_devicetype",
                    "description": "A short description",
                    "type": "integer",
                    "max_value": 255,                               # depending on the type the are other possibilities
                    "min_value": 0,
                    "xpl": True                                     # if the params is in the xpl messages : True. For example, an interval will not be in a xpl message
                    "default": "A default value"                    # optionally a default value for this parameter
                }
            ]
        }
    },


    "commands": {                                                   # commands who are call by device_types, A command can be call by many device_types
       "cmd_one": {
           "name": "The cmd name",
           "return_confirmation": true,                             # at true a confirmation xpl message should be send at command return. It is strongly recommended to use confirmation
           "parameters": [                                              # parameters that are common to ALL xpl messages for this command
               {
                   "key": "key1_cmd_one",                           # key copied to the xpl messages, this a dynamic key that can be set in each Xpl message (dynamic)
                                                                    # key appears in the xpl as : {key1_cmd_two : dynamic value}
                   "data_type": "DT_Switch",                        # link to Domogik Type of Data used by widgets in UI.
                   "conversion": "from_DT_Switch_to_level"          # if needed define proper conversion function in file <plugindirectory>/conversion/<fooplugin>.py
                                                                    # conversion have utility when hard device receive a particular scale or data type who are not directly compatible with Domogik Types
               }
           ],
           "xpl_command": "xpl_cmd_one"                             # link to xpl_command format to use
        },
        "cmd_two": {
           "name": "The cmd name",
           "return_confirmation": false,
           "parameters": [
               {
                   "key": "key1_cmd_two",
                   "data_type": "DT_Scale",
                   "conversion": "from_DT_Scale_to_level"
               }
           ],
           "xpl_command": "xpl_cmd_two"
        }
    },

    sensors": {                                                     # sensors who are used by device_types, A sensors can be use by many device_types
        "sensor_one": {
            "name": "The sensor name",
            "data_type": "DT_Switch",
            "conversion": "from_level_to_DT_Switch",                # if needed define proper conversion fuction in file <plugindirectory>/conversion/<fooplugin>.py
                                                                    # conversion have utility when hard device send a particular scale or data type who are not directly compatible with Domogik Types
            "incremental" : false,                                  # does the sensor is about incremental values
            "timeout" : 120,                                        # value in seconds to indicate when a value is too old and so the device may be offline/broken/...
            "history": {                                            # how is managed the sensor history.
                "store": true,
                "duplicate": false,
                "max": 0,
                "expire": 0,
                "round_value": 0
            }
         },
        "sensor_two": {
            "name": "The sensor name",
            "data_type": "DT_Scaling",
            "conversion": "from_level_to_DT_Scaling",
            "incremental" : false,
            "timeout" : 0,
            "history": {
                "store": true,
                "duplicate": false,
                "max": 0,
                "expire": 0,
                "round_value": 0
            }
     },

    "xpl_commands": {                                               # declaration for linking commands to xpl schema and keys
         "xpl_cmd_one": {
            "name": "The xplCmd name",
            "schema": "fooplugin.basic",                            # schema that must respect the xpl-protocol
            "xplstat_name": "xplstat_name_one",
            "parameters": {                                         # parameters that are common to ALL xpl messages for this xpl_command
                "static": [                                         # key copied to the xPL message. Static keys will always be the same, whatever the device is
                    {
                        "key": "static_key1_xpl_cmd_one",           # key appears in the xpl as : {static_key1_xpl_cmd_one : static value}
                        "value" : "value_static_key1_xpl_cmd_one"
                    },
                    {
                        "key": "static_key2_xpl_cmd_one",
                        "value" : "value_static_key2_xpl_cmd_one"
                    }
                ],
                "device":                                           # key copied to the xPL message. Satic value defined on a per device basis (input is requested) in the user interface.
                                                                    # The device key is used for parameters that depend on a device, that require an input by the user,
                                                                    # but that do not change once the device is created, once the device is cerated these parameters
                                                                    # are static, but the value is requested during the device creation process.
                                                                    # an example of a device parameter is the datatype in knx, this can be different for xplstat and xplcommand,
                                                                    # but it will stay the same once the device is created


                    {
                        "key": "device_key1_xpl_cmd_one",           # key appears in the xpl as : {device_key1_xpl_cmd_one}
                        "description": "A description",             # A description that will be displayed in the admin interface
                        "type": "string"                            # The type of the field (used for the userinterface)
                        "default": "A default value"                # optionally a default value for this parameter
                    },
                    {
                        "key": "device_key2_xpl_cmd_one",           # key appears in the xpl as : {device_key1_xpl_cmd_one}
                        "description": "A description",             # A description that will be displayed in the admin interface
                        "type": "string"                            # The type of the field (used for the userinterface)
                        "default": "A default value"                # optionally a default value for this parameter
                    }
                ]
            }
         },
         "xpl_cmd_two": {
            "name": "The xplCmd name",
            "schema": "fooplugin.basic",
            "xplstat_name": "",
            "parameters": {
                "static": [
                    {
                        "key": "static_key1_xpl_cmd_two",
                        "value" : "value_static_key1_xpl_cmd_two"
                    }
                ],
                "device": [
                    {
                        "key": "device_key1_xpl_cmd_two",           # key appears in the xpl as : {device_key1_xpl_cmd_one : static value}
                        "description": "A description",             # A description that will be displayed in the admin interface
                        "type": "string"                            # The type of the field (used for the userinterface)
                        "default": "A default value"                # optionally a default value for this parameter
                    }
                ]
            }
         }
    },

    "xpl_stats": {                                                  # declaration for linking stats to xpl schema and keys.
       "xplstat_name_one": {
            "name": "The xplstat name",
            "schema": "sensor.basic",                               # schema that must respect the xpl-protocol
            "parameters": {                                         # parameters that are common to ALL xpl messages for this xpl_stats
                "static": [                                         # key copied to the xPL message. Static keys will always be the same, whatever the device is
                    {
                        "key": "static_key1_xplstat_one",           # key appears in the xpl as : {static_key1_xplstat_one : static value}
                        "value" : "value_static_key1_xplstat_one"
                    }],
                "device": [
                    {
                        "key": "device_key1_xpl_cmd_two",           # key appears in the xpl as : {device_key1_xpl_cmd_one}
                        "description": "A description",             # A description that will be displayed in the admin interface
                        "type": "string"                            # The type of the field (used for the userinterface)
                        "default": "A default value"                # optionally a default value for this parameter
                    }
                ],
                "dynamic": [                                        # key copied to the xPL message. keys which are values for the sensors. They are stored in the sensor history table.
                    {
                         "key": "dynamic_key1_xpl_xplstat_one",     # key appears in the xpl as : {dynamic_key1_xpl_xplstat_one : dynamic value}
                         "ignore_values": "",
                         "sensor": "sensor_one"                     # link to sensor type to set Domogik Data Type and optimal conversion
                    },
                    {
                         "key": "dynamic_key2_xpl_xplstat_one",
                         "ignore_values": "",
                         "sensor": "sensor_two"
                    }
                ]
            }
       }
    }

xpl-cmnd
{
    hop1 = 1
    source = xpl-rest.domogik
    target = *
}
fooplugin.basic
{
    key1_devicetype_one = static-value_string                   # ex : {"deviceaddr" : "x0103"}
    key2_devicetype_one = static-value-integer                  # ex : {"channel" : "a1"}
    key1_cmd_one = dynamic-value-DT_Switch                      # ex : {"command" : "on"}
    static_key1_xpl_cmd_one = value_static_key1_xpl_cmd_one     # ex : {"zone" : "bedroom"}
    static_key2_xpl_cmd_one = value_static_key2_xpl_cmd_one     # ex : {"timer" : 15}
    device_key1_xpl_cmd_one = value_device_key1_xpl_cmd_one     # ex : {"bauds" : 40000}
    device_key2_xpl_cmd_one = value_device_key2_xpl_cmd_one     # ex : {"sleep" : 2}
}

xpl-trig
{
    hop = 1
    source = domogik-fooplugin.foomachine
    target = *
}
sensor.basic
{
    key1_devicetype = static-value_string                                   # ex : {"deviceaddr" : "x0103"}
    key2_devicetype = static-value-integer                                  # ex : {"channel" : "a1"}
    static_key1_xplstat_one = value_static_key1_xplstat_one                 # ex : {"zone" : "bedroom"}}
    device_key1_xpl_xplstat_one = value_device_key1_xpl_xplstat_one         # ex : {"bauds": 40000
    dynamic_key1_xpl_xplstat_one = dynamic-value-sensor-one-DT_Switch       # ex : {"state" : "on"}
    dynamic_key2_xpl_xplstat_one = dynamic-value-sensor-two-DT_Scaling      # ex : {"battery" : 80}
}

xpl-cmnd
{
    hop1 = 1
    source = xpl-rest.domogik
    target = *
}
fooplugin.basic
{
    key1_devicetype_one = static-value_string
    key2_devicetype_one = static-value-integer
    key1_cmd_two = dynamic-value-DT_Scale
    static_key1_xpl_cmd_two = value_static_key1_xpl_cmd_two
    device_key1_xpl_cmd_two = value_device_key1_xpl_cmd_two

}

Purpose¶

The sensors section will quickly describe each sensor with:

• a name
• a data type. See the data types documentation for more informations
• if needed, some conversion options

Example 1¶

This example is the diskfree plugin. See the device types documentation for more informations.

For each sensor, we define a name and no conversion option. Then, depending of the sensor we use 2 data types:

• DT_Byte : this one is used for values in byte, which are the values returned by the sensors get_free_space, get_used_space and get_total_space.
• DT_Scaling is used for values in percent from 0 to 100, which corresponds to the value returned by the sensor get_percent_used.

"sensors": {
    "get_total_pace": {
        "name": "Total Space",
        "data_type": "DT_Byte",
        "conversion": "",
        "incremental": false,
        "timeout" : 0,
        "history": {
            "store": true,
            "duplicate": false,
            "max": 0,
            "expire": 0,
            "round_value": 0
        }
    },
    ...
}

Description¶

Each item has several properties:

• name : it is a string used to display this sensor on the UI

• data_type : the data type used for this sensor

• incremental : if set to True, store the difference between the last value and the current value, check the incremental section for more info.

• timeout : this is just an information which could be used by user interfaces to notify if a value is too old (this may indicate that the sensor is down/offline/broken. This is a value in seconds. If the value is 0, then we consider there is no issue if the last value is very old.

• conversion : if not an empty string, what conversion function to call before storing the value in the db, the function should return the data in the format according to the data_type. More informations in the conversion chapter.

• history : some extra parameters that can be used to define what to store in the history table

  • store : can be True or False, if True the values will be stored in the sensor_history table
  • max : max number of records that will be stored in the history table, if 0 the max number is infinite.
  • expire : how long the history needs to be kept, if 0 the stats will be kept forever. Its counted in days, so a value of 10 means keep the history for 10 days.
  • round_value : a number that will be used for the reduced stats storage. This will only be evaluated if store is True
  • duplicate : if set to true, duplicate values following each other will be stored, if set to false this will not happen

What happens if a sensor value needs to be store¶

A couple of steps are taken during sensor lookups, they are described below:

1. First check if we can match the xpl source to a known plugin
2. Search for a matching xpl_stat message thats stored in the db (can be multiple)
3. Record all sensors + the value to store per xpl_stat
4. For every sensor/value pair check if the value is in the ignore_values, go to step 11
5. For every sensor/value pair run the conversion function if defined
6. For every sensor/value pair handle the incremental field if needed, value = <received value> - <last stored value>
7. For every sensor/value pair handle the duplicate field if needed
8. For every sensor/value pair handle the formula if needed
9. For every sensor/value pair handle the round value
10. for every sensor/value store the value if sensor.store is set
11. Send an mq pub message device-stats with the device id, sensor id and the value

Incremental explanation¶

The parameter is used for sensors like a kWH sensor, these sensors typically send over an absolute value, if we would store this value we would just get a climing chart, so the charts would not be representative. To solve this we introduced the incremental sensor type.

This means that only the difference between the last stored value will be stored, to explain it exactly w’ll work with an example.

For a sensor that has incremental set tue, the following values are received: 1. 10 2. 11 3. 12 4. 15 5. 16 6. 18

So on the receival of the first value, incremental will just keep this value in memory, so it can calculate the difference on the next received values.

On step 2 we will store the difference between the value received on step 1 and the one received in step 2, so we will store 1. The same happens on all the next steps. As a result we will get the following stored values: 1. 0 2. 1 3. 1 4. 3 5. 1 6. 2

This will result in a chart that will really display the used kWH during the time period between the 2 steps.

Round Value explanation¶

The round value is used to reduce the number of stored history values.

As an example why this would be needed: Some device can collect data every 10 seconds, in 24 hours, the device can collect 8640 items a day. If we have 10 such a devices it would result in 86400 items a day or 2678400 items a week. This would be to much data and is not useful. To solve this problem we introduced the round_value key. Basically the round_value key will delete values that fall within a predefined range. An example explains this the best:

Below is a list of received values from a sensor: 1. 10 2. 11 3. 12 4. 9 5. 15 6. 16 7. 18 8. 19 9. 20

So lets see what will happen if round_value is set to 2:

On step 1 and 2 the round value will do nothing as it needs at least 2 stored values to work. On step 3 the round value will do its first action, it will see that the difference between value from step 1 is smaller or equal to round_value key, meaning that it will delete the value received in step 2. On step 4 it will kick in action again, at this point the data in the table is the following * 10 * 12 The received value is 9, as the round_value is set to 2, it will not delete anything. The same happend is step 5 and 6. On step 7 the difference is again <= round_value, so the value received in step 6 is deleted. So if we see what data we will have after value 9 is received:

• 10
• 12
• 9
• 15
• 18
• 20

In this example this will save use 66% of storage space, so for the example in the beginning of this section we will go from 2678400 items a week to 1767744 items a week.

For this system to work its very important that the round_value is set to a logical number for that type of sensor.

Formula Field¶

This is a field that is NOT in the json, but its a sensor parameter. This means it can be updated after sensor creation. With the formula field you can apply a formula to a value that will be stored. The formula calculation is one of the last steps in the sensor history storage.

The formula needs an input parameter and thats the value coming from XPL. To use this value in you formula use VALUE string, it will be replaced before the formula is applied.

example: VALUE - 5 This will store the received XPL value - 5

If the formula is not a valid formula (means it can not be evaluated) the exception is logged in the db_api.log and the original value will be stored in the DB

Purpose¶

The xpl_command section will describe all xPL messages that must be sent by Domogik to trigger an action in the plugin.

Example¶

This example is the velbus plugin. See the device types documentation for more informations.

"xpl_commands": {
     "set_level_bin": {
        "name": "blah",
        "schema": "lighting.basic",
        "xplstat_name": "get_level_bin",
        "parameters": {
            "static": [],
            "device": []
        }
     },
     ...

Purpose¶

The xpl_stats section will describe all xPL messages that may be received by some sensors. No difference is made between xpl-stat and xpl-trig messages : as some xpl client implement these messages not as required in the xPL specification, we process them in the same (this is not an issue as basically xpl-trig and xpl-stat contains the same data : xpl-trig is just used for events and data changes and xpl-stat for all other usages).

Example 1¶

This example is the diskfree plugin. See the device types documentation for more informations.

"xpl_stats": {
   "get_percent_used": {
        "name": "Percent used",
        "schema": "sensor.basic",
        "parameters": {
                "static": [
                    {
                            "key": "type",
                            "value": "percent_used"
                    }
                ],
                "device": [],
                "dynamic": [
                    {
                         "key": "current",
                         "ignore_values": "",
                         "sensor": "get_percent_used"
                    }
                ]
        }
   },
   "get_used_space": {
           ...
   },
   "get_total_space": {
           ...
   },
   "get_free_space": {
           ...
   }
},

Purpose¶

First, as already said in the chapter about creating the plugin repository on GitHub, there are some rules about a plugin name : don’t forget to respect them.

The binary part of a plugin will in all cases be a single file. Assuming your plugin name is myplugin, the file name will be bin/myplugin.py.

This file is the gateway between the library and Domogik. Basically it will:

• import the needed libraries
• check if the plugin is configured
• get the plugin configuration values
• instantiate the library class
• use the library and thanks to callback functions communicate wih Domogik
• set the plugin as ready when it is fully started

Depending on your plugin, more actions may be done:

• after getting the configuration, get the devices list from the database (over the MQ). This is needed when the plugin can’t know the devices addresses, for example a weather plugin don’t know in which town you live, so it will check the weather for already created devices.
• register a newly detected device. For example, if a new sensor is plugged on your onewire network or if you add a new temperature sensor compliant with RFXCOM devices, the related plugins (onewire and rfxcom) will be able to detect the new devices, and if the device is not known, it will be registered, so a notification will be send to the user interfaces over MQ.

XplPlugin and its useful functions¶

..todo ::

overview

Template of the binary part¶

#!/usr/bin/python
# -*- coding: utf-8 -*-

""" This file is part of B{Domogik} project (U{http://www.domogik.org}).

License
=======

B{Domogik} is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

B{Domogik} is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with Domogik. If not, see U{http://www.gnu.org/licenses}.

Plugin purpose
==============

A quick description of the plugin

Implements
==========

MyClassName()
-  __init__()
-  a_function()

@author: The developer name <the.developer@email.com>
@copyright: (C) 2007-2013 Domogik project
@license: GPL(v3)
@organization: Domogik
"""

from domogik.xpl.common.plugin import XplPlugin
from domogik.xpl.common.xplmessage import XplMessage

from domogik_packages.plugin_diskfree.lib.diskfree import Disk
import threading
import traceback

from domogik.xpl.common.plugin import XplPlugin # this one is needed as all the xPL plugins extends from the class XplPlugin. This class will provide you some useful functions to get the configuration parameters values, ...
from domogik.xpl.common.xplmessage import XplMessage # this one is needed to create a xPL message.

from domogik_packages.plugin_diskfree.lib.diskfree import Disk

import threading
import traceback

class MypluginManager(XplPlugin):
    """ A description about the class
    """

    def __init__(self):
        """ The constructor of your class. This function  will be called when the class is instantiated.
        """
        XplPlugin.__init__(self, name='myplugin')
        # do some other actions

if __name__ == "__main__":
    MypluginManager()

Focus on the start process¶

# check if the plugin is configured. If not, this will stop the plugin and log an error
if not self.check_configured():
    return

self.interval = self.get_config("interval")

# get the devices list
self.devices = self.get_device_list(quit_if_no_device = True)

# loop on all found devices
for a_device in self.devices:
    try:
        # get the configuration values for the key 'the_key' for the sensor 'the_sensor' of the device
        the_value = self.get_parameter_for_feature(a_device, "xpl_stats", "the_sensor", "the_key")

        # do something
    except:
        # if there is an error, log it
        self.log.error(traceback.format_exc())
        # if the error is something blocking (a hardware gateway unavailable for example), you may want the plugin to stop.
        # if so, uncomment the following lines
        #self.force_leave()
        #return

# get the devices list
self.devices = self.get_device_list(quit_if_no_device = True)

# instantiate the class of the library
# notice that we sent as parameters some callbacks :
# * the logger object (self.log)
# * the function to send a xPL messages
# * a function to help to stop the plugin
disk_manager = Disk(self.log, self.send_xpl, self.get_stop())

# loop on all found devices
threads = {}
for a_device in self.devices:
    try:
        ### feature get_total_space
        # get the path to check for the sensor 'get_total_space' of the device. The configuration key is 'device'
        path = self.get_parameter_for_feature(a_device, "xpl_stats", "get_total_space", "device")
        # get the interval between each check for the sensor 'get_total_space' of the device. The configuration key is 'interval'
        interval = self.get_parameter_for_feature(a_device, "xpl_stats", "get_total_space", "interval")

        self.log.info("Start monitoring total space for '%s'" % path)
        # start a thread
        # the thread must be named to be explicit in the logs
        thr_name = "{0}-{1}".format(a_device['name'], "get_total_space")
        # create the thread
        threads[thr_name] = threading.Thread(None,
                                       disk_manager.get_total_space,
                                      thr_name,
                                      (path, interval,),
                                      {})
        # start the thread
        threads[thr_name].start()
        self.register_thread(threads[thr_name])


        # [...] 3 other features are managed like this in the plugin

    except:
        # if there is an error, log it
        self.log.error(traceback.format_exc())
        # we don't quit plugin if an error occurred
        # a disk can have been unmounted for a while
        #self.force_leave()
        #return

Perform updating devices process¶

After declaring an instance of an handling update method of devices, you can register it as callback

# register a callback device.update MQ message.
self.register_cb_update_devices(myHandleDeviceUpdate)

Your myHandleDeviceUpdate method will be called each device.update event comming from MQ pub message. That method must have devices list as parameters. Example of refreshing method to register

# A methode to handle updated devices by callback, in Class declaration :
def myHandleDeviceUpdate(self, devices):
    for hardDevice in self._myHardDevices:
        hardDevice.refreshAllDmgDevice(devices)
    self.log.info(u"All hard devives are updated from domogik devices")

Updating global parameters of device

# @param paramId: db id of global parameters getting from dict device['parameters']['myGlobalParam']['id']
# @param value : New parameter value.
# @return : True if success else False.
self.udpate_device_param(param_id, value)

# Your new overwrite methode
def on_message(self, msgid, content):
    Plugin.on_message(self, msgid, content)
    ....

Focus on the end : plugin ready¶

At the end of your __init__ function, just add these 2 lines:

self.ready()
self.log.info("Plugin ready :)")

Focus on devices tools¶

@param name :  the name of DT_Type
@return : dict DT_Type himself, empty dict if not find.
myDataType = self.get_data_type(name)

#    @param productId : part of productId set in json plugin. Search it in lower case if your <myProductId> is contained in product['id'] string.
#    @return : dict of product defined in json plugin with picture file name added. If not find empty dict.
#        {
#            "name" : The name of product,
#            "id" : product id, this is base name of picture,
#            "documentation" : Link to manufacturer documentation, manual, specification,
#            "type": Device_type linked to this product,
#            "picture" : if exist, file name of picture representing product without full path, else None
#       }
self.get_product_by_id(self, myProductId):

#    @param id : command id to find set in json plugin.
#    @return : dict of commands if find else empty dict.
command = self.get_command_by_id(myCommandId):

Focus on the logs¶

There are 4 methods that enable you to log informations in the log files. The default logfiles are located in the directory configured in /etc/domogik/domogik.cfg as log_dir_path (default is /var/log/domogik/). The filename is myplugin.log.

Here are the 4 methods:

• self.log.error("a message"): log an error message
• self.log.warning("a message"): log a warning message
• self.log.info("a message"): log an information message
• self.log.debug("a message"): log a debug message

Focus on xPL : send xPL messages and listen for xPL messages¶

def send_xpl(self, arg1, arg2, ...):
    """ Send xPL message on network
    """
    self.log.debug("Sending xPL message for arg1={0}, arg2={1}...".format(arg1, arg2))
    msg = XplMessage()
    msg.set_type("xpl-stat")
    msg.set_schema("aschema.basic")
    msg.add_data({"key1" : arg1})
    msg.add_data({"key2" : arg2})
    # ...
    self.myxpl.send(msg)

def send_xpl(self, address, type, value):
    """ Send xPL message on network
    """
    self.log.debug("Values for {0} on {1} : {2}".format(type, address, value))
    msg = XplMessage()
    msg.set_type("xpl-stat")
    msg.set_schema("sensor.basic")
    msg.add_data({"device" : address})
    msg.add_data({"type" : type})
    msg.add_data({"current" : value})
    self.myxpl.send(msg)

my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

Focus on launching threads¶

When you want to launch an infinite task, you can use a thread. In this thread you can have an infinite loop with a timer in it or an infinite reading on something.

To be sure that the thread could be stopped when the plugin is requested to stop, you will need to use a stop flag in the library. So you need to give this stop flag to the library, for example when instantiating the library class. Example:

my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

The function self.get_stop() will get the stop flag of the plugin and so give it to my_object. Then, in the methods of the library class, you will be able to handle this flag in the loops. You can read the library documentation to see how to handle it.

Then, you can create your thread. Example:

import threading
# [...]

class MypluginManager(XplPlugin):
    # [...]

    def a_function(self):
        # [...]

        # instantiate the class of the library
        my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

        # set the thread name
        thr_name = "a_name_for_my_thread"
        # create the thread
        my_thread = threading.Thread(None,
                                      my_object.the_function,
                                      thr_name,
                                      (arg1, arg2,),
                                      {})
        # start the thread
        my_thread.start()
        self.register_thread(my_thread)

Notice that a function self.register_thread() has been called. This function is important is it register the thread in a list. When the plugin is requested to stop, this list is used to help killing the existing threads! If your plugin doesn’t stop, please check that all your threads are registered.

Focus on the json file¶

If you need, for some reasons to access the json file content, please notice that:

• you should really ask yourself if the data you are looking for is not available somewhere else (over MQ or over REST).
• the json content is available in self.json_data. Don’t write in it!!!

Purpose¶

Some plugins can see all devices for the related hardware or service. For example, rfxcom plugin can catch all messages of all devices seen by the Rfxcom product. The onewire can also see all devices when reading the onewire bus. When a device which is not known from Domogik is detected, a message is sent over the MQ and catched by the user interfaces for being displayed.

How to implement this ?¶

# get the devices list
# for this plugin, if no devices are created we won't be able to use devices.
# but.... if we stop the plugin right now, we won't be able to detect existing device and send events about them
# so we don't stop the plugin if no devices are created
self.devices = self.get_device_list(quit_if_no_device = False)

self.rfxcom_manager = Rfxcom(..., self.device_detected, ...)

class Rfxcom:
    """ Rfxcom
    """

    def __init__(self, ..., cb_device_detected, ...):
        """ Some blah blah
            @param ...
            @param cb_device_detected : callback to handle detected devices
            @param ...
        """
        ...

        self.cb_device_detected = cb_device_detected

        ...

# handle device features detection
for feature in ['temperature', 'humidity']:
    self.cb_device_detected(device_type = "rfxcom.temperature_humidity",
                            type = "xpl_stats",
                            feature = feature,
                            data = {"device" : address,
                                    "reference" : model})

"device_types": {
    "rfxcom.temperature": {
        "description": "",
        "id": "rfxcom.temperature",
        "name": "Temperature sensors",
        "commands": [],
        "sensors": ["temperature", "battery", "rssi"],
        "parameters": [
            {
                "key": "device",
                "xpl" : true,
                "description": "Device address. Example: th9 0xFFFF",
                "type": "string"
            }
        ]
    },

Purpose¶

The library part of a plugin can be the first step of your plugin. It you write it in the good way, this library may be usable out of Domogik!

In most cases, a library will contain a class. This class will have a constructor, some functions for actuators features (these functions will be called from the binary part when some xPL messages are catched), some functions for the sensor part (in most cases, it will be an eternal loop to listen to the hardware or to call some services). The sensor functions will be able to send xPL messages by calling some callback functions which refers to functions in the binary part.

Template¶

..todo ::

• inifinite loop
• timer

Focus on the logs¶

You already learned how to use the log functions in the library part documentation. Let’s see how to do this in the library part.

..todo ::

TODO :)

Focus on xPL : send xPL messages and listen for xPL messages¶

my_object = MyLibraryClass(self.log, self.send_xpl, self.get_stop())

class MyLibraryClass:
    """ My class
    """

    def __init__(self, log, callback, stop):
        """ Init MyLibraryClass
            @param log : log instance
            @param callback : callback
            @param stop : stop flag
        """
        self._log = log
        self._callback = callback
        self._stop = stop

def a_function(self):
    """ My function
    """
    # do some things
    arg1 = "abc"
    arg2 = 234
    self._callback(arg1, arg2)

Focus on devices¶

README.md¶

The README.md file should contain a quick description of the plugin, and invite the user to find more informations on http://docs.domogik/org/

The .md extension is for Markdown which is a text markup language. You can find more information about Markdown on wikipedia

You may create the file like this:

$ echo " # Purpose

This is a package for Domogik : http://www.domogik.org

Domogik is an open source home automation solution.

# Documentation

You can find the documentation source in the **docs/** folder. When the package will be installed, the documentation will be available in the **Documentation** menu of the Domogik administration for this package.
You may also find online documentation for this plugin. You will be able to find the documentation url on http://repo-public.domogik.org/dashboard

# Install the package

To install this package on your Domogik system, you can go in this GitHub repository releases page and get the link to a release .zip file. Then you just have to do :

    dmg_package -i http://path.to/the/file.zip" > README.md

CHANGELOG¶

The real changelog file must be located in docs/changelog.txt. To allow the user to find it quickly, please create a CHANGELOG in the root directory like this:

echo "The changelog informations are available in docs/changelog.txt" > CHANGELOG

Global rule¶

Here is the main way to choose the version of a Domogik package : <major>.<minor>. If needed, you may use alpha and beta notations : 1.0a1, 1.0b3. In all cases, the version number must respect the PEP 386.

First release of the package¶

The first release of your package should be named 0.1. The following releases should also be named 0.x until the package is fully functional : then you will switch to 1.0 version number. You may want to use some alpha, beta or candidates releases like 1.0a1, 1.0b3, 1.0c1 before using the final 1.0 version number.

Next releases of the package¶

After the 1.0 has been published, if you need to make bugfixes or little evolutions, you may use 1.1, 1.2, ... If you want, you can also use alpha, beta and candidates : 1.1c1.

If you have have to make big evolutions in your package, you may switch to the next major version : 2.0.

Purpose¶

The start.sh file is only an helper for the developers and people who want to test the plugin from the command line. It will override the PYTHONPATH environment variable and launch the plugin.

Create the start.sh¶

Assuming your plugin name is myplugin, create the start.sh file like this (you just need to update the first line):

$ PLUGIN_ID=myplugin
$ echo "export PYTHONPATH=/var/lib/domogik && /usr/bin/python bin/$PLUGIN_ID.py -f" > start.sh
$ chmod u+x start.sh

Use it¶

To launch the plugin in foreground (it is better during the plugin development), just launch start.sh:

$ cd /var/lib/domogik/domogik_packages/plugin_myplugin
$ ./start.sh

The plugin will be launched in foreground. To stop it, you can use (ctrl)-(C).

When should the tests be launched ?¶

Well, after each plugin update! But as this could be time consuming, you can automate this step thanks to the testrunner.py tool and Travis, the continuous integration service.

File tree¶

The following files are mandatory for the tests:

tests/
  # the 0* files are just helpers for the developers
  001_configure.py     # this python file is used by the developers to quickly configure the plugin
  002_create_device.py # this python file is used by the developers to quickly create some test devices

  # all the other files are related to the plugin tests
  tests.json          # this is a file which describe all the test files. It is used for tests automation

Then, depending on your plugin, you can have only one test file:

tests/
  ..
  tests.py

Or several files:

tests/
  ..
  test_feature_A.py
  test_feature_B.py
  test_feature_C.py

tests.json¶

This file is very important! It will be used by the testrunner.py tool.

Example:

{
    "tests" : {
        "alter_configuration_or_setup" : true,
        "need_hardware" : false,
        "criticity" : "high"
    }
}

• alter_configuration_or_setup: true if the test need to alter the plugin configuration or some devices. The test should not be run on a production environment! false if the test doesn’t alter anything and can be run safely on a production environment.
• need_hardware: true if some hardware is needed by the test. Please note that the testrunner.py tool will never run the tests that need some hardware.
• criticity: high, medium or low.

Each test file must be listed in the tests.json file.

A test file¶

A test file is made of 2 parts:

• a class which inherits from PluginTestCase. This class will contain all the test cases related to the plugin.
• the main part which will do some actions and launch the test cases.

Here is a sample file from the teleinfo plugin. This sample file has only one dummy test defined. This is the minimal file you must prepare before creating the tests. This test file will test only global features :

• deletion and creation of devices
• plugin configuration
• plugin startup
• xpl hbeat
• plugin stop successfully

#!/usr/bin/python
# -*- coding: utf-8 -*-

from domogik.xpl.common.plugin import XplPlugin
from domogik.tests.common.plugintestcase import PluginTestCase
from domogik.tests.common.testplugin import TestPlugin
from domogik.tests.common.testdevice import TestDevice
from domogik.tests.common.testsensor import TestSensor
from domogik.common.utils import get_sanitized_hostname
from datetime import datetime
import unittest
import sys
import os
import traceback

class TeleinfoTestCase(PluginTestCase):

    def test_0100_dummy(self):
        self.assertTrue(True)

if __name__ == "__main__":
    ### global variables
    device = "/dev/teleinfo"
    interval = 60

    # set up the xpl features
    xpl_plugin = XplPlugin(name = 'test',
                           daemonize = False,
                           parser = None,
                           nohub = True,
                           test  = True)

    # set up the plugin name
    name = "teleinfo"

    # set up the configuration of the plugin
    # configuration is done in test_0010_configure_the_plugin with the cfg content
    # notice that the old configuration is deleted before
    cfg = { 'configured' : True }

    ### start tests
    # load the test devices class
    td = TestDevice()

    # delete existing devices for this plugin on this host
    client_id = "{0}-{1}.{2}".format("plugin", name, get_sanitized_hostname())
    try:
        td.del_devices_by_client(client_id)
    except:
        print(u"Error while deleting all the test device for the client id '{0}' : {1}".format(client_id, traceback.format_exc()))
        sys.exit(1)

    # create a test device
    try:
        #device_id = td.create_device(client_id, "test_device_teleinfo", "teleinfo.electric_meter")

        params = td.get_params(client_id, "teleinfo.electric_meter")

        # fill in the params
        params["device_type"] = "teleinfo.electric_meter"
        params["name"] = "test_device_teleinfo"
        params["reference"] = "reference"
        params["description"] = "description"
        # global params
        for the_param in params['global']:
            if the_param['key'] == "interval":
                the_param['value'] = interval
            if the_param['key'] == "device":
                the_param['value'] = device
        print params['global']
        # xpl params
        pass # there are no xpl params for this plugin
        # create
        td.create_device(params)

    except:
        print(u"Error while creating the test devices : {0}".format(traceback.format_exc()))
        sys.exit(1)

    ### prepare and run the test suite
    suite = unittest.TestSuite()
    # check domogik is running, configure the plugin
    suite.addTest(TeleinfoTestCase("test_0001_domogik_is_running", xpl_plugin, name, cfg))
    suite.addTest(TeleinfoTestCase("test_0010_configure_the_plugin", xpl_plugin, name, cfg))

    # start the plugin
    suite.addTest(TeleinfoTestCase("test_0050_start_the_plugin", xpl_plugin, name, cfg))


    # do the specific plugin tests
    suite.addTest(TeleinfoTestCase("test_0100_dummy", xpl_plugin, name, cfg))

    # do some tests comon to all the plugins
    suite.addTest(TeleinfoTestCase("test_9900_hbeat", xpl_plugin, name, cfg))
    suite.addTest(TeleinfoTestCase("test_9990_stop_the_plugin", xpl_plugin, name, cfg))

    # quit
    res = unittest.TextTestRunner().run(suite)
    if res.wasSuccessful() == True:
        rc = 0   # tests are ok so the shell return code is 0
    else:
        rc = 1   # tests are ok so the shell return code is != 0
    xpl_plugin.force_leave(return_code = rc)

A test file : the class which inherits from PluginTestCase¶

In this class, you will define the tests that will be executed on the plugin.

There is a norm to name the functions in this class: test_9999_thetestname.

• test_0xxx_xxx : these functions are reserved, declared in PluginTestCase and related to the preparation of the plugin.
• test_9xxx_xxx : these functions are reserved, declared in PluginTestCase and related to the end of the plugin tests (hbeat test, plugin stop).
• test_1xxx_xxx to test_8xxx_xxx : these functions are free for use.

class DiskfreeTestCase(PluginTestCase):

    def test_0100_dummy(self):
        self.assertTrue(True)

self.assertTrue(self.wait_for_xpl(xpltype = "xpl-stat",
                                  xplschema = "sensor.basic",
                                  xplsource = "domogik-{0}.{1}".format(self.name, get_sanitized_hostname()),
                                  data = {"type" : "total_space",
                                          "device" : path,
                                          "current" : du_total},
                                  timeout = interval * 60))

current_value = self.xpl_data.data['current']

print(u"Check that the value of the xPL message has been inserted in database")
sensor = TestSensor(device_id, "get_total_space")
self.assertTrue(sensor.get_last_value()[1] == self.xpl_data.data['current'])

# get the first message
self.assertTrue(self.wait_for_xpl(xpltype = "xpl-stat",
                                  ...
                                  timeout = 2 * 60))
# get the time of the first message
msg1_time = datetime.now()

# get the second message
self.assertTrue(self.wait_for_xpl(xpltype = "xpl-stat",
                                  ...
                                  timeout = 2 * 60))
# get the time of the second message
msg2_time = datetime.now()

# check if the interval between the 2 messages is OK
self.assertTrue(self.is_interval_of(2 * 60, msg2_time - msg1_time))

def test_0110_total_space(self):
    """ check if the xpl messages about total space are OK
        Sample message :
        xpl-stat
        {
        hop=1
        source=domogik-diskfree.darkstar
        target=*
        }
        sensor.basic
        {
        device=/home
        type=total_space
        current=19465224
        }
    """
    global interval
    global path
    global device_id


    # get the current total space on the device
    du = os.statvfs(path)
    du_total = (du.f_blocks * du.f_frsize) / 1024

    # do the test
    print(u"Check that a message about total space is sent. The message must be received each {0} minute(s)".format(interval))

    self.assertTrue(self.wait_for_xpl(xpltype = "xpl-stat",
                                      xplschema = "sensor.basic",
                                      xplsource = "domogik-{0}.{1}".format(self.name, get_sanitized_hostname()),
                                      data = {"type" : "total_space",
                                              "device" : path,
                                              "current" : du_total},
                                      timeout = interval * 60))
    print(u"Check that the value of the xPL message has been inserted in database")
    sensor = TestSensor(device_id, "get_total_space")
    self.assertTrue(sensor.get_last_value()[1] == self.xpl_data.data['current'])
    msg1_time = datetime.now()

    print(u"Check there is a second message is sent and the interval between them")
    self.assertTrue(self.wait_for_xpl(xpltype = "xpl-stat",
                                      xplschema = "sensor.basic",
                                      xplsource = "domogik-{0}.{1}".format(self.name, get_sanitized_hostname()),
                                      data = {"type" : "total_space",
                                              "device" : path,
                                              "current" : du_total},
                                      timeout = interval * 60))
    msg2_time = datetime.now()
    self.assertTrue(self.is_interval_of(interval * 60, msg2_time - msg1_time))

A test file : the main part¶

Here are the actions that can be done in the main part:

• if needed, define some global variables (polling interval, ...). Example:

  ### global variables
  interval = 1
  path = "/home"
  

• set up the xpl features for the test file. A XplPlugin instance will be created with some special parameters (please always use these parameters, even the generic name). Example:

  # set up the xpl features
  xpl_plugin = XplPlugin(name = 'test',
                         daemonize = False,
                         parser = None,
                         nohub = True,
                         test  = True)
  

• set up the plugin name:

  # set up the plugin name
  name = "diskfree"
  

• define the configuration of the plugin. If no configuration is required for the plugin, at least you must set up the configured key to True. Example:

  # set up the configuration of the plugin
  # configuration is done in test_0010_configure_the_plugin with the cfg content
  # notice that the old configuration is deleted before
  cfg = { 'configured' : True }
  

• start the common tests by setting up the TestDevice class which helps to manage the devices. Example:

  ### start tests
  # load the test devices class
  td = TestDevice()
  

• if needed, delete all the existing devices of the plugin on the current host. If you do this, you must set alter_configuration_or_setup to True in the json. Example:

  # delete existing devices for this plugin on this host
  client_id = "{0}-{1}.{2}".format("plugin", name, get_sanitized_hostname())
  try:
      td.del_devices_by_client(client_id)
  except:
      print(u"Error while deleting all the test device for the client id '{0}' : {1}".format(client_id, traceback.format_exc()))
      sys.exit(1)
  

• if needed, create some devices. If you do this, you must set alter_configuration_or_setup to True in the json. Notice that the device parameters should come from the global variables defined before. Example (here 2 global parameters are defined but no xpl parameters are defined):

  # create a test device
  try:
      #device_id = td.create_device(client_id, "test_device_teleinfo", "teleinfo.electric_meter")
  
      params = td.get_params(client_id, "teleinfo.electric_meter")
  
      # fill in the params
      params["device_type"] = "teleinfo.electric_meter"
      params["name"] = "test_device_teleinfo"
      params["reference"] = "reference"
      params["description"] = "description"
      # global params
      for the_param in params['global']:
          if the_param['key'] == "interval":
              the_param['value'] = interval
          if the_param['key'] == "device":
              the_param['value'] = device
      print params['global']
      # xpl params
      pass # there are no xpl params for this plugin
      # create
      td.create_device(params)
  
  except:
      print(u"Error while creating the test devices : {0}".format(traceback.format_exc()))
      sys.exit(1)
  

• then, call the common tests related to the plugin. These tests are common to all plugins and are defined in the class PluginTestCase. The first one will just check if Domogik is running (if not, the plugin will not be able to start). The second one will configure the plugin and the last one will start the plugin. Example:

  ### prepare and run the test suite
  suite = unittest.TestSuite()
  # check domogik is running, configure the plugin
  suite.addTest(DiskfreeTestCase("test_0001_domogik_is_running", xpl_plugin, name, cfg))
  suite.addTest(DiskfreeTestCase("test_0010_configure_the_plugin", xpl_plugin, name, cfg))
  
  # start the plugin
  suite.addTest(DiskfreeTestCase("test_0050_start_the_plugin", xpl_plugin, name, cfg))
  

• launch all the tests you created in the YourpluginTestCase class. Example:

  # do the specific plugin tests
  suite.addTest(DiskfreeTestCase("test_0110_total_space", xpl_plugin, name, cfg))
  suite.addTest(DiskfreeTestCase("test_0120_free_space", xpl_plugin, name, cfg))
  suite.addTest(DiskfreeTestCase("test_0130_used_space", xpl_plugin, name, cfg))
  suite.addTest(DiskfreeTestCase("test_0140_percent_used", xpl_plugin, name, cfg))
  

• launch some common tests related to the plugin stopping process. The first one will check that the plugin sends hbeat messages and can take several minutes! The second one will try to stop the plugin and check if the plugin can be stopped. Example:

  # do some tests common to all the plugins
  suite.addTest(DiskfreeTestCase("test_9900_hbeat", xpl_plugin, name, cfg))
  suite.addTest(DiskfreeTestCase("test_9990_stop_the_plugin", xpl_plugin, name, cfg))
  

• and finally get the status of the tests. If there were some errors, the python test file will return 1. This is very important for the continuous integration tools and testrunner. Example:

  # quit
  res = unittest.TextTestRunner().run(suite)
  if res.wasSuccessful() == True:
      rc = 0   # tests are ok so the shell return code is 0
  else:
      rc = 1   # tests are ok so the shell return code is != 0
  xpl_plugin.force_leave(return_code = rc)
  

if __name__ == "__main__":

    ### global variables

if __name__ == "__main__":

    test_folder = os.path.dirname(os.path.realpath(__file__))

    ### global variables

# specific configuration for test mdode (handled by the manager for plugin startup)
cfg['test_mode'] = True
cfg['test_option'] = "{0}/tests_hchp_data.json".format(test_folder)

teleinfo_list[device] = Teleinfo(self.log, self.send_xpl, self.get_stop(), device, interval)

teleinfo_list[device] = Teleinfo(self.log, self.send_xpl, self.get_stop(), device, interval, self.options.test_option)