Welcome to HAPI’s documentation!

The Hydoponic Automation Platform Initiative (HAPI) is a suite of tools that will allow people to grow a variety of food in diverse environments. Our primary focus is on building intelligent automation for hydroponics and aquaponics. HAPI is currently under heavy development.

To get started with HAPI check out the System Overview. You can find out more about the project on the website.

Contents:

System Overview

The HAPI system is based on low cost micro-controllers providing sufficient computing power to monitor and control all local production resources. These resources are comprised of the sensors that monitor the environment and growing conditions, and the controls that affect these conditions. The results of these activities are published using a standard messaging protocol called MQTT. The information that is published can then be used to analyze the conditions, monitor the controls, and change the operation of the system.

Below is a draft of the HAPI system architecture. Please note it is not fully up to date

_images/system-overview.png

Components

HAPImodule

HAPImodules are devices with a high computational capability that run system, sensor and control code. Currently the supported hardware for Smart Modules is a Raspberry Pi Zero W or 3B. Multiple HAPImodules can exist within the system and collaborate to determine the tasks performed by each module. One HAPImodule is the MQTT Broker (Server) and is responsible for storing all the data being transmitted. If for some reason this Server is unreachable (configuration problems or hardware failure, for instance), any other HAPImodule within the system can step up and take its place.

Features:

  • Focused on the needs of Urban and Controlled-Environment Agriculture
  • Supports multiple sites for a single user
  • Open source designs and code (user isn’t stuck with proprietary technology)
  • Wide & growing range of sensor support
  • User-definable Alerts
  • User-definable Schedule
  • Control equipment based on schedule, sensor values or manually.
  • WiFi Capable (configured via FMS Application)
  • Seamlessly integrate with other Smart Modules
  • Integrated, battery-backed, real-time clock
  • Simple, durable design and enclosure
  • Waterproof, weather-resistant enclosures
  • Optional data push to cloud visualization services
  • Automatic fail-over of communications and scheduling functions (fault-tolerant mesh)
  • Minimal User Configuration (Set WiFi info and you’re done)

HAPInode

The hardware for the HAPInode is based on arduino. Since the main focus of HAPI is a system wide automation, arduinos that can easily incorporate WiFi and Bluetooth Low Energy are preferred. Conditional defines in the arduino software allow for ESP32, ESP8266 and mega2560 w/Ethernet modules. The preferred development module is (https://learn.sparkfun.com/tutorials/esp32-thing-hookup-guide), which includes the necessary WiFi, Bluetooth, external power and external battery interfaces.

Features:

  • Run on Arduino, ESP8266 (e.g. NodeMCU) or ESP32 (e.g. WROOM32)
  • Targeted to DIY/Makers, Technicians, Experimenters, Hobbyists
  • User-configurable sensor and control configurations
  • Seamlessly integrate with other HAPI Smart Modules and Nodes
  • Open source designs and code

Facility Management System (FMS) Android App

The FMS is an Android application for managing and monitoring the HAPI system.

  • Configure WiFi settings for Smart Modules
  • See all sensor values at a touch
  • Manage schedule for collecting sensor data, weather data, system health and checking alerts
  • Set Alert parameters and notification preferences
  • View data in Standard or Metric Units
  • Open source designs and code
  • Integration with Weather Underground (optional via free subscription)
  • Access to Visualization Dashboards (optional via paid subscription)

System Features

Job scheduling

The HAPI platform features an in-process scheduler for running periodic jobs. Jobs can be control functions, such as turning a light or pump on. Jobs can also serve monitoring purposes, such as gathering sensor data. Job information is stored in the database, in the table schedule. Any Smart Module can run the job scheduling code (i.e. “become the Scheduler”). HAPI Nodes cannot become the Scheduler.

Technical Description

When a Smart Module starts, it publishes on message on the MQTT topic SCHEDULER/QUERY, essentially asking for the Scheduler to respond. If another module on the network is running the job scheduling code, it responds to the to the message and no further action is taken. If there is no response to the message, the starting module runs the job scheduling code itself, announces that it is now the Scheduler and listens on the SCHEDULER/QUERY topic for other Smart Modules that are discovering the scheduler.

The HAPI Scheduler uses [the schedule package](https://pypi.python.org/pypi/schedule) created by Daniel Badr. Documentation can be [found here](https://schedule.readthedocs.io/en/stable/).

Auto-Discovery

All modules (Smart or Dumb) have the capability to automatically detect and connect to one another without user configuration or intervention. To accomplish this, the HAPI platform uses a zero-configuration networking implementation called Avahi. Similar to Apple’s Bonjour, Avahi allows modules to dynamically name themselves and discover neighboring modules.

Technical Description

For Smart Modules, the original Raspian images are modified to include the avahi daemon. When a module boots, it automatically runs this daemon. When a Smart Module starts, to attempts to contact the MQTT broker, “mqttbroker.local”. This machine is acts as the communications hub for all HAPI-based facilities. If the Smart Module’s search for the MQTT broker is not successful, it changes it’s own name to “mqttbroker.local” and becomes the communications hub for the site.

HAPImodule

Release 1 of the HAPI system uses Raspberry Pi Zeros (HAPiZ) as the Smart Modules that run the system code, the sensor code and the control code. Multiple HAPiZ devices can exist within the system and collaborate to determine the tasks performed by each module.

Hardware Setup

A minimum HAPiZ module consists of the following components:

  • Raspberry Pi Zero

  • Real-time-clock (DS3231)

  • Temperature and humidity sensor (DHT22)

    _images/DHT22.jpg

  • Water temperature sensor (DS18B20)

    _images/DS18B20.jpg

Prototype Diagram

_images/HAPImodule.png

Software Setup

Note: Soon we’ll introduce configuration via regular file and/or database. For now all configuration are hardcoded.

  1. Install Raspbian on the Raspberry Pi
  1. Install dependencies on the Pi

  • Dependencies:

    • Avahi (daemon) configured to publish MQTT service.

      • Example config

      • Restart to apply changes:

        systemctl restart avahi-daemon.service

    • MQTT Mosquitto with default configuration.

    • Influxdb with default configuration.

    • Grafana [Optional] (highly recommended).

  • There is also a script from the repo for installing the first two system dependencies and then setup the python environment:

    sudo apt-get install git
cd ~
git clone https://github.com/mayaculpa/hapi.git
cd ~/hapi/src/smart_module
./INSTALL.sh
    • It is good practice to look over scripts you download from the Internet before running them.

Usage

Start the program:

python smart_module.py

You should get output like this:

$ python smart_module.py
2017-05-15 22:37:55.089210 - communicator.log - INFO - Communicator initialized
Mock Smart Module hosting asset  HSM-WT123-MOCK wt Environment
2017-05-15 22:37:55.091207 - smartmodule.log - INFO - Performing Discovery...
2017-05-15 22:37:55.091782 - smartmodule.log - INFO - Waiting Broker information on attempt: 1.
2017-05-15 22:37:56.092877 - smartmodule.log - INFO - MQTT Broker: ArchMain.local. IP: 192.168.0.99.
2017-05-15 22:37:56.093225 - communicator.log - INFO - Connecting to ArchMain.local. at 192.168.0.99.
2017-05-15 22:37:57.094778 - communicator.log - INFO - Connected with result code 0
$SYS/broker/clients/total 0
$SYS/broker/clients/total 1
2017-05-15 22:38:02.648088 - smartmodule.log - INFO - No Scheduler found. Becoming the Scheduler.
2017-05-15 22:38:02.648342 - scheduler.log - INFO - Loading Schedule Data...
2017-05-15 22:38:02.648997 - scheduler.log - INFO - Schedule Data Loaded.
2017-05-15 22:38:02.649105 - scheduler.log - INFO -   Loading seconds job: System Status.
2017-05-15 22:38:02.649160 - scheduler.log - INFO -   Loading seconds job: Check Alert.
2017-05-15 22:38:02.649627 - smartmodule.log - INFO - Scheduler program loaded.
2017-05-15 22:38:02.650200 - smartmodule.log - INFO - Site data loaded.
Running command self.smart_module.on_query_status()
Running command self.smart_module.on_check_alert()
STATUS/QUERY I might need to know how you are!
ASSET/QUERY/HSM-WT123-MOCK Is it warm here?
STATUS/RESPONSE [{'memory': {'cached': 913498112, 'used': 2294038528, 'free': 533913600}, 'disk': {'total': 52472872960, 'free': 36725215232, 'used': 13051768832}, 'network': {'packet_recv': 558630, 'packet_sent': 601295}, 'time': 1494898693.364454, 'hostname': 'ArchMain', 'boot': '2017-05-15 17:09:17', 'cpu': {'percentage': 3.2}, 'clients': 1}]
ASSET/RESPONSE/HSM-WT123-MOCK 8.0
2017-05-15 22:38:13.892977 - alert.log - INFO - Fetching alert param. from database
2017-05-15 22:38:13.893555 - alert.log - INFO - ALERT DETECTED. Value: 8.0.
2017-05-15 22:38:14.283729 - smartmodule.log - INFO - Wrote to analytic database: [{'fields': {'unit': 'C', 'value': '8.0'}, 'tags': {'site': u'HPF-0', 'asset': 'Indoor Temperature'}, 'time': '2017-05-15 22:38:14.010127', 'measurement': 'Environment'}].

An important note: we’re currently using sqlite3 database to load schedule jobs and others information. You can check/use a demo of the database here: database-example For now you should place it on the same folder as smart_module.py and name it as hapi_core.db.

HAPInode

Hardware Setup

The hardware for the HAPInode is based on arduino. Since the main focus of HAPI is a system wide automation, arduinos that can easily incorporate WiFi and Bluetooth Low Energy are preferred. Conditional defines in the arduino software allow for ESP32, ESP8266 and mega2560 w/Ethernet modules. The preferred development module is https://learn.sparkfun.com/tutorials/esp32-thing-hookup-guide, which includes the necessary WiFi, Bluetooth, external power and external battery interfaces.

Prototype Diagram

_images/HAPInode.png

Software Setup

Using the Arduino IDE download and flash the board with the HAPInode code.

MQTT Topics

In the HAPI system devices are either modules or nodes. Both types of devices can have sensors and controls attached to them. Modules are based on Raspberry Pis, while nodes are based on arduinos. The communication of sensor data or control commands uses the MQTT protocol. This protocol has two important operations called publishing and subscribing. In the HAPI system, a local broker is responsible for these operations.

The HAPI philosophy is that any module can assume the role of local broker. It then has the role of publishing controls based on a schedule, on sensor values or on exceptions, as well as co-ordinating the collection and storage of the sensor data.

A HAPI device can publish its sensor data and subscribe to controls for actions. A HAPI device could also subscribe to sensor data for processing and generate controls based on schedules or on these data values.

A HAPImodule can both publish and subscribe to data as well as publish and subscribe to controls. A HAPImodule can also assume the role of the MQTT local broker. A HAPInode can only publish data and subscribe to controls.

Device Naming

Each device has a unique ID, called the deviceID, derived from its type and the unique mac address that is hardcoded into the device at the time of manufacture. Each sensor or control is called an asset and has an assetID that is unique to the function of the asset. The deviceID and the assetID are used to uniquely identify the topic that sensor data is published to or that control information is received from.

HAPImodule

HAPImodules have a deviceID of the form HN1xxxxxx, where HN1 identifies that this is a module (Raspberry Pi Zero) and xxxxxx is the low three bytes of the mac address of its WiFi module. (The high three bytes identify the manufacturer of the WiFi integrated circuit). In python, the mac address is found using: from uuid import getnode as get_mac mac = get_mac()

HAPInode

Each node has a unique ID derived from its type (a node and its arduino type) and the unique mac address that is hardcoded into the device at the time of manufacture. HAPInodes have a name of the form HNnxxxxxx, where HNn identifies that this is a node (ESP32(HN5), ESP8266(HN4), or mega2560(HN3)) and xxxxxx is the low three bytes of the mac address of its WiFi module. (The high three bytes identify the manufacturer of the WiFi integrated circuit). In arduino with a WiFi module, the mac address is found using: `mac = WiFi(mac); `

MQTT topics

Each topic is built from the nature of the topic, the activity to be undertaken, and optionally, the deviceID and the assetID. The first part is the nature of the topic, e.g. STATUS, ASSET, EXCEPTION, CONFIG The next part is the activity associated with that topic, e.g. QUERY, RESPONSE, SET, CLEAR

Sample topics

Status Query

Query the status of ALL devices topic: STATUS/QUERY/ message: {“device”: “*”}

Status Response

topic: STATUS/RESPONSE/HN1123456 Message contains JSON encoded fields identifying the device, name, and assets. message: {“device”: “HN1123456”, “name”: “TomatoBay1”, “Asset”: “[hum, tmp, wtm, lux]”} Note that multiple messages may be generated to identify all the assets associated with a device, as the maximum MQTT payload length is limited to approximately 96 bytes, or 128 byte message length.

Asset Query

Query the value of the hum (humidity) asset of all devices with humidity assets topic: ASSET/QUERY/+/hum message: {“device”: “*”}

Asset Response

topic: ASSET/RESPONSE/HN1123456/hum Message contains JSON encoded fields identifying the device, time, asset value, and units. message: {“device”: “HN1123456”, “time”: 123456, “hum”: 67, “units”: “%”}