Welcome!¶
OxiTopped is a small suite of utilies for extracting data from an OxiTop OC110 data logger via a serial (RS-232) port and dumping it to a specified file in various formats. Options are provided for controlling the output, and for listing the content of the device.
Disclaimer¶
OxiTopped is not affiliated with, or endorsed by WTW GmbH in any way. This is a personal project to provide an interface to the OxiTop OC110 on alternate platforms.
Warning
OxiTopped is currently incomplete. My understanding of the serial protocol used to communicate with the device is probably sufficient to retrieve data from pressure mode runs (including manual measurements, although this is not currently implemented), but retrieval of data from BOD mode runs should be considered experimental at best.
Contents¶
Installation¶
oxitopped is distributed in several formats. The following sections detail installation on a variety of platforms.
Pre-requisites¶
Where possible, I endeavour to provide installation methods that provide all pre-requisites automatically - see the following sections for platform specific instructions.
If your platform is not listed (or you’re simply interested in what oxitopped depends on): oxitopped depends primarily on pyserial. If you wish to use the GUI you will also need PyQt4 installed.
Additional optional dependencies are:
- xlwt - required for Excel writing support
- matplotlib - required for graphing support
Ubuntu Linux¶
For Ubuntu Linux, it is simplest to install from the Waveform PPA as follows (this also ensures you are kept up to date as new releases are made):
$ sudo add-apt-repository ppa://waveform/ppa
$ sudo apt-get update
$ sudo apt-get install oxitopped
Microsoft Windows¶
On Windows it is simplest to install from the standalone MSI installation package available from the oxitopped homepage. Be aware that the installation package requires administrator privileges.
Apple Mac OS X¶
XXX To be written
Other Platforms¶
If your platform is not covered by one of the sections above, oxitopped is
available from PyPI and can therefore be installed with the Python distribute
pip
tool:
$ pip install oxitopped
Theoretically this should install the mandatory pre-requisites, but optional pre-requisites require suffixes like the following:
$ pip install "oxitopped[GUI,XLS]"
Please be aware that at this time, the PyQt package does not build “nicely”
under pip
. If it is available from your distro’s package manager I strongly
recommend using that as your source of this pre-requisite.
If PyQt is not provided by your distro (or you’re on some esoteric platform without a package manager), you can try following the instructions on the Veusz wiki for building PyQt (and SIP) under a virtualenv sandbox.
Development¶
If you wish to develop oxitopped, you can install the pre-requisites, construct a virtualenv sandbox, and check out the source code from GitHub with the following command lines:
# Install the pre-requisites
$ sudo apt-get install python-matplotlib python-xlwt python-qt4 python-virtualenv python-sphinx make git
# Construct and activate a sandbox with access to the packages we just
# installed
$ virtualenv --system-site-packages sandbox
$ source sandbox/bin/activate
# Check out the source code and install it in the sandbox for development and testing
$ git clone https://github.com/waveform-computing/oxitopped.git
$ cd oxitopped
$ make develop
The above instructions assume you are on Ubuntu Linux. Please feel free to extend this section with instructions for alternate platforms.
oxitoplist¶
This utility lists the sample results stored on a connected OxiTop Data Logger. If bottle-serial values are specified, the details of those bottles and all heads attached to them will be displayed, otherwise a list of all available bottle serials provided. The bottle-serial values may include *, ?, and [] wildcards.
Synopsis¶
$ oxitoplist [options] [bottle-serial]...
Description¶
-
--version
¶
show program’s version number and exit
-
-h
,
--help
¶
show this help message and exit
-
-q
,
--quiet
¶
produce less console output
-
-v
,
--verbose
¶
produce more console output
-
-l
LOGFILE
,
--log-file
=LOGFILE
¶ log messages to the specified file
-
-P
,
--pdb
¶
enables debug mode (runs under PDB)
-
-p
PORT
,
--port
=PORT
¶ specify the port which the OxiTop Data Logger is connected to. This will be something like /dev/ttyUSB0 on Linux or COM1 on Windows
-
-r
,
--readings
¶
if specified, output readings for each head after displaying bottle details
-
-a
,
--absolute
¶
if specified with –readings, output absolute pressure values instead of deltas against the first value
-
-m
POINTS
,
--moving-average
=POINTS
¶ if specified with –readings, output a moving average over the specified number of points instead of actual readings
Examples¶
The basic usage of oxitoplist is to dump a list of the bottles stored on the connected device:
$ oxitoplist -p /dev/ttyUSB0
Serial ID Started Finished Complete Mode Heads
--------- --- ---------- ---------- -------- ------------ -----
110222-06 999 2011-02-22 2011-03-08 Yes Pressure 14d 1
121119-03 3 2012-11-19 2012-11-22 Yes Pressure 3d 1
120323-01 1 2012-03-23 2012-04-20 Yes Pressure 28d 2
3 results returned
If one or more bottle-serial numbers are listed on the command line (which may include wildcards), the details of the bottles listed are output instead:
$ oxitoplist -p /dev/ttyUSB0 12*
Serial 121119-03
ID 3
Started 2012-11-19 13:53:04
Finished 2012-11-19 13:53:04
Readings Interval 0:12:00
Completed Yes
Mode Pressure 3d
Bottle Volume 510.0ml
Sample Volume 432.0ml
Dilution 1+0
Desired no. of Values 360
Actual no. of Values 0
Heads 1
Serial 120323-01
ID 1
Started 2012-03-23 17:32:23
Finished 2012-03-23 17:32:23
Readings Interval 1:52:00
Completed Yes
Mode Pressure 28d
Bottle Volume 510.0ml
Sample Volume 432.0ml
Dilution 1+0
Desired no. of Values 360
Actual no. of Values 361
Heads 2
The -r
option can be used to include the readings from selected
bottles. These are excluded by default as it’s probably more useful to use
oxitopdump for those purposes:
$ oxitoplist -p /dev/ttyUSB0 -r 110222-06
Serial 110222-06
ID 999
Started 2011-02-22 16:54:55
Finished 2011-02-22 16:54:55
Readings Interval 0:56:00
Completed Yes
Mode Pressure 14d
Bottle Volume 510.0ml
Sample Volume 432.0ml
Dilution 1+0
Desired no. of Values 360
Actual no. of Values 361
Heads 1
Head
Timestamp 60108
------------------- -----
2011-02-22 16:54:55 0.0
2011-02-22 17:50:55 -5.0
2011-02-22 18:46:55 -5.0
2011-02-22 19:42:55 -5.0
2011-02-22 20:38:55 -5.0
2011-02-22 21:34:55 -5.0
2011-02-22 22:30:55 -6.0
2011-02-22 23:26:55 -5.0
2011-02-23 00:22:55 -5.0
...
2011-03-08 11:18:55 -8.0
2011-03-08 12:14:55 -8.0
2011-03-08 13:10:55 -8.0
2011-03-08 14:06:55 -8.0
2011-03-08 15:02:55 -8.0
2011-03-08 15:58:55 -9.0
2011-03-08 16:54:55 -8.0
Readings are always given in chronological order and are delta readings by
default. If you want the absolute pressure readings, use the -a
option.
oxitopdump¶
This utility dumps the sample readings stored on a connected OxiTop Data Logger to files in CSV or Excel format. If bottle-serial values are specified, the details of those bottles and all heads attached to them will be exported, otherwise a list of all available bottles is exported. The bottle-serial values may include *, ?, and [] wildcards. The filename value may include references to bottle attributes like {bottle.serial} or {bottle.id}.
Synopsis¶
$ oxitopdump [options] [bottle-serial]... filename
Description¶
-
--version
¶
show program’s version number and exit
-
-h
,
--help
¶
show this help message and exit
-
-q
,
--quiet
¶
produce less console output
-
-v
,
--verbose
¶
produce more console output
-
-l
LOGFILE
,
--log-file
=LOGFILE
¶ log messages to the specified file
-
-P
,
--pdb
¶
enables debug mode (runs under PDB)
-
-p
PORT
,
--port
=PORT
¶ specify the port which the OxiTop Data Logger is connected to. This will be something like
/dev/ttyUSB0
on Linux or COM1 on Windows
-
-a
,
--absolute
¶
if specified, export absolute pressure values instead of deltas against the first value
-
-m
POINTS
,
--moving-average
=POINTS
¶ if specified, export a moving average over the specified number of points instead of actual readings
-
-H
,
--header
¶
if specified, a header row will be written in the output file
-
-R
,
--row-colors
¶
if specified, alternate row coloring will be used in the output file (.xls only)
-
-C
DELIMITER
,
--column-delimiter
=DELIMITER
¶ specifies the column delimiter in the output file. Defaults to
,
(.csv only)
-
-L
LINETERMINATOR
,
--line-terminator
=LINETERMINATOR
¶ specifies the line terminator in the output file. Defaults to
dos
(.csv only)
-
-Q
QUOTECHAR
,
--quote-char
=QUOTECHAR
¶ specifies the character used for quoting strings in the output file. Defaults to
"
(.csv only)
-
-U
QUOTING
,
--quoting
=QUOTING
¶ specifies the quoting behaviour used in the output file. Defaults to minimal (.csv only). Can be none, all, minimal, or nonnumeric
-
-T
TIMESTAMP_FORMAT
,
--timestamp-format
=TIMESTAMP_FORMAT
¶ specifies the formatting of timestamps in the output file. Defaults to
%Y-%m-%d %H:%M:%S
(.csv only)
Examples¶
When oxitopdump is invoked without specifying a bottle-serial the list of bottles will be exported to the specified filename. Typically you will want to use oxitoplist to discover the content of the connected device before exporting the readings for a specific bottle like so:
$ oxitoplist -p /dev/ttyUSB0
Serial ID Started Finished Complete Mode Heads
--------- --- ---------- ---------- -------- ------------ -----
110222-06 999 2011-02-22 2011-03-08 Yes Pressure 14d 1
121119-03 3 2012-11-19 2012-11-22 Yes Pressure 3d 1
120323-01 1 2012-03-23 2012-04-20 Yes Pressure 28d 2
3 results returned
$ oxitopdump -p /dev/ttyUSB0 120323-01 readings.csv
$ cat readings.csv
0,2012-03-23 17:32:23,0:00:00,0.0,0.0
1,2012-03-23 19:24:23,1:52:00,-12.0,-5.0
2,2012-03-23 21:16:23,3:44:00,-13.0,-5.0
3,2012-03-23 23:08:23,5:36:00,-13.0,-5.0
4,2012-03-24 01:00:23,7:28:00,-13.0,-5.0
...
357,2012-04-20 11:56:23,"27 days, 18:24:00",-16.0,-8.0
358,2012-04-20 13:48:23,"27 days, 20:16:00",-17.0,-8.0
359,2012-04-20 15:40:23,"27 days, 22:08:00",-17.0,-9.0
360,2012-04-20 17:32:23,"28 days, 0:00:00",-16.0,-8.0
If you specify multiple bottle-serials or if you specify a bottle-serial
with wildcards which matches multiple bottles, you will need to specify a
filename containing a substitution template like {bottle.serial}
so that
each bottle is output to a unique file. For example:
$ oxitopdump -p /dev/ttyUSB0 12* readings_{bottle.serial}.xls
$ ls *.xls
readings_120323-01.xls readings_121119-03.xls
Various options are provided for customizing the output of the formats available. For example, to include a header row and force space separation:
$ oxitopdump -p /dev/ttyUSB0 -H -D " " 11* test.csv
$ head test.csv
No. Timestamp Offset "Head 60108"
0 "2011-02-22 16:54:55" 0:00:00 0.0
1 "2011-02-22 17:50:55" 0:56:00 -5.0
2 "2011-02-22 18:46:55" 1:52:00 -5.0
3 "2011-02-22 19:42:55" 2:48:00 -5.0
4 "2011-02-22 20:38:55" 3:44:00 -5.0
5 "2011-02-22 21:34:55" 4:40:00 -5.0
6 "2011-02-22 22:30:55" 5:36:00 -6.0
7 "2011-02-22 23:26:55" 6:32:00 -5.0
8 "2011-02-23 00:22:55" 7:28:00 -5.0
oxitopview¶
The oxitopview utility encapsulates the functionality of the command line utilities (with the exception of oxitopemu) with a graphical interface. If matplotlib is installed it also provides a rudimentary graphing facility for data obtained from the unit.
Downloading Data¶
After starting the application from your platform’s launcher, click on the Connect icon (the first icon on the toolbar) or select . A dialog will appear prompting you for the serial port that the OxiTop OC110 unit is attached to.
If your platform supports enumeration of serial ports you may be able to use
the drop-down list to select from the available ports on your computer.
Otherwise, you will need to manually enter the name of the serial port to which
the device is attached. On Windows this will typically be something like
COM1
(or some higher number if you are using a USB-attached serial port),
while on UNIX-based platforms (like Mac OS X and Linux) it will typically be
something like /dev/ttyUSB0
(for a USB-attached serial port).
Ensure the unit is connected and turned on, then click OK. The application will attempt to open the serial port, and download the list of bottles stored on the device.
Note
Currently, it is frequently necessary to “nudge” the OC110 to get it to enter serial mode. Immediately after clicking OK, press the Up/Down buttons on the OC110 to wake it up and get it to enter serial mode; when in serial mode, the unit displays “V.24 Modus” on the screen. If you do not see this, then you will need to retry.
The status bar at the bottom of the application indicates serial port activity (currently rather crudely) with a Communicating label. Once the bottle list has downloaded it is displayed in a table:
Each line represents a single set of readings (which for BOD mode runs may incorporate multiple heads). Double-click on a row to query the device for the readings from that run. Once again, the unit will be queried over the serial port, and the Communicating label will flash on the status bar. Once the readings have been downloaded, the details will be shown in a new tab within the application:
Most of the controls in the tab are read-only, with the exception of two:
- The Moving average spinner allows you to calculate a center-based moving average over the specified number of readings.
- The Show absolute readings check-box allows you to toggle between delta pressure readings (the default) and absolute pressure readings.
Note that multiple sets of readings can be opened simultaneously. Each will appear in its own tab. For readings involving multiple heads (BOD mode only), the columns of the readings table will be colored the same as the corresponding line in the graph:
Note
Also be aware that, once a set of readings has been downloaded, the application caches it (for as long as it is running) so that re-opening the tab will be near instantaneous. To force the application to re-download the readings from the unit, use the
menu option.Exporting Data¶
You can export the available set of readings from the main window, or an individual set of readings from a readings tab. To do so, simply select the
menu entry. A file-save dialog will appear, prompting you to enter a filename in which to save the data. The name of the file will determine the format the data is exported in (or, on some platforms you can select the format from the file-type drop-down at the bottom of the dialog). Currently two export formats are supported:
- CSV - Comma Separated Values. A simple text-based format which doesn’t allow any formatting or multiple data-sets, but is easily accessible and universally supported. When data readings are exported in this format, only the readings will be included - not data about the bottle head (start and stop times, etc.)
- Excel - specifically the old-style .xls binary Excel format. This format permits multiple sheets and more formatting options, but is not open and not universally supported (although it is widely supported). When data readings are exported in this format, data about the bottle head (start and stop times, etc.) will be included in a separate sheet.
After selecting an export filename and clicking on OK, you will be presented with a format-dependent dialog to select additional export options. For CSV, this includes the record and field delimiters to use (it is generally best to stick to the defaults of DOS-style line breaks and comma field separators), and how to format timestamps (the default is the unambiguous ISO8601 YYYY-MM-DD style):
For Excel exports, the dialog simply determines whether to include a header row and whether to alternately color data-rows:
Note
When exporting bottle readings, the Show absolute readings and Moving average controls will affect the exported data. If you want to export the raw readings from the device, remember to reset these controls prior to exporting.
Exporting Graphs¶
Currently, there is no facility for exporting the graphs drawn by the application. If you wish to plot the data for publication, you are encouraged to use the CSV export facility described above and import the data into a good scientific graphing application like the excellent Veusz (Excel’s graphing facilities are rudimentary at best and, lacking decent vector facilities, are no good for publishing).
oxitopemu¶
The oxitopemu utility emulates an OxiTop OC110 device, or at least the serial port data retrieval portion anyway. This utility is of niche interest; it is intended for developers wishing to work on OxiTopped without having to have an actual OC110 to hand.
Synopsis¶
$ oxitopemu [options] bottles-xml
Description¶
-
--version
¶
show program’s version number and exit
-
-h
,
--help
¶
show this help message and exit
-
-q
,
--quiet
¶
produce less console output
-
-v
,
--verbose
¶
produce more console output
-
-l
LOGFILE
,
--log-file
=LOGFILE
¶ log messages to the specified file
-
-P
,
--pdb
¶
run under PDB (debug mode)
-
-p
PORT
,
--port
=PORT
¶ specify the port which the OxiTop Data Logger is connected to. This will be something like /dev/ttyUSB0 on Linux or COM1 on Windows. Default: /dev/ttyUSB0
-
-t
TIMEOUT
,
--timeout
=TIMEOUT
¶ specify the number of seconds to wait for data from the serial port. Default: 3
-
-d
,
--daemon
¶
if specified, start the emulator as a background daemon
Usage and Notes¶
Simply install the emulator on a small machine with a serial port (personally I
use a RaspberryPi with a USB to Serial adapter), then use a null-modem
between the machine running the client and the machine running the emulator. A
default set of bottle definitions in XML format is included in the package as
example.xml
under the main package’s installation directory.
If you have the python-daemon package installed (it’s included in the dependencies of the Linux packages, and is bundled with the Windows installer) you can run the emulator in daemon mode.
The main purpose of the emulator is to test the applications in a setting with
a “real” serial interface. For testing command compatibility, there is no need
to use oxitopemu directly; the emulation code is used internally by each of
the clients when the TEST
port is specified. In this case, an emulated
null-modem is used to connect the emulation code to the client.
License¶
This documentation is part of oxitopped.
oxitopped is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
oxitopped is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with oxitopped. If not, see <http://www.gnu.org/licenses/>.