Huygens Remote Manager documentation

This is the entry point for the Huygens Remote Manager documentation. Please use the links below to browse the Installation instructions and the User manual.

Contents

Installation instructions

These are the detailed installation and administration instructions for the Huygens Remote Manager.

Conventions

This page lists some conventions and notations used in this documentation.

Using sudo

Throughout this document, we’re assuming that commands which require root permissions can be issued by using sudo. If your system is not configured for this, you can either login directly as root (considered as “bad practice”) or use the su - command to switch to the root user.

We highly recommend the usage of sudo though!

Operating systems

The recommended linux distributions are:

Icon	Distribution name
	Debian
	Ubuntu
	CentOS
	Fedora (not tested for HRM ver. > 3.5)

We will use tabs to group the code and instructions for respective operating system families

Tab name	Corresponding operating system family
Debian	Debian, Ubuntu
RHEL	Fedora, CentOS

Warning

Please notice that with release 3.1, we dropped support for Mac OS X. HRM 3.0 is still known to work on Mac OS X from 10.5 (Leopard) onward, but no effort will be made to make future versions of the HRM compatible with Mac OS X. Also notice that the HRM was never tested on Mavericks.

Variables

Variable	Description	(Example) value
$WWW_ROOT	Web server document root	/var/www/html
$HRM_HOME	HRM root (home) folder	$WWW_ROOT/hrm
$HRM_CONFIG	HRM configuration folder	$HRM_HOME/config
$HRM_SAMPLES	HRM configuration samples folder	$HRM_HOME/config/samples
$HRM_RESRC	HRM resources folder	$HRM_HOME/resources
$HRM_SETUP	HRM setup folder	$HRM_HOME/setup
$HRM_BIN	HRM executables folder	$HRM_HOME/bin
$HRM_USER	HRM customization folder	$HRM_HOME/user
$HRM_LOG	HRM logging folder	/var/log/hrm
$HRM_DATA	HRM data folder	/data/hrm_data

If you want to use those variables in your interactive shell later on, just copy-paste the following lines into your session and adjust the values accordingly:

WWW_ROOT=/var/www/html                 # Web server document root
HRM_HOME=$WWW_ROOT/hrm                 # HRM root (home) folder
HRM_CONFIG=$HRM_HOME/config            # HRM configuration folder
HRM_SAMPLES=$HRM_HOME/config/samples   # HRM configuration samples folder
HRM_RESRC=$HRM_HOME/resources          # HRM resources folder
HRM_SETUP=$HRM_HOME/setup              # HRM setup folder
HRM_BIN=$HRM_HOME/bin                  # HRM executables folder
HRM_USER=$HRM_HOME/user                # HRM customization folder
HRM_LOG=/var/log/hrm                   # HRM logging folder
HRM_DATA=/data/hrm_data                # HRM data folder

Note

Please notice that $WWW_ROOT is /var/www/html in and in as of version 14.04 LTS. In earlier versions of , however, $WWW_ROOT used to be /var/www.

Get the HRM

Current version

Current version of the HRM is 3.6.x.

Download archive

Download the latest version of HRM, unpack the code and install it following the Manual installation instructions.

Download the archive.

Note

We are working on installation scripts to automate the process!

Clone git repository

To get the latest stable release directly from our git repository, run the following in a shell:

$ git clone -b master --single-branch https://github.com/aarpon/hrm
$ cd hrm
$ ./setup/setup_release.sh

If you want to follow the HRM development, you can checkout the read-only HRM repository and and configure your working copy for development:

$ git clone https://github.com/aarpon/hrm
$ cd hrm
$ git checkout devel
$ ./setup/setup_devel.sh

Find more detailed instructions in the installation pages.

Warning

Please keep in mind that the latest development snapshot is unstable software and should not be deployed on production machines!

Automated installation

We will release the installation scripts soon. The documentation will follow!

Manual installation

Install the prerequisites

The HRM requires a few prerequisites for its functions.

Operating system

If you have not yet read the conventions, please consult the Operating systems.

Note

For those who plan on using the (Optional) OMERO support it is strongly recommended to set up the HRM and OMERO using the same Linux distribution and version, simply to avoid Ice version incompatibilities, which will prevent the HRM-OMERO connector from working. Details can be found in the OMERO system administrator documentation.

PLEASE MIND: this does NOT mean you should install HRM and OMERO on the same machine! You should just run the same Linux installation on them.

Huygens Core

The HRM is an interface to Scientific Volume Imaging’s Huygens Core. Huygens Core is is a fully scriptable compute engine intended to run image processing and deconvolution jobs on large 64 bit multiprocessor servers in headless mode, i.e. without a specific graphical interface. The HRM provides such an interface for multi-user, batch access to Huygens Core.

Note

If the web and the processing server are not on the same machine, you will need an additional Huygens Core for the web server with a reader license (free of charge).

Apache2 web server

Debian

RHEL

sudo apt-get install apache2

sudo yum install httpd

Web pages can be installed globally or per-user.

Debian

RHEL

The Apache2 global document root is /var/www, or /var/www/html in more recent versions (14.04 LTS and newer).

The Apache2 global document root is /var/www/html.

If you plan to install the HRM in a specific user directory, use /home/<hrm_user>/public_html.

Apache2 access handling

HRM uses .htaccess files to prevent access to configuration files. Make sure to configure Apache2 to use them by setting the AllowOverride and Require directives in configuration files.

Debian

RHEL

/etc/apache2/sites-available/000-default.conf

If you are installing HRM in your user directory also put the directives in /etc/apache2/mods-available/userdir.conf Make sure to enable the userdir mod first by running the follwing line in the shell:

sudo a2enmod userdir

See also Enabling use of Apache htaccess files.

/etc/httpd/conf/httpd.conf

If you are installing HRM in your user directory also put the directives in: /etc/httpd/conf.d/userdir.conf

See also Apache Userdir with SELinux on Fedora 23/22, CentOS/RHEL 7.2/6.7/5.11.

Please notice that the Require directive is required from Apache version 2.4. Here is the code snippet for the config files

<Directory /var/www/html/hrm>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

PHP

Note

Minimum required PHP version is 5.5.

The HRM is made of two parts, a web interface and a queue manager, both written in PHP but with different requirements. The web interface requires the PHP 5 module for Apache2, the queue manager requires the PHP 5 command line interpreter.

Debian

RHEL

sudo apt-get install libapache2-mod-php5 php5 php5-cli php5-common php5-json php5-xml

Note

JSON support for PHP was moved into a separate package php5-json in Ubuntu 14.04LTS; in older versions, JSON support is part of the core php5 package.

sudo yum install php php-cli php-common php-process php-json php-xml

Production php.ini settings

Please configure the HRM machine for production. Edit the php.ini configuration file

Debian

RHEL

/etc/php5/apache2/php.ini

/etc/php.ini

In there set at least the values below (more information can be found in the php.ini file itself).

display_errors = Off
display_startup_errors = Off
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT

A relational database

The HRM officially supports two relational databases: MySQL and PostgreSQL.

MySQL

Debian

RHEL

sudo apt-get install php5-mysql mysql-server

sudo yum install php-mysql php-pdo mysql-server

Note

It is recommended to install a database management tool like phpmyadmin.

PostgreSQL

Debian

RHEL

sudo apt-get install php5-pgsql postgresql

sudo yum install php-pgsql postgresql-server postgresql-contrib

You will need to manually enable PostgreSQL:

sudo systemctl enable postgresql

Note

It is recommended to install a database manageent tool like phppgadmin.

Some additional information:

• This is a good tutorial from the Ubuntu Community on how to set up PostgreSQL to use with the HRM.

(Optional) LDAP support

If you plan to configure the HRM to use either Active Directory authentication or LDAP authentication, you will need to install the php-ldap package as well:

Debian

RHEL

sudo apt-get install php5-ldap

sudo yum install php-ldap

Sendmail (postfix)

HRM uses the PHP mail() function to notify the users:

“For the Mail functions to be available, PHP must have access to the sendmail binary on your system during compile time. If you use another mail program, such as qmail or postfix, be sure to use the appropriate sendmail wrappers that come with them.” More…

Debian

RHEL

sudo apt-get install postfix

sudo yum install postfix

Note

If your mail server is set up correctly and still PHP cannot send e-mails, SELinux might be blocking it. Query the status of httpd_can_sendmail as follows:

$ /usr/sbin/getsebool httpd_can_sendmail

If httpd_can_sendmail is off, you can enable it with:

$ /usr/sbin/setsebool -P httpd_can_sendmail on

PHP date() and default timezone

Please make sure to set the default timezone in php.ini as follows:

[Date]
; Defines the default timezone used by the date functions
; http://php.net/date.timezone
date.timezone = "Europe/Zurich"

Otherwise you will get the following warning every time the PHP function date() is called within the HRM:

PHP Warning: date(): It is not safe to rely on the system's timezone settings. You are
required to use the date.timezone setting or the date_default_timezone_set() function. (...)

Click here for the full list of supported time zones.

SSH

If the queue manager and the image processing server are not on the same machine (see installation instructions), HRM transfers files via ssh between the two using sudo. To allow HRM to login and run commands as sudo via remote, it is necessary to comment out the line 'Defaults requiretty' in the /etc/sudoers file.

Compressors

The HRM compresses files to be downloaded (such as deconvolution results). Several options are possible (and more can be added in the configuration files), but by default the HRM uses zip.

Debian

RHEL

sudo apt-get install zip

sudo yum install zip

(Optional) OMERO support

If you plan to use the OMERO connector, you will need to download the “server” package from the OMERO website that matches your existing OMERO installation and the Ice version installed on your HRM system. To make it work on your system, you also need Python 2.6 or 2.7 and (which is installed by default on Fedora and Ubuntu) and Java 7.

Warning

The connector requires at least the 5.0 series of OMERO, the latest version is only tested against OMERO 5.1 and newer!

Note

It is NOT required to do any installation or configuration of the downloaded OMERO package! HRM just needs this package to communicate with your existing OMERO server. Thus, this package allows HRM to communicate with any OMERO server connected to the network.

Required OMERO Connector Components

This section explains the steps required if you want to use the optional OMERO connector.

As an example on how to download these packages, the commands used to fetch OMERO 5.0.3 and Ice 3.4 are shown below. For other combinations please have a look at the OMERO download site. We recommend placing the downloaded OMERO “server” package into a subdirectory of /opt/OMERO, as follows:

Debian

RHEL

sudo apt-get install python-zeroc-ice libicessl34 openjdk-7-jre
wget http://downloads.openmicroscopy.org/omero/5.0.3/artifacts/OMERO.server-5.0.3-ice34-b41.zip -O /tmp/OMERO.server.zip
sudo mkdir -pv /opt/OMERO
cd /opt/OMERO
sudo unzip /tmp/OMERO.server.zip
rm /tmp/OMERO.server.zip

sudo yum install ice-python java-1.7.0-openjdk
wget http://downloads.openmicroscopy.org/omero/5.0.3/artifacts/OMERO.server-5.0.3-ice34-b41.zip -O /tmp/OMERO.server.zip
sudo mkdir -pv /opt/OMERO
cd /opt/OMERO
sudo unzip /tmp/OMERO.server.zip
rm /tmp/OMERO.server.zip

Note

From HRM 3.3.0 on, the OMERO connector requires the Python package argparse, which is included by default from Python 2.7 on. In case you are running an older Python version, you have to install the package yourself. Please refer to the installation instructions of your distribution on how to set up Python argparse.

Please follow the instructions in configuring the OMERO connector about the optional configuration variable PYTHON_EXTLIB in case you have to place the package in a non-default location.

This configuration variable also applies to the following section!

Additional Python Packages (Optional)

This section is about packages that are not mandatory for using the connector, but add more functionality to it.

If the Python Imaging Library (PIL) or Pillow is installed, the connector will download thumbnails as well to use them as previews in the HRM side bar. Those “simple” previews can of course be replaced with the HRM ones by clicking on the “Re-generate Preview” link.

In case the Beautiful Soup library is available on the system, the connector will generate parameter summaries and attach them as a human-readable comment to any image uploaded to OMERO.

Debian

RHEL

sudo apt-get install python-imaging python-bs4

sudo yum install python-imaging python-beautifulsoup4

Install the HRM

… From an Archive

Download or checkout the HRM as explained here.

Unpack the downloaded archive to web server document root directory. This is the directory where Apache2 finds the html and php files to serve.

up to 13.10

sudo unzip hrm_x.y.z.zip -d /var/www

and 14.04 LTS and later

sudo unzip hrm_x.y.z.zip -d /var/www/html

where x.y.z is a placeholder for the HRM version.

Note

You can of course extract or clone the HRM somewhere else: just add the location to the Apache2 configuration (httpd.conf).

… Using Git

The advantage of using git is that later upgrades are easy to perform, and all the modifications are documented this way. You need to install git on the machine.

sudo apt-get install git

Then you can change directory to the hrm document root and check out the git repository.

cd $WWW_ROOT
git clone https://github.com/aarpon/hrm.git
git tag -l

In this project tags are used to mark the different version in the master branch. the last command (see above) gives you a list of all the available versions of the HRM. Usually you can checkout the highest version numbers, except if there are certain compatibility issues.

git checkout <tag>

Once the latest release is checked out, you might want to create a local branch before starting to configure the HRM. Per default the $HRM_HOME/config is in the .gitignore file, so if you want to put your configurations under version control you have to modify the .gitignore first.

git checkout -b deployed (optional)

Configure the HRM

Mandatory configuration steps

Set up HRM user and group and set permissions

In the following instructions, HRM_DATA points to the data folder that contains all user subfolders (for example: /data/hrm_data) and HRM_LOG is the log folder for the HRM (for example: /var/log/hrm). Please see also the conventions. It is important that those folder have the correct ownership and permissions set for HRM to work properly. Please follow these steps.

Create a Unix group hrm and user hrm on the web server machine.

$ sudo groupadd --system hrm
$ sudo useradd hrm --system --gid hrm

Create the log directory:

sudo mkdir ${HRM_LOG}

Make sure hrm owns (and has full read-write access) to HRM_DATA and HRM_LOG. This is done by setting the group ownership of HRM_DATA and HRM_LOG to hrm:

sudo chown -R hrm:hrm ${HRM_DATA}
sudo chmod -R u+s,g+ws ${HRM_DATA}
sudo chown -R hrm:hrm ${HRM_LOG}
sudo chmod -R u+s,g+ws ${HRM_LOG}

Add the Apache user ( www-data, apache) to the hrm group:

# www-data in Ubuntu, apache in Fedora
sudo usermod www-data --append --groups hrm

Note

You might have to restart your server for the group changes to be activated.

Edit hrm.conf

The hrm.conf file is used by both the Web server and the Queue Manager.

Copy the sample file

The sample configuration file $HRM_SAMPLES/hrm.conf.sample must be copied to /etc/hrm.conf and then edited as explained in the following sections.

sudo cp $HRM_SAMPLES/hrm.conf.sample /etc/hrm.conf

Edit the configuration file

This is the content of the sample configuration file:

#!/bin/bash
#
# HRM configuration file
#
# This file is part of the Huygens Remote Manager
# Copyright and license notice: see license.txt

# HRM install directory
HRM_HOME="/path/to/hrm/home"

# HRM image share
HRM_DATA="/path/to/hrm/data"

# Source and destination folder names
HRM_SOURCE="src"
HRM_DEST="dst"

# User to run the HRM daemon with rights to
# execute /bin/mkdir, /bin/chown, /bin/chmod and
# /bin/rm on HRM_DATA (see above) for user management.
SUSER="hrm"

# HRM log directory
HRM_LOG="/var/log/hrm"

# Interaction with OMERO (if switched on in hrm/config).
OMERO_PKG="/opt/OMERO/OMERO.server"
OMERO_HOSTNAME="full.qualified.name.of.omero.server"
OMERO_PORT="4064"

# Set a custom PHP CLI binary if necessary:
# PHP_CLI="/usr/local/php/bin/php"

Explanation

• HRM_HOME points to $WWW_ROOT/hrm (for example: /var/www/html/hrm).
• HRM_DATA points to the data folder that contains all user subfolders (for example: /data/hrm_data).
• HRM_SOURCE and HRM_DEST are the source and destination subfolders inside the user directory (for example: src and dst). The source folder for an hypothetical user ‘john’ would then be /data/hrm_data/john/src.
• SUSER is the Unix user that runs the Queue Manager (for example: hrm).
• HRM_LOG is the log folder for the HRM (for example: /var/log/hrm)
• (optional) OMERO_PKG, OMERO_HOSTNAME and OMERO_PORT: see OMERO connector for details.
• (optional) PHP_CLI is the path tho the php CLI executable if it is not in the path or another one should be used (for example: /usr/local/php/bin/php).

Edit hrm_{server|client}_config.inc

Copy the sample files

Copy $HRM_SAMPLES/hrm_server_config.inc.sample to $HRM_CONFIG/hrm_server_config.inc, this file is used by the queue manager:

sudo cp -v $HRM_SAMPLES/hrm_server_config.inc.sample $HRM_CONFIG/hrm_server_config.inc

In a single-machine HRM installation, the server and client configuration files are identical. It is therefore recommended to create a symlink for the client configuration file that points to the server config file. This prevents out-of-sync configurations:

sudo ln -s hrm_server_config.inc $HRM_CONFIG/hrm_client_config.inc

Edit the configuration files

Here we will assume a one-machine installation, and we will therefore show just one file ($HRM_CONFIG/hrm_server_config.inc).

<?php
// This file is part of the Huygens Remote Manager
// Copyright and license notice: see license.txt

// This configuration file is used by the QUEUE MANAGER.

//==============================================================================
// Database settings
//==============================================================================

// The database type ('mysqli' or 'postgres'; mysql is deprecated).
// Please notice that it is highly recommended to use the mysqli driver for
// PHP 5.5 and above. In PHP >= 5.6, the mysql driver is deprecated and will
// overflow your log files when used!
$db_type = "mysqli";

?>

Note

Although the database abstraction library (ADOdb) we use in the HRM can interface with a large number of databases, we currently only support MySQL and PostgreSQL. Other DBs may work, but we haven’t tested them.

Note

Also, please use the ‘mysqli’ driver instead of ‘mysql’ as in previous versions of the HRM. From PHP 5.6, the ‘mysql’ driver is deprecated!

<?php
...
// The name of the database host machine (this may be localhost if it is the
// machine on which the web server runs)
$db_host = "localhost";

// The name of the database used by HRM
$db_name = "hrm";

// The name of the user with which the database is accessed
$db_user = "hrm";

// The password of the account with which the database is accessed
$db_password = "dbpasswd";

//==============================================================================
// Huygens settings
//==============================================================================

// Huygens server default user
$huygens_user = "hrm";
$huygens_group = "hrm";
?>

The $huygens_user is used in case the processing machine (the one where Huygens Core is installed) does not have direct access to the file server (i.e. the files to be deconvolved must be copied from the file server to the processing machine via ssh). For this to work, you will need to set up password-less ssh connection between file and processing server for the $huygens_user (see for example here) and also set the variable $copy_images_to_huygens_server to true (see below).

<?php
...
// Path to a local Huygens Core executable, to handle image information and
// thumbnails. This installatation of hucore doesn't require a full license
// unless this is also the computation server. For image information and
// thumbnails mostly freeware functions will be used, but support for certain
// propietary file formats requires a B flag in the license string. (See
// http://support.svi.nl/wiki/FileFormats).
//
// In combination with Huygens 3.5.1 or higher, HRM can also provide an
// estimate of the Signal-To-Noise ratio (SNR) of an image, interactively. At
// least that same simple license is required for this to work.
$local_huygens_core = "/usr/local/bin/hucore";
?>

The web interface uses hucore for some operations on the input and result files (for example to generate image previews or estimate the SNR interactively). An additional copy of hucore needs therefore to be installed on the web server. However, since all these functions used are accessible in reader mode, only a reader license is needed (free of charge).

<?php
//==============================================================================
// File server settings
//==============================================================================

// File server host name
$image_host = "localhost";

// File server default user
$image_user = "hrm";
$image_group = "hrm";

// File server base folder
$image_folder = "/data/hrm_data";

// File server image source folder
$image_source = "src";

// File server image destination folder
$image_destination = "dst";

// File server base folder as seen from the Huygens server machines
$huygens_server_image_folder = "/data/hrm_data";
?>

This is information needed for the web interface and the queue manager to login to the file server. If the file server is not on the same machine, its host name must be given.

<?php
...
// Allow HTTP transfer of the restored (download) images
$allowHttpTransfer = true;

// Allow HTTP transfer of original (upload) images.
// Limitations in upload size must be configured in the below parameters
// $max_upload_limit AND $max_post_limit. If these are set to 0, they are
// instead read from the PHP settings. If set to higher values, make sure
// that the PHP settings allow for such transfer limits. These PHP settings
// are normally located in php.ini, with the options upload_max filesize AND
// post_max size.
// Unzipping large archives may also require high max_execution_time and
// memory_limit variables.
$allowHttpUpload = true;
$max_upload_limit = 0; // Maximum file size for uploads in Mb
$max_post_limit = 0;   // Maximum post size for uploads in Mb

// Archiver for downloading the results via the web browser:
// $compressBin is the archiver, including options.  %DEST% will be replaced
// with the user's destination path; can be used as an option or part of the
// command to change directories with 'cd'. Separate commands with "\n".
// $packExcludePath controls whether the archiving command excludes the whole
// file path per file when this is taken into account in the $compressBin
// command itself by using %DEST% properly.

$compressExt = ".zip";

switch ($compressExt) {
    case ".tgz":
    case ".tar.gz":
        $compressBin = "cd %DEST% \n /bin/tar chzf ";
        $packExcludePath = true;
        $dlMimeType = "application/x-force-download";
        break;
    case ".tar":
        $compressBin = "cd %DEST% \n /bin/tar cfh ";
        $packExcludePath = true;
        $dlMimeType = "application/x-force-download";
        break;
    case ".zip":
        $compressBin = "cd %DEST% \n /usr/bin/zip ";
        $packExcludePath =true;
        $dlMimeType = "application/x-force-download";
        break;
    default:
        $compressExt = ".zip";
        $compressBin = "cd %DEST% \n/usr/bin/zip -D";
        $packExcludePath = true;
        $dlMimeType = "application/x-force-download";
}

// Tools to decompress uploaded archives. This is an array, each key linked to
// an archive extension.  %DEST% will be replaced with the user's destination
// path; can be used as an option or part of the command to change directories
// with 'cd'.
// More formats are possible, just add more commands to the array.

$decompressBin['zip'] = "cd %DEST% \n /usr/bin/unzip -o ";
$decompressBin['tgz'] = "cd %DEST% \n /usr/bin/tar xzf ";
$decompressBin['tar'] = "cd %DEST% \n /usr//bin/tar xf ";
$decompressBin['tar.gz'] = "cd %DEST% \n /usr/bin/tar xzf ";
?>

If $allowHttpTransfer is true, the results of deconvolution can be downloaded through the web interface. If it is false, then the results can only be accessed by other means (e.g. via network shares). Downloaded files are compressed to reduce bandwidth load: $compressExt defines the type of compression (default is zip).

Note

For the compressed file to be served by the web server, it has to be re-read in. The HRM uses chunked reading, that should make sure that even large files can be read without problems. Should the users still encounter problems downloading files from the HRM, then try increasing the value of memory_limit in php.ini.

Example:

memory_limit = 128M

If $allowHttpUpload is true, an HTTP uploader will be in place to allow uploading of the files to be deconvolved through the web interface. Multi-file upload (with directory structures) is indirectly possible by first compressing the files into an archive that the HRM will automatically extract at the end upload process. Default supported formats are zip, tgz, tar and tar.gz (as long as the corresponding executables are installed on the system) but more can be added by extending the $decompressBin array.

Warning

Make sure to change the values of upload_max_filesize AND post_max_size in php.ini: with the default values, only extremely small files can be uploaded!

Example:

post_max_size = 1024M
upload_max_filesize = 1024M

Note

Notice that despite some script executions are relatively fast (like file extraction from zip archives, image previews generation, or SNR estimation), they may take a long time if the images are very large or if deconvolution jobs are already running in the web server. You may need to increase also PHP’s max_execution_time directive in php.ini.

Example:

max_execution_time = 120

<?php
...
//==============================================================================
// HRM configuration parameters
//==============================================================================

// URL
$hrm_url = "http://localhost/hrm";

// Install dir
$hrm_path = "/var/www/html/hrm";

// Logging
// Please create a directory for the logging (default and recommended is /var/log/hrm)
// and make sure to grant read/write access to the web server user.
$log_verbosity = 1;         // 0=quiet, 1=normal, 2=debug
$logdir  = "/var/log/hrm";
$logfile = "hrm.log";
$logfile_max_size = 100;    // maximum size of the logfile in MB
?>

The $logdir variable points to the directory where the log files created by the web server user and the queue manager should reside. Default (and recommended) directory for the logs is /var/log/hrm. Please create this directory and make sure to grant the relevant users read/write access to it!

<?php
...
// Email
$send_mail = true;
$email_sender = "hrm@localhost";
$email_admin = "hrm@localhost";
// Comma (',') is the standard separator for lists of email addresses.
// Please see: http://tools.ietf.org/html/rfc2822#section-3.6.3
// Unfortunately, Microsoft Outlook uses semicolon (';') by default and
// ',' only optionally. Please see: http://support.microsoft.com/kb/820868
// on how to configure Outlook to support comma-separated lists.
// If your users are mostly using Outlook, you can tell the HRM to use ';'
// instead of the standard ',' for distribution lists.
$email_list_separator = ",";    // Valid options are ',' and ';'.

Enable and configure the e-mail service.

<?php
...
// Supported authentication types. Possible values are 'integrated', 'active_dir', 'ldap'.
// These values replace the previous ones (but the old ones are still recognized):
// 'integrated' replaces 'MYSQL';
// 'active_dir' replaces 'ACTIVE_DIR';
// 'ldap' replaces 'LDAP'.
// You can use more than one authentication mechanism by listing them all in the
// $authenticateAgainst array below.
//
// e.g. $authenticateAgainst = array("active_dir", integrated");
//
// The first entry in the array is the default authentication mode that will be used
// for new users. Administrators can then change the authentication mode for individual
// users in the user management pages.
//
// Make sure to properly configure all authentication modes
// (see sample files in config/samples).
$authenticateAgainst = array("integrated");
?>

Setting the authentication mode to integrated enables the embedded user management system (independent of the actual database you are using, e.g. postgresql). Use this if you don’t have any other user management system in place. Alternatively, simple authentication against Microsoft’s Active Directory and OpenLDAP is possible.

In case one or more of the alternative authentication types are selected (i.e. active_dir or ldap), some additional configuration will be required (see later).

Note

The super administrator ‘admin’ always has authentication mode ‘integrated’!

<?php
...
// If true, use DES password encryption; if false, use MD5.
// This variable is DEPRECATED and will be removed in the next HRM version!
$useDESEncryption = false;

// If true, the queue manager and the image processing server run on the same
// machine
$imageProcessingIsOnQueueManager = true;

// If true the images are copied to the huygens server machine
$copy_images_to_huygens_server = false;
?>

Set $imageProcessingIsOnQueueManager to false if the queue manager is not on the same machine as the Huygens Core that runs deconvolution.

Set $copy_images_to_huygens_server to true if the files have to be copied to the machine where the Huygens Core is installed via ssh (using password-less ssh connection!). If Huygens Core has direct access to the file area (e.g. through NFS mount), set this to false.

Note

The use of the variable $useDESEncryption is deprecated and will be removed in the next HRM version!

<?php
...
// Thumbnails require Huygens Core 3.3.2 or higher.

// Notice that despite some script executions are relatively fast (like
// thumbnail generation and SNR estimation), they still may take a long time if
// the images are very large, or if deconvolution jobs are already running in
// the local server. You may need to increase PHP's max_execution_time
// directive in php.ini, and restart the web server.

// If true, allows generates thumbnails and previews for the originals and the
// results during the deconvolution (i.e. in the computation servers). These
// are maximum intensity projections (MIP) of the stacks along the main axes,
// so they provide xy, xz and zy views.
$useThumbnails = true;

// If true, allows on-demand generation of a preview of the original images in
// this web server prior to the deconvolution. For this to work the variable
// $local_huygens_core must be set correctly above. Notice that not all file
// formats are readable in a freeware Huygens Core and for full functionality
// you may need a license string. Otherwise just let the original thumbnails
// be generated during the deconvolution itself with the previous option.
// If this is true and the correct version of Huygens is installed, the SNR
// estimator tools is also enabled (see $enable_code_for_huygens above).
// This requires $useThumbnails = true;
$genThumbnails = true;

// Moreover, if $genThumbnails is true and Huygens 3.5.1 or higher is
// installed (see $enable_code_for_huygens above), the Signal To Noise ratio
// can be estimated visually from raw images, in the restoration parameters
// editor.

// If larger than zero, generates an AVI movie for 3D stacks that allows
// browsing along the XY slices, and for time series to browse along MIPs in
// time. The movie will have this maximum number of pixels in the largest
// dimension.
// This requires $useThumbnails = true;
$movieMaxSize = 300;

// If true, (and $useThumbnails is also true), it generates not only the MIP
// previews but also Simulated Fluorescence Process top-view renderings of the
// original and the restored images (http://support.svi.nl/wiki/SFP).
$saveSfpPreviews = true;

// If non-zero, save stack and time series previews of the original and restored
// data side by side, to allow comparisons. (Requires $useThumbnails and
// Huygens Core 3.5.2). This is the max size in pixels for each of the images's
// slicer.
$maxComparisonSize = 300;

These were several options for preview generations.

<?php
...
// A shell command that executes ping four times and stops afterwards
$ping_command = 'ping -w 4';    // use this on linux systems
// $ping_command = 'ping -t 4'; // use this on macosx systems
// $ping_command = 'ping';  // use this on cygwin

// The parameter for the ping command after the hostname
$ping_parameter = '';           // use this on linux systems
//$ping_parameter = '56 4'; // use this on cygwin
?>

This is used by the queue manager to test whether the processing machine is reachable.

Note

In HRM it is possible to use a machine for each of the components, i.e. web server, database server, one or more processing machines, file server and queue manager. Configuration files are needed only on the web server and the machine running the queue manager. Make sure to fill the various path and user fields from the perspective of the machine that hosts each of the configuration files!

Configure authentication

The HRM curently supports three user authentication mechanisms; if your institution relies on Microsoft’s Active Directory or a generic LDAP server, the HRM can be configured to use them to test user credentials and obtain limited user information (e-mail address angd group). Otherwise, the HRM implements a simple, internal user management module.

You can enable any combination of authentication mechanisms. The first one is the default for new users. Valid examples are:

<?php
...
// Only integrated authentication
$authenticateAgainst = array("integrated");

// Authenticate against Active Directory by default;
// some users may be checked against integrated authentication
$authenticateAgainst = array("active_dir", "integrated");

// Integrated authentication by default;
// some users may be authenticated against ldap
$authenticateAgainst = array("integrated", "ldap");

// This is also possible, however unlikely in reality:
$authenticateAgainst = array("active_dir", "ldap", "integrated");
...

The following pages explain the required configuration steps for the support authentication mechanisms.

Internal authentication

To enable internal authentication, please add “integrated” to the $authenticateAgainst array in $HRM_CONFIG/hrm_client_config.inc and in $HRM_CONFIG/hrm_server_config.inc

<?php
...
$authenticateAgainst = array("integrated");
...

This is the default value, so nothing needs to be changed in a fresh installation to use internal authentication.

Active Directory authentication

To enable Active Directory authentication, please add “active_dir” to the $authenticateAgainst array in $HRM_CONFIG/hrm_client_config.inc and in $HRM_CONFIG/hrm_server_config.inc

<?php
...
$authenticateAgainst = array("active_dir");
...

Then, copy $HRM_SAMPLES/active_directory.config.inc.sample to $HRM_CONFIG/active_directory.config.inc and edit it.

<?php
// This file is part of the Huygens Remote Manager
// Copyright and license notice: see license.txt

// See http://adldap.sourceforge.net/wiki/doku.php?id=api_configuration for
// detailed help on configuring adLdap.

// The account suffix for your domain
$ACCOUNT_SUFFIX = "@mydomain.local";

// The base dn for your domain
$BASE_DN = "DC=mydomain,DC=local";

// Array of domain controllers. Specifiy multiple controllers if you would
// like the class to balance the LDAP queries amongst multiple servers
$DOMAIN_CONTROLLERS = array ("dc01.mydomain.local");

// Domain controller port
$AD_PORT = 389;

// User name suffix
$AD_USERNAME_SUFFIX = "";

// User name suffix matching string
$AD_USERNAME_SUFFIX_REPLACE_MATCH = "";

// User name suffix replacing string
$AD_USERNAME_SUFFIX_REPLACE_STRING = "";

// Optional account with higher privileges for searching (otherwise
// leave it to NULL). This should be set to a domain admin account (only
// read operations are performed!)
$AD_USERNAME = NULL;
$AD_PASSWORD = NULL;

// Tweak to get the real primary group from Active Directory. It works if
// the user's primary group is domain users.
// http://support.microsoft.com/?kbid=321360
$REAL_PRIMARY_GROUP = true;

// Use SSL (LDAPS)
$USE_SSL = false;

// Use TLS: if you wish to use TLS you should ensure that $USE_SSL is
// set to false and vice-versa
$USE_TLS = false;

// When querying group memberships, do it recursively
$RECURSIVE_GROUPS = true;

// Group filtering and authorization.
//
// Users in Active Directory usually belong to several groups. Many of those groups
// will not be relevant for the administrator of the HRM (groups like 'Domain Users',
// 'Building 12', 'Staff') and the important ones, like the research group, are hidden
// somewhere in this list.
//
// The $VALID_GROUPS array can be used to specify only those groups that are interesting
// for the HRM and that will be used as the user group in the HRM and will show up in the
// usage statistics. For example, only the research groups at the institution might be
// included in this list.
//
// Additionally, not all research groups might be allowed to use the HRM. The
// $AUTHORIZED_GROUPS array can be used to specify those groups that are allowed to use
// the HRM.
//
// Please notice that $AUTHORIZED_GROUPS must not strictly be a subset of $VALID_GROUPS.
//
// Example 1
// =========
//
// $AUTHORIZED_GROUPS = array('group_einstein', 'group_bohr');
// $VALID_GROUPS = array('group_einstein', 'group_bohr');
//
// User 'john' has groups {'Domain Users', 'Building 12', 'Staff', 'group_einstein'}
//
// 'john' is allowed to log in and the group used in HRM is 'group_einstein'.
//
// User 'stephanie' has groups {'Domain Users', 'Building 12', 'Staff', 'group_bohr'}
//
// 'stephanie' is allowed to log in and the group used in HRM is 'group_bohr'.
//
// User 'jeff' has groups {'Domain Users', 'Building 1', 'Admin Staff', 'group_admin'}
//
// 'jeff' is not allowed to log in ('group_admin' is not in $AUTHORIZED_GROUPS)'.
//
// Example 2
// =========
//
// $AUTHORIZED_GROUPS = array('hrm');
// $VALID_GROUPS = array('group_einstein', 'group_bohr');
//
// User 'john' has groups {'Domain Users', 'Building 12', 'Staff', 'group_einstein', 'hrm'}
//
// 'john' is allowed to log in and the group used in HRM is 'group_einstein'.
//
// User 'stephanie' has groups {'Domain Users', 'Building 12', 'Staff', 'group_bohr'}
//
// 'stephanie' is not allowed to log in (she does not belong to group 'hrm').
//
// User 'jeff' has groups {'Domain Users', 'Building 1', ''Admin Staff', 'group_admin'}
//
// 'jeff' is not allowed to log in (she does not belong to group 'hrm').
//
// Example 3
// =========
//
// $AUTHORIZED_GROUPS = array();
// $VALID_GROUPS = array('group_einstein', 'group_bohr');
//
// No restriction on which groups can log in to HRM. Group filtering works as in Examples 1
// and 2.
//
// Example 4
// =========
//
// $AUTHORIZED_GROUPS = array();
// $VALID_GROUPS = array();
//
// If neither valid nor authorized groups are defined, all groups are allowed to log in and
// the first returned group will be used in HRM.
//
// User 'john' has groups {'Domain Users', 'Building 12', 'Staff', 'group_einstein', 'hrm'}
//
// john's group in HRM will be 'Domain Users'.
//
// Example 5
// =========
//
// $AUTHORIZED_GROUPS = array(... any number of entries ...);
// $VALID_GROUPS = array();
//
// If $VALID_GROUPS is empty and $AUTHORIZED_GROUPS contains one or more entries,
// $VALID_GROUPS will be reset to be the same as $AUTHORIZED_GROUPS (the end behavior
// will be as in Example 1).

// Groups to use in HRM (filter)
$VALID_GROUPS = array();

// Groups authorized to log in to HRM.
$AUTHORIZED_GROUPS = array();

Special case: Active Directory forests

Warning

For most installations, you can skip the entire rest of this section and proceed with the next one by simply leaving the variables $AD_USERNAME_SUFFIX, $AD_USERNAME_SUFFIX_REPLACE_MATCH and $AD_USERNAME_SUFFIX_REPLACE_STRING with their default values (empty, i.e. "").

This part of the config is meant for the special case of an Active Directory with multiple domains (commonly referred to as forest) which usually consist of a top-level domain (the forest root domain) and several sub-domains. If the HRM users belong to domains in all levels of the forest it would be required to enter the full domain name for logging in, which can become quite tedious (and therefore error-prone).

Consider the following domain setup, a root domain called SATURN with two sub-domains RHEA and TETHYS. Those “plain” domain names are usually referred to as NetBIOS Domain Names. A hierarchical representation could look like this:

SATURN
  |
  +-- RHEA
  |
  +-- TETHYS

Now, assuming a top-level suffix of .private, in Microsoft’s Active Directory naming scheme this is translated into fully qualified domain names (FQDN’s) as follows. The root domain (SATURN) gets prefixed by ads. and followed by the top-level .private suffix, whereas the two subdomains just get postfixed by the FQDN of the root domain. The graphical representation from above would therefore translate into this:

ads.saturn.private
  |
  +-- rhea.ads.saturn.private
  |
  +-- tethys.ads.saturn.private

To make authentication work in such a scenario, it is required to authenticate against the GLOBAL CATALOG of the Active Directory. This is an LDAP service provided only on the domain controllers of the root domain on the special port 3268. The usernames used to authenticate have to be of the form username@fully.qualified.domain.name to provide the global catalog with the information to which sub-domain a certain account belongs as it is possible to have the same username for different accounts in different domains of a single domain forest.

Now given the HRM configuration option $ACCOUNT_SUFFIX would be set to .saturn.private, a user foo from the domain SATURN could log in using something like foo@ads whereas a user bar from RHEA would have to type bar@rhea.ads. This is obviously screaming for problems as normal users don’t understand where and why to put those weird suffixes, especially as they’ve never heard of something like ads before.

Using the sample config from below, users can simply log on by using their regular account name followed by an @ sign and the (short) NetBIOS domain name, e.g.

foo@saturn

or

bar@rhea

which is way easier to explain and remember for them, especially as the short domain names are usually visible for them when logging into their workstation.

<?php
...
$ACCOUNT_SUFFIX = "";
$BASE_DN = "DC=ads, DC=saturn, DC=private";

// set this to a domain controller of the *ROOT* domain:
$DOMAIN_CONTROLLERS = array("dc01.ads.saturn.private");

// the port is required to connect to the global catalog:
$AD_PORT = 3268;

// give the FQDN of the root domain with a leading dot here:
$AD_USERNAME_SUFFIX = ".ads.saturn.private";

// account names from the root domain will end up wrong, looking like this
// (note the leading "@" and the double occurence of the root domain):
$AD_USERNAME_SUFFIX_REPLACE_MATCH = "@saturn.ads.saturn.private";

// they have to be rewritten using the correct suffix instead (removing the
// first occurence of the root domain name):
$AD_USERNAME_SUFFIX_REPLACE_STRING = "@ads.saturn.private";
?>

Please feel free to contact the HRM developers in case you need help setting up authentication in a multi-domain forest scenario!

LDAP authentication

To enable LDAP authentication, please add “ldap” to the $authenticateAgainst array in $HRM_CONFIG/hrm_client_config.inc and in $HRM_CONFIG/hrm_server_config.inc

<?php
...
$authenticateAgainst = array("ldap");
...

Then, copy $HRM_SAMPLES/ldap.config.inc.sample to $HRM_CONFIG/ldap.config.inc and edit it.

<?php
// This file is part of the Huygens Remote Manager
// Copyright and license notice: see license.txt

// The machine on which the ldap server is running
$ldap_host = "localhost";

// The port for the ldap connection
$ldap_port = "389";

// Use SSL (LDAPS)
$ldap_use_ssl = false;

// Use TLS: if you wish to use TLS you should ensure that $ldap_use_ssl is
// set to false and vice-versa
$ldap_use_tls = false;

// The ldap root
$ldap_root = "dc=root,dc=country";

// The base for the manager DN (either cn or uid)
$ldap_manager_base_DN = "cn";

// The ldap manager (username only!)
$ldap_manager = "manager";

// The ldap manager password
$ldap_password = "secret";

// The ldap manager OU: use this in case the ldap_manager is in some special OU
// that distinguishes it from the other users. $ldap_manager_ou will be
// prepended to $ldap_user_search_DN.
// Set it to "" if $ldap_user_search_DN can be used for binding
$ldap_manager_ou = "ou=special_user";

// The user search DN (without ldap root)
$ldap_user_search_DN = "cn=users";

// Group filtering and authorization.
//
// Users in LDAP usually belong to several groups. Many of those groups
// will not be relevant for the administrator of the HRM (groups like 'Domain Users',
// 'Building 12', 'Staff') and the important ones, like the research group, are hidden
// somewhere in this list.
//
// The $ldap_valid_groups array can be used to specify only those groups that are
// interesting for the HRM and that will be used as the user group in the HRM and will
// show up in the usage statistics. For example, only the research groups at the institution
// might be included in this list.
//
// Additionally, not all research groups might be allowed to use the HRM. The
// $ldap_authorized_groups array can be used to specify those groups that are allowed
// to use the HRM.
//
// Please notice that $ldap_authorized_groups must not strictly be a subset of
// $ldap_valid_groups.
//
// Example 1
// =========
//
// $ldap_authorized_groups = array('group_einstein', 'group_bohr');
// $ldap_valid_groups = array('group_einstein', 'group_bohr');
//
// User 'john' has groups {'Domain Users', 'Building 12', 'Staff', 'group_einstein'}
//
// 'john' is allowed to log in and the group used in HRM is 'group_einstein'.
//
// User 'stephanie' has groups {'Domain Users', 'Building 12', 'Staff', 'group_bohr'}
//
// 'stephanie' is allowed to log in and the group used in HRM is 'group_bohr'.
//
// User 'jeff' has groups {'Domain Users', 'Building 1', 'Admin Staff', 'group_admin'}
//
// 'jeff' is not allowed to log in ('group_admin' is not in $ldap_authorized_groups)'.
//
// Example 2
// =========
//
// $ldap_authorized_groups = array('hrm');
// $ldap_valid_groups = array('group_einstein', 'group_bohr');
//
// User 'john' has groups {'Domain Users', 'Building 12', 'Staff', 'group_einstein', 'hrm'}
//
// 'john' is allowed to log in and the group used in HRM is 'group_einstein'.
//
// User 'stephanie' has groups {'Domain Users', 'Building 12', 'Staff', 'group_bohr'}
//
// 'stephanie' is not allowed to log in (she does not belong to group 'hrm').
//
// User 'jeff' has groups {'Domain Users', 'Building 1', ''Admin Staff', 'group_admin'}
//
// 'jeff' is not allowed to log in (she does not belong to group 'hrm').
//
// Example 3
// =========
//
// $ldap_authorized_groups = array();
// $ldap_valid_groups = array('group_einstein', 'group_bohr');
//
// No restriction on which groups can log in to HRM. Group filtering works as in Examples 1
// and 2.
//
// Example 4
// =========
//
// $ldap_authorized_groups = array();
// $ldap_valid_groups = array();
//
// If neither valid nor authorized groups are defined, all groups are allowed to log in and
// the first returned group will be used in HRM.
//
// User 'john' has groups {'Domain Users', 'Building 12', 'Staff', 'group_einstein', 'hrm'}
//
// john's group in HRM will be 'Domain Users'.
//
// Example 5
// =========
//
// $ldap_authorized_groups = array(... any number of entries ...);
// $ldap_valid_groups = array();
//
// If $ldap_valid_groups is empty and $ldap_authorized_groups contains one or more entries,
// $ldap_valid_groups will be reset to be the same as $ldap_authorized_groups (the end
// behavior will be as in Example 1).

// Groups to use in HRM (filter)
$ldap_valid_groups = array();

// Groups authorized to log in to HRM.
$ldap_authorized_groups = array();

Set up the HRM database

Preparing the database for the HRM consists of two steps, first the corresponding user needs to be created and configured and then the database itself has to be created and prepared for the HRM.

Create the HRM database user

Unless the database user for the HRM is already existing, you need to create it and assign the corresponding permissions.

MySQL

Note

On Ubuntu 18.04 LTS you might need to follow these instructions to set the mysql root password.

The following command will create a new MySQL user hrm and grant the permissions to a database hrm@localhost using the mysql command line tool. Of course, you can also use a database management tool like phpmyadmin to perform this task if you prefer. Make sure to adjust the password for the new database user so it matches the one from your configuration files.

# start the mysql command line client and connect as root:
mysql -u root -p
# Create the database if it does not exist yet:
CREATE DATABASE hrm CHARACTER SET utf8;
# now from within the client, create the database user:
CREATE USER 'hrm'@'localhost' IDENTIFIED BY 'dbpasswd';
# grant permissions to the user:
GRANT ALL ON hrm.* to 'hrm'@'localhost';
exit;

PostgreSQL

su postgres -c "createuser -e -P -d -A -S -R -N hrm"
su postgres -c "createdb hrm"

MD5 host authentication has to be enabled explicitly on Fedora. This can be done using the following commands:

sudo -s
echo -e "host all hrm 127.0.0.1/32 md5\n" >> /var/lib/pgsql/data/pg_hba.conf
echo -e "host all hrm ::1/127 md5\n" >> /var/lib/pgsql/data/pg_hba.conf
service postgresql restart

On Ubuntu MD5 host authentication is enabled by default.

Create or update the database

Run the php script $HRM_SETUP/dbupdate.php from the shell to create and populate the HRM database. The same command can be used to perform an upgrade of the database when the HRM version is upgraded:

cd $HRM_SETUP
php dbupdate.php

If the database does not exist, it will be created using the information stored in the $HRM_CONFIG/hrm_client_config.inc and filled with content for the latest revision. If it exists, it will be updated from whichever revision it currently has.

Review processing server configuration

In the table server of the HRM database, two default values are set for the fields name (i.e. the name of the server running Huygens Core) and huscript_path (i.e. the path pointing to the hucore executable).

name has default value localhost, but can also be something like remote.server.com. huscript_path has default value /usr/local/bin/hucore.

If you need to change the default values, you can use your favorite SQL client and execute a query similar to this (for MySQL):

UPDATE server SET name = 'remote.server.com', huscript_path = '/usr/bin/hucore';

Note

The HRM requires the hucore binary to work, not huscript. Before hucore was introduced HRM was indeed using huscript; the huscript_path entry in the database was kept not to break existing HRM installations.

Warning

The server table contains two other entries: status and job. These are used by the interface at runtime, please do not set them manually!

Install the HRM daemon

The HRM daemon consists of two parts, the actual queue manager (a PHP process) and a shell-based wrapper in $HRM_BIN/hrm_queuemanager that prepares the environment, checks the config, launches the queue manager and cleans up after the process has terminated.

The configuration file for the HRM daemon is /etc/hrm.conf (see Edit hrm.conf)

Warning

You should never start the PHP process directly as unpredicted effects will happen, in worst case including damage to the HRM database.

Depending on whether your Linux installation is using a “classical” System-V like init system (including upstart) or the more modern systemd, you will need to follow different instructions. Please note that systemd is the default in recent versions of Ubuntu and Fedora, as well as its close relatives Debian and CentOS/RHEL.

Install the HRM daemon on System-V-like init systems

Warning

Please make sure to only follow EITHER the instructions given in this chapter here for installing the daemon on System-V or upstart based systems, OR the ones for systemd, but NEVER BOTH.

If unsure, use the instructions here!

Install the init script

To launch the HRM daemon at boot, we are providing an init script in $HRM_RESRC/sysv-init-lsb/hrmd which can be placed in /etc/init.d/. The script makes use of the LSB init functions and therefore requires /lib/lsb/init-functions to be available in your distribution (which is true for Ubuntu and Fedora, as well as most other major up-to-date Linux distributions).

For installing the init script, type the following commands in a shell:

sudo cp -v $HRM_RESRC/sysv-init-lsb/hrmd /etc/init.d/hrmd
sudo chmod +x /etc/init.d/hrmd

Start / stop the daemon at boot / shutdown

Ubuntu: To set up the appropriate run-level links for starting and stopping the daemon during boot and shutdown, use this command:

sudo update-rc.d hrmd defaults

This will configure your system to run the init script in the default run-levels. Please note that issuing this command does NOT actually start the daemon until the next reboot. To start it right now follow the steps in the section below.

Fedora

Recent versions of Fedora do not install the Linux Standard Base (LSB) specification by default, as they are using systemd init. Therefore we recommend using the systemd unit file, described in Install the HRM daemon as a systemd service.

Note

If you want to use the classical init script approach nevertheless, please make sure the package redhat-lsb.x86_64 is installed, i.e. by running:

sudo dnf install redhat-lsb.x86_64

Start manually

Make sure to set the proper value of SUSER in /etc/hrm.conf (see Edit hrm.conf).

Start a shell and type:

sudo /etc/init.d/hrmd start

If the queue manager started correctly, you should see:

Forking background process...
Reporting stdout to '/var/log/hrm/log.txt' and stderr to '/var/log/hrm/error_log.txt'.

To check if the queue manager daemon is running, use the status argument. In case the service is operational, it will show a message like this:

[ ok ] HRM is running (/var/www/hrm/bin/hrm_queuemanager), PID: 1234.

The queue manager can be started, stopped and restarted by using:

sudo /etc/init.d/hrmd start
sudo /etc/init.d/hrmd stop
sudo /etc/init.d/hrmd restart

Install the HRM daemon as a systemd service

If your Linux distribution is using the systemd init system, we recommend to not using the “classical” init script (described in Install the HRM daemon on System-V-like init systems). Instead, we are providing a unit file that can be used with systemd directly.

Warning

Please make sure to only follow EITHER the instructions given in this chapter here for installing the daemon on systemd based systems, OR the ones for System-V/upstart, but NEVER BOTH.

If unsure, or in case of problems with systemd, use the instructions for System-V!

Change the default configuration in the unit file

By default, the unit file is configured to run as the hrm user and to depend on mysql.service. If needed, you can change those values before you proceed with the installation.

[Unit]
Description=HRM (Huygens Remote Manager) Queue Manager Service
# For both 'Requires=' and 'After=', please set one of mysql.service,
# mariadb.service (fork of mysql), postgresql.service.
Requires=mysql.service
Wants=network-online.target
After=mysql.service network.target network-online.target

[Service]
# If needed, change 'User=' and 'Group=' to point to the correct values.
User=hrm
Group=hrm
ExecStart=/var/www/html/hrm/bin/hrm_queuemanager --detach
Type=forking
PIDFile=/var/log/hrm/hrmd.pid

[Install]
WantedBy=multi-user.target

Please change the values of Requires=, After= to point to the correct database you are using for HRM (one of mariadb.service, mysql.service, postgresql.service) and User= and Group= if you are planning to run the Queue Manager as a different user (please see also set-up-the-hrm-user-and-group).

Install the unit file

To launch the HRM daemon as a systemd service, the unit file has to be copied into /etc/systemd/system/ and systemd has to be notified of the new unit. This is done with these commands:

sudo cp -v $HRM_RESRC/systemd/hrmd.service /etc/systemd/system/
sudo systemctl daemon-reload

Start / stop the daemon at boot / shutdown

To tell systemd to start the service at boot and stop it during shutdown, use this command:

sudo systemctl enable hrmd.service

Start / stop / check the HRM daemon manually

To start the daemon manually, type in a shell:

sudo systemctl start hrmd.service

Note that in case of a successful start, there will be no output. If the daemon did not start correctly, systemd will produce a number of output lines telling you about how to debug the problem.

To check if the queue manager daemon is running, use the status command.

sudo systemctl status hrmd.service

In case the service is operational, it will show a message like this:

* hrmd.service - HRM (Huygens Remote Manager) Queue Manager Service
   Loaded: loaded (/etc/systemd/system/hrmd.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-02-11 15:05:22 CET; 18ms ago
  Process: 7782 ExecStart=/var/www/html/hrm/bin/hrm_queuemanager --detach (code=exited, status=0/SUCCESS)
 Main PID: 7785 (hrm_queuemanage)
   CGroup: /system.slice/hrmd.service
           |-7785 /bin/bash /var/www/html/hrm/bin/hrm_queuemanager
           |-7791 php -q /var/www/html/hrm/run/runHuygensRemoteManager.php
           `-7792 sleep 1

The queue manager can be started, stopped and restarted by using:

sudo systemctl start hrmd.service
sudo systemctl stop hrmd.service
sudo systemctl restart hrmd.service

Change the admin password!

The admin user can access the administration backend of the HRM with the username and default password specified in the $HRM_RESRC/passwd.README file.

Please, immediately go to the account page and change the admin password!

Optional configuration steps

Customize the login page

You can customize the login page by renaming the $HRM_USER/login_user.inc.sample file into $HRM_USER/login_user.inc and adding information that you would like to appear on the HRM login page, such as contact information, quick instructions, etc.

Warning

Please use valid HTML code: no validation is performed on your $HRM_USER/login_user.inc HTML code, and you are responsible for any vulnerability you introduce into the HRM!

Example

<h2>Contact information</h2>
<ul>
    <li><a href="mailto:admin@facility.edu?Subject=Help!">HRM admin</a></li>
</ul>

<h2>HRM custom instructions</h2>
<ul>
    <li><a href="http://www.facility.edu/hrm_instr.pdf">Using the HRM at our facility</a></li>
</ul>

With a bit of HTML and CSS skills you can make your login page shine!

Export data share

The HRM offers data upload and download through the broswer. For large datasets, however, it is recommended to set up more performant ways to get data in and out of the HRM file server.

One approach consists of exporting the $HRM_DATA folder as a samba share. See for instance here on how to setup samba.

What follows is a sample configuration for exporting the $HRM_DATA folder (add it to the /etc/samba/smb.conf file):

[hrm_data]
    comment = HRM data folder
    path = /data/hrm_data
    force user = hrm
    force group = hrm
    writeable = Yes
    create mask = 0666
    directory mask = 0777
    guest ok = No
    strict locking = no

OMERO connector

Please make sure to have the OMERO prerequisites set up correctly. To enable the OMERO connector, then set the $omero_transfers variable to true in $HRM_CONFIG/hrm_client_config.inc and in $HRM_CONFIG/hrm_server_config.inc.

<?php
...
//======================================================================
// HRM + Omero
//======================================================================

// Switch on/off (true/false) data transfers between HRM and Omero.
$omero_transfers = true;

?>

In addition, the following values need to be set accordingly in /etc/hrm.conf. By default, those settings are commented out, so make sure to remove the comment sign # in front of those lines!

Assuming your OMERO server is running on a host called omero.mynetwork.xy and using the versions shown in the example in the OMERO prerequisites, this would look as below. Please make sure to adjust OMERO_PKG and OMERO_HOSTNAME so it is matching your current setup.

# Interaction with OMERO (if switched on in hrm/config).
OMERO_PKG="/opt/OMERO/OMERO.server-5.0.3-ice34-b41.linux"
OMERO_HOSTNAME="omero.mynetwork.xy"
OMERO_PORT="4064"

Optionally, you can configure the OMERO connector to look for additional Python packages in a non-standard location (i.e. not within the regular PYTHONPATH). To do so, uncomment the following variable and adjust it accordingly:

PYTHON_EXTLIB="/opt/OMERO/python-extlibs"

Enable GPUs

Log on to HRM as admin and go to Servers and GPUs . The form in Servers and GPUs allows you to register GPUs (as well as other processing servers) in HRM for further parallelization of the HRM processing queue.

To add a new entry, enter the machine name and the absolute path of HuCore on that machine.

To register a machine with no GPU cards enter a negative integer for the GPU ID. Use, for example, GPU ID = -1.

To register a machine with one or more GPU cards simply add different entries, one per GPU. All these entries should have the same Server Name and HuCore Path, but different GPU IDs. As follows,

HRM will consider each GPU card as a separate server. The HRM queue will process as many images in parallel as the total number of registered GPU cards.

The GPU IDs can be retrieved via HuCore. For example, the following machine reports 2 GPUs with ID 0 (Titan X) and ID 1 (Quadro P5000):

> hucore
% huOpt gpu -query names
    {0: TITAN X (Pascal)} {1: Quadro P5000}

Add processing machines

This section describes the specific steps required to configure the HRM to run on multiple machines. In particular, we address the following setup:

• Machine 1: runs the web server (and database) and the queue manager
• Machine 2: runs Huygens Core
• The data folder is not directly shared (via NFS or Samba) between the two machines.

Note

There could be more than on processing machine running Huygens Core in your institution: set up all additional machines like Machine 2.

Configuration steps

Public-key authentication

Image data for deconvolution is transferred from Machine 1 to Machine 2 via secure copy (scp). For this, a specified user on Machine 2 must be created/chosen, and remote access from Machine 1 via password-less, public key authentication must be set up for this user.

Note

For the sake of the example, we will call this specific user hrm. You can pick another user, just make sure to replace hrm with your user name in what follows.

On Machine 2, create a user with name hrm and group hrm that will be used for remote authentication and data transfer. Configure password-less, public key authentication as explained on this page. The public key is uploaded to Machine 2, whereas the private key is stored on Machine 1 for the selected user (e.g. in /home/hrm/.ssh/).

HRM configuration files

On Machine 1, configure the HRM as usual (as in the single-server configuration) but set the following variables in $HRM_CONFIG/hrm_client_config.inc and $HRM_CONFIG/hrm_server_config.inc as shown:

<?php
...
$copy_images_to_huygens_server = true;
$imageProcessingIsOnQueueManager = false;
$huygens_user = "hrm"   // This is the user that will log in to Machine 2
$huygens_group = "hrm"  // And this is its group

Replace hrm with your own user name and group if different.

Register processing machines

Log on to HRM as admin and go to Servers and GPUs . Register the processing servers in HRM by adding an entry for every processing machine, specifiying the absolute Huygens Core path on that machine, as well as the machine name. For details on the GPU ID field see Enable GPUs.

Data folder

On Machine 2, create the data folder pointed at by $huygens_server_image_folder. This folder should be owned by $huygens_user (e.g. “hrm”) and have group ownership as set by $huygens_group (e.g. “hrm”).

Warning

As of version 3.2.0, the HRM expects specific file permissions on the file area! Please see Update the data folder permissions below. The deprecated fall-back mechanism of version 3.2 no longer works in 3.3, 3.4, 3.5 and 3.6!!

Note

These instructions always explain the steps required to upgrade from last stable release to current stable release. However, each section also provides links to instructions for older versions.

Version upgrade

If you have upgraded the HRM in the past, you will know that some steps must be performed in addition to replacing the old HRM code with the new one: some entries might have been added or changed in the configuration files (hrm_{server|client}_config.inc), and the database structure might have been changed.

Stop the Queue Manager

Significant parts of the configuration as well as the database are usually changed during an upgrade, so the Queue Manager needs to be stopped first.

Shut down the Queue Manager with:

sudo /etc/init.d/hrmd stop

if you are using System-V or upstart, or with:

sudo systemctl stop hrmd.service

if you are using systemd.

In some rare situations, the Queue Manager might get stuck. To ensure the stop command did work properly, run the following check.

On System-V or upstart:

sudo /etc/init.d/hrmd status

On systemd:

sudo systemctl status hrmd.service

Alternatively, you can use:

ps aux | grep -i [r]unHuygens

that should return empty (nothing).

Download and extract the new HRM release

To install the new HRM version you need to download the .zip file from the website or github as explained in downloading the standard archive.

Warning

Please do not extract the new archive on top of the previous HRM installation! See details below.

Clean up previous installations

Because of changes in tre structure of the code and of the external dependences (starting from version 3.4), we highly recommend not to extract the new archive on top of the old one.

Please rename the old hrm folder, extract the code into a fresh ${HRM_ROOT} and move the configuration files from the old config subfolder into the new ${HRM_ROOT}/config.

You might also want to reinstall the hrmd or hrmd.service scripts.

Note

Please follow these instructions first if you are upgrading from older versions.

Update the configuration files

There were no configuration changes between versions 3.4.x and 3.5.x of HRM.

Check the configuration files

An easy way to check for modifications is by running the $HRM_HOME/resources/checkConfig.php script. From the shell, run:

cd $HRM_HOME
php resources/checkConfig.php config/hrm_server_config.inc
php resources/checkConfig.php config/hrm_client_config.inc

There were two configuration changes between versions 3.5.x and 3.6 of HRM. The output of the checkConfig.php script should be:

Checking against HRM v3.6.x.

* * * Error: variable default_output_format not set or empty.
* * * Error: variable min_free_mem_launch_requirement not set or empty.
Check completed with errors! Please fix your configuration!

Please make sure to fix all problems you might have! The sample files and the Manual installation instructions will help you set the correct parameters.

Note

Please follow these instructions first if you are upgrading from older versions.

Update the database

Newer versions of the HRM might use slightly different/updated versions of the database back-end than previous ones.

HRM version	Database version
3.6	17
3.5	16
3.4	15
3.3	14
3.2	13
3.1	12
3.0.3	11
3.0	10
2.1	9
2.0	8
1.2.3	7

For this reason, the first time you run the HRM after an update you will be told that the database must be updated and that you are not allowed to continue until this has been done!

Note

Database updates are supported across HRM versions, i.e. it is possible to upgrade the database from revision 7 to 17 in one step.

The following describes two possible ways to update the database.

Note

Although we test this procedure quite carefully, it is highly recommended to backup the database before updating!

Updating from the web interface

Login to the HRM as the admin user: you will be brought directly to the Database update page. Click on the update button. If everything works properly (as it should…), the following message should be displayed.

Needed database revision for HRM v3.6 is number 17.
Current database revision is number 16.
Updating...

Database successfully updated to revision 17.

The database is now at the latest revision.

Updating from the console

Alternatively, the database can be updated from the console (see create or update database). Please pay attention to what the update process will report! The output should be the same as the one listed in the previous section, but if the update fails, you might want to report it.

Re-start the Queue Manager

After processing the described upgrade steps, the Queue Manager needs to be started again, with:

sudo /etc/init.d/hrmd start

if you are using System-V or upstart, or with:

sudo systemctl start hrmd.service

if you are using systemd.

Upgrade from previous releases

The following pages are linked to from the relevant sections above, but are listed here again for convenience.

Clean up previous installations (previous versions)

Warning

These instructions refer to older versions of HRM!

3.4 to 3.5

It is highly recommended not to extract the new archive on top of the old one.

Please rename the old hrm folder, extract the code into a fresh ${HRM_ROOT} and move the configuration files from the old config subfolder into the new ${HRM_ROOT}/config.

You might also want to reinstall the hrmd or hrmd.service scripts.

3.3 to 3.4

In version 3.4, we started a major reorganization of the code structure of HRM and it is therefore highly recommended not to extract the new archive on top of the old one.

Please rename the old hrm folder, extract the code into a fresh ${HRM_ROOT} and move the configuration files from the old config subfolder into the new ${HRM_ROOT}/config.

You might also want to reinstall the hrmd or hrmd.service scripts.

3.2 to 3.3

HRM 3.3 uses new init scripts. Please delete the old files $HRM_BIN/hrm_user_manager, $HRM_BIN/hrm_user_manager_old, $HRM_BIN/hrmd and $HRM_BIN/ome_hrm and then follow the instructions in Upgrade the init script.

Upgrade the init script

Note

If your Linux installation is using the systemd init system, please have a look at the instructions about how to set up the HRM daemon with systemd. Please make sure to remove the old init script at /etc/init.d/hrmd in this case!

Otherwise, proceed as described here for the init script.

This step is basically identical to the initial installation of the init script as described in installing the daemon. You need to copy the new script from $HRM_RESRC/sysv-init-lsb/hrmd to /etc/init.d/ and make sure it is executable. This can be done using the following commands:

sudo cp -v $HRM_RESRC/sysv-init-lsb/hrmd /etc/init.d/hrmd
sudo chmod +x /etc/init.d/hrmd

Update the data folder permissions

As of HRM 3.2.0, the system users running the Queue Manager and the web server are expected to have full read-write access to $HRM_DATA. The supported way of doing this is explained in setting up the HRM user and group. Briefly:

• a user hrm and its corresponding group hrm are created
• the web server user ( www-data, apache) is added to the hrm group
• the variable SUSER is set to hrm in /etc/hrm.conf
• the $HRM_DATA and $HRM_LOG group ownership is set to hrm with the setgid bit set

For detailed instructions, please see setting up the HRM user and group.

Warning

With HRM 3.2 it was possible to preserve the old behavior by setting a configuration variable $change_ownership. This is not supported with HRM 3.3 any more!

3.1 to 3.2

rm -v inc/ActiveDirectory.inc.php inc/Ldap.inc.php

Update the configuration files (previous versions)

Warning

These instructions refer to older versions of HRM!

3.4 to 3.5

There were no changes in the configuration files between versions 3.4 and 3.5 of HRM.

3.3 to 3.4

$authenticateAgainst

The support for various authentication mechanisms was extended in HRM 3.4. This comes with a change in configuration: the $authenticateAgainst variable is now an array and its values have also changed (although, temporarily, the old ones are still supported). Example:

<?php
...
$authenticateAgainst = array("active_dir", "integrated");
...

$useDESEncryption

The configuration variable $useDESEncryption (that was not used) must be removed from the configuration files!

Note

Please follow these instructions first if you are upgrading from older versions.

3.2 to 3.3

The $change_ownership option is not supported in version 3.3 any more. If the variable is present in the configuration files hrm_{server|client}_config.inc it will be silently ignored. For consistency reasons it is therefore recommend to remove this setting from the config file(s).

Group authentication

As of HRM 3.3, external authentication via Active Directory or generic LDAP now supports group authorization. An additional array $AUTHORIZED_GROUPS can be set to define the set of Active Directory or generic LDAP groups that are granted access to HRM.

3.1 to 3.2

As of version 3.2 of HRM, the system users running the Queue Manager and the web server are expected to have direct read-write access to the data folders. If this is not the case for your setup and you rely on adding the web server user to /etc/sudoers, please notice that this behavior is deprecated in 3.2 but can still be enabled by adding:

$change_ownership=true;

in hrm_{server|client}_config.inc. The variable $change_ownership defaults to false if not explicitly set to true in the configuration, in compliance to the new behavior.

Note

As of HRM 3.3, this variable will be ignored and the new behavior will be enforced!

3.0.x to 3.1

No changes in the configuration files.

2.1.x to 3.0

From HRM 2.1.x to HRM 3.0, one variable was added in the configuration files.

$omero_transfers={true|false}

1.2.x to 2.0

From HRM 1.2.x to HRM 2.0, three variables were added in the configuration files:

[+] max_upload_limit
[+] max_post_limit
[+] email_list_separator

Moreover, three variables were removed:

[-] resultImagesOwnedByUser
[-] resultImagesRenamed
[-] enable_code_for_huygens

Note

If you are upgrading straight from HRM 1.2.x, please notice that as of HRM 2.0 configuration and sample files were moved as per following table.

Config files (new)	Sample files (new)	Config files (1.x)	Sample files (1.x)
$HRM_HOME/config	$HRM_HOME/config/samples	$HRM_HOME/inc	$HRM_HOME/resources

Check the configuration files (previous versions)

Warning

These instructions refer to older versions of HRM!

An easy way to check for modifications is by running the $HRM_HOME/resources/checkConfig.php script. From the shell, run:

cd $HRM_HOME
php resources/checkConfig.php config/hrm_server_config.inc

3.4.x to 3.5

Checking the 3.4 files with the 3.5.x checkConfig.php script will result in the following output:

Check against HRM v3.5.x.
Check completed successfully! Your configuration file is valid!

3.3.x to 3.4

Checking the 3.3 files with the 3.4.x checkConfig.php script will result in the following output:

Check against HRM v3.4.x.
* * * Error: variable useDESEncryption must be removed from the configuration files!
* * * Error: variable 'authenticateAgainst' must be an array!
* * * Moreover, please change 'MYSQL' into 'integrated'.
Check completed with errors! Please fix your configuration!

3.2.x to 3.3

Checking the 3.2.x files with the 3.3 checkConfig.php script will result in the following output:

Check against HRM v3.3.x.
* * * Error: variable change_ownership must be removed from the configuration files!
Check completed with errors! Please fix your configuration!

3.1.x to 3.2.x

Checking the 3.1.x files with the 3.2.x checkConfig.php script will result in the following output:

Check against HRM v3.2.x.
Check completed successfully! Your configuration file is valid!

3.0.x to 3.1.x

Checking the 3.0.x files with the 3.1.x checkConfig.php script will result in the following output:

Check against HRM v3.1.x.
Check completed successfully! Your configuration file is valid!

2.1.x to 3.0.x

Checking the 2.1.x files with the 3.0 checkConfig.php script will result in the following output:

Check against HRM v3.0.x.
* * * Error: variable omero_transfers not set or empty.
Check completed with errors! Please fix your configuration!

1.2.x to 2.1.x

Checking the 1.2.x files with the 2.1.x checkConfig.php script will result in the following output:

Check against HRM v2.1.x.
* * * Error: variable max_upload_limit not set or empty.
* * * Error: variable max_post_limit not set or empty.
* * * Error: variable email_list_separator not set or empty.
* * * Error: variable resultImagesOwnedByUser must be removed from the configuration files!
* * * Error: variable resultImagesRenamed must be removed from the configuration files!
* * * Error: variable enable_code_for_huygens must be removed from the configuration files!
Check completed with errors! Please fix your configuration!

Please make sure to fix all problems. The sample files and the Manual installation instructions will help you set the correct parameters.

User manual

These are the detailed instructions for the Huygens Remote Manager User Manual.

Introduction

What is HRM

HRM is a collaborative open-source interface to Huygens Core for multi-user, web-based, scheduled, batch deconvolution.

The Huygens Remote Manager (HRM) enables large groups of users to work with Huygens on a central system through an easy-to-use interface. HRM is web-based and therefore easily accessible from your workstation, home PC, tablet or smart phone.

An HRM server can distribute work among several processing machines smartly, which allows for intelligent use of hardware and fast processing times. It features an intelligent queue that switches among the jobs of all users.

Who makes HRM

HRM is an open source project, this means that the code is freely accessible and it can be adapted and modified to any needs by any user.

HRM is a joined collaboration of Huygens users from:

• Montpellier RIO Imaging,
• Facility for Advanced Imaging and Microscopy at the Friedrich Miescher Institute (FMI, Basel),
• The BioImaging and Optics Platform at the Ecole Polytechnique Fédérale de Lausanne,
• The Department of Biosystems Science and Engineering at the ETH Zürich,
• The Leibniz Institute for Neurobiology (Magdeburg),
• The Combinatorial Neuroimaging facility (Magdeburg),
• The Imaging Core Facility at the Biozentrum (University of Basel).
• The University of Fribourg.
• The Microscopy and Image Analysis Platform of the University of Freiburg.
• The University of Manchester.

Scientific Volume Imaging participates in this project by contributing its experience in deconvolution and software engineering.

Where to find HRM

HRM is a free and open source project, and can be found on http://huygens-rm.org. The source code is hosted on https://github.com/aarpon/hrm.

More information about HRM and links to other HRM resources can be found in this online article of the SVI Wiki. Instructions for online testing, downloading and installing HRM are also linked on that page.

Installing HRM on a regular web server is not very difficult. Please consult the Installation instructions and the SVI Wiki.

Getting started

Here is a quick guide to get you started with the HRM/HuCore deconvolution. In order to optimize results, please refer to section Advanced deconvolution in HRM.

Select UI theme in HRM

HRM offers two UI themes (dark, light) that can be selected from the login page.

The theme selected at the moment of logging in is preserved during the entire session. In this manual screenshots from both themes are displayed interchangeably.

Create an account in HRM

• If your imaging facility already offers deconvolution via HRM, proceed to the facility’s HRM login page.
• If HRM is not available at your imaging facility you can still familiarize with HRM by using the HRM demo server offered by SVI.

Follow the registration link on the HRM login page and complete the short form. Names are case sensitive. Upon registration the new HRM account needs to be validated by the administrator.

Upload a raw image

Upon successful logon the HRM Home panel will be displayed:

To upload images click on the Raw images icon . The following, intuitive upload tool allows for multiple uploads from your local device:

Deconvolve

To submit a deconvolution job proceed to Launch jobs from the home panel. The process of submitting the job is split into the following 5 steps:

• Step 1/5 - Select images: click on the “Image File Format” drop down widget to select the file type of the images to deconvolve. All uploaded files of that format will belisted. Use the icons to add/remove images to/from the selection. When selecting a file, a preview can be generated by using the link on the right hand side panel. The image preview also contains information about the dimensions, sampling and number of channels.

• Step 2/5 - Image parameters: A parameter template describing the imaging conditions of acquisition system. The template can be created from scratch or derived from an existing one. Templates shared by other HRM users (including the administrator) can also be used. Choosing the correct parameters is absolutely crucial for obtaining high-quality deconvolution results.

  Use an existing parameter template (easy): When selecting an existing parameter template the contents are displayed in the panel on the right hand side. Make sure that the microscope type and the number of channels of the template match the images selected in Step 1/5.

  Create a new parameter template (elaborated): Click the New template button . Type a name for the new template and click ‘Create’. The template editor opens, select the correct number of channels and PSF type. Move on to the optical parameters and select the correct microscope type. For a detailed explanation on the remaining parameters please refer to section Advanced deconvolution in HRM.

• Step 3/5 - Restoration parameters: As in the previous step it’s possible to choose between an existing template or creating a new one.

  Use existing parameter template (easy): Select a parameter template and verify its contents on the preview shown on the right hand side panel. Particularly, pay attention to the value of the Signal to Noise ratio (SNR) (see Signal to Noise ratio for further help).

  Create a new parameter template (elaborated): Click the New template button . Type a name for the new template and click ‘Create’. Once in the template editor select a deconvolution algorithm (CMLE is a good choice) and an SNR. Follow the instructions of the SNR estimator as a guide for the SNR of the selected images. Set the background mode to automatic, the number of iterations to 40 and the quality change to 0.01. For more detailed fine-tuning follow the instructions at Advanced deconvolution in HRM.

• Step 4/5 - Analysis parameters: This step is only active when 2 or more channels have been specified in the previous steps. When active, the Analysis step allows the user to execute Colocalization analysis on the deconvolved images.

  Use existing parameter template (easy): Select the analysis parameter template of your choice and verify that the values displayed on the preview on the right hand side panel match your analysis needs.

  Create a new parameter template (elaborated): Click the New template button . Type a name for the new template and click ‘Create’. Select whether colocalization should be enabled. For a colocalization run select the channels to be analyzed as well as the colocalization coefficients of interest. Set the threshold to “Automatic estimation” and the Colocalization map to “Pearson”. In order to further fine-tune the Colocalization parameters please refer to Advanced deconvolution in HRM.

• Step 5/5 - Launch job: Select the output file format for the restored images. ICS and HDF5 are the recommended formats because they provide good dynamic range and the necessary meta data infrastructure to save all the image parameters. Plus, they allow for good compression levels that minimize the amount of disk space needed for storage. Next, review the parameter summaries as well as the image selection. If all the settings are correct then proceed to submit the job to the server by clicking .

Queue

After submitting the job HRM returns to the Home panel. Access the queue by clicking . A typical processing queue looks like the following:

The queue is handled in an intelligent way, allowing jobs from different users to be processed at different times.

Results

When a deconvolution and analysis job is finished a notification email is sent with a link to the result. Alternatively, go to the HRM home panel and click on the Results icon . The contents of the Results folder will be listed. Select the image that you would like to visualize and a preview will be displayed on the right hand side panel:

For a detailed view of both the raw image and the restored image click on .

A results page opens showing MIP, SFP and slicer comparisons of the raw image versus the deconvolved image. Summary tables show information about possible scaling factors, colocalization results, image parameters and deconvolution parameters. Also, movies along the Z direction and/or through time will be available for download.

Tips & Tricks

• When a deconvolution run gives optimal results download the processed data from the detailed view by clicking on .
• Click at steps 2, 3 or 4 to share any of the templates with one or more colleagues.
• To optimize the deconvolution results it is necessary to understand the different properties of the image and some of the mechanisms behind deconvolution. This introductory chapter skipped details that can make a difference on images with spherical aberration, extreme SNR values, undersampling, etc. For a detailed explanation please see Advanced deconvolution in HRM.

Advanced deconvolution in HRM

The Home panel

Once logged in HRM, the home panel is displayed:

From this page the user can manage deconvolution jobs as well as other HRM administrative tasks. The following shortcuts are available:

• Launch jobs : Start new jobs for deconvolution, stabilization, chromatic aberration correction, colocalization, etc.
• Queue status : See all jobs. Manage owned jobs.
• Raw images: Upload new data for deconvolution.
• Results: View and download deconvolved data.
• Statistics: Summary of job statistics.
• Account: View and change login data.

Launch a job

Select images (1/5)

In step 1 - Select Images - the user can add images for deconvolution. A batch of images with the same properties can be selected with Ctrl+click or Shift+click and added in one go.

Select the file format of the batch from the drop down widget and use to add images to the batch selection.

Note

Keep in mind that the images of a batch must share the same microscopic parameters in order to obtain good deconvolution results because they will be restored with the same settings. This is generally true if the images are recorded with the same setup.

When an image is selected a preview is displayed on the right panel whenever possible. If not yet available, the user can click on to generate the preview on the fly.

Note

Image previews also display the image dimensions and sampling sizes in an overlay. This will be useful to know in the subsequent steps.

The Image Parameters (2/5)

In step 2 - Image Parameters - the microscope settings can be specified and saved. The image parameters are grouped in templates. A parameter template can be reused in future deconvolution jobs. This will be useful when there are more images recorded under the same conditions. Moreover, the parameter templates can be shared with one or more HRM users.

The following selections are possible at this stage:

• Admin image templates: parameter templates created by the HRM administrator to be used as references. These templates cannot be changed directly. Instead, they can be copied to Your image templates (see below) for further changes. For copying to Your image templates select the admin template and use .

• Your image templates: the image templates that the user can use directly. All templates in this area may be selected, edited, removed, etc.

Note

show tooltips describing their actions w.r.t templates. Namely, create, edit, clone, import from an image, import from a Huygens template, share, mark as favourite and delete.

An image parameter template consists of several microscopic pararameters relevant for deconvolution. To get a preview of the template contents select the template, the contents will be displayed on the right panel.

Note

When editing a template help links are displayed at key locations to point you to specific articles of the SVI-wiki.

The following parameters can be edited in a template.

Number of channels

Number of fluorescent channels in the image.

Note

Transmission channels can NOT be deconvolved with the algorithms currently available in HRM and should be skipped!

PSF Type

To perform image deconvolution a Point Spread Function (PSF) is needed.

The Huygens software can compute a theoretical PSF from the image parameters used during deconvolution or it can use a measured PSF.

Depending on the user choice HRM will, at a later stage, ask for:

• the path to the file containing the PSF if you selected Measured,
• the settings to correct for spherical aberration if you selected Theoretical and there is a refractive index mismatch.

In most cases a theoretical PSF works fine.

Microscope type

Choose among the supported microscope types: widefield, confocal, spinning disk, multiphoton, STED, STED 3D, SPIM, Rescan, and Array detector (Airyscan). Depending on the Huygens license some of these options might not show.

Note

HRM requires the microscope type as input from the user. HRM will NOT load the microscope type from the selected images automatically.

Import metadata

Some parameters are always present in the meta data of specific file formats and can be imported from the image at run time, while deconvolving the image, to save you time.

Note

When a parameter is missing from the meta data HRM needs input from the user. This is highlighted by a message like this:

Note

When a parameter is present in the meta data HRM can work without the input from the user. This is indicated with

At the end of the deconvolution job, HRM informs on the values of all used parameters as well as their origin (user input or meta data).

Numerical aperture

The Numerical Aperture (NA) describes the amount of light coming from the focus that the objective can collect.

The NA depends on the half angle of the maximum cone of light that can enter or exit the lens. It is directly linked to the resolution of the objective.

Usually, the NA can be found engraved on the objective next to the magnification.

Note

Depending on the selected file format this parameter may be skipped, as explained in Import metadata.

Wavelengths

Excitation and emission wavelengths of each channel. For the emission wavelength the central value of the emission spectrum of the fluorophore can be considered.

Note

Make sure to insert these values in the same order as they were acquired.

Objective Type

Whether a dry (air) or immersion objective (water, glycerol, oil) was used. For other immersion types it is possible to enter their refractive indexes directly.

Note

Several file formats report this value in the meta data. To leave empty please click on .

Sample medium

Whether glycerol, polyvinyl alcohol, vectashield or water was used to embed the sample.

For other embedding media it is also possible to specify their refractive indexes directly.

Note

Few file formats report this value in the meta data. To leave empty please click on .

Voxel Size

The voxel size or sampling size is a very important parameter in image deconvolution.

Note

According to the Nyquist criterion its value should not be larger than half the optical resolution of the imaging system.

HRM offers several utilities to help you set up this parameter correctly.

Ideally, the optimal voxel sizes are calculated before acquiring the images. By doing so, one can ensure that the images are compliant with the Nyquist criterion. Click on to determine how large the voxel sizes should be. By using these values, there will be no lost information during the image acquisition and the deconvolution will be able to restore the original signal better.

If the optimal voxel sizes have not been determined before the acquisition stage one can calculate the resulting sizes in widefield and spinning disk microscopy by using . Here the voxel sizes are calculated from the CCD camera element, the objective magnification, etc., although no optimal voxel size can be guaranteed.

Note

Make sure to set a voxel size consistent with the Nyquist criterion. Undersampled images will often show artifacts after deconvolution.

Lastly, for deconvolving time series set the time interval between subsequent frames.

Among other things, this will help HRM to decide when to stabilize the image.

Using a measured PSF

This option will only appear if ‘measured PSF’ was selected as PSF Type in a previous step.

Distilled PSFs need to be created beforehand using Huygens Professional and uploaded to the Raw images folder.

Note

One PSF image per channel. Mltichannel PSFs are not supported.

When presented with the Distilled PSF file selection window,

browse for the distilled PSF appropriate for the channel(s) of the image.

Select the PSF file and click close. Repeat for all channels.

Afterwards, press save and return to the image parameter selection page.

Note

Files that can’t work as PSFs for the selected image(s) are crossed off and displayed in red.

Pinhole Radius

Pinholes are used in confocal, spinning disk, STED, STED 3D and Rescan microscopy to get rid of out-of-focus light. Since there’s no pinhole in widefield, two photon and SPIM microscopes HRM skips this parameter in those cases.

Note

HRM needs the value of the pinhole backprojected on the specimen.

Further help on how to calculate the backprojected pinhole radius can be found on .

Pinhole Spacing

Spinning disk microscopy uses more than one pinhole to speed up image acquisition. The backprojected distance between these pinholes also plays a role in image deconvolution. HRM skips this parameter for all other microscope types.

Click on for further help on the calculation of the backprojected value.

STED parameters

If the selected microscope type is STED or STED 3D then a few STED parameters need to be specified to describe the STED acquisition. When working with other microscope types HRM will skip these parameters.

First, select the STED depletion mode: Pulsed, Continous wave gated detection, Continous wave non gated detection or Off/Confocal.

Note

STED and confocal microscopy are often combined. That’s why even if the selected microscope type is STED it’s still possible to set specific channels as confocal at this point.

Set the STED saturation factor of each channel. Typical values are in the range of 10 to 50. The higher this factor, the more suppression of fluorescence away from the optical axis, the more resolution.

Next, set the STED depletion wavelength of each channel. Notice that Channel 1 is grayed out in these figures because it was set as confocal.

The STED immunity fraction tells Huygens the percentage of fluorescent molecules that are not susceptible to the STED laser. Typical values are in the range 0% to 10%.

Finally, depending on whether the microscope type is STED 3D or not, set the percentage of STED 3D per channel. This tells Huygens how much power was used in the Z depletion beam. The remaining power is used for the vortex path.

SPIM parameters

If the selected microscope type is SPIM then a few SPIM parameters need to be specified to describe the SPIM acquisition. When working with other microscope types HRM will skip these parameters.

First, select the SPIM excitation mode of each channel: Gaussian light sheet, Gaussian MultiView light sheet, High fill factor (cylinder), or High fill factor (scanning beam).

Set the width of the Gaussian light sheet for the channels acquired with one of the Gaussian modes. The field will be grayed out for the other channels.

The width is defined as the distance between the two (symmetric) points of the Gaussian profile where the intensity values drop to 2/e^2 of the peak value.

Set the SPIM Focus Offset of the light sheet. This is the distance along the optical axis of the detection lens between the focus of the illumination lens and the focus of the detection lens.

Similarly, set the SPIM Center Offset of the light sheet. This is the distance along the optical axis of the illumination lens between the focus of the illumination lens and the focus of the detection lens.

Set the SPIM NA for the channels aquired with the one of the High Fill Factor modes. This is the Numerical Aperture of the illumination lens.

Then, the fill factor of the illumination lens, for the channels acquired with one of the High Fill Factor modes. This is the ratio between the beam width and the diameter of the objective pupil in the illumination lens.

Lastly, the SPIM Illumination Direction. This can be left, right, bottom, top for any of the non-MuVi (Multi-View) excitation modes; and right+left or top+bottom for the Gaussian MuVi excitation mode.

Spherical Aberration correction

Spherical aberration is mainly caused by a mismatch between the refractive indexes of the objective immersion medium and the sample embedding medium.

HRM will automatically detect this mismatch and ask whether the aberration should be corrected for when using a theoretical PSF.

Note

Spherical aberration can be corrected for by creating different theoretical PSFs (with slightly changing shapes) at different depths: depth-dependent PSF.

When using a depth-dependent PSF one also needs to specify whether the first plane in the dataset is closest or farthest to/from the coverslip.

The Correction mode allows for a more detailed choice as to how the depth-dependent PSF should vary with depth.

• Automatic correction: will generate a depth-dependent PSF where the PSF is automatically selected at each depth, as explained above.
• Advanced correction: allows to specify further details about the depth-dependent PSF.

The Advanced correction will further enable a number of advanced options.

• Depth-dependent PSF on few bricks: a different PSF will be generated for each Z brick in which the original data is split.
• Depth-dependent PSF slice by slice: a different PSF will be generated for each slice.
• Depth-dependent PSF on few slabs: several PSFs will be generated for each Z brick. This option is memory-consuming but the best choice when the spherical aberration is very strong.

Note

Splitting the original data into bricks while deconvolving not only helps to correct for spherical aberration but it also minimizes the amount of RAM memory needed for deconvolution.

The Restoration Parameters (3/5)

In step 3 - Restoration Parameters - the restoration settings can be specified and saved. As in The Image Parameters (2/5) the restoration parameters are grouped in templates. A template can be reused in future deconvolution jobs. This will be useful when other images need to be restored with the same settings. Moreover, the templates can be shared among HRM users, fostering collaboration and re-usage of good restoration settings.

The following selections are possible at this stage:

• Admin restoration templates: parameter templates created by the HRM administrator to be used as references. They can be copied and edited by the HRM users. In order to use them, they first need to be copied to Your restoration templates by using .
• Your restoration templates: the restoration templates that the user can use directly. All templates in this area may be selected, edited, removed, etc.

Note

show tooltips describing their actions w.r.t templates. Namely, create, edit, clone, import from a Huygens template, share, mark as favourite and delete.

A restoration parameter template consists of a number of options which provide Huygens Core with information on the restoration procedure. To get a preview of the template contents select the template, the contents will be displayed on the right panel.

Note

When editing a template help links are displayed at key locations to point you to specific articles of the SVI-wiki.

The following parameters can be edited in a template.

Deconvolution algorithm

Choose one of the deconvolution algorithms from the drop down widget.

At the moment HRM offers three algorithms for the restoration of the images.

• Classic Maximum Likelihood Estimation (CMLE): CMLE is the method of choice under most circumstances. In case of doubt, CMLE should be used.
• Quick Maximum Likelihood Estimation (QMLE): QMLE is faster than CMLE, but gives slightly less precise solutions in some cases. One may consider using QMLE in compute-intensive situations, for example, when deconvolving widefield 3D-time series.
• Good’s roughness Maximum Likelihood Estimation (GMLE): GMLE is a good alternative for high-noise images, as STED or confocal.

Note

All channels of an image will be deconvolved with the same algorithm.

Signal to Noise ratio

In the Huygens Software the Signal-to-Noise ratio (SNR) is treated as a regularization parameter, i.e., a means to control the sharpness of the deconvolution result.

Yet, please be aware of the effect of extreme SNR values on image deconvolution: too high SNR values can produce restoration artifacts; too low SNR values can lead to smooth structures.

Note

After a little practice it is possible to assess the SNR range for deconvolution visually. In general, the SNR should increase with the quality and amount of signal of the raw data.

There are different acquisition settings that have an impact on the quality of the recorded signal. Changing these in the acquisition system will most likely require a modification of the SNR value for deconvolution. These acquisition settings are:

• Gain / offset
• Time exposure / scanning velocity
• Summing / averaging
• Laser power
• Spectral detection range

Because these settings are typically different for each channel HRM will require an SNR value per channel, like this:

As a guide, these are typical SNR values for different microscopy tecniques:

• Confocal: between 20 and 40.
• STED: between 10 and 20.
• Widefield: between 40 and 60.

HRM offers an automatic SNR estimator to determine suitable SNR values for the raw images. Click on to use the SNR estimator.

In a new page you will be asked to select an SNR estimation method.

The Auto method gets good estimations on images without a baseline. When working with images that contain a baseline the Legacy method can be used instead.

Next, select an image representative of the batch selected in Select images (1/5). Press for the SNR estimation:

The output of the SNR estimator shows a region of the original data next to 4 different noise simulations of the same region. In bold letters the SNR of the simulation that most resembles the original image.

By pressing the estimated SNR values are copied into the restoration parameter template.

Crop image

HRM can intelligently crop your images without losing important data.

Select from:

• Apply conservative crop: to save computation time.
• Do not crop the image: if computation time is not an issue.

Note

The time needed to deconvolve an image increases more than proportionally with the image volume. Thus, cropping the image will speed up its restoration and the processing of the HRM queue.

Stabilization in Z

This option will only be shown when restoring STED images (including STED 3D). Due to the high lateral resolution in STED, image acquisition in more sensitive to drift in STED than in other microscopy modes.

Note

It is strongly recommended to select Z stabilization on STED images.

The stabilization is performed before deconvolution. This significantly improves the restoration results.

Z stabilization is also recommended on STED 3D images although Huygens can automatically skip it depending on the PSF shape of each particular case.

Array Detector Reduction Mode

This option will only be shown when restoring Array Detector images (Airyscan, SPAD, etc). Array Detector images contain one acquisition per detector. The user can tell Huygens how to combine the data from all the detectors for deconvolution.

The following options are possible:

• All: the images from all detectors are centered and combined, thus increasing the SNR. The resulting, combined image is then deconvolved.
• No: the independent detectors are not combined prior to deconvolution. They are used as separate inputs of a ‘parallel’ deconvolution.
• Core all: Same as All but taking into account the central detectors only.
• Core no: Same as No but taking into account the central detectors only.
• SuperY: Create an image with 4 times as many samples in Y. This is mostly useful when dealing with images recorded with the Zeiss Airyscan in fast mode.
• SuperXY: Create an image with double the samples in X and Y. Thus, producing an image with 4 times more samples. This is a good option when dealing with images that have been acquired well under the Nyquist sampling rate.
• Auto: Automatic selection of one of the above mentioned modes depending on the microscopic parameters and detector model of the image.

In all cases the SNR value specified in Signal to Noise ratio is for the central detector. Huygens will scale the SNR for the combined detectors automatically.

Background mode

The background intensity is a nearly constant value that is present everywhere in the image.

Three options are available for background correction. These options can return slightly different values, therefore this choice can affect the deconvolution result:

• Automatic background estimation: This estimation usually works well. The background is estimated within a region with a low mean value.
• In/near object: The background is estimated around intensity peaks. This option can be interesting, for example, when having bright little objects in a cell with a strong cytoplasmic background.
• Remove constant absolute value: To make sure that the same background level is removed from all the images in the batch, insert manually a measured mean background for each channel. This option is typically useful for those interested in doing fluorescence quantification or stitching.

Stopping criteria

The restoration algorithms offered in HRM, CMLE, QMLE, and GMLE, are iterative methods. This means that the algorithms compute sequential solutions which converge to a stable result.

Deconvolution will stop when either of the following two conditions is met.

• Number of iterations: sets the maximum number of iterations that Huygens will compute.
• Quality change: how much the results of two consecutive iterations differ. If two subsequent results differ less than this factor then convergence has been reached.

Chromatic Aberration Correction

This option will only be shown when restoring multi channel images. In order to let HRM know that an image batch contains multi channel images please set the number of channels correctly, as explained in Number of channels.

The restoration template editor contains a section for post-deconvolution operations. Options for these operations (chromatic aberration correction and stabilization of time series) only show up when applicable, i.e, multichannel images and time series.

Therefore, the following section will only be displayed when restoring multi channel images:

Choose a reference channel:

The chromatic aberration can be described by an x-y-z shift + rotation + scale (zoom) of each channel with respect to the reference channel. Therefore, by definition the reference channel has no x-y-z shift, rotation or scale. For the other channels, type in the shifts, rotations and scales.

The table will contain one entry per channel.

For an accurate estimation of the chromatic aberration one can use a local installation of Huygens Professional or Huygens Essential, see Chromatic Aberration Corrector.

Use the estimated x-y-z shifts, rotations and scale values reported by Huygens Professional or Essential to fill out the table. All images recorded with the same settings and the same microscope can be corrected with the same chromatic aberration values.

Note

For skipping this correction simply leave the table fields empty.

Stabilization of time series

This option will only be shown when restoring time series. In order to let HRM know that an image batch contains time series please set the time interval correctly, as explained in Voxel Size.

The restoration template editor contains a section for post-deconvolution operations. Options for these operations (chromatic aberration correction and stabilization of time series) only show up when applicable, i.e, multichannel images and time series.

Therefore, the following option will only be displayed when restoring a time series:

Set a stabilization method:

• Center of mass: works best if the image contains a single large object. No objects should cross the image borders.
• Correlation: for general x-y-z translations and axial rotations. Adjacent time frames will be compared. The software will try to find the best alignment by maximizing structural overlap.
• Model based: if the geometry of the imaged object did not change much during the acquisition.

Choose whether to detect and correct rotations:

Lastly, specify the cropping mode for the resulting, stabilized time series:

• Original: to crop the result to the size of the raw data.
• Full: to preserve the size of the stabilized data.
• Tight: to crop away large background regions.

Note

Depending on the cropping mode the dimensions of the result and the raw data will differ. This will become noticeable when comparing the data later. See Results: detailed view.

Analysis parameters (4/5)

In step 4 - Analysis Parameters - the analysis settings for your images can be specified and saved. As always, the analysis parameters are grouped in templates. A parameter template can be reused in future deconvolution and analysis jobs. This will be useful when the same type of analysis has to be performed on other images. Moreover, the parameter templates can be shared among HRM users, fostering collaboration and re-usage of good analysis settings.

The following selections are possible at this stage:

• Admin analysis templates: parameter templates created by the HRM administrator to be used as references. They can be copied and edited by the HRM users. In order to use them, they first need to be copied to Your analysis templates by using .
• Your analysis templates: the analysis templates that the user can use directly. All templates in this area may be selected, edited, removed, etc.

Note

show tooltips stating their action w.r.t templates. Namely, create, edit, clone, share, mark as favourite and delete.

An analysis parameter template consists of a number of options that provide Huygens Core with information about the type of analysis to execute on the images. To get a preview of the template contents select the template, the contents will be displayed on the right panel.

Note

When editing a template help links are displayed at key locations to point you to specific articles of the SVI-wiki.

The following parameters can be edited in a template.

Colocalization

Colocalization is a type of analysis which measures the amount of overlap between the objects of two channels.

Here simply select whether or not colocalization analysis should be performed after deconvolution.

Note

Colocalization can only be performed on multichannel images.

Channels

Select the channels to be analyzed.

All the combinations of the selected channels will be studied. For example, the selection shown in the figure above will result in the analysis of the following channels:

• 0 and 1,
• 0 and 2,
• 1 and 2.

Colocalization coefficients

HRM can report all the colocalization coefficients that are most widely used in fluorescence microscopy.

Select here the colocalization coefficients for the analysis:

A colocalization report with all these findings will be ready when the job is finished.

For further information on specific colocalization coefficients please read this article from the SVI wiki.

Threshold

In order to discard background signals from the colocalization analysis it is possible to set a threshold level.

The automatic option typically works well for most purposes. Yet, it’s possible to specify a threshold per channel to discard particular features of each colour.

Colocalization map

While a colocalization coefficient quantifies the level of overlap between two whole channels, a colocalization map displays the value of a specific colocalization coefficient for each voxel of the two channels.

Therefore, a colocalization map is a 3D image of the level of overlap between the two channels, according to a specific coefficient.

It is possible to choose between the 4 most widely used colocalization maps. The maps will be displayed in the colocalization report after the job is finished.

Note

The colocalization maps help us to assess the colocalization regions visually.

As an example the below figure shows the deconvolution result of channels 0 and 3 next to the colocalization map (Pearson coefficient) of the same channels.

Launch the job (5/5)

In this last step several summaries display all the chosen parameters.

Also, the output format for the result images can be selected at the top of the page:

• ICS, ICS2 and HDF5: support multi-channel images, 32 bit dynamic range (preserve all image details) and all the microscopic metadata parameters. Furthermore, they support high compression levels. Since deconvolved images often contain background regions with near 0 intensities the compression algorithms usualy manage to greatly reduce the size of the images. For all these reasons these are the formats recommended for further work with Huygens.
• TIFF: this format can be used for analysis such as counting or segmentation. For quantification one needs to scale the TIFF image back to its original dynamic range. The corresponding scaling factors are reported by HRM and can be found in Parameter summaries.
• IMS, OME-XML, R3D: formats offered for compatibility with other programs.

Next, review all the selected parameters before submitting the job to the server. It’s possible to go back to the image, restoration, and analysis template editors by clicking on the corresponding links.

Click on to change the image parameters.

Click on to change the restoration parameters.

Click on to change the analysis parameters.

To change the image selection click on .

Finally submit the job by pressing .

To start a new job with deconvolution, chromatic aberration, stabilization, or colocalization analysis click on Launch jobs in the home panel.

Starting a new job is split into 5 main steps:

• (1/5) Selection of raw images.
• (2/5) Image parameters: enter parameters about the acquisition system and the image(s).
• (3/5) Restoration parameters: enter parameters for deconvolution, chromactic aberration, and stabilization.
• (4/5) Analysis parameters: enter parameters for colocalization analysis.
• (5/5) Launch: overview and selection of the output file format.

The Queue status

After submitting a job HRM returns to the home panel. The Queue Status button shows how many queued jobs you own, as follows:

Click on the button to get a detailed view of the job queue.

HRM manages the deconvolution of multiple jobs owned by different users through a smart queue. The job being processed is marked in green.

The server field will show the name of the machine and, when applicable, the number of the GPU card processing the image. HRM can process multiple jobs at the same time.

Note

The total number of processing machines and GPU cards defines the jobs than HRM can process in parallel.

To delete any of your jobs, select them in the queue table and click .

Note

Depending on the HRM configuration settings, a notification email will be sent to the job owner when the job is finished.

If the job’s result seems wrong, try to verify if there is a mistake in the settings. Contact the HRM administrator otherwise.

Results: detailed view

When the job finishes, the files are placed in the Results folder, accessible via the home panel.

Click on an image result for further inspection as explained in Results .

The detailed view shows the following information:

• MIP of original vs deconvolved,
• SFP of original vs deconvolved,
• Z Slicer of original vs deconvolved,
• T Slicer of original vs deconvolved,
• Stack movie of deconvolved,
• T movie of deconvolved,
• T SFP movie of deconvolved,
• Colocalization results,
• Parameter summaries,
• Huygens log.

Note

The number of tools shown in the detailed view depends on the type of restoration, the image geometry and the HRM configuration.

Comparing MIPs

A comparison of the Maximum Intensity Projections of the raw image and the restored image shows the effect of deconvolution. This can be assessed in both XY and XZ projections.

Comparing SFPs

A comparison of the Simulated Fluoresence Process in both the raw image and the restored image also shows the effect of deconvolution and difference in noise.

Comparing Z Slices

The slicer allows for a comparision of the raw image and the restored image slice by slice along the Z direction at any depth.

Comparing T frames

The time slicer allows for a comparision of the raw image and the restored image frame by frame.

Z, T and T-SFP movies

Click on the corresponding links to download any of the movies to your local computer.

Note

These automatically made movies are quite useful for prepraring presentations.

Colocalization

The selected colocalization coefficients of each two channels are listed in a table with an entry per frame.

Also, 2D histograms show the correlations between any two channels.

Lastly, the colocalization maps show the regions where colocalization between any two channels is strongest.

Parameter summaries

HRM also gives feedback on all the parameter values used during deconvolution. This is particularly useful when the parameters are read from the image metadata.

Notice that green entries indicate parameters loaded from the image metadata, whereas violet entries indicate parameters defined by the user.

If the image had to be scaled to fit in the output file format (some formats have very limited dynamic ranges) HRM will show the scaling factors for further quantification:

Huygens log

The exact steps followed by Huygens to process the data can be found on the Huygens log, under the link .

Note

All these results can be donwloaded along with the deconvolved dataset by clicking at the top of the page.

Tips & Tricks

• File uploads: When using the browser uploader it is possible to upload compressed files containing more images. HRM will uncompress the file and set the contents in your raw folder.
• File uploads: Add more files to your selection by clicking . Use drag&drop from your desktop into the browser.
• Image previews: Select an image and click on Generate preview to visualize the image contents plus the dimensions and sampling sizes. The colours used in the preview depict the wavelengths provided by the image metadata.
• Voxel sizes: Whenever possible HRM highlights in yellow the ideal voxel sizes according to the Nyquist criterion.
• SNR estimation: Notice that it’s possible to zoom in the thumbnails displayed by the SNR estimator by moving the mouse pointer over the thumbnails. This can be useful to confirm the correctness of the SNR
• SNR estimation: The SNR can highly influence the deconvolution result. On the one hand, if the deconvolution result looks too smooth and details are missing, a higher SNR value can be used. On the other hand, if the result looks too grainy a lower SNR value may be better.
• Colocalization: Filter out unsignificant colocalization coefficients by using the Highlight button of the colocalization section in the detailed view.
• Select images (1/5): Tick the “When applicable, load file series automatically” checkbox if several independent files should be loaded into a single image.

This section explains how to:

• upload/download images.
• create and launch deconvolution jobs.
• inspect deconvolution results.
• access colocalization coefficients and maps.
• import/export data from/to OMERO.
• view user statistics.

For instructions on how to request an HRM account please see Create an account in HRM.

Other features in HRM

Communication with an OMERO server

HRM can import and export images from/to a local or remote OMERO server.

Note

This feature is not active by default. Ask the HRM administrator for more information about the HRM/OMERO connector.

If the connection between HRM and OMERO is enabled the OMERO logo will be is displayed in the Raw images and Results folders.

Click on . The following window will ask you to enter your OMERO login credentials.

Upon logging in successfully a file tree will be displayed listing all the images in the OMERO repository that you are allowed to visualize, like this:

Note

The OMERO credentials and file tree will be used throughout the HRM session. When the session is closed HRM will remove the OMERO credentials from the cache.

The OMERO tree is requested automatically only the first time in every HRM session. This saves waiting time in subsequent import/export operations with OMERO. To force a tree rebuild click .

The following actions are possible:

• Import data from OMERO: In the Raw images folder select an image from the OMERO tree and click . The image will be imported to HRM with extension .ome.tiff. Select more images with Ctrl + click for importing multiple images in one go.
• Export data to OMERO: In the Results folder select the HRM image to be exported. Select the target OMERO dataset. Click . The image will be attached to the OMERO project with an annotation describing all the parameters used for deconvolution. Select more images from the Results folder for exporting multiple images in one go.

Statistics

The user statistics can be reached from the HRM home panel.

Click on Statistics to create reports about the deconvolution and analysis jobs within a specific time window.

The following reports are possible:

• Input file format.
• Output file format.
• Type of PSF used during deconvolution.
• Microscope type.
• Run time per user.

Account

Click on in the home panel to change any of the details linked to your HRM account.

The role of the administrator in HRM

Technical features

HRM consists of two main components: a web based interface and a queue manager.

The web interface allows:

• the management of users by the system administrator;
• the management of parameter sets (templates) that all users can copy or use directly;
• the creation of deconvolution jobs, including image selection, setting microscopic, restoration and analysis parameters;
• inspecting the job queue status, and allowing the users to delete their own jobs from it;
• previewing and analyzing deconvolved images, including a slicer, MIP and SFP renderers, movies and colocalization results;
• sharing templates among users;
• visualizing user and jobs statistics.

HRM is equipped with a simple http file uploader/downloader to send raw images from the user’s local machine to the HRM server, as well as to retrieve the deconvolution results from the server. The server administrator can set up a limit for these transactions.

The queue manager, running the background, dispatches:

• the jobs created via the web interface to any of the dedicated servers running Huygens Core.
• the emails to inform the users that the job is finished and that the restored datasets are available.

Installation and requirements

To install HRM the following pre-requisites (at least) must be fulfilled:

• Operating system: Any recent Linux distribution. Ubuntu and Fedora are the recommended distro’s for HRM.
• Huygens Core: HRM is just an interface and needs Huygens Core to perform deconvolution on the raw images. Note that Huygens Core needs a license.
• Apache2 web server.
• PHP version ≥ 5.6: Both the HRM queue manager and the web interface are written in PHP and need PHP to operate.
• MySQL or PostgreSQL: A relational database management system is required to store deconvolution parameters, job descriptions and, optionally, user accounts.
• A file server: To temporarily store input and restored datasets.

The setup is highly configurable, since the file server, the processing servers and the queue manager can either be all hosted by the same machine or be distributed across two, three or more computers.

To follow the steps for a guided installation please see the installation page at the HRM project’s site.

Administrator’s options

Log in to HRM with the credentials of the administrator. Instead of the regular user home panel HRM will display the administrator home panel, like this:

There are a number of extra options that are available to the administrator only:

• Manage users: Add, remove, edit, enable and disable users. Alternatively, HRM allows for LDAP/Active Directory user management.
• Queue status: Manage jobs from all users.
• Global statistics: See statistics from all users and jobs, including microscope types, PSFs, research groups, dates, etc.
• Database update: Carry out a database update when a new HRM release requires it.
• System summary: Get an overview of all the versions and configuration options that play a role in HRM.
• Check for updates: Get automatic feedback on whether you are running the latest HRM version.
• Create global templates: The administrator can create global templates accessible to all users. This is meaningful for lowering the threshold for those beginning users who need assistance to start deconvolving their raw data. Global templates can be created at any of the states where templates are required: The Image Parameters (2/5), The Restoration Parameters (3/5) and Analysis parameters (4/5).
• Servers & GPUs: Add and remove processing machines and GPU cards. This has an effect on the total number of jobs that can run in parallel.
• HuCore licenses: Get an overview of the installed Huygens license.

Version upgrade

Note

Before upgrading HRM please proceed to backup the HRM database, images and source files.

To upgrade to a new version, download the latest version from here. Then follow the intructions on specific version upgrades.

Database update

Note

The HRM database should NOT be deleted and replaced by a newer version.

The HRM database contains the existing users, statistics, parameter templates, etc. To update these contents to the latest release click on Database Update from the admin home panel and follow the instructions.

In order for this update to work properly it is advised to upgrade the HRM source code before upgrading the database.

System configuration

HRM can be configured during the installation. The configuration settings are saved in a configuration file. Some settings, as the maximum upload limit, need to be adjusted both in HRM and in PHP.

For example, if the PHP configuration file states a max upload file size of 256MB, this is applicable to all programs using PHP. Suppose the HRM settings state that the max upload limit is 200MB, since 200MB is lower than 256MB, the HRM configuration does not conflict with the PHP configuration and 200MB is the maximum allowed file size.

Note

The PHP configuration file: /path/to/php/php.ini

The HRM configuration file: /path/to/hrm/config/hrm_client_config.inc

How HRM communicates with Huygens Core

For each deconvolution task in the job queue the HRM queue manager automatically generates a Huygens Batch template for Huygens Core that

• loads the raw image from a source directory,
• applies the microscopic parameters to it as defined by the user or reads the microscopic parameters from the image metadata,
• optionally loads another image containing the microscope Point Spread Function,
• deconvolves the image using the restoration parameters chosen by the user,
• optionally applies a chromatic aberration correction,
• optionally stabilizes the time series,
• optionally computes the colocalization coefficients of multi-channel images and generates colocalization maps,
• stores the resulting restored image in a destination directory,
• generates several visualizations of the raw and deconvolved images for the user to see the effect of the restoration,
• and finally writes a tag in the destination directory to inform the HRM queue manager that the job is finished.

When the job is finished the queue manager optionally sends the user an e-mail announcing the end of the job and its status. The administrator may configure this.

Multiple jobs can be processed in parallel depending on how HRM is configured, the multiprocessing capabilities of the server, the number of available computation servers and GPU cards.

Tips & Tricks

• Use the global statistics to get an idea on which microscopes are used most, which file formats, PSFs, etc.
• On the home panel, click on System summary to get an overview of your current HRM settings. Expand this information with the PHP settings and/or test the HRM e-mails by clicking on .
• When performing an HRM upgrade disable all users first. Upon successfully upgrading HRM enable the users back again.
• If you experience unexpected behaviour from the HRM queue manager, such as a deleted job which persists in the queue, restart the HRM daemon.
• Encourage your users to save deconvolved data in ICS or HDF5 formats since the dynamic range and metadata parameters are conserved (unlike in TIFF). Moreover both ICS and HDF5 allow for good compression rates.

Clean a Stalled Queue in HRM

It could happen that the queue breaks or freezes and the database is polluted with outdated information.

To fix this, one must currently go and clean up a few places in the hrm database as well as restart the queue manager. Here is a short checklist of what to do.

1. shut down the hrm daemon by using

sudo /etc/init.d/hrmd stop

or

sudo systemctl stop hrmd

2. go to the hrm database and do the following

   1. Remove broken or killed jobs from the job_* tables
   2. Make sure that the server table has a status of ‘free’ and job set to NULL

   Both measures can be achieved with the following SQL script:

DELETE FROM job_analysis_parameter WHERE setting IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_analysis_setting WHERE name IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_files WHERE job IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_parameter WHERE setting IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_parameter_setting WHERE name IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_task_parameter WHERE setting IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_task_setting WHERE name IN (SELECT id FROM job_queue WHERE status="broken" OR status="kill");
DELETE FROM job_queue  WHERE status="broken" OR status="kill";
UPDATE server SET status= 'free', job = NULL;

3. restart the hrm daemon

sudo /etc/init.d/hrmd start