OBiBa Agate Documentation
Targeted at individual studies and study consortia, OBiBa software stack (Opal, Mica etc.) provides a software solution for epidemiological data management, analysis and publication. While Opal, the core data warehouse application, provides all the necessary tools to import, transform and describe data, Mica provides everything needed to build personalized web data portals and publish content of research activities of both studies and consortia.
Agate is the OBiBa’s central authentication server which intends to be easy to install and to use. Agate centralizes also some user related services such as profile management, and a notification system using emails.
Introduction
Users, Groups and Applications
The following diagram describes the domain handled by Agate. Each entity of this domain can be edited individually in the Agate Web Application administration interface.

User
A user is described by some properties. Among these properties, the user name and email must be unique in the system: when signing in, a user can provide its name or email. The authentication is done by providing a password, which is stored in the Agate database in a digested form.
A user can belong to some groups.
A user can have access to some applications. If no application is provided, the user can only access to Agate. Otherwise, listed applications will have the user authenticated by Agate.
Group
A group is uniquely identified by its name. A group can be associated to one or applications.
Members of a group can have access to the applications associated to it.
Application
An application has a name and a key. Each time an external application wants to use the services of Agate, it must provide in the request its name and key. This allows Agate to check the validity of the actions to be performed and the information to be returned.
Its redirect URI is used when authenticating through the OpenID Connect Flow to validate the source application.
Authentication Flow
When a user tries to sign-in an application X, this application delegates the user authentication to Agate. If successful, a ticket is created in Agate (to track user activity) and user session local to the application X is created. This local user session allows the application to not query Agate each time user authorization check is requested.

Architecture, Servers and Clients
The architecture of Mica is split in several servers:
Mica server: holds the domain and controls what is to be published, provides the web portal front-end and uses Agate as its user directory.
Opal server: holds the data with their dictionary and provide statistics services,
Agate server: user directory for data access requests management.
Mica, Opal and Agate are applications developed by OBiba. Each of these OBiBa servers expose web services to allow easy interconnection. The Mica web portal is the final application which leverages each server specific domain and functionalities in one.
The following diagram shows how these servers are linked together:

Agate Server
Agate application is used for:
having a user directory shared between OBiBa’s applications,
having centralized services such as profile management and email notifications.
Mica Server
Mica application is used for:
defining and publishing network, study and dataset catalogues,
search for variables.
Installation and configuration guides can be found in the section Mica Server Administrator Guide.
Editors and reviewers of the Mica web portal content can access to the web interface of this server as described in the Mica Web Application User Guide.
Mica server is a client of Opal and Agate servers.
Opal Server
Opal application is used for:
defining data dictionaries (variables),
storing data,
providing data summary statistics.
Opal offers well established security controls, allowing to NOT expose individual-level data. Note also that the Opal server is only accessed by the Mica server, reducing the risk of data compromisation from a malicious end user.
Installation and configuration guides can be found in the Opal Server Administrator Guide.
Mica expects at least one Opal server when some datasets are defined. Additional Opal servers can also be identified to access to distributed datasets.
Installation
Agate is a stand-alone Java server application that requires MongoDB as database engine.
Requirements
Server Hardware Requirements
Component |
Requirement |
---|---|
CPU |
Recent server-grade or high-end consumer-grade processor |
Disk space |
8GB or more. |
Memory (RAM) |
Minimum: 4GB, Recommended: >4GB |
Server Software Requirements
Software |
Version |
Download link |
Usage |
---|---|---|---|
Java |
21 |
Java runtime environment |
|
MongoDB |
<= 6.1.x |
Database engine |
While Java is required by Agate server application, MongoDB can be installed on another server.
Install
Agate is distributed as a Debian/RPM package and as a zip file. The resulting installation has default configuration that makes Agate ready to be used (as soon as a MongoDB server is available). Once installation is done, see Configuration instructions.
Debian Package Installation
Agate is available as a Debian package from OBiBa Debian repository. To proceed installation, do as follows:
Install Debian package. Follow the instructions in the repository main page for installing Agate.
Manage Agate Service: after package installation, Agate server is running: see how to manage the Service.
RPM Package Installation
Agate is available as a RPM package from OBiBa RPM repository. To proceed installation, do as follows:
Install RPM package. Follow the instructions in the RPM repository main page for installing Agate.
Manage Agate Service: after package installation, Agate is running: see how to manage the Service.
Zip Distribution Installation
Agate is also available as a Zip file. To install Agate zip distribution, proceed as follows:
Unzip the Agate distribution. Note that the zip file contains a root directory named agate-x.y.z-dist (where x, y and z are the major, minor and micro releases, respectively). You can copy it wherever you want. You can also rename it.
Create an
AGATE_HOME
environment variableSeparate Agate home from Agate distribution directories (recommended). This will facilitate subsequent upgrades.
Set-up example for Linux:
mkdir agate-home
cp -r agate-x-dist/conf agate-home
export AGATE_HOME=`pwd`/agate-home
./agate-x-dist/bin/agate
Launch Agate. This step will create/update the database schema for Agate and will start Agate: see Regular Command.
For the administrator accounts, the credentials are “administrator” as username and “password” as password. See User Directories Configuration to change it.
Docker Image Installation
OBiBa is an early adopter of the Docker technology, providing its own images from the Docker Hub repository.
A typical docker-compose file (including a MongoDB database) would be:
version: '3'
services:
agate:
image: obiba/agate
ports:
- "8881:8081"
links:
- mongo
environment:
- AGATE_ADMINISTRATOR_PASSWORD=password
- MONGO_HOST=mongo
- MONGO_PORT=27017
- RECAPTCHA_SITE_KEY=6Lfo7gYTAAAAAOyl8_MHuH-AVBzRDtpIuJrjL3Pb
- RECAPTCHA_SECRET_KEY=6Lfo7gYTAAAAADym-vSDvPBeBCXaxIprA0QXLk_b
volumes:
- /tmp/agate:/srv
mongo:
image: mongo
Then environment variables that are exposed by this image are:
Environment Variable |
Description |
---|---|
|
|
|
Agate administrator password, required and set at first start. |
|
MongoDB server host (optional). |
|
MongoDB server port, default is |
|
MongoDB database name, default is |
|
reCAPTCHA v2 site key |
|
reCAPTCHA v2 secret key |
Upgrade
The upgrade procedures are handled by the application itself.
Debian Package Upgrade
If you installed Agate via the Debian package, you may update it using the command:
apt-get install agate
RPM Package Upgrade
If you installed Agate via the RPM package, you may update it using the command:
yum install agate
Zip Distribution Upgrade
Follow the Installation of Agate Zip distribution above but make sure you don’t overwrite your agate-home directory.
Execution
Server launch
Service
When Agate is installed through a Debian/RPM package, Agate server can be managed as a service.
Options for the Java Virtual Machine can be modified if Agate service needs more memory. To do this, modify the value of the environment variable JAVA_ARGS
in the file /etc/default/agate.
Main actions on Agate service are: start
, stop
, status
, restart
. For more information about available actions on Agate service, type:
service agate help
The Agate service log files are located in /var/log/agate directory.
Manually
The Agate server can be launched from the command line. The environment variable AGATE_HOME
needs to be setup before launching Agate manually.
Environment variable |
Required |
Description |
---|---|---|
|
yes |
Path to the Agate “home” directory. |
|
no |
Options for the Java Virtual Machine. For example: -Xmx4096m -XX:MaxPermSize=256m |
To change the defaults update: bin/agate
or bin/agate.bat
Make sure Command Environment is setup and execute the command line (bin directory is in your execution PATH)):
agate
Executing this command upgrades the Agate server and then launches it.
The Agate server log files are located in AGATE_HOME/logs directory. If the logs directory does not exist, it will be created by Agate.
Usage
To access Agate with a web browser the following urls may be used (port numbers may be different depending on HTTP Server Configuration):
http://localhost:8081 will provide a connection without encryption,
https://localhost:8444 will provide a connection secured with ssl.
Troubleshooting
If you encounter an issue during the installation and you can’t resolve it, please report it in our Agate Issue Tracker.
Agate logs can be found in /var/log/agate. If the installation fails, always refer to this log when reporting an error.
Configuration
The file AGATE_HOME/conf/application.yml is to be edited to match your server needs. This file is written in YAML format allowing to specify a hierarchy within the configuration keys. The YAML format uses indentations to express the different levels of this hierarchy. The file is already pre-filled with default values (to be modified to match your configuration), just be aware that you should not modify the indentations. In the following documentation, the configuration keys will be presented using the dot-notation (levels are separated by dots) for readability.
HTTP Server Configuration
Agate server is a web application and as such, you need to specify on which ports the web server should listen to incoming requests.
Property |
Description |
---|---|
|
HTTP port number. Generally speaking this port should not be exposed to the web. Use the https port instead. |
|
Web server host name. |
|
The URL’s context path, starting with a /. For instance when setting /auth, the base URL will be https://example.org/auth. |
|
HTTPS port number. |
MongoDB Server Configuration
Agate server will store its data (system configuration, networks, studies, datasets, etc.) in a MongoDB database. You must specify how to connect to this database.
Property |
Description |
---|---|
|
MongoDB URI. Read Standard Connection String Format to learn more. |
By default MongoDB does not require any user name, it is highly recommended to configure the database with a user. This can be done by enabling the Client Access Control procedure.
Follow these steps to enable the Client Access Control on your server:
create a user with the proper roles on the target databases
restart the MongoDB service with Client Access Control enabled
Note
Once the MongoDB service runs with Client Access Control enabled, all database connections require authentication.
MongoDB User Creation Example
The example below creates the agateadmin user for agate database:
use admin
db.createUser( {
user: "agateadmin", pwd: "agateadmin",
roles: [
{ "role" : "readWrite", "db" : "agate" },
{ "role" : "dbAdmin", "db" : "agate" },
{ "role" : "readAnyDatabase", "db": "admin" }
]
});
Here is the required configuration snippet in /etc/agate/application.yml for the above user:
spring:
data:
mongodb:
uri: mongodb://agateadmin:agateadmin@localhost:27017/agate?authSource=admin
Note
Agate requires either clusterMonitor or readAnyDatabase role on the admin database for validation operations. The first role is useful for a cluster setup and the latter if your MongoDB is on a single server.
reCAPTCHA Configuration
Agate uses reCAPTCHA service to protect the sign-up page from spam and abuse. See reCAPTCHA Guide to create a key pair. Note that only reCAPTCHA version 2 is supported.
Property |
Description |
---|---|
|
External service that verifies the reCAPTCHA key pair. Default is |
|
reCAPTCHA secret key, used to authorize the communication between Agate and the reCAPTCHA server. |
|
reCAPTCHA site key, used to invoke reCAPTCHA service on the application’s site. |
Cross Site Resource Forgery (CSRF)
CSRF attacks can be mitigated by a built-in interceptor. Default behavior allows connections (http or https) from localhost
and 127.0.0.1
. Requests from pages served by Opal should be allowed as well (https only), unless network settings or proxies modify or do not report the referer URL.
Property |
Description |
---|---|
|
Comma separated list of client |
User Directories
The security framework that is used by Agate for authentication, authorization etc. is Shiro. Configuring Shiro for Agate is done via the file AGATE_HOME/conf/shiro.ini. See also Shiro ini file documentation.
Note
Default configuration is a static user ‘administrator’ with password ‘password’ (or the one provided while installing Agate Debian/RPM package).
By default Agate server has several built-in user directories (in the world of Shiro, a user directory is called a realm):
a file-based user directory (shiro.ini file),
the internal user directory persisted in the MongoDB database.
Although it is possible to register some additional user directories, this practice is currently not recommended. It is also not recommended to use this file-based user directory for adding users. It is mainly dedicated to define a default system super-user. For a better security, user passwords are encrypted with a one way hash such as sha256. The example shiro.ini file below demonstrates how encryption is configured.
# =======================
# Shiro INI configuration
# =======================
[main]
# Objects and their properties are defined here,
# Such as the securityManager, Realms and anything else needed to build the SecurityManager
[users]
# The 'users' section is for simple deployments
# when you only need a small number of statically-defined set of User accounts.
#
# Password here must be encrypted!
# Use shiro-hasher tools to encrypt your passwords:
# DEBIAN:
# cd /usr/share/agate/tools && ./shiro-hasher -p
# UNIX:
# cd <AGATE_DIST_HOME>/tools && ./shiro-hasher -p
# WINDOWS:
# cd <AGATE_DIST_HOME>/tools && shiro-hasher.bat -p
#
# Format is:
# username=password[,role]*
administrator = $shiro1$SHA-256$500000$dxucP0IgyO99rdL0Ltj1Qg==$qssS60kTC7TqE61/JFrX/OEk0jsZbYXjiGhR7/t+XNY=,agate-administrator
[roles]
# The 'roles' section is for simple deployments
# when you only need a small number of statically-defined roles.
# Format is:
# role=permission[,permission]*
agate-administrator = *
Passwords must be encrypted using shiro-hasher tools (included in Agate tools directory):
cd /usr/share/agate/tools
./shiro-hasher -p
Notification Emails
Agate offers a notification emails service to the registered applications. Based on email templates, an application can request Agate to send emails to one or more of its users. Agate is using email templates for sending its notifications (email confirmation, reset password etc.).
Some templates are provided by default: see default templates directory. To override these default templates, the new templates are to be defined in the AGATE_HOME/conf/templates/notifications/ directory, using the same file names and directory structure.
The email templates specific to an application are located in the directory <templates folder>/notifications/<application name>.
The template engine used for building the email messages is FreeMarker. The default templates are in HTML format, but they could also be written in plain text.
Reverse Proxy Configuration
Agate server can be accessed through a reverse proxy server.
Apache
Example of Apache directives that:
redirects HTTP connection on port 80 to HTTPS connection on port 443,
specifies acceptable protocols and cipher suites,
refines organization’s specific certificate and private key.
<VirtualHost *:80>
ServerName agate.your-organization.org
ProxyRequests Off
ProxyPreserveHost On
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
RewriteEngine on
ReWriteCond %{SERVER_PORT} !^443$
RewriteRule ^/(.*) https://agate.your-organization.org:443/$1 [NC,R,L]
</VirtualHost>
<VirtualHost *:443>
ServerName agate.your-organization.org
SSLProxyEngine on
SSLEngine on
SSLProtocol All -SSLv2 -SSLv3
SSLHonorCipherOrder on
# Prefer PFS, allow TLS, avoid SSL, for IE8 on XP still allow 3DES
SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+AESG CM EECDH EDH+AESGCM EDH+aRSA HIGH !MEDIUM !LOW !aNULL !eNULL !LOW !RC4 !MD5 !EXP !PSK !SRP !DSS"
# Prevent CRIME/BREACH compression attacks
SSLCompression Off
SSLCertificateFile /etc/apache2/ssl/cert/your-organization.org.crt
SSLCertificateKeyFile /etc/apache2/ssl/private/your-organization.org.key
ProxyRequests Off
ProxyPreserveHost On
ProxyPass / https://localhost:8444/
ProxyPassReverse / https://localhost:8444/
</VirtualHost>
For performance, you can also activate Apache’s compression module (mod_deflate) with the following settings (note the json content type setting) in file /etc/apache2/mods-available/deflate.conf:
<IfModule mod_deflate.c>
<IfModule mod_filter.c>
# these are known to be safe with MSIE 6
AddOutputFilterByType DEFLATE text/html text/plain text/xml
# everything else may cause problems with MSIE 6
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/json
</IfModule>
</IfModule>
Recommended security headers are (to be added to the apache2.conf
file, requires headers
module):
# Security Headers, see https://securityheaders.com/
Header set Strict-Transport-Security "max-age=63072000"
Header set X-Frame-Options DENY
Header set X-XSS-Protection 1;mode=block
Header set X-Content-Type-Options nosniff
Header set Content-Security-Policy "frame-ancestors 'none'"
Header set Referrer-Policy "same-origin"
Header set Permissions-Policy "fullscreen=(self)"
Header onsuccess edit Set-Cookie ^(.+)$ "$1;HttpOnly;Secure;SameSite=Strict"
Public Pages Configuration
Starting from Agate 2.0, the administration user interface is distinct from the public pages, i.e. pages that are to be accessed by regular users. These pages are based on templates that can be customized, extended or overridden. The template engine that is used is FreeMarker which has a clean and powerful syntax.
Page Templates
Main Pages
The main public pages are:
Page |
Description |
---|---|
|
The home page |
|
The user profile page for updating personal information and password |
|
The login page |
|
The user registration page |
|
The user registration page, with form pre-filled with personal information extracted from a OpenID Connect server |
|
The page to confirm user’s registration (and validate email) and set the user password |
|
The page to ask for password reset |
|
The page to update the password after a reset was triggered |
|
The welcome page after a user has registered |
The templates structure is organized in a way that it should not be necessary to override these pages definitions. Instead of that, it is recommended to change/extend the theme/style as described in this guide.
Some template variables (date formats, branding, favicon etc.) are also defined in libs/settings.ftl and can be altered in the file models/settings.ftl that would be added in your configuration folder as follows:
AGATE_HOME
└── conf
└── templates
└── models
└── settings.ftl
General settings
Variable |
Description |
---|---|
|
The format in which the date-time values should be rendered. |
|
The format in which the date values should be rendered. |
|
The location of the favicon, to be modified to match your own. |
|
The location of your organization’s logo. |
|
CSS classes to apply to the logo. |
|
Logical to show/hide a text aside of the logo. |
|
CSS classes to apply to the text aside of the logo. |
|
The location of the AdminLTE theme if this one has been modified (see the Theme section in this documentation). |
Home page settings
Variable |
Description |
---|---|
|
The link applied to the logo. Default is the data portal (as specified in the Administration > General section), but it could also be the organization’s main portal. |
User Profile page settings
Variable |
Description |
---|---|
|
Logical to show/hide the role to which the user belongs. |
|
Logical to show/hide the groups to which the user belongs. |
|
Logical to show/hide the applications in which the user can sign. |
Adding Pages
It is possible to add new pages, for providing additional information or guidance to the regular user. This can be done as follows:
Install a new page templates
Add a new menu entry
1. Install custom page template
The new template page is to be declared in the configuration folder:
AGATE_HOME
└── conf
└── templates
└── custom.ftl
You can check at the provided templates to make your template fit in the site theme and structure. The profile page template could be a good starting point.
FreeMarker will look at its context to resolve variable values. For a custom page the objects available in the context are:
Object |
Description |
---|---|
|
The Agate configuration |
|
The user object (if user is logged in) |
|
The URL query parameters as a map of strings |
This custom template page can load any CSS or JS file that might be useful. These files can be served directly by adding them as follows (there are no restrictions regarding the naming and the structure of these files, as soon as they are located in the static folder):
AGATE_HOME
└── conf
└── static
├── custom.css
└── custom.js
The URL of this custom page will be for instance: https://agate.example.org/page/custom
.
2. Custom menu entry
To link to a custom page (or an external page), some templates can be defined to extend the default menus: left menu can be extended on its right and right menu can be extended on its left. The corresponding templates are:
AGATE_HOME
└── conf
└── templates
└── models
├── navbar-menus-left.ftl
└── navbar-menus-right.ftl
Check at the default left and right menus implementation as a reference.
Theme and Style
Theme
The default theme is the one provided by the excellent AdminLTE framework. It is based on Bootstrap and JQuery. In order to overwrite this default theme, the procedure is the following:
Build a custom AdminLTE distribution
Install this custom distribution
Change the template settings so that pages refer to this custom distribution instead of the default one
1. Build custom AdminLTE
This requires some knowledge in CSS development in a Node.js environment:
Download AdminLTE source (source code or a released version)
Reconfigure Sass variables
Rebuild AdminLTE (see instructions in the README file, contributions section)
2. Install custom AdminLTE
The objective is to have the web server to serve this new set of stylesheet and javascript files. This is achieved by creating the folder AGATE_HOME/conf/static and copying the AdminLTE custom distribution in that folder. Not all the AdminLTE are needed, only the dist and plugins ones. The folder tree will look like:
AGATE_HOME
└── conf
└── static
└── admin-lte
├── dist
└── plugins
3. Template settings
Now that the custom AdminLTE distribution is installed in the web server environment, this new location must be declared in the page templates. The default templates settings are defined in the libs/settings.ftl template file. See the adminLTEPath variable. This variable can be altered by defining a custom settings.ftl file as follows:
AGATE_HOME
└── conf
└── templates
└── models
└── settings.ftl
In this custom settings.ftl file the new AdminLTE distribution location will be declared:
<#assign adminLTEPath = "/admin-lte"/>
Style
As an alternative to theming, it is also possible to alter the style of the pages by loading your own stylesheet and tweaking the pages’ layout using javascript (and JQuery). The procedure is the following:
Install custom CSS and/or JS files
Custom the templates to include these new CSS and/or JS assets
1. Install custom CSS/JS
The objective is to have the web server to serve this new set of stylesheet and javascript files. This is achieved by creating the folder AGATE_HOME/conf/static and copying any CSS/JS files that will be included in the template pages. The folder tree will look like:
AGATE_HOME
└── conf
└── static
├── custom.css
└── custom.js
2. Custom templates
For the CSS files, the models/head.ftl template allows to extend the HTML pages “head” tag content with custom content. For the JS files, the models/scripts.ftl template allows to extend the HTML pages “script” tags. The folder tree will look like:
AGATE_HOME
└── conf
└── templates
└── models
├── head.ftl
└── scripts.ftl
Where the head.ftl template will be:
<link rel="stylesheet" href="/custom.css"/>
And the scripts.ftl template will be:
<script src="/custom.js"/>
Translations
The translations are performed in the following order, for a given locale
:
check for the message key in the messages_<locale>.properties (at different locations)
check for the message key in the <locale> JSON object as defined the Administration > Translations section of the administration interface
For the messages_* properties, the translations can be added/overridden as follows:
AGATE_HOME
└── conf
└── translations
├── notifications
│ ├── messages_fr.properties
│ └── messages_en.properties
├── messages_fr.properties
└── messages_en.properties
Note that the notification emails translations are located at a different place than the ones for the public pages. Note also that you can declare only the messages_* properties files that are relevant (language and public pages vs. notification emails) and the content of these files can contain only the translation keys that you want to override.
Web Introduction
The Agate Web Application is the administration web interface of the Agate server. It is NOT the end-user web portal and therefore firewall policies can (or should) be applied to restrict access to administrators or content editors.
See the Users, Groups and Applications presentation page for a detailed description of the type of documents that can be edited through this web interface.
The following manuals are available:
Users Management: add, edit users
Groups Management: add, edit groups
Applications Management: add, edit applications
Tickets Management: track user sessions
Administration: configure server settings
Requirements
This web interface is a javascript application requiring a modern web browser. There is no requirement regarding the operating system.
Users Management
The user pages are: the list of users page, the list of users requesting to join page, and user view and edit pages. See also User domain documentation.
Permissions
Users with the agate-administrator
role have access to these pages.
Operations
Add a user
Agate administrators can create users. The “General information” section contains system defined properties as well as configured attributes defined by the administrator. The “Access” section contains information related to the Role in agate, Groups and Applications for the user. Some user specific attributes can be defined too.
Edit a user
All the information for a user but his user name can be edited.
Delete a user
A user can be deleted.
Reset a user’s password
Click the reset password button to send the user, an email with details on how to reset his password.
Approve/Reject a user request
User requests can be approved or rejected. When a user sends a request, it is created with a status pending. If the request is rejected the user is removed, otherwise his status becomes approved.
Groups Management
Users can grouped in groups associated with a list of applications. Members of a group get access to the applications associated with it. See also Group domain documentation.
The group pages are: the list of groups page and group view and edit pages.
Permissions
Users with agate-administrator
role can access these pages.
Operations
Add group
Creates a group defined by a unique name.
Edit group
The description and the list of associated applications can be edited.
Delete group
A group can be deleted if there are no users associated with it.
Applications Management
An application is an external system that can use Agate as a central authentication system. Once an application is registered in agate, it can use its credentials (name and key) to connect with agate. See also Application domain documentation.
When Agate delegates the authentication to an external Open ID Connect Realm or when using OAuth2 service (see OAuth2 Introduction), the redirect URI must be set so that Agate performs the redirect to a known application after successful authentication. Wildcard *
can be used in this configured redirect URI.
The application pages are: the list of applications page and application view and edit pages.
Permissions
Users with agate-administrator
role can access these pages.
Operations
Add an application
Creates a new application that can access agate with the defined name and key. The application name has to be unique in agate.
Edit an application
Edits an application’s properties. The name can not be changed.
Delete an application
An application can be deleted only if there are no groups or users associated with it.
Tickets Management
Tickets are used to track the requests done on a specific user by the applications. A ticket is identified by a token which is an obscure identifier used by the applications internally.
Permissions
Users with agate-administrator
role have access to this page.
Operations
Delete ticket
A ticket can be deleted to clear the history of requests done on a specific user by the applications.
Realms Management
Authentication Delegation
Agate is able to delegate authentication to alternate identity provider systems. Note that even if the authentication happens in this third party application, the user still need to have a profile declared in Agate. The sign-up process extracts the user information, if some are available, to assist with the creation of this profile, but afterwards only the authentication service is used.
Realm Types
Open ID Connect Realm
A realm that uses the OpenID Connect (OIDC) protocol to authenticate users. OpenID Connect Flow explains the typical authentication flow when using this type of realm.
To register Agate as a client of the OIDC provider it will be necessary to provide its callback URL which is: https://agate.example.org/auth/callback/
.
Note
For Agate to authenticate for an Application, the redirect URI of the Application must be set (see Applications Management).
An example of well known open source ID provider that can be declared as an OIDC realm is Keycloak. Keycloak has also a strong user federation feature, which we recommend to use instead of using the following other realm types (LDAP etc.).
There following fields are required:
An ID provider must be identified by a Name,
The Agate application has been registered in the this provider: these are the Client ID and Client Secret fields.
The Discovery URI must follow the OpenID Connect configuration discovery specifications.
The optional fields are:
Title is a human-readable name that will be displayed in the provider’s signin button in the login page. If missing, the name of the ID provider will be used.
Groups are the group that are to be automatically applied to any users signing in through this ID provider.
Account Login address allows the user to go to it’s personal profile page in the ID provider interface (to chenge its password for instance) from the Opal login page.
Scope is the scope value(s) to be sent to the ID provider to initiate the OpenID Connect dialog. This is provider dependent but usually
openid
is enough.User Information Mapping specify which field values of the UserInfo object will be applied to the new Agate user.
Groups by Claim is an optional field name in the UserInfo object (that is returned by the ID provider) that contains the group names to which the user belongs. These will be automatically applied to the user’s profile. Such field is not one of the standard claims and needs to be explicitly set. The expected value type associated to this claim is either an array of strings, or a string which group names are separated by spaces (or commas).
Groups by JS is an optional Javascript code chunk that will process the UserInfo object to extract a group name or an array of group names to which the Agate user will belong.
Public URL is the public base URL of the server that will be used when sending notification emails and building an OpenID Connect callback URL.
Note that the groups mapping (by claim or JS) is executed at each sign in. Then if the user was associated to a new group in the OIDC provider, this group will be automatically applied to the corresponding Agate’s user as well. If the group does not exist yet, it will be created (without associated application). Removing an OIDC user from a group does not remove the Agate user from the group with same name.
LDAP Realm
A realm that authenticates users by using Lightweight Directory Access Protocol to query a Directory Access Agent. This realm uses a user’s Distinguished Name (DN) template to build queries.
Active Directory Realm
A realm tailored to a Microsoft Active Directory environment. This realm queries by using a combination of a search filter and search base.
SQL Database Realm
mysql
, mariadb
and postgresql
are supported.
This realm queries the user’s password with the salt style used by the database.
Salt styles include:
NO_SALT
: used when the password is in plain text.CRYPT
: uses the database’s underlying cryptographic method to decrypt the password.COLUMN
: the salt column must be the second column included in the query.EXTERNAL
: uses the specified algorithm to decrypt the password.
Administration
The Administration section is available to users with the role agate-administrator
. This menu gives access to server configuration and status.
Properties
The following general configuration properties can be modified:
Property |
Description |
---|---|
Name |
The name of the organization using this instance of Agate server. It will be used when sending notification emails. |
Public URL |
Public base URL of the server. It will be used when sending notification emails and in the OAuth2 settings. |
Portal URL |
The organization main portal, to go back to the main site form the Agate’s public pages. |
Domain |
The session cookie domain, required for single sign-on to operate. |
Short term timeout |
Ticket expiration timeout in hours. |
Long term timeout |
Ticket expiration timeout in hours when “remember me” option is selected. |
Inactive timeout |
User account expiration timeout in days. |
Sign up enabled |
Whether a user can self register from Agate public pages. This does not prevent from Mica to expose the sign-up feature. |
Sign up form offers to choose the username |
User name will be extracted from user email. |
Encryption Keys
This section presents the tool related to the encryption through HTTPS of transactions between Agate and its clients by means of a trusted or a self-signed certificate.
Note
In the instruction below, when you are told to cut and paste the content of the certificate, private key or of an .pem file, make sure that you copy all content, that is including the lines containing -----BEGIN XXXXXXXX-----
and -----END XXXXXXXX-----
.
Create a (self-signed) certificate
Useful when in testing phase, not recommended in production.
Click on the Add Keys drop-down.
Select Create.
Fill in the form and click on Save.
Click on the Download Certificate button under the section title Encryption Keys.
Your certificate (.pem file) should automatically be downloaded on your computer.
Import a certificate
It is recommended to use a valid key pair in production.
Click on the Add Keys drop-down
Select Import. Here you may use (1) certificate and (2) private key that you created using third party software e.g., OpenSSL. Note that both the certificate and the private key must be in PEM format.
Save.
Finally, in order for the changes to be taken in account you need to restart Agate server.
User Attributes
Additional user attributes can be declared. They will appear in the user form (including sign-up).
Python Introduction
Agate Python client, a command line scripting tool written in Python, enables automation of tasks in a Agate server.
Requirements
Python 3.7+ must be installed on the system. See more about Python.
Installation
The Agate Python Client is available on the official Python Package Index.
sudo pip install obiba-agate
Note
Previous versions were available as system packages. Make sure to remove them before installing the package with pip
.
# on Debian systems
sudo apt-get remove agate-python-client
# on RPM systems
sudo yum remove agate-python-client
Note
This python package depends on the pycurl package which has some system dependencies. One simple solution
is to install the pycurl
system package before.
# on Debian systems
sudo apt-get install python3-pycurl
# on RPM systems
sudo yum install python3-pycurl
Usage
To get the options of the command line:
agate --help
This command will display which sub-commands are available. Further, given a subcommand obtained from command above, its help message can be displayed via:
agate <subcommand> --help
This command will display available subcommands.
User Commands
User management commands.
Add User
Add a new user.
agate add-user <CREDENTIALS> [OPTIONS] [EXTRA]
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
The user name, required and unique. |
|
The user email, required and unique. |
|
The user password, required if realm is not specified or if it is Agate’s one. |
|
The realm in which the user will authenticate, optional (default is Agate’s realm). |
|
The user first name. |
|
The user last name. |
|
The applications in which the user can sign-in, space separated. |
|
The groups to which the user belongs, space separated. |
|
The role of the user. Default is “agate-user”, which gives only the right to user to access to its own profile. Other possible value is “agate-administrator”. |
|
Only active users can sign-in. Default value is “ACTIVE”. Other possible values are: “PENDING”, “APPROVED” or “INACTIVE”. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Add a new user.
agate add-user -ag http://localhost:8081 -u administrator -p password --name user1 --email user1@example.org --upassword CHANGEME --applications mica
Add a new user from a Keycloak’s server that is registered as as an OpenID Connect realm.
agate add-user -ag http://localhost:8081 -u administrator -p password --name user1 --email user1@example.org --realm keycloak --groups mica-user --groups opal-user
Delete User
Delete a user.
agate delete-user <CREDENTIALS> [OPTIONS] [EXTRA]
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
The user name, mutually exclusive with email. |
|
The user email, mutually exclusive with name. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Delete a user by its name.
agate delete-user -ag http://localhost:8081 -u administrator -p password --name user1
Group Commands
Group management commands.
Add Group
Add a new group.
agate add-group <CREDENTIALS> [OPTIONS] [EXTRA]
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
The group name, required and unique. |
|
The description of the group, optional. |
|
The applications in which the users members of the group can sign-in, space separated. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Add a new group.
agate add-group -ag http://localhost:8081 -u administrator -p password --name researchers --applications mica
Delete Group
Delete a group.
agate delete-group <CREDENTIALS> [OPTIONS] [EXTRA]
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
The group name, required. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Delete a group.
agate delete-group -ag http://localhost:8081 -u administrator -p password --name researchers
Application Commands
Application management commands.
Add Application
Add a new application.
agate add-application <CREDENTIALS> [OPTIONS] [EXTRA]
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
The application name, required and unique. |
|
The description of the application, optional. |
|
The application key, required. |
|
Callback URL to the application’s server, required in the OAuth context. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Add a new application.
agate add-application -ag http://localhost:8081 -u administrator -p password --name someapp --key ABCDEFGH1234
Delete Application
Delete an application.
agate delete-application <CREDENTIALS> [OPTIONS] [EXTRA]
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
The application name, required. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Delete an application.
agate delete-application -ag http://localhost:8081 -u administrator -p password --name someapp
Other Commands
Other commands for advanced users.
Web Services
This command is for advanced users wanting to directly access to the REST API of Agate server.
agate rest ws <CREDENTIALS> [OPTIONS] [EXTRA]
Arguments
Argument |
Description |
---|---|
|
Web service path, for instance: /user/xxx |
Credentials
Authentication is done by username/password credentials.
Option |
Description |
---|---|
|
Agate server base url. |
|
User name. User with appropriate permissions is expected depending of the REST resource requested. |
|
User password. |
Options
Option |
Description |
---|---|
|
HTTP method: GET (default), POST, PUT, DELETE, OPTIONS. |
|
Accept header (default is application/json). |
|
Content-Type header (default is application/json). |
|
Pretty JSON formatting of the response. |
Extras
Option |
Description |
---|---|
|
Show the command help’s message |
|
Verbose output |
Example
Get all users.
agate rest -ag http://localhost:8081 -u administrator -p password -j /users
OAuth2 Introduction
Agate exposes web services that implements the OAuth2 protocol. OAuth2 is an open authorization protocol which enables applications to access each others data. The authorization refers to the fact that these data are accessed on behalf of a resource owner.
For more details, the OAuth2 specifications are available at RFC6749. See also the OpenID Connect specifications built on top of OAuth2.
Roles
The OAuth2 protocol defines several roles:
the resource owner is the person or application that owns the data that is to be shared. In our case a user on Agate could be a resource owner. The resource they own is their data. The resource owner is depicted in the diagram as a person, which is probably the most common situation. The resource owner could also be an application.
the resource server is the server hosting the resources. For instance, Opal or Mica are resource servers.
the client application is the application requesting access to the resources stored on the resource server. These resources are owned by the resource owner. A client application could be an X-ray images analyser that extracts the image data from Opal.
the authorization server is the server authorizing the client application to access the resources of the resource owner: this is the role of Agate. The authorization server and the resource server can be the same server, but it doesn’t have to. When Agate is also the resource server, the resource that is accessed is the user profile.

Client ID, Client Secret and Redirect URI
Before a client application can request access to resources on a resource server, the client application must first register with the authorization server associated with the resource server. In Agate this is done by adding the client as a new Application.
The registration is typically a one-time task. Once registered, the registration remains valid, unless the client application registration is revoked by the Agate’s administrator.
At registration the client application is assigned a client ID and a client secret (password) by the authorization server. The client ID and secret is unique to the client application on that authorization server. In terms of Agate’s domain, the client ID is the application’s name and the client secret is the application’s key.
During the registration the client needs to provide a redirect URI. This redirect URI is used when a resource owner grants authorization to the client application. When a resource owner has successfully authorized the client application via the authorization server, the resource owner is redirected back to the client application, to the redirect URI.
Scopes
The scopes are space-separated the application IDs, optionally qualified by a permission. As an example, if an application registered in Agate with ID foo declares the read permission (= the permission to access to the resource granted by foo is read-only), then the authorization scope will be foo:read. If no action is specified, Agate will assume that foo grants full access to the resource. The permissions are specific to the application and it is the responsibility of the resource server to handle them as announced.
Flows
Authorization Code Grant Flow: when a client application wants access to the resources of a resource owner, hosted on a resource server, the client application must first obtain an authorization code grant from the authorization server (Agate).
Resource Owner Password Credentials Grant Flow: suitable for clients capable of obtaining the resource owner’s credentials (username and password),
OpenID Connect Flow: when client wants to get the user information from Agate (authorization and resource are the same).
Resource Owner Password Credentials Grant Flow
Summary
The resource owner password credentials grant is suitable for clients capable of obtaining the resource owner’s credentials (username and password, typically using an interactive form). This implies that the resource owner has a trust relationship with the client application, such as the device operating system or a highly privileged application.
Agate’s implementation of this flow is very limited. The access token obtained with this flow does not provide authorization to access the resource applications. This flow’s main use case is to authenticate the resource owner.
Access Token Issuing
Request
The REST end point to be used is:
POST https://agate.example.org/ws/oauth2/token
The form parameters to be sent within the body of the request are:
Parameter |
Description |
---|---|
|
Client application name (required). |
|
Client application secret key (required). |
|
The expected value is: password (required). |
|
The user name. |
|
The user password. |
Response
The response is a JSON object with the following properties:
Property |
Description |
---|---|
|
The access token. Agate provides signed tokens that implement the JSON Web Token specification. |
|
What you can do with this token; in the case of Agate the value for this property is bearer. |
|
Information about the expiration time (in seconds) before the token expires. |
An example of response would be:
{
"access_token": "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJlZGl0b3IiLCJpc3MiOiJhZ2F0ZTo1NmZjMzg0MmNjZjJjMWM3ZWM1YzVkMTQiLCJpYXQiOjE0NTk0NTg0NTgsImV4cCI6MTQ1OTQ4NzI1OCwianRpIjoiNTZmZDkxOWFjY2YyYzFjN2VjNWM1ZDE2IiwiYXVkIjpbIm1pY2EiLCJ0b3RvIl0sImNvbnRleHQiOnsic2NvcGVzIjpbIm1pY2EiXSwidXNlciI6eyJuYW1lIjoiSnVsaWUiLCJncm91cHMiOlsibWljYS1lZGl0b3IiXSwiZmlyc3RfbmFtZSI6Ikp1bGllIn19fQ.PqlLSZegdPLM2byp0jsgWV-XM3Xed8DP4I03kbUUEeo",
"token_type": "bearer",
"expires_in": 28799
}
Being a JSON Web Token (JWT), the access token can be decoded. There are three parts in a JWT: the header, the payload and the signature. This could give for example:
{
"alg": "HS256"
}
.
{
"sub": "editor",
"iss": "agate:56fc3842ccf2c1c7ec5c5d14",
"iat": 1459458458,
"exp": 1459487258,
"jti": "56fd919accf2c1c7ec5c5d16",
"aud": [
"client_app"
],
"context": {
"user": {
"name": "Julie LaTendresse",
"groups": [
"mica-editor"
],
"first_name": "Julie",
"last_name": "Latendresse"
}
}
}
.
[signature]
The JWT payload contains some basic details on the subject (in addition to the standard claims). These are available in the context object (which is a claim specific to Agate). The properties of the context are:
Property |
Description |
---|---|
|
The user full name for display. |
|
The user first name (if any). |
|
The user last name (if any). |
|
The user groups. |
Note that this step can be repeated as many times as necessary, using the same authorization code that was granted at step 1.
Errors
When an error is encountered during this step, the JSON object returned contains the description of the error, for example:
{
"error_description":"Authorization with code '3b1d664fb09407972d4c212081789c6f' does not exist",
"error":"NoSuchAuthorizationException"
}
OpenID Connect Flow
Summary
OpenID connect is an extension on top of OAuth2, so the authorization and token endpoints are the same as described in OAuth2 Introduction. Currently the OpenID Connect implementation in Agate only supports the authorization code flow.
Agate implements the OpenID Connect configuration discovery specification (scopes, endpoints, algos etc.). The discovery request would look like:
GET https://agate.example.org/.well-known/openid-configuration
Step 2. ID Token Issuing
This second step is similar to the access token issuing. When the authorization includes the openid scope, the response will contain an additional id_token in JWT format. An example of the reponse is:
{
"access_token": "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2VyMSIsImlzcyI6ImFnYXRlOjU3MDUyMjA2ZTRiMGRlNDNlYzE5NzM2YiIsImlhdCI6MTQ2MDA0MTU4NCwiZXhwIjoxNDYwMDcwMzg0LCJqdGkiOiI1NzA2Nzc3MGU0YjBmZjM3ODJkYmQ2MjIiLCJhdWQiOlsiZHJ1cGFsIl0sImNvbnRleHQiOnsic2NvcGVzIjpbIm9wZW5pZCJdLCJ1c2VyIjp7Im5hbWUiOiJKb2hubnkgQi4gR29vZCIsImxhc3RfbmFtZSI6Ikdvb2QiLCJncm91cHMiOlsibWljYS11c2VyIl0sImZpcnN0X25hbWUiOiJKb2hubnkgQi4ifX19.7SblBktnvXaoBFL61Rx_jb6PXXYPr4TFMlyi4ZYP5xE",
"id_token": "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2VyMSIsImlzcyI6ImFnYXRlOjU3MDUyMjA2ZTRiMGRlNDNlYzE5NzM2YiIsImdpdmVuX25hbWUiOiJKb2hubnkgQi4iLCJmYW1pbHlfbmFtZSI6Ikdvb2QiLCJuYW1lIjoiSm9obm55IEIuIEdvb2QiLCJlbWFpbCI6ImpvaG55Lmdvb2RAZXhhbXBsZS5jb20iLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsImlhdCI6MTQ1OTk3Mzc1OCwiZXhwIjoxNDY3NzQ5NzU4LCJhdWQiOiJkcnVwYWwifQ.1IqjodUNGZ8pKnlxmjzR0XcDgs8Hnl-ufeFsSNH3qaA",
"expires_in": 28799,
"token_type": "bearer"
}
The id_token represents the following structure (when using scope=openid email profile):
{
"alg": "HS256"
}
.
{
"sub": "user1",
"iss": "agate:57052206e4b0de43ec19736b",
"given_name": "Johnny B.",
"family_name": "Good",
"name": "Johnny B. Good",
"email": "johny.good@example.com",
"email_verified": false,
"iat": 1459973758,
"exp": 1467749758,
"aud": "someapp"
}
.
[signature]
Step 3. ID Resource Access
In addition to the id_token included in the access token response, the user information can be retrieved from the UserInfo end point. This step is similar to the resource access one (the resource is then the user information).
An example of ID resource request is:
curl -X GET --header "Authorization: Bearer ACCESS_TOKEN" https://agate.example.org/ws/oauth2/userinfo
The response is in JSON format and contains the user profile claims. An example of a response is:
{
"family_name": "Good",
"sub": "user1",
"iss": "agate:57052206e4b0de43ec19736b",
"email_verified": false,
"given_name": "Johnny B.",
"email": "johny.good@example.com",
"name": "Johnny B. Good"
}
Partners and Funders
The development of this application was made possible thanks to the support of our partners and funders:
Support
Please visit OBiBa support page.