Global/Universal¶

• name: the name of the host. Must be unique for the cluster.
• namespace: path within the configuration store where Patroni will keep information about the cluster. Default value: “/service”
• scope: cluster name

Bootstrap configuration¶

• dcs: This section will be written into /<namespace>/<scope>/config of a given configuration store after initializing of new cluster. This is the global configuration for the cluster. If you want to change some parameters for all cluster nodes - just do it in DCS (or via Patroni API) and all nodes will apply this configuration.

  • loop_wait: the number of seconds the loop will sleep. Default value: 10

  • ttl: the TTL to acquire the leader lock. Think of it as the length of time before initiation of the automatic failover process. Default value: 30

  • maximum_lag_on_failover: the maximum bytes a follower may lag to be able to participate in leader election.

  • postgresql:

    • use_pg_rewind:whether or not to use pg_rewind
    • use_slots: whether or not to use replication_slots. Must be False for PostgreSQL 9.3. You should comment out max_replication_slots before it becomes ineligible for leader status.
    • recovery_conf: additional configuration settings written to recovery.conf when configuring follower.
    • parameters: list of configuration settings for Postgres. Many of these are required for replication to work.

• initdb: List options to be passed on to initdb.

  • - data-checksums: Must be enabled when pg_rewind is needed on 9.3.
  • - encoding: UTF8: default encoding for new databases.
  • - locale: UTF8: default locale for new databases.

• pg_hba: list of lines that you should add to pg_hba.conf.

  • - host all all 0.0.0.0/0 md5.
  • - host replication replicator 127.0.0.1/32 md5: A line like this is required for replication.

• users: Some additional users users which needs to be created after initializing new cluster

  • admin: the name of user

    • password: zalando:

    • options: list of options for CREATE USER statement

      • - createrole
      • - createdb

• post_init: An additional script that will be executed after initializing the cluster. The script receives a connection string URL (with the cluster superuser as a user name). The PGPASSFILE variable is set to the location of pgpass file.

Consul¶

• host: the host:port for the Consul endpoint.

Etcd¶

• host: the host:port for the etcd endpoint.

Exhibitor¶

• hosts: initial list of Exhibitor (ZooKeeper) nodes in format: ‘host1,host2,etc...’. This list updates automatically whenever the Exhibitor (ZooKeeper) cluster topology changes.
• poll_interval: how often the list of ZooKeeper and Exhibitor nodes should be updated from Exhibitor
• port: Exhibitor port.

PostgreSQL¶

• authentication:

  • superuser:

    • username: name for the superuser, set during initialization (initdb) and later used by Patroni to connect to the postgres.
    • password: password for the superuser, set during initialization (initdb).

  • replication:

    • username: replication username; the user will be created during initialization. Replicas will use this user to access master via streaming replication
    • password: replication password; the user will be created during initialization.

• callbacks: callback scripts to run on certain actions. Patroni will pass the action, role and cluster name. (See scripts/aws.py as an example of how to write them.)

  • on_reload: run this script when configuration reload is triggered.
  • on_restart: run this script when the cluster restarts.
  • on_role_change: run this script when the cluster is being promoted or demoted.
  • on_start: run this script when the cluster starts.
  • on_stop: run this script when the cluster stops.

• connect_address: IP address + port through which Postgres is accessible from other nodes and applications.

• create_replica_methods: an ordered list of the create methods for turning a Patroni node into a new replica. “basebackup” is the default method; other methods are assumed to refer to scripts, each of which is configured as its own config item.

• data_dir: The location of the Postgres data directory, either existing or to be initialized by Patroni.

• bin_dir: Path to PostgreSQL binaries. (pg_ctl, pg_rewind, pg_basebackup, postgres) The default value is an empty string meaning that PATH environment variable will be used to find the executables.

• listen: IP address + port that Postgres listens to; must be accessible from other nodes in the cluster, if you’re using streaming replication. Multiple comma-separated addresses are permitted, as long as the port component is appended after to the last one with a colon, i.e. listen: 127.0.0.1,127.0.0.2:5432. Patroni will use the first address from this list to establish local connections to the PostgreSQL node.

• pgpass: path to the .pgpass password file. Patroni creates this file before executing pg_basebackup, the post_init script and under some other circumstances. The location must be writable by Patroni.

• recovery_conf: additional configuration settings written to recovery.conf when configuring follower.

• custom_conf : path to an optional custom postgresql.conf file, that will be used in place of postgresql.base.conf. The file must exist on all cluster nodes, be readable by PostgreSQL and will be included from its location on the real postgresql.conf. Note that Patroni will not monitor this file for changes, nor backup it. However, its settings can still be overriden by Patroni’s own configuration facilities - see dynamic configuration for details.

• parameters: list of configuration settings for Postgres. Many of these are required for replication to work.

• pg_ctl_timeout: How long should pg_ctl wait when doing start, stop or restart. Default value is 60 seconds.

• use_pg_rewind: try to use pg_rewind on the former leader when it joins cluster as a replica.

• remove_data_directory_on_rewind_failure: If this option is enabled, Patroni will remove postgres data directory and recreate replica. Otherwise it will try to follow the new leader. Default value is false.

• replica_method for each create_replica_method other than basebackup, you would add a configuration section of the same name. At a minimum, this should include “command” with a full path to the actual script to be executed. Other configuration parameters will be passed along to the script in the form “parameter=value”.

REST API¶

• connect_address: IP address and port to access the REST API.

• listen: IP address and port that Patroni will listen to, to provide health-check information for HAProxy.

• Optional:

  • authentication:

    • username: Basic-auth username to protect unsafe REST API endpoints.
    • password: Basic-auth password to protect unsafe REST API endpoints.

  • certfile: Specifies the file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.

  • keyfile: Specifies the file with the secret key in the PEM format.

ZooKeeper¶

• hosts: list of ZooKeeper cluster members in format: [‘host1:port1’, ‘host2:port2’, ‘etc...’].

Global/Universal¶

• PATRONI_CONFIGURATION: it is possible to set the entire configuration for the Patroni via PATRONI_CONFIGURATION environment variable. In this case any other environment variables will not be considered!
• PATRONI_NAME: name of the node where the current instance of Patroni is running. Must be unique for the cluster.
• PATRONI_NAMESPACE: path within the configuration store where Patroni will keep information about the cluster. Default value: “/service”
• PATRONI_SCOPE: cluster name

Bootstrap configuration¶

It is possible to create new database users right after the successful initialization of a new cluster. This process is defined by the following variables:

• PATRONI_<username>_PASSWORD=’<password>’
• PATRONI_<username>_OPTIONS=’list,of,options’

Example: defining PATRONI_admin_PASSWORD=strongpasswd and PATRONI_admin_OPTIONS='createrole,createdb' will cause creation of the user admin with the password strongpasswd that is allowed to create other users and databases.

Consul¶

• PATRONI_CONSUL_HOST: the host:port for the Consul endpoint.

Etcd¶

• PATRONI_ETCD_HOST: the host:port for the etcd endpoint.

Exhibitor¶

• PATRONI_EXHIBITOR_HOSTS: initial list of Exhibitor (ZooKeeper) nodes in format: ‘host1,host2,etc...’. This list updates automatically whenever the Exhibitor (ZooKeeper) cluster topology changes.
• PATRONI_EXHIBITOR_PORT: Exhibitor port.

PostgreSQL¶

• PATRONI_POSTGRESQL_LISTEN: IP address + port that Postgres listens to. Multiple comma-separated addresses are permitted, as long as the port component is appended after to the last one with a colon, i.e. listen: 127.0.0.1,127.0.0.2:5432. Patroni will use the first address from this list to establish local connections to the PostgreSQL node.
• PATRONI_POSTGRESQL_CONNECT_ADDRESS: IP address + port through which Postgres is accessible from other nodes and applications.
• PATRONI_POSTGRESQL_DATA_DIR: The location of the Postgres data directory, either existing or to be initialized by Patroni.
• PATRONI_POSTGRESQL_BIN_DIR: Path to PostgreSQL binaries. (pg_ctl, pg_rewind, pg_basebackup, postgres) The default value is an empty string meaning that PATH environment variable will be used to find the executables.
• PATRONI_POSTGRESQL_PGPASS: path to the .pgpass password file. Patroni creates this file before executing pg_basebackup and under some other circumstances. The location must be writable by Patroni.
• PATRONI_REPLICATION_USERNAME: replication username; the user will be created during initialization. Replicas will use this user to access master via streaming replication
• PATRONI_REPLICATION_PASSWORD: replication password; the user will be created during initialization.
• PATRONI_SUPERUSER_USERNAME: name for the superuser, set during initialization (initdb) and later used by Patroni to connect to the postgres. Also this user is used by pg_rewind.
• PATRONI_SUPERUSER_PASSWORD: password for the superuser, set during initialization (initdb).

REST API¶

• PATRONI_RESTAPI_CONNECT_ADDRESS: IP address and port to access the REST API.
• PATRONI_RESTAPI_LISTEN: IP address and port that Patroni will listen to, to provide health-check information for HAProxy.
• PATRONI_RESTAPI_USERNAME: Basic-auth username to protect unsafe REST API endpoints.
• PATRONI_RESTAPI_PASSWORD: Basic-auth password to protect unsafe REST API endpoints.
• PATRONI_RESTAPI_CERTFILE: Specifies the file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.
• PATRONI_RESTAPI_KEYFILE: Specifies the file with the secret key in the PEM format.

ZooKeeper¶

• PATRONI_ZOOKEEPER_HOSTS: comma separated list of ZooKeeper cluster members: “‘host1:port1’,’host2:port2’,’etc...’”. It is important to quote every single entity!

REST API¶

We provide a REST API endpoint for working with dynamic configuration.

$ curl -s localhost:8008/config | jq .
{
  "ttl": 30,
  "loop_wait": 10,
  "retry_timeout": 10,
  "maximum_lag_on_failover": 1048576,
  "postgresql": {
    "use_slots": true,
    "use_pg_rewind": true,
    "parameters": {
      "hot_standby": "on",
      "wal_log_hints": "on",
      "wal_keep_segments": 8,
      "wal_level": "hot_standby",
      "max_wal_senders": 5,
      "max_replication_slots": 5,
      "max_connections": "100"
    }
  }
}

$ curl -s -XPATCH -d \
        '{"loop_wait":5,"ttl":20,"postgresql":{"parameters":{"max_connections":"101"}}}' \
        http://localhost:8008/config | jq .
{
  "ttl": 20,
  "loop_wait": 5,
  "maximum_lag_on_failover": 1048576,
  "retry_timeout": 10,
  "postgresql": {
    "use_slots": true,
    "use_pg_rewind": true,
    "parameters": {
      "hot_standby": "on",
      "wal_log_hints": "on",
      "wal_keep_segments": 8,
      "wal_level": "hot_standby",
      "max_wal_senders": 5,
      "max_replication_slots": 5,
      "max_connections": "101"
    }
  }
}

$ curl -s http://localhost:8008/patroni | jq .
{
  "pending_restart": true,
  "database_system_identifier": "6287881213849985952",
  "postmaster_start_time": "2016-06-13 13:13:05.211 CEST",
  "xlog": {
    "location": 2197818976
  },
  "patroni": {
    "scope": "batman",
    "version": "1.0"
  },
  "state": "running",
  "role": "master",
  "server_version": 90503
}

$ curl -s -XPATCH -d \
        '{"postgresql":{"parameters":{"max_connections":null}}}' \
        http://localhost:8008/config | jq .
{
  "ttl": 20,
  "loop_wait": 5,
  "retry_timeout": 10,
  "maximum_lag_on_failover": 1048576,
  "postgresql": {
    "use_slots": true,
    "use_pg_rewind": true,
    "parameters": {
      "hot_standby": "on",
      "unix_socket_directories": ".",
      "wal_keep_segments": 8,
      "wal_level": "hot_standby",
      "wal_log_hints": "on",
      "max_wal_senders": 5,
      "max_replication_slots": 5
    }
  }
}

$ curl -s -XPUT -d \
        '{"maximum_lag_on_failover":1048576,"retry_timeout":10,"postgresql":{"use_slots":true,"use_pg_rewind":true,"parameters":{"hot_standby":"on","wal_log_hints":"on","wal_keep_segments":8,"wal_level":"hot_standby","unix_socket_directories":".","max_wal_senders":5}},"loop_wait":3,"ttl":20}' \
        http://localhost:8008/config | jq .
{
  "ttl": 20,
  "maximum_lag_on_failover": 1048576,
  "retry_timeout": 10,
  "postgresql": {
    "use_slots": true,
    "parameters": {
      "hot_standby": "on",
      "unix_socket_directories": ".",
      "wal_keep_segments": 8,
      "wal_level": "hot_standby",
      "wal_log_hints": "on",
      "max_wal_senders": 5
    },
    "use_pg_rewind": true
  },
  "loop_wait": 3
}

The goal¶

Under certain circumstances Patroni needs to temporary step down from managing the cluster, while still retaining the cluster state in DCS. Possible use cases are uncommon activities on the cluster, such as major version upgrades or corruption recovery. During those activities nodes are often started and stopped for the reason unknown to Patroni, some nodes can be even temporary promoted, violating the assumption of running only one master. Therefore, Patroni needs to be able to “detach” from the running cluster, implementing an equivalent of the maintenance mode in Pacemaker.

The implementation¶

When Patroni runs in a paused mode, it does not change the state of PostgreSQL, except for the following cases:

• For each node, the member key in DCS is updated with the current information about the cluster. This causes Patroni to run read-only queries on a member node if the member is running.
• For the Postgres master with the leader lock Patroni updates the lock. If the node with the leader lock stops being the master (i.e. is demoted manually), Patroni will release the lock instead of promoting the node back.
• Manual unscheduled restart, reinitialize and manual failover are allowed. Manual failover is only allowed if the node to failover to is specified. In the paused mode, manual failover does not require a running master node.
• If ‘parallel’ masters are detected by Patroni, it emits a warning, but does not demote the masters without the leader lock.
• If there is no leader lock in the cluster, the running master acquires the lock. If there is more than one master node, then the first master to acquire the lock wins. If there are no masters altogether, Patroni does not try to promote any replicas. There is an exception in this rule: if there is no leader lock because the old master has demoted itself due to the manual promotion, then only the candidate node mentioned in the promotion request may take the leader lock. When the new leader lock is granted (i.e. after promoting a replica manually), Patroni makes sure the replicas that were streaming from the previous leader will switch to the new one.
• When Postgres is stopped, Patroni does not try to start it. When Patroni is stopped, it does not to stop Postgres instance it is managing.

User guide¶

patronictl supports pause and resume commands.

One can also issue a PATCH request to the {namespace}/{cluster}/config key with {"pause": true/false/null}