3. PulpDist Repository Management Client¶
The 1.x series of the upstream Pulp project does not provide a management client for repositories that use the preliminary support for the v2 repo plugin model. Accordingly, PulpDist comes with a command line interface for working with these repositories.
3.1. Invoking the Client¶
At this stage, there is no separately installed executable script to manage repositories. Instead, a feature of the CPython interpreter is used to invoke the appropriate module as a command line script:
$ python -m pulpdist.manage_repos --help
Before using the command line client to manage a Pulp server, it is necessary to create the login credentials for the Pulp server. PulpDist supports three mechanisms for that:
- Kerberos tickets (default, but requires a patched version of Pulp)
- Pulp’s native credential caching mechanism
The former just requires an active Kerberos ticket that will be recognised
by the server. The credentials for the latter can be acquired with the
upstream pulp-admin
client (entering the appropriate password when
prompted):
$ pulp-admin --host <HOST> auth login --username <USER>
To request use of these credentials (rather than a Kerberos ticket), pass
the --auth pulp
option to the client.
Like pulp-admin
the PulpDist repo management client defaults to using the
fully qualified domain name of the current host as the target server. This can
be overridden by passing a different hostname via the --host
option.
3.1.1. Synchronisation Management Commands¶
sync
: Request immediate synchronisation of repositoriesenable
: Configure repositories to respond to sync requestsdisable
: Configure repositories to ignore sync requestscron_sync
: See Scheduling sync operations with cron
3.1.2. Repository Status Queries¶
list
: Display id and name for repositoriesinfo
: Display details for repositoriesstatus
: Display repository synchronisation statushistory
: Display repository synchronisation historylog
: Display most recent synchronisation logstats
: Display most recent synchronisation statistics
3.1.3. Repository Management Commands¶
init
: Create or update repositories on the serverdelete
: Remove repositories from the servervalidate
: Check the validity of a repository definition fileexport
: Create a site definition file from an existing repository (Not Yet Implemented)
3.2. Limiting commands to selected repositories¶
The --repo
option accepts repository identifiers and allows a command
to run against the named repository. It may be supplied multiple times to
run a command against multiple repositories.
The --mirror
option accepts local mirror identifiers and allows a command
to run against the named local mirror. It may be supplied multiple times to
run a command against multiple repositories.
The --tree
option accepts remote tree identifiers and allows a
command to run against repositories that were configured from a site
configuration file to sync with a particular remote tree. It may be
supplied multiple times to run a command against mirrors of multiple trees.
The --source
option accepts remote source identifiers and allows a
command to run against repositories that were configured from a site
configuration file to sync with a tree from that remote source. It may be
supplied multiple times to run a command against repositories from multiple
sources.
The --server
option accepts remote server identifiers and allows a
command to run against repositories that were configured from a site
configuration file to sync with a tree from that remote server. It may be
supplied multiple times to run a command against repositories from multiple
servers.
The --site
option accepts site identifiers and allows a command to run
against repositories that were configured from a site configuration file
based on the specified site settings. It may be supplied multiple times to
run a command against multiple local “sites”. This option is only useful if
repositories are configured against more than one site on the specified Pulp
server.
If no specific repositories are identified, most commands default to affecting every repository defined on the server, or, if the command accepts a configuration file, every repository named in the file.
By default, the command line client uses the metadata stored on the server to
identify the available repositories. If this metadata is incomplete or invalid,
the --ignoremeta
option can be passed before the command to be executed. In
this mode, the Pulp server will be treated as containing only raw repo
definitions, allowing listing and manipulation of repos that would otherwise be
ignored (due to the fact they aren’t recorded in the stored metadata).
3.3. Scheduling sync operations¶
3.3.1. Scheduling sync operations with Pulp¶
Eventually, PulpDist will use the native Pulp task scheduler for sync operations. However, this is not yet supported by Pulp for plugin based repositories (such as those used by PulpDist).
3.3.2. Scheduling sync operations with cron¶
As the versions of Pulp currently supported by PulpDist do not provide native sync scheduling support for plugin based repositories), PulpDist offers a simple alternative mechanism based on cron (or any similar tool that can be used to periodically execute a Python script).
The relevant command is:
python -m pulpdist.manage_repos cron_sync
This tool is designed to be run once per hour (if a previous instance for
the same Pulp host is still running, the new instance will immediately exit).
For more immediate synchronisation, the sync
command should be invoked
directly.
The command first retrieves the list of repository definitions from the
Pulp server and queries each one for a ["notes"]["pulpdist"]["sync_hours"]
setting in the metadata.
If sync operations on the repository are currently enabled, the repository
does not already have a sync operation in progress, the sync_hours
setting
is found and is non-zero,and the current time (in hours) relative to midnight
is a multiple of the sync_hours
setting, then a new thread is spawned to
request immediate synchronisation of the repository through the Pulp REST API.
Otherwise, the repository is ignored until the next check for new sync operations.
As long as any sync operations are still in progress, the client will periodically query the server for updated information, scheduling sync operations as appropriate.
As soon as all sync operations are complete (regardless of success or failure), the client will terminate.
The following options can be set to control the sync operation:
--threads
: maximum number of concurrent sync operations (default: 4)--day
: rsync bandwidth limit to apply during the day (6 am - 6 pm)--night
: rsync bandwidth limit to apply at night (6 pm - 6 am)
By default, no bandwidth limits are applied.
Note
Support for bandwidth limiting is not yet implemented
3.4. The repository definition file format¶
The init
and validate
commands provided by manage_repos
both
require a repository definition file. The export
command generates a
respository definition file describing the server contents.
These are JSON files that specify the information needed to create the repositories on the Pulp server, and appropriately configure the associated importer plugins. See PulpDist Site Configuration for more details.
3.5. PulpDist metadata in Pulp¶
When PulpDist repositories are initialised from a site configuration file,
a pulpdist-meta
repo is automatically created to record the full contents
of the original site configuration. This information is stored in the “notes”
field for that repository.
Additional information is also recorded in the notes
field of each created
Pulp repo to support some features of the PulpDist command line client. This
additional metadata is stored in the format:
pulpdist
: Top-level mapping entry to identify pulpdist related metadatasync_hours
: The remote treesync_hours
setting (if any)site_id
: The site settings used to configure this repomirror_id
: The local mirror name for this repotree_id
: The remote tree mirrored by this reposource_id
: The remote source for this treeserver_id
: The remote server for this tree
The repo_id
of the associated Pulp repository is built from the
mirror_id
and site_id
of the local mirror definition.