Welcome to Pacifica CLI’s documentation!

The Pacifica CLI tool provides a user command line interface to Pacifica Core services. The CLI allows users to upload and download data in Pacifica.

Installation

The Pacifica software is available through PyPi so creating a virtual environment to install is what is shown below. Please keep in mind compatibility with the Pacifica Core services.

Installation in Virtual Environment

These installation instructions are intended to work on both Windows, Linux, and Mac platforms. Please keep that in mind when following the instructions.

Please install the appropriate tested version of Python for maximum chance of success.

Linux and Mac Installation

mkdir ~/.virtualenvs
python -m virtualenv ~/.virtualenvs/pacifica
. ~/.virtualenvs/pacifica/bin/activate
pip install pacifica-cli

Windows Installation

This is done using PowerShell. Please do not use Batch Command.

mkdir "$Env:LOCALAPPDATA\virtualenvs"
python.exe -m virtualenv "$Env:LOCALAPPDATA\virtualenvs\pacifica"
& "$Env:LOCALAPPDATA\virtualenvs\pacifica\Scripts\activate.ps1"
pip install pacifica-cli

Configuration

The configure subcommand generates a local configuration file for the user. It will read the system configuration to preseed its defaults and asks the user to enter the values required. An example configuration is located here.

The system configuration is processed first and the two directories are, /etc/pacifica-cli/config.ini then PYTHON_PREFIX/pacifica-cli/config.ini. Which ever is found first the client uses that as the system default.

The user configuration is processed second, if found. The directory the client looks in by default is ~/.pacifica-cli/config.ini. The ~ translates to the users home directory on any platform.

System Metadata

The metadata is managed by a JSON configuration file referenced by an environment variable UPLOADER_CONFIG. By default the environment variable is set to uploader.json. However, it could be managed at a system level or changed on the command line by the --config option.

The contents of the metadata configuration file is complex and should be read from here. Please get your systems administrator to help create this file for you. An example to start from is here.

CLI Options

These are the command line options reference all the options are present here and have a description.

Global Options

These options are common to all commands and affects behavior of the cli.

--verbose This options requires an option which tells python logging what messages to print.
--config This specifies the system metadata configuration for use when uploading data.

Download Sub-Command Options

The download sub-command requires interfacing with the cartd service. The cart requires file metadata about what to download and there is several methods for getting that file metadata.

--destination Download the files to this folder (default to cwd).
--cloudevent Use the cloudevents file from the notifications service to download.
--transaction-id
 Setup a cart from this transaction ID.

Upload Sub-Command Options

There are two sets of options to the upload sub-command. The first comes from the missing values in the metadata configuration json file. The second are common for all uploads and will be documented further.

--follow-links This will follow symlinked directories as you walk a directory tree building the list of files to upload.
--nowait The uploader will not wait for the successful or failed ingest of the upload. Instead you will have to query for this information later.
--local-retry Upload an already generated bundle by the –local-save option. This option will honor the –local-save and –tar-in-tar options as well.
--local-save The uploader will save off a copy of the uploaded bundle of data. The path is defined as the argument to the option.
--local-compress
 Use a compression algorithm (gzip, bzip2) when creating the local save.
--tar-in-tar A second bundler will be running generating a TAR in a TAR for upload.
--dry-run Do just the query portion of the upload and print off what the metadata will be set to. This will not upload or generate a local save or retry.
--interactive Interact with the results of the query engine to manually select the values for each metadata entry requested.
--do-not-upload
 This forces the upload process to stop before uploading. This works well with the –local-save option.

Example Usage

There are many things to consider when packaging up yours or someone elses data. We will help by going over some senerios we have experienced in the past and their solutions.

Example Configuration Output

$ pacifica-cli configure
Generating New Configuration.

Endpoints are an HTTP URL that looks similar to a website but
are designed for an uploader to interact with.

What are the endpoint URLs for the following...

Upload URL (https://ingest.example.com/upload):
Upload Status URL (https://ingest.example.com/get_state):
Upload Policy URL (https://policy.example.com/uploader):
Download URL (https://cartd.example.com):
Download Policy URL (https://policy.example.com/status/transactions/by_id):

CA certificate bundle is the path to your certificate authority bundle.

Use this if you have a custom site SSL Certificate for your Site.

Valid values:
- True: verify the SSL server certificiate using system bundle
- False: do not verify the SSL server certificate (not recommended)
- a/path/to/a/cacert/bundle: custom path to the server certificate

CA Certificate Bundle (True):

There are three kinds of authentication types supported.

- clientssl - This is where you have an SSL client key and cert
- basic     - This is a username and password
- None      - Do not perform any authentication

Authentication Type (None): basic
Username (None): jdoe
Password (None): password

Example Usage

Download Examples

$ pacifica-cli download --destination down --transaction-id 1234

Upload Examples

$ pacifica-cli upload --interactive config.cfg

Instrument ID - Select an ID
=====================================

54 Nittany Liquid Probes - NMR PROBES: Nittany Liquid Probes
104 Transmogriscope - Nanoscale Transmogriscope
Select ID (54): 54

Proposal ID - Select an ID
=====================================

1234a - Pacifica Development (active no close)
1235 - Pacifica Development (no close or end)
1236e - Pacifica Development (expired closed and end)
1237 - Pacifica Development (expired closed no end)
1238 - Pacifica Development (pre-active)
Select ID (1234a): 1234a
Done 10240.
Waiting job to complete (1).
Done.
{
    "created": "2017-09-26 01:32:34",
    "exception": "",
    "job_id": 1,
    "state": "OK",
    "task": "ingest metadata",
    "task_percent": "100.00000",
    "updated": "2017-09-26 01:32:36"
}

CLI Python Module

Configure Module

Methods for configuring the client.

pacifica.cli.configure.configure_url_endpoints(global_ini)

Query and set the URL endpoints.

pacifica.cli.configure.configure_ca_bundle(global_ini)

Query for the ca bundle when using https.

pacifica.cli.configure.configure_auth(global_ini)

Query and set the authentication configuration.

Query Module

These are the query methods used for interactive query.

pacifica.cli.query.execute_pager(content)

Find the appropriate pager default is embedded python pager.

pacifica.cli.query.filter_results(md_update, query_obj, regex)

Filter the results of query_obj by regex and save result back into md_update.

pacifica.cli.query.find_leaf_node(md_update)

Find a leaf node that has all deps resolved.

pacifica.cli.query.format_query_results(md_update, query_obj)

Format the query results and return some data structures.

pacifica.cli.query.interactive_select_loop(md_update, query_obj, default_id)

While loop to ask users what they want to select.

pacifica.cli.query.paged_content(title, display_data, valid_ids)

Display the data yielding results.

pacifica.cli.query.parse_command(exe)

Walk the system path and return executable path.

pacifica.cli.query.query_main(md_update, args)

Query from the metadata configuration.

pacifica.cli.query.set_query_obj(dep_meta_ids, md_update, obj)

Return the query object or false.

pacifica.cli.query.set_results(md_update, query_obj, default_id, interactive=False)

Set results of the query and ask if interactive.

pacifica.cli.query.set_selected_id(selected_id, default_id, valid_ids)

Return the selected ID validating it first.

Methods Module

Upload Module

Utils Module

Utilities module for common methods.

pacifica.cli.utils.compressor_generator(compressor_type)

Return a compressor based on type, bzip2, gzip.

pacifica.cli.utils.system_config_path(config_file)

Return the system configuration path.

pacifica.cli.utils.user_config_path(config_file)

Return the global configuration path.

The main cli module describes the high level interface (the CLI).

Indices and tables