Juicer¶

Juicer is a command-line interface to the Pulp REST API which provides a shopping cart style approach to uploading and promoting groups of packages, files, or docker images through multiple environments.

Getting Started¶

Installation¶

Juicer was built to work with Pulp version 2.6.0. We assume that you have a server up and running which Juicer can talk to. Installation instructions for Pulp are available here.

RPM Install¶

RPM packages are available for RHEL/Centos 7 and Fedora 21 & 22.

dnf copr enable abutcher/juicer
dnf install -y juicer

Source Install¶

sudo python ./setup.py install

Configuration¶

Juicer is configured through a ~/.config/juicer/config file. The config is broken into sections by environment and may also contain an optional DEFAULT section, from which the defaults for all following sections are supplied.

The standard flow of this sample infrastructure goes from devel to prod; meaning that we upload our packages to devel and test them accordingly in our development environment before we promote them to prod.

[DEFAULT]
username: testuser
password: testpass
port: 443
verify_ssl: False
ca_path: /home/testuser/certs/pulp.crt
cert_filename: /home/testuser/certs/client.crt
cart_seeds: testhost.re.example.com:27017
start_in: re

[re]
hostname: testhost.re.example.com
promotes_to: qa

[qa]
hostname: testhost.qa.example.com

Usage¶

usage: juicer [-h] [-v] [-V] {cart,rpm,repo,role,user,hello} ...

manage pulp and release carts

optional arguments:
-h, --help            show this help message and exit
-v, --verbose         show verbose output
-V, --version         show program's version number and exit

commands:
'juicer COMMAND -h' for individual help topics

{cart,rpm,repo,role,user,hello}
cart                cart operations
rpm                 rpm operations
repo                repo operations
role                role operations
user                user operations
hello               test your connection to the pulp server

Create a repository¶

Creating a repository without specifying --in will automatically create the repository in every configured environment.

juicer repo create my-repository

Or, a repository can be created in specific environments.

juicer repo create my-repository --in devel

Note

Repositories created by juicer have a relative path which includes the environments they were created in. If a repository was created in devel, it would be available at https://<pulp-host>/pulp/repos/devel/.

The Pulp repo_id of a repository created by juicer will be display_name-environment. A repository named test-repo created in the devel environment would have a repo_id of test-repo-devel.

This was done so that multiple environments can co-exist on a single Pulp node.

Create a cart¶

A cart is composed of repositories and packages.

juicer cart create my-cart -r my-repository ~/rpmbuild/RPMS/noarch/*.rpm

Multiple packages and repositories can be specified.

juicer cart create my-cart -r my-repository ~/rpmbuild/RPMS/noarch/*.rpm \
                           -r my-other-repository ./awesome.rpm /tmp/woah.rpm

Packages don’t have to be local.

juicer cart create my-cart -r my-repository http://dang.com/rpms/omg.rpm

You can even provide an apache directory index (example: http://lnx.cx/~tbielawa/rpms/) as a source. The directory listing will be searched for links ending in .rpm. All matches will be added to the cart!

juicer cart create my-dir-cart -r my-repository http://son.com/rpms/
juicer cart show my-dir-cart

{
    "_id": "my-dir-cart",
    "repos_items": {
        "my-repository": [
            "http://son.com/rpms/megafrobber-1.0.3-2.noarch.rpm",
            "http://son.com/rpms/defrobnicate-ng-3.2.1-0.noarch.rpm",
        ]
    }
}

If you need to get more specific you can use fnmatch.fnmatch wildcard matching with apache directory indexes.

juicer cart create my-dir-cart -r my-repository http://son.com/rpms/mega*rpm
juicer cart show my-dir-cart

{
    "_id": "my-dir-cart",
    "repos_items": {
        "my-repository": [
            "http://son.com/rpms/megafrobber-1.0.3-2.noarch.rpm",
        ]
    }
}

Push a cart to an environment¶

Pushing a cart will upload all of its items to the specified environment.

juicer cart push my-cart --in qa

Note

A cart can be saved remotely once it has been pushed. This can be useful if the release engineer needs to swap mid-release. Add cart_seeds (insecure mongo endpoint) to juicer configuration to enable remote saves. Remote carts can be pulled with juicer cart pull.

To further illustrate remote cart saving, we can delete our local cart and pull it down again.

juicer cart delete my-cart --local
juicer cart pull my-cart
juicer cart show my-cart

juicer cart pull will overwrite a local cart file if it exists.

Plugins¶

Juicer will execute pre and post upload plugins if they exist. Example use cases include signing packages, kicking off builds, or anything that you’d want to do with a cart’s files before and after uploading.

Plugins are read from /usr/share/juicer/plugins/pre and /usr/share/juicer/plugins/post.
Plugins must be named after the class they contain.
Plugins classes will be initialized with item_type (rpm, docker, or iso), the environment, and a list of items that have already been synced to the local filesystem.
Plugin classes must contain a run member function in which all of the work will be done.

An Example Plugin¶

Here is an example plugin stored in /usr/share/juicer/plugins/pre/myplugin.py that displays each item and its size.

import os

class myplugin:
    def __init__(self, item_type, environment, items):
        self.item_type = item_type
        self.environment = environment
        self.items = items

        def run(self):
            print("Item type: {}".format(self.item_type))
            print("Environment: {}".format(self.environment))
            for item in self.items:
                print("File: {}, Size: {}".format(item, os.path.getsize(item.path)))

Running the Tests ¶

The Juicer test suite is invoked via the Makefile. The following is an example of how to run the ci target manually.

This will install dependencies within an isolated Python virtualenv. In addition to running our tests, PEP8 style formatting is also checked.

Once the command make ci exits and returns control to the shell we can scroll up a few lines and review the results of our unit tests and code-coverage report (move your cursor over the window above and scroll up/down to see for yourself).

Running Juicer Locally ¶

Once the ci Makefile target has been ran, we can enter the python virtual environment and run juicer by running the highlighted commands in the following block.

[~/juicer] 23:30:27  (master)
$ . juicerenv/bin/activate

[~/juicer] 23:30:41  (master)
$ juicer -h
usage: juicer [-h] [-q] [-v] [-V] {cart,rpm,repo,role,user,hello} ...

manage pulp and release carts

optional arguments:
  -h, --help            show this help message and exit
  -q, --quiet           show no output
  -v, --verbose         show verbose output
  -V, --version         show program's version number and exit

commands:
  'juicer COMMAND -h' for individual help topics

  {cart,rpm,repo,role,user,hello}
  cart                cart operations
  rpm                 rpm operations
  repo                repo operations
  role                role operations
  user                user operations
  hello               test your connection to the pulp server

Code Style and Formatting ¶

Please conform to PEP 0008 for code formatting. This specification outlines the style that is required for patches.

Your code must follow this (or note why it can’t) before patches will be accepted. There is one consistent exception to this rule:

E501

Line too long

The pep8 tests for juicer include a --ignore option to automatically exclude E501 errors from the tests.

Argument and Command Style ¶

Arguments should be expressed as they are in the following example. Usage strings are all lower case except for argument metavars which are in caps. Description strings are all lower case.

$ juicer cart create -h
usage: juicer cart create CARTNAME [-r REPONAME ITEM ... [-r REPONAME ITEM ...]] [-h]

positional arguments:
CARTNAME              cart name

optional arguments:
 -h, --help            show this help message and exit
 -r REPONAME [ITEM ...]
                       destination repo name, items

Output should read as a garden variety sentence.

$ juicer cart create test -r test-repo ~/rpmbuild/RPMS/noarch/*
Saved cart 'test'

Documentation ¶

I assume you came here to learn how to update the project documentation?

Hello, you have just become my new best friend. I think we’re going to get along really well together.

Section Headers ¶

When marking up section headers please refer to the HEADERS file in the docsite directory. This file shows the order we apply section header markup. Please follow it exactly, doing so will help us avoid silly rendering errors.

Start at the top and work your way down as you nest deeper and deeper

#    with overline, for parts
*    with overline, for chapters
=    for sections
-    for subsections
^    for subsubsections
"    for paragraphs
|    for anything smaller

Word Wrapping ¶

Please do word-wrap your documentation contributions! In emacs this is as simple as pressing M-q in a paragraph you want to auto-word-wrap (the emacs function is called fill-paragraph. You can run it manually with M-x fill-paragraph <RET> if you prefer.

If you use vi(m), then I’m sorry. I cannot assist you with your word-wrapping needs a this time. Please feel free to submit a pull request to update these docs with vi(m) automatic word-wrapping instructions!

Fear not – pull-requests won’t be rejected just because they aren’t word-wrapped. You just earn major karma with us if you word-wrap your contributions :-). Thanks!

Building The Docs ¶

So you want to build the documentation locally? Aren’t you in luck, I think that’s a surpurb idea as well. Building the docs is a fairly straight-forward process. All you may have to do is install some requirements first:

From yum:

python-sphinx
python-sphinx_rtd_theme

Optionally, you may install these requirements from pip:

Sphinx
sphinx_rtd_theme

Once you have the requirements installed you can attempt to build the documentation from source

Switch into the docsite directory and run make html:

$ cd ./docsite
$ make html
sphinx-build -b html -d build/doctrees   source build/html
Making output directory...
Running Sphinx v1.1.3
loading pickled environment... not yet created
building [html]: targets for 3 source files that are out of date
updating environment: 3 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded.

Build finished. The HTML pages are in build/html.

If the docs built correctly then you can open them in your default browser with this command (while still in the docsite directory):

$ xdg-open ./build/html/index.html