Welcome to Augeias’s documentation!

Augeias is a small, RESTful, webapplication that allows you to store digital objects in an object store. While it allows you to communicate with you objects as a service, it also decouples the storage implementation from the interface.

For this purpose, Augeias, allows you to define multiple collections. Every collection can be served by a different storage module. So, one collection might store it’s objects on the filesystem while another one might store it’s objects in an Amazon S3 environment. The client that communicates with Augeias doesn’t need to know where the digit alobjects it’s receiving are stored.

Installation

Getting an installation of Augeias running is very simple. As always we recommend working with a separate virtualenvironment for this.

$ mkvirtualenv augeias_demo
$ pip install -U augeias

Once Augeias is installed, you can call upon a pyramid scaffold to generate the demo site.

$ pcreate -s augeias augeias_demo
$ cd augeias_demo

Just a few more steps and you’ve got a demo version running.

$ pip install -r requirements-dev.txt
$ python setup.py develop
$ pserve development.ini

The Augeias demo instance is now running on your localhost at port 6543. To reach it, open your browser and surf to the address http://localhost:6543.

This basic version comes with one configured collection default.

Services

The augeias provides services on Collections, Containers and Objects. You can think of them as a three-leveled hierarchy. At the top there are the collections. Each collection contains containers. And within each container are 1 or more objects.

GET /collections

Show a list of all collections. Every collection is a set of containers. They might have some sort of meaning as defined by the implementer. In practice quite often there will be a collection for every application.

Example request

GET /collections HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response

HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        'collection_key': 'default',
        'uri': 'https://storage.onroerenderfgoed.be/collections/default'
    } , {
        'collection_key': 'my_collection',
        'uri': 'https://storage.onroerenderfgoed.be/collections/my_collection'
    }
]
Request Headers

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Content-Type – This service currently always returns application/json

POST /collections/{collection_key}/containers

Create a new container. The server will generate a random container key.

Example request:

POST /collections/mine/containers HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 201 Created
Content-Type: application/json
Location: https://storage.onroerenderfgoed.be/collections/mine/containers/6ed5a007-41cf-49ed-8cb8-184fa5f48e42

{
    'container_key': '6ed5a007-41cf-49ed-8cb8-184fa5f48e42',
    'uri': 'https://storage.onroerenderfgoed.be/collections/mine/containers/6ed5a007-41cf-49ed-8cb8-184fa5f48e42'
}
Parameters

  • collection_key – Key for the collection within which the container will be placed.

Request Headers

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Location – The url where the newly added container can be found.

  • Content-Type – This service currently always returns application/json

Status Codes
PUT /collections/{collection_key}/containers/{container_key}

Create a new container. Allowsyou to choose your own container key. If the container already exists, nothing will happen.

Example request:

PUT /collections/mine/containers/abcd HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 201 Created
Content-Type: application/json
Location: https://storage.onroerenderfgoed.be/collections/mine/containers/abcd

{
    'container_key': 'abcd'
    'uri': 'https://storage.onroerenderfgoed.be/collections/mine/containers/abcd'
}
Parameters

  • collection_key – Key for the collection within which the container will be placed.

  • container_key – Key for the container that will be created.

Request Headers

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Location – The url where the newly added container can be found.

  • Content-Type – This service currently always returns application/json

Status Codes

  • 200 OK – The container already existed.

  • 201 Created – The container was added succesfully.

  • 404 Not Found – The collection collection_key does not exist.

DELETE /collections/{collection_key}/containers/{container_key}

Remove a container and all the objects in it.

Example request:

DELETE /collections/mine/containers/abcd HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{
    'container_key': 'abcd'
    'uri': 'https://storage.onroerenderfgoed.be/collections/mine/containers/abcd'
}
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container that will be deleted.

Request Headers

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Content-Type – This service currently always returns application/json

Status Codes

  • 200 OK – The container was deleted.

  • 404 Not Found – The collection collection_key does not exist or the container container_key does not exist within this collection.

GET /collections/{collection_key}/containers/{container_key}

Show all objects present in this container.

Example request:

GET /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06 HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

[
    "square",
    "small",
    "medium",
    "original",
    "large",
    "full"
]
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container that will be queried.

Request Headers

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Content-Type – This service currently always returns application/json

Status Codes

  • 200 OK – The container exists.

  • 404 Not Found – The collection collection_key or the container container_key does not exist.

HEAD /collections/{collection_key}/containers/{container_key}/{object_key}

Fetch metadata on a object without actually fetching the object.

Example request:

HEAD /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/full HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 200 OK
Date: Fri, 30 Oct 2015 07:11:44 GMT
Server: Apache/2.4.7 (Ubuntu)
Content-type: image/jpeg
Content-Length: 23562
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container where the object lives.

  • object_key – Key for the object that will be fetched

Status Codes

  • 200 OK – The object was found.

  • 404 Not Found – The collection collection_key or the container container_key or the object_key does not exist.

GET /collections/{collection_key}/containers/{container_key}/{object_key}

Fetch an object from a container.

Example request:

GET /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/full HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 200 OK
Date: Fri, 30 Oct 2015 07:11:44 GMT
Server: Apache/2.4.7 (Ubuntu)
Content-type: image/jpeg
Content-Length: 23562

<snipped>
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container where the object lives.

  • object_key – Key for the object that will be fetched

Status Codes

  • 200 OK – The object was found.

  • 404 Not Found – The collection collection_key or the container container_key or the object_key does not exist.

GET /collections/{collection_key}/containers/{container_key}/{object_key}/meta

Fetch object info (mimetype, size, time last modification).

Example request:

GET /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/full/meta HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 200 OK
Content-type: image/jpeg

{
        "time_last_modification": "2017-08-18T13:52:25.970242",
        "mime": "image/jpeg",
        "size": 11370
}
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container where the object lives.

  • object_key – Key for the object that will be fetched

Status Codes

  • 200 OK – The object was found.

  • 404 Not Found – The collection collection_key or the container container_key or the object_key does not exist.

POST /collections/{collection_key}/containers/{container_key}

Create a new object. The server will generate a random object key.

Example request:

POST /collections/mine/containers/mine_container HTTP/1.1
Host: augeias.onroerenderfgoed.be
Content-Type: application/octet-stream
Accept: application/json

Example response:

HTTP/1.1 201 Created
Content-Type: application/json
Location: https://storage.onroerenderfgoed.be/collections/mine/containers/mine_container/6ed5a007-41cf-49ed-8cb8-184fa5f48e42

{
    'container_key': 'mine_container',
    'object_key': '6ed5a007-41cf-49ed-8cb8-184fa5f48e42',
    'collection_key': 'mine'
    'uri': 'https://storage.onroerenderfgoed.be/collections/mine/containers/mine_container/6ed5a007-41cf-49ed-8cb8-184fa5f48e42'
}
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container where the object lives.

  • object_key – Key for the object that will be created or updated.

Request Headers

  • Content-Type – application/json or application/octet-stream

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Content-Type – This service currently always returns application/json

Status Codes

  • 201 Created – The object and the key were created.

  • 404 Not Found – The collection collection_key or the container container_key does not exist.

PUT /collections/{collection_key}/containers/{container_key}/{object_key}

Add or update an object in a container.

If an object with this key already exists, it will be overwritten. If not, it will be created.

Example request:

PUT /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/circle HTTP/1.1
Host: augeias.onroerenderfgoed.be
Content-Type: application/octet-stream
Accept: application/json

Exmaple response:

HTTP/1.1 200 OK
Content-Type: application/json

{
    'uri': 'https://id.erfgoed.net/storage/collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/circle',
    'object_key': 'circle',
    'container_key': 'a311efb7-f125-4d0a-aa26-69d3657a2d06',
    'collection_key': 'mine'
}
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container where the object lives.

  • object_key – Key for the object that will be created or updated.

Request Headers

  • Content-Type – application/json or application/octet-stream

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Content-Type – This service currently always returns application/json

Status Codes

  • 200 OK – The object was updated.

  • 201 Created – There was no object present with this key, it was created.

  • 404 Not Found – The collection collection_key or the container container_key does not exist.

PUT /collections/{collection_key}/containers/{container_key}/{object_key}

Copy an object from one store location into another within the same Augeias instance. The input json data must contain host url, collection_key, container_key and object_key of the object that needs to be copied.

If an object with this key already exists, it will be overwritten. If not, it will be created.

Example request:

PUT /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/circle HTTP/1.1
Host: augeias.onroerenderfgoed.be
Content-Type: application/json
Accept: application/json

{
  "host_url": "http://augeias.onroerenderfgoed.be",
  "collection_key": "temp",
  "container_key": "container_id",
  "object_key": "circletemp"
}

Exmaple response:

HTTP/1.1 200 OK
Content-Type: application/json

{
    "uri": "https://id.erfgoed.net/storage/collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/circle",
    "object_key": "circle",
    "container_key": "a311efb7-f125-4d0a-aa26-69d3657a2d06",
    "collection_key": "mine"
}
Parameters

  • collection_key – Key for the collection where the container lives.

  • container_key – Key for the container where the object lives.

  • object_key – Key for the object that will be created or updated.

Request Headers

  • Content-Type – application/json or application/octet-stream

  • Accept – The response content type depends on this header. Currently only application/json is supported.

Response Headers

  • Content-Type – This service currently always returns application/json

Status Codes

  • 200 OK – The object was copied.

  • 201 Created – There was no object present with this key, it was created.

  • 400 Bad Request – Validation failure. The input data of the object in the json body is not correct.

  • 404 Not Found – The collection collection_key or the container container_key does not exist.

DELETE /collections/{collection_key}/containers/{container_key}/{object_key}

Delete an object from a container.

Example request:

DELETE /collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/full HTTP/1.1
Host: augeias.onroerenderfgoed.be
Accept: application/json

Example response:

HTTP/1.1 200 OK
Content-Type: application/json

{
    "uri": "https://id.erfgoed.net/storage/collections/mine/containers/a311efb7-f125-4d0a-aa26-69d3657a2d06/circle",
    "object_key":"full",
    "container_key":"a311efb7-f125-4d0a-aa26-69d3657a2d06",
    "collection_key": "mine"
}
Status Codes

  • 200 OK – The object was deleted.

  • 404 Not Found – The collection collection_key or the container container_key or the object_key does not exist.

API Documentation

Collections

Contains models that deal with collections.

class augeias.collections.model.Collection(name, object_store, **kwargs)[source]

A collection is a set of containers that can have separate logic and configuration applied to it.

Currently a collection allows making two chief distinctions:

  • Where data whilebe stored, by setting the object_store

  • How URI’s for the collection and it’s contents will be generated by setting a uri_generator

Stores

Errors

This module contains exceptions that can be generated by stores.

exception augeias.stores.error.NotFoundException(value='Nothing matching the key value was found in the store')[source]

Indicates that an object or container could not be found.

Check the error message for more detailed information.

Store Interface

This module defines the interface every store needs to adhere to.

class augeias.stores.StoreInterface.IStore[source]

This interface handles object-storage. Implementations of this interface can be made for different object-storages Currently this interface is only implemented for PairTreeFileSystemStore

abstract create_container(container_key)[source]

Create a new container in the data store.

Parameters

container_key (str) – Key of the container to create.

abstract create_object(container_key, object_key, object_data)[source]

Save a new object in the data store

Parameters

  • container_key (str) – Key of the container to create an object in.

  • object_key (str) – Key of the object to create.

  • object_data (str) – The data for the object to create.

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

abstract delete_container(container_key)[source]

Delete a container and all it’s objects in the data store.

Parameters

container_key (str) – Key of the container to delete.

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

abstract delete_object(container_key, object_key)[source]

Delete an object from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to delete.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

abstract get_container_data(container_key, translations=None)[source]

Find a container and return a zip file of its contents.

Parameters

  • container_key – Key of the container which must be retrieved.

  • translations – Dict of object IDs and file names to use for them.

Returns

a zip file containing all files of the container.

abstract get_object(container_key, object_key)[source]

Retrieve an object from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to retrieve.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

abstract get_object_info(container_key, object_key)[source]

Retrieve object info (mimetype, size, time last modification) from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to retrieve.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

abstract list_object_keys_for_container(container_key)[source]

List all object keys for a container in the data store.

Parameters

container_key (str) – Key of the container to list the objects for.

Returns

A list of container keys.

Return type

lst

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

abstract update_object(container_key, object_key, object_data)[source]

Update an object in the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to update.

  • object_data (str) – New data for the object.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

Implementations

This module provide a simple filesystem based store

class augeias.stores.PairTreeFileSystemStore.PairTreeFileSystemStore(store_dir, uri_base='urn:x-vioe:')[source]

Provides a filesystem based store.

Will store your digital objects on disk using a PairTree.

create_container(container_key)[source]

Create a new container in the data store.

Parameters

container_key (str) – Key of the container to create.

create_object(container_key, object_key, object_data)[source]

Save a new object in the data store

Parameters

  • container_key (str) – Key of the container to create an object in.

  • object_key (str) – Key of the object to create.

  • object_data (str) – The data for the object to create.

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

delete_container(container_key)[source]

Delete a container and all it’s objects in the data store.

Parameters

container_key (str) – Key of the container to delete.

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

delete_object(container_key, object_key)[source]

Delete an object from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to delete.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

get_container_data(container_key, translations=None)[source]

Find a container and return a zip file of its contents.

Parameters

  • container_key – Key of the container which must be retrieved.

  • translations – Dict of object IDs and file names to use for them.

Returns

a zip file containing all files of the container.

get_object(container_key, object_key)[source]

Retrieve an object from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to retrieve.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

get_object_info(container_key, object_key)[source]

Retrieve object info (mimetype, size, time last modification) from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to retrieve.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

list_object_keys_for_container(container_key)[source]

List all object keys for a container in the data store.

Parameters

container_key (str) – Key of the container to list the objects for.

Returns

A list of container keys.

Return type

lst

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

update_object(container_key, object_key, object_data)[source]

Update an object in the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to update.

  • object_data (str) – New data for the object.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

class augeias.stores.CephStore.CephStore[source]
create_container(container_key)[source]

Create a new container in the data store.

Parameters

container_key (str) – Key of the container to create.

create_object(container_key, object_key, object_data)[source]

Save a new object in the data store

Parameters

  • container_key (str) – Key of the container to create an object in.

  • object_key (str) – Key of the object to create.

  • object_data (str) – The data for the object to create.

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

delete_container(container_key)[source]

Delete a container and all it’s objects in the data store.

Parameters

container_key (str) – Key of the container to delete.

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

delete_object(container_key, object_key)[source]

Delete an object from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to delete.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

get_container_data(container_key, translations=None)[source]

Find a container and return a zip file of its contents.

Parameters

  • container_key – Key of the container which must be retrieved.

  • translations – Dict of object IDs and file names to use for them.

Returns

a zip file containing all files of the container.

get_object(container_key, object_key)[source]

Retrieve an object from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to retrieve.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

get_object_info(container_key, object_key)[source]

Retrieve object info (mimetype, size, time last modification) from the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to retrieve.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

list_object_keys_for_container(container_key)[source]

List all object keys for a container in the data store.

Parameters

container_key (str) – Key of the container to list the objects for.

Returns

A list of container keys.

Return type

lst

Raises

augeias.stores.error.NotFoundException – When the container could not be found.

update_object(container_key, object_key, object_data)[source]

Update an object in the data store.

Parameters

  • container_key (str) – Key of the container that the object lives in.

  • object_key (str) – Key of the object to update.

  • object_data (str) – New data for the object.

Raises

augeias.stores.error.NotFoundException – When the object or container could not be found.

Uri

This module provides utilities for working with URIS.

class augeias.uri.DefaultUriGenerator(pattern='https://storage.onroerenderfgoed.be/')[source]

Generate a URI specific to storageprovider.

Used for providers that do not implement a specific UriGenerator.

generate_collection_uri(collection)[source]

Generate a URI for collections based on parameters passed.

Parameters

collection – The collection.

Return type

string

generate_container_uri(collection, container)[source]

Generate a URI for collection and container based on parameters passed.

Parameters

  • collection – The collection.

  • container – The container.

Return type

string

generate_object_uri(collection, container, object)[source]

Generate a URI for collections, container and object based on parameters passed.

Parameters

  • collection – The collection.

  • container – The container.

  • object – The object.

Return type

string

class augeias.uri.UriGenerator[source]

An abstract class for generating URIs.

abstract generate_collection_uri(collection)[source]

Generate a URI for a collection based on parameters passed.

abstract generate_container_uri(collection, container)[source]

Generate a URI for a container based on parameters passed.

abstract generate_object_uri(collection, container, object)[source]

Generate a URI for a object based on parameters passed.

class augeias.uri.UriPatternGenerator(pattern, separator='/')[source]

Generate a URI based on a simple pattern.

generate_collection_uri(collection)[source]

Generate a URI for a collection based on parameters passed.

Parameters

collection – The collection.

Return type

string

generate_container_uri(collection, container)[source]

Generate a URI for collection and container based on parameters passed.

Parameters

  • collection – The collection.

  • container – The container.

Return type

string

generate_object_uri(collection, container, object)[source]

Generate a URI for collections, container and object based on parameters passed.

Parameters

  • collection – The collection.

  • container – The container.

  • object – The object.

Return type

string

Views

exception augeias.views.ValidationFailure(msg)[source]

History

0.7.0 (22-12-2020)

  • Py3 migratie + library update (#90)

0.6.0 (09-06-2020)

  • Lijst met vertaling van id naar bestandsnaam meesturen download zip (#80)

0.5.0 (28-11-2018)

  • Download zip of entire container (#48)

  • Update list of supported python version (#51)

0.4.1 (26-10-2017)

  • pairtree update to 0.8.1

0.4.0 (24-10-2017)

  • Added python 3 compatibility

0.3.1 (12-10-2017)

  • Fix pairtree dependency==0.7.1-T

0.3.0 (22-08-2017)

  • Add ‘copy’ functionality to update_object endpoint. In this case the view accepts the (augeias) location of the file object (host url, collection_key, container_key and object_key) and copies it to a new location on the same augeias instance. (#17)

  • Add Endpoint to create object and key. (#20)

  • Add Endpoint to retrieve meta data (mimetype (#13), size, time last modification) from object.

0.2.0 (07-11-2015)

  • Add a scaffold for generating an Augeias instance. Use by running pcreate -s augeias. (#6)

  • Update docs with information about making a HEAD request for an object’s metdata. (#9)

  • Added permission hooks so it’s possible for an Augeias instance to add authentication. (#2)

  • Removed some OE specific configuration and made it more generic. (#8)

  • Added some installation instructions.

0.1.0 (20-10-2015)

  • Initial version

  • Has as single, filesystem based, store.

Glossary

URI

A Uniform Resource Identifier.

URN

A URN is a specific form of a URI.

Indices and tables