XmlModels2

XmlModels allows you to define Models similar in nature to Django models that are backed by XML endpoints rather than a database. Using a familiar declarative definition, the fields map to values in the XML document by means of XPath expressions. With support for querying external REST APIs using a django-esque approach, we have strived to make writing and using xml backed models as close to django database models as we can, within the limitations of the available API calls.

User Documentation

Mapping XML to Models

XML is mapped to xml_models.Model via Fields. Each field requires an xpath expression that determines which node or attribute to get the data from. Each field also has an optional default value for when no value can be retrieved from the XML.

Basic Fields

TThe available field mappings are

  • CharField – returns string data
  • IntField – returns integers
  • DateField – returns a date from using the supplied date_format mask or the default ISO8601 format
  • FloatField – returns a floating point number
  • BoolField – returns a boolean
  • OneToOneField – returns a xml_model.Model subclass
  • CollectionField – returns a collection of either one of the above types, or an xml_model.Model subclass

Most of these fields are fairly self explanatory. The CollectionField and OneToOneField is where it gets interesting. This is what allows you to map instances or collections of nested entities, such as:-

<Person id="112">
  <firstName>Chris</firstName>
  <lastName>Tarttelin</lastName>
  <occupation>Code Geek</occupation>
  <website>http://www.pyruby.com</website>
  <contact-info>
    <contact type="telephone">
      <info>(555) 555-5555</info>
      <description>Cell phone, but no calls during work hours</description>
    </contact>
    <contact type="email">
      <info>me@here.net</info>
      <description>Where possible, contact me by email</description>
    </contact>
    <contact type="telephone">
      <info>1-800-555-5555</info>
      <description>Toll free work number for during office hours.</description>
    </contact>
  </contact-info>
</Person>

This can be mapped using a Person and a ContactInfo model:-

class Person(Model):
  id = IntField(xpath="/Person/@id")
  firstName = CharField(xpath="/Person/firstName")
  lastName = CharField(xpath="/Person/lastName")
  contacts = CollectionField(ContactInfo, order_by="contact_type", xpath="/Person/contact-info/contact")

class ContactInfo(Model):
  contact_type = CharField(xpath="/contact/@type")
  info = CharField(xpath="/contact/info")
  description = CharField(xpath="/contact/description", default="No description supplied")

This leads to the usage of a person as :-

>>> person.contacts[0].info
me@here.com

Collections

When querying collections or lists, it is assumed that a collection of zero or more results are returned wrapped in an enclosing collection tag.

As some REST APIs may return lists wrapped in one or more layers of metadata, Models may also define a collection_node attribute. this allows the XML processor to find the relevant node.

Note

collection_node is the tag name only and not an xpath expression.

For example, given the following XML

<reponse status="200">
  <metadata count="2">
  <collection>
    <model ... />
    <model ... />
  </collection>
  </metadata>
</response>

We would need to define a Model with a collection_node like so

class SomeModel(Model):
  fieldA = CharField(xpath="/some/node")

  collection_node = 'collection'

Querying Models

Querying is based on Django Model querying. Each Model has a class attribute objects which is the entry point for querying. However the querying ability is limited in scope and only supports very basic filtering.

To do anything interesting you will need to define finders on your models. There are no assumptions made about the nature of the REST API e.g. it is not even assumed that an id attribute can be queried.

Finders

An external REST api will present a limited number of options for querying data. Because the different options do not have to follow any specific convention, the model must define what finders are available and what parameters they accept. This still attempts to follow a Django-esque approach

class Person(xml_models.Model:
    ...
    finders = { (firstName, lastName): "http://person/firstName/%s/lastName/%s",
                (id,): "http://person/%s"}

The above defines two query options. The following code exercises these options

>>> people = Person.objects.filter(firstName='Chris', lastName='Tarttelin')
>>> people.count()
1
>>> person = Person.objects.get(id=123)
>>> person.firstName
Chris

Self-signed HTTPS Endpoints

Self-signed, or endpoints signed by a non-standard CA, are supported by setting xml_models.VERIFY as per the requests documentation for [SSL certificate verification](http://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification).

For example, to not verify a self-signed certificate you can use the following:

API Documentation

XmlModels is based on Django database models in terms of functionality and interface.

Manager

class xml_models.xml_models.ModelManager(model, finders)

Handles what can be queried for, and acts as the entry point for querying.

The API and usage is intended to be familiar to Django users however it does not support the complete Django ModelManager API.

count()

Get a count

Returns:int
filter(**kw)

Filter models by key-value pairs.

Example:
Model.objects.filter(attr1=value1,attr2=value2)

How the actual HTTP request is handled is determined by Finders.

Parameters:kw – key value pairs of field name and value
Returns:lazy query
filter_custom(url)

Set a URL to be called when querying

Parameters:url – full URL
Returns:lazy query
get(**kw)

Get a single object.

This can be called directly with key-value pairs or after setting some filters

Parameters:kw
Returns:

Model

class xml_models.xml_models.Model(xml=None, dom=None)

A model is a representation of the XML source, consisting of a number of Fields. It can be constructed with either an xml string, or an etree.Element.

Example:
class Person(xml_models.Model):
    namespace="urn:my.default.namespace"
    name = xml_models.CharField(xpath"/Person/@Name", default="John")
    nicknames = xml_models.CollectionField(CharField, xpath="/Person/Nicknames/Name")
    addresses = xml_models.CollectionField(Address, xpath="/Person/Addresses/Address")
    date_of_birth = xml_models.DateField(xpath="/Person/@DateOfBirth", date_format="%d-%m-%Y")

If you define Finders on your model you will also be able to retreive models from an API endpoint using a familiar Django-esque object manager style of access with chainable filtering etc.

to_tree()

etree.Element representation of Model

Return type:lxml.etree.Element
to_xml()

XML representation of Model

Return type:string
validate_on_load()

Perform validation when the model is instantiated.

Override on your model to perform validation when the XML data is first passed in.

Note

You will need to raise appropriate exceptions as no checking of the return value occurs

Fields

class xml_models.xml_models.BaseField(**kw)

Base class for Fields. Should not be used directly

class xml_models.xml_models.CharField(**kw)

Returns the single value found by the xpath expression, as a string

parse(xml, namespace)
Parameters:
  • xml – the etree.Element to search in
  • namespace – not used yet
Return type:

string

class xml_models.xml_models.IntField(**kw)

Returns the single value found by the xpath expression, as an int

parse(xml, namespace)
Parameters:
  • xml – the etree.Element to search in
  • namespace – not used yet
Return type:

DateTime, may be timezone aware or naive

class xml_models.xml_models.FloatField(**kw)

Returns the single value found by the xpath expression, as a float

parse(xml, namespace)
Parameters:
  • xml – the etree.Element to search in
  • namespace – not used yet
Return type:

float

class xml_models.xml_models.DateField(date_format=None, **kw)

Returns the single value found by the xpath expression, as a datetime.

By default, expects dates that match the ISO8601 date format. If a date_format keyword arg is supplied, that will be used instead. date_format should conform to strptime formatting options.

If the XML contains UTC offsets then a timezone aware datetime object will be returned.

parse(xml, namespace)
Parameters:
  • xml – the etree.Element to search in
  • namespace – not used yet
Return type:

DateTime, may be timezone aware or naive

class xml_models.xml_models.OneToOneField(field_type, **kw)

Returns a subclass of Model from the xpath expression.

parse(xml, namespace)
Parameters:
  • xml – the etree.Element to search in
  • namespace – not used yet
Return type:

as defined by self.field_type

class xml_models.xml_models.CollectionField(field_type, order_by=None, **kw)

Returns a collection found by the xpath expression.

Requires a field_type to be supplied, which can either be a field type, e.g. IntField, which returns a collection ints, or it can be a Model type e.g. Person may contain a collection of Address objects.

parse(xml, namespace)

Find all nodes matching the xpath expression and create objects from each the matched node.

If order_by has been defined then the resulting list will be ordered.

Parameters:
  • xml – the etree.Element to search in
  • namespace – not used yet
Return type:

as defined by self.field_type

Installation

The simplest approach is to to use pip install xml_models2

A simple example

Just to get started, this is an example of taking an XML representation of an Address that might be returned from a GET request to an external REST api.

<Address id="2">
  <number>22</number>
  <street>Acacia Avenue</street>
  <city>Maiden</city>
  <country>England</country>
  <postcode>IM6 66B</postcode>
</Address>

class Address(xml_models.Model):
  id=xml_models.IntField(xpath="/Address/@id")
  number = xml_models.IntField(xpath="/Address/number")
  street = xml_models.CharField(xpath="/Address/street")
  city = xml_models.CharField(xpath="/Address/city")
  country = xml_models.CharField(xpath="/Address/country")
  postcode = xml_models.CharField(xpath="/Address/postcode")

  finders = {(id,): 'http://adresses/%s'}

This example would be used as follows:-

>>> address = Address.objects.get(id=2)
>>> print "address is %s, %s" % (address.number, address.street)
"22, Acacia Avenue"

Heritage

This project is a fork of [Django REST Models](http://djangorestmodel.sourceforge.net/)

Indices and tables