pyrs schema

Contents

MicroService framework :: Schema

https://travis-ci.org/palankai/pyrs-schema.svg?branch=master https://coveralls.io/repos/palankai/pyrs-schema/badge.svg?branch=master&service=github Documentation Status

What is this package for

I’ve used different python frameworks for data serialisation many times. Mostly when I had to implement an API for my work. I felt many times those frameworks did good job but not extensible enough. Also writing easily an API which is satisfy every expectations of projects, without coupled restrictions sometimes really hard.

Nutshell

from pyrs import schema

class UserSchema(schema.Object):
    version = schema.Version(version='1.0')
    username = schema.StringField(required=True)
    password = schema.StringField(required=True, tags=['writeonly'])
    email = schema.EmailField(title='Registered email address')

writer = schema.JSONWriter(UserSchema)
jsonstring = writer.write(data) # The validation also happen

schemawriter = schema.JSONSchemaWriter()
jsonschemastr = writer.write(UserSchema)

Features

  • Easy schema definition
  • Schema validation
  • Decoupled serialisation, validation
  • Extensible API

Installation

The code is tested with python 2.7, 3.3, 3.4.

$ pip install pyrs-schema

Dependencies

See requirements.txt. But The goal is less dependency as possible. The main dependency is the python jsonchema The validation is using that package.

Notice that even it’s a JSON schema validator this work still can be used for any (compatible) schema validation.

Important caveats

This code is in beta version. I working hard on write stable as possible API in the first place but while this code in 0.x version you should expect some major modification on the API.

The ecosystem

This work is part of pyrs framework. The complete framework follow the same intention to implement flexible solution.

Contribution

I really welcome any comments! I would be happy if you fork my code or create pull requests. I’ve already really strong opinions what I want to achieve and how, though any help would be welcomed.

Feel free drop a message to me!

Base module

class pyrs.schema.base.Base(**attrs)[source]

Bases: pyrs.schema.base.Schema

get_jsonschema(context=None)[source]
get_name(default=None)[source]
class pyrs.schema.base.DeclarativeMetaclass[source]

Bases: type

classmethod get_inherited(mcls, cls, name, base, remove_if_none=False)[source]
classmethod update_attrs(mcls, attrs, name, clsname)[source]
classmethod update_fields(mcls, attrs, name, base)[source]
class pyrs.schema.base.Schema(_jsonschema=None, **attrs)[source]

Bases: object

get_attr(name, default=None, expected=None, throw=True)[source]
get_jsonschema(context=None)[source]
get_tags()[source]
has_attr(name, expected=None, throw=True)[source]
has_tags(tags)[source]
parent
root
to_python(value, path='', context=None)[source]

Convert the value to a real python object

to_raw(value, context=None)[source]

Convert the value to a dict of primitives

Basic types

This module introduce the basic schema types.

class pyrs.schema.types.Array(**attrs)[source]

Bases: pyrs.schema.base.Base

Successful validation of an array instance with regards to these two keywords is determined as follows:

if “items” is not present, or its value is an object, validation of the instance always succeeds, regardless of the value of “additional”

if the value of “additional” is boolean value true or an object, validation of the instance always succeeds;

if the value of “additional” is boolean value false and the value of “items” is an array, the instance is valid if its size is less than, or equal to, the size of “items”.

Array specific options:
min_items:
An array instance is valid against “min_items” if its size is greater than, or equal to, the value of this keyword.
max_items:
An array instance is valid against “max_items” if its size is less than, or equal to, the value of this keyword.
unique_items:
If this keyword has boolean value false, the instance validates successfully. If it has boolean value true, the instance validates successfully if all of its elements are unique.
get_jsonschema(context=None)[source]
class pyrs.schema.types.Boolean(**attrs)[source]

Bases: pyrs.schema.base.Base

class pyrs.schema.types.Date(**attrs)[source]

Bases: pyrs.schema.types.String

to_python(value, context=None)[source]
to_raw(value, context=None)[source]
class pyrs.schema.types.DateTime(**attrs)[source]

Bases: pyrs.schema.types.String

to_python(value, context=None)[source]
to_raw(value, context=None)[source]
class pyrs.schema.types.Duration(**attrs)[source]

Bases: pyrs.schema.types.String

to_python(value, context=None)[source]
to_raw(value, context=None)[source]
class pyrs.schema.types.Enum(**attrs)[source]

Bases: pyrs.schema.base.Base

JSON generic enum class

Parameters:enum (list) – list of possible values
get_jsonschema(context=None)[source]

Ensure the generic schema, remove types

Returns:Gives back the schema
Return type:dict
class pyrs.schema.types.Integer(**attrs)[source]

Bases: pyrs.schema.types.Number

Integer specific agruments:
maximum, exclusive_max:
The value of maximum MUST be a number. The value of exclusive_max MUST be a boolean. If “exclusiveMaximum” is present, “maximum” MUST also be present. Successful validation depends on the presence and value of exclusive_max. If it iss is not present, or has boolean value false, then the instance is valid if it is lower than, or equal to, the value of maximum. If exclusive_max has boolean value true, the instance is valid if it is strictly lower than the value of maximum
minimum, exclusive_min:
The value of minimum MUST be a number. The value of exclusive_min MUST be a boolean. If “exclusiveMinimum” is present, “minimum” MUST also be present. Successful validation depends on the presence and value of exclusive_min. If it iss is not present, or has boolean value false, then the instance is valid if it is greater than, or equal to, the value of minimum. If exclusive_min is present and has boolean value true, the instance is valid if it is strictly greater than the value of minimum.
multiple:
The value MUST be an number. This number MUST be strictly greater than 0. A numeric instance is valid against multiple if the result of the division of the instance by this keyword’s value is an integer.
class pyrs.schema.types.Number(**attrs)[source]

Bases: pyrs.schema.base.Base

Number specific agruments:
maximum, exclusive_max:
The value of maximum MUST be a number. The value of exclusive_max MUST be a boolean. If exclusive_max is present, maximum MUST also be present. Successful validation depends on the presence and value of exclusive_max. If it iss is not present, or has boolean value false, then the instance is valid if it is lower than, or equal to, the value of maximum. If exclusive_max has boolean value true, the instance is valid if it is strictly lower than the value of maximum
minimum, exclusive_min:
The value of minimum MUST be a number. The value of exclusive_min MUST be a boolean. If exclusive_min is present, minimum MUST also be present. Successful validation depends on the presence and value of exclusive_min. If it iss is not present, or has boolean value false, then the instance is valid if it is greater than, or equal to, the value of minimum. If exclusive_min is present and has boolean value true, the instance is valid if it is strictly greater than the value of minimum.
multiple:
The value MUST be an number. This number MUST be strictly greater than 0. A numeric instance is valid against multiple if the result of the division of the instance by this keyword’s value is an integer.
get_jsonschema(context=None)[source]
class pyrs.schema.types.Object(extend=None, **attrs)[source]

Bases: pyrs.schema.base.Base

Declarative schema object

Object specific attributes:
additional:
boolean value: enable or disable extra items on the object schema: items which are valid against the schema allowed to extend false by default
min_properties:
An object instance is valid against min_properties if its number of properties is greater than, or equal to, the value.
max_properties:
An object instance is valid against max_properties if its number of properties is less than, or equal to, the value.
pattern:
Should be a dict where the keys are valid regular excpressions and the values are schema instances. The object instance is valid if the extra properties (which are not listed as property) valid against the schema while name is match on the pattern.

Be careful, the pattern sould be explicit as possible, if the pattern match on any normal property the validation should be successful against them as well.

A normal object should looks like the following:

class Translation(types.Object):
    keyword = types.String()
    value = types.String()

    class Attrs:
        additional = False
        patterns = {
            'value_[a-z]{2}': types.String()
        }
extend(properties, context=None)[source]

Extending the exist same with new properties. If you want to extending with an other schema, you should use the other schame properties

fields
get_jsonschema(context=None)[source]
to_python(value, context=None)[source]

Convert the value to a real python object

to_raw(value, context=None)[source]

Convert the value to a JSON compatible value

class pyrs.schema.types.Ref(**attrs)[source]

Bases: pyrs.schema.base.Base

get_jsonschema(context=None)[source]
class pyrs.schema.types.String(**attrs)[source]

Bases: pyrs.schema.base.Base

String specific arguments:
pattern:
The value of this keyword MUST be a string. This string SHOULD be a valid regular expression, according to the ECMA 262 regular expression dialect. A string instance is considered valid if the regular expression matches the instance successfully. Recall: regular expressions are not implicitly anchored.
minlen (int >=0):
The value of this keyword MUST be an integer. This integer MUST be greater than, or equal to, 0. A string instance is valid against this keyword if its length is greater than, or equal to, the value of this keyword.
maxlen (int >=minlen):
The value of this keyword MUST be an integer. This integer MUST be greater than, or equal to, 0. A string instance is valid against this keyword if its length is less than, or equal to, the value of this keyword.
blank (bool):
The value of blank MUST be a boolean. Successful validation depends on presence and value of min_len. If min_len is present and its value is greather than 0 this keyword has no effect. If min_len is not present or its value is 0 the value of min_len will be set to 1.
get_jsonschema(context=None)[source]
class pyrs.schema.types.Time(**attrs)[source]

Bases: pyrs.schema.types.String

to_python(value, context=None)[source]
to_raw(value, context=None)[source]
class pyrs.schema.types.TimeDelta(**attrs)[source]

Bases: pyrs.schema.types.Number

to_python(value, context=None)[source]
to_raw(value, context=None)[source]

Schema IO

This module introduce the base classes for reading and writing data based on schema. The preferred way is using reader is writer rather than using the schema itself. It gives more flexibility and more extensibility.

class pyrs.schema.schemaio.JSONFormReader(schema, context=None)[source]

Bases: pyrs.schema.schemaio.JSONReader

read(data)[source]
class pyrs.schema.schemaio.JSONReader(schema, context=None)[source]

Bases: pyrs.schema.schemaio.Reader

read(data)[source]
class pyrs.schema.schemaio.JSONSchemaDictValidator(schema, context=None)[source]

Bases: pyrs.schema.schemaio.JSONSchemaValidator

validate(data)[source]
class pyrs.schema.schemaio.JSONSchemaValidator(schema, context=None)[source]

Bases: pyrs.schema.schemaio.Validator

validate(data)[source]
class pyrs.schema.schemaio.JSONSchemaWriter(context=None)[source]

Bases: pyrs.schema.schemaio.SchemaWriter

extract(schema, context=None)[source]
write(schema, context=None)[source]
class pyrs.schema.schemaio.JSONWriter(schema, context=None)[source]

Bases: pyrs.schema.schemaio.Writer

write(data)[source]
class pyrs.schema.schemaio.Reader(schema, context=None)[source]

Bases: pyrs.schema.schemaio.SchemaIO

Reader abstract class At least the read method should be implemented

sw = Reader(CustomSchema())
data = sw.write(<custom datastructure>)
read(data)[source]

with self.schema select the proper schema and read the data, validate the input and gives back the decoded value

class pyrs.schema.schemaio.SchemaIO(schema, context=None)[source]

Bases: object

The schema IO gives chance to Schema remain independent from the serialisation method. Even the schema provide conversion still just based on primitive values.

class pyrs.schema.schemaio.SchemaWriter(context=None)[source]

Bases: object

Abstract implementation of schema writer. The main purpose of this class to ensure different useage of the schema. Add extra value if it’s necessary which can’t be implemented by the schema itself.

write(schema)[source]
class pyrs.schema.schemaio.Validator(schema, context=None)[source]

Bases: pyrs.schema.schemaio.SchemaIO

Abstract base class of validators.

validate(data)[source]
class pyrs.schema.schemaio.Writer(schema, context=None)[source]

Bases: pyrs.schema.schemaio.SchemaIO

Writer abstract class At least the write method should be implemented

sw = Writer(CustomSchema())
encoded_data = sw.write({'custom': 'value'})
write(data)[source]

With self.schema select the proper schema, encode the given data then gives it back.

pyrs.schema.schemaio.select_json_validator(schema, context=None)[source]

Formats

pyrs.schema.formats.date_format_checker(instance)[source]
pyrs.schema.formats.datetime_format_checker(instance)[source]
pyrs.schema.formats.duration_format_checker(instance)[source]
pyrs.schema.formats.format_checker(name, raises=())[source]
pyrs.schema.formats.parse_datetime(datetimestring)[source]

Parses ISO 8601 date-times into datetime.datetime objects. This function uses parse_date and parse_time to do the job, so it allows more combinations of date and time representations, than the actual ISO 8601:2004 standard allows.

pyrs.schema.formats.time_format_checker(instance)[source]

Exceptions

class ValidationError

Simple import from jsonschema errors

exception pyrs.schema.exceptions.FormatError(message='Unrecognised input format', value=None)[source]

Bases: pyrs.schema.exceptions.SchemaError

Cover serialization and deserialization errors and related parse errors. It would be raised when the object cannot be converted.

exception pyrs.schema.exceptions.ParseError(message, value, error='ParseError')[source]

Bases: pyrs.schema.exceptions.SchemaError

Cover serialization and deserialization errors and related parse errors. It would be raised when the object cannot be converted.

exception pyrs.schema.exceptions.SchemaError(message, value, error=None)[source]

Bases: Exception

Core exception, you can use it to catch all kind of errors. Unlikely to raised directly. Could contain multiple errors.

exception pyrs.schema.exceptions.ValidationError(message, value, invalid, against, path=None)[source]

Bases: pyrs.schema.exceptions.ValidationErrors

Cover a single validation error

exception pyrs.schema.exceptions.ValidationErrors(message, value, errors=None)[source]

Bases: pyrs.schema.exceptions.SchemaError

Cover the validation errors.

Changelog

The [!] sign marks the incompatible changes.

0.7.3

Fixes
  • Fix JSONReader / Form reader regarding dict input, dict schema

0.7.2

Fixes
  • Get rid of the validation on FormReader

0.7.1

Minor modifications
  • exceptions.* also imported in the pyrs.schema

0.7

This release a major release with lots of modification on API. One of the main concept is decouple responsibilities of schema, means serialisation, deserialisation, validation. Regarding this I removed the load, dump, validate functions from schema. Also make possible the further improvement the get_schema renamed to get_jsonschema.

Major improvements
  • JSONSchemaValidator introduced
  • JSONReader introduced
  • JSONFormReader introduced
  • JSONWriter introduced
  • validate removed from Schema (and the whole validation) [!]
  • load removed from Schema [!]
  • dump removed from Schema [!]
  • to_dict renamed to to_raw [!]
  • get_schema renamed to get_jsonschema [!]
  • Changed to MIT license
Minor modifications
  • ValidationErrors introduced and has became the unified umbrella error
  • SchemaWriter and JSONSchemaWriter introduced
  • JSONSchemaDictValidator introduced (for dict based schemas)
  • Cover document changed regarding the new API
  • Number types functionality extended
  • String type functionality extended
  • More tests for date and time related types
  • Doc and test removed from build
  • Changelog introduced

License

The MIT License (MIT)

Copyright (c) 2015 Csaba Palankai

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Indices and tables