Welcome to DStore!¶
DStore (DataStore) is a Python Object Relational Mapper (ORM) that allows easy description of data models.
The source code can be found on GitHub at https://github.com/MarkLark/dstore
The Python Package can be found on PyPi at https://pypi.python.org/pypi/DStore
Minimal Example¶
from dstore import MemoryStore, Model, var, mod
class Car( Model ):
_namespace = "cars.make"
_vars = [
var.RowID,
var.String( "manufacturer", 32, mods = [ mod.NotNull() ] ),
var.String( "make", 32, mods = [ mod.NotNull() ] ),
var.Number( "year", mods = [ mod.NotNull(), mod.Min( 1950 ), mod.Max( 2017 ) ] ),
]
# Create the MemoryStore instance, and add Models to it
store = MemoryStore( [ Car ] )
store.init_app()
store.connect()
store.create_all()
# Create a new Car, then retrieve it using filter and all
Car( manufacturer = "Holden", make = "Commodore", year = 2010 ).add()
holdens = Car.filter( manufacturer = "Holden" )
cars = Car.all()
# Destroy all instances and shut down the application
store.destroy_all()
store.disconnect()
store.destroy_app()
Introduction To DStore¶
DStore (DataStore) is a Python Object Relational Mapper (ORM) that allows easy description of data models.
Installing¶
From PyPi¶
DStore is available from the PyPi repository at DStore.
This means that all you have to do to install DStore is run the following in a console:
$ pip install dstore
From Source¶
DStore can also be installed from source by downloading from GitHub and running setup.py.
$ wget https://github.com/MarkLark/dstore/archive/master.tar.gz
$ tar xvf master.tar.gz
$ cd dstore-master
$ python setup.py install
Requirements¶
DStore does not rely on any other Python Packages.
It has also been thoroughly tested to work on the following Python Versions:
- 2.7
- 3.3
- 3.4
- 3.5
- 3.6
Minimal Example¶
from dstore import MemoryStore, Model, var, mod
class Car( Model ):
_namespace = "cars.make"
_vars = [
var.RowID,
var.String( "manufacturer", 32, mods = [ mod.NotNull() ] ),
var.String( "make", 32, mods = [ mod.NotNull() ] ),
var.Number( "year", mods = [ mod.NotNull(), mod.Min( 1950 ), mod.Max( 2017 ) ] ),
]
# Create the MemoryStore instance, and add Models to it
store = MemoryStore( [ Car ] )
store.init_app()
store.connect()
store.create_all()
# Create a new Car, then retrieve it using filter and all
Car( manufacturer = "Holden", make = "Commodore", year = 2010 ).add()
holdens = Car.filter( manufacturer = "Holden" )
cars = Car.all()
# Destroy all instances and shut down the application
store.destroy_all()
store.disconnect()
store.destroy_app()
How It Works¶
DStore works by defining the models that you wish to use. It then provides a layer for storing the model instances.
Models¶
Models are used to describe the data types you wish to use.
Base Model¶
The base model
Variables¶
Variables within a Model
-
class
dstore.var.
Variable
(name, default=None, mods=None)[source]¶ The base class used for creating Variable types This also provides a validate method which iterates over the modifiers and runs their validate method on this variable instance.
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
RowID¶
RowID denotes an instance ID, and is not a Class but an instance of dstore.var.Number:
from dstore import var, mod
RowID = Number( "id", mods = [ mod.AutoIncrement(), mod.PrimaryKey(), mod.Unique() ])
In terms of a MySQL it equates to:
id INT NOT NULL AUTO_INCREMENT,
UNIQUE (id),
PRIMARY KEY (id)
Usage
from dstore import Model, var
class Car( Model ):
_namespace = "cars.make"
_vars = [
var.RowID
]
Number¶
-
class
dstore.var.
Number
(name, default=None, mods=None)[source]¶ This variable type is used to store an integer value.
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var, mod var.Number( "year", 1950, mods = [ mod.NotNull() ] )
Boolean¶
-
class
dstore.var.
Boolean
(name, default=None, mods=None)[source]¶ This variable is used to store a boolean value.
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var var.Boolean( "is_available", False )
Character¶
-
class
dstore.var.
Character
(name, length, default=None, mods=None)[source]¶ This variable is used to store a string of static length.
Parameters: - name – The name of the Variable instance inside a Model Class
- length – The length of the string
- default – The default value to give this variable
- mods – A list of modifiers. The mod dstore.mod.Length( length ) is always added to this list
Usage:
from dstore import var var.Character( "is_available", 4 )
Binary¶
-
class
dstore.var.
Binary
(name, length, default=None, mods=None)[source]¶ This variable is used to store binary data.
Parameters: - name – The name of the Variable instance inside a Model Class
- length – The length of the data
- default – The default value to give this variable
- mods – A list of modifiers. The mod dstore.mod.Length( length ) is always added to this list
Usage:
from dstore import var var.Binary( "data", 32 )
String¶
-
class
dstore.var.
String
(name, length, default=None, mods=None)[source]¶ This variable is used to store a string of variable length.
Parameters: - name – The name of the Variable instance inside a Model Class
- length – The length of the string
- default – The default value to give this variable
- mods – A list of modifiers. The mod dstore.mod.Length( length ) is always added to this list
Usage:
from dstore import var var.String( "username", 32 )
Text¶
-
class
dstore.var.
Text
(name, default=None, mods=None)[source]¶ This variable is used to store text.
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var var.Text( "description" )
Float¶
-
class
dstore.var.
Float
(name, default=None, mods=None)[source]¶ This variable is used to store a floating point integer.
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var var.Float( "price" )
Enum¶
-
class
dstore.var.
Enum
(name, values, default=None, mods=None)[source]¶ This variable is used to store a an enum.
Parameters: - name (str) – The name of the Variable instance inside a Model Class
- values (list[str]) – The list of available choices
- default (str or None) – The default value to give this variable
- mods (list[dstore.mod.Mod]) – A list of modifiers. The mod dstore.mod.InEnum( values ) is always added to this list
Usage:
from dstore import var, mod var.Enum( "car_type", [ "truck", "bus", "ute", "car", "motorbike" ], mods = [ mod.NotNull() ] ) # or var.Enum( name = "car_type", values = [ "truck", "bus", "ute", "car", "motorbike" ], default = "car", mods = [ mod.NotNull() ] )
ForeignKey¶
-
class
dstore.var.
ForeignKey
(namespace)[source]¶ This variable is used to link the Model Instance to another Model Instance
Parameters: namespace – The namespace of the Model Class this variable references This variable type is a subclass of dstore.var.Number, with the following mod:
- dstore.mod.ForeignKey
The name given to this variable is the name of the Model Class being referenced with a suffix of ‘_id”
For example:
from dstore import var var.ForeignKey( "cars.make" )
Is the same as:
from dstore import var var.Number( "cars_make_id", mods = [ dstore.mod.ForeignKey( "cars.make" ) )
Date¶
-
class
dstore.var.
Date
(name, default=None, mods=None)[source]¶ This variable is used to store a date value (without a timezone offset).
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var var.Date( "build_date" )
Time¶
-
class
dstore.var.
Time
(name, default=None, mods=None)[source]¶ This variable is used to store a time value (without a timezone offset).
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var var.Time( "build_time" )
DateTime¶
-
class
dstore.var.
DateTime
(name, default=None, mods=None)[source]¶ This variable is used to store a date and time value (without a timezone offset).
Parameters: - name – The name of the Variable instance inside a Model Class
- default – The default value to give this variable
- mods – A list of modifiers, i.e. dstore.mod.NotNull()
Usage:
from dstore import var var.DateTime( "built_on" )
Modifiers¶
-
class
dstore.mod.
Mod
[source]¶ The base class used for creating a Modifier.
A Modifier is a way to validate Model instance Variables upon add and/or update.
PrimaryKey¶
NotNull¶
AutoIncrement¶
Unique¶
ForeignKey¶
Min¶
Max¶
Length¶
Storage¶
DStore provides an abstraction layer to store the model instances.
Memory¶
Introduction¶
The default storage in DStore is the MemoryStore.
This simply stores the model instances in an array in memory.
Keep in mind that this storage type is runtime only, meaning that the data is wiped when the application closes.
Usage¶
from dstore import MemoryStore
store = MemoryStore( models )
MySQL¶
Introduction¶
This storage type allows model instances to be stored in a MySQL DataBase.
In order to use this storage type, you need to install the Python Package for it.
Installing¶
You have two choices to install dstore-mysql, using pip or from source.
From PyPi¶
DStore is available from the PyPi repository at DStore.
This means that all you have to do to install PyMan is run the following in a console:
$ pip install dstore-mysql
From Source¶
DStore can also be installed from source by downloading from GitHub and running setup.py.
$ wget https://github.com/MarkLark/dstore-mysql/archive/master.tar.gz
$ tar xvf master.tar.gz
$ cd dstore-mysql-master
$ python setup.py install
Requirements¶
DStore-MySQL requires the Python package MySQL-python.
Usage¶
from dstore_mysql import MySQLStore
store = MySQLStore( models )
MongoDB¶
Introduction¶
This storage type allows model instances to be stored in a Mongo DataBase.
In order to use this storage type, you need to install the Python Package for it.
Installing¶
You have two choices to install dstore-mongo, using pip or from source.
From PyPi¶
DStore-Mongo package is available from the PyPi repository.
This means that all you have to do to install PyMan is run the following in a console:
$ pip install dstore-mongo
From Source¶
DStore-Mongo can also be installed from source by downloading from GitHub and running setup.py.
$ wget https://github.com/MarkLark/dstore-mongo/archive/master.tar.gz
$ tar xvf master.tar.gz
$ cd dstore-mongo-master
$ python setup.py install
Usage¶
from dstore_mongo import MongoStore
store = MongoStore( models )
CRUD¶
The Data Store provides functionality to Create, Read, Update, Delete and Filter the Model instances.
Create¶
Creating Model Instances
Read¶
Reading Model Instances
Update¶
Updating Model Instances
Delete¶
Deleting Model Instances
Filter¶
Filtering Model Instances
Security - DStore-ACL¶
Introduction¶
DStore-ACL is a Security Layer for DStore. Simply by provide the ACL when describing your Models, and DStore-ACL will do the rest.
Installing¶
You have two choices to install dstore-acl, using pip or from source.
From PyPi¶
DStore is available from the PyPi repository at DStore.
This means that all you have to do to install PyMan is run the following in a console:
$ pip install dstore-acl
From Source¶
DStore can also be installed from source by downloading from GitHub and running setup.py.
$ wget https://github.com/MarkLark/dstore-acl/archive/master.tar.gz
$ tar xvf master.tar.gz
$ cd dstore-acl-master
$ python setup.py install
Web API - Flask-DStore¶
Introduction¶
Flask-DStore is a Web API and Javascript Client.
The API routes, logic and client code is automatically generated for you.
Installing¶
You have two choices to install dstore-mysql, using pip or from source.
From PyPi¶
DStore is available from the PyPi repository at https://pypi.python.org/pypi/flask-dstore.
This means that all you have to do to install PyMan is run the following in a console:
$ pip install flask-dstore
From Source¶
DStore can also be installed from source by downloading from GitHub and running setup.py.
$ wget https://github.com/MarkLark/flask-dstore/archive/master.tar.gz
$ tar xvf master.tar.gz
$ cd flask-dstore-master
$ python setup.py install
Minimal Example¶
from flask import Flask
from dstore import MemoryStore, Model, var, mod
from flask_dstore import API
class Car( Model ):
_namespace = "cars.make"
_vars = [
var.RowID,
var.String( "manufacturer", 32, mods = [ mod.NotNull() ] ),
var.String( "make", 32, mods = [ mod.NotNull() ] ),
var.Number( "year", mods = [ mod.NotNull(), mod.Min( 1950 ), mod.Max( 2017 ) ] ),
]
# Create the app instances
app = Flask( __name__ )
store = MemoryStore( [ Car ] )
api = API( store, app )
# While inside the Flask app context, create all storage and add a car
with app.app_context():
store.create_all()
Car( manufacturer = "Holden", make = "Commodore", year = 2005 ).add()
# Run the Flask dev. server
app.run()
# Now destroy all data
with app.app_context():
store.destroy_all()
store.destroy_app()
Javascript Client¶
Flask-DStore provides a javascript library that allows you to easily create, read, update and delete model instances within Javascript, for every Model type registered, a Factory class is created.
To include this library, add the following to your base html template
<script src="/dstore-client.js"></script>
<script src="/dstore-models.js"></script>
<script src="/dstore-view.js"></script>
DS.Factory¶
This Factory contains the following methods:
-
class
DS.
Factory
()¶ -
load_all
()¶ Load all Model instances from the server into browser memory
-
load
(id)¶ Load a specific Model Instance from the server into the browser memory
Arguments: - id (int) – The instance ID to retrieve
-
get
(id)¶ Get a Model Instance from browser memory, null if it doesn’t exist
Arguments: - id (int) – The instance ID to retrieve
-
add
(args)¶ Add a new Model Instance into browser memory.
You need to execute save on the returned object to save the instance to the server.
Arguments: - args – A dictionary of values to store in the Model Instance
-
DS.Model¶
The Model class is as follows:
Example Usage¶
You don’t directly use the DS.Factory and DS.Model class’ directly. Instead Flask-DStore generates instances of these class’ for every Model type registered.
These are then stored under the ‘ds’ namespace, proceeded by the namespace of the Model types.
The following are examples of how to use this library with the following Model:
from dstore import MemoryStore, Model, var, mod
class Car( Model ):
_namespace = "cars.make"
_vars = [
var.RowID,
var.String( "manufacturer", 32, mods = [ mod.NotNull() ] ),
var.String( "make", 32, mods = [ mod.NotNull() ] ),
var.Number( "year", mods = [ mod.NotNull(), mod.Min( 1950 ), mod.Max( 2017 ) ] ),
]
Load All¶
ds.cars.make.load_all();
car = ds.cars.make.add({ manufacturer: "Holden", make: "Commodore", year: 2010 });
car.save();
Create¶
ds.cars.make.load_all();
car = ds.cars.make.add({ manufacturer: "Holden", make: "Commodore", year: 2010 });
car.save();
Update¶
ds.cars.make.load_all();
car = ds.cars.make.get(1);
car.year = 2011;
car.save();
Delete¶
ds.cars.make.load_all();
car = ds.cars.make.get(1);
car.delete();
Events¶
Table of Contents
Introduction¶
DStore makes the use of an Event Manager in the Store and Models themselves.
These events allow you to hook custom code before or after an action takes place.
In fact, this is exactly how DStore-ACL works to provide a security layer to DStore Models.
Listening To Events¶
Listening to an event is as easy as adding your method to the event in question.
For example, to listen to before_init_app on the store:
from dstore import MemoryStore
def before_init_app( event, store ):
print( "Before init store %s" % store.name )
def after_init_app( event, store ):
print( "After init store %s" % store.name )
store = MemoryStore()
store.events.before_init_app += before_init_app
store.events.after_init_app += after_init_app
store.init_app()
Store Events¶
To listen to a store event, you add your method to the store’s event object.
store.events.before_init_app += method_to_call
destroy_app¶
This event is fired before and after you execute destroy_app on a store
register_models¶
This event is fired before and after all models have been registered.
This happens automatically when init_app is run on the store
register_model¶
This event is fired when a single Model is being registered on the store.
-
before_register_models
(event, store, model) -
after_register_models
(event, store, model) -
store
The Data Store that register_models is being run on
-
model
¶ The Model Class that is being registered
-
create_all¶
This event is fired before and after you execute create_all on a store
destroy_all¶
This event is fired before and after you execute destroy_all on a store
disconnect¶
This event is fired before and after you execute disconnect on a store
Model Events¶
To listen to a store event, you add your method to the store’s event object.
from dstore import MemoryStore, Model, var, mod
class Car( Model ):
_namespace = "cars.make"
_vars = [
var.RowID,
var.String( "manufacturer", 32, mods = [ mod.NotNull() ] ),
var.String( "make", 32, mods = [ mod.NotNull() ] ),
var.Number( "year", mods = [ mod.NotNull(), mod.Min( 1950 ), mod.Max( 2017 ) ] ),
]
def car_before_add( event, model, instance ):
print( "Attempting to add a new %s instance" % model._namespace )
Car.events.before_add += car_before_add
validate¶
This event is fired before and after you attempt to add or update a Model Instance (i.e. on validation)
all¶
This event is fired before and after you attempt to get all Model instances
-
before_all
(event, model)¶ -
model
The Model Class of the instance to be added or updated
-
-
after_all
(event, model, instances)¶ -
model
The Model Class of the instance to be added or updated
-
instances
The list of all instances
-
get¶
This event is fired before and after you attempt to get a Model instance
-
before_get
(event, model, row_id)¶ -
model
The Model Class of the instance to be added or updated
-
row_id
¶ The ID of the instance to retrieve
-
-
after_get
(event, model, instance)¶ -
model
The Model Class of the instance to be added or updated
-
instance
The Model instance retrieved
-
create¶
This event is fired before and after you attempt to create the storage for the Model instances
destroy¶
This event is fired before and after you attempt to destroy the storage for the Model instances
filter¶
This event is fired before and after you attempt to get a filtered list of the Model instances
-
before_filter
(event, model, params)¶ -
model
The Model Class to filter for instances
-
params
¶ A dictionary of the parameters used to filter the list
-
-
after_filter
(event, model, instances, params)¶ -
model
The Model Class to filter for instances
-
instances
The filtered list of Model instances
-
params
A dictionary of the parameters used to filter the list
-