Welcome to PandaCoreData’s documentation!¶

So, if you already played games like Factorio, Rimworld or Dwarf Fortress and tried to mod them, you will find that it’s absurdly easy to mod those games for simple things like changing the balancing, adding items, changing descriptions and etc. Because the data files of the games are simple raws and not binaries. This library pretty much makes the task of using raws as simple as possible. Raws in our case is plain text files like xml, json, yaml and etc that is commonly used to hold data, for now the library supports yaml and json.

The library might have modding in mind, however it pretty much be used for any sort of software or game engine that uses python.

Internally the library use dataclasses to handle the data and TinyDB to load them from raws.

Contents

Getting Started
- Installation
- Quick Start
- Data Types
  - Models
  - Raws
- Data Core

Getting Started ¶

First of all we need to install the library so we can use it! So here we go:

Installation ¶

Python have a lot of options if you want to install a package. Since the library is available in Pypi repository all you need to run most of the time is:

pip install panda_core_data

That command should work with Windows, Mac or Linux and will install the stable version.

Alternativelly, you can install the development version with this command:

pip install -U pip git+https://github.com/Cerberus1746/PandaCoreData.git@development

Also, this badge attests if the unit tests in the development branch are working, if it’s red, something in the package is not working and you are better off using just the stable version. It’s automatically updated on each commit. Also, don’t worry, I get automatically notified if the tests fail.

https://travis-ci.org/Cerberus1746/PandaCoreData.svg?branch=development

Actually I advise you to always use the stable version. Unless you want to test new features and such.

You can check the docs of the development version here: https://pandacoredata.readthedocs.io/en/development/

And the badge that checks if the documentation of the development version is working is this:

Quick Start ¶

Once installed, the following command is available:

panda_core_data_commands -o folder_name -re json

Which will create all the necessary folder structure and a example Model together with their raws inside the supplied folder_name. You can also set the -re argument as either json or yaml which will create an example raw based on the supplied extension.

Data Types ¶

The library has mostly 4 types of data structures, Model and Template, and then we have their raw files being raw templates and raw models.

Once you create a class and inherit a Model or Template, the class is automatically transformed into a dataclass.

Models ¶

Models are pretty much what you will use inside your game to use the data. You can instance multiple times Model classes each containing a different set of data and accessing them based on how you created the model. But by default you set the fields like you do with a dataclass and the automatic finders will create a model instance with the data from the raw.

For learning purposes. Let’s consider you executed the quick start command with the folder named tutorial. Then go to the file /tutorial/mods/core/models/example_model.py rename however you like and write it like this:

from panda_core_data.model import Model

class Items(Model, data_name="items"):
    name: str
    description: str
    cost: int

Mostly, we are just setting the data_name parameter to make the I in low caps there’s more parameters in _add_into(). Also remember, if you inherit Template or Model, the class will turn into a dataclass, so you can instance the model like this for example:

Items("Copper", "Fragile material", 1) # The args are in the field order
Items(name="Copper", description="Fragile material", cost=1) # as kwargs

But that’s not the point of our library, the point is to have easy way to load data from raw files. So let’s go to the folder /tutorial/mods/core/raws/models/ and rename the folder model_name to the name of your model which in our current case is items if you didn’t set the param data_name the model name will be Items with a capital I because the library will set the same name as the class.

Since models can be instanced multiple times, it will read all raw files inside the folder that have the same name as the model (if it’s inside the folder /mods/core/raws/models/ in this case) and load a instance with the data of the raw.

Raws ¶

The raws are pretty much plain text files that holds data for our instances. The available formats the package support are yaml and json and soon we will add support for xml

So let’s go to the file /tutorial/mods/core/raws/models/items/example_model_raw.yaml rename it to whatever name you’d like, for the tutorial let’s name it copper.yaml and set it’s contents to:

data:
    - name: "Copper"
    - description: "Fragile material"
    - value: 1

Also, now as in version 0.0.2 the package supports json, so alternatively you can use the example below. The json code would be able to work with yaml extension tho, but, I would advise against it because the pyaml package would attempt to decode a mix of json and yaml code, and yaml is slower than json.

{"data": [
    {"name": "Copper"},
    {"description": "Fragile material"},
    {"value": 1}
]}

And the data of our instance will be the same as if you were using yaml syntax.

To load the raw you can do like this:

copper = Items.instance_from_raw("/tutorial/mods/core/raws/models/items/copper.yaml")

Needless to say you need to fix the path to the file. Because I’m not in your computer and I don’t know if you use gentoo with a custom kernel having the root folder named popcorn (I don’t even know if it’s possible to change the root folder, but if I could I would totally name it to popcorn).

Also, in this case, the raw file can be anywhere in the disk, or different atoms in your SSD, because, of course, who would still use a disk (me). It can be inside the folder popcorn/ if you’d like.

But guess what, we don’t need to worry to call every single raw or even to import our model inside our game, because we have:

Data Core ¶

DataCore is the class we use to access all the types, instances and data. It’s use is (hopefully) simple. Let’s edit the file /tutorial/main.py to this:

from os.path import join, dirname, abspath
from dataclasses import fields
from panda_core_data import data_core

def main():
    # Let's automatically get the folder named Popco- mods, I mean.
    mods_folder = join(dirname(abspath(__file__)), "mods")

    # Templates are something we will cover in the future, but for now
    # let's set them to False
    data_core(mods_folder, templates_folder=False)

    # If we use a for with a model class, we will get all instances of it.
    for instance in data_core.get_model_type("items"):
        print(f"\nValues for: {instance.name}")
        # Remember that I said our class turned into a dataclass?
        # We can iter along the fields now.
        for field in fields(instance):
             print(f"\t{field.name}: {getattr(instance, field.name)}")

if __name__ == '__main__':
    main()

This will output the values of our raw file without calling it, without even importing our model and etc etc etc. So much if you like you can create another file in /tutorial/mods/core/raws/models/items/ and the instance will automatically be created. Also, the package will automatically choose the correct parser based on the extension of the raw file. So you are able to mix json and yaml files together. But please, don’t unless you have a good reason for that.

CoreData¶

DataCore¶

class panda_core_data.DataCore(*args, name=None, replace=False, **kwargs)¶

Bases: panda_core_data.data_core_bases.data_model.DataModel, panda_core_data.data_core_bases.data_template.DataTemplate

Class where everything is kept.

__call__(mods_path, **kwargs)¶

Automatically import all data types based on the paths in the params.

Parameters:

mods_path (str) – Absolute root folder to the root mods folder
core_mod_folder (str) – Name of the core mod folder. The base mod.
raws_folder (str) – Name of the raw folder.
models_folder (str) – Name of the models folder.
templates_folder (bool or str) – Name of the templates folder.
raw_models_folder (str or bool) – Name of the raws that are related to the models. Default is ‘models_folder’ param.
raw_templates_folder (str or bool) – Name of the raws that are related to the templates. Default is the ‘templates_folder’ param

Raises:

PCDFolderNotFound – If any of the folders are invalid.

DataTemplate¶

class panda_core_data.data_core_bases.data_template.DataTemplate(*args, **kwargs)¶

Bases: panda_core_data.data_core_bases.base_data.BaseData

all_template_instances¶

Gets all the template instances

Yield Template:	returns a generator of all instanced templates.

all_templates¶

Get all Template types

Return list(Template):
	return a list of template types.

get_template_type(template_name, **kwargs)¶

Get Data type from a list of all Template types.

Return Template:
Parameters:	template_name (str) – The name of the Template default (bool) – Default value to be returned if the template type couldn’t be found.
	the `Template`

recursively_instance_template(path, *args, **kwargs)¶

Instance Template recursively based on the raws inside the folders.

Return list(Template):
Parameters:	path (str) – Starting path to search for raws.
	returns all the instanced template from the path.

DataModel¶

class panda_core_data.data_core_bases.data_model.DataModel(*args, **kwargs)¶

Bases: panda_core_data.data_core_bases.base_data.BaseData

all_model_instances¶

Gets all the model instances.

Yield Model:	returns a generator of all instanced templates.

all_models¶

Get all Model types

Return list(Model):
	return a list of model types.

get_model_type(model_name: str, **kwargs)¶

Get Data type from a list of all Model types.

Parameters:	model_name (str) – The name of the Model default (bool) – Default value to be returned if the model type couldn’t be found.
Return Model:	the `Model`

recursively_instance_model(path, *args, **kwargs)¶

Instance Model recursively based on the raws inside the folders.

Return list(Model):
Parameters:	path (str) – Starting path to search for raws.
	returns all the instanced model from the path.

BaseData¶

class panda_core_data.data_core_bases.base_data.BaseData(excluded_extensions=False)¶

classmethod __init_subclass__()¶

This function checks if a method is lacking inside any class that inherits this, and also automatically creates docstrings into those methods based on the original method.

Parameters:	cls (DataType) – Child class

add_module(path)¶

Automatically import the module from the python file and add it’s directory to sys.path if it wasn’t in there before.

Parameters:	path (Path or str) – The path to the python file.
Return module:	Returns the imported module.

static all_datas()¶

Get all DataType types

Return list(DataType):
	return a list of data types.

static get_data_type(data_name, data_dict, default=None)¶

Get Data type from a list of all DataType types.

Return DataType:
Parameters:	data_name (str) – The name of the DataType default (bool) – Default value to be returned if the data type couldn’t be found.
	the `DataType`

recursively_add_module(path)¶

Recursively add a module with DataType from the supplied path.

Return list(module):
Parameters:	path (str) – Path to the data module
	Returns the loaded modules.

static recursively_instance_data()¶

Instance DataType recursively based on the raws inside the folders.

Return list(DataType):
Parameters:	path (str) – Starting path to search for raws.
	returns all the instanced data from the path.

Exceptions¶

Module containing all cusstom exceptions used by the package. All exceptions related to this package has the PCD prefix.

Please, if you used one of our methods and it raised an exception that doesn’t have our PCD prefix and it came from inside our files send us a ticket in this link: https://github.com/Cerberus1746/PandaCoreData/issues

author:	Leandro (Cerberus1746) Benedet Garcia

exception panda_core_data.custom_exceptions.PCDDataCoreIsNotUnique¶: Exception raised if the data core is not unique.

exception panda_core_data.custom_exceptions.PCDDuplicatedModuleName¶: Exception raised if a module with the same name was already imported

exception panda_core_data.custom_exceptions.PCDDuplicatedTypeName¶: Exception raised if a Model or Template Type already exists with the same name.

exception panda_core_data.custom_exceptions.PCDFolderIsEmpty¶: Exception raised if the folder is empty.

exception panda_core_data.custom_exceptions.PCDInvalidBaseData¶: Exception raised if a base doesn’t have a method.

exception panda_core_data.custom_exceptions.PCDInvalidPath¶: Exception raised if the file is invalid.

exception panda_core_data.custom_exceptions.PCDInvalidPathType¶: Exception raised if a invalid path type was requested.

exception panda_core_data.custom_exceptions.PCDInvalidRaw¶: Exception raised if a raw is invalid

exception panda_core_data.custom_exceptions.PCDRawFileNotSupported¶: Exception raised if the package can’t read the extension.

exception panda_core_data.custom_exceptions.PCDTypeError¶: Exception raised if a invalid type was found.

Data Types¶

DataType¶

class panda_core_data.data_type.DataType(*args, **kwargs)¶

Base for all the model types, be it template or model

Internally it uses the TinyDB database

__getattr__(name)¶

This is here just to make this method back to the default that was overwritten by TinyDB

Parameters:	name (str) – name of the attribute to get

static __new__(cls, *_, db_file=False, **__)¶

Method that handles the instancing of the models and templates, this is necessary because dataclasses create a custom __init__ method. Which we doesn’t use at all if a raw file is supplied.

Parameters:	cls (Model or Template) – the type to be instanced path (str) – path to the raw file to be loaded, if False, the class will be instanced like a normal dataclass

__repr__()¶: Return repr(self).

__setattr__(attr_name, value)¶

This is here just to make this method back to the default that was overwritten by TinyDB

Parameters:	name (str) – name of the attribute to write value (any) – value of the attribute.

static _add_into(data_type, data_type_dict, **kwargs)¶

You can use the prefix dataclass_ with a dataclass parameter to configure it. They can be found in this link: https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass

For example:

from panda_core_data.model import Model

class ModelName(Model, dataclass_init=False):
    out: int

    def __init__(self, num):
        self.out = num ** num

Will make the library not create a __init__ method and use yours instead.

Parameters:	cls (Model or Template) – class type to be added template_name (str) – The name of the template, if not supplied, the class name is used. dependency_list (list(str)) – `Template` to be used as dependency.

add_dependencies()¶: add all dependencies for the model

has_dependencies¶

If the model has any dependencies

Return bool:	If the instance have dependencies or not.

load_db(db_file, *init_args, default_table='data', **kwargs)¶

Method that load raw files and assign each field to an attribute.

Parameters:	db_file (str) – Path to a raw file storage (tinydb.storages.Storage) – storage class to be used, it needs to inherit `Storage` default_table (str) – default main field in the raw file.

Model¶

class panda_core_data.model.Model(*args, **kwargs)¶

Bases: panda_core_data.data_type.DataType

classmethod __init_subclass__(core_name=None, **kwargs)¶: Method that automatically registers class types into data_core. You can use the same parameters as _add_into()

Template¶

class panda_core_data.model.Template(*args, **kwargs)¶

Bases: panda_core_data.data_type.DataType

Class that will be used to make ModelTemplates

classmethod __init_subclass__(core_name=None, **kwargs)¶: Method that automatically registers class types into data_core. You can use the same parameters as _add_into()

Storages¶

Storages are classes based on TinyDB that handles file management, raw parsing and other things like that, if you’d like to create support for more file types, all you have to do is to inherit the BaseDB and follow the instructions in the API of that class.

Directory Utils¶

panda_core_data.storages.auto_convert_to_pathlib(path)¶

Check if the path is valid and automatically convert it into a Path object

Parameters:	path (str or `Path`) – source folder
Returns:	The Path object
Return type:	`Path`
Raises:	PCDFolderNotFound – If the folder is invalid PCDInvalidPath – If the file is invalid

panda_core_data.storages.get_extension(path)¶

Get file extension from the path

Return str:	The file extension

panda_core_data.storages.get_raw_extensions()¶

Get all available extensions the package supports

Return list(str):
	A list of available extensions

panda_core_data.storages.get_storage_from_extension(extension)¶

Returns the storage based on the file extension

Returns:	Returns a storage object that handles the raw file
Return type:	`Storage`

panda_core_data.storages.is_excluded_extension(path, exclude_ext)¶

Check if the file has an ignored extension

Parameters:	path (str or `Path`) – source folder exclude_ext (list(str)) – If the path should be from a folder or file
Return bool:	returns True if it’s a excluded extension, False otherwise

panda_core_data.storages.raw_glob_iterator(path, excluded_ext=False)¶

Iterate along the path yielding the raw file.

:yields Path: The file path

YAMLDB¶

class panda_core_data.storages.yaml_db.YamlDB(*args, **kwargs)¶

Parser storage class used to read yaml files

__init__(*args, **kwargs)¶

Open file as Data Base

Parameters:	str (path) – path pointing to a yaml file

read()¶: Method used by TinyDB to read the file

write(data)¶

Write the current state of the database to the storage.

Any kind of serialization should go here.

Parameters:	data (dict) – The current state of the database.

JsonDB¶

class panda_core_data.storages.json_db.JsonDB(path, **kwargs)¶

read()¶

Read the last stored state.

Any kind of deserialization should go here. Return None here to indicate that the storage is empty.

Return type:	dict

write(data)¶

Write the current state of the database to the storage.

Any kind of serialization should go here.

Parameters:	data (dict) – The current state of the database.

BaseDB¶

class panda_core_data.storages.base_db.BaseDB(path, **kwargs)¶

Base storage class that reads which extensions are available to feed the path handling functions

To create a new storage, you will need to inherit this class, create a extensions variable containing a list of extensions the storage will support for example:

extensions = ["yml", "yaml"]

then implement a read and write method using the methods base_read() and base_write() all you need to do is follow the instructions contained in them

__init__(path, **kwargs)¶

Create a new instance.

Parameters:	path (str) – Path to file

classmethod __init_subclass__()¶: Automatically generate an extension list containing the available raw extensions available together with their storage.

base_read(load_method, use_handle)¶

Base method used by children classes to read the file and transforms the string into a list of dictionaries, a good example of this method is the built in python json.load() however, since it needs a string as an input (or handler) you would need to set the parameter use_handler so the string, which is the contents of the raw file, will be passed to that method. For example the read method of our yaml parser:

def read(self):
    return self.base_read(yaml.safe_load, True)

And since the function yaml.safe_load() needs a string as an input, we set use_handle to True.

An example of list of dictionaries would be like this:

{"data": [
    {
        'field_name': 'value',
        'another_field': 10,
    },
    {
        'field_name': 'value',
        'another_field': 10,
    },
]}

The dict keys are fields of a dataclass and the value, well, values.

Parameters:	load_method (`function`) – method used to transform the raw file into a list of dictionaries. use_handle (bool) – TinyDB offers a handle (More specifically, the handle of the class `JSONStorage`) to load the file and turn into a string automatically if you’d like to use it, just set this parameter to True.

base_write(write_method, data, use_handle)¶

Transforms the data dictionary to a raw representation.

Parameters:	data (dict(str, object)) – data dictionary.

Utils¶

created:	2019-07-29
author:	Leandro (Cerberus1746) Benedet Garcia

panda_core_data.utils.check_if_valid_instance(an_object, the_type)¶

Check if an_object is a instance from the_type.

Parameters:	an_object (any) – source object. the_type (type) – type object.
Raises:	PCDTypeError – if the type is not a instance or wrong type.

Change Logs¶

0.0.2¶

Support for Json was added
auto_convert_to_pathlib() do not need the is_file parameter anymore
DataCore has a new parameter excluded_extensions
Now, it’s possible to save the Model or Template to the raw with the save_to_file() method
PCDInvalidRaw and PCDDuplicatedModuleName exceptions was created
instance_data() and folder_contents() were added.
panda_core_data_commands has a new option -re which you can choose between json or yaml
Both exceptions PCDFolderNotFound and PCDFileNotFound was merged into PCDInvalidPath
Package Storage was created

Welcome to PandaCoreData’s documentation!¶

Getting Started¶

Installation¶

Quick Start¶

Data Types¶

Models¶

Raws¶

Data Core¶

CoreData¶

DataCore¶

DataTemplate¶

DataModel¶

BaseData¶

Exceptions¶

Data Types¶

DataType¶

Model¶

Template¶

Storages¶

Directory Utils¶

YAMLDB¶

JsonDB¶

BaseDB¶

Utils¶

Change Logs¶

0.0.2¶

Indices and tables¶

Getting Started ¶

Installation ¶

Quick Start ¶

Data Types ¶

Models ¶

Raws ¶

Data Core ¶