shapenet

shapenet provides a PyTorch Implementation of Super-Realtime Facial Landmark Detection by Deep Regression of Shape Model Parameters

ShapeNet

The ShapeNet package provides a PyTorch implementation of our paper “Super-Realtime Facial Landmark Detection and Shape Fitting by Deep Regression of Shape Model Parameters”.

It cointains:

JIT

The jit subpackage contains simplified implementations of some modules (networks and layers) to export them via torch.jit

Layers

The following layers have been simplified for torch.jit:

HomogeneousShapeLayer
class HomogeneousShapeLayer(shapes, n_dims, use_cpp=False)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Module to Perform a Shape Prediction (including a global homogeneous transformation)

forward(params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb2e8>)[source]

Performs the actual prediction

Parameters:params (torch.Tensor) – input parameters
Returns:predicted shape
Return type:torch.Tensor
num_params

Property to access these layer’s number of parameters

Returns:number of parameters
Return type:int
HomogeneousTransformationLayer
class HomogeneousTransformationLayer(n_dims: int, use_cpp=False)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Wrapper Class to Wrap the Python and C++ API into a combined python API

forward(shapes: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb7b8>, params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb7f0>)[source]

Selects individual parameters from params and forwards them through the actual layer implementation

Parameters:
  • shapes (torch.Tensor) – shapes to transform
  • params (torch.Tensor) – parameters specifying the affine transformation
Returns:

the transformed shapes in cartesian coordinates
Return type:

torch.Tensor
num_params
_HomogeneousTransformationLayerPy
class _HomogeneousTransformationLayerPy(n_dims)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Module to perform homogeneous transformations in 2D and 3D (Implemented in Python)

_ensemble_2d_matrix(rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcba58>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcba90>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcbac8>)[source]

ensembles the homogeneous transformation matrix for 2D

Parameters:
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one parameter)
  • translation_params (torch.Tensor) – parameters specifying the translation (two parameters)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (one parameter) (currently only isotropic scaling supported)
Returns:

2D transformation matrix
Return type:

torch.Tensor
_ensemble_3d_matrix(rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcbb00>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcbb38>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcbb70>)[source]

ensembles the homogeneous transformation matrix for 3D

Parameters:
  • rotation_params (torch.Tensor) – parameters specifying the rotation (three parameters)
  • translation_params (torch.Tensor) – parameters specifying the translation (three parameters)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (one parameter) (currently only isotropic scaling supported)
Returns:

3D transformation matrix
Return type:

torch.Tensor
_ensemble_trafo(rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb9b0>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb9e8>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcba20>)[source]

ensembles the transformation matrix in 2D and 3D

Parameters:
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one per DoF)
  • translation_params (torch.Tensor) – parameters specifying the translation (one per dimension)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (currently only isotropic scaling supported)
Returns:

transformation matrix
Return type:

torch.Tensor
forward(shapes: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb898>, rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb8d0>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb908>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb940>)[source]

ensembles the homogeneous transformation matrix and applies it to the shape tensor

Parameters:
  • shapes (torch.Tensor) – shapes to transform
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one per DoF)
  • translation_params (torch.Tensor) – parameters specifying the translation (one per dimension)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (currently only isotropic scaling supported)
Returns:

the transformed shapes in cartesian coordinates
Return type:

torch.Tensor
ShapeLayer
class ShapeLayer(shapes, use_cpp=False)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

forward(shape_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfcb048>)[source]
num_params

AbstractShapeNetwork

class AbstractShapeNetwork(**kwargs)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Abstract JIT Network

static norm_type_to_class(norm_type)[source]

AbstractFeatureExtractor

class AbstractFeatureExtractor(in_channels, out_params, norm_class, p_dropout=0)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Abstract Feature Extractor Class all further feature extractors should be derived from

static _build_model(in_channels, out_features, norm_class, p_dropout)[source]

Build the actual model structure

Parameters:
  • in_channels (int) – number of input channels
  • out_features (int) – number of outputs
  • norm_class (Any) – class implementing a normalization
  • p_dropout (float) – dropout probability
Returns:

ensembled model
Return type:

torch.jit.ScriptModule
forward(input_batch)[source]

Feed batch through network

Parameters:input_batch (torch.Tensor) – batch to feed through network
Returns:extracted features
Return type:torch.Tensor

Conv2dRelu

class Conv2dRelu(*args, **kwargs)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Block holding one Conv2d and one ReLU layer

forward(input_batch)[source]

Forward batch though layers

Parameters:input_batch (class:torch.Tensor) – input batch
Returns:class – result
Return type:torch.Tensor

Img224x224Kernel7x7SeparatedDims

class Img224x224Kernel7x7SeparatedDims(in_channels, out_params, norm_class, p_dropout=0)[source]

Bases: shapenet.jit.abstract_network.AbstractFeatureExtractor

static _build_model(in_channels, out_params, norm_class, p_dropout)[source]

Build the actual model structure

Parameters:
  • in_channels (int) – number of input channels
  • out_params (int) – number of outputs
  • norm_class (Any) – class implementing a normalization
  • p_dropout (float) – dropout probability
Returns:

ensembled model
Return type:

torch.jit.ScriptModule
forward(input_batch)

Feed batch through network

Parameters:input_batch (torch.Tensor) – batch to feed through network
Returns:extracted features
Return type:torch.Tensor

ShapeNetwork

class ShapeNetwork(layer_cls, layer_kwargs, in_channels=1, norm_type='instance', img_size=224, tiny=False, feature_extractor=None, **kwargs)[source]

Bases: shapenet.jit.abstract_network.AbstractShapeNetwork

Network to Predict a single shape

forward(input_images)[source]

Forward input batch through network and shape layer

Parameters:input_images (torch.Tensor) – input batch
Returns:predicted shapes
Return type:torch.Tensor
model
static norm_type_to_class(norm_type)

Layer

The layer subpackage contains implementations of all provided layers

ShapeLayer

class ShapeLayer(shapes, use_cpp=False)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Wrapper to compine Python and C++ Implementation under Single API

forward(shape_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfbac18>)[source]

Forwards parameters to Python or C++ Implementation

Parameters:shape_params (torch.Tensor) – parameters for shape ensembling
Returns:Ensempled Shape
Return type:torch.Tensor
num_params

Property to access these layer’s parameters

Returns:number of parameters
Return type:int

_ShapeLayerPy

class _ShapeLayerPy(shapes)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Python Implementation of Shape Layer

forward(shape_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfbacc0>)[source]

Ensemble shape from parameters

Parameters:shape_params (torch.Tensor) – shape parameters
Returns:ensembled shape
Return type:torch.Tensor
num_params

Property to access these layer’s parameters

Returns:number of parameters
Return type:int

_ShapeLayerCpp

class _ShapeLayerCpp(shapes, verbose=True)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

C++ Implementation of Shape Layer

forward(shape_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfbad68>)[source]

Ensemble shape from parameters

Parameters:shape_params (torch.Tensor) – shape parameters
Returns:ensembled shape
Return type:torch.Tensor
num_params

Property to access these layer’s parameters

Returns:number of parameters
Return type:int

HomogeneousTransformationLayer

class HomogeneousTransformationLayer(n_dims: int, use_cpp=False)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Wrapper Class to Wrap the Python and C++ API into a combined python API

forward(shapes: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3198>, params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc31d0>)[source]

Actual prediction

Parameters:
  • shapes (torch.Tensor) – shapes before applied global transformation
  • params (torch.Tensor) – parameters specifying the global transformation
Returns:

Transformed shapes
Return type:

torch.Tensor
num_params

_HomogeneousTransformationLayerPy

class _HomogeneousTransformationLayerPy(n_dims)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Module to perform homogeneous transformations in 2D and 3D (Implemented in Python)

_ensemble_2d_matrix(rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3550>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3588>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc35c0>)[source]

ensembles the homogeneous transformation matrix for 2D

Parameters:
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one parameter)
  • translation_params (torch.Tensor) – parameters specifying the translation (two parameters)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (one parameter) (currently only isotropic scaling supported)
Returns:

2D transformation matrix
Return type:

torch.Tensor
_ensemble_3d_matrix(rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc35f8>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3630>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3668>)[source]

ensembles the homogeneous transformation matrix for 3D

Parameters:
  • rotation_params (torch.Tensor) – parameters specifying the rotation (three parameters)
  • translation_params (torch.Tensor) – parameters specifying the translation (three parameters)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (one parameter) (currently only isotropic scaling supported)
Returns:

3D transformation matrix
Return type:

torch.Tensor
_ensemble_trafo(rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc34a8>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc34e0>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3518>)[source]

ensembles the transformation matrix in 2D and 3D

Parameters:
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one per DoF)
  • translation_params (torch.Tensor) – parameters specifying the translation (one per dimension)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (currently only isotropic scaling supported)
Returns:

transformation matrix
Return type:

torch.Tensor
forward(shapes: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc33c8>, rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3400>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3438>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3470>)[source]

ensembles the homogeneous transformation matrix and applies it to the shape tensor

Parameters:
  • shapes (torch.Tensor) – shapes to transform
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one per DoF)
  • translation_params (torch.Tensor) – parameters specifying the translation (one per dimension)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (currently only isotropic scaling supported)
Returns:

the transformed shapes in cartesian coordinates
Return type:

torch.Tensor

_HomogeneousTransformationLayerCpp

class _HomogeneousTransformationLayerCpp(n_dims, verbose=True)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Module to perform homogeneous transformations in 2D and 3D (Implemented in C++)

forward(shapes: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3278>, rotation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc32b0>, translation_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc32e8>, scale_params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfc3320>)[source]

ensembles the homogeneous transformation matrix and applies it to the shape tensor

Parameters:
  • shapes (torch.Tensor) – shapes to transform
  • rotation_params (torch.Tensor) – parameters specifying the rotation (one per DoF)
  • translation_params (torch.Tensor) – parameters specifying the translation (one per dimension)
  • scale_params (torch.Tensor) – parameter specifying the global scaling factor (currently only isotropic scaling supported)
Returns:

the transformed shapes in cartesian coordinates
Return type:

torch.Tensor

HomogeneousShapeLayer

class HomogeneousShapeLayer(shapes, n_dims, use_cpp=False)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Module to Perform a Shape Prediction (including a global homogeneous transformation)

forward(params: <sphinx.ext.autodoc.importer._MockObject object at 0x7fb7fdfba4a8>)[source]

Performs the actual prediction

Parameters:params (torch.Tensor) – input parameters
Returns:predicted shape
Return type:torch.Tensor
num_params

Property to access these layer’s number of parameters

Returns:number of parameters
Return type:int

Networks

The networks subpackage contains implementations of all provided networks

AbstractShapeNetwork

class AbstractShapeNetwork(**kwargs)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Abstract base Class to provide a convenient norm_class_mapping

static norm_type_to_class(norm_type)[source]

helper function to map a string to an actual normalization class

Parameters:norm_type (str) – string specifying the normalization class
Returns:Normalization Class (subclass of torch.nn.Module)
Return type:type

AbstractFeatureExtractor

class AbstractFeatureExtractor(in_channels, out_params, norm_class, p_dropout=0)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Abstract Feature Extractor Class all further feature extracotrs should be derived from

static _build_model(in_channels, out_features, norm_class, p_dropout)[source]

Build the actual model structure

Parameters:
  • in_channels (int) – number of input channels
  • out_features (int) – number of outputs
  • norm_class (Any) – class implementing a normalization
  • p_dropout (float) – dropout probability
Returns:

ensembled model
Return type:

torch.nn.Module
forward(input_batch)[source]

Feed batch through network

Parameters:input_batch (torch.Tensor) – batch to feed through network
Returns:exracted features
Return type:torch.Tensor

Conv2dRelu

class Conv2dRelu(*args, **kwargs)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Block holding one Conv2d and one ReLU layer

forward(input_batch)[source]

Forward batch though layers

Parameters:input_batch (torch.Tensor) – input batch
Returns:result
Return type:torch.Tensor

Img224x224Kernel7x7SeparatedDims

class Img224x224Kernel7x7SeparatedDims(in_channels, out_params, norm_class, p_dropout=0)[source]

Bases: shapenet.networks.abstract_network.AbstractFeatureExtractor

static _build_model(in_channels, out_params, norm_class, p_dropout)[source]

Build the actual model structure

Parameters:
  • in_channels (int) – number of input channels
  • out_params (int) – number of outputs
  • norm_class (Any) – class implementing a normalization
  • p_dropout (float) – dropout probability
Returns:

ensembled model
Return type:

torch.nn.Module
forward(input_batch)

Feed batch through network

Parameters:input_batch (torch.Tensor) – batch to feed through network
Returns:exracted features
Return type:torch.Tensor

ShapeNetwork

class ShapeNetwork(layer_cls, layer_kwargs, in_channels=1, norm_type='instance', img_size=224, feature_extractor=None, **kwargs)[source]

Bases: shapenet.networks.abstract_network.AbstractShapeNetwork

Network to Predict a single shape

static closure(model, data_dict: dict, optimizers: dict, criterions={}, metrics={}, fold=0, **kwargs)[source]

closure method to do a single backpropagation step

Parameters:
  • model (ShapeNetwork) – trainable model
  • data_dict (dict) – dictionary containing the data
  • optimizers (dict) – dictionary of optimizers to optimize model’s parameters
  • criterions (dict) – dict holding the criterions to calculate errors (gradients from different criterions will be accumulated)
  • metrics (dict) – dict holding the metrics to calculate
  • fold (int) – Current Fold in Crossvalidation (default: 0)
  • **kwargs – additional keyword arguments
Returns:

  • dict – Metric values (with same keys as input dict metrics)
  • dict – Loss values (with same keys as input dict criterions)
  • list – Arbitrary number of predictions as torch.Tensor
Raises:

AssertionError – if optimizers or criterions are empty or the optimizers are not specified
forward(input_images)[source]

Forward input batch through network and shape layer

Parameters:input_images (torch.Tensor) – input batch
Returns:predicted shapes
Return type:torch.Tensor
model
static norm_type_to_class(norm_type)

helper function to map a string to an actual normalization class

Parameters:norm_type (str) – string specifying the normalization class
Returns:Normalization Class (subclass of torch.nn.Module)
Return type:type

CustomGroupNorm

class CustomGroupNorm(n_features, n_groups=2)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

Custom Group Norm which adds n_groups=2 as default parameter

forward(x)[source]

Forward batch through network

Parameters:x (torch.Tensor) – batch to forward
Returns:normalized results
Return type:torch.Tensor

Scripts

The scripts subpackage contains basic scripts for:

prepare_all_data

prepare_all_data()[source]

Prepares all Datasets from commandline arguments

See also

_prepare_ibug_dset(), _prepare_cats()

prepare_helen_dset

prepare_helen_dset()[source]

Prepares the HELEN Dataset from commandline arguments

See also

_prepare_ibug_dset() iBug Datasets HELEN Dataset

prepare_lfpw_dset

prepare_lfpw_dset()[source]

Prepares the LFPW Dataset from commandline arguments

See also

_prepare_ibug_dset() iBug Datasets LFPW Dataset

prepare_cat_dset

prepare_cat_dset()[source]

Prepares the Cat Dataset from commandline arguments

See also

_prepare_cats() Cat Dataset

_prepare_ibug_dset

_prepare_ibug_dset(zip_file, dset_name, out_dir, remove_zip=False, normalize_pca_rot=True)[source]

Prepares an ibug dataset (from a given zipfile)

Parameters:
  • zip_file (str) – the zip archive containing the data
  • dset_name (str) – the dataset’s name
  • out_dir (str) – the output directory
  • remove_zip (bool, optional) – whether or not to remove the ZIP file after finishing the preparation
  • normalize_pca_rot (bool, optional) – whether or not to normalize the data’s rotation during PCA

See also

iBug Datasets

_prepare_cats

_prepare_cats(out_dir, remove_zip=False, normalize_pca_rot=False, **split_options)[source]

Prepares the cat dataset (with multiprocessing)

Parameters:
  • out_dir (str) – the output directory
  • remove_zip (bool, optional) – whether or not to remove the ZIP file after finishing the preparation
  • normalize_pca_rot (bool, optional) – whether or not to normalize the data’s rotation during PCA

See also

Cat Dataset

_process_single_cat_file

_process_single_cat_file(file, target_dir)[source]

Processes a single file of the cat dataset

Parameters:
  • file (str) – the file to process
  • target_dir (str) – the target directory

_make_pca

_make_pca(data_dir, out_file, normalize_rot=False, rotation_idxs=())[source]

Creates a PCA from data in a given directory

Parameters:
  • data_dir (str) – directory containing the image and landmark files
  • out_file (str) – file the pca will be saved to
  • normalize_rot (bool, optional) – whether or not to normalize the data’s rotation
  • rotation_idxs (tuple, optional) – indices for rotation normalization, msut be specified if normalize_rot=True

train_shapenet

train_shapenet()[source]

Trains a single shapenet with config file from comandline arguments

See also

delira.training.PyTorchNetworkTrainer

predict

predict()[source]

Predicts file directory with network specified by files to output path

create_jit_net_from_config_and_weight

create_jit_net_from_config_and_weight(config_dict, weight_file)[source]

Creates a JIT Network from config dict and weight file

Parameters:
  • config_dict (dict) – dict containing network configuration
  • weight_file (str) – path to file containing weights
Returns:

jitted network
Return type:

torch.jit.ScriptModule

main

main()[source]

Utils

The utils subpackage contains general utilities, whcih did not fit into any other subpackage

Config

class Config(verbose=False)[source]

Bases: object

Implements parser for configuration files

now

now()[source]

Return current time as YYYY-MM-DD_HH-MM-SS

Returns:current time
Return type:string

Indices and tables