mpi_array documentation

Build Status AppVeyor Status Coveralls Status Documentation Status MIT License
Release:0.1.0dev1-202-g6a6c707
Version:0.1.0
Date:Nov 15, 2017

Contents

Introduction

The mpi_array python package provides a numpy.ndarray API to a Partitioned Global Address Space array which utilizes MPI (via mpi4py) for parallelism.

Quick Start Example

The following quickstart.py script creates a zero-initialised array and performs some element assignments:

import mpi_array as mpia

# creates zero-initialized PGAS (distributed) array
dary = mpia.zeros((1000, 1000, 1000), dtype="uint16")

# Add one to all elements of array
dary += 1

# Assign to slice
dary[250:750, :, 250:750] = 8

# ufuncs
dary[...] = mpia.power(dary, 1.0/3.0)

The quickstart.py script can be executed serially (single process) as:

python quickstart.py

or in parallel (using 8 processes) as:

mpirun -n 8 python quickstart.py

Installation

Using pip from latest github source:

pip install --user git+git://github.com/mpi-array/mpi_array.git#egg=mpi_array

Requirements

Requires:

Testing

Run tests (unittest test-cases and doctest docstring test-cases) using:

python -m mpi_array.tests

or, from the source tree, run:

python setup.py test

Run tests with parallelism:

mpirun -n 8 python -m mpi_array.tests

Travis CI at:

Documentation

Latest sphinx generated documentation at readthedocs.org:

and at github gh-pages:

Sphinx documentation can be built from the source:

python setup.py build_sphinx

with the HTML generated in docs/_build/html.

Latest source code

Source at github:

clone with:

git clone https://github.com/mpi-array/mpi_array.git

License information

See the file LICENSE.txt for terms & conditions, for usage and a DISCLAIMER OF ALL WARRANTIES.

Terminology

Distributing arrays amongst nodes

The mpi_array Package

Python package for multi-dimensional distributed arrays ( Partitioned Global Address Space ) with numpy.ndarray-like API.

Classes and Functions

Attributes

The mpi_array.benchmarks Package

Runtime benchmarking.

Modules

benchmark Manages finding, running and recoding benchmark results.
bench_creation Benchmarks for array creation.
bench_ufunc Benchmarks for ufuncs.
core Core utilities for benchmark implementations.
utils Miscellaneous benchmark utilities.

The mpi_array.comms Module

MPI communicators and windows for locales.

Classes

LocaleCommsInfo Communicators associated with a locale.
LocaleComms([peer_comm, intra_locale_comm, ...]) MPI communicators for inter and intra locale data exchange.
CartLocaleCommsInfo Communicators associated with a cartesian topology locales.
CartLocaleComms([ndims, dims, peer_comm, ...]) Defines cartesian communication topology for locales.
CommsAndDistribution A 3 element tuple (locale_comms, distribution, this_locale) describing the apportionment of array elements over MPI processes.
ThisLocaleInfo Pair of communicator rank values (inter_locale_rank, peer_rank) which indicates that the rank inter_locale_rank of the inter_locale_comm communicator corresponds to the peer_rank rank of the peer_comm communicator.
RmaWindowBuffer(is_shared, shape, dtype, ...) Details of the buffer allocated on a locale.

Factory Functions

create_locale_comms_info([peer_comm, ...]) Creates a LocaleCommsInfo associated with the specified communicators.
get_locale_comms_info([peer_comm, ...]) Finds or creates a LocaleCommsInfo associated with the specified communicators.
create_cart_locale_comms_info([ndims, dims, ...]) Creates a CartLocaleCommsInfo associated with the specified communicators.
get_cart_locale_comms_info([ndims, dims, ...]) Finds or creates a CartLocaleCommsInfo associated with the specified communicators.
create_locale_comms([locale_type, ...]) Factory function for creating a LocaleComms object.
create_block_distribution(shape[, ...]) Factory function for creating mpi_array.distrbution.BlockPartition distribution and associated CartLocaleComms.
create_cloned_distribution(shape[, ...]) Factory function for creating mpi_array.distrbution.ClonedDistribution distribution and associated LocaleComms.
create_single_locale_distribution(shape[, ...]) Factory function for creating mpi_array.distrbution.SingleLocaleDistribution distribution and associated LocaleComms.
create_distribution(shape[, distrib_type, ...]) Factory function for creating mpi_array.distribution.Distribution and associated LocaleComms.

Attributes

LT_PROCESS Single process locale type
LT_NODE Node (NUMA) locale type
DT_BLOCK Hyper-block partition distribution type
DT_SLAB Hyper-slab partition distribution type
DT_CLONED Entire array repeated on each locale.
DT_SINGLE_LOCALE Entire array on single locale, no array elements on other locales.

The mpi_array.comms_test Module

Module defining mpi_array.comms unit-tests. Execute as:

python -m mpi_array.comms_test

or:

mpirun -n 4 python -m mpi_array.comms_test

The mpi_array.distribution Module

Apportionment of arrays over locales.

Classes

GlobaleExtent([slice, start, stop, halo, struct]) Indexing extent for an entire globale array.
HaloSubExtent([globale_extent, slice, halo, ...]) Indexing extent for single region of a larger globale extent.
LocaleExtent([peer_rank, inter_locale_rank, ...]) Indexing extent for single region of array residing on a locale.
CartLocaleExtent([peer_rank, ...]) Indexing extents for single tile of cartesian domain distribution.
Distribution(globale_extent, locale_extents) Describes the apportionment of array extents amongst locales.
ClonedDistribution(globale_extent, num_locales) Distribution where entire globale extent elements occur on every locale.
SingleLocaleDistribution(globale_extent, ...) Distribution where entire globale extent elements occur on just a single locale.
BlockPartition(globale_extent, dims, ...[, ...]) Block partition of an array (shape) over locales.

The mpi_array.distribution_test Module

Module defining mpi_array.distribution unit-tests. Execute as:

python -m mpi_array.distribution_test

The mpi_array.globale Module

Defines gndarray class and factory functions for creating multi-dimensional distributed arrays (Partitioned Global Address Space).

Classes

gndarray A Partitioned Global Address Space array with numpy.ndarray API.
PerAxisRmaHaloUpdater(locale_extents, dtype, ...) Helper class for performing halo data transfer using RMA via MPI windows (mpi4py.MPI.Win objects).
RmaRedistributeUpdater(dst, src[, casting]) Helper class for redistributing array to new distribution.

Functions

copyto(dst, src[, casting]) Copy the elements of the src array to corresponding elements of the dst array.

The mpi_array.globale_test Module

Module defining mpi_array.globale unit-tests. Execute as:

python -m mpi_array.globale_test

and with parallelism:

mpirun -n  2 python -m mpi_array.globale_test
mpirun -n  4 python -m mpi_array.globale_test
mpirun -n 27 python -m mpi_array.globale_test

The mpi_array.globale_creation Module

Defines mpi_array.globale.gndarray creation functions.

Ones and zeros

empty([shape, dtype, order, ...]) Creates array of uninitialised elements.
empty_like(ary[, dtype, order, subok]) Return a new array with the same shape and type as a given array.
eye(N[, M, k, dtype]) Not implemented.
identity(n[, dtype]) Not implemented.
ones([shape, dtype, comms_and_distrib, order]) Creates array of one-initialised elements.
ones_like(ary, *args, **kwargs) Return a new one-initialised array with the same shape and type as a given array.
zeros([shape, dtype, order, comms_and_distrib]) Creates array of zero-initialised elements.
zeros_like(ary, *args, **kwargs) Return a new zero-initialised array with the same shape and type as a given array.
full([shape, fill_value]) Return a new array of given shape and type, filled with fill_value.
full_like(ary, fill_value, *args, **kwargs) Return a new array with the same shape and type as a given array.

From existing data

array(a[, dtype, copy, order, subok, ndmin]) Create a mpi_array.globale.gndarray from an existing array-like object.
asarray(a[, dtype, order]) Converts a (potentially via a copy) to a mpi_array.globale.gndarray.
asanyarray(a[, dtype, order]) Convert the input to an ndarray, but pass mpi_array.globale.gndarray subclasses through.
copy(ary, **kwargs) Return an array copy of the given object.

The mpi_array.globale_creation_test Module

Module for testing creation/factory functions which generate instances of mpi_array.globale.gndarray. Execute as:

python -m mpi_array.globale_creation_test

and with parallelism:

mpirun -n  2 python -m mpi_array.globale_creation_test
mpirun -n  4 python -m mpi_array.globale_creation_test
mpirun -n 27 python -m mpi_array.globale_creation_test

Classes

GndarrayCreationTest([methodName]) unittest.TestCase for mpi_array.globale.gndarray() instance generation.

The mpi_array.globale_ufunc Module

Defines numpy.ufunc functions for mpi_array.globale.gndarray.

Classes

GndarrayArrayUfuncExecutor(array_like_obj, ...) Instances execute a ufunc for a mpi_array.globale.gndarray.

Functions

get_dtype_and_ndim(array_like) Returns (dtype, ndim) pair for the given array_like argument.
ufunc_result_type(ufunc_types, inputs[, ...]) Attempts to calculate the result type from given ufunc inputs and ufunc types (numpy.ufunc.types).
broadcast_shape(*shape_args) Returns the numpy broadcast shape for the give shape arguments.
shape_extend_dims(ndim, shape) Returns shape pre-prepended with ones so returned 1D array has length ndim.
gndarray_array_ufunc(array_like_obj, ufunc, ...) The implementation for mpi_array.globale.gndarray.__array_ufunc__().

The mpi_array.globale_ufunc_test Module

Module defining mpi_array.globale unit-tests. Execute as:

python -m mpi_array.globale_ufunc_test

and with parallelism:

mpirun -n  2 python -m mpi_array.globale_ufunc_test
mpirun -n  4 python -m mpi_array.globale_ufunc_test
mpirun -n 27 python -m mpi_array.globale_ufunc_test

The mpi_array.indexing Module

Various calculations for array indexing and array indexing extents.

Classes and Functions

IndexingExtent([slice, start, stop, struct])
HaloIndexingExtent([slice, start, stop, ...]) Indexing bounds with ghost (halo) elements, for a single tile of domain decomposition.
calc_intersection_split(dst_extent, ...) Calculates intersection between dst_extent and {src_extent}.

The mpi_array.indexing_test Module

Module defining mpi_array.indexing unit-tests. Execute as:

python -m mpi_array.indexing_test

Classes

IndexingExtentTest([methodName]) unittest.TestCase for mpi_array.indexing.IndexingExtentTest.
HaloIndexingExtentTest([methodName]) unittest.TestCase for mpi_array.indexing.HaloIndexingExtentTest.

The mpi_array.init Module

Initialisation which needs to occur prior to MPI_Init.

Parts of this source borrows from the airspeed velocity (asv) file benchmark.py.

See the LICENSE.

Functions

create_linux_process_time() Uses ctypes to create a time.process_time() on the 'Linux' platform.
create_darwin_process_time() Uses ctypes to create a time.process_time() on the 'darwin' (OSX) platform.
initialise_process_time_timer() Loads (or creates) time.process_time() function and caches the function in mpi_array.init._process_time.
get_process_time_timer() The best timer we can use is time.process_time(), but it is not available in the Python stdlib until Python 3.3.

The mpi_array.license Module

License and copyright info.

License

Copyright (C) 2017 The Australian National University.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Functions

version() Returns mpi_array version string.
license() Returns the mpi_array license string.
copyright() Returns the mpi_array copyright string.

The mpi_array.locale Module

Defines LndarrayProxy class and factory functions for creating multi-dimensional arrays where memory is allocated using mpi4py.MPI.Win.Allocate_shared() or mpi4py.MPI.Win.Allocate().

Classes

lndarray Sub-class of numpy.ndarray which requires buffer to be specified for instantiation.
LndarrayProxy Proxy for lndarray instances.
PartitionViewSlices Stores multiple tuple-of-slice objects indicating the slice (tile) of the lndarray on which a intra_locale_comm rank MPI process operates.

Factory Functions

empty([shape, dtype, comms_and_distrib, ...]) Creates array of uninitialised elements.
empty_like(ary[, dtype]) Return a new array with the same shape and type as a given array.
zeros([shape, dtype, comms_and_distrib, order]) Creates array of zero-initialised elements.
zeros_like(ary, *args, **kwargs) Return a new zero-initialised array with the same shape and type as a given array.
ones([shape, dtype, comms_and_distrib, order]) Creates array of one-initialised elements.
ones_like(ary, *args, **kwargs) Return a new one-initialised array with the same shape and type as a given array.
copy(ary) Return an array copy of the given object.

Utilities

NdarrayMetaData(offset, strides, order) Encapsulates, strides, offset and order argument of LndarrayProxy.__new__().

The mpi_array.locale_test Module

Module defining mpi_array.locale unit-tests. Execute as:

python -m mpi_array.locale_test

Classes

WinLndarrayTest([methodName]) Tests for mpi_array.locale.win_lndarray.
LndarrayTest([methodName]) unittest.TestCase for mpi_array.locale.lndarray.
LndarrayProxyTest([methodName]) unittest.TestCase for mpi_array.locale.LndarrayProxy.

The mpi_array.logging Module

Default initialisation of python logging.

Some simple wrappers of python built-in logging module for mpi_array logging.

Classes and Functions

SplitStreamHandler([outstr, errstr, splitlevel]) A python logging.handlers Handler class for splitting logging messages to different streams depending on the logging-level.
initialise_loggers(names[, log_level, ...]) Initialises specified loggers to generate output at the specified logging level.
get_rank_logger(name[, comm, ranks]) Returns logging.Logger object for message logging.
get_root_logger(name[, comm, root_rank]) Returns a logging.Logger object with time-stamp, comm.Get_name() and comm.Get_rank() in the message.
LoggerFactory() Factory for generating logging.Logger instances.

Attributes

mpi_array.logging.logger_factory = <mpi_array.logging.LoggerFactory object>

Factory for creating logging.Logger objects. Can set value to different instance in order to customise logging output.

The mpi_array.rtd Module

Sets up mock modules for readthedocs.org sphinx builds. See this FAQ.

Functions

initialise_mock_modules(module_name_list) Updates system modules (sys.modules.update()) with unittest.mock.MagicMock objects.

Attributes

mpi_array.rtd.MOCK_MODULES = ['mpi4py', 'mpi4py.MPI']

List of module names

The mpi_array.tests Module

Module for running all mpi_array unit-tests, including unittest test-cases and doctest tests for module doc-strings and sphinx (RST) documentation. Execute as:

python -m mpi_array.tests

The mpi_array.types Module

Convert numpy.dtype to mpi4pi.MPI.Datatype.

Functions

to_datatype(dtyp) Converts a numpy.dtype to a mpi4py.MPI.Datatype.

Utilities

create_lookup() Creates a collections.defaultdict of (numpy.dtype, mpi4py.MPI.Datatype) key-value pairs.
create_datatype(dtyp) Creates a mpi4py.MPI.Datatype from a given numpy.dtype.
find_or_create_datatype(dtyp) Converts a numpy.dtype to a mpi4py.MPI.Datatype.

The mpi_array.types_test Module

Module defining mpi_array.types unit-tests. Execute as:

python -m mpi_array.types_test

The mpi_array.utils Module

Various utilities.

Functions

get_shared_mem_usage_percent_string([...]) Returns a string indicating the current percentage of available shared memory which is allocated.
log_shared_memory_alloc(logger, pfx, ...[, ...]) Generates logging message which indicates amount of shared-memory allocated using call to mpi4py.MPI.Win.Allocate_shared().
log_memory_alloc(logger, pfx, ...[, buffer]) Generates logging message which indicates amount of memory allocated using call to mpi4py.MPI.Win.Allocate().

The mpi_array.unittest Module

Some simple wrappers of python built-in unittest module for mpi_array unit-tests.

Classes and Functions

TestCase([methodName]) Extends unittest.TestCase with the assertArraySplitEqual().
TestProgram(*args, **kwargs) A command-line program that runs a set of tests, extends unittest.TestProgram by using logging rather than standard stream.
TextTestRunner(*args, **kwargs) A test runner class that displays results in textual form.
TextTestResult(stream, descriptions, verbosity)
main(module_name[, log_level, ...]) Like unittest.main(), initialises logging.Logger objects and instantiates a TestProgram to discover and run TestCase objects.

The mpi_array.update Module

Helper classes for calculating sub-extent intersections in order to perform remote array element copying/updates.

Classes and Functions

ExtentAndRegion(locale_extent[, region_extent]) Container for mpi_array.distribution.LocaleExtent and an update region (mpi_array.indexing.IndexingExtent).
MpiExtentAndRegion(locale_extent, region_extent)
ExtentUpdate(dst_extent_info, src_extent_info) Source and destination indexing info for updating a sub-extent region.
PairExtentUpdate(dst_extent, src_extent, ...) Source and destination indexing info for updating a sub-extent region.
MpiPairExtentUpdate(dst_extent, src_extent, ...) Source and destination indexing info for updating the whole of a halo portion.
MpiPairExtentUpdateDifferentDtypes(...) Over-rides MpiPairExtentUpdate.do_get() to buffer-copy and subsequent casting when source and destination arrays have different numpy.dtype.
HaloSingleExtentUpdate(dst_extent, ...) Source and destination indexing info for updating a halo portion.
MpiHaloSingleExtentUpdate(dst_extent, ...) Source and destination indexing info for updating the whole of a halo portion.
UpdatesForRedistribute(dst_distrib, src_distrib) Collection of update extents for re-distribution of array elements from one distribution to another.
RmaUpdateExecutor(inter_win, dst_lndarray, ...) Performs one-sided fetch of data from remote (source) locale arrays to update destination locale array.

The mpi_array.update_test Module

Module defining mpi_array.update unit-tests. Execute as:

python -m mpi_array.update_test

Classes

MpiPairExtentUpdateTest([methodName]) Tests for mpi_array.distribution.MpiPairExtentUpdate.
MpiHaloSingleExtentUpdateTest([methodName]) Tests for mpi_array.distribution.MpiHaloSingleExtentUpdate.
HalosUpdateTest([methodName]) Tests for mpi_array.distribution.HalosUpdate.
UpdatesForRedistributeTest([methodName]) Tests for mpi_array.update.UpdatesForRedistribute.