Named tuples¶

putil.eng.ENGPOWER(suffix, exp)¶

Constructor for engineering notation suffix

putil.eng.NUMCOMP(mant, exp)¶

Constructor for number components representation

Functions¶

putil.eng.no_exp(number)¶

Converts a number to a string guaranteeing that the result is not expressed in scientific notation

Parameters:	number (integer or float) – Number to convert
Return type:	string

Raises:	RuntimeError (Argument `number` is not valid)

putil.eng.peng(number, frac_length, rjust=True)¶

Converts a number to engineering notation. The absolute value of the number (if it is not exactly zero) is bounded to the interval [1E-24, 1E+24)

Parameters:	number (integer or float) – Number to convert frac_length (NonNegativeInteger) – Number of digits of fractional part rjust (boolean) – Flag that indicates whether the number is right-justified (True) or not (False)
Return type:	string

Raises:	RuntimeError (Argument `frac_length` is not valid) RuntimeError (Argument `number` is not valid) RuntimeError (Argument `rjust` is not valid)

The supported engineering suffixes are:

Exponent	Name	Suffix
1E-24	yocto	y
1E-21	zepto	z
1E-18	atto	a
1E-15	femto	f
1E-12	pico	p
1E-9	nano	n
1E-6	micro	u
1E-3	milli	m
1E+0
1E+3	kilo	k
1E+6	mega	M
1E+9	giga	G
1E+12	tera	T
1E+15	peta	P
1E+18	exa	E
1E+21	zetta	Z
1E+24	yotta	Y

For example:

>>> import putil.eng
>>> putil.eng.peng(1235.6789E3, 3, False)
'1.236M'

putil.eng.peng_float(snum)¶

Returns the floating point equivalent of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	string

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_float(putil.eng.peng(1235.6789E3, 3, False))
1236000.0

putil.eng.peng_frac(snum)¶

Returns the fractional part of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	integer

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_frac(putil.eng.peng(1235.6789E3, 3, False))
236

putil.eng.peng_int(snum)¶

Returns the integer part of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	integer

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_int(putil.eng.peng(1235.6789E3, 3, False))
1

putil.eng.peng_mant(snum)¶

Returns the mantissa of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	float

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_mant(putil.eng.peng(1235.6789E3, 3, False))
1.236

putil.eng.peng_power(snum)¶

Returns engineering suffix and floating point equivalent of the suffix when a number is represented in engineering notation. putil.eng.peng() lists the correspondence between suffix and floating point exponent.

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	named tuple in which the first item is the engineering suffix and the second item is the floating point equivalent of the suffix when the number is represented in engineering notation.

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_power(putil.eng.peng(1235.6789E3, 3, False))
EngPower(suffix='M', exp=1000000.0)

putil.eng.peng_suffix(snum)¶

Returns the suffix of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	string

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_suffix(putil.eng.peng(1235.6789E3, 3, False))
'M'

putil.eng.peng_suffix_math(suffix, offset)¶

Returns an engineering suffix based on a starting suffix and an offset of number of suffixes

Parameters:	suffix (EngineeringNotationSuffix) – Engineering suffix offset (integer) – Engineering suffix offset
Return type:	string

Raises:	RuntimeError (Argument `offset` is not valid) RuntimeError (Argument `suffix` is not valid) ValueError (Argument `offset` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_suffix_math('u', 6)
'T'

putil.eng.pprint_vector(vector, limit=False, width=None, indent=0, eng=False, frac_length=3)¶

Formats a list of numbers (vector) or a Numpy vector for printing. If the argument vector is None the string 'None' is returned

Parameters:	vector (list of integers or floats, Numpy vector or None) – Vector to pretty print or None limit (boolean) – Flag that indicates whether at most 6 vector items are printed (all vector items if its length is equal or less than 6, first and last 3 vector items if it is not) (True), or the entire vector is printed (False) width (integer or None) – Number of available characters per line. If None the vector is printed in one line indent (boolean) – Flag that indicates whether all subsequent lines after the first one are indented (True) or not (False). Only relevant if width is not None eng (boolean) – Flag that indicates whether engineering notation is used (True) or not (False) frac_length (integer) – Number of digits of fractional part (only applicable if eng is True)
Raises:	ValueError (Argument `width` is too small)
Return type:	string

For example:

>>> from __future__ import print_function
>>> import putil.eng
>>> header = 'Vector: '
>>> data = [1e-3, 20e-6, 300e+6, 4e-12, 5.25e3, -6e-9, 700, 8, 9]
>>> print(
...     header+putil.eng.pprint_vector(
...         data,
...         width=30,
...         eng=True,
...         frac_length=1,
...         limit=True,
...         indent=len(header)
...     )
... )
Vector: [    1.0m,   20.0u,  300.0M,
                     ...
           700.0 ,    8.0 ,    9.0  ]
>>> print(
...     header+putil.eng.pprint_vector(
...         data,
...         width=30,
...         eng=True,
...         frac_length=0,
...         indent=len(header)
...     )
... )
Vector: [    1m,   20u,  300M,    4p,
             5k,   -6n,  700 ,    8 ,
             9  ]
>>> print(putil.eng.pprint_vector(data, eng=True, frac_length=0))
[    1m,   20u,  300M,    4p,    5k,   -6n,  700 ,    8 ,    9  ]
>>> print(putil.eng.pprint_vector(data, limit=True))
[ 0.001, 2e-05, 300000000.0, ..., 700, 8, 9 ]

putil.eng.round_mantissa(arg, decimals=0)¶

Rounds the fractional part of a floating point number mantissa or Numpy vector of floating point numbers to a given number of digits. Integers are not altered. The mantissa used is that of the floating point number(s) when expressed in normalized scientific notation

Parameters:	arg (integer, float, Numpy vector of integers or floats, or None) – Input data decimals (integer) – Number of digits to round the fractional part of the mantissa to.
Return type:	same as arg

For example:

>>> import putil.eng
>>> putil.eng.round_mantissa(012345678E-6, 3)
12.35
>>> putil.eng.round_mantissa(5, 3)
5

putil.eng.to_scientific_string(number, frac_length=None, exp_length=None, sign_always=False)¶

Converts a number or a string representing a number to a string with the number expressed in scientific notation. Full precision is maintained if the number is represented as a string

Parameters:

number (number or string) – Number to convert frac_length (integer or None) – Number of digits of fractional part, None indicates that the fractional part of the number should not be limited exp_length (integer or None) – Number of digits of the exponent; the actual length of the exponent takes precedence if it is longer sign_always (boolean) – Flag that indicates whether the sign always precedes the number for both non-negative and negative numbers (True) or only for negative numbers (False)

Return type:

string

For example:

>>> import putil.eng
>>> putil.eng.to_scientific_string(333)
'3.33E+2'
>>> putil.eng.to_scientific_string(0.00101)
'1.01E-3'
>>> putil.eng.to_scientific_string(99.999, 1, 2, True)
'+1.0E+02'

putil.eng.to_scientific_tuple(number)¶

Returns mantissa and exponent of a number when expressed in scientific notation. Full precision is maintained if the number is represented as a string

Parameters:	number (integer, float or string) – Number
Return type:	named tuple in which the first item is the mantissa (string) and the second item is the exponent (integer) of the number when expressed in scientific notation

For example:

>>> import putil.eng
>>> putil.eng.to_scientific_tuple('135.56E-8')
NumComp(mant='1.3556', exp=-6)
>>> putil.eng.to_scientific_tuple(0.0000013556)
NumComp(mant='1.3556', exp=-6)

Context managers¶

class putil.exdoc.ExDocCxt(exclude=None, pickle_fname=None, in_callables_fname=None, out_callables_fname=None)¶

Bases: object

Context manager to simplify exception tracing; it sets up the tracing environment and returns a putil.exdoc.ExDoc object that can the be used in the documentation string of each callable to extract the exceptions documentation with either putil.exdoc.ExDoc.get_sphinx_doc() or putil.exdoc.ExDoc.get_sphinx_autodoc().

Parameters:

exclude (list of strings or None) – Module exclusion list. A particular callable in an otherwise fully qualified name is omitted if it belongs to a module in this list. If None all callables are included pickle_fname (FileName or None) – File name to pickle traced exception handler (useful for debugging purposes). If None all pickle file is created in_callables_fname (FileNameExists or None) – File name that contains traced modules information. File can be produced by either the putil.pinspect.Callables.save() or putil.exh.ExHandle.save_callables() methods out_callables_fname (FileNameExists or None) – File name to save traced modules information to in JSON format. If the file exists it is overwritten

Raises:

OSError (File [in_callables_fname] could not be found) RuntimeError (Argument `in_callables_fname` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `out_callables_fname` is not valid) RuntimeError (Argument `pickle_fname` is not valid)

For example:

>>> from __future__ import print_function
>>> import putil.eng, putil.exdoc
>>> with putil.exdoc.ExDocCxt() as exdoc_obj:
...     value = putil.eng.peng(1e6, 3, False)
>>> print(exdoc_obj.get_sphinx_doc('putil.eng.peng'))
.. Auto-generated exceptions documentation for putil.eng.peng

:raises:
 * RuntimeError (Argument \`frac_length\` is not valid)

 * RuntimeError (Argument \`number\` is not valid)

 * RuntimeError (Argument \`rjust\` is not valid)

Classes¶

class putil.exdoc.ExDoc(exh_obj, depth=None, exclude=None)¶

Bases: object

Generates exception documentation with reStructuredText mark-up

Parameters:	exh_obj (putil.exh.ExHandle) – Exception handler containing exception information for the callable(s) to be documented depth (non-negative integer or None) – Default hierarchy levels to include in the exceptions per callable (see putil.exdoc.ExDoc.depth). If None exceptions at all depths are included exclude (list of strings or None) – Default list of (potentially partial) module and callable names to exclude from exceptions per callable (see putil.exdoc.ExDoc.exclude). If None all callables are included
Return type:	putil.exdoc.ExDoc
Raises:	RuntimeError (Argument `depth` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `exh_obj` is not valid) RuntimeError (Exceptions database is empty) RuntimeError (Exceptions do not have a common callable) ValueError (Object of argument `exh_obj` does not have any exception trace information)

get_sphinx_autodoc(depth=None, exclude=None, width=72, error=False, raised=False)¶

Returns an exception list marked up in reStructuredText automatically determining callable name

Parameters:

depth (non-negative integer or None) – Hierarchy levels to include in the exceptions list (overrides default depth argument; see putil.exdoc.ExDoc.depth). If None exceptions at all depths are included exclude (list of strings or None) – List of (potentially partial) module and callable names to exclude from exceptions list (overrides default exclude argument, see putil.exdoc.ExDoc.exclude). If None all callables are included width (integer) – Maximum width of the lines of text (minimum 40) error (boolean) – Flag that indicates whether an exception should be raised if the callable is not found in the callables exceptions database (True) or not (False) raised (boolean) – Flag that indicates whether only exceptions that were raised (and presumably caught) should be documented (True) or all registered exceptions should be documented (False)

Raises:

RuntimeError (Argument `depth` is not valid) RuntimeError (Argument `error` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `raised` is not valid) RuntimeError (Argument `width` is not valid) RuntimeError (Callable not found in exception list: [name]) RuntimeError (Unable to determine callable name)

get_sphinx_doc(name, depth=None, exclude=None, width=72, error=False, raised=False)¶

Returns an exception list marked up in reStructuredText

Parameters:

name (string) – Name of the callable (method, function or class property) to generate exceptions documentation for depth (non-negative integer or None) – Hierarchy levels to include in the exceptions list (overrides default depth argument; see putil.exdoc.ExDoc.depth). If None exceptions at all depths are included exclude (list of strings or None) – List of (potentially partial) module and callable names to exclude from exceptions list (overrides default exclude argument; see putil.exdoc.ExDoc.exclude). If None all callables are included width (integer) – Maximum width of the lines of text (minimum 40) error (boolean) – Flag that indicates whether an exception should be raised if the callable is not found in the callables exceptions database (True) or not (False) raised (boolean) – Flag that indicates whether only exceptions that were raised (and presumably caught) should be documented (True) or all registered exceptions should be documented (False)

Raises:

RuntimeError (Argument `depth` is not valid) RuntimeError (Argument `error` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `raised` is not valid) RuntimeError (Argument `width` is not valid) RuntimeError (Callable not found in exception list: [name])

depth¶

Gets or sets the default hierarchy levels to include in the exceptions per callable. For example, a function my_func() calls two other functions, get_data() and process_data(), and in turn get_data() calls another function, open_socket(). In this scenario, the calls hierarchy is:

my_func            <- depth = 0
├get_data          <- depth = 1
│└open_socket      <- depth = 2
└process_data      <- depth = 1

Setting depth=0 means that only exceptions raised by my_func() are going to be included in the documentation; Setting depth=1 means that only exceptions raised by my_func(), get_data() and process_data() are going to be included in the documentation; and finally setting depth=2 (in this case it has the same effects as depth=None) means that only exceptions raised by my_func(), get_data(), process_data() and open_socket() are going to be included in the documentation.

Return type:	non-negative integer
Raises:	RuntimeError (Argument `depth` is not valid)

exclude¶

Gets or sets the default list of (potentially partial) module and callable names to exclude from exceptions per callable. For example, ['putil.ex'] excludes all exceptions from modules putil.exh and putil.exdoc (it acts as r'putil.ex*'). In addition to these modules, ['putil.ex', 'putil.eng.peng'] excludes exceptions from the function putil.eng.peng().

Return type:	list
Raises:	RuntimeError (Argument `exclude` is not valid)

Functions¶

putil.exh.get_exh_obj()¶

Returns the global exception handler

Return type:	putil.exh.ExHandle if global exception handler is set, None otherwise

putil.exh.get_or_create_exh_obj(full_cname=False, exclude=None, callables_fname=None)¶

Returns the global exception handler if it is set, otherwise creates a new global exception handler and returns it

Parameters:	full_cname (boolean) – Flag that indicates whether fully qualified function/method/class property names are obtained for functions/methods/class properties that use the exception manager (True) or not (False). There is a performance penalty if the flag is True as the call stack needs to be traced. This argument is only relevant if the global exception handler is not set and a new one is created exclude (list of strings or None) – Module exclusion list. A particular callable in an otherwise fully qualified name is omitted if it belongs to a module in this list. If None all callables are included callables_fname (FileNameExists or None) – File name that contains traced modules information. File can be produced by either the putil.pinspect.Callables.save() or putil.exh.ExHandle.save_callables() methods
Return type:	putil.exh.ExHandle
Raises:	OSError (File [callables_fname] could not be found RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `callables_fname` is not valid) RuntimeError (Argument `full_cname` is not valid)

putil.exh.del_exh_obj()¶

Deletes global exception handler (if set)

putil.exh.set_exh_obj(obj)¶

Sets the global exception handler

Parameters:	obj (putil.exh.ExHandle) – Exception handler
Raises:	RuntimeError (Argument `obj` is not valid)

Classes¶

class putil.exh.ExHandle(full_cname=False, exclude=None, callables_fname=None)¶

Bases: object

Exception handler

Parameters:	full_cname (boolean) – Flag that indicates whether fully qualified function/method/class property names are obtained for functions/methods/class properties that use the exception manager (True) or not (False). There is a performance penalty if the flag is True as the call stack needs to be traced exclude (list of strings or None) – Module exclusion list. A particular callable in an otherwise fully qualified name is omitted if it belongs to a module in this list. If None all callables are included callables_fname (FileNameExists or None) – File name that contains traced modules information. File can be produced by either the putil.pinspect.Callables.save() or putil.exh.ExHandle.save_callables() methods
Return type:	putil.exh.ExHandle
Raises:	OSError (File [callables_fname] could not be found RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `callables_fname` is not valid) RuntimeError (Argument `full_cname` is not valid) ValueError (Source for module [module_name] could not be found)

__add__(other)¶

Merges two objects.

Raises:	RuntimeError (Incompatible exception handlers) TypeError (Unsupported operand type(s) for +: putil.exh.ExHandle and [other_type])

For example:

>>> import copy, putil.exh, putil.eng, putil.tree
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj1 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> putil.eng.peng(100, 3, True) # Trace some exceptions
' 100.000 '
>>> obj2 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj3 = copy.copy(putil.exh.get_exh_obj())
>>> obj1 == obj2
False
>>> obj1 == obj3
False
>>> obj1 == obj2+obj3
True

__bool__()¶

Returns False if exception handler does not have any exception defined, True otherwise.

Note

This method applies to Python 3.x

For example:

>>> from __future__ import print_function
>>> import putil.exh
>>> obj = putil.exh.ExHandle()
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: False
>>> def my_func(exhobj):
...     exhobj.add_exception('test', RuntimeError, 'Message')
>>> my_func(obj)
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: True

__copy__()¶

Copies object. For example:

>>> import copy, putil.exh, putil.eng
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> obj1 = putil.exh.get_exh_obj()
>>> obj2 = copy.copy(obj1)
>>> obj1 == obj2
True

__eq__(other)¶

Tests object equality. For example:

>>> import copy, putil.exh, putil.eng
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> obj1 = putil.exh.get_exh_obj()
>>> obj2 = copy.copy(obj1)
>>> obj1 == obj2
True
>>> 5 == obj1
False

__iadd__(other)¶

Merges an object into an existing object.

Raises:	RuntimeError (Incompatible exception handlers) TypeError (Unsupported operand type(s) for +: putil.exh.ExHandle and [other_type])

For example:

>>> import copy, putil.exh, putil.eng, putil.tree
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj1 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> putil.eng.peng(100, 3, True) # Trace some exceptions
' 100.000 '
>>> obj2 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj3 = copy.copy(putil.exh.get_exh_obj())
>>> obj1 == obj2
False
>>> obj1 == obj3
False
>>> obj2 += obj3
>>> obj1 == obj2
True

__nonzero__()¶

Returns False if exception handler does not have any exception defined, True otherwise.

Note

This method applies to Python 2.7

For example:

>>> from __future__ import print_function
>>> import putil.exh
>>> obj = putil.exh.ExHandle()
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: False
>>> def my_func(exhobj):
...     exhobj.add_exception('test', RuntimeError, 'Message')
>>> my_func(obj)
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: True

__str__()¶

Returns a string with a detailed description of the object’s contents. For example:

>>> from __future__ import print_function
>>> import docs.support.exh_example
>>> docs.support.exh_example.my_func('Tom')
My name is Tom
>>> print(str(docs.support.exh_example.EXHOBJ)) 
Name    : .../illegal_name
Type    : TypeError
Message : Argument `name` is not valid
Function: None

add_exception(exname, extype, exmsg)¶

Adds an exception to the handler

Parameters:

exname (string) – Exception name; has to be unique within the namespace, duplicates are eliminated extype (Exception type object, i.e. RuntimeError, TypeError, etc.) – Exception type; must be derived from the Exception class exmsg (string) – Exception message; it can contain fields to be replaced when the exception is raised via putil.exh.ExHandle.raise_exception_if(). A field starts with the characters '*[' and ends with the characters ']*', the field name follows the same rules as variable names and is between these two sets of characters. For example, *[fname]* defines the fname field

Raises:

RuntimeError (Argument `exmsg` is not valid) RuntimeError (Argument `exname` is not valid) RuntimeError (Argument `extype` is not valid)

raise_exception_if(exname, condition, edata=None)¶

Raises exception conditionally

Parameters:

exname (string) – Exception name condition (boolean) – Flag that indicates whether the exception is raised (True) or not (False) edata (dictionary, iterable of dictionaries or None) – Replacement values for fields in the exception message (see putil.exh.ExHandle.add_exception() for how to define fields). Each dictionary entry can only have these two keys: field (string) – Field name value (any) – Field value, to be converted into a string with the format string method If None no field replacement is done

Raises:

RuntimeError (Argument `condition` is not valid) RuntimeError (Argument `edata` is not valid) RuntimeError (Argument `exname` is not valid) RuntimeError (Field [field_name] not in exception message) ValueError (Exception name [name] not found’)

save_callables(callables_fname)¶

Saves traced modules information to a JSON file. If the file exists it is overwritten

Parameters:	callables_fname (FileName) – File name
Raises:	RuntimeError (Argument `callables_fname` is not valid)

callables_db¶

Returns the callables database of the modules using the exception handler, as reported by putil.pinspect.Callables.callables_db()

callables_separator¶

Returns the character ('/') used to separate the sub-parts of fully qualified function names in putil.exh.ExHandle.callables_db() and name key in putil.exh.ExHandle.exceptions_db()

exceptions_db¶

Returns the exceptions database. This database is a list of dictionaries that contain the following keys:

• name (string) – Exception name of the form '[callable_identifier]/[exception_name]'. The contents of [callable_identifier] depend on the value of the argument full_cname used to create the exception handler.

  If full_cname is True, [callable_identifier] is the fully qualified callable name as it appears in the callables database (putil.exh.ExHandle.callables_db()).

  If full_cname is False, then [callable_identifier] is a decimal string representation of the callable’s code identifier as reported by the id() function.

  In either case [exception_name] is the name of the exception provided when it was defined in putil.exh.ExHandle.add_exception() (exname argument)

• data (string) – Text of the form '[exception_type] ([exception_message])[raised]' where [exception_type] and [exception_message] are the exception type and exception message, respectively, given when the exception was defined by putil.exh.ExHandle.add_exception() (extype and exmsg arguments); and raised is an asterisk ('*') when the exception has been raised via putil.exh.ExHandle.raise_exception_if(), the empty string ('') otherwise

Context managers¶

putil.misc.ignored(*exceptions)¶

Executes commands and selectively ignores exceptions (Inspired by “Transforming Code into Beautiful, Idiomatic Python” talk at PyCon US 2013 by Raymond Hettinger)

Parameters:	exceptions (Exception object, i.e. RuntimeError, OSError, etc.) – Exception type(s) to ignore

For example:

# misc_example_1.py
from __future__ import print_function
import os, putil.misc

def ignored_example():
    fname = 'somefile.tmp'
    open(fname, 'w').close()
    print('File {0} exists? {1}'.format(
        fname, os.path.isfile(fname)
    ))
    with putil.misc.ignored(OSError):
        os.remove(fname)
    print('File {0} exists? {1}'.format(
        fname, os.path.isfile(fname)
    ))
    with putil.misc.ignored(OSError):
        os.remove(fname)
    print('No exception trying to remove a file that does not exists')
    try:
        with putil.misc.ignored(RuntimeError):
            os.remove(fname)
    except:
        print('Got an exception')

>>> import docs.support.misc_example_1
>>> docs.support.misc_example_1.ignored_example()
File somefile.tmp exists? True
File somefile.tmp exists? False
No exception trying to remove a file that does not exists
Got an exception

class putil.misc.Timer(verbose=False)¶

Bases: object

Profiles blocks of code by calculating elapsed time between the context manager entry and exit time points. Inspired by Huy Nguyen’s blog

Parameters:	verbose (boolean) – Flag that indicates whether the elapsed time is printed upon exit (True) or not (False)
Returns:	putil.misc.Timer
Raises:	RuntimeError (Argument `verbose` is not valid)

For example:

# misc_example_2.py
from __future__ import print_function
import putil.misc

def timer(num_tries, fpointer):
    with putil.misc.Timer() as tobj:
        for _ in range(num_tries):
            fpointer()
    print('Time per call: {0} seconds'.format(
        tobj.elapsed_time/(2.0*num_tries)
    ))

def sample_func():
    count = 0
    for num in range(0, count):
        count += num

>>> from docs.support.misc_example_2 import *
>>> timer(100, sample_func) 
Time per call: ... seconds

elapsed_time¶

Returns elapsed time (in seconds) between context manager entry and exit time points

Return type:	float

class putil.misc.TmpFile(fpointer=None)¶

Bases: object

Creates a temporary file and optionally sets up hooks for a function to write data to it

Parameters:	fpointer (function object or None) – Pointer to a function that writes data to file. If the argument is not None the function pointed to receives exactly one argument, a file-like object
Returns:	temporary file name
Raises:	RuntimeError (Argument `fpointer` is not valid)

Warning

The file name returned uses the forward slash (/) as the path separator regardless of the platform. This avoids problems with escape sequences or mistaken Unicode character encodings (\user for example). Many functions in the os module of the standard library ( os.path.normpath() and others) can change this path separator to the operating system path separator if needed

For example:

# misc_example_3.py
from __future__ import print_function
import putil.misc

def write_data(file_handle):
    file_handle.write('Hello world!')

def show_tmpfile():
    with putil.misc.TmpFile(write_data) as fname:
        with open(fname, 'r') as fobj:
            lines = fobj.readlines()
    print('\n'.join(lines))

>>> from docs.support.misc_example_3 import *
>>> show_tmpfile()
Hello world!

File¶

putil.misc.make_dir(fname)¶

Creates the directory of a fully qualified file name if it does not exist

Parameters:	fname (string) – File name

Equivalent to these Bash shell commands:

$ dir=$(dirname ${fname})
$ mkdir -p ${dir}

Parameters:	fname (string) – Fully qualified file name

putil.misc.normalize_windows_fname(fname)¶

Fix potential problems with a Microsoft Windows file name. Superfluous backslashes are removed and unintended escape sequences are converted to their equivalent (presumably correct and intended) representation, for example r'\x07pps' is transformed to r'\\apps'. A file name is considered network shares if the file does not include a drive letter and they start with a double backslash ('\\')

Parameters:	fname (string) – File name
Return type:	string

Membership¶

putil.misc.isalpha(obj)¶

Tests if the argument is a string representing a number

Parameters:	obj (any) – Object
Return type:	boolean

For example:

>>> import putil.misc
>>> putil.misc.isalpha('1.5')
True
>>> putil.misc.isalpha('1E-20')
True
>>> putil.misc.isalpha('1EA-20')
False

putil.misc.ishex(obj)¶

Tests if the argument is a string representing a valid hexadecimal digit

Parameters:	obj (any) – Object
Return type:	boolean

putil.misc.isiterable(obj)¶

Tests if the argument is an iterable

Parameters:	obj (any) – Object
Return type:	boolean

putil.misc.isnumber(obj)¶

Tests if the argument is a number (complex, float or integer)

Parameters:	obj (any) – Object
Return type:	boolean

putil.misc.isreal(obj)¶

Tests if the argument is a real number (float or integer)

Parameters:	obj (any) – Object
Return type:	boolean

Miscellaneous¶

putil.misc.flatten_list(lobj)¶

Recursively flattens a list

Parameters:	lobj (list) – List to flatten
Return type:	list

For example:

>>> import putil.misc
>>> putil.misc.flatten_list([1, [2, 3, [4, 5, 6]], 7])
[1, 2, 3, 4, 5, 6, 7]

putil.misc.pprint_ast_node(node, annotate_fields=True, include_attributes=False, indent=' ')¶

Emulates the AST module dump function but with prettier printing. From Alex Leone’s blog

Parameters:	node (AST object) – root abstract syntax tree node annotate_fields (boolean) – Flag that indicates whether name and values for fields are shown (True) or not (False); the latter is required if code is to be evaluated include_attributes (boolean) – Flag that indicates whether line numbers and column offsets are dumped (True) or not (False) indent (string) – Characters to use for indenting output sub-nodes and structures
Return type:	string
Raises:	RuntimeError (Argument `node` is not valid)

Numbers¶

putil.misc.gcd(vector)¶

Calculates the greatest common divisor (GCD) of a list of numbers or a Numpy vector of numbers. The computations are carried out with a precision of 1E-12 if the objects are not fractions. When possible it is best to use the fractions data type with the numerator and denominator arguments when computing the GCD of floating point numbers.

Parameters:	vector (list of numbers or Numpy vector of numbers) – Vector of numbers

putil.misc.normalize(value, series, offset=0)¶

Scales a value to the range defined by a series

Parameters:	value (number) – Value to normalize series (list) – List of numbers that defines the normalization range offset (number) – Normalization offset, i.e. the returned value will be in the range [offset, 1.0]
Return type:	number
Raises:	RuntimeError (Argument `offset` is not valid) RuntimeError (Argument `series` is not valid) RuntimeError (Argument `value` is not valid) ValueError (Argument `offset` has to be in the [0.0, 1.0] range) ValueError (Argument `value` has to be within the bounds of the argument `series`)

For example:

>>> import putil.misc
>>> putil.misc.normalize(15, [10, 20])
0.5
>>> putil.misc.normalize(15, [10, 20], 0.5)
0.75

putil.misc.per(arga, argb, prec=10)¶

Calculates the percentage difference between two numbers or the element-wise percentage difference between two lists of numbers or Numpy vectors. If any of the numbers in the arguments is zero the value returned is 1E+20

Parameters:	arga (float, integer, list of floats or integers, or Numpy vector of floats or integers) – First number, list of numbers or Numpy vector argb (float, integer, list of floats or integers, or Numpy vector of floats or integers) – Second number, list of numbers or or Numpy vector prec (integer) – Maximum length of the fractional part of the result
Return type:	Float, list of floats or Numpy vector, depending on the arguments type
Raises:	RuntimeError (Argument `arga` is not valid) RuntimeError (Argument `argb` is not valid) RuntimeError (Argument `prec` is not valid) TypeError (Arguments are not of the same type)

putil.misc.pgcd(numa, numb)¶

Calculate the greatest common divisor (GCD) of two numbers

Parameters:	numa (number) – First number numb (number) – Second number
Return type:	number

For example:

>>> import putil.misc, fractions
>>> putil.misc.pgcd(10, 15)
5
>>> str(putil.misc.pgcd(0.05, 0.02))
'0.01'
>>> str(putil.misc.pgcd(5/3.0, 2/3.0))[:6]
'0.3333'
>>> putil.misc.pgcd(
...     fractions.Fraction(str(5/3.0)),
...     fractions.Fraction(str(2/3.0))
... )
Fraction(1, 3)
>>> putil.misc.pgcd(
...     fractions.Fraction(5, 3),
...     fractions.Fraction(2, 3)
... )
Fraction(1, 3)

String¶

putil.misc.binary_string_to_octal_string(text)¶

Returns a binary-packed string in octal representation aliasing typical codes to their escape sequences

Parameters:	text (string) – Text to convert
Return type:	string

Code	Alias	Description
0	\0	Null character
7	\a	Bell / alarm
8	\b	Backspace
9	\t	Horizontal tab
10	\n	Line feed
11	\v	Vertical tab
12	\f	Form feed
13	\r	Carriage return

For example:

>>> import putil.misc, struct, sys
>>> def py23struct(num):
...    if sys.hexversion < 0x03000000:
...        return struct.pack('h', num)
...    else:
...        return struct.pack('h', num).decode('ascii')
>>> nums = range(1, 15)
>>> putil.misc.binary_string_to_octal_string(
...     ''.join([py23struct(num) for num in nums])
... ).replace('o', '')  
'\\1\\0\\2\\0\\3\\0\\4\\0\\5\\0\\6\\0\\a\\0\\b\\0\\t\\0\\...

putil.misc.char_to_decimal(text)¶

Converts a string to its decimal ASCII representation, with spaces between characters

Parameters:	text (string) – Text to convert
Return type:	string

For example:

>>> import putil.misc
>>> putil.misc.char_to_decimal('Hello world!')
'72 101 108 108 111 32 119 111 114 108 100 33'

putil.misc.elapsed_time_string(start_time, stop_time)¶

Returns a formatted string with the elapsed time between two time points. The string includes years (365 days), months (30 days), days (24 hours), hours (60 minutes), minutes (60 seconds) and seconds. If both arguments are equal, the string returned is 'None'; otherwise, the string returned is [YY year[s], [MM month[s], [DD day[s], [HH hour[s], [MM minute[s] [and SS second[s]]]]]]. Any part (year[s], month[s], etc.) is omitted if the value of that part is null/zero

Parameters:	start_time (datetime) – Starting time point stop_time (datetime) – Ending time point
Return type:	string
Raises:	RuntimeError (Invalid time delta specification)

For example:

>>> import datetime, putil.misc
>>> start_time = datetime.datetime(2014, 1, 1, 1, 10, 1)
>>> stop_time = datetime.datetime(2015, 1, 3, 1, 10, 3)
>>> putil.misc.elapsed_time_string(start_time, stop_time)
'1 year, 2 days and 2 seconds'

putil.misc.pcolor(text, color, indent=0)¶

Returns a string that once printed is colorized

Parameters:	text (string) – Text to colorize color (string) – Color to use, one of 'black', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'white' or 'none' (case insensitive) indent (integer) – Number of spaces to prefix the output with
Return type:	string
Raises:	RuntimeError (Argument `color` is not valid) RuntimeError (Argument `indent` is not valid) RuntimeError (Argument `text` is not valid) ValueError (Unknown color [color])

putil.misc.quote_str(obj)¶

Adds extra quotes to a string. If the argument is not a string it is returned unmodified

Parameters:	obj (any) – Object
Return type:	Same as argument

For example:

>>> import putil.misc
>>> putil.misc.quote_str(5)
5
>>> putil.misc.quote_str('Hello!')
'"Hello!"'
>>> putil.misc.quote_str('He said "hello!"')
'\'He said "hello!"\''

putil.misc.strframe(obj, extended=False)¶

Returns a string with a frame record (typically an item in a list generated by inspect.stack()) pretty printed

Parameters:	obj (tuple) – Frame record extended (boolean) – Flag that indicates whether contents of the frame object are printed (True) or not (False)
Return type:	string

Examples¶

# pcsv_example_1.py
import putil.misc, putil.pcsv

def main():
    with putil.misc.TmpFile() as fname:
        ref_data = [
            ['Item', 'Cost'],
            [1, 9.99],
            [2, 10000],
            [3, 0.10]
        ]
        # Write reference data to a file
        putil.pcsv.write(fname, ref_data, append=False)
        # Read the data back
        obj = putil.pcsv.CsvFile(fname)
    # After the object creation the I/O is done,
    # can safely remove file (exit context manager)
    # Check that data read is correct
    assert obj.header() == ref_data[0]
    assert obj.data() == ref_data[1:]
    # Add a simple row filter, only look at rows that have
    # values 1 and 3 in the "Items" column
    obj.rfilter = {'Item':[1, 3]}
    assert obj.data(filtered=True) == [ref_data[1], ref_data[3]]

if __name__ == '__main__':
    main()

# pcsv_example_2.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as ifname:
        with ctx() as rfname:
            with ctx() as ofname:
                # Create first (input) data file
                input_data = [
                    ['Item', 'Cost'],
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10]
                ]
                putil.pcsv.write(ifname, input_data, append=False)
                # Create second (replacement) data file
                replacement_data = [
                    ['Staff', 'Rate', 'Days'],
                    ['Joe', 10, 'Sunday'],
                    ['Sue', 20, 'Thursday'],
                    ['Pat', 15, 'Tuesday']
                ]
                putil.pcsv.write(rfname, replacement_data, append=False)
                # Replace "Cost" column of input file with "Rate" column
                # of replacement file for "Items" 2 and 3 with "Staff" data
                # from Joe and Pat. Save resulting data to another file
                putil.pcsv.replace(
                    ifname=ifname,
                    idfilter=('Cost', {'Item':[1, 3]}),
                    rfname=rfname,
                    rdfilter=('Rate', {'Staff':['Joe', 'Pat']}),
                    ofname=ofname
                )
                # Verify that resulting file is correct
                ref_data = [
                    ['Item', 'Cost'],
                    [1, 10],
                    [2, 10000],
                    [3, 15]
                ]
                obj = putil.pcsv.CsvFile(ofname)
                assert obj.header() == ref_data[0]
                assert obj.data() == ref_data[1:]

if __name__ == '__main__':
    main()

# pcsv_example_3.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as fname1:
        with ctx() as fname2:
            with ctx() as ofname:
                # Create first data file
                data1 = [
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10]
                ]
                putil.pcsv.write(fname1, data1, append=False)
                # Create second data file
                data2 = [
                    ['Joe', 10, 'Sunday'],
                    ['Sue', 20, 'Thursday'],
                    ['Pat', 15, 'Tuesday']
                ]
                putil.pcsv.write(fname2, data2, append=False)
                # Concatenate file1 and file2. Filter out
                # second column of file2
                putil.pcsv.concatenate(
                    fname1=fname1,
                    fname2=fname2,
                    has_header1=False,
                    has_header2=False,
                    dfilter2=[0, 2],
                    ofname=ofname,
                    ocols=['D1', 'D2']
                )
                # Verify that resulting file is correct
                ref_data = [
                    ['D1', 'D2'],
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10],
                    ['Joe', 'Sunday'],
                    ['Sue', 'Thursday'],
                    ['Pat', 'Tuesday']
                ]
                obj = putil.pcsv.CsvFile(ofname)
                assert obj.header() == ref_data[0]
                assert obj.data() == ref_data[1:]

if __name__ == '__main__':
    main()

# pcsv_example_4.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as fname1:
        with ctx() as fname2:
            with ctx() as ofname:
                # Create first data file
                data1 = [
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10]
                ]
                putil.pcsv.write(fname1, data1, append=False)
                # Create second data file
                data2 = [
                    ['Joe', 10, 'Sunday'],
                    ['Sue', 20, 'Thursday'],
                    ['Pat', 15, 'Tuesday']
                ]
                putil.pcsv.write(fname2, data2, append=False)
                # Merge file1 and file2
                putil.pcsv.merge(
                    fname1=fname1,
                    has_header1=False,
                    fname2=fname2,
                    has_header2=False,
                    ofname=ofname
                )
                # Verify that resulting file is correct
                ref_data = [
                    [1, 9.99, 'Joe', 10, 'Sunday'],
                    [2, 10000, 'Sue', 20, 'Thursday'],
                    [3, 0.10, 'Pat', 15, 'Tuesday'],
                ]
                obj = putil.pcsv.CsvFile(ofname, has_header=False)
                assert obj.header() == list(range(0, 5))
                assert obj.data() == ref_data

if __name__ == '__main__':
    main()

# pcsv_example_5.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as ifname:
        with ctx() as ofname:
            # Create first data file
            data = [
                ['Ctrl', 'Ref', 'Result'],
                [1, 3, 10],
                [1, 4, 20],
                [2, 4, 30],
                [2, 5, 40],
                [3, 5, 50]
            ]
            putil.pcsv.write(ifname, data, append=False)
            # Sort
            putil.pcsv.dsort(
                fname=ifname,
                order=[{'Ctrl':'D'}, {'Ref':'A'}],
                has_header=True,
                ofname=ofname
            )
            # Verify that resulting file is correct
            ref_data = [
                [3, 5, 50],
                [2, 4, 30],
                [2, 5, 40],
                [1, 3, 10],
                [1, 4, 20]
            ]
            obj = putil.pcsv.CsvFile(ofname, has_header=True)
            assert obj.header() == ['Ctrl', 'Ref', 'Result']
            assert obj.data() == ref_data

if __name__ == '__main__':
    main()

Identifying (filtering) columns¶

Several class methods and functions in this module allow column and row filtering of the CSV file data. It is necessary to identify columns for both of these operations and how these columns can be identified depends on whether the file has or does not have a header as indicated by the has_header boolean constructor argument:

• If has_header is True the first line of the file is taken as the header. Columns can be identified by name (a string that has to match a column value in the file header) or by number (an integer representing the column number with column zero being the leftmost column)
• If has_header is False columns can only be identified by number (an integer representing the column number with column zero being the leftmost column)

For example, if a file myfile.csv has the following data:

Then when the file is loaded with putil.pcsv.CsvFile('myfile.csv', has_header=True) the columns can be referred to as 'Ctrl' or 0, 'Ref' or 1, or 'Result' or 2. However if the file is loaded with putil.pcsv.CsvFile('myfile.csv', has_header=False) the columns can be referred only as 0, 1 or 2.

Filtering rows¶

Several class methods and functions of this module allow row filtering of the CSV file data. The row filter is described in the CsvRowFilter pseudo-type

Swapping or inserting columns¶

The column filter not only filters columns but also determines the order in which the columns are stored internally in an putil.pcsv.CsvFile object. This means that the column filter can be used to reorder and/or duplicate columns. For example:

# pcsv_example_6.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as ifname:
        with ctx() as ofname:
            # Create input data file
            data = [
                ['Ctrl', 'Ref', 'Result'],
                [1, 3, 10],
                [1, 4, 20],
                [2, 4, 30],
                [2, 5, 40],
                [3, 5, 50]
            ]
            putil.pcsv.write(ifname, data, append=False)
            # Swap 'Ctrl' and 'Result' columns, duplicate
            # 'Ref' column at the end
            obj = putil.pcsv.CsvFile(
                fname=ifname,
                dfilter=['Result', 'Ref', 'Ctrl', 1],
            )
            assert obj.header(filtered=False) == ['Ctrl', 'Ref', 'Result']
            assert (
                obj.header(filtered=True)
                ==
                ['Result', 'Ref', 'Ctrl', 'Ref']
            )
            obj.write(
                ofname,
                header=['Result', 'Ref', 'Ctrl', 'Ref2'],
                filtered=True,
                append=False
            )
            # Verify that resulting file is correct
            ref_data = [
                [10, 3, 1, 3],
                [20, 4, 1, 4],
                [30, 4, 2, 4],
                [40, 5, 2, 5],
                [50, 5, 3, 5]
            ]
            obj = putil.pcsv.CsvFile(ofname, has_header=True)
            assert obj.header() == ['Result', 'Ref', 'Ctrl', 'Ref2']
            assert obj.data() == ref_data

if __name__ == '__main__':
    main()

Empty columns¶