Named tuples¶

putil.eng.ENGPOWER(suffix, exp)¶

Constructor for engineering notation suffix

putil.eng.NUMCOMP(mant, exp)¶

Constructor for number components representation

Functions¶

putil.eng.no_exp(number)¶

Converts a number to a string guaranteeing that the result is not expressed in scientific notation

Parameters:	number (integer or float) – Number to convert
Return type:	string

Raises:	RuntimeError (Argument `number` is not valid)

putil.eng.peng(number, frac_length, rjust=True)¶

Converts a number to engineering notation. The absolute value of the number (if it is not exactly zero) is bounded to the interval [1E-24, 1E+24)

Parameters:	number (integer or float) – Number to convert frac_length (NonNegativeInteger) – Number of digits of fractional part rjust (boolean) – Flag that indicates whether the number is right-justified (True) or not (False)
Return type:	string

Raises:	RuntimeError (Argument `frac_length` is not valid) RuntimeError (Argument `number` is not valid) RuntimeError (Argument `rjust` is not valid)

The supported engineering suffixes are:

Exponent	Name	Suffix
1E-24	yocto	y
1E-21	zepto	z
1E-18	atto	a
1E-15	femto	f
1E-12	pico	p
1E-9	nano	n
1E-6	micro	u
1E-3	milli	m
1E+0
1E+3	kilo	k
1E+6	mega	M
1E+9	giga	G
1E+12	tera	T
1E+15	peta	P
1E+18	exa	E
1E+21	zetta	Z
1E+24	yotta	Y

For example:

>>> import putil.eng
>>> putil.eng.peng(1235.6789E3, 3, False)
'1.236M'

putil.eng.peng_float(snum)¶

Returns the floating point equivalent of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	string

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_float(putil.eng.peng(1235.6789E3, 3, False))
1236000.0

putil.eng.peng_frac(snum)¶

Returns the fractional part of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	integer

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_frac(putil.eng.peng(1235.6789E3, 3, False))
236

putil.eng.peng_int(snum)¶

Returns the integer part of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	integer

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_int(putil.eng.peng(1235.6789E3, 3, False))
1

putil.eng.peng_mant(snum)¶

Returns the mantissa of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	float

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_mant(putil.eng.peng(1235.6789E3, 3, False))
1.236

putil.eng.peng_power(snum)¶

Returns engineering suffix and floating point equivalent of the suffix when a number is represented in engineering notation. putil.eng.peng() lists the correspondence between suffix and floating point exponent.

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	named tuple in which the first item is the engineering suffix and the second item is the floating point equivalent of the suffix when the number is represented in engineering notation.

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_power(putil.eng.peng(1235.6789E3, 3, False))
EngPower(suffix='M', exp=1000000.0)

putil.eng.peng_suffix(snum)¶

Returns the suffix of a number represented in engineering notation

Parameters:	snum (EngineeringNotationNumber) – Number
Return type:	string

Raises:	RuntimeError (Argument `snum` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_suffix(putil.eng.peng(1235.6789E3, 3, False))
'M'

putil.eng.peng_suffix_math(suffix, offset)¶

Returns an engineering suffix based on a starting suffix and an offset of number of suffixes

Parameters:	suffix (EngineeringNotationSuffix) – Engineering suffix offset (integer) – Engineering suffix offset
Return type:	string

Raises:	RuntimeError (Argument `offset` is not valid) RuntimeError (Argument `suffix` is not valid) ValueError (Argument `offset` is not valid)

For example:

>>> import putil.eng
>>> putil.eng.peng_suffix_math('u', 6)
'T'

putil.eng.pprint_vector(vector, limit=False, width=None, indent=0, eng=False, frac_length=3)¶

Formats a list of numbers (vector) or a Numpy vector for printing. If the argument vector is None the string 'None' is returned

Parameters:	vector (list of integers or floats, Numpy vector or None) – Vector to pretty print or None limit (boolean) – Flag that indicates whether at most 6 vector items are printed (all vector items if its length is equal or less than 6, first and last 3 vector items if it is not) (True), or the entire vector is printed (False) width (integer or None) – Number of available characters per line. If None the vector is printed in one line indent (boolean) – Flag that indicates whether all subsequent lines after the first one are indented (True) or not (False). Only relevant if width is not None eng (boolean) – Flag that indicates whether engineering notation is used (True) or not (False) frac_length (integer) – Number of digits of fractional part (only applicable if eng is True)
Raises:	ValueError (Argument `width` is too small)
Return type:	string

For example:

>>> from __future__ import print_function
>>> import putil.eng
>>> header = 'Vector: '
>>> data = [1e-3, 20e-6, 300e+6, 4e-12, 5.25e3, -6e-9, 700, 8, 9]
>>> print(
...     header+putil.eng.pprint_vector(
...         data,
...         width=30,
...         eng=True,
...         frac_length=1,
...         limit=True,
...         indent=len(header)
...     )
... )
Vector: [    1.0m,   20.0u,  300.0M,
                     ...
           700.0 ,    8.0 ,    9.0  ]
>>> print(
...     header+putil.eng.pprint_vector(
...         data,
...         width=30,
...         eng=True,
...         frac_length=0,
...         indent=len(header)
...     )
... )
Vector: [    1m,   20u,  300M,    4p,
             5k,   -6n,  700 ,    8 ,
             9  ]
>>> print(putil.eng.pprint_vector(data, eng=True, frac_length=0))
[    1m,   20u,  300M,    4p,    5k,   -6n,  700 ,    8 ,    9  ]
>>> print(putil.eng.pprint_vector(data, limit=True))
[ 0.001, 2e-05, 300000000.0, ..., 700, 8, 9 ]

putil.eng.round_mantissa(arg, decimals=0)¶

Rounds the fractional part of a floating point number mantissa or Numpy vector of floating point numbers to a given number of digits. Integers are not altered. The mantissa used is that of the floating point number(s) when expressed in normalized scientific notation

Parameters:	arg (integer, float, Numpy vector of integers or floats, or None) – Input data decimals (integer) – Number of digits to round the fractional part of the mantissa to.
Return type:	same as arg

For example:

>>> import putil.eng
>>> putil.eng.round_mantissa(012345678E-6, 3)
12.35
>>> putil.eng.round_mantissa(5, 3)
5

putil.eng.to_scientific_string(number, frac_length=None, exp_length=None, sign_always=False)¶

Converts a number or a string representing a number to a string with the number expressed in scientific notation. Full precision is maintained if the number is represented as a string

Parameters:

number (number or string) – Number to convert frac_length (integer or None) – Number of digits of fractional part, None indicates that the fractional part of the number should not be limited exp_length (integer or None) – Number of digits of the exponent; the actual length of the exponent takes precedence if it is longer sign_always (boolean) – Flag that indicates whether the sign always precedes the number for both non-negative and negative numbers (True) or only for negative numbers (False)

Return type:

string

For example:

>>> import putil.eng
>>> putil.eng.to_scientific_string(333)
'3.33E+2'
>>> putil.eng.to_scientific_string(0.00101)
'1.01E-3'
>>> putil.eng.to_scientific_string(99.999, 1, 2, True)
'+1.0E+02'

putil.eng.to_scientific_tuple(number)¶

Returns mantissa and exponent of a number when expressed in scientific notation. Full precision is maintained if the number is represented as a string

Parameters:	number (integer, float or string) – Number
Return type:	named tuple in which the first item is the mantissa (string) and the second item is the exponent (integer) of the number when expressed in scientific notation

For example:

>>> import putil.eng
>>> putil.eng.to_scientific_tuple('135.56E-8')
NumComp(mant='1.3556', exp=-6)
>>> putil.eng.to_scientific_tuple(0.0000013556)
NumComp(mant='1.3556', exp=-6)

Context managers¶

class putil.exdoc.ExDocCxt(exclude=None, pickle_fname=None, in_callables_fname=None, out_callables_fname=None)¶

Bases: object

Context manager to simplify exception tracing; it sets up the tracing environment and returns a putil.exdoc.ExDoc object that can the be used in the documentation string of each callable to extract the exceptions documentation with either putil.exdoc.ExDoc.get_sphinx_doc() or putil.exdoc.ExDoc.get_sphinx_autodoc().

Parameters:

exclude (list of strings or None) – Module exclusion list. A particular callable in an otherwise fully qualified name is omitted if it belongs to a module in this list. If None all callables are included pickle_fname (FileName or None) – File name to pickle traced exception handler (useful for debugging purposes). If None all pickle file is created in_callables_fname (FileNameExists or None) – File name that contains traced modules information. File can be produced by either the putil.pinspect.Callables.save() or putil.exh.ExHandle.save_callables() methods out_callables_fname (FileNameExists or None) – File name to save traced modules information to in JSON format. If the file exists it is overwritten

Raises:

OSError (File [in_callables_fname] could not be found) RuntimeError (Argument `in_callables_fname` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `out_callables_fname` is not valid) RuntimeError (Argument `pickle_fname` is not valid)

For example:

>>> from __future__ import print_function
>>> import putil.eng, putil.exdoc
>>> with putil.exdoc.ExDocCxt() as exdoc_obj:
...     value = putil.eng.peng(1e6, 3, False)
>>> print(exdoc_obj.get_sphinx_doc('putil.eng.peng'))
.. Auto-generated exceptions documentation for putil.eng.peng

:raises:
 * RuntimeError (Argument \`frac_length\` is not valid)

 * RuntimeError (Argument \`number\` is not valid)

 * RuntimeError (Argument \`rjust\` is not valid)

Classes¶

class putil.exdoc.ExDoc(exh_obj, depth=None, exclude=None)¶

Bases: object

Generates exception documentation with reStructuredText mark-up

Parameters:	exh_obj (putil.exh.ExHandle) – Exception handler containing exception information for the callable(s) to be documented depth (non-negative integer or None) – Default hierarchy levels to include in the exceptions per callable (see putil.exdoc.ExDoc.depth). If None exceptions at all depths are included exclude (list of strings or None) – Default list of (potentially partial) module and callable names to exclude from exceptions per callable (see putil.exdoc.ExDoc.exclude). If None all callables are included
Return type:	putil.exdoc.ExDoc
Raises:	RuntimeError (Argument `depth` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `exh_obj` is not valid) RuntimeError (Exceptions database is empty) RuntimeError (Exceptions do not have a common callable) ValueError (Object of argument `exh_obj` does not have any exception trace information)

get_sphinx_autodoc(depth=None, exclude=None, width=72, error=False, raised=False, no_comment=False)¶

Returns an exception list marked up in reStructuredText automatically determining callable name

Parameters:

depth (non-negative integer or None) – Hierarchy levels to include in the exceptions list (overrides default depth argument; see putil.exdoc.ExDoc.depth). If None exceptions at all depths are included exclude (list of strings or None) – List of (potentially partial) module and callable names to exclude from exceptions list (overrides default exclude argument, see putil.exdoc.ExDoc.exclude). If None all callables are included width (integer) – Maximum width of the lines of text (minimum 40) error (boolean) – Flag that indicates whether an exception should be raised if the callable is not found in the callables exceptions database (True) or not (False) raised (boolean) – Flag that indicates whether only exceptions that were raised (and presumably caught) should be documented (True) or all registered exceptions should be documented (False) no_comment (boolean) – Flag that indicates whether a reStructuredText comment labeling the callable (method, function or class property) should be printed (False) or not (True) before the exceptions documentation

Raises:

RuntimeError (Argument `depth` is not valid) RuntimeError (Argument `error` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `no_comment` is not valid) RuntimeError (Argument `raised` is not valid) RuntimeError (Argument `width` is not valid) RuntimeError (Callable not found in exception list: [name]) RuntimeError (Unable to determine callable name)

get_sphinx_doc(name, depth=None, exclude=None, width=72, error=False, raised=False, no_comment=False)¶

Returns an exception list marked up in reStructuredText

Parameters:

name (string) – Name of the callable (method, function or class property) to generate exceptions documentation for depth (non-negative integer or None) – Hierarchy levels to include in the exceptions list (overrides default depth argument; see putil.exdoc.ExDoc.depth). If None exceptions at all depths are included exclude (list of strings or None) – List of (potentially partial) module and callable names to exclude from exceptions list (overrides default exclude argument; see putil.exdoc.ExDoc.exclude). If None all callables are included width (integer) – Maximum width of the lines of text (minimum 40) error (boolean) – Flag that indicates whether an exception should be raised if the callable is not found in the callables exceptions database (True) or not (False) raised (boolean) – Flag that indicates whether only exceptions that were raised (and presumably caught) should be documented (True) or all registered exceptions should be documented (False) no_comment (boolean) – Flag that indicates whether a reStructuredText comment labeling the callable (method, function or class property) should be printed (False) or not (True) before the exceptions documentation

Raises:

RuntimeError (Argument `depth` is not valid) RuntimeError (Argument `error` is not valid) RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `no_comment` is not valid) RuntimeError (Argument `raised` is not valid) RuntimeError (Argument `width` is not valid) RuntimeError (Callable not found in exception list: [name])

depth¶

Gets or sets the default hierarchy levels to include in the exceptions per callable. For example, a function my_func() calls two other functions, get_data() and process_data(), and in turn get_data() calls another function, open_socket(). In this scenario, the calls hierarchy is:

my_func            <- depth = 0
├get_data          <- depth = 1
│└open_socket      <- depth = 2
└process_data      <- depth = 1

Setting depth=0 means that only exceptions raised by my_func() are going to be included in the documentation; Setting depth=1 means that only exceptions raised by my_func(), get_data() and process_data() are going to be included in the documentation; and finally setting depth=2 (in this case it has the same effects as depth=None) means that only exceptions raised by my_func(), get_data(), process_data() and open_socket() are going to be included in the documentation.

Return type:	non-negative integer
Raises:	RuntimeError (Argument `depth` is not valid)

exclude¶

Gets or sets the default list of (potentially partial) module and callable names to exclude from exceptions per callable. For example, ['putil.ex'] excludes all exceptions from modules putil.exh and putil.exdoc (it acts as r'putil.ex*'). In addition to these modules, ['putil.ex', 'putil.eng.peng'] excludes exceptions from the function putil.eng.peng().

Return type:	list
Raises:	RuntimeError (Argument `exclude` is not valid)

Functions¶

putil.exh.addex(extype, exmsg, condition=None, edata=None)¶

Adds an exception in the global exception handler

Parameters:	extype (Exception type object, i.e. RuntimeError, TypeError, etc.) – Exception type; must be derived from the Exception class exmsg (string) – Exception message; it can contain fields to be replaced when the exception is raised via putil.exh.ExHandle.raise_exception_if(). A field starts with the characters '*[' and ends with the characters ']*', the field name follows the same rules as variable names and is between these two sets of characters. For example, '*[fname]*' defines the fname field condition (boolean or None) – Flag that indicates whether the exception is raised (True) or not (False). If None the flag is not used an no exception is raised edata – Replacement values for fields in the exception message (see putil.exh.ExHandle.add_exception() for how to define fields). Each dictionary entry can only have these two keys: field (string) – Field name value (any) – Field value, to be converted into a string with the format string method If None no field replacement is done
Return type:	(if condition is not given or None) function
Raises:	RuntimeError (Argument `condition` is not valid) RuntimeError (Argument `edata` is not valid) RuntimeError (Argument `exmsg` is not valid) RuntimeError (Argument `extype` is not valid)

putil.exh.addai(argname, condition=None)¶

Adds an exception of the type RuntimeError('Argument `*[argname]*` is not valid') in the global exception handler where *[argname]* is the value of the argname argument

Parameters:	argname (string) – Argument name condition (boolean or None) – Flag that indicates whether the exception is raised (True) or not (False). If None the flag is not used and no exception is raised
Return type:	(if condition is not given or None) function
Raises:	RuntimeError (Argument `argname` is not valid) RuntimeError (Argument `condition` is not valid)

putil.exh.get_exh_obj()¶

Returns the global exception handler

Return type:	putil.exh.ExHandle if global exception handler is set, None otherwise

putil.exh.get_or_create_exh_obj(full_cname=False, exclude=None, callables_fname=None)¶

Returns the global exception handler if it is set, otherwise creates a new global exception handler and returns it

Parameters:	full_cname (boolean) – Flag that indicates whether fully qualified function/method/class property names are obtained for functions/methods/class properties that use the exception manager (True) or not (False). There is a performance penalty if the flag is True as the call stack needs to be traced. This argument is only relevant if the global exception handler is not set and a new one is created exclude (list of strings or None) – Module exclusion list. A particular callable in an otherwise fully qualified name is omitted if it belongs to a module in this list. If None all callables are included callables_fname (FileNameExists or None) – File name that contains traced modules information. File can be produced by either the putil.pinspect.Callables.save() or putil.exh.ExHandle.save_callables() methods
Return type:	putil.exh.ExHandle
Raises:	OSError (File [callables_fname] could not be found RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `callables_fname` is not valid) RuntimeError (Argument `full_cname` is not valid)

putil.exh.del_exh_obj()¶

Deletes global exception handler (if set)

putil.exh.set_exh_obj(obj)¶

Sets the global exception handler

Parameters:	obj (putil.exh.ExHandle) – Exception handler
Raises:	RuntimeError (Argument `obj` is not valid)

Classes¶

class putil.exh.ExHandle(full_cname=False, exclude=None, callables_fname=None)¶

Bases: object

Exception handler

Parameters:	full_cname (boolean) – Flag that indicates whether fully qualified function/method/class property names are obtained for functions/methods/class properties that use the exception manager (True) or not (False). There is a performance penalty if the flag is True as the call stack needs to be traced exclude (list of strings or None) – Module exclusion list. A particular callable in an otherwise fully qualified name is omitted if it belongs to a module in this list. If None all callables are included callables_fname (FileNameExists or None) – File name that contains traced modules information. File can be produced by either the putil.pinspect.Callables.save() or putil.exh.ExHandle.save_callables() methods
Return type:	putil.exh.ExHandle
Raises:	OSError (File [callables_fname] could not be found RuntimeError (Argument `exclude` is not valid) RuntimeError (Argument `callables_fname` is not valid) RuntimeError (Argument `full_cname` is not valid) ValueError (Source for module [module_name] could not be found)

__add__(other)¶

Merges two objects.

Raises:	RuntimeError (Incompatible exception handlers) TypeError (Unsupported operand type(s) for +: putil.exh.ExHandle and [other_type])

For example:

>>> import copy, putil.exh, putil.eng, putil.tree
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj1 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> putil.eng.peng(100, 3, True) # Trace some exceptions
' 100.000 '
>>> obj2 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj3 = copy.copy(putil.exh.get_exh_obj())
>>> obj1 == obj2
False
>>> obj1 == obj3
False
>>> obj1 == obj2+obj3
True

__bool__()¶

Returns False if exception handler does not have any exception defined, True otherwise.

Note

This method applies to Python 3.x

For example:

>>> from __future__ import print_function
>>> import putil.exh
>>> obj = putil.exh.ExHandle()
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: False
>>> def my_func(exhobj):
...     exhobj.add_exception('test', RuntimeError, 'Message')
>>> my_func(obj)
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: True

__copy__()¶

Copies object. For example:

>>> import copy, putil.exh, putil.eng
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> obj1 = putil.exh.get_exh_obj()
>>> obj2 = copy.copy(obj1)
>>> obj1 == obj2
True

__eq__(other)¶

Tests object equality. For example:

>>> import copy, putil.exh, putil.eng
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> obj1 = putil.exh.get_exh_obj()
>>> obj2 = copy.copy(obj1)
>>> obj1 == obj2
True
>>> 5 == obj1
False

__iadd__(other)¶

Merges an object into an existing object.

Raises:	RuntimeError (Incompatible exception handlers) TypeError (Unsupported operand type(s) for +: putil.exh.ExHandle and [other_type])

For example:

>>> import copy, putil.exh, putil.eng, putil.tree
>>> exhobj = putil.exh.set_exh_obj(putil.exh.ExHandle())
>>> putil.eng.peng(100, 3, True)
' 100.000 '
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj1 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> putil.eng.peng(100, 3, True) # Trace some exceptions
' 100.000 '
>>> obj2 = copy.copy(putil.exh.get_exh_obj())
>>> putil.exh.del_exh_obj()
>>> exhobj = putil.exh.get_or_create_exh_obj()
>>> tobj = putil.tree.Tree().add_nodes([{'name':'a', 'data':5}])
>>> obj3 = copy.copy(putil.exh.get_exh_obj())
>>> obj1 == obj2
False
>>> obj1 == obj3
False
>>> obj2 += obj3
>>> obj1 == obj2
True

__nonzero__()¶

Returns False if exception handler does not have any exception defined, True otherwise.

Note

This method applies to Python 2.7

For example:

>>> from __future__ import print_function
>>> import putil.exh
>>> obj = putil.exh.ExHandle()
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: False
>>> def my_func(exhobj):
...     exhobj.add_exception('test', RuntimeError, 'Message')
>>> my_func(obj)
>>> if obj:
...     print('Boolean test returned: True')
... else:
...     print('Boolean test returned: False')
Boolean test returned: True

__str__()¶

Returns a string with a detailed description of the object’s contents. For example:

>>> from __future__ import print_function
>>> import docs.support.exh_example
>>> putil.exh.del_exh_obj()
>>> docs.support.exh_example.my_func('Tom')
My name is Tom
>>> print(str(putil.exh.get_exh_obj())) 
Name    : ...
Type    : TypeError
Message : Argument `name` is not valid
Function: None

add_exception(exname, extype, exmsg)¶

Adds an exception to the handler

Parameters:

exname (non-numeric string) – Exception name; has to be unique within the namespace, duplicates are eliminated extype (Exception type object, i.e. RuntimeError, TypeError, etc.) – Exception type; must be derived from the Exception class exmsg (string) – Exception message; it can contain fields to be replaced when the exception is raised via putil.exh.ExHandle.raise_exception_if(). A field starts with the characters '*[' and ends with the characters ']*', the field name follows the same rules as variable names and is between these two sets of characters. For example, '*[fname]*' defines the fname field

Return type:

tuple

The returned tuple has the following items:

• callable id (string) first returned item, identification (as reported by the id built-in function) of the callable where the exception was added
• exception definition (tuple), second returned item, first item is the exception type and the second item is the exception message
• callable name (string), third returned item, callable full name (encoded with the ExHandle.encode_call() method

Raises:	RuntimeError (Argument `exmsg` is not valid) RuntimeError (Argument `exname` is not valid) RuntimeError (Argument `extype` is not valid)

decode_call(call)¶

Replaces callable tokens with callable names

Parameters:	call (string) – Encoded callable name
Return type:	string

encode_call(call)¶

Replaces callables with tokens to reduce object memory footprint. A callable token is an integer that denotes the order in which the callable was encountered by the encoder, i.e. the first callable encoded is assigned token 0, the second callable encoded is assigned token 1, etc.

Parameters:	call (string) – Callable name
Return type:	string

raise_exception_if(exname, condition, edata=None)¶

Raises exception conditionally

Parameters:

exname (string) – Exception name condition (boolean) – Flag that indicates whether the exception is raised (True) or not (False) edata (dictionary, iterable of dictionaries or None) – Replacement values for fields in the exception message (see putil.exh.ExHandle.add_exception() for how to define fields). Each dictionary entry can only have these two keys: field (string) – Field name value (any) – Field value, to be converted into a string with the format string method If None no field replacement is done

Raises:

RuntimeError (Argument `condition` is not valid) RuntimeError (Argument `edata` is not valid) RuntimeError (Argument `exname` is not valid) RuntimeError (Field [field_name] not in exception message) ValueError (Exception name [name] not found’)

save_callables(callables_fname)¶

Saves traced modules information to a JSON file. If the file exists it is overwritten

Parameters:	callables_fname (FileName) – File name
Raises:	RuntimeError (Argument `callables_fname` is not valid)

callables_db¶

Returns the callables database of the modules using the exception handler, as reported by putil.pinspect.Callables.callables_db()

callables_separator¶

Returns the character ('/') used to separate the sub-parts of fully qualified function names in putil.exh.ExHandle.callables_db() and name key in putil.exh.ExHandle.exceptions_db()

exceptions_db¶

Returns the exceptions database. This database is a list of dictionaries that contain the following keys:

• name (string) – Exception name of the form '[callable_identifier]/[exception_name]'. The contents of [callable_identifier] depend on the value of the argument full_cname used to create the exception handler.

  If full_cname is True, [callable_identifier] is the fully qualified callable name as it appears in the callables database (putil.exh.ExHandle.callables_db()).

  If full_cname is False, then [callable_identifier] is a decimal string representation of the callable’s code identifier as reported by the id() function.

  In either case [exception_name] is the name of the exception provided when it was defined in putil.exh.ExHandle.add_exception() (exname argument)

• data (string) – Text of the form '[exception_type] ([exception_message])[raised]' where [exception_type] and [exception_message] are the exception type and exception message, respectively, given when the exception was defined by putil.exh.ExHandle.add_exception() (extype and exmsg arguments); and raised is an asterisk ('*') when the exception has been raised via putil.exh.ExHandle.raise_exception_if(), the empty string ('') otherwise

Context managers¶

putil.misc.ignored(*exceptions)¶

Executes commands and selectively ignores exceptions (Inspired by “Transforming Code into Beautiful, Idiomatic Python” talk at PyCon US 2013 by Raymond Hettinger)

Parameters:	exceptions (Exception object, i.e. RuntimeError, OSError, etc.) – Exception type(s) to ignore

For example:

# misc_example_1.py
from __future__ import print_function
import os, putil.misc

def ignored_example():
    fname = 'somefile.tmp'
    open(fname, 'w').close()
    print('File {0} exists? {1}'.format(
        fname, os.path.isfile(fname)
    ))
    with putil.misc.ignored(OSError):
        os.remove(fname)
    print('File {0} exists? {1}'.format(
        fname, os.path.isfile(fname)
    ))
    with putil.misc.ignored(OSError):
        os.remove(fname)
    print('No exception trying to remove a file that does not exists')
    try:
        with putil.misc.ignored(RuntimeError):
            os.remove(fname)
    except:
        print('Got an exception')

>>> import docs.support.misc_example_1
>>> docs.support.misc_example_1.ignored_example()
File somefile.tmp exists? True
File somefile.tmp exists? False
No exception trying to remove a file that does not exists
Got an exception

class putil.misc.Timer(verbose=False)¶

Bases: object

Profiles blocks of code by calculating elapsed time between the context manager entry and exit time points. Inspired by Huy Nguyen’s blog

Parameters:	verbose (boolean) – Flag that indicates whether the elapsed time is printed upon exit (True) or not (False)
Returns:	putil.misc.Timer
Raises:	RuntimeError (Argument `verbose` is not valid)

For example:

# misc_example_2.py
from __future__ import print_function
import putil.misc

def timer(num_tries, fpointer):
    with putil.misc.Timer() as tobj:
        for _ in range(num_tries):
            fpointer()
    print('Time per call: {0} seconds'.format(
        tobj.elapsed_time/(2.0*num_tries)
    ))

def sample_func():
    count = 0
    for num in range(0, count):
        count += num

>>> from docs.support.misc_example_2 import *
>>> timer(100, sample_func) 
Time per call: ... seconds

elapsed_time¶

Returns elapsed time (in seconds) between context manager entry and exit time points

Return type:	float

class putil.misc.TmpFile(fpointer=None)¶

Bases: object

Creates a temporary file and optionally sets up hooks for a function to write data to it

Parameters:	fpointer (function object or None) – Pointer to a function that writes data to file. If the argument is not None the function pointed to receives exactly one argument, a file-like object
Returns:	temporary file name
Raises:	RuntimeError (Argument `fpointer` is not valid)

Warning

The file name returned uses the forward slash (/) as the path separator regardless of the platform. This avoids problems with escape sequences or mistaken Unicode character encodings (\\user for example). Many functions in the os module of the standard library ( os.path.normpath() and others) can change this path separator to the operating system path separator if needed

For example:

# misc_example_3.py
from __future__ import print_function
import putil.misc

def write_data(file_handle):
    file_handle.write('Hello world!')

def show_tmpfile():
    with putil.misc.TmpFile(write_data) as fname:
        with open(fname, 'r') as fobj:
            lines = fobj.readlines()
    print('\n'.join(lines))

>>> from docs.support.misc_example_3 import *
>>> show_tmpfile()
Hello world!

File¶

putil.misc.make_dir(fname)¶

Creates the directory of a fully qualified file name if it does not exist

Parameters:	fname (string) – File name

Equivalent to these Bash shell commands:

$ dir=$(dirname ${fname})
$ mkdir -p ${dir}

Parameters:	fname (string) – Fully qualified file name

putil.misc.normalize_windows_fname(fname)¶

Fix potential problems with a Microsoft Windows file name. Superfluous backslashes are removed and unintended escape sequences are converted to their equivalent (presumably correct and intended) representation, for example r'\x07pps' is transformed to r'\\apps'. A file name is considered network shares if the file does not include a drive letter and they start with a double backslash ('\\')

Parameters:	fname (string) – File name
Return type:	string

Membership¶

putil.misc.isalpha(obj)¶

Tests if the argument is a string representing a number

Parameters:	obj (any) – Object
Return type:	boolean

For example:

>>> import putil.misc
>>> putil.misc.isalpha('1.5')
True
>>> putil.misc.isalpha('1E-20')
True
>>> putil.misc.isalpha('1EA-20')
False

putil.misc.ishex(obj)¶

Tests if the argument is a string representing a valid hexadecimal digit

Parameters:	obj (any) – Object
Return type:	boolean

putil.misc.isiterable(obj)¶

Tests if the argument is an iterable

Parameters:	obj (any) – Object
Return type:	boolean

putil.misc.isnumber(obj)¶

Tests if the argument is a number (complex, float or integer)

Parameters:	obj (any) – Object
Return type:	boolean

putil.misc.isreal(obj)¶

Tests if the argument is a real number (float or integer)

Parameters:	obj (any) – Object
Return type:	boolean

Miscellaneous¶

putil.misc.flatten_list(lobj)¶

Recursively flattens a list

Parameters:	lobj (list) – List to flatten
Return type:	list

For example:

>>> import putil.misc
>>> putil.misc.flatten_list([1, [2, 3, [4, 5, 6]], 7])
[1, 2, 3, 4, 5, 6, 7]

Numbers¶

putil.misc.gcd(vector)¶

Calculates the greatest common divisor (GCD) of a list of numbers or a Numpy vector of numbers. The computations are carried out with a precision of 1E-12 if the objects are not fractions. When possible it is best to use the fractions data type with the numerator and denominator arguments when computing the GCD of floating point numbers.

Parameters:	vector (list of numbers or Numpy vector of numbers) – Vector of numbers

putil.misc.normalize(value, series, offset=0)¶

Scales a value to the range defined by a series

Parameters:	value (number) – Value to normalize series (list) – List of numbers that defines the normalization range offset (number) – Normalization offset, i.e. the returned value will be in the range [offset, 1.0]
Return type:	number
Raises:	RuntimeError (Argument `offset` is not valid) RuntimeError (Argument `series` is not valid) RuntimeError (Argument `value` is not valid) ValueError (Argument `offset` has to be in the [0.0, 1.0] range) ValueError (Argument `value` has to be within the bounds of the argument `series`)

For example:

>>> import putil.misc
>>> putil.misc.normalize(15, [10, 20])
0.5
>>> putil.misc.normalize(15, [10, 20], 0.5)
0.75

putil.misc.per(arga, argb, prec=10)¶

Calculates the percentage difference between two numbers or the element-wise percentage difference between two lists of numbers or Numpy vectors. If any of the numbers in the arguments is zero the value returned is 1E+20

Parameters:	arga (float, integer, list of floats or integers, or Numpy vector of floats or integers) – First number, list of numbers or Numpy vector argb (float, integer, list of floats or integers, or Numpy vector of floats or integers) – Second number, list of numbers or or Numpy vector prec (integer) – Maximum length of the fractional part of the result
Return type:	Float, list of floats or Numpy vector, depending on the arguments type
Raises:	RuntimeError (Argument `arga` is not valid) RuntimeError (Argument `argb` is not valid) RuntimeError (Argument `prec` is not valid) TypeError (Arguments are not of the same type)

putil.misc.pgcd(numa, numb)¶

Calculate the greatest common divisor (GCD) of two numbers

Parameters:	numa (number) – First number numb (number) – Second number
Return type:	number

For example:

>>> import putil.misc, fractions
>>> putil.misc.pgcd(10, 15)
5
>>> str(putil.misc.pgcd(0.05, 0.02))
'0.01'
>>> str(putil.misc.pgcd(5/3.0, 2/3.0))[:6]
'0.3333'
>>> putil.misc.pgcd(
...     fractions.Fraction(str(5/3.0)),
...     fractions.Fraction(str(2/3.0))
... )
Fraction(1, 3)
>>> putil.misc.pgcd(
...     fractions.Fraction(5, 3),
...     fractions.Fraction(2, 3)
... )
Fraction(1, 3)

String¶

putil.misc.binary_string_to_octal_string(text)¶

Returns a binary-packed string in octal representation aliasing typical codes to their escape sequences

Parameters:	text (string) – Text to convert
Return type:	string

Code	Alias	Description
0	\0	Null character
7	\a	Bell / alarm
8	\b	Backspace
9	\t	Horizontal tab
10	\n	Line feed
11	\v	Vertical tab
12	\f	Form feed
13	\r	Carriage return

For example:

>>> import putil.misc, struct, sys
>>> def py23struct(num):
...    if sys.hexversion < 0x03000000:
...        return struct.pack('h', num)
...    else:
...        return struct.pack('h', num).decode('ascii')
>>> nums = range(1, 15)
>>> putil.misc.binary_string_to_octal_string(
...     ''.join([py23struct(num) for num in nums])
... ).replace('o', '')  
'\\1\\0\\2\\0\\3\\0\\4\\0\\5\\0\\6\\0\\a\\0\\b\\0\\t\\0\\...

putil.misc.char_to_decimal(text)¶

Converts a string to its decimal ASCII representation, with spaces between characters

Parameters:	text (string) – Text to convert
Return type:	string

For example:

>>> import putil.misc
>>> putil.misc.char_to_decimal('Hello world!')
'72 101 108 108 111 32 119 111 114 108 100 33'

putil.misc.elapsed_time_string(start_time, stop_time)¶

Returns a formatted string with the elapsed time between two time points. The string includes years (365 days), months (30 days), days (24 hours), hours (60 minutes), minutes (60 seconds) and seconds. If both arguments are equal, the string returned is 'None'; otherwise, the string returned is [YY year[s], [MM month[s], [DD day[s], [HH hour[s], [MM minute[s] [and SS second[s]]]]]]. Any part (year[s], month[s], etc.) is omitted if the value of that part is null/zero

Parameters:	start_time (datetime) – Starting time point stop_time (datetime) – Ending time point
Return type:	string
Raises:	RuntimeError (Invalid time delta specification)

For example:

>>> import datetime, putil.misc
>>> start_time = datetime.datetime(2014, 1, 1, 1, 10, 1)
>>> stop_time = datetime.datetime(2015, 1, 3, 1, 10, 3)
>>> putil.misc.elapsed_time_string(start_time, stop_time)
'1 year, 2 days and 2 seconds'

putil.misc.pcolor(text, color, indent=0)¶

Returns a string that once printed is colorized

Parameters:	text (string) – Text to colorize color (string) – Color to use, one of 'black', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'white' or 'none' (case insensitive) indent (integer) – Number of spaces to prefix the output with
Return type:	string
Raises:	RuntimeError (Argument `color` is not valid) RuntimeError (Argument `indent` is not valid) RuntimeError (Argument `text` is not valid) ValueError (Unknown color [color])

putil.misc.quote_str(obj)¶

Adds extra quotes to a string. If the argument is not a string it is returned unmodified

Parameters:	obj (any) – Object
Return type:	Same as argument

For example:

>>> import putil.misc
>>> putil.misc.quote_str(5)
5
>>> putil.misc.quote_str('Hello!')
'"Hello!"'
>>> putil.misc.quote_str('He said "hello!"')
'\'He said "hello!"\''

putil.misc.strframe(obj, extended=False)¶

Returns a string with a frame record (typically an item in a list generated by inspect.stack()) pretty printed

Parameters:	obj (tuple) – Frame record extended (boolean) – Flag that indicates whether contents of the frame object are printed (True) or not (False)
Return type:	string

Functions¶

putil.pcontracts.all_disabled()¶

Wraps PyContracts all_disabled() function. From the PyContracts documentation: “Returns true if all contracts are disabled”

putil.pcontracts.disable_all()¶

Wraps PyContracts disable_all() function. From the PyContracts documentation: “Disables all contract checks”

putil.pcontracts.enable_all()¶

Wraps PyContracts enable_all() function. From the PyContracts documentation: “Enables all contract checks. Can be overridden by an environment variable”

putil.pcontracts.get_exdesc()¶

Retrieves the contract exception(s) message(s). If the custom contract is specified with only one exception the return value is the message associated with that exception; if the custom contract is specified with several exceptions, the return value is a dictionary whose keys are the exception names and whose values are the exception messages.

Raises:	RuntimeError (Function object could not be found for function [function_name])
Return type:	string or dictionary

For example:

# pcontracts_example_2.py
import putil.pcontracts

@putil.pcontracts.new_contract('Only one exception')
def custom_contract_a(name):
    msg = putil.pcontracts.get_exdesc()
    if not name:
        raise ValueError(msg)

@putil.pcontracts.new_contract(ex1='Empty name', ex2='Invalid name')
def custom_contract_b(name):
    msg = putil.pcontracts.get_exdesc()
    if not name:
        raise ValueError(msg['ex1'])
    elif name.find('[') != -1:
        raise ValueError(msg['ex2'])

In custom_contract1() the variable msg contains the string 'Only one exception', in custom_contract2() the variable msg contains the dictionary {'ex1':'Empty name', 'ex2':'Invalid name'}.

Decorators¶

putil.pcontracts.contract(**contract_args)¶

Wraps PyContracts contract() decorator (only the decorator way of specifying a contract is supported and tested). A RuntimeError exception with the message 'Argument `*[argument_name]*` is not valid' is raised when a contract is breached ('*[argument_name]*' is replaced by the argument name the contract is attached to) unless the contract is custom and specified with the putil.pcontracts.new_contract() decorator. In this case the exception type and message are controlled by the custom contract specification.

putil.pcontracts.new_contract(*args, **kwargs)¶

Defines a new (custom) contract with custom exceptions.

Raises:

RuntimeError (Attempt to redefine custom contract `*[contract_name]*`) TypeError (Argument `contract_exceptions` is of the wrong type) TypeError (Argument `contract_name` is of the wrong type) TypeError (Contract exception definition is of the wrong type) TypeError (Illegal custom contract exception definition) ValueError (Empty custom contract exception message) ValueError (Contract exception messages are not unique) ValueError (Contract exception names are not unique) ValueError (Multiple replacement fields to be substituted by argument value)

The decorator argument(s) is(are) the exception(s) that can be raised by the contract. The most general way to define an exception is using a 2-item tuple with the following members:

• exception type (type) – Either a built-in exception or sub-classed from Exception. Default is RuntimeError
• exception message (string) – Default is 'Argument `*[argument_name]*` is not valid', where the token *[argument_name]* is replaced by the argument name the contract is attached to

The order of the tuple elements is not important, i.e. the following are valid exception specifications and define the same exception:

@putil.pcontracts.new_contract(ex1=(RuntimeError, 'Invalid name'))
def custom_contract1(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.new_contract(ex1=('Invalid name', RuntimeError))
def custom_contract2(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

The exception definition simplifies to just one of the exception definition tuple items if the other exception definition tuple item takes its default value. For example, the same exception is defined in these two contracts:

@putil.pcontracts.new_contract(ex1=ValueError)
def custom_contract3(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.new_contract(ex1=(
    ValueError,
    'Argument `*[argument_name]*` is not valid'
))
def custom_contract4(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

and these contracts also define the same exception (but different from that of the previous example):

@putil.pcontracts.new_contract(ex1='Invalid name')
def custom_contract5(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.new_contract(ex1=('Invalid name', RuntimeError))
def custom_contract6(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

In fact the exception need not be specified by keyword if the contract only uses one exception. All of the following are valid one-exception contract specifications:

@putil.pcontracts.new_contract(
    (OSError, 'File could not be opened')
)
def custom_contract7(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.new_contract('Invalid name')
def custom_contract8(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.new_contract(TypeError)
def custom_contract9(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

No arguments are needed if a contract only needs a single exception and the default exception type and message suffice:

@putil.pcontracts.new_contract()
def custom_contract10(arg):
    if not arg:
        raise ValueError(putil.pcontracts.get_exdesc())

For code conciseness and correctness the exception message(s) should be retrieved via the putil.pcontracts.get_exdesc() function.

A PyContracts new contract can return False or raise a ValueError exception to indicate a contract breach, however a new contract specified via the putil.pcontracts.new_contract() decorator has to raise a ValueError exception to indicate a contract breach.

The exception message can have substitution “tokens” of the form *[token_name]*. The token *[argument_name]* is substituted with the argument name the contract it is attached to. For example:

@putil.pcontracts.new_contract((
    TypeError,
    'Argument `*[argument_name]*` has to be a string'
))
def custom_contract11(city):
    if not isinstance(city, str):
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.contract(city_name='custom_contract11')
def print_city_name(city_name):
    return 'City: {0}'.format(city_name)

>>> from __future__ import print_function
>>> from docs.support.pcontracts_example_3 import print_city_name
>>> print(print_city_name('Omaha'))
City: Omaha
>>> print(print_city_name(5))   
Traceback (most recent call last):
    ...
TypeError: Argument `city_name` has to be a string

Any other token is substituted with the argument value. For example:

@putil.pcontracts.new_contract((
    OSError, 'File `*[fname]*` not found'
))
def custom_contract12(fname):
    if not os.path.exists(fname):
        raise ValueError(putil.pcontracts.get_exdesc())

@putil.pcontracts.contract(fname='custom_contract12')
def print_fname(fname):
    print('File name to find: {0}'.format(fname))

>>> from __future__ import print_function
>>> import os
>>> from docs.support.pcontracts_example_3 import print_fname
>>> fname = os.path.join(os.sep, 'dev', 'null', '_not_a_file_')
>>> print(print_fname(fname))   
Traceback (most recent call last):
    ...
OSError: File `..._not_a_file_` not found

Examples¶

# pcsv_example_1.py
import putil.misc, putil.pcsv

def main():
    with putil.misc.TmpFile() as fname:
        ref_data = [
            ['Item', 'Cost'],
            [1, 9.99],
            [2, 10000],
            [3, 0.10]
        ]
        # Write reference data to a file
        putil.pcsv.write(fname, ref_data, append=False)
        # Read the data back
        obj = putil.pcsv.CsvFile(fname)
    # After the object creation the I/O is done,
    # can safely remove file (exit context manager)
    # Check that data read is correct
    assert obj.header() == ref_data[0]
    assert obj.data() == ref_data[1:]
    # Add a simple row filter, only look at rows that have
    # values 1 and 3 in the "Items" column
    obj.rfilter = {'Item':[1, 3]}
    assert obj.data(filtered=True) == [ref_data[1], ref_data[3]]

if __name__ == '__main__':
    main()

# pcsv_example_2.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as fname1:
        with ctx() as fname2:
            with ctx() as ofname:
                # Create first (input) data file
                input_data = [
                    ['Item', 'Cost'],
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10]
                ]
                putil.pcsv.write(fname1, input_data, append=False)
                # Create second (replacement) data file
                replacement_data = [
                    ['Staff', 'Rate', 'Days'],
                    ['Joe', 10, 'Sunday'],
                    ['Sue', 20, 'Thursday'],
                    ['Pat', 15, 'Tuesday']
                ]
                putil.pcsv.write(fname2, replacement_data, append=False)
                # Replace "Cost" column of input file with "Rate" column
                # of replacement file for "Items" 2 and 3 with "Staff" data
                # from Joe and Pat. Save resulting data to another file
                putil.pcsv.replace(
                    fname1=fname1,
                    dfilter1=('Cost', {'Item':[1, 3]}),
                    fname2=fname2,
                    dfilter2=('Rate', {'Staff':['Joe', 'Pat']}),
                    ofname=ofname
                )
                # Verify that resulting file is correct
                ref_data = [
                    ['Item', 'Cost'],
                    [1, 10],
                    [2, 10000],
                    [3, 15]
                ]
                obj = putil.pcsv.CsvFile(ofname)
                assert obj.header() == ref_data[0]
                assert obj.data() == ref_data[1:]

if __name__ == '__main__':
    main()

# pcsv_example_3.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as fname1:
        with ctx() as fname2:
            with ctx() as ofname:
                # Create first data file
                data1 = [
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10]
                ]
                putil.pcsv.write(fname1, data1, append=False)
                # Create second data file
                data2 = [
                    ['Joe', 10, 'Sunday'],
                    ['Sue', 20, 'Thursday'],
                    ['Pat', 15, 'Tuesday']
                ]
                putil.pcsv.write(fname2, data2, append=False)
                # Concatenate file1 and file2. Filter out
                # second column of file2
                putil.pcsv.concatenate(
                    fname1=fname1,
                    fname2=fname2,
                    has_header1=False,
                    has_header2=False,
                    dfilter2=[0, 2],
                    ofname=ofname,
                    ocols=['D1', 'D2']
                )
                # Verify that resulting file is correct
                ref_data = [
                    ['D1', 'D2'],
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10],
                    ['Joe', 'Sunday'],
                    ['Sue', 'Thursday'],
                    ['Pat', 'Tuesday']
                ]
                obj = putil.pcsv.CsvFile(ofname)
                assert obj.header() == ref_data[0]
                assert obj.data() == ref_data[1:]

if __name__ == '__main__':
    main()

# pcsv_example_4.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as fname1:
        with ctx() as fname2:
            with ctx() as ofname:
                # Create first data file
                data1 = [
                    [1, 9.99],
                    [2, 10000],
                    [3, 0.10]
                ]
                putil.pcsv.write(fname1, data1, append=False)
                # Create second data file
                data2 = [
                    ['Joe', 10, 'Sunday'],
                    ['Sue', 20, 'Thursday'],
                    ['Pat', 15, 'Tuesday']
                ]
                putil.pcsv.write(fname2, data2, append=False)
                # Merge file1 and file2
                putil.pcsv.merge(
                    fname1=fname1,
                    has_header1=False,
                    fname2=fname2,
                    has_header2=False,
                    ofname=ofname
                )
                # Verify that resulting file is correct
                ref_data = [
                    [1, 9.99, 'Joe', 10, 'Sunday'],
                    [2, 10000, 'Sue', 20, 'Thursday'],
                    [3, 0.10, 'Pat', 15, 'Tuesday'],
                ]
                obj = putil.pcsv.CsvFile(ofname, has_header=False)
                assert obj.header() == list(range(0, 5))
                assert obj.data() == ref_data

if __name__ == '__main__':
    main()

# pcsv_example_5.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as ifname:
        with ctx() as ofname:
            # Create first data file
            data = [
                ['Ctrl', 'Ref', 'Result'],
                [1, 3, 10],
                [1, 4, 20],
                [2, 4, 30],
                [2, 5, 40],
                [3, 5, 50]
            ]
            putil.pcsv.write(ifname, data, append=False)
            # Sort
            putil.pcsv.dsort(
                fname=ifname,
                order=[{'Ctrl':'D'}, {'Ref':'A'}],
                has_header=True,
                ofname=ofname
            )
            # Verify that resulting file is correct
            ref_data = [
                [3, 5, 50],
                [2, 4, 30],
                [2, 5, 40],
                [1, 3, 10],
                [1, 4, 20]
            ]
            obj = putil.pcsv.CsvFile(ofname, has_header=True)
            assert obj.header() == ['Ctrl', 'Ref', 'Result']
            assert obj.data() == ref_data

if __name__ == '__main__':
    main()

Identifying (filtering) columns¶

Several class methods and functions in this module allow column and row filtering of the CSV file data. It is necessary to identify columns for both of these operations and how these columns can be identified depends on whether the file has or does not have a header as indicated by the has_header boolean constructor argument:

• If has_header is True the first line of the file is taken as the header. Columns can be identified by name (a string that has to match a column value in the file header) or by number (an integer representing the column number with column zero being the leftmost column)
• If has_header is False columns can only be identified by number (an integer representing the column number with column zero being the leftmost column)

For example, if a file myfile.csv has the following data:

Then when the file is loaded with putil.pcsv.CsvFile('myfile.csv', has_header=True) the columns can be referred to as 'Ctrl' or 0, 'Ref' or 1, or 'Result' or 2. However if the file is loaded with putil.pcsv.CsvFile('myfile.csv', has_header=False) the columns can be referred only as 0, 1 or 2.

Filtering rows¶

Several class methods and functions of this module allow row filtering of the CSV file data. The row filter is described in the CsvRowFilter pseudo-type

Swapping or inserting columns¶

The column filter not only filters columns but also determines the order in which the columns are stored internally in an putil.pcsv.CsvFile object. This means that the column filter can be used to reorder and/or duplicate columns. For example:

# pcsv_example_6.py
import putil.misc, putil.pcsv

def main():
    ctx = putil.misc.TmpFile
    with ctx() as ifname:
        with ctx() as ofname:
            # Create input data file
            data = [
                ['Ctrl', 'Ref', 'Result'],
                [1, 3, 10],
                [1, 4, 20],
                [2, 4, 30],
                [2, 5, 40],
                [3, 5, 50]
            ]
            putil.pcsv.write(ifname, data, append=False)
            # Swap 'Ctrl' and 'Result' columns, duplicate
            # 'Ref' column at the end
            obj = putil.pcsv.CsvFile(
                fname=ifname,
                dfilter=['Result', 'Ref', 'Ctrl', 1],
            )
            assert obj.header(filtered=False) == ['Ctrl', 'Ref', 'Result']
            assert (
                obj.header(filtered=True)
                ==
                ['Result', 'Ref', 'Ctrl', 'Ref']
            )
            obj.write(
                ofname,
                header=['Result', 'Ref', 'Ctrl', 'Ref2'],
                filtered=True,
                append=False
            )
            # Verify that resulting file is correct
            ref_data = [
                [10, 3, 1, 3],
                [20, 4, 1, 4],
                [30, 4, 2, 4],
                [40, 5, 2, 5],
                [50, 5, 3, 5]
            ]
            obj = putil.pcsv.CsvFile(ofname, has_header=True)
            assert obj.header() == ['Result', 'Ref', 'Ctrl', 'Ref2']
            assert obj.data() == ref_data

if __name__ == '__main__':
    main()