CLI Argument Management¶

The system can handle managing arguments passed via the CLI as well as ones in config files or databases. When configuring an option, you have the ability to pass a dictionary of parameters to the option keyword “cli_option”. This dictionary will tell the system to allow that option to be configured by the CLI, defines the help for the cli, and tells the system how to handle the response.

• flags: (required) a flag or list of flags to be accepted. (these must be unique across the system).

  Note

  If only a single string is passed instead of a dictionary, it is assumed to be the flag and the rest of the data will be set at defaults.

• data_flag: If present, this will set the “store_true” or “store_false” action in argparse, this means that

  if True is set, and the cli flag is set, the option will be set to True, and vice-versa. If a default is used, it must be a boolean.

• nargs: [int] This will allow that number of arguments to be added to a list config item, so for example,

  if nargs 3 is passed, the following argument would be treated as one:

  program.py -arg 1 3 4
  

  This would return [‘1’,‘2’,‘3’], however if nargs 2 was passed, this would return [‘1’,‘2’] and the ‘3’ would be treated as another positional argument. If a default is used in this case, it must be a list and have this many items.

• choices: a list of choices that are available (and will be validated against). If a default is used, it

  must be present in the list.

• default: if not present, will use the option default, if present and None, will not have a default,

  if present and set to somethign, will use that as a default. .. note:: if a default is present, either passed or using the option default, it must also fit the other arguments.

• required: [True/False] this option MUST be passed on the command line.

• help: this is a descriotion string to be used on the command line. if this is not present,

  the option description will be used.

cli_option: ({'flags':['f', 'foo'],
              'data_flag':False,
              'nargs':2,
              'choices':['bar','barr','b'],
              'required':False,
              'help':'This defines if this is fu or bar'})

Migration of Versions¶

The system can help with migrating the data between versions. To handle migrations, first create a migration definition dictionary. and pass it to the migration manager during section initialization.

Migrations are handled on a section by section basis, and are performed when running a read operation from a storage manager (other than the cli manager) So be aware that in a multi-storage environment,

This dictionary could have the following keys:

{'action':'remove','option_name':<option_name>}

('remove','option_name')

{'action':'rename',
'option_name':<old option name>,
'new_option_name':<new option name>,
['interpolation_str':<optional interpolation string>]}

('rename','old_option_name','new_option_name'[, 'interpolation_str'])

{'action':'copy',
'option_name':<existing option name>,
'new_option_name',<new option name>,
['interpolation_str':<interpolation_string>]}

{'action':'copy',
'option_name':<existing section name.existing option name>,
'new_option_name',<new option name>,
['interpolation_str':<interpolation_string>]}

('copy','old_option_name','new_option_name' [,'interpolation_str'])
('copy','old_section.old_option_name,'new_option_name' [, 'interpolation_str'])

{'action':'pass',
'option_name':<option_name>)

{'action':'interpolate',
'option_name':<option_name>
['interpolation_str':<interpolation string>]}

('interpolate','option_name' [,'interpolation_str'])

Validation of data¶

If the data is going to be manipulated by users, the data can also be validated for accuracy. This is done by

Managing Data Storage Locations¶

Ths system can support multiple storage locations for the data, including handling data stored on local disk INI files, databases, and even passing the data in dictionary or list format.

Interpolation Options¶

The interpolation system can handles replacement on the fly. This is done within the options themselves, as well as during migrations.

At its most basic, interpolation allows you to have a string that contains a key string that is replaced with the value of another configuration option. For example:

::

# setup the configuration manager

config = ConfigManager() config.add(‘section1’) section1 = config[‘section1’]

# add some options for the data directory and data file

section1[‘data_dir’] = ‘my_path/data’ section1[‘data_filename’] = ‘my_data_file.db’

# add an option that will return the full file and path for the database:

section1[‘database’] = ‘%(data_dir)/%(data_filename)’

# when you call the interpolated option section1[‘database’]

‘mypath/data/my_data_file.db’

..note:: there are better approaches to handling paths in the system that woudl work better with cross platform systems,

see :py:module:`pathlib`

Interpolation can also handle cross section variables, so you can use %(section_name.option_name) if needed, as well as recursive variables where you are including a interpolated option in another interpolated option (up to a max of 10 levels).

if you need to use “%” in your option values, you can use escape it with “%%”

..note:: you can disable interpolation by passing None to the “interpolator” keyward arg for the ConfigManager

ConfigManager Class¶

class AdvConfigMgr.ConfigManager(version=None, migrations=None, storage_config=None, storage_managers=None, default_storage_managers=None)[source]¶

Parameters:

allow_no_value – allow empty values. empty_lines_in_values – allow_add_from_storage – allow adding sections and options directly from the storage allow_create_on_set – allows sections to be created when the set command is used. the set command can only be used to set the values of options, so this REQUIRES using dot-notation. no_sections – this will disable all sections and all options will be accessible from the base manager object. (this creates a section named “default_section”.) .. warning:: converting from simple configurations to sections may require manual data minipulation! section_defaults – a dictionary of settings used as defaults for all sections created interpolation – can be defined if interpolation is requested or required. section_option_sep (str) – defines a seperator character to use for getting options using the dot_notation style of query. defaults to ‘.’, in which case options can be queried by calling ConfigManager[section.option] in addition to ConfigManager[section][option]. If this is set to None, dot notation will be disabled. cli_program – the name of the program for the cli help screen (by default this will use the program run to launch the app) cli_desc – the text to show above the arguments in the cli help screen. cli_epilog – the text to show at the end of the arguments in the cli help screen. raise_error_on_locked_edit – if True, will raise an error if an attempt to change locked options, if False (default) the error is suppressed and the option will not be changed. storage_managers (BaseStorageManager) – a list of storage managers to use, if none are passed, the configuration will not be able to be saved. cli_parser_name – the name of the cli parser if not the default. set to None if the CLI parser is not to be used. cli_group_by_section – True if cli arguments shoudl be grouped by section for help screens. version_class (Version or str or None) – ‘loose’, ‘strict’, a subclass of the Version class or None for no versioning. version (str) – the version number string. version_option_name (str) – the option named used to store the version for each section, {section} will be replaced by the name of the section. version_allow_unversioned (bool) – if True, the system will import unversioned data, if false, the version of the data must be specified when importing any data. version_enforce_versioning (bool) – if True, the system will raise an error if no version is set at the section or base level. version_disable_cross_section_copy (bool) – if True, cross section copy will not work. Used when you have plugins from different authors and you want to segment them. version_make_migrations (list) – this is a list of migrations that can be performed, (see migration) kwargs – if “no_sections” is set, all section options can be passed to the ConfigManager object. data_dict (ConfigDict) – any dictionary that can support the number of levels needed. version – the sering version number. migrations (list) – a list of migration dictionaries. storage_config (dict) – a dictionary of storage configuration dictionaries. These would be in the format of {‘storage_name’:{<config_dict>},’storage_name’:{<config_dict>}}. See specific storage managers for details of the config dict entries. storage_managers – a list or set of storage managers to use, if not passed, the file storage manager is used. default_storage_managers (list) – a list or string indicating the default storage manager(s) names to use if no names are passed during read/write operations. if None (the default) all conifgured storage managers are polled.

add(*args, **kwargs)[source]¶

Adds configuration options:

add('section1', 'section2', 'section3')
add(section_def_dict1, section_def_dict2, section_def_dict3)
add([list of section_def_dicts])
add('section_name.option_name', 'section_name.option_name')

add(section_name1='option_name1', section_name2='option_name2')
add(section_name=section_def_dict, section_name2=section_def_dict)
add(section_name=[list_of_option_names or dicts], section_name=(list of option names or dicts]

section_def_dict keys

Key	Default	Description
‘name’	None	The name of the section
‘verbose_name’	None	The verbose name of the section
‘description’	None	A long description of the section
‘storage’	None	The name of the storage location (if used)
‘keep_if_empty’	False	Keep the section even if all options ahve been deleted
‘store_default’	False	Store defaults in storage medium
‘locked’	False	If True, do not allow any changes to the section
‘allow_create_on_load’	True	Allow new options to be created directly from the storage medium for example, if you hand edit the ini file and add new options
‘option_defaults’	None	Allows a dict to be passed with defaults for any new options in this section this will replace any system wide option defaults specified.
‘options’	None	Provides a list of options to be added to the section,

Note

When no sections are used, this will redirect to ConfigSection.add()

add_section(section, force_add_default=False, **kwargs)[source]¶

Create a new section in the configuration.

Raise DuplicateSectionError if a section by the specified name already exists.

read(sections=None, storage_names=None, override_tags=False, data=None, **kwargs)[source]¶

runs the read from storage process for the selected or configured managers

Parameters:

storage_names – If None, will read from all starnard storage managers, if a string or list, will read from the selected ones following the configured tag settings. sections – If None, will read from all sections, if string or list, will read from the selected ones following the configured tag settings. override_tags – if True, this will override the configured storage tag settings allowing things like exporting the full config etc. data – if a single storage tag is passed, then data can be passed to that storage manager for saving. this will raise an AssignmentError if data is not None and more than one storage tag is passed.

write(sections=None, storage_names=None, override_tags=False, **kwargs)[source]¶

runs the write to storage process for the selected or configured managers

Parameters:

storage_names – If None, will write to all starnard storage managers, if a string or list, will write to the selected ones following the configured tag settings. sections – If None, will write to all sections, if string or list, will write to the selected ones following the configured tag settings. override_tags – if True, this will override the configured storage tag settings allowing things like exporting the full config etc.

Returns:

if ONLY one storage_tag is passed, this will return the data from that manager if present.

ConfigSection Class¶

class AdvConfigMgr.ConfigSection(manager, name, verbose_name=None, description=None, storage_write_to=None, storage_read_from_only=None, store_default=False, locked=False, allow_create_on_load=True, allow_create_on_set=True, option_defaults=None, cli_section_title=None, cli_section_desc=None, version=None, version_option_name=None, version_migrations=None, options=None)[source]¶

A single section of a config

Parameters:

manager (ConfigManager) – A pointer to the config manager name (str) – The name of the section verbose_name (str) – The verbose name of the section description (str) – A long description of the section storage_write_to (str) – The tag of the storage location to save to, if None, the section will be saved to the default location, if ‘-‘ it will not be saved in save_all operations, if “*”, section will be saved to all configured storage locations. Sections can be saved manually to any storage if needed. storage_read_from_only (str or list) – options from storage with tags in this list will be read. If None (default) then options in storage will be always be used. This allows restricting options to specific storage locations. CLI options, if configured, will always overwrite data from storage. store_default (bool) – store defaults in storage medium locked (bool) – if True, do not allow any changes to the section allow_create_on_load (bool) – if True, options will be created if they are in the stored data, if false, they must be configured first. allow_create_on_set (bool) – if True, options can be created using a dictionary set proces, if False, they need to be configured first. cli_section_title (str) – the name of the section in the CLI (if sectioned), Defaults to verbose_name cli_section_desc (str) – the description for the section in the CLI (if sectioned), Defaults to description version (str) – the version number of the section, if None, this will take the version number of the ConfigManager object. version_option_name (str) – allows for overriding the ConfigManager’s version_option_name setting.

add(*args, **kwargs)[source]¶

Adds new option definition to the system

args, kwargs: config options can also be passed in args/kwargs, in a number of formats.

Examples:

This does not set any default values:

add('option1', 'option2', 'option3')

Seperate dictionaries:

add(full_option_dict, full_option_dict)

A list of dictionaries:

add([full_option_dict, full_option_dict])

A list of sets with option_name and default value.:

add([('option1',default_value),('option2',default_value)]

If default value is a dict, this will not work, if a dict is passed, it is assumed to be a full_option_dict:

add(option1=default_value1, option2=default_value2, option3=default_value3)

add(option1={full_option_dict}, option2={full_option_dict))

These can be mixed, so the following would be valid:

add('option1', full_option_dict, [full_option_dict, full_option_dict], [('option2',default_value)])

full_option_dict Example (with defaults):

‘name’: ‘<name of option>’, ‘default_value’: _UNSET, ‘datatype’: None, ‘verbose_name’: None, ‘description’: None, ‘cli_option’: None, ‘validations’: None, ‘do_not_change’: False, ‘do_not_delete’: False, ‘required_after_load’: False,

Note

• If a default value is a dictionary, it must be passed within a full option dict.

• See ConfigOption for option_dict parameters.

• If a full option dict is passed as an arg (not kwarg) it must contain a ‘name’ key.

• Args and kwargs can be mixed if needed... for example this is also a valid approach:

  add(option1, <full_option_dict>, option3=default_value, option4={full_option_dict}
  

• If options are repeated in the same commane, kwargs will take precdence over args, and new options will overwrite old ones.

• if there are existing options in the section with the same name, an error will be raised.

clear(options, forgiving=False)[source]¶

Will set the option to the default value or to unset as long as long as the section is not locked.

Parameters:	options (str or list) – option name or list of option names forgiving (bool) – True will return False if the option is not found, False, will raise NoOptionError.
Returns:	True if all deletes passed, False if not. if False, a list of the failed options is stored in self.last_failure_list
Return type:	bool
Raises:	NoOptionError – If the option does not exist. and forgiving is False.

delete(options, force=False, forgiving=False)[source]¶

Will delete a list of options.

Parameters:	options – Option name or list of option names. force (bool) – True will delete the object even if it has a default_value without checking for value or lock. forgiving (bool) – True will return False if the option is not found, False, will raise NoOptionError.
Type:	str or list
Returns:	True if all deletes passed, False if not. if False, a list of the failed options is stored in ConfigSection.last_failure_list
Return type:	bool

from_read(option, value, raw=False, validate=True)[source]¶

adds data from a storage module to the system, this ignores the ‘do_not_add’ flag. :param str option: option name :param value: the value to add :param bool raw: True if the interpolation needs to be bypassed. :param bool validate: True if validation should happen. :return:

get(option, fallback=Empty Value, raw=False)[source]¶

gets the value of an option

Parameters:	option – the name of the option fallback – a value to return if the option is empty raw – a flag to skip interpolation
Returns:

items(raw=False)[source]¶

Return a list of (name, value) tuples for each option in a section.

All interpolations are expanded in the return values, based on the defaults passed into the constructor, unless the optional argument ‘raw’ is true.

Parameters:	raw (bool) – True if data should not be interpolated.

load(option, value, *args, **kwargs)[source]¶

Loads value into system, can create new options if “allow_create_on_load” is set.

Parameters:	name – Option Name value – Option Value args – as per ConfigOption class (passed through) kwargs – as per ConfigOption class (passed through)

section_ok_after_load¶

validates that all options that are required “after_load” have either a set or default value :return: :rtype boolean:

set(option, value, raw=False, validate=True, force=False)[source]¶

Sets an option value.

Parameters:	option – the name of the option to set value – the value to set raw – if set to True will bypass the interpolater validate – if False will bypass the validation steps force – if True will bypass the lock checks, used for loading data.
Returns:	the interpolated value or default value.

to_write(option, raw=False, as_string=False)[source]¶

gets data from the system to save to a storage module :param bool raw: :param bool as_string: :return:

ConfigOption Class¶

class AdvConfigMgr.ConfigOption(section, name, *args, **kwargs)[source]¶

An individual option in the config

Parameters:

section (ConfigSection) – A pointer to the ConfigSection object that this is a part of name (str) – The name of the config object. This is transformed by the optionxform method in the main config manager. by default this is converted to lowercase default_value (object) – Default= _UNSET the default value for the item. If set to _UNSET this is considered to not have a default. (this allows None to be a valid default setting. data_type (str) – Default=None: This is the type of data that is stored in the option. this accepts : None, ‘str’, ‘int’, ‘float’, ‘list’, ‘dict’ additional data types can be defined using the DataTypeBase class. If set to None and there is a default value set, this will take the datatype of the default value, otherwise it will be set to ‘str’ verbose_name (str or None) – Default=None This is the long name for the option (that can show up in the options configuration screen or help screen) This is set to a title case version of the option name with spaces replacing ‘_’ description (str or None) – Default=None This is the long description for the option, available in the help screens. cli_option (str or None) – Default=None This allows the option to be changed via the CLI on startup, this would be a string, tuple or dictionary of options that configure how the cli commands will be handled. validations (object) – Default=None: This is a set of validation classes to be run for any options saved. keep_if_empty (bool) – Default=True: If set to False the option will be deleted when the value is cleared AND there is no set default value. do_not_change (bool) – Default=False If set to True, this will not allow the user to change the option after initial loading. do_not_delete (bool) – Default=False If set to True, this will not allow the user to delete the option. required_after_load (bool) – Default = False, If set to true, the app should not start without this being set. if there is a CLI_option available, the app should prompt the user for that option, if not, the app should fail with a usefull message. autoconvert (bool) – will attempt to autoconvert values to the datatype, this can be disabled if needed. (some types of data may not autoconvert correctly.)

from_read(value, raw=False, validate=True, from_string=False)[source]¶

adds data from a storage module to the system, this ignores the ‘do_not_add’ flag.

Parameters:	value – the value to add raw – if set to True will bypass the interpolater validate – if False will bypass the validation steps from_string – if True will convert from string
Returns:	the interpolated value or default value.

get(raw=False, as_string=False)[source]¶

Gets the current value or default interpolated value.

Parameters:	raw – if set to True will bypass the interpolater
Returns:	the interpolated value or default value.

set(value, raw=False, validate=True, force=False)[source]¶

Sets the current value.

Parameters:	value – the value to set raw – if set to True will bypass the interpolater validate – if False will bypass the validation steps force – if True will bypass the lock checks
Returns:	the interpolated value or default value.

to_write(raw=False, as_string=False)[source]¶

gets data from the system to save to a storage module,

Parameters:	raw – if set to True will bypass the interpolater as_string – returns the value as a strong (passing through the datatype module to_string method)
Returns:	the interpolated value or default value.
Returns:	the interpolated value or default value.

ConfigStorage Class¶