lisa.conf.MultiSrcConf#

class lisa.conf.MultiSrcConf(conf=None, src='user', add_default_src=True)[source]#

Bases: MultiSrcConfABC, Loggable, Mapping

Base class providing layered configuration management.

Parameters:

conf (collections.abc.Mapping) – collections.abc.Mapping to initialize the configuration with. This must be optional, in which case it is assumed the object will contain a default base configuration.
src – Name of the source added when passing conf
src – str

The class inherits from collections.abc.Mapping, which means it can be used like a readonly dict. Writing to it is handled by a different API that allows naming the source of values that are stored.

Each configuration key can be either a leaf key, that holds a value, or a level key that allows to defined nested levels. The allowed keys is set by the STRUCTURE class attribute.

Each leaf key can hold different values coming from different named sources. By default, the last added source will have the highest priority and will be served when looking up that key. A different priority order can be defined for a specific key if needed.

See also

KeyDescBase

This base class will modify the docstring of subclasses, using it as an str.format template with the following placeholders:

{generated_help}: snippet of ResStructuredText containing the list of allowed keys.

{yaml_example}: example snippet of YAML

It will also create the types specified using newtype in the KeyDesc, along with a getter to expose it to exekall.

Note

Since the dosctring is interpreted as a template, “{” and “}” characters must be doubled to appear in the final output.

Attention

The layout of the configuration is typically guaranteed to be backward-compatible in terms of accepted shape of input, but layout of the configuration might change. This means that the path to a given key could change as long as old input is still accepted. Types of values can also be widened, so third party code re-using config classes from lisa might have to evolve along the changes of configuration.

Warning

Arbitrary code can be executed while loading an instance from a YAML or Pickle file. To include untrusted data in YAML, use the !untrusted tag along with a string

Attributes

`DEFAULT_SRC`	Source added automatically using `add_src()` under the name ‘default’ when instances are built.
`ATTRIBUTES_SERIALIZATION` ^inherited	Attributes to be treated specially during serialization.
`DEFAULT_SERIALIZATION_FMT` ^inherited	Default format used when serializing objects.
`YAML_ENCODING` ^inherited	Encoding used for YAML files.
`__reversed__` ^inherited	See `collections.abc.Mapping.__reversed__`

Properties

`STRUCTURE`	Class attribute defining the structure of the configuration file, as a instance of `TopLevelKeyDescBase`.
`as_yaml_map` ^inherited	Give a mapping suitable for storing in a YAML configuration file.
`logger` ^inherited	Convenience short-hand for `self.get_logger()`.

Methods

`__copy__()`	Shallow copy of the nested configuration tree, without duplicating the leaf values.
`__deepcopy__()`
`__getitem__()`
`__getstate__()`	Filter instances of `DeferredValue` that are not computed already since their runtime parameters will probably not be available after deserialization.
`__iter__()`
`__len__()`
`add_src()`	Add a source of configuration.
`eval_deferred()`	Evaluate instances of `DeferredValue` that can be used for values that are expensive to compute.
`force_src()`	Force the source priority list for a given key.
`force_src_nested()`	Force the source priority list for all the keys defined in the nested mapping `key_src_map`.
`from_map()`	Create a new configuration instance, using the output of `to_map()`.
`get_help()`
`get_key()`	Get the value of the given key. It returns a deepcopy of the value.
`get_nested_key()`	Same as `get_key()` but works on a list of keys to access nested mappings.
`get_src_map()`	Get a mapping of all sources for the given `key`, in priority order (first item is the highest priority source).
`items()`	Override the default definition of `collections.abc.Mapping.items()` to allow not evaluating deferred values if necessary.
`pretty_format()`	Give a pretty string representation of the configuration.
`resolve_src()`	Get the source name that will be used to serve the value of `key`.
`set_default_src()`	Set the default source priority list.
`to_map()`	Export the configuration as a mapping.
`__class_getitem__()` ^inherited	See `collections.abc.Iterable.__class_getitem__()`
`__contains__()` ^inherited	See `collections.abc.Mapping.__contains__()`
`__setstate__()` ^inherited
`from_path()` ^inherited	Deserialize an object from a file.
`from_yaml_map()` ^inherited	Allow reloading from a plain mapping, to avoid having to specify a tag in the configuration file. The content is hosted under the top-level key specified in `STRUCTURE`.
`from_yaml_map_list()` ^inherited	Create a mapping of configuration classes to instance, by loading them from the list of paths using `from_yaml_map()` and merging them.
`get()` ^inherited	See `collections.abc.Mapping.get()`
`get_logger()` ^inherited	Provides a `logging.Logger` named after `cls`.
`keys()` ^inherited	See `collections.abc.Mapping.keys()`
`log_locals()` ^inherited	Debugging aid: log the local variables of the calling function.
`to_path()` ^inherited	Serialize the object to a file.
`to_yaml()` ^inherited	Return a YAML string with the serialized object.
`to_yaml_map()` ^inherited	Write a configuration file, with the key descriptions in comments.
`to_yaml_map_str()` ^inherited	Return the content of the file that would be create by `to_yaml_map()` in a string.
`values()` ^inherited	See `collections.abc.Mapping.values()`

Attributes#

MultiSrcConf.DEFAULT_SRC = {}#: Source added automatically using add_src() under the name ‘default’ when instances are built.

MultiSrcConf.ATTRIBUTES_SERIALIZATION = {'allowed': [], 'ignored': [], 'placeholders': {}}#

Inherited attribute, see lisa.utils.Serializable.ATTRIBUTES_SERIALIZATION

Attributes to be treated specially during serialization.

MultiSrcConf.DEFAULT_SERIALIZATION_FMT = 'yaml'#

Inherited attribute, see lisa.utils.Serializable.DEFAULT_SERIALIZATION_FMT

Default format used when serializing objects.

MultiSrcConf.YAML_ENCODING = 'utf-8'#

Inherited attribute, see lisa.utils.Serializable.YAML_ENCODING

Encoding used for YAML files.

MultiSrcConf.__reversed__ = None#

Properties#

abstract property MultiSrcConf.STRUCTURE[source]#: Class attribute defining the structure of the configuration file, as a instance of TopLevelKeyDescBase

property MultiSrcConf.as_yaml_map#

Inherited property, see lisa.conf.MultiSrcConfABC.as_yaml_map

Give a mapping suitable for storing in a YAML configuration file.

property MultiSrcConf.logger#

Inherited property, see lisa.utils.Loggable.logger

Convenience short-hand for self.get_logger().

Methods#

MultiSrcConf.__copy__()[source]#: Shallow copy of the nested configuration tree, without duplicating the leaf values.

MultiSrcConf.__deepcopy__(memo)[source]#

MultiSrcConf.__getitem__(key)[source]#

MultiSrcConf.__getstate__()[source]#

Filter instances of DeferredValue that are not computed already since their runtime parameters will probably not be available after deserialization.

If needed, call eval_deferred() before serializing.

MultiSrcConf.__iter__()[source]#

MultiSrcConf.__len__()[source]#

MultiSrcConf.add_src(src, conf, filter_none=False, fallback=False, inplace=True)[source]#

Add a source of configuration.

Parameters:

src (str) – Name of the soruce to add
conf (collections.abc.Mapping) – Nested mapping of key/values to overlay
filter_none (bool) – Ignores the keys that have a None value. That simplifies the creation of the mapping, by having keys always present. That should not be used if None value for a key is expected, as opposit to not having that key set at all.
fallback (bool) – If True, the source will be added as a fallback, which means at the end of the priority list. By default, the source will have the highest priority and will be used unless a key-specific priority override is setup.
inplace (bool) – If True, the object is modified. If False, a mutated copy is returned and the original object is left unmodified.

This method provides a way to update the configuration, by importing a mapping as a new source.

MultiSrcConf.eval_deferred(cls=<class 'lisa.conf.DeferredValue'>, src=None, resolve_src=True, error='raise')[source]#

Evaluate instances of DeferredValue that can be used for values that are expensive to compute.

Parameters:

cls (subclass of DeferredValue) – Only evaluate values of instances of that class. This can be used to have different categories of DeferredValue by subclassing.
src (str or None) – If not None, only evaluate values that were added under that source name.
resolve_src (bool) – If True, resolve the source of each key and only compute deferred values for this source.
error (str or None) – If 'raise' or None, exception are raised as usual. If log, the exception is logged at error level.

MultiSrcConf.force_src(key, src_prio)[source]#

Force the source priority list for a given key

Parameters:

key (str) – name of the key. Only leaf keys are allowed here, since level keys have no source on their own.
src_prio (collections.abc.Sequence(str) or None) – List of sources in priority order (first is highest priority). Special value None can be used to remove the key-specific priority override, so the default priority list will be used instead.

MultiSrcConf.force_src_nested(key_src_map)[source]#

Force the source priority list for all the keys defined in the nested mapping key_src_map

Parameters:: key_src_map (collections.abc.Mapping) – nested mapping of keys to priority list of sources

classmethod MultiSrcConf.from_map(mapping, add_default_src=True)[source]#: Create a new configuration instance, using the output of to_map()

classmethod MultiSrcConf.get_help(*args, **kwargs)[source]#

MultiSrcConf.get_key(key, src=None, eval_deferred=True, quiet=False)[source]#

Get the value of the given key. It returns a deepcopy of the value.

The special key .. can be used to refer to the parent in the hierarchy.

Parameters:

key (str) – name of the key to lookup
src (str or None) – If not None, look up the value of the key in that source
eval_deferred (bool) – If True, evaluate instances of DeferredValue if needed
quiet (bool) – Avoid logging the access

Note

Using the indexing operator self[key] is preferable in most cases , but this method provides more parameters.

MultiSrcConf.get_nested_key(key, *args, **kwargs)[source]#

Same as get_key() but works on a list of keys to access nested mappings.

Parameters:: key (list(str)) – List of nested keys.

MultiSrcConf.get_src_map(key)[source]#: Get a mapping of all sources for the given key, in priority order (first item is the highest priority source).

MultiSrcConf.items(eval_deferred=True)[source]#: Override the default definition of collections.abc.Mapping.items() to allow not evaluating deferred values if necessary.

MultiSrcConf.pretty_format(eval_deferred=False)[source]#

Give a pretty string representation of the configuration.

Parameters:: eval_deferred (bool) – If True, evaluate all deferred values before printing.

MultiSrcConf.resolve_src(key)[source]#: Get the source name that will be used to serve the value of key.

MultiSrcConf.set_default_src(src_prio)[source]#

Set the default source priority list.

Parameters:: src_prio (collections.abc.Sequence(str)) – list of source names, first is the highest priority

Adding sources using add_src() in the right order is preferable, but the default priority order can be specified using that method.

MultiSrcConf.to_map()[source]#

Export the configuration as a mapping.

The return value should preserve key-specific priority override list, which is not done if directly passing that instance to dict().

classmethod MultiSrcConf.__class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

MultiSrcConf.__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

MultiSrcConf.__setstate__(dct)#: Inherited method, see lisa.utils.Serializable.__setstate__()

classmethod MultiSrcConf.from_path(filepath, fmt=None)#

Inherited method, see lisa.utils.Serializable.from_path()

Deserialize an object from a file.

classmethod MultiSrcConf.from_yaml_map(path, add_default_src=True)#

Inherited method, see lisa.conf.MultiSrcConfABC.from_yaml_map()

Allow reloading from a plain mapping, to avoid having to specify a tag in the configuration file. The content is hosted under the top-level key specified in STRUCTURE.

classmethod MultiSrcConf.from_yaml_map_list(path_list, add_default_src=True)#

Inherited method, see lisa.conf.MultiSrcConfABC.from_yaml_map_list()

Create a mapping of configuration classes to instance, by loading them from the list of paths using from_yaml_map() and merging them.

MultiSrcConf.get()#: Inherited method, see collections.abc.Mapping.get()

classmethod MultiSrcConf.get_logger(suffix=None)#

Inherited method, see lisa.utils.Loggable.get_logger()

Provides a logging.Logger named after cls.

MultiSrcConf.keys()#: Inherited method, see collections.abc.Mapping.keys()

classmethod MultiSrcConf.log_locals(var_names=None, level='debug')#

Inherited method, see lisa.utils.Loggable.log_locals()

Debugging aid: log the local variables of the calling function.

MultiSrcConf.to_path(filepath, fmt=None)#

Inherited method, see lisa.utils.Serializable.to_path()

Serialize the object to a file.

MultiSrcConf.to_yaml()#

Inherited method, see lisa.utils.Serializable.to_yaml()

Return a YAML string with the serialized object.

MultiSrcConf.to_yaml_map(path)#

Inherited method, see lisa.conf.MultiSrcConfABC.to_yaml_map()

Write a configuration file, with the key descriptions in comments.

MultiSrcConf.to_yaml_map_str(**kwargs)#

Inherited method, see lisa.conf.MultiSrcConfABC.to_yaml_map_str()

Return the content of the file that would be create by to_yaml_map() in a string.

MultiSrcConf.values()#: Inherited method, see collections.abc.Mapping.values()

lisa.conf.MultiSrcConf#

Attributes#

Properties#

Methods#

This Page