Internal API#

Warning

This API is for now considered completely internal and could change at any time, except for items listed in Public API

Engine#

exception exekall.engine.NoOperatorError[source]#

Bases: Exception

Exception raised when no operator has been found to build objects of a needed type.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see Exception.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.IndentationManager(style)[source]#

Bases: object

Manage the indentation level in a generated script.

indent()[source]#

dedent()[source]#

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ValueDB(froz_val_seq_list, adaptor_cls=None)[source]#

Bases: object

Serializable object that contains a graph of FrozenExprVal.

This allows storing all the objects computed for each subexpression that was executed for later inspection.

Parameters:

froz_val_seq_list (list(FrozenExprValSeq)) – List of FrozenExprValSeq to store at the root of the database.
adaptor_cls (type) – A subclass of exekall.customization.AdaptorBase that was used when setting up the expressions. This class can provide hooks that are called when deserializing the database.

The values of each expression is recorded in a root list of FrozenExprValSeq.

PICKLE_PROTOCOL = 4#

classmethod merge(db_list, roots_from=None)[source]#

Merge multiple databases together.

Parameters:

db_list (list(ValueDB)) – Lists of DB to merge
roots_from (ValueDB) – Only get the root values from the specified DB. The other DBs are only used to provide subexpressions values for expressions already present in that DB. That DB is also appended to db_list.

When two different FrozenExprVal are available for a given UUID, the one that contains the most information will be selected. This allows natural removal of values created using an PrebuiltOperator when the original object is also available.

classmethod from_path(path, relative_to=None)[source]#

Deserialize a ValueDB from a file.

At the moment, it is assumed to be an LZMA compressed Pickle file.

Parameters:

path (str or pathlib.Path) – Path to the file containing the serialized ValueDB.
relative_to (str or pathlib.Path) – If provided, path is interpreted as being relative to that folder, or to the containing folder if it is a file. This is mainly used to ease generation of scripts that can provide relative_to=__file__ so that the artifact folder can be moved around easily.

__reduce_ex__(protocol)[source]#: Provide custom serialization that will call the adaptor’s hooks.

to_path(path, optimize=True)[source]#

Write the DB to the given file.

Parameters:

path (pathlib.Path or str) – path to file to write the DB into
optimize (bool) – Optimize the representation of the DB. This may increase the dump time and memory consumption, but should speed-up loading/file size.

get_by_uuid(uuid)[source]#: Get a FrozenExprVal by its UUID.

get_by_predicate(predicate, flatten=True, deduplicate=False)[source]#

Get FrozenExprVal matching the predicate.

Parameters:

predicate (collections.abc.Callable) – Predicate callable called with an instance of FrozenExprVal. If it returns True, the value is selected.
flatten (bool) – If False, return a set of frozenset of objects. There is a frozenset set for each expression result that shared their parameters. If False, the top-level set is flattened into a set of objects matching the predicate.
deduplicate (bool) – If True, there won’t be duplicates across nested sets.

get_roots(flatten=True)[source]#

Get all the root FrozenExprVal.

Parameters:: flatten (bool) – If True, a set of FrozenExprVal is returned, otherwise a set of frozensets of FrozenExprVal is returned. Each set will correspond to an expression, and values inside the frozenset will correspond to the values of that expression.

Root values are the result of full expression (as opposed to subexpressions).

prune_by_predicate(predicate)[source]#

Create a new ValueDB with all FrozenExprVal matching the predicate replaced by a terminal PrunedFrozVal.

Parameters:: predicate (collections.abc.Callable) – Predicate callable called with an instance of FrozenExprVal. If it returns True, the value is pruned out.

This allows trimming a ValueDB to a smaller size by removing non-necessary content.

get_all(**kwargs)[source]#

Get all FrozenExprVal contained in this database.

Variable keyword arguments:: Forwarded to ValueDB.get_by_predicate()

get_by_type(cls, include_subclasses=True, **kwargs)[source]#

Get all FrozenExprVal contained in this database which value has the specified type.

Parameters:

cls (type) – Class to match.
include_subclasses (bool) – If True, the check is done using isinstance, otherwise an exact type check is done using is.

Variable keyword arguments:

Forwarded to ValueDB.get_by_predicate()

Note

If a subexpressions had a exekall._utils.NoValue value, it will not be selected as type matching is done on the value itself, not the return type of the callable used for that sub expression.

get_by_id(id_pattern, qual=False, full_qual=False, **kwargs)[source]#

Get all FrozenExprVal contained in this database which ID matches the given pattern.

Parameters:

id_pattern (str) – fnmatch.fnmatch() pattern used to match the ID.
qual (bool) – If True, the match will be performed on the qualified ID.
full_qual (bool) – If True, the match will be performed on the fully qualified ID.

Variable keyword arguments:

Forwarded to ValueDB.get_by_predicate()

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ScriptValueDB(db, var_name='db')[source]#

Bases: object

Class tying together a generated script and a ValueDB.

Parameters:

db (ValueDB) – ValueDB used.
var_name (str) – Name of the variable used to represent the ValueDB in the generated script.

get_snippet(expr_val, attr)[source]#

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

exception exekall.engine.CycleError[source]#

Bases: Exception

Exception raised when a cyclic dependency is detected when building the Expression out of a set of callables.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see Exception.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

exception exekall.engine.AlreadyVisitedException[source]#

Bases: Exception

Exception raised when a node has already been visited during a graph traversal.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see Exception.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ExprHelpers[source]#

Bases: Mapping

Helper class used by all expression-like classes.

It mainly implements the mapping protocol, with keys being parameters and values being subexpressions computing the value of these parameters.

__getitem__(k)[source]#

__len__()[source]#

__iter__()[source]#

fold(f, init=None, visit_once=False)[source]#

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

Parameters:

f (collections.abc.Callable) –
Function to execute for each instance. It must take two parameters:
- as first parameter: the return value of the previous invocation of f. For the first call, init value is used.
- as second parameter: the instance of ExprHelpers that is being visited.
init (object) – Initial value passed to f.
visit_once (bool) – If True, each ExprHelpers will only be visited once.

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

get()#: Inherited method, see collections.abc.Mapping.get()

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.ExpressionBase(op, param_map)[source]#

Bases: ExprHelpers

Base class of all expressions proper.

Parameters:

op (Operator) – Operator to call for that expression
param_map (collections.OrderedDict) – Mapping of parameter names to other ExpressionBase. The mapping must maintain its order, so that it is possible to get the parameter list in the order it was defined in the sources.

classmethod cse(expr_list)[source]#: Apply a flavor of common subexpressions elimination to the list of ExpressionBase.

clone_by_predicate(predicate)[source]#

Create a new ExpressionBase, with the outer levels cloned and the inner sub expressions shared with the original one.

Parameters:: predicate (collections.abc.Callable) – Predicate called on ExpressionBase used to know at what level the sub expressions should not be cloned anymore, but instead shared with the original ExpressionBase. All parents of a shared expression will be shared no matter what to ensure consistent expressions.

format_structure(full_qual=True, graphviz=False)[source]#

Format the expression in a human readable way.

Parameters:

full_qual (bool) – If True, use fully qualified IDs.
graphviz – If True, return a graphviz description suitable for the dot graph rendering tool.
graphviz – bool

get_id(*args, marked_expr_val_set={}, **kwargs)[source]#

Return the ID of the expression.

Parameters:

marked_expr_val_set (set(ExpressionBase)) – If True, return a two-line strings with the second line containing marker characters under the ID of all ExpressionBase specified in that set.
with_tags (bool) – Add the tags extracted from the values of each ExprVal.
remove_tags (set(str)) – Do not add the specified tags values.
qual (bool) – If True, return the qualified ID.
full_qual (bool) – If True, return the fully qualified ID.
style (str or None) – If "rst", return a Sphinx reStructuredText string with references to types.
hidden_callable_set (set(collections.abc.Callable)) – Hide the ID of all callables given in that set, including their parent’s ID.

get_script(*args, **kwargs)[source]#

Return a script equivalent to that ExpressionBase.

Variable keyword arguments:: Forwarded to get_all_script().

classmethod get_all_script(expr_list, prefix='value', db_path='VALUE_DB.pickle.xz', db_relative_to=None, db=None, adaptor_cls=None)[source]#

Return a script equivalent to executing the specified ExpressionBase.

Parameters:

expr_list (list(ExpressionBase)) – List of ExpressionBase to turn into a script
prefix (str) – Prefix used to name variables containing the values of expressions of expr_list.
db_path (str) – Path to the serialized ValueDB that contains the FrozenExprVal of the expressions in expr_list. This is used to generate commented-out code allowing to deserialize values instead of calling the operator again.
relative_to (str) – Passed to ValueDB.from_path() when the ValueDB is opened at the beginning of the script. This can typically be set to __file__, so that the script will be able to refer to the ValueDB using a relative path.
db (ValueDB or None) – ValueDB containing the FrozenExprVal that were computed when computing expressions of expr_list. If None is provided, a new ValueDB object will be built assuming expr_list is a list of ComputableExpression.
adaptor_cls (type) – If db=None, used to build a new ValueDB.

EXPR_DATA_VAR_NAME = 'EXPR_DATA'#

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

get()#: Inherited method, see collections.abc.Mapping.get()

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.ComputableExpression(op, param_map, data=None)[source]#

Bases: ExpressionBase

Expression that also contains its computed values.

Parameters:: data (ExprData or None) – ExprData to use when computing the values of the expression. The data of the root ComputableExpression will be used for all the subexpressions as well during the execution.

See also

ExpressionBase

Instances of this class contains values, whereas Expression do not.

classmethod from_expr(expr, **kwargs)[source]#

Build an instance from an ExpressionBase

Variable keyword arguments:: Forwarded to __init__

classmethod from_expr_list(expr_list)[source]#: Build an a list of instances from a list of ExpressionBase.

Note

Common Subexpression Elimination using ExpressionBase.cse() will be applied on the resulting list.

get_id(mark_excep=False, marked_expr_val_set={}, **kwargs)[source]#

Return the ID of the expression.

Parameters:

mark_excep (bool) – Mark expressions listed by get_excep().
marked_expr_val_set (bool) – If mark_excep=False mark these exceptions, otherwise it is ignored.

See also

ExpressionBase.get_id()

find_expr_val_seq_list(param_map)[source]#

Return a list of ExprValSeq that were computed using the given parameters.

Parameters:: param_map (collections.OrderedDict) – Mapping of parameter names to values

Note

param_map will be checked as a subset of the parameters.

classmethod execute_all(expr_list, *args, **kwargs)[source]#

Execute all expressions of expr_list after applying Common Expression Elimination and yield tuples of (ExpressionBase, ExprVal).

Parameters:: expr_list (list(ExpressionBase)) – List of expressions to execute
Variable keyword arguments:: Forwarded to execute().

prepare_execute()[source]#

Prepare the expression for execution.

This includes appropriate cloning of expressions using ConsumerOperator and ExprData.

Note

Calling this method manually is only useful to get more accurate graphs when showing the structure of the expression, since it is done in any case by execute().`

execute(post_compute_cb=None)[source]#

Execute the expression and yield its ExprVal.

Parameters:: post_compute_cb (collections.abc.Callable) – Callback called after every computed value. It takes two parameters: 1) the ExprVal that was just computed and 2) a boolean that is True if the ExprVal was merely reused and False if it was actually computed.

Note

The prepare_execute() is called prior to executing.

get_all_vals()[source]#: Get all ExprVal that were computed for that expression.

get_excep()[source]#: Get all ExprVal containing an exception that are reachable from ExprVal computed for that expression.

EXPR_DATA_VAR_NAME = 'EXPR_DATA'#

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

clone_by_predicate(predicate)#

Inherited method, see exekall.engine.ExpressionBase.clone_by_predicate()

Create a new ExpressionBase, with the outer levels cloned and the inner sub expressions shared with the original one.

classmethod cse(expr_list)#

Inherited method, see exekall.engine.ExpressionBase.cse()

Apply a flavor of common subexpressions elimination to the list of ExpressionBase.

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

format_structure(full_qual=True, graphviz=False)#

Inherited method, see exekall.engine.ExpressionBase.format_structure()

Format the expression in a human readable way.

get()#: Inherited method, see collections.abc.Mapping.get()

classmethod get_all_script(expr_list, prefix='value', db_path='VALUE_DB.pickle.xz', db_relative_to=None, db=None, adaptor_cls=None)#

Inherited method, see exekall.engine.ExpressionBase.get_all_script()

Return a script equivalent to executing the specified ExpressionBase.

get_script(*args, **kwargs)#

Inherited method, see exekall.engine.ExpressionBase.get_script()

Return a script equivalent to that ExpressionBase.

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.ClassContext(op_map, cls_map)[source]#

Bases: object

Collect callables and types that put together will be used to create Expression.

Parameters:

op_map (dict(type, list(Operator))) – Mapping of types to list of Operator that can produce that type.
cls_map (dict(type, list(type))) – Mapping of types to a list of compatible types. A common “compatibility” relation is issubclass. In that case, keys are classes and values are the list of all (direct and indirect) subclasses.

COMPAT_CLS(class_or_tuple, /)#: Callable defining the compatibility relation between two classes. It will be called on two classes and shall return True if the classes are compatible, False otherwise.

classmethod from_op_set(op_set, forbidden_pattern_set={}, restricted_pattern_set={}, compat_cls=<built-in function issubclass>)[source]#

Build an ClassContext out of a set of Operator.

Parameters:

op_set (set(Operator)) – Set of Operator to consider.
forbidden_pattern_set (set(str)) – Set fnmatch.fnmatch() type name patterns that are not allowed to be produced.
compat_cls (collections.abc.Callable) – Callable defining the compatibility relation between two classes. It will be called on two classes and shall return True if the classes are compatible, False otherwise.

Parm restricted_pattern_set:

Set of fnmatch.fnmatch() Operator ID pattern. Operators matching that pattern will be the only one allowed to produce the type they are producing, or any other compatible type.

build_expr_list(result_op_set, non_produced_handler='raise', cycle_handler='raise')[source]#

Build a list of consistent Expression.

Parameters:

result_op_set (set(Operator)) – Set of Operator that will constitute the roots of expressions.
non_produced_handler (str or collections.abc.Callable) –
Handler to be used when a needed type is produced by no Operator:
- raise: will raise a NoOperatorError exception
- ignore: the expression will be ignored
- a callback: called with the following parameters:
  __qualname__ of the type that cannot be produced.
  
  name of the Operator for which a value of the type was needed.
  
  name of the parameter for which a value of the type was needed.
  
  a stack (tuple) of callables which is the path leading from the root expression to the operator for which the type was needed.
cycle_handler (str or collections.abc.Callable) –
Handler to be used when a cycle is detected in the built Expression:
- raise: will raise a CycleError exception
- ignore: the expression will be ignored
- a callback: called with a tuple of callables constituting the cycle.

All combinations of compatible classes and operators will be generated.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.Expression(op, param_map)[source]#

Bases: ExpressionBase

Static subclass ExpressionBase tying Operator with its parameters.

Instances of this class do not contain any computed values, they can be considered as read-only structure.

See also

ComputableExpression.

validate(cls_map)[source]#

Check that the Expression does not involve two classes that are compatible.

This ensures that only one class of each “category” will be used in each expression, so that all references to that class will point to the same expression after ExpressionBase.cse() is applied.

EXPR_DATA_VAR_NAME = 'EXPR_DATA'#

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

clone_by_predicate(predicate)#

Inherited method, see exekall.engine.ExpressionBase.clone_by_predicate()

Create a new ExpressionBase, with the outer levels cloned and the inner sub expressions shared with the original one.

classmethod cse(expr_list)#

Inherited method, see exekall.engine.ExpressionBase.cse()

Apply a flavor of common subexpressions elimination to the list of ExpressionBase.

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

format_structure(full_qual=True, graphviz=False)#

Inherited method, see exekall.engine.ExpressionBase.format_structure()

Format the expression in a human readable way.

get()#: Inherited method, see collections.abc.Mapping.get()

classmethod get_all_script(expr_list, prefix='value', db_path='VALUE_DB.pickle.xz', db_relative_to=None, db=None, adaptor_cls=None)#

Inherited method, see exekall.engine.ExpressionBase.get_all_script()

Return a script equivalent to executing the specified ExpressionBase.

get_id(*args, marked_expr_val_set={}, **kwargs)#

Inherited method, see exekall.engine.ExpressionBase.get_id()

Return the ID of the expression.

get_script(*args, **kwargs)#

Inherited method, see exekall.engine.ExpressionBase.get_script()

Return a script equivalent to that ExpressionBase.

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

exception exekall.engine.AnnotationError[source]#

Bases: Exception

Exception raised when there is a missing or invalid PEP 484 annotation.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see Exception.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

exception exekall.engine.PartialAnnotationError[source]#

Bases: AnnotationError

Exception raised when there is a missing PEP 484 annotation, but other parameters are annotated.

This usually indicates a missing annotation in a function otherwise supposed to be annotated.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see Exception.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ForcedParamType[source]#

Bases: object

Base class for types placeholders used when forcing the value of a parameter using Operator.force_param().

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.UnboundMethod(callable_, cls)[source]#

Bases: object

Wrap a function in a similar way to Python 2 unbound methods.

Parameters:

callable (collections.abc.Callable) – method to wrap.
cls (type) – Class on which the method is available.

Note

It is generally assumed that if a given method is wrapped in an UnboundMethod, all subclasses will also have that method wrapped the same way.

Note

UnboundMethod can wrap things such as staticmethod or classmethod as well, it is simply a wrapper used to attach the class of origin.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.Operator(callable_, non_reusable_type_set=None, tags_getter=None)[source]#

Bases: object

Wrap a callable.

Parameters:

callable (collections.abc.Callable) – callable to represent.
non_reusable_type_set (set(type)) – Set of non reusable types. If the callable produces a subclass of these types, it will be considered as non-reusable.
tags_getter (collections.abc.Callable) – Callback used to get the tags for the objects returned by the callable. It takes the object as argument, and is expected to return a mapping of tags names to values.

property callable_globals[source]#: Returns a dictionnary of global variables as seen by the callable.

force_param(param_value_map, tags_getter=None)[source]#

Force the value of a given parameter of the callable.

Parameters:

param_value_map (dict(str, list(object))) – Mapping of parameter names to list of values that this parameter should take.
tags_getter (collections.abc.Callable) – Callable used to return the tags for the values of the parameter. Same as Operator’s __init__ parameter.

property resolved_callable[source]#: Fully unwrapped callable. If the callable is a class, it’s __init__ will be returned.

property unwrapped_callable[source]#: Fully unwrapped callable.

See also

functools.wraps()

get_name(*args, **kwargs)[source]#: Get the name of the callable, or None if no name can be retrieved.

get_id(full_qual=True, qual=True, style=None)[source]#

Get the ID of the operator.

Parameters:

full_qual (bool) – Fully qualified name, including the module name.
qual (bool) – Qualified name.
style (str or None) – If rst, a Sphinx reStructuredText string is returned.

property name[source]#: Same as get_name()

property mod_name[source]#

Name of the module the callable is defined in.

If the callable is an UnboundMethod, the module of its class is returned.

Note

The callable is first unwrapped.

property src_loc[source]#: Get the source location of the unwrapped callable.

See also

exekall._utils.get_src_loc()

property value_type[source]#: Annotated return type of the callable.

property is_genfunc[source]#: True if the callable is a generator function.

property is_class[source]#: True if the callable is a class.

property is_static_method[source]#: True if the callable is a staticmethod.

property is_method[source]#: True if the callable is a plain method.

property is_cls_method[source]#: True if the callable is a classmethod.

property is_factory_cls_method[source]#: True if the callable is a factory classmethod, i.e. a classmethod that returns objects of the class it is defined in (or of a subclass of it), or has a return annotation of typing.Self.

make_expr_val_iter(expr, param_map)[source]#: Make an iterator that will yield the computed ExprVal.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.PrebuiltOperator(obj_type, obj_list, id_=None, **kwargs)[source]#

Bases: Operator

Operator that injects prebuilt objects.

Parameters:

obj_type (type) – Type of the objects that are injected.
obj_list (list(object)) – List of objects to inject
id (str or None) – ID of the operator.

Variable keyword arguments:

Forwarded to Operator constructor.

get_name(*args, **kwargs)[source]#: Get the name of the callable, or None if no name can be retrieved.

get_id(*args, style=None, **kwargs)[source]#

Get the ID of the operator.

Parameters:

full_qual (bool) – Fully qualified name, including the module name.
qual (bool) – Qualified name.
style (str or None) – If rst, a Sphinx reStructuredText string is returned.

property src_loc[source]#: Get the source location of the unwrapped callable.

See also

exekall._utils.get_src_loc()

property is_genfunc[source]#: True if the callable is a generator function.

property is_method[source]#: True if the callable is a plain method.

make_expr_val_iter(expr, param_map)[source]#: Make an iterator that will yield the computed ExprVal.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

property callable_globals#

Inherited property, see exekall.engine.Operator.callable_globals

Returns a dictionnary of global variables as seen by the callable.

force_param(param_value_map, tags_getter=None)#

Inherited method, see exekall.engine.Operator.force_param()

Force the value of a given parameter of the callable.

property is_class#

Inherited property, see exekall.engine.Operator.is_class

True if the callable is a class.

property is_cls_method#

Inherited property, see exekall.engine.Operator.is_cls_method

True if the callable is a classmethod.

property is_factory_cls_method#

Inherited property, see exekall.engine.Operator.is_factory_cls_method

True if the callable is a factory classmethod, i.e. a classmethod that returns objects of the class it is defined in (or of a subclass of it), or has a return annotation of typing.Self.

property is_static_method#

Inherited property, see exekall.engine.Operator.is_static_method

True if the callable is a staticmethod.

property mod_name#

Inherited property, see exekall.engine.Operator.mod_name

Name of the module the callable is defined in.

property name#

Inherited property, see exekall.engine.Operator.name

Same as get_name().

property resolved_callable#

Inherited property, see exekall.engine.Operator.resolved_callable

Fully unwrapped callable. If the callable is a class, it’s __init__ will be returned.

property unwrapped_callable#

Inherited property, see exekall.engine.Operator.unwrapped_callable

Fully unwrapped callable.

property value_type#

Inherited property, see exekall.engine.Operator.value_type

Annotated return type of the callable.

class exekall.engine.ConsumerOperator(consumer=None)[source]#

Bases: PrebuiltOperator

Placeholder operator used to represent the consumer of the an expression asking for it.

Parameters:: consumer (collections.abc.Callable) – Callable that will consume the value of the expression refering to its consumer.

get_name(*args, **kwargs)[source]#: Get the name of the callable, or None if no name can be retrieved.

property consumer[source]#

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

property callable_globals#

Inherited property, see exekall.engine.Operator.callable_globals

Returns a dictionnary of global variables as seen by the callable.

force_param(param_value_map, tags_getter=None)#

Inherited method, see exekall.engine.Operator.force_param()

Force the value of a given parameter of the callable.

get_id(*args, style=None, **kwargs)#

Inherited method, see exekall.engine.PrebuiltOperator.get_id()

Get the ID of the operator.

property is_class#

Inherited property, see exekall.engine.Operator.is_class

True if the callable is a class.

property is_cls_method#

Inherited property, see exekall.engine.Operator.is_cls_method

True if the callable is a classmethod.

property is_factory_cls_method#

Inherited property, see exekall.engine.Operator.is_factory_cls_method

True if the callable is a factory classmethod, i.e. a classmethod that returns objects of the class it is defined in (or of a subclass of it), or has a return annotation of typing.Self.

property is_genfunc#

Inherited property, see exekall.engine.PrebuiltOperator.is_genfunc

True if the callable is a generator function.

property is_method#

Inherited property, see exekall.engine.PrebuiltOperator.is_method

True if the callable is a plain method.

property is_static_method#

Inherited property, see exekall.engine.Operator.is_static_method

True if the callable is a staticmethod.

make_expr_val_iter(expr, param_map)#

Inherited method, see exekall.engine.PrebuiltOperator.make_expr_val_iter()

Make an iterator that will yield the computed ExprVal.

property mod_name#

Inherited property, see exekall.engine.Operator.mod_name

Name of the module the callable is defined in.

property name#

Inherited property, see exekall.engine.Operator.name

Same as get_name().

property resolved_callable#

Inherited property, see exekall.engine.Operator.resolved_callable

Fully unwrapped callable. If the callable is a class, it’s __init__ will be returned.

property src_loc#

Inherited property, see exekall.engine.PrebuiltOperator.src_loc

Get the source location of the unwrapped callable.

property unwrapped_callable#

Inherited property, see exekall.engine.Operator.unwrapped_callable

Fully unwrapped callable.

property value_type#

Inherited property, see exekall.engine.Operator.value_type

Annotated return type of the callable.

class exekall.engine.ExprDataOperator(data=None)[source]#

Bases: PrebuiltOperator

Placeholder operator for ExprData.

The ExprData that will be used is the same throughout an expression, and is the one of the root expression.

property data[source]#

property uuid_list[source]#

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

property callable_globals#

Inherited property, see exekall.engine.Operator.callable_globals

Returns a dictionnary of global variables as seen by the callable.

force_param(param_value_map, tags_getter=None)#

Inherited method, see exekall.engine.Operator.force_param()

Force the value of a given parameter of the callable.

get_id(*args, style=None, **kwargs)#

Inherited method, see exekall.engine.PrebuiltOperator.get_id()

Get the ID of the operator.

get_name(*args, **kwargs)#

Inherited method, see exekall.engine.PrebuiltOperator.get_name()

Get the name of the callable, or None if no name can be retrieved.

property is_class#

Inherited property, see exekall.engine.Operator.is_class

True if the callable is a class.

property is_cls_method#

Inherited property, see exekall.engine.Operator.is_cls_method

True if the callable is a classmethod.

property is_factory_cls_method#

Inherited property, see exekall.engine.Operator.is_factory_cls_method

True if the callable is a factory classmethod, i.e. a classmethod that returns objects of the class it is defined in (or of a subclass of it), or has a return annotation of typing.Self.

property is_genfunc#

Inherited property, see exekall.engine.PrebuiltOperator.is_genfunc

True if the callable is a generator function.

property is_method#

Inherited property, see exekall.engine.PrebuiltOperator.is_method

True if the callable is a plain method.

property is_static_method#

Inherited property, see exekall.engine.Operator.is_static_method

True if the callable is a staticmethod.

make_expr_val_iter(expr, param_map)#

Inherited method, see exekall.engine.PrebuiltOperator.make_expr_val_iter()

Make an iterator that will yield the computed ExprVal.

property mod_name#

Inherited property, see exekall.engine.Operator.mod_name

Name of the module the callable is defined in.

property name#

Inherited property, see exekall.engine.Operator.name

Same as get_name().

property resolved_callable#

Inherited property, see exekall.engine.Operator.resolved_callable

Fully unwrapped callable. If the callable is a class, it’s __init__ will be returned.

property src_loc#

Inherited property, see exekall.engine.PrebuiltOperator.src_loc

Get the source location of the unwrapped callable.

property unwrapped_callable#

Inherited property, see exekall.engine.Operator.unwrapped_callable

Fully unwrapped callable.

property value_type#

Inherited property, see exekall.engine.Operator.value_type

Annotated return type of the callable.

class exekall.engine.ExprValSeq(expr, iterator, param_map, post_compute_cb=None)[source]#

Bases: object

Sequence of ExprVal produced by an ComputableExpression.

Parameters:

expr (ComputableExpression) – ComputableExpression that was used to compute the values.
iterator (collections.abc.Iterator) – Iterator that yields the ExprVal. This is used when the expressions are being executed.
param_map (collections.OrderedDict) – Ordered mapping of parameters name to ExprVal used to compute the recored ExprVal.
post_compute_cb (collections.abc.Callable) – See ComputableExpression.execute()

Since ComputableExpression can represent generator functions, they are allowed to create multiple ExprVal.

classmethod from_one_expr_val(expr_val)[source]#: Build an ExprValSeq out of a single ExprVal.

See also

ExprValSeq for parameters description.

classmethod from_expr(expr, param_map, **kwargs)[source]#: Build an ExprValSeq out of a single ComputableExpression.

See also

ExprValSeq for parameters description.

iter_expr_val()[source]#

Iterate over the iterator and yield ExprVal.

post_compute_cb will be called when a value is computed or reused.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ExprValParamMap[source]#

Bases: OrderedDict

Mapping of parameters to ExprVal used when computing the value of a ComputableExpression.

is_partial(ignore_error=False)[source]#

Return True if the map is partial, i.e. some parameters don’t have a value.

That could be because one of them could not be computed due to an exception, or because it was skipped since it could not lead to a result anyway.

classmethod from_gen_map(param_gen_map)[source]#

Build a ExprValParamMap out of a mapping of parameters names and expressions to generators.

Parameters:: param_gen_map (collections.OrderedDict) – Mapping of tuple(param_name, param_expr) to an iterator that is ready to generate the possible values for the generator.

Generators are assumed to only yield once.

See also

from_gen_map_product() for cases where the generator is expected to yield more than once.

classmethod from_gen_map_product(param_gen_map)[source]#

Yield collections.OrderedDict for each combination of parameter values.

Parameters:: param_gen_map (collections.OrderedDict) – Mapping of tuple(param_name, param_expr) to an iterator that is ready to generate the possible values for the generator.

__class__#: alias of type

__class_getitem__()#: Inherited method, see dict.__class_getitem__()

__delitem__()#: Inherited method, see collections.OrderedDict.__delitem__()

__ge__()#: Inherited method, see collections.OrderedDict.__ge__()

__gt__()#: Inherited method, see collections.OrderedDict.__gt__()

__hash__ = None#

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__ior__()#: Inherited method, see collections.OrderedDict.__ior__()

__iter__()#: Inherited method, see collections.OrderedDict.__iter__()

__le__()#: Inherited method, see collections.OrderedDict.__le__()

__lt__()#: Inherited method, see collections.OrderedDict.__lt__()

__ne__()#: Inherited method, see collections.OrderedDict.__ne__()

__new__()#: Inherited method, see dict.__new__()

__or__()#: Inherited method, see collections.OrderedDict.__or__()

__reduce__()#: Inherited method, see collections.OrderedDict.__reduce__()

__reversed__()#: Inherited method, see collections.OrderedDict.__reversed__()

__ror__()#: Inherited method, see collections.OrderedDict.__ror__()

__setitem__()#: Inherited method, see collections.OrderedDict.__setitem__()

__sizeof__()#: Inherited method, see collections.OrderedDict.__sizeof__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

clear()#: Inherited method, see collections.OrderedDict.clear()

copy()#: Inherited method, see collections.OrderedDict.copy()

fromkeys()#: Inherited method, see collections.OrderedDict.fromkeys()

items()#: Inherited method, see collections.OrderedDict.items()

keys()#: Inherited method, see collections.OrderedDict.keys()

move_to_end()#: Inherited method, see collections.OrderedDict.move_to_end()

pop()#: Inherited method, see collections.OrderedDict.pop()

popitem()#: Inherited method, see collections.OrderedDict.popitem()

setdefault()#: Inherited method, see collections.OrderedDict.setdefault()

update()#: Inherited method, see collections.OrderedDict.update()

values()#: Inherited method, see collections.OrderedDict.values()

class exekall.engine.ExprValBase(param_map, value, excep, uuid, duration, log)[source]#

Bases: ExprHelpers

Base class for classes representing the value of an expression.

Parameters:

param_map (dict(str, ExprValBase)) – Map of parameter names of the Operator that gave this value to their ExprValBase value.
value (Exception) – Value that was computed. If no value was computed, exekall._utils.NoValue will be used.
excep – Exception that was raised while computing the value. If no excpetion was raised, exekall._utils.NoValue will be used.
uuid (str) – UUID of the ExprValBase
duration (float or None) – Time it took to compute the value or the exception in seconds.
log (ExprValLog) – Log collected during the computation of the value.

property cumulative_duration[source]#: Sum of the duration of all ExprValBase that were involved in the computation of that one.

get_full_log(level)[source]#

Reconstruct a consistent log output at the given level by stitching the logs of all parent ExprValBase that were involved in the computation of that value.

Parameters:: level (str) – Logging level to reconstruct.

get_by_predicate(predicate)[source]#

Get a list of parents ExprValBase for which the predicate returns True.

get_excep()[source]#: Get all the parents ExprValBase for which an exception was raised.

get_by_type(cls, include_subclasses=True, **kwargs)[source]#

Get a list of parents ExprValBase having a value of the given type.

Parameters:

cls (type) – Type to look for.
include_subclasses (bool) – If True, the check is done using isinstance, otherwise an exact type check is done using is.

format_structure(full_qual=True)[source]#

Format the value and its parents in a human readable way.

Parameters:: full_qual (bool) – If True, use fully qualified IDs.

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

get()#: Inherited method, see collections.abc.Mapping.get()

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.FrozenExprVal(param_map, value, excep, uuid, duration, log, callable_qualname, callable_name, recorded_id_map, tags)[source]#

Bases: ExprValBase

Serializable version of ExprVal.

Parameters:

uuid (str) – UUID of the ExprVal
duration (float or None) – Time it took to compute the value or the exception in seconds.
log (ExprValLog) – Log collected during the computation of the value.
callable_qualname (str) – Qualified name of the callable that was used to compute the value, including module name.
callable_name (str) – Name of the callable that was used to compute the value.
recorded_id_map (dict) – Mapping of ExprVal.get_id() parameters to the corresponding ID. The parameters

The most important instance attributes are:

`value`	Value that was computed, or `NoValue` if it was not computed. This could be because of an exception when computing it, or because computing the value was skipped.
`excep`	Exception that was raised when trying to compute the value, or `NoValue`.
`uuid`	String UUID of that value. This is unique and can be used to correlate with logs, or deduplicate across multiple `ValueDB`.
`duration`	Time it took to compute the value in seconds.

Since it is a subclass of ExprValBase, the FrozenExprVal value of the parameters of the callable that was used to compute it can be accessed using the subscript operator [].

Instances of this class will not refer to the callable that was used to create the values, and will record the ID of the values instead of recomputing it from the graph of ExpressionBase. This allows manipulating FrozenExprVal as standalone objects, with minimal references to the code, which improves robustness against API change that would make deserializing them impossible.

See also

ExprValBase

property callable_[source]#

Callable that produced the value.

If it cannot be found, an AttributeError is raised.

property type_[source]#

Type of the value, as reported by the return annotation of the callable that produced it.

If the callable cannot be found, the actual value type is used. If the value type is compatible with the return annotation of the callable (i.e. is a subclass), the value type is returned as well.

property type_names[source]#

classmethod from_expr_val(expr_val, hidden_callable_set=None)[source]#

Build a FrozenExprVal from one ExprVal.

Parameters:: hidden_callable_set (set(collections.abc.Callable)) – Set of callables that should not appear in the ID.

get_id(full_qual=True, qual=True, with_tags=True, remove_tags={})[source]#: Return recorded IDs generated using ExprVal.get_id().

get_tags()[source]#

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

property cumulative_duration#

Inherited property, see exekall.engine.ExprValBase.cumulative_duration

Sum of the duration of all ExprValBase that were involved in the computation of that one.

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

format_structure(full_qual=True)#

Inherited method, see exekall.engine.ExprValBase.format_structure()

Format the value and its parents in a human readable way.

get()#: Inherited method, see collections.abc.Mapping.get()

get_by_predicate(predicate)#

Inherited method, see exekall.engine.ExprValBase.get_by_predicate()

Get a list of parents ExprValBase for which the predicate returns True.

get_by_type(cls, include_subclasses=True, **kwargs)#

Inherited method, see exekall.engine.ExprValBase.get_by_type()

Get a list of parents ExprValBase having a value of the given type.

get_excep()#

Inherited method, see exekall.engine.ExprValBase.get_excep()

Get all the parents ExprValBase for which an exception was raised.

get_full_log(level)#

Inherited method, see exekall.engine.ExprValBase.get_full_log()

Reconstruct a consistent log output at the given level by stitching the logs of all parent ExprValBase that were involved in the computation of that value.

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.PrunedFrozVal(froz_val)[source]#

Bases: FrozenExprVal

Placeholder introduced by ValueDB.prune_by_predicate() when a FrozenExprVal is pruned.

property cumulative_duration[source]#: Sum of the duration of all ExprValBase that were involved in the computation of that one.

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

property callable_#

Inherited property, see exekall.engine.FrozenExprVal.callable_

Callable that produced the value.

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

format_structure(full_qual=True)#

Inherited method, see exekall.engine.ExprValBase.format_structure()

Format the value and its parents in a human readable way.

classmethod from_expr_val(expr_val, hidden_callable_set=None)#

Inherited method, see exekall.engine.FrozenExprVal.from_expr_val()

Build a FrozenExprVal from one ExprVal.

get()#: Inherited method, see collections.abc.Mapping.get()

get_by_predicate(predicate)#

Inherited method, see exekall.engine.ExprValBase.get_by_predicate()

Get a list of parents ExprValBase for which the predicate returns True.

get_by_type(cls, include_subclasses=True, **kwargs)#

Inherited method, see exekall.engine.ExprValBase.get_by_type()

Get a list of parents ExprValBase having a value of the given type.

get_excep()#

Inherited method, see exekall.engine.ExprValBase.get_excep()

Get all the parents ExprValBase for which an exception was raised.

get_full_log(level)#

Inherited method, see exekall.engine.ExprValBase.get_full_log()

Reconstruct a consistent log output at the given level by stitching the logs of all parent ExprValBase that were involved in the computation of that value.

get_id(full_qual=True, qual=True, with_tags=True, remove_tags={})#

Inherited method, see exekall.engine.FrozenExprVal.get_id()

Return recorded IDs generated using ExprVal.get_id().

get_tags()#: Inherited method, see exekall.engine.FrozenExprVal.get_tags()

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

property type_#

Inherited property, see exekall.engine.FrozenExprVal.type_

Type of the value, as reported by the return annotation of the callable that produced it.

property type_names#: Inherited property, see exekall.engine.FrozenExprVal.type_names

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.FrozenExprValSeq(froz_val_list, param_map)[source]#

Bases: Sequence

Sequence of FrozenExprVal analogous to ExprValSeq.

Parameters:

froz_val_list (list(FrozenExprVal)) – List of FrozenExprVal.
param_map (dict) – Parameter map that was used to compute the ExprVal. See ExprValSeq.

Since it inherits from collections.abc.Sequence, it can be iterated over directly.

__getitem__(k)[source]#

__len__()[source]#

classmethod from_expr_val_seq(expr_val_seq, **kwargs)[source]#

Build a FrozenExprValSeq from an ExprValSeq.

Variable keyword arguments:: Forwarded to FrozenExprVal.from_expr_val().

classmethod from_expr_list(expr_list, **kwargs)[source]#

Build a list of FrozenExprValSeq from an list of ComputableExpression.

Parameters:: expr_list (list(ComputableExpression)) – List of ComputableExpression to extract the ExprVal from.
Variable keyword arguments:: Forwarded to from_expr_val_seq().

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Sequence.__contains__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see collections.abc.Sequence.__iter__()

__new__()#: Inherited method, see object.__new__()

__reversed__()#: Inherited method, see collections.abc.Sequence.__reversed__()

__slots__ = ()#

count()#: Inherited method, see collections.abc.Sequence.count()

index()#: Inherited method, see collections.abc.Sequence.index()

class exekall.engine.ExprValLog(log_map, utc_datetime)[source]#

Bases: object

Logging output created when computing an ExprValBase.

Parameters:

log_map (dict(str, str)) – Mapping of log level name to log content.
utc_datetime (datetime.datetime) – UTC timestamp as a datetime object corresponding to the beginning of the log.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ExprVal(expr, param_map, value=NoValue, excep=NoValue, uuid=None, duration=None, log=None)[source]#

Bases: ExprValBase

Value computed when executing ComputableExpression.

Parameters:

expr (ExpressionBase) – Expression for which this value was computed
uuid (str) – UUID of the value.

See also

ExprValBase for the other parameters.

get_tags()[source]#: Return a dictionary of the tags.

format_tags(remove_tags={})[source]#

Return a formatted string for the tags of that ExprVal.

Parameters:: remove_tags (set(str)) – Do not include those tags

classmethod validate(expr_val_list)[source]#: Check that the list contains only one ExprVal for each ComputableExpression, unless it is non reusable.

get_id(*args, with_tags=True, **kwargs)[source]#: See ExpressionBase.get_id.

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

property cumulative_duration#

Inherited property, see exekall.engine.ExprValBase.cumulative_duration

Sum of the duration of all ExprValBase that were involved in the computation of that one.

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

format_structure(full_qual=True)#

Inherited method, see exekall.engine.ExprValBase.format_structure()

Format the value and its parents in a human readable way.

get()#: Inherited method, see collections.abc.Mapping.get()

get_by_predicate(predicate)#

Inherited method, see exekall.engine.ExprValBase.get_by_predicate()

Get a list of parents ExprValBase for which the predicate returns True.

get_by_type(cls, include_subclasses=True, **kwargs)#

Inherited method, see exekall.engine.ExprValBase.get_by_type()

Get a list of parents ExprValBase having a value of the given type.

get_excep()#

Inherited method, see exekall.engine.ExprValBase.get_excep()

Get all the parents ExprValBase for which an exception was raised.

get_full_log(level)#

Inherited method, see exekall.engine.ExprValBase.get_full_log()

Reconstruct a consistent log output at the given level by stitching the logs of all parent ExprValBase that were involved in the computation of that value.

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.UnEvaluatedExprVal(expr)[source]#

Bases: ExprVal

Placeholder ExprVal created when computing the value was known to not lead to anything useful.

__class__#: alias of ABCMeta

classmethod __class_getitem__()#: Inherited method, see collections.abc.Iterable.__class_getitem__()

__contains__()#: Inherited method, see collections.abc.Mapping.__contains__()

__getitem__(k)#: Inherited method, see exekall.engine.ExprHelpers.__getitem__()

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__iter__()#: Inherited method, see exekall.engine.ExprHelpers.__iter__()

__len__()#: Inherited method, see exekall.engine.ExprHelpers.__len__()

__new__()#: Inherited method, see object.__new__()

__reversed__ = None#

__slots__ = ()#

property cumulative_duration#

Inherited property, see exekall.engine.ExprValBase.cumulative_duration

Sum of the duration of all ExprValBase that were involved in the computation of that one.

fold(f, init=None, visit_once=False)#

Inherited method, see exekall.engine.ExprHelpers.fold()

Fold the function f over the instance and all its parents listed in param_map attribute, deep first.

format_structure(full_qual=True)#

Inherited method, see exekall.engine.ExprValBase.format_structure()

Format the value and its parents in a human readable way.

format_tags(remove_tags={})#

Inherited method, see exekall.engine.ExprVal.format_tags()

Return a formatted string for the tags of that ExprVal.

get()#: Inherited method, see collections.abc.Mapping.get()

get_by_predicate(predicate)#

Inherited method, see exekall.engine.ExprValBase.get_by_predicate()

Get a list of parents ExprValBase for which the predicate returns True.

get_by_type(cls, include_subclasses=True, **kwargs)#

Inherited method, see exekall.engine.ExprValBase.get_by_type()

Get a list of parents ExprValBase having a value of the given type.

get_excep()#

Inherited method, see exekall.engine.ExprValBase.get_excep()

Get all the parents ExprValBase for which an exception was raised.

get_full_log(level)#

Inherited method, see exekall.engine.ExprValBase.get_full_log()

Reconstruct a consistent log output at the given level by stitching the logs of all parent ExprValBase that were involved in the computation of that value.

get_id(*args, with_tags=True, **kwargs)#

Inherited method, see exekall.engine.ExprVal.get_id()

See ExpressionBase.get_id.

get_tags()#

Inherited method, see exekall.engine.ExprVal.get_tags()

Return a dictionary of the tags.

items()#: Inherited method, see collections.abc.Mapping.items()

keys()#: Inherited method, see collections.abc.Mapping.keys()

classmethod validate(expr_val_list)#

Inherited method, see exekall.engine.ExprVal.validate()

Check that the list contains only one ExprVal for each ComputableExpression, unless it is non reusable.

values()#: Inherited method, see collections.abc.Mapping.values()

class exekall.engine.Consumer[source]#

Bases: object

Placeholder type used in PEP 484 annotations by callables to refer to the callable that will use their value.

Note

This leads to cloning the expression refering to its consumer for each different consumer.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

class exekall.engine.ExprData[source]#

Bases: dict

Placeholder type used in PEP 484 annotations by callables to refer to the expression-wide data dictionnary.

__class__#: alias of type

__class_getitem__()#: Inherited method, see dict.__class_getitem__()

__hash__ = None#

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see dict.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

fromkeys()#: Inherited method, see dict.fromkeys()

Customization#

class exekall.customization.AdaptorBase(args)[source]#

Bases: object

Base class of all adaptors.

Parameters:: args (argparse.Namespace) – Command line argument namespace as returned by argparse.ArgumentParser.parse_args(). Depending on the subcommand used, it could be the arguments of exekall run or exekall compare (or any other subcommand).

An adaptor is a class providing a number of hooks to customize the behaviour of exekall on a given codebase. It should be implemented in a module called exekall_customize in order to be found by exekall run.

name = 'default'#

get_non_reusable_type_set()[source]#

Return a set of non-reusable types.

Defaults to an empty set.

static get_tags(value)[source]#

Returns a dictionnary of tag names to their value.

Default to empty set of tags, unless value is a numbers.Number in which case the value of the number is used.

filter_op_set(op_set)[source]#

Returns a potentially filtered set of exekall.engine.Operator.

This allows removing some operators from the ones that will be used to build expressions. Defaults to filtering out operators without any parameter, since the “spark” values are usually introduced by the adaptor directly, instead of calling functions from the code base.

format_expr_list(expr_list, verbose=0)[source]#

Return a string that is printed right after the list of executed expressions.

Parameters:: expr_list (list(exekall.engine.ExpressionBase)) – List of exekall.engine.ExpressionBase that will be executed. Note that this list has not yet undergone CSE, cloning for multiple iterations or other transformations.

This can be used to add some information about the expressions that are about to be executed.

get_prebuilt_op_set()[source]#

Returns a set of exekall.engine.PrebuiltOperator.

This allows injecting any “spark” value that is needed to build the expressions, like configuration objects. These values are usually built out of the custom CLI parameters added by the adaptor.

get_hidden_op_set(op_set)[source]#

Returns the set of hidden exekall.engine.Operator.

This allows hiding parts of the IDs that would not add much information but clutter them.

static register_run_param(parser)[source]#

Parameters:: parser (argparse.ArgumentParser) – Parser of the run subcommand to add arguments onto.

static register_compare_param(parser)[source]#

Parameters:: parser (argparse.ArgumentParser) – Parser of the compare subcommand to add arguments onto.

compare_db_list(db_list)[source]#

Compare databases listed in db_list.

Parameters:: db_list (list(exekall.engine.ValueDB)) – List of exekall.engine.ValueDB to compare.

This is called by exekall compare to actually do something useful.

static register_show_param(parser)[source]#

Parameters:: parser (argparse.ArgumentParser) – Parser of the show subcommand to add arguments onto.

show_db(db)[source]#

Show the content of the database.

Parameters:: db (exekall.engine.ValueDB) – exekall.engine.ValueDB to show.

This is called by exekall show to actually do something useful.

static get_default_type_goal_pattern_set()[source]#: Returns a set of patterns that will be used as the default value for exekall run --goal.

classmethod reload_db(db, path=None)[source]#

Hook called when reloading a serialized exekall.engine.ValueDB. The returned database will be used.

Parameters:

db (exekall.engine.ValueDB) – exekall.engine.ValueDB that has just been deserialized.
path (str or None) – Path of the file of the serialized database if available.

finalize_expr(expr)[source]#: Finalize an exekall.engine.ComputableExpression right after all its values have been computed.

format_result(expr_val)[source]#: Format an exekall.engine.ExprVal that is the result of an expression. It should return a (short) string that will be displayed at the end of the computation.

get_summary(result_map)[source]#

Return the summary of an exekall run session as a string.

Parameters:: result_map (dict(exekall.engine.ComputableExpression, list(exekall.engine.ExprVal))) – Dictionary of expressions to the list of their values.

get_run_exit_code(result_map)[source]#

Return an integer used as exekall run exit status.

Parameters:: result_map (dict(exekall.engine.ComputableExpression, exekall.engine.ExprVal)) – Dictionary of expressions to the list of their values.

classmethod get_adaptor_cls(name=None)[source]#: Return the adaptor class that has the name name.

Note

This is not intended to be overriden by subclasses.

__class__#: alias of type

__init_subclass__()#: Inherited method, see object.__init_subclass__()

__new__()#: Inherited method, see object.__new__()

__subclasshook__()#: Inherited method, see object.__subclasshook__()

Utils#

exekall.utils.get_callable_set(module_set, verbose=False)[source]#

Get the set of callables defined in all modules of module_set.

We ignore any callable that is defined outside of the modules’ package.

Parameters:: module_set (set(types.ModuleType)) – Set of modules to scan.

exekall.utils.sweep_param(callable_, param, start, stop, step=1)[source]#

Used to generate a stream of numbers or strings to feed to a callable.

Parameters:

callable (collections.abc.Callable) – Callable the numbers will be used by.
param (str) – Name of the parameter of the callable the numbers will be providing values for.
start (str) – Starting value.
stop (str) – End value (inclusive)
step (str) – Increment step.

If start == stop, only that value will be yielded, and it can be of any type.

The type used will either be one that is annotated on the callable, or the one from the default value if no annotation is available, or float if no default value is found. The value will then be built by passing the string to the type as only parameter.

exekall.utils.get_method_class(function)[source]#: Get the class in which a function is defined.

exekall._utils.LOGGING_OUT_LEVEL = 60#

Log level used for the OUT level.

This allows sending all the output through the logging module instead of using print(), so it can easily be recorded to a file

exekall._utils.NoValue = NoValue#: Singleton with similar purposes as None.

Internal API#

Engine#

Customization#

Utils#

This Page