.. title:: utils lisa.utils ========== .. toctree:: :hidden: LISA_CACHE_HOME LISA_HOME LISA_HOST_ABI TASK_COMM_MAX_LEN ArtifactPath ComposedContextManager DirCache ExekallTaggable FrozenDict HideExekallID LazyMapping Loggable PartialInit Serializable SerializeViaConstructor SimpleHash UnboundMethodType UnknownTagPlaceholder bothmethod instancemethod mappable DelegateToAttr add annotations_from_signature chain_cm checksum compose consume curry deduplicate delegate_getattr deprecate destroyablecontextmanager dispatch_kwargs docstring_update ffill filter_values fixedpoint fold foldr get_call_site get_common_ancestor get_common_prefix get_doc_url get_nested_key get_obj_name get_parent_namespace get_short_doc get_sphinx_role get_subclasses group_by_value groupby grouper guess_format ignore_exceps import_all_submodules is_link_dead is_monotonic is_running_ipython is_running_sphinx kwargs_dispatcher kwargs_forwarded_to loopify lru_memoized measure_time memoized mp_spawn_pool namedtuple newtype non_recursive_property optional_kwargs order_as resolve_dotted_name set_nested_key setup_logging show_doc sig_bind sphinx_nitpick_ignore sphinx_register_nitpick_ignore split_paragraphs subprocess_detailed_excep subprocess_log take unzip_into update_params_from update_wrapper_doc value_range get_cls_name get_sphinx_name ContextManagerDestroyed ContextManagerExcep ContextManagerExit ContextManagerNoExcep .. module:: lisa.utils .. withrefctx:: :module: lisa.utils .. automodule:: lisa.utils :no-index: :no-members: :no-inherited-members: :no-undoc-members: :no-private-members: :no-special-members: .. rubric:: Globals .. list-table:: :align: left * - :data:`~lisa.utils.LISA_CACHE_HOME` - .. withrefctx:: :module: lisa.utils Base folder used for caching files. * - :data:`~lisa.utils.LISA_HOME` - .. withrefctx:: :module: lisa.utils The detected location of your LISA installation. * - :data:`~lisa.utils.LISA_HOST_ABI` - .. withrefctx:: :module: lisa.utils ABI of the machine that imported that module. * - :data:`~lisa.utils.TASK_COMM_MAX_LEN` - .. withrefctx:: :module: lisa.utils Value of ``TASK_COMM_LEN - 1`` macro in the kernel, to account for ``\0`` terminator. .. rubric:: Classes .. list-table:: :align: left * - :class:`~lisa.utils.ArtifactPath` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ArtifactPath Path to a folder that can be used to store artifacts of a function. This must be a clean folder, already created on disk. * - :class:`~lisa.utils.ComposedContextManager` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ComposedContextManager Compose context managers together. * - :class:`~lisa.utils.DirCache` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.DirCache Provide a folder-based cache. * - :class:`~lisa.utils.ExekallTaggable` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ExekallTaggable Allows tagging the objects produced in exekall expressions ID. * - :class:`~lisa.utils.FrozenDict` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.FrozenDict Read-only mapping that is therefore hashable. * - :class:`~lisa.utils.HideExekallID` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.HideExekallID Hide the subclasses in the simplified ID format of exekall. * - :class:`~lisa.utils.LazyMapping` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.LazyMapping Lazy Mapping dict-like class for elements evaluated on the fly. * - :class:`~lisa.utils.Loggable` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.Loggable A simple class for uniformly named loggers. * - :class:`~lisa.utils.PartialInit` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.PartialInit Allow partial initialization of instances with curry-like behaviour for the constructor. * - :class:`~lisa.utils.Serializable` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.Serializable A helper class for YAML serialization/deserialization. * - :class:`~lisa.utils.SerializeViaConstructor` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.SerializeViaConstructor Base class providing serialization to objects that typically cannot due to unpicklable attributes. * - :class:`~lisa.utils.SimpleHash` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.SimpleHash Base class providing a basic implementation of ``__eq__`` and ``__hash__``: two instances are equal if their ``__dict__`` and ``__class__`` attributes are equal. * - :class:`~lisa.utils.UnboundMethodType` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.UnboundMethodType Dummy class to be used to check if a function is a method defined in a class or not::. * - :class:`~lisa.utils.UnknownTagPlaceholder` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.UnknownTagPlaceholder * - :class:`~lisa.utils.bothmethod` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.bothmethod Decorator to allow a method to be used both as an instance method and a classmethod. * - :class:`~lisa.utils.instancemethod` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.instancemethod Decorator providing a hybrid of a normal method and a classmethod:. * - :class:`~lisa.utils.mappable` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.mappable Decorator that allows the decorated function to be mapped on an iterable::. .. rubric:: Functions .. list-table:: :align: left * - :func:`~lisa.utils.DelegateToAttr` - .. withrefctx:: :module: lisa.utils Implement delegation of attribute lookup to attribute named ``attr`` on instances of the classes specified by ``attr_classes``. * - :func:`~lisa.utils.add` - .. withrefctx:: :module: lisa.utils Same as :func:`sum` but works on any object that defines ``__add__``. * - :func:`~lisa.utils.annotations_from_signature` - .. withrefctx:: :module: lisa.utils Build a PEP484 ``__annotations__`` dictionary from a :class:`inspect.Signature`. * - :func:`~lisa.utils.chain_cm` - .. withrefctx:: :module: lisa.utils Chain the context managers returned by the given callables. * - :func:`~lisa.utils.checksum` - .. withrefctx:: :module: lisa.utils Compute a checksum on a given file-like object. * - :func:`~lisa.utils.compose` - .. withrefctx:: :module: lisa.utils Compose multiple functions such that ``compose(f, g)(x) == g(f(x))``. * - :func:`~lisa.utils.consume` - .. withrefctx:: :module: lisa.utils Advance the iterator n-steps ahead. If ``n`` is None, consume entirely. * - :func:`~lisa.utils.curry` - .. withrefctx:: :module: lisa.utils Currify the given function such that ``f(x, y) == curry(f)(x)(y)``. * - :func:`~lisa.utils.deduplicate` - .. withrefctx:: :module: lisa.utils Deduplicate items in the given sequence and return a list. :param seq: Sequence to deduplicate :type Seq: collections.abc.Sequence. * - :func:`~lisa.utils.delegate_getattr` - .. withrefctx:: :module: lisa.utils Somewhat equivalent to ``x..``. * - :func:`~lisa.utils.deprecate` - .. withrefctx:: :module: lisa.utils Mark a class, method, function etc as deprecated and update its docstring. * - :func:`~lisa.utils.destroyablecontextmanager` - .. withrefctx:: :module: lisa.utils Similar to :func:`contextlib.contextmanager` but treats all cases of ``yield`` as an exception. * - :func:`~lisa.utils.dispatch_kwargs` - .. withrefctx:: :module: lisa.utils Dispatch the provided ``kwargs`` mapping to the ``funcs`` functions, based on their signature. * - :func:`~lisa.utils.docstring_update` - .. withrefctx:: :module: lisa.utils Create a class to inherit from in order to add a snippet of doc at the end of the docstring of all direct and indirect subclasses::. * - :func:`~lisa.utils.ffill` - .. withrefctx:: :module: lisa.utils Forward fill an iterator with the last selected value. * - :func:`~lisa.utils.filter_values` - .. withrefctx:: :module: lisa.utils Yield value from ``iterable`` unless they are in ``values``. * - :func:`~lisa.utils.fixedpoint` - .. withrefctx:: :module: lisa.utils Find the fixed point of a function ``f`` with the initial parameter ``init``. * - :func:`~lisa.utils.fold` - .. withrefctx:: :module: lisa.utils Fold the given function over ``xs``, with ``init`` initial accumulator value. * - :func:`~lisa.utils.foldr` - .. withrefctx:: :module: lisa.utils Right-associative version of :func:`fold`. * - :func:`~lisa.utils.get_call_site` - .. withrefctx:: :module: lisa.utils Get the location of the source that called that function. * - :func:`~lisa.utils.get_common_ancestor` - .. withrefctx:: :module: lisa.utils Pick the most derived common ancestor between the classes, assuming single inheritance. * - :func:`~lisa.utils.get_common_prefix` - .. withrefctx:: :module: lisa.utils Return the common prefix of the passed iterables as an iterator. * - :func:`~lisa.utils.get_doc_url` - .. withrefctx:: :module: lisa.utils Return an URL to the documentation about the given object. * - :func:`~lisa.utils.get_nested_key` - .. withrefctx:: :module: lisa.utils Get a key in a nested mapping. * - :func:`~lisa.utils.get_obj_name` - .. withrefctx:: :module: lisa.utils Get a prettily-formated name for the object given as parameter. * - :func:`~lisa.utils.get_parent_namespace` - .. withrefctx:: :module: lisa.utils Return the enclosing namespace of ``obj`` (a class or a module). * - :func:`~lisa.utils.get_short_doc` - .. withrefctx:: :module: lisa.utils Get the short documentation paragraph at the beginning of docstrings. * - :func:`~lisa.utils.get_sphinx_role` - .. withrefctx:: :module: lisa.utils Return the reStructuredText Sphinx role of a given object. * - :func:`~lisa.utils.get_subclasses` - .. withrefctx:: :module: lisa.utils Get all indirect subclasses of the class. * - :func:`~lisa.utils.group_by_value` - .. withrefctx:: :module: lisa.utils Group a mapping by its values. * - :func:`~lisa.utils.groupby` - .. withrefctx:: :module: lisa.utils Equivalent of :func:`itertools.groupby`, with a pre-sorting so it works as expected. * - :func:`~lisa.utils.grouper` - .. withrefctx:: :module: lisa.utils Collect data into fixed-length chunks or blocks. * - :func:`~lisa.utils.guess_format` - .. withrefctx:: :module: lisa.utils Guess the file format from a `path`, using the mime types database. * - :func:`~lisa.utils.ignore_exceps` - .. withrefctx:: :module: lisa.utils Wrap a context manager and handle exceptions raised in ``__enter__()`` and ``__exit__()``. * - :func:`~lisa.utils.import_all_submodules` - .. withrefctx:: :module: lisa.utils Import all submodules of a given package. * - :func:`~lisa.utils.is_link_dead` - .. withrefctx:: :module: lisa.utils Check if link is dead. If dead, returns a truthy value, otherwise a falsy one. * - :func:`~lisa.utils.is_monotonic` - .. withrefctx:: :module: lisa.utils Return ``True`` if the given sequence is monotonic, ``False`` otherwise. * - :func:`~lisa.utils.is_running_ipython` - .. withrefctx:: :module: lisa.utils Returns True if running in IPython console or Jupyter notebook, False otherwise. * - :func:`~lisa.utils.is_running_sphinx` - .. withrefctx:: :module: lisa.utils Returns True if the module is imported when Sphinx is running, False otherwise. * - :func:`~lisa.utils.kwargs_dispatcher` - .. withrefctx:: :module: lisa.utils Decorate a function so that it acts as an argument dispatcher between multiple other functions. * - :func:`~lisa.utils.kwargs_forwarded_to` - .. withrefctx:: :module: lisa.utils Similar to :func:`functools.wraps`, except that it will only fixup the signature. * - :func:`~lisa.utils.loopify` - .. withrefctx:: :module: lisa.utils Try to factor an iterable into a prefix that is repeated a number of times. * - :func:`~lisa.utils.lru_memoized` - .. withrefctx:: :module: lisa.utils Decorator to memoize the result of a callable, based on :func:`functools.lru_cache`. * - :func:`~lisa.utils.measure_time` - .. withrefctx:: :module: lisa.utils Context manager to measure time in seconds. * - :func:`~lisa.utils.memoized` - .. withrefctx:: :module: lisa.utils Decorator to memoize the result of a callable, based on :func:`functools.lru_cache`. * - :func:`~lisa.utils.mp_spawn_pool` - .. withrefctx:: :module: lisa.utils Create a context manager wrapping :class:`multiprocessing.pool.Pool` using the ``spawn`` method, which is safe even in multithreaded applications. * - :func:`~lisa.utils.namedtuple` - .. withrefctx:: :module: lisa.utils Same as :func:`collections.namedtuple`, with :class:`collections.abc.Mapping` behaviour. * - :func:`~lisa.utils.newtype` - .. withrefctx:: :module: lisa.utils Make a new class inheriting from ``cls`` with the given ``name``. * - :func:`~lisa.utils.non_recursive_property` - .. withrefctx:: :module: lisa.utils Create a property that raises an :exc:`AttributeError` if it is re-entered. * - :func:`~lisa.utils.optional_kwargs` - .. withrefctx:: :module: lisa.utils Decorator used to allow another decorator to both take keyword parameters when called, and none when not called::. * - :func:`~lisa.utils.order_as` - .. withrefctx:: :module: lisa.utils Reorder the iterable of ``items`` to match the sequence in ``order_as``. Items present in ``items`` and not in ``order_as`` will be appended at the end, in appearance order. * - :func:`~lisa.utils.resolve_dotted_name` - .. withrefctx:: :module: lisa.utils Resolve a dotted name, importing all modules necessary. * - :func:`~lisa.utils.set_nested_key` - .. withrefctx:: :module: lisa.utils Set a key in a nested mapping. * - :func:`~lisa.utils.setup_logging` - .. withrefctx:: :module: lisa.utils Initialize logging used for all the LISA modules. * - :func:`~lisa.utils.show_doc` - .. withrefctx:: :module: lisa.utils Show the online LISA documentation about the given object. * - :func:`~lisa.utils.sig_bind` - .. withrefctx:: :module: lisa.utils Similar to :meth:`inspect.Signature.bind` but expands variable keyword arguments so that the resulting dictionary can be used directly in a function call. * - :func:`~lisa.utils.sphinx_nitpick_ignore` - .. withrefctx:: :module: lisa.utils Set of objects to ignore without warning when cross referencing in Sphinx. * - :func:`~lisa.utils.sphinx_register_nitpick_ignore` - .. withrefctx:: :module: lisa.utils Register an object with a name that cannot be resolved and therefore cross referenced by Sphinx. * - :func:`~lisa.utils.split_paragraphs` - .. withrefctx:: :module: lisa.utils Split `string` into a list of paragraphs. * - :func:`~lisa.utils.subprocess_detailed_excep` - .. withrefctx:: :module: lisa.utils Context manager that will replace :class:`subprocess.CalledProcessError` by a subclass that shows more details. * - :func:`~lisa.utils.subprocess_log` - .. withrefctx:: :module: lisa.utils Similar to :func:`subprocess.check_output` but merges stdout and stderr and logs them using the :mod:`logging` module as it goes. * - :func:`~lisa.utils.take` - .. withrefctx:: :module: lisa.utils Yield the first ``n`` items of an iterator, if ``n`` positive, or last items otherwise. * - :func:`~lisa.utils.unzip_into` - .. withrefctx:: :module: lisa.utils Unzip a given ``iterator`` into ``n`` variables. * - :func:`~lisa.utils.update_params_from` - .. withrefctx:: :module: lisa.utils Decorator to update the signature of the decorated function using annotation and default values from the specified ``f`` function. * - :func:`~lisa.utils.update_wrapper_doc` - .. withrefctx:: :module: lisa.utils Equivalent to :func:`functools.wraps` that updates the signature by taking into account the wrapper's extra *keyword-only* parameters and the given description. * - :func:`~lisa.utils.value_range` - .. withrefctx:: :module: lisa.utils Equivalent to builtin :class:`range` function, but works for floats as well. .. rubric:: Exceptions .. list-table:: :align: left * - :class:`~lisa.utils.ContextManagerDestroyed` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ContextManagerDestroyed Exception raised in context managers created by :func:`destroyablecontextmanager` when no exception was raised during ``yield`` per say but the context manager was destroyed without calling ``__exit__``. * - :class:`~lisa.utils.ContextManagerExcep` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ContextManagerExcep Exception raised when an exception was raised during ``yield`` in a context manager created with :func:`destroyablecontextmanager`. * - :class:`~lisa.utils.ContextManagerExit` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ContextManagerExit Dummy exception raised in the generator wrapped by :func:`destroyablecontextmanager` when anything else than :exc:`GeneratorExit` happened during ``yield``. * - :class:`~lisa.utils.ContextManagerNoExcep` - .. withrefctx:: :module: lisa.utils :class: lisa.utils.ContextManagerNoExcep Exception raised when no exception was raised during ``yield`` in a context manager created with :func:`destroyablecontextmanager`.