.. title:: datautils lisa.datautils ============== .. toctree:: :hidden: NO_INDEX DataAccessor DataFrameAccessor SeriesAccessor SignalDesc Timestamp df_add_delta df_combine df_combine_duplicates df_convert_to_nullable df_deduplicate df_delta df_dereference df_filter df_filter_task_ids df_find_redundant_cols df_make_empty_clone df_merge df_refit_index df_split_signals df_squash df_update_duplicates df_window df_window_signals series_align_signal series_combine series_convert series_deduplicate series_dereference series_derivate series_envelope_mean series_integrate series_local_extremum series_mean series_refit_index series_rolling_apply series_update_duplicates series_window series_tunnel_mean .. module:: lisa.datautils .. withrefctx:: :module: lisa.datautils .. automodule:: lisa.datautils :no-index: :no-members: :no-inherited-members: :no-undoc-members: :no-private-members: :no-special-members: .. rubric:: Globals .. list-table:: :align: left * - :data:`~lisa.datautils.NO_INDEX` - .. withrefctx:: :module: lisa.datautils .. rubric:: Classes .. list-table:: :align: left * - :class:`~lisa.datautils.DataAccessor` - .. withrefctx:: :module: lisa.datautils :class: lisa.datautils.DataAccessor Proxy class that allows extending the :class:`pandas.DataFrame` API. * - :class:`~lisa.datautils.DataFrameAccessor` - .. withrefctx:: :module: lisa.datautils :class: lisa.datautils.DataFrameAccessor Proxy class that allows extending the :class:`pandas.DataFrame` API. * - :class:`~lisa.datautils.SeriesAccessor` - .. withrefctx:: :module: lisa.datautils :class: lisa.datautils.SeriesAccessor Proxy class that allows extending the :class:`pandas.DataFrame` API. * - :class:`~lisa.datautils.SignalDesc` - .. withrefctx:: :module: lisa.datautils :class: lisa.datautils.SignalDesc Define a signal to be used by various signal-oriented APIs. * - :class:`~lisa.datautils.Timestamp` - .. withrefctx:: :module: lisa.datautils :class: lisa.datautils.Timestamp Nanosecond-precision timestamp. It inherits from ``float`` and as such can be manipulating as a floating point number of seconds. The ``nanoseconds`` attribute allows getting the exact timestamp regardless of the magnitude of the float, allowing for more precise computation. .. rubric:: Functions .. list-table:: :align: left * - :func:`~lisa.datautils.df_add_delta` - .. withrefctx:: :module: lisa.datautils Add a column containing the delta of the given other column. * - :func:`~lisa.datautils.df_combine` - .. withrefctx:: :module: lisa.datautils Same as :meth:`pandas.DataFrame.combine` on a list of series rather than just two. * - :func:`~lisa.datautils.df_combine_duplicates` - .. withrefctx:: :module: lisa.datautils Combine the duplicated rows using ``func`` and remove the duplicates. * - :func:`~lisa.datautils.df_convert_to_nullable` - .. withrefctx:: :module: lisa.datautils Convert the columns of the dataframe to their equivalent nullable dtype, when possible. * - :func:`~lisa.datautils.df_deduplicate` - .. withrefctx:: :module: lisa.datautils Same as :func:`series_deduplicate` but for :class:`pandas.DataFrame`. * - :func:`~lisa.datautils.df_delta` - .. withrefctx:: :module: lisa.datautils pre_df and post_df containing paired/consecutive events indexed by time, df_delta() merges the two dataframes and adds a ``delta`` column containing the time spent between the two events. A typical usecase would be adding pre/post events at the entry/exit of a function. * - :func:`~lisa.datautils.df_dereference` - .. withrefctx:: :module: lisa.datautils Similar to :func:`series_dereference`. * - :func:`~lisa.datautils.df_filter` - .. withrefctx:: :module: lisa.datautils Filter the content of a dataframe. * - :func:`~lisa.datautils.df_filter_task_ids` - .. withrefctx:: :module: lisa.datautils Filter a dataframe using a list of :class:`lisa.analysis.tasks.TaskID`. * - :func:`~lisa.datautils.df_find_redundant_cols` - .. withrefctx:: :module: lisa.datautils Find the columns that are redundant to ``col``, i.e. that can be computed as ``df[x] = df[col].map(dict(...))``. * - :func:`~lisa.datautils.df_make_empty_clone` - .. withrefctx:: :module: lisa.datautils Make an empty clone of the given dataframe. * - :func:`~lisa.datautils.df_merge` - .. withrefctx:: :module: lisa.datautils Merge a list of :class:`pandas.DataFrame`, keeping the index sorted. * - :func:`~lisa.datautils.df_refit_index` - .. withrefctx:: :module: lisa.datautils Same as :func:`series_refit_index` but acting on :class:`pandas.DataFrame`. * - :func:`~lisa.datautils.df_split_signals` - .. withrefctx:: :module: lisa.datautils Yield subset of ``df`` that only contain one signal, along with the signal identification values. * - :func:`~lisa.datautils.df_squash` - .. withrefctx:: :module: lisa.datautils Slice a dataframe of deltas in [start:end] and ensure we have an event at exactly those boundaries. * - :func:`~lisa.datautils.df_update_duplicates` - .. withrefctx:: :module: lisa.datautils Same as :func:`series_update_duplicates` but on a :class:`pandas.DataFrame`. * - :func:`~lisa.datautils.df_window` - .. withrefctx:: :module: lisa.datautils Same as :func:`series_window` but acting on a :class:`pandas.DataFrame`. * - :func:`~lisa.datautils.df_window_signals` - .. withrefctx:: :module: lisa.datautils Similar to :func:`df_window` with ``method='pre'`` but guarantees that each signal will have a values at the beginning of the window. * - :func:`~lisa.datautils.series_align_signal` - .. withrefctx:: :module: lisa.datautils Align a signal to an expected reference signal using their cross-correlation. * - :func:`~lisa.datautils.series_combine` - .. withrefctx:: :module: lisa.datautils Same as :meth:`pandas.Series.combine` on a list of series rather than just two. * - :func:`~lisa.datautils.series_convert` - .. withrefctx:: :module: lisa.datautils Convert a :class:`pandas.Series` with a best effort strategy. * - :func:`~lisa.datautils.series_deduplicate` - .. withrefctx:: :module: lisa.datautils Remove duplicate values in a :class:`pandas.Series`. * - :func:`~lisa.datautils.series_dereference` - .. withrefctx:: :module: lisa.datautils Replace each value in ``series`` by the value at the corresponding index by the source indicated by ``series``'s value. * - :func:`~lisa.datautils.series_derivate` - .. withrefctx:: :module: lisa.datautils Compute a derivative of a :class:`pandas.Series` with respect to another series. * - :func:`~lisa.datautils.series_envelope_mean` - .. withrefctx:: :module: lisa.datautils Compute the average between the mean of local maximums and local minimums of the series. * - :func:`~lisa.datautils.series_integrate` - .. withrefctx:: :module: lisa.datautils Compute the integral of `y` with respect to `x`. * - :func:`~lisa.datautils.series_local_extremum` - .. withrefctx:: :module: lisa.datautils Returns a series of local extremum. * - :func:`~lisa.datautils.series_mean` - .. withrefctx:: :module: lisa.datautils Compute the average of `y` by integrating with respect to `x` and dividing by the range of `x`. * - :func:`~lisa.datautils.series_refit_index` - .. withrefctx:: :module: lisa.datautils Slice a series using :func:`series_window` and ensure we have a value at exactly the specified boundaries, unless the signal started after the beginning of the required window. * - :func:`~lisa.datautils.series_rolling_apply` - .. withrefctx:: :module: lisa.datautils Apply a function on a rolling window of a series. * - :func:`~lisa.datautils.series_update_duplicates` - .. withrefctx:: :module: lisa.datautils Update a given series to avoid duplicated values. * - :func:`~lisa.datautils.series_window` - .. withrefctx:: :module: lisa.datautils Select a portion of a :class:`pandas.Series`.