lisa.trace.Trace#

class lisa.trace.Trace(*args, df_fmt=None, **kwargs)[source]#

Bases: TraceBase

This class provides a way to access event dataframes and ties together various low-level moving pieces to make that happen.

Parameters:

  • trace_path (str or None) – File containing the trace

  • events (TraceEventCheckerBase or list(str) or None) –

    events to be parsed. Since events can be loaded on-demand, that is optional but still recommended to improve trace parsing speed.

    See also

    lisa.trace.TraceBase.df_event() for event formats accepted.

  • events_namespaces (list(str or None)) – List of namespaces of the requested events. Each namespace will be tried in order until the event is found. The None namespace can be used to specify no namespace. The full event name is formed with <namespace>__<event>.

  • strict_events (bool) – When True, all the events specified in events have to be present, and any other events will be assumed to not be present. This allows early failure and avoid the cost of lazy detection of events in very large traces.

  • plat_info (lisa.platforms.platinfo.PlatformInfo) – Platform info describing the target that this trace was collected on.

  • normalize_time (bool) – Make the first timestamp in the trace 0 instead of the system timestamp that was captured when tracing.

  • parser (object or None) – Optional trace parser to use as a backend. It must implement the API defined by TraceParserBase, and will be called as parser(path=trace_path, events=events, needed_metadata={'time-range', ...}) with the events that should be parsed. Other parameters must either have default values, or be pre-assigned using partially-applied constructor (for subclasses of lisa.utils.PartialInit) or functools.partial(). By default, .txt files will be assumed to be in human-readable format output directly by the kernel (or trace-cmd report without -R). Support for this format is limited and some events might not be parsed correctly (or at least without custom event parsers).

  • plots_dir (str) – directory where to save plots

  • sanitization_functions (object) – This parameter is only for backward compatibility with existing code, use lisa.trace.TraceBase.get_view() with process_df parameter instead.

  • max_mem_size (int or None) – Maximum memory usage to be used for dataframe cache. Note that the peak memory usage can exceed that, as the cache can not forcefully evict an object from memory (it can only drop references to it). When None, use illimited amount of memory.

  • swap_dir (str or None) – Swap directory used to store dataframes evicted from the cache. When None, a hidden directory along the trace file is used.

  • enable_swap (bool) – If True, the on-disk swap is enabled.

  • max_swap_size (int or None) – Maximum size of the swap directory. When None, the max size is the size of the trace file.

Supporting more events in text parsers:

Note

trace.dat parser can now fully infer the dataframe schema from the binary trace.dat and does not require (nor allow) any manual setting.

Subclasses of TraceParserBase can usually auto-detect the event format, but there may be a number of reasons to pass a custom event parser:

  • The event format produced by a given kernel differs from the description bundled with the parser, leading to incorrect parse (missing field).

  • The event cannot be parsed in raw format in case text output of trace-cmd is used, because of a const char* field displayed as a pointer for example.

    See also

    For events not following the regular field syntax, use CustomFieldsTxtEventParser

  • Automatic detection can take a heavy performance toll. This is why parsers needing descriptions will come with pre-defined descritption of most used events.

Custom event parsers can be passed as extra parameters to the parser, which can be set manually:

# Event parsers provided by TxtTraceParser can also be overridden
# with user-provided ones in case they fail to parse what a given
# kernel produced
event_parsers = [
    TxtEventParser('foobar', fields={'foo': int, 'bar': 'string'}, raw=True)
]

# Pre-fill the "event_parsers" parameter of
# TxtEventParser.from_dat() using a partial application.
#
# Note: you need to choose the parser appropriately for the
# format of the trace, since the automatic filetype detection is
# bypassed.
parser = TxtTraceParser.from_dat(event_parsers=event_parsers)
trace = Trace('foobar.dat', parser=parser)

Warning

Custom event parsers that are not types or functions (such as partially-applied values) will tie the on-disc swap entries to the parser Trace instance, incurring higher pandas.DataFrame load time when a new Trace object is created.

Attributes

base_trace

The original TraceBase this view is based on.

Properties

ana

Allows calling an analysis method on the trace, sharing the dataframe cache.

basetime

Absolute timestamp when the tracing started.

cpus_count

Number of CPUs on which data was gathered in that trace.

end

The timestamp of the last trace event.

endtime

Absolute timestamp when the tracing stopped.

events

Preloaded events as a TraceEventCheckerBase.

events_namespaces

Namespaces events will be looked up in.

normalize_time

True if the trace timestamps were normalized to start at 0.

start

The timestamp of the first trace event.

trace_state

State of the trace object that might impact the output of dataframe getter functions like lisa.trace.TraceBase.df_event().

available_events inherited

Set of available events on that trace.

logger inherited

Convenience short-hand for self.get_logger().

time_range inherited

Duration of that trace (difference between start and end).

window inherited

Same as (trace.start, trace.end).

Methods

__enter__()

__exit__()

__getattr__

Delegate attribute lookup to a private attribute.

df_event()

Get a dataframe containing all occurrences of the specified trace event in the parsed trace.

from_target()

Context manager that can be used to collect a lisa.trace.TraceBase directly from a lisa.target.Target without needing to setup an FtraceCollector.

get_event_sources()

Get the possible sources events of a given event.

get_metadata()

Get metadata from the underlying trace parser.

get_view()

Get a view on a trace.

has_events()

Returns True if the specified event is present in the parsed trace, False otherwise.

__getitem__() inherited

Slice the trace with the given time range.

get_logger() inherited

Provides a logging.Logger named after cls.

log_locals() inherited

Debugging aid: log the local variables of the calling function.

show() inherited

Open the parsed trace using the most appropriate native viewer.

Attributes#

Trace.base_trace#

The original TraceBase this view is based on.

Properties#

property Trace.ana[source]#

Allows calling an analysis method on the trace, sharing the dataframe cache.

Example

Call lisa.analysis.LoadTrackingAnalysis.df_task_signal() on a trace:

df = trace.ana.load_tracking.df_task_signal(task='foo', signal='util')

The trace.ana proxy can also be called like a function to define default values for analysis methods:

ana = trace.ana(task='big_0-3')
ana.load_tracking.df_task_signal(signal='util')

# Equivalent to:
ana.load_tracking.df_task_signal(task='big_0-3', signal='util')

# The proxy can be called again to override the value given to some
# parameters, and the the value can also be overridden when calling the
# method:
ana(task='foo').df_task_signal(signal='util')
ana.df_task_signal(task='foo', signal='util')
property Trace.basetime[source]#

Absolute timestamp when the tracing started.

This might differ from start as the latter can be affected by various normalization or windowing features.

property Trace.cpus_count#

Number of CPUs on which data was gathered in that trace.

This will typically be the number of CPUs on the target, but might sometimes differ depending on the file format of the trace.

property Trace.end[source]#

The timestamp of the last trace event.

property Trace.endtime[source]#

Absolute timestamp when the tracing stopped.

This might differ from end as the latter can be affected by various normalization or windowing features.

Note

With some parsers, that might be the timestamp of the last recorded event instead if the trace end timestamp was not recorded.

property Trace.events#

Preloaded events as a TraceEventCheckerBase.

property Trace.events_namespaces#

Namespaces events will be looked up in.

property Trace.normalize_time#

True if the trace timestamps were normalized to start at 0.

property Trace.start[source]#

The timestamp of the first trace event.

property Trace.trace_state[source]#

State of the trace object that might impact the output of dataframe getter functions like lisa.trace.TraceBase.df_event().

It must be hashable and serializable to JSON, so that it can be recorded when analysis methods results are cached to the swap.

property Trace.available_events#

Inherited property, see lisa.trace.TraceBase.available_events

Set of available events on that trace.

property Trace.logger#

Inherited property, see lisa.utils.Loggable.logger

Convenience short-hand for self.get_logger().

property Trace.time_range#

Inherited property, see lisa.trace.TraceBase.time_range

Duration of that trace (difference between start and end).

property Trace.window#

Inherited property, see lisa.trace.TraceBase.window

Same as (trace.start, trace.end).

property Trace.analysis[source]#

Deprecated since version 3.0.

analysis() is deprecated and will be removed in version 4.0, use lisa.trace.Trace.ana instead

Methods#

Trace.__enter__()[source]#
Trace.__exit__(*args)[source]#
Trace.__getattr__(attr)#

Delegate attribute lookup to a private attribute.

Trace.df_event(event, *, df_fmt=None, **kwargs)[source]#

Get a dataframe containing all occurrences of the specified trace event in the parsed trace.

Parameters:

  • event (str) –

    Trace event name.

    In addition to actual events, the following formats for meta events are supported:

    • trace_printk@: The event format is described by the bprint event format string, and the field values are decoded from the variable arguments buffer. Note that:

      • The field values must be in the buffer, i.e. the format string is only used as the event format, no “literal value” will be extracted from it.

      • The event must have fields. If not, trace_printk() will emit a bputs event that will be ignored at the moment. We need to get a bprint event.

      • Field names must be unique.

      // trace.df_event('trace_printk@myevent')
void foo(void) {
    trace_printk("myevent: field1=%s field2=%i", "foo", 42);
}

    • userspace@: the event is generated by userspace:

      # trace.df_event('userspace@myevent')
echo "myevent: field1=foo field2=42" > /sys/kernel/debug/tracing/trace_marker

      Note that the field names must be unique.

    Note

    All meta event names are expected to be valid C language identifiers. Usage of other characters will prevent correct parsing.

  • signals (list(SignalDesc)) – List of signals to fixup if signals_init == True. If left to None, lisa.datautils.SignalDesc.from_event() will be used to infer a list of default signals.

  • compress_signals_init (bool) – Give a timestamp very close to the beginning of the sliced dataframe to rows that are added by signals_init. This allows keeping a very close time span without introducing duplicate indices.

classmethod Trace.from_target(target, events=None, buffer_size=10240, filepath=None, **kwargs)[source]#

Context manager that can be used to collect a lisa.trace.TraceBase directly from a lisa.target.Target without needing to setup an FtraceCollector.

Example:

from lisa.trace import Trace
from lisa.target import Target

target = Target.from_default_conf()

with Trace.from_target(target, events=['sched_switch', 'sched_wakeup']) as trace:
    target.execute('echo hello world')
    # DO NOT USE trace object inside the `with` statement

trace.ana.tasks.plot_tasks_total_residency(filepath='plot.png')
Parameters:

  • target (Target) – Target to connect to.

  • events (list(str)) – ftrace events to collect and parse in the trace.

  • buffer_size (int) – Size of the ftrace ring buffer.

  • filepath (str or None) – If set, the trace file will be saved at that location. Otherwise, a temporary file is created and removed as soon as the parsing is finished.

Variable keyword arguments:

Forwarded to Trace.

classmethod Trace.get_event_sources(*args, **kwargs)[source]#
Trace.get_metadata()#

Get metadata from the underlying trace parser.

See also

TraceParserBase.get_metadata()

Trace.get_view(window=None, *, df_fmt=None, **kwargs)[source]#

Get a view on a trace.

Various aspects of the trace can be altered depending on the parameters, such as cropping time-wise to fit in window.

Parameters:

  • window (tuple(float or None, float or None) or None) – Crop the dataframe to include events that are inside the given window. This includes the event immediately preceding the left boundary if there is no exact timestamp match. This can also include more rows before the beginning of the window based on the signals required by the user. A None boundary will extend to the beginning/end of the trace.

  • signals (list(lisa.datautils.SignalDesc) or None) – List of lisa.datautils.SignalDesc to use when selecting rows before the beginning of the window. This allows ensuring that all the given signals have a known value at the beginning of the window.

  • compress_signals_init (bool or None) – If True, the timestamp of the events before the beginning of the window will be compressed to be either right before the beginning of the window, or at the exact timestamp of the beginning of the window (depending on the dataframe library chosen, since pandas cannot cope with more than one row for each timestamp).

  • normalize_time (bool or None) – If True, the beginning of the window will become timestamp 0. If no window is used, the beginning of the trace is taken as T=0. This allows easier comparison of traces that were generated with absolute timestamps (e.g. timestamp related to the uptime of the system). It also allows comparing various slices of the same trace.

  • events_namespaces (list(str or None)) – List of namespaces of the requested events. Each namespace will be tried in order until the event is found. The None namespace can be used to specify no namespace. The full event name is formed with <namespace><event>.

  • events (list(str) or lisa.trace.TraceEventCheckerBase or None) – Preload the given events when creating the view. This can be advantageous as a single instance of the parser will be spawned, so if the parser supports it, multiple events will be parsed in one trace traversal.

  • strict_events – If True, will raise an exception if the events specified cannot be loaded from the trace. This allows failing early in trace processing.

  • strict_events – bool or None

  • process_df (Callable[[str, polars.LazyFrame], polars.LazyFrame] or None) –

    Function called on each dataframe returned by lisa.trace.TraceBase.df_event(). The parameters are as follow:

    1. Name of the event being queried.

    2. A polars.LazyFrame of the event.

    It is expected to return a polars.LazyFrame as well.

  • df_fmt (str or None) –

    Format of the dataframes returned by lisa.trace.TraceBase.df_events(). One of:

    • "pandas": pandas.DataFrame.

    • "polars-lazyframe": polars.LazyFrame.

    • None: defaults to "pandas" for backward-compatibility.

Variable arguments:

Forwarded to the contructor of the view.

Trace.has_events()#

Returns True if the specified event is present in the parsed trace, False otherwise.

Parameters:

events (str or list(str) or TraceEventCheckerBase) – trace event name or list of trace events

Trace.__getitem__(window)#

Inherited method, see lisa.trace.TraceBase.__getitem__()

Slice the trace with the given time range.

Trace.get_logger()#

Inherited method, see lisa.utils.Loggable.get_logger()

Provides a logging.Logger named after cls.

Trace.log_locals()#

Inherited method, see lisa.utils.Loggable.log_locals()

Debugging aid: log the local variables of the calling function.

Trace.show()#

Inherited method, see lisa.trace.TraceBase.show()

Open the parsed trace using the most appropriate native viewer.