Analysis plots

import os
from pathlib import Path

import holoviews as hv

from lisa.trace import Trace
from lisa.analysis.tasks import TaskID
from lisa.platforms.platinfo import PlatformInfo

# Enable bokeh as default backend, for interactive plots.

# THIS MUST BE DONE AFTER ALL IMPORTS.
# Otherwise there might be issues that lead to
# not displaying plots until hv.extension() is called again.

hv.extension('bokeh')

ⓘ

Load the trace

base_path = Path(os.getenv('LISA_HOME'))
doc_path = Path(
    base_path,
    'doc',
    'traces'
)

# Load a trace.dat ftrace binary file, see typical_experiment.ipynb on how to collect one using LISA
trace = Trace(
    doc_path / 'trace.dat',
    # Platform information contain all the knowledge that LISA means about the plaform,
    # such as CPU max frequencies, kernel symbols etc
    plat_info=PlatformInfo.from_yaml_map(doc_path / 'plat_info.yml')
)

DataFrame libraries

LISA supports two dataframe (table) libraries:

• polars: https://docs.pola.rs/api/python/stable/reference/index.html

• pandas: https://pandas.pydata.org/docs/index.html

Polars is a more modern alternative to pandas and most of the internal machinery of LISA has been moved to polars. At this point, pandas is available for backward compatibility and some internal code still has not been converted, but eventually there will not be any direct dependencies on pandas anymore. Since most dataframe-producing APIs are related to the Trace class, the switch between the two libraries can be achieved at that level:

import polars as pl

# This creates a view of the trace that will provide polars.LazyFrame dataframes.
# It is also possible to create the trace object for polars directly with
# Trace(..., df_fmt='polars-lazyframe'). The result is the same.
trace = trace.get_view(df_fmt='polars-lazyframe')

List of tasks in the trace, defined as a unique combination of a PID and a task name.

It is also possible to create name-only and pid-only TaskID in the analysis APIs by assigning None to the other field.

trace.ana.tasks.task_ids

[TaskID(pid=0, comm='swapper/0'),
 TaskID(pid=0, comm='swapper/4'),
 TaskID(pid=0, comm='swapper/5'),
 TaskID(pid=0, comm='swapper/1'),
 TaskID(pid=0, comm='swapper/3'),
 TaskID(pid=0, comm='swapper/2'),
 TaskID(pid=1, comm='init'),
 TaskID(pid=10, comm='kworker/0:1'),
 TaskID(pid=11, comm='ksoftirqd/0'),
 TaskID(pid=12, comm='rcu_preempt'),
 TaskID(pid=13, comm='migration/0'),
 TaskID(pid=26, comm='migration/3'),
 TaskID(pid=31, comm='migration/4'),
 TaskID(pid=36, comm='migration/5'),
 TaskID(pid=43, comm='kworker/1:1'),
 TaskID(pid=44, comm='kworker/2:1'),
 TaskID(pid=48, comm='kcompactd0'),
 TaskID(pid=57, comm='kworker/5:1'),
 TaskID(pid=60, comm='kworker/4:1'),
 TaskID(pid=85, comm='kworker/u12:1'),
 TaskID(pid=147, comm='kworker/3:2'),
 TaskID(pid=3807, comm='sshd'),
 TaskID(pid=5715, comm='trace-cmd'),
 TaskID(pid=5716, comm='sshd'),
 TaskID(pid=5716, comm='sh'),
 TaskID(pid=5717, comm='sshd'),
 TaskID(pid=5717, comm='sh'),
 TaskID(pid=5717, comm='shutils'),
 TaskID(pid=5718, comm='shutils'),
 TaskID(pid=5718, comm='busybox'),
 TaskID(pid=5719, comm='sshd'),
 TaskID(pid=5719, comm='sh'),
 TaskID(pid=5719, comm='shutils'),
 TaskID(pid=5720, comm='shutils'),
 TaskID(pid=5720, comm='busybox'),
 TaskID(pid=5720, comm='true'),
 TaskID(pid=5721, comm='shutils'),
 TaskID(pid=5721, comm='busybox'),
 TaskID(pid=5722, comm='shutils'),
 TaskID(pid=5722, comm='busybox'),
 TaskID(pid=5722, comm='true'),
 TaskID(pid=5723, comm='shutils'),
 TaskID(pid=5723, comm='busybox'),
 TaskID(pid=5724, comm='shutils'),
 TaskID(pid=5724, comm='busybox'),
 TaskID(pid=5724, comm='true'),
 TaskID(pid=5725, comm='shutils'),
 TaskID(pid=5725, comm='busybox'),
 TaskID(pid=5726, comm='shutils'),
 TaskID(pid=5726, comm='busybox'),
 TaskID(pid=5726, comm='true'),
 TaskID(pid=5727, comm='shutils'),
 TaskID(pid=5727, comm='busybox'),
 TaskID(pid=5728, comm='shutils'),
 TaskID(pid=5728, comm='busybox'),
 TaskID(pid=5728, comm='true'),
 TaskID(pid=5729, comm='shutils'),
 TaskID(pid=5729, comm='busybox'),
 TaskID(pid=5730, comm='shutils'),
 TaskID(pid=5730, comm='busybox'),
 TaskID(pid=5730, comm='true'),
 TaskID(pid=5731, comm='shutils'),
 TaskID(pid=5731, comm='busybox'),
 TaskID(pid=5732, comm='sshd'),
 TaskID(pid=5732, comm='sh'),
 TaskID(pid=5732, comm='rt-app'),
 TaskID(pid=5733, comm='rt-app'),
 TaskID(pid=5733, comm='big_0-0'),
 TaskID(pid=5734, comm='rt-app'),
 TaskID(pid=5734, comm='big_1-1'),
 TaskID(pid=5735, comm='rt-app'),
 TaskID(pid=5735, comm='small_0-2'),
 TaskID(pid=5736, comm='rt-app'),
 TaskID(pid=5736, comm='small_1-3'),
 TaskID(pid=5737, comm='rt-app'),
 TaskID(pid=5737, comm='small_2-4'),
 TaskID(pid=5738, comm='sshd'),
 TaskID(pid=5738, comm='sh'),
 TaskID(pid=5738, comm='shutils'),
 TaskID(pid=5739, comm='shutils'),
 TaskID(pid=5739, comm='busybox'),
 TaskID(pid=5740, comm='sshd'),
 TaskID(pid=5740, comm='sh'),
 TaskID(pid=5741, comm='sshd'),
 TaskID(pid=5741, comm='sh'),
 TaskID(pid=5741, comm='trace-cmd')]

task = TaskID(pid=None, comm='small_2-4')

# Default values for all all analysis methods parameters can be changed on the analysis proxy object.
# This can de-clutter the code, but the result could be changed after an update in case a plot method gains a parameter.
ana = trace.ana(
    #tasks=[task],
    #task=task,
)

Holoviews

All plot methods are implemented using the holoviews library: https://holoviews.org/getting_started/index.html

For more advanced plot customizations https://holoviews.org/user_guide/Applying_Customizations.html

Holoviews is an abstraction layer on top of matplotlib and bokeh (and plotly, but this backend is not supported in LISA). This means that all the styling options are backend specific.

# Call a plot method that will return a holoviews object.
util_fig = ana.load_tracking.plot_task_signals(task, signals=['util'])
util_fig

Set a style option

… on the Curve elements of the plot. Style options are backend-specific.

The recommended backend is “bokeh”, as it will provide the best interactivity.

util_fig.options('Curve', color='red')

Save plot to a file

path = base_path / 'util.html'
ana.load_tracking.plot_task_signals(task, signals=['util'], filepath=path)
print(path)

/home/ubuntu/builds/t3_Sy8ex/0/tooling/lisa/tmp/tmp.LBBYiKkXXK-worktree/util.html

Kernelshark-like plot.

Using output='ui' allows displaying dataframes under the plot, with scrolling synchronised with the cursor.

Caveat: The object created attempts to emulate the holoviews plot API, but if you get some unexpected exceptions when combining plots, revert to output=None.

Note: Dataframes can be costly to create, but most of them are then cached on disk for later use, so repeated creation of the same plot should be quick.

activation_fig = ana.tasks.plot_tasks_activation(output='ui')
activation_fig

Create a Layout

• If you don’t want the X axis range to be synchronise, use .options(shared_axes=False).

• .cols(1) allows getting the plots on top of eachother, rather than side by side.

layout = activation_fig + util_fig
layout.cols(1)

Create an Overlay

It only works well if the axes have the same units, otherwise there will be scaling issues

capa_fig = ana.load_tracking.plot_task_required_capacity(task=task)
capa_fig * util_fig

Display custom DataFrame

Any number of dataframes can be linked to the plot. They will each get their marker and tab.

util_df = ana.load_tracking.df_task_signal(task=task, signal='util')
events_df = ana.notebook.df_all_events()

ana.load_tracking.plot_task_signals(
    task=task,
    signals=['util'],
    link_dataframes={'util': util_df, 'all events': events_df},
    output='ui',
)