Quickstart: SQL-based analysis and trace-based metrics
This quickstart explains how to use trace_processor as well as its Python API to programmatically query the trace contents through SQL and compute trace-based metrics.
Trace Processor
TraceProcessor is a multi-format trace importing and query engine based on SQLite. It comes both as a C++ library and as a standalone executable: trace_processor_shell (or just trace_processor).
Setup
# Download prebuilts (Linux and Mac only) curl -LO https://get.perfetto.dev/trace_processor chmod +x ./trace_processor # Start the interactive shell ./trace_processor trace.perfetto-trace # Start a local trace processor instance to replace wasm module in the UI ./trace_processor trace.perfetto-trace --httpd
NOTE: In HTTP mode the trace will be loaded into the trace_processor and the UI will connect and issue queries over TCP. This can allow arbitrary sized traces to be loaded since there are no memory constraints, unlike the WASM module. In addition, this can improve performance in the UI as it issues SQL queries.
See Trace Processor docs for the full TraceProcessor guide.
Sample queries
For more exhaustive examples see the SQL section of the various Data sources docs.
Slices
Slices are stackable events which have name and span some duration of time.
> SELECT ts, dur, name FROM slice
ts dur name
-------------------- -------------------- ---------------------------
261187017446933 358594 eglSwapBuffersWithDamageKHR
261187017518340 357 onMessageReceived
261187020825163 9948 queueBuffer
261187021345235 642 bufferLoad
261187121345235 153 query
...
Counters
Counters are events with a value which changes over time.
> SELECT ts, value FROM counter
ts value
-------------------- --------------------
261187012149954 1454.000000
261187012399172 4232.000000
261187012447402 14304.000000
261187012535839 15490.000000
261187012590890 17490.000000
261187012590890 16590.000000
...
Scheduler slices
Scheduler slices indicate which thread was scheduled on which CPU at which time.
> SELECT ts, dur, cpu, utid FROM sched
ts dur cpu utid
-------------------- -------------------- -------------------- --------------------
261187012170489 267188 0 390
261187012170995 247153 1 767
261187012418183 12812 2 2790
261187012421099 220000 6 683
261187012430995 72396 7 2791
...
Trace-based metrics
Trace Processor offers also a higher-level query interface that allows to run pre-baked queries, herein called “metrics”. Metrics are generally curated by domain experts, often the same people who add the instrumentation points in the first place, and output structured JSON/Protobuf/text. Metrics allow to get a summarized view of the trace without having to type any SQL or having to load the trace in the UI.
The metrics` schema files live in the /protos/perfetto/metrics directory. The corresponding SQL queries live in /src/trace_processor/metrics.
Run a single metric
Let's run the android_cpu metric. This metrics computes the total CPU time and the total cycles (CPU frequency * time spent running at that frequency) for each process in the trace, breaking it down by CPU (core) number.
./trace_processor --run-metrics android_cpu trace.perfetto-trace android_cpu { process_info { name: "/system/bin/init" threads { name: "init" core { id: 1 metrics { mcycles: 1 runtime_ns: 570365 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902017 } } core { id: 3 metrics { mcycles: 0 runtime_ns: 366406 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902908 } } ... } ... } process_info { name: "/system/bin/logd" threads { name: "logd.writer" core { id: 0 metrics { mcycles: 8 runtime_ns: 33842357 min_freq_khz: 595200 max_freq_khz: 1900800 avg_freq_khz: 1891825 } } core { id: 1 metrics { mcycles: 9 runtime_ns: 36019300 min_freq_khz: 1171200 max_freq_khz: 1900800 avg_freq_khz: 1887969 } } ... } ... } ... }
Running multiple metrics
Multiple metrics can be flagged using comma separators to the --run-metrics flag. This will output a text proto with the combined result of running both metrics.
$ ./trace_processor --run-metrics android_mem,android_cpu trace.perfetto-trace android_mem { process_metrics { process_name: ".dataservices" total_counters { anon_rss { min: 19451904 max: 19890176 avg: 19837548.157829277 } file_rss { min: 25804800 max: 25829376 avg: 25827909.957489081 } swap { min: 9289728 max: 9728000 avg: 9342355.8421707246 } anon_and_swap { min: 29179904 max: 29179904 avg: 29179904 } } ... } ... } android_cpu { process_info { name: "/system/bin/init" threads { name: "init" core { id: 1 metrics { mcycles: 1 runtime_ns: 570365 min_freq_khz: 1900800 max_freq_khz: 1900800 avg_freq_khz: 1902017 } } ... } ... } ... }
JSON and binary output
The trace processor also supports binary protobuf and JSON as alternative output formats. This is useful when the intended reader is an offline tool.
Both single and multiple metrics are supported as with proto text output.
./trace_processor --run-metrics android_mem --metrics-output=binary trace.perfetto-trace
<binary protobuf output>
./trace_processor --run-metrics android_mem,android_cpu --metrics-output=json trace.perfetto-trace
{
"android_mem": {
"process_metrics": [
{
"process_name": ".dataservices",
"total_counters": {
"anon_rss": {
"min": 19451904.000000,
"max": 19890176.000000,
"avg": 19837548.157829
},
"file_rss": {
"min": 25804800.000000,
"max": 25829376.000000,
"avg": 25827909.957489
},
"swap": {
"min": 9289728.000000,
"max": 9728000.000000,
"avg": 9342355.842171
},
"anon_and_swap": {
"min": 29179904.000000,
"max": 29179904.000000,
"avg": 29179904.000000
}
},
...
},
...
]
}
"android_cpu": {
"process_info": [
{
"name": "\/system\/bin\/init",
"threads": [
{
"name": "init",
"core": [
{
"id": 1,
"metrics": {
"mcycles": 1,
"runtime_ns": 570365,
"min_freq_khz": 1900800,
"max_freq_khz": 1900800,
"avg_freq_khz": 1902017
}
},
...
]
...
}
...
]
...
},
...
]
...
}
}
Python API
The API can be run without requiring the trace_processor binary to be downloaded or installed.
Setup
$ pip install perfetto
NOTE: The API is only compatible with Python3.
Example functions
See the Python API section of Trace Processor (SQL) to get more details on all available functions.
Query
from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.perfetto-trace') qr_it = tp.query('SELECT name FROM slice') for row in qr_it: print(row.name)
Output
eglSwapBuffersWithDamageKHR onMessageReceived queueBuffer bufferLoad query ...
Query as Pandas DataFrame
from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.perfetto-trace') qr_it = tp.query('SELECT ts, name FROM slice') qr_df = qr_it.as_pandas_dataframe() print(qr_df.to_string())
Output
ts name
-------------------- ---------------------------
261187017446933 eglSwapBuffersWithDamageKHR
261187017518340 onMessageReceived
261187020825163 queueBuffer
261187021345235 bufferLoad
261187121345235 query
...
Metric
from perfetto.trace_processor import TraceProcessor tp = TraceProcessor(trace='trace.perfetto-trace') cpu_metrics = tp.metric(['android_cpu']) print(cpu_metrics)
Output
metrics {
android_cpu {
process_info {
name: "/system/bin/init"
threads {
name: "init"
core {
id: 1
metrics {
mcycles: 1
runtime_ns: 570365
min_freq_khz: 1900800
max_freq_khz: 1900800
avg_freq_khz: 1902017
}
}
core {
id: 3
metrics {
mcycles: 0
runtime_ns: 366406
min_freq_khz: 1900800
max_freq_khz: 1900800
avg_freq_khz: 1902908
}
}
...
}
...
}
...
}
}
Next steps
There are several options for exploring more of the trace analysis features Perfetto provides:
- The trace conversion quickstart gives an overview on how to convert Perfetto traces to legacy formats to integrate with existing tooling.
- The Trace Processor documentation gives more information about how to work with trace processor including details on how to write queries and how tables in trace processor are organized.
- The metrics documentation gives a more in-depth look into metrics including a short walkthrough on how to build an experimental metric from scratch.
- The SQL table reference gives a comprehensive guide to the all the available tables in trace processor.
- The common tasks page gives a list of steps on how new metrics can be added to the trace processor.