Plotter Parser

The Mess plotting utilities are Python scripts located in the utils/ directory.

Key Features:

• Data Aberration Correction: Automatically cleans up noise and spikes in the raw measurements.
• Data Parsing: Converts the large set of raw .txt output files into concise CSV and JSON summaries.
• Visualization: Generates high-quality PDF and PNG plots of the bandwidth-latency curves.

---

Setup

Create a virtual environment and install dependencies:

cd utils/
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

---

plotter.txt

The plotter.txt file is a required configuration file that lives inside each measurement directory (e.g., measuring/multisequential/plotter.txt). It is auto-generated by the Mess benchmark during execution and provides the hardware parameters needed by the plotting and parsing utilities.

Purpose

The plotter uses raw hardware performance counter values (counts) in the measurement output files. plotter.txt supplies the parameters needed to:

1. Convert counts to bytes: The CACHE_LINE_SIZE tells the parser how many bytes each memory access count represents, enabling the conversion from raw counter values to bandwidth in GB/s.
2. Compute theoretical peak bandwidth: The memory subsystem parameters (MEM_FREQ, N_CHANNELS, BUS_WIDTH) are used to calculate the maximum theoretical bandwidth achievable by the hardware, which is drawn as a reference line on the charts.
3. Compute accurate latency: The CPU_FREQ and TLB_NS values are used to convert cycle counts into nanoseconds and subtract the TLB miss penalty from the raw latency measurements, producing corrected memory access latency values.

Fields

Field	Example	Description
TLB_NS	2.34041	Measured sTLB hit latency in nanoseconds, used to subtract TLB penalty from latency
CACHE_LINE_SIZE	64	Cache line size in bytes, used to convert counter counts into bytes
MEMORY_BINDING	local	NUMA binding mode (local or remote)
MEM_FREQ	4800 MT/s	Memory frequency in megatransfers per second
N_CHANNELS	8	Number of memory channels
BUS_WIDTH	64	Memory bus width in bits
CPU_FREQ	2.978992	Measured CPU frequency in GHz, used to convert cycles to nanoseconds

For remote NUMA measurements, the memory subsystem fields are replaced with interconnect parameters (e.g., UPI_FREQ, N_DATA_LANES, NVLINK_BW_GB_S).

Example

TLB_NS=2.34041
CACHE_LINE_SIZE=64
MEMORY_BINDING=local
MEM_FREQ=4800 MT/s
N_CHANNELS=8
BUS_WIDTH=64
CPU_FREQ=2.978992

Generation

This file is created and updated automatically by the benchmark:

• At benchmark start: Hardware detection writes TLB_NS, CACHE_LINE_SIZE, MEMORY_BINDING, and the memory subsystem parameters.
• After first stable measurement: The measured CPU_FREQ is appended once the first outlier-free data point is collected, ensuring an accurate frequency value under load.

Important

The plotting utilities will refuse to run if plotter.txt is missing from the measurement directory. If you need to re-plot data from a system where this file was lost, you must recreate it manually with the correct hardware parameters.

---

plotter.py

Generates memory bandwidth-latency curve visualizations and parsed data summaries (CSV/JSON/PDF/PNG) from Mess measurement results.

Usage

python3 plotter.py [OPTIONS] <directory> [<directory2> ...]

Features:

• Single Directory: Plots the bandwidth-latency curves for a single benchmark run.
• Multiple Directories: Automatically overlays bandwidth-latency curves from multiple directories for easy comparison (e.g., comparing different machines or configurations).

Options

Option	Description
--mode {perread,perkernel}	Group bandwidth-latency curves by measured read ratio (default) or by kernel
--step {1,2,4,5,10,50}	Plot bandwidth-latency curves in steps of N load ratios (default: 1)

Examples

# Single directory
python3 plotter.py /path/to/measuring/multisequential

# Multiple directories (overlay comparison)
python3 plotter.py /path/to/system1 /path/to/system2

# Plot every 5th read ratio (less cluttered)
python3 plotter.py --step 5 /path/to/measuring/multisequential

Output Files

Generated in processed/ subdirectory:

File	Description
memory_curves.pdf	Vector plot (publication quality)
memory_curves.png	Raster plot
memory_curves.csv	Raw data in CSV format
memory_curves.json	Compact JSON format

---

app_plotter.py

Overlays application profiler data (from Mess Profiler) on top of memory bandwidth-latency curves. This visualizes where your application sits on the memory performance envelope.

Usage

python3 app_plotter.py -c <curves_path> -p <profiler_output> [OPTIONS]

Options

| Option | Description | | ---------------- | -------------------------------------------------------------------------- | --- | | -c, --curves | Path to measurement directory (with bw/lat subdirs) or memory_curves.csv | | | -p, --profiler | Path to mess-profiler output CSV | | -o, --output | Output PDF path (default: app_on_curves.pdf in profiler dir) | | -n, --name | Application name for the plot title | | --cmap | Colormap for bandwidth-latency curves (default: Blues) |

Examples

# Using measurement directory (recommended)
python3 app_plotter.py -c /path/to/measuring/multisequential -p profile.csv -n "My App"

# Using pre-generated CSV
python3 app_plotter.py -c /path/to/memory_curves.csv -p profile.csv

# Custom output location
python3 app_plotter.py -c /path/to/curves -p profile.csv -o results/app_plot.pdf

How It Works

1. Loads bandwidth-latency curves from Mess measurement data
2. For each profiler sample, calculates read ratio from ReadBytes / (ReadBytes + WriteBytes)
3. Finds the bandwidth-latency curve with the closest read ratio
4. Interpolates latency from that curve at the sample's bandwidth
5. Plots application samples as black dots on top of the bandwidth-latency curves

Output Files

File	Description
app_on_curves.pdf	Plot with application overlay
app_on_curves.png	Raster version
app_on_curves.csv	Enriched data with calculated read ratios and estimated latencies

---

parse_runtimes.py

Reads Mess run logs and summarizes total runtime, useful for comparing benchmark configurations.

Usage

python3 parse_runtimes.py <log_file_or_directory>

---

Workflow Example

Complete workflow from benchmark to visualization:

# 1. Run benchmark (measurement files are saved by default)
./build/bin/mess

# 2. Generate bandwidth-latency curves plot
cd utils/
python3 plotter.py ../measuring/multisequential

# 3. Profile your application
cd ..
./build/bin/mess-profiler -s 100ms -o app_bw.csv ./my_application

# 4. Overlay application on curves
cd utils/
python3 app_plotter.py -c ../measuring/multisequential -p ../app_bw.csv -n "My Application"

---

See Also

• Mess Benchmark - Running the benchmark
• Mess Profiler - Profiling applications
• Understand output - Output file formats