GTPin: Memorytrace Sample Tool

The Memorytrace tool generates a dynamic trace of memory addresses that are accessed by the kernel

The trace is provided for each kernel, for each Draw/Enqueue granularity, and for each individual HW thread.

Running the Memorytrace tool

The Memorytrace tool (as well as all GTPin tracing tools) works in two phases, which should be run separately:

• Pre-processing phase (phase 1). A phase in which the tool determines the number of required records to be saved.
• Trace gathering phase (phase 2). A phase in which the actual trace is collected.

To run the pre-processing phase of the Memorytrace tool in its default configuration, use this command:

Profilers/Bin/gtpin -t memorytrace --phase 1 -- app

NOTE: You may run this phase only once per application.

To run the trace gathering phase of the Memorytrace tool (in its default configuration), use this command:

Profilers/Bin/gtpin -t memorytrace --phase 2 -- app

How to understand Memorytrace results

When you run the in-house GTPin Memorytrace tool in its default configuration for pre-processing (phase 1), the tool generates a directory called: GTPIN_PROFILE_MEMORYTRACE0. In addition the tool creates the following two files in the current directory:

• memorytrace_pre_process.txt: Provides estimation of trace sizes that can be generated by kernels executed on the device. The memorytrace_pre_process.txt file contains the per-kernel buffer capacity required to hold any trace that can be generated by kernel dispatches. For example, if the kernel generates tarces between 100 and 150 bytes in different dispatches, the buffer space for that kernel should be large enough to hold 150 bytes.

This file is an input to the trace gathering phase. It has the following format:

BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 142606336

where, for each kernel, the required trace size is provided.

• memorytrace_pre_process_dispatch.txt: Provides information about trace sizes and threads profiled in kernel dispatches.

This file contains informational data only, and has the following format:

BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 142606336 262144  OpenCL 0  0
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  1
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 142606336 262144  OpenCL 0  2
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  3
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  4
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 142606336 262144  OpenCL 0  5
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  6
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  7
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  8
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 142606336 262144  OpenCL 0  9
BitonicSort___CS_asmf641279bbb4bc39f_simd32_7a6d7d0b064fb7e1 100034048 131072  OpenCL 0  10

where each line corresponds to a single kernel dispatch. The fields have the following meaning (from left to right):

• Kernel name
• The size of the trace required to profile this kernel dispatch
• Number of dispatched threads
• The platform the application supports: Intel® oneAPI Level Zero* (Level Zero*), OpenCL*, Microsoft DX11*, Microsoft DX12*, or Vulkan*.
• The execution descriptor (per platform meta data)
  • Level Zero - Device Enqueue.
  • OpenCL - Device Enqueue.
  • DX11 - Device Draw.
  • DX12 - Device CommandList Draw PSO.
  • Vulkan - Device CommandList Draw PSO.
    where:
    • Device - The ID of the logical device on which the kernel was run.
    • Enqueue - The ID of the Enqueue command.
    • Draw - The ID of the Draw command.
    • CommandList - The ID of the Command List.
    • PSO - The ID of the PSO (Pipeline State Object).

When the Memorytrace tool is run for the trace gathering phase (phase 2), the tool generates the directory: GTPIN_PROFILE_MEMORYTRACE1. GTPin saves the profiling results in the folder: GTPIN_PROFILE_MEMORYTRACE1\Session_Final. The traces for each kernel is saved in a separate sub-folder that has the same name as the kernel. Each Draw/Enqueue command has a separate trace, which is saved in a corresponding sub-directory, as shown in the following screenshot:

How to uncompress Memorytrace and read the trace

Each trace is saved in a compressed binary format within a file called memorytrace_compressed.bin, as shown above. To uncompress the trace, you must run a Profilers\Scripts\uncompress_memorytrace.py Python Software Foundation Python* script, in the following manner (Python 3.5 or above is required):

python3 Profilers\Scripts\uncompress_memtrace.py --input_dir GTPIN_PROFILE_MEMORYTRACE1\Session_Final\BitonicSort\device_0__enqueue_0 --gen 9 -v

NOTE: the -v parameter is optional. It provides additional metadata (such as access size, access type, address width, operand width, etc) for each memory access.

Running the script opens the compressed trace into separate traces, one per HW thread, as shown in the following screenshot:

where the trace generated on each HW thread is saved in a text file named memorytrace___s_0_ss_0_eu_1_tid_5.out. The file name indicates the HW thread topology ID (Slice (S), DualSubSlice (DSS), SubSlice (SS), Execution Unit (EU), and HW thread ID (TID)). The resulting trace is provided in the following format:

 BBL-ID   DP  SEND-OFFSET    RW      ADDRESS      SIZE
======================================================
    16    1    0x00000a58     R      0x00004800             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004810             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004820             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004830             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004840             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004850             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004860             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004870             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004880             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x00004890             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x000048a0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x000048b0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x000048c0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x000048d0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x000048e0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a58     R      0x000048f0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004900             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004910             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004920             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004930             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004940             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004950             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004960             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004970             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004980             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x00004990             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x000049a0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x000049b0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x000049c0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x000049d0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x000049e0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    16    1    0x00000a68     R      0x000049f0             16  // R  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x00004800             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x00004820             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x00004840             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x00004860             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x00004880             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x000048a0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x000048c0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000ca8     W      0x000048e0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x00004900             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x00004920             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x00004940             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x00004960             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x00004980             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x000049a0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x000049c0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    21    1    0x00000cb8     W      0x000049e0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x00004810             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x00004830             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x00004850             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x00004870             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x00004890             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x000048b0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x000048d0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000d98     W      0x000048f0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x00004910             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x00004930             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x00004950             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x00004970             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x00004990             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x000049b0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x000049d0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     
    23    1    0x00000da8     W      0x000049f0             16  // W  Scatter  BTS   32-bit access  SIMD16  BTI = 00  Operand = DW  Access size = 16     

----EOT----

Each line corresponds to a single memory access. The different fields in the output have the following meanings (from left to right):

• The basic-block ID in which the memory access instruction is contained.
• The number of the data port through which the access is done.
• The offset of the SEND instruction that is performing the access. Since each instruction in the GEN architecture is a vector instruction, the number of lines printed for each specific instruction is equal to the number of active channels (or actual accesses) that the instruction performs.
• The access type (Read or Write).
• The access address (an offset to the surface) to which the access is performed.
• The access size in bytes.
• Additional meta information that describes the access instruction (if the -v option is provided).

An EOT indication separates the consequent dispatches of different SW threads of the kernel from the same HW thread.

To map the basic block IDs and instruction offsets to the kernel code, you must look in the Session_Final\ISA sub-folder where GEN assembly of all the kernels are saved.

(Back to the list of all GTPin Sample Tools)

memorytrace.h

/*========================== begin_copyright_notice ============================
Copyright (C) 2018-2025 Intel Corporation

SPDX-License-Identifier: MIT
============================= end_copyright_notice ===========================*/

/*!
 * @file Memorytrace tool definitions
 */

#ifndef MEMORYTRACE_H_
#define MEMORYTRACE_H_

#include <list>
#include <map>
#include <vector>

#include "gtpin_api.h"
#include "gtpin_tool_utils.h"
#include "kernel_weight.h"
#include "gen_send_decoder.h"
#include "gt_basic_defs.h"

using namespace gtpin;

/*
 * The trace of memory accesses consists of records, each of which holds data pertaining to a single
 * execution of a basic block that accesses memory.
 * Each trace record comprises two data portions:
 *  - Header that details architectural state during this BBL execution. The MemTraceRecordHeader
 *    structure defines layout of the data elements in the header
 *  - Array of address payload register values in SEND instructions of this BBL. The values are
 *    stored in the order of the corresponding instructions in the basic block. This portion of the
 *    record is of a variable length - different BBLs may access different number of data elements.
 * Both portions (header and address payloads) are aligned with the GRF register size.
 */

/* ============================================================================================= */
// Struct MemTraceRecordHeader
/* ============================================================================================= */
/*!
 * Structure of the trace record header.
 * The header details architectural state during execution of a BBL that accesses memory. In the
 * trace record, the header is followed by the array of address payload registers.
 */
#pragma pack(push, 1)
struct MemTraceRecordHeader 
{
    uint16_t bblId;     ///< BBL identifier
    uint16_t sr0;       ///< LSB-16 of the State register sr0.0:ud
    uint32_t tileId;    ///< Tile ID
    uint32_t ce;        ///< Channel Enable register ce:ud
    uint32_t dm;        ///< Dispatch Mask register dm:ud
    uint32_t res;       ///< Reserved
    uint32_t tm00;      ///< tm0.0:ud
    uint32_t tm01;      ///< tm0.1:ud

    /// @return Size of this structure aligned with the GRF register size in the specified GEN model 
    static uint32_t AlignedSize(const IGtGenModel& genModel)
    {
        return (uint32_t)AlignUp(sizeof(MemTraceRecordHeader), genModel.GrfRegSize());
    }
};
#pragma pack(pop)

/* ============================================================================================= */
// Struct MemIns
/* ============================================================================================= */
/// SEND instruction descriptor
struct MemIns
{
    InsId       id;         ///< Instruction ID
    uint32_t    offset;     ///< Offset ot the instruction within the kernel
    DcSendMsg   msg;        ///< Decoded SEND message instruction
};
using MemInsList = std::list<MemIns>;

/* ============================================================================================= */
// Class BblMemAccessInfo
/* ============================================================================================= */
/*!
 * Class that retrieves information about memory accesses in the basic block, and provides an
 * interface for querying this information
 */
class BblMemAccessInfo
{
public:
    BblMemAccessInfo() : _recordSize(0) {}

    /// Populate this object with the information about memory accesses in the specified basic block
    BblMemAccessInfo(const IGtKernelInstrument& kernelInstrument, const IGtBbl& bbl) : BblMemAccessInfo() { Build(kernelInstrument, bbl); }
    BblMemAccessInfo& Build(const IGtKernelInstrument& kernelInstrument, const IGtBbl& bbl);

    /// @return List of memory instructions in the basic block
    const MemInsList& MemInstructions() const { return _memInstructions; }

    /// @return Size, in bytes, of the trace record in the basic block
    uint32_t RecordSize() const { return _recordSize; }

    /// @return true if basic block does not contain any memory instruction of interest
    bool IsEmpty() const { return (_recordSize == 0); }

private:
    MemInsList  _memInstructions;   ///< List of memory instructions in the basic block
    uint32_t    _recordSize;        ///< Size, in bytes, of the trace record in the basic block
};

/* ============================================================================================= */
// Class KernelMemAccessInfo
/* ============================================================================================= */
/*!
 * Class that retrieves information about memory accesses in the kernel, and provides an
 * interface for querying this information
 */
class KernelMemAccessInfo
{
public:
    KernelMemAccessInfo() : _maxRecordSize(0) {}

    /// Populate this object with the information about memory accesses in the specified kernel
    explicit KernelMemAccessInfo(const IGtKernelInstrument& kernelInstrument) { Build(kernelInstrument); }
    KernelMemAccessInfo& Build(const IGtKernelInstrument& kernelInstrument);

    /// @return Information about memory accesses in the kernel, indexed by BBL identifiers
    using MemAccessMap = std::map<BblId, BblMemAccessInfo>;
    const MemAccessMap& GetMemAccessMap() const { return _memAccessMap; }

    /// @return Information about memory accesses in the specified BBL, or NULL if requested information is not found
    const BblMemAccessInfo* GetBblInfo(BblId bblId) const;

    /// @return Number of basic blocks that access memory
    uint32_t NumMemBbls() const { return (uint32_t)_memAccessMap.size(); }

    /// @return Maximum size, in bytes, of the trace record in the kernel
    uint32_t MaxRecordSize() const { return _maxRecordSize; }

private:
    MemAccessMap _memAccessMap;  ///< Information about memory accesses, indexed by BBL identifiers
    uint32_t     _maxRecordSize; ///< Max. size, in bytes, of the trace record in the kernel
};

/* ============================================================================================= */
// Class MemTraceDispatch
/* ============================================================================================= */
/*!
 * Class that holds memory trace collected during a single kernel dispatch
 */
class MemTraceDispatch
{
public:
    /// Construct a MemTraceDispatch object with the empty trace
    explicit MemTraceDispatch(const IGtKernelDispatch& dispatch) : _isTrimmed(false) { dispatch.GetExecDescriptor(_kernelExecDesc); }
    
    /// Read the entire trace from the specified profile buffer into this object
    bool ReadTrace(const GtProfileTrace& traceAccessor, const IGtProfileBuffer& profileBuffer);

    const GtKernelExecDesc& KernelExecDesc() const { return _kernelExecDesc; } ///< @return Descriptor of this kernel dispatch
    uint32_t        Size()      const { return (uint32_t)_rawTrace.size(); }   ///< @return Trace size in bytes
    const uint8_t*  Data()      const { return _rawTrace.data(); }             ///< @return Trace data collected in this dispatch
    uint8_t*        Data()            { return _rawTrace.data(); }             ///< @return Trace data collected in this dispatch
    bool            IsEmpty()   const;                                         ///< @return true if the trace is empty
    bool            IsTrimmed() const { return _isTrimmed; }                   ///< @return true if the trace has been trimmed

private:
    GtKernelExecDesc        _kernelExecDesc; ///< Kernel execution descriptor
    std::vector<uint8_t>    _rawTrace;       ///< Trace data collected in this kernel dispatch
    bool                    _isTrimmed;      ///< true if the trace has been trimmed to avoid buffer overflow
};

/* ============================================================================================= */
// Class MemTraceKernel
/* ============================================================================================= */
/*!
 * Class that contains
 *  - Static information about memory accesses in the kernel
 *  - Collection of memory traces recorded by kernel dispatches
 */
class MemTraceKernel
{
public:
    MemTraceKernel() = default;

    /// Construct a MemTraceKernel object intended to hold traces of the specified kernel
    explicit MemTraceKernel(const IGtKernelInstrument& kernelInstrument, uint32_t numTiles);

    /*!
     * Read a trace recorded by the specified kernel dispatch. Create and add the corresponding MemTraceDispatch
     * instance to this object
     */
    MemTraceDispatch& AddMemTrace(IGtKernelDispatch& kernelDispatch);

    std::string           Name()            const { return _name; }               ///< @return Kernel's name
    std::string           ExtendedName()    const { return _extName; }            ///< @return Kernel's extended name
    std::string           UniqueName()      const { return _uniqueName; }         ///< @return Kernel's unique name
    const GtGpuPlatform   Platform()        const { return _platform; }           ///< @return Kernel's platform
    const IGtGenModel&    GenModel()        const { return GetGenModel(_genId); } ///< @return Kernel's GEN model
    const GtProfileTrace& TraceAccessor()   const { return _traceAccessor; }      ///< @return Trace accessor
    void  DumpAsm()                         const;                                ///< Dump kernel's assembly text to file

     /// @return true, if tracing of this kernel is enabled
    uint32_t IsEnabled() const { return (_traceAccessor.MaxTraceSize() != 0); }

    /// @return Static information about kernel's memory accesses
    const KernelMemAccessInfo& GetMemAccessInfo() const { return _memAccessInfo; }

    /// @return Traces collected in kernel's dispatches
    typedef std::list<MemTraceDispatch>  Traces;
    const Traces& GetTraces() const { return _traces; }

    /// @return Number of tiles
    uint32_t NumTiles() const { return _numTiles; }
private:
    std::string         _name;              ///< Kernel's name
    std::string         _uniqueName;        ///< Kernel's unique name
    std::string         _extName;           ///< Kernel's extended name
    GtGpuPlatform       _platform;          ///< Kernel's platform
    GtGenModelId        _genId;             ///< Identifier of the GEN model, the kernel is compiled for
    std::string         _asmText;           ///< Kernel's assembly text
    KernelMemAccessInfo _memAccessInfo;     ///< Static information about kernel's memory accesses
    GtProfileTrace      _traceAccessor;     ///< Trace accessor
    Traces              _traces;            ///< Traces collected in kernel's dispatches
    uint32_t            _numTiles;          ///< The number of supported tiles
};

/* ============================================================================================= */
// Class MemTrace
/* ============================================================================================= */
/*!
 * Implementation of the IGtTool interface for the Memorytrace tool
 */
class MemTrace : public GtTool
{
public:
    /// Implementation of the IGtTool interface
    const char* Name() const { return "memorytrace"; }

    void OnKernelBuild(IGtKernelInstrument& instrumentor);
    void OnKernelRun(IGtKernelDispatch& dispatcher);
    void OnKernelComplete(IGtKernelDispatch& dispatcher);

public:
    static void      OnFini();      ///< Callback function registered with atexit()
    static MemTrace* Instance();    ///< @return Single instance of this class

private:
    MemTrace() = default;
    MemTrace(const MemTrace&) = delete;
    MemTrace& operator = (const MemTrace&) = delete;
    ~MemTrace() = default;

    /*!
     * Generate instrumentation for the specified basic block
     * @param[in] instrumentor      Interface of the kernel being instrumented
     * @param[in] bbl               Basic block to be instrumented
     * @param[in] memTraceKernel    Object that holds information about memory accesses in the kernel
     * @return success/failure status
     */
    bool InstrumentBbl(IGtKernelInstrument& instrumentor, const IGtBbl& bbl, const MemTraceKernel& memTraceKernel);

    /*!
     * Generate procedure that allocates space for the new trace record in the trace and stores the record header
     * for the specified basic block. The procedure sets _offsetReg equal to the offset of the location within the
     * profile buffer immediately following the record header.
     * If new record can not be allocated due to the trace capacity limitations, the procedure zeroes _offsetReg.
     *
     * @param[in, out]  proc            Procedure, the generated code is appended to
     * @param[in]       coder           GEN code generator
     * @param[in]       bbl             Basic block being instrumented
     * @param[in]       memTraceKernel  Object that holds information about memory accesses in the kernel
     * @param[in]       recordSize      Size of the new record, in bytes
     * @param[in]       firstSendInBbl  Indicates whether this is the first Send instruction in BBL
     */
    void StoreRecordHeader(GtGenProcedure& proc, const IGtGenCoder& coder, const IGtBbl& bbl,
                           const MemTraceKernel& memTraceKernel, uint32_t recordSize, bool firstSendInBbl = true);

    /*!
     * Generate procedure that stores the specified range of GRF registers in the trace.
     * On entry, the procedure assumes that _offsetReg holds offset of the location within the profile buffer
     * at which address payload is about to be stored.
     * On exit, the procedure advances _offsetReg to the location just after the stored register range.
     * If input value of _offsetReg is zero. no registers are stored, and _offsetReg retains value zero.
     * 
     * @param[in, out]  proc            Procedure, the generated code is appended to
     * @param[in]       coder           GEN code generator
     * @param[in]       firstRegNum     First GRF register to be stored
     * @param[in]       numRegs         Number of registers to be stored
     */
    void StoreRegRange(GtGenProcedure& proc, const IGtGenCoder& coder, uint32_t firstRegNum, uint32_t numRegs);

private:
    std::map<GtKernelId, MemTraceKernel>    _kernels;  ///< Collection of traces per kernel

    GtReg _addrReg;     ///< Virtual register that holds address within profile buffer
    GtReg _dataReg;     ///< Virtual register that holds data to be read from/written to profile buffer 
    GtReg _offsetReg;   ///< Virtual register that holds offset within the trace
    GtReg _tileIdReg;   ///< Virtual register that holds tile ID
};

/* ============================================================================================= */
// Class MemoryTracePreProcessor
/* ============================================================================================= */
/*!
 * Class that computes per-kernel trace sizes in the preprocessing phase, and provides access to
 * this data in the trace gathering phase
 */
class MemoryTracePreProcessor : public KernelWeight
{
public:
    uint64_t TraceSize(const std::string& extKernelName) const; ///< Given extended kernel name, return the trace size in bytes
    static MemoryTracePreProcessor* Instance();                 ///< @return Single instance of this class
    static void OnFini();                                       ///< Callback function registered with atexit()

private:
    MemoryTracePreProcessor();
    MemoryTracePreProcessor(const MemoryTracePreProcessor&) = delete;
    MemoryTracePreProcessor& operator = (const MemoryTracePreProcessor&) = delete;

private:
    /// KernelWeight interface overrides (@see description of KernelWeight functions)
    uint32_t GetBblWeight(IGtKernelInstrument& kernelInstrument, const IGtBbl& bbl) const;
    void AggregateDispatchCounters(KernelWeightCounters& kc, KernelWeightCounters dc) const;

private:
    KernelWeightProfileData     _kernelCounters; ///< Per-kernel counters of required trace records; collected in preprocessing phase

    static const char* _kernelPreProcessFileName;   ///< Name of the file that contains preprocessing data per kernel
    static const char* _dispatchPreProcessFileName; ///< Name of the file that contains preprocessing data per kernel dispatch
};

/* ============================================================================================= */
// Class MemoryTracePostProcessor
/* ============================================================================================= */
/*!
 * Function object that processes kernel traces - stores them in files within the profile directory:
 *
 *    kernel_name
 *    |
 *        |- kernel_dispatch_1
 *           |- memorytrace_compressed.bin
 *        |- kernel_dispatch_2
 *           |- memorytrace_compressed.bin
 * The .bin trace files can be uncompressed by the uncompress_memtrace.exe utility.
 *
 * Format of .bin trace files:
 * - Static information:
 *     - Number of BBLs that access memory
 *     - For each BBL that accesses memory:
 *        - BBL ID
 *        - Number of SEND instructions in this basic block
 *        - For each SEND instruction:
 *            - Decoded information including offset, address model, address type, address payload size, etc
 * 
 * - Dynamic trace data:
 *      - Number of HW threads in which the trace was collected
 *      - For each HW thread:
 *          - HW Thread ID (in the format of sr0.0)
 *          - Number of records collected for this HW thread
 *          - All the records collected for this HW thread
 */
class MemoryTracePostProcessor
{
public:
    /// Construct a MemoryTracePostProcessor object for the specified collection of kernel traces
    MemoryTracePostProcessor(const IGtCore& gtpinCore, const MemTraceKernel&  memTraceKernel);

    /// Process all kernel traces associated with this object - store them in files within the profile directory
    bool operator()();

private:
    /// Store the specified trace in the specified file stream
    void StoreTrace(const MemTraceDispatch& trace, std::ofstream& fs);

    /// Store static information about memory accesses in the kernel
    void StoreMemAccessInfo(std::ofstream& fs);

    /// Store Global Thread Identifier
    void StoreGlobalTid(uint32_t gtid, std::ofstream& fs);

    /// Store the specified value in the specified file stream in the binary format
    template <typename T> void Store(const T& val, std::ofstream& fs) { fs.write((const char*)&val, sizeof(val)); }

private:
    struct TraceRecord                              ///< Reference to the trace record
    {
        const MemTraceRecordHeader* header;         ///< Pointer to the header of the record
        uint32_t                    size;           ///< Size of the record in bytes, including header
    };
    using TraceRecordList = std::list<TraceRecord>;           ///< List of references to trace records
    using PerTileTraceRecords = std::vector<TraceRecordList>; ///< Per tile trace records
    ///The SEND instruction description stored in the trace
    struct MemInsInfo
    {
        explicit MemInsInfo(const MemIns& memIns);    ///< Construct packed variant of the specified MemIns structure

        uint32_t    offset;             ///< Offset of the instruction within the kernel
        uint32_t    isWrite;            ///< Is write (true) or read (false) operation
        uint32_t    isBlock2D;          ///< Is block 2D load or store access
        uint32_t    isScatter;          ///< Is scatter
        uint32_t    isBTS;              ///< Is Binding Table State address model
        uint32_t    isSLM;              ///< Is Shared Local Memory access
        uint32_t    isScratch;          ///< Is access to scratch block
        uint32_t    isAtomic;           ///< Is atomic operation
        uint32_t    isFence;            ///< Is fence
        uint32_t    addressWidth;       ///< Address width (32bit or 64bit)
        uint32_t    simdWidth;          ///< SIMD width
        uint32_t    bti;                ///< BTI value
        uint32_t    addrPayloadLength;  ///< Address payload length
        uint32_t    dataPort;           ///< Data port ID: 0=DP0, 1=DP1, 2=UGM, 3=UGML, 4=TGM, 5=SLM, 6-all others
        uint32_t    isEOT;              ///< Is SEND.EOT
        uint32_t    isMedia;            ///< Is media block access
        uint32_t    elementSize;        ///< Data element size in bytes
        uint32_t    numElements;        ///< Number of elements
        uint32_t    execSize;           ///< Instruction execution size
        uint32_t    channelOffset;      ///< Channel mask offset
    };

private:
    const MemTraceKernel*            _kernel;            ///< Kernel&traces to be processed
    const KernelMemAccessInfo*       _memAccessInfo;     ///< Static information about memory accesses in the kernel
    std::string                      _kernelDir;         ///< Directory to store kernel's trace files
    std::vector<PerTileTraceRecords> _threadTraceRecords;///< Lists of trace records, indexed by the thread ID

    static const char*               _traceFileName;     ///< Name of the file to store trace in
};

#endif

memorytrace.cpp

/*========================== begin_copyright_notice ============================
Copyright (C) 2018-2025 Intel Corporation

SPDX-License-Identifier: MIT
============================= end_copyright_notice ===========================*/

/*!
 * @file Implementation of the Memorytrace tool
 */

#include <fstream>
#include <string>

#include "memorytrace.h"
#include "gtpin_tool_utils.h"

using namespace gtpin;
using namespace std;

/* ============================================================================================= */
// Configuration
/* ============================================================================================= */
Knob<int>  gKnobMaxTraceBufferInMB("max_buffer_mb", 3072, "memorytrace - the max allowed size of the trace buffer per kernel in MB\n");
Knob<int>  gKnobPhase("phase", 0, "tracing tool - processing phase\n { 1 - pre-processing, 2 - processing - trace gathering} ");
Knob<bool> gKnobTimeStamp("include_timestamp", false, "true - includes time stamp, false - doesn't include time stamp");

/* ============================================================================================= */
// BblMemAccessInfo implementation
/* ============================================================================================= */
BblMemAccessInfo& BblMemAccessInfo::Build(const IGtKernelInstrument& kernelInstrument, const IGtBbl& bbl)
{
    const IGtCfg&       cfg             = kernelInstrument.Cfg();
    uint32_t            addrPayloadSize = 0;
    uint32_t            headerSize      = 0;
    const IGtGenModel&  genModel        = kernelInstrument.Kernel().GenModel();

    _memInstructions.clear();
    for (auto insPtr : bbl.Instructions())
    {
        const IGtIns& ins = *insPtr;

        if ((ins.Id() < uint32_t(knobMinInstrumentIns)) || (ins.Id() > uint32_t(knobMaxInstrumentIns)))
        {
            continue;
        }
        if (ins.IsSendMessage())
        {
            DcSendMsg msg = DcSendMsg::Decode(ins.GetGedIns());
            if (msg.IsValid() || ins.IsEot())
            {
                addrPayloadSize += (msg.AddrPayloadLength() * genModel.GrfRegSize());   // Accumulate SEND address payloads
                headerSize = MemTraceRecordHeader::AlignedSize(genModel);               // Add one trace record header per BBL
                _memInstructions.emplace_back(MemIns{ins.Id(), cfg.GetInstructionOffset(ins), std::move(msg)});
            }
        }
    }
    _recordSize = addrPayloadSize + headerSize;
    if (!_memInstructions.empty() && gKnobTimeStamp)
    {
        _recordSize = addrPayloadSize + (uint32_t)_memInstructions.size() * headerSize;
    }
    return *this;
}

/* ============================================================================================= */
// KernelMemAccessInfo implementation
/* ============================================================================================= */
KernelMemAccessInfo& KernelMemAccessInfo::Build(const IGtKernelInstrument& kernelInstrument)
{
    const IGtCfg&  cfg = kernelInstrument.Cfg();

    _memAccessMap.clear();
    _maxRecordSize = 0;
    for (auto bblPtr : cfg.Bbls())
    {
        const IGtBbl& bbl = *bblPtr;
        BblMemAccessInfo bblMemAccessInfo(kernelInstrument, bbl);
        if (!bblMemAccessInfo.IsEmpty())
        {
            _maxRecordSize = std::max(_maxRecordSize, bblMemAccessInfo.RecordSize());
            _memAccessMap.emplace(bbl.Id(), std::move(bblMemAccessInfo));
        }
    }
    return *this;
}

const BblMemAccessInfo* KernelMemAccessInfo::GetBblInfo(BblId bblId) const 
{
    auto it = _memAccessMap.find(bblId);
    return (it == _memAccessMap.end()) ? nullptr: &(it->second);
}

/* ============================================================================================= */
// MemTraceDispatch implementation
/* ============================================================================================= */
bool MemTraceDispatch::ReadTrace(const GtProfileTrace& traceAccessor, const IGtProfileBuffer& profileBuffer)
{
    uint32_t traceSize = traceAccessor.Size(profileBuffer);
    _rawTrace.resize(traceSize);
    _isTrimmed = traceAccessor.IsTruncated(profileBuffer);
    return traceAccessor.Read(profileBuffer, _rawTrace.data(), 0, traceSize);
}

bool MemTraceDispatch::IsEmpty() const
{
    return _rawTrace.size() < sizeof(MemTraceRecordHeader);
}

/* ============================================================================================= */
// MemTraceKernel implementation
/* ============================================================================================= */
MemTraceKernel::MemTraceKernel(const IGtKernelInstrument& kernelInstrument, uint32_t numTiles) : _numTiles(numTiles)
{
    const IGtKernel& kernel = kernelInstrument.Kernel();
    const IGtCfg&    cfg    = kernelInstrument.Cfg();

    _name       = GlueString(kernel.Name());
    _extName    = ExtendedKernelName(kernel);
    _platform   = kernel.GpuPlatform();
    _genId      = kernel.GenModel().Id();
    _asmText    = CfgAsmText(cfg);
    _uniqueName = kernel.UniqueName();

    // Build static information about memory accesses in the kernel
    _memAccessInfo.Build(kernelInstrument);

    // Initialize trace accessor. The trace capacity is expected to be computed during the preprocessing phase.
    uint64_t traceCapacity =  MemoryTracePreProcessor::Instance()->TraceSize(_extName);
    if (traceCapacity == 0)
    {
        // Unknown trace capacity
        GTPIN_WARNING("MEMORYTRACE: unknown trace capacity for kernel " + _name + ". Assuming the kernel is filtered out. "
            "Allocating a buffer of 8KB size. If the kernel is supposed to run, expect buffer overflow. "
            "In this case, please re-run phase 1 and make sure the kernel is not filtered out.");
        traceCapacity = 0x2000;
    }
    else
    {
        traceCapacity += 0x2000; // Add some space to account for possible fluctuation of trace sizes between phases
        if (traceCapacity > UINT32_MAX)
        {
            GTPIN_WARNING("MEMORYTRACE: The kernel " + _name + " exceedeed maximum trace capacity.");
            traceCapacity = UINT32_MAX;
        }
    }
    if (traceCapacity > (uint64_t(gKnobMaxTraceBufferInMB) * 0x100000))
    {
        GTPIN_WARNING("MEMORYTRACE: required capacity (" + DecStr(traceCapacity) + ") for kernel " + _name + " is too big - cut to " + DecStr(gKnobMaxTraceBufferInMB) + "MB. "
                      "Expect the final trace to contain partial data.");
        traceCapacity = uint64_t(gKnobMaxTraceBufferInMB) * 0x100000;
    }
    uint32_t maxRecordSize = _memAccessInfo.MaxRecordSize();
    _traceAccessor = GtProfileTrace((uint32_t)traceCapacity, maxRecordSize);
    _traceAccessor.Allocate(kernelInstrument.ProfileBufferAllocator());
}

MemTraceDispatch& MemTraceKernel::AddMemTrace(IGtKernelDispatch& kernelDispatch)
{
    // Create a new MemTraceDispatch object and store the entire trace within this object
    _traces.emplace_back(kernelDispatch);
    MemTraceDispatch& memTraceDispatch  = _traces.back();
    if (!memTraceDispatch.ReadTrace(_traceAccessor, *kernelDispatch.GetProfileBuffer()))
    {
        GTPIN_ERROR_MSG("MEMORYTRACE: Failed to read profile buffer for kernel " + _name);
    }
    return memTraceDispatch;
}

void MemTraceKernel::DumpAsm() const
{
    DumpKernelAsmText(_name, _uniqueName, _asmText);
}

/* ============================================================================================= */
// MemTrace implementation
/* ============================================================================================= */
MemTrace* MemTrace::Instance()
{
    static MemTrace instance;
    return &instance;
}

void MemTrace::OnKernelBuild(IGtKernelInstrument& instrumentor)
{
    const IGtKernel& kernel = instrumentor.Kernel();
    uint32_t numTiles = (instrumentor.Coder().IsTileIdSupported()) ? GTPin_GetCore()->GenArch().MaxTiles(kernel.GpuPlatform()) : 1;


    // Handle common instrumentation knobs
    HandleCommonInstrumnetationKnobs(instrumentor);
    // Create new KernelData object and add it to the data base
    auto ret = _kernels.emplace(piecewise_construct, forward_as_tuple(kernel.Id()), forward_as_tuple(instrumentor, numTiles));
    if (ret.second)
    {
        MemTraceKernel& memTraceKernel = (*ret.first).second;
        if (!memTraceKernel.IsEnabled())
        {
            GTPIN_WARNING("MEMORYTRACE: The trace won't be generated for kernel " + memTraceKernel.Name());
            return;
        }

        const IGtCfg&   cfg   = instrumentor.Cfg();
        IGtVregFactory& vregs = instrumentor.Coder().VregFactory();

        // Initialize virtual registers
        _addrReg   = vregs.MakeMsgAddrScratch();
        _dataReg   = vregs.MakeMsgDataScratch(VREG_TYPE_HWORD);
        _offsetReg = vregs.Make(VREG_TYPE_DWORD);
        _tileIdReg = vregs.Make(VREG_TYPE_DWORD);

        GtGenProcedure loadTileIdProc;
        instrumentor.Coder().LoadTileId(loadTileIdProc, _tileIdReg);

        // Instrument kernel entries
        instrumentor.InstrumentEntries(loadTileIdProc);

        // Instrument basic blocks
        for (auto bblPtr : cfg.Bbls())
        {
            InstrumentBbl(instrumentor, *bblPtr, memTraceKernel);
        }
    }
}

void MemTrace::OnKernelRun(IGtKernelDispatch& dispatcher)
{
    bool isProfileEnabled = false;

    const IGtKernel& kernel = dispatcher.Kernel();
    GtKernelExecDesc execDesc; dispatcher.GetExecDescriptor(execDesc);
    if (kernel.IsInstrumented() && IsKernelExecProfileEnabled(execDesc, kernel.GpuPlatform(), kernel.Name().Get()))
    {
        auto it = _kernels.find(kernel.Id());
        if (it != _kernels.end())
        {
            const MemTraceKernel&  memTraceKernel = it->second;
            if (memTraceKernel.IsEnabled())
            {
                IGtProfileBuffer*     buffer        = dispatcher.CreateProfileBuffer(); GTPIN_ASSERT(buffer);
                const GtProfileTrace& traceAccessor = memTraceKernel.TraceAccessor();
                if (traceAccessor.Initialize(*buffer))
                {
                    isProfileEnabled = true;
                }
                else
                {
                    GTPIN_ERROR_MSG("MEMORYTRACE: Failed to write into memory buffer for kernel " + string(kernel.Name()));
                }
            }
        }
    }
    dispatcher.SetProfilingMode(isProfileEnabled);
}

void MemTrace::OnKernelComplete(IGtKernelDispatch& dispatcher)
{
    if (!dispatcher.IsProfilingEnabled())
    {
        return; // Do nothing with unprofiled kernel dispatches
    }

    const IGtKernel& kernel = dispatcher.Kernel();
    auto it = _kernels.find(kernel.Id());
    if (it != _kernels.end())
    {
        // Read the trace from the profile buffer
        MemTraceKernel&  memTraceKernel = it->second;
        memTraceKernel.AddMemTrace(dispatcher);
    }
}

bool MemTrace::InstrumentBbl(IGtKernelInstrument& instrumentor, const IGtBbl& bbl, const MemTraceKernel& memTraceKernel)
{
    const KernelMemAccessInfo&  kernelMemAccessInfo = memTraceKernel.GetMemAccessInfo();
    const BblMemAccessInfo*     bblInfoPtr          = kernelMemAccessInfo.GetBblInfo(bbl.Id());
    if ((bblInfoPtr == nullptr) || bblInfoPtr->IsEmpty())
    {
        return false; // The basic block does not contain any memory instruction of interest
    }

    const IGtGenCoder&      coder         = instrumentor.Coder();
    const IGtCfg&           cfg           = instrumentor.Cfg();
    const BblMemAccessInfo& bblInfo       = *bblInfoPtr;

    // Generate code that allocates space for the new record in the trace and stores the trace record header.
    // Insert this procedure before the first memory access in the basic block.
    GtGenProcedure headerProc;
    const IGtIns&  firstMemIns = cfg.GetInstruction(bblInfo.MemInstructions().front().id);
    StoreRecordHeader(headerProc, coder, bbl, memTraceKernel, bblInfo.RecordSize());
    instrumentor.InstrumentInstruction(firstMemIns, GtIpoint::Before(), headerProc);

    // Generate code that stores address payload registers of SEND instructions in the basic block
    for (const auto& insInfo : bblInfo.MemInstructions())
    {
        GtGenProcedure      proc;
        const IGtIns&       ins                 = cfg.GetInstruction(insInfo.id);
        const DcSendMsg&    msg                 = insInfo.msg;
        GtRegNum            src0RegNum          = msg.Src0();
        GtRegNum            src1RegNum          = msg.Src1();
        uint32_t            addrPayloadLength   = msg.AddrPayloadLength();
        uint32_t            src0Length          = msg.Src0Length();

        if (gKnobTimeStamp && (ins.Id() != firstMemIns.Id()))
        {
            GtGenProcedure headerProc;
            StoreRecordHeader(headerProc, coder, bbl, memTraceKernel, bblInfo.RecordSize(), false);
            instrumentor.InstrumentInstruction(ins, GtIpoint::Before(), headerProc);
        }

        // If size of the SRC0 register range is greater or equal to the address payload length, the SRC0 range contains
        // the entire address payload.
        // Otherwise the address payload is split between SRC0 and SRC1 register ranges
        if (src0Length >= addrPayloadLength)
        {
            StoreRegRange(proc, coder, src0RegNum, addrPayloadLength);
        }
        else
        {
            StoreRegRange(proc, coder, src0RegNum, src0Length);
            if (src1RegNum.IsValid())
            {
                StoreRegRange(proc, coder, src1RegNum, addrPayloadLength - src0Length);
            }
        }
        instrumentor.InstrumentInstruction(ins, GtIpoint::Before(), proc);
    }

    return true;
}

void MemTrace::StoreRecordHeader(GtGenProcedure& proc, const IGtGenCoder& coder, const IGtBbl& bbl,
                                 const MemTraceKernel& memTraceKernel, uint32_t recordSize, bool firstSendInBbl)
{
    /// @return Subregister of _dataReg intended to hold the value of the MemTraceRecordHeader field whose offset is specified
    auto fieldReg = [&](uint32_t fieldOffset) -> GtReg { return GtReg(_dataReg, sizeof(uint32_t), fieldOffset / sizeof(uint32_t)); };

    GtReg idFieldReg     = fieldReg(offsetof(MemTraceRecordHeader, bblId));
    GtReg tileIdFieldReg = fieldReg(offsetof(MemTraceRecordHeader, tileId));
    GtReg ceFieldReg     = fieldReg(offsetof(MemTraceRecordHeader, ce));
    GtReg dmFieldReg     = fieldReg(offsetof(MemTraceRecordHeader, dm));
    GtReg tm0FieldReg    = fieldReg(offsetof(MemTraceRecordHeader, tm00));

    IGtInsFactory&  insF = coder.InstructionFactory();
    GtPredicate     predicate(FlagReg(0));

    // Set values of MemTraceRecordHeader fields in _dataReg 
    if (firstSendInBbl)
    {
        proc += insF.MakeShl(idFieldReg, StateReg(0), 16);                      // idFieldReg[16:31] = sr0.0
        proc += insF.MakeAdd(idFieldReg, idFieldReg, GtImmU32(bbl.Id()));       // idFieldReg[0:15]  = bbl.Id()
        proc += insF.MakeMov(tileIdFieldReg, _tileIdReg);                       // Tile ID
        proc += insF.MakeMov(ceFieldReg, ChannelEnableReg());                   // ceFieldReg        = ChannelEnableReg()
        proc += insF.MakeMov(dmFieldReg, DispatchMaskReg());                    // dmFieldReg        = DispatchMaskReg()
    }
    if (gKnobTimeStamp)
    {
        proc += insF.MakeMov(tm0FieldReg, GtRegRegion(TimeStampReg(), GtStride(2, 2, 1), GED_DATA_TYPE_ud), { 2 });
    }

    // Allocate new record in the trace.
    if (firstSendInBbl)
    {
        // Set _offsetReg = offset of the allocated record in the profile buffer, _addrReg = address of the allocated record
        memTraceKernel.TraceAccessor().ComputeNewRecordOffset(coder, proc, recordSize, _offsetReg);
    }
    coder.ComputeAddress(proc, _addrReg, _offsetReg);

    // Zero _offsetReg if the trace buffer is overflowed (predicate == true)
    proc += insF.MakeMov(_offsetReg, 0).SetPredicate(predicate);

    //if (!predicate) { STORE buffer[_offsetReg] = _dataReg;  _offsetReg += aligned-header-size}
    uint32_t alignedHeaderSize = MemTraceRecordHeader::AlignedSize(memTraceKernel.GenModel());
    coder.StoreMemBlock(proc, _addrReg, _dataReg, alignedHeaderSize, !predicate);
    proc += insF.MakeAdd(_offsetReg, _offsetReg, alignedHeaderSize).SetPredicate(!predicate);

    if (!proc.empty()) { proc.front()->AppendAnnotation(__func__); }
}

void MemTrace::StoreRegRange(GtGenProcedure& proc, const IGtGenCoder& coder, uint32_t firstRegNum, uint32_t numRegs)
{
    if (numRegs == 0) { return; }

    IGtInsFactory&  insF        = coder.InstructionFactory();
    uint32_t        grfRegSize  = insF.GenModel().GrfRegSize();
    GtReg           flagReg     = FlagReg(0);
    GtPredicate     predicate(flagReg);
    uint32_t blockSize = numRegs * grfRegSize;

    // predicate = (_offsetReg != 0) = trace is not overflowed
    proc += insF.MakeCmp(GED_COND_MODIFIER_nz, flagReg, _offsetReg, 0, {16});
    
    // store address payload in addrReg = buffer[offsetReg]
    coder.ComputeAddress(proc, _addrReg, _offsetReg);
    coder.StoreRegRange(proc, _addrReg, firstRegNum, numRegs, predicate);
   
    proc += insF.MakeAdd(_offsetReg, _offsetReg, blockSize).SetPredicate(predicate);;

    if (!proc.empty()) { proc.front()->AppendAnnotation(__func__); }
}

void MemTrace::OnFini()
{
    MemTrace& me = *Instance();
    IGtCore* gtpinCore = GTPin_GetCore();
    for (auto& ref : me._kernels)
    {
        const MemTraceKernel&  memTraceKernel = ref.second;
        MemoryTracePostProcessor(*gtpinCore, memTraceKernel)();
        memTraceKernel.DumpAsm();
    }
}

/* ============================================================================================= */
// MemoryTracePreProcessor implementation
/* ============================================================================================= */
const char* MemoryTracePreProcessor::_kernelPreProcessFileName   = "memorytrace_pre_process.txt";
const char* MemoryTracePreProcessor::_dispatchPreProcessFileName = "memorytrace_pre_process_dispatch.txt";

MemoryTracePreProcessor::MemoryTracePreProcessor()
{
    if (gKnobPhase == 2)
    {
        // Read the data collected during the preprocessing phase
        std::ifstream is(_kernelPreProcessFileName);
        GTPIN_ASSERT_MSG(is, string("File ") + _kernelPreProcessFileName + " does not exist. The trace won't be generated");
        is >> _kernelCounters;
    }
    else if (gKnobPhase == 1)
    {
        // Create pre_process files or remove old pre_process files's content if they exist
        CreateCleanFile(_kernelPreProcessFileName);
        CreateCleanFile(_dispatchPreProcessFileName);
    }
}

MemoryTracePreProcessor* MemoryTracePreProcessor::Instance()
{
    static MemoryTracePreProcessor instance;
    return &instance;
}

void MemoryTracePreProcessor::OnFini()
{
    MemoryTracePreProcessor&  tool = *Instance();
    tool.DumpKernelProfiles(_kernelPreProcessFileName);
    tool.DumpDispatchProfiles(_dispatchPreProcessFileName);
}

uint64_t MemoryTracePreProcessor::TraceSize(const string& extKernelName) const
{
    auto it = _kernelCounters.find(extKernelName);
    return ((it == _kernelCounters.end()) ? 0 : it->second.weight);
}

uint32_t MemoryTracePreProcessor::GetBblWeight(IGtKernelInstrument& kernelInstrument, const IGtBbl& bbl) const
{
    // For the memorytrace tool, the weight of the BBL is the trace record size in this BBL
    return BblMemAccessInfo(kernelInstrument, bbl).RecordSize();
}

void MemoryTracePreProcessor::AggregateDispatchCounters(KernelWeightCounters& kc, KernelWeightCounters dc) const
{
    kc.weight = std::max(kc.weight, dc.weight);
    kc.freq += dc.freq;
}

/* ============================================================================================= */
// MemoryTracePostProcessor implementation
/* ============================================================================================= */
const char* MemoryTracePostProcessor::_traceFileName = "memorytrace_compressed.bin";

MemoryTracePostProcessor::MemoryTracePostProcessor(const IGtCore& gtpinCore, const MemTraceKernel& memTraceKernel) :
    _kernel(&memTraceKernel), _memAccessInfo(&memTraceKernel.GetMemAccessInfo()),
    _kernelDir(JoinPath(string(gtpinCore.ProfileDir()), memTraceKernel.UniqueName())) {}

bool MemoryTracePostProcessor::operator()()
{
    if (!MakeDirectory(_kernelDir))
    {
        GTPIN_WARNING("MEMORYTRACE: Could not create directory " + _kernelDir);
        return false;
    }

    // Process traces recorded in kernel dispatches
    for (const MemTraceDispatch& trace : _kernel->GetTraces())
    {
        if (!trace.IsEmpty())
        {
            if (trace.IsTrimmed())
            {
                GTPIN_WARNING("MEMORYTRACE: Detected trace buffer overflow in kernel " + _kernel->Name());
            }

            string subdir   = trace.KernelExecDesc().ToString(_kernel->Platform(), ExecDescFileNameFormat());
            string dir      = MakeSubDirectory(_kernelDir, subdir);
            string filePath = JoinPath(dir, _traceFileName);

            ofstream fs(filePath, std::ios::binary);
            if (!fs)
            {
                GTPIN_WARNING("MEMORYTRACE: Could not create file " + filePath);
                continue;
            }
            StoreTrace(trace, fs);
        }
    }
    return true;
}

void MemoryTracePostProcessor::StoreTrace(const MemTraceDispatch& trace, std::ofstream& fs)
{
    const uint8_t* traceData         = trace.Data();
    uint32_t       traceSize         = trace.Size();
    uint32_t       alignedHeaderSize = MemTraceRecordHeader::AlignedSize(_kernel->GenModel());

    // Associate trace records with threads - populate _threadTraceRecords array
    const GtStateRegAccessor& sra = _kernel->GenModel().StateRegAccessor();
    uint32_t maxThreads = _kernel->GenModel().MaxThreads(); // Max number of HW threads

    _threadTraceRecords.resize(_kernel->NumTiles());

    for (uint32_t tile = 0; tile < _kernel->NumTiles(); tile++)
    {
        _threadTraceRecords[tile].clear();
        _threadTraceRecords[tile].resize(maxThreads);
    }

    std::vector<uint32_t> numProfiledThreads; // Number of profiled (active) threads
    numProfiledThreads.resize(_kernel->NumTiles(), 0);

    for (uint32_t recordOffset = 0; recordOffset + sizeof(MemTraceRecordHeader) <= traceSize;)
    {
        // Retrive thread ID and BBL ID from the record header
        const MemTraceRecordHeader* header = (const MemTraceRecordHeader*)(traceData + recordOffset);
        uint32_t tid    = sra.GetGlobalTid(header->sr0);
        uint32_t bblId  = header->bblId;
        uint32_t tileId = header->tileId; GTPIN_ASSERT(tileId < _kernel->NumTiles());
        const BblMemAccessInfo* bblInfoPtr = _memAccessInfo->GetBblInfo(bblId); GTPIN_ASSERT(bblInfoPtr != nullptr);
        uint32_t recordSize = bblInfoPtr->RecordSize();
        if (recordOffset + recordSize > traceSize)
        {
            break; // end of trace
        }

        auto& tileThreadRecords = _threadTraceRecords[tileId];
        auto& threadTraceRecords = tileThreadRecords[tid];

        // Add a new trace record reference to _threadTraceRecords
        if (threadTraceRecords.empty()) { ++numProfiledThreads[tileId]; } // Increment thread count on the first relevant record
        threadTraceRecords.emplace_back(TraceRecord{ header, recordSize });

        recordOffset += recordSize;
    }

    uint32_t registerSize = _kernel->GenModel().GrfRegSize();

    StoreMemAccessInfo(fs);         // Store static information about memory accesses in the kernel
    Store(registerSize, fs);        // Store register size

    uint32_t timeStampIncluded = gKnobTimeStamp ? 1 : 0;

    Store(timeStampIncluded, fs);   // Store whether timestamp is included

    // Compute and store the number of involved tiles
    uint32_t numOfTiles = 0; 
    for (uint32_t i = 0; i < numProfiledThreads.size(); i++)
    {
        numOfTiles += (numProfiledThreads[i] == 0) ? 0 : 1;
    }
    Store(numOfTiles, fs);

    for (uint32_t tileId = 0; tileId < _threadTraceRecords.size(); tileId++)
    {
        if (numProfiledThreads[tileId] == 0) { continue; }

        Store(tileId, fs);

        Store(numProfiledThreads[tileId], fs);

        // Store per-thread traces
        for (uint32_t tid = 0; tid < maxThreads; tid++)
        {
            const auto& tileThreadRecords = _threadTraceRecords[tileId];
            const auto& traceRecordList = tileThreadRecords[tid];

            if (traceRecordList.empty()) { continue; }

            StoreGlobalTid(tid, fs);    // Store Global Thread Identifier

            uint32_t numRecords = (uint32_t)traceRecordList.size();
            Store(numRecords, fs);      // Store #records collected in the thread

            // Store trace records
            for (const auto& record : traceRecordList)
            {
                const auto& header   = *(record.header);
                uint32_t    bblId    = header.bblId;
                uint32_t    execMask = header.ce & header.dm;

                Store(bblId, fs);       // Store BBL ID
                Store(execMask, fs);    // Store dynamic execution mask

                if (timeStampIncluded)
                {
                    Store(header.tm00, fs);  // Store tm0.0
                    Store(header.tm01, fs);  // Store tm0.1
                }
                // Store address paylads
                if (record.size > alignedHeaderSize)
                {
                    fs.write((const char*)(record.header) + alignedHeaderSize, record.size - alignedHeaderSize);
                }
            }
        }
    }
}

void MemoryTracePostProcessor::StoreMemAccessInfo(std::ofstream& fs)
{
    // Store static information about memory accesses in BBLs
    uint32_t numBbls = _memAccessInfo->NumMemBbls();
    Store(numBbls, fs);                                 // Store the number of BBLs that access memory

    for (const auto& entry : _memAccessInfo->GetMemAccessMap())
    {
        uint32_t                bblId               = entry.first;
        const BblMemAccessInfo& bblInfo             = entry.second;
        uint32_t                numMemInstructions  = (uint32_t)bblInfo.MemInstructions().size();

        Store(bblId, fs);                               // Store BBL ID
        Store(numMemInstructions, fs);                  // Store the number of memory instructions in BBL
        for (const auto& memIns : bblInfo.MemInstructions())
        {
            MemInsInfo memInsInfo(memIns);
            Store(memInsInfo, fs);                    // Store the memory instruction descriptor
        }
    }
}

void MemoryTracePostProcessor::StoreGlobalTid(uint32_t gtid, std::ofstream& fs)
{
    const GtStateRegAccessor& sra = _kernel->GenModel().StateRegAccessor();
    uint32_t sr0 = sra.SetGlobalTid(0, gtid);

    auto storeSr0Field = [&](const ScatteredBitFieldU32& sbf)
    {
        uint32_t val = (sbf.IsEmpty() ? UINT32_MAX : sbf.GetValue(sr0));
        Store(val, fs);
    };

    storeSr0Field(sra.SliceIdField());
    storeSr0Field(sra.DualSubSliceIdField());
    storeSr0Field(sra.SubSliceIdField());
    storeSr0Field(sra.EuIdField());
    storeSr0Field(sra.ThreadSlotField());
}

MemoryTracePostProcessor::MemInsInfo::MemInsInfo(const MemIns& memIns)
{
    offset              = memIns.offset;
    isWrite             = memIns.msg.IsWrite() && (memIns.msg.Opcode() != GED_DP_OPCODE_LOAD_2D_BLOCK);
    isBlock2D           = (memIns.msg.Opcode() == GED_DP_OPCODE_LOAD_2D_BLOCK) || (memIns.msg.Opcode() == GED_DP_OPCODE_STORE_2D_BLOCK);
    isScatter           = memIns.msg.IsScatter();
    isBTS               = memIns.msg.IsBts();
    isSLM               = memIns.msg.IsSlm();
    isScratch           = memIns.msg.IsScratch();
    isAtomic            = memIns.msg.IsAtomic();
    isFence             = memIns.msg.IsMemFence();
    addressWidth        = (memIns.msg.IsA64() || isBlock2D) ? 64 : 32;
    simdWidth           = isBlock2D ? 1 : memIns.msg.SimdWidth();
    bti                 = memIns.msg.Bti();
    elementSize         = memIns.msg.ElementSize();
    numElements         = memIns.msg.NumElements();
    addrPayloadLength   = memIns.msg.AddrPayloadLength();
    dataPort            = (memIns.msg.IsDp0() ? 0 :
                          (memIns.msg.IsDp1() ? 1 :
                          (memIns.msg.IsUgm() ? 2 :
                          (memIns.msg.IsUgml()? 3 :
                          (memIns.msg.IsTgm() ? 4 :
                          (memIns.msg.IsSlm() ?  5 : 6))))));
    isEOT               = memIns.msg.IsEot();
    isMedia             = memIns.msg.IsMedia();
    execSize            = memIns.msg.ExecSize();
    channelOffset       = memIns.msg.ChannelOffset();
}

/* ============================================================================================= */
// GTPin_Entry
/* ============================================================================================= */
EXPORT_C_FUNC void GTPin_Entry(int argc, const char *argv[])
{
    ConfigureGTPin(argc, argv);
    if (gKnobPhase == 1)
    {
        MemoryTracePreProcessor::Instance()->Register();
        atexit(MemoryTracePreProcessor::OnFini);
    }
    else
    {
        GTPIN_ASSERT_MSG((gKnobPhase == 2), "MEMORYTRACE: Invalid phase value. Should be 1 or 2, provided " + std::to_string(gKnobPhase));
        MemTrace::Instance()->Register();
        atexit(MemTrace::OnFini);
    }
}

(Back to the list of all GTPin Sample Tools)