The Intel® Trace Analyzer and Collector employs the use of the Structured Trace File Format (STF) in order to profile MPI applications in a scalable and efficient manner.
The development of STF was motivated by the observation that the conventional approach of handling trace data in a single trace file is not suitable for large applications or systems, where the trace file can quickly grow into the tens of gigabytes range. It is impractical to squeeze such huge amounts of data into a single display. Mechanisms must be provided to start at a coarser level and then resolve the display into more detailed information. Additionally, the ability to request and inspect only parts of the data becomes essential with the amount of trace data growing.
These requirements necessitated a more powerful data organization than the previous Intel® Trace Collector tracefile format can provide. In response to this, the STF has been developed. The aim of the STF is to provide a file format which:
- can arbitrarily be partitioned into several files, each one containing a specific subset of the data
- allows fast random access and easy extraction of data
- is extensible, portable, and upward compatible
- is clearly defined and structured
- can efficiently exploit parallelism for reading and writing
- is as compact as possible
The specification outlying the technical requirements for the STF is now available for general download. This document describes how the scalability requirements are being met while keeping the file format transparent to the Intel Trace Analyzer and Collector user.