rustc supports doing profile-guided optimization (PGO).
This chapter describes what PGO is and how the support for it is
The basic concept of PGO is to collect data about the typical execution of a program (e.g. which branches it is likely to take) and then use this data to inform optimizations such as inlining, machine-code layout, register allocation, etc.
There are different ways of collecting data about a program's execution.
One is to run the program inside a profiler (such as
perf) and another
is to create an instrumented binary, that is, a binary that has data
collection built into it, and run that.
The latter usually provides more accurate data.
rustc current PGO implementation relies entirely on LLVM.
LLVM actually supports multiple forms of PGO:
- Sampling-based PGO where an external profiling tool like
perfis used to collect data about a program's execution.
- GCOV-based profiling, where code coverage infrastructure is used to collect profiling information.
- Front-end based instrumentation, where the compiler front-end (e.g. Clang) inserts instrumentation intrinsics into the LLVM IR it generates.
- IR-level instrumentation, where LLVM inserts the instrumentation intrinsics itself during optimization passes.
rustc supports only the last approach, IR-level instrumentation, mainly
because it is almost exclusively implemented in LLVM and needs little
maintenance on the Rust side. Fortunately, it is also the most modern approach,
yielding the best results.
So, we are dealing with an instrumentation-based approach, i.e. profiling data is generated by a specially instrumented version of the program that's being optimized. Instrumentation-based PGO has two components: a compile-time component and run-time component, and one needs to understand the overall workflow to see how they interact.
Generating a PGO-optimized program involves the following four steps:
- Compile the program with instrumentation enabled (e.g.
rustc -Cprofile-generate main.rs)
- Run the instrumented program (e.g.
./main) which generates a
- Convert the
.profrawfile into a
.profdatafile using LLVM's
- Compile the program again, this time making use of the profiling data
rustc -Cprofile-use=merged.profdata main.rs)
Depending on which step in the above workflow we are in, two different things can happen at compile time:
As mentioned above, the profiling instrumentation is added by LLVM.
rustc instructs LLVM to do so by setting the appropriate
flags when creating LLVM
// `PMBR` is an `LLVMPassManagerBuilderRef` unwrap(PMBR)->EnablePGOInstrGen = true; // Instrumented binaries have a default output path for the `.profraw` file // hard-coded into them: unwrap(PMBR)->PGOInstrGen = PGOGenPath;
rustc also has to make sure that some of the symbols from LLVM's profiling
runtime are not removed by marking the with the right export level.
In the final step of the workflow described above, the program is compiled
again, with the compiler using the gathered profiling data in order to drive
rustc again leaves most of the work to LLVM here,
basically just telling the LLVM
where the profiling data can be found:
unwrap(PMBR)->PGOInstrUse = PGOUsePath;
LLVM does the rest (e.g. setting branch weights, marking functions with
Instrumentation-based approaches always also have a runtime component, i.e. once we have an instrumented program, that program needs to be run in order to generate profiling data, and collecting and persisting this profiling data needs some infrastructure in place.
In the case of LLVM, these runtime components are implemented in
compiler-rt and statically linked into any instrumented
rustc version of this can be found in
basically packs the C code from
compiler-rt into a Rust crate.
In order for
profiler_builtins to be built,
profiler = true must be set
Since the PGO workflow spans multiple compiler invocations most testing happens
in run-make tests (the relevant tests have
pgo in their name).
There is also a codegen test that checks that some expected
instrumentation artifacts show up in LLVM IR.
Clang's documentation contains a good overview on PGO in LLVM here: https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization