When the compiler sees a reference to an external crate, it needs to load some information about that crate. This chapter gives an overview of that process, and the supported file formats for crate libraries.
A crate dependency can be loaded from an
rmeta file. A
key point of these file formats is that they contain
metadata. This metadata allows the compiler to discover enough
information about the external crate to understand the items it contains,
which macros it exports, and much more.
rlib is an archive file, which is similar to a tar file. This file
format is specific to
rustc, and may change over time. This file contains:
- Object code, which is the result of code generation. This is used during
regular linking. There is a separate
.ofile for each codegen unit. The codegen step can be skipped with the
-C linker-plugin-ltoCLI option, which means each
.ofile will only contain LLVM bitcode.
- LLVM bitcode, which is a binary representation of LLVM's intermediate
representation, which is embedded as a section in the
.ofiles. This can be used for Link Time Optimization (LTO). This can be removed with the
-C embed-bitcode=noCLI option to improve compile times and reduce disk space if LTO is not needed.
rustcmetadata, in a file named
- A symbol table, which is generally a list of symbols with offsets to the object file that contain that symbol. This is pretty standard for archive files.
dylib is a platform-specific shared library. It includes the
metadata in a special link section called
.rustc in a compressed format.
rmeta file is custom binary format that contains the metadata for the
crate. This file can be used for fast "checks" of a project by skipping all
code generation (as is done with
cargo check), collecting enough information
for documentation (as is done with
cargo doc), or for
pipelining. This file is created if the
--emit=metadata CLI option is used.
rmeta files do not support linking, since they do not contain compiled
The metadata contains a wide swath of different elements. This guide will not
go into detail of every field it contains. You are encouraged to browse the
CrateRoot definition to get a sense of the different elements it contains.
Everything about metadata encoding and decoding is in the
Here are a few highlights of things it contains:
- The version of the
rustccompiler. The compiler will refuse to load files from any other version.
- The Strict Version Hash (SVH). This helps ensure the correct dependency is loaded.
- The Crate Disambiguator. This is a hash used to disambiguate between different crates of the same name.
- Information about all the source files in the library. This can be used for a variety of things, such as diagnostics pointing to sources in a dependency.
- Information about exported macros, traits, types, and items. Generally, anything that's needed to be known when a path references something inside a crate dependency.
- Encoded MIR. This is optional, and only encoded if needed for code
cargo checkskips this for performance reasons.
The Strict Version Hash (SVH, also known as the "crate hash") is a 64-bit hash that is used to ensure that the correct crate dependencies are loaded. It is possible for a directory to contain multiple copies of the same dependency built with different settings, or built from different sources. The crate loader will skip any crates that have the wrong SVH.
The SVH is also used for the incremental compilation session filename, though that usage is mostly historic.
The hash includes a variety of elements:
- Hashes of the HIR nodes.
- All of the upstream crate hashes.
- All of the source filenames.
- Hashes of certain command-line flags (like
-C metadatavia the Crate Disambiguator, and all CLI options marked with
finalize_and_compute_crate_hash for where the hash is actually
CrateDisambiguator is a 128-bit hash used to distinguish between
different crates of the same name. It is a hash of all the
-C metadata CLI
options computed in
compute_crate_disambiguator. It is used in a variety
of places, such as symbol name mangling, crate loading, and much more.
By default, all Rust symbols are mangled and incorporate the disambiguator
hash. This allows multiple versions of the same crate to be included together.
Cargo automatically generates
-C metadata hashes based on a variety of
factors, like the package version, source, and the target kind (a lib and bin
can have the same crate name, so they need to be disambiguated).
Crate loading can have quite a few subtle complexities. During name
resolution, when an external crate is referenced (via an
extern crate or
path), the resolver uses the
CrateLoader which is responsible for finding
the crate libraries and loading the metadata for them. After the dependency
is loaded, the
CrateLoader will provide the information the resolver needs
to perform its job (such as expanding macros, resolving paths, etc.).
To load each external crate, the
CrateLoader uses a
actually find the correct files for one specific crate. There is some great
documentation in the
locator module that goes into detail on how loading
works, and I strongly suggest reading it to get the full picture.
The location of a dependency can come from several different places. Direct
dependencies are usually passed with
--extern flags, and the loader can look
at those directly. Direct dependencies often have references to their own
dependencies, which need to be loaded, too. These are usually found by
scanning the directories passed with the
-L flag for any file whose metadata
contains a matching crate name and SVH. The loader
will also look at the sysroot to find dependencies.
As crates are loaded, they are kept in the
CStore with the crate metadata
wrapped in the
CrateMetadata struct. After resolution and expansion, the
CStore will make its way into the
GlobalCtxt for the rest of
One trick to improve compile times is to start building a crate as soon as the
metadata for its dependencies is available. For a library, there is no need to
wait for the code generation of dependencies to finish. Cargo implements this
technique by telling
rustc to emit an
rmeta file for each
dependency as well as an
rlib. As early as it can,
rmeta file to disk before it continues to the code generation
phase. The compiler sends a JSON message to let the build tool know that it
can start building the next crate if possible.
The crate loading system is smart enough to know when it
rmeta file to use that if the
rlib is not there (or has only been
This pipelining isn't possible for binaries, because the linking phase will require the code generation of all its dependencies. In the future, it may be possible to further improve this scenario by splitting linking into a separate command (see #64191).