Model files

In short: MappedFile maps a whole file read-only and is shared by everything that views it. SafetensorsReader parses the 8-byte length and JSON header, and SafetensorsModel joins the shards of a Hugging Face checkpoint through its index. GgufReader parses llama.cpp's container (versions 2 and 3), reverses each tensor's dims into rml's row-major order, and hands out raw byte spans for any ggml type or Tensor views for the few rml has dtypes for. Parsing touches only headers; tensor bytes are paged in by the OS the first time a kernel reads them. Every header field is bounds checked, so a truncated or hostile file is an error, never a read past the mapping. Checkpoints write a model's parameters, buffers and optimizer state as one safetensors file and load it back strictly.

The concepts are on /ml-models: checkpointing and quantization. The module is five libraries, one per header, each with a test (BUILD.bazel). rml's architecture notes state the rule this module implements: weights are never copied into the heap, so resident memory is the pages the OS has faulted in (architecture.md L66–70).

Mapping a file

The header gives the reason in three lines: the OS pages mapped weights in on first touch and can drop clean pages under memory pressure, so a model larger than free RAM still loads, and loading costs nothing until a tensor is used (mapped_file.h L4–8).

Nothing in rml calls madvise at this commit. rml's memory notes plan layer-by-layer golden runs of an 8B model at bf16 that drop each layer's pages with madvise(MADV_DONTNEED); that is planned (models.md L41–43, see Models and memory).

How weights stay mapped

A tensor that views a mapping must keep it alive, and a reader must be able to go away first. rml does this with one shared owner. Tensor::FromBuffer(data, shape, dtype, owner) wraps external memory with a shared_ptr<const void> keep-alive handle (tensor.h L88–89, Tensors and dtypes), and both readers pass their MappedFile as that owner.

The mapping is read-only but Tensor's data pointer is not const, so both readers const_cast it and document that callers must not write through it (safetensors.cc L191–193, gguf.cc L577–579). A write would fault, which is why LoadCheckpoint copies values into the model's own tensors rather than keeping views (below).

safetensors

The format is an 8-byte little-endian header length, a JSON object mapping each tensor name to {dtype, shape, data_offsets} plus an optional __metadata__ object of string pairs, then the raw little-endian bytes (safetensors.h L4–11).

Sharded checkpoints

Large Hugging Face checkpoints split their tensors across files such as model-00001-of-00004.safetensors, with model.safetensors.index.json mapping each name to its shard under weight_map. SafetensorsModel maps every shard, so opening an 8B-parameter model costs only the headers (safetensors_model.h L4–10).

• What it opens. A .safetensors file, an index .json, or a directory, where it prefers the index and falls back to model.safetensors (safetensors_model.cc L48–60).
• The index. weight_map must be an object of strings, and each shard must be a plain file name: a path with a directory part is InvalidArgument, so an index cannot point outside its own directory (L24–44).
• Agreement. Each shard is opened once, the first time the index names it, and every name the index assigns to a shard must actually be in that shard's header (L62–76).
• Lookup. names() is sorted (a std::map), and Get forwards to the shard's reader (L79–90).

GGUF

GGUF is llama.cpp's single-file container: metadata (architecture, hyperparameters, the tokenizer's vocabulary) and every tensor in one file. The header comment is the format description rml works from (gguf.h L4–21):

• "GGUF", u32 version, u64 tensor count, u64 metadata count;
• metadata: a string key, a u32 value type and the value, where an array holds an element type, a u64 count and the elements, and a string is a u64 length and its bytes;
• the tensor table: a string name, u32 n_dims, u64 dims, a u32 ggml type and a u64 offset into the data section;
• padding to general.alignment (default 32), then the data.

GGUF lists dims innermost first, so a PyTorch weight of shape [out, in] is stored as {in, out}; the reader reverses them into rml's row-major Shape. The format is read in place as little-endian, which a static_assert on the host's byte order guarantees (gguf.cc L27–30).

A worked example: the hand-built file

gguf_test builds a file byte by byte from the spec, one string key and one 2 × 3 f32 tensor (gguf_test.cc L43–63). Its layout:

The reader

GgufReader::Open walks the mapping with a small Cursor that copies each field out with memcpy and refuses to run past the end (gguf.cc L101–128). The checks, in order (L413–499):

Metadata values are parsed into GgufValue: integers and bools in ints, floats in floats, strings in strings, with the element type kept for arrays (gguf.h L42–83). Tensors are only recorded as GgufTensorInfo (row-major shape, type, offset, byte size); their bytes are not read.

The writer

rml also writes GGUF: the llm tests and the decoder benchmark build their files with it. The raw form takes GgufTensorData (name, shape, ggml type, bytes) and an alignment (gguf.cc L326–396): always version 3; a power-of-two alignment, written as a leading general.alignment key only when it is not 32 (passing that key yourself is an error); each tensor checked for whole-block rows, rank 1 to 4 and the exact byte count; dims written innermost first; and every tensor's offset rounded up to the alignment, with the header and gaps zero-padded. The Tensor overload makes each tensor contiguous and maps its dtype to a ggml type (L398–411).

Training checkpoints

A checkpoint is one safetensors file holding a model's parameters and buffers, an optimizer's state and free-form metadata, so training can stop and resume exactly. Keys are model. plus the dotted parameter or buffer path and optim. plus the optimizer's state name, and any safetensors tool can read the file (checkpoint.h L4–11).

The state each optimizer saves is on Losses and optimizers, and the modules whose parameters are named here on Neural network layers. No example or other module calls the checkpoint functions at this commit; their test is the user.

Who uses it

One edge in WeightSource::Raw: when a safetensors tensor is misaligned, Get returns a heap copy, and Raw keeps only a span of that copy's bytes while the copy itself goes out of scope at the end of the function, so the span would outlive its storage. Aligned files, which is what rml's writer and the usual Hugging Face files produce, take the view path and are unaffected (weight_source.cc L143–148). How the decoder uses these views is on The decoder, the GGUF tokenizer on Tokenizers, and the block formats behind Bytes on Quantization.

Tests