Every schema

Conventions

• All files are proto3 and compiled with prost into jj_lib::protos::<package>, except default_index and git_store, which the jj-default-index and jj-git-backend crates compile into their own private protos modules (source). In proto3, an unset scalar reads as its zero value, which is why so much of the conversion code checks for an empty bytes or string to detect old formats.
• IDs are raw bytes, not hex. Commit, tree, file, view and operation IDs are all stored this way.
• Content-addressed objects are named by a hash of the Rust value, not of the proto bytes. The simple backend and the simple op store both compute the ID as blake2b_hash(&value) over the Rust struct's ContentHash, then write the encoded proto to a file named after that ID (commit, view, operation).
• Deprecated fields are kept so old repos still load. New code writes only the current fields; readers fall back to the deprecated ones when the current ones are empty.

simple_store.proto

The native object format of SimpleBackend, a backend that stores everything as individual proto files. Git-backed repos never use these messages: commits and trees are Git objects there. Full file.

syntax = "proto3";

package simple_store;

message TreeValue {
  message File {
    bytes id = 1;
    bool executable = 2;
    bytes copy_id = 3;
  }

  oneof value {
    File file = 2;
    bytes symlink_id = 3;
    bytes tree_id = 4;
  }
}

message Tree {
  message Entry {
    string name = 1;
    TreeValue value = 2;
  }

  repeated Entry entries = 1;
}

message Commit {
  repeated bytes parents = 1;
  repeated bytes predecessors = 2;
  // Alternating positive and negative terms
  repeated bytes root_tree = 3;
  // Labels for the terms of a conflict.
  repeated string conflict_labels = 10;
  bytes change_id = 4;
  string description = 5;

  message Timestamp {
    int64 millis_since_epoch = 1;
    int32 tz_offset = 2;
  }
  message Signature {
    string name = 1;
    string email = 2;
    Timestamp timestamp = 3;
  }
  Signature author = 6;
  Signature committer = 7;
  optional bytes secure_sig = 9;
}

TreeValue and TreeValue.File

There's no variant for Git submodules: writing TreeValue::GitSubmodule to the simple backend panics with cannot store git submodules (tree_value_to_proto, tree_value_from_proto).

Tree and Tree.Entry

tree_to_proto / tree_from_proto

Commit, Commit.Signature, Commit.Timestamp

Signing works on the proto itself: write_commit encodes the commit, passes those bytes to the signing function, and stores the result in secure_sig. When reading, commit_from_proto takes secure_sig out first and re-encodes the remainder, which reproduces exactly the bytes that were signed (write, convert, signatures).

git_store.proto

The Git backend stores commits as real Git commits. That leaves out what Git can't represent, and this one message carries it. The serialized bytes go into a stacked table under .jj/repo/store/extra, keyed by the Git commit ID. Entries are added both when jj writes a commit and when it imports one from Git (write, import). Full file.

syntax = "proto3";

package git_store;

message Commit {
  repeated bytes predecessors = 2;
  bytes change_id = 4;

  // Alternating positive and negative terms. Set only for conflicts.
  // Resolved trees are stored in the git commit
  repeated bytes root_tree = 1;
  // TODO(#1624): delete when we assume that all commits use this format
  bool uses_tree_conflict_format = 10;

  bool is_open = 8 [deprecated = true];
  bool is_pruned = 9 [deprecated = true];
}

Everything else about a commit's jj view lives in the Git commit itself. A conflicted root tree is written as a jj:trees header of space-separated hex tree IDs, which must hold an odd number of at least three, so jj and Git can never disagree about a resolved commit's contents. Conflict labels go in a jj:conflict-labels header, and the change ID can go in a change-id header written as reverse hex. When a Git commit has no change ID, jj derives a synthetic one from the commit ID (jj:trees, headers on write, serialize_extras / deserialize_extras).

simple_op_store.proto

The operation log. Every operation and every view is one file under .jj/repo/op_store/operations/ or views/, named by the hex of its 64-byte ID. The root operation and root view are never stored: both are all-zero IDs, and the store synthesizes them on read (read/write). Full file.

syntax = "proto3";

package simple_op_store;

message RefConflictLegacy {
  repeated bytes removes = 1 [deprecated = true];
  repeated bytes adds = 2 [deprecated = true];
}

// This is redundant with `RefTargetTerm`. It exists for historical reasons.
// TODO: Consider deprecating it.
message RefConflict {
  message Term {
    optional bytes value = 1;
  }
  repeated Term removes = 1;
  repeated Term adds = 2;
}

// RefTarget that may be serialized in legacy form.
message RefTarget {
  // New `RefConflict` type represents both `commit_id` and `conflict_legacy`.
  oneof value {
    bytes commit_id = 1 [deprecated = true];
    RefConflictLegacy conflict_legacy = 2 [deprecated = true];
    RefConflict conflict = 3;
  }
}

// RefTarget term that should be serialized in native form, in which positive
// and negative terms alternate.
message RefTargetTerm {
  optional bytes value = 1;
}

enum RemoteRefState {
  New = 0;
  Tracked = 1;
}

message RemoteBookmark {
  string remote_name = 1;
  RefTarget target = 2;
  // Introduced in jj 0.11.
  optional RemoteRefState state = 3;
}

message Bookmark {
  string name = 1;
  // Unset if the bookmark has been deleted locally.
  RefTarget local_target = 2;
  // Deprecated since jj 0.34.
  repeated RemoteBookmark remote_bookmarks = 3 [deprecated = true];
}

message GitRef {
  string name = 1;
  // This field is just for historical reasons (before we had the RefTarget
  // type). New GitRefs have (only) the target field.
  // TODO: Delete support for the old format.
  bytes commit_id = 2 [deprecated = true];
  RefTarget target = 3;
}

message GitHead {
  string name = 1;
  RefTarget target = 2;
}

message RemoteRef {
  string name = 1;
  repeated RefTargetTerm target_terms = 2;
  RemoteRefState state = 3;
}

message Tag {
  string name = 1;
  RefTarget target = 2;
}

message View {
  repeated bytes head_ids = 1;
  reserved 4;
  bytes wc_commit_id = 2 [deprecated = true];
  map<string, bytes> wc_commit_ids = 8;
  // Local bookmarks and remote bookmarks in legacy form.
  repeated Bookmark bookmarks = 5;
  repeated Tag local_tags = 6;
  // Introduced in jj 0.34.
  repeated RemoteView remote_views = 11;
  // Only a subset of the refs. For example, does not include refs/notes/.
  repeated GitRef git_refs = 3;
  // This field is just for historical reasons (before we had the RefTarget
  // type). New Views have (only) the target field.
  // TODO: Delete support for the old format.
  bytes git_head_legacy = 7 [deprecated = true];
  RefTarget git_head = 9 [deprecated = true];
  // Whether "@git" tags have been migrated to remote_views.
  bool has_git_refs_migrated_to_remote_tags = 12;
  // Per-workspace Git HEAD targets, keyed by workspace name.
  repeated GitHead git_heads = 13;
  reserved 10;
}

message RemoteView {
  string name = 1;
  repeated RemoteRef bookmarks = 2;
  repeated RemoteRef tags = 3;
}

message Operation {
  bytes view_id = 1;
  repeated bytes parents = 2;
  OperationMetadata metadata = 3;
  // Introduced in jj 0.30.
  repeated CommitPredecessors commit_predecessors = 4;
  // Whether or not `commit_predecessors` is recorded.
  bool stores_commit_predecessors = 5;
}

// TODO: Share with store.proto? Do we even need the timezone here?
message Timestamp {
  int64 millis_since_epoch = 1;
  int32 tz_offset = 2;
}

message OperationMetadata {
  Timestamp start_time = 1;
  Timestamp end_time = 2;
  string description = 3;
  string hostname = 4;
  string username = 5;
  bool is_snapshot = 7;
  optional string workspace_name = 8;
  map<string, string> attributes = 6;
}

message CommitPredecessors {
  bytes commit_id = 1;
  repeated bytes predecessor_ids = 2;
}

Operation

predecessor map and operation conversions

OperationMetadata, Timestamp, CommitPredecessors

metadata conversions

View

view_to_proto · view_from_proto

Writing is deterministic: view_to_proto sorts head_ids (source), and the generated wc_commit_ids and OperationMetadata.attributes are BTreeMaps, so map entries go out sorted by key (source). The same view or operation always encodes to the same bytes; readers accept any order.

Refs: RefTarget, RefConflict, RefTargetTerm, RefConflictLegacy

A ref target in Rust is RefTarget = Merge<Option<CommitId>>: a normal ref, an absent one, or a conflicted one. The proto has accumulated three encodings of it:

A missing RefTarget message reads as absent (terms, ref_target_to_proto, ref_target_from_proto).

Bookmarks, tags, remotes and Git refs

legacy bookmark views · remote views and refs · RemoteRefState

local_working_copy.proto

The state of one workspace's files on disk. TreeState is the whole .jj/working_copy/tree_state file; Checkout is the .jj/working_copy/checkout file. Full file.

syntax = "proto3";

package local_working_copy;

enum FileType {
  Normal = 0;
  Symlink = 1;
  Executable = 2;
  Conflict = 3 [deprecated = true];
  GitSubmodule = 4;
}

message MaterializedConflictData {
  // TODO: maybe we should store num_sides here as well
  uint32 conflict_marker_len = 1;
}

message FileState {
  int64 mtime_millis_since_epoch = 1;
  uint64 size = 2;
  FileType file_type = 3;
  MaterializedConflictData materialized_conflict_data = 5;
  reserved 4;
}

message FileStateEntry {
  string path = 1;
  FileState state = 2;
}

message SparsePatterns {
  repeated string prefixes = 1;
}

message TreeState {
  bytes legacy_tree_id = 1 [deprecated = true];
  // Alternating positive and negative terms if there's a conflict, otherwise a
  // single (positive) value
  repeated bytes tree_ids = 5;
  // Labels for the terms of a conflict.
  repeated string conflict_labels = 7;
  repeated FileStateEntry file_states = 2;
  bool is_file_states_sorted = 6;
  SparsePatterns sparse_patterns = 3;
  WatchmanClock watchman_clock = 4;
}

message WatchmanClock {
  oneof watchman_clock {
    string string_clock = 1;
    int64 unix_timestamp = 2;
  }
}

message Checkout {
  // The operation at which the working copy was updated.
  bytes operation_id = 2;
  // An identifier for this workspace. It is used for looking up the current
  // working-copy commit in the repo view. Currently a human-readable name.
  // TODO: Is it better to make this a UUID and a have map that to a name in
  // config? That way users can rename a workspace.
  string workspace_name = 3;
  reserved 1;
}

TreeState

read · save · sparse patterns

FileState, FileStateEntry, FileType, MaterializedConflictData

file state conversions

SparsePatterns, WatchmanClock, Checkout

CheckoutState load/save

default_index.proto

The commit index is stored as immutable segment files. This single message links an operation to the segments that make up the index as of that operation. It's written to .jj/repo/index/op_links/<operation id hex>. Full file.

syntax = "proto3";

package default_index;

// Control file to link operation to index segments.
message SegmentControl {
  // Hash (or file name) of the commit index segment.
  bytes commit_segment_id = 1;
  // First commit stored in the changed-path index segments. Unset if
  // changed-path index is disabled.
  optional uint32 changed_path_start_commit_pos = 2;
  // Hashes (or file names) of the changed-path index segments.
  repeated bytes changed_path_segment_ids = 3;
}

load · write

simple_workspace_store.proto

The repo-level registry of workspaces, kept in .jj/repo/workspace_store/index. Adding, forgetting and renaming a workspace each take a file lock around the read-modify-write. Full file.

syntax = "proto3";

package simple_workspace_store;

message Workspace {
  string name = 1;
  bytes path = 2;
}

message Workspaces {
  repeated Workspace workspaces = 1;
}

read/write · add

secure_config.proto

jj keeps per-repo and per-workspace config in a config directory identified by a generated config ID. Next to that config's config.toml it writes metadata.binpb, which records which repo or workspace the config belongs to. jj checks the stored path against the actual one when loading. Full file.

syntax = "proto3";

package secure_config;

// Metadata about the configuration files for jj
message ConfigMetadata {
  // The absolute path to the workspace's .jj or .jj/repo directory for
  // workspaces and repos respectively.
  // Access via file_util::path_to/from_bytes
  optional bytes path = 1;
}

metadata read/write · path validation