jj-vfs-poc / Commits as directory trees

Commits as directory trees

Every path below commits/<id>/ or workspaces/<name>/ is a CommitTreeFile: a commit plus a path inside it. Each call asks jj-lib for the value at that path and turns it into a directory, a file or a symlink.

On this page
  1. The type
  2. Looking a path up
  3. How each kind of path is presented
  4. Listing a directory
  5. Reading content
  6. Sizes and attributes
  7. Conflicts
  8. Tests
  9. Limitations

In short: a CommitTreeFile is built for every request. It looks the path up in the commit's MergedTree. Directories, regular files and symlinks work. Executable files show as ordinary files, submodules as files that fail on every access, and conflicted paths can't be read or stat'ed. A file's size is found by reading the whole file, and timestamps are always the Unix epoch.

This page covers src/commit_tree_file.rs, the only VirtualFile that reads repository content. For jj's own model of trees and conflicts, see Trees and Conflicts in the jj reference. The jj-lib links here are pinned to v0.43.0.

The type

ItemType/SignatureMeaningSource
CommitTreeFilestruct { commit: Commit, path: PathBuf }One path inside one commit. The path may name nothing; that is only found out when a method runs.L23
newasync fn new(repo: &ReadonlyRepo, commit_id: CommitId, path: PathBuf) -> Result<Self, JjError>Loads the commit with Store::get_commit_async; fails only if the commit can't be read.L28
readasync fn read(&self) -> Result<Pin<Box<dyn AsyncRead + Send>>, JjError>A reader over a regular file's content.L42
listasync fn list(&self) -> Result<DirectoryStream, JjError>A directory's entries with their types.L60
read_linkasync fn read_link(&self) -> Result<PathBuf, JjError>A symlink's target.L107
attributesasync fn attributes(&self) -> Result<FileAttributes, JjError>Type, size and timestamps.L128
file_typeasync fn file_type(&self) -> Result<FileType, JjError>Directory, file or symlink.L155

AllCommitsPathMapper creates one for commits/<id>/… and for workspaces/<name>/…, passing the remaining segments as path (L56, L73). Nothing is cached: each FUSE request resolves its path again, so each one loads the commit again. commits/<id> itself is the commit's root, with an empty path.

Looking a path up

Every method starts the same way. It converts path with RepoPathBuf::from_relative_path, which rejects components like .. with InvalidPath. Then it reads the commit's tree:

  • read, read_link and file_type call MergedTree::path_value. It walks the merged trees down to the parent directory and returns the Merge<Option<TreeValue>> for the last component. The lookup goes through Merge<Tree>::value, which resolves trivial merges, so a path that is the same on every side of a conflicted commit still comes back resolved. The root path returns the commit's tree IDs unchanged.
  • Each method then calls as_resolved, which only succeeds for a single-term merge. Anything else is treated as a conflict, described below.
  • list instead calls MergedTree::trees and Merge<Tree>::sub_tree_recursive to get the directory's own trees. If that finds no directory, it looks at path_value to choose between NotFound and NotADirectory (L67).

How each kind of path is presented

A resolved value is one of the four TreeValue variants, or absent. Errors become errnos through the FUSE adapter's mapping (fuse.rs).

Value at the pathListed asgetattr / lookupreadreadlink
Treedirectorydirectory, size 0, mode 755EISDIREINVAL
File, not executablefileregular file, mode 644, size from reading itits contentEINVAL
File, executablefilethe same: executable is never read, so the mode is still 644its contentEINVAL
Symlinksymlinksymlink, size = target length in bytesEISDIRits target
GitSubmodulefileEISDIR: typed as a file, then sizing it calls read, which refusesEISDIREINVAL
absentnot listedENOENTENOENTENOENT
conflictedfileEIO (“Conflicted path”)ENOTDIREINVAL

Sources: file_type (L155), read (L48), read_link (L113), attributes (L129), the listing (L86) and the mode, which to_fuse picks from the type alone (fuse.rs). EISDIR for a symlink or a submodule comes from NotAFile; it doesn't mean they are directories.

How CommitTreeFile presents one path The path inside the commit is converted to a RepoPathBuf; an invalid path fails with EIO. Then MergedTree::path_value returns a merge of tree values. If the merge has more than one term it is a conflict, and the path can't be read or stat'ed. If it is resolved, the value decides: a Tree becomes a directory, a File becomes a regular file with mode 644 whether or not it is executable, a Symlink becomes a symlink, a GitSubmodule becomes a file that fails with EISDIR, and an absent value is ENOENT. RepoPathBuf::from_relative_path “..” and similar → InvalidPath → EIO MergedTree::path_value Merge<Option<TreeValue>>, trivial merges resolved more than one term conflict: EIO / ENOTDIR as_resolved() → one value Tree directory, 755 File file, 644 even if +x Symlink symlink → target GitSubmodule “file”, EISDIR absent (None) ENOENT Derived from the code at the pin. Only directories, regular files and symlinks are covered by upstream tests.

Listing a directory

list gets the directory as a Merge<Tree>: one tree when it is resolved, one per term when it is conflicted. Where a term has no directory at that path, jj-lib substitutes an empty tree (sub_tree). It then:

  1. takes the names from every term's entries_non_recursive and drops repeats with itertools' unique, so a name present on any side is listed once (L76);
  2. builds each name's Merge<Option<TreeValue>> by mapping Tree::value over the terms (L82);
  3. types it as above if that merge has one term, and as a file otherwise (L86);
  4. collects everything into a Vec and returns it as a stream (L102).

Step 2 doesn't use Merge<Tree>::value, so it never resolves trivial merges. In a conflicted directory every term is kept, and every entry is listed as a file, even subdirectories and files that are the same on every side. A lookup of such an entry goes through path_value, which does resolve trivial merges, so it can report a different type than the listing did. In a resolved directory there is one term and the types are right.

The names come out in the order of the first term's entries, followed by names only the other terms have. Listing is the slow path for large directories: the Vec holds every entry, and readdir calls list again for each batch and skips to its offset. Upstream's TODOs at L80 and L102 note the lack of pagination.

Reading content

read passes the resolved File's id to Store::read_file and returns the backend's AsyncRead as is (L56). read_link does the same for a Symlink with Store::read_symlink and returns the target string as a path (L120).

The reader always starts at byte 0. For a FUSE read(offset, size), PathMappedVfs::read discards the first offset bytes and then reads size (vfs.rs), so reading a file from start to end in chunks reads the earlier part again for every chunk.

Sizes and attributes

FieldValueSource
size of a fileThe byte count from reading the whole file into a sink. Every getattr and lookup of a file reads all of it; upstream's TODO says jj-lib would need a way to get the size without reading.L132
size of a directory0.L140
size of a symlinkThe target's length in bytes.L141
created, modifiedUNIX_EPOCH, marked TODO. The commit's own timestamps aren't used.L150
mode, links, owner, atimeSet by the FUSE adapter from the type: 755 or 644, 2 or 1 links, uid and gid 1000, atime the current time.FileAttr

attributes calls file_type first, so it looks the path up twice, and for a file a third time inside read.

Conflicts

jj stores a conflicted commit's tree as several trees (see tree merges). Here, a path whose value doesn't resolve, even trivially, is refused: read returns NotADirectory, file_type an I/O error “Conflicted path”, and read_link NotASymlink. Upstream's TODOs at L48 and L163 plan to show conflicts. Conflict markers are never materialized.

Derived from the code, not tested upstream: directories count too. The root of a conflicted commit has one tree per term, and so does every directory on the way down to a conflicted file, unless jj can resolve it trivially. file_type refuses those as well, so getattr on commits/<id> of a conflicted commit fails with EIO, even though list could list it.

Tests

All tests use the repository from test_helpers::setup_test_repo: file1.txt (“hello content 1”), dir/file2.txt and a symlink symlink → file1.txt, in one resolved commit.

TestWhat it pins downSource
test_list_directory_rootThe root lists dir, file1.txt, symlink in that order, typed directory, file, symlink.L187
test_list_directory_subdir lists one file, file2.txt.L206
test_file_sizeThe size is the content's length.L221
test_read_fileread returns the content.L232
test_invalid_pathsome/../path is InvalidPath for list and read.L245
test_not_foundMissing paths are NotFound for list and read.L274
test_is_directoryReading a directory is NotAFile.L305
test_list_file_not_a_directoryListing a file is NotADirectory.L319
test_file_typeDirectory, file and symlink are typed correctly.L333
test_read_symlink_fileThe symlink's target is file1.txt.L359

No test covers executable files, submodules or conflicts.

Limitations

LimitationEffectSource
Conflicts are refused.Conflicted files can't be read or stat'ed, and neither can the directories above them.L161
Conflicted directories list everything as files.The listing's types disagree with lookup.L82
The executable bit is dropped.Scripts can't be run from the mount.fuse.rs
Submodules aren't supported.They appear in listings as files that fail with EISDIR.L52
Sizes need a full read.Every stat of a file reads the whole file.L135
No seeking.Each read chunk re-reads everything before it.vfs.rs
Listings aren't paginated.Each readdir batch builds the whole listing again.L102
No timestamps.Every file and directory is dated 1970-01-01.L150
Nothing is cached.Every request loads the commit and walks the trees again.L30