jj-vfs-poc / The mount's namespace

The mount's namespace

What a mounted repository looks like, how AllCommitsPathMapper turns each path into a VirtualFile, and which snapshot of the repository every lookup sees.

On this page
  1. 1. The tree
  2. 2. Path resolution
  3. Where paths come from
  4. get_entry, step by step
  5. Commit IDs: full only, and how they fail
  6. All the errors resolution can return
  7. 3. The directory types
  8. How CommitsDirectory lists commits
  9. 4. Repo snapshot semantics
  10. 5. Worked example
  11. What cat <mountpoint>/commits/<A>/file1.txt does
  12. 6. Limitations the code makes visible

In short: the mount has two top-level directories. commits/ holds one directory per visible commit, named by its full hex commit ID. workspaces/ holds one directory per workspace, showing that workspace's working-copy commit. Inside either, you see the commit's tree as files, directories and symlinks. Every request re-resolves its full path from scratch against one ReadonlyRepo, loaded once at mount time. The mount never sees later operations, so it is a frozen snapshot.

All of this is decided by one type, AllCommitsPathMapper, the only PathMapper in the crate. A PathMapper has a single method, get_entry(path) → Box<dyn VirtualFile>, and four VirtualFile implementors sit behind it: StaticDirectory, CommitsDirectory, WorkspacesDirectory and CommitTreeFile. The traits themselves are on the virtual file layer page.

1. The tree

This is the complete namespace at the pinned commit. Nothing else exists: any other name at the top level is ENOENT, and there are no hidden files, no .jj directory and no metadata files.

Directory tree of a jjfsd mount The mount root contains exactly two directories, commits and workspaces, served by a StaticDirectory. commits is a CommitsDirectory with one subdirectory per visible commit, named by its full 40-character hex ID, including the all-zero root commit. workspaces is a WorkspacesDirectory with one subdirectory per workspace name. Each commit or workspace directory is a CommitTreeFile showing that commit's tree: directories, regular files and symlinks. <mountpoint>/ commits/ 0000000000000000000000000000000000000000/ <full hex commit id>/ dir/ file1.txt symlink -> file1.txt … one directory per visible commit workspaces/ default/ … one per workspace, same view as its @ commit StaticDirectory CommitsDirectory CommitTreeFile, path "" CommitTreeFile, path "" CommitTreeFile, path "file1.txt" WorkspacesDirectory

The contents of a commit directory (here dir/, file1.txt and symlink, the tree the upstream test helper builds) are whatever that commit's tree holds. The commits as directory trees page covers how. The all-zero entry is jj's root commit, which has an empty tree. It is listed because all() includes it, and it resolves because the Git backend answers the all-zero ID with a synthetic root commit (source).

2. Path resolution

Where paths come from

The kernel never sends a path, only an inode number and sometimes a name. Before anything reaches the path mapper, reply_async! in JjFuse asks InodeMap::get_path to rebuild the path by following parent links up to inode 1. The result is relative and has no leading slash: the root is the empty path "", and a file inside a commit is commits/<id>/dir/file2.txt. For lookup, JjFuse pushes the requested name onto the parent's path first. Every VirtualFilesystem method of PathMappedVfs then starts the same way: self.path_mapper.get_entry(path).await?, and then a single trait call on the result. Nothing is cached between requests: every getattr, readdir, read and readlink resolves the full path again.

get_entry, step by step

AllCommitsPathMapper::get_entry walks the path with path.iter() and looks at no more than two components itself:

  1. No components (the root): return a fresh StaticDirectory listing commits and workspaces, both as directories (source).
  2. First component: convert it to UTF-8, or fail with InvalidPath. Then match on it. "commits" and "workspaces" go on to step 3, and anything else is NotFound (source, source).
  3. Under commits (source): with no second component, return a CommitsDirectory. Otherwise parse the second component with CommitId::try_from_hex, where a string that isn't valid hex gives NotFound. Then build a CommitTreeFile from the commit ID and all remaining components (segments.collect()) as the path inside the commit.
  4. Under workspaces (source): with no second component, return a WorkspacesDirectory. Otherwise look the name up in the repo view's wc_commit_ids(), where an unknown workspace gives NotFound. Then build a CommitTreeFile for that workspace's working-copy commit and the remaining components, exactly as in step 3.

CommitTreeFile::new immediately loads the commit with Store::get_commit_async, so a bad commit ID fails during resolution, before any file operation. It does not look at the path inside the commit. A missing file inside a real commit only fails later, when the operation asks the tree for that path.

Commit IDs: full only, and how they fail

The directory name must be the commit's full hex ID: 40 digits in a Git-backed repository, which is any repository made with jj git init or jj git clone. There is no prefix resolution, no change IDs, and no revset or bookmark names. The failure modes differ in ways a user can see. Every row below was checked by running the pinned code against a fresh Git-backed repository:

Second component under commits/What happensJjErrorerrno
Not valid hex, or an odd number of digitstry_from_hex returns NoneNotFoundENOENT
Valid hex, but not 40 digits (a prefix)The Git backend rejects the length with BackendError::InvalidHashLength (source). The local backend the upstream tests use reports ObjectNotFound instead.JjLibBackendErrorEIO
40 hex digits naming no objectThe backend's object lookup fails with BackendError::ObjectNotFoundJjLibBackendErrorEIO
The ID of any commit object in the storeResolves. The store is not checked against the view, so this includes commits that commits/ doesn't list——

So ls <mount>/commits/abcd reports "Input/output error", not "No such file or directory": every error other than NotFound, NotADirectory, NotAFile and NotASymlink becomes EIO in From<JjError> for Errno.

All the errors resolution can return

ItemType/SignatureMeaningSource
InvalidPathJjError → EIOA component isn't valid UTF-8, or the path inside a commit isn't a valid repo path.jj_error.rs
NotFoundJjError → ENOENTUnknown top-level name, non-hex commit directory, unknown workspace, or (later) a missing path inside a commit. Also an inode the InodeMap doesn't know.jj_error.rs
JjLibBackendErrorJjError → EIOAny jj-lib backend failure, including a commit ID of the wrong length or one that doesn't exist.jj_error.rs

3. The directory types

All three are cheap wrappers built fresh by get_entry on every request. None of them implements read or read_link, so those fall through to the VirtualFile defaults and return NotAFile (EISDIR) and NotASymlink (EINVAL).

Itemlistattributesfile_type
StaticDirectory The Vec<DirectoryEntry> it was built with, cloned into a stream, in that order. size 0, Directory, created and modified at UNIX_EPOCH (marked TODO) Directory
CommitsDirectory Every commit in ResolvedRevsetExpression::all() evaluated on the repo, as a directory named commit_id.hex(). Entries whose ID fails to load are silently skipped. size 0, Directory, UNIX_EPOCH Directory
WorkspacesDirectory The keys of view.wc_commit_ids(), a BTreeMap, so sorted by name, each as a directory. size 0, Directory, UNIX_EPOCH Directory

How CommitsDirectory lists commits

CommitsDirectory::list evaluates jj-lib's all(), which resolves to the ancestors of the visible heads (source), so that is every visible commit, including the root. In jj terms, it is the same set as jj log -r 'all()'. The source explains the next step: "Revset does not implement Send", so the stream can't cross threads. Instead of streaming, it calls futures::executor::block_on inside the async function to collect every ID into a Vec, and then streams that. This blocks a tokio worker thread for the whole evaluation, and the source flags both the blocking and the missing pagination as temporary.

The upstream test test_repo_commits pins the count: a fresh repo with one test commit lists 3 entries ("root commit + initial workspace commit + our test commit"), all directories. test_repo_workspaces pins the single default workspace.

4. Repo snapshot semantics

main loads the workspace, calls RepoLoader::load_at_head exactly once (merging divergent operation heads if there are any), and moves the resulting Arc<ReadonlyRepo> into AllCommitsPathMapper::new. That Arc is never replaced. Every directory type clones it, and no code path reloads the repo. So for the whole life of the mount:

  • commits/ lists the visible commits as of mount time. Commits created, rewritten or abandoned later don't change the listing. Abandoned ones stay listed, and new ones never appear.
  • workspaces/ is frozen too. Workspaces added later don't appear, and workspaces/<name>/ keeps showing the working-copy commit as of mount time, even after jj new or jj edit moves @. It is not a live view of the working copy. In particular, it never shows uncommitted edits on disk, which jj only records when it snapshots.
  • Lookup by commit ID bypasses the view. commits/<full id>/ resolves any commit the store can read, so a hidden commit's tree can still be reached by ID even though ls commits/ doesn't show it.

The kernel adds one more layer of staleness: JjFuse replies with a 1-second TTL on every entry and attribute (source), so the kernel may answer repeated lookups from its cache for up to a second. Since the snapshot never changes, this costs nothing in correctness.

5. Worked example

Take a repository with three commits besides the root: A, which adds file1.txt ("hello content 1"); B, a child of A, the working-copy commit of the default workspace; and C, another child of A, the working-copy commit of a second workspace, docs. Mounted with jjfsd <mountpoint> <workspace root>, these are exactly the paths that exist (<A> standing for A's 40-digit ID, and so on):

<mountpoint>/
├── commits/
│   ├── 0000000000000000000000000000000000000000/   (empty: the root commit's tree)
│   ├── <A>/file1.txt
│   ├── <B>/file1.txt
│   └── <C>/file1.txt
└── workspaces/
    ├── default/file1.txt                            (= commits/<B>)
    └── docs/file1.txt                               (= commits/<C>)

The commit directories are listed newest first, with the root last, which is the order the revset stream yields them. The workspaces are sorted by name. If B and C changed nothing, all three file1.txts have the same content, but they are separate paths with separate inodes, and each is read from the store independently.

What cat <mountpoint>/commits/<A>/file1.txt does

Starting from a cold kernel cache, the kernel resolves the path one component at a time with lookup, opens the file, and reads until it gets a short read. Each FUSE call arrives on a fuser thread, and JjFuse immediately spawns the work onto the tokio runtime and returns. The reply is sent from inside that task.

Sequence of calls for cat on a file in a commit directory cat opens the file. The kernel sends lookup for commits under inode 1, for the commit ID under inode 2, and for file1.txt under inode 3. For each, JjFuse spawns a tokio task that asks the InodeMap for the parent path and a child inode number, then asks PathMappedVfs for attributes, which calls AllCommitsPathMapper.get_entry. For the commit directory and the file this builds a CommitTreeFile, which loads the commit from jj-lib's Store. For the file, attributes reads the whole file to count its size. The kernel's open is answered by fuser's default. Then the kernel sends read for inode 4 at offset 0. The task gets the path back from the InodeMap, resolves it again, reads the file from the Store, skips to the offset and returns up to size bytes. A second read at offset 15 returns no data, which ends the file. tokio task (spawned per request) cat kernel JjFuse InodeMap PathMappedVfs+ get_entry CommitTreeFile jj-lib Store open(".../file1.txt") lookup(1, "commits") get_path(1) = "", get_ino → 2 get_attributes("commits") → CommitsDirectory entry(ino 2, dir) lookup(2, "<A>") get_ino → 3 get_attributes("commits/<A>") new(repo, A, "") get_commit_async entry(ino 3, dir) lookup(3, "file1.txt") get_ino → 4, get_attributes(".../file1.txt") path_value, read_file ×2 entry(ino 4, file, 15 bytes) open: fuser's default, file handle 0 read() read(4, offset 0, size) get_path(4) read("commits/<A>/file1.txt", 0, size) get_commit, path_value, read_file data(15 bytes) read(4, offset 15, size) data(0 bytes) EOF

Three details show up here that a reader might not expect:

  • Getting the file's size reads the whole file. CommitTreeFile::attributes calls file_type (one path_value), then for a file calls read() and copies the stream into a sink to count its bytes, with a TODO noting jj-lib would need changes to avoid this. read() does its own path_value, which is why the lookup row says read_file after it.
  • Each read starts from the beginning. PathMappedVfs::read opens a new reader, copies offset bytes into a sink ("TODO: handle proper seek()"), then takes size bytes. Reading a large file in chunks costs time proportional to the square of its length.
  • Inodes are allocated before the path is known to exist. lookup calls get_ino before get_attributes, so a lookup that ends in ENOENT still leaves a new inode in the map, and forget is left to fuser's no-op default (fuser source). Inodes are never freed.

6. Limitations the code makes visible

LimitationWhere it showsSource
Read-only mount. There are no write operations at all, and the mount is created with MountOption::RO and file system name jjfs.Writes fail. JjFuse implements only init, lookup, getattr, read, readdir and readlink.main.rs, fuse.rs
Frozen snapshot: the repo is loaded once at mount.New commits, moved working copies and new workspaces never appear.main.rs
Full commit IDs only. Prefixes fail with EIO, not ENOENT.commits/<prefix>path_mapper_all_commits.rs
Conflicted paths can't be read, typed or sized.read returns NotADirectory (ENOTDIR) on a conflict. file_type, and so getattr, returns an InvalidData I/O error ("Conflicted path"), which is EIO. In a listing, conflicted entries show as regular files.read, file_type, list
Submodules (and any other value kind) are shown as files but can't be read.Listing says file, and read returns NotAFile (EISDIR).read, list
Executable bit not shown. Every non-directory is 0644, every directory 0755, owned by uid/gid 1000.ls -lfuse.rs
No timestamps. Created and modified are UNIX_EPOCH everywhere, and access time is "now".ls -l, statfuse.rs
No pagination. commits/ collects every visible commit (blocking a worker thread), and commit directories collect their whole listing before replying.Large repositoriescommits_directory.rs, commit_tree_file.rs
Whole-file reads for size and for every read offset.Large filesattributes, read
Inodes are never forgotten.Memory grows with every distinct path ever looked up, including failed ones.inode_map.rs

One upstream test is named for a case it doesn't exercise. test_all_commit_trees_mapper_invalid_commit_id resolves the path invalid_commit_hex at the top level, so it hits the "unknown top-level name" arm, not commit-ID parsing. Nothing in the suite covers a bad ID under commits/.