jj / How jj sparse set works

How jj sparse set works

jj sparse set changes which parts of the working-copy commit are written to disk. It adds or removes path prefixes in the workspace's sparse patterns, then checks out or deletes just the files whose inclusion changed. Commits and the operation log are untouched.

On this page
  1. 1. Command-line surface
  2. 2. Configuration
  3. 3. End-to-end flow
  4. 4. Core architecture touchpoints
  5. 5. Transaction and operation
  6. 6. Algorithm and invariants
  7. Computing the list
  8. Updating the files
  9. 7. Errors and edge cases
  10. 8. Worked example

In short: jj loads the workspace, which snapshots it as usual, and locks the working copy. It starts from the current patterns, or from none with --clear, removes the --remove paths, adds the --add paths, and sorts the result. The working copy then writes the files that are newly covered and deletes the ones no longer covered. It saves its state under the same operation ID and prints Added N files, modified 0 files, removed M files. No operation is recorded.

The command is cmd_sparse_set. It and its siblings jj sparse edit and jj sparse reset each compute a new pattern list and hand it to the shared update_sparse_patterns_with. Its help text's example: jj sparse set --clear --add README.md --add lib to keep only those two (source).

1. Command-line surface

FlagTypeDefaultMeaning
--add PATHrepo path, repeatablenoneA prefix to include (source).
--remove PATHrepo path, repeatablenoneA prefix to drop. It must match an existing pattern exactly. Conflicts with --clear (source).
--clearboolfalseStart from no patterns, so only --add's remain (source).

SparseSetArgs

Paths are parsed by RepoPathBuf::from_relative_path, so they are relative to the workspace root, not the current directory. . alone means the whole repo; any other . or .. component is rejected. With no flags at all, the patterns stay as they are.

2. Configuration

None. The patterns aren't configuration: each workspace keeps its own list in its working-copy state, the sparse_patterns field (3) of the TreeState message, as a list of prefixes (source; see protobufs). A new workspace's list comes from jj workspace add --sparse-patterns, which copies the current workspace's by default.

3. End-to-end flow

jj sparse set: call flow cmd_sparse_set loads the workspace through the shared lifecycle, which snapshots it. The shared update_sparse_patterns_with locks the working copy and reads the current patterns. The closure starts from those patterns, or none with --clear, removes the --remove paths, adds the --add paths, and sorts them. The locked working copy's set_sparse_patterns checks out files under the newly added prefixes and deletes files under the removed ones. It finishes with the old operation ID, so no operation is recorded, and prints the checkout stats. Shared lifecycle: load workspace and repo, snapshot the working copy (CLI page) update_sparse_patterns_with: start_working_copy_mutation (lock); read the current patterns new = (--clear ? {} : current − --remove) ∪ --add, sorted a HashSet, so duplicates collapse TreeState::set_sparse_patterns added = new − old: check out from the tree removed = old − new: delete from disk store the new patterns finish(old operation ID) save the tree state; no new operation print "Added N files, modified 0 files, removed M files" result: disk has exactly the files under the new prefixes the working-copy commit still holds every file (working-copy page)

4. Core architecture touchpoints

StructureRead / writeHowExplained on
Sparse patterns (TreeState)read, writeReplaced by the new list and saved with the working-copy state (source).working-copy
File states (TreeState)writeEntries are added for checked-out files and dropped for deleted ones.working-copy
Files on diskwriteCreated under newly included prefixes, deleted under removed ones.working-copy
Working-copy commit's treereadThe source of the checked-out files. It is never rewritten.trees
Operation log, ViewuntouchedThe working copy is saved with the operation ID it already had (source).operations

5. Transaction and operation

  • Transaction: none. Only the working-copy lock is taken (source).
  • Operation: none. finish gets old_operation_id(), so the checkout state's operation ID doesn't change and only the tree state is rewritten (source, finish).
  • Before it: loading may record a snapshot working copy operation. That snapshot runs first, so edits to files about to be deleted are already in the commit.
  • jj undo does not reverse it, since there is no operation to undo. Run jj sparse set again, or jj sparse reset.

6. Algorithm and invariants

Computing the list

The closure starts from an empty HashSet. Unless --clear is given, it copies the current patterns in and then removes each --remove path. It then inserts each --add path and returns the set sorted (source). Removal is by exact equality: --remove lib/foo doesn't narrow a lib pattern, and --remove lib leaves lib/foo in place if both were listed. Adds win over removes of the same path, since they are applied last.

Updating the files

Each pattern is a path prefix: lib covers lib and everything under it, and the root covers everything (PrefixMatcher). TreeState::set_sparse_patterns builds two difference matchers from the old and new patterns:

  1. Added (new minus old): update from the empty tree to the current tree under it, which writes those files.
  2. Removed (old minus new): update from the current tree to the empty tree under it, which deletes them.
  3. Store the new patterns. Assertions check that nothing was "modified"; the stats report only additions, removals, and additions skipped because something already occupied the path on disk.

Invariants: the working-copy commit and every other commit are unchanged. After the command, the files on disk match the current tree under the new prefixes. Files outside them, even untracked ones, are not touched by the deletion step, which only removes tracked files. The pattern list is kept sorted.

7. Errors and edge cases

MessageWhenSource
clap: invalid value '../file2' for '--add <ADD>': Invalid component ".." in repo-relative path "../file2"A path with .., or . other than on its own.repo_path.rs, test
clap: --remove cannot be used with --clearBoth given.set.rs
Failed to update working copy paths (internal error)Writing or deleting files failed.mod.rs
Warning: <n> of those updates were skipped because there were conflicting changes in the working copy.A newly included file's path was already occupied on disk.cli_util.rs
This command must be able to update the working copy.Run with --at-op, --ignore-working-copy or --no-integrate-operation.cli_util.rs
Concurrent working copy operation. Try again.The working copy changed between loading and locking.cli_util.rs
(no output)No file was added or removed, for instance when no flags are given.cli_util.rs

8. Worked example

From upstream's test_sparse_manage_patterns. @ has file1, file2 and file3, and the patterns start as .. The commands run from a subdirectory, which makes no difference because paths are workspace-relative:

jj sparse set: patterns and files on disk, step by step Start: patterns are dot, and file1, file2 and file3 are on disk. After --remove dot: no patterns, no files on disk, removed 3, but the commit still has all three. After --add file2 --add file3: patterns file2 and file3, added 2. After --add file1 --remove file2 --remove file3: pattern file1, added 1, removed 2. After --clear --add file2: pattern file2, added 1, removed 1. the commit always holds file1, file2, file3 (start) patterns: . on disk: file1 file2 file3 --remove . patterns: (none) on disk: (nothing) removed 3 --add file2 file3 patterns: file2, file3 on disk: file2 file3 added 2 +file1 −file2 −file3 patterns: file1 on disk: file1 added 1, removed 2 --clear --add file2 patterns: file2 on disk: file2 added 1, removed 1

Upstream's snapshot for the first step (source):

$ jj sparse set --remove .
Added 0 files, modified 0 files, removed 3 files

jj sparse list then prints nothing, the three files are gone from disk, and jj file list still prints all three: they are in the commit, just not checked out. In the fourth step one call adds file1 and removes the other two, and the stats count both directions.