jj / How jj file search works

How jj file search works

jj file search is a grep over one commit's tree. It walks the same filtered tree as jj file list, reads each file's contents from the store, and tests every line against a string pattern. Conflicted files are searched on each of their sides.

On this page
  1. 1. Command-line surface
  2. 2. Configuration
  3. 3. End-to-end flow
  4. 4. Core architecture touchpoints
  5. 5. What it reads, and snapshotting
  6. 6. Algorithm and invariants
  7. 7. Errors and edge cases
  8. 8. Example output

In short: jj resolves -r (default @), turns the positional filesets into a matcher, and parses --pattern as kind:pattern, with the kind defaulting to regex. For each matching entry it materializes the value. A file's bytes are split on \n, and each matching line is printed as path:line (or path:N:line with -n, or just the path with --name-only). Symlinks, submodules and non-file conflicts are skipped. Nothing is written.

The command is cmd_file_search, and the line printer is write_matches. The help text: "Search for content in files", followed by "Prints each line that matches the specified pattern, prefixed by the file path. Use --name-only to print only the file paths" (source).

1. Command-line surface

ArgumentTypeDefaultMeaning
-p, --pattern PATTERN (required)string patternkind regexThe pattern each line is tested against. A glob has to match the whole line, so use 'glob:*foo*' to find foo anywhere (source).
FILESETS (positional, repeatable)filesetall filesOnly search files these match. Several are unioned (source).
-r, --revision REVSETrevset@The commit whose tree is searched. It must resolve to exactly one commit (source).
--name-onlyflagoffPrint each matching file's path once instead of its lines. Can't be combined with -n (source).
-n, --line-numberflagoffPrefix each match with its 1-based line number (source).

FileSearchArgs

As with jj file list, the help calls the positional arguments "prefixes", but each is a full fileset expression, parsed by parse_file_patterns (cli_util.rs, fileset.rs). Upstream's fileset docs cover the language.

--pattern is split at its first colon (source). The part before the colon has to be one of the eight kinds from_str_kind knows (str_util.rs), so foo:bar is an error rather than a regex. Colons after the first are part of the pattern, so regex-i:foo:bar searches for foo:bar. To search for a literal colon in a regex, spell the kind out.

KindA line matches when
regex (default), regex-iThe regular expression matches somewhere in the line. It runs on bytes, so non-UTF-8 files are searched too.
substring, substring-iThe line contains the text.
glob, glob-iThe whole line matches the glob. A glob with no wildcard characters becomes exact.
exact, exact-iThe whole line equals the text, so exact: finds blank lines.

StringPattern, glob, match_lines. exact-i and substring-i fold ASCII case only; regexes are Unicode-aware (source).

2. Configuration

There is no template and no config key of its own: the output format is hard-coded, and a TODO says so (source). Paths are printed relative to the current directory with format_file_path, so running in dir/ prints file3:… rather than dir/file3:…. revset-aliases apply when resolving -r, fileset-aliases when parsing the filesets, and ui.pager controls paging. The matched line is written as raw bytes, without colour.

3. End-to-end flow

jj file search: call flow cmd_file_search loads the workspace through the shared lifecycle, resolves -r (default @) to one commit, takes its tree and builds a matcher from the filesets. It parses --pattern by splitting at the first colon into kind and pattern, defaulting to regex, and builds a StringMatcher. It requests the pager and streams tree.entries_matching(matcher). Entries are materialized from the store and files read, several at once but kept in order, then dispatched by kind: files and file conflicts are searched line by line and printed. Finally print_unmatched_explicit_paths warns about literal paths absent from the tree. Shared lifecycle: load workspace and repo, snapshot the working copy (CLI page) resolve_single_rev(-r or @) → commit.tree() parse_file_patterns(FILESETS) → all() if empty fileset_expression.to_matcher() --pattern → split_once(':') from_str_kind(pattern, kind or "regex") pattern.to_matcher() → StringMatcher request_pager; tree.entries_matching(matcher) the same walk as jj file list: subtrees read concurrently, yielded in path order then materialize + read_all, buffered(store.concurrency()), order kept materialize_tree_value → dispatch file: read_all, then match lines file conflict: match each add side see the algorithm below print_unmatched_explicit_paths tree.path_value(p) for each literal path absent → Warning: No matching entries for paths: … (stderr, exit 0)

4. Core architecture touchpoints

StructureRead / writeHowExplained on
CommitreadResolved from -r; only its tree is used.commits
TreesreadWalked with entries_matching, exactly as jj file list does (source).file list, trees
File contentsreadEvery matched file's blob is read in full through the store (source).backends
ConflictsreadA conflict of files is simplified and each term's contents read; only the add sides are searched (source).conflicts
Filesets and matchersreadThe positional arguments become a Matcher (source).upstream docs
Revset enginereadResolves -r to one commit (source).revsets
Working copyread, may snapshotSnapshotted by the shared lifecycle, so unsaved edits on disk are searched in @.working-copy

5. What it reads, and snapshotting

  • No transaction, no operation of its own. The command only reads the repo it loaded (source).
  • Snapshot: workspace_helper snapshots the working copy first (source). If files changed on disk, that snapshot records its own operation, so jj file search sees what is on disk now, not the last snapshot.
  • Whole blobs, from the store. Even for @, contents come from the commit's tree objects, not the files on disk. Each file is read to the end into memory (read_all), and binary files are searched like any other. Symlink targets are read too, by the materializer, but never searched.
  • Concurrent reads, ordered output. Up to the backend's concurrency() entries are materialized and read at once, with buffered, which still hands results back in tree order. Matching and printing happen one file at a time on the command's side (source).
  • jj undo has nothing to revert, except the snapshot if one was taken.

6. Algorithm and invariants

Each entry the walk yields is passed to materialize_tree_value (source), and a resolved file's bytes are read in the same buffered step (source). The command then branches on the result:

jj file search: what happens to each kind of entry materialize_tree_value turns a tree entry into one of several kinds. A resolved file is read in full and passed to write_matches, or with --name-only its path is printed if any line matches. A conflict whose terms are all files or absent becomes a file conflict; only its add sides are searched, each one separately, and base sides and conflict markers never are. A read refused by the backend becomes AccessDenied and prints a warning. Symlinks, Git submodules and other conflicts are skipped silently. Absent and tree values are not expected from the walk, which descends into trees, and the command panics on them. materialize_tree_value(path, value) reads the blob(s) from the store File read_all → write_matches --name-only: path if any line matches FileConflict contents.adds(): each side searched separately; bases and markers never are AccessDenied Warning: Skipping '…' due to permission error: … then carries on Symlink, GitSubmodule, OtherConflict skipped silently; OtherConflict is any conflict with a non-file term, e.g. file vs. symlink Absent, Tree panic: "Entry for … in file listing"; the walk descends into trees instead of yielding them

Line matching. write_matches splits the contents with split_inclusive('\n'), numbering pieces from 1. It strips the trailing \n before testing, so patterns can anchor to the line's end, then prints the prefix and the original line, adding a newline when the last line has none (source). --name-only uses match_lines, which splits the same way, and stops at the first match (source). Only \n separates lines, so a CRLF line is tested with its \r still attached.

Conflicts. try_materialize_file_conflict_value applies only when every term is a file or absent. It simplifies the merge, dropping pairs that cancel out, and reads each remaining term; an absent side reads as empty (source, get_file_contents). The command then searches only the adds(): the contents each side wants, never the base it diverged from, and never the marker text written to the working copy. With -n, numbering restarts on each side, because there is no line numbering shared across sides (source).

Invariants: output follows tree order (depth-first, name order) however the reads finish, and within a file, line order (sides in merge order for a conflict). Each matching line is printed once per side it appears on; with --name-only each path at most once. Literal paths that name nothing get one warning afterwards (explicit_paths). An entry that matches the filesets but no lines produces no output.

7. Errors and edge cases

MessageWhenSource
Error: Invalid string pattern kind `foo:` (exit 2)The text before the first colon isn't a known kind. It is a CLI error, like a bad flag.str_util.rs, search.rs
Glob or regex syntax error (exit 2)The pattern doesn't compile; the globset or regex error is shown as-is.str_util.rs
Warning: Skipping '<path>' due to permission error: …The backend refused to read a file. The search continues.search.rs, conflicts.rs
Warning: No matching entries for paths: <paths>A literal path in the filesets is absent from the tree. The command still succeeds.cli_util.rs
Revset `…` resolved to more than one revision (or to none)-r isn't exactly one commit.revset_util.rs
Fileset parse errorAn argument isn't a valid fileset, or names a path outside the workspace.cli_util.rs
Usage error--name-only together with -n, or no --pattern; clap rejects it before the command runs.search.rs
Backend errorA blob or subtree can't be read for another reason; the command stops after the lines already printed.search.rs

A pattern that matches nothing prints nothing and exits 0; there is no "no matches" exit status as in grep.

8. Example output

From upstream's test_file_search. @- adds file1 (-foo-) and file2 (-bar-); @ adds dir/file3 (-foobar-): 

$ jj file search --pattern 'glob:*foo*'
dir/file3:-foobar-
file1:-foo-

$ jj file search --pattern 'glob:foo'      # a glob must match the whole line

$ jj file search --pattern 'glob:*foo*' -r @-
file1:-foo-

$ jj file search --pattern 'glob:*foo*' file9
Warning: No matching entries for paths: file9

$ jj file search --pattern foo:bar
Error: Invalid string pattern kind `foo:`

Every matching line, with line numbers, or only the path:

$ printf 'hit-one\nmiss\nhit-two\nhit-three\n' > multi
$ jj file search -n --pattern hit multi
multi:1:hit-one
multi:3:hit-two
multi:4:hit-three

$ jj file search --name-only --pattern hit multi
multi

From test_file_search_conflicts: file1 conflicts with add sides -foo- and -baz- and base -bar-. Each side is searched on its own, so both report line 1. The base and the conflict markers are never matched: 

$ jj file search -n --pattern 'regex:-(foo|baz)-'
file1:1:-foo-
file1:1:-baz-

$ jj file search --pattern 'glob:*bar*'

$ jj file search --pattern 'glob:*%%%*'