jj / How jj run works

How jj run works

jj run checks each selected commit out into its own scratch working copy, runs a command there, and amends the commit with whatever the command left behind. It can also just run a check across many commits without changing any of them.

On this page
  1. 1. Command-line surface
  2. 2. Configuration
  3. 3. End-to-end flow
  4. 4. Core architecture touchpoints
  5. 5. Transaction and operation
  6. 6. Algorithm and invariants
  7. The workspace pool
  8. One job
  9. Scheduling
  10. Rewriting
  11. 7. Errors and edge cases
  12. 8. Worked example

In short: jj resolves the revisions (default revsets.run) and orders them parents first. Unless --ignore-changes is given, it refuses immutable ones. It then runs the command on up to --jobs commits at once, each in a reusable slot under .jj/run/default/, with JJ_CHANGE_ID, JJ_COMMIT_ID and JJ_WORKSPACE_ROOT set. After each command it snapshots the slot. A failure stops the run and rewrites nothing, unless --ignore-errors is given. At the end one transaction rewrites every commit whose tree changed and rebases the rest of their descendants. The operation is run: rewrite <n> commits.

The command is cmd_run. Its help text names the environment variables and gives jj run -j 4 -- pre-commit run .github/pre-commit.yaml as an example (source). Unlike jj fix, which pipes single files through formatters in memory, jj run materializes whole trees on disk, so it can run builds and test suites.

1. Command-line surface

FlagTypeDefaultMeaning
COMMAND (positional)stringrequiredThe program to run in each commit (source).
ARGS (positional)stringsnoneIts arguments. Put -- first to pass arguments starting with - (source).
-r, --revision, --revisions REVSETSrevsetsrevsets.runThe commits to run on (source).
-xboolhiddenDoes nothing. It exists to match git rebase -x (source).
-j, --jobs Nintegerrun.jobs, else 1How many commands run at once, which is also the number of slots (source).
--rootboolfalseRun from each slot's root instead of the subdirectory jj run was started in (source).
--cleanboolfalseWipe each slot before checking out, so no build artifacts are reused (source).
--restore-descendantsboolfalseKeep descendants' content, not their diffs (source).
--passthroughboolfalseConnect the command's stdout and stderr to the terminal instead of capturing them. Only one job is allowed (source).
--ignore-changesboolfalseRun without rewriting anything, which also allows immutable commits. Conflicts with --restore-descendants (source).
--ignore-errorsboolfalseKeep going after a failing command. Failed commits aren't changed, successful ones still are, and jj run itself still succeeds (source).
--sparse-patternscopy | full | emptycopyWhich sparse patterns the slots use: the current workspace's, everything, or nothing (source, values).

RunArgs

2. Configuration

KeyDefaultEffect
revsets.runreachable(@, mutable())The commits used when no -r is given (default, use).
run.jobsunset, meaning 1Number of parallel jobs when --jobs isn't given. Zero or a negative number is an error (source).
snapshot.auto-trackall()Which new files a command creates get added to the commit (field, matcher).
revset-aliases."immutable_heads()"built inDecides which commits may be rewritten, unless --ignore-changes is given.

snapshot.max-new-file-size does not apply: slots use a fixed limit of 64,000 bytes, so a larger new file stays untracked. The code comment beside the value says "64 MB" (source). Gitignore rules come from the workspace's base ignores plus the .gitignore files in each checked-out tree.

3. End-to-end flow

jj run: call flow cmd_run creates .jj/run/default, loads the workspace through the shared lifecycle, resolves the revisions (default revsets.run) and reverses them so parents come first. Unless --ignore-changes is given it checks they are rewritable. It resolves the job count, rejects --passthrough with more than one job, works out the subdirectory and sparse patterns, and starts a transaction. A producer, run_inner, keeps up to jobs tasks in flight on a multi-threaded runtime. Each task, rewrite_commit, acquires a slot from the workspace pool, checks the commit out, runs the command with the JJ environment variables, snapshots the slot and sends back the new tree. A consumer prints each job's output, stops on the first failure unless --ignore-errors is given, and records changed trees. If nothing changed, or with --ignore-changes, the transaction finishes with nothing to record. Otherwise transform_descendants rewrites the changed commits and rebases the others, and tx.finish records "run: rewrite N commits". Shared lifecycle: load workspace and repo, snapshot the working copy (CLI page) resolve -r (or revsets.run), reverse to parents-first; check_rewritable unless --ignore-changes resolve_jobs: --jobs, then run.jobs, then 1; --passthrough needs exactly one job subdir (unless --root), auto-track matcher, sparsity; start_transaction; tokio runtime; WorkspacePool run_inner (producer) spawn rewrite_commit in commit order, at most jobs in flight; send each result to the channel; on failure: shut down the others rewrite_commit (one per commit) pool.acquire: lock a slot, check the commit out skip if the subdir is missing; run the command snapshot the slot; persist its tree state success: new tree; failure: none consumer: print each job's stdout and stderr; "Skipped commit …" for a missing subdir a failure ends the run with an error (no rewrites) unless --ignore-errors; changed trees are remembered nothing changed or --ignore-changes: finish with nothing to record; else transform_descendants tx.finish(ui, "run: rewrite N commits"): write the operation, update the working copy (CLI page)

4. Core architecture touchpoints

StructureRead / writeHowExplained on
Selected commitsread, maybe rewrittenEach one whose snapshot differs from its tree gets a new tree. It keeps its change ID and description, and you become the committer.commits
Slot TreeStatescreated, loaded, savedOne per slot in .jj/run/default/N/state/, kept between runs so a checkout only touches changed files (source).working-copy
Slot lock fileslocked.jj/run/default/N.lock, held for one job. Concurrent jj run processes share the pool this way (source).—
Trees and filesread, writtenChecked out into slots; snapshots write new file and tree objects. Conflicted trees are kept as they are.trees
DescendantsrewrittenCommits outside the run set whose parents changed are rebased or, with --restore-descendants, reparented.transactions
View: heads, bookmarks, wc_commit_idswriteFollow the rewritten commits.view
OperationwriteOne operation, only if something was rewritten.operations
Your working copyread, then maybe writeSnapshotted first but never used to run the command. If @ was rewritten, its new tree is checked out.working-copy

5. Transaction and operation

  • Transaction: one, started before any command runs and committed only after all of them have finished, so the result is all or nothing (start).
  • Operation description: run: rewrite <n> commits, where n counts the commits whose own tree changed (source).
  • No changes: with --ignore-changes, or when no tree changed, jj finishes the transaction as run: nothing changed. Since the repo is untouched, tx.finish just prints Nothing changed. and records no operation (source, finish).
  • Failure: without --ignore-errors, the first failing command returns an error before the rewrite step. The transaction is dropped, so even commands that already succeeded change nothing (source).
  • Slots live outside the operation log. Their files and tree state persist after the run, whatever the outcome.
  • jj undo restores the previous view, bringing back the original commits.

6. Algorithm and invariants

The workspace pool

WorkspacePool has --jobs slots, numbered from 1, under .jj/run/default/, next to .jj/repo rather than inside it. Every slot is checked out and snapshotted with the same settings: your ui.conflict-marker-style, no end-of-line conversion, auto executable-bit handling and no fsmonitor (source, marker style). acquire does this:

  1. It tries each slot's lock without blocking, sleeping from 10 ms, doubling up to 250 ms, until one is free (source).
  2. If the slot has a saved tree_state and --clean wasn't given, it loads it and deletes the file. The missing file marks the slot as dirty until the job saves it again, so a crash mid-job makes the next user wipe the slot (source).
  3. Otherwise it deletes the slot's working copy and state and starts from an empty in-memory tree state (source).
  4. It sets the sparse patterns, or the root path for full, and checks the commit's tree out. Because it diffs against the saved state, only changed files are touched and ignored build artifacts survive (source).
  5. For a reused slot, it snapshots once more to find files that a previous commit ignored but this one doesn't, deletes them, and checks out again (source).

One job

rewrite_commit runs the command in the slot, or in the same subdirectory you ran jj run from unless --root is given. If that subdirectory doesn't exist in the commit, the commit is skipped (source, subdir). The command gets JJ_WORKSPACE_ROOT (the slot directory), JJ_CHANGE_ID, JJ_COMMIT_ID, and a null stdin. It is killed if its task is dropped. Output is buffered and printed whole when the job finishes, so parallel jobs never interleave (source).

Then jj snapshots the slot using the snapshot.auto-track matcher. If the command failed, it deletes the untracked, non-ignored files it left behind and discards the result. Either way it saves the slot's tree state. A successful snapshot becomes the job's new tree, conflicts included (source).

Scheduling

run_inner spawns jobs in parents-first order on a multi-threaded tokio runtime, with at most jobs in flight (runtime). Results come back in completion order through a channel to a consumer that runs at the same time and prints each one as it arrives (source). On a failure without --ignore-errors, the producer shuts down the remaining tasks, which kills their processes, and the consumer returns the error. Every job checks out its commit's original tree, so a commit never sees a command's changes to its ancestors.

Rewriting

A job's tree counts as changed only if its trees and conflict labels differ from the original (source). jj then walks the selected commits and their descendants with transform_descendants, parents first (source):

  • Changed, default: rebase the commit onto its rewritten parents, then 3-way merge rebased + (command result − original). That way the command's diff and any ancestors' changes both carry through (source).
  • Changed, --restore-descendants: rebase it and set its tree to the command result exactly, ignoring what changed in its ancestors (source).
  • Unchanged, or outside the run set: if its parents changed, rebase it, or reparent it with --restore-descendants. These count as rebased descendants (source).

Invariants: nothing is rewritten unless every command succeeded or --ignore-errors was given. Commands never run in your own working copy. Change IDs and descriptions are kept. Only trees change.

7. Errors and edge cases

MessageWhenSource
the command '<command and args>' failed with <status>, with hint Failed revision: …A command exited unsuccessfully, without --ignore-errors. Nothing is rewritten.run.rs
Skipped commit <id>: directory does not exist: <subdir>The commit doesn't contain the directory jj run was started from.run.rs
cannot use --passthrough with more than one job--passthrough with --jobs or run.jobs above 1.run.rs
invalid value for `run.jobs`: <n> (must be a positive integer)--jobs 0, or run.jobs zero or negative.run.rs
failed to checkout the commit …, failed to snapshot the working copy for commit …, failed to create path …, failed to delete path …Slot setup or snapshot failed.run.rs
Commit … is immutable (or The root commit … is immutable)A selected commit is immutable and --ignore-changes wasn't given.cli_util.rs
Nothing changed.No command changed a tree, or --ignore-changes.cli_util.rs
Rewrote <n> commits. then Rebased <m> descendant commits[ (while preserving their content)].Status after rewriting.run.rs
clap: --ignore-changes cannot be used with --restore-descendantsBoth given.run.rs

8. Worked example

From upstream's test_run_parallel_changes_propagate_to_descendants. A, B and C each add one file, and @ is empty on top. Running jj run --jobs=3 -r=..@ -- sh -c 'touch newfile-$JJ_CHANGE_ID.txt' runs the command on the four commits, at most three at a time. Each job sees only its commit's original tree, so each adds exactly one new file named after its own change ID:

jj run --jobs=3: each job adds one file, rewriting merges them upward Left: three slots run the command on A, B and C (the slot each gets depends on timing) at the same time, each on that commit's original tree, and each job adds newfile named after its change ID. Right: after rewriting parents first, A has a.txt and its own new file; B has a.txt, b.txt and the new files of A and B; C has a.txt, b.txt, c.txt and all three new files. Each commit's diff is still one original file plus its own new file, because the default rewrite merges the rebased tree with the command's diff. jobs in .jj/run/default/ after the rewrite slot 1: A's tree (a) + newfile-qpvuntsm….txt slot 2: B's tree (a, b) + newfile-rlvkpnrz….txt slot 3: C's tree (a, b, c) + newfile-kkmpptxz….txt @ (empty) runs in whichever slot frees up first, and adds a file too A qpvuntsm a.txt, newfile-qpvuntsm B rlvkpnrz + b.txt, newfile-rlvkpnrz C kkmpptxz + c.txt, newfile-kkmpptxz

Upstream then lists @- (that is, C):

a.txt
b.txt
c.txt
newfile-kkmpptxzrspxrzommnulwmwkkqwworpl.txt
newfile-qpvuntsmwlqtpsluzzsnyyzlmlwvmlnu.txt
newfile-rlvkpnrzqnoowoytxnquwvuryrwnrmlp.txt

B and C are rebased onto their rewritten parents, and the 3-way merge adds their own command result on top, so every descendant ends up with all its ancestors' new files. With --restore-descendants, C's tree would instead be exactly what its command produced, without A's and B's new files.

Two failure cases from the same file: test_run_failure_rewrites_nothing fails on B only, and the log afterwards is identical to before, even though the other commands succeeded. test_run_failure_shows_output shows the command's captured output printed first, then Error: the command 'fake-formatter --stdout hello stdout …' failed with exit status: 1 and Hint: Failed revision: qpvuntsm 26d8ff9b A.