jj / How jj git fetch works

How jj git fetch works

jj git fetch runs git fetch for each selected remote, then imports the changed remote-tracking refs into jj's view. Along the way it updates tracked local bookmarks, abandons commits the remote dropped, and records commits the remote rewrote as rewrites.

On this page
  1. 1. Command-line surface
  2. 2. Configuration
  3. 3. End-to-end flow
  4. 4. Core architecture touchpoints
  5. 5. Transaction and operation
  6. 6. Algorithm and invariants
  7. Refspecs
  8. Missing remote refs
  9. Import
  10. 7. Errors and edge cases
  11. 8. Worked example

In short: jj picks the remotes (from --remote, --all-remotes, git.fetch, the only remote, or origin). For each remote it builds a bookmark expression and a tag expression, and turns them into forced Git refspecs. It shells out to git fetch --porcelain --prune --no-write-fetch-head --refmap= --no-tags, dropping and retrying refspecs the remote doesn't have. Fetched bookmarks land in jj's own refs/jj/remote-bookmarks/ namespace, not refs/remotes/. jj then diffs those refs against the view and imports the changes. Tracked remote bookmarks are merged into local ones. Commits no longer reachable from any ref are abandoned, and remotely rewritten commits get synthetic predecessors. All of this is one operation, fetch from git remote(s) <names>. In a colocated workspace, the new remote bookmarks are exported to refs/remotes/ when that operation is written.

The command is cmd_git_fetch. Most of the work happens in jj-lib's GitFetch and the shared import code in import_refs_inner. Remote bookmarks and their tracking state live in the view (view page). Abandoning and rebasing descendants is the usual rewrite machinery, run when the transaction finishes (CLI page).

1. Command-line surface

FlagTypeDefaultMeaning
-b, --branch, --bookmark BRANCHname patterns, repeatablefrom configFetch only these branches. Globs, but only * is expanded. Union and ~ (negative intersection) are the only operators (source).
-t, --tag TAGname patterns, repeatablefrom configFetch only these tags, with the same pattern rules (source).
--trackedbooloffFetch only bookmarks and tags already tracked from each remote. Conflicts with --branch/--tag (source).
--remote REMOTEname patterns, repeatablegit.fetch, else the only remote, else originRemotes to fetch from. Globs by default, for example --remote '*' (source).
--all-remotesbooloffEvery remote. Conflicts with --remote (source).

If either --branch or --tag is given, the other one defaults to nothing, not to the config. So --branch main fetches no tags (source). The global --no-integrate-operation flag is respected: the fetch and import run, the operation is written but left uncommitted, and nothing is exported to Git (source, test).

GitFetchArgs

2. Configuration

KeyDefaultEffect
git.fetchunsetRemote name pattern, or a list of them, used when --remote isn't given (source, docs).
remotes.<name>.fetch-bookmarksunsetWhich branches to fetch from that remote. When it's unset, the remote's fetch refspecs are read from Git config instead (source, docs).
remotes.<name>.fetch-tagsunset (all tags)Which tags to fetch (source).
remotes.<name>.auto-track-bookmarksunsetNewly seen remote bookmarks whose names match start out tracked. Otherwise they're untracked (default_remote_ref_state_for). auto-track-created-bookmarks doesn't apply here (source).
git.abandon-unreachable-commitstrueAbandon commits that only the changed remote refs kept reachable (default, docs).
git.record-synthetic-predecessorstrueLink newly imported commits to older commits with the same change ID (docs).
git.executable-pathgitThe Git binary jj runs (GitSettings).

With no explicit choice, the remote is guessed: a single existing remote is used, with Hint: Fetching from the only existing remote: <name> if it isn't called origin. Otherwise jj uses origin (source).

3. End-to-end flow

jj git fetch: call flow cmd_git_fetch loads the workspace and picks the remotes, warning about exact names that don't exist and failing if none match. It starts a transaction and, per remote, builds a bookmark and tag expression from flags, config or Git's refspecs, expanding them into forced refspecs. GitFetch::fetch runs git fetch for each remote, retrying without refspecs whose source is missing and deleting their local refs; branches land under refs/jj/remote-bookmarks/R/. GitFetch::import_refs diffs those fetched refs against the view and runs import_refs_inner: import the new heads, update remote refs and merge tracked ones into local bookmarks, abandon unreachable commits, record synthetic predecessors. The stats are printed, missing branches warned about, and tx.finish rebases descendants, exports the remote bookmarks to refs/remotes in a colocated workspace, and records the operation. load workspace (snapshot); remotes = --all-remotes | --remote | git.fetch | only remote | origin exact names that don't exist → "Warning: No matching remotes for names: …" none match → "No git remotes to fetch from" start_transaction; per remote: --tracked → names already tracked bookmarks: --branch | fetch-bookmarks | Git refspecs tags: --tag | fetch-tags | all expand_fetch_refspecs +refs/heads/G:refs/jj/remote-bookmarks/R/G +refs/tags/G:refs/jj/remote-tags/R/G negative patterns → ^refs/heads/G GitFetch::fetch, once per remote (git subprocess) git fetch --porcelain --prune --no-write-fetch-head --refmap= --no-tags -- R <refspecs> remote lacks a source ref → drop that refspec, delete its destination ref, retry rejected updates → error GitFetch::import_refs → diff_fetched_refs_to_import (fetched patterns only) → import_refs_inner import new head commits into the backend and index; add them as visible heads set each remote ref; tracked → merge_local_bookmark / merge_local_tag (base = old tracked target) abandon commits only old remote targets reached (git.abandon-unreachable-commits) same change ID, new commit → synthetic predecessor and rewrite (git.record-synthetic-predecessors) print "bookmark: name@remote [new|updated|deleted] tracked|untracked", abandoned, rewritten; tx.finish: rebase descendants; colocated → export refs/remotes under the lock; write the operation

Every remote is fetched before anything is imported, so the import sees all remotes at once (source). Just before the fetch loop it prints Fetching from Git remotes: <r1, r2, …> on the status stream (source). If any git fetch fails, the command errors out before the transaction is committed. Refs that git already wrote stay in the Git repository.

4. Core architecture touchpoints

StructureRead / writeHowExplained on
Git repository refswritten by git, then readgit fetch writes refs/jj/remote-bookmarks/<remote>/* for branches and refs/jj/remote-tags/<remote>/* for tags (namespaces). Tags never land in refs/tags. Only in a colocated workspace, when the transaction finishes, are the fetched bookmarks exported to refs/remotes/<remote>/* (source).commits
View.remote_viewsread, writeEach changed remote bookmark or tag gets its new target. It keeps its tracking state, or gets the default state if it is new (source).view
View.local_bookmarks, local_tagswriteFor tracked refs, a 3-way merge: local, the old remote target as base, and the new remote target. This can move, delete or conflict a local bookmark.view
View.git_refswrite (colocated)The import leaves it alone: diff_fetched_refs_to_import returns no Git-ref changes (source). In a colocated workspace, the export at the end records the refs/remotes refs it writes.view
Commit store and indexwriteNew Git commits are imported in bulk, indexed, and added as heads (source).commits
Rewrite recordswriterecord_abandoned_commit, set_rewritten_commit, set_divergent_rewrite and set_predecessors. Descendants are rebased when the transaction finishes.transactions
OperationwriteOne operation for the whole fetch.operations

5. Transaction and operation

  • Transaction: one, started after the remotes are chosen and before any refspec is built (source).
  • Without integration: with --no-integrate-operation the operation is written but not committed, and nothing is exported to Git. The fetched refs stay in jj's own namespaces (test).
  • Operation description: fetch from git remote(s) <r1, r2, …>, the matched remotes joined with a comma and a space, the same list the status line prints (join, finish). Upstream's evolog shows it as fetch from git remote(s) origin (test).
  • Commits: fetched commits are imported unchanged. Abandoned commits are hidden, and their local descendants are rebased. With synthetic predecessors, local descendants of remotely rewritten commits are rebased onto the new versions.
  • Working copy: if @ was rebased or abandoned, it is updated when the transaction finishes, as with any rewrite.
  • jj undo restores the previous view. The fetched Git objects stay in the Git store.

6. Algorithm and invariants

Refspecs

split_into_positive_negative_patterns requires each expression to be (p1 | p2 | …) ~ (n1 | …): at most one union of positive patterns, minus unions of negative ones. Each pattern must convert to a Git glob containing none of : ^ ? [ ] (source, pattern_to_refspec_glob). Positive patterns become forced refspecs into jj's namespaces. Negative ones become ^refs/… exclusions. Without fetch-bookmarks, the remote's Git refspecs are reused if they have the shape +refs/heads/X:refs/remotes/<remote>/X. Anything else is skipped with a warning (load_default_fetch_bookmarks).

Missing remote refs

git fetch fails as a whole when one explicit source ref doesn't exist, and reports only one at a time. So fetch loops: run git, drop the failing refspec, and retry. --prune doesn't delete a ref named explicitly by a refspec, so when a refspec is dropped jj deletes its destination ref itself through gix, which is how a branch that disappeared upstream goes away locally. A failed deletion is Failed to prune stale refs. If no refspecs are left, nothing is fetched from that remote. --refmap= stops Git from also updating refs through the remote's configured remote.<name>.fetch mappings, so only the refspecs jj passed are applied (spawn_fetch).

Import

The import only considers refs matching what was fetched from each remote, so --branch a1 doesn't notice that a2 was deleted upstream (source). diff_fetched_refs_to_import compares the refs under refs/jj/remote-bookmarks/ and refs/jj/remote-tags/ with the view's remote bookmarks and tags. A known ref missing from Git becomes an update to absent.

Abandonment (abandon_unreachable_commits) hides pinned..old_remote_heads. Here pinned means local bookmarks and tags, plus untracked remote refs that are still visible, plus the root. It only runs when some changed ref had an old target, and only for commits that are currently visible. Synthetic predecessors (record_synthetic_predecessors) match change IDs between the commits the remote dropped and the newly reachable ones. A match marks the old commit as rewritten into the new one, or as a divergent rewrite if there are several new ones. Rewritten commits are then removed from the abandoned list (source).

Invariants: remote-tracking refs always mirror what the remote last reported, even when the local merge conflicts. Untracked remote bookmarks never move local bookmarks. Fetched commits are never modified.

7. Errors and edge cases

MessageWhenSource
No git remotes to fetch fromThe remote selection matched no configured remote.fetch.rs
Warning: No matching remotes for names: …Exact remote names that don't exist.fetch.rs
Warning: No matching branches found on any specified/configured remote: …An exact --branch name that exists on none of the remotes, before or after the fetch.fetch.rs
Warning: Ignored refspec `…` from `…`: <reason>A Git-configured refspec jj can't use: fetch-only, non-forced, not under refs/heads/, not into refs/remotes/<remote>/, or renaming.git.rs
Invalid branch pattern provided. When fetching, branch names and globs may not contain the characters …A pattern with : ^ ? [ ], or one that isn't a glob.git.rs
Cannot use `&` for positive expressions, and the other two shape errorsAn expression that isn't a union minus unions.git.rs
Failed to update refs: …Git rejected a ref update. Remote refs are forced, so this is rare.git.rs
Failed to prune stale refsDeleting the local ref of a refspec the remote no longer has failed.git.rs
No git remote named '…'The remote disappeared from Git config between selection and fetch.git.rs
Warning: Failed to import some Git refs:Refs whose targets couldn't be imported. Refs under the reserved git remote name add a hint to rename the remote.git_util.rs

8. Worked example

From upstream's test_git_fetch_remotely_rewritten. The local repo has book@origin (untracked) on eedc2709 "bookmarked", whose parent is 97604bbe "original", and the working copy @ sits on top. On the remote, "original" is re-described as "modified", which rewrites both commits. Then jj git fetch runs:

jj git fetch after a remote rewrite Before: root, 97604bbe "original", eedc2709 "bookmarked" with book@origin, then @ 257ea01f. After jj git fetch: root, f30445f7 "modified", 3ee37bc8 "bookmarked" with book@origin, then @ 0818b176. The two old commits were matched by change ID and recorded as rewritten into the new ones, so the working copy was rebased instead of left on abandoned commits. before after jj git fetch root 97604bbe original eedc2709 bookmarked · book@origin @ 257ea01f root f30445f7 modified 3ee37bc8 bookmarked · book@origin @ 0818b176 rebased

Upstream's snapshot:

$ jj git fetch
Fetching from Git remotes: origin
bookmark: book@origin [updated] untracked
Updated 2 rewritten commits.
Rebased 1 descendant commits.
Working copy  (@) now at: royxmykx 0818b176 (empty) (no description set)
Parent commit (@-)      : kkmpptxz 3ee37bc8 book@origin | (empty) bookmarked

Both new commits carry the old change IDs, so the commits only eedc2709 reached are mapped onto them instead of being abandoned. jj evolog then shows each new commit with the predecessor it replaced, created by the fetch from git remote(s) origin operation. Compare test_git_fetch_removed_bookmark: there, a tracked bookmark deleted on the remote deletes the local bookmark, and its commit is reported as Abandoned 1 commits that are no longer reachable.